Fluentd and Nebius Cloud Monitoring Integration

Input and output integration overview

The Fluentd Input Plugin gathers metrics from Fluentd’s in_monitor plugin endpoint. It provides insights into various plugin metrics while allowing for custom configurations to reduce series cardinality.

This plugin allows users to effortlessly send aggregated metrics to Nebius Cloud Monitoring, leveraging the cloud’s monitoring solutions.

Integration details

Fluentd

This plugin gathers metrics from the Fluentd plugin endpoint provided by the in_monitor plugin. It reads data from the /api/plugin.json resource and allows exclusion of specific plugins based on their type.

Nebius Cloud Monitoring

The Nebius Cloud Monitoring plugin serves as an intermediary to send custom metrics to the Nebius Cloud Monitoring service. It is designed specifically to facilitate the monitoring of applications and services running within the Nebius ecosystem. This plugin is especially useful for users of the Nebius Cloud Platform who need to leverage cloud-based monitoring capabilities without significant configuration overhead. The plugin’s integration relies on Google Cloud metadata, allowing it to automatically fetch the necessary authentication credentials from the Compute instance it operates within. Key technical considerations include the management of reserved labels to ensure metrics are recorded correctly without conflicts.

Configuration

Fluentd

[[inputs.fluentd]]
  ## This plugin reads information exposed by fluentd (using /api/plugins.json endpoint).
  ##
  ## Endpoint:
  ## - only one URI is allowed
  ## - https is not supported
  endpoint = "http://localhost:24220/api/plugins.json"

  ## Define which plugins have to be excluded (based on "type" field - e.g. monitor_agent)
  exclude = [
    "monitor_agent",
    "dummy",
  ]

Nebius Cloud Monitoring

[[outputs.nebius_cloud_monitoring]]
  ## Timeout for HTTP writes.
  # timeout = "20s"

  ## Nebius.Cloud monitoring API endpoint. Normally should not be changed
  # endpoint = "https://monitoring.api.il.nebius.cloud/monitoring/v2/data/write"

Input and output integration examples

Fluentd

1. Basic Configuration: Set up the Fluentd Input Plugin to gather metrics from your Fluentd instance’s monitoring endpoint, ensuring you are able to track performance and usage statistics.
2. Excluding Plugins: Use the exclude option to ignore specific plugins’ metrics that are not necessary for your monitoring needs, streamlining data collection and focusing on what matters.
3. Custom Plugin ID: Implement the @id parameter in your Fluentd configuration to maintain a consistent plugin_id, which helps avoid issues with high series cardinality during frequent restarts.

Nebius Cloud Monitoring

1. Dynamic Application Monitoring: Integrate this plugin with your application to continuously send metrics related to resource usage, such as CPU and memory utilization, to Nebius Cloud Monitoring. By doing so, you can gain insights into the performance of your application, allowing for adjustments in real-time based on the metrics received.

2. Incident Response Automation: Use the Nebius Cloud Monitoring plugin to automatically send alerts and metrics when certain thresholds are reached. For instance, if a particular service’s uptime drops below a certain percentage, the plugin can be configured to report this directly to the monitoring service, enabling quicker incident response and resolution.

3. Comparative Service Analysis: Set up the plugin to send metrics from multiple cloud instances running different versions of the same application to Nebius Cloud Monitoring. This approach allows for a comparative analysis of resource usage and performance, helping teams determine which version performs best under similar workloads.

4. Aggregated Metrics Dashboard: Use this plugin to create a centralized dashboard displaying metrics from various services across your cloud instances. By aggregating different application metrics into one interface, stakeholders can assess the overall health and performance of their cloud environment easily.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.