OpenTelemetry and Nebius Cloud Monitoring Integration

Input and output integration overview

This plugin receives traces, metrics, and logs from OpenTelemetry clients and agents via gRPC, enabling comprehensive observability of applications.

This plugin allows users to effortlessly send aggregated metrics to Nebius Cloud Monitoring, leveraging the cloud’s monitoring solutions.

Integration details

OpenTelemetry

The OpenTelemetry plugin is designed to receive telemetry data such as traces, metrics, and logs from clients and agents implementing OpenTelemetry via gRPC. This plugin initiates a gRPC service that listens for incoming telemetry data, making it distinct from standard plugins that collect metrics at defined intervals. The OpenTelemetry ecosystem aids developers in observing and understanding their applications’ performance by providing a vendor-neutral way to instrument, generate, collect, and export telemetry data. Key features of this plugin include customizable connection timeouts, adjustable maximum message sizes for incoming data, and options for specifying span, log, and profile dimensions to tag the incoming metrics. With this flexibility, organizations can tailor their telemetry collection to meet precise observability requirements and ensure seamless data integration into systems like InfluxDB.

Nebius Cloud Monitoring

The Nebius Cloud Monitoring plugin serves as an intermediary to send custom metrics to the Nebius Cloud Monitoring service. It is designed specifically to facilitate the monitoring of applications and services running within the Nebius ecosystem. This plugin is especially useful for users of the Nebius Cloud Platform who need to leverage cloud-based monitoring capabilities without significant configuration overhead. The plugin’s integration relies on Google Cloud metadata, allowing it to automatically fetch the necessary authentication credentials from the Compute instance it operates within. Key technical considerations include the management of reserved labels to ensure metrics are recorded correctly without conflicts.

Configuration

OpenTelemetry

[[inputs.opentelemetry]]
  ## Override the default (0.0.0.0:4317) destination OpenTelemetry gRPC service
  ## address:port
  # service_address = "0.0.0.0:4317"

  ## Override the default (5s) new connection timeout
  # timeout = "5s"

  ## gRPC Maximum Message Size
  # max_msg_size = "4MB"

  ## Override the default span attributes to be used as line protocol tags.
  ## These are always included as tags:
  ## - trace ID
  ## - span ID
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  # span_dimensions = ["service.name", "span.name"]

  ## Override the default log record attributes to be used as line protocol tags.
  ## These are always included as tags, if available:
  ## - trace ID
  ## - span ID
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  ## When using InfluxDB for both logs and traces, be certain that log_record_dimensions
  ## matches the span_dimensions value.
  # log_record_dimensions = ["service.name"]

  ## Override the default profile attributes to be used as line protocol tags.
  ## These are always included as tags, if available:
  ## - profile_id
  ## - address
  ## - sample
  ## - sample_name
  ## - sample_unit
  ## - sample_type
  ## - sample_type_unit
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  # profile_dimensions = []

  ## Override the default (prometheus-v1) metrics schema.
  ## Supports: "prometheus-v1", "prometheus-v2"
  ## For more information about the alternatives, read the Prometheus input
  ## plugin notes.
  # metrics_schema = "prometheus-v1"

  ## Optional TLS Config.
  ## For advanced options: https://github.com/influxdata/telegraf/blob/v1.18.3/docs/TLS.md
  ##
  ## Set one or more allowed client CA certificate file names to
  ## enable mutually authenticated TLS connections.
  # tls_allowed_cacerts = ["/etc/telegraf/clientca.pem"]
  ## Add service certificate and key.
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"

Nebius Cloud Monitoring

[[outputs.nebius_cloud_monitoring]]
  ## Timeout for HTTP writes.
  # timeout = "20s"

  ## Nebius.Cloud monitoring API endpoint. Normally should not be changed
  # endpoint = "https://monitoring.api.il.nebius.cloud/monitoring/v2/data/write"

Input and output integration examples

OpenTelemetry

1. Unified Monitoring Across Services: Use the OpenTelemetry plugin to collect and consolidate telemetry data from various microservices within a Kubernetes environment. By instrumenting each service with OpenTelemetry, you can utilize this plugin to gather a holistic view of application performance and dependencies in real-time, enabling faster troubleshooting and improved reliability of complex systems.

2. Enhanced Debugging with Traces: Implement this plugin to capture end-to-end traces of requests flowing through multiple services. For instance, when a user initiates a transaction that triggers several backend services, the OpenTelemetry plugin can record detailed traces that highlight performance bottlenecks, giving developers the necessary insights to debug issues and optimize their code.

3. Dynamic Load Testing and Performance Monitoring: Leverage the capabilities of this plugin during load testing phases by collecting live metrics and traces under simulated higher loads. This approach helps to evaluate the resilience of the application components and identify potential performance degradations preemptively, ensuring a smooth user experience in production.

4. Integrated Logging and Metrics for Real-Time Monitoring: Combine the OpenTelemetry plugin with logging frameworks to gather real-time logs alongside metric data, creating a powerful observability platform. For example, integrate it within a CI/CD pipeline to monitor builds and deployments, while collecting logs that help diagnose failures or performance issues in real-time.

Nebius Cloud Monitoring

1. Dynamic Application Monitoring: Integrate this plugin with your application to continuously send metrics related to resource usage, such as CPU and memory utilization, to Nebius Cloud Monitoring. By doing so, you can gain insights into the performance of your application, allowing for adjustments in real-time based on the metrics received.

2. Incident Response Automation: Use the Nebius Cloud Monitoring plugin to automatically send alerts and metrics when certain thresholds are reached. For instance, if a particular service’s uptime drops below a certain percentage, the plugin can be configured to report this directly to the monitoring service, enabling quicker incident response and resolution.

3. Comparative Service Analysis: Set up the plugin to send metrics from multiple cloud instances running different versions of the same application to Nebius Cloud Monitoring. This approach allows for a comparative analysis of resource usage and performance, helping teams determine which version performs best under similar workloads.

4. Aggregated Metrics Dashboard: Use this plugin to create a centralized dashboard displaying metrics from various services across your cloud instances. By aggregating different application metrics into one interface, stakeholders can assess the overall health and performance of their cloud environment easily.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.