OpenTelemetry and Librato Integration

Input and output integration overview

This plugin receives traces, metrics, and logs from OpenTelemetry clients and agents via gRPC, enabling comprehensive observability of applications.

The Librato plugin for Telegraf is designed to facilitate seamless integration with the Librato Metrics API, allowing for efficient metric reporting and monitoring.

Integration details

OpenTelemetry

The OpenTelemetry plugin is designed to receive telemetry data such as traces, metrics, and logs from clients and agents implementing OpenTelemetry via gRPC. This plugin initiates a gRPC service that listens for incoming telemetry data, making it distinct from standard plugins that collect metrics at defined intervals. The OpenTelemetry ecosystem aids developers in observing and understanding their applications’ performance by providing a vendor-neutral way to instrument, generate, collect, and export telemetry data. Key features of this plugin include customizable connection timeouts, adjustable maximum message sizes for incoming data, and options for specifying span, log, and profile dimensions to tag the incoming metrics. With this flexibility, organizations can tailor their telemetry collection to meet precise observability requirements and ensure seamless data integration into systems like InfluxDB.

Librato

The Librato plugin enables Telegraf to send metrics to the Librato Metrics API. To authenticate, users must provide an api_user and api_token, which can be acquired from the Librato account settings. This integration allows for efficient monitoring and reporting of custom metrics within the Librato platform. The plugin also utilizes a source_tag option that can enrich the metrics with contextual information from Point Tags; however, it does not currently support sending associated Point Tags. It is essential to note that any point value sent that cannot be converted to a float64 type will be skipped, ensuring that only valid metrics are processed and sent to Librato. The plugin also supports secret-store options for managing sensitive authentication credentials securely, facilitating best practices in credential management.

Configuration

OpenTelemetry

[[inputs.opentelemetry]]
  ## Override the default (0.0.0.0:4317) destination OpenTelemetry gRPC service
  ## address:port
  # service_address = "0.0.0.0:4317"

  ## Override the default (5s) new connection timeout
  # timeout = "5s"

  ## gRPC Maximum Message Size
  # max_msg_size = "4MB"

  ## Override the default span attributes to be used as line protocol tags.
  ## These are always included as tags:
  ## - trace ID
  ## - span ID
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  # span_dimensions = ["service.name", "span.name"]

  ## Override the default log record attributes to be used as line protocol tags.
  ## These are always included as tags, if available:
  ## - trace ID
  ## - span ID
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  ## When using InfluxDB for both logs and traces, be certain that log_record_dimensions
  ## matches the span_dimensions value.
  # log_record_dimensions = ["service.name"]

  ## Override the default profile attributes to be used as line protocol tags.
  ## These are always included as tags, if available:
  ## - profile_id
  ## - address
  ## - sample
  ## - sample_name
  ## - sample_unit
  ## - sample_type
  ## - sample_type_unit
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  # profile_dimensions = []

  ## Override the default (prometheus-v1) metrics schema.
  ## Supports: "prometheus-v1", "prometheus-v2"
  ## For more information about the alternatives, read the Prometheus input
  ## plugin notes.
  # metrics_schema = "prometheus-v1"

  ## Optional TLS Config.
  ## For advanced options: https://github.com/influxdata/telegraf/blob/v1.18.3/docs/TLS.md
  ##
  ## Set one or more allowed client CA certificate file names to
  ## enable mutually authenticated TLS connections.
  # tls_allowed_cacerts = ["/etc/telegraf/clientca.pem"]
  ## Add service certificate and key.
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"

Librato

[[outputs.librato]]
  ## Librato API Docs
  ## http://dev.librato.com/v1/metrics-authentication
  ## Librato API user
  api_user = "[email protected]" # required.
  ## Librato API token
  api_token = "my-secret-token" # required.
  ## Debug
  # debug = false
  ## Connection timeout.
  # timeout = "5s"
  ## Output source Template (same as graphite buckets)
  ## see https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#graphite
  ## This template is used in librato's source (not metric's name)
  template = "host"

Input and output integration examples

OpenTelemetry

1. Unified Monitoring Across Services: Use the OpenTelemetry plugin to collect and consolidate telemetry data from various microservices within a Kubernetes environment. By instrumenting each service with OpenTelemetry, you can utilize this plugin to gather a holistic view of application performance and dependencies in real-time, enabling faster troubleshooting and improved reliability of complex systems.

2. Enhanced Debugging with Traces: Implement this plugin to capture end-to-end traces of requests flowing through multiple services. For instance, when a user initiates a transaction that triggers several backend services, the OpenTelemetry plugin can record detailed traces that highlight performance bottlenecks, giving developers the necessary insights to debug issues and optimize their code.

3. Dynamic Load Testing and Performance Monitoring: Leverage the capabilities of this plugin during load testing phases by collecting live metrics and traces under simulated higher loads. This approach helps to evaluate the resilience of the application components and identify potential performance degradations preemptively, ensuring a smooth user experience in production.

4. Integrated Logging and Metrics for Real-Time Monitoring: Combine the OpenTelemetry plugin with logging frameworks to gather real-time logs alongside metric data, creating a powerful observability platform. For example, integrate it within a CI/CD pipeline to monitor builds and deployments, while collecting logs that help diagnose failures or performance issues in real-time.

Librato

1. Real-time Application Monitoring: Utilize Librato to collect performance metrics from a web application in real-time. This setup involves sending response times, error rates, and user interactions to Librato, allowing developers to monitor the application’s health and performance metrics closely. By analyzing these metrics, teams can quickly identify and address performance bottlenecks or application failures before they impact end users.

2. Infrastructure Metrics Aggregation: Leverage this plugin to gather and send metrics from various infrastructure components, such as servers or containers, to Librato for centralized monitoring. Configuring the plugin to send CPU, memory usage, and disk I/O metrics enables system administrators to have a comprehensive view of infrastructure performance, assisting in capacity planning and resource optimization strategies.

3. Custom Metrics for Business Operations: Feed business-specific metrics, such as sales transactions or user sign-ups, to the Librato service using this plugin. By tracking these custom metrics, businesses can gain insights into their operational performance and make data-driven decisions to enhance their strategies, marketing efforts, or product development initiatives.

4. Anomaly Detection in Metrics: Implement monitoring tools that utilize machine learning for anomaly detection. By continuously sending real-time metrics to Librato, teams can analyze trends and automatically flag unusual behavior, such as sudden spikes in latency or unusual traffic patterns, enabling timely intervention and troubleshooting.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.