OpenTelemetry and Clarify Integration

Input and output integration overview

This plugin receives traces, metrics, and logs from OpenTelemetry clients and agents via gRPC, enabling comprehensive observability of applications.

The Clarify plugin allows users to publish Telegraf metrics directly to Clarify, enabling enhanced analysis and monitoring capabilities.

Integration details

OpenTelemetry

The OpenTelemetry plugin is designed to receive telemetry data such as traces, metrics, and logs from clients and agents implementing OpenTelemetry via gRPC. This plugin initiates a gRPC service that listens for incoming telemetry data, making it distinct from standard plugins that collect metrics at defined intervals. The OpenTelemetry ecosystem aids developers in observing and understanding their applications’ performance by providing a vendor-neutral way to instrument, generate, collect, and export telemetry data. Key features of this plugin include customizable connection timeouts, adjustable maximum message sizes for incoming data, and options for specifying span, log, and profile dimensions to tag the incoming metrics. With this flexibility, organizations can tailor their telemetry collection to meet precise observability requirements and ensure seamless data integration into systems like InfluxDB.

Clarify

This plugin facilitates the writing of Telegraf metrics to Clarify, a platform for managing and analyzing time series data. By transforming metrics into Clarify signals, this output plugin enables seamless integration of collected telemetry data into the Clarify ecosystem. Users must obtain valid credentials, either through a credentials file or basic authentication, to configure the plugin. The configuration also provides options for fine-tuning how metrics are mapped to signals in Clarify, including the ability to specify unique identifiers using tags. Given that Clarify supports only floating point values, the plugin ensures that any unsupported types are effectively filtered out during the publishing process. This comprehensive connectivity aligns with use cases in monitoring, data analysis, and operational insights.

Configuration

OpenTelemetry

[[inputs.opentelemetry]]
  ## Override the default (0.0.0.0:4317) destination OpenTelemetry gRPC service
  ## address:port
  # service_address = "0.0.0.0:4317"

  ## Override the default (5s) new connection timeout
  # timeout = "5s"

  ## gRPC Maximum Message Size
  # max_msg_size = "4MB"

  ## Override the default span attributes to be used as line protocol tags.
  ## These are always included as tags:
  ## - trace ID
  ## - span ID
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  # span_dimensions = ["service.name", "span.name"]

  ## Override the default log record attributes to be used as line protocol tags.
  ## These are always included as tags, if available:
  ## - trace ID
  ## - span ID
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  ## When using InfluxDB for both logs and traces, be certain that log_record_dimensions
  ## matches the span_dimensions value.
  # log_record_dimensions = ["service.name"]

  ## Override the default profile attributes to be used as line protocol tags.
  ## These are always included as tags, if available:
  ## - profile_id
  ## - address
  ## - sample
  ## - sample_name
  ## - sample_unit
  ## - sample_type
  ## - sample_type_unit
  ## Common attributes can be found here:
  ## - https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv
  # profile_dimensions = []

  ## Override the default (prometheus-v1) metrics schema.
  ## Supports: "prometheus-v1", "prometheus-v2"
  ## For more information about the alternatives, read the Prometheus input
  ## plugin notes.
  # metrics_schema = "prometheus-v1"

  ## Optional TLS Config.
  ## For advanced options: https://github.com/influxdata/telegraf/blob/v1.18.3/docs/TLS.md
  ##
  ## Set one or more allowed client CA certificate file names to
  ## enable mutually authenticated TLS connections.
  # tls_allowed_cacerts = ["/etc/telegraf/clientca.pem"]
  ## Add service certificate and key.
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"

Clarify

[[outputs.clarify]]
  ## Credentials File (Oauth 2.0 from Clarify integration)
  credentials_file = "/path/to/clarify/credentials.json"

  ## Clarify username password (Basic Auth from Clarify integration)
  username = "i-am-bob"
  password = "secret-password"

  ## Timeout for Clarify operations
  # timeout = "20s"

  ## Optional tags to be included when generating the unique ID for a signal in Clarify
  # id_tags = []
  # clarify_id_tag = 'clarify_input_id'

Input and output integration examples

OpenTelemetry

1. Unified Monitoring Across Services: Use the OpenTelemetry plugin to collect and consolidate telemetry data from various microservices within a Kubernetes environment. By instrumenting each service with OpenTelemetry, you can utilize this plugin to gather a holistic view of application performance and dependencies in real-time, enabling faster troubleshooting and improved reliability of complex systems.

2. Enhanced Debugging with Traces: Implement this plugin to capture end-to-end traces of requests flowing through multiple services. For instance, when a user initiates a transaction that triggers several backend services, the OpenTelemetry plugin can record detailed traces that highlight performance bottlenecks, giving developers the necessary insights to debug issues and optimize their code.

3. Dynamic Load Testing and Performance Monitoring: Leverage the capabilities of this plugin during load testing phases by collecting live metrics and traces under simulated higher loads. This approach helps to evaluate the resilience of the application components and identify potential performance degradations preemptively, ensuring a smooth user experience in production.

4. Integrated Logging and Metrics for Real-Time Monitoring: Combine the OpenTelemetry plugin with logging frameworks to gather real-time logs alongside metric data, creating a powerful observability platform. For example, integrate it within a CI/CD pipeline to monitor builds and deployments, while collecting logs that help diagnose failures or performance issues in real-time.

Clarify

1. Automated Data Monitoring: By integrating the Clarify plugin with sensor data collection, organizations can automate the monitoring of environmental conditions, such as temperature and humidity. The plugin processes metrics in real-time, sending updates to Clarify where they can be analyzed for trends, alerts, and historical tracking. This use case makes it easier to maintain optimal conditions in data centers or production environments, reducing the risk of equipment failures.

2. Performance Metrics Analysis: Companies can leverage this plugin to send application performance metrics to Clarify. By transmitting key indicators such as response times and error rates, developers and operations teams can utilize Clarify’s capabilities to visualize and analyze application performance over time. This insight can drive improvements in user experience and help identify areas in need of optimization.

3. Sensor Data Aggregation: Utilizing the plugin to push data from multiple sensors to Clarify allows for a comprehensive view of physical environments. This aggregation is particularly beneficial in sectors such as agriculture, where metrics from various sensors can be correlated to decision-making about resource allocations, pest control, and crop management. The plugin ensures the data is accurately mapped and transformed for effective analysis.

4. Real-Time Alerts and Notifications: Implement the Clarify plugin to trigger real-time alerts based on predefined thresholds within the metrics being sent. For instance, if temperature readings exceed certain levels, alerts can be generated and sent to operational staff. This proactive approach allows for immediate responses to potential issues, enhancing operational reliability and safety.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.