Hashicorp Nomad and Apache Druid Integration

Input and output integration overview

This plugin allows users to collect metrics from Hashicorp Nomad agents in distributed environments.

This plugin allows Telegraf to send JSON-formatted metrics to Apache Druid over HTTP, enabling real-time ingestion for analytical queries on high-volume time-series data.

Integration details

Hashicorp Nomad

The Hashicorp Nomad input plugin is designed to gather metrics from every Nomad agent within a cluster. By deploying Telegraf on each node, it can connect to the local Nomad agent, typically available at ‘http://127.0.0.1:4646’. With this setup, users can systematically collect and monitor metrics related to the performance and status of their Nomad environment, ensuring they maintain a healthy and efficient cluster operational state. This plugin enables visibility into the operational aspects of Nomad, which is essential for maintaining reliable cloud infrastructure.

Apache Druid

This configuration uses Telegraf’s HTTP output plugin with json data format to send metrics directly to Apache Druid, a real-time analytics database designed for fast, ad hoc queries on high-ingest time-series data. Druid supports ingestion via HTTP POST to various components like the Tranquility service or native ingestion endpoints. The JSON format is ideal for structuring Telegraf metrics into event-style records for Druid’s columnar and time-partitioned storage engine. Druid excels at powering interactive dashboards and exploratory queries across massive datasets, making it an excellent choice for real-time observability and monitoring analytics when integrated with Telegraf.

Configuration

Hashicorp Nomad

[[inputs.nomad]]
  ## URL for the Nomad agent
  # url = "http://127.0.0.1:4646"

  ## Set response_timeout (default 5 seconds)
  # response_timeout = "5s"

  ## Optional TLS Config
  # tls_ca = /path/to/cafile
  # tls_cert = /path/to/certfile
  # tls_key = /path/to/keyfile

Apache Druid

[[outputs.http]]
  ## Druid ingestion endpoint (e.g., Tranquility, HTTP Ingest, or Kafka REST Proxy)
  url = "http://druid-ingest.example.com/v1/post"

  ## Use POST method to send events
  method = "POST"

  ## Data format for Druid ingestion (expects JSON format)
  data_format = "json"

  ## Optional headers (may vary depending on Druid setup)
  # [outputs.http.headers]
  #   Content-Type = "application/json"
  #   Authorization = "Bearer YOUR_API_TOKEN"

  ## Optional timeout and TLS settings
  timeout = "10s"
  # tls_ca = "/path/to/ca.pem"
  # tls_cert = "/path/to/cert.pem"
  # tls_key = "/path/to/key.pem"
  # insecure_skip_verify = false

Input and output integration examples

Hashicorp Nomad

1. Cluster Health Monitoring: Use the Hashicorp Nomad plugin to aggregate metrics across all nodes in a Nomad deployment. By monitoring health metrics such as allocation status, job performance, and resource utilization, operations teams can gain insights into the overall health of their deployment, quickly identify and resolve issues, and optimize resource allocation based on real-time data.

2. Performance Analytics for Job Execution: Leverage the metrics provided by Nomad to analyze job execution times and resource consumption. This use case enables developers to adjust job parameters effectively, optimize task performance, and illustrate trends over time, ultimately leading to increased efficiency and reduced costs in resource allocation.

3. Alerting on Critical Conditions: Implement alerting mechanisms based on metrics scraped from Nomad agents. By setting thresholds for critical metrics like CPU usage or failed job allocations, teams can proactively respond to potential issues before they escalate, ensuring higher uptime and reliability for applications running on the Nomad platform.

4. Integration with Visualization Tools: Use the data collected by the Hashicorp Nomad plugin to feed into visualization tools for real-time dashboards. This setup allows teams to monitor cluster workloads, job states, and system performance at a glance, facilitating better decision-making and strategic planning based on visual insights into the Nomad environment.

Apache Druid

1. Real-Time Application Monitoring Dashboard: Use Telegraf to collect metrics from application servers and send them to Druid for immediate analysis and visualization in dashboards. Druid’s low-latency querying allows users to interactively explore system behavior in near real-time.

2. Security Event Aggregation: Aggregate and forward security-related metrics such as failed logins, port scans, or process anomalies to Druid. Analysts can build dashboards to monitor threat patterns and investigate incidents with millisecond-level granularity.

3. IoT Device Analytics: Collect telemetry from edge devices via Telegraf and send it to Druid for fast, scalable processing. Druid’s time-partitioned storage and roll-up capabilities are ideal for handling billions of small JSON events from sensors or gateways.

4. Web Traffic Behavior Exploration: Use Telegraf to capture web server metrics (e.g., requests per second, latency, error rates) and forward them to Druid. This enables teams to drill down into user behavior by region, device, or request type with subsecond query performance.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.