OpenStack and M3DB Integration

Input and output integration overview

This plugin collects metrics from essential OpenStack services, facilitating the monitoring and management of cloud infrastructures.

This plugin allows Telegraf to stream metrics to M3DB using the Prometheus Remote Write protocol, enabling scalable ingestion through the M3 Coordinator.

Integration details

OpenStack

The OpenStack plugin allows users to collect performance metrics from various OpenStack services such as CINDER, GLANCE, HEAT, KEYSTONE, NEUTRON, and NOVA. It supports multiple OpenStack APIs to fetch critical metrics related to these services, enabling comprehensive monitoring and management of cloud resources. As organizations increasingly adopt OpenStack for their cloud infrastructure, this plugin plays a vital role in providing insights into resource usage, availability, and performance across the cloud environment. Configuration options allow for customized polling intervals and filtering unwanted tags to optimize performance and cardinals.

M3DB

This configuration uses Telegraf’s HTTP output plugin with prometheusremotewrite format to send metrics directly to M3DB through the M3 Coordinator. M3DB is a distributed time series database designed for scalable, high-throughput metric storage. It supports ingestion of Prometheus remote write data via its Coordinator component, which manages translation and routing into the M3DB cluster. This approach enables organizations to collect metrics from systems that aren’t natively instrumented for Prometheus (e.g., Windows, SNMP, legacy systems) and ingest them efficiently into M3’s long-term, high-performance storage engine. The setup is ideal for high-scale observability stacks with Prometheus compatibility requirements.

Configuration

OpenStack

[[inputs.openstack]]
  ## The recommended interval to poll is '30m'

  ## The identity endpoint to authenticate against and get the service catalog from.
  authentication_endpoint = "https://my.openstack.cloud:5000"

  ## The domain to authenticate against when using a V3 identity endpoint.
  # domain = "default"

  ## The project to authenticate as.
  # project = "admin"

  ## User authentication credentials. Must have admin rights.
  username = "admin"
  password = "password"

  ## Available services are:
  ## "agents", "aggregates", "cinder_services", "flavors", "hypervisors",
  ## "networks", "nova_services", "ports", "projects", "servers",
  ## "serverdiagnostics", "services", "stacks", "storage_pools", "subnets",
  ## "volumes"
  # enabled_services = ["services", "projects", "hypervisors", "flavors", "networks", "volumes"]

  ## Query all instances of all tenants for the volumes and server services
  ## NOTE: Usually this is only permitted for administrators!
  # query_all_tenants = true

  ## output secrets (such as adminPass(for server) and UserID(for volume)).
  # output_secrets = false

  ## Amount of time allowed to complete the HTTP(s) request.
  # timeout = "5s"

  ## HTTP Proxy support
  # http_proxy_url = ""

  ## Optional TLS Config
  # tls_ca = /path/to/cafile
  # tls_cert = /path/to/certfile
  # tls_key = /path/to/keyfile
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  ## Options for tags received from Openstack
  # tag_prefix = "openstack_tag_"
  # tag_value = "true"

  ## Timestamp format for timestamp data received from Openstack.
  ## If false format is unix nanoseconds.
  # human_readable_timestamps = false

  ## Measure Openstack call duration
  # measure_openstack_requests = false

M3DB

# Configuration for sending metrics to M3
[outputs.http]
  ## URL is the address to send metrics to
  url = "https://M3_HOST:M3_PORT/api/v1/prom/remote/write"

  ## HTTP Basic Auth credentials
  username = "admin"
  password = "password"

  ## Data format to output.
  data_format = "prometheusremotewrite"

  ## Outgoing HTTP headers
  [outputs.http.headers]
    Content-Type = "application/x-protobuf"
    Content-Encoding = "snappy"
    X-Prometheus-Remote-Write-Version = "0.1.0"

Input and output integration examples

OpenStack

1. Cross-Cloud Management: Leverage the OpenStack plugin to monitor and manage multiple OpenStack clouds from a single Telegraf instance. By aggregating metrics across different clouds, organizations can gain insights into resource utilization and optimize their cloud architecture for cost and performance.

2. Automated Scaling Based on Metrics: Integrate the metrics gathered from OpenStack into an automated scaling solution. For example, if the plugin detects that a specific service’s performance is degraded, it can trigger auto-scaling rules to launch additional instances, ensuring that system performance remains optimal under varying workloads.

3. Performance Monitoring Dashboard: Use data collected by the OpenStack Telegraf plugin to power real-time monitoring dashboards. This setup provides visualizations of key metrics from OpenStack services, enabling stakeholders to quickly identify trends, pinpoint issues, and make data-driven decisions in managing their cloud infrastructure.

4. Reporting and Analysis of Service Availability: By utilizing the metrics collected from various OpenStack services, teams can generate detailed reports on service availability and performance over time. This information can help identify recurring issues, improve service delivery, and make informed decisions regarding changes in infrastructure or service configuration.

M3DB

1. Large-Scale Cloud Infrastructure Monitoring: Deploy Telegraf agents across thousands of virtual machines and containers to collect metrics and stream them into M3DB through the M3 Coordinator. This provides reliable, long-term visibility with minimal storage overhead and high availability.

2. Legacy System Metrics Ingestion: Use Telegraf to gather metrics from older systems that lack native Prometheus exporters (e.g., Windows servers, SNMP devices) and forward them to M3DB via remote write. This bridges modern observability workflows with legacy infrastructure.

3. Centralized App Telemetry Aggregation: Collect application-specific telemetry using Telegraf’s plugin ecosystem (e.g., exec, http, jolokia) and push it into M3DB for centralized storage and query via PromQL. This enables unified analytics across diverse data sources.

4. Hybrid Cloud Observability: Install Telegraf agents on-prem and in the cloud to collect and remote-write metrics into a centralized M3DB cluster. This ensures consistent visibility across environments while avoiding the complexity of running Prometheus federation layers.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.