Google Cloud Storage and SigNoz Integration

Input and output integration overview

The Google Cloud Storage plugin collects metrics from specified Google Cloud Storage buckets, providing insight into storage usage and performance.

This configuration turns any Telegraf agent into a Remote Write publisher for SigNoz, streaming rich metrics straight into the SigNoz backend with a single URL change.

Integration details

Google Cloud Storage

The Google Cloud Storage Telegraf plugin enables the collection of metrics from specified Google Cloud Storage buckets. As organizations increasingly rely on cloud storage solutions for their data management, the ability to monitor the performance and utilization of these resources becomes essential. This plugin is particularly useful for tracking how storage is used, understanding data patterns, and ensuring operational efficiency. By integrating with Google Cloud Storage APIs, it allows users to gather insights from their cloud environments, feeding metrics directly into monitoring systems for further analysis. The plugin supports various configuration options, enabling users to customize the data collection process based on their specific needs.

SigNoz

SigNoz is an open source observability platform that stores metrics, traces, and logs. When you deploy SigNoz, its signoz-otel-collector-metrics service exposes a Prometheus Remote Write receiver (default :13133/api/v1/write). By configuring Telegraf’s Prometheus plugin to point at this endpoint, you can push any Telegraf collected metrics, SNMP counters, cloud services, or business KPIs—directly into SigNoz. The plugin natively serializes metrics in the Remote Write protobuf format, supports external labels, metadata export, retries, and TLS or bearer-token auth, so it fits zero-trust and multi-tenant SigNoz clusters. Inside SigNoz, the data lands in ClickHouse tables that back Metrics Explorer, alert rules, and unified dashboards. This approach lets organizations unify Prometheus and OTLP pipelines, enables long-term retention powered by ClickHouse compression, and avoids vendor lock-in while retaining PromQL-style queries.

Configuration

Google Cloud Storage

[[inputs.google_cloud_storage]]
  bucket = "my-bucket"
  # key_prefix = "my-bucket"
  offset_key = "offset_key"
  objects_per_iteration = 10
  data_format = "influx"
  # credentials_file = "path/to/my/creds.json"

SigNoz

[[outputs.prometheusremotewrite]]
  ## SigNoz OTEL-Collector metrics endpoint (Prometheus Remote Write receiver)
  ## Default port is 13133 when you install SigNoz with the Helm chart
  url = "http://signoz-otel-collector-metrics.monitoring.svc.cluster.local:13133/api/v1/write"

  ## Add identifying labels so you can slice & dice the data later
  external_labels = { host = "${HOSTNAME}", agent = "telegraf" }

  ## Forward host metadata for richer dashboards (SigNoz maps these to ClickHouse columns)
  send_metadata = true

  ## ----- Authentication (comment out what you don’t need) -----
  # bearer_token   = "$SIGNOZ_TOKEN"          # SaaS tenant token
  # basic_username = "signoz"                 # Basic auth (self-hosted)
  # basic_password = "secret"

  ## ----- TLS options (for SaaS or HTTPS self-hosted) -----
  # tls_ca                  = "/etc/ssl/certs/ca.crt"
  # tls_cert                = "/etc/telegraf/certs/telegraf.crt"
  # tls_key                 = "/etc/telegraf/certs/telegraf.key"
  # insecure_skip_verify    = false

  ## ----- Performance tuning -----
  max_batch_size = 10000      # samples per POST
  timeout        = "10s"
  retry_max      = 3

Input and output integration examples

Google Cloud Storage

1. Automated Backup Monitoring: Utilize the Google Cloud Storage plugin to regularly monitor the status of backup files stored in a Cloud Storage bucket. By configuring the plugin to track file metrics, organizations can automate alerts if backup sizes deviate from expected patterns, ensuring that data protection processes are functioning properly and any anomalies are promptly addressed.

2. Cost Optimization Insights: Integrate this plugin into a cost management tool to analyze the usage patterns of Cloud Storage. By collecting metrics on file sizes and access frequencies, teams can optimize their storage solutions and make informed decisions about data retention policies, potentially reducing unnecessary storage costs and improving resource allocation.

3. Compliance and Auditing: Use the plugin to generate metrics that aid in compliance verification for data stored in Google Cloud Storage. By providing detailed insights into data access and storage usage, organizations can ensure adherence to regulatory requirements, helping in audits and aligning with best practices for data governance.

4. Performance Benchmarking: Deploy the plugin to benchmark the performance of data retrieval and storage operations in Google Cloud Storage. By analyzing metrics over time, teams can identify performance bottlenecks or inefficiencies, allowing them to optimize their applications and infrastructure that depend on cloud storage services.

SigNoz

1. Multi-Cluster Federated Monitoring: Drop a Telegraf DaemonSet into each Kubernetes cluster, tag metrics with cluster=<name>, and Remote Write them to a central SigNoz instance. Ops teams get a single PromQL window across prod, staging, and edge clusters without running Thanos sidecars.

2. Factory-Floor Edge Gateway: A rugged Intel NUC on the shop floor runs Telegraf to scrape Modbus PLCs and environmental sensors. It batches readings every 5 seconds and pushes them over an intermittent 4G link to SigNoz SaaS. ClickHouse compression keeps costs low while AI-based outlier detection in SigNoz flags overheating motors before failure.

3. SaaS Usage Metering: Telegraf runs alongside each micro-service, exporting per-tenant counters (api_calls, gigabytes_processed). Remote Write streams the data to SigNoz where a scheduled ClickHouse materialized view aggregates usage for monthly billing—no separate metering stack required.

4. Autoscaling Feedback Loop: Combine Telegraf’s Kubernetes input with the Remote Write output to publish granular pod CPU and queue-length metrics into SigNoz. A custom SigNoz alert fires when P95 latency breaches 200 ms and a GitOps controller reads that alert to trigger a HorizontalPodAutoscaler tweak—closing the loop between observability and automation.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.