Google Cloud Stackdriver and Google Cloud Monitoring Integration
Powerful performance with an easy integration, powered by Telegraf, the open source data connector built by InfluxData.
5B+
Telegraf downloads
#1
Time series database
Source: DB Engines
1B+
Downloads of InfluxDB
2,800+
Contributors
Table of Contents
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Input and output integration overview
This plugin enables the collection of monitoring data from Google Cloud services through the Stackdriver Monitoring API. It is designed to help users monitor their cloud infrastructure’s performance and health by gathering relevant metrics.
The Stackdriver plugin allows users to send metrics directly to a specified project in Google Cloud Monitoring, facilitating robust monitoring capabilities across their cloud resources.
Integration details
Google Cloud Stackdriver
The Stackdriver Telegraf plugin allows users to query timeseries data from Google Cloud Monitoring using the Cloud Monitoring API v3. With this plugin, users can easily integrate Google Cloud monitoring metrics into their monitoring stacks. This API provides a wealth of insights about resources and applications running in Google Cloud, including performance, uptime, and operational metrics. The plugin supports various configuration options to filter and refine the data retrieved, enabling users to customize their monitoring setup according to their specific needs. This integration facilitates a smoother experience in maintaining the health and performance of cloud resources and assists teams in making data-driven decisions based on historical and current performance statistics.
Google Cloud Monitoring
This plugin writes metrics to a project in Google Cloud Monitoring, which used to be known as Stackdriver. Authentication is a prerequisite and can be achieved via service accounts or user credentials. The plugin is designed to group metrics by a namespace
variable and metric key, facilitating organized data management. However, users are encouraged to use the official
naming format for enhanced query efficiency. The plugin supports additional configurations for managing metric representation and allows tags to be treated as resource labels. Notably, it imposes certain restrictions on the data it can accept, such as not allowing string values or points that are out of chronological order.
Configuration
Google Cloud Stackdriver
[[inputs.stackdriver]]
## GCP Project
project = "erudite-bloom-151019"
## Include timeseries that start with the given metric type.
metric_type_prefix_include = [
"compute.googleapis.com/",
]
## Exclude timeseries that start with the given metric type.
# metric_type_prefix_exclude = []
## Most metrics are updated no more than once per minute; it is recommended
## to override the agent level interval with a value of 1m or greater.
interval = "1m"
## Maximum number of API calls to make per second. The quota for accounts
## varies, it can be viewed on the API dashboard:
## https://cloud.google.com/monitoring/quotas#quotas_and_limits
# rate_limit = 14
## The delay and window options control the number of points selected on
## each gather. When set, metrics are gathered between:
## start: now() - delay - window
## end: now() - delay
#
## Collection delay; if set too low metrics may not yet be available.
# delay = "5m"
#
## If unset, the window will start at 1m and be updated dynamically to span
## the time between calls (approximately the length of the plugin interval).
# window = "1m"
## TTL for cached list of metric types. This is the maximum amount of time
## it may take to discover new metrics.
# cache_ttl = "1h"
## If true, raw bucket counts are collected for distribution value types.
## For a more lightweight collection, you may wish to disable and use
## distribution_aggregation_aligners instead.
# gather_raw_distribution_buckets = true
## Aggregate functions to be used for metrics whose value type is
## distribution. These aggregate values are recorded in in addition to raw
## bucket counts; if they are enabled.
##
## For a list of aligner strings see:
## https://cloud.google.com/monitoring/api/ref_v3/rpc/google.monitoring.v3#aligner
# distribution_aggregation_aligners = [
# "ALIGN_PERCENTILE_99",
# "ALIGN_PERCENTILE_95",
# "ALIGN_PERCENTILE_50",
# ]
## Filters can be added to reduce the number of time series matched. All
## functions are supported: starts_with, ends_with, has_substring, and
## one_of. Only the '=' operator is supported.
##
## The logical operators when combining filters are defined statically using
## the following values:
## filter ::= {AND AND AND }
## resource_labels ::= {OR }
## metric_labels ::= {OR }
## user_labels ::= {OR }
## system_labels ::= {OR }
##
## For more details, see https://cloud.google.com/monitoring/api/v3/filters
#
## Resource labels refine the time series selection with the following expression:
## resource.labels. =
# [[inputs.stackdriver.filter.resource_labels]]
# key = "instance_name"
# value = 'starts_with("localhost")'
#
## Metric labels refine the time series selection with the following expression:
## metric.labels. =
# [[inputs.stackdriver.filter.metric_labels]]
# key = "device_name"
# value = 'one_of("sda", "sdb")'
#
## User labels refine the time series selection with the following expression:
## metadata.user_labels."" =
# [[inputs.stackdriver.filter.user_labels]]
# key = "environment"
# value = 'one_of("prod", "staging")'
#
## System labels refine the time series selection with the following expression:
## metadata.system_labels."" =
# [[inputs.stackdriver.filter.system_labels]]
# key = "machine_type"
# value = 'starts_with("e2-")'
</code></pre>
Google Cloud Monitoring
[[outputs.stackdriver]]
## GCP Project
project = "project-id"
## Quota Project
## Specifies the Google Cloud project that should be billed for metric ingestion.
## If omitted, the quota is charged to the service account’s default project.
## This is useful when sending metrics to multiple projects using a single service account.
## The caller must have the `serviceusage.services.use` permission on the specified project.
# quota_project = ""
## The namespace for the metric descriptor
## This is optional and users are encouraged to set the namespace as a
## resource label instead. If omitted it is not included in the metric name.
namespace = "telegraf"
## Metric Type Prefix
## The DNS name used with the metric type as a prefix.
# metric_type_prefix = "custom.googleapis.com"
## Metric Name Format
## Specifies the layout of the metric name, choose from:
## * path: 'metric_type_prefix_namespace_name_key'
## * official: 'metric_type_prefix/namespace_name_key/kind'
# metric_name_format = "path"
## Metric Data Type
## By default, telegraf will use whatever type the metric comes in as.
## However, for some use cases, forcing int64, may be preferred for values:
## * source: use whatever was passed in
## * double: preferred datatype to allow queries by PromQL.
# metric_data_type = "source"
## Tags as resource labels
## Tags defined in this option, when they exist, are added as a resource
## label and not included as a metric label. The values from tags override
## the values defined under the resource_labels config options.
# tags_as_resource_label = []
## Custom resource type
# resource_type = "generic_node"
## Override metric type by metric name
## Metric names matching the values here, globbing supported, will have the
## metric type set to the corresponding type.
# metric_counter = []
# metric_gauge = []
# metric_histogram = []
## NOTE: Due to the way TOML is parsed, tables must be at the END of the
## plugin definition, otherwise additional config options are read as part of
## the table
## Additional resource labels
# [outputs.stackdriver.resource_labels]
# node_id = "$HOSTNAME"
# namespace = "myapp"
# location = "eu-north0"
Input and output integration examples
Google Cloud Stackdriver
-
Integrating Cloud Metrics into Custom Dashboards: With this plugin, teams can funnel metrics from Google Cloud into personalized dashboards, allowing for real-time monitoring of application performance and resource utilization. By customizing the visual representation of cloud metrics, operations teams can easily identify trends and anomalies, enabling proactive management before issues escalate.
-
Automated Alerts and Analysis: Users can set up automated alerting mechanisms leveraging the plugin’s metrics to track resource thresholds. This capability allows teams to act swiftly in response to performance degradation or outages by providing immediate notifications, thus reducing the mean time to recovery and ensuring continued operational efficiency.
-
Cross-Platform Resource Comparison: The plugin can be used to draw metrics from various Google Cloud services and compare them with on-premise resources. This cross-platform visibility helps organizations make informed decisions about resource allocation and scaling strategies, as well as optimize cloud spending versus on-premise infrastructure.
-
Historical Data Analysis for Capacity Planning: By collecting historical metrics over time, the plugin empowers teams to conduct thorough capacity planning. Understanding past performance trends facilitates accurate forecasting for resource needs, leading to better budgeting and investment strategies.
Google Cloud Monitoring
-
Multi-Project Metric Aggregation: Use this plugin to send aggregated metrics from various applications across different projects into a single Google Cloud Monitoring project. This use case helps centralize metrics for teams managing multiple applications, providing a unified view for performance monitoring and enhancing decision-making. By configuring different quota projects for billing, organizations can ensure proper cost management while benefiting from a consolidated monitoring strategy.
-
Anomaly Detection Setup: Integrate the plugin with a machine learning-based analytics tool that identifies anomalies in the collected metrics. Using the historical data provided by the plugin, the tool can learn normal baseline behavior and promptly alert the operations team when unusual patterns arise, enabling proactive troubleshooting and minimizing service disruptions.
-
Dynamic Resource Labeling: Implement dynamic tagging by utilizing the tags_as_resource_label option to adaptively attach resource labels based on runtime conditions. This setup allows metrics to provide context-sensitive information, such as varying environmental parameters or operational states, enhancing the granularity of monitoring and reporting without changing the fundamental metric structure.
-
Custom Metric Visualization Dashboards: Leverage the data collected by the Google Cloud Monitoring output plugin to feed a custom metrics visualization dashboard using a third-party framework. By visualizing metrics in real-time, teams can achieve better situational awareness, notably by correlating different metrics, improving operational decision-making, and streamlining performance management workflows.
Feedback
Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.
Powerful Performance, Limitless Scale
Collect, organize, and act on massive volumes of high-velocity data. Any data is more valuable when you think of it as time series data. with InfluxDB, the #1 time series platform built to scale with Telegraf.
See Ways to Get Started
Related Integrations
Related Integrations
HTTP and InfluxDB Integration
The HTTP plugin collects metrics from one or more HTTP(S) endpoints. It supports various authentication methods and configuration options for data formats.
View IntegrationKafka and InfluxDB Integration
This plugin reads messages from Kafka and allows the creation of metrics based on those messages. It supports various configurations including different Kafka settings and message processing options.
View IntegrationKinesis and InfluxDB Integration
The Kinesis plugin allows for reading metrics from AWS Kinesis streams. It supports multiple input data formats and offers checkpointing features with DynamoDB for reliable message processing.
View Integration