Burrow is a method for monitoring Kafka consumers, specifically for keeping track of consumer status in Apache Kafka. Consumer status is determined by evaluating the consumer’s behavior over a sliding window to determine whether the:

  • consumers are committing offsets
  • offset commits are increasing
  • lag is increasing
  • lag is increasing in a consistent or fluctuating manner

Burrow monitors every consumer group that is committing offsets to either Kafka or Zookeeper, and monitors every topic and partition consumed by those groups.

Why use a Telegraf plugin for Burrow?

Monitoring your Apache Kafka infrastructure is important because it could be your core pipeline for your application’s data, and therefore will help ensure 100% availability. The Burrow Telegraf Plugin can help achieve this by monitoring whether or not the consumers are keeping up with the messages — through giving you a view of the offsets that the consumers are committing and the broker’s state.

You can collect these metrics into your InfluxDB instance and build alerts based on thresholds that you set to help you maintain this critical piece of your application stack.

How to monitor Kafka consumers using the Burrow Telegraf Plugin

Burrow Telegraf Plugin configurations allow you to set response times, limit concurrent connections, filter clusters, consumer groups, and topics.

Key Burrow metrics to use for monitoring

Some of the important Burrow metrics that you should proactively monitor include:

  • For burrow_group (one event per each consumer group) and burrow_partition (one event per each topic partition)
    • status
    • status_code
    • partition_count
    • offset
    • total_lag
    • lag
  • burrow_topic (one event per topic offset)
    • offset

For more information, please check out the documentation.

Project URL   Documentation

Related Resources

Kafka Monitoring Template

Try using the pre-built Kafka Monitoring Template to help you get started quickly.

RabbitMQ Telegraf plugin

This plugin gathers statistics from your RabbitMQ monitoring cluster. It reads metrics from RabbitMQ servers via the RabbitMQ Management Plugin.

ActiveMQ Telegraf Plugin

Use this ActiveMQ Telegraf Plugin to send metrics from the entire stack of your applications and infrastructure to InfluxDB.

Scroll to Top