How to Reduce Telegraf Binary Size with a Customized Telegraf Agent

By Jay Clifford / Samantha Wang / Product, Use Cases
Nov 04, 2022

Is Telegraf too big for your device? Too many plugins in one binary getting you down?

Let me introduce you to the Telegraf custom builder – the new tool for reducing Telegraf’s overall memory and disk footprint. In this blog, we will discuss the “what, why, when” and also how to use the new custom builder.

What, why, and when?

What?

The Telegraf custom builder is a tool written in Go and found within the tools directory of the Telegraf repo. The custom builder allows you to select which plugins to build the Telegraf binary with. It takes your Telegraf configuration file as input, scans the configuration files for the plugin files being defined, and builds a new binary including only these plugins (more on this in the how-to section).

Why?

The Telegraf binary has grown substantially over the last few years. The diagram below shows the growth of the Telegraf binary on a Linux amd64 build from versions 1.14.0 to 1.24.0. With this growth, many users have requested ways to build a slimmer Telegraf binary with just the plugins they want to download on their lightweight IoT devices.

Although stripping the Telegraf binary is not a new concept, it’s never been an easy one for the Telegraf community to perform. It required knowledge of the source code and specific build configuration files. The Telegraf custom builder is here to streamline this process for all users. This custom builder also provides scope for Telegraf to be used more performantly on resource-constrained devices, even some embedded devices/controllers which support running the Telegraf binary. The Telegraf custom builder also helps with the future scalability of Telegraf. Naturally, the more plugins the community contributes, the more Telegraf bloats in size (the nature of adding more features over time). This tool provides the scope to build a binary relevant to your use case.

When?

The Telegraf custom builder is a powerful tool to use within the right situations and stages of your solution design. For instance, it makes sense to use the custom builder when you have made a firm decision on which plugins you will be using. The custom builder also makes sense if you are comfortable building the Telegraf binary from its source.

Before you start:

Create your Telegraf configuration with the plugins you want to use. This could be stored locally or remotely. In this blog, we’ll be using the Telegraf configuration.

How to use it

In this section, let’s use the Telegraf custom builder to build Telegraf for this demo: MQTT Emergency Generator Simulator.

0. Before you start Make sure your Go version is up to date. At the time of publishing, this tool requires Go version 1.18.0+. Create your Telegraf configuration with the plugins you want to use. This could be stored locally or remotely. In this blog we’ll be using the Telegraf configuration from the MQTT Emergency Generator Simulator.
1. Clone the Telegraf repo

   git clone https://github.com/influxdata/telegraf.git

2. Build custom builder
   We now need to build the custom builder. To do this open a terminal within the top-level directory of the Telegraf repository and enter the command:

   make build_tools

3. Point the custom builder at your Telegraf configuration(s) that include the plugins you want to build
   Depending on how you have built/stored your Telegraf config(s), you have a few options here:
   • Single Telegraf config

     ./tools/custom_builder/custom_builder --config /etc/telegraf.conf

   • Single Telegraf config + Telegraf config directory

     /tools/custom_builder/custom_builder --config     /etc/telegraf.conf --config-dir /etc/telegraf/telegraf.d

   • Remote Telegraf config

     ./tools/custom_builder/custom_builder --config http://myserver/telegraf.conf

4. For our Emergency Generator demo, we can find the Telegraf configuration here. So let's try out the remote feature with:

   ./tools/custom_builder/custom_builder --config https://raw.githubusercontent.com/InfluxCommunity/InfluxDB-Roadshow-Training/master/telegraf/telegraf.conf

   This will run both phases of the custom builder (configuration and building), resulting in the creation of your customized Telegraf binary.
5. View customized Telegraf binary
   You should now be able to find your new smaller Telegraf binary within the top level of your Telegraf repository.
   The table below gives a comparison of the overall binary size on MacOS:

   MQTT Emergency Generator custom build	Full binary
   MQTT input plugin, InfluxDB_V2 output plugin, JSON_V2 parser	All Plugins
   17.6 MB	207.9 MB

6. Add or remove plugins from your customized Telegraf If you wish to add or remove plugins from your customized Telegraf build, you can just edit your configuration and rerun the `./tools/custom_builder/custom_builder --config path/to/config.conf` command.

Conclusion

With Telegraf 2.0 in development, we are making a conscious effort to provide a Telegraf binary that delivers great functionality on all devices, including those with constrained resources. The Telegraf custom builder provides you with the flexibility to save vital memory and storage space by only bringing the plugins you need to the deployment phase. We are excited to hear your feedback on the new tool. Head over to the InfluxData Slack and Community forums. Let’s continue the discussion there with the rest of the team!