Making the Most of MQTT – Native Collector or Telegraf?

When it comes to IoT data, MQTT is a superstar. With so many IoT devices generating data out in the world, developers need ways to access it. After all, data lies at the heart of every application. But data doesn’t just magically manifest itself into your datastore, and building the right data pipeline can make or break an application. Data collection is not a one-size-fits-all problem to solve. Developers need a lot of flexibility when it comes to building these solutions so they can account for the full range of data sources generating the data they need.

InfluxDB offers half-a-dozen different ways to collect data. The newest option is the native collector feature. Perhaps the most popular option is Telegraf, InfluxData’s open source, plugin-based data collection agent. So, when it comes to MQTT, when would you want to use the native collector and when would you use Telegraf?

The Native MQTT collector

Native collector is a feature in InfluxDB Cloud 2.0 that allows you to create direct, cloud-to-cloud connections between cloud services and InfluxDB. In the case of the Native MQTT collector integration, you configure InfluxDB to function as a MQTT client that subscribes directly to MQTT brokers.

Sound complicated? It’s not.

Using Native MQTT involves a simple, three-step configuration process. You just need to input the information for your MQTT broker, identify the topics you want to subscribe to and the buckets where you want to send that data in InfluxDB, and define your parsing rules.

Once you do that, InfluxDB will immediately begin to collect data from your MQTT broker. So, make sure you’re ready to start data ingestion when you finish the configuration process.

This cloud-to-cloud data pipeline is ideal for cloud-first solutions. Some applications don’t have the resources to install and run a collection agent like Telegraf. Sure, you could create a work-around, like running a virtual machine or creating a local solution but both options involve an intermediary layer that could affect costs, eat up development cycles, and introduce potential points of failure to your application.

Native MQTT does data collection and it does it really well. But there are other users that may want or need more flexibility when dealing with MQTT brokers. That’s why Telegraf coupled with the MQTT plugin remains a valuable solution.

MQTT and Telegraf

The Telegraf MQTT plugin makes it easy to collect your MQTT data without the need to write a custom script to connect to your MQTT broker. Telegraf also gives you extensive control to process and transform MQTT messages as they arrive.

Telegraf’s plugin system gives you a lot of options when it comes to collecting data. You can use the same agent to collect data from different protocols or sources, not just MQTT. It has many easy-to-configure processor and aggregator plugins that allow you to clean your data before it ever hits your datastore. Telegraf also has output plugins, so you can send your clean MQTT data to InfluxDB and to any other location where you need that data, simultaneously.

Unlike the Native Collector, you need to have a place to install Telegraf, but we designed it to be extremely lightweight and to use minimal resources. So, if you have the space, Telegraf is very robust.

Final thoughts

It’s tempting to pit native collector against Telegraf, but, using MQTT as an example, we can see that these are simply two different data collection methods, each with their own benefits. In our commitment to meet developers where they are, native collector and Telegraf both provide access to MQTT data. Neither one is necessarily right or wrong. Instead, it’s really up to you – the developer – to determine which approach is better for you based on the specific application needs and your available resources.