Faster MQTT Data Collection with InfluxDB
By
Jason Myers /
Product, Use Cases
Nov 01, 2022
Navigate to:
This article was originally published in The New Stack and is reposted here with permission.
Native MQTT eliminates the need to write custom code, orchestrate additional technology layers or incorporate additional hosting services.
MQTT is a powerhouse within the Internet of Things (IoT) space. Its pub/sub model and lack of defined payload structure make it infinitely adaptable to the needs of modern sensors, devices and systems. IoT data is also time-series data. Time-stamped data enables businesses and applications to track real-time and historical change, and it can also contribute to forecasting and prediction.
InfluxDB just introduced a new Native Collector feature designed to accelerate collecting from cloud-based pub/sub services. The first integration available for this new feature is the native MQTT collector.
Native MQTT allows InfluxDB users to subscribe to MQTT topics directly from the InfluxDB Cloud interface. The InfluxDB Cloud service then communicates directly with their MQTT broker to immediately begin receiving any message posted to those topics.
Configuring Native MQTT
Native MQTT is easy to set up, and it doesn’t require installing any additional software in your software stack or any custom coding. It’s a true no-code solution. So, let’s take a look at basic configuration, as well as some of the additional parsing options available to you.
To get MQTT data flowing into your InfluxDB instance, Native MQTT requires three basic configuration steps.
- Broker details: Specify the IP address, port and authentication parameters for your MQTT message broker.
- Topic name: Provide the name(s) for the topic(s) you want to subscribe to.
- Parsing rules: Set up parsing rules to map elements in your MQTT messages to the different elements of InfluxDB’s line protocol data model: measurements, timestamps, fields, and tags.
Boom! That’s it.
Data parsing options
When it comes to data parsing, you have several options. MQTT allows for the use of wild cards. The + and # wildcard characters allow you to dynamically subscribe to one or many specific topics on a particular broker. This use of wildcards adheres to the MQTT specification for topic names and topic filters.
You can even combine wildcards with specific write destinations to optimize the workflows for your MQTT brokers. Doing so allows you to dynamically write data from specific topics into specific InfluxDB buckets. For example, you can write all data from energy-related topics to an “Energy” bucket, and all data from weather-related topics to a “Weather” bucket.
Finally, because MQTT allows for unstructured data payloads, the Native MQTT feature allows you to dynamically parse JSON and strong payloads using JSON path expressions and regular expressions, respectively. This means you can mine your MQTT data for timestamp, measurement, field and tag sets regardless of text format.
Typical configuration only takes a few minutes. Once completed, any messages published to your MQTT broker will flow into InfluxDB as time-series data. At that point, you can initiate any necessary queries, visualizations and alerts, or set up additional tasks and transformations to your data using the Flux scripting language.
Conclusion
Native MQTT gives developers a way to collect time-series data in the cloud with a single step. Eliminating the need to write custom code, orchestrate additional technology layers or incorporate additional hosting services means that developers can spend more time actually using their collected data and less time configuring or managing infrastructure.
For users who want or need a truly managed, serverless, cloud-to-cloud solution, Native MQTT provides it.
To learn more about Native MQTT and InfluxDB, visit the InfluxDB Native Collectors page.
Interested in using Python with InfluxDB and MQTT? Check out this Python MQTT tutorial.
