Stream Processing with InfluxDB

Stream processing — the processing of data in real-time — is critical in many scenarios, such as Industry 4.0, application performance monitoring (APM) and infrastructure monitoring. Often, significant volumes of data come in at high rates and applications must process data quickly and efficiently, without neglecting to combine them with the domain knowledge that is necessary to solve the user’s information needs.

Stream processing unifies applications and analytics by processing data as it arrives, in real-time, and detects conditions within a short period of time from when data is received. The key strength of stream processing is that it can provide insights faster, within seconds or even milliseconds.

In order to do stream processing, two main components are necessary: data stream management and complex event processing. Respectively, these handle consuming and producing streaming data, and detecting relevant patterns in incoming streams.

Stream processing naturally fits with time series data, as most continuous data series are time series data. As such, streaming data processing is a type of time series data processing. In order to do both properly, you’ll need a purpose-built database to ingest, store and process it. This is exactly what InfluxDB is: an open source, purpose-built time series database. And this is why, given its high-write throughput and the scalability it allows, InfluxDB suits stream processing.