Stream Processing with InfluxDB

Why InfluxDB for stream processing?

Stream processing unifies applications and analytics by processing data as it arrives, in real-time, and detects conditions within a short period of time from when data is received. The key strength of stream processing is that it can provide insights faster, often within milliseconds to seconds.

With that being said, stream processing naturally fits with time series data, as most continuous data series are time series data. And time series data needs a purpose-built database to ingest, store and process it. This is exactly what InfluxDB is. And this is why, given its high-write throughput and the scalability it allows, InfluxDB suits stream processing.

Providing material and contextual analysis regarding a significant amount of data, coming in at high rates, is difficult. Therefore, big stream processing perspective follows the quest for the optimal trade-off between latency and throughput.

Stream processing, from an information-need perspective, requires considering order, context and responsiveness.

For these three characteristics, state-of-the-art stream processing provides the necessary theoretical foundations. But a language that reaches the right level of expressiveness is needed. This is where Flux, InfluxDB’s scripting and query language, comes in.

Flux is:

• Turing-complete and meets the need, in stream processing, for a programming language that reaches the right level of expressiveness.
• Functional, which makes it inherently declarative; it enables planning before execution and enables logical/physical optimizations.
• Task-driven. The target programs are Continuous Queries that analyze input streams, gluing together applications.
• Portable and testable. It is designed to be shared and easy-to-read.
• Can serve the design of SQL-like, domain-specific languages on top, like InfluxQL.

Components of the InfluxData Platform

Flux

Flux is InfluxData’s new functional data scripting language designed for querying, analyzing, and acting on time series data. Its takes the power of InfluxQL and the functionality of TICKscript and combines them into a single, unified syntax.

InfluxDB

InfluxDB is a time series database designed to handle high-write and query loads. It is an integral component of the TICK Stack. InfluxDB is meant to be used as a backing store for any use case involving large amounts of time-stamped data, including DevOps monitoring, application metrics, IoT sensor data, and real-time analytics.

Kapacitor

Kapacitor is an open source data processing framework that makes it easy to create alerts, run ETL jobs and detect anomalies. Kapacitor is the final piece of the TICK Stack.