StreamNative

Founded by the original creators of Apache Pulsar, StreamNative provides a cloud-native, scalable, and secure messaging and event streaming solution powered by Apache Pulsar. StreamNative Cloud empowers teams and simplifies operations with a fully managed, flexible Pulsar®-as-a-Service offering.

Enriching your streaming data

StreamNative and InfluxData

StreamNative makes it easy to ingest, transform, filter and enrich a large variety of streaming data sources into InfluxDB via native Apache Pulsar connector and MQTT Telegraf Input Plugin. Combined as a team, StreamNative and InfluxDB allow any streaming sources including logs, sensors and events to be routed, buffered, aggregated, multiplexed, transformed and filtered to InfluxDB while also populating any number of other enterprise data stores.

Get Connector

Integrating Apache Pulsar with InfluxDB

StreamNative’s Pulsar to InfluxDB open source connector streams data from Apache Pulsar clusters. It creates a fast, scalable data pipeline to InfluxDB Buckets. The FLiP-InfluxDB connector is a no-code solution to send data. The InfluxDB sink connector pulls messages from Pulsar topics and persists the messages to InfluxDB.

To integrate Apache Pulsar with InfluxDB, you merely need to download the InfluxDB connector to your Pulsar nodes.

Steps:

  • Download Pulsar to InfluxDB connector.
  • Copy the nar to the connectors directory.
  • Create an Influx token.
  • Build a configuration file.
  • Create the influxdb sink pointing to the nar and configuration file utilizing REST endpoint or command line interface.
  • Send data.
  • Visualize in InfluxDB Cloud.

For a demo of how to filter and enrich data from Apache Pulsar to InfluxDB, check out this Pulsar Function that outputs Line Protocol data.

StreamNative -InfluxDB-CPU

StreamNative-InfluxDB- Data Explorer

Key resources

Unified messaging and streaming to InfluxDB

StreamNative provides a platform for ingesting data from a variety of sources, filtering, routing and transforming them in real-time to feed InfluxDB via its no-code data sink.

StreamNative can also act as a source for InfluxDB to read from with Telegraf via the MQTT protocol for seamless IoT integration.

StreamNative and Apache Pulsar are typically used to ingest, route, transform and enrich data streams for devices, databases, events, logs, sensors and REST endpoints. This data can be streamed one event at a time or in bulk to InfluxDB depending on the client’s needs.

End-to-End straming Edge App

Streaming Edge App

Typical Dashboard

Typical Dashboard

Typical Dashboard

VIDEO

Using FLiPN with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021

Timothy Spann from StreamNative leads this video walkthrough about how his team ran deep learning models, and capture real-time GPS and sensor data on edge devices. They used Pulsar, Apache NiFi + Edge Flow Manager + MiniFi Agents with Apache MXNet, OpenVino, TensorFlow Lite, and other Deep Learning Libraries on edge devices including Raspberry Pi with Movidius 2, Google Coral TPU and NVidia Jetson Nano. Their low-coding IoT applications provided easy edge routing, transformation, data acquisition and alerting before they decide what data to stream real-time to their data space. These edge applications classified images and sensor readings real-time at the edge and then sent Deep Learning results to Flink SQL and Apache NiFi for transformation, parsing, enrichment, querying, filtering and merging data to InfluxDB.

Available as InfluxDB Open Source, InfluxDB Cloud & InfluxDB Enterprise

Compare