How ju:niz Energy Improves Renewable Energy Storage with MQTT, Modbus, and InfluxDB Cloud Dedicated

Overview

Bridging telemetry from the edge to the cloud

As a company, ju:niz Energy is a pioneer of decentralized energy in Germany. But the ju:niz Energy engineering team needed new technology to fully support the business’s data needs. Their workload included data originating from multiple edge locations. Edge devices generate this data rapidly and in high volume, and varying degrees of connection reliability created additional consistency challenges for their data. To address these problems, ju:niz Energy implemented InfluxDB OSS and InfluxDB Cloud Dedicated. By using InfluxDB at the edge and in the cloud, ju:niz Energy engineers unlocked Edge Data Replication (EDR) capabilities. EDR provided a secure, durable way to send data from the edge to the cloud despite unreliable connectivity. InfluxDB Cloud Dedicated handled ju:niz Energy’s workload without bottlenecking. InfluxDB Cloud Dedicated’s reliable query performance provided the platform for ju:niz Energy engineers to build a more streamlined workflow.

Challenge

Managing distributed battery energy storage systems without losing visibility

As a company, ju:niz Energy sets the standard for the economical use of energy and the decentral production, storage, and conversion of renewable energy. To successfully achieve their goals, ju:niz Energy engineers need both fresh and historical data. Control reserve and trading are an example of how ju:niz Energy engineers use data to optimize their battery systems. ju:niz Energy engineers also use data to monitor their storage facilities to ensure that environmental conditions such as temperature and humidity are suitable for ongoing operation.

ju:niz Energy’s technical infrastructure was operational but not optimal. To continue growing and leading Germany’s decentralized energy transition, ju:niz Energy engineers needed new technology as innovative as their energy standards.

Technical Challenge

Keeping edge locations and centralized cloud monitoring in sync

ju:niz Energy’s legacy hardware and software systems were unable to keep up with ju:niz Energy’s data needs. Unreliable syncing between edge locations and ju:niz Energy’s centralized monitoring database caused data loss. The lack of storage space, in both edge database instances and the cloud, led to tight retention policies and forced data eviction. These challenges brought ju:niz Energy engineers to their first InfluxDB OSS upgrade, from 1.x to 2.0. At this time, ju:niz Energy engineers also added InfluxDB Cloud v1.

The InfluxDB OSS 2.0 and Cloud v1 upgrade brought both benefits and challenges. Cloud v1’s multi-node setup meant ju:niz Energy engineers finally had a highly available database complete with snapshots and backups of their data. Higher storage capacity meant the end of tight retention policies. The introduction of Edge Data Replication (EDR) introduced a new way for ju:niz Energy engineers to safely, reliably send data from edge locations to the cloud.

But there were also challenges. ju:niz Energy engineers were unable to use the tooling for InfluxDB OSS 2.0 in their older facilities due to the remaining legacy hardware. This meant data syncs depended upon less reliable tools and unstable connections. Because of this, not all of ju:niz Energy’s data made it to the cloud. Even with data occasionally missing, the Cloud v1 cluster bottlenecked regularly, and the CPU load hovered around 70-80%. Though there were other factors leading to the slower performance, such as inconsistent data and the ju:niz Energy code files themselves, this was not the final result the ju:niz Energy engineering teams hoped for.

Even after experiencing the challenges stemming from the first migration, the ju:niz Energy engineering team wanted to continue using InfluxDB products rather than native Postgres or Timescale. ju:niz Energy tested a Postgres database with one of their plants. Not only did the Postgres database bottleneck, but the database also required ju:niz Energy engineers to allocate resources toward optimizing indexes, drivers, and cluster performance. Migrating to a SQL database wasn’t an option.

Shortly after ju:niz Energy added Cloud v1, InfluxDB launched InfluxDB Cloud Dedicated, a fully-managed, single tenant product in the InfluxDB 3.0 product suite. Cloud Dedicated caters to large workloads, such as ju:niz Energy’s.

Solution

Edge-to-cloud InfluxDB deployment

ju:niz Energy engineers began sending data from all its plants to InfluxDB Cloud Dedicated. Because of the more advanced capabilities of the Cloud Dedicated cluster, the plants with legacy hardware were able to send data using EDR. Even while receiving all ju:niz Energy’s fresh data, the Cloud Dedicated cluster kept pace with data ingest while CPU and memory remained at normal levels. There were no spikes or bottlenecks. The process of migrating older data from Cloud 1 and OSS 1.x remains underway and InfluxDB engineers are working on advanced migration tooling.

Architecture

How ju:niz manages BESS telemetry with InfluxDB

The diagram below depicts the architecture and data flow from ju:niz Energy’s newer plants.

ju:niz Energy engineers controls its plants with iEMS SPS controllers. The iEMS SPS controllers hold all ju:niz Energy’s data. This includes data from its plants, such as climate and fire detection data, and batteries. Open source Telegraf, the data collection agent for InfluxDB, connects to the iEMS SPS controller via the Modbus protocol, reads data from the device directly, then writes the data to InfluxDB OSS. Telegraf is a low-code, plugin-based solution that doesn’t have any dependences for, or require modifications to the iEMS SPS. ju:niz Energy engineers also use Telegraf to sample the data in a one-second resolution from their plant server. A single plant can have multiple Telegraf instances, which allows parallel data sampling. Each plant has an instance of InfluxDB OSS running locally. ju:niz Energy sends all data from the local InfluxDB OSS instances to the central, AWS-hosted Cloud Dedicated cluster using EDR.

Thanks to InfluxDB Cloud Dedicated’s high-ratio data compression, ju:niz Energy engineers store all their data in the cloud. Previously, they found it less expensive to store more data locally. InfluxDB Cloud Dedicated performs well regardless of the query size, time range, or number of included fields. ju:niz Energy engineers can now view the results of several queries in a single Grafana dashboard. They can change the technical units, time frames, and/or datapoints they’re querying instantly. Having reliable query performance enabled ju:niz Energy engineers to redefine their Grafana alerting systems.

With InfluxDB Cloud Dedicated’s performant queries, ju:niz Energy engineers have greater context for reducing false positives and more success with setting precision time-based conditions for Grafana alarms. With a reduction in false positives and alarms triggering as soon as events meet specific thresholds, ju:niz Energy replaced their manual support ticket generation process with an automatic system facilitated by Opsgenie. ju:niz Energy’s new incident management workflow reduced the workload of all teams working in its plants and systems and further streamlined their incident management workflow.

Result

Low-cost monitoring without sacrificing visibility

During the construction phase of a new plant, ju:niz Energy engineers needed a temporary, low-cost, easy-to-build, solution to record the climate conditions of battery rooms. Even though this solution was temporary it was important because it monitored the climate surrounding the batteries under warranty. The only additional supplies ju:niz Energy bought were LAN sensors. Using the MQTT protocol, ju:niz Energy engineers sent the sensor data to their local instance InfluxDB OSS using Telegraf. Once in InfluxDB, ju:niz Energy engineers used EDR to transmit the local data to the cloud. For a very low cost, ju:niz Energy engineers built a temporary solution with the same functionality as their permanent builds.