How Teréga Replaced Legacy Data Historians with InfluxDB, AWS, and IO-Base

Overview

From Digital Transformation to New SaaS Offering

Teréga modernized its legacy on-premises IT system to better suit its changing business model. They did so by building a cloud-native data historian, IO-Base, with the InfluxDB time series database as the foundation. Teréga also created Indabox, a proprietary gateway that connects to machine sensors and control devices to collect, process, and assemble data for secure transfer to IO-Base. Teréga found the addition of real-time metrics, unrestricted data ingestion, and better ecosystem integration so successful that it made IO-Base available as a SaaS platform, through its subsidiary, Teréga Solutions, so other companies can also experience its benefits.

Challenge

Terega’s New Multi-Energy Business Model Required a New Digital Foundation

The energy sector is experiencing major upheaval and facing unprecedented challenges from climate change and the move towards a decarbonized energy system. While the reliance on fossil fuels shrinks, energy demands remain the same. To solve this, Teréga created a mixed, multi-energy business model. Adding biomethane and hydrogen to more traditional energy sources allowed them to optimize the use of fossil fuels, and make good use of the link between energy and gas.

To accomplish their goals and achieve energy transition objectives at territorial, national, and European levels, Teréga needed new tech. Being ready on the front lines wasn’t enough. They also had to be ready on the digital side.

Technical Challenge

Legacy Data Historians Could Not Meet Cloud-Native Requirements

Teréga’s technical infrastructure was an elaborate system of legacy on-prem servers and firewalls with limited cloud resources. Each industrial site had its own server system designed to collect the massive amounts of time series data flowing from their operational technology’s sensors and industrial control devices. An intricately organized series of firewalls, or in some places even tougher demilitarized zones, sat between the on-prem and cloud elements. This high level of security was necessary because Teréga manages critical French energy systems. However, these security measures created longer data transit times, which impacted real-time data ingestion and analysis. In addition to slow ingest rates, the legacy system was difficult and expensive to maintain, and made building new projects challenging.

But Teréga tried to bring their on-prem infrastructure and legacy historian, OSI Pi, into the future. With the goal of becoming a cloud-native organization, they attempted to replicate their database in the cloud. Their first attempt was a full replica, followed by a more precise “cherry picked” version. Neither were successful at transitioning Teréga to a cloud-native organization. Instead of continuing to try to retrofit their old solution to the cloud, Teréga chose to find a new system.

Teréga’s team of engineers surveyed the market in search of a cloud-native data historian to suit their industrial needs. Their search didn’t yield fruitful results. Of the few cloud-native data historians in the marketplace, none were specialized for industrial data. Teréga needed a high-performing, scalable, resilient solution to handle their massive amount of time series data. They decided to build their own historian. For the foundation, they turned to InfluxDB.

Solution

Secure Cloud-Native Historian Built on InfluxDB

Teréga replaced its entire on-prem IT infrastructure with Indabox, a new data collection tool they built to meet their specific needs. Indabox creates a unidirectional data flow, which ensures hackers and bad actors can’t connect to their critical infrastructure. Indabox is now the sole piece of IT equipment needed at their industrial sites. It connects directly to the sensors and machine controllers through various protocols (Modbus, Ethernet/IP, OPC-UA, and others). Indabox collects, processes, and sends the data securely and directly to the cloud. In the event of an internet outage, a local cache stores data until Indabox can reconnect and send the stored data to the cloud.

Teréga replaced OSI Pi with InfluxDB Cloud as the foundation for its custom historian. InfluxDB Cloud’s fully managed cloud infrastructure allowed Teréga to leverage their engineers’ expertise with database deployments and InfluxDB’s ability to manage the database instances. InfluxDB’s ability to create metrics from raw event data in real-time during ingestion really set the database apart from others on the market. Real-time metrics simplified deployment and support for over 100k metrics. The metrics include details on processes related to pipeline management (pressure, gas quality, valve status), machine sensors (electricity consumption, vibration, process timing), more general categories (temperature, electricity and gas meters), among other areas. Other qualities such as excellent integration with other tools and software systems, like Grafana, and an established position in the market led to additional confidence in terms of longevity and ongoing support.

Teréga hosts InfluxDB on Amazon Web Services (AWS) because its robust, reliable infrastructure provides the high availability and scalability Teréga requires. AWS also provides strong security measures, including encryption, access control, and monitoring for the protection and privacy of data.

IO-Base is the hardware/software system Teréga created. After the successful completion of IO-Base, Teréga now has a fully cloud-native data historian specifically designed to operate and monitor critical industrial activities. IO-Base is highly available and runs 24/7. Indabox is the only technology needed to connect the onsite operational technology stack and IO-Base in the cloud. Indabox connects to the machine sensors and industrial controls through required protocols. Indabox then transmits the data securely via HTTPS from the industrial site to the API in JSON format in a unidirectional site-to-cloud data flow.

Though InfluxDB is highly secure, Teréga layered supplementary security measures to protect France’s critical services. Teréga engineers made the database inaccessible from the outside by creating private networking and building another layer between the database and their API, which is only accessible via a secure transit gateway. Teréga enhanced authentication using OAuth2.

By creating a cache for recent data, the Redis integration enables faster access for real-time visualization, data transformations, and alerts, which contribute to the overall processing efficiency of IO-Base. Teréga also modified InfluxDB’s standard data hierarchy by leveraging DynamoDB. Adding this layer on top of InfluxDB allows IO-Base’s end users to dynamically reorganize their data without making any modifications to their InfluxDB instance.

Teréga engineers built IO-Base with a variety of user interfaces, with selectable characteristics based on user preference. These front-end applications enable data retrieval, visualizations, user management, and the creation of new metrics using formulas. Teréga built these front-end applications with Angular

Result

50% Lower TCO and 12x More Frequent Data Collection

IO-Base paired with Indabox yielded several positive results for Teréga. The global Total Cost of Ownership for time series data is 50% lower than with Teréga’s legacy system. IO-Base allows Teréga to create new projects and onboard new customers with ease. Rather than having to build an intricate system of servers and firewalls for every new user, all a new customer or project site needs to get started is an Indabox unit. Indabox and IO-Base, paired with InfluxDB, created an environment where vital data is available, structured, and ready for use.

Because InfluxDB has no data ingestion restrictions or limits, Teréga collects data more frequently. With the old system, Teréga collected data at one-minute intervals. The new system, however, collects data every five seconds. This provides deeper insights, exposing issues previously undetectable. Teréga, once unable to detect and therefore troubleshoot and solve any issues lasting less than 60 seconds, no longer faces that challenge thanks to InfluxDB’s ingest speeds. InfluxDB also offers high-level support. InfluxDB’s support team helped Teréga with data migration as they set up their new solution

IO-Base software-as-a-service (SaaS) platform

From Internal Platform to New SaaS Offering

Teréga’s experience with IO-Base and Indabox were so positive that it made both available for third party use through its subsidiary, Teréga Solutions. Indabox is available for purchase and IO-Base is now a SaaS platform. In addition to the tools and templates Teréga uses in the energy sector, they built tools and templates that fit with other business sectors as well.