Summary

We recently sat down with founder and principal consultant at recultiv8, Coenraad Pretorius, who drew on his years of data engineering experience in the manufacturing and energy sectors to share key industrial IoT insights. In this article, I list the top takeaways; you can also watch the full webinar recording here.

The challenge with traditional data historians

Industrial systems generate large volumes of time series data from machines, sensors, and control systems. Historically, this data has been managed using proprietary data historian platforms.

These systems often lead to the following challenges:

  • Complexity: Traditional stacks involve many tightly coupled components: SCADA systems, OPC servers, historians, data extraction tools, and analytics layers. Each layer requires specialized skills, making systems difficult to debug, extend, or modernize.
  • High cost: Per-tag licensing, annual maintenance fees, and specialized training significantly increase the total cost of ownership, particularly as systems scale.
  • Slow time to insight: Extracting and analyzing data often takes days or weeks, delaying decisions and limiting optimization opportunities.
  • The analytics gap: Traditional historians prioritize data storage, not data analysis. Common pain points include proprietary query languages, reliance on Excel exports, overloaded BI integrations, and additional licensing for advanced features. As a result, time to action is measured in days or weeks rather than hours, which is an unacceptable delay for modern industrial operations.

Data Historian Technical Architecture

Data Historian Traditional Architecture

A modern, open architecture: edge + cloud

To address these limitations, Coenraad presented a modern architecture built around InfluxDB 3, open source tooling, and cloud analytics. The core idea is a clear separation of responsibilities that leads to improved performance, security, cost efficiency, and scalability while keeping systems simpler and easier to operate.

  • Edge systems handle real-time ingestion, short-term storage, and operational dashboards close to the data source.
  • Cloud systems handle long-term storage, historical analysis, and advanced analytics without impacting operational performance.

Modern IIoT Technical Architecture

Modern Stack Overview

Example from Coenraad’s case study

Typical deployment setup

  • Four OPC UA servers
  • 10k+ tags
  • Windows-based servers
  • Telegraf running as Windows service

Configuration approach

  • Split config files (agent, inputs, outputs)
  • Custom Starlark processor for schema management
  • Environment variables for cloud credentials

Result: Rapid implementation of the modern data stack using open source solution resulted in saving $70k (once off) and $5 (annually).

Why this approach works

This modern stack delivers several practical benefits:

  • Simpler systems built with familiar tools like SQL and Python that most developers are familiar with.
  • Faster dashboards move from multi-second load times to near instant response as detailed in this blog post.
  • Lower costs are incurred by replacing proprietary licensing with open source and consumption-based services.
  • Flexible data pipelines use Telegraf to ingest data from industrial protocols such as OPC UA, MQTT, and Modbus into InfluxDB Core with optional streaming to the cloud.

Recap

The difference is fairly cut and dry: traditional data historians often limit agility and slow down insights, while modern industrial data stacks focus on speed, openness, and maintainability by separating edge operations from cloud analytics and using familiar, developer-friendly tools. For industrial and IIoT teams, modernizing the data pipeline is now foundational. To learn more, read the Teréga case study and connect with our community in the InfluxDB forums.

FAQ

What exactly is a "data historian," for readers unfamiliar with industrial systems?
A data historian is a specialized database designed to capture and store time-stamped process data from sensors, PLCs, and control systems, typically sampled at intervals from milliseconds to minutes. Unlike a general-purpose database, it's built to handle very high write volumes. Historians have been the standard tool in manufacturing, energy, and process industries for decades, largely because they predate general-purpose time series databases.

What is per-tag licensing, and why does it get so expensive at scale?
Per-tag licensing charges based on the number of individual data points (tags) a system monitors, so cost scales directly and often steeply with plant size. Enterprise deployments with hundreds of thousands of tags can push licensing costs well past six figures annually. This is a big part of why total cost of ownership balloons as industrial operations add sensors and expand monitoring over time.

What is OPC UA, and why does it show up so often in industrial data collection?
OPC UA (Open Platform Communications Unified Architecture) is an open, vendor-neutral protocol for exchanging real-time data between industrial devices, SCADA systems, and software platforms. It's become a de facto standard for interoperability in manufacturing and process industries because it lets equipment from different vendors communicate through one consistent interface.

What's the practical difference between MQTT and OPC UA as protocols for collecting industrial data?
OPC UA is generally used for structured, semantic-rich data exchange directly with industrial control systems and historians, with built-in security and data modeling features. MQTT, by contrast, is a lightweight publish/subscribe messaging protocol originally designed for constrained IoT devices and unreliable networks, prioritizing low overhead over rich data modeling. Many modern architectures use both, since OPC UA handles core plant floor systems while MQTT often covers more distributed or resource-constrained sensors.