{"version":"https://jsonfeed.org/version/1","title":"InfluxData Blog","home_page_url":"https://www.influxdata.com/blog/","feed_url":"https://www.influxdata.com/blog/feed.json","description":"The place for technical guides, customer observability \u0026 IoT use cases, product info, and news on leading time series platform InfluxDB, Telegraf, SQL, \u0026 more.","items":[{"id":"https://www.influxdata.com/blog/core-ai-user-mumu","url":"https://www.influxdata.com/blog/core-ai-user-mumu","title":"How Mumu Migrated From Prometheus to InfluxDB and Tripled Their Metric Coverage","content_html":"\u003cp\u003eWhen a team uses an internal Slack channel for everything from contact form submissions to deployment alerts and server warnings, the notification engine quickly becomes critical infrastructure. When the same team builds that engine as a product for other teams to use, the bar gets even higher.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://usemumu.com/\"\u003eMumu\u003c/a\u003e is an all-in-one productivity platform for modern teams. While most companies stitch together separate SaaS tools for org charts, agile estimation, internal Q\u0026amp;A, skill mapping, recognition, and notifications, Mumu offers all of those as connected modules under a single subscription. The premise is that your organizational structure shouldn’t be replicated across five different databases; it should live in one place and flow into every workflow your team uses.\u003c/p\u003e\n\n\u003cp\u003eIn this blog, we will go over why the Mumu team rebuilt their monitoring stack on \u003ca href=\"https://www.influxdata.com/products/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=core-ai-user-mumu\u0026amp;utm_content=blog\"\u003eInfluxDB 3\u003c/a\u003e and how the migration went.\u003c/p\u003e\n\n\u003ch2 id=\"why-pull-based-monitoring-stopped-making-sense\"\u003eWhy pull-based monitoring stopped making sense\u003c/h2\u003e\n\n\u003cp\u003eLike many teams running their own infrastructure, Mumu started off using Prometheus as its primary monitoring solution. The main problem over time was the fundamental mismatch between Prometheus’s pull-based data collection model and the type of data Mumu was working with. Rather than telemetry data that can be scraped at a regular interval, Mumu often needs to track discrete events like user triggered actions, scripts completing, and a pipeline finishing. As a result, push-based delivery for tracking events made more sense.\u003c/p\u003e\n\n\u003cp\u003eThat architectural mismatch wasn’t the only problem. As Mumu evaluated alternatives like Betterstack, VictoriaMetrics, PostHog, Graphite, Datadog, and New Relic, issues related to transparency became a concern. Several of the SaaS solutions came with documentation that made it genuinely hard to understand what was happening under the hood, particularly around how metrics were ingested and stored. For a team that ships fast and needs to be able to debug its own pipeline, that was a dealbreaker.\u003c/p\u003e\n\n\u003ch2 id=\"why-influxdb-3-was-the-right-option\"\u003eWhy InfluxDB 3 was the right option\u003c/h2\u003e\n\n\u003cp\u003eTwo things about InfluxDB stood out during evaluation. The first was that self-hosting was effortless. Mumu runs its own dedicated servers, and spinning up an InfluxDB 3 Core instance using the official Docker image took almost no time or configuration overhead.\u003c/p\u003e\n\n\u003cp\u003eThe second factor was the push-based HTTP API. \u003ca href=\"https://docs.influxdata.com/influxdb3/core/api/\"\u003eInfluxDB’s Line Protocol HTTP API\u003c/a\u003e lets Mumu’s services emit metrics at the exact line of code where an event occurred with no sidecar, no exposition format, no scrape interval, just a POST request at the moment an event happened.\u003c/p\u003e\n\n\u003cp\u003eIn hindsight, the team’s biggest evaluation lesson was that they should have built a small proof of concept with InfluxDB earlier. The time spent evaluating other tools wasn’t wasted as it gave them context and confidence in the final decision, but InfluxDB’s simplicity would have been apparent within an afternoon.\u003c/p\u003e\n\n\u003ch2 id=\"migration-process\"\u003eMigration process\u003c/h2\u003e\n\n\u003cp\u003eThe migration involved three phases over a 3-month period: dual writing to InfluxDB and the existing Prometheus setup, validation, and finally, decommissioning the Prometheus infrastructure.\u003c/p\u003e\n\n\u003ch4 id=\"phase-1-dual-writing-via-vector\"\u003ePhase 1: Dual-Writing via Vector\u003c/h4\u003e\n\n\u003cp\u003eThe first move was to make the same metrics flow into both systems at once. Mumu added InfluxDB as a second sink alongside Prometheus in their existing Vector pipeline, so every metric was being written to both simultaneously. That parallel run is what made the eventual cutover risk-free by allowing the team to confirm performance and validate both systems against each other.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eMigrating to InfluxDB is made easy using AI agents.\u003c/strong\u003e\u003c/p\u003e\n\n\u003cp\u003eAbout 80% of the migration work, such as Vector configuration changes, the dual-write sink setup, and the boilerplate around the new HTTP drivers in Go and TypeScript, was  generated by coding agents.\u003c/p\u003e\n\n\u003cp\u003eA migration becomes a lot less daunting when the routine work compresses into hours, but what really made this work was what the agent had to work with on the InfluxDB side. InfluxDB 3 exposes a full REST API with a published \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/api/\"\u003eOpenAPI specification\u003c/a\u003e. When an AI agent can read that contract directly, it doesn’t have to guess at parameter names from stale blog posts or hallucinate endpoint shapes from vague documentation. It reads the spec, generates correct client code, and gets the integration right on the first pass. InfluxDB also has an \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/admin/mcp-server/\"\u003eMCP server\u003c/a\u003e for integrating with AI agents, although it wasn’t used by Mumu.\u003c/p\u003e\n\n\u003cp\u003eThis is an important property in a world where agents are doing more and more of the integration work. The systems that will be easiest to adopt over the next few years are not necessarily the ones with the most features, they’re the ones whose APIs are legible to machines.\u003c/p\u003e\n\n\u003ch4 id=\"phase-2-validation\"\u003ePhase 2: Validation\u003c/h4\u003e\n\n\u003cp\u003eRunning two systems in parallel only helps if you actually compare them, and this is where the team spent its caution wisely. The validation approach was deliberately simple: they duplicated their Grafana panels side by side, with one panel pulling from Prometheus and an identical panel pulling from InfluxDB. When two panels showing the same metric look identical for weeks on end, confidence accumulates quickly.\u003c/p\u003e\n\n\u003cp\u003eBeyond visual parity, four things got specific attention:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eRetention policies\u003c/strong\u003e: Confirming data was being stored at the right granularity and for the expected duration.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eTag cardinality\u003c/strong\u003e: Making sure the tagging strategy wouldn’t cause write-performance problems at scale. Keeping cardinality low on high-volume metric streams is a lesson the team internalized early.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eBatch write behavior\u003c/strong\u003e: Validating that the NestJS batching logic produced correct time series data with no gaps or duplicates.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eDashboard parity\u003c/strong\u003e: Rebuilding key Grafana dashboards from scratch against InfluxDB to confirm they told the same story as their Prometheus equivalents.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch4 id=\"phase-3-cutover\"\u003ePhase 3: Cutover\u003c/h4\u003e\n\n\u003cp\u003eBecause Mumu’s Go and TypeScript codebases already had proper abstractions and interfaces for metric delivery, writing a new driver that sent metrics to InfluxDB via the HTTP API required almost no changes to the application code. The abstraction layer in their app code meant that swapping the metrics backend was a contained, well-scoped task rather than a sprawling refactor.\u003c/p\u003e\n\n\u003cp\u003eThe TypeScript driver came in at around 160 lines of code, largely because the team leaned on the official InfluxDB 3 client library package. The Go implementation was slightly longer due to manual HTTP handling, retry logic, and error handling, but was still a straightforward, bounded piece of work. Once the drivers were in place, the team decommissioned Prometheus for business metrics and declared the migration complete.\u003c/p\u003e\n\n\u003ch2 id=\"benefits-of-influxdb-3from-150-to-560-metrics\"\u003eBenefits of InfluxDB 3—from 150 to 560 metrics\u003c/h2\u003e\n\n\u003cp\u003eBefore InfluxDB, Mumu collected around 150 metrics. Today, they collect 560 metrics, and that number is constantly increasing.\u003c/p\u003e\n\n\u003cp\u003eThat growth didn’t come from a dedicated instrumentation initiative. There was no mandate, no quarter-long observability push, it happened organically because \u003cstrong\u003eadding a new metric became a one-line HTTP call\u003c/strong\u003e. When friction drops that far, engineers instrument things they would previously have skipped.\u003c/p\u003e\n\n\u003cp\u003eThe number is less a measure of throughput than a measure of how much the team’s relationship with its own data changed once the cost of asking a question fell to nearly zero. And because metric delivery was suddenly cheap, Mumu started instrumenting things that would have seemed impractical before:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eOperational automation\u003c/strong\u003e: Mumu sends a metric for every command executed on their servers, with automations built on top using MsgGO, and certain commands automatically trigger a Slack alert. The result is passive visibility into operational activity with no manual reporting required.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eCI/CD observability\u003c/strong\u003e: They emit metrics from their Bitbucket pipelines, including how long each pipeline runs. Over time, this has established a baseline for normal build duration, making regressions easy to spot.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003ePer-developer environments\u003c/strong\u003e: Every developer tags their metrics with an \u003ccode class=\"language-markup\"\u003eenv\u003c/code\u003e field set to their local environment name, such as \u003ccode class=\"language-markup\"\u003elocal:john\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003elocal:kate\u003c/code\u003e, and so on. Each developer can observe their own environment in Grafana, test new instrumentation locally before it ships, and build personal dashboards, all without polluting shared production data.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eGeneral script instrumentation\u003c/strong\u003e: Bash scripts, custom CLI commands, and database migration durations during deployments are all now tracked, where before each would have demanded disproportionate effort.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"sql-on-time-series-data\"\u003eSQL on time series data\u003c/h2\u003e\n\n\u003cp\u003eVolume was not the only shift. InfluxDB 3’s SQL support meaningfully improved Mumu’s ability to build dashboards and debug metric data. Before the migration, querying time series data meant learning a specialized query language and reasoning about its particular semantics. With SQL, any engineer on the team can write an ad-hoc query to investigate a metric anomaly, validate that a new event is being tracked correctly, or prototype a Grafana panel without consulting documentation.\u003c/p\u003e\n\n\u003cp\u003eThe qualitative win is harder to put a number on, but is equally important: the metrics are now trusted. Engineering time that used to go into questioning whether a dashboard was telling the truth now goes into acting on what it shows.\u003c/p\u003e\n\n\u003ch2 id=\"architecture-overview\"\u003eArchitecture overview\u003c/h2\u003e\n\n\u003cp\u003eMumu runs entirely on its own dedicated servers, giving the team full control, room for hardware-level optimization, and predictable costs. The application layer runs on Docker, with services written in Go and NestJS; Go handles core infrastructure-level work while NestJS handles application layer operations.\u003c/p\u003e\n\n\u003cp\u003eMetrics reach InfluxDB along two paths. Vector collects and transforms log-based metrics from the server environment and forwards them to InfluxDB. The Go and NestJS services send business and application metrics directly over the HTTP API.\u003c/p\u003e\n\n\u003cp\u003eThe tagging strategy reflects that split. Server-level and infrastructure metrics carry richer tag sets like \u003ccode class=\"language-markup\"\u003eenv\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003econtainer\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003eprocess_name\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003eprocess_instance\u003c/code\u003e, and \u003ccode class=\"language-markup\"\u003eservice\u003c/code\u003e. Business and application metrics are tagged more lightly, typically just \u003ccode class=\"language-markup\"\u003eenv\u003c/code\u003e plus a small number of domain-specific identifiers. On the NestJS side, metrics are batched and flushed either every 60 seconds or when the batch size crosses a configured threshold, which is a configuration the team continues to tune to balance data freshness against RAM usage and write overhead. Grafana sits on top of it all, querying InfluxDB directly.\u003c/p\u003e\n\n\u003ch2 id=\"future-plans-for-utilizing-influxdb-3\"\u003eFuture plans for utilizing InfluxDB 3\u003c/h2\u003e\n\n\u003cp\u003eFor Mumu, InfluxDB has unlocked more than just an internal observability story. The team is now actively building toward making it a first-class part of their product surface.\u003c/p\u003e\n\n\u003cp\u003eThe most immediate project is integrating InfluxDB as a delivery target inside MsgGO. Today, MsgGO routes messages to Slack, Telegram, Discord, Email, SMS, and Webhooks. Adding InfluxDB as a target means any system already sending events through MsgGO like  contact forms, deployment notifications, server alerts, and application events, can now route structured event data directly into InfluxDB with no additional integration work. Since Mumu uses MsgGO heavily inside its own infrastructure, this would pay off immediately in its own workflows, with the customer-facing benefits coming close behind.\u003c/p\u003e\n\n\u003cp\u003eFurther out, the team is evaluating whether to move user-activity statistics that are currently kept as activity records in a NoSQL database into InfluxDB. That data is inherently time series in nature, and putting it in InfluxDB would let them expose richer usage analytics inside the Mumu dashboard without needing a separate query infrastructure. And they want to lean on InfluxDB’s SQL interface to drive product decisions, using internal usage metrics to understand which modules see the most engagement, where users drop off, and how feature adoption shifts after a release.\u003c/p\u003e\n","date_published":"2026-06-25T08:00:00+00:00","authors":[{"name":"Charles Mahler"}]},{"id":"https://www.influxdata.com/blog/telegraf-enterprise-ga","url":"https://www.influxdata.com/blog/telegraf-enterprise-ga","title":"Telegraf Enterprise Now Generally Available: Manage Telegraf Fleets at Scale","content_html":"\u003cp\u003e\u003cstrong\u003eSummary: Telegraf Enterprise is now generally available. It combines Telegraf Controller, a centralized management console for Telegraf, with official support from InfluxData. Open source Telegraf remains unchanged. Telegraf Controller is free to start with built-in limits, while a Telegraf Enterprise license unlocks higher-scale limits, audit logging, LDAP/OIDC integration, and commercial support.\u003c/strong\u003e\u003c/p\u003e\n\n\u003cp\u003eTelegraf has become the standard for collecting telemetry across cloud, edge, and physical infrastructure. With more than five billion downloads and 400+ official plugins, Telegraf is the open source standard to connect virtually any data source to any destination.\u003c/p\u003e\n\n\u003cp\u003eOver the years, we’ve seen Telegraf evolve from a lightweight collection agent into a foundational part of production infrastructure. But as deployments grow, the challenge shifts from collecting telemetry to managing the systems that collect it. Large Telegraf deployments require consistent configurations across environments, clear visibility into fleet health, and safe rollout of changes to thousands of agents. Many teams rely on scripts, internal tools, and manual processes to manage this, but these approaches quickly break down as deployments scale.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://www.influxdata.com/products/telegraf-enterprise/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=telegraf-enterprise-ga\u0026amp;utm_content=blog\"\u003eTelegraf Enterprise\u003c/a\u003e is now generally available to address these challenges, giving teams a centralized way to manage configurations, monitor fleet health, and operate Telegraf deployments with tens of thousands of agents from a single system.\u003c/p\u003e\n\n\u003ch2 id=\"centralized-control-for-large-telegraf-deployments\"\u003eCentralized control for large Telegraf deployments\u003c/h2\u003e\n\n\u003cp\u003eAs Telegraf becomes more deeply embedded in production environments, visibility and consistency become increasingly important. Teams need to understand which agents are healthy, what configurations are running across environments, and whether changes have been deployed successfully. They also need a way to manage differences between regions, customers, and environments without creating hundreds of nearly identical configurations.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eTelegraf Enterprise combines Telegraf Controller, a centralized management console for Telegraf, with official InfluxData support\u003c/strong\u003e. Telegraf Controller provides a centralized way to manage those deployments. Teams can create and manage configurations centrally, assign them to agents, and monitor fleet health from a single interface. Configuration templates and parameter substitution make it possible to standardize shared configurations while still allowing environment-specific values where needed.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg class=\"py-10\" src=\"//images.ctfassets.net/o7xu9whrs0u9/2k07vZhwHTnp5F2uhSWUVj/0b624827c8409d6db8fec263c5cdefb9/telegraf-enterprise-agents.png\" alt=\"telegraf-enterprise-agents\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eFor organizations operating large Telegraf deployments, consistency is just as important as collection. Keeping configurations aligned across thousands of agents, while continuing to work with existing automation systems, can quickly become an operational challenge.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg class=\"py-10\" src=\"//images.ctfassets.net/o7xu9whrs0u9/30YgKR4570eWgHjrLjv177/947f9741de71f9494a4f60e97fb6c3b4/telegraf-config-builder.png\" alt=\"telegraf-config-builder\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe visual configuration builder makes it easier to create and review configurations across Telegraf’s 400+ plugin ecosystem without manually authoring every line of TOML.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg class=\"py-10\" src=\"//images.ctfassets.net/o7xu9whrs0u9/5pgPpQFjD9aNpl0Jry5Ynx/746773079100ec12f0a61cbb27ee5efa/telegraf-enterprise-configs.png\" alt=\"telegraf-enterprise-configs\" /\u003e\u003c/p\u003e\n\n\u003cp\u003e“We run thousands of Telegraf agents across diverse customer environments. Telegraf Controller will help us use our existing automation tools to keep agent configurations consistent and up to date across our fleet as we continue expanding our observability platform.” – Poul H. Sørensen, Senior Systems Consultant at Orange Business\u003c/p\u003e\n\n\u003cp\u003eOpen source Telegraf remains unchanged. The agent, the plugin ecosystem, and community continue as they always have. Telegraf Controller’s free tier supports up to 20 configs and 100 agents, making it easy to get started with centralized fleet management.\u003c/p\u003e\n\n\u003ch2 id=\"what-an-enterprise-license-buys-you\"\u003eWhat an Enterprise license buys you\u003c/h2\u003e\n\n\u003cp\u003eAs your fleet grows and more people touch your Telegraf configurations, the free tier’s limits start becoming constraints. Telegraf Enterprise removes those constraints and scales to your needs, offering:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eRaised scale limits\u003c/strong\u003e: The free version of Telegraf Controller makes it easy to get started and evaluate the product. A Telegraf Enterprise license raises the agent and configuration limits based on your licensed entitlement, allowing Telegraf Controller to grow with the size of your fleet.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eAudit logging\u003c/strong\u003e: Telegraf Enterprise records security-relevant and administrative events, including configuration changes, permission updates, agent actions, login activity, and license changes. This provides clearer operational history for troubleshooting, compliance, and security investigations.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eIdentity provider integration\u003c/strong\u003e: Organizations can integrate Telegraf Enterprise with LDAP and OIDC rather than maintaining a separate user directory. For organizations using SSO or MFA through an identity provider, Telegraf Controller can integrate into that existing model.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"official-support-for-telegraf\"\u003eOfficial support for Telegraf\u003c/h2\u003e\n\n\u003cp\u003eTelegraf Enterprise adds an official InfluxData support path for organizations running Telegraf as part of their critical production infrastructure. Customers get support for both Telegraf Controller and the Telegraf agent, including installation, configuration, operational guidance, and troubleshooting assistance.\u003c/p\u003e\n\n\u003ch2 id=\"upgrade-in-place\"\u003eUpgrade in place\u003c/h2\u003e\n\n\u003cp\u003eIf you’re already running Telegraf Controller, there’s no separate Enterprise deployment or migration process. Add a valid license to your existing installation and the Enterprise capabilities unlock in place. The same deployment, agents, and configurations remain in place while additional scale, security, and support capabilities become available.\u003c/p\u003e\n\n\u003ch2 id=\"get-started\"\u003eGet started\u003c/h2\u003e\n\n\u003cp\u003eTelegraf Controller is available today with a free tier. \nTelegraf solved the problem of collecting telemetry from almost anywhere; Telegraf Enterprise helps teams operate that collection layer as it grows into critical infrastructure.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://docs.influxdata.com/telegraf/controller/install/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=telegraf-enterprise-ga\u0026amp;utm_content=blog\"\u003e\u003cstrong\u003eDownload and install Telegraf Controller\u003c/strong\u003e\u003c/a\u003e or \u003ca href=\"http://influxdata.com/contact-sales-telegraf-enterprise/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=telegraf-enterprise-ga\u0026amp;utm_content=blog\"\u003e\u003cstrong\u003econtact us about Telegraf Enterprise\u003c/strong\u003e\u003c/a\u003e.\u003c/p\u003e\n","date_published":"2026-06-24T06:00:00+00:00","authors":[{"name":"Scott Anderson"}]},{"id":"https://www.influxdata.com/blog/why-relational-databases-fail","url":"https://www.influxdata.com/blog/why-relational-databases-fail","title":"Why Relational Databases Fail Satellite Telemetry","content_html":"\u003cp\u003eSatellite operations depend on telemetry as the primary interface to systems that teams cannot directly inspect. Once a spacecraft reaches orbit, signals such as battery levels, temperature, signal strength, and fault codes become the foundation for understanding system health and maintaining control.\u003c/p\u003e\n\n\u003cp\u003eTelemetry streams continuously, so the underlying data system becomes a critical control point that needs to handle a constant, heavy flow of data. When that system cannot ingest, query, and manage data efficiently, dashboards lag, investigations slow, and the clarity teams rely on begins to erode.\u003c/p\u003e\n\n\u003ch2 id=\"relational-data-vs-satellite-telemetry\"\u003eRelational data vs. satellite telemetry\u003c/h2\u003e\n\n\u003cp\u003eRelational data organizes information into structured records with consistent fields and defined relationships. A business application might represent customers, orders, and products as related datasets, making it easy to answer questions such as which customer placed an order or which products were included in a purchase. This relational model works well when information is relatively stable and relationships between records are clearly defined.\u003c/p\u003e\n\n\u003cp\u003eSatellites are dynamic systems. Operators are not tracking static records or one-time transactions, but monitoring systems that change continuously. Telemetry arrives as a stream of time-stamped measurements, or \u003ca href=\"https://www.influxdata.com/time-series-database/#what-is-time-series\"\u003etime series data\u003c/a\u003e. Each value depends on when the system recorded it and its change relative to earlier readings.\u003c/p\u003e\n\n\u003cp\u003eA single battery reading of 80% may appear normal. In isolation, it provides limited insight. If that value declined from 95% over the past hour, the same reading indicates a different condition. Operators need the sequence, rate of change, and surrounding context to determine whether a change reflects normal behavior, an emerging anomaly, or a sign of impending failure.\u003c/p\u003e\n\n\u003cp\u003eAs telemetry volume grows, individual readings lose meaning without historical continuity. Teams need to understand how values change over time, not just what they are at one moment. As that requirement grows, relational systems show their limits.\u003c/p\u003e\n\n\u003ch2 id=\"where-relational-databases-start-to-strain\"\u003eWhere relational databases start to strain\u003c/h2\u003e\n\n\u003cp\u003ePostgreSQL- and MySQL-style databases can support early telemetry workloads, especially when teams already use them for operational data. The strain begins when a transactional, row-oriented database becomes the primary store for high-volume telemetry.\u003c/p\u003e\n\n\u003cp\u003eFor these systems, telemetry challenges typically appear in three areas: query speed, data lifecycle management, and storage efficiency.\u003c/p\u003e\n\n\u003ch4 id=\"speed-and-query-performance\"\u003eSpeed and Query Performance\u003c/h4\u003e\n\n\u003cp\u003eSatellite operations rely on dashboards, alerts, and analytics to maintain visibility into system health. When a subsystem begins overheating or signal strength drops during a contact window, teams need recent data immediately.\u003c/p\u003e\n\n\u003cp\u003eRelational databases such as PostgreSQL and MySQL can store telemetry, but they don’t optimize for high-volume time series workloads. As measurements accumulate, queries must process larger datasets, indexes expand, and the database spends more resources balancing continuous writes with analytical reads.\u003c/p\u003e\n\n\u003cp\u003eSlow queries affect more than latency. Delayed dashboards and alerts reduce the time teams have to detect anomalies, investigate issues, and make operational decisions. Teams may need additional infrastructure and ongoing tuning to keep systems responsive, increasing the cost of managing mission data.\u003c/p\u003e\n\n\u003ch4 id=\"time-awareness-and-data-lifecycle\"\u003eTime Awareness and Data Lifecycle\u003c/h4\u003e\n\n\u003cp\u003eTelemetry does not retain the same value over time. Recent measurements are critical for real-time monitoring, \u003ca href=\"https://www.influxdata.com/glossary/anomaly-detection/\"\u003eanomaly detection\u003c/a\u003e, and fault investigation, where operators need full-resolution data. Older data remains useful, but teams more often use it for trend analysis, reporting, and long-term performance evaluation.\u003c/p\u003e\n\n\u003cp\u003ePostgreSQL- and MySQL-style transactional databases do not manage this telemetry lifecycle by default. Teams can use partitions, scheduled jobs, archive tables, or custom pipelines to retain, move, or summarize older records, but those processes require separate design, maintenance and governance. Other relational systems handle historical analysis differently. Analytical databases and data warehouses can support large-scale scans and reporting, but continuous telemetry creates separate tradeoffs around ingest speed, data freshness, cost, and operational responsiveness.\u003c/p\u003e\n\n\u003cp\u003eFor telemetry stores,  lifecycle management becomes ongoing operational work. Without a clear strategy, raw measurements continue to accumulate, historical datasets grow harder to manage and queries may process more full-resolution data than operators need for long-term analysis.\u003c/p\u003e\n\n\u003ch4 id=\"storage-and-scale\"\u003eStorage and Scale\u003c/h4\u003e\n\n\u003cp\u003eSatellite telemetry generates large volumes of measurements, many of which share the same contextual information. Spacecraft identifiers, subsystem names, and sensor metadata often repeat across millions of records while only the timestamp and value change.\u003c/p\u003e\n\n\u003cp\u003eTelemetry schemas can also become large and sparse as missions add more sensors, subsystems, and measurement types. Not every field applies to every reading, but the database still has to store, index, and manage the growing structure around those measurements. As the number of sources and dimensions increases, data volume grows rapidly, and \u003ca href=\"https://www.influxdata.com/glossary/cardinality/\"\u003ecardinality\u003c/a\u003e expands, multiplying the number of unique series the system must store.\u003c/p\u003e\n\n\u003cp\u003ePostgreSQL- and MySQL-style transactional databases are not optimized for this pattern. Row-oriented storage works well when each row represents a distinct transaction or operational record. Telemetry behaves differently. It repeats similar context across continuous streams of time-ordered measurements, which can make compression less efficient and increase storage overhead.\u003c/p\u003e\n\n\u003cp\u003eOver time, organizations may allocate more infrastructure simply to retain and manage telemetry. What begins as a manageable archive can become increasingly expensive to maintain, especially when long-term historical data remains important for analysis and mission planning.\u003c/p\u003e\n\n\u003cp\u003eThese limitations stem from storage models designed for transactional records rather than continuous, time-ordered measurements.\u003c/p\u003e\n\n\u003ch2 id=\"time-series-databases-a-better-fit-for-time-series-data\"\u003eTime series databases: A better fit for time-series data\u003c/h2\u003e\n\n\u003cp\u003eTime series databases are purpose-built for data that changes over time. Instead of organizing information around static records and relationships, they structure data around timestamps, time ranges, and continuous streams of measurements.\u003c/p\u003e\n\n\u003cp\u003eThis design matches how satellite telemetry behaves. Operators need to monitor recent readings, compare values across time windows, identify trends, and investigate anomalies using historical context. The database must support both high-ingest workloads and fast access to time-based data.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB is a time series database built for these requirements. It provides a data layer optimized for telemetry, helping satellite teams power real-time dashboards, alerts, anomaly detection, and long-term analysis while avoiding many of the performance and scalability challenges that emerge when telemetry is stored in a relational database.\u003c/p\u003e\n\n\u003ch4 id=\"maintaining-query-speed-at-scale\"\u003eMaintaining Query Speed at Scale\u003c/h4\u003e\n\n\u003cp\u003eInfluxDB 3 maintains query performance as telemetry volumes grow. Its architecture combines a real-time columnar engine with technologies designed for analytical workloads, helping teams retrieve and analyze large datasets efficiently.\u003c/p\u003e\n\n\u003cp\u003eTelemetry is stored in a columnar format, allowing queries to read only the fields they need instead of scanning entire records. Because data is organized around time, queries can focus on relevant time ranges rather than searching across the full dataset.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB 3 also uses \u003ca href=\"https://www.influxdata.com/glossary/apache-datafusion/\"\u003eApache DataFusion\u003c/a\u003e to power SQL queries. DataFusion applies filters early and processes data efficiently through \u003ca href=\"https://www.influxdata.com/glossary/batch-processing-explained/\"\u003ebatches\u003c/a\u003e, reducing the amount of information that must be scanned and moved during query execution.\u003c/p\u003e\n\n\u003cp\u003eFor satellite operations, these optimizations help keep dashboards, alerts, and investigations responsive even as telemetry volumes increase. Teams can access recent measurements and historical trends without the growing query overhead that often affects relational systems handling large-scale time series data.\u003c/p\u003e\n\n\u003ch4 id=\"managing-data-over-time\"\u003eManaging Data Over Time\u003c/h4\u003e\n\n\u003cp\u003eTelemetry does not retain the same value over time. Recent measurements are critical for real-time monitoring, anomaly detection, and fault investigation, where operators need full-resolution data. Older telemetry remains useful, but teams more often use it for trend analysis, reporting, and long-term performance evaluation.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB supports this lifecycle through retention and downsampling. Retention policies define how long data remains available, while downsampling converts older high-frequency telemetry into lower-resolution aggregates that preserve long-term trends.\u003c/p\u003e\n\n\u003cp\u003eThis approach helps teams manage telemetry according to how it’s actually used. Recent data can remain detailed and readily accessible for operational workflows, while historical data can be summarized to reduce storage requirements and query costs.\u003c/p\u003e\n\n\u003cp\u003eBy automating retention and downsampling, InfluxDB reduces the need for custom cleanup scripts, archive processes, and manual data management. Teams can spend less time maintaining telemetry pipelines while keeping storage growth and query overhead under control.\u003c/p\u003e\n\n\u003ch4 id=\"reducing-storage-overhead-at-scale\"\u003eReducing Storage Overhead at Scale\u003c/h4\u003e\n\n\u003cp\u003eTelemetry adds both fresh and repeated data. \u003cstrong\u003eAs missions add sensors, measurement types, and metadata, telemetry schemas can become large, sparse, and expensive to store\u003c/strong\u003e. InfluxDB 3 organizes telemetry in a columnar format built on \u003ca href=\"https://www.influxdata.com/glossary/apache-arrow/\"\u003eApache Arrow\u003c/a\u003e and \u003ca href=\"https://www.influxdata.com/glossary/apache-parquet/\"\u003eApache Parquet\u003c/a\u003e, which helps store similar values together and improve compression.\u003c/p\u003e\n\n\u003cp\u003eThis matters for satellite workloads. Spacecraft, subsystem, and sensor labels may repeat across large volumes of readings, while different measurements may rely on different fields. A storage model designed for time series data can compress repeated values and sparse data more efficiently than a transactional model that treats each reading like a separate record.\u003c/p\u003e\n\n\u003cp\u003eStronger compression helps reduce storage overhead as telemetry volume and cardinality grow, allowing teams to retain historical context without carrying the full storage cost of every raw measurement, repeated label, and unused field. InfluxDB 3 also supports architectures that separate compute from storage, giving organizations more flexibility to scale storage and query resources independently as data grows.\u003c/p\u003e\n\n\u003cp\u003eIt’s a model that helps slow the growth of storage overhead: satellite teams can keep more telemetry available for analysis while reducing the infrastructure cost of storing large, repetitive telemetry datasets.\u003c/p\u003e\n\n\u003ch2 id=\"processing-telemetry-beyond-storage\"\u003eProcessing telemetry beyond storage\u003c/h2\u003e\n\n\u003cp\u003eThe value of satellite telemetry lies in understanding and responding to real-time spacecraft behavior. Operational impact comes from turning continuous streams of measurements into timely insights that help teams maintain visibility and make informed decisions with confidence.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB is the foundation that makes that possible. By combining scalable telemetry storage with built-in processing capabilities, it helps satellite teams support real-time monitoring, accelerate analysis, and automate workflows that keep mission operations running smoothly.\u003c/p\u003e\n\n\u003cp\u003eWhether the goal is improving spacecraft health monitoring, reducing investigation time, scaling telemetry infrastructure, or preserving long-term mission insight, InfluxDB helps transform telemetry data into a continuous source of operational value.\u003c/p\u003e\n\n\u003cp\u003e2Get started with \u003ca href=\"https://www.influxdata.com/products/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=why_relational_databases_fail\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Core OSS\u003c/a\u003e or \u003ca href=\"https://www.influxdata.com/products/influxdb3-enterprise/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=why_relational_databases_fail\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Enterprise\u003c/a\u003e to build a telemetry platform designed for time series workloads.\u003c/p\u003e\n","date_published":"2026-06-19T08:00:00+00:00","authors":[{"name":"Allyson Boate"}]},{"id":"https://www.influxdata.com/blog/influxdb-3-10","url":"https://www.influxdata.com/blog/influxdb-3-10","title":"What’s New in InfluxDB 3.10: Performance Beta Expanded with New Enterprise Features ","content_html":"\u003cp\u003eIn our last release, we introduced a beta of performance updates designed for heavier, more complex time series workloads. InfluxDB 3.10 expands that beta to include enterprise features that give teams more control as they scale and manage larger workloads in InfluxDB 3.\u003c/p\u003e\n\n\u003cp\u003eThis release adds end-to-end backup and restore, row-level deletes, bulk import from Parquet, user management, and an RBAC preview to the previous performance beta. It also includes cross-database plugin queries, a new readiness endpoint, and compaction improvements for InfluxDB 3 Enterprise. Together, these updates help teams evaluate the next phase of InfluxDB 3 performance and scale with more of the operational tooling they need to manage real workloads.\u003c/p\u003e\n\n\u003cp\u003eWe’re inviting customers to test this next phase of InfluxDB 3 performance and scale, share feedback, and help shape the path to general availability.\u003c/p\u003e\n\n\u003ch2 id=\"expanded-capabilities-for-the-performance-beta\"\u003eExpanded capabilities for the performance beta\u003c/h2\u003e\n\n\u003cp\u003eThe performance improvements \u003ca href=\"https://www.influxdata.com/blog/influxdb-3-9/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=influxdb_3_10_expanded_beta\u0026amp;utm_content=blog\"\u003ewe previewed in InfluxDB 3.9\u003c/a\u003e continue to mature in beta. These updates are designed for teams testing heavier time series workloads, including higher ingest, wider schemas, sparse data, and more demanding recent-data queries.\u003c/p\u003e\n\n\u003cp\u003eThe beta remains opt-in and is not yet the default, so existing deployments continue running unaffected unless teams explicitly enable it with the \u003ccode class=\"language-markup\"\u003e--use-pacha-tree\u003c/code\u003e flag.\u003c/p\u003e\n\n\u003cp\u003eOnce users opt in to the beta, InfluxDB 3.10 adds operational capabilities, so teams can do more than test raw performance. They can protect data, recover from known-good states, remove bad or unnecessary rows, and bring existing Parquet datasets into InfluxDB 3 for evaluation. That matters for teams working with real production patterns, where data rarely arrives perfectly clean and workloads rarely stay fixed.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eThe following capabilities are available only when using the performance beta, enabled with the -\u003ccode class=\"language-markup\"\u003e-use-pacha-tree\u003c/code\u003e flag\u003c/strong\u003e:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eEnd-to-end backup and restore\u003c/strong\u003e: You can now run full backups that capture cluster state and compacted data for easy rollbacks. Restores run asynchronously, allowing you to recover data into a fresh store for disaster recovery or roll a live cluster back to an earlier point in time. We’ll soon be adding incremental restores to give you even more control over your restore points.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eRow-level deletes\u003c/strong\u003e: Teams can now remove specific rows based on time ranges or tag predicates rather than dropping entire tables when data needs to be purged. The compactor applies these changes asynchronously in the background, allowing teams to clean up production data without interrupting operations.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eBulk import from Parquet\u003c/strong\u003e: A new bulk-import function makes it easier to bring historical or external data into InfluxDB 3. Teams can point InfluxDB at a generic Parquet file or an entire directory, use simple column mappings, and ingest each file as an independent import job.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"user-authentication-and-rbac-preview\"\u003eUser authentication and RBAC preview\u003c/h2\u003e\n\n\u003cp\u003eAs time series workloads become more central to production systems, access control becomes increasingly important. The same database may support operators monitoring live systems, developers building applications, analysts exploring historical data, and automated services writing or transforming telemetry.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB 3.10 Enterprise introduces a preview of multi-user authentication and role-based access control (RBAC). This feature is turned off by default for this release.\u003c/p\u003e\n\n\u003cp\u003eWhen enabled, operators can configure traditional username and password logins that issue JWTs, or opt for external identity management through OAuth and OIDC. 3.10 also introduces built-in roles, including Admin, Auditor, and Member, to enforce proper boundaries across your teams. Best of all, your existing token workflows will continue to work exactly as they do today without any breaking changes.\u003c/p\u003e\n\n\u003ch2 id=\"general-updates-and-improvements\"\u003eGeneral updates and improvements\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB 3.10 also includes several capabilities that work across all deployments to streamline data pipelines and improve cluster management.\u003c/p\u003e\n\n\u003ch4 id=\"cross-database-plugin-queries\"\u003eCross-Database Plugin Queries\u003c/h4\u003e\n\n\u003cp\u003eTime series data often moves through stages: raw telemetry, cleaned data, downsampled rollups, forecasts, anomaly scores, and application-ready views. Those stages may live in different databases, but teams still need to connect them without building unnecessary external pipelines.\u003c/p\u003e\n\n\u003cp\u003eIn 3.10, Processing Engine plugins are no longer restricted to querying their own database. Now, a plugin can query any database residing on that node. This unlocks read-from-one, write-to-another data pipelines, such as reading raw telemetry from a staging database and writing compacted rollups or machine learning forecasts to a production database.\u003c/p\u003e\n\n\u003ch4 id=\"readiness-endpoint\"\u003eReadiness Endpoint\u003c/h4\u003e\n\n\u003cp\u003eProduction deployments need health checks that reflect whether a node can actually serve traffic, not just whether a process is running. InfluxDB 3.10 adds a new /ready endpoint. Instead of a basic uptime check, this endpoint verifies whether the node can successfully reach its underlying object store, giving operators a more reliable signal for traffic routing.\u003c/p\u003e\n\n\u003ch4 id=\"improved-compaction\"\u003eImproved Compaction\u003c/h4\u003e\n\n\u003cp\u003eUnder heavy ingest, compaction needs to keep pace with incoming writes so query nodes can access optimized data. When compaction stalls, the path from raw writes to efficient queries slows down. InfluxDB 3.10 introduces parallel compaction, ensuring that query nodes access fully compacted data more quickly and serve queries faster.\u003c/p\u003e\n\n\u003ch2 id=\"get-started-with-influxdb-310\"\u003eGet started with InfluxDB 3.10\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB 3.10 is available now. To get started, download the latest version or pull the newest Docker image for Core or Enterprise.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://www.influxdata.com/products/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=influxdb_3_10_expanded_beta\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Core\u003c/a\u003e remains free and open source under MIT and Apache 2 licenses, optimized for recent data and local workloads. \u003ca href=\"https://www.influxdata.com/products/influxdb3-enterprise/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=influxdb_3_10_expanded_beta\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Enterprise\u003c/a\u003e adds long-range querying, clustering, advanced security, and full operational tooling for production deployments.\u003c/p\u003e\n\n\u003cp\u003eCheck out the docs (\u003ca href=\"https://docs.influxdata.com/influxdb3/core/release-notes/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=influxdb_3_10_expanded_beta\u0026amp;utm_content=blog\"\u003eCore\u003c/a\u003e, \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/release-notes/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=influxdb_3_10_expanded_beta\u0026amp;utm_content=blog\"\u003eEnterprise\u003c/a\u003e), try the release in your environment, and share your feedback in Discord or the Community Slack. We want your feedback as the performance beta continues to mature.\u003c/p\u003e\n","date_published":"2026-06-17T12:00:00+00:00","authors":[{"name":"Peter Barnett"}]},{"id":"https://www.influxdata.com/blog/generate-synthetic-data","url":"https://www.influxdata.com/blog/generate-synthetic-data","title":"Generate Synthetic Time Series Data in InfluxDB 3","content_html":"\u003cp\u003eGetting InfluxDB 3 up and running is a pretty lightweight process with the \u003ca href=\"https://docs.influxdata.com/influxdb3/core/install/#quick-install-for-linux-and-macos\"\u003einstallation script\u003c/a\u003e. Getting time series data into it is the next step, and for exploration, basic testing, or scenarios where you don’t have a stream of time series data ready to write, that can be a point of friction.\u003c/p\u003e\n\n\u003cp\u003eThat hurdle is particularly high when you want to test the rest of the system around the data you’d be writing: dashboards, alerts, replication, network connectivity, edge devices, server sizing, or Processing Engine workflows—you don’t always have the ability to start writing production data into a freshly-installed database, or you may not have that data yet.\u003c/p\u003e\n\n\u003cp\u003eTwo new InfluxDB 3 plugins help with exactly that: the Bird Tracking Simulator and the Signal Generator. Both are scheduled plugins that generate data directly to InfluxDB 3, making it easy to start writing realistic sample data with a single trigger. The Bird Tracking Simulator creates synthetic bird telemetry, while the Signal Generator creates configurable waveform data for sensor-like or metric-like use cases.\u003c/p\u003e\n\n\u003ch2 id=\"why-generate-sample-data-this-way\"\u003eWhy generate sample data this way?\u003c/h2\u003e\n\n\u003cp\u003eA lot of InfluxDB workflows are easier to understand once data is actively moving through the system:a dashboard is easier to build when the line and the most recent datapoint keep changing, an alert is easier to validate when values cross a threshold, edge replication is easier to test when writes are arriving continuously, and a small server or single-board computer is easier to evaluate when you can watch how it behaves under a steady stream of points.\u003c/p\u003e\n\n\u003cp\u003eThese plugins are meant to make that first step simple. Create a database, create a trigger, and InfluxDB 3 starts generating data on a schedule. From there, you can query it, visualize it, replicate it, downsample it, or use it as input for other Processing Engine plugins.\u003c/p\u003e\n\n\u003ch2 id=\"bird-tracking-simulator\"\u003eBird Tracking Simulator\u003c/h2\u003e\n\n\u003cp\u003eThe Bird Tracking Simulator generates a stream of synthetic bird telemetry. On its first run, it creates a persistent flock of named birds, assigns each bird a variety of tags, such as species, name, and range, and stores the flock in the Processing Engine cache. Each scheduled execution of the plugin advances the flock by updating a number of measurements, including speed, heading, latitude, and longitude, with the birds going on \u003ca href=\"https://en.wikipedia.org/wiki/Random_walk\"\u003erandom walks\u003c/a\u003e within a predefined range for each species.\u003c/p\u003e\n\n\u003cp\u003eThe shape of the data is useful for a few reasons. It has multiple entities,  tags, and geospatial fields. It changes over time in a way that is easy to inspect visually. That makes it a good fit for testing dashboards, map panels, edge replication, and basic query patterns that group or filter by tags.\u003c/p\u003e\n\n\u003cp\u003eThe plugin writes to the \u003ccode class=\"language-markup\"\u003ebird_tracking\u003c/code\u003e measurement. Its configuration is also intentionally small, specified with simple trigger arguments: \u003ccode class=\"language-markup\"\u003ebird_count\u003c/code\u003e controls how many persistent birds are tracked, and \u003ccode class=\"language-markup\"\u003epoints_per_bird\u003c/code\u003e controls how many movement points each bird emits per scheduled run. The defaults are 25 birds and 1 point per bird. The number of data points the plugin generates is a simple product of these two options and the trigger specification for how often the plugin runs.\u003c/p\u003e\n\n\u003cp\u003eThe plugin requires \u003ca href=\"https://pypi.org/project/Faker/\"\u003eFaker\u003c/a\u003e, so install that first:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 install package Faker\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThen create a database and a scheduled trigger:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create database sample_data\n\ninfluxdb3 create trigger \\\n  --database sample_data \\\n  --path \"gh:influxdata/bird_data_simulator/bird_data_simulator.py\" \\\n  --trigger-spec \"every:10s\" \\\n  --trigger-arguments bird_count=10,points_per_bird=10 \\\n  bird_tracking_demo\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eAfter the trigger has run a few times, query the generated data:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query \\\n  --database sample_data \\\n  \"SELECT * FROM bird_tracking ORDER BY time DESC LIMIT 5\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eFor a denser stream, increase the flock size, increase the points per bird, or adjust the trigger interval. That gives you a simple way to create a steady stream of entity-oriented time series data. You can use it to populate dashboards, test writes across a network, or quickly confirm that a new InfluxDB 3 setup is receiving, storing, and querying data as expected.\u003c/p\u003e\n\n\u003ch2 id=\"signal-generator\"\u003eSignal Generator\u003c/h2\u003e\n\n\u003cp\u003eThe Signal Generator achieves many of the same things, but by generating numeric signals rather than named entities. The default preset produces a signal centered around 30, with a slow sine trend, Gaussian noise, and occasional spikes. It uses only the Python standard library, supports configurable measurement names, field names, tags, and point resolution, and can compose multiple waveform types together. Supported waveform types include sine, square, triangle, sawtooth, noise, and spike.\u003c/p\u003e\n\n\u003cp\u003eThat makes it useful for testing dashboards, threshold checks, alerting behavior, anomaly detection, and any workflow that requires a predictable yet non-static stream of numeric values. A line with trend, noise, and the occasional spike gives you something closer to the patterns you usually care about when working with time series data.\u003c/p\u003e\n\n\u003cp\u003eThe simplest version uses the default preset:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create database signals\n\ninfluxdb3 create trigger \\\n  --database signals \\\n  --path \"gh:influxdata/signal_generator/signal_generator.py\" \\\n  --trigger-spec \"every:10s\" \\\n  signal_basic\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eOnce the trigger has run, query the latest generated values:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query \\\n  --database signals \\\n  \"SELECT time, value FROM signal ORDER BY time DESC LIMIT 10\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eYou can also aggregate the generated signal over time:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003einfluxdb3 query \\\n  --database signals \\\n  \"SELECT\n     time_bucket(time, INTERVAL '1 minute') AS minute,\n     AVG(value) AS avg_value,\n     MIN(value) AS min_value,\n     MAX(value) AS max_value\n   FROM signal\n   WHERE time \u0026gt; now() - INTERVAL '1 hour'\n   GROUP BY minute\n   ORDER BY minute DESC\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eFor custom waveforms, the plugin can be configured with JSON arguments through InfluxDB 3 Explorer or the Processing Engine API. That lets you define signals for different measurements, fields, and tags, or stack waveforms together to create the shape you want.\u003c/p\u003e\n\n\u003cp\u003eFor example, you might create one signal that looks like a temperature sensor, another that behaves like CPU utilization, and another that emits occasional spikes to test an alerting path. Because each trigger can have its own configuration, you can build out a small set of synthetic streams that exercise different parts of your system.\u003c/p\u003e\n\n\u003ch2 id=\"lightweight-data-generation-for-influxdb-3\"\u003eLightweight data generation for InfluxDB 3\u003c/h2\u003e\n\n\u003cp\u003eThe Bird Tracking Simulator and Signal Generator are small plugins, but they solve a useful problem: they make it easy to get fresh time series data flowing through InfluxDB 3 with very little setup, allowing you to test your deployment and ensure data is flowing to and from every system downstream of your InfluxDB instance.\u003c/p\u003e\n\n\u003cp\u003eUse the Bird Tracking Simulator when you want moving, entity-oriented telemetry with tags and location fields. Use the Signal Generator when you want numeric signal data for dashboards, alerts, thresholds, and processing workflows.\u003c/p\u003e\n\n\u003cp\u003eCheck out the plugins in the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=generate_synthetic_data\u0026amp;utm_content=blog\"\u003eInfluxDB 3 plugin repository\u003c/a\u003e, try them on the hardware you already have, and use them as a quick way to exercise InfluxDB 3, the Processing Engine, and the systems connected to them.\u003c/p\u003e\n","date_published":"2026-06-12T08:00:00+00:00","authors":[{"name":"Cole Bowden"}]},{"id":"https://www.influxdata.com/blog/satellite-telemetry-itar-architecture","url":"https://www.influxdata.com/blog/satellite-telemetry-itar-architecture","title":"Satellite Telemetry, ITAR, and Data Residency: Building Architecture for Speed and Control","content_html":"\u003cp\u003eSatellite mission operators depend on \u003ca href=\"https://www.influxdata.com/telemetry-workloads/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=satellite_telemetry_itar_architecture\u0026amp;utm_content=blog\"\u003etelemetry\u003c/a\u003e to understand spacecraft health, ground system performance, and mission status in real-time. Operation signals help teams identify risks, investigate anomalies, and keep operations moving.\u003c/p\u003e\n\n\u003cp\u003eWhen a spacecraft enters safe mode or signal strength drops during a contact window, teams need trusted telemetry immediately. But mission data moves quickly across operational systems, and every handoff makes it harder to control.\u003c/p\u003e\n\n\u003cp\u003eHow can teams keep telemetry fast, useful, and available while maintaining control over sensitive mission data?\u003c/p\u003e\n\n\u003ch2 id=\"why-itar-and-data-residency-matter-for-telemetry\"\u003eWhy ITAR and data residency matter for telemetry\u003c/h2\u003e\n\n\u003cp\u003eFor satellite operators, sensitive mission data raises two practical questions: who can access the data, and where can it go? ITAR and data residency requirements bring those questions into the monitoring conversation.\u003c/p\u003e\n\n\u003cp\u003eITAR, or International Traffic in Arms Regulations, controls how certain defense-related products, services, and technical data can be shared. In practice, these rules help prevent sensitive information from moving to unauthorized people, systems, or environments.\u003c/p\u003e\n\n\u003cp\u003eTechnical data does not always stay attached to hardware, but can show how spacecraft systems operate, perform, fail, or respond under real conditions. Battery temperature, solar panel output, signal strength, contact window performance, and subsystem fault codes help teams monitor spacecraft health; those readings may also reveal performance limits or operational patterns that require closer review.\u003c/p\u003e\n\n\u003cp\u003eLegal and compliance teams determine which telemetry falls under ITAR or other export-control requirements. Classification alone does not protect the data. Engineering and infrastructure teams need systems that enforce those decisions as telemetry moves through daily operations.\u003c/p\u003e\n\n\u003ch4 id=\"data-movement-creates-the-control-challenge\"\u003eData Movement Creates the Control Challenge\u003c/h4\u003e\n\n\u003cp\u003eData residency adds the next layer. Telemetry may originate from spacecraft, ground systems, payload systems, and mission infrastructure, then move through ground stations, dashboards, cloud tools, analytics workflows, vendor systems, and long-term archives. Each stop creates another place where mission data may live, get copied, or become accessible.\u003c/p\u003e\n\n\u003cp\u003eWhen teams cannot trace readings across those systems, compliance reviews take longer and ownership becomes harder to prove. Security teams may struggle to confirm who accessed the data, where copies exist, and which system serves as the source of truth. Mission teams may also lose time reconstructing event timelines across systems.\u003c/p\u003e\n\n\u003cp\u003eThe architecture needs to answer the practical questions behind the compliance review: where mission data lives, who can access it, how long teams retain it, and how it moves across systems.\u003c/p\u003e\n\n\u003ch2 id=\"what-data-sprawl-costs-mission-teams\"\u003eWhat data sprawl costs mission teams\u003c/h2\u003e\n\n\u003cp\u003eData \u003ca href=\"https://www.influxdata.com/blog/breaking-data-silos-influxdb-3/\"\u003esilos\u003c/a\u003e and sprawl create the most risk when mission teams need to act quickly. During an anomaly, engineers need a clear sequence of events: what changed, when it changed, and which systems contributed.\u003c/p\u003e\n\n\u003cp\u003eA ground station contact window can expose the cost. A satellite reports rising battery temperature, irregular power draw, and a sudden safe mode event. To identify the cause, engineers need to trace battery temperature, power draw, command history, subsystem fault codes, and communications data from the minutes leading up to the event.\u003c/p\u003e\n\n\u003cp\u003eWhen each data source lives in a different tool, the response slows. One team checks a dashboard, another pulls logs from cloud storage, and another reviews an exported file from an analytics workflow. Engineers spend critical time reconciling records instead of isolating the issue.\nThose disconnected workflows create operational and governance costs at the same time. Mission teams lose speed during anomaly response. Compliance and security teams lose visibility into where sensitive telemetry lives, who can access it, and which copies exist.\u003c/p\u003e\n\n\u003ch2 id=\"how-influxdb-3-supports-controlled-telemetry-monitoring\"\u003eHow InfluxDB 3 supports controlled telemetry monitoring\u003c/h2\u003e\n\n\u003cp\u003eWith InfluxDB 3, satellite teams can bring high-volume operational signals from spacecraft, ground systems, and infrastructure into a shared time series architecture. When mission information lives across disconnected dashboards, logs, and exports, engineers have to piece together telemetry from multiple systems. With a shared time series architecture, engineers can analyze time-organized signals in one place, compare readings against historical baselines, and respond faster when anomalies occur.\u003c/p\u003e\n\n\u003ch4 id=\"how-the-architecture-works\"\u003eHow the Architecture Works\u003c/h4\u003e\n\n\u003cp\u003eA controlled data architecture starts with ingestion. Telemetry from spacecraft, ground systems, payload systems, and mission infrastructure can flow through \u003ca href=\"https://www.influxdata.com/time-series-platform/telegraf/\"\u003eTelegraf agents\u003c/a\u003e, \u003ca href=\"https://www.influxdata.com/mqtt/\"\u003eMQTT pipelines\u003c/a\u003e, or other approved collection paths into InfluxDB 3 as time-stamped data. Tags add operational context, such as spacecraft ID, subsystem, ground station, or signal source.\u003c/p\u003e\n\n\u003cp\u003eOnce telemetry enters the database, teams can query across systems without moving data into separate files or one-off tools. Engineers can compare current signal strength against previous contact windows, review power draw before a safe mode event, and correlate reaction wheel performance with temperature changes over time.\u003c/p\u003e\n\n\u003cp\u003eDashboards and alerts can use the same telemetry record. Operators can monitor live spacecraft health, trigger alerts when values drift from expected ranges, and investigate anomalies with historical context. \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/internals/data-retention/\"\u003eRetention\u003c/a\u003e and \u003ca href=\"https://www.influxdata.com/blog/downsampling-plugin-influxdb-3/\"\u003edownsampling\u003c/a\u003e extend the workflow over time, helping teams keep high-resolution telemetry where detail matters and preserve long-term trends as data ages.\u003c/p\u003e\n\n\u003ch4 id=\"deployment-flexibility-for-sensitive-data\"\u003eDeployment Flexibility for Sensitive Data\u003c/h4\u003e\n\n\u003cp\u003eFor satellite operators working with sensitive data, flexibility matters. Without deployment control, organizations risk signals moving outside approved environments, granting access to the wrong users or systems, and creating copies that complicate internal reviews.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB 3 Core gives teams a self-managed option for real-time telemetry ingest and recent-data queries in edge, on-premises, or private cloud environments. Self-managed deployment helps teams keep time series workloads closer to mission operations, ground systems, or other reviewed infrastructure.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB 3 Enterprise builds on that foundation for production workloads. High availability helps maintain access to mission data during critical operations. Read replicas can support dashboards, investigations, and analytics traffic without putting extra pressure on ingest workloads. Multi-node deployment options help teams separate ingest, query, and compaction as data volumes grow.\nWhile InfluxDB 3 does not determine whether telemetry falls under ITAR or make an organization compliant by default, it aligns telemetry workflows with internal requirements for storage, access, retention, and deployment.\u003c/p\u003e\n\n\u003ch4 id=\"eutelsat-oneweb-satellite-telemetry-at-scale\"\u003eEutelsat OneWeb: Satellite Telemetry at Scale\u003c/h4\u003e\n\n\u003cp\u003eEutelsat OneWeb puts this deployment flexibility into play.  The company operates a hybrid constellation with more than 600 LEO satellites, each producing more than 50,000 values.  Across the constellation, the operations team processes more than 1 million data points per second.\u003c/p\u003e\n\n\u003cp\u003eAt this scale, they needed a platform that could handle high-volume time series data, support real-time monitoring, and help engineers analyze spacecraft and ground-segment behavior in one place.\u003c/p\u003e\n\n\u003cp\u003eThe company built a telemetry stack with InfluxDB as the centralized time series engine, \u003ca href=\"https://www.influxdata.com/time-series-platform/telegraf/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=satellite_telemetry_itar_architecture\u0026amp;utm_content=blog\"\u003eTelegraf\u003c/a\u003e agents across the ground segment, and Grafana for dashboards, alerting, and cross-system correlation. InfluxDB supports more than 15 million unique series.\u003c/p\u003e\n\n\u003cp\u003eThis architecture gives the team a unified way to explore spacecraft and ground-segment data side by side. Engineers can monitor satellite health, correlate time series data across systems, trigger alerts from InfluxDB queries, and replay events for root-cause analysis. With that shared operational timeline, the team can analyze mission behavior across spacecraft and ground-segment systems.\u003c/p\u003e\n\n\u003cp\u003eRead the full \u003ca href=\"https://www.influxdata.com/customer/eutelsat/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=satellite_telemetry_itar_architecture\u0026amp;utm_content=blog\"\u003ecase study\u003c/a\u003e to check out how Eutelsat OneWeb uses InfluxDB to manage satellite telemetry.\u003c/p\u003e\n\n\u003ch2 id=\"the-bottom-line\"\u003eThe bottom line\u003c/h2\u003e\n\n\u003cp\u003eSatellite telemetry needs to move fast, but sensitive mission data also needs control. When telemetry spreads across disconnected systems, teams lose time during anomaly response and confidence during compliance review. A unified time series architecture helps satellite operators keep telemetry queryable, comparable, and governed across live operations and historical analysis.\u003c/p\u003e\n\n\u003cp\u003eTo get started, explore \u003ca href=\"https://www.influxdata.com/products/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=satellite_telemetry_itar_architecture\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Core OSS\u003c/a\u003e or \u003ca href=\"https://www.influxdata.com/products/influxdb3-enterprise/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=satellite_telemetry_itar_architecture\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Enterprise\u003c/a\u003e  to see how time series architecture can support real-time mission visibility, historical analysis, and controlled data workflows.\u003c/p\u003e\n","date_published":"2026-06-11T08:00:00+00:00","authors":[{"name":"Allyson Boate"}]},{"id":"https://www.influxdata.com/blog/predictive-maintenance-plugin-tutorial","url":"https://www.influxdata.com/blog/predictive-maintenance-plugin-tutorial","title":"Building a Predictive Maintenance Plugin with the InfluxDB 3 Processing Engine ","content_html":"\u003cp\u003ePredictive maintenance is one of the most compelling use cases for time series data. Instead of waiting for equipment to fail or servicing it on a fixed calendar regardless of condition, you watch the live sensor data and act when it indicates that a failure is coming. That “watch the data and act” loop is exactly what the InfluxDB 3 Processing Engine was built for.\u003c/p\u003e\n\n\u003cp\u003eIn this tutorial, we’ll build a working predictive maintenance plugin from scratch. We’ll install \u003ca href=\"https://docs.influxdata.com/influxdb3/core/install/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=predictive_maintenance_plugin_tutorial\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Core\u003c/a\u003e, load a well-known public dataset of jet engine sensor data, write a Python plugin that runs inside the database to estimate each engine’s Remaining Useful Life (RUL), and have it raise maintenance alerts automatically. By the end, you’ll have an end-to-end system that you can adapt to pumps, motors, HVAC units, CNC machines, or any other instrumented asset.\u003c/p\u003e\n\n\u003ch2 id=\"what-were-building\"\u003eWhat we’re building\u003c/h2\u003e\n\n\u003cp\u003eHere’s the architecture at a glance:\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003eSensor data lands in InfluxDB 3 Core. We’ll use \u003ca href=\"https://www.kaggle.com/datasets/bishals098/nasa-turbofan-engine-degradation-simulation\"\u003eNASA’s C-MAPSS turbofan engine degradation dataset\u003c/a\u003e, replayed into a \u003ccode class=\"language-markup\"\u003esensors\u003c/code\u003e table as if it were arriving live from a fleet of engines.\u003c/li\u003e\n  \u003cli\u003eA \u003ca href=\"https://docs.influxdata.com/influxdb3/core/plugins/\"\u003escheduled plugin\u003c/a\u003e runs every minute. It queries the most recent sensor readings per engine, computes a health/degradation score, and converts that into an estimated Remaining Useful Life.\u003c/li\u003e\n  \u003cli\u003eThe plugin writes its conclusions back into the database. RUL estimates go into a \u003ccode class=\"language-markup\"\u003erul_estimates\u003c/code\u003e table, and when an engine crosses a danger threshold, the plugin writes a row into a \u003ccode class=\"language-markup\"\u003emaintenance_alerts\u003c/code\u003e table and logs a warning.\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe key idea is that the analysis logic lives \u003cem\u003eembedded in the database\u003c/em\u003e. There’s no separate service to deploy, scale, or keep in sync. When data arrives, the engine acts on it.\u003c/p\u003e\n\n\u003ch2 id=\"prerequisites\"\u003ePrerequisites\u003c/h2\u003e\n\n\u003cp\u003eBefore starting, make sure you have:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eA Linux or macOS machine (Windows works too via the installer or Docker; commands below assume a Unix-like shell)\u003c/li\u003e\n  \u003cli\u003eCommand-line access\u003c/li\u003e\n  \u003cli\u003ePython 3 installed locally\u003c/li\u003e\n  \u003cli\u003eThe \u003ccode class=\"language-markup\"\u003etrain_FD001.txt\u003c/code\u003e file from the C-MAPSS dataset\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThat’s it. InfluxDB 3 Core itself is a single binary and brings its own bundled Python for plugins.\u003c/p\u003e\n\n\u003ch2 id=\"install-influxdb-3-core\"\u003eInstall InfluxDB 3 Core\u003c/h2\u003e\n\n\u003cp\u003eThe quickest path is the official install script, which always pulls the latest release:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003ecurl -O https://www.influxdata.com/d/install_influxdb3.sh \\\n  \u0026amp;\u0026amp; sh install_influxdb3.sh core\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eWhen the script finishes, confirm it worked:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 --version\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eIf the \u003ccode class=\"language-markup\"\u003einfluxdb3\u003c/code\u003e command isn’t found, the installer’s output tells you what to do—usually it’s a matter of sourcing your shell config (for example \u003ccode class=\"language-markup\"\u003esource ~/.bashrc or source ~/.zshrc\u003c/code\u003e) so the new binary is on your \u003ccode class=\"language-markup\"\u003ePATH\u003c/code\u003e.\u003c/p\u003e\n\n\u003ch4 id=\"docker\"\u003eDocker\u003c/h4\u003e\n\n\u003cp\u003eIf you’d rather containerize, the Processing Engine is enabled by default in the Docker image (the plugin directory defaults to \u003ccode class=\"language-markup\"\u003e/plugins\u003c/code\u003e):\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003edocker run -it -p 8181:8181 --name influxdb3-core \\\n  --volume ~/.influxdb3_data:/var/lib/influxdb3 \\\n  --volume ~/.influxdb3_plugins:/plugins \\\n  influxdb:3-core influxdb3 serve \\\n  --node-id my_host \\\n  --object-store file \\\n  --data-dir /var/lib/influxdb3 \\\n  --plugin-dir /plugins\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eFor the rest of this tutorial we’ll assume the local binary install. The commands translate directly to Docker by prefixing \u003ccode class=\"language-markup\"\u003edocker exec -it influxdb3-core\u003c/code\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"start-influxdb-with-the-processing-engine-enabled\"\u003eStart InfluxDB with the Processing Engine enabled\u003c/h2\u003e\n\n\u003cp\u003eThe Processing Engine activates only when you tell InfluxDB where your plugins live, using the \u003ccode class=\"language-markup\"\u003e--plugin-dir\u003c/code\u003e flag.\u003c/p\u003e\n\n\u003cp\u003eFirst create a directory to hold plugins, then start the server:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003emkdir -p ~/influxdb3/plugins\n\ninfluxdb3 serve \\\n  --node-id host01 \\\n  --object-store file \\\n  --data-dir ~/.influxdb3 \\\n  --plugin-dir ~/influxdb3/plugins\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eA few notes on these flags:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--node-id\u003c/code\u003e is a unique name for this server instance. It forms part of the storage path.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--object-store file\u003c/code\u003e keeps everything on a local disk under\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--data-dir\u003c/code\u003e. In production you’d point this at S3 or another object store; InfluxDB 3 uses a “diskless” architecture where object storage is the source of truth.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--plugin-dir\u003c/code\u003e is the directory the engine scans for plugin files. This is the switch that turns the Processing Engine on.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eLeave this server running in its own terminal. Open a second terminal for the remaining commands.\u003c/p\u003e\n\n\u003cp\u003eIt’s worth noting that the \u003ccode class=\"language-markup\"\u003einfluxdb3\u003c/code\u003e binary depends on an adjacent \u003ccode class=\"language-markup\"\u003epython/\u003c/code\u003e directory that ships alongside it. If you extracted from a tarball manually, keep the binary and that \u003ccode class=\"language-markup\"\u003epython/\u003c/code\u003e folder in the same parent directory and add the parent to your \u003ccode class=\"language-markup\"\u003ePATH\u003c/code\u003e, don’t move the binary out on its own, or plugins won’t run.\u003c/p\u003e\n\n\u003ch2 id=\"create-an-admin-token\"\u003eCreate an admin token\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB 3 Core uses token authentication. Several operations require an admin token. Create one with the following command:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create token --admin\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThis prints a token string once. Copy it somewhere safe; you can’t recover it later (you’d have to regenerate). For convenience in this session, export it:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003eexport INFLUXDB3_AUTH_TOKEN=\"paste-your-token-here\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThe CLI automatically picks up \u003ccode class=\"language-markup\"\u003eINFLUXDB3_AUTH_TOKEN\u003c/code\u003e, so you won’t have to pass \u003ccode class=\"language-markup\"\u003e--token\u003c/code\u003e on every command.\u003c/p\u003e\n\n\u003ch2 id=\"create-a-database\"\u003eCreate a database\u003c/h2\u003e\n\n\u003cp\u003eCreate a database to hold our data:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3 create database engine_fleet\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eVerify it exists:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3 show databases\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eYou should see \u003ccode class=\"language-markup\"\u003eengine_fleet\u003c/code\u003e in the list.\u003c/p\u003e\n\n\u003ch2 id=\"load-the-dataset-into-influxdb\"\u003eLoad the dataset into InfluxDB\u003c/h2\u003e\n\n\u003cp\u003eThe \u003ca href=\"https://www.kaggle.com/datasets/bishals098/nasa-turbofan-engine-degradation-simulation\"\u003eC-MAPSS\u003c/a\u003e file is a flat, headerless, space-separated table. We need to turn each row into a time series point. There’s one wrinkle worth thinking through: the data has a \u003ccode class=\"language-markup\"\u003etime_cycles\u003c/code\u003e counter per engine, not real timestamps. To make this behave like a live stream, we’ll synthesize timestamps by mapping each engine cycle to one second of wall-clock time, anchored to “now minus the engine’s lifetime.” That way recent cycles look recent, which is what a scheduled “look at the last N minutes” plugin expects.\u003c/p\u003e\n\n\u003cp\u003eHere’s a small loader script. It uses the InfluxDB 3 Python client to write line protocol in batches. First install the client into your local Python (this is separate from the engine’s bundled Python), using \u003ca href=\"https://docs.astral.sh/uv/\"\u003euv\u003c/a\u003e or \u003ca href=\"https://docs.python.org/3/library/venv.html\"\u003evenv\u003c/a\u003e for package management if desired::\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003epip install influxdb3-python pandas\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eSave the following as \u003ccode class=\"language-markup\"\u003eload_cmapss.py\u003c/code\u003e, adjusting \u003ccode class=\"language-markup\"\u003eDATA_FILE\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003eTOKEN\u003c/code\u003e as needed:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003e# load_cmapss.py\nimport time\nfrom datetime import datetime, timedelta, timezone\n\nimport pandas as pd\nfrom influxdb_client_3 import InfluxDBClient3, Point\n\n# ---- Configuration ----\nDATA_FILE = \"train_FD001.txt\"\nHOST = \"http://localhost:8181\"\nDATABASE = \"engine_fleet\"\nTOKEN = \"paste-your-admin-token\"   # or read from env\n\n# ---- Column names (NASA C-MAPSS layout) ----\nindex_cols = [\"unit_number\", \"time_cycles\"]\nsetting_cols = [\"setting_1\", \"setting_2\", \"setting_3\"]\nsensor_cols = [f\"s_{i}\" for i in range(1, 22)]\ncol_names = index_cols + setting_cols + sensor_cols\n\n# ---- Read the space-separated file ----\ndf = pd.read_csv(DATA_FILE, sep=r\"\\s+\", header=None, names=col_names)\ndf = df.astype({\"unit_number\": int, \"time_cycles\": int})\n\nprint(f\"Loaded {len(df)} rows across {df['unit_number'].nunique()} engines\")\n\n# ---- Synthesize timestamps: 1 cycle == 1 second, anchored so the\n#      last cycle of each engine lands at 'now'. ----\nnow = datetime.now(timezone.utc)\nclient = InfluxDBClient3(host=HOST, database=DATABASE, token=TOKEN)\n\npoints = []\nBATCH = 5000\n\n# Per-engine max cycle so we can anchor each engine's final cycle to \"now\"\nmax_cycle = df.groupby(\"unit_number\")[\"time_cycles\"].transform(\"max\")\ndf[\"ts\"] = [\n    now - timedelta(seconds=int(mc - tc))\n    for mc, tc in zip(max_cycle, df[\"time_cycles\"])\n]\n\nfor row in df.itertuples(index=False):\n    p = (\n        Point(\"sensors\")\n        .tag(\"unit_number\", str(row.unit_number))\n        .field(\"time_cycles\", int(row.time_cycles))\n    )\n    for c in setting_cols + sensor_cols:\n        p = p.field(c, float(getattr(row, c)))\n    p = p.time(row.ts)\n    points.append(p)\n\n    if len(points) \u0026gt;= BATCH:\n        client.write(points)\n        points = []\n\nif points:\n    client.write(points)\n\nclient.close()\nprint(\"Done writing sensor data.\")\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eRun it:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003epython load_cmapss.py\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eA few design choices worth calling out:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003e\u003ccode class=\"language-markup\"\u003eunit_number\u003c/code\u003e is a tag, everything else is a field.\u003c/strong\u003e Tags are indexed and are how you separate one engine’s series from another. Sensor values and the cycle counter are fields.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eWe anchor each engine’s final cycle to “now.”\u003c/strong\u003e This means every engine in the fleet looks like it just reached end-of-life, which is convenient for demonstrating alerts. In a real deployment your data already has real timestamps and you’d skip all of this.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eBatching matters.\u003c/strong\u003e  Writing 20,000+ points one at a time is slow; batches of a few thousand keep the loader quick.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eConfirm the data landed:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query --database engine_fleet \\\n  \"SELECT COUNT(*) FROM sensors\"\nAnd take a peek at one engine's recent readings:\ninfluxdb3 query --database engine_fleet \\\n  \"SELECT time, unit_number, time_cycles, s_2, s_4, s_11 \\\n   FROM sensors WHERE unit_number = '1' \\\n   ORDER BY time DESC LIMIT 5\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003ch2 id=\"an-overview-of-the-plugin-model\"\u003eAn overview of the plugin model\u003c/h2\u003e\n\n\u003cp\u003eBefore writing code, let’s get the mental model straight.\u003c/p\u003e\n\n\u003cp\u003eA plugin is a Python file (or a directory with an \u003ccode class=\"language-markup\"\u003e__init__.py\u003c/code\u003e for multi-file plugins) placed in your plugin directory. It defines a function whose signature matches the trigger type:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eData-write plugin: \u003ccode class=\"language-markup\"\u003edef process_writes(influxdb3_local, table_batches, args=None):\u003c/code\u003e\u0026lt;/pre\u0026gt;\u003c/li\u003e\n  \u003cli\u003eScheduled plugin: \u003ccode class=\"language-markup\"\u003edef process_scheduled_call(influxdb3_local, call_time, args=None):\u003c/code\u003e\u0026lt;/pre\u0026gt;\u003c/li\u003e\n  \u003cli\u003eHTTP-request plugin: \u003ccode class=\"language-markup\"\u003edef process_request(influxdb3_local, query_parameters, request_headers, request_body, args=None):\u003c/code\u003e\u0026lt;/pre\u0026gt;\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eA trigger is the database resource that connects an event to a plugin. You create it with \u003ccode class=\"language-markup\"\u003einfluxdb3 create trigger\u003c/code\u003e, specifying a \u003cem\u003etrigger spec\u003c/em\u003e that defines when the plugin runs.\u003c/p\u003e\n\n\u003cp\u003eWhatever the type, every plugin receives \u003ccode class=\"language-markup\"\u003einfluxdb3_local\u003c/code\u003e which is the shared API object that’s your gateway to the database. The methods we’ll use:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3_local.query(sql)\u003c/code\u003e - run a SQL query and get results back as a list of dictionaries.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3_local.write(line)\u003c/code\u003e - write a point back into the database, built with the \u003ccode class=\"language-markup\"\u003eLineBuilder\u003c/code\u003e helper.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3_local.info(msg)\u003c/code\u003e/ \u003ccode class=\"language-markup\"\u003e.warn(msg)\u003c/code\u003e / \u003ccode class=\"language-markup\"\u003e.error(msg)\u003c/code\u003e - log to stdout and the system.processing_engine_logs table.\u003c/li\u003e\n  \u003cli\u003eThe engine also offers an in-memory cache for keeping state between runs (useful for things like tracking a rolling baseline), though we’ll keep our first version stateless.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003eLineBuilder\u003c/code\u003e is the recommended way to construct a point inside a plugin:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003eline = LineBuilder(\"my_table\")\nline.tag(\"device\", \"pump_7\")\nline.float64_field(\"value\", 42.0)\nline.int64_field(\"count\", 3)\ninfluxdb3_local.write(line)\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003ePlugins receive trigger arguments as a \u003ccode class=\"language-markup\"\u003eDict[str, str]\u003c/code\u003e in \u003ccode class=\"language-markup\"\u003eargs\u003c/code\u003e, which is how we’ll pass tunable thresholds without editing code.\u003c/p\u003e\n\n\u003ch2 id=\"predictive-maintenance-plugin-logic\"\u003ePredictive maintenance plugin logic\u003c/h2\u003e\n\n\u003cp\u003eWe need to turn raw sensor readings into a Remaining Useful Life estimate. A full production system might run a trained LSTM or gradient-boosted model, but a tutorial plugin should be transparent and dependency-light, so we’ll use a degradation-index approach that captures the real idea behind RUL prediction without a black box:\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003ePick the sensors that carry degradation signals. In FD001, several sensors drift steadily as the high-pressure compressor wears. Sensors s_2, s_3, s_4, s_8, s_11, s_13, and s_15 trend upward over an engine’s life; s_7, s_12, s_20, and s_21 trend downward. (Some sensors in FD001 are flat and carry no information—we ignore those.)\u003c/li\u003e\n  \u003cli\u003eNormalize each chosen sensor against the healthy-baseline range so they’re comparable, flipping the downward-trending ones so “more degraded” always means “higher.”\u003c/li\u003e\n  \u003cli\u003eAverage them into a single health index between roughly 0 (factory-fresh) and 1 (failure imminent).\u003c/li\u003e\n  \u003cli\u003eMap the index to an RUL estimate. Using the dataset convention that degradation becomes meaningfully predictable in roughly the final 125 cycles, we estimate \u003ccode class=\"language-markup\"\u003eRUL ≈ 125 × (1 − health_index)\u003c/code\u003e.\u003c/li\u003e\n  \u003cli\u003eAlert when the estimated RUL drops below a configurable threshold.\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThis is intentionally simple and explainable. The plugin structure is identical whether the scoring function is ten lines of arithmetic or a 50-megabyte neural network,  you’d just swap the body of \u003ccode class=\"language-markup\"\u003ecompute_health_index\u003c/code\u003e.\u003c/p\u003e\n\n\u003cp\u003eTo normalize, the plugin needs each sensor’s healthy and degraded reference values. Rather than hard-code magic numbers, we’ll derive them once at the top of each run from the fleet itself with the early-life readings approximate “healthy” and the latest readings approximate “degraded.”\u003c/p\u003e\n\n\u003ch2 id=\"creating-the-plugin\"\u003eCreating the plugin\u003c/h2\u003e\n\n\u003cp\u003eCreate the plugin file directly in your plugin directory. Save this as \u003ccode class=\"language-markup\"\u003e~/influxdb3/plugins/predictive_maintenance.py\u003c/code\u003e.\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003e# predictive_maintenance.py\n#\n# Scheduled plugin: estimates Remaining Useful Life (RUL) for each engine\n# from the most recent sensor readings, writes the estimates back into the\n# database, and raises alerts when RUL drops below a threshold.\n\n# Sensors that rise as the engine degrades\nRISING = [\"s_2\", \"s_3\", \"s_4\", \"s_8\", \"s_11\", \"s_13\", \"s_15\"]\n# Sensors that fall as the engine degrades\nFALLING = [\"s_7\", \"s_12\", \"s_20\", \"s_21\"]\n\n# Convention from the C-MAPSS literature: degradation becomes meaningfully\n# predictable in roughly the final 125 cycles.\nMAX_PREDICTABLE_RUL = 125.0\n\ndef _safe_float(value, default=0.0):\n    try:\n        return float(value)\n    except (TypeError, ValueError):\n        return default\n\ndef compute_baselines(influxdb3_local):\n    \"\"\"Derive healthy and degraded reference values for each sensor.\n\n    Healthy  ~ average of the earliest cycles across the fleet.\n    Degraded ~ average of the latest cycles across the fleet.\n    \"\"\"\n    sensors = RISING + FALLING\n    avg_cols = \", \".join([f\"AVG({s}) AS {s}\" for s in sensors])\n\n    healthy_rows = influxdb3_local.query(\n        f\"SELECT {avg_cols} FROM sensors WHERE time_cycles = 15\"\n    )\n    degraded_rows = influxdb3_local.query(\n        f\"SELECT {avg_cols} FROM sensors WHERE time_cycles = 175\"\n    )\n\n    if not healthy_rows or not degraded_rows:\n        return None\n\n    healthy = healthy_rows[0]\n    degraded = degraded_rows[0]\n\n    baselines = {}\n    for s in sensors:\n        lo = _safe_float(healthy.get(s))\n        hi = _safe_float(degraded.get(s))\n        baselines[s] = (lo, hi)\n    return baselines\n\ndef normalize(value, lo, hi):\n    \"\"\"Scale a reading to 0..1 between the healthy (lo) and degraded (hi)\n    references. Clamps to the [0, 1] range.\"\"\"\n    if hi == lo:\n        return 0.0\n    frac = (value - lo) / (hi - lo)\n    return max(0.0, min(1.0, frac))\n\ndef compute_health_index(reading, baselines):\n    \"\"\"Combine the signal-bearing sensors into a single 0..1 degradation\n    index. 0 = healthy, 1 = failure imminent.\"\"\"\n    scores = []\n\n    for s in RISING:\n        lo, hi = baselines[s]\n        scores.append(normalize(_safe_float(reading.get(s)), lo, hi))\n\n    for s in FALLING:\n        lo, hi = baselines[s]\n        # Falling sensors: degraded value is lower, so invert the scale.\n        scores.append(1.0 - normalize(_safe_float(reading.get(s)), hi, lo))\n\n    if not scores:\n        return 0.0\n    return sum(scores) / len(scores)\n\ndef process_scheduled_call(influxdb3_local, call_time, args=None):\n    # --- Read tunables from trigger arguments ---\n    args = args or {}\n    rul_threshold = float(args.get(\"rul_threshold\", \"30\"))\n    lookback = args.get(\"lookback\", \"10m\")\n\n    influxdb3_local.info(\n        f\"Predictive maintenance run starting. \"\n        f\"rul_threshold={rul_threshold}, lookback={lookback}\"\n    )\n\n    # --- Build per-sensor baselines from the fleet ---\n    baselines = compute_baselines(influxdb3_local)\n    if baselines is None:\n        influxdb3_local.warn(\"Not enough data to compute baselines; skipping run.\")\n        return\n\n    # --- Get the most recent reading per engine ---\n    sensor_select = \", \".join(RISING + FALLING)\n    latest = influxdb3_local.query(\n        f\"\"\"\n        SELECT unit_number, time_cycles, {sensor_select}\n        FROM sensors\n        WHERE time \u0026gt;= now() - INTERVAL '{lookback}'\n        ORDER BY time DESC\n        \"\"\"\n    )\n\n    if not latest:\n        influxdb3_local.warn(\"No recent sensor data in lookback window.\")\n        return\n\n    # Keep only the newest row per engine (results are time-desc ordered).\n    seen = set()\n    newest_per_engine = []\n    for row in latest:\n        unit = row.get(\"unit_number\")\n        if unit not in seen:\n            seen.add(unit)\n            newest_per_engine.append(row)\n\n    alerts = 0\n    for row in newest_per_engine:\n        unit = str(row.get(\"unit_number\"))\n        cycles = int(_safe_float(row.get(\"time_cycles\")))\n\n        health = compute_health_index(row, baselines)\n        est_rul = round(MAX_PREDICTABLE_RUL * (1.0 - health), 1)\n\n        # --- Write the RUL estimate ---\n        est = LineBuilder(\"rul_estimates\")\n        est.tag(\"unit_number\", unit)\n        est.float64_field(\"health_index\", round(health, 4))\n        est.float64_field(\"estimated_rul\", est_rul)\n        est.int64_field(\"time_cycles\", cycles)\n        influxdb3_local.write(est)\n\n        # --- Raise an alert if the engine is in the danger zone ---\n        if est_rul = rul_threshold:\n            alerts += 1\n            severity = \"critical\" if est_rul = rul_threshold / 2 else \"warning\"\n\n            alert = LineBuilder(\"maintenance_alerts\")\n            alert.tag(\"unit_number\", unit)\n            alert.tag(\"severity\", severity)\n            alert.float64_field(\"estimated_rul\", est_rul)\n            alert.float64_field(\"health_index\", round(health, 4))\n            influxdb3_local.write(alert)\n\n            influxdb3_local.warn(\n                f\"[{severity.upper()}] Engine {unit}: estimated RUL \"\n                f\"{est_rul} cycles (health index {round(health, 2)}). \"\n                f\"Schedule maintenance.\"\n            )\n\n    influxdb3_local.info(\n        f\"Run complete. Scored {len(newest_per_engine)} engines, \"\n        f\"raised {alerts} alert(s).\"\n    )\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eLet’s walk through the important parts.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eBaselines from the data help us avoid any magic numbers.\u003c/strong\u003e - \u003ccode class=\"language-markup\"\u003ecompute_baselines\u003c/code\u003e asks the database for the average sensor values during early life (cycles ≤ 15, “healthy”) and late life (cycles ≥ 175, “degraded”). This adapts automatically to your fleet and means you don’t have to know the absolute scale of s_4 in advance.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eA single, explainable health index.\u003c/strong\u003e \u003ccode class=\"language-markup\"\u003ecompute_health_index\u003c/code\u003e normalizes each signal-bearing sensor onto a 0–1 scale and averages them. Rising and falling sensors are both oriented so that 1 always means “more degraded.” This is the piece you’d replace with a trained model in production — the surrounding plumbing stays identical.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eNewest reading per engine.\u003c/strong\u003e The query pulls everything in the lookback window ordered newest-first, then we keep the first row we see for each \u003ccode class=\"language-markup\"\u003eunit_number\u003c/code\u003e. Because of how we loaded the data (each engine’s last cycle anchored to “now”), the most recent rows are the most degraded.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eWriting conclusions back.\u003c/strong\u003e Every engine gets a row in \u003ccode class=\"language-markup\"\u003erul_estimates\u003c/code\u003e. Engines past the threshold also get a row in \u003ccode class=\"language-markup\"\u003emaintenance_alerts\u003c/code\u003e, tagged with a \u003ccode class=\"language-markup\"\u003eseverity\u003c/code\u003e derived from how deep into the danger zone they are, plus a logged warning you’ll see in the server terminal.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eConfigurable via trigger arguments.\u003c/strong\u003e \u003ccode class=\"language-markup\"\u003erul_threshold\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003elookback\u003c/code\u003e come from \u003ccode class=\"language-markup\"\u003eargs\u003c/code\u003e, so you can retune behavior by recreating the trigger.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"creating-the-trigger\"\u003eCreating the trigger\u003c/h2\u003e\n\n\u003cp\u003eNow connect the plugin to a schedule. We’ll run it every minute, with a 30-cycle RUL alert threshold and a 10-minute lookback window:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create trigger \\\n  --trigger-spec \"every:1m\" \\\n  --path \"predictive_maintenance.py\" \\\n  --trigger-arguments \"rul_threshold=30,lookback=10m\" \\\n  --database engine_fleet \\\n  pdm_scheduler\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eBreaking that down:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--trigger-spec \"every:1m\"\u003c/code\u003e runs the plugin once a minute. You could also use \u003ccode class=\"language-markup\"\u003ecron:\u003c/code\u003e for calendar schedules, for example \u003ccode class=\"language-markup\"\u003ecron:0 0 8 * * *\u003c/code\u003e for 8am daily (the format includes seconds).\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--path \"predictive_maintenance.py\"\u003c/code\u003e is the filename relative to your plugin directory. (For a multi-file plugin you’d point this at the directory containing \u003ccode class=\"language-markup\"\u003e__init__.py\u003c/code\u003e instead.)\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e--trigger-arguments\u003c/code\u003e passes our tunables as key=value pairs.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003epdm_scheduler\u003c/code\u003e is the trigger’s name.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThe trigger is enabled by default and starts running on the next interval boundary.\u003c/p\u003e\n\n\u003ch2 id=\"testing-out-the-plugin\"\u003eTesting out the plugin\u003c/h2\u003e\n\n\u003cp\u003eWithin a minute, the plugin fires. Check the server terminal and you’ll see the \u003ccode class=\"language-markup\"\u003einfo\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003ewarn\u003c/code\u003e log lines. Now query what it produced.\u003c/p\u003e\n\n\u003cp\u003eThe latest RUL estimates, most-degraded first:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query --database engine_fleet \\\n  \"SELECT unit_number, estimated_rul, health_index, time_cycles \\\n   FROM rul_estimates \\\n   ORDER BY time DESC, estimated_rul ASC LIMIT 15\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThe maintenance alerts:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query --database engine_fleet \\\n  \"SELECT time, unit_number, severity, estimated_rul, health_index \\\n   FROM maintenance_alerts \\\n   ORDER BY time DESC LIMIT 20\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eIf you want to confirm the plugin is registered and see its file details:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 show plugins\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eAnd the engines flagged critical right now:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query --database engine_fleet \\\n  \"SELECT unit_number, estimated_rul FROM maintenance_alerts \\\n   WHERE severity = 'critical' ORDER BY estimated_rul ASC\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eYou should see a spread of RUL estimates across the fleet, with the most-degraded engines surfacing as warnings or criticals.\u003c/p\u003e\n\n\u003ch2 id=\"inspecting-logs-and-iterating\"\u003eInspecting logs and iterating\u003c/h2\u003e\n\n\u003cp\u003ePlugin logs go both to the server’s stdout and to a system table, which is handy for debugging without scrolling the terminal:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 query --database engine_fleet \\\n  \"SELECT * FROM system.processing_engine_logs ORDER BY time DESC LIMIT 20\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eWhen you change the plugin code, you don’t need to recreate the trigger. Edit the file and push the update, preserving the trigger’s configuration and history:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 update trigger \\\n  --database engine_fleet \\\n  --trigger-name pdm_scheduler \\\n  --path \"predictive_maintenance.py\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eFor local development you can also develop a plugin on your own machine and upload it with \u003ccode class=\"language-markup\"\u003e--upload\u003c/code\u003e when creating or updating a trigger, which copies the file to the server for you (this requires an admin token). And if you want to dry-run a scheduled plugin without waiting for the interval, there’s \u003ccode class=\"language-markup\"\u003einfluxdb3 test schedule_plugin\u003c/code\u003e.\u003c/p\u003e\n\n\u003cp\u003eTo pause the system without deleting anything:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 disable trigger --database engine_fleet pdm_scheduler\n# ...and later...\ninfluxdb3 enable trigger --database engine_fleet pdm_scheduler\u003c/code\u003e\u003c/pre\u003e\n\n\u003ch4 id=\"error-handling\"\u003eError Handling\u003c/h4\u003e\n\n\u003cp\u003eBy default, plugin errors are logged and the trigger keeps going. For a critical pipeline you might prefer automatic retries or auto-disable. Set this when creating the trigger with \u003ccode class=\"language-markup\"\u003e--error-behavior retry\u003c/code\u003e or \u003ccode class=\"language-markup\"\u003e--error-behavior disable\u003c/code\u003e (the default is log).\u003c/p\u003e\n\n\u003ch2 id=\"what-to-build-next\"\u003eWhat to build next\u003c/h2\u003e\n\n\u003cp\u003eYou now have a complete, self-contained predictive maintenance loop running inside InfluxDB 3 \nCore. Some natural extensions:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eSwap in a real model.\u003c/strong\u003e Replace \u003ccode class=\"language-markup\"\u003ecompute_health_index\u003c/code\u003e with a trained regressor or classifier. Install the library with \u003ccode class=\"language-markup\"\u003einfluxdb3 install package\u003c/code\u003e, load your serialized model at the top of the plugin, and predict per engine. The trigger, the writes, and the alerts don’t change.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eAdd a notifier.\u003c/strong\u003e Have the alert branch call an external service like Slack, PagerDuty, email directly from Python. InfluxData also publishes an official notifier plugin you can compose with.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eUse the engine’s cache for stateful detection.\u003c/strong\u003e Track a rolling baseline or a per-engine trend slope across runs instead of recomputing fleet baselines each time, using the in-memory cache to persist state between executions.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eLower the latency.\u003c/strong\u003e Move from a scheduled trigger to a data-write trigger if you need to react the moment a reading crosses a line.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eBrowse the official plugin library.\u003c/strong\u003e InfluxData maintains a public repository of plugins (anomaly detection via MAD, threshold/deadman checks, Prophet forecasting, downsampling, and more) that you can reference directly in a trigger with the \u003ccode class=\"language-markup\"\u003egh:\u003c/code\u003e prefix, or copy and adapt.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThe broader point is the pattern: with the InfluxDB 3 Processing Engine, the intelligence lives in the database, next to the data, reacting as the data moves. For time-series-heavy domains like industrial IoT and predictive maintenance, that proximity is exactly what you want.\u003c/p\u003e\n","date_published":"2026-06-09T08:00:00+00:00","authors":[{"name":"Charles Mahler"}]},{"id":"https://www.influxdata.com/blog/river-plugins-influxdb-3","url":"https://www.influxdata.com/blog/river-plugins-influxdb-3","title":"Anomaly Detection and Forecasting That Learns From Every Write in InfluxDB","content_html":"\u003cp\u003eFor many operational time series workloads, machine learning can’t operate in the historical way, where data is compiled once and models are trained offline. Sensor readings, infrastructure metrics, application telemetry, energy data, industrial measurements, and financial ticks all share a basic property: the next datapoint is more useful when the system can respond to it immediately (or at least close to immediately). When a model learns in the same flow that ingests data and reacts to incoming data as it’s written, things like anomaly detection, short-horizon forecasting, and adaptive thresholding all become a lot more useful.\u003c/p\u003e\n\n\u003cp\u003eEnter three new \u003ca href=\"https://riverml.xyz/dev/\"\u003eRiver\u003c/a\u003e-based plugins for InfluxDB 3:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/river_anomaly_detector/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=river_plugins_influxdb_3\u0026amp;utm_content=blog\"\u003eRiver Anomaly Detector\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/river_auto_profiler/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=river_plugins_influxdb_3\u0026amp;utm_content=blog\"\u003eRiver Auto-Profiler\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/river_forecaster/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=river_plugins_influxdb_3\u0026amp;utm_content=blog\"\u003eRiver Forecaster\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThese plugins are built for the InfluxDB 3 Processing Engine and leverage River, a Python library for online machine learning. If you’re unfamiliar with it, the Processing Engine is an embedded Python VM that runs inside InfluxDB 3 and can execute plugin code on writes, schedules, or HTTP requests; it also provides an in-memory cache for stateful applications. For write-triggered plugins, InfluxDB 3 can send batches of data to a plugin as data is flushed through the write-ahead log. River, meanwhile, is designed for models that learn from streaming data incrementally, including anomaly detection, drift detection, and time series forecasting.\u003c/p\u003e\n\n\u003cp\u003eThat combination makes InfluxDB 3 and River a perfect match. These plugins bring small, per-series, constantly-updating models directly into the write path, then write the resulting profiles, anomalies, and forecasts back into InfluxDB tables where they can be queried like any other time series data or combined with other triggers and plugins to kick off informed actions. Better yet, they do this all within InfluxDB, eliminating the need for extra infrastructure, servers, and data pipelines to fuel your ML models. In this blog, we’re going to talk about all three new River plugins for InfluxDB, and how they can not only simplify your ML stack, but lead to faster insights through online modeling.\u003c/p\u003e\n\n\u003ch2 id=\"river-anomaly-detector-multiple-ways-to-detect-problematic-data\"\u003eRiver Anomaly Detector: multiple ways to detect problematic data\u003c/h2\u003e\n\n\u003cp\u003eLet’s start with the River Anomaly Detector.\nAt a high level, the plugin monitors numeric fields and writes anomaly rows to the table \u003ccode class=\"language-markup\"\u003e_anomalies.{source_table}\u003c/code\u003e. It supports rolling Z-score detection, seasonal detection, and \u003ca href=\"https://riverml.xyz/dev/api/drift/ADWIN/\"\u003eadaptive window (ADWIN)\u003c/a\u003e-based drift detection. Each unique combination of table, tags, and field has its own detector state, and models are updated incrementally as new observations arrive.\u003c/p\u003e\n\n\u003cp\u003eThe rolling detector tracks an exponentially weighted mean and variance, then flags values that exceed the learned range by a configurable number of standard deviations. The seasonal detector maintains separate time-based buckets, either 24 hour-of-day buckets or 168 hour-of-week buckets, and it compares new values against the bucket that matches the current timestamp. The ADWIN detector watches for changes in the statistical properties of the stream, which is useful when the issue is a behavior shift rather than a single spike.\u003c/p\u003e\n\n\u003cp\u003eDetectors learn from every observation, and you can choose to enable one, two, or all three detector modes, but only the detectors active in the current mode can vote on whether a point is anomalous. This allows you to train the various forms of anomaly detection, then modify your trigger once enough data has been provided to the model. This way, a stream can accumulate enough seasonal history before the seasonal detector is allowed to participate in decisions, and ADWIN can keep learning even when the current mode is using a simpler rolling Z-score path. You don’t even need to specify which detector to use; run the plugin with a minimal trigger such as the following:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 install package river\n\ninfluxdb3 create trigger \\\n --database mydb \\\n --plugin-filename gh:influxdata/river_anomaly_detector/river_anomaly_detector.py \\\n --trigger-spec \"all_tables\" \\\n anomaly_detector\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eWith that trigger in place, the plugin starts monitoring numeric fields in all incoming tables. The default behavior is to “auto-tune” and read recommendations from the River Auto Profiling plugin, which we’ll dive into more in the next section.\nA typical query against the output might look like this:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eSELECT\n time,\n host,\n field_name,\n original_value,\n detector_mode,\n rolling_mean,\n rolling_std,\n rolling_deviation,\n rolling_threshold,\n seasonal_bucket,\n drift_detected\nFROM \"_anomalies.cpu\"\nORDER BY time DESC\nLIMIT 20;\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThat output is deliberately detailed: you get the original value, the active detector mode, rolling statistics, seasonal statistics when available, and ADWIN drift information. The plugin only writes rows where a datapoint is determined to be an anomaly, so the anomaly table is an event stream rather than a full copy of the source data.\u003c/p\u003e\n\n\u003cp\u003eFor a stable metric, the detector may run with a lower Z-score threshold because small deviations are meaningful. For a noisy metric, it may use a higher threshold to avoid producing noise. For a metric with a strong weekly pattern, seasonal buckets can separate “expected at 2 p.m. on Monday” from “expected in general.” For a metric that is drifting, ADWIN can participate so the detector can respond to changes in the stream rather than treating every shift as a one-off outlier.\u003c/p\u003e\n\n\u003cp\u003eYou can also make the mode explicit. For example, if you only care about a specific table and you know it has strong daily or weekly seasonality on a couple fields you want to monitor, you can specify this and force the plugin to only use seasonal detection:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create trigger \\\n --database mydb \\\n --plugin-filename gh:influxdata/river_anomaly_detector/river_anomaly_detector.py \\\n --trigger-spec \"table:cpu\" \\\n --trigger-arguments \\\n   'include_fields=temperature humidity' \\\n   'rolling_std_threshold=3.0' \\\n   'enable_seasonal=true' \\\n anomaly_detector_cpu\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThe detector can encode understanding in the database itself, close to the data, while still leaving room for explicit configuration when you know exactly what you want. Whether you want adaptive detection, rolling Z-scores, seasonal detection, or some combination of the three, the River Anomaly Detector plugin can make it happen, and it’s truly as simple as defining the tables and fields you want it to operate on.\u003c/p\u003e\n\n\u003ch2 id=\"river-auto-profiler-the-control-plane-for-per-series-tuning\"\u003eRiver Auto-Profiler: the control plane for per-series tuning\u003c/h2\u003e\n\n\u003cp\u003eThe Auto-Profiler is the least flashy of the three plugins, but it is the one that makes the anomaly detector more practical at scale.\u003c/p\u003e\n\n\u003cp\u003eA static threshold is easy to understand and easy to deploy. It is also usually wrong somewhere. A metric that barely moves, a metric with high variance, a metric with a weekly pattern, and a metric undergoing a slow drift should not all use the same anomaly detection settings. You can tune each field by hand, but that does not scale well when your schema grows or when the behavior of a series changes over time.\u003c/p\u003e\n\n\u003cp\u003eThe Auto-Profiler addresses that by incrementally profiling each numeric series and writing recommendations to \u003ccode class=\"language-markup\"\u003e_meta.series_profiles\u003c/code\u003e. It tracks exponentially weighted mean and variance, skewness, \u003ca href=\"https://en.wikipedia.org/wiki/Kurtosis\"\u003ekurtosis\u003c/a\u003e, write interval, and seasonal variance buckets. After a short calibration phase, it writes profile snapshots that include a pattern label, recommended detector mode, threshold, fading factors, seasonality strength, trend strength, and maturity flags.\u003c/p\u003e\n\n\u003cp\u003eCreating the trigger is intentionally simple:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create trigger \\\n --database mydb \\\n --plugin-filename gh:influxdata/river_auto_profiler/river_auto_profiler.py \\\n --trigger-spec \"all_tables\" \\\n auto_profiler\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThen you can inspect the current profile state:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eSELECT\n time,\n source_table,\n host,\n field_name,\n observations,\n pattern_label,\n recommended_detector_mode,\n recommended_threshold,\n recommended_fading_factor,\n seasonality_strength,\n trend_strength,\n profile_mature,\n seasonality_ready\nFROM \"_meta.series_profiles\"\nORDER BY time DESC\nLIMIT 20;\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThe classifier is intentionally transparent. It labels streams as stable, noisy, trending, seasonal, or bursty and maps those labels to detector modes such as \u003ccode class=\"language-markup\"\u003ezscore_low\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003ezscore_high\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003ezscore_adaptive\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003ezscore_conservative seasonal\u003c/code\u003e, or \u003ccode class=\"language-markup\"\u003ezscore_conservative adwin\u003c/code\u003e. Seasonality detection uses hourly or weekly variance buckets, and trend detection compares fast and slow exponentially weighted means.\u003c/p\u003e\n\n\u003cp\u003eThat transparency is useful. If a profile says a series is seasonal, you can look at the \u003ccode class=\"language-markup\"\u003eseasonality_strength\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003eseasonal_buckets_filled\u003c/code\u003e fields. If it says a series is trending, you can inspect \u003ccode class=\"language-markup\"\u003etrend_strength\u003c/code\u003e. If the profile is not mature yet, downstream consumers can fall back to safer defaults.\u003c/p\u003e\n\n\u003cp\u003eThe Auto-Profiler also tunes thresholds using observed exceedance behavior. It tracks how often observations fall outside \u003ccode class=\"language-markup\"\u003emean ± threshold × std\u003c/code\u003e and adjusts the threshold toward an approximate target anomaly rate, with bounds to avoid unbounded sensitivity changes. That does not make anomaly detection “automatic” in the magical sense. It makes it adaptive in the operational sense: the system can use what it has learned about each series to pick a more appropriate starting point.\u003c/p\u003e\n\n\u003cp\u003eThe most important integration is with the anomaly detector. With both plugins deployed, the profiler writes per-series recommendations, and the anomaly detector consumes those recommendations to adapt to your data, weed out anomalies, and minimize noise.\u003c/p\u003e\n\n\u003ch2 id=\"river-forecaster-short-horizon-forecasts-from-the-write-stream\"\u003eRiver Forecaster: short-horizon forecasts from the write stream\u003c/h2\u003e\n\n\u003cp\u003eThe River Forecaster takes the same online-learning pattern and applies it to forecasting.\nThe plugin uses River’s \u003ca href=\"https://riverml.xyz/dev/api/time-series/SNARIMAX/\"\u003eSNARIMAX\u003c/a\u003e model, an online \u003ca href=\"https://www.ibm.com/think/topics/arima-model\"\u003eARIMA\u003c/a\u003e-style model, to learn from incoming values and periodically write multi-step forecasts to \u003ccode class=\"language-markup\"\u003e_forecasts.{source_table}\u003c/code\u003e. Like the anomaly detector, it keeps a separate model per table/tag/field series. Unlike the anomaly detector, it requires explicit table selection through \u003ccode class=\"language-markup\"\u003einclude_tables\u003c/code\u003e, which is a necessary guardrail for forecast cardinality.\u003c/p\u003e\n\n\u003cp\u003eA trigger for the forecasting plugin might look like this:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create trigger \\\n --database mydb \\\n --plugin-filename gh:influxdata/river_forecaster/river_forecaster.py \\\n --trigger-spec \"all_tables\" \\\n --trigger-arguments \\\n   \"include_tables=system_cpu system_memory\" \\\n   \"include_fields=idle used available\" \\\n   \"max_series=100\" \\\n   \"default_horizon=24\" \\\n   \"log_forecasts=true\" \\\n forecaster\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eBy default, the model waits for a warm-up period before producing forecasts. The forecast horizon is also derived from the stream. The plugin tracks each model’s write interval and uses it to choose how many future steps are needed to cover a target forecast window, which defaults to one hour. If a series is written every 10 seconds, the forecast horizon can be much longer than a series written every five minutes, because the time window is the same, but the step size is different.\u003c/p\u003e\n\n\u003cp\u003eYou can query forecasts like this:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eSELECT\n time,\n host,\n field_name,\n horizon_step,\n forecast_value,\n horizon_total,\n observations\nFROM \"_forecasts.system_cpu\"\nORDER BY time DESC\nLIMIT 12;\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eEach forecast row receives a future timestamp based on the last observed timestamp plus \u003ccode class=\"language-markup\"\u003ehorizon_step * write_interval_seconds\u003c/code\u003e. The output includes the forecasted value, the step number, total horizon length, and the number of observations the model has learned from.\u003c/p\u003e\n\n\u003cp\u003eThe forecaster also self-throttles per model. It produces a forecast, stores those predictions in memory, and then compares them step by step against incoming actuals. A new forecast is produced once the previous forecast horizon has been consumed, so evaluation covers the full horizon rather than a partial subset.\u003c/p\u003e\n\n\u003cp\u003eThat makes the plugin useful for near-term operational questions. For the example above (forecasting CPU and system memory utilization), these questions might be:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eIs memory usage expected to cross a threshold in the next hour?\u003c/li\u003e\n  \u003cli\u003eIs a sensor trending toward a range that usually precedes maintenance?\u003c/li\u003e\n  \u003cli\u003eIs the current CPU-idle forecast consistent with what the anomaly detector is seeing?\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThese are not the only forecasting questions you might ask of time series data, but they are the kind that fit well in a write-triggered architecture.\u003c/p\u003e\n\n\u003ch2 id=\"using-the-river-plugins-with-influxdb\"\u003eUsing the River plugins with InfluxDB\u003c/h2\u003e\n\n\u003cp\u003eWith the simplicity of InfluxDB 3’s embedded Processing Engine, getting started with notoriously difficult or complicated ML tasks is now simpler, faster, and easier than ever. For all three of these River ML plugins, you can view the code and README files within the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=river_plugins_influxdb_3\u0026amp;utm_content=blog\"\u003eInfluxDB 3 plugin repository\u003c/a\u003e, with full documentation for configuration and recommended use. Installing them is as simple as defining and starting the triggers within the processing engine.\u003c/p\u003e\n\n\u003cp\u003eMake sure you’re intentional about which tables and fields you monitor. You should think about cold starts, profile maturity, and whether a given field has enough regularity for short-horizon forecasting. These questions exist whenever you’re deploying ML techniques on data—this isn’t magic.\u003c/p\u003e\n\n\u003cp\u003eOnce you have the plugins up and running, everything should work seamlessly as described above, adapting and learning from the data written to your tables. All three plugins’ outputs remain in InfluxDB. Profiles are written to  \u003ccode class=\"language-markup\"\u003e_meta.series_profiles\u003c/code\u003e. Any detected anomalies are written to \u003ccode class=\"language-markup\"\u003e_anomalies.{source_table}\u003c/code\u003e. Forecasts are written to \u003ccode class=\"language-markup\"\u003e_forecasts.{source_table}\u003c/code\u003e.\u003c/p\u003e\n\n\u003cp\u003eThat means the operational burden is small, and you can query profiles, anomalies, forecasts, and raw data with SQL. You can visualize your River ML-generated data in dashboards alongside the raw and processed data you’ve written into InfluxDB, join different datasets together, and inspect behavior, anomalies, and forecasts without needing to involve any other services or platforms. You don’t need additional hardware or pipelines or self-written plugins to extract your time series data into an offline system for ML. It may just be the easiest way to leverage your time series data for machine learning out there. By embedding these plugins within InfluxDB, your training workflows and ML insights can now live closer to the raw data than ever before.\u003c/p\u003e\n","date_published":"2026-06-04T08:00:00+00:00","authors":[{"name":"Cole Bowden"},{"name":"Ryan Nelson"}]},{"id":"https://www.influxdata.com/blog/modernize-your-historian-6-signs","url":"https://www.influxdata.com/blog/modernize-your-historian-6-signs","title":"6 Signs Your Historian Renewal Should Be a Modernization Conversation","content_html":"\u003cp\u003eRenewal notices don’t arrive with a subject line that says “modernization conversation enclosed.” They show up as a line item in procurement, usually with a larger number than the previous year. Most teams sign and move on.\u003c/p\u003e\n\n\u003cp\u003eThe ones that pause tend to find the renewal moment is a useful forcing function, not to rip out the historian, but to ask whether the current architecture can support what the business needs over the next three years.\u003c/p\u003e\n\n\u003cp\u003eIf any of the following sounds familiar, that audit is worth running before you sign.\u003c/p\u003e\n\n\u003ch2 id=\"sign-1-your-renewal-quote-is-higher-and-so-is-your-tag-count\"\u003eSign 1: Your renewal quote is higher, and so is your tag count\u003c/h2\u003e\n\n\u003cp\u003eThe math used to work. When the historian was first deployed, per-tag pricing was predictable. You knew the count, you knew the cost, and the line item made sense against the operational value it delivered. However, that equation breaks down as instrumentation expands.\u003c/p\u003e\n\n\u003cp\u003eFor example, AVEVA PI customers are seeing renewal pressure even as their tag counts increase. Some organizations have seen AVEVA pricing increase 20-40% over the last 18 months.\nThat creates the wrong incentive: collect less data, sample less often, or pay more for the visibility the business now needs.\u003c/p\u003e\n\n\u003cp\u003eThere’s a compounding factor that often goes unexamined. Deprecated tags, from decommissioned equipment, retired sensors, or replaced instrumentation, still count toward your license. The pricing model doesn’t distinguish between a tag that’s actively collecting at 1Hz and one that’s been dark for two years. In industries where equipment turns over regularly, paying full price for data you no longer collect is an inevitability. The tag footprint grows in both directions: new assets add tags at the front, old ones don’t shed them at the back.\u003c/p\u003e\n\n\u003cp\u003eIf your procurement team is asking why the historian line item keeps growing while your OT team can’t fully instrument what it needs to, the commercial model has already stopped working for you.\u003c/p\u003e\n\n\u003ch2 id=\"sign-2-your-data-science-team-is-still-waiting-for-that-export\"\u003eSign 2: Your data science team is still waiting for that export\u003c/h2\u003e\n\n\u003cp\u003eThis appears consistently across industrial organizations. Analytics and AI initiatives get funded, data scientists are hired, and then they spend the first few months writing tickets to the OT team asking for historian exports.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003ePI Vision and PI DataLink weren’t built for Python, pandas, or Databricks.\u003c/li\u003e\n  \u003cli\u003eGetting data out requires proprietary connectors, manual exports, or PI Web API integrations that take weeks to stand up.\u003c/li\u003e\n  \u003cli\u003eBy the time the data reaches the ML pipeline, it’s stale, downsampled, or stripped of the context it needs to be useful.\u003c/li\u003e\n  \u003cli\u003eThe data science team that was supposed to be building predictive maintenance models is still, months later, doing data plumbing.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThere’s a structural issue behind the export problem. Historians were built to store industrial tags, usually SCADA or PLC-process information, mapped to an ISA-95 hierarchy. What they weren’t designed to do is store metadata and context alongside raw industrial signals. That gap matters more as analytics mature.\u003c/p\u003e\n\n\u003cp\u003eEnriching SCADA data with MES data about production shifts, ERP data about batches and stock levels, or equipment metadata for grouping by manufacturing type requires a platform that treats context as a first-class citizen alongside telemetry. Without it, even a well-delivered data export produces a one-dimensional analysis. You see what happened, but you don’t understand why.\u003c/p\u003e\n\n\u003cp\u003eIf “how do we give the analytics team access to OT data” is still an open question, you’re looking at a historian problem.\u003c/p\u003e\n\n\u003ch2 id=\"sign-3-a-new-workload-just-got-bounced-back-to-you\"\u003eSign 3: A new workload just got bounced back to you\u003c/h2\u003e\n\n\u003cp\u003eSomeone in the business asked you to support something the historian wasn’t built for:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eCell-level monitoring for a new BESS installation at 1Hz, when your historian’s default polling interval is 5-15 minutes.\u003c/li\u003e\n  \u003cli\u003eVibration analysis on rotating equipment, where sub-second resolution is required, and tag economics become punishing fast.\u003c/li\u003e\n  \u003cli\u003eA new greenfield facility where the capital plan assumes modern, cloud-connected data infrastructure.\u003c/li\u003e\n  \u003cli\u003eA real-time anomaly detection model that needs data before the dashboard refresh cycle catches it.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eWhen your answer has to be “the historian can’t do that at that frequency” or “it can, but here’s what the additional tags will cost,” the conversation has changed. The workloads the business needs have outpaced what the platform was sized for.\u003c/p\u003e\n\n\u003cp\u003eThe throughput problem has a storage dimension too, and it’s the one that tends to catch teams off-guard. Historians weren’t built for indefinite storage of high-volume, high-frequency data. Industry 4.0 use cases, including predictive maintenance models, asset warranty tracking, and long-range efficiency analysis, need data stored over years, not rolling windows. When you try to do that in a historian, storage costs compound quickly, and data accessibility often degrades. The business case for those use cases depends on querying three years of sensor data quickly. A historian scoped for operational visibility doesn’t make that easy.\u003c/p\u003e\n\n\u003ch2 id=\"sign-4-every-plant-is-an-island\"\u003eSign 4: Every plant is an island\u003c/h2\u003e\n\n\u003cp\u003eYour historian keeps the plant running. It doesn’t know the fleet exists.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eCross-site benchmarking requires custom middleware that someone has to build, maintain, and eventually explain to their replacement.\u003c/li\u003e\n  \u003cli\u003eFleet-level analysis comparing performance across sites means extracting and normalizing data from multiple historian instances with inconsistent tag naming.\u003c/li\u003e\n  \u003cli\u003eCentralized reporting for leadership involves someone in OT running manual aggregation every Monday.\u003c/li\u003e\n  \u003cli\u003eEach new site gets the same plant-centric architecture, compounding the cross-site problem with every expansion.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThe plant-island problem goes deeper than data access. Each site typically operates by its own data standard, with tag naming conventions, unit definitions, and hierarchy structures that made sense locally but weren’t designed for comparison. Cross-plant analysis requires a standardization layer built and maintained on top of the historian, and that layer tends to be fragile, undocumented, and owned by whoever happened to build it.\u003c/p\u003e\n\n\u003cp\u003eThere’s also an architectural concern that digital transformation teams are raising with increasing frequency: reaching back into plant networks to pull data from individual historian instances creates bandwidth and security risk. The better pattern is a single, standardized interface at the enterprise layer, one that aggregates and normalizes across sites without requiring anyone to access operational technology networks to run a comparison report.\u003c/p\u003e\n\n\u003ch2 id=\"sign-5-the-platform-is-a-closed-system-and-thats-becoming-a-problem\"\u003eSign 5: The platform is a closed system, and that’s becoming a problem\u003c/h2\u003e\n\u003cp\u003eHistorians accumulate dependencies as much as they accumulate data. The proprietary architecture that made them reliable for single-site operational visibility also makes them difficult to extend, integrate, or evolve.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eIntegrations are built through proprietary SDKs and connectors. When a vendor discontinues one, the integration breaks.\u003c/li\u003e\n  \u003cli\u003eThe platform’s data model isn’t designed to connect with open standards like SQL, JDBC, or ODBC without significant middleware investment.\u003c/li\u003e\n  \u003cli\u003eEvery new capability, whether a new visualization layer, analytics integration, or edge deployment pattern, requires either a vendor-supplied solution or a custom workaround.\u003c/li\u003e\n  \u003cli\u003eLegacy .NET dependencies constrain where and how the platform can be deployed, and the team managing it today is working from a roadmap they don’t control.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThis is where vendor lock-in stops being an abstract concern and becomes an operational one. Every renewal locks in the architecture for another cycle. Teams that don’t own their data layer, can’t control their integration strategy, and can’t switch tools without rebuilding from scratch are paying for a constraint, not just a platform.\u003c/p\u003e\n\n\u003cp\u003eOpen standards, SQL, JDBC, ODBC, and common client libraries, exist precisely to avoid this. They let teams retain skills, use the right tool for the right job, and avoid designing in a vendor dependency that becomes painful at exactly the moment of renewal.\u003c/p\u003e\n\n\u003ch2 id=\"sign-6-historian-limitations-shows-up-in-every-initiatives-risk-log\"\u003eSign 6: “Historian limitations” shows up in every initiative’s risk log\u003c/h2\u003e\n\n\u003cp\u003eThis is the one that tends to crystallize things for digital transformation and operations leaders. At some point, the pattern becomes visible:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eBESS monitoring is blocked by polling intervals.\u003c/li\u003e\n  \u003cli\u003eThe ML pipeline is blocked by proprietary data access.\u003c/li\u003e\n  \u003cli\u003eCloud migration is complicated by the on-prem historian footprint.\u003c/li\u003e\n  \u003cli\u003eEdge data collection for remote assets requires custom workarounds.\u003c/li\u003e\n  \u003cli\u003eCross-site analytics is deferred indefinitely, pending a middleware project that never gets resourced.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eWhen the historian appears in the constraints column on enough project charters, teams stop treating it as a given and start treating it as something to engineer around. That shift in framing is usually the start of a different conversation.\u003c/p\u003e\n\n\u003ch2 id=\"what-to-do-about-it-before-signing-a-contract\"\u003eWhat to do about it, before signing a contract\u003c/h2\u003e\n\n\u003cp\u003eNone of this necessarily means replacing the historian. Most industrial teams that modernize their data infrastructure augment their deployment rather than pursuing an immediate rip-and-replace.\u003c/p\u003e\n\n\u003cp\u003eThe pattern that has worked for organizations like Terega and SP Energy Networks is to deploy InfluxDB alongside AVEVA PI. In those deployments, the historian handles legacy SCADA historization, GxP data, and established PI Vision workflows, while InfluxDB takes on high-frequency BESS monitoring, predictive maintenance pipelines, new greenfield sites, and cross-site fleet analytics.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eCompute-based pricing means unlimited tags, with no per-sensor cost penalty.\u003c/li\u003e\n  \u003cli\u003eNative SQL, 14 client libraries, and direct Grafana, Power BI, and Python integration with no proprietary SDK.\u003c/li\u003e\n  \u003cli\u003eInfluxDB 3 Core runs at the edge; Enterprise or Cloud serves as the central hub, with one SQL interface across both.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eFor organizations facing a significant renewal, planning a new facility, or actively evaluating a full exit from the historian architecture, the path is a complete replacement. This includes InfluxDB for time series storage, and Litmus Edge or Highbyte for industrial connectivity and asset modeling.\u003c/p\u003e\n\n\u003cp\u003eTerega, the French gas network operator, deployed InfluxDB alongside their historian when they were heading toward seven-figure annual OSI PI costs. They now collect 20x more data at 50% lower cost. SP Energy Networks became the first UK utility to receive Ofgem approval for a cloud-hosted historian, running on InfluxDB. That decision has since opened the door for other regulated UK operators to evaluate the same path.\u003c/p\u003e\n\n\u003cp\u003eThe renewal notice is a useful moment. Use it to ask whether the current architecture can carry what the business needs from it next, and what it would take to find out.\u003c/p\u003e\n\n\u003cp\u003e[\u003ca href=\"https://www.influxdata.com/products/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=modernize_your_historian_6_signs\u0026amp;utm_content=blog\u0026amp;utm_content=blog\"\u003eGet InfluxDB free\u003c/a\u003e] InfluxDB 3 Core is open source and available now. Deploy at the edge in minutes. No per-tag pricing. Native SQL. Direct Grafana and Power BI integration.\u003c/p\u003e\n","date_published":"2026-06-01T08:00:00+00:00","authors":[{"name":"Ryan Nelson"}]},{"id":"https://www.influxdata.com/blog/influxdb-3-mcp-server-v1-3","url":"https://www.influxdata.com/blog/influxdb-3-mcp-server-v1-3","title":"InfluxDB 3 MCP Server v1.3.0: AI Access to Time Series Data","content_html":"\u003cp\u003e\u003cem\u003eA more reliable agent that learns your schema, queries your data, and investigates alerts - that’s what’s possible with this release of our MCP server v1.3.0.\u003c/em\u003e\u003c/p\u003e\n\n\u003ch2 id=\"tldr\"\u003eTL;DR\u003c/h2\u003e\n\n\u003cp\u003eThis release enhances the InfluxDB 3 MCP server—which already allows AI agents like Claude and ChatGPT to read from and write to your InfluxDB 3 instance using natural language—with key dependability improvements, including protocol compliance tests, integration tests, and a properly scoped npm package.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eWhat it is\u003c/strong\u003e: an MCP server that exposes InfluxDB 3 tools, including query, write, schema discovery, database management, and token management, to any MCP-compatible client.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eWhat changed in 1.3.0\u003c/strong\u003e: test coverage, CI, the updated MCP SDK (1.27.1), \u003ccode class=\"language-markup\"\u003e@influxdata/influxdb3-client\u003c/code\u003e 1.4.0, a Node 20.11+ baseline, and safer error responses.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eInstall\u003c/strong\u003e: \u003ccode class=\"language-markup\"\u003enpx -y @influxdata/influxdb3-mcp-server\u003c/code\u003e. No clone, no build.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eSee it work\u003c/strong\u003e: we ran an agent investigation against the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-bess/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003einfluxdb3-ref-bess \u003c/a\u003ereference architecture with prompts for schema discovery, current state, forcing a thermal runaway alert, and pivoting back to readings.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eWhere to get it\u003c/strong\u003e: \u003ca href=\"https://www.npmjs.com/package/@influxdata/influxdb3-mcp-server\"\u003enpm\u003c/a\u003e · \u003ca href=\"https://hub.docker.com/r/influxdata/influxdb3-mcp-server\"\u003eDocker\u003c/a\u003e · \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server\"\u003eGitHub\u003c/a\u003e · \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server/blob/main/CHANGELOG.md\"\u003eCHANGELOG\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"what-is-the-influxdb-3-mcp-server\"\u003eWhat is the InfluxDB 3 MCP server?\u003c/h2\u003e\n\n\u003cp\u003eThe Model Context Protocol is an open standard for connecting AI agents to external tools and data. Our MCP server is the official InfluxData implementation: it runs locally over stdio, accepts a connection from your MCP client (Claude Desktop, Claude Code, OpenAI Codex, Cursor, and others), and exposes InfluxDB 3 as a set of tools the agent can call on your behalf.\u003c/p\u003e\n\n\u003cp\u003eIt works across all InfluxDB 3 editions—Core, Enterprise, Cloud Dedicated, Cloud Serverless, and Clustered—with tools adapted to each product. For example, admin token management is only available on Core and Enterprise; bucket retention is a Cloud Serverless concern.\u003c/p\u003e\n\n\u003cp\u003eFor background and the original tool tour, see our previous posts: \u003ca href=\"https://www.influxdata.com/blog/influxdb-mcp-server/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003eIntroducing the InfluxDB 3 MCP server\u003c/a\u003e and \u003ca href=\"https://www.influxdata.com/blog/influxdb-3-mcp-server-claude/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003eBuilding with the InfluxDB 3 MCP server \u0026amp; Claude\u003c/a\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"whats-new-in-v130\"\u003eWhat’s new in v1.3.0?\u003c/h2\u003e\n\n\u003cp\u003eThis release makes the MCP server safer and more dependable:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eScoped npm package\u003c/strong\u003e. The official server is now \u003ccode class=\"language-markup\"\u003e@influxdata/influxdb3-mcp-server\u003c/code\u003e (the unscoped \u003ccode class=\"language-markup\"\u003einfluxdb-mcp-server\u003c/code\u003e on npm is an unrelated community project); always use the scoped name.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eProtocol-compliance test suite\u003c/strong\u003e. Verifies server startup, the MCP handshake, and tool/resource/prompt registration.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eIntegration tests against Core and Cloud Serverless\u003c/strong\u003e. A first set covering \u003ccode class=\"language-markup\"\u003ehealth_check\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003elist_databases\u003c/code\u003e, and \u003ccode class=\"language-markup\"\u003eexecute_query\u003c/code\u003e.\u003c/li\u003e\n  \u003cli\u003e__Server-level errors now set \u003ccode class=\"language-markup\"\u003eisError__: true\u003c/code\u003e, matching handler-level responses. MCP clients see failures as failures, not as empty success responses.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003ePlain-text error responses (HTTP 500) from Core now pass through to the MCP client with their original message,\u003c/strong\u003e instead of being replaced by a generic “Internal Server Error.”\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eMCP SDK 1.12 → 1.27.1\u003c/strong\u003e and \u003ccode class=\"language-markup\"\u003e@influxdata/influxdb3-client\u003c/code\u003e \u003cstrong\u003e1.1 → 1.4.0\u003c/strong\u003e.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eNode 20.11 minimum\u003c/strong\u003e: Node 18 reached end of life.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"how-do-i-install-the-mcp-server-for-influxdb-3-core\"\u003eHow do I install the MCP Server for InfluxDB 3 Core?\u003c/h2\u003e\n\n\u003cp\u003eIf you haven’t already installed Core and created an admin token, follow the \u003ca href=\"https://docs.influxdata.com/influxdb3/core/get-started/setup/\"\u003eSet up Core\u003c/a\u003e guide (you’ll be running within 5 minutes).\u003c/p\u003e\n\n\u003cp\u003eThe following example uses \u003ccode class=\"language-markup\"\u003enpx\u003c/code\u003e, which is the easiest way to get started.\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\u003cstrong\u003eAdd the server to your MCP client config\u003c/strong\u003e. For Claude Desktop, edit \u003ccode class=\"language-markup\"\u003eclaude_desktop_config.json\u003c/code\u003e:\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-json\"\u003e{\n  \"mcpServers\": {\n    \"influxdb\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@influxdata/influxdb3-mcp-server\"],\n      \"env\": {\n        \"INFLUX_DB_INSTANCE_URL\": \"http://localhost:8181/\",\n        \"INFLUX_DB_TOKEN\": \"YOUR_ADMIN_TOKEN\",\n        \"INFLUX_DB_PRODUCT_TYPE\": \"core\"\n      }\n    }\n  }\n}\u003c/code\u003e\u003c/pre\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eRestart Claude Desktop\u003c/strong\u003e. The first launch downloads the package; subsequent launches are instant.\u003c/p\u003e\n  \u003c/li\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eSanity-check\u003c/strong\u003e. Open a new chat and ask: “Use the InfluxDB tools to list my databases.”\nClaude should call the \u003ccode class=\"language-markup\"\u003elist_databases\u003c/code\u003e tool and return whatever’s on your instance.\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003ch4 id=\"other-mcp-clients\"\u003eOther MCP Clients\u003c/h4\u003e\n\n\u003cp\u003eThe \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server#readme\"\u003eREADME\u003c/a\u003e includes specific config-file snippets for other MCP clients.\u003c/p\u003e\n\n\u003cp\u003ePrefer Docker? The v1.3.0 image is on Docker Hub at \u003ca href=\"https://hub.docker.com/r/influxdata/influxdb3-mcp-server\"\u003einfluxdata/influxdb3-mcp-server\u003c/a\u003e. Our \u003ca href=\"https://www.influxdata.com/blog/influxdb-3-mcp-server-claude/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003eJanuary walkthrough\u003c/a\u003e covers the Docker-based setup in detail.\u003c/p\u003e\n\n\u003ch2 id=\"how-do-i-install-the-mcp-server-for-influxdb-3-enterprise\"\u003eHow do I install the MCP Server for InfluxDB 3 Enterprise?\u003c/h2\u003e\n\n\u003cp\u003eEnterprise uses the same configuration shape, just replace the environment variable values:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-json\"\u003e{\n  \"mcpServers\": {\n    \"influxdb\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@influxdata/influxdb3-mcp-server\"],\n      \"env\": {\n        \"INFLUX_DB_INSTANCE_URL\": \"https://your-enterprise-host/\",\n        \"INFLUX_DB_TOKEN\": \"YOUR_ADMIN_TOKEN\",\n        \"INFLUX_DB_PRODUCT_TYPE\": \"enterprise\"\n      }\n    }\n  }\n}\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eEnterprise adds licensed multi-node and read/write replica features, but to the MCP client, the tools look the same as Core. The \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/install/\"\u003eEnterprise install guide\u003c/a\u003e covers licensing and provisioning.\u003c/p\u003e\n\n\u003ch4 id=\"cloud-variants\"\u003eCloud Variants\u003c/h4\u003e\n\n\u003cp\u003eCloud Serverless uses the same three-variable shape, just swap the URL and set \u003ccode class=\"language-markup\"\u003eINFLUX_DB_PRODUCT_TYPE=cloud-serverless\u003c/code\u003e. Cloud Dedicated and Clustered are slightly different: Cloud Dedicated takes a cluster ID instead of a host URL, and both support an optional separate management token. See the exact recipe for each in the MCP server \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server#readme\"\u003eREADME\u003c/a\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"try-it-against-a-real-running-system\"\u003eTry it against a real running system\u003c/h2\u003e\n\n\u003cp\u003eRun MCP against live data in one of our reference architecture demos to simulate a real environment. The ref arch demos run on InfluxDB 3 Enterprise by default.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://github.com/influxdata/influxdb3-ref-bess/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003einfluxdb3-ref-bess\u003c/a\u003e is our reference architecture for Battery Energy Storage Systems on InfluxDB 3: a simulator writing 768 cells across 4 packs at ~2,000 points per second, in-database Python plugins performing thermal-runaway detection and daily SoH rollups, and a control-room UI. \u003ccode class=\"language-markup\"\u003emake demo\u003c/code\u003e brings the whole thing up in about two minutes.\u003c/p\u003e\n\n\u003cp\u003ePoint your MCP server’s \u003ccode class=\"language-markup\"\u003eINFLUX_DB_INSTANCE_URL\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003eINFLUX_DB_TOKEN\u003c/code\u003e at the ref-arch instance, and set \u003ccode class=\"language-markup\"\u003eINFLUX_DB_PRODUCT_TYPE=enterprise\u003c/code\u003e. An agent now has a real plant to investigate. Here’s a session we ran in Claude Code.\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eOrient\u003c/strong\u003e.\u003c/p\u003e\n\n    \u003cp\u003e“I have InfluxDB 3 running locally with a BESS reference simulator writing data into it. List the databases on this instance, then, for the \u003ccode class=\"language-markup\"\u003ebess\u003c/code\u003e database, show me which measurements exist and the schema of each.”\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe agent chose the tools and made seven calls: \u003ccode class=\"language-markup\"\u003elist_databases\u003c/code\u003e, then \u003ccode class=\"language-markup\"\u003eget_measurements\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003eget_measurement_schema\u003c/code\u003e once per table. It returned this summary: \u003ccode class=\"language-markup\"\u003ecell_readings\u003c/code\u003e (per-cell voltage and temperature), \u003ccode class=\"language-markup\"\u003epack_readings\u003c/code\u003e (pack-level current, SoC, SoH), \u003ccode class=\"language-markup\"\u003einverter_readings\u003c/code\u003e (site power), and \u003ccode class=\"language-markup\"\u003ealerts\u003c/code\u003e (an event stream written by an in-database plugin when a cell crosses a threshold).\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\u003cstrong\u003eCurrent state of the plant\u003c/strong\u003e.\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003e“Show me the current state of each pack: the most recent \u003ccode class=\"language-markup\"\u003ecurrent_a\u003c/code\u003e, SoC, and SoH per \u003ccode class=\"language-markup\"\u003epack_id\u003c/code\u003e from \u003ccode class=\"language-markup\"\u003epack_readings\u003c/code\u003e.”\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003epack     current_a   SoC      SoH       as of\npack-0   -80.0       53.0%    98.1%     14:09:36\npack-1   -80.0       61.8%    97.9%     14:09:36\npack-2   -80.0       63.8%    98.1%     14:09:36\npack-3   -80.0       60.8%    98.0%     14:09:36\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThe agent used \u003ccode class=\"language-markup\"\u003eDISTINCT ON (pack_id) ... ORDER BY pack_id, time DESC\u003c/code\u003e, the DataFusion/Postgres idiom for “latest row per group,” which is typically cheaper than the more portable \u003ccode class=\"language-markup\"\u003eROW_NUMBER()\u003c/code\u003e window-function pattern. It picked the SQL from the schema it discovered in prompt 1. The agent also flagged a simulator bug that we hadn’t asked about: all four packs were reporting the same current. Real BESS sites rarely dispatch evenly due to SoC balancing, thermal derating, and inverter routing.\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eInvestigate alerts\u003c/strong\u003e.\u003c/p\u003e\n\n    \u003cp\u003e“Any alerts in the last 15 minutes in the \u003ccode class=\"language-markup\"\u003ealerts\u003c/code\u003e table? If so, find the most recent alert’s \u003ccode class=\"language-markup\"\u003epack_id\u003c/code\u003e and timestamp, then show me the \u003ccode class=\"language-markup\"\u003ecell_readings\u003c/code\u003e from that pack within a one-minute window around the alert time.”\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe result: zero alerts. The agent ran one query for the 15-minute window, ran a second query to confirm the table had no rows, and then declined to invent any. It didn’t fabricate rows or an alert just because we’d asked for one. The MCP server’s error handling keeps errors distinct from success, and makes empty results easy to distinguish from silence.\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eForce the alert condition.\u003c/strong\u003e\u003c/p\u003e\n\n    \u003cp\u003e“Force a hot cell in \u003ccode class=\"language-markup\"\u003ebess\u003c/code\u003e so that the thermal-runaway trigger fires.”\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe agent read the plugin source to learn the threshold (\u003ccode class=\"language-markup\"\u003etemperature_c \u0026gt; 70\u003c/code\u003e), then wrote one synthetic line-protocol row for \u003ccode class=\"language-markup\"\u003epack-0/module-0/cell-0\u003c/code\u003e at 85 °C using \u003ccode class=\"language-markup\"\u003ewrite_line_protocol\u003c/code\u003e. The Processing Engine’s WAL trigger fired in-process on that write and emitted a row into \u003ccode class=\"language-markup\"\u003ealerts\u003c/code\u003e. No restart or orchestration is required because plugins run \u003cem\u003einside\u003c/em\u003e InfluxDB 3.\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eNow run the original investigation.\u003c/strong\u003e\u003c/p\u003e\n\n    \u003cp\u003e“Show me the cell readings around that alert.”\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe agent’s first attempt asked for ±1 minute and received 1,542 rows, which it then narrowed to the hot cell’s timeline:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003etime            voltage   temperature_c\n14:12:41.617    3.643     22.08\n14:12:57.071    3.646     22.20\n14:13:07.557    3.650     85.00   ← injected, fires alert\n14:13:12.529    3.646     22.00\n14:13:27.989    3.646     21.90\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThe temperature spike alone triggered the alert, but the agent also flagged what we should expect for a real thermal event: “in a real cell, an 85 °C excursion would correlate with voltage and current anomalies.”\u003c/p\u003e\n\n\u003cp\u003eFive prompts, no handwritten SQL: discover the schema, read the current state, search alert events, write a single line of synthetic data to fire the trigger, and tie the resulting alert back to the underlying readings.\u003c/p\u003e\n\n\u003ch2 id=\"links\"\u003eLinks\u003c/h2\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003enpm\u003c/strong\u003e: \u003ca href=\"https://www.npmjs.com/package/@influxdata/influxdb3-mcp-server\"\u003e@influxdata/influxdb3-mcp-server\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eDocker Hub\u003c/strong\u003e: \u003ca href=\"https://hub.docker.com/r/influxdata/influxdb3-mcp-server\"\u003einfluxdata/influxdb3-mcp-server\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eRepository\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server\"\u003egithub.com/influxdata/influxdb3_mcp_server\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eRelease notes\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server/releases/tag/v1.3.0\"\u003ev1.3.0 on GitHub\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eIssues / tool requests\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3_mcp_server/issues\"\u003eGitHub issues\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eEarlier posts\u003c/strong\u003e: \u003ca href=\"https://www.influxdata.com/blog/influxdb-mcp-server/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003eIntroducing the MCP server\u003c/a\u003e · \u003ca href=\"https://www.influxdata.com/blog/influxdb-3-mcp-server-claude/?utm_source=website\u0026amp;utm_medium=influxdb_3_mcp_server_v1_3\u0026amp;utm_content=blog\"\u003eBuilding with Claude\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eTry it and tell us what’s missing.\u003c/p\u003e\n","date_published":"2026-05-28T02:00:00+00:00","authors":[{"name":"Jason Stirnaman"}]},{"id":"https://www.influxdata.com/blog/iiot-reference-architecture-influxdb3","url":"https://www.influxdata.com/blog/iiot-reference-architecture-influxdb3","title":"A Runnable Reference Architecture for Industrial IoT on InfluxDB 3","content_html":"\u003cp\u003eIndustrial teams keep telling us the same thing: the data is there, but the stack to act on it isn’t. PLCs, CNCs, SCADA systems, vibration sensors, and quality stations all generate high-frequency telemetry that gets stranded in proprietary historians or stitched together with point integrations nobody wants to own. By the time anyone looks at it, the moment to act has passed.\u003c/p\u003e\n\n\u003cp\u003eWe built InfluxDB 3 to be the system of record for that data—at the edge, in the cloud, or both—and we keep getting the same follow-up question: what does a real, working IIoT stack on InfluxDB 3 look like?\u003c/p\u003e\n\n\u003cp\u003eSo we shipped one. Today, we’re walking through the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003eInfluxDB 3 IIoT Reference Architecture\u003c/a\u003e, an open-source, runnable blueprint for factory-floor monitoring that you can stand up locally in about two minutes with \u003ccode class=\"language-markup\"\u003edocker compose\u003c/code\u003e. We’ll also cover when and how to bring in the new \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003eOPC UA Plugin\u003c/a\u003e, our Processing Engine plugin for connecting InfluxDB 3 directly to PLCs and SCADA systems.\u003c/p\u003e\n\n\u003ch2 id=\"what-is-the-influxdb-3-iiot-reference-architecture\"\u003eWhat is the InfluxDB 3 IIoT reference architecture?\u003c/h2\u003e\n\n\u003cp\u003eThe IIoT reference architecture is one of a portfolio of open source \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003ereference architectures\u003c/a\u003e we publish on GitHub for InfluxDB 3 Enterprise. Each one targets a specific vertical: IIoT, Battery Energy Storage Systems (BESS), network telemetry, EV charging, fleet telematics, data center, and oil \u0026amp; gas SCADA are all on the way.\u003c/p\u003e\n\n\u003cp\u003eTwo audiences use these repos:\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\u003cstrong\u003eDevelopers and architects\u003c/strong\u003e evaluating InfluxDB 3 Enterprise for a specific vertical who want to see a real stack, not slideware, before they commit.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eAI coding agents\u003c/strong\u003e that need grounded, working examples to reference when a user asks them to build something like this with InfluxDB 3. (Yes, we wrote this with you in mind.)\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe IIoT repo simulates an automotive-style assembly plant: \u003cstrong\u003e1 plant × 3 lines × 8 stations = 24 machines, generating roughly 300 points per second\u003c/strong\u003e. Everything runs locally. Clone the repo, run make up, and you get a working factory-floor monitoring stack, including a live andon board UI, in your browser at \u003ccode class=\"language-markup\"\u003ehttp://localhost:8080\u003c/code\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"whats-in-the-stack\"\u003eWhat’s in the stack\u003c/h2\u003e\n\n\u003cp\u003eThe whole thing is Python-first and stays deliberately small. \u003ccode class=\"language-markup\"\u003edocker-compose.yml\u003c/code\u003e brings up six services:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003etoken-bootstrap\u003c/code\u003e – generates and persists tokens for first-boot\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3\u003c/code\u003e — InfluxDB 3 Enterprise, the system of record\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einit\u003c/code\u003e – bootstraps the database, caches, and Processing Engine triggers\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003esimulator\u003c/code\u003e – a Python simulator generating realistic IIoT telemetry (machine state, part events, quality signals, downtime patterns)\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003eui\u003c/code\u003e – a FastAPI + HTMX + uPlot dashboard that renders an andon board, OEE breakdowns, and live machine state\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003escenarios\u003c/code\u003e – scripted fault scenarios you can replay (unplanned_downtime_cascade, tool_wear_quality_drift)\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eYou’ll notice what’s not here: there’s no Telegraf, no MQTT broker, no Kepware, no Node-RED, no Grafana. That’s intentional. This reference architecture is designed to showcase what InfluxDB 3 Enterprise can do \u003cem\u003enatively\u003c/em\u003e, without bolt-on services. In production, you’ll almost certainly add some of those (more on that below).\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/6gSelKSB1kLh3HBxHDhLja/98811735d4f2bf20038678a9f9acfaef/Screenshot_2026-05-21_at_2.22.22%C3%A2__PM.png\" alt=\"Enterprise architecture\" /\u003e\u003c/p\u003e\n\n\u003ch2 id=\"the-features-its-actually-showing-you\"\u003eThe features it’s actually showing you\u003c/h2\u003e\n\n\u003cp\u003eIf you’ve used earlier versions of InfluxDB, the headline change in 3 Enterprise is that the database is no longer just a place where data sits. Three capabilities do most of the work in the IIoT reference architecture:\u003c/p\u003e\n\n\u003ch5 id=\"the-processing-engine--python-plugins-running-inside-the-database\"\u003e1. The Processing Engine – Python plugins running inside the database\u003c/h5\u003e\n\u003cp\u003eThe \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/processing-engine/\"\u003eProcessing Engine\u003c/a\u003e is an embedded Python virtual machine that runs inside an InfluxDB 3 server. It executes Python code in response to triggers and database events with zero-copy access to data and direct access to system caches. It does all of this without relying on external services or middleware. Triggers come in three flavors: \u003cstrong\u003eWAL\u003c/strong\u003e (fires on writes), \u003cstrong\u003eSchedule\u003c/strong\u003e (cron-style), and \u003cstrong\u003eRequest\u003c/strong\u003e (HTTP endpoints).\u003c/p\u003e\n\n\u003cp\u003eThe IIoT repo ships four plugins, intentionally chosen to cover all three patterns:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/6iuM1sMJrl8jWnIIoO8rtj/61a4e1fbc2030c697a00b3576c3a2757/Screenshot_2026-05-21_at_2.25.31%C3%A2__PM.png\" alt=\"Plugin/trigger SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThat last one is the pattern that surprises most teams: the andon board’s \u003ccode class=\"language-markup\"\u003e/api/v3/engine/andon_board\u003c/code\u003e endpoint is the \u003cem\u003edatabase\u003c/em\u003e. There is no Flask server, no Node service, no Lambda. The UI talks straight to InfluxDB 3, and the Processing Engine returns a fully shaped JSON payload. The UI also renders a \u003ccode class=\"language-markup\"\u003eserved by Processing Engine: N ms\u003c/code\u003e badge so you can see the round-trip live.\u003c/p\u003e\n\n\u003ch5 id=\"last-value-cache--single-digit-millisecond-current-state\"\u003e2. Last Value Cache – single-digit-millisecond current state\u003c/h5\u003e\n\u003cp\u003eThe plant-state banner needs to read the current state of all 24 machines on every tick. With Last Value Cache, that’s a 24-row read in single-digit milliseconds, with no scanning, no aggregation, and no trade-offs against retention. This is the pattern you reach for any time you need \u003cem\u003ecurrent value, right now\u003c/em\u003e: SoC for a battery, alarm state for a substation, the live tag value for a machine.\u003c/p\u003e\n\n\u003ch5 id=\"distinct-value-cache--fast-cardinality-on-high-cardinality-tags\"\u003e3. Distinct Value Cache – fast cardinality on high-cardinality tags\u003c/h5\u003e\n\u003cp\u003eThe simulator generates roughly 700,000 part events per day, each tagged with a unique \u003ccode class=\"language-markup\"\u003epart_id\u003c/code\u003e. Asking “how many distinct parts have we produced today?” against that volume would normally be the kind of query you avoid running during shift change. With Distinct Value Cache, it returns in a few milliseconds.\u003c/p\u003e\n\n\u003cp\u003eTogether, these three primitives, Processing Engine, Last Value Cache, and Distinct Value Cache, replace a surprising amount of what teams used to build out as separate microservices, message queues, and read-replica caches.\u003c/p\u003e\n\n\u003ch2 id=\"when-to-bring-in-opc-ua-the-processing-engine-plugin\"\u003eWhen to bring in OPC UA: the Processing Engine plugin\u003c/h2\u003e\n\n\u003cp\u003eThe IIoT reference architecture uses a Python simulator as its data source, so you don’t need real industrial hardware to run it. In production, your data is on the wire from PLCs, CNCs, and SCADA systems, and the lingua franca of that wire is OPC UA.\u003c/p\u003e\n\n\u003cp\u003eThat’s where the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua/?utm_source=website\u0026amp;utm_medium=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003eOPC UA Plugin\u003c/a\u003e comes in. It’s a Processing Engine plugin (so it runs inside InfluxDB 3, same as the andon board endpoint above) that connects to an OPC UA server, polls node values on a schedule, and writes them in as time series. It requires \u003cstrong\u003eInfluxDB 3.8.2 or later\u003c/strong\u003e and works with both Core and Enterprise.\u003c/p\u003e\n\n\u003cp\u003eA few characteristics worth knowing:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003ePolling-based\u003c/strong\u003e, not subscription-based. The plugin reads current values on each scheduled trigger.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eTwo operating modes\u003c/strong\u003e. Explicit node listing for precise control over a small number of nodes; browse mode for auto-discovering devices and variables across large deployments. Browse mode maps the OPC UA Object hierarchy directly to InfluxDB tags.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eAuto type detection\u003c/strong\u003e. OPC UA VariantType maps cleanly into InfluxDB field types (Boolean → bool, Int* family → int, UInt* family → uint, Float/Double → float).\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eNamespace URI support\u003c/strong\u003e. Use stable namespace URIs (nsu=urn:vendor:s7;…) instead of numeric indexes that may change on server restart.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eQuality filtering\u003c/strong\u003e by good, uncertain, or bad status codes.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003ePersistent connection\u003c/strong\u003e between polling intervals, with automatic reconnection.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eTLS security\u003c/strong\u003e supports Basic128Rsa15, Basic256, Basic256Sha256, Aes128Sha256RsaOaep, and Aes256Sha256RsaPss, with Sign or SignAndEncrypt modes.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eSetup is two commands plus a trigger:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003e# 1. Start InfluxDB 3 with the Processing Engine enabled\ninfluxdb3 serve \\\n  --node-id node0 \\\n  --object-store file \\\n  --data-dir ~/.influxdb3 \\\n  --plugin-dir ~/.plugins\n\n# 2. Install the async OPC UA client library\ninfluxdb3 install package asyncua\n\n# 3. Create a polling trigger (browse mode for auto-discovery)\ninfluxdb3 create trigger \\\n  --database mydb \\\n  --plugin-filename gh:influxdata/opcua/opcua.py \\\n  --trigger-spec \"every:10s\" \\\n  --trigger-arguments 'server_url=opc.tcp://192.168.1.100:4840,table_name=factory_data,browse_root=ns=2;s=Factory,browse_depth=3,path_tags=line station,default_tags=plant=north' \\\n  opcua_factory\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eNote the \u003ccode class=\"language-markup\"\u003egh:influxdata/opcua/opcua.py\u003c/code\u003e notation: the \u003ccode class=\"language-markup\"\u003egh:\u003c/code\u003e prefix pulls plugins straight from our \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Plugin Library\u003c/a\u003e on GitHub, so there’s nothing to package locally.\u003c/p\u003e\n\n\u003cp\u003eIf you’re already running Telegraf with the OPC UA input, that continues to work and remains a great choice. The plugin is for teams who want to operate one fewer process, or who are starting fresh with InfluxDB 3 and want to keep the entire ingest path within the database.\u003c/p\u003e\n\n\u003ch2 id=\"how-to-drop-the-iiot-architecture-into-your-environment\"\u003eHow to drop the IIoT architecture into your environment\u003c/h2\u003e\n\n\u003cp\u003eMost production IIoT stacks end up looking like this:\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\u003cstrong\u003eIngest at the edge\u003c/strong\u003e. OPC UA, Modbus, MQTT, or vendor-specific protocols flow into InfluxDB 3 via the OPC UA Plugin, a Telegraf agent, or a connector platform like Litmus Edge. Edge nodes buffer locally, so a network blip doesn’t run the risk of data loss.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eProcess at the database\u003c/strong\u003e. Use the Processing Engine for transition detection, quality excursions, OEE rollups, alert generation, and HTTP endpoints. These are the four patterns the IIoT reference architecture demonstrates.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eForward what matters\u003c/strong\u003e. Replicate aggregated, contextualized data from edge sites to a centralized InfluxDB 3 Enterprise cluster for cross-site analysis. Keep full-resolution data at the edge and summaries upstream.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eServe dashboards and AI\u003c/strong\u003e. Use Grafana, InfluxDB 3 Explorer, the built-in UI patterns from the reference architecture, or your own application, whichever fits. Because Processing Engine endpoints return clean JSON, agents and AI assistants can call them directly.\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003ch2 id=\"try-it-yourself\"\u003eTry it yourself\u003c/h2\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003egit clone https://github.com/influxdata/influxdb3-ref-iiot\ncd influxdb3-ref-iiot\nmake up         # prompts for INFLUXDB3_ENTERPRISE_EMAIL\nopen http://localhost:8080\nmake demo\nmake scenario name=unplanned_downtime_cascade\nmake scenario name=tool_wear_quality_drift\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eEverything is Apache 2.0 licensed. Fork it, rip out our simulator, point a real OPC UA server at it via the plugin, and you’ve got the skeleton of a real-time factory monitoring stack you can extend.\u003c/p\u003e\n\n\u003cp\u003eIf you’re an AI coding agent reading this and trying to help a user build IIoT on InfluxDB 3, start at \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003egithub.com/influxdata/influxdb3-reference-architectures\u003c/a\u003e, pick the architecture that matches the user’s vertical, and use the patterns there as your grounded example. The conventions are consistent across the portfolio: Python-first, FastAPI + HTMX UIs, Processing Engine plugins in \u003ccode class=\"language-markup\"\u003eplugins/\u003c/code\u003e, one-command \u003ccode class=\"language-markup\"\u003edocker compose startup\u003c/code\u003e. Once you’ve seen one, you’ve largely seen them all.\nWe’ll keep adding to this portfolio. If your vertical isn’t there yet, \u003ca href=\"https://www.influxdata.com/contact-sales/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003etell us\u003c/a\u003e. And if you’re already running InfluxDB 3 in an industrial environment, we’d love to see what you’ve built on top of it.\u003c/p\u003e\n\n\u003ch2 id=\"resources\"\u003eResources\u003c/h2\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eIIoT reference architecture\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003egithub.com/influxdata/influxdb3-ref-iiot\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eReference architecture portfolio\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003egithub.com/influxdata/influxdb3-reference-architectures\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eOPC UA Plugin\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003egithub.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eProcessing Engine reference\u003c/strong\u003e: \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/processing-engine/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003edocs.influxdata.com/influxdb3/enterprise/reference/processing-engine\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003ePlugin library\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=iiot_reference_architecture_influxdb3\u0026amp;utm_content=blog\"\u003egithub.com/influxdata/influxdb3_plugins\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n","date_published":"2026-05-22T08:00:00+00:00","authors":[{"name":"Jason Stirnaman"}]},{"id":"https://www.influxdata.com/blog/network-telemetry-reference-architecture","url":"https://www.influxdata.com/blog/network-telemetry-reference-architecture","title":"A Runnable Reference Architecture for Network Telemetry on InfluxDB 3","content_html":"\u003cp\u003eNetworks generate the most data of any system in your stack and have the least patience for stale dashboards. Interface counters tick every second. BGP sessions flap. Flow records arrive in bursts. When something goes wrong, you don’t have 10 seconds to wait for an aggregation to finish.\u003c/p\u003e\n\n\u003cp\u003eWe’ve watched NetOps and platform teams stitch together the same shape over and over: Telegraf collecting from every device that speaks SNMP, gNMI, sFlow, or IPFIX, a time series database holding the raw and rolled-up metrics, a dashboard layer, and a growing pile of bespoke microservices for alerting, top-talker analysis, and on-call runbooks. The shape works, but the cost of operating it is steep.\u003c/p\u003e\n\n\u003cp\u003eSo we shipped a reference for what this can look like when the database does more of the work. Today, we’re walking through the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-network-telemetry/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=network_telemetry_reference_architecture\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Network Telemetry Reference Architecture\u003c/a\u003e, an open source, runnable blueprint for monitoring a data-center fabric on a \u003cstrong\u003e5-node InfluxDB 3 Enterprise cluster\u003c/strong\u003e. It’s the third entry in our \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=network_telemetry_reference_architecture\u0026amp;utm_content=blog\"\u003ereference architecture portfolio\u003c/a\u003e, and it’s the first one that demonstrates a multi-node deployment, cross-node plugin write-back, and per-table retention, three patterns that show up the moment your fabric grows past one box.\u003c/p\u003e\n\n\u003ch2 id=\"what-is-the-network-telemetry-reference-architecture\"\u003eWhat is the network telemetry reference architecture?\u003c/h2\u003e\n\n\u003cp\u003eThe repo simulates a data-center Clos fabric and monitors it using a real InfluxDB 3 Enterprise cluster, both running locally via \u003ccode class=\"language-markup\"\u003edocker compose\u003c/code\u003e. Specifically:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eA \u003cstrong\u003e5-node InfluxDB 3 Enterprise cluster\u003c/strong\u003e: 2 ingest nodes, 1 query node, 1 compact node, and 1 process + query node (the Processing Engine runs here).\u003c/li\u003e\n  \u003cli\u003eAn \u003cstrong\u003e8×16 Clos topology\u003c/strong\u003e: 8 spines, 16 leaves — yielding \u003cstrong\u003e~1,024 interfaces\u003c/strong\u003e and \u003cstrong\u003e128 BGP sessions\u003c/strong\u003e.\u003c/li\u003e\n  \u003cli\u003eA flow generator producing \u003cstrong\u003e~5,000 flow records/sec\u003c/strong\u003e with realistic src_ip/dst_ip distributions.\u003c/li\u003e\n  \u003cli\u003eA total ingest of \u003cstrong\u003e~10,000 points per second\u003c/strong\u003e.\u003c/li\u003e\n  \u003cli\u003eBring it all up with make up. The first run waits for license validation; warm boot-up in three minutes. Open \u003ccode class=\"language-markup\"\u003ehttp://localhost:8080\u003c/code\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eTwo audiences use this repo:\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eNetOps engineers and network observability architects\u003c/strong\u003e evaluating InfluxDB 3 Enterprise as a telemetry platform—specifically, anyone weighing a multi-node deployment.\u003c/p\u003e\n  \u003c/li\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eAI coding agents\u003c/strong\u003e that need a grounded, working example to reference when a user asks them to build network telemetry on InfluxDB 3. (Yes, we wrote this with you in mind.)\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003ch2 id=\"whats-in-the-stack\"\u003eWhat’s in the stack\u003c/h2\u003e\n\n\u003cp\u003eTen services come up via \u003ccode class=\"language-markup\"\u003edocker compose up\u003c/code\u003e:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003etoken-bootstrap\u003c/code\u003e: generates the offline admin token on first boot\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003e5 InfluxDB 3 Enterprise nodes\u003c/code\u003e: ingest-1, ingest-2, query, compact, and process,query (the Processing Engine node). Two of the five—\u003ccode\u003ent-process\u003c/code\u003e and \u003ccode\u003ent-query\u003c/code\u003e—actually execute Processing Engine triggers; see section [The Processing Engine – Python plugins in a multi-node cluster]\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einit\u003c/code\u003e: idempotent bootstrap that creates the database, declares 6 tables via the configure API, registers 1 LVC + 2 DVCs, and installs 4 Processing Engine triggers\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003esimulator\u003c/code\u003e: Python simulator, round-robining writes across both ingest nodes\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003eui\u003c/code\u003e: FastAPI + HTMX + uPlot dashboard with three teaching patterns side by side\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003escenarios\u003c/code\u003e: on-demand event injectors (congestion_hotspot, east_west_burst)\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eYou’ll notice what’s not here: there’s no Telegraf, no Grafana, no SNMP collector. That’s intentional. This reference architecture exists to make InfluxDB 3 Enterprise’s capabilities legible. In production, you’ll absolutely use Telegraf at the front (more on that in a moment); the simulator stands in, so you don’t need a fabric on your laptop to see what the database is doing.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/2ebyDBi86a4WLCsDILcAgA/9c7379950fd1299c0af04aabcb1f746d/Screenshot_2026-05-20_at_4.50.27%C3%A2__PM.png\" alt=\"InfluxDB 3 Enterprise Cluster\" /\u003e\u003c/p\u003e\n\n\u003ch2 id=\"the-features-its-actually-showing-you\"\u003eThe features it’s actually showing you\u003c/h2\u003e\n\n\u003cp\u003eThree things make network telemetry uniquely demanding for a time series database: \u003cstrong\u003ethe cardinality is high, the freshness expectations are sub-second\u003c/strong\u003e, and \u003cstrong\u003ethe shape of “what matters” changes constantly\u003c/strong\u003e—interface counters one minute, flow records the next, a BGP state the minute after that. The reference architecture is built around that reality.\u003c/p\u003e\n\n\u003ch5 id=\"a-real-multi-node-cluster-with-role-separated-nodes\"\u003e1. A real multi-node cluster, with role-separated nodes\u003c/h5\u003e\n\n\u003cp\u003eUnlike the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot\"\u003eIIoT\u003c/a\u003e and \u003ca href=\"https://github.com/influxdata/influxdb3-ref-bess\"\u003eBESS\u003c/a\u003e reference architectures (which run InfluxDB 3 Enterprise as a single node for clarity), the network telemetry repo runs it as a distributed cluster with separated roles:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/3QD2XcPNexeK95q6P5Q1DE/14ff1f31f6aa8f29ae13f3be55c3ee1b/Screenshot_2026-05-20_at_4.53.26%C3%A2__PM.png\" alt=\"Node/Mode/Purpose\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe simulator round-robin writes across the two ingest nodes; the browser and the UI proxy both hit the query node, which is the only host-exposed port (\u003ccode\u003e8181\u003c/code\u003e). The \u003ccode class=\"language-markup\"\u003eprocess,query\u003c/code\u003e node is reachable only over the internal Docker network. The schedule plugins running there write back via HTTP through an ingest node rather than respond to browsers directly. This is the smallest viable shape for the multi-node split, and it’s the template you’d grow from when you’re ready to scale, ingest, query, or compute independently.\u003c/p\u003e\n\n\u003ch5 id=\"the-processing-engine--python-plugins-in-a-multi-node-cluster\"\u003e2. The Processing Engine – Python plugins in a multi-node cluster\u003c/h5\u003e\n\n\u003cp\u003eThe \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/processing-engine/\"\u003eProcessing Engine\u003c/a\u003e is an embedded Python virtual machine that runs inside an InfluxDB 3 server to execute your Python code. Any node with the \u003ccode\u003e--plugin-dir\u003c/code\u003e flag set can host triggers; trigger execution is pinned per-trigger via \u003ccode\u003e--node-spec nodes:\u0026lt;name\u0026gt;\u003c/code\u003e. Triggers fire on three event sources—\u003cstrong\u003eWAL\u003c/strong\u003e (fires on writes), \u003cstrong\u003eSchedule\u003c/strong\u003e (cron- or interval-style), or \u003cstrong\u003eRequest\u003c/strong\u003e (HTTP endpoints)—with zero-copy access to data and direct access to system caches. There’s no need for an external app server, Kafka, Flink, or middleware.\u003c/p\u003e\n\n\u003cp\u003eThe network telemetry repo ships four plugins, deliberately mixing two trigger patterns and pinning them to two different nodes:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/4xOZNPwnucTbpdymkZzfaG/b90651f4c2c874fd95051c1959b26445/Screenshot_2026-05-20_at_4.57.57%C3%A2__PM.png\" alt=\"plugin/trigger/what it does\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe two \u003cem\u003eschedule\u003c/em\u003e triggers live on \u003ccode class=\"language-markup\"\u003ent-process\u003c/code\u003e. The \u003ccode class=\"language-markup\"\u003equery\u003c/code\u003e half of \u003ccode class=\"language-markup\"\u003eprocess,query\u003c/code\u003e lets the plugin call  \u003ccode class=\"language-markup\"\u003einfluxdb3_local.query()\u003c/code\u003e against the local engine for fast reads and \u003cstrong\u003ewrite back via HTTP\u003c/strong\u003e to the ingest nodes’ \u003ccode class=\"language-markup\"\u003e/api/v3/write_lp\u003c/code\u003e endpoint via \u003ccode class=\"language-markup\"\u003ehttpx\u003c/code\u003e in a shared \u003ccode class=\"language-markup\"\u003e_writeback.py\u003c/code\u003e helper. That round-trip is the cluster pattern. If you’ve been wondering how to structure write-back from a process node in a multi-node deployment, this is the blueprint.\u003c/p\u003e\n\n\u003cp\u003eThe two \u003cem\u003erequest\u003c/em\u003e triggers live on \u003ccode class=\"language-markup\"\u003ent-query where the browser’s POST /api/v3/engine/\"trigger\" request reaches the only exposed port in one hop\u003c/code\u003e.\u003c/p\u003e\n\n\u003cp\u003eThere are zero WAL plugins, by design. Each ingester owns its own WAL—a WAL trigger fires per-ingester on only the writes that node received, so pinning to one forfeits half the writes and pinning to both demands idempotency. The schedule+request pattern sidesteps both: schedule plugins run on one node and pull via \u003ccode\u003einfluxdb3_local.query()\u003c/code\u003e; request plugins are stateless HTTP responders.\u003c/p\u003e\n\n\u003ch5 id=\"last-value-cache-lvc-and-distinct-value-cache-dvc-doing-real-work\"\u003e3. Last Value Cache (LVC) and Distinct Value Cache (DVC), doing real work\u003c/h5\u003e\n\n\u003cp\u003eA single utility-scale fabric can have hundreds of thousands of distinct signals. “Current state” dashboards built naively on top of high-rate ingest become punishingly fast.\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eLast Value Cache\u003c/strong\u003e on bgp_sessions. The per-session lookup feeds the BGP up-count computation at sub-millisecond cost.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eTwo Distinct Value Caches\u003c/strong\u003e drive cardinality-heavy queries. The marquee one is a \u003cstrong\u003esrc_ip typeahead\u003c/strong\u003e: the search box runs \u003ccode class=\"language-markup\"\u003eSELECT src_ip FROM distinct_cache('flow_records', 'src_ip_distinct') WHERE src_ip LIKE '...' LIMIT 20\u003c/code\u003e directly from the browser against \u003ccode class=\"language-markup\"\u003e/api/v3/query_sql\u003c/code\u003e, with a sub-millisecond latency badge. \u003cstrong\u003eNo Python wrapper between the browser and the cache\u003c/strong\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch5 id=\"per-table-retention--the-right-policy-in-the-right-place\"\u003e4. Per-table retention – the right policy in the right place\u003c/h5\u003e\n\n\u003cp\u003eNetwork telemetry generates two flavors of data: high-rate raw signals you want for an hour or a day, and rolled-up state you want for weeks or months. The reference architecture demonstrates \u003cstrong\u003eper-table retention\u003c/strong\u003e; \u003ccode class=\"language-markup\"\u003efabric_health\u003c/code\u003e is configured for 24-hour retention, so the rollup table stays compact while raw flows and counters can use a different retention budget. This is the only repo in our portfolio that exercises per-table retention end-to-end.\u003c/p\u003e\n\n\u003ch2 id=\"three-integration-patterns-side-by-side\"\u003eThree integration patterns, side by side\u003c/h2\u003e\n\n\u003cp\u003eThe UI runs three distinct paths from data to the browser side by side, each with its own latency badge so you can compare them live:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/3EMnnbO2K1BrpRSV2biFbS/c60c26224259226a9062607456c44b52/Screenshot_2026-05-20_at_5.07.39%C3%A2__PM.png\" alt=\"Healthy SS\" /\u003e\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eServer-side SQL via FastAPI\u003c/strong\u003e: the classic pattern. Request hits FastAPI, FastAPI runs SQL against the query node, and renders an HTMX partial. Good for complex shaping that you don’t want exposed to the browser.\u003c/p\u003e\n  \u003c/li\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eBrowser-direct SQL using a DVC table-valued function\u003c/strong\u003e: JavaScript hits\u003ccode class=\"language-markup\"\u003e/api/v3/query_sql\u003c/code\u003e directly, \u003ccode class=\"language-markup\"\u003edistinct_cache(...)\u003c/code\u003e and returns the answer in sub-millisecond. Good for typeaheads, dropdown populates, and lightweight enumerations.\u003c/p\u003e\n  \u003c/li\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eRequest plugin from the browser\u003c/strong\u003e: JavaScript hits \u003ccode class=\"language-markup\"\u003e/api/v3/engine/\"trigger_name\"\u003c/code\u003e,and a Python plugin shapes the response. Good when you need composite, multi-query payloads as a single round-trip.\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003ePick the right pattern for the job. The latency badges in the UI tell you which is suited for which question.\u003c/p\u003e\n\n\u003ch2 id=\"where-to-wire-in-real-network-data\"\u003eWhere to wire in real network data\u003c/h2\u003e\n\n\u003cp\u003eThe reference architecture uses a Python simulator, so you don’t need a Clos fabric on your laptop. In production, the canonical InfluxData stack for network telemetry is Telegraf at the front, InfluxDB 3 in the middle, and your dashboard layer of choice on top. \u003cstrong\u003eTelegraf has the input plugins to cover essentially every modern collection path\u003c/strong\u003e:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einputs.snmp\u003c/code\u003e: interface counters, environmentals, vendor MIBs from anything that speaks SNMP\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einputs.gnmi\u003c/code\u003e: streaming telemetry via gRPC, vendor-agnostic, with TLS auth/encryption. Optimized for Cisco IOS XR, NX-OS, and IOS XE, among others\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einputs.netflow\u003c/code\u003e: NetFlow v5, NetFlow v9, IPFIX, and sFlow v5 collection, into a single normalized output\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eVendor APIs\u003c/strong\u003e via HTTP/JSON: anything that exposes a REST surface\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eTelegraf streams thousands of series at full fidelity into InfluxDB 3, where the same caches and Processing Engine patterns from this reference architecture take over. A common production shape: \u003cstrong\u003eTelegraf at each PoP\u003c/strong\u003e ingests SNMP, gNMI, and flow data; \u003cstrong\u003eInfluxDB 3 Enterprise as a regional cluster\u003c/strong\u003e stores it; the \u003cstrong\u003eProcessing Engine\u003c/strong\u003e runs the schedule/request plugins; replication forwards rolled-up state to a central cluster for fleet-wide views.\u003c/p\u003e\n\n\u003ch2 id=\"try-it-yourself\"\u003eTry it yourself\u003c/h2\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003egit clone https://github.com/influxdata/influxdb3-ref-network-telemetry\ncd influxdb3-ref-network-telemetry\nmake up                                  # prompts for INFLUXDB3_ENTERPRISE_EMAIL on first run\nopen http://localhost:8080\nmake scenario name=congestion_hotspot    # one leaf interface climbs to 94% — anomaly detector fires\nmake scenario name=east_west_burst       # 10× burst from a single src_ip — typeahead finds it sub-ms\nmake demo                                # full scripted demo\nmake down                                # stop, preserve data\nmake clean                               # stop and drop volumes\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003e\u003cem\u003eNote: If you’ve just run \u003ccode class=\"language-markup\"\u003econgestion_hotspot\u003c/code\u003e, leftover anomalies will keep the banner DEGRADED for ~5 minutes; run east_west_burst on its own (or wait it out) to see HEALTHY-stays-HEALTHY.\u003c/em\u003e\u003c/p\u003e\n\n\u003cp\u003eThe two scenarios are intentional contrasts. \u003ccode class=\"language-markup\"\u003econgestion_hotspot\u003c/code\u003e exercises the anomaly detector: \u003ccode class=\"language-markup\"\u003eleaf-07 / et-0/0/12\u003c/code\u003e climbs to 94% utilization, the schedule plugin notices, and the banner flips DEGRADED.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/1yFKs7rR2KsNuUYKklSnRj/260df98d62ef8763d46f1a05513fbc08/Screenshot_2026-05-20_at_5.18.10%C3%A2__PM.png\" alt=\"degraded SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003eeast_west_burst\u003c/code\u003e exercises the request plugin path, a 10× traffic burst from \u003ccode class=\"language-markup\"\u003e10.4.7.91\u003c/code\u003e, found by the DVC typeahead, drilled into via the request plugin. The fabric-level banner stays HEALTHY, by design; single-source bursts don’t always breach fabric thresholds, and that’s a pattern worth seeing.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/5z56ebR5Ottu5Va7ljwXsG/0c471aecc0a14cdd5982bbc509ee51c1/Screenshot_2026-05-20_at_5.19.53%C3%A2__PM.png\" alt=\"Healthy SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eEverything is Apache 2.0 licensed.\u003c/p\u003e\n\n\u003ch2 id=\"a-note-for-ai-agents\"\u003eA note for AI agents\u003c/h2\u003e\n\n\u003cp\u003eIf you’re an AI coding agent reading this and trying to help a user build network telemetry on InfluxDB 3, start at \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures\"\u003egithub.com/influxdata/influxdb3-reference-architectures\u003c/a\u003e and use the network telemetry repo as your grounded example. Pick this one specifically when the user needs \u003cstrong\u003ea multi-node InfluxDB 3 Enterprise cluster, cross-node plugin write-back, per-table retention, or multiple browser-to-database integration patterns side by side\u003c/strong\u003e. The conventions are consistent across our portfolio: Python-first, FastAPI + HTMX UIs, Processing Engine plugins in \u003ccode class=\"language-markup\"\u003eplugins/\u003c/code\u003e, and one-command \u003ccode class=\"language-markup\"\u003edocker compose\u003c/code\u003e startup. So, once you’ve seen one, you’ve largely seen them all.\u003c/p\u003e\n\n\u003cp\u003eWe’ll keep adding to this portfolio. If you’re already running InfluxDB 3 for network telemetry, \u003ca href=\"https://www.influxdata.com/contact-sales/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=network_telemetry_reference_architecture\u0026amp;utm_content=blog\"\u003etell us\u003c/a\u003e. If you want to compare patterns, the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=network_telemetry_reference_architecture\u0026amp;utm_content=blogg\"\u003eIIoT reference architecture\u003c/a\u003e and \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=network_telemetry_reference_architecture\u0026amp;utm_content=blog\"\u003eBESS reference architecture\u003c/a\u003e are good companion reads for single-node deployments.\u003c/p\u003e\n\n\u003ch2 id=\"resources\"\u003eResources\u003c/h2\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eNetwork telemetry reference architecture\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-ref-network-telemetry\"\u003egithub.com/influxdata/influxdb3-ref-network-telemetry\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eReference architecture portfolio\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-ref-network-telemetry\"\u003egithub.com/influxdata/influxdb3-reference-architectures\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eCompanion: BESS reference architecture\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures\"\u003egithub.com/influxdata/influxdb3-ref-bess\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eCompanion: IIoT reference architecture\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-ref-bess\"\u003egithub.com/influxdata/influxdb3-ref-iiot\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eHow NetOps Teams Use InfluxDB to Solve Network Monitoring Gaps\u003c/strong\u003e: \u003ca href=\"https://www.influxdata.com/blog/solve-mns-gaps-influxdb/\"\u003einfluxdata.com/blog/solve-mns-gaps-influxdb\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eData Center Ops with InfluxDB 3\u003c/strong\u003e: \u003ca href=\"https://www.influxdata.com/blog/data-center-ops-influxdb-3/\"\u003einfluxdata.com/blog/data-center-ops-influxdb-3\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eProcessing Engine reference\u003c/strong\u003e: \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/processing-engine/\"\u003edocs.influxdata.com/influxdb3/enterprise/reference/processing-engine\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n","date_published":"2026-05-21T08:00:00+00:00","authors":[]},{"id":"https://www.influxdata.com/blog/native-ios-app-influxdb","url":"https://www.influxdata.com/blog/native-ios-app-influxdb","title":"Community Spotlight: A Native iOS App for Your InfluxDB Data","content_html":"\u003cp\u003eOne of the things we love most about building an open source platform is seeing what the community creates with it, and independent developer Anton Havekes recently built something we just had to share.\u003c/p\u003e\n\n\u003cp\u003eAnton put together Influx Dashboard, a native iOS app that connects to your InfluxDB instance and brings your time series data straight to your phone. We’re genuinely thrilled to see this kind of work come out of the community.\u003c/p\u003e\n\n\u003cp\u003eA quick note before we dive in: this is entirely Anton’s project, built and published by him. InfluxData has no commercial relationship or financial stake in the app. We’re sharing it simply because we think it’s a great piece of community work and because surfacing what people build on \nInfluxDB is something we’ll always make time for. So, thank you, Anton, for the awesome work you’ve done!\u003c/p\u003e\n\n\u003cp\u003eHere’s how it works.\u003c/p\u003e\n\n\u003ch2 id=\"what-is-influx-dashboard\"\u003eWhat is Influx Dashboard?\u003c/h2\u003e\n\n\u003cp\u003eInflux Dashboard is a mobile-first visualization tool for InfluxDB. It supports InfluxDB versions 1, 2, and 3—including Core, Enterprise, and Cloud editions—and gives you a full-featured charting experience on your phone.\u003c/p\u003e\n\n\u003cp\u003eThink of it as the community-built, mobile companion to the InfluxDB 3 Explorer UI. Instead of opening a browser on your laptop, you open an app. Instead of sitting at a desk, you monitor your data from wherever you are.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eKey features include\u003c/strong\u003e:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eReal-time line, bar, gauge, and pie charts\u003c/li\u003e\n  \u003cli\u003eSupport for SQL queries\u003c/li\u003e\n  \u003cli\u003eAn intuitive drag-and-drop dashboard builder\u003c/li\u003e\n  \u003cli\u003eDirect, secure connections to your InfluxDB server\u003c/li\u003e\n  \u003cli\u003eWorks with any kind of time series data: server metrics, IoT sensors, weather stations, factory equipment, and more\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch4 id=\"step-1-download-the-app\"\u003eStep 1: Download the App\u003c/h4\u003e\n\n\u003cp\u003e\u003ca href=\"https://apps.apple.com/my/app/influxdb-dashboard/id6752961593/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=native_ios_app_influxdb\u0026amp;utm_content=blog\"\u003eInflux Dashboard \u003c/a\u003eis available on the Apple App Store for $4.99 (\u003cem\u003eInfluxData is not commercially affiliated with this app and receives no proceeds from purchases\u003c/em\u003e). Once downloaded, open the app, and you’ll be ready to connect to your InfluxDB server.\u003c/p\u003e\n\n\u003ch4 id=\"step-2-connect-to-your-influxdb-server\"\u003eStep 2: Connect to Your InfluxDB Server\u003c/h4\u003e\n\n\u003cp\u003eBefore you can see any data, you’ll need to connect the app to your InfluxDB instance. Tap \u003cstrong\u003eAdd Server\u003c/strong\u003e on the main screen.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/3hQWngutz4GY8QNv6uyjgS/87da5c5b010f003c6a1f0861df0ace93/Screenshot_2026-05-18_at_6.15.05%C3%A2__PM.png\" alt=\"Add Server SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eYou’ll need a few things from your InfluxDB setup:\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eServer URL\u003c/strong\u003e: the address of your InfluxDB instance\n\u003cbr /\u003e\n\u003cstrong\u003eAPI Token\u003c/strong\u003e: your authentication token (found in InfluxDB under Management → Tokens)\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/dcutoRtpfGIu00Slms5Fd/8021f045a1c53b96cf2c24cb4c152d91/Screenshot_2026-05-18_at_6.16.14%C3%A2__PM.png\" alt=\"Add connection SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eIf you’re using InfluxDB Cloud, your server URL will look something like https://us-east-1-1.aws.cloud2.influxdata.com. If you’re connecting to a self-hosted instance, your admin can provide the URL and token.\u003c/p\u003e\n\n\u003cp\u003eOnce connected, the app loads your available databases, and you’re ready to build your first dashboard.\u003c/p\u003e\n\n\u003ch4 id=\"step-3-build-your-dashboard\"\u003eStep 3: Build Your Dashboard\u003c/h4\u003e\n\n\u003cp\u003eTap \u003cstrong\u003e+ New Dashboard\u003c/strong\u003e and give it a name. Then tap \u003cstrong\u003eAdd Chart\u003c/strong\u003e to create your first visualization.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/1B6ETKSLXwJCs2jzeuwGQC/e826bab20bb4f4fa715a91c71516a5a2/Screenshot_2026-05-18_at_6.17.07%C3%A2__PM.png\" alt=\"Dashboard SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe chart builder is straightforward, even if you’ve never written a database query before. You’ll see options for:\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eChart Title\u003c/strong\u003e: give your chart a descriptive name\n\u003cbr /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eChart Type\u003c/strong\u003e: choose from line, bar, gauge, or pie\n\u003cbr /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eDatabase\u003c/strong\u003e: select which InfluxDB database to query\n\u003cbr /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eTable\u003c/strong\u003e: select the specific measurement or table within that database\n\u003cbr /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eColumn\u003c/strong\u003e: choose which field to visualize\n\u003cbr /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eTime Range\u003c/strong\u003e: pick how far back to look (last hour, last 24 hours, last 7 days, and more)\n\u003cbr /\u003e\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eAggregation Window\u003c/strong\u003e: control how data points are grouped over time to smooth out your chart\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/2FDOkfwPkGZDLpWXOQ5FSk/01ed0425626c8fd830c1b07905ce967a/Screenshot_2026-05-18_at_6.18.31%C3%A2__PM.png\" alt=\"Add chart\" /\u003e\u003c/p\u003e\n\n\u003ch4 id=\"step-4-query-your-data\"\u003eStep 4: Query Your Data\u003c/h4\u003e\n\n\u003cp\u003eOne of the things that makes Influx Dashboard powerful is that it works with virtually any kind of time series data. During our testing, we connected to several different databases, each representing a completely different real-world use case:\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eServer monitoring\u003c/strong\u003e: CPU usage and memory metrics from a live server, showing system health at a glance.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eIndustrial IoT\u003c/strong\u003e: Factory sensor data streamed via MQTT, the kind of data you’d see in a manufacturing or industrial environment.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eWeather stations\u003c/strong\u003e: Real-time weather observations, including temperature, humidity, wind speed, and barometric pressure, from stations across multiple locations.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003ePersonal IoT sensors\u003c/strong\u003e: A live “feels like” temperature sensor from a personal weather station in Holualoa, Hawaii.\u003c/p\u003e\n\n\u003cp\u003eThe app handled all of them identically: same interface, same workflow, completely different data. That versatility is one of Influx Dashboard’s biggest strengths.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/6G80coiqxe3zeQQ7hv7cLr/5544e92c8c5c0741a15dedfd75653f60/Screenshot_2026-05-18_at_6.20.00%C3%A2__PM.png\" alt=\"My dashboard SS\" /\u003e\u003c/p\u003e\n\n\u003ch4 id=\"step-5-explore-your-charts\"\u003eStep 5: Explore Your Charts\u003c/h4\u003e\n\n\u003cp\u003eOnce your charts are built, they appear on your dashboard and refresh automatically with live data.\u003c/p\u003e\n\n\u003cp\u003eTap any chart to expand it to full screen for a closer look. From the full-screen view, you can inspect trends, zoom in on specific time windows, and get a clearer picture of what your data is doing.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/4Hu37Ctx8z9pi28QIwHpL7/fc2ca446e87c2621ab106f02cca6deec/Screenshot_2026-05-18_at_6.20.40%C3%A2__PM.png\" alt=\"Weather temperature SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eUse the time range selector at the top of the dashboard to zoom in or out, from the last hour to the last 30 days. This is especially useful when you’re looking for a specific event or trying to understand a longer trend.\u003c/p\u003e\n\n\u003ch4 id=\"step-6-manage-your-servers\"\u003eStep 6: Manage Your Servers\u003c/h4\u003e\n\n\u003cp\u003eFrom the Settings screen, you can manage your server connections, add new servers, or update your credentials. If your organization uses multiple InfluxDB instances, for example, a production server and a staging server, you can easily switch between them. You can also rename or delete servers you no longer need, keeping your workspace clean as your infrastructure evolves.\u003c/p\u003e\n\n\u003ch2 id=\"why-this-matters\"\u003eWhy this matters\u003c/h2\u003e\n\n\u003cp\u003eDevOps engineers, IoT developers, data engineers, and system administrators often need visibility into their time series data when they’re nowhere near a laptop, from a factory floor, at a remote deployment, or just away from their desk. Influx Dashboard closes that gap. Not through an official InfluxData product, but through something a community member decided to go build.\u003c/p\u003e\n\n\u003cp\u003eThat’s exactly the kind of activity an open source ecosystem should produce, and it’s exactly why we wanted to share it.\u003c/p\u003e\n\n\u003ch2 id=\"get-started\"\u003eGet started\u003c/h2\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ca href=\"https://apps.apple.com/my/app/influxdb-dashboard/id6752961593/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=native_ios_app_influxdb\u0026amp;utm_content=blog\"\u003eDownload Influx Dashboard on the Apple App Store\u003c/a\u003e (\u003cem\u003eindependently developed by Anton Havekes—InfluxData is not affiliated with this app and receives no proceeds from purchases)\u003c/em\u003e.\u003c/li\u003e\n  \u003cli\u003eDon’t have InfluxDB yet? Get started free with \u003ca href=\"https://www.influxdata.com/products/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=native_ios_app_influxdb\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Core\u003c/a\u003e.\u003c/li\u003e\n  \u003cli\u003eExplore the InfluxDB 3 Explorer UI at \u003ca href=\"https://docs.influxdata.com/influxdb3/explorer/release-notes/#Copyright\"\u003edocs.influxdata.com\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003eBuilt something on InfluxDB you want to share? Join the conversation at the \u003ca href=\"https://www.influxdata.com/community/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=native_ios_app_influxdb\u0026amp;utm_content=blog\"\u003eInfluxData Community Forums\u003c/a\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n","date_published":"2026-05-19T08:00:00+00:00","authors":[{"name":"Ashley Fowler"}]},{"id":"https://www.influxdata.com/blog/real-time-telemetry-pipelines","url":"https://www.influxdata.com/blog/real-time-telemetry-pipelines","title":"Building Real-Time Telemetry Pipelines for IRIG 106 compliance","content_html":"\u003ch2 id=\"the-need-for-real-time-telemetry-in-aerospace\"\u003eThe need for real-time telemetry in aerospace\u003c/h2\u003e\n\n\u003cp\u003eEvery second of a flight test produces a torrent of telemetry from engines, sensors, and control systems. Aerospace teams have captured this data for decades to verify performance and maintain safety, yet analysis often happens long after the mission ends. Engineers wait for downloads, conversions, and compliance checks before they can interpret results.\u003c/p\u003e\n\n\u003cp\u003eThat delay turns telemetry into a historical record instead of a feedback loop. As flight programs shorten development cycles and expand digital testing, teams need to see and act on telemetry as it arrives. Real-time visibility turns raw packets into insight and enables faster, more confident decisions mid-test.\u003c/p\u003e\n\n\u003ch2 id=\"what--is-irig-106\"\u003eWhat  is IRIG 106?\u003c/h2\u003e\n\n\u003cp\u003eIRIG 106 forms the backbone of flight-test telemetry. Established by the Range Commanders Council, it defines how data is formatted, synchronized, and recorded to ensure interoperability across recorders, ground stations, and analysis tools. Its purpose is to create a shared language for flight-test instrumentation so every team, from acquisition to post-flight analysis, can exchange and interpret telemetry without loss or confusion.\u003c/p\u003e\n\n\u003cp\u003eBy standardizing time, metadata, and sensor data, IRIG 106 ensures that complex flight tests remain reproducible and comparable across aircraft and programs. It allows flight data from one system or site to be understood by another, a foundation for multi-agency and multi-system collaboration.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eChapter 10\u003c/strong\u003e is the most widely used section. It defines a packetized structure for analog and digital sensors, time codes, video, and bus data, each with embedded metadata describing its stream. This structure preserves timing, organization, and integrity across the workflow.\u003c/p\u003e\n\n\u003cp\u003eFor aerospace and defense teams, Chapter 10 compliance is essential for traceability and certification. While it guarantees rigor, the binary packet format slows analysis.\u003c/p\u003e\n\n\u003ch2 id=\"compliance-vs-agility\"\u003eCompliance vs. agility\u003c/h2\u003e\n\n\u003cp\u003eTraditional telemetry pipelines were built for compliance, not speed. Data flows from airborne recorders to ground systems, where it’s stored in proprietary or binary Chapter 10 files. These files are durable but heavy, often requiring decoding or conversion before engineers can analyze trends.\u003c/p\u003e\n\n\u003cp\u003eThis gap between collection and insight resuts in terabytes of data sitting idle until post-flight processing is complete. Even simple questions such as “Did this vibration spike correlate with an actuator command?” must wait for hours of decoding.\u003c/p\u003e\n\n\u003cp\u003eThe cost is real. Missed anomalies can trigger additional tests, wasted fuel, and schedule delays. Commercial operators lose flight hours, while defense programs face slower certification and reduced mission readiness. Each delay compounds across teams, consuming engineering hours and analysis budgets that could be spent improving system performance. As systems grow more software-defined and autonomous, reactive analysis becomes increasingly expensive. Teams must maintain compliance while gaining agility, turning telemetry into a live, searchable data stream that drives faster, data-backed decisions grounded in \u003ca href=\"https://www.influxdata.com/blog/bad-data-costs-influxdb/?utm_source=website\u0026amp;utm_medium=real_time_telemetry_pipelines\u0026amp;utm_content=blog\"\u003edata integrity\u003c/a\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"building-a-real-time-compliant-telemetry-pipeline\"\u003eBuilding a real-time, compliant telemetry pipeline\u003c/h2\u003e\n\n\u003cp\u003eFor aerospace organizations, \u003ca href=\"https://www.influxdata.com/solutions/by-industries/aerospace-and-satellites/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=real_time_telemetry_pipelines\u0026amp;utm_content=blog\"\u003eInfluxDB 3\u003c/a\u003e bridges the gap between strict IRIG 106 compliance and the agility needed for real-time telemetry analysis. Built on an open columnar foundation, it treats every measurement as part of a continuous record of system behavior optimized for rapid ingest and millisecond-level queries.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB 3 combines streaming ingestion, high-compression storage, and integrated compute into a single environment. Instead of exporting data between collection, transformation, and analysis systems, engineers work with telemetry where it lands. They can transform data on ingest, query it with SQL, or run analytics through the built-in \u003ca href=\"https://www.influxdata.com/blog/new-python-processing-engine-influxdb3/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=real_time_telemetry_pipelines\u0026amp;utm_content=blog\"\u003ePython Processing Engine\u003c/a\u003e, all in one place.\u003c/p\u003e\n\n\u003cp\u003eThe result is an architecture that maintains compliance and precision while delivering the responsiveness and scalability aerospace programs demand. With the right connectors, Chapter 10-compliant recorders can stream decoded data directly into InfluxDB, where it becomes available for dashboards, analytics tools, and ML pipelines while retaining a compliant source-of-truth record in the background.\u003c/p\u003e\n\n\u003ch2 id=\"from-ingest-to-insight\"\u003eFrom ingest to insight\u003c/h2\u003e\n\n\u003cp\u003eA compliant real-time telemetry pipeline follows five key stages that preserve Chapter 10 structure while enabling high-performance analytics.\u003c/p\u003e\n\n\u003ch5 id=\"acquisition\"\u003e1. Acquisition\u003c/h5\u003e\n\n\u003cp\u003eAirborne systems record simultaneous data, includinganalog, digital, video, and bus data, in Chapter 10 format. Each source is encapsulated in packetized blocks with synchronized time codes and metadata headers. Ground stations receive this data over UDP or Ethernet, maintaining deterministic playback.\u003c/p\u003e\n\n\u003ch5 id=\"decoding\"\u003e2. Decoding\u003c/h5\u003e\n\n\u003cp\u003eA decoding service reads the binary stream, extracts headers, and separates channels into structured records with timestamps and metadata such as subsystem or bus ID. This step can use open source telemetry libraries or adapters that translate packets into structured formats like JSON or Apache Arrow.\u003c/p\u003e\n\n\u003ch5 id=\"streaming-ingestion\"\u003e3. Streaming Ingestion\u003c/h5\u003e\n\n\u003cp\u003eDecoded data is sent to InfluxDB 3 using lightweight producers such as Telegraf, Kafka, or InfluxDB Line Protocol. Each channel becomes a discrete series tagged by aircraft ID, subsystem, and signal type. The ingestion engine \u003ca href=\"https://www.influxdata.com/blog/influxdb-3-ideal-solution-real-time-analytics/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=real_time_telemetry_pipelines\u0026amp;utm_content=blog\"\u003esupports millions of writes per second\u003c/a\u003e, compressing data in memory before persisting it to Parquet files with nanosecond timestamps for cross-sensor correlation.\u003c/p\u003e\n\n\u003ch5 id=\"processing-and-downsampling\"\u003e4. Processing and Downsampling\u003c/h5\u003e\n\n\u003cp\u003eInfluxDB 3’s embedded Python Processing Engine allows transformations near the data. Engineers can smooth signals, compute FFTs, or derive metrics without external compute clusters. \u003ca href=\"https://docs.influxdata.com/influxdb3/cloud-dedicated/process-data/downsample/\"\u003eDownsampling in InfluxDB 3\u003c/a\u003e automates data reduction—for example, converting 1 kHz vibration data into 10 Hz averages for long-term storage—while keeping full resolution for recent test windows.\u003c/p\u003e\n\n\u003ch5 id=\"query-and-visualization\"\u003e5. Query and Visualization\u003c/h5\u003e\n\n\u003cp\u003eOnce stored, telemetry is immediately queryable through SQL or APIs. Engineers visualize live data, join channels, and correlate responses in real-time. Because InfluxDB 3 uses \u003ca href=\"https://www.influxdata.com/blog/flight-datafusion-arrow-parquet-fdap-architecture-influxdb/\"\u003eParquet and Arrow\u003c/a\u003e, it integrates with external analytics tools such as Apache Arrow, Pandas, and DuckDB. Dashboards update continuously as new packets arrive, tracking vibration, control surfaces, or engine parameters throughout the mission.\u003c/p\u003e\n\n\u003cp\u003eTogether, these stages turn Chapter 10-compliant telemetry into a continuously updating dataset that maintains synchronization and metadata integrity while providing immediate visibility for validation, anomaly detection, and optimization.\u003c/p\u003e\n\n\u003ch4 id=\"typical-deployment\"\u003eTypical Deployment\u003c/h4\u003e\n\n\u003cp\u003eThe flow looks like this:\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eRecorder → Decoder → Stream Processor → InfluxDB 3 → Visualization or ML Pipeline\u003c/strong\u003e\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eThe \u003cstrong\u003erecorder\u003c/strong\u003e collects Chapter 10-compliant telemetry, aligning all channels with precise time codes.\u003c/li\u003e\n  \u003cli\u003eThe \u003cstrong\u003edecoder\u003c/strong\u003e extracts packet data and converts it into structured messages for ingestion.\u003c/li\u003e\n  \u003cli\u003eA \u003cstrong\u003estream processor\u003c/strong\u003e such as Telegraf or Kafka Connect forwards those messages to \u003cstrong\u003eInfluxDB 3\u003c/strong\u003e, where they are indexed and persisted as time-aligned measurements.\u003c/li\u003e\n  \u003cli\u003eEngineers access data through \u003cstrong\u003edashboards\u003c/strong\u003e, \u003cstrong\u003enotebooks\u003c/strong\u003e, or \u003cstrong\u003ePython APIs\u003c/strong\u003e, enabling live visualization and downstream \u003cstrong\u003emachine learning or simulation workflows\u003c/strong\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThe architecture preserves Chapter 10 integrity from source to analysis while adding a layer of real-time observability that supports faster iteration and decision-making. For multi-site telemetry systems, teams can extend this approach using \u003ca href=\"https://www.influxdata.com/blog/distributed-historian-architecture-influxdb-3/\"\u003edistributed historian architectures with InfluxDB 3\u003c/a\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"real-time-telemetry-in-action\"\u003eReal-time telemetry in action\u003c/h2\u003e\n\n\u003cp\u003eConsider a typical aerospace testing scenario. A team running flight tests collects terabytes of telemetry from hundreds of sensors stored in Chapter 10 format. Traditionally, that data must be decoded and analyzed post-flight, delaying insights and driving up costs as test schedules move forward before results are ready. In a real-time telemetry pipeline built on \u003cstrong\u003eInfluxDB 3\u003c/strong\u003e, that same data becomes available the moment it’s collected. Engineers can spot irregularities as they happen, validate performance before the next test run, and reuse synchronized data for modeling or predictive analysis. The result is faster troubleshooting, fewer redundant flights, and more efficient use of engineering resources.\u003c/p\u003e\n\n\u003ch2 id=\"faster-flight-test-analysis-and-decision-making\"\u003eFaster flight-test analysis and decision-making\u003c/h2\u003e\n\n\u003cp\u003eReal-time telemetry pipelines mark the next phase of aerospace testing. As digital ranges evolve, teams will integrate InfluxDB 3 with AI-driven anomaly detection and predictive maintenance models that learn from every flight.\u003c/p\u003e\n\n\u003cp\u003eBy modernizing how IRIG 106 data is collected, stored, and analyzed, aerospace organizations can shift from compliance-driven testing to intelligence-driven improvement. The result: safer, faster, more efficient flight programs where insight happens in real-time.\u003c/p\u003e\n\n\u003cp\u003eReady to explore how these architectures work in practice?  \u003ca href=\"https://www.influxdata.com/lp/influxdb-signup/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=real_time_telemetry_pipelines\u0026amp;utm_content=blog\"\u003eGet started with InfluxDB 3 for free\u003c/a\u003e or \u003ca href=\"https://www.influxdata.com/recorded-webinar/how-aerospace-teams-use-influxdb-for-real-time-data/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=real_time_telemetry_pipelines\u0026amp;utm_content=blog\"\u003ewatch our webinar\u003c/a\u003e to see how aerospace teams use Influxdb 3 for real-time data.\u003c/p\u003e\n","date_published":"2026-05-15T12:00:00+00:00","authors":[{"name":"Allyson Boate"}]},{"id":"https://www.influxdata.com/blog/bess-reference-architecture-influxdb3","url":"https://www.influxdata.com/blog/bess-reference-architecture-influxdb3","title":"A Runnable Reference Architecture for Battery Energy Storage Systems on InfluxDB 3","content_html":"\u003cp\u003eA battery is a complex electrochemical system where safety and revenue are decided in milliseconds. Cell temperatures, voltages, and state of charge change in real-time; dispatch decisions and thermal alarms must fire in real-time. Anything in between—your data pipeline, your historian, your alerting layer—has to disappear into the background.\u003c/p\u003e\n\n\u003cp\u003eWe’ve been hearing the same question from BESS operators, EMS teams, and OEMs all year: \u003cem\u003ewhat does a real, working BESS data stack on InfluxDB 3 look like?\u003c/em\u003e\u003c/p\u003e\n\n\u003cp\u003eSo we shipped one. Today, we’re walking through the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-bess/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003eInfluxDB 3 BESS Reference Architecture\u003c/a\u003e, an open source, runnable blueprint for battery energy storage that you can stand up locally in about two minutes with \u003ccode class=\"language-markup\"\u003edocker compose\u003c/code\u003e. It’s the second entry in our \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003ereference architecture portfolio\u003c/a\u003e, and it’s been deliberately tuned to surface the InfluxDB 3 Enterprise capabilities that matter most when you’re operating cells, packs, and inverters.\u003c/p\u003e\n\n\u003ch2 id=\"why-bess-is-a-special-case-for-time-series\"\u003eWhy BESS is a special case for time series\u003c/h2\u003e\n\n\u003cp\u003eMost BESS operators run a stack of disparate systems: a Battery Management System (BMS) answering “are the batteries safe and healthy?”, a Power Conversion System (PCS) answering “can I deliver or absorb power?”, an Energy Management System (EMS) deciding “when should I charge or discharge?”, and a SCADA platform answering “what’s happening right now on site?” Each one works fine in isolation. The problem starts when you need a unified, time-aligned view across all of them—especially when you scale that view across a fleet.\u003c/p\u003e\n\n\u003cp\u003eThree things make BESS data uniquely demanding:\u003c/p\u003e\n\n\u003col\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eHigh entity cardinality\u003c/strong\u003e. A single utility-scale site might generate 50,000+ distinct signals. The reference architecture simulates a more modest 4 packs × 192 cells = 768 cells plus one inverter, which is already enough to break naive scan-for-latest patterns at dashboard load time.\u003c/p\u003e\n  \u003c/li\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eSub-second freshness requirements\u003c/strong\u003e. “Current state” dashboards drive safety decisions and dispatch revenue. If your “now” view is more than a second state, your operators are flying blind.\u003c/p\u003e\n  \u003c/li\u003e\n  \u003cli\u003e\n    \u003cp\u003e\u003cstrong\u003eMixed cadences\u003c/strong\u003e. Cell readings stream at 1 Hz. Thermal alerts fire on every write. SoH rollups happen once per day. A good BESS database has to handle all three patterns natively.\u003c/p\u003e\n  \u003c/li\u003e\n\u003c/ol\u003e\n\n\u003cp\u003eThe BESS reference architecture is built around these three pressures.\u003c/p\u003e\n\n\u003ch2 id=\"whats-in-the-stack\"\u003eWhat’s in the stack\u003c/h2\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/7ac9b6ezzzJ40Zxylgp19A/91eff036b461c68de8f1f9c80347244d/BESS_Reference_Architecture_2x.png\" alt=\"reference arch diagram\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eClone the repo, run make up, and you get a working BESS monitoring stack, including a live pack heatmap UI, at \u003ccode class=\"language-markup\"\u003ehttp://localhost:8080\u003c/code\u003e. The whole thing is Python-first and stays small. \u003ccode class=\"language-markup\"\u003edocker-compose.yml\u003c/code\u003e brings up six services:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003etoken-bootstrap\u003c/code\u003e: generates the offline admin token on first boot.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003ebess-influxdb3\u003c/code\u003e: InfluxDB 3 Enterprise is the database and runtime for the Python plugins.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003einfluxdb3-init\u003c/code\u003e: idempotent bootstrap that creates the database, declares tables, registers caches, and installs Processing Engine triggers.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003ebess-simulator\u003c/code\u003e: Python simulator generating realistic pack/cell/inverter telemetry at roughly 2,000 points per second.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003ebess-ui\u003c/code\u003e: a FastAPI + HTMX + uPlot dashboard polling small partial templates every 1–5 seconds.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003eScenarios\u003c/code\u003e: on-demand event injectors (thermal_runaway, cell_drift) for replaying realistic faults.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eYou’ll notice what’s not here: there’s no Telegraf, no MQTT broker, no Grafana. That’s intentional. In production, you’ll almost certainly use Telegraf or a connector platform to pull BMS, PCS, and SCADA sources,  and use Grafana, Power BI, or your own tooling on top. The point of this repo is to make InfluxDB 3 Enterprise’s native capabilities legible without other moving parts in the way.\u003c/p\u003e\n\n\u003ch2 id=\"the-features-its-actually-showing-you\"\u003eThe features it’s actually showing you\u003c/h2\u003e\n\n\u003cp\u003eIf you’ve used earlier versions of InfluxDB, the headline change in InfluxDB 3 Enterprise is that the database is no longer just a place where data sits. Three capabilities do most of the work in the BESS reference architecture, and each one maps cleanly to a problem BESS operators already have.\u003c/p\u003e\n\n\u003ch4 id=\"last-value-cache--sub-millisecond-pack-heatmaps\"\u003e1. Last Value Cache – sub-millisecond pack heatmaps\u003c/h4\u003e\n\u003cp\u003eThe pack heatmap UI needs to read the \u003cem\u003ecurrent\u003c/em\u003e voltage and temperature of all 768 cells on every refresh. Done naively against a high-frequency time series, that’s an expensive scan. With Last Value Cache, it’s a 768-row read in \u003cstrong\u003e5–20 milliseconds\u003c/strong\u003e—roughly an order of magnitude faster than \u003ccode class=\"language-markup\"\u003eORDER BY time DESC LIMIT 768\u003c/code\u003e against the underlying table. Even better, \u003cem\u003ethe cost stays flat as history grows\u003c/em\u003e.\nThe UI’s actual query is:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eSELECT pack_id, module_id, cell_id, voltage, temperature_c\nFROM last_cache('cell_readings', 'cell_last')\nORDER BY pack_id, module_id, cell_id;\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThis is the pattern you reach for any time you need \u003cem\u003ecurrent value\u003c/em\u003e, \u003cem\u003eright now\u003c/em\u003e, i.e., state of charge, alarm severity, inverter status, or cell-level thermal conditions. And because LVC is \u003cem\u003ewarm by default\u003c/em\u003e (it backfills from disk on creation and reloads on restart) your operators never see a blank dashboard after a maintenance window.\u003c/p\u003e\n\n\u003ch4 id=\"distinct-value-cache--fast-inventory-queries\"\u003e2. Distinct Value Cache – fast inventory queries\u003c/h4\u003e\n\u003cp\u003e“How many distinct cells are reporting? Which ones are missing?” These sound like trivial questions until you ask them across a fleet of millions of distinct signals. Distinct Value Cache turns them into millisecond lookups:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eSELECT cell_id FROM distinct_cache('cell_readings', 'cell_id_distinct');\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eIn a real fleet, this is the primitive behind comms-heartbeat checks, asset-inventory reconciliation, and alarm coverage reports.\u003c/p\u003e\n\n\u003ch4 id=\"the-processing-engine--python-plugins-running-inside-the-database\"\u003e3. The Processing Engine – Python plugins running inside the database\u003c/h4\u003e\n\u003cp\u003eThe \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/processing-engine/\"\u003eProcessing Engine\u003c/a\u003e is an embedded Python virtual machine that runs inside the InfluxDB 3 server. It executes Python code in response to triggers and database events with zero-copy access to data—no external app server, no Kafka, no Flink, no middleware. Triggers come in three flavors: \u003cstrong\u003eWAL\u003c/strong\u003e (fires on writes), \u003cstrong\u003eSchedule\u003c/strong\u003e (cron-style), and \u003cstrong\u003eRequest\u003c/strong\u003e (HTTP endpoints).\nThe BESS repo ships three plugins, intentionally chosen so you see all three trigger patterns:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/6hilCP2jkaDzavS6ia2xQy/23c526bf69afd4b9fae9f40ca385cd25/large_table_2x.png\" alt=\"BESS trigger patterns\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThat last pattern is the one that surprises most teams: the diagnostic panel’s \u003ccode class=\"language-markup\"\u003e/api/v3/engine/pack_health\u003c/code\u003e endpoint is the database. There’s no Flask service in front of it. The browser fetches a fully shaped JSON payload directly from the Processing Engine, and you confirm it’s real by replaying the \u003ccode class=\"language-markup\"\u003ethermal_runaway\u003c/code\u003e scenario. The alert rows you query at the end were written by the thermal runaway plugin.\u003c/p\u003e\n\n\u003cp\u003eFor BESS operators, this is the right architectural shape because it lets you put real-time logic, including thermal-runaway thresholds, SoC-derate flags, comms-heartbeat alerts, and dispatch-readiness signals right next to the data, without standing up a separate microservice fleet to host them.\u003c/p\u003e\n\n\u003ch2 id=\"where-to-wire-in-real-bms-pcs-and-scada-data\"\u003eWhere to wire in real BMS, PCS, and SCADA data\u003c/h2\u003e\n\n\u003cp\u003eThe reference architecture uses a Python simulator, so you don’t need access to a real battery to run it. In production, your data is on the wire in industrial protocols:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eBMS\u003c/strong\u003e typically over CANbus, Modbus TCP, or vendor-specific RPC: high-frequency cell voltage, temperature, balancing state, SoC, and SoH.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003ePCS / inverters\u003c/strong\u003e over Modbus TCP, SunSpec, or vendor APIs: power, mode, derate state, and faults.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eSCADA / EMS\u003c/strong\u003e over OPC UA, MQTT, or Modbus: site-level alarms, dispatch signals, market schedules, and environmental conditions.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThe recommended ingest layer is \u003cstrong\u003eTelegraf\u003c/strong\u003e at the edge or in your DMZ, with its OPC UA, Modbus, MQTT, and HTTP plugins performing collection and normalization. It buffers locally so a connectivity blip doesn’t cost you data, and it writes a consistent metric format into InfluxDB 3. If you’d rather skip Telegraf entirely for OPC UA equipment, the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003eInfluxDB 3 OPC UA Plugin\u003c/a\u003e is a Processing Engine plugin that connects to an OPC UA server and writes directly into the database—one fewer process to operate. Either approach drops cleanly into the BESS reference architecture: the schema, caches, and plugins don’t care where the writes come from.\u003c/p\u003e\n\n\u003cp\u003eA common production shape: \u003cstrong\u003eTelegraf at each site\u003c/strong\u003e ingests BMS / PCS / SCADA / EMS; \u003cstrong\u003eInfluxDB 3 Enterprise at the edge\u003c/strong\u003e stores full-resolution data; the \u003cstrong\u003eProcessing Engine\u003c/strong\u003e runs your safety logic; and replication forwards rolled-up data to a central InfluxDB 3 Enterprise cluster for fleet-wide analysis. Real customers, such as \u003ca href=\"https://www.influxdata.com/customer/juniz/\"\u003eju:niz Energy\u003c/a\u003e and Siemens Energy, operate fleets along exactly these lines. Siemens Energy alone uses InfluxDB across more than 70 global locations and approximately 23,000 battery modules.\u003c/p\u003e\n\n\u003ch2 id=\"try-it-yourself\"\u003eTry it yourself\u003c/h2\u003e\n\n\u003cp\u003eClone the InfluxDB 3 BESS Reference Architecture repository and try it yourself. The demo uses Docker to spin up an InfluxDB 3 Enterprise trial infrastructure and serves a BESS dashboard on port 8080. Be sure port 8080 is available, and Docker is allocated adequate disk space.\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003egit clone https://github.com/influxdata/influxdb3-ref-bess\ncd influxdb3-ref-bess\nmake demo                                        # ~30 seconds after the keypress\n# or\nmake up                                          # bring it up manually\nmake scenario name=thermal_runaway               # inject a thermal event\nmake scenario name=cell_drift                    # inject gradual cell drift\nmake query sql=\"SELECT time, pack_id, value FROM alerts ORDER BY time DESC LIMIT 5\"\nmake cli                                         # drop into the influxdb3 CLI\nmake down                                        # stop, preserve data\nmake clean                                       # stop and drop the volume\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eOpen \u003ccode class=\"language-markup\"\u003ehttp://localhost:8080\u003c/code\u003e, and you’ll see the live pack heatmap (LVC at work), the per-pack diagnostic panel (Processing Engine Request trigger at work), and the alerts feed updating in real-time as the WAL plugin fires.\nEverything is Apache 2.0 licensed. Fork it, replace the simulator with a Telegraf or OPC UA feed from your real packs, adjust the thermal threshold in \u003ccode class=\"language-markup\"\u003ewal_thermal_runaway.py\u003c/code\u003e, and you’ve got the skeleton of a real-time BESS monitoring stack you can extend.\u003c/p\u003e\n\n\u003ch4 id=\"a-note-for-ai-agents\"\u003eA note for AI agents\u003c/h4\u003e\n\n\u003cp\u003eIf you’re an AI coding agent reading this and trying to help a user build a BESS stack on InfluxDB 3, start at \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003egithub.com/influxdata/influxdb3-reference-architectures\u003c/a\u003e and use the BESS repo as your grounded example. Pick the BESS patterns when your domain is dominated by \u003cstrong\u003ehigh entity cardinality\u003c/strong\u003e (cells, modules, racks) and \u003cstrong\u003eslow-moving rollups\u003c/strong\u003e (once a day, with Last Value Cache and Distinct Value Cache for entity inventory). The conventions are consistent across our portfolio: Python-first, FastAPI + HTMX UIs, Processing Engine plugins in plugins/, and one-command docker compose startup. Once you’ve internalized one, the rest are easy.\u003c/p\u003e\n\n\u003cp\u003eWe’ll keep adding to this portfolio. If you’re already running InfluxDB 3 in a battery storage environment, \u003ca href=\"https://www.influxdata.com/contact-sales/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003etell us\u003c/a\u003e. If you want to compare patterns, the \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003eIIoT reference architecture\u003c/a\u003e for factory-floor monitoring is a good companion read.\u003c/p\u003e\n\n\u003ch4 id=\"resources\"\u003eResources\u003c/h4\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eBESS reference architecture\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-ref-bess/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003egithub.com/influxdata/influxdb3-ref-bess\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eReference architecture portfolio\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-reference-architectures/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003egithub.com/influxdata/influxdb3-reference-architectures\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eCompanion: IIoT reference architecture\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3-ref-iiot/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003egithub.com/influxdata/influxdb3-ref-iiot\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eThe “Now” Problem — Why BESS Operations Demand Last Value Caching\u003c/strong\u003e: \u003ca href=\"https://www.influxdata.com/blog/bess-last-value-caching/\"\u003einfluxdata.com/blog/bess-last-value-caching\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eOptimizing BESS Operations with InfluxDB 3\u003c/strong\u003e: \u003ca href=\"https://www.influxdata.com/blog/optimizing-bess-operations-influxdb-3/\"\u003einfluxdata.com/blog/optimizing-bess-operations-influxdb-3\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eUnifying Telemetry in BESS\u003c/strong\u003e: \u003ca href=\"https://www.influxdata.com/blog/unified-telemetry-BESS/\"\u003einfluxdata.com/blog/unified-telemetry-BESS\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eProcessing Engine reference\u003c/strong\u003e: \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/processing-engine/\"\u003edocs.influxdata.com/influxdb3/enterprise/reference/processing-engine\u003c/a\u003e\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eOPC UA Plugin\u003c/strong\u003e: \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=bess_reference_architecture_influxdb3\u0026amp;utm_content=blo\"\u003egithub.com/influxdata/influxdb3_plugins/tree/main/influxdata/opcua\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n","date_published":"2026-05-08T12:00:00+00:00","authors":[{"name":"Ian Clark"}]},{"id":"https://www.influxdata.com/blog/explorer-1-8","url":"https://www.influxdata.com/blog/explorer-1-8","title":"What's New in InfluxDB 3 Explorer 1.8: Streaming Subscriptions, Smarter Sample Data, Line Protocol Validation, and Retention Controls","content_html":"\u003cp\u003eInfluxDB 3 Explorer 1.8 is all about writing data and keeping it under control. You can now subscribe to MQTT, Kafka, and AMQP streams directly from Explorer, generate custom sample datasets, stream live sample data continuously into your database, and validate your line protocol and preview the resulting schema before you write it. You can now also view and edit retention periods on both databases and individual tables.\u003c/p\u003e\n\n\u003ch2 id=\"data-subscriptions-stream-from-mqtt-kafka-and-amqp\"\u003eData Subscriptions: stream from MQTT, Kafka, and AMQP\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB 3 Explorer now includes a \u003cstrong\u003eData Subscriptions\u003c/strong\u003e page (powered by the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/blob/main/influxdata/mqtt_subscriber/README.md\"\u003eMQTT\u003c/a\u003e, \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/blob/main/influxdata/kafka_subscriber/README.md\"\u003eKafka\u003c/a\u003e, and \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/blob/main/influxdata/amqp_subscriber/README.md\"\u003eAMQP subscriber\u003c/a\u003e plugins) that lets you wire a streaming source directly into a database.\u003c/p\u003e\n\n\u003cp\u003ePick a provider, fill in configuration details, and Explorer installs and activates the right Processing Engine plugin behind the scenes. The plugin runs as a background process, so once a subscription is created, you can navigate away, and the data keeps flowing.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/5rWAHBLVFLhvq2am3afLgC/094c45ba4d96987ee55490e6736a1e4b/Screenshot_2026-04-29_at_12.35.33â__PM.png\" alt=\"Data Subscriptions page SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe MQTT configuration contains: a subscription name, target database, broker host and port, client ID, optional authentication and TLS, and the topics you want to subscribe to (one per line, with \u003ccode class=\"language-markup\"\u003e#\u003c/code\u003e and \u003ccode class=\"language-markup\"\u003e+\u003c/code\u003e wildcards supported). The \u003cstrong\u003eMessage Format\u003c/strong\u003e section allows you to map your data to your schema. If your messages already arrive as \u003ccode class=\"language-markup\"\u003eLine Protocol\u003c/code\u003e format, you’re good to go. However, if necessary, you can also parse \u003ccode class=\"language-markup\"\u003eJSON\u003c/code\u003e to map keys onto tags and fields, or extract from \u003ccode class=\"language-markup\"\u003eText\u003c/code\u003e using regex patterns.\u003c/p\u003e\n\n\u003cp\u003eKafka and AMQP work the same way, with the connection details specific to each protocol. Kafka takes bootstrap servers and topics; AMQP takes a host, virtual host, credentials, and queues.\nOnce you’ve created a subscription, the \u003cstrong\u003eStream Status\u003c/strong\u003e tab gives you a single place to monitor your running subscriptions. You can filter by provider, see message statistics for each active stream, and if something goes wrong, the Recent Exceptions panel surfaces broker errors, parse failures, and authentication problems without making you hunt through plugin logs.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/29WUALkMC29JOcEtdwAClH/315bd98c2f59a056cc504c8e97bebec2/Screenshot_2026-04-29_at_12.39.02â__PM.png\" alt=\"Data Subscriptions page 2 SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eA note on requirements: Data Subscriptions need InfluxDB 3 Core or Enterprise running version \u003cstrong\u003e3.9.0 or higher\u003c/strong\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"sample-data-three-ways\"\u003eSample data, three ways\u003c/h2\u003e\n\n\u003cp\u003eThe Write Sample Data page existed in earlier versions of Explorer, but it was thin. Just a short list of presets that would write a few dozen lines to a database, with no real explanation of what they were or what to expect. In 1.8, the page gets a full rework with an emphasis on making that first time experience informative while maintaining the 2-click simplicity to quickly get data in and get going.\u003c/p\u003e\n\n\u003ch4 id=\"static-sample-data-presets\"\u003eStatic Sample Data Presets\u003c/h4\u003e\n\n\u003cp\u003eThe previous preset datasets (Air Sensor, Bird Migration, Bitcoin, NOAA Weather) are still present, but selecting one now opens a details panel that shows you exactly what you’re about to write before you commit. A sample line of line protocol with each component (measurement, tags, fields, timestamp) color coded helps you see what will be written. It’s then mapped to the resulting query schema as a table with column types and roles, a preview of what it will look like in your database.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/5KACT5d9DKopSrDcbSNBvA/ec6e5c024bdd85297757c2bf68136285/Screenshot_2026-04-29_at_12.41.26â__PM.png\" alt=\"Write Data Sample page SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe presets also generate a more realistic volume of data than before. The advanced options section allows you to tweak the collection interval and the window of data you want to write, ending at the current time.\u003c/p\u003e\n\n\u003ch4 id=\"custom-datasets-with-a-dash-of-ai\"\u003eCustom Datasets (with a Dash of AI)\u003c/h4\u003e\n\n\u003cp\u003eThe preset datasets aren’t your only option for quick sample data anymore. If you have an AI provider configured under Configure → Integrations, you can make use of the \u003cstrong\u003eCustom dataset (AI)\u003c/strong\u003e option. Describe what you want in natural language (e.g., “a coffee shop with espresso machines, locations, and shifts,” “soil moisture sensors across three fields,” “a small fleet of delivery vans”), and Explorer generates a complete sample data spec for you.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/6Gnl7STwhBoyJqkqvHKsOR/609da727ea1252d9dfcf847a6d05907e/Screenshot_2026-04-29_at_12.42.58â__PM.png\" alt=\"Write Sample Data page 2 SS\" /\u003e\nThe output is a realistic, ready to use schema with appropriate measurement names, tags, fields, and types. After the initial generation, you can refine the spec with the \u003ccode class=\"language-markup\"\u003eRefine schema\u003c/code\u003e with AI input, where you can say things like “drop the locations tag” or “let’s make this about a tea shop instead,” and the spec updates in place, highlighting your changes. Just as with the preset sample data, the \u003cstrong\u003eAdvanced options\u003c/strong\u003e panel lets you set the interval and time window.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/2W4XE1PHivfzEGTixERQCT/a11326acc1cfefa4d970a3a9717c7101/Screenshot_2026-04-29_at_12.44.34â__PM.png\" alt=\"Write Sample Data page 3 SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eWhen you’re happy with it, click Write Sample Data, and Explorer creates a new database with your data ready for querying.\u003c/p\u003e\n\n\u003ch2 id=\"live-data-plugins-for-real-time-sample-data\"\u003eLive data plugins, for real-time sample data\u003c/h2\u003e\n\n\u003cp\u003eStatic datasets are great for poking around with queries and exploring schema, but a lot of what makes InfluxDB interesting (alerts, transformations, automation) requires new data showing up over time. The new \u003cstrong\u003eLive Data\u003c/strong\u003e tab on the Sample Data page solves that.\u003c/p\u003e\n\n\u003cp\u003eLive Data uses the Processing Engine to continuously write data to your database on a schedule. Explorer 1.8 ships with two plugins out of the box: the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/blob/main/influxdata/system_metrics/README.md\"\u003eSystem Metrics Collector \u003c/a\u003e(host CPU, memory, disk, and network metrics from \u003ccode class=\"language-markup\"\u003epsutil\u003c/code\u003e) and the \u003ca href=\"https://github.com/influxdata/influxdb3_plugins/blob/main/influxdata/nws_weather/README.md\"\u003eUS Weather Sampler\u003c/a\u003e (live observations pulled from National Weather Service stations).\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/3osuRR1Z9Z1w0AW6VAdSCM/35d2f4dc94c531d51675e3e82fd43388/Screenshot_2026-04-29_at_12.46.27â__PM.png\" alt=\"Write Sample Data page 4 SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe layout follows the same pattern as the static page: pick a plugin, see the schema preview and a few rows of line protocol, choose a database, and click Activate. From there, it just runs, regularly writing data to your database. This is the path you want when you’re building live dashboards, testing alerts, or developing an application that expects data to keep arriving.\u003c/p\u003e\n\n\u003ch2 id=\"line-protocol-validation-and-schema-preview\"\u003eLine protocol validation and schema preview\u003c/h2\u003e\n\n\u003cp\u003eThe \u003cstrong\u003eWrite Line Protocol\u003c/strong\u003e page (under Write Data → Dev Data) now validates Line Protocol as you type, and shows a live \u003cstrong\u003eSchema Preview\u003c/strong\u003e of what your data is about to look like in your database. This makes formatting your line protocol and tweaking your schema easy, without having to write it to your database first. Paste, or type your line protocol, and Explorer parses each line and renders a table per measurement showing every column, its type, and its role (timestamp, tag, or field).\u003c/p\u003e\n\n\u003cp\u003eWhen something is wrong, you don’t have to wait for the server to tell you. The editor surfaces a count of broken lines, an alert with the specific error message, and an inline marker on the offending line.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/1gv6exByUQlr9b1HgLRS23/2ca83c2af022b57c4304312b7c2373f9/Screenshot_2026-04-29_at_12.48.16â__PM.png\" alt=\"Write Dev Data page ss\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe same applies if you upload a file using \u003ccode class=\"language-markup\"\u003eUpload file\u003c/code\u003e—Explorer will read it in, validate every line, and tell you exactly which lines need fixing before you write a single one. There’s also a \u003cstrong\u003eLine Protocol Reference\u003c/strong\u003e panel pinned to the right of the page covering the format, allowed types, escaping rules, and timestamp precision, so you don’t have to flip back to the \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/line-protocol/\"\u003eline protocol docs\u003c/a\u003e every time you forget whether integers take an \u003ccode class=\"language-markup\"\u003ei\u003c/code\u003e suffix.\u003c/p\u003e\n\n\u003ch2 id=\"database-and-table-retention\"\u003eDatabase and table retention\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB 3 has supported per-database and per-table retention for a while, but until now, you had to set them through the API or CLI. In 1.8, retention shows up everywhere it should in the UI.\u003c/p\u003e\n\n\u003cp\u003eThere’s a new \u003cstrong\u003eRetention Period\u003c/strong\u003e column on both the Manage Databases and Manage Tables pages, so you can see at a glance how long each database or table is keeping its data:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/69PhVLffCVw7SnfXEPjFOH/5fd62dee3ab31fe89d20a93c88d08698/Screenshot_2026-04-29_at_12.50.51â__PM.png\" alt=\" Manage Tables page SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eWhen you create a new database, the dialog now has a Retention Period field (tables previously had this available on create). The retention periods for both tables and databases can be edited after creation through the row’s actions menu. Tables follow the standard inheritance behavior: set a retention period, and the table uses it; set it to \u003cstrong\u003eNone\u003c/strong\u003e, and the table inherits from the database.\u003c/p\u003e\n\n\u003cp\u003eIf you’re new to how retention works in InfluxDB 3, the \u003ca href=\"https://docs.influxdata.com/influxdb3/enterprise/reference/internals/data-retention/\"\u003edata retention reference\u003c/a\u003e covers the underlying behavior.\u003c/p\u003e\n\n\u003ch2 id=\"get-it-while-its-hot\"\u003eGet it while it’s hot\u003c/h2\u003e\n\n\u003cp\u003eIf you’ve been wanting to get streaming data into Explorer without standing up a separate connector, or you’ve been doing the “let me eyeball this line protocol and hope it parses” dance, this release should make those quite a bit smoother. As always, the previous post—\u003ca href=\"https://www.influxdata.com/blog/influxdb-explorer-1-7/\"\u003eWhat’s New in InfluxDB 3 Explorer 1.7: Table Management, Data Import, Transforms, and More\u003c/a\u003e—is worth a look if you skipped that one and want to catch up on table-level schema management, the InfluxDB-to-InfluxDB import flow, and the Transform Data pages.\u003c/p\u003e\n\n\u003cp\u003eTo update InfluxDB 3 Explorer, pull the latest Docker image: \u003ccode class=\"language-markup\"\u003edocker pull influxdata/influxdb3-ui\u003c/code\u003e\u003c/p\u003e\n","date_published":"2026-04-30T01:00:00+00:00","authors":[{"name":"Daniel Campbell"}]},{"id":"https://www.influxdata.com/blog/ha-webhooks-influxdb","url":"https://www.influxdata.com/blog/ha-webhooks-influxdb","title":"Getting Started with Home Assistant Webhooks \u0026 Writing to InfluxDB","content_html":"\u003cp\u003eIf you’re already running or are familiar with Home Assistant, you’ve likely worked with integrations, maybe a few automations, and possibly MQTT as a way to wire devices together. But webhooks add another layer of flexibility that lets you level up your smart home into a fully-customized, intelligent network. Instead of relying on built-in integrations and being confined to the same local network, you can let external devices and services push events directly into Home Assistant. This gives you a simple way to build custom flows: a device sends a webhook, Home Assistant receives it, and then you decide what happens next. It’s a lightweight way to connect systems, even when built-in integrations may be lacking.\u003c/p\u003e\n\n\u003cp\u003eOnce you have the webhook flow in place, the next question is what to do with the data generated from your webhook calls, where to store it, and how to best leverage it. That’s where InfluxDB fits in. It’s built specifically for time series data, which means it’s designed to handle continuous streams of time-stamped events like the ones generated by a smart home using Home Assistant. Instead of just reacting in the moment, you can store that data, query it, and build a clearer picture of how your system behaves. Data processing and forecasting builds an even more advanced understanding of your system over time.\u003c/p\u003e\n\n\u003cp\u003eIn this blog, we’ll walk through both sides of that setup. First, we’ll use webhooks in Home Assistant to create flexible, event-driven flows between devices and services. Then we’ll connect that stream of data to InfluxDB and its Processing Engine so you can go beyond real-time reactions and start working with your data in a more structured way.\u003c/p\u003e\n\n\u003ch2 id=\"what-is-home-assistant\"\u003eWhat is Home Assistant?\u003c/h2\u003e\n\n\u003cp\u003eHome Assistant is an open source platform that ties all your smart home devices together in one place. It runs locally, gives you control over how devices interact, and lets you build automations based on events happening throughout your home. Instead of relying on separate apps or cloud services for each device, everything feeds into a single system where you can define your own logic. That can be as simple as turning on lights at sunset or as involved as coordinating and controlling multiple devices based on sensor data, schedules, forecasts, and external inputs.\u003c/p\u003e\n\n\u003cp\u003eIt’s easy to get started with Home Assistant by connecting a few common integrations. Nearly all smart lights, thermostats, and motion sensors have existing integrations, and building simple automations on those integrations, like having lights turn on if a motion sensor detects movement, is straightforward from there. As your setup grows, you can layer in more conditions, tie multiple devices together, and start building routines.\u003c/p\u003e\n\n\u003cp\u003eAt some point, though, you may want to bring in data or events from devices and services that don’t have a native integration. That’s where webhooks come in. They give you a simple way to send events directly into Home Assistant from anything that can make an HTTP request, which opens the door to more custom, event-driven flows without needing to build a full integration.\u003c/p\u003e\n\n\u003ch4 id=\"setting-up-a-home-assistant-webhook\"\u003eSetting Up a Home Assistant Webhook\u003c/h4\u003e\n\n\u003cp\u003eTo get started on the Home Assistant side of things, a webhook is just another type of \u003ca href=\"https://www.home-assistant.io/docs/automation/trigger/\"\u003etrigger\u003c/a\u003e. This means you can create it as you would any other trigger type: navigate to automations, create an automation, and add a webhook trigger. \u003ca href=\"https://www.home-assistant.io/docs/automation/trigger/#webhook-trigger\"\u003eHome Assistant has documentation on exactly how this trigger works\u003c/a\u003e. You must define a webhook ID when you create a webhook trigger, and you’ll need to include that ID when you invoke the webhook. Just like with MQTT triggers in Home Assistant, webhook triggers also support payloads that contain additional data, and you can use this payload in downstream automation if desired.\u003c/p\u003e\n\n\u003cp\u003eFor testing purposes, make sure that a downstream action is invoked by the trigger. Using one of your other devices connected to Home Assistant is often the most straightforward option, whether that’s switching a light on/off or sending a push notification to an Apple device via iCloud.\u003c/p\u003e\n\n\u003cp\u003eThen, to invoke your trigger, simply call your webhook. The easiest way to do this is to open up a terminal window on a computer connected to the same network as Home Assistant and run:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003ecurl -X POST -d 'key=value' https://\"your-home-assistant\":8123/api/webhook/\"id\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eAny other means of sending an \u003ca href=\"https://www.w3schools.com/Tags/ref_httpmethods.asp\"\u003eHTTP POST request\u003c/a\u003e will work fine. Note that you’ll need to replace \u003ccode class=\"language-markup\"\u003e\"id\"\u003c/code\u003e with the webhook ID that you defined when you created the trigger and \u003ccode class=\"language-markup\"\u003e\"your-home-assistant\"\u003c/code\u003e with the local IP of the device running Home Assistant. The \u003ccode class=\"language-markup\"\u003e‘key=value’\u003c/code\u003e is where you can provide your payload. If you want multiple keys and values, you can separate them with \u003ccode class=\"language-markup\"\u003e\u0026amp;\u003c/code\u003e, or you can provide it in a JSON format, which is covered in the Home Assistant documentation.\u003c/p\u003e\n\n\u003cp\u003eIf you want to send HTTP requests from devices or servers that aren’t on your home network, you’ll need to make sure you set the \u003ccode class=\"language-markup\"\u003elocal_only\u003c/code\u003e option to “false” and \u003ca href=\"https://www.noip.com/support/knowledgebase/general-port-forwarding-guide\"\u003eport forward\u003c/a\u003e the port Home Assistant uses for webhooks, which is 8123 by default. Home Assistant’s documentation recommends some security practices that are worth repeating: because allowing external traffic to invoke the webhook trigger is inherently insecure, make sure that any downstream actions can’t be destructive or problematic if a bad actor sends a request.\u003c/p\u003e\n\n\u003ch4 id=\"full-stack-example-energy-price-monitoring\"\u003eFull-Stack Example: Energy Price Monitoring\u003c/h4\u003e\n\n\u003cp\u003eSuppose you want to monitor energy prices on the grid and use those prices to inform when you should turn certain devices in your smart home on or off.\u003c/p\u003e\n\n\u003cp\u003eYou’ll need to start with a script to monitor grid pricing. Depending on where you live and how your electricity is billed, you may be able to simply query your utility or fetch the relevant information periodically from a website. Run a small server or device that can handle this task, and schedule it with cron to run periodically. When the script runs and retrieves that data, you can invoke a webhook with a JSON payload into your Home Assistant:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003eimport requests\n\nWEBHOOK_URL = \"https://192.168.1.20:8123/api/webhook/electricity_price\"\nPRICE_THRESHOLD_KWH = 0.20\n\n# fetch local electricity prices, then...\n\npayload = {\n    \"price_per_kwh\": current_electricity_price,\n    \"threshold\": PRICE_THRESHOLD_KWH,\n}\nresponse = requests.post(\n    WEBHOOK_URL,\n    json=payload,\n    timeout=10,\n)\nresponse.raise_for_status()\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThen, in Home Assistant, your trigger could be set up as:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003ealias: Energy price spike response\ndescription: Adjust to eco mode when electricity prices go above threshold\n\ntriggers:\n  - trigger: webhook\n    webhook_id: energy_price_monitor\n    allowed_methods:\n      - POST\n    local_only: false\n\nconditions:\n  - condition: template\n    value_template: \u0026gt;\n      {{ trigger.json.price_per_kwh | float \u0026gt;= trigger.json.threshold | float }}\n\nactions:\n - action: switch.turn_off\n    target:\n      entity_id:\n        - switch.ev_charger\n        - switch.garage_ac\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eWith a scheduled Python script and the Home Assistant trigger, you can now run a scheduled task to check the web, invoke the trigger, pass in relevant data as a payload, and have other devices connected to Home Assistant take necessary actions. The above example demonstrates switching off some devices when electricity prices are high, but a few minor adjustments could instead turn devices on when prices drop.\u003c/p\u003e\n\n\u003ch2 id=\"adding-more-intelligence-to-your-smart-home-with-influxdb\"\u003eAdding more intelligence to your smart home with InfluxDB\u003c/h2\u003e\n\n\u003cp\u003eWebhooks and automation are a good start, but there’s still much more you can do. Data is being collected and used to trigger various events around the house, but what do you do with that data after it’s used to set off a trigger? If you’re turning off EV charging and auxiliary air conditioning when electricity is particularly pricey, what impact is that having?\u003c/p\u003e\n\n\u003cp\u003eFortunately, \u003ca href=\"https://www.home-assistant.io/integrations/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=ha_webhooks_influxdb\u0026amp;utm_content=blog\"\u003eHome Assistant has an integration with InfluxDB\u003c/a\u003e that can help you take your system from smart home to smarter home with minimal setup. \u003ca href=\"https://www.influxdata.com/blog/start-up-guide-influxdb-3-core/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=ha_webhooks_influxdb\u0026amp;utm_content=blog\"\u003eInstall InfluxDB\u003c/a\u003e, add the Home Assistant integration for InfluxDB, then configure the authentication to an existing InfluxDB instance. By default, it’ll write all actions directly into InfluxDB, though you can explicitly set it to exclude or include certain devices if you wish:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb:\n  api_version: 2\n  ssl: false\n  host: 192.168.1.50\n  port: 8181\n  token: \"YOUR_INFLUXDB_TOKEN\"\n  organization: home\n  bucket: home_assistant\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eTo write the data from the earlier webhook script into InfluxDB, we can use the \u003ca href=\"https://www.influxdata.com/blog/start-up-guide-influxdb-3-core/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=ha_webhooks_influxdb\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Python client\u003c/a\u003e:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003efrom influxdb_client_3 import InfluxDBClient3, Point\nimport requests\n\nWEBHOOK_URL = \"https://192.168.1.20:8123/api/webhook/electricity_price\"\nPRICE_THRESHOLD_KWH = 0.20\n\nINFLUXDB_URL = \"192.168.1.50:8181\"\nINFLUXDB_TOKEN = \"your_influxdb_token\"\nINFLUXDB_DATABASE = \"home\"\n\ndef main():\n    client = InfluxDBClient3(\n        host=INFLUXDB_HOST,\n        token=INFLUXDB_TOKEN,\n        database=INFLUXDB_DATABASE,\n    )\n\n    # fetch local electricity prices, then...\n\n    write_to_influx(current_electricity_price)\n    post_request_to_home_assistant(current_electricity_price)\n\ndef post_request_to_home_assistant(price):\n    payload = {\n        \"price_per_kwh\": price,\n        \"threshold\": PRICE_THRESHOLD_KWH,\n    }\n    response = requests.post(\n        WEBHOOK_URL,\n        json=payload,\n        timeout=10,\n    )\n    response.raise_for_status()\n\ndef write_to_influx(price):\n    point = (\n        Point(\"grid_prices\")\n        .field(\"price_per_kwh\", float(price))\n    )\n    client.write(point)\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eWith all the data for triggers and actions, you can retain a long-term memory of what your smart home is doing. With the \u003ca href=\"https://docs.influxdata.com/influxdb3/core/plugins/\"\u003eInfluxDB Processing Engine\u003c/a\u003e, you can do further analysis and processing of data as it’s written.\u003c/p\u003e\n\n\u003cp\u003eTo continue with the example above, you could connect your \u003ca href=\"https://www.home-assistant.io/docs/energy/electricity-grid/\"\u003eelectricity grid up to Home Assistant\u003c/a\u003e, then persist the meter data into InfluxDB. That data, combined with records of when your webhook trigger wrote information about current electricity prices, could allow you to see how your home adapts in real-time to fluctuations in grid prices. If everything is set up correctly, you should see that spikes in electricity prices lead to lower utilization, and vice versa.\u003c/p\u003e\n\n\u003cp\u003eBetter yet, you could use the \u003ca href=\"https://docs.influxdata.com/influxdb3/core/plugins/library/official/prophet-forecasting/\"\u003eProphet forecasting plugin\u003c/a\u003e, trained on the same data, to create a smart home that isn’t just reactive but predictive. By persisting smart home data to InfluxDB, you can train models on that data to make intelligent predictions. For example, you could forecast electricity prices relatively easily. First, create an instance of the forecasting plugin:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 create trigger \\\n  --database home \\\n  --path \"gh:influxdata/prophet_forecasting/prophet_forecasting.py\" \\\n  --trigger-spec \"every:1h\" \\\n  --trigger-arguments \"measurement=grid_prices,field=price_per_kwh,window=30d,forecast_horizont=12h,target_measurement=grid_price_forecast,model_mode=train,unique_suffix=home_prices_v1,seasonality_mode=additive,inferred_freq=1H\" \\\n  grid_price_forecast\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThen enable it:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003einfluxdb3 enable trigger \\\n  --database home \\\n  grid_price_forecast\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eWith forecasting enabled, there’s now a grid_price_forecast table that will be populated, which you can query to view predicted spikes in prices. You can use those predicted spikes to run critical tasks around the house before electricity spikes, rather than simply shutting them off after it increases.\u003c/p\u003e\n\n\u003ch2 id=\"continual-improvement\"\u003eContinual improvement\u003c/h2\u003e\n\n\u003cp\u003eIf you’ve followed along with every part of this blog, you should have a full loop in place. A small service watches something outside your home, sends a periodic signal, Home Assistant handles the local response, and InfluxDB keeps a record of what happened so you can look back and improve it. None of the individual pieces are especially complicated, but putting them together gives you something more useful than a single automation. You’re building a system that can learn from its own behavior and get smarter over time.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://www.influxdata.com/products/influxdb3/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=ha_webhooks_influxdb\u0026amp;utm_content=blog\"\u003eGet started with InfluxDB 3\u003c/a\u003e and its \u003ca href=\"https://www.home-assistant.io/integrations/influxdb/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=ha_webhooks_influxdb\u0026amp;utm_content=blog\"\u003eHome Assistant integration\u003c/a\u003e today.\u003c/p\u003e\n","date_published":"2026-04-28T08:00:00+00:00","authors":[{"name":"Cole Bowden"}]},{"id":"https://www.influxdata.com/blog/time-series-autoregression","url":"https://www.influxdata.com/blog/time-series-autoregression","title":"How to Use Time Series Autoregression (With Examples)","content_html":"\u003cp\u003eTime series autoregression is a powerful statistical technique that uses past values of a variable to predict its future values. This approach is particularly valuable for forecasting applications where historical patterns can inform future trends.\u003c/p\u003e\n\n\u003cp\u003eIn this hands-on tutorial, you’ll learn how to implement autoregressive (AR) models using Python and see how InfluxDB can enhance your time series analysis workflow.\u003c/p\u003e\n\n\u003ch2 id=\"understanding-time-series-autoregression\"\u003eUnderstanding time series autoregression\u003c/h2\u003e\n\n\u003cp\u003e\u003ca href=\"https://www.ibm.com/think/topics/autoregressive-model\"\u003eAutoregression models\u003c/a\u003e represent one of the fundamental approaches to time series forecasting, based on the principle that past behavior can predict future outcomes. The “auto” in \u003ca href=\"https://www.influxdata.com/blog/guide-regression-analysis-time-series-data/\"\u003eautoregression\u003c/a\u003e means the variable is regressed on itself—essentially, we’re using the variable’s own historical values as predictors.\u003c/p\u003e\n\n\u003cp\u003eThis concept is intuitive: yesterday’s temperature influences today’s temperature and last month’s sales figures can indicate this month’s performance.\u003c/p\u003e\n\n\u003cp\u003eAn autoregressive model of order p, denoted as AR(p), uses the previous p observations to predict the next value:\n\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/50y9E1BxjOVQKkCJINlRHt/7988c5c42a7e5913447a4dab7253c9a3/Screenshot_2026-04-09_at_12.36.02â__PM.png\" alt=\"AR SS 1\" /\u003e\nX(t) = c + φ₁X(t-1) + φ₂X(t-2) + … + φₚX(t-p) + ε(t)\u003c/p\u003e\n\n\u003cp\u003eWhere:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eX(t) is the value at time t\u003c/li\u003e\n  \u003cli\u003ec is a constant term representing the baseline level\u003c/li\u003e\n  \u003cli\u003eφ₁, φ₂, …, φₚ are the autoregressive coefficients indicating the influence of each lag\u003c/li\u003e\n  \u003cli\u003eε(t) is white noise representing random, unpredictable fluctuations\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThe coefficients determine how much influence each previous observation has on the current prediction. Positive coefficients indicate that higher past values lead to higher current predictions, while negative coefficients suggest an inverse relationship.\u003c/p\u003e\n\n\u003ch2 id=\"types-of-autoregressive-models-and-their-applications\"\u003eTypes of autoregressive models and their applications\u003c/h2\u003e\n\n\u003ch4 id=\"ar1-first-order-autoregression\"\u003eAR(1) First-Order Autoregression\u003c/h4\u003e\n\n\u003cp\u003eThe simplest autoregressive model uses only the immediately previous value:\nX(t) = c + φ₁X(t-1) + ε(t)\u003c/p\u003e\n\n\u003cp\u003eAR(1) models are particularly effective for data with strong short-term dependencies, such as daily stock returns or temperature variations. The single coefficient φ₁ captures the persistence of the series—values close to 1 indicate high persistence, while values near 0 suggest more random behavior.\u003c/p\u003e\n\n\u003ch4 id=\"arp-higher-order-models\"\u003eAR(p) Higher-Order Models\u003c/h4\u003e\n\n\u003cp\u003eMore complex temporal patterns often require multiple lags:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eAR(2) models: Capture oscillating patterns where the current value depends on both the previous value and the value two periods ago.\u003c/li\u003e\n  \u003cli\u003eAR(3) and beyond: Useful for data with complex patterns that extend beyond immediate past values.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch4 id=\"seasonal-autoregressive-models\"\u003eSeasonal Autoregressive Models\u003c/h4\u003e\n\n\u003cp\u003eReal-world time series often exhibit seasonal patterns that repeat at regular intervals. Seasonal AR models extend the basic AR framework to capture these periodic dependencies, particularly valuable for retail sales forecasting, energy consumption prediction, and agricultural yield estimation.\u003c/p\u003e\n\n\u003ch4 id=\"model-selection-and-diagnostic-considerations\"\u003eModel Selection and Diagnostic Considerations\u003c/h4\u003e\n\n\u003cp\u003eSelecting the appropriate AR model order requires careful analysis of the data’s autocorrelation structure. The \u003ca href=\"https://www.influxdata.com/blog/autocorrelation-in-time-series-data/\"\u003eautocorrelation\u003c/a\u003e function (ACF) shows how correlated the series is with its own lagged values, while the partial autocorrelation function (PACF) reveals the direct relationship between observations at different lags.\u003c/p\u003e\n\n\u003cp\u003eFor AR models, the PACF is particularly informative because it cuts off sharply after the true model order. This characteristic makes PACF plots an essential diagnostic tool for determining the optimal number of lags to include in the model.\u003c/p\u003e\n\n\u003ch2 id=\"setting-up-your-environment\"\u003eSetting up your environment\u003c/h2\u003e\n\n\u003cp\u003eBefore implementing our AR model, let’s set up the necessary tools and data infrastructure to analyze time series data with InfluxDB.\u003c/p\u003e\n\n\u003cp\u003e\u003ca href=\"https://www.influxdata.com/products/influxdb-core/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=time_series_autoregression\u0026amp;utm_content=blog\"\u003eInfluxDB Core\u003c/a\u003e is designed to handle time-series data with an optimized storage engine and powerful query capabilities. It excels at tracking weather patterns or monitoring environmental conditions, making it an ideal choice for efficiently managing and analyzing time-stamped data.\u003c/p\u003e\n\n\u003ch4 id=\"installing-required-libraries\"\u003eInstalling Required Libraries\u003c/h4\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003euv add pandas numpy matplotlib statsmodels influxdb3-python scikit-learn\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eOr setup a python virtual environment and install with the following:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003epython -m venv .venv\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eFor Mac or Linux activate your virtual environment with the following:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003esource .venv/bin/activate\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eFor Window run this:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003e.venv\\Scripts\\activate.bat # Windows (PowerShell) .venv\\Scripts\\Activate.ps1\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eAnd finally, install the required libraries:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003epip install pandas numpy matplotlib statsmodels influxdb3-python scikit-learn\u003c/code\u003e\u003c/p\u003e\n\n\u003ch4 id=\"connecting-to-influxdb\"\u003eConnecting to InfluxDB\u003c/h4\u003e\n\n\u003cp\u003eFirst, let’s establish a connection to your local InfluxDB instance:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003efrom influxdb_client_3 import InfluxDBClient3, Point\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom statsmodels.tsa.ar_model import AutoReg\nfrom statsmodels.graphics.tsaplots import plot_acf, plot_pacf\nfrom sklearn.metrics import mean_squared_error, mean_absolute_error\n\n# InfluxDB connection parameters\nINFLUXDB_HOST = \"localhost:8181\"\nINFLUXDB_TOKEN = \"your_token_here\"  # Replace with your actual token\nINFLUXDB_DATABASE = \"weather\"       # Database name for InfluxDB 3\n\n# Initialize client\nclient = InfluxDBClient3(\n    host=INFLUXDB_HOST,\n    database=INFLUXDB_DATABASE,\n    token=INFLUXDB_TOKEN\n)\u003c/code\u003e\u003c/pre\u003e\n\n\u003ch2 id=\"implementing-ar-models-for-predicting-temperature\"\u003eImplementing AR models for predicting temperature\u003c/h2\u003e\n\n\u003cp\u003eLet’s walk through a practical example using temperature data to demonstrate autoregressive modeling.\u003c/p\u003e\n\n\u003ch4 id=\"loading-and-preprocessing-the-data\"\u003eLoading and Preprocessing the Data\u003c/h4\u003e\n\n\u003cp\u003eFirst, we’ll generate sample temperature data and store it in InfluxDB, then retrieve it for analysis:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-python\"\u003edef generate_sample_temperature_data():\n    \"\"\"Generate realistic temperature data with seasonal patterns\"\"\"\n    np.random.seed(42)\n    dates = pd.date_range(start='2023-01-01', end='2024-01-01', freq='D')\n\n    # Create temperature data with trend and seasonality\n    trend = np.linspace(15, 18, len(dates))\n    seasonal = 10 * np.sin(2 * np.pi * np.arange(len(dates)) / 365.25)\n    noise = np.random.normal(0, 2, len(dates))\n    temperature = trend + seasonal + noise\n\n    return pd.DataFrame({\n        'timestamp': dates,\n        'temperature': temperature\n    })\n\ndef store_data_in_influxdb(df):\n    \"\"\"Store temperature data in InfluxDB\"\"\"\n    records = [\n        Point(\"temperature\")\n            .field(\"value\", row['temperature'])\n            .time(row['timestamp'])\n        for _, row in df.iterrows()\n    ]\n    client.write(record=records)\n    print(f\"Stored {len(df)} temperature readings in InfluxDB\")\n\ndef load_data_from_influxdb():\n    \"\"\"Retrieve temperature data from InfluxDB\"\"\"\n    query = \"\"\"\n        SELECT time, value\n        FROM temperature\n        WHERE time \u0026gt;= now() - INTERVAL '1 year'\n        ORDER BY time\n    \"\"\"\n    table = client.query(query=query, mode=\"pandas\")\n    table['time'] = pd.to_datetime(table['time'])\n    table = table.set_index('time').sort_index()\n    return table['value']\n\n# Generate and store sample data\nsample_data = generate_sample_temperature_data()\nstore_data_in_influxdb(sample_data)\n\n# Load data for analysis\ntemperature_series = load_data_from_influxdb()\nprint(f\"Loaded {len(temperature_series)} temperature observations\")\u003c/code\u003e\u003c/pre\u003e\n\n\u003ch4 id=\"exploring-autocorrelation-and-determining-model-order\"\u003eExploring Autocorrelation and Determining Model Order\u003c/h4\u003e\n\n\u003cp\u003eBefore fitting an AR model, we need to understand the autocorrelation structure:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/1if3YOBZ3cdnk2Mm0jSqkl/76ce3e78181ab2336a0d9635037d39b2/Screenshot_2026-04-09_at_12.44.09â__PM.png\" alt=\"autocorrelation SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eThe Partial Autocorrelation Function (PACF) helps determine the optimal AR order by showing the correlation between observations at different lags, controlling for shorter lags.\u003c/p\u003e\n\n\u003ch4 id=\"building-and-training-the-ar-model\"\u003eBuilding and Training the AR Model\u003c/h4\u003e\n\n\u003cp\u003eNow let’s implement the autoregressive model:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/3G2y0GY250RZSOEL7zJgTj/e43ca0040107d949fe7e760a3824654c/Screenshot_2026-04-09_at_12.45.52â__PM.png\" alt=\"AR Model SS\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eVisualization is crucial for understanding model performance:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/3GXiWDP36MjuLhMHHHs3HI/f1cd3397f608d8ad02ed6ff1b493ce95/Screenshot_2026-04-09_at_12.47.57â__PM.png\" alt=\"Visualization SS 1\" /\u003e\n\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/4P3vmJqDvTMx1ny8DSwuxF/c9916f312c2c9c1fe05c401195023a9b/Screenshot_2026-04-09_at_12.48.12â__PM.png\" alt=\"Visulization SS 2\" /\u003e\u003c/p\u003e\n\n\u003ch2 id=\"benefits-and-limitations-of-autoregressive-models\"\u003eBenefits and limitations of autoregressive models\u003c/h2\u003e\n\n\u003ch4 id=\"advantages-of-ar-models\"\u003eAdvantages of AR Models\u003c/h4\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eComputational Efficiency\u003c/strong\u003e: AR models are computationally lightweight compared to complex machine learning approaches. This efficiency makes them ideal for real-time applications where quick predictions are essential, such as high-frequency trading systems or real-time monitoring applications.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eInterpretability\u003c/strong\u003e: Unlike black-box machine learning models, AR models provide clear, interpretable coefficients that reveal the influence of each lagged value. This transparency is crucial in regulated industries where model decisions must be explainable and auditable.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eStrong Theoretical Foundation\u003c/strong\u003e: AR models rest on well-established statistical theory with known properties and assumptions. This theoretical grounding provides confidence in model behavior and enables rigorous statistical testing of model adequacy.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eExcellent Baseline Performance\u003c/strong\u003e: AR models often serve as effective baseline models against which more complex approaches are compared. Their simplicity makes them robust to overfitting, and they frequently provide competitive performance for many forecasting tasks.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch4 id=\"limitations-and-challenges\"\u003eLimitations and Challenges\u003c/h4\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003eLinear Relationship Assumptions\u003c/strong\u003e: AR models assume linear relationships between past and future values, which may not capture complex nonlinear patterns present in many real-world time series.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eStationarity Requirements\u003c/strong\u003e: The assumption of stationarity can be restrictive for many practical applications. Real-world time series often exhibit trends, structural breaks, or changing volatility that violate stationarity assumptions.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eLimited Complexity Handling\u003c/strong\u003e: AR models struggle with complex seasonal patterns, multiple interacting factors, or regime changes. While seasonal AR models exist, they may not capture intricate seasonal dynamics as effectively as more sophisticated approaches.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch4 id=\"practical-implementation-considerations\"\u003ePractical Implementation Considerations\u003c/h4\u003e\n\n\u003cp\u003eWhen implementing AR models in practice, several key considerations ensure successful deployment. Data preprocessing often requires careful attention to stationarity testing and transformation.\u003c/p\u003e\n\n\u003cp\u003eModel validation requires time-aware cross-validation techniques that respect the temporal structure of the data. Traditional random sampling approaches can introduce data leakage, where future information inadvertently influences past predictions.\u003c/p\u003e\n\n\u003cp\u003eParameter selection involves balancing model complexity with predictive accuracy. Information criteria like AIC and BIC provide systematic approaches to order selection, while out-of-sample testing validates the chosen specification.\u003c/p\u003e\n\n\u003ch2 id=\"time-series-analysis-with-influxdb\"\u003eTime series analysis with InfluxDB\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB provides several critical advantages for time series autoregression workflows that extend beyond simple data storage. As a purpose-built time series database, InfluxDB addresses many challenges associated with managing and analyzing temporal data at scale.\u003c/p\u003e\n\n\u003ch4 id=\"optimized-storage-and-performance\"\u003eOptimized Storage and Performance\u003c/h4\u003e\n\n\u003cp\u003eInfluxDB’s columnar storage format and specialized compression algorithms reduce storage requirements for time series data. This efficiency becomes crucial when working with high-frequency data or maintaining long historical records necessary for robust AR model training.\u003c/p\u003e\n\n\u003ch4 id=\"real-time-data-processing\"\u003eReal-time Data Processing\u003c/h4\u003e\n\n\u003cp\u003eModern forecasting applications often require real-time model updates as new data arrives. InfluxDB’s streaming capabilities enable continuous data ingestion, allowing AR models to incorporate the latest observations immediately.\u003c/p\u003e\n\n\u003ch4 id=\"scalable-query-operations\"\u003eScalable Query Operations\u003c/h4\u003e\n\n\u003cp\u003eAs time series datasets grow, query performance becomes a limiting factor. InfluxDB’s indexing strategies and query optimization target temporal queries, enabling fast aggregations and data retrieval operations common in AR model preprocessing.\u003c/p\u003e\n\n\u003ch4 id=\"native-time-series-functions\"\u003eNative Time Series Functions\u003c/h4\u003e\n\n\u003cp\u003eInfluxDB includes built-in functions for common time series operations like moving averages and lag calculations. These functions can preprocess data directly within the database.\u003c/p\u003e\n\n\u003ch2 id=\"production-deployment-and-best-practices\"\u003eProduction deployment and best practices\u003c/h2\u003e\n\n\u003cp\u003eDeploying AR models in production environments requires attention to several operational aspects. Model monitoring becomes crucial as data patterns evolve over time, potentially degrading model performance. InfluxDB’s ability to store both input data and model predictions simplifies the creation of monitoring dashboards.\u003c/p\u003e\n\n\u003cp\u003ePerformance considerations include monitoring prediction accuracy over time and detecting concept drift.\u003c/p\u003e\n\n\u003ch2 id=\"capping-it-off\"\u003eCapping it off\u003c/h2\u003e\n\n\u003cp\u003eTime series autoregression provides a powerful and interpretable foundation for forecasting applications across diverse domains. The combination of statistical rigor, computational efficiency, and clear interpretability makes AR models an essential tool in the time series analyst’s toolkit.\u003c/p\u003e\n\n\u003cp\u003eWhile AR models have limitations in handling complex nonlinear patterns, their strengths in capturing temporal dependencies make them invaluable for both standalone applications and as components in more complex forecasting systems.\u003c/p\u003e\n\n\u003cp\u003eThe integration of AR modeling with modern time series infrastructure like \u003ca href=\"https://www.influxdata.com/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=time_series_autoregression\u0026amp;utm_content=blog\"\u003eInfluxDB\u003c/a\u003e creates opportunities for robust, scalable forecasting solutions. By leveraging InfluxDB’s specialized capabilities alongside the proven statistical foundations of autoregressive modeling, practitioners can build production-ready forecasting systems that deliver reliable predictions.\u003c/p\u003e\n","date_published":"2026-04-22T08:00:00+00:00","authors":[{"name":"Charles Mahler"}]},{"id":"https://www.influxdata.com/blog/litmus-and-influxdata-partnership","url":"https://www.influxdata.com/blog/litmus-and-influxdata-partnership","title":"From Edge to Enterprise: How Litmus and InfluxDB Are Modernizing the Industrial Data Stack","content_html":"\u003cp\u003eToday at Hannover Messe, InfluxData is announcing a strategic partnership with Litmus to address one of the most persistent challenges in industrial data: \u003cstrong\u003egetting reliable, contextualized telemetry from the shop floor into production systems\u003c/strong\u003e.\u003c/p\u003e\n\n\u003cp\u003eLitmus bridges the gap between OT systems and modern IT infrastructure, while InfluxDB serves as the industrial data hub, giving organizations both real-time operational visibility and enterprise-scale historical analysis in a unified architecture.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/ZK8Y3Nel8ihgcMLPyAleL/171b1f00ed9918d40f48afdab4c87199/Screenshot_2026-04-17_at_2.00.54â__PM.png\" alt=\"Influx + Litmus logo\" /\u003e\u003c/p\u003e\n\n\u003cp\u003eBy integrating \u003ca href=\"https://litmus.io/litmus-edge\"\u003eLitmus Edge\u003c/a\u003e with \u003ca href=\"https://www.influxdata.com/products/influxdb3-enterprise/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=litmus_and_influxdata_partnership\u0026amp;utm_content=blog\"\u003eInfluxDB 3 Enterprise\u003c/a\u003e, teams can collect and contextualize data at the source, then write it into a time series engine built for high-resolution data. Litmus handles connectivity and data normalization at the edge. InfluxDB provides high-throughput ingestion, real-time querying, and cost-efficient long-term storage, deployable at the edge, in the enterprise layer, or both.\u003c/p\u003e\n\n\u003cp\u003eThe result is a system that captures every signal, retains its context, and makes it immediately usable\u003c/p\u003e\n\n\u003ch2 id=\"the-industrial-data-problem\"\u003eThe industrial data problem\u003c/h2\u003e\n\n\u003cp\u003eSomething has shifted in industrial sectors. Modernization is no longer a roadmap item, but it’s starting to hit real constraints. The pull: industrial AI initiatives, predictive maintenance, cross-site analytics, digital twins, offer attractive value propositions. The push: legacy data historians are buckling under the demands of modern industrial operations, and the cost of extension is becoming harder to justify.\u003c/p\u003e\n\n\u003cp\u003eOT environments are notoriously fragmented. PLCs, CNCs, SCADA systems, and sensors operate across different protocols, vendors, and network boundaries. Getting that data into a usable, consistent format still requires heavy integration, time, and cost.\u003c/p\u003e\n\n\u003cp\u003eTraditional Historians made progress on the industrial data problem, but they weren’t built for what comes next. They struggle to preserve context across systems, degrade under high-frequency ingest and query load, and make cross-site analysis slow and expensive. This forces teams into trade-offs between fidelity, scale, and cost.\u003c/p\u003e\n\n\u003cp\u003e\u003cstrong\u003eThat’s the core issue: the value of industrial data is in its resolution and context. Most systems weren’t designed to retain either at scale.\u003c/strong\u003e\u003c/p\u003e\n\n\u003ch2 id=\"how-litmus-and-influxdb-work-together\"\u003eHow Litmus and InfluxDB work together\u003c/h2\u003e\n\n\u003cp\u003eTo move forward, teams need an architecture built for how industrial data actually behaves: high-frequency, distributed, and context-dependent. Litmus Edge and InfluxDB 3 Enterprise provide that foundation by collecting and structuring data at the edge, then making it available centrally without losing resolution or context.\u003c/p\u003e\n\n\u003cp\u003eHere’s how that looks in practice:\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/5OMDcrZFgEbU1ZBcZ8Uy8G/870217aff5fd191fde503594b80db336/Screenshot_2026-04-17_at_2.03.15â__PM.png\" alt=\"Litmus + IDB architecture\" /\u003e\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003cstrong\u003e250+ prebuilt industrial connectors\u003c/strong\u003e. Out-of-the-box connectivity to industrial data sources, including legacy systems and proprietary protocols. No custom integration required.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eCollect and contextualize at scale\u003c/strong\u003e. Normalize and contextualize telemetry from the source, with unlimited cardinality that preserves full context without compromising query performance.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eCentralized data, not silos\u003c/strong\u003e. Bring telemetry from tools, teams, and sites into a single architecture, from single-site monitoring to cross-plant analytics, without a data consolidation project.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eBuffered, store-and-forward data transfer\u003c/strong\u003e. Buffer and transmit data from remote sites with intermittent connectivity, with no loss or manual recovery.\u003c/li\u003e\n  \u003cli\u003e\u003cstrong\u003eRetain more, spend less\u003c/strong\u003e. Keeps high-resolution data accessible long-term with object storage, without driving up storage costs as you scale.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/7fPG6jqxIE4VktLXwV8SbR/4520cfd13bd2e3f1b503de0ef732f5ea/Screenshot_2026-04-17_at_2.04.58â__PM.png\" alt=\"Litmus quote 1\" /\u003e\u003c/p\u003e\n\n\u003ch2 id=\"the-edge-collect-contextualize-buffer\"\u003eThe edge: collect, contextualize, buffer\u003c/h2\u003e\n\n\u003cp\u003eLitmus Edge acts as the intelligence layer between your machines and the rest of your data architecture. With 250+ native connectors spanning OPC-UA, Modbus, MQTT, FANUC, Siemens S7, and more, it connects directly to industrial sources (PLCs, CNCs, DCS, SCADA systems, sensors, and beyond) without custom integration.\u003c/p\u003e\n\n\u003cp\u003eBut connectivity alone isn’t enough. Raw signals without context aren’t useful. Litmus Edge tags, enriches, and structures data at the point of collection so a temperature reading is tied to an asset, production line, facility, and product run. By the time it leaves the edge, it’s already queryable.\u003c/p\u003e\n\n\u003ch2 id=\"the-industrial-data-hub-centralize-scale-retain\"\u003eThe industrial data hub: Centralize, scale, retain\u003c/h2\u003e\n\n\u003cp\u003eInfluxDB 3 serves as the system of record for industrial time series data, whether deployed at the edge, centralized in the enterprise layer, or both.\u003c/p\u003e\n\n\u003cp\u003eAt the site level, InfluxDB runs locally alongside Litmus Edge, ingesting full-resolution telemetry and serving low-latency queries for real-time operations. It operates autonomously, so if connectivity to the central hub is interrupted, data is buffered locally and automatically forwarded when the connection is restored. There’s no data loss or manual intervention.\u003c/p\u003e\n\n\u003cp\u003eAt the enterprise level, a centralized InfluxDB cluster aggregates data from every site into a single query layer across assets, plants, and time horizons. This creates a consistent, high-resolution data layer that can be used across operations, analytics, and industrial AI.\u003c/p\u003e\n\n\u003cp\u003e\u003cimg src=\"//images.ctfassets.net/o7xu9whrs0u9/27iTqGpIQNfbNF1D1C9PUU/b6a34c5dc5099af641a34a9f803cf32f/Screenshot_2026-04-17_at_2.05.49â__PM.png\" alt=\"Litmus quote 2\" /\u003e\u003c/p\u003e\n\n\u003ch2 id=\"the-bridge-to-higher-level-analytics\"\u003eThe bridge to higher-level analytics\u003c/h2\u003e\n\n\u003cp\u003eWith high-resolution, contextualized data available across systems, teams can move beyond basic monitoring. Predictive maintenance, anomaly detection, and cross-site analytics all depend on full-fidelity data. Industrial AI at the edge depends on low-latency access to it. Without that foundation, these systems don’t operate reliably. That’s what this architecture enables.\u003c/p\u003e\n\n\u003ch2 id=\"get-started\"\u003eGet started\u003c/h2\u003e\n\n\u003cp\u003eWhether you’re starting a greenfield initiative or hitting the limits of your current industrial data infrastructure, we’d love to talk.\u003c/p\u003e\n\n\u003cp\u003eReach out to \u003ca href=\"https://www.influxdata.com/contact-sales/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=litmus_and_influxdata_partnership\u0026amp;utm_content=blog\"\u003econnect to an expert\u003c/a\u003e or join the conversation in the \u003ca href=\"https://community.influxdata.com/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=litmus_and_influxdata_partnership\u0026amp;utm_content=blog\"\u003eInfluxData Community Forums\u003c/a\u003e where our team and broader community are active.\u003c/p\u003e\n\n\u003cp\u003eIf you’re attending Hannover Messe, \u003ca href=\"https://www.influxdata.com/event/meet-influxdb-at-hannover-messe-2026/?utm_source=website\u0026amp;utm_medium=direct\u0026amp;utm_campaign=litmus_and_influxdata_partnership\u0026amp;utm_content=blog\"\u003ecome find me at the Litmus booth\u003c/a\u003e (Stand A09 in Hall 16) and see the architecture running end-to-end.\u003c/p\u003e\n","date_published":"2026-04-20T00:00:00+00:00","authors":[{"name":"Ben Corbett"}]},{"id":"https://www.influxdata.com/blog/mqtt-data-pipeline-influxdb","url":"https://www.influxdata.com/blog/mqtt-data-pipeline-influxdb","title":"Setting Up an MQTT Data Pipeline with InfluxDB","content_html":"\u003cp\u003eIn this blog, we’re going to take a look at how you can set up a fully-functioning, robust data pipeline to centralize your data into an InfluxDB instance by collecting and sending messages with the MQTT protocol. We’ll start with a brief overview of the technologies and protocols used in the pipeline, then dive into how you can connect, configure, and test them to ensure your data pipeline is fully functional. It’s going to be a long post, so let’s jump right in.\u003c/p\u003e\n\n\u003ch2 id=\"what-is-mqtt\"\u003eWhat is MQTT?\u003c/h2\u003e\n\n\u003cp\u003eMQTT is an industry-standard, lightweight protocol for moving messages through a network of devices. It functions by having a broker, or multiple brokers, receive messages from individual devices (publishing clients) across the network, and publish those messages to external systems (destination clients) that are connected and listening to the broker. By categorizing messages into “topics,” systems that subscribe to specific topics can opt to receive only messages they’re interested in.\u003c/p\u003e\n\n\u003cp\u003eAs a lightweight protocol with a number of prominent open source implementations, MQTT is an industry standard for a variety of use cases. It’s particularly common in Internet of Things (IoT) and Industrial IoT (IIoT) applications, but can be leveraged anywhere you have a distributed network of devices generating data or messages. This includes fleet management, home automation, real-time telemetry on computer hardware, and practically any use case where sensors generate data points periodically.\u003c/p\u003e\n\n\u003ch2 id=\"why-use-influxdb-for-mqtt-data\"\u003eWhy use InfluxDB for MQTT data?\u003c/h2\u003e\n\n\u003cp\u003eIf you’ve already concluded that the MQTT protocol is the right way to move your data from various devices into a centralized broker, odds are that you’re working with time series data. Time series data has a couple of key characteristics: it’s a sequence of data collected in chronological order, and all data points contain a timestamp. Most commonly, this also means there’s a large volume of data. Hundreds or thousands of sensors generating new data points every second can quickly turn into millions or billions of records per day. As the scale of data increases, the need for a specialized, purpose-built solution to handle this volume grows, too.\u003c/p\u003e\n\n\u003cp\u003eThat’s where InfluxDB, the industry-leading time series database, comes in. InfluxDB is purpose-built for the time series data common in MQTT use case scenarios, delivering unparalleled performance and a number of dedicated features to make managing and working with your time series data as easy as possible.\u003c/p\u003e\n\n\u003cp\u003ePerformance is critical because ingesting millions or billions of data points per day can strain most databases. Because time series databases like InfluxDB are optimized to handle that firehose of continuous data, they can scale to handle and ingest it with greater efficiency and lower costs. A custom-built storage engine eliminates snags that most other types of databases encounter, such as index maintenance and contention locks. Last-value caches and engine optimizations for timestamp-based filtering makes retrieving recent data extremely efficient, so fresh data being written into InfluxDB can be queried in less than 10 milliseconds, minimizing time to insight (or as we like to call it, “time to awesome”). This ensures a real-time view of the data generated across your network of devices.\u003c/p\u003e\n\n\u003cp\u003eTime series functionality also makes managing and working with this data much easier, regardless of if performance at scale is a concern. DataFusion, the SQL query engine embedded into InfluxDB 3, makes it easy to query with a language most data professionals and AI agents already know. With dedicated time-based functions, queries that look like this in a general purpose database:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eWITH hours AS (\n  SELECT generate_series(\n    date_trunc('hour', now() - interval '24 hours'),\n    date_trunc('hour', now()),\n    interval '1 hour'\n  ) AS hour_bucket\n),\nsensors AS (\n  SELECT DISTINCT sensor_id FROM sensor_data\n),\nhour_sensor AS (\n  SELECT h.hour_bucket, s.sensor_id\n  FROM hours h\n  CROSS JOIN sensors s\n),\nagg AS (\n  SELECT\n    sensor_id,\n    date_trunc('hour', time) AS hour_bucket,\n    percentile_cont(0.95) WITHIN GROUP (ORDER BY temperature) AS p95\n  FROM sensor_data\n  WHERE time \u0026gt;= now() - interval '24 hours'\n  GROUP BY sensor_id, hour_bucket\n)\nSELECT\n  hs.hour_bucket,\n  hs.sensor_id,\n  COALESCE(a.p95, 0) AS p95\nFROM hour_sensor hs\nLEFT JOIN agg a USING (hour_bucket, sensor_id)\nORDER BY hs.sensor_id, hs.hour_bucket;\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eCan be shortened to this in InfluxDB:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-sql\"\u003eSELECT\n  date_bin_gapfill(INTERVAL '1 hour', time) AS hour,\n  sensor_id,\n  interpolate(percentile(temperature, 95)) AS p95\nFROM sensor_data\nWHERE time \u0026gt;= NOW() - INTERVAL '24 hours'\nGROUP BY hour, sensor_id;\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eAdmittedly, this is a cherry-picked example for a complicated function most users won’t use every day, but there are plenty that aren’t. The InfluxDB 3 processing engine comes with a host of built-in plugins for processing and transforming data as it’s written, monitoring and anomaly detection, forecasting, and alerting. Retention policies can be set at a database or table level, ensuring you keep data as long as it’s useful, and the downsampling plugin for the processing engine can help you keep your data at a lower resolution once it’s past the end of that policy. InfluxDB also has tons of connections to the ecosystem of data visualization tools, clients, and, critical for the purposes of this tutorial, integrates seamlessly with Telegraf, the data collection agent we’ll be using to move data from our MQTT broker into InfluxDB.\u003c/p\u003e\n\n\u003ch2 id=\"the-mqtt---influxdb-pipeline\"\u003eThe MQTT -\u0026gt; InfluxDB pipeline\u003c/h2\u003e\n\n\u003cp\u003eThe architecture of this data pipeline is relatively straightforward, with data flowing in one direction throughout:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eDevices, sensors, and anything generating raw data are set up as an MQTT publishing client connected to the broker.\u003c/li\u003e\n  \u003cli\u003eThe MQTT broker receives the raw data from the various publishers and forwards it.\u003c/li\u003e\n  \u003cli\u003eTelegraf subscribes to the published topics and then writes data into InfluxDB.\u003c/li\u003e\n  \u003cli\u003eThe InfluxDB processing engine handles all necessary transformations and makes the data immediately available for querying and visualization.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eSo let’s jump into specifics.\u003c/p\u003e\n\n\u003ch4 id=\"setting-up-the-mqtt-broker-and-clients\"\u003eSetting Up the MQTT Broker and Clients\u003c/h4\u003e\n\n\u003cp\u003eThe first thing you’re going to need to do is install the MQTT technology of your choice on every device that’s going to be a publishing client, as well as on the server you want to act as your broker. Eclipse Mosquitto is a common open source option for MQTT that we’ll use in this guide, but any other MQTT client, such as HiveMQ, Paho, MQTTX, MQTT Explorer, or EasyMQTT, will also work great for this tutorial. The exact commands will differ depending on what you’re using, but the concepts will remain the same, as it’s a standardized protocol.\u003c/p\u003e\n\n\u003cp\u003eTo install Eclipse Mosquitto:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eOn Linux, run: \u003ccode class=\"language-markup\"\u003esnap install mosquitto\u003c/code\u003e\u003c/li\u003e\n  \u003cli\u003eOn Mac: Install \u003ca href=\"https://brew.sh/\"\u003eHomebrew\u003c/a\u003e, then run \u003ccode class=\"language-markup\"\u003ebrew install mosquitto\u003c/code\u003e\u003c/li\u003e\n  \u003cli\u003eOn Windows: Go to the \u003ca href=\"https://mosquitto.org/download/\"\u003emosquitto download page\u003c/a\u003e and install from there\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eWhen you install Mosquitto, the installer will then tell you the exact file path that the configuration file sits in. You’ll want to configure your broker first, and you should set up authentication if you don’t want to allow unauthenticated connections. A lack of authentication can be fine if you’re running everything on a local network where you’re not doing any port forwarding, but it’s not recommended if your devices are communicating over the internet.\u003c/p\u003e\n\n\u003cp\u003eThere are \u003cem\u003emany\u003c/em\u003e different ways to set up authentication with Mosquitto—one of the simplest is \u003ca href=\"https://mosquitto.org/man/mosquitto_passwd-1.html\"\u003ecreating a password file with the \u003ccode class=\"language-markup\"\u003emosquitto-passwd\u003c/code\u003e command\u003c/a\u003e, but you can read a full list of options on \u003ca href=\"https://mosquitto.org/documentation/authentication-methods/\"\u003etheir documentation page for authentication methods\u003c/a\u003e. Whatever you settle on, if you decide to use some form of authentication, you’ll need to add the following line to your Mosquitto configuration file.:\u003c/p\u003e\n\n\u003cp\u003e\u003ccode class=\"language-markup\"\u003eallow_anonymous false\u003c/code\u003e\u003c/p\u003e\n\n\u003cp\u003eThere are \u003ca href=\"https://mosquitto.org/man/mosquitto-conf-5.html\"\u003emany other configuration options in the documentation\u003c/a\u003e, and what you set and configure will depend on your use case, but some you may want to consider are:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003epersistence false\u003c/code\u003e - Because we’re writing to InfluxDB, we don’t need to persist messages to disk.\u003c/li\u003e\n  \u003cli\u003e\u003ccode class=\"language-markup\"\u003elog_dest stdout\u003c/code\u003e - For setting up, testing, and debugging, outputting logs directly to the terminal makes things easier.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eAnd of course, make sure your listener is configured on the same port for all devices. The default is 1883, but you can change this if desired.\u003c/p\u003e\n\n\u003cp\u003eOnce you configure your broker, you can set up your publishing clients, and with whatever data you’re measuring, they can publish messages to the broker with the command:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003emosquitto_pub -h \"host\" -t \"topic\" -m \"value\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eIf you’re running this all on a local network, your host will be \u003ccode class=\"language-markup\"\u003elocalhost\u003c/code\u003e; otherwise, it’ll be the address where your broker is running. The value should be whatever you’re measuring and publishing at that moment.\u003c/p\u003e\n\n\u003cp\u003eYour topic can be whatever is appropriate to label that value. If you have different devices and different types of measurements for each device, it’s recommended to nest your topics and organize them in a way that makes logical sense. For example, if you have many different devices measuring, say, temperature and velocity, your topic arrangement may look like:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003e/sensors/vehicles/v1/device1/temp\u003c/li\u003e\n  \u003cli\u003e/sensors/vehicles/v1/device1/velocity\u003c/li\u003e\n  \u003cli\u003e/sensors/vehicles/v1/device2/temp\u003c/li\u003e\n  \u003cli\u003e/sensors/vehicles/v1/device2/velocity\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eAs long as you have a unique topic structure for each type of value being sent, we can parse and sort this into tags and fields with InfluxDB. For further information on setting up MQTT topics, there are plenty of great \u003ca href=\"https://www.cedalo.com/blog/mqtt-topics-and-mqtt-wildcards-explained\"\u003eguides on the matter\u003c/a\u003e.\u003c/p\u003e\n\n\u003cp\u003eWith your clients and broker configured, your clients publishing messages, and your broker receiving and forwarding those messages, you should be all set up for the MQTT portion of this data pipeline.\u003c/p\u003e\n\n\u003ch2 id=\"installing-influxdb\"\u003eInstalling InfluxDB\u003c/h2\u003e\n\n\u003cp\u003eThe next step is to move your MQTT data into InfluxDB. The first step is to install InfluxDB. You can \u003ca href=\"https://docs.influxdata.com/influxdb3/core/install/\"\u003echeck out our docs on installing it here\u003c/a\u003e, but the simplest and easiest way to get started is to run the install scripts provided by InfluxData with:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003ecurl -O https://www.influxdata.com/d/install_influxdb3.sh \\\n\u0026amp;\u0026amp; sh install_influxdb3.sh\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThese should work on every operating system and provide you with some simple options to get started with InfluxDB 3 Core or Enterprise. The installation script should also give you an admin token, which you’ll want to store somewhere safe so you can use it for authentication. If you’d like to further configure your InfluxDB 3 instance, the installation script should tell you where all files and configuration files were installed for further adjusting, though it should run fine out of the box.\u003c/p\u003e\n\n\u003cp\u003eIf you have Docker installed, you can also install the InfluxDB Explorer UI as part of this process, giving you an easy way to view, manage, and query your InfluxDB 3 instance. You can reach it by navigating to \u003ccode class=\"language-markup\"\u003elocalhost:8888\u003c/code\u003e in your browser, entering \u003ccode class=\"language-markup\"\u003ehost.docker.internal:8181\u003c/code\u003e for the server address, and providing the admin token.\u003c/p\u003e\n\n\u003ch4 id=\"installing-and-configuring-telegraf\"\u003eInstalling and Configuring Telegraf\u003c/h4\u003e\n\n\u003cp\u003eWith InfluxDB 3 installed and running, the last step to get the data pipeline operational is to install and configure Telegraf to connect our MQTT broker to InfluxDB. Telegraf installation varies by operating system and Linux distribution, so check out the \u003ca href=\"https://docs.influxdata.com/telegraf/v1/install/#download-and-install-telegraf\"\u003eTelegraf documentation on installation to find the right files or command to run\u003c/a\u003e.\u003c/p\u003e\n\n\u003cp\u003eIf you’re on Mac or Linux, this will generate a default configuration file for you:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eOn Mac, install via Homebrew: \u003ccode class=\"language-markup\"\u003e/usr/local/etc/telegraf.conf\u003c/code\u003e\u003c/li\u003e\n  \u003cli\u003eOn Linux: \u003ccode class=\"language-markup\"\u003e/etc/telegraf/telegraf.conf\u003c/code\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eOtherwise, you’ll need to create an empty configuration file or generate one with \u003ccode class=\"language-markup\"\u003etelegraf config \u0026gt; telegraf.conf\u003c/code\u003e. Once you have located or created your configuration file, all that’s left to do is connect Telegraf to your MQTT Broker and InfluxDB.\u003c/p\u003e\n\n\u003cp\u003eInfluxDB is very easy to configure a connection to, and you can add these lines to the config file:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003e[[outputs.influxdb_v2]]\n  urls = [\"InfluxDB address \u0026amp; port\"]\n  token = \"admin token\"\n  organization = \"org name\"\n  bucket = \"destination database\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cul\u003e\n  \u003cli\u003eThe InfluxDB address and port should be wherever you have InfluxDB installed. If you’re running on a local network, this will be \u003ccode class=\"language-markup\"\u003ehttp://127.0.0.1:8181\u003c/code\u003e; otherwise, it’ll be the IP and port.\u003c/li\u003e\n  \u003cli\u003eToken is the admin token you copied from installation.\u003c/li\u003e\n  \u003cli\u003eOrganization can be whatever you’d like to name it.\u003c/li\u003e\n  \u003cli\u003eBucket should be the name of the database you’re writing all your MQTT data to. You don’t have to create the database first.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eSetting up a connection to your MQTT broker is also straightforward:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003e[[inputs.mqtt_consumer]]\n  servers = [\"broker address\"]\n  topics = [\"list of topics\"]\n  data_format = \"value\"\n  data_type = \"data_type\"\n\n  ## if you have username and password authentication for MQTT\n  username = \"username\"\n  password = \"password\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cul\u003e\n  \u003cli\u003eThe broker address is one again the address and port for where your MQTT broker is running. For a local network, this will be \u003ccode class=\"language-markup\"\u003etcp://127.0.0.1:1883\u003c/code\u003e\u003c/li\u003e\n  \u003cli\u003eTopics is a comma-separated list of topics that you’re writing to.\u003c/li\u003e\n  \u003cli\u003eData type is the primitive data type being written: integer, float, long, string, or boolean.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003cp\u003eThis is all you need in your configuration file to have the full pipeline running! If you run telegraf with \u003ccode class=\"language-markup\"\u003etelegraf --config telegraf.conf\u003c/code\u003e, you should be able to send a message from an MQTT publisher and view that data in InfluxDB.\u003c/p\u003e\n\n\u003cp\u003eHowever, you can make some improvements in Telegraf’s configuration to help parse and organize your data by topic. By default, this writes each topic into a single tag column to the same table, with a monolithic “value” column for all your values, which isn’t a very good data model. With topic parsing and pivot processing added to the configuration, we can specify what part of the topic should define what table the data is written into, turn every level of the topic into a tag, and pivot on the last level of the topic so that each raw value is its own field:\u003c/p\u003e\n\n\u003cpre class=\"\"\u003e\u003ccode class=\"language-bash\"\u003e[[inputs.mqtt_consumer]]\n  servers = [\"broker address\"]\n  topics = [\"/sensors/#\"]\n  data_format = \"value\"\n  data_type = \"data_type\"\n\n  ## if you have username and password authentication for MQTT\n  username = \"username\"\n  password = \"password\"\n\n  [[inputs.mqtt_consumer.topic_parsing]]\n    measurement = \"/measurement/_/_/_/_\"\n    tags = \"/_/device_type/version/device_name/field\"\n  [[processors.pivot]]\n    tag_key = \"field\"\n    value_key = \"value\"\u003c/code\u003e\u003c/pre\u003e\n\n\u003cp\u003eThis takes a value from the /sensors/vehicles/v1/device1/temp topic and writes it to the sensors table. The tag columns populate with \u003ccode class=\"language-markup\"\u003edevice_type = vehicles\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003eversion = v1\u003c/code\u003e, \u003ccode class=\"language-markup\"\u003edevice_name = device1\u003c/code\u003e, and temp is written as a field with the value of temp set to whatever your MQTT publisher wrote. You can modify this configuration as appropriate for your topics, and \u003ca href=\"https://docs.influxdata.com/telegraf/v1/input-plugins/mqtt_consumer/\"\u003ethe documentation provides full information on everything that can be done\u003c/a\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"further-improvements\"\u003eFurther improvements\u003c/h2\u003e\n\n\u003cp\u003eWith MQTT data being published, parsed, and written into InfluxDB, you’ve fully set up an MQTT data pipeline! However, there’s a lot more you can do:\u003c/p\u003e\n\n\u003cul\u003e\n  \u003cli\u003eView and query your data with the InfluxDB Explorer UI, as discussed earlier.\u003c/li\u003e\n  \u003cli\u003eConnect any one of the many \u003ca href=\"https://docs.influxdata.com/influxdb3/core/tags/client-libraries/\"\u003eclient libraries\u003c/a\u003e to access your data and use it for downstream applications, or to a data visualization tool for dashboarding and insight into what’s being written.\u003c/li\u003e\n  \u003cli\u003eUse the \u003ca href=\"https://docs.influxdata.com/influxdb3/core/plugins/\"\u003eInfluxDB 3 processing engine\u003c/a\u003e for further transformations and processing of your data as it’s written.\u003c/li\u003e\n  \u003cli\u003eSet up alerts, monitoring, forecasting, and more with the processing engine, too.\u003c/li\u003e\n\u003c/ul\u003e\n\n\u003ch2 id=\"the-final-product\"\u003eThe final product\u003c/h2\u003e\n\n\u003cp\u003eBy integrating MQTT, Telegraf, and InfluxDB, you’ve constructed a robust, fully-functioning data pipeline capable of efficiently centralizing real-time telemetry. The lightweight MQTT protocol ensures that messages from your distributed network flow reliably to the broker, while Telegraf acts as the collection agent for seamless ingestion and transformation. Finally, InfluxDB provides the purpose-built storage and specialized features needed to query and visualize your data in minimal time. This architecture establishes a solid foundation for turning raw event streams into meaningful insights, minimizing your time to awesome.\u003c/p\u003e\n","date_published":"2026-04-17T08:00:00+00:00","authors":[{"name":"Cole Bowden"}]}]}