New Plugins, Faster Writes, and Easier Configuration: What’s New with the InfluxDB 3 Processing Engine

By Ryan Nelson / Gary Fowler
Apr 07, 2026 Developer Product

The Processing Engine is one of the most powerful features in InfluxDB 3. It lets you run Python code at the database—transforming data on ingest, running scheduled jobs, or serving HTTP requests—without spinning up external services or building middleware. You define the logic, attach it to a trigger, and the database handles the rest.

Since launching the Processing Engine, we’ve been building out both the engine itself and the ecosystem of plugins that run on it. Today, we want to walk you through some exciting recent additions: new plugins for data ingestion, import, and validation; some general improvements to the engine; and a better configuration experience using InfluxDB 3 Explorer.

A quick refresher: Processing Engine plugins

If you’re already familiar with the Processing Engine, feel free to skip ahead. For those newer to the concept, here’s the short version.

A plugin is a Python script that runs inside InfluxDB 3 in response to a trigger. There are three trigger types: data writes (react to incoming data as it’s written), scheduled events (run on a timer or cron expression), and HTTP requests (expose a custom API endpoint). Plugins have direct access to the database: they can query and write without having to egress and ingress the data to a different machine or location. Plugins can also talk to other systems, letting you utilize data from other places or systems.

You can write your own plugins from scratch to solve problems specific to your environment. That’s the whole point of embedding Python in the database: your logic, your rules, running right next to your data.

But we also know that not everyone wants to start from a blank page. That’s why we maintain an official plugin library with production-ready plugins for common time series tasks, such as downsampling, anomaly detection, forecasting, state change monitoring, and sending notifications to Slack, email, or SMS.

These official plugins are designed to work in two ways. You can install them and use them as-is, configuring them through trigger arguments or TOML files to fit your setup. Or you can treat them as templates: fork one, customize the logic, and build something tailored to your exact workflow. Either way, they’re meant to get you moving faster.

One more thing worth mentioning: if you’re thinking about building a custom plugin but aren’t sure where to start, AI tools like Claude can be very effective. Point Claude to the Processing Engine documentation and the plugin library repo for examples, describe what you want your plugin to do, and let it generate a first draft. We’ve seen simple plugins created in a single shot, from description to working code, and even more complex plugins come together quickly when the AI has good examples to work from. It’s a great way to get past the blank-page problem and into something you can iterate on.

New plugins: data ingestion, import, and validation

We’ve recently added several new plugins to the library that address some of the most common requests we’ve been hearing from the community. These are available now in beta—they’re fully functional, but we want to see them tested across more environments before we call them production-ready. Give them a try and let us know how they work for you.

InfluxDB Import Plugin

If you’re running an older version of InfluxDB and want to bring your data into InfluxDB 3, the new Import Plugin makes that significantly easier. It supports importing from InfluxDB v1, v2, or v3 instances over HTTP, with features you’d expect from a serious import tool: automatic data sampling for optimal batch sizing, pause/resume for long-running imports, progress tracking, tag/field conflict detection and resolution, configurable time ranges and table filtering, and a dry run mode so you can preview what an import will look like before committing to it.

The plugin runs as an HTTP trigger, so you control the entire import lifecycle (start, pause, resume, cancel, check status) through simple HTTP requests. That means you can kick off a large import, pause it during peak hours, and pick it up later from exactly where it left off. For small or medium-sized InfluxDB databases, some might even use this as a migration tool to move to InfluxDB 3.

Data subscription plugins: MQTT, Kafka, and AMQP

These three plugins let new InfluxDB 3 users start getting data into InfluxDB 3 fast and without coding. They let you subscribe to external message brokers and begin automatically ingesting that data into InfluxDB 3.

The MQTT Subscriber Plugin connects to an MQTT broker, subscribes to topics you specify, and transforms incoming messages into time series data. It supports JSON, Line Protocol, and custom text formats with regex parsing, and uses persistent sessions to ensure reliable message delivery between executions.

The Kafka Subscriber Plugin does the same for Kafka topics. It uses consumer groups for reliable delivery, supports configurable offset commit policies (commit on success for data integrity, or commit always for maximum throughput), and handles JSON, Line Protocol, and text formats.

The AMQP Subscriber Plugin rounds out the trio with support for RabbitMQ and other AMQP-compatible brokers. Like the others, it supports multiple message formats, flexible acknowledgment policies, and comprehensive error tracking.

OPC UA Plugin

For industrial environments, the new OPC UA Plugin connects directly to PLCs, SCADA systems, and other OPC UA-enabled equipment. It polls node values on a schedule and writes them into InfluxDB 3 with automatic data type detection. You can list specific nodes for precise control, or use browse mode to auto-discover devices and variables across large deployments. The plugin maintains a persistent connection between polling intervals and supports quality filtering, namespace URI resolution, and TLS security.

Now, you might be thinking: “I’m already using Telegraf to interface with my streaming data services or OPC UA, why do I need these?” If Telegraf is working well for you, that’s great; there’s no need to change what isn’t broken. But if you’re newer to InfluxDB and aren’t yet a Telegraf user, these plugins give you another way to quickly get data flowing into InfluxDB 3 without adding another component to your stack.

All three plugins share a consistent configuration model: you can set them up with CLI arguments for simple cases or TOML configuration files for more complex mapping scenarios. They all include built-in error tracking (logging parse failures to dedicated exception tables) and write statistics so you can monitor ingestion health over time.

Schema Validator Plugin

One of the benefits of InfluxDB is that you don’t have to pre-define a schema. Data gets written as it is received. But for some use cases our customers have, they do want to constrain incoming data to conform to a specific schema.

The Schema Validator Plugin addresses that challenge, ensuring only clean, well-structured data makes it into your production tables. You define a JSON schema that specifies allowed measurements, required and optional tags and fields, data types, and allowed values. The plugin sits on a WAL flush trigger and validates every incoming row against your schema. Rows that pass get written to your target database or table; rows that fail get rejected (and optionally logged so you can see what’s being filtered out).

A typical pattern is to write raw data into a single database or table, let the validator check it, and have clean data land in a separate database or table. It’s a straightforward way to build a reliable data pipeline without external tooling.

Processing Engine general improvements

Alongside the new plugins, we’ve made several improvements to the Processing Engine itself that give plugin authors more control over write behavior, throughput, and concurrency.

Synchronous writes with durability control. New synchronous write functions let you choose between two modes: wait for the write to persist to the WAL before returning (for cases where you need to query the data immediately after writing), or return immediately for maximum throughput. This means you can treat bulk telemetry data as a fast path while ensuring that coordination states, such as job checkpoints or configuration flags, are immediately durable and queryable.

Batch writes. If your plugin writes thousands of points, the overhead isn’t in the data itself; it’s in the repeated write calls. The new batch write capability lets you group many records into a single write operation, which can dramatically improve throughput and make memory usage more predictable.

Asynchronous request handling. Request-based triggers now support concurrent execution. Previously, request handlers processed one request at a time, which meant a slow request would block everything behind it. With asynchronous mode enabled, the engine can handle multiple requests concurrently, so status checks, control commands, and other lightweight requests stay responsive even while a heavy operation is running.

These improvements work together in practice. The Import Plugin, for example, uses batch writes with fast-path durability for bulk data transfer, synchronous durable writes for checkpoints and state, and async request handling to keep its pause/resume/status endpoints responsive during long-running imports.

Easier plugin configuration in Explorer

We’ve also been improving InfluxDB 3 Explorer to make configuring plugins simpler, especially for the plugins in the library.

Until now, configuring a plugin meant passing all the right parameters as startup arguments to the Python script or specifying them in a TOML file. That works, but it requires you to know exactly which parameters a plugin expects—which means reading the documentation first.

We’re adding dedicated UI configuration forms for some of the plugins in Explorer. Instead of assembling a string of key-value pairs, you’ll see a form with all the available options laid out, along with descriptions and example values. Required fields are clearly marked, and the form handles the formatting for you. It’s the same configuration under the hood, just a much more approachable way to get there.

This is especially helpful for plugins with more involved configuration, like the data subscription plugins. where you’re specifying broker connections, authentication, message format mappings, and field type definitions. The form-based approach removes the guesswork and lets you get a plugin running without bouncing back and forth between the docs and your terminal. So far, we have built a specific configuration for the Import, Basic Transformation, and Downsampling plugins.

This is what it looks like for the Import plugin:

This is what the Basic Transformation and Downsample configuration looks like:

Look for these to become available in Explorer in the next couple of months.

What’s next

We are continuing to improve the Processing Engine and the Plugin Library. We have an OPC UA plugin about ready for you to try, as well as some additional anomaly detection and forecasting plugins. And, we are building UI configuration for the data subscription plugins mentioned above to make them even easier to configure.

Try them out

All new plugins are now available in beta in the InfluxDB 3 Plugin Library. They require InfluxDB 3 v3.8.2 or later. Install them from the CLI using the gh: prefix, or browse and install them directly from InfluxDB 3 Explorer’s Plugin Library.

We’re releasing these as a beta because we want your feedback. We’ve tested them thoroughly internally, but real-world environments are always more diverse and more demanding than any test suite. If you run into issues, have ideas for improvements, or build something cool on top of these plugins, we’d love to hear from you: drop into the InfluxData Discord, post on the Community Forums, or open an issue on GitHub.