Edge Data Replication: Contributions and Status Updates for InfluxDB 3

By Anais Dotis-Georgiou / Developer
Jun 03, 2025

Navigate to:

If you’ve ever stood up multiple edge InfluxDB instances in remote locations and wished you could consolidate their data into a centralized instance for analysis, you’re not alone. That’s exactly why we designed Edge Data Replication (EDR) in InfluxDB v2. Now, with InfluxDB 3 Core and 3 Enterprise, we’re seeing new ways to handle replication using the brand-new Python Processing Engine.

In this post, I’ll walk you through:

  • What EDR looked like in InfluxDB v2 and why it matters for TSDBs
  • The new Python plugin engine for InfluxDB 3 Core and 3 Enterprise
  • A community-built EDR plugin ready for immediate use
  • Setup instructions and requirements
  • A sneak peek at official Support and how to find both official and community plugins

EDR in InfluxDB v2

In InfluxDB v2, Edge Data Replication gave you a built-in way to forward writes from an edge instance (like InfluxDB OSS) to a remote InfluxDB Cloud bucket. It handled durability, retries, compression, filtering, and optional aggregation. This made it easy to collect data from the edge and then stream it back to a centralized system for backup, analysis, or visualization.

In InfluxDB 3 Core and Enterprise, the architecture has changed. There’s no built-in EDR. Instead, we have a Python-based Processing Engine, which is even more flexible.

Introducing the Python Processing Engine for InfluxDB 3

InfluxDB 3 Core and Enterprise now support a Python-based plugin engine. It lets you run Python code inside the database, triggered by either:

  • a WAL flush (new data arriving)
  • a schedule (like a cron job)
  • or a manual request (an HTTP API call you define)

This means you can process data as it’s written, transform or downsample it, send alerts—or, yes, replicate it elsewhere. To activate it, you pass --plugin-dir when starting the server and drop your .py plugin files in that directory. We’ll walk through how to use a data replicator plugin later in this post.

A DevRel-contributed data replicator plugin

The official EDR plugin for InfluxDB 3 is expected to be released soon—it’s currently undergoing some final testing. In light of that, I wanted to share Developer Advocate @suyashcjoshi’s early version of a data replicator plugin, which is available today, and inspired development details of the official plugin.

You can find his InfluxDB 3 Custom Data Replication Plugin in our public plugin repository. It works with both InfluxDB 3 Core and 3 Enterprise and includes:

  • Replication to InfluxDB Cloud (Serverless or Dedicated) or InfluxDB 3 Enterprise
  • Table filtering
  • Optional downsampling via aggregate_interval
  • A compressed queue (edr_queue.jsonl.gz) for durability

Like all plugins, it works with any data source, whether you’re writing via Telegraf, a script, or your own app.

Requirements

Here’s what you’ll need to run this Plugin:

  • InfluxDB 3 Core or 3 Enterprise
  • A free-trail version of InfluxDB Cloud Serverless
  • InfluxDB 3 Python client: pip install influxdb3-python
  • Python 3.7+
  • Optional: Telegraf for simulating system metrics

How to use the plugin

Here’s a quick-start guide for testing this plugin locally and replicating the data to InfluxDB Cloud Serverless.

  1. Start InfluxDB with the Plugin Directory. 
    mkdir -p ~/.plugins
cp data-replicator.py ~/.plugins/

influxdb3 serve \

  --node-id host01 \
  --object-store file \
  --data-dir ~/.influxdb3 \
  --plugin-dir ~/.plugins

  2. Download and install the InfluxDB 3 Python Client.

    3. influxdb3 install package influxdb3-python

  3. (Optional) Run Telegraf. You can elect to run the Telegraf config that’s provided in the data replicator directory to quickly write system stats from your machine to your local InfluxDB 3 instance. You simply have to provide an authorization token. Run the following commands to generate a token and run the Telegraf config. 
    influxdb 3 create token 
telegraf --config telegraf.conf

  4. Create and enable the Replication Trigger. Here’s how you replicate all tables to a cloud instance: 
    influxdb3 create trigger \
  -d mydb \
  --plugin-filename data-replicator.py \
  --trigger-spec "all_tables" \
  --trigger-arguments "host=YOUR_HOST,token=YOUR_TOKEN,database=TARGET_DB" \
  data_replicator_trigger
    influxdb3 enable trigger --database mydb data_replicator_trigger

If you want to replicate only some tables or downsample the data to one-minute average values, you can add the following trigger arguments to the “create trigger” command above.

--trigger-arguments "tables=cpu,aggregate_interval=1m"

What’s next: official EDR and Downsampling Plugin status

We’re working on an officially-supported EDR Plugin that will:

  • Data Replication: Replicate data from a local InfluxDB 3 instance to a remote one.
  • Filtering: Specify which tables to replicate and which fields to exclude.
  • Renaming: Rename tables and fields during replication.
  • Downsampling: When enabled, downsample all data within the specified time window for scheduled triggers or for each individual run for data write triggers.
  • Queue Management: Use a compressed JSONL queue file for reliable delivery.
  • Retry Logic: Handle errors and rate limits with retry mechanisms.

We’re also developing a robust Downsampling Plugin that you can use alongside the EDR plugin that offers the following features:

  • Flexible Downsampling: Aggregate data over specified time intervals (e.g., seconds, minutes, hours, days, weeks, months, quarters, or years) using functions like avg, sum, min, max, or derivative.
  • Field and Tag Filtering: Select specific fields for aggregation and filter by tag values.
  • Scheduler and HTTP Support: Run periodically via InfluxDB triggers or on-demand via HTTP requests.
  • Retry Logic: Configurable retries for robust write operations
  • Batch Processing: Process large datasets in configurable time batches for HTTP requests.
  • Backfill Support: Downsample historical data within a specified time window.
  • Task ID Tracking: Each execution generates a unique task_id included in logs and error messages for traceability in late-arriving data.
  • Metadata Columns: Each downsampled record includes three additional columns to verify accurate downsampling and easily re-downsample data in the face of late-arriving data.

Final thoughts

Edge Data Replication isn’t just back; it’s more flexible than ever thanks to the Python Processing Engine. With InfluxDB 3 and the new Processing Engine, you can write plugins that transform, downsample, and replicate data in real-time, right from the edge. You’ll also be able to take the official plugins and easily adapt them to meet your own needs. I hope this post encourages you to contribute a plugin of your own. Check out our Getting Started Guide for Core and Enterprise, and share your feedback with our development team on Discord in the #influxdb3_core channel, Slack in the #influxdb3_core channel, or our Community Forums.