Arista LANZ and DuckDB Integration

Input and output integration overview

The Arista LANZ plugin is designed for reading latency and congestion metrics from Arista LANZ, helping users monitor their network performance effectively.

This plugin enables Telegraf to write structured metrics into DuckDB using SQLite-compatible SQL connections, supporting lightweight local analytics and offline metric analysis.

Integration details

Arista LANZ

This plugin provides a consumer for use with Arista Networks’ Latency Analyzer (LANZ). Metrics are read from a stream of data via TCP through port 50001 on the switches management IP. The data is in Protobuffers format, allowing for efficient transportation and parsing of data. LANZ is utilized to monitor network latency and congestion in real-time, which is vital for maintaining optimal performance in networking environments. The underlying technology, Arista’s latency analysis, provides insights into various network operations and infrastructure behaviors, making it a crucial tool for network engineering and management.

DuckDB

Use the Telegraf SQL plugin to write metrics into a local DuckDB database. DuckDB is an in-process OLAP database designed for efficient analytical queries on columnar data. Although it does not provide a traditional client-server interface, DuckDB can be accessed via SQLite-compatible drivers in embedded mode. This allows Telegraf to store time series metrics in DuckDB using SQL, enabling powerful analytics workflows using familiar SQL syntax, Jupyter notebooks, or integration with data science tools like Python and R. DuckDB’s columnar storage and vectorized execution make it ideal for compact and high-performance metric archives.

Configuration

Arista LANZ

[[inputs.lanz]]
  ## URL to Arista LANZ endpoint
  servers = [
    "tcp://switch1.int.example.com:50001",
    "tcp://switch2.int.example.com:50001",
  ]

DuckDB

[[outputs.sql]]
  ## Use the SQLite driver to connect to DuckDB via Go's database/sql
  driver = "sqlite3"

  ## DSN should point to the DuckDB database file
  dsn = "file:/var/lib/telegraf/metrics.duckdb"

  ## SQL INSERT statement with placeholders for metrics
  table_template = "INSERT INTO metrics (timestamp, name, value, tags) VALUES (?, ?, ?, ?)"

  ## Optional: manage connection pooling
  # max_open_connections = 1
  # max_idle_connections = 1
  # conn_max_lifetime = "0s"

  ## DuckDB does not require TLS or authentication by default

Input and output integration examples

Arista LANZ

1. Real-Time Latency Monitoring: This plugin can be used to set up a monitoring dashboard that tracks real-time latency metrics across multiple interfaces. By gathering and visualizing this data, network admins can swiftly identify and rectify latency issues before they impact service quality. The challenge lies in efficiently handling the influx of metrics from various sources without overwhelming the infrastructure or incurring excessive processing delays.

2. Congestion Analysis for Traffic Engineering: Users can leverage the LANZ plugin to analyze congestion records, enabling the optimization of network traffic flows. By applying historical pattern recognition to the metrics collected, IT teams can make informed decisions on traffic management strategies, thus improving overall network efficiency. This requires implementing robust data storage and analysis capabilities to derive actionable insights from the raw metrics.

3. Integration with Alerting Systems: Integrate the metrics from this plugin with alerting systems to automatically notify network engineers of any significant changes in latency or congestion. By setting thresholds based on historical data trends, this use case enhances proactive incident management, allowing teams to address potential issues proactively. The technical challenge here is establishing the right balance in threshold settings to minimize false positives while ensuring genuine issues are flagged promptly.

4. Network Optimization Reports: Utilize the metrics gathered through the LANZ plugin to generate periodic reports that detail network performance, latency trends, and congestion events. These reports can help stakeholders understand network health over time and guide infrastructure investments. The challenge involves structuring and formatting the output data to make it comprehensible and actionable for various audiences.

DuckDB

1. Embedded Metric Warehousing for Notebooks: Write metrics to a local DuckDB file from Telegraf and analyze them in Jupyter notebooks using Python or R. This workflow supports reproducible analytics, ideal for data science experiments or offline troubleshooting.

2. Batch Time-Series Processing on the Edge: Use Telegraf with DuckDB on edge devices to log metrics locally in SQL format. The compact storage and fast analytical capabilities of DuckDB make it ideal for batch processing and low-bandwidth environments.

3. Exploratory Querying of Historical Metrics: Accumulate system metrics over time in DuckDB and perform exploratory data analysis (EDA) using SQL joins, window functions, and aggregates. This enables insights that go beyond what typical time-series dashboards provide.

4. Self-Contained Metric Snapshots: Use DuckDB as a portable metrics archive by shipping .duckdb files between systems. Telegraf can collect and store data in this format, and analysts can later load and query it using the DuckDB CLI or integrations with tools like Tableau and Apache Arrow.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.