Suricata and CrateDB Integration

Input and output integration overview

This plugin reports internal performance counters of the Suricata IDS/IPS engine and processes the incoming data to fit Telegraf’s format.

The CrateDB plugin facilitates the writing of metrics to a CrateDB database, leveraging its PostgreSQL-compatible protocol to ensure a seamless experience for users.

Integration details

Suricata

The Suricata plugin captures and reports internal performance metrics from the Suricata IDS/IPS engine, which includes a wide range of statistics such as traffic volume, memory usage, uptime, and counters for flows and alerts. This plugin listens for JSON-formatted log outputs from Suricata, allowing it to parse and format the data for integration with Telegraf. It operates as a service input plugin, meaning it actively waits for metrics or events from Suricata rather than collecting metrics at predefined intervals. The plugin supports configurations for different metrics versions allowing for enhanced flexibility and detailed data gathering.

CrateDB

This plugin writes to CrateDB via its PostgreSQL protocol, allowing for metrics to be efficiently stored in a scalable database. CrateDB is designed for high-speed analytics, supporting time-series data and complicated queries, making it ideal for applications that require fast ingestion and analysis of large datasets. By utilizing the PostgreSQL protocol, the CrateDB output plugin ensures compatibility with existing PostgreSQL client libraries and tools, enabling a smooth integration for users who are already familiar with PostgreSQL’s ecosystem. The plugin provides options such as automatic table creation, connection parameters, and query timeouts, offering flexibility in how metrics are handled and stored within the database.

Configuration

Suricata

[[inputs.suricata]]
  ## Source
  ## Data sink for Suricata stats log. This is expected to be a filename of a
  ## unix socket to be created for listening.
  # source = "/var/run/suricata-stats.sock"

  ## Delimiter
  ## Used for flattening field keys, e.g. subitem "alert" of "detect" becomes
  ## "detect_alert" when delimiter is "_".
  # delimiter = "_"

  ## Metric version
  ## Version 1 only collects stats and optionally will look for alerts if
  ## the configuration setting alerts is set to true.
  ## Version 2 parses any event type message by default and produced metrics
  ## under a single metric name using a tag to differentiate between event
  ## types. The timestamp for the message is applied to the generated metric.
  ## Additional tags and fields are included as well.
  # version = "1"

  ## Alerts
  ## In metric version 1, only status is captured by default, alerts must be
  ## turned on with this configuration option. This option does not apply for
  ## metric version 2.
  # alerts = false

CrateDB

[[outputs.cratedb]]
  ## Connection parameters for accessing the database see
  ##   https://pkg.go.dev/github.com/jackc/pgx/v4#ParseConfig
  ## for available options
  url = "postgres://user:password@localhost/schema?sslmode=disable"

  ## Timeout for all CrateDB queries.
  # timeout = "5s"

  ## Name of the table to store metrics in.
  # table = "metrics"

  ## If true, and the metrics table does not exist, create it automatically.
  # table_create = false

  ## The character(s) to replace any '.' in an object key with
  # key_separator = "_"

Input and output integration examples

Suricata

1. Network Traffic Analysis: Utilize the Suricata plugin to track detailed metrics about network intrusion attempts and performance, aiding in real-time threat detection and response. By visualizing captured alerts and flow statistics, security teams can quickly pinpoint vulnerabilities and mitigate risks.

2. Performance Monitoring Dashboard: Create a dashboard using the Suricata Telegraf plugin metrics to monitor the health and performance of the IDS/IPS engine. This use case provides an overview of memory usage, captured packets, and alert statistics, allowing teams to maintain optimal operating conditions.

3. Automated Security Reporting: Leverage the plugin to generate regular reports on alert statistics and traffic patterns, helping security analysts to identify long-term trends and prepare strategic defense initiatives. Automated reports also ensure that the security posture of the network is continually assessed.

4. Real-time Alert Handling: Integrate Suricata’s alert metrics within a broader incident response automation framework. By incorporating the inputs from the Suricata plugin, organizations can develop smart triggers for alerting and automated response workflows that enhance reaction times to potential threats.

CrateDB

1. Real-Time Analytics for IoT Devices: Collect and store metrics from thousands of IoT devices. By setting up a dynamic metrics table for each device, users can perform real-time analytics on the collected data, enabling quick insights into device performance, patterns, and potential failures. This setup benefits from CrateDB’s ability to handle high-throughput data ingestion while providing the necessary analytics capabilities to derive actionable insights.

2. Website Performance Monitoring: Track key performance metrics from web applications, such as request latency and user activity. By storing metrics in CrateDB, teams can leverage the power of SQL-like queries to analyze traffic patterns, user engagement, and server performance over time, leading to optimized application performance and enhanced user experiences.

3. Financial Transaction Analysis: Manage large volumes of financial transaction data for real-time fraud detection and analysis. With CrateDB’s scalable infrastructure, users can store, query, and analyze transaction metrics efficiently, allowing for the detection of anomalies and illicit activities based on transaction patterns and trends.

Feedback

Thank you for being part of our community! If you have any general feedback or found any bugs on these pages, we welcome and encourage your input. Please submit your feedback in the InfluxDB community Slack.