Choosing the right database is a critical choice when building any software application. All databases have different strengths and weaknesses when it comes to performance, so deciding which database has the most benefits and the most minor downsides for your specific use case and data model is an important decision. Below you will find an overview of the key concepts, architecture, features, use cases, and pricing models of M3 and StarRocks so you can quickly see how they compare against each other.

The primary purpose of this article is to compare how M3 and StarRocks perform for workloads involving time series data, not for all possible use cases. Time series data typically presents a unique challenge in terms of database performance. This is due to the high volume of data being written and the query patterns to access that data. This article doesn’t intend to make the case for which database is better; it simply provides an overview of each database so you can make an informed decision.

M3 vs StarRocks Breakdown


 
Database Model

Time series database

Data warehouse

Architecture

The M3 stack can be deployed on-premises or in the cloud, using containerization technologies like Kubernetes or as a managed service on platforms like AWS or GCP

StarRocks can be deployed on-premises, in the cloud, or in a hybrid environment, depending on your infrastructure preferences and requirements.

License

Apache 2.0

Apache 2.0

Use Cases

Monitoring, observability, IoT, Real-time analytics, large-scale metrics processing

Business intelligence, analytics, real-time data processing, large-scale data storage

Scalability

Horizontally scalable, designed for high availability and large-scale deployments

Horizontally scalable, with support for distributed storage and query processing

Looking for the most efficient way to get started?

Whether you are looking for cost savings, lower management overhead, or open source, InfluxDB can help.

M3 Overview

M3 is a distributed time series database written entirely in Go. It is designed to collect a high volume of monitoring time series data, distribute storage in a horizontally scalable manner, and efficiently leverage hardware resources. M3 was initially developed by Uber as a scalable remote storage backend for Prometheus and Graphite and later open-sourced for broader use.

StarRocks Overview

StarRocks is an open source high-performance analytical data warehouse that enables real-time, multi-dimensional, and highly concurrent data analysis. It features an MPP (Massively Parallel Processing) architecture and is equipped with a fully vectorized execution engine and a columnar storage engine that supports real-time updates.


M3 for Time Series Data

M3 is specifically designed for time-series data. It is a distributed and scalable time-series database optimized for handling large volumes of high-resolution data points, making it an ideal solution for storing, querying, and analyzing time-series data.

M3’s architecture focuses on providing fast and efficient querying capabilities, as well as high ingestion rates, which are essential for working with time-series data. Its horizontal scalability and high availability ensure that it can handle the demands of large-scale deployments and maintain performance as data volumes grow.

StarRocks for Time Series Data

StarRocks is primarily focused on data warehousing workloads but can be used for time series data. StarRocks can be used for real time analytics and historical data analysis.


M3 Key Concepts

  • Time Series Compression: M3 has the ability to compress time series data, resulting in significant memory and disk savings. It uses two compression algorithms, M3TSZ and protobuf encoding, to achieve efficient data compression.
  • Sharding: M3 uses virtual shards that are assigned to physical nodes. Timeseries keys are hashed to a fixed set of virtual shards, making horizontal scaling and node management seamless.
  • Consistency Levels: M3 provides variable consistency levels for read and write operations, as well as cluster connection operations. Write consistency levels include One (success of a single node), Majority (success of the majority of nodes), and All (success of all nodes). Read consistency level is One, which corresponds to reading from a single nod

StarRocks Key Concepts

  • MPP Architecture: StarRocks utilizes an MPP architecture, which enables parallel processing and distributed execution of queries, allowing for high-performance and scalability.
  • Vectorized Execution Engine: StarRocks employs a fully vectorized execution engine that leverages SIMD (Single Instruction, Multiple Data) instructions to process data in batches, resulting in optimized query performance.
  • Columnar Storage Engine: The columnar storage engine in StarRocks organizes data by column, which improves query performance by only accessing the necessary columns during query execution.
  • Cost-Based Optimizer (CBO): StarRocks includes a fully-customized cost-based optimizer that evaluates different query execution plans and selects the most efficient plan based on estimated costs.
  • Materialized View: StarRocks supports intelligent materialized views, which are precomputed summaries of data that accelerate query performance by providing faster access to aggregated data.


M3 Architecture

M3 is designed to be horizontally scalable and handle high data throughput. It uses fileset files as the primary unit of long-term storage, storing compressed streams of time series values. These files are flushed to disk after a block time window becomes unreachable. M3 has a commit log, equivalent to the commit log or write-ahead-log in other databases, which ensures data integrity. Client Peer streaming is responsible for fetching blocks from peers for bootstrapping purposes. M3 also implements caching policies to optimize efficient reads by determining which flushed blocks are kept in memory.

StarRocks Architecture

StarRock’s architecture includes a fully vectorized execution engine and a columnar storage engine for efficient data processing and storage. It also incorporates features like a cost-based optimizer and materialized views for optimized query performance. StarRocks supports real-time and batch data ingestion from a variety of sources and enables direct analysis of data stored in data lakes without data migration

Free Time-Series Database Guide

Get a comprehensive review of alternatives and critical requirements for selecting yours.

M3 Features

Commit Log

M3 uses a commit log to ensure data integrity, providing durability for write operations.

Peer Streaming

M3’s client peer streaming fetches data blocks from peers for bootstrapping purposes, optimizing data retrieval and distribution.

Caching Mechanisms

M3 implements various caching policies to efficiently manage memory usage, keeping frequently accessed data blocks in memory for faster reads.

StarRocks Features

Multi-Dimensional Analysis

StarRocks supports multi-dimensional analysis, enabling users to explore data from different dimensions and perspectives.

High Concurrency

StarRocks is designed to handle high levels of concurrency, allowing multiple users to execute queries simultaneously.

Materialized View

StarRocks supports materialized views, which provide precomputed summaries of data for faster query performance.


M3 Use Cases

Monitoring and Observability

M3 is particularly suitable for large-scale monitoring and observability tasks, as it can store and manage massive volumes of time-series data generated by infrastructure, applications, and microservices. Organizations can use M3 to analyze, visualize, and detect anomalies in the metrics collected from various sources, enabling them to identify potential issues and optimize their systems.

IoT and Sensor Data

M3 can be used to store and process the vast amounts of time-series data generated by IoT devices and sensors. By handling data from millions of devices and sensors, M3 can provide organizations with valuable insights into the performance, usage patterns, and potential issues of their connected devices. This information can be used for optimization, predictive maintenance, and improving the overall efficiency of IoT systems.

Financial Data Analysis

Financial organizations can use M3 to store and analyze time-series data related to stocks, bonds, commodities, and other financial instruments. By providing fast and efficient querying capabilities, M3 can help analysts and traders make more informed decisions based on historical trends, current market conditions, and potential future developments.

StarRocks Use Cases

Real-Time Analytics

StarRocks is well-suited for real-time analytics scenarios, where users need to analyze data as it arrives, enabling them to make timely and data-driven decisions.

Ad-Hoc Queries

With its high-performance and highly concurrent data analysis capabilities, StarRocks is ideal for ad-hoc querying, allowing users to explore and analyze data interactively.

Data Lake Analytics

StarRocks supports analyzing data directly from data lakes without the need for data migration. This makes it a valuable tool for organizations leveraging data lakes for storage and analysis.


M3 Pricing Model

M3 is an open source database and can be used freely, although you will have to account for the cost of managing your infrastructure and the hardware used to run M3. Chronosphere is the co-maintainer of M3 along with Uber and also offers a hosted observability that uses M3 as the backend storage layer.

StarRocks Pricing Model

StarRocks can be deployed on your own hardware using the open source project. There are also a number of commercial vendors offering managed services to run StarRocks in the cloud.