Graphite vs MongoDB

Choosing the right database is a critical choice when building any software application. All databases have different strengths and weaknesses when it comes to performance, so deciding which database has the most benefits and the most minor downsides for your specific use case and data model is an important decision. Below you will find an overview of the key concepts, architecture, features, use cases, and pricing models of Graphite and MongoDB so you can quickly see how they compare against each other.

The primary purpose of this article is to compare how Graphite and MongoDB perform for workloads involving time series data, not for all possible use cases. Time series data typically presents a unique challenge in terms of database performance. This is due to the high volume of data being written and the query patterns to access that data. This article doesn’t intend to make the case for which database is better; it simply provides an overview of each database so you can make an informed decision.

Graphite Overview

Graphite is an open-source monitoring and graphing tool created in 2006 by Orbitz and open sourced in 2008. Graphite is designed for storing time series data and is widely used for collecting, storing, and visualizing metrics from various sources, such as application performance, system monitoring, and business analytics.

MongoDB Overview

MongoDB is a popular, open-source NoSQL database launched in 2009. Designed to handle large volumes of unstructured and semi-structured data, MongoDB offers a flexible, schema-less data model, horizontal scalability, and high performance. Its ease of use, JSON-based document storage, and support for a wide range of programming languages have contributed to its widespread adoption across various industries and applications.

Graphite for Time Series Data

Graphite is specifically designed and optimized for time series data. It uses the Whisper database format, which efficiently stores and manages time series data by automatically aggregating and expiring data based on user-defined retention policies. Graphite supports a wide range of functions for querying, transforming, and aggregating time series data, enabling users to create custom graphs and dashboards. However, as Graphite focuses exclusively on time series data, it may not be suitable for other types of data or use cases that require more advanced data modeling or querying capabilities.

MongoDB for Time Series Data

Although MongoDB is a general-purpose NoSQL database, it can be used for storing and processing time series data. The flexible data model of MongoDB allows for easy adaptation to the evolving structure of time series data, such as the addition of new metrics or the modification of existing ones. MongoDB provides built-in support for time-to-live (TTL) indexes, which automatically expire old data after a specified time period, making it suitable for managing large volumes of time series data with a limited storage capacity. MongoDB has also recently added a custom columnar storage engine and time series collection for time series use cases, meant to improve performance over the default MongoDB storage engine in terms of data compression and query performance.

Graphite Key Concepts

• Metric: A metric in Graphite represents a time series data point, consisting of a path (name), timestamp, and value.
• Series: A series is a collection of metrics that are all related to the same thing. For example, you might have a series for CPU usage, a series for memory usage, and a series for disk usage.
• Whisper: Whisper is a fixed-size, file-based time series database format used by Graphite. It automatically manages data retention and aggregation.
• Carbon: Carbon is the daemon responsible for receiving, caching, and storing metrics in Graphite. It listens for incoming metrics and writes them to Whisper files.
• Graphite-web: Graphite-web is the web application that provides a user interface for visualizing and querying the stored time series data.

MongoDB Key Concepts

Some key terminology and concepts specific to MongoDB include:

• Database: A MongoDB database is a container for collections, which are groups of related documents.
• Collection: A collection in MongoDB is analogous to a table in relational databases, holding a set of documents.
• Document: A document in MongoDB is a single record, stored in a JSON-like format called BSON (Binary JSON). Documents within a collection can have different structures.
• Field: A field is a key-value pair within a document, similar to an attribute or column in a relational database.
• Index: An index in MongoDB is a data structure that improves the query performance on specific fields within a collection.

Graphite Architecture

Graphite’s architecture consists of several components, including Carbon, Whisper, and Graphite-web. Carbon is responsible for receiving metrics from various sources, caching them in memory, and storing them in Whisper files. Whisper is a file-based time series database format that efficiently manages data retention and aggregation. Graphite-web is the web application that provides a user interface for querying and visualizing the stored time series data. Graphite can be deployed on a single server or distributed across multiple servers for improved performance and scalability.

MongoDB Architecture

MongoDB’s architecture is centered around its flexible, document-based data model. As a NoSQL database, MongoDB supports a schema-less structure, which allows for the storage and querying of diverse data types, such as nested arrays and documents. MongoDB can be deployed as a standalone server, a replica set, or a sharded cluster. Replica sets provide high availability through automatic failover and data redundancy, while sharded clusters enable horizontal scaling and load balancing by distributing data across multiple servers based on a shard key.