OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.
There are many open source projects which power OpenStack and core services namely Swift (object storage), Keystone (Identity), Nova (compute), Neutron (networking), Cinder (block storage) and Glance (image service). OpenStack has quickly evolved as a platform of choice for many enterprises building their private or hybrid clouds even with the dominance of public clouds.
There are many organizations that don’t want to lock themselves into a single vendor, regardless of how good or bad this vendor is. Or, there are still many cases in which controlling the infrastructure will give you a business advantage in controlling your product margins. That is, using specialized infrastructure that is more optimized for the kind of workload and customers that you are serving, than just general purpose infrastructure.
Having said that, the most important point is that OpenStack is not a product and shouldn’t be measured as such. It’s an ecosystem with strong foundations behind it.
When there is a common baseline of infrastructure, it’s much easier for an entire industry to plug into it and support it individually. This applies to all of the layers of the stack starting from storage, and network through more high level services such as big data services, and even analytics.
In this context, the big cost saving is that all the major infrastructure providers have been adding support for OpenStack, which drives the cost down for three reasons:
Even though there is wide adoption, maturity of OpenStack is often up for discussion as it is primarily being powered by open source projects. Key challenges in building OpenStack management solutions include:
Due to these challenges, a solution with full visibility and control into various layers of OpenStack is what every enterprise adopter wants to build. However, the ability to collect metrics data from many distributed components can be a challenge without a big ecosystem. Scaling the backend to “datacenter level” requires very high concurrency and storage in range of billions of data points.
InfluxDB has become synonymous with OpenStack deployments with both major platform and services vendors using it as integral part of their OpenStack solutions. This is because of targeted features like:
An OpenStack deployment means having to collect data from disparate systems, services and components. InfluxData’s Telegraf collector supports 30+ inputs and 10+ outputs and can be easily extended to support your sources of data. Telegraf makes collecting data in a format InfluxDB can consume, simple. Here’s why:
However, the InfluxData platform is extensible by design so you can easily integrate other collection agents like collectd, in conjunction with Telegraf.
Learn more about Telegraf
The most popular data in an OpenStack monioring system is going to be in a time-series format. InfluxDB is designed from the ground up to handle just time-series data and to do it better than any other database. InfluxDB is the “I” in the TICK stack. More specifically, InfluxDB is an open source database written in Go to handle time-series data with high availability and high performance requirements. InfluxDB installs in minutes without external dependencies, yet is flexible and scalable enough for complex deployments. Here’s why InfluxDB is the best choice for storing a custom monitoring solution’s time-series data:
Learn more about InfluxDB
If you don’t already have a dashboarding or graphing UI in place, InfluxData provides Chronograf. It’s the “C” in the TICK stack. Chronograf is a downloadable binary you install behind your firewall to collaboratively, yet securely, perform ad-hoc visualizations on your time-series data. Features include:
Another visualization UI choice that offers tight integration with InfluxDB is the open source Grafana project. Either choice makes connecting to and visualizing time-series data, simple.
Inevitably, you are going to want to either alert on or in some way process the time-series data being sent by the components in your OpenStack deployment. You’ll want to do this either before it gets written to InfluxDB or when it is retrieved. To address this need, the InfluxData platform ships with the open source Kapacitor project. Kapacitor is the “K” in the the TICK stack. It’s an alerting and data processing engine specifically designed for time-series data. It lets you define your own custom pipeline to aggregate, select, transform or otherwise process data and then store it back in InfluxDB or trigger an event. Features include:
Learn more about Kapacitor