InfluxData Customer Success Story: Robinhood
A pioneer of commission-free investing, Robinhood is on a mission to democratize finance for all and believes the financial system should be built to work for everyone. Robinhood is democratizing the financial systems by offering commission-free investing and trading with the use of your phone or desktop. As exciting as that sounds to the outside world, internally, the team at Robinhood had to understand the different risk vectors and build engineering solutions to mitigate these risks. To build a real-time risk monitoring system, Robinhood chose InfluxDB and Faust, an open-source Python stream processing library for Kafka streams.
To monitor and mitigate risk, Robinhood wanted to set up intelligent, real-time alerts on critical metrics without the need for constant manual dashboard tracking. Since staring at a screen to track thousands of time series 24/7 to take immediate action isn’t practical or scalable, Robinhood needed to build an anomaly detection system. The first anomaly detection solution that Robinhood tried was threshold-based alerting, yet that solution in the case of complex time series would require the same effort as checking the dashboard 24/7. Therefore, they needed an anomaly detection algorithm.
They sought to leverage historical data to determine — given an incoming data point — what a reasonable threshold would be, in order to define the anomaly state. So, they resorted to a concept derived from statistics called normal distribution. Defining your threshold from a standard deviation for anomaly detection is advantageous because it can help you detect anomalies on data that is non-stationary. They get a list of aggregated data points for every minute in the day for the past 30 days. They utilize the time series that they aggregate to compute the mean and the standard deviation over the past 30 days, and then use that as a boundary for a time series threshold.
Robinhood chose InfluxDB because it is lightweight, schemaless, allows indexing via a speciﬁc ﬁeld in the data, and provides fast data ingestion (writes) and aggregation (reads). Other InfluxDB attributes that led them to select it were that InfluxDB is available in an Enterprise edition that provides high availability (InfluxDB Enterprise); has a complete stack (encompassing data collection, analysis, and visualization); and has a large and active community. Using InfluxDB as its architecture’s time series database, Robinhood built an end-to-end anomaly detection system in a short period of time that runs well in production.
“InfluxDB has a very awesome stack, which gave us everything we needed when constructing the system.”
Allison Wang, Software Engineer, Robinhood