InfluxData Blog

The platform for building time series applications in less time with less code.

Daniël Heres

Daniël Heres is Apache Arrow PMC, and Software Engineer Query Engine at Coralogix. He was previously Data / ML Engineer at Godatadriven and Data / ML Engineer at bol.com.

Aggregating Millions of Groups Fast in Apache Arrow DataFusion

By Andrew Lamb, Raphael Taylor-Davies, Daniël Heres / Aug 01, 2023 / Developer, Apache Arrow

TLDR Grouped aggregations are a core part of any analytic tool, creating understandable summaries of huge data volumes. Apache Arrow DataFusion’s parallel aggregation capability is 2-3x faster in version 28.0.0 for queries with a large number (10,000 or more) of groups....

Try InfluxDB Cloud

The most powerful time series database as a service is a click away.

Try It Free