Using the New Flux Usage API to Calculate Pricing for InfluxDB Cloud

By John Corrigan / Jul 16, 2021 / Community, InfluxDB Cloud, Developer, Flux

InfluxDB Cloud offers a transparent usage-based pricing model that only charges users on the work performed, with no minimums or long-term commitments. This puts YOU in charge of what you spend. However, with four separate pricing vectors, it’s not always easy to see exactly where that cost is going, or how to estimate your potential spend based on your data usage.

How does the Usage-Based Plan pricing work for InfluxDB Cloud?

Pricing is based on your usage of InfluxDB Cloud. Your monthly usage is calculated based on the following four vectors:

• Writes: The total amount of uncompressed data ingested (measured in MB) in InfluxDB Cloud via line protocol, the API, the client libraries, or Telegraf.
• Query Count: The total count of individual query executions as generated by Dashboards, Tasks, Alerts, Data Explorer, etc. measured in queries per month.
• Storage (GB per hour): Storage, measured in GB per hour, consumed by the data retained in your InfluxDB Cloud account. For example, 10 GB stored for 1 hour, and 1 GB stored for 10 hours, are both counted as 10 GB-hours.
• Data Out: the total volume of data returned to the user in response to queries (measured in GB per month).

You may already be familiar with tracking your data usage on the Usage page, where the running totals for the four billing metrics are shown for the current billing period, as well as controls for displaying each individual metric for one of the three pre-set time ranges (past 24h, past 7d, past 30d). In order to turn these usage numbers into pricing data, you would need to compare your usage data to the price per individual data metric (shown in the table above). This approach is covered in detail in this Cloud Pricing article.

<figcaption> InfluxDB Cloud Usage page</figcaption>

This approach has some obvious shortcomings:

• Manual calculations? Ain't no one got time for that!
• You're limited to the current billing period.
• You can't see the cost related to specific events.

Get detailed information using the Flux Usage API

To get more detailed information on your data usage from an InfluxDB Cloud instance, you can use the experimental/usage Flux function which maps to the usage endpoint in the InfluxDB Cloud API. The experimental/usage package contains two functions: usage.from() and usage.limits().

The usage.from() function returns usage data for your InfluxDB Cloud Organization. By default, the function returns aggregated usage data as an hourly sum. This default downsampling enables you to query usage data over long periods of time efficiently. If you want to view raw high-resolution data, query your data over a short time range (up to one hour) and set raw:true.

To demonstrate how you can utilize the data usage metrics as well as the limit events data, we’ve created the InfluxDB Cloud Usage Dashboard community template. By delivering this as a dashboard, we’ve given you the ability to customize the information to fit your needs. [You can read more about the full template in this article.]

Add pricing calculations to your Usage Dashboard in InfluxDB Cloud

Adding pricing details to your dashboard is a relatively simple process, and can provide some insight into how your costs are divided between the different data metrics. The Usage Dashboard already provides you with individual totals for each metric, so it’s simply a matter of multiplying that value by the individual pricing vector for that data metric. Let’s walk through the steps for adding a ‘Est. Cost’ cell for the ‘Data in’ metric:

• Click the gear icon for the "Data In (total)" cell and select the Clone option.
• On the newly cloned cell, click the gear icon, and choose Configure.
• With the query builder now open, locate the map() function which converts the data into the actual metric value (converting into 'MB' in this case):

|> map(fn: (r) => ({r with write_mb: math.round(x: float(v: r._value) / 10000.0) / 100.0 }))

• Simply add the pricing value for the data metric to the calculation (for 'Data In' this would be 0.002 per MB):

|> map(fn: (r) => ({r with write_mb: math.round(x: float(v: r._value) / 10000.0) / 100.0 * 0.002 }))

• You can now Customize the cell to display as a dollar value in whatever color you like.

<figcaption> This is what the modified cell might look like</figcaption>

Below is the complete query for the Data In pricing metric calculation:

import "csv"
import "math"
import "date"
import "strings"
import "experimental/json"
import "experimental/usage"
 
usage.from(
start: v.timeRangeStart,
stop: v.timeRangeStop,
)
|> filter(fn: (r) =>
r._measurement == "http_request"
and (r.endpoint == "/api/v2/write" or r.endpoint == "/write")
and r._field == "req_bytes"
)
|> group()
|> keep(columns: ["_value", "_field", "_time"])
|> fill(column: "_value", value: 0)
|> map(fn: (r) =>
({r with
write_mb: math.round(x: float(v: r._value) / 10000.0) / 100.0 * 0.002
}))
|> sort(columns: ["_time"])
|> group()
|> sum(column: "write_mb")

You can follow these steps for each of the four data metric totals to create individual pricing amounts.

Create a cell with the total price

In order to create a ‘Total’ sum of all four data metrics, you can create a new cell, copy the functions for each data metric into four separate variables and do a union() to combine the values and sum.

import "csv"
import "math"
import "date"
import "strings"
import "experimental/json"
import "experimental/usage"
 
cost_writes = usage.from(
start: v.timeRangeStart,
stop: v.timeRangeStop,
)
|> filter(fn: (r) =>
r._measurement == "http_request"
and (r.endpoint == "/api/v2/write" or r.endpoint == "/write")
and r._field == "req_bytes"
)
|> group()
|> keep(columns: ["_value", "_field", "_time"])
|> fill(column: "_value", value: 0)
|> sum(column: "_value")
|> map(fn: (r) => ({ r with _value: float(v: r._value)*0.002/1000.0/1000.0}))
 
 
cost_queries = usage.from(
start: v.timeRangeStart,
stop: v.timeRangeStop,
)
 
|> filter(fn: (r) =>
r._measurement == "query_count"
)
|> group(columns: ["_time"])
|> sum()
|> group()
|> sort(columns: ["_time"])
|> group()
|> sum(column: "_value")
|> map(fn: (r) => ({ r with _value: float(v: r._value)/10000.0}))
 
cost_storage = usage.from(
start: v.timeRangeStart,
stop: v.timeRangeStop,
)
|> filter(fn: (r) => r._measurement == "storage_usage_bucket_bytes" and r._field == "gauge")
|> range(start: 2021-03-01T00:00:00.000Z)
|> aggregateWindow(every: 1h, fn: mean)
|> fill(column: "_value", value: 0.0)
|> group()
|> sum()
|> map(fn: (r) =>
({r with
_value: math.round(x: float(v: r._value) * 0.002 / 10000000.0) / 100.0
})
)
 
cost_reads = usage.from(
start: v.timeRangeStart,
stop: v.timeRangeStop,
)
|> filter(fn: (r) =>
r._measurement == "http_request"
and (r.endpoint == "/api/v2/write" or r.endpoint == "/read")
and r._field == "req_bytes"
)
|> group()
|> keep(columns: ["_value", "_field", "_time"])
|> fill(column: "_value", value: 0)
|> sum(column: "_value")
|> map(fn: (r) => ({ r with _value: float(v: r._value)*0.09/1000.0/1000.0/1000.0}))
 
 
union (tables: [cost_writes, cost_reads, cost_queries, cost_storage])
|> sum()

<figcaption> Usage Dashboard with estimated costs</figcaption>

Adding cost to hourly metrics

If you’re looking for more finely detailed analysis of your costs, you could even add pricing details to the hourly metric calculations. Wondering what that sudden spike in Query Count might have actually cost? Add an additional calculation to your graph data to compare items side-by-side.

import "experimental/usage"
 
count = usage.from(
start: v.timeRangeStart,
stop: v.timeRangeStop,
)
|> range(start: v.timeRangeStart)
|> filter(fn: (r) =>
r._measurement == "query_count"
)
|> keep(columns: ["_value", "_field", "_time"])
|> sort(columns: ["_time"], desc: false)
|> aggregateWindow(every: 1h, fn: sum)
|> toFloat()
 
count
|> yield(name: "query count")
 
 
count
|> map(fn: (r) => ({ r with _value: float(v: r._value)/10000.0}))
|> yield(name: "cost - cents")

Selecting your time period

Once you’ve added the pricing metrics you wish to track, you can select the time period you want to see data for, as with any dashboard. If you’re interested in viewing the information for your current month-to-date billing period, you can simply set the time range of the dashboard to a Custom Time Range and select the first day of the current month as your starting date, and the current day as your stopping point.

Note: The current usage.from() API is limited to returning data from the last 30-day time period.

<figcaption> Setting custom time range for your dashboard</figcaption>

Final thoughts on calculating pricing with the Flux Usage API and the InfluxDB Cloud Usage Template

I hope this post inspires you to take advantage of the InfluxDB Cloud Usage Template. If you are using a community template or Flux and need help, please ask for some in our community site or Slack channel. If you’re developing a cool IoT application on top of InfluxDB, we’d love to hear about it, so make sure to share your story! Additionally, please share your thoughts, concerns or questions in the comments section. We’d love to get your feedback and help you with any problems you run into!