TL;DR InfluxDB Tech Tips - Using InfluxDB To Write and Export Data In Line Protocol
By Regan Kuchan / Nov 03, 2016 / Tech Tips
In this weekly post we recap the most interesting InfluxDB and TICK-stack related issues, using InfluxDB to write and export data in line protocol, workarounds, how-tos and Q&A from GitHub, IRC and the InfluxDB Google Group that you might have missed in the last week or so.
Write data from a file
Q: I have a lot of points in line protocol that I’d like to write to InfluxDB with the HTTP API. Is there a way to do this besides separating each point in the request body with a newline?
A: You can use the HTTP API to write data to InfluxDB from a file. For example:
$ curl -i -XPOST "http://localhost:8086/write?db=mydb" --data-binary @bridges.txt
A sample of the data in
bridges,type=cable_stayed visitors=607 1478133069000000000 bridges,type=cable_stayed visitors=608 1478133071000000000 bridges,type=suspension visitors=231 1478133069000000000 bridges,type=suspension visitors=234 1478133071000000000
Q: I’d like to export data from InfluxDB in line protocol format. How can I do this?
influx_inspect tool allows you to do just that. The example below exports data from the
mydb database and the
autogen retention policy to a file called
influx_inspect export -datadir "/var/lib/influxdb/data" -waldir "/var/lib/influxdb/wal" -out "export" -database mydb -retention autogen
A sample of the data in
# INFLUXDB EXPORT: 1677-09-21T00:12:43Z - 2262-04-11T23:47:16Z # DDL CREATE DATABASE mydb WITH NAME autogen # DML # CONTEXT-DATABASE:mydb # CONTEXT-RETENTION-POLICY:autogen # writing tsm data mymeas,color=blue value=800 1478129348953855972 mymeas,color=blue value=900 1478129351427777943 # writing wal data bridges,type=cable_stayed visitors=607 1478133069000000000 bridges,type=cable_stayed visitors=608 1478133071000000000 bridges,type=suspension visitors=231 1478133069000000000 bridges,type=suspenstion visitors=234 1478133071000000000
Note that the example uses syntax available in the 1.1 version of the
influx_inspect tool. The release candidate for version 1.1 is available on the downloads page.
Set a (nearly) infinite shard group duration
Q: I’m trying to create a retention policy with an infinite shard group duration. The
CREATE RETENTION POLICY query seems to succeed but the shard group duration is set to seven days, not
> CREATE RETENTION POLICY "ever_after" ON "final_frontier" DURATION INF REPLICATION 1 SHARD DURATION INF > SHOW RETENTION POLICIES ON "final_frontier" name duration shardGroupDuration replicaN default ---- -------- ------------------ -------- ------- autogen 0s 168h0m0s 1 true ever_after 0s 168h0m0s 1 false
A: Shard group durations cannot be infinite. As a workaround, we recommend specifying a
1000w shard group duration to achieve an extremely long shard group duration. For more on shard duration management, see the Schema Design documentation.
There’s an open issue on GitHub for InfluxDB to return an error if the
CREATE RETENTION POLICY query attempts to set the shard group duration to
For more InfluxDB tips, see our Frequently Asked Questions page and feel free to post your questions in the InfluxDB users group!
- Download and get started with InfluxDB!
- Schedule a FREE 20 minute consultation with a Solutions Architect to review your InfluxDB project.
- Attend one of our FREE virtual training seminars.
- Got a question and need an immediate answer from the InfluxData Support team? Support subscriptions with unlimited incidents start at just $399 a month. Check out all the support options here.