InfluxData Blog - Russ Savage

Release Announcement: InfluxDB CLI 2.2.0

Russ Savage (InfluxData) — Fri, 22 Oct 2021 11:43:45 -0700

A new release for InfluxDB Command Line Tools (CLI) is now available.

InfluxDB CLI 2.2.0 — release notes

This release includes the following new features and bug fixes.

Features

This release makes it easier to create API tokens with the influx CLI, adds support for viewing more than 20 buckets using influx bucket list, and adds a shorthand flag for bucket (-b) to influx delete.

Create an All-Access token in the Influx CLI

We added the ability to use the Influx CLI to create an All-Access API token with read and write permissions to all resources in an Organization.

View more buckets in the Influx CLI

We updated influx bucket list with pagination to support displaying more than 20 buckets. By default, buckets are fetched in batches of 20; set --page-size to override this default value. You may also limit the total number of buckets to display with --limit (by default, there’s no limit).

New bucket shorthand for influx delete

Add the shorthand flag -b for --bucket to influx delete.

Bug fixes

We now detect and warn when the Operator token is changed using influx restore (either setting a new --active config or updating the INFLUX_TOKEN variable).
The CLI now automatically sets the newly-created connection configuration as active in influx setup.
We have embedded the timezone data into Windows builds to avoid errors.

The latest open source release can be found on our downloads page.

If you find issues or have questions, please join our InfluxDB Community Slack, post them in our InfluxDB GitHub Repo or our Community Site and we will take a look.

Release Announcement: InfluxDB OSS 2.0.9

Russ Savage (InfluxData) — Mon, 11 Oct 2021 15:50:13 -0700

A new maintenance release for InfluxDB OSS is available now. Thank you to all the community members who have downloaded and used this latest release.

InfluxDB 2.0.9 — release notes

Upcoming changes to CLI packaging

Beginning with the next minor version (2.1.0), the InfluxDB CLI (influx command) will no longer be packaged in releases from the InfluxDB repository (influxdata/influxdb). Future versions of the InfluxDB CLI will instead be released from the influx-cli repository. Users who wish to adopt the new InfluxDB CLI today can download the latest release from the InfluxData Downloads Portal.

Breaking change for Mac OS Users

This release upgrades the project to go version 1.16. Because of the version bump to go, the macOS build for this release requires at least version 10.12 Sierra to run.

Features

New Resources API - Added a new route /api/v2/resources that returns a list of known resources to the platform. This makes it easier to automate token creation.
Flux Updates - Updated to Flux v0.130.0. Also added support for influxdb.cardinality() function. We also added --flux-log-enabled option to influxd to show detailed logs for Flux queries.
Performance Improvements - We optimized series iteration for queries that can be answered without inspecting TSM data and for queries with predicates that contain multiple measurements.

Fixes and improvements

This maintenance release of InfluxDB OSS includes the following fixes and improvements:

We now correctly filter requests to /api/v2/authorizations by organization and user parameters.
Updated to include curl as a dependency in influxdb2 packages.
Added a message to set the Flux content-type when a query fails to parse as JSON.
Discarded excessive errors over DefaultMaxSavedErrors (100) to prevent out-of-memory crashes.
Upgraded golang.org/x/sys to avoid panics on Macs.
Implemented hard limits on field size (MaxFieldValueLength = 1048576) while parsing line protocol.
Resolved the compaction queue stats flutter issue.
We now ensure the TSI index compacts log files that meet one of the following criteria:
- The log file hasn't been updated (no new series have been added to the shard) for 4 (or more) hours (to change this duration, specify a new storage-compact-full-write-cold-duration)
- The log file is one (or more) megabytes (to update this size, specify a new storage-max-index-log-file-size)
Repaired bad port dropping return value names.
We now use consistent path separator in permission string representation.
(Windows only) Added a copy for snapshot files being backed up.
Fixed an issue where updating an inactive task no longer schedules it. Thanks @raffs!
We now preserve comments in Flux queries when saving task definitions.
Fixed X-Influxdb-Build and X-Influxdb-Version response headers for the /ping endpoint.
Upgraded influxql to latest version and fixed predicate handling for SHOW TAG VALUES meta queries.

If you find issues or have questions, please join our InfluxDB Community Slack, post them in our InfluxDB GitHub Repo or our Community Site and we will take a look.

Release Announcement: InfluxDB CLI 2.1.1

Russ Savage (InfluxData) — Fri, 24 Sep 2021 11:27:52 -0700

A new release for InfluxDB Command Line Tools (CLI) is now available.

InfluxDB CLI 2.1.1 — release notes

Key highlights for this InfluxDB CLI release include:

Features

This release upgrades the project to Go version 1.17.

Bug fixes

Fixed the shell completion for top-level influx commands.
Added global --http-debug flag to the help text.
Resolved an issue where we were incorrectly setting empty strings for IDs in permission resources.
Added some detection and erroring out when we see incorrect positional args.
We now respect value of --host flag when writing CLI configs during setup.

The latest open source release can be found on our downloads page.

If you find issues or have questions, please join our InfluxDB Community Slack, post them in our InfluxDB GitHub Repo or our Community Site and we will take a look.

New Bucket Schema Option Can Protect You From Unwanted Schema Changes

Russ Savage (InfluxData) — Thu, 26 Aug 2021 02:00:34 -0700

One of the best things about getting started with InfluxDB over traditional relational databases is the fact that you don’t need to pre-define your schema in order to write data. This means you can create a bucket and write data in seconds, which can be pretty powerful to developers who care way more about the application they’re building than the mechanics of storing the data.

As your application grows, and more users are adding their own data, it can become tricky to ensure that the schema doesn’t get muddled up with junk. Some customers actually end up developing some sort of protection service to ensure only the right points make it to their production databases.

To help solve this problem for our users, we have introduced a new option to explicitly define the schema of the data that can be written to a bucket. The schema is specific to each measurement written to a bucket, which allows it to grow as new measurements are added.

In this post, I’m going to walk you through some of the concepts of the new explicit schema feature using one of my favorite datasets, the USGS Earthquake data. If you’re feeling antsy, feel free to jump directly to our documentation to learn all the details.

To follow along with this post, you’ll need a free InfluxDB Cloud account as well as the latest InfluxDB Cloud Command Line Tool (CLI) available on our Downloads page. You can quickly set up your CLI to work with cloud by following the instructions here — just make sure the version of the CLI is 2.1 or greater. You can check using explicit.

Creating a bucket with an explicit schema

In order to work with an explicit bucket schema, you need to make sure that you create a new bucket with the schema type set to explicit (the default for new buckets is to allow any schema, aka implicit). You can do this using the influx command line tool.

influx bucket create \
--name usgs \
--description "Only USGS data allowed in here" \
--retention 7d \
--schema-type explicit

ID			Name			Retention	Shard group duration	Organization ID		Schema Type
6f5781a53eed10f5	usgs	168h0m0s	n/a			03921cbebef90000	explicit

When you log into your InfluxDB Cloud account through your browser, you can see the new Bucket created with the Explicit Schema Type.

Adding measurement schemas to your bucket

Now that the bucket is created, we can start to add schemas for each of the measurements we want to store in that bucket. Any data that doesn’t match one of the measurement schemas will be rejected from the bucket.

Measurement schemas are just text files in either CSV, JSON, or NDJSON formats that tell InfluxDB what tags, fields, and types to allow in the measurement. Some example files are in the documentation. For this USGS dataset, you can find the schema files in our sample data repository. Here’s the one for the earthquake measurement in NDJSON format.

{"name": "time", "type": "timestamp"}
{"name": "alert", "type": "field", "dataType": "string"}
{"name": "cdi", "type": "field", "dataType": "float"}
{"name": "depth", "type": "field", "dataType": "float"}
{"name": "detail", "type": "field", "dataType": "string"}
{"name": "dmin", "type": "field", "dataType": "float"}
{"name": "felt", "type": "field", "dataType": "integer"}
{"name": "gap", "type": "field", "dataType": "float"}
{"name": "ids", "type": "field", "dataType": "string"}
{"name": "lat", "type": "field", "dataType": "float"}
{"name": "lon", "type": "field", "dataType": "float"}
{"name": "mag", "type": "field", "dataType": "float"}
{"name": "mmi", "type": "field", "dataType": "float"}
{"name": "nst", "type": "field", "dataType": "integer"}
{"name": "place", "type": "field", "dataType": "string"}
{"name": "rms", "type": "field", "dataType": "float"}
{"name": "sig", "type": "field", "dataType": "integer"}
{"name": "sources", "type": "field", "dataType": "string"}
{"name": "status", "type": "field", "dataType": "string"}
{"name": "tsunami", "type": "field", "dataType": "integer"}
{"name": "types", "type": "field", "dataType": "string"}
{"name": "url", "type": "field", "dataType": "string"}
{"name": "code", "type": "tag"}
{"name": "id", "type": "tag"}
{"name": "magType", "type": "tag"}
{"name": "net", "type": "tag"}
{"name": "title", "type": "tag"}

The format here is pretty straightforward. The three types are timestamp, field, and tag and the only type you will need to add the data type to are the fields, since tags are always strings.

To create these schema files, I’ve found that it’s useful to ingest the data into a temporary bucket with an implicit schema type, then use the schema package in Flux to inspect the tags (schema.measurementTagKeys) and field keys (schema.measurementFieldKeys) in the data.

If you aren’t sure about the data types for the fields, you can also run a simple Flux query to fetch that information.

import "influxdata/influxdb/schema"
from(bucket: "temp_bucket")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r._measurement == "earthquake")
  |> schema.fieldsAsCols()
  |> group()
  |> limit(n: 1)

You can see the column types in the raw data view of the Data Explorer, just under the grouping information.

Once you have your schema files ready to go, you can use the InfluxDB Cloud CLI tool to add that measurement schema to the bucket.

influx bucket-schema create --bucket usgs --name "earthquake" --columns-file ./schemas/earthquake-schema.ndjson --columns-format ndjson
ID			Measurement Name	Bucket ID
0808dd6fa216b000	earthquake		923f47af748a6453

For full documentation of the command, you can use the -h option or check out our documentation. You can repeat the process to add additional measurement schemas to the same bucket. For this example, there are five measurements for the USGS data, and once they’re all added, you can verify them using the list command. If you need the full output, you can add the --json flag to the end of the command as well, which is also useful for automation.

influx bucket-schema list -n usgs
ID			Measurement Name	Bucket ID
0808dd50522a1000	chemical explosion	923f47af748a6453
0808dd6fa216b000	earthquake		923f47af748a6453
0808dd84f0551000	explosion		923f47af748a6453
0808dd96d876c000	ice quake		923f47af748a6453
0808ddaf8156a000	quarry blast		923f47af748a6453

Writing data to an explicit schema bucket

Once the schemas are set up and ready to go, you can test that things are working by first trying to write some invalid line protocol to the bucket, to verify it is rejected.

influx write --bucket usgs "m,f=f1 v=1"
Error: failed to write data: 400 Bad Request: unable to parse 'm,f=f1 v=1': schema: measurement "m" not permitted by schema

This verifies that incorrect data will get rejected from the bucket, but let’s write some data as well to make sure our schemas are correct. Then we can query to verify that data was written.

influx write -b usgs -u https://raw.githubusercontent.com/influxdata/influxdb2-sample-data/master/usgs-earthquake-data/all_week.lp

influx query 'from(bucket: "usgs") |> range(start: -1d) |> count() |> group() |> sum()'
Result: _result
Table: keys: []
                _value:int
--------------------------
                      5076

You might get a different number of results, but anything greater than zero means things are working.

As you start to write more data to the same bucket, you can add additional measurement schemas or update existing ones, as long as you are only adding tags or fields. You can’t edit a measurement schema in such a way that it invalidates the existing data in the bucket.

Conclusion

Using explicit bucket schemas can help protect your bucket against unwanted writes. Once you go through the process of creating the schema files, you can manage the bucket schemas quickly and easily using the latest influx CLI tooling. These explicit schemas are also compatible with InfluxDB Templates, so you can also manage these as part of your GitOps workflows.

If you have any questions or comments about explicit schema buckets, please join us on our Community Slack workspace or you can file issues directly in GitHub.

Managing Secrets in the Browser in InfluxDB Cloud

Russ Savage (InfluxData) — Thu, 19 Aug 2021 02:00:43 -0700

Directly embedding passwords and API keys into the code you write is a bad practice. Of course, everyone knows this, but I’ll be the first to admit that it still happens now and then. In the world of source control and shared codebases, leaking a password can be a huge problem that costs your team time and money.

Of course, today many companies leverage a secrets management system to lower the probability of something like this happening.

InfluxDB Cloud ships with a built-in secrets management system powered by the industry-leading Vault secret store to provide every customer with a secure secret store. Users can store secrets, list their keys, and delete secrets using our fully documented API.

Now, that capability is available directly in the InfluxDB Cloud web interface so that you can quickly add or update secrets for your Flux scripts without having to work with the API.

Use cases for storing secrets

Users usually use InfluxDB Cloud secrets store for keeping track of sensitive connection information such as passwords, 3rd party API keys, and other security credentials when writing Flux scripts.

For example, suppose you need to bring data into InfluxDB Cloud that is stored in Snowflake. Flux has a sql.from function that allows you to connect to many popular SQL databases, including Snowflake. When doing so, you will need to provide credentials for that connection. Those credentials should be stored in the secrets store and fetched by the Flux script directly.

import "sql"
import "influxdata/influxdb/secrets"

username = secrets.get(key: "SNOWFLAKE_USER")
password = secrets.get(key: "SNOWFLAKE_PASS")
account = secrets.get(key: "SNOWFLAKE_ACCT")

sql.from(
  driverName: "snowflake",
  dataSourceName: "${username}:${password}@${account}/db/exampleschema?warehouse=wh",
  query: "SELECT * FROM example_table"
)

Examples for working with the Flux secrets package are available in our documentation.

Adding a new secret

To get to the secrets management page in your browser, simply log into your InfluxDB Cloud account and navigate to the setting page (the gearbox icon in the left navigation menu) and look for the “Secrets” tab. From there, adding a secret is as easy as clicking the “Add Secret” button.

Keep in mind that once a secret is created, it cannot be viewed or updated again. In order to change the value, you will need to delete and create a new secret. Any Flux scripts that were using that secret will need to be updated as well with the new secret key.

You can then leverage that secret in any Flux script or Task that runs on our InfluxDB Cloud platform.

Conclusion

One of the major goals of InfluxDB is to be secure by default, and we aim to give developers the right tools to ensure that the code they write and the applications they build on our platform can be secure as well.

As we move forward, we are looking for ways to make it easy to access these secret keys from anywhere you write Flux code in our platform.

If you have any questions or feedback, please join us on our Community Slack Workspace or on GitHub.

Release Announcement: InfluxDB OSS 2.0.8

Russ Savage (InfluxData) — Fri, 13 Aug 2021 14:19:37 -0700

A new maintenance release for InfluxDB OSS is available now. Thank you to all the community members who have downloaded and used this latest release.

InfluxDB 2.0.8 — release notes

Upcoming changes to CLI packaging

Breaking change for Mac OS Users

This release upgrades the project to go version 1.16. Because of the version bump to go, the macOS build for this release requires at least version 10.12 Sierra to run.

Features

Disable the Browser UI - Added --ui-disabled option to influxd to allow for running with the UI disabled.

Fixes and improvements

This maintenance release of InfluxDB OSS includes the following fixes and improvements:

Upgraded to latest version of influxdata/cron so that tasks can be created with interval of every: 1w.
We now return an error instead of panicking when InfluxQL statement rewrites fail.
Added migration for restored KV snapshots to latest schema before using them.
We now specify which fields are missing when rejecting an incomplete onboarding request.
systemd unit should block on startup until http endpoint is ready
Fixed display and parsing of influxd upgrade CLI prompts in Windows PowerShell.
Prevented silently dropped writes when there are overlapping shards.
Invalid requests to /api/v2 subroutes now return 404 instead of a list of links.
Flux metaqueries for _field take fast path if _measurement is the only predicate.
Copy names from mmapped memory before closing iterator.
Flux has been upgraded to v0.124.0.

If you find issues or have questions, please join our InfluxDB Community Slack, post them in our InfluxDB GitHub Repo or our Community Site and we will take a look.

Introducing Ranged Annotations in InfluxDB Cloud

Russ Savage (InfluxData) — Mon, 09 Aug 2021 04:00:42 -0700

Adding annotations to your data is a great way to share context with other members of your team. In May, we added the ability to annotate individual points in your data. Today, we have added the ability to add ranged annotations to your dashboard graphs.

We’ve also reworked some of the interactions with annotations based on user feedback so that they can be added quickly and easily. To learn more about working with annotations, check out our documentation.

Adding ranged annotations

Adding ranged annotations to your data is simple. To get started, open any of your InfluxDB Cloud dashboards and look for the Annotations button at the top.

As the informational popover says, to add an annotation to a graph, simply hold down the shift key and click where you’d like to place the annotation. A box should open that allows you to enter a message. If you’d like to add a ranged annotation, simply hold down shift, then click and drag across an area of a graph you’d like to annotate.

The annotations you add will be visible to anyone else in your organization when they have the annotations button turned on in their dashboard. Annotations can be added to any line graph.

Conclusion

Annotations are a quick and easy way to share additional contextual information about the data being shown in a graph. This can help a team triage and troubleshoot faster because the important insights, hints, and information is available on the dashboard itself.

We’d love to hear what you think about the new annotations feature of InfluxDB Cloud. Join us in our Community Slack workspace and let us know!

Use InfluxDB with GitHub Actions for GitOps, CI/CD, and Data Transformation

Russ Savage (InfluxData) — Thu, 05 Aug 2021 01:00:25 -0700

GitHub Actions are a powerful way to add automation to any source code repository. When you take that power and connect it with InfluxDB, you get an amazing combination that allows you to automate data generation, manage GitOps workflows, and a whole lot more.

This post will highlight some of the interesting ways to use InfluxDB and GitHub Actions.

Introducing the InfluxDB GitHub Action

I’ve been playing around with GitHub Actions a ton lately and noticed that the marketplace was missing one for working with InfluxDB. So I decided to remedy that and create a new InfluxDB Action that anyone can use. The process was very simple, and the documentation to create new actions is robust.

There are a few examples of how to set it up in the readme on the project. You can configure a specific version of InfluxDB, or use the keywords “latest” or “nightly” to always grab new versions (not recommended for production). The action will download InfluxDB and the InfluxDB CLI Tooling, unzip and move it into your path, then start and configure InfluxDB (unless you tell it not to). From there, you can run any influx command you would normally run.

So we have an InfluxDB instance up and running, but what can we do with it? Well, turns out, we can do a lot.

GitOps integrations

I’ve written previously about how to set up a GitOps workflow with InfluxDB, but with this GitHub action, it’s even easier. I’ve updated the version of the GitHub action configuration from that blog post to use the new InfluxDB Action.

For these GitOps flows, I don’t need a local instance of InfluxDB running, since all I’m doing is using the CLI tooling to deploy resources into InfluxDB Cloud, so I’ve set influxdb_start: false on line 28. It moves the influx command into the path so I can leverage it as I normally would.

Another way the InfluxDB Action can help in your GitOps workflows is through validating that the changes you’ve made to your InfluxDB Template files are valid. We do this internally with our Community Templates repository, where all changes to a Template are validated by loading a local InfluxDB instance and installing the template. If the template installs without issue, then it is assumed to be correct, and the test passes. Check it out here in our Community Templates repository.

Data validation and conversion

Recently, I’ve been playing around with GitHub’s Flat Data project and looking for ways to leverage it for projects and demos. In a nutshell, GitHub Flat Data allows you to quickly configure a GitHub action to fetch and commit a public data feed into your GitHub repository at a regular interval. The example they use is Bitcoin data.

So I updated the InfluxDB Data Examples repository and started loading in some public data streams that I had used in the past, namely, NOAA National Buoy Data Center data and USGS Earthquake data.

Next, I created a few post processing scripts to convert the data files into something that was usable for InfluxDB. For the buoy data, I enriched the data with the active station information and converted it into a CSV file, and for the Earthquake data, I converted it directly into line protocol.

Now that I had these data feeds regularly being updated and in a format that InfluxDB could read easily, I started playing around with using GitHub actions for doing some data conversions.

First, I grabbed the CSV file and used the InfluxDB command line interface (CLI) to parse and generate a line protocol version of the data. Here’s the command for converting that CSV data into line protocol.

/usr/local/bin/influx write dryrun -f $GITHUB_WORKSPACE/noaa-ndbc-data/latest-observations.csv --format csv --header "#constant measurement,ndbc" --header "#datatype double,double,double,double,double,double,long,double,double,double,double,double,double,double,tag,double,double,double,tag,tag,tag,tag,string,string,string,string,dateTime:number" > $GITHUB_WORKSPACE/noaa-ndbc-data/latest-observations.lp

I’m using the dryrun command here to spit out the line protocol data to a file. Check out our documentation for more information and examples about parsing CSV data with the InfluxDB Command Line Tooling.

The line protocol data is helpful, but I didn’t see any Flux capabilities for reading it directly, so I wanted to convert that to Flux’s Annotated CSV format so that I could leverage the csv.from(url: "" capability to pull the data into my InfluxDB Cloud account.

The quickest way to do that is to just ingest the data into the InfluxDB OSS instance running in the background and then query it back out again into its own file.

/usr/local/bin/influx write -f $GITHUB_WORKSPACE/noaa-ndbc-data/latest-observations.lp -b dummy
          /usr/local/bin/influx query "from(bucket: \"dummy\") |> range(start: -1y) |> drop(columns: [\"_start\",\"_stop\"])" --raw > $GITHUB_WORKSPACE/noaa-ndbc-data/latest-observations-annotated.csv

The range of 1y (one year) here is just a fake range since I know this data is all very recent.

Now, with this annotated CSV file, it’s really easy to pull that data into my InfluxDB Cloud account with the following query:

import "experimental/csv"
csv.from(url: "https://raw.githubusercontent.com/influxdata/influxdb2-sample-data/master/noaa-ndbc-data/latest-observations-annotated.csv")
  |> filter(fn: (r) => r._field == "dewpoint_temp_degc")

And visualize it with the Map graph type:

I did a similar thing for the USGS earthquake data.

This data is also now available in your InfluxDB Cloud instance using the new Sample Data library.

Conclusion

GitHub Actions are a powerful way to automate CI/CD pipelines for your projects, but it can also be used to automate data collection and transformation. We are using it internally in a variety of ways, and we’d love to hear how you’re using it.

Please join us on our Community Slack channel and let us know what you think!

Visualize Geotemporal Data with InfluxDB Cloud's New Map Graph

Russ Savage (InfluxData) — Mon, 02 Aug 2021 02:00:40 -0700

We recently introduced a new Map graph type into InfluxDB Cloud to help users visualize time series data that includes position.

Above is a graph showing the most recent earthquakes in California, where the color of the marker indicates their magnitude.

In this post, I’m going to walk through the ways to ingest geotemporal data into InfluxDB Cloud, and how to use the new Maps visualization type.

Before we continue, you’ll need to sign up for a free InfluxDB Cloud account if you haven’t already.

Ingesting geotemporal data

There are many different ways to ingest geotemporal data into InfluxDB Cloud, but there are recommended schema design considerations to think about.

If you are ingesting data with latitude and longitude information that changes over time, we recommend storing those values as fields within InfluxDB, so that it doesn’t continually increase your cardinality. This can also make it easier to track the same object moving over time, since you can group by the name of the object stored as defined by one or more tags such as sensor name, device ID, MAC address, etc.

Even if the latitude and longitude values never change, and have the same cardinality as your other tags like sensor name, we would still recommend you store the information as fields, so that you don’t need to worry about converting the values when doing calculations, since all tags are stored as strings.

If you do many queries where you need to group by the coordinates of the data, then we recommend you encode the latitude and longitude values using S2 Geometry. S2 takes the planet, and divides it into cells of customizable size (known as levels), such that you have a bounded set of values when working with the data. This way, the cell identifier (commonly tagged as s2_cell_id) can be added as a tag to the dataset without leading to runaway cardinality.

The Flux geo package and the new Maps graph type are designed to work with either latitude and longitude stored as fields, or the S2 cells stored as tags in your dataset. Adding the s2_cell_id as a single tag provides a performance boost when working with geotemporal data as all tag values are indexed. When ingesting your data, you can pre-calculate the S2 cell ids using the s2geo processor in Telegraf and store it with the dataset.

If you aren’t using Telegraf, the Flux geo package provides the ability to calculate the s2_cell_id on the fly, but the performance of the resulting query will be slightly slower as a result. If you’ve already been ingesting geotemporal data without this s2 cell information, don’t worry. You can also leverage Tasks to define a background process that would pre-calculate the s2 cells at different levels and store them with your dataset.

For playing with the Map type, we offer a few different sample datasets you can use. You can leverage the USGS Earthquake dataset (usgs) or the NOAA National Buoy Data Center dataset (noaa). You can access them using the new Sample data Flux package.

import "influxdata/influxdb/sample"

sample.data(set: "usgs")

Graphing the data with the maps

Once the data has been ingested into InfluxDB Cloud, you can begin working with that data when building out visualizations.

Simply select the series that you are interested in, ensure that you have selected either the latitude and longitude fields in your data, or the s2_cell_id tag, and select the Map visualization type from the graph options dropdown.

If all goes well, you should see a Map display with your points displayed. You might see an error if your fields are not named lat and lon or your tag isn’t named s2_cell_id. That’s ok. You can open the config options for the graph type and select the correct column or fields to use for the display.

Changing Map Marker colors

The Map graph type allows you to configure a value to use to change the colors of the markers on the map. In order for this to work, the data will need to use the s2_cell_id tag for the coordinates, so that you can use the specific field value to set the marker colors.

If your data already includes the s2_cell_id column, you can just use that. If you only have the latitude and longitude values as fields, you can leverage the shapeData function in the Geo package to calculate s2_cell_id you need. Finally, you can map the value column you would like to use in your threshold back to the _value column. The example below provides a sample query using this approach.

Once you have your data in the correct format, you can then configure threshold values to adjust the colors of the markers on the graph.

Here is an example of using the magnitude field of the earthquake to change the marker colors of the map.

Conclusion

We hope you enjoy this new ability to analyze your data using the Map graph type. Looking at the data geographically can unlock new insights or allow you to find patterns that you may have otherwise overlooked.

We are always looking for feedback on how you are using the capabilities of our product, or how we can improve. Please join us in our community Slack workspace or open feature requests directly in GitHub to let us know what you are interested in.