InfluxData Blog - Ignacio Van Droogenbroeck

How to Monitor Kubernetes K3s Using Telegraf and InfluxDB Cloud

Ignacio Van Droogenbroeck (InfluxData) — Thu, 08 Dec 2022 07:00:00 +0000

This article was originally published in The New Stack and is reposted here with permission.

A Helm chart can simplify our lives and enable us to see what is happening with our K3s cluster using an external system.

Lightweight Kubernetes, known as K3s, is an installation of Kubernetes half the size in terms of memory footprint.

Do you need to monitor your nodes running K3s to know the status of your cluster? Do you also need to know how your pods perform, the resources they consume, as well as network traffic? In this article, I will show you how to monitor K3s with Telegraf and InfluxDB Cloud.

I run a blog and a few other resources on Kubernetes. Specifically, these run in a cluster of three nodes in DigitalOcean that run K3s, and I use Telegraf and InfluxDB to monitor everything.

I’m going to demonstrate how to monitor the cluster to make sure that everything is running as expected and how to identify something if it is not.

To monitor the cluster I use two components:

InfluxDB Cloud: It’s ideal to do monitoring from the outside because if we do it from the inside and the node goes down, then so does the monitoring solution, and that doesn’t make any sense. You can get a free InfluxDB account here: https://cloud2.influxdata.com/signup/

Next, we need to install a Helm chart from Telegraf, specifically this one, because it does not have Docker engine support, which if you run K3s, doesn’t need it.

Let’s do it…

Configuring InfluxDB Cloud

The first thing we must do is create an account in InfluxDB Cloud. Next, we go to the Data section, click on Buckets, and then on Create Bucket.

Name the bucket and click on Create.

This is what our list of buckets should look like. After successfully creating the bucket, we create an access token to be able to write data to that bucket. To do that we go to the Tokens tab.

In this section, we click on Generate Token and choose the Read/Write Token option.

We specify a name, choose the bucket we want to associate with this token, and click on Save.

Once this is done, the new token appears in the token list.

To finish this part, we are going to need our Org ID and the URL to point our Telegraf to.

The Org ID is the email you used to sign up for InfluxDB Cloud. I get the URL from the address bar. In my case, when I set up my InfluxDB Cloud account, I chose the western United States. So my URL looks like this:

https://us-west-2-1.aws.cloud2.influxdata.com

Now that we configured InfluxDB Cloud, we need to configure the nodes.

As I mentioned above, we are going to use a Helm Chart. I modified this Helm Chart to adapt to K3s, because by default it tries to monitor Docker, which isn’t used in this Kubernetes distribution.

If you don’t have Helm installed, you can install it by running this command:

$ curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Once installed, download the values.yaml file here.

You can also grab the raw file and download it directly to the master node with a wget.

$ wget https://raw.githubusercontent.com/xe-nvdk/awesome-helm-charts/main/telegraf-ds-k3s/values.yaml

Now, we have to modify this file a bit. We need to open it and modify the Output section. By default the file looks like this:

## Exposed telegraf configuration
## ref: https://docs.influxdata.com/telegraf/v1.13/administration/configuration/
config:
  # global_tags:
  #   cluster: "mycluster"
  agent:
    interval: "10s"
    round_interval: true
    metric_batch_size: 1000
    metric_buffer_limit: 10000
    collection_jitter: "0s"
    flush_interval: "10s"
    flush_jitter: "0s"
    precision: ""
    debug: false
    quiet: false
    logfile: ""
    hostname: "$HOSTNAME"
    omit_hostname: false
  outputs:
    - influxdb:
        urls:
          - "http://influxdb.monitoring.svc:8086"
        database: "telegraf"
        retention_policy: ""
        timeout: "5s"
        username: ""
        password: ""
        user_agent: "telegraf"
        insecure_skip_verify: false
  monitor_self: false

But since we are going to use InfluxDB Cloud, we must make some adjustments. The modified version will look something like this:

## Exposed telegraf configuration
## ref: https://docs.influxdata.com/telegraf/v1.13/administration/configuration/
config:
  # global_tags:
  #   cluster: "mycluster"
  agent:
    interval: "1m"
    round_interval: true
    metric_batch_size: 1000
    metric_buffer_limit: 10000
    collection_jitter: "0s"
    flush_interval: "10s"
    flush_jitter: "0s"
    precision: ""
    debug: false
    quiet: false
    logfile: ""
    hostname: "$HOSTNAME"
    omit_hostname: false
  outputs:
    - influxdb_v2:
        urls:
          - "https://us-west-2-1.aws.cloud2.influxdata.com"
        bucket: "kubernetes"
        organization: "miemail@midominio.com"
        token: "WIX6Fy-v10zUIag_dslfjasfljadsflasdfjasdlñjfasdlkñfj=="
        timeout: "5s"
        insecure_skip_verify: false
  monitor_self: false

If we need to adjust other values, like the collection interval, you can do it by changing the interval value. For example, I don’t need the data every 10 seconds, so I changed it to 1 minute.

Now we come to the moment of truth! We are going to install the Helm Chart and see if everything works as expected. Depending on your K3s configuration, you might need to pass the cluster configuration as a KUBECONFIG environment variable.

$  export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Once that’s done, we’re going to add the Awesome-Helm-Charts repo.

$ helm repo add awesome-helm-charts https://xe-nvdk.github.io/awesome-helm-charts/

Then we update the content of the repos that we configured.

$ helm repo update

Finally, we’ll install the repo, passing it the configuration we just modified in the values.yaml file

$ helm upgrade --install telegraf-ds-k3s -f values.yaml awesome-helm-charts/telegraf-ds-k3s

The terminal should return something similar to this:

Release "telegraf-ds-k3s" does not exist. Installing it now.
NAME: telegraf-ds-k3s
LAST DEPLOYED: Fri Jun 25 22:47:22 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
To open a shell session in the container running Telegraf run the following:

- kubectl exec -i -t --namespace default $(kubectl get pods --namespace default -l app.kubernetes.io/name=telegraf-ds -o jsonpath='{.items[0].metadata.name}') /bin/sh

To tail the logs for a Telegraf pod in the Daemonset run the following:

- kubectl logs -f --namespace default $(kubectl get pods --namespace default -l app.kubernetes.io/name=telegraf-ds -o jsonpath='{ .items[0].metadata.name }')

To list the running Telegraf instances run the following:

- kubectl get pods --namespace default -l app.kubernetes.io/name=telegraf-ds -w

This output shows that the Helm chart deployed successfully. Keep in mind that this is a DaemonSet, which automatically installs the Helm Chart on each of the nodes in this cluster.

To check that everything is running properly use the following command:

$ kubectl get pods

We see that our pod is alive and kicking.

NAME                    READY   STATUS    RESTARTS   AGE
telegraf-ds-k3s-w8qhc   1/1     Running   0          2m29s

If you want to make sure that the log is working as expected, then run:

$ kubectl logs -f telegraf-ds-k3s-w8qhc

The terminal should output something like this:

2021-06-26T02:55:22Z I! Starting Telegraf 1.18.3
2021-06-26T02:55:22Z I! Using config file: /etc/telegraf/telegraf.conf
2021-06-26T02:55:22Z I! Loaded inputs: cpu disk diskio kernel kubernetes mem net processes swap system
2021-06-26T02:55:22Z I! Loaded aggregators:
2021-06-26T02:55:22Z I! Loaded processors:
2021-06-26T02:55:22Z I! Loaded outputs: influxdb_v2
2021-06-26T02:55:22Z I! Tags enabled: host=k3s-master
2021-06-26T02:55:22Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"k3s-master", Flush Interval:10s

Everything seems fine, but now comes the moment of truth. We go to our InfluxDB Cloud account, navigate to the Explore section, and we should see some measurements and, of course, some data when selecting the bucket.

As you can see, this process isn’t as complicated as it might seem. The Helm chart simplifies our lives and from now on we can see what is happening with our cluster using an external system.

Three Ways to Keep Cardinality Under Control When Using Telegraf

Ignacio Van Droogenbroeck (InfluxData) — Thu, 20 May 2021 04:00:01 -0700

This article will show how we kept cardinality under control with a few tweaks in the Telegraf configuration. If you’re not yet familiar with it, Telegraf is the native and open-source plugin-driver metrics collection agent of InfluxDB.

As you may know, cardinality is the combination of measurements, tags, sets, fields, and values in a time-series database, and having high cardinality can be a challenge. No worries, though — here are three ways (based on my previous SRE experience) to keep cardinality under control.

One of these is selecting the interval collection based on our needs, the specific metrics we need, and how long we need them.

Let’s take a closer look at these three tweaks and how they helped.

Data collection interval

When we needed to re-architect the monitoring system, we asked ourselves: What’s our SLA, and for what duration do we want to store the data? The default interval of Telegraf is ok, but was it too short for us? Do we need data from more than 100 servers hitting every 10 seconds? We didn’t, and based on our policies, receiving data every 60 seconds was enough. So that was one of the first things we tweaked.

Here’s our Telegraf configuration for this tweak.

[agent]
  interval = "60s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "60s"
  flush_jitter = "0s"
  precision = ""

  debug = false
  quiet = false
  logfile = ""

  hostname = ""
  omit_hostname = false

Adjusting the interval is an excellent way to keep cardinality under control but isn’t the only one. Let’s see the team’s (and my own) favorite way, and that is…

Filtering metrics

Filtering metrics can be another excellent way to keep cardinality under control; sometimes, depending on our needs and scenario, we don’t need all the metrics that an input plugin can offer. Take a look at all these fields collected when we have the Mem Telegraf Input Plugin running.

active (integer, Darwin, FreeBSD, Linux, OpenBSD)
available (integer)
available_percent (float)
buffered (integer, FreeBSD, Linux)
cached (integer, FreeBSD, Linux, OpenBSD)
commit_limit (integer, Linux)
committed_as (integer, Linux)
dirty (integer, Linux)
free (integer, Darwin, FreeBSD, Linux, OpenBSD)
high_free (integer, Linux)
high_total (integer, Linux)
huge_pages_free (integer, Linux)
huge_page_size (integer, Linux)
huge_pages_total (integer, Linux)
inactive (integer, Darwin, FreeBSD, Linux, OpenBSD)
laundry (integer, FreeBSD)
low_free (integer, Linux)
low_total (integer, Linux)
mapped (integer, Linux)
page_tables (integer, Linux)
shared (integer, Linux)
slab (integer, Linux)
sreclaimable (integer, Linux)
sunreclaim (integer, Linux)
swap_cached (integer, Linux)
swap_free (integer, Linux)
swap_total (integer, Linux)
total (integer)
used (integer)
used_percent (float)
vmalloc_chunk (integer, Linux)
vmalloc_total (integer, Linux)
vmalloc_used (integer, Linux)
wired (integer, Darwin, FreeBSD, OpenBSD)
write_back (integer, Linux)
write_back_tmp (integer, Linux)

It’s a lot, right? Ask yourself: do you need all these values? We didn’t because the data that was important to us was used_percent and available.

For the other input plugins, we asked ourselves the same question. So we filtered and collected only the data that mattered to us, and we actively monitored. Our configuration looks like this:

[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false
  fieldpass = ["usage_guest", "usage_system", "usage_idle"]

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "overlay", "aufs", "squashfs"]
  fieldpass = ["used, "free"]

[[inputs.diskio]]
  fieldpass = ["read_bytes", "write_bytes", "read_time", "read_write", "read", "writes"]

[[inputs.mem]]
  fieldpass = ["used_percent", "available"]

[[inputs.net]]
  fieldpass = ["bytes_sent", "bytes_recv", "err_in", "err_out", "drop_in", "drop_out"]

[[inputs.processes]]
  fieldpass = ["running", "zombie", "sleeping", "total"]

[[inputs.swap]]
  fieldpass = ["total", "used", "free"]

[[inputs.system]]
  fieldpass = ["load1", "load15", "load5", "uptime"]

This configuration reduces the fields of collection from 109 to 30. That’s a reduction of more than 72% of the data collected.

The other policy that we applied and helped control cardinality was…

Retention policy and downsampling

Customizing the database retention policy in InfluxDB is an excellent way to control cardinality. Ask yourself (one more time) if you need to have three years of data. Do you see yourself querying disk usage of a server that ran three years ago? Even one year ago?

In our case, we set the life of the data in accordance with PCI compliance and our internal policies. We need to retain the information about our systems for one year. For immediate analysis, we need to retain the data for the last 14 days. So we architected our monitoring system having, in our main buckets, the data for the previous 14 days, and every same quantity of days, we dump the older data into another bucket with a retention policy of one year.

To conclude...

As you can see, with a few tweaks in the Telegraf configuration, such as interval and the quantity of data that we collect, plus the retention and downsampling data available at the time, we were able to keep cardinality under control. What do you think of these tweaks, and what other tricks do you have to keep cardinality under control? Let us know through the comments below.

How to Control Crowds with Python, OpenCV and InfluxDB

Ignacio Van Droogenbroeck (InfluxData) — Fri, 09 Apr 2021 04:00:35 -0700

As I mentioned a while ago, I am learning to develop in Python; it seems that it is a super easy language to interpret, and so far, I have not found a “no” when trying to do things with it. I am delighted, and I keep learning.

In this learning process, I started to play with OpenCV (Open Source Computer Vision Library) to detect faces. OpenCV is an open-source, BSD-licensed library that includes several hundreds of computer vision algorithms. It did not take me long to understand how everything worked, and I began to think about InfluxDB time series use cases and how to integrate other technologies.

From that search came the idea of integrating OpenCV with InfluxDB to count the times it detects a face. Time series and biometrics are a perfect fit, I think, opening up numerous applications such as, but not limited to, the following:

Register the number of people coming through a specific entry point in a store at a particular moment of the day
Register the number of cars or other vehicles that pass for an avenue.
Count how many people stop and watch a specific type of product to understand if it's attractive to them
Measure customer conversion in retail and hospitality establishments, such as comparing restaurant guests walking in vs. plates sold
Overwatch crowds within a defined space and time (resources allocation, traffic light management, infrastructure improvements, etc.)

The last is very important considering the global context we live in. Controlling crowds to ensure public safety in a pandemic era has been challenging for law enforcement agencies worldwide.

Let's see some of the code...

The code of this project can be found in this GitHub repository. The relevant files are:

live.py
influxdb_config.ini
haarcascade_frontalface_default.xml

The file live.py looks as shown below. The only thing to modify, if desired, is the name of the bucket to match the bucket that we created in InfluxDB.

# This script will detect faces via your webcam and push the data to an InfluxDB instance.
# By Ignacio Van Droogenbroeck @hectorivand
 
import cv2
 
from datetime import datetime
 
from influxdb_client import Point, InfluxDBClient
from influxdb_client.client.write_api import SYNCHRONOUS
 
# Specify the InfluxDB bucket
 
bucket = "crowd-counter"
 
# In this file, you must set the parameters to connect to InfluxDB.
 
client = InfluxDBClient.from_config_file('influxdb_config.ini')
 
write_api = client.write_api(write_options=SYNCHRONOUS)
query_api = client.query_api()
 
cap = cv2.VideoCapture(0)
 
# Create the haar cascade
faceCascade = cv2.CascadeClassifier("haarcascade_frontalface_default.xml")
 
while(True):
	# Capture frame-by-frame
	ret, frame = cap.read()
 
	# Our operations on the frame come here
	gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
 
	# Detect faces in the image
	faces = faceCascade.detectMultiScale(
		gray,
		scaleFactor=1.2,
		minNeighbors=5,
		minSize=(30, 100)
	)
 
	# Write the quantity of faces detected on InfluxDB
 
	p = Point("public-count").tag("cameras", "entry").field("people", '{0}'.format(len(faces)))
	write_api.write(bucket=bucket, record=p)
 
	# Draw a rectangle around the faces
	for (x, y, w, h) in faces:
		cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
 
	# Display the resulting frame
	cv2.imshow('frame', frame)
	if cv2.waitKey(1) & 0xFF == ord('q'):
		break
 
# When everything is done, release the capture
cap.release()
cv2.destroyAllWindows()

The other file to modify is the influxdb_config.ini. This file has the data to connect to InfluxDB. As you can see, you need to define the values to connect to InfluxDB.

[influx2]
url=$INFLUX_URL
org=$INFLUX_ORG
token=$INFLUX_TOKEN
timeout=6000
verify_ssl=True

If you don’t have an instance of InfluxDB OSS running, I recommend you create an InfluxDB Cloud account (InfluxDB Cloud is a managed, elastic, serverless edition of InfluxDB). The free tier is more than enough to perform some tests. If you’ve decided to use this solution in a production environment, you can keep your Cloud account.

The last important file is the haarcascade_frontalface_default.xml. This file provides the models to detect faces. In the OpenCV: Cascade Classifier tutorial, I recommend reading about haarcascade and why it is an excellent option to detect objects without the need to train a TensorFlow model.

Now we have everything set up, so let’s run the code.

Let's count some faces...

Once this script is running, if you run the live.py script, it will activate your webcam to start counting faces and send that data to InfluxDB.

As you can see, this face detection approach is not so complicated. It can be used to control traffic flow on a highway or avenue.

Well, you might be thinking — all is beautiful, you have a pretty face, the cars look good too, but show me how to control crowds using this technology. Fair enough! Let me reply to your comment with this screenshot.

In this case, the script detected 110 faces and sent that data to InfluxDB. Imagine this example running through several devices in parks, restaurants or fairs.

As you may notice, I used a video because, well, I don’t have a crowd in my home! But the output will be the same; the only difference in this case is the input of the data (the video vs. a live feed).

Conclusion

This software solution isn’t perfect; it is part of my learning path. If you find some issues, I encourage you to fork and propose improvements in the GitHub Repository.

I hope this article has just given you a whole new use case for deploying InfluxDB.

Monitoring DigitalOcean Billing with InfluxDB

Ignacio Van Droogenbroeck (InfluxData) — Wed, 24 Feb 2021 04:00:54 -0700

I’ve always had a good experience using DigitalOcean, a cloud infrastructure provider which offers developers cloud services that help deploy and scale applications that run simultaneously on multiple computers. I’ve used DigitalOcean a lot for my personal projects — for example, to host my personal blog, its stats, and a NextCloud instance, all running in Kubernetes.

Monitoring your server is always a good idea. When you have workloads in the cloud, monitoring turns critical because while you’re using a Cloud infrastructure, you remain responsible for the health of the servers/services you host. The Cloud provider mostly monitors their own infrastructure, but you’re on your own to monitor the software you run. If your application stops responding, this becomes a problem you have to solve.

Monitoring simplified with InfluxDB Templates

An InfluxDB Template, the DigitalOcean Billing Monitoring Template lets you quickly define your entire monitoring configuration (data sources, dashboards, and alerts) in one easily-shared, open-source text file that can be imported into InfluxDB with a single command.

Template import is built straight into the InfluxDB Cloud UI — no software to download or command line syntax to memorize. All you have to do is copy a link to the Template you want, paste it into the page in your InfluxDB Cloud UI, and you’re done.

Controlling DigitalOcean account balance

The pricing of DigitalOcean is straightforward, with monthly billing cycles that depend on the price plan tier you’re on. I’m always charging my DigitalOcean account in advance, so the template allows me to control my balance — to understand how much my resources cost per day and how much I owe — without entering the DigitalOcean dashboard.

I use this data right next to the monitoring of my Kubernetes cluster running in DigitalOcean. I created this dashboard using the DigitalOcean API, and simply to have system monitoring and billing in only one place.

Using the DigitalOcean Billing Monitoring Template

You can use this InfluxDB Template to actively monitor your consumption of resources to manage your DigitalOcean Account spend. The template’s dashboard monitors current balance, consumption to date, and balance to date.

To gather monitoring data for the template, I use the Exec Telegraf Input Plugin (which parses supported Telegraf input data formats) and then use the InfluxDB Output Plugin to carry that JSON data to InfluxDB.

To install this template and get a headstart on monitoring, sign up for your free InfluxDB Cloud account.

HAProxy Monitoring (the InfluxDB Way)

Ignacio Van Droogenbroeck (InfluxData) — Tue, 02 Feb 2021 04:00:30 -0700

My personal experience with HAProxy dates back to my work with a previous company, where we used HAProxy to do load balancing between pairs of servers with specific roles. Those servers are the core of the major payment gateway in Uruguay, where thousands of users use them every day to pay their bills, recharge their mobile phones, pay parking fees, and even play lottery numbers.

Before I created The HAProxy Monitoring Template — an InfluxDB Template — we were blind in terms of understanding how HAProxy was performing or even if our users viewed any error when they tried to do some transactions.

What happens if you don't monitor HAProxy

If you don’t monitor HAProxy, your users are the alert method to understand if something isn’t working as expected. Before starting to monitor using InfluxDB, we had a hard time responding when developers or customers told us: “The application is working slow.” We had to review each part of the architecture and ask ourselves: Is the application, bandwidth, or backend to blame — what’s going on? We had to make a diagnosis in this almost blind scenario. This takes time, and time is money.

When you need to analyze the server’s and your application’s performance, you can use InfluxDB. We used the built-in tool to get metrics and bring them to InfluxDB, create a dashboard and alerts to monitor the service’s performance, and if, for example, our users and developers received an HTTP code 200 or something else — we’d know. They didn’t need to run in circles screaming “Fire, fire, fire!” when something didn’t work as expected.

The above scenario is what led me to create the The HAProxy Monitoring Template profiled below. But first, a quick refresher on HAProxy and InfluxDB Templates.

Overview of HAProxy load balancing

HAProxy is an open-source TCP/HTTP load balancer software and proxy server for TCP and HTTP-based applications. It can run on Linux, Solaris, and FreeBSD to improve the performance and reliability of the server environment by distributing workloads across multiple servers. HAProxy is used by some of the world’s highest-traffic websites. HAProxy load balancing algorithms determine which server, in a backend, will be selected when load balancing.

Overview of InfluxDB Templates

InfluxDB Templates let you quickly define your entire monitoring configuration (data sources, dashboards, and alerts) for any technology in one easily-shared, open-source text file that can be imported into InfluxDB with a single command. Most InfluxDB Templates are community-contributed, so if you’re interested in submitting your own InfluxDB Template, learn how to build one.

Collecting HAProxy metrics with InfluxDB

The HAProxy Monitoring Template has a preconfigured dashboard.

The template uses the HAProxy Telegraf Plugin to collect metrics about HAProxy and store them in InfluxDB, which is then presented in the included dashboard:

Telegraf is InfluxDB's open-source, plugin-driven metrics collection agent.
The Telegraf plugin gathers statistics using the stats socket or HTTP statistics page of a HAProxy server.
The HAProxy Monitoring Template webpage lists the HAProxy metrics you can monitor using this dashboard.

Installing the template is easy:

Once your InfluxDB credentials have been properly configured in the CLI, you can install the HAProxy monitoring template using the Quick Install command.
Once installed, the data for the dashboard will be populated by the included Telegraf configuration, which includes the relevant HAProxy metrics.
General instructions on using InfluxDB Templates can be found in the use a template document.

It’s worth noting that this template can be used with the Linux System Monitoring Template. To install the HAProxy Monitoring Template, sign up for your free InfluxDB Cloud account.

A great way to get started with InfluxDB

Developers need to have visibility of how their applications are performing. It’s also important to store that data in a system that allows you to manipulate it, and InfluxDB can help. This is why InfluxDB Templates are a great way to get started with InfluxDB and time series storage.

Are you developing an application or device that gets important data to your customers? Count on InfluxDB; it’s not only important to get the data. The value resides in what we can do with that data to improve customers’ business. For example, can your application/device measure the quantity of water used in the farm? You can store that data in InfluxDB. You, the customer, can understand if the water is being used efficiently. Do you have an IoT device that counts the number of people in front of a business within a period of time? InfluxDB also helps, and your customers can do promotions in a specific period of the day to get more customers.

I can spend the entire day talking to you about how InfluxDB can help bring value to your IoT or monitoring solutions, and maybe I will do just that — in a future blog post!

Automating SSL Certificate Expiration Monitoring

Ignacio Van Droogenbroeck (InfluxData) — Tue, 19 Jan 2021 10:00:01 -0700

In my previous work experience, monitoring certificate validation was critical to our team. These certificates were used to sign commercial transactions between the payment gateway (us) and other providers. That check was manual and depended on the calendar of one person. So, if that person forgets to notify the team about the upcoming expiration of one certificate and doesn’t start the procedure of getting the new one, well, the platform starts to fail. This is what led me to create the x509 SSL Certificate Monitoring Template, an InfluxDB Template.

What happens when an x509 SSL certificate expires?

X.509 certificates have an expiration date that can prevent your website and applications from working properly, and thereby presenting your users with an alarming warning that the site’s security certificate has expired.

When a certificate expires, several things can happen. Your users will receive a message that your site is not secure. If yours is an e-commerce site, that can’t be good for business. Also, the certificates can be used to authenticate between systems and/or to sign commercial transactions.

One expired certificate means financial loss and bad customer experience. So one can just imagine the negative snowball effect of having many customers encounter several expired certificates.

How the X.509 SSL Certificate Monitoring Template works

To avoid being faced with certificate expiration — and the business disruption and undermined credibility it can cause — it is a best practice to check the expiration dates on a regular basis. The X.509 SSL Certificate Monitoring Template does just that – monitors SSL certificate expiration.

Like other InfluxDB Templates, this template lets you quickly define your entire monitoring configuration (data sources, dashboards, and alerts) in one easily-shared, open-source text file that can be imported into InfluxDB with a single command.

The template uses the x509 Telegraf plugin, and can handle many certificates at a time. It can be used in a corporate environment (for enterprises that handle a large number of certificates) to monitor certificates from a URL or directly in the certificate store. To install this template, sign up for your free InfluxDB Cloud account.

Free to use and simple to set up for your use case

This template can be used for free with an InfluxDB account. That’s one of the main advantages of using this template, and it’s very simple to set up and understand — no need to configure or write a script.

While alerting isn’t built into the template, it would be straightforward to set alerts using familiar technologies, given how well InfluxDB integrates with incident management systems, to get notifications before a certificate expires.

More security-related InfluxDB Templates

Other security-related InfluxDB Templates include:

The Fail2ban Monitoring Template, which helps you to determine if there are more (or less) IP addresses that you might want to add to your block list.
The Endpoint Security State Template, which can be used to monitor certificates and authentication of determined endpoints.

Want more security-themed premade dashboards? Most InfluxDB Templates are community-contributed, so you could be the one to build the next template (and we’ll share it with the community in an article like this one)! Here’s how to build and submit your own InfluxDB Template.

Why I Joined InfluxData - Ignacio Van Droogenbroeck

Ignacio Van Droogenbroeck (InfluxData) — Wed, 23 Dec 2020 04:00:12 -0700

A few things pop up in my head to quickly answer why I joined this company, and I want to share them with you.

People & values

“People” is the first thing that comes to mind when I ask myself why I joined InfluxData.

In 2020, I started contributing to InfluxDB and had the opportunity to meet great persons with whom I had a cool work synergy and relationship throughout the year. That’s one of the reasons that I joined this company.

But, not only that, the values and that sentence in specific that I found on the careers page was very inspirational for me:

“Join a team that believes in a healthy balance of work and home life — one that believes we succeed together.”

They got me. I’m a team player, and “succeed together” is a powerful statement for me.

Technology

I started to work directly with InfluxDB v2.0, and I found it very easy to understand, deploy and play with. I saw a tool not only to monitor systems but also to understand a lot of other types of data that helps companies and users make better decisions.

InfluxDB is the technology that helps you to understand the following statement: “If you can’t measure it, you can’t improve it.”

Open source & community

One of my dreams was to work in an Open Source company, and now, I’m living that dream.

It wasn’t only for me and my dream. I also saw how important the community is (that helps shape InfluxDB) to the company. InfluxData recognizes the community’s hard work contributing to code, content, and support in the Community Slack.

As part of this community, even before I joined InfluxData, I contributed to developing InfluxDB Templates to accelerate the adoption of v2.x. One of the most popular here is the CS:GO template, but I also created a template to track COVID19 in South America and a few more to do system monitoring.

Due to these contributions, I have been invited to be part of the InfluxAces program. This program was created to recognize contributors to InfluxDB with code, content and knowledge sharing with the community.

Opportunity to learn more and challenge to impact through innovation

I love learning, reading, trying new things every day. At InfluxData, I’m learning a lot, working with a great team where we feel challenged to keep going and impact our customers worldwide with innovative solutions.

Do you want to join us?

We’re looking for you. Take a look at our careers page to know about the opportunities, but I warn you:

"We don't want people who just want jobs but who want careers that fuel their personal and professional growth."

Monitoring COVID-19 in South America with InfluxDB

Ignacio Van Droogenbroeck (InfluxData) — Tue, 22 Dec 2020 11:33:07 -0700

Monitoring COVID-19 metrics has become essential as the Coronavirus pandemic continues to impact countries across the globe. Mitigating virus impact and spread requires ongoing tracking of metrics such as cases and fatalities. This is what the InfluxDB COVID-19 Monitoring Template does.

Introducing the InfluxDB COVID-19 Monitoring Template

The template provides a dashboard that includes information about COVID-19 focused on the South American region; primarily Argentina, Bolivia, Brazil, Chile, Paraguay and Uruguay. I created it, as a contributor to the InfluxDB Community, before I joined InfluxData as a Solution Architect.

I developed the template because I wanted to track the COVID-19 situation, not only in Uruguay where I live, but also in the South American region. I have family in Argentina, and friends in Chile and Paraguay, so I looked to monitor all the region. Groups that would find this template useful are those who want to know how other countries are ‘performing’ in the development of the pandemic.

Previously, I had built a COVID-19 monitoring Chronograf dashboard. Once InfluxDB Templates for the InfluxDB UI (which includes a built-in visualization engine) were launched, I then built the InfluxDB Template for the same purpose. The template’s dashboard is shown below.

Dashboard information includes current data, new cases, graph about totals of cases and deaths, as well as — for reference — overall worldwide information about cases and deaths.

As for how this template compares to other open source COVID-19 monitoring dashboards, the only dashboard I’ve seen with the kind of information I want is the Johns Hopkins University (JHU) dashboard. The ‘problem’ with this dashboard was that I couldn’t group the data for the countries I’m interested in. To install this InfluxDB template and get started with COVID-19 monitoring today, sign up for your free InfluxDB Cloud account.

The growing popularity of InfluxDB Templates

InfluxDB Templates let you quickly define your entire monitoring configuration (data sources, dashboards, and alerts) for any technology in one easily-shared, open-source text file that can be imported into InfluxDB with a single command. Most of the templates are community-contributed, and here are the guidelines for building and submitting an InfluxDB Template.

Using InfluxDB Templates is straightforward. You can start to track wherever you want, in most cases in a minute or so, and most of the templates don’t require more than defining a few things such as environment variables. To start, just import a template from the Community Templates on GitHub, export the HOST, TOKEN, and ORG as variables, copy the URL for the Telegraf config, run it in your machine or a container, and you’re done.

Since I started contributing InfluxDB templates, I’ve seen the number of forks in the repository go up. Also, I really like seeing that InfluxDB use cases are not only focused on systems monitoring but also on tracking game progress, like Fortnite and CS:GO, currencies and home automation things.