Advanced: Write Your Own Telegraf Plugin

Coming soon! Our webinar just ended. Check back soon to watch the video.

Webinar Date: 2018-08-16 08:00:00 (Pacific Time)

Telegraf is a plugin-driven server agent for collecting & reporting metrics and there are many plugins already written to source data from a variety of services and systems. However, there may be instances where you need to write your own plugin to source data from your particular systems. In this session, Noah Crowley will provide you with the steps on how to write Telegraf plugins. This will require an understanding of the Go programming language.

Watch the Webinar

Watch the webinar “Write Your Own Telegraf Plugin” by filling out the form and clicking on the download button on the right. This will open the recording.

Transcript

Here is an unedited transcript of the webinar “Write Your Own Telegraf Plugin.” This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors.

Speakers:
• Chris Churilo: Director Product Marketing, InfluxData
• Noah Crowley: Developer Evangelist, InfluxData

Chris Churilo 00:00:01.552 All right. Welcome, everybody, to our training this morning. We are just getting set up waiting for the various attendees to join us, and we’ll get started at three minutes after the hour. Just want to let everybody know that we have InfluxDays coming up in New York City on the 13th of February. And in addition to that, we will be opening up call for papers for the next InfluxDays, which will be in London in June. So we’ll be sending that note out pretty soon. And we also have a brand new Dev Evangelist team, and Noah, who will be our presenter today, is actually a part of that team. And the nice thing is that they already have kicked off a set of meetups around the world. And you should be also getting email notifications about that soon. So we have meetups in New York City, where Noah lives; London; Denver; and San Francisco; and Raleigh. Another thing that I want to remind everybody is that we have a community site that I encourage everyone to go to for your questions, at community.influxdata.com, and Noah, and all of the evangelists, as well as our developers do a pretty good job of sitting on that site and making sure that they answer your questions. So if after today’s session you have a lot more questions, please feel free to go to that site, and the team will definitely be happy to make sure that they can serve you guys. We’ll get started a minute. As always, I record these sessions and post them to our website so you can take another listen to them. And at any point during the presentation, you can just put your questions in the Q&A panel or in the chat panel. Whatever is easy for you to get to, it’s not a problem. And if you really would like to use your voice to actually ask your questions, just raise your hand and I’ll make sure that I unmute you so you can ask Noah directly. We want to make sure we make it as easy as possible. And today’s topic is actually a lot of fun. It’s quite a popular topic, and we often have people listening to this webinar over and over because it’s all about writing your own Telegraf plugin. And if you have written anything in Go, then you know it’s a pretty fun language. And it’s an excuse, if you haven’t written anything in Go, to check out Go, as well as to take a stab at writing your own plugin.

Chris Churilo 00:02:30.998 So I know it’s three minutes after the hour, and I promised that we would get started; so we will. So welcome, everybody. My name is Chris Churilo, and today, we are doing a training on writing your own Telegraf plugin. We record these sessions, and I’ll post them before the end of the day to our website, and you’ll get an email with the link to the webinar. And if you have any questions about this webinar, or if you even have any other questions about any of the components of InfluxData, so the database, Kapacitor, Chronograf, just go ahead and put your questions in the Q&A panel or in the chat panel, and Noah and I will do our best to get those answered to you before the end of our session today. And if not, what I typically do is, if you guys stump me, then I’ll just take that question and I’ll actually post it to the community site on your behalf. And then internally, I kind of nudge some of the developers to help me out with some of the questions. So we will get those questions answered. And I have to say that, sometimes the Telegraf plugin questions are a little bit of a white head because I might not be familiar with the technology that you’re trying to pull data from or write to. But we have a great group of people that can definitely help. So with that, I’m going to turn it over to Noah. So, Noah, you want to give your introduction and kick off the training?

Noah Crowley 00:03:54.416 Yeah, absolutely. Thank you, Chris. So welcome, everybody. We’ll be writing a Telegraf plugin this morning. Like Chris said, this is a really great project to get started with if you’re just learning Go. We do a little bit of work with interfaces, and you can pull some data in from something you’re familiar with and send it off to InfluxDB. So it’s a really great getting-started project. So quickly, we’ll go through a brief overview of Telegraf, what it is, how it fits into our stack, and how you can use it with other open source tools. We’ll touch on some examples of Telegraf plugins, what’s already out there, what you can monitor, sort of, out of the box. We’ll go briefly into the Telegraf plugin architecture. The whole agent is plugin-driven, which makes it really easy for the community to add different functionality to it. And then we’ll go through a brief demo of actually writing a Telegraf plugin so that you can see how that’s done.

Noah Crowley 00:04:57.750 So Telegraf is one component of our TICK Stack. The TICK Stack is InfluxData’s platform for metrics and events. At its core is InfluxDB, which is our time series database. And then in addition, we have Kapacitor, which is a stream processing engine in Chronograf, which is our visualization platform. But today, we’re going to be talking mostly about Telegraf. Telegraf is our collection agent. It lives out on your machines, it collects metrics and events from various sources and it ships them off to any type of time series store that you want to use. So InfluxDB is great, but we can also ship to Graphite, we can also ship to OpenTSDB. Part of the reason that we wrote Telegraf as our own plugin was so that we would have something that worked with the data format that is native to InfluxDB, and also, to some other Time Series Databases. So if you’re familiar with Graphite, you might know that each metric has its own name space, and that’s usually a period delineated stream that describes what the metric is that you’re using, and then it has a timestamp and value associated with it. What we do in InfluxDB, and what we do, and what OpenTSDB and other time series databases do is, they actually use tags to tag the various metrics to give you additional degrees of chronality to be able to look at various aspects of the metrics as you’re filtering them, and querying, and things like that. So it lets you be a little bit more efficient with how you store things on disks, and how you query them. So Telegraf is custom designed for that.

Noah Crowley 00:06:38.096 Telegraf is open source. It’s our most open source tool. We get a lot of contributions from the community. And like I said, it works with a lot of different applications. So it’s not just something that you have to use with a fixed stack, but you’re welcome to use it with whatever time series database or pipeline that you happen to be using. It can do either push or pull metrics, depending on which output plugin you’re using. So that’s a topic that comes up a lot in the monitoring conversation. And we support both methodologies. But Telegraf is primarily used to batch metrics from multiple sources together in order to reduce the amount of atomic write requests. This is really important when you’re getting massive, massive numbers of metrics into InfluxDB, or your time series database. You don’t want to overload the server with continuously trying to write individual points to disk. Writing to disk isn’t super-fast, and so if you can batch these things together and send them over and write them all together as a batch, then that’s much more efficient. We have a large collection of plugins. They’re broken down into inputs, outputs, aggregators, and processors. Those are pretty self-explanatory. The inputs are whatever you use to collect metrics and events from your various sources. The outputs spit it out to your time series database, or your message queue, or wherever you want them to be going. We also have the ability to aggregate and process the data. So we can compute means, medians, histograms, things like that. And we can do some processing, in terms of adding tags, or changing names, and etc. etc.

Noah Crowley 00:08:18.004 So Telegraf can act as an agent. It can sit out there and do its own thing, periodically collecting metrics. It can act as a collector. So it can be, sort of, a passive sink we use, and the metrics to that, and it receives them and passes them on. Or you can insert it pretty much anywhere if you’re in just pipeline. We see it being used for, sort of, a fan-out pattern, where multiple Telegraf agents will feed into a single message queue, or something like that. But whatever your pipeline looks like, it’s most likely that Telegraf will be able to easily feed into that. We do the metrics ingestion from a number of different places. We can grab metrics from your whole system, CPU, I/O, network, usually reading from the file system for those things. We’ve got a large number of plugins for our common programs: Hadoop, Postgres, Redis, etc. Over 140 plugins, at this point in time. You can go on to the Telegraf repo on GitHub and check those out and see just exactly what we have. We have a bunch of third-party APIs: Mailchimp, CloudWatch, Google Analytics, etc. And then we also have output plugins. So those are our various ways of getting the data that’s being brought into Telegraf, out into so sort of database, data store, message queues, etc. We have output plugins for InfluxDB, for Graphite, for OpenTSDB, Datadog, Librato, Kafka, MQTT, NSQ, Prometheus, and many others. So really, this is something that you can drop into your infrastructure, no matter what you’re using, and pretty much be guaranteed to be able to get data out of your applications and into your pipeline as quickly as possible.

Noah Crowley 00:10:12.281 So like I mentioned, we have over 140 plugins. It’s definitely worth taking a look at that list and seeing if the pieces of your infrastructure are on there. And if they’re not, the you’re in the right place to write a new plugin to add those things. There are a lot of metrics collection agents out there. Telegraf has a few benefits, like I mentioned earlier. It’s specifically written to handle metrics that have tags, as opposed to metrics that are uniquely-names based. We think that’s a really huge advantage. And if your time series database supports that, it makes a lot of sense to use an agent that works with that paradigm as well. It’s easy to contribute functionality. The application is written in Go. Go is a great language for collaboration because it’s fairly strict about the way it needs to be written. It means that you can go online and take a look at the examples that are already out there, easily understand them, and write your own. And the fact that it’s written in Go means that it compiles into a single binary with no external dependencies, which is really nice, in terms of deployments, and distribution, and things like that. It makes things a lot more simpler than if you were to have to manage a Python installation, or something else like that. And like I mentioned before, we have really strong community contributions with Telegraf. Most of our plugins are actually community contributed. This is something that people are using. And they find functionality that they need, a lot of the times, because it’s so easy. They just go ahead and add it themselves. So we’ve seen really great growth in the community in the contributions and plugins that we’re getting. And of course, if you’re writing something, if you’re working on this, you can reach out for help either on our community site, or on GitHub issues, and one of our engineers, or some of the dev op team will get in touch with you and help you with that.

Noah Crowley 00:12:11.309 So let’s just touch on a few examples of Telegraf plugins so that you can sort of see what’s out there and what you might end up using. We’ve got StatsD, which is really popular. A lot of places are using StatsD as a format for sending their metrics either from their applications or from other collection agents. If you’ve instrumented your application, and you’re sending StatsD information, StatsD metrics out of it, you can just drop in Telegraf and place the StatsD, and you can get to work. Postgres, really popular database. We support that. It pulls information from the built-in PG Stats database, and the pg_stat_bgwriter views. We support Apache. Again, super popular. Collecting data from the /server-status?auto endpoint and storing it in InfluxDB, or your time series database. win_perf_counters are in there, for those Windows users. So you can gather performance information about Windows machines. We also, of course, have built-in collections for Linux and Unix-based machines as well.

Noah Crowley 00:13:21.059 Couple of more Telegraf plugins. One thing that’s really popular right now is Prometheus. We can happily expose any of our metrics in a Prometheus format. So the prometheus_client is an output plugin that creates an endpoint. It functions the same way that any Prometheus metrics endpoint would work. If you already have a Prometheus starter, you can immediately start scrapping metrics from that. The Histogram plugin is an aggregator. So I wanted to throw in one example of those. But as metrics are coming in, this will allow you to compute histograms for that data containing the counts of filed values within various ranges. We’ve got an Nginx plugin along the same lines as Apache. And of course, we’re working on something for Kubernetes. So this plugin is still, currently, in the experimental changes, but we’re putting a lot of effort into it. It talks to the Kubelet API using the /stats/summary endpoint. And the reason it’s experimental is because of some of the used patters that you see with something like Kubernetes, is where you’re bringing up a lot of ephemeral containers, you’re spinning them up, you’re killing them. Maybe somebody is bringing up an entire dev environment, something like that. So you quickly see in the number of metrics and series that you’re collecting start to balloon. That’s the benefit of Kubernetes, is having the ability to do that kind of ephemeral workloads. But it does post some challenges for storing those metrics in your databases. I know, if you’re using some of the hosted solutions that only provide a certain number of metrics per plan, you can quickly eat through those if you’re using something like Kubernetes is generating a lot of ephemeral series. So as we work on this and get it out there, I think this is going to be really valuable for any of you who are starting to play with orchestrators and containers at a significant scale.

Noah Crowley 00:15:28.210 If you’re looking to install it, you can download the binaries at influxdata.com. Telegraf has the built-in ability to generate a sample configuration, so that’s the example code you see here. We’re calling Telegraf, we’re telling it to generate a SampleConfig using the input filter haproxy and an output of influxdb. We’re exporting that to a file, and then we’re launching Telegraf using that configuration. That file has our configuration. Telegraf normally runs as a Linux service. So once you’ve created that configuration, if you store it in actelegraf, telegraf.com, you can just start throwing in service using starters Telegraf. As an aside, if you go to our website in the Telegraf documentation, you can actually find more specific guides for individual oases that are out there. So if you’re on Ubuntu, if you’re running Red Hat, if you’re running Mac OS 10, we have packages available in a large number of package managers. So if you are on one of these operating systems, just working as a developer, the package manager is probably the quickest way to get up and running. So OS 10, it’s as simple as installing Homebrew, updating and install Telegraf. But the binaries will work regardless of which system you’re on or what you’re working with.

Noah Crowley 00:17:03.182 So I’ll talk really quickly about the plugin architecture of Telegraf. Like I mentioned before, Telegraf was built with the idea of plugins, from the very beginning. The functionality in Telegraf, inputting data, outputting data, whether you’re collecting it from a system, or a database, or the network, these things are very modular. They can be easily swapped in, one for the other. And the way that we do that is using interfaces; Go interfaces. So each type of plugin has an interface associated with it. If you go to the Git repo, there’s a file in there called contributing.md, which actually has a really great detailed guide about contributing plugins back to our repo, back to the application. And within this guide, it goes over the various types of plugins and their interfaces. So we have input plugins, which are used for receiving data or pulling data from variety of sources. They have a well-defined interface that needs to be fulfilled. We can also have service input plugins. So an input plugin is defined as something that is executed continuously. So once every 10 seconds, or something like that, the input plugin, or the service input plugin runs continuously. So it’s always running. As metrics come into it, it collects them and prepares them for output. We have output plugins. Again, there’s our simple interface that you can take a look at, and we’ll touch in more detail on the interfaces when we get into the demo, but this is just to give you an overview. This is, sort of, the output example interface. We also have processor plugins, which has an interface and aggregator plugins which have an interface. So the entire plugin architecture is defined around these interfaces. And as long as you go in, write a few files, fulfill those interfaces, you’ll have a working plugin that does what you need it to do.

Noah Crowley 00:19:16.597 So for the input interface, in particular, this is what it looks like. There’s a number of methods, but the only one that really does anything, in terms of processing or receiving the data, is down here at the bottom, which is the gather method. The other two are used for generating your config files, SampleConfig will return the default configuration of the input. So that might include something like a IP address, if you’re reaching out to a network instance. It might include a file name if you’re processing log files, or something like that. Description returns a one-sentence description of the input. So if we’re doing an Apache plugin, then it’ll talk, “Hey. This is the Apache plugin. We’re pulling stats from this endpoint.” Very simple. Very straightforward. But that’s so you can easily identify what each plugin is doing in the configuration file. And then finally, there’s the gather interface, which is the accumulator that adds the metrics that the input gathers. So every time the input plugin runs, it’ll run the accumulator interface. It’ll take any metrics that it’s gathered and it’ll send them off into the rest of the Telegraf application. So let’s dive a little bit more into that and actually write an input plugin for ourselves.

Noah Crowley 00:20:45.359 So you’ll need Go installed on your computer. Good to have the latest version. You can go ahead to goline.org, they’ve got great install guides, and you can install that there. But I think anything above 1.5 will work if you already have that installed on your machine. You need a desire to write a Telegraf plugin. So for a lot of you out there, I think, maybe, you’ve taken a look at what we have, and there’s something specific to your use case that you need, or you want to make modifications to existing plugins, or something like that. And that’s why you’re here at this webinar today. So for the example, we’re going to work on a plugin called Trig. It’s a demo plugin that we’ve created that really just writes cosine and sine waves to the database. But it’ll give you enough insight into how we fulfilled the interfaces, and what needs to be done to write an input plugin. So let’s get started. The first thing to do is actually get the source code of Telegraf. You can use Go’s built-in tooling to do this. First thing you type into your command line is “go get github.com/influxdata/telegraf.” This will pull down the source code into your Go graphs, which is your Go work space. All programming done in Go is usually done within the same work space. And these tools automatically write your call down source code and build packages within that work space so that everything you’re working on knows about each other, and knows about dependencies, and things like that.

Noah Crowley 00:22:32.105 So once you’ve gotten the code using “go get,” you can change directory to “GOPATH,” which will be filled in by the [inaudible] github.com/influxdata/telegraf. And then once you’re in that directory, you can create a new branch and check it out using “git checkout -b mySweetPlugin” and go ahead and type “make” after that. That’ll actually go through the build process and make sure that everything is configured correctly, and that you have all dependencies, and your environment is set up. And at the end of it, it’ll create a new Telegraf binary at GOPATH/bin/telegraf. You can run that, and you shouldn’t receive any errors when you do so. The next step is to start making the files that you’re going to be working on. So you can change to the plugins directory, and the inputs sub-directory within that. That’s where we’ll be doing most of our work. We’ll make a new directory for the plugin, called “trig”. And then within that directory, we’re going to create a new file using the “touch” command. So we type “touch trig/trig.go” and that will create a new Go file in there, to which we’ll “add boilerplate”. So once you’ve added the boilerplate, you can do “cat trig/trig.go” to make sure you put it in the right place. But this is what the boilerplate looks like. And we’ll go through this in a little bit of detail.

Noah Crowley 00:24:13.081 This is the Go file, or the code for the trig plugin. The first lines that you need are import statements. These are importing data from the rest of the Telegraf application, the main application, as well as some information about the various inputs. You’ll define a struct, which is the main data structure for this object, for this trig object. And then we’ll define a number of methods that get bound to that struct. So the first one is SampleConfig. We talked about that before. This is going to inject a string. So a simple, one-line description of your plugin that can be used in the configuration files. Sorry, I switched that. The SampleConfig is the actual data that’s going to go into your configuration files, and the next method description is the simple one-line description of what the plugin actually does. Finally, the last method that we need in order to fulfill this interface is the gather method. So we’ll put that in there as well. And at the end, we have a function called init, which is called a Telegraf application itself at startup. And this basically just registers the plugin that you’re working on with Telegraf, so that it knows that it exists, and that it can call it, and that it can use it. Without this init function, Telegraf wouldn’t be able to find the plugin that you’ve created. So in order for that init function to be called, Telegraf needs to know about the plugin that you’re creating. There’s a file called all.go within input/all in the plugins directory. And this just contains the list of all the plugins that have currently been contributed to Telegraf. Telegraf will scan through this list, it’ll find each of the individual plugins, and it’ll import them into the main Telegraf package. This makes sure that they can run after starup. Without this line, Telegraf will not know about your plugin, and it won’t be able to run it. So make sure that you add that.

Noah Crowley 00:26:40.696 The next thing that we’re going to do is add any configuration variables that we might need. This goes back to the idea that Telegraf can dynamically construct your configuration files by aggregating the configurations for all of its plugins. So here, in this example, you have “var TrigConfig”, and within that variable, you have “amplitude = 10.0”. This will set the amplitude of the wave form that we’re generating. If this were a plugin that were accessing resources over the network, we might add a default IP address there, maybe localhost. At the very least, we would define a space for the user to add that information themselves. But localhost is usually a good way to go for networking plugins. If it’s going to be reading files from the disk, again, that information can go there. If you want, you can actually put that variable within the SampleConfig function itself. But we like to just leave it outside for stylistic purposes; it makes it a little bit easier to read. So we have the variable here. We have “amplitude = 10.0” and we’re returning that variable when the SampleConfig function is called.

Noah Crowley 00:28:09.256 We’ll add a simple description of our plugin to go along with those configuration variables. The description is included above the plugin configuration in that configuration file. In this example, “inserts sine and cosine waves for demonstration purposes” and then you’ll set the amplitude comment, and then “amplitude = 10.0” within your configuration itself. So let’s test out the config stuff, even before we get to writing the actual code for the plugin itself. We can rebuild Telegraf by typing “make” from the root of the Telegraf directory. That’ll build the new binary and put it into your GOPATH/bin. And from there, we can type “telegraf” and pass it a few arguments to generate a configuration file, specifically, for our trig plugin. So those arguments are the first in SampleConfig, which will tell Telegraf to generate a configuration file. The second is the input filter, which is which plugins you want to add to that configuration file. And the third argument is output filter, which is which plugins on the output side you want to add to that configuration file. And then the debug argument just gives you a little bit more information as Telegraf runs. So you type that into your console. You should see an output that shows a bunch of data for the default Telegraf configuration, and then at the very end of it, you should see a section for inputs with the information that we’ve just added to our trig plugin. So it should have the one-line description, followed by the configuration variables.

Noah Crowley 00:30:10.841 Now that we have our configuration set up, we can start adding properties to our struct. Sorry, this is actually the wrong code example.

[silence]

Noah Crowley 00:30:49.520 Okay. I’m sorry about that. The slides are a little out of order. But let’s talk again about—all right. So we’re going to have to come back to that.

[silence]

Noah Crowley 00:31:32.164 Okay. So I can’t find the slide at the moment. But basically, we need to add a property to our struct that will actually hold the configuration variables itself. So right here, as we generated our configuration, part of the configuration is to have that amplitude in there. Our trig application is going to need to have a variable to store the state of the plugin between the collection intervals. So the amplitude is going to define how big the wait is, and the state is going to define where the point of the wave is at any given instance in time. I’ll come back to this. I think there’s a slide at the end with all the code; and I’ll point this out when we get to it.

Noah Crowley 00:32:22.827 So the next thing that we’re going to touch on is the Gather function. The Gather function is at the heart of the plugin. It’s run every time Telegraf collects metrics. So within Telegraf’s configuration, there’s something called the interval. The interval defines the amount of time that elapses between each one of the gather function. So by default, I think it’s up to 10 seconds. You can find it under the agent heading. And then under the agent heading, there will be a variable for interval 10 seconds, and you can change that to whatever you need it to be. There’s a second variable in there called flash interval, which is how often Telegraf writes that data out to the database, or the message dealer, or wherever you happen to be sending it. So one thing to keep in mind was that your collection interval should always be lower than your flash interval. So you want to be collecting data at 1-second intervals, but then potentially sending it up to the database at 5- or 10-second intervals, or something like that. Within the configuration, you can define different intervals for different plugins. But the most important part is that, every time that interval elapses, the telegraf.Accumulator.AddFields function needs to be called within your Gather function. So your Gather function is going to run, it’s going to do whatever work is required to gather that data, and then the telegraf.Accumulator function is to be called within that to add a new point for InfluxDB, or for whatever time series database you end up using.

Noah Crowley 00:34:12.845 So this is our Gather function for the Trig example. It has two main properties. We have the sinner function, which computes a sine wave based on, say, this X variable here, which is what I mentioned just a minute ago; which is what’s storing the state of the wave between individual points, and then multiplying that by the amplitude that we provided earlier to determine the height of the wave. We need to call each individual field to make sure that the measurements are tagged appropriately—sorry, the measurements are stored in the appropriate field. And then we need to define which tags we’re going to add to those fields. So once that’s done, we can increment X, which is that variable which is storing the state between intermediate calls. And we can call the accumulator and add that information, these various points, into the accumulator. So AddFields, we have the fields for the sine function, we have the field for the cosine function, and then we have any tags that we ended up defining. Okay, that, we can “return nil” because this function is returning an error, and everything seems to be working all right. If this were a plugin that were gathering data from the network, or from a file, or something like that, you might have to a little bit more error handling here, rather than just returning nil.

Noah Crowley 00:36:07.361 Okay. I apologize, this is the starting slide. Got a little out of order. But within the innit function, we’ve defined that X variable within our struct, and in the innit function we’re setting the initial state for the X variable to 0. So the sine wave will start with an initial value, 0, and every time the Gather function runs, we’ll increment that X variable by 1.0. And that will give us our sine wave. So now that all that’s written, we want to compile and test our plugin. The first thing that we’re going to do is rebuild. We can do that using the “make”. After that, we’ll generate a new sample config using the same command as we generated before. This time we’re going to store it in Telegraf.conf.test. Anything with .test at the end of it is ignored by default by the gitignore in the repo. So that’s a good place to store your configuration files as you’re testing them out, if you don’t want them to get committed. And then we’ll go ahead and launch Telegraf using the configuration file that we just generated, as well as the debug option. You should start seeing data being returned from that command. Telegraf will start up. The appropriate outputs and inputs will be loaded, and you’ll see the agent begin to collect data every 10 seconds. So the configuration will be flash interval of 10 seconds, and it should start running immediately. And after 10 seconds have passed, you should see the first entry here gather metrics from one input, and the amount of time that it took to gather this metrics.

Noah Crowley 00:38:13.091 The next thing to do, to make sure that the data has actually arrived, if you’re shipping this to Chronograf and InfluxDB, the you can open up Chronograf and go into the Data Explorer itself. So this is actually a live version of Chronograf, that I have running on my own machine. Within the Data Explorer, you can come over here and select the Telegraf database and the Trig plugin, and you’ll see those two fields that we just defined earlier while we were writing our Trig plugin. We can select data from both of those, and you can start seeing that a sine wave is coming in to InfluxDB. And we’re appropriately saving and visualizing that data. If you’re looking to actually submit those for inclusion in Telegraf as the open source project, you’re going to need to write some tests. Tests are really important. They guard against regressions or changes in the code that might start causing it to output different kinds of data and break other people’s integrations. So getting tests in there is really important. For out sine and cosine generation, they’re a little bit trivial. We’re simply going to generate some data and add it to the accumulator. But again, if you’re writing a more substantial plugin, testing is a really important part of it. And we do require that tests be added before you can contribute the function to our story.

Noah Crowley 00:40:00.047 A couple of other requirements, if you’re looking to submit a plugin: you’ll need a README.md file. This should provide some information about your plugin: how it works, what it’s doing, enough so that if someone’s visiting the repo and is interested in using your plugin, they can get a good idea of how it works. We’ll require your LICENSE file be in there. We’ll require you to sign our CLA, which is our contributor agreement, just so that the codes can be guaranteed to be open source, and other people can use it. And then it’s really important for us to get a sample of the output and input format for the plugin. The reason for this is that, a lot of the time, the engineers that are working on Telegraf might not be particularly familiar with the particular application, or the way that you’re collecting data. And in order to verify that the plugin is working as expected, we need some examples to compare that to. So really important that you help us out a little bit on that. We really want to grow these plugins and the community to a point which is way beyond what any of our engineers could handle on their own, and getting help from the community in terms of how the plugins are supposed to work, and how these third-party applications export data, is a really integral part of that. But if you’ve done those three things: if you’ve signed our CLA and you’ve written a plugin, then you are, at this point, ready to create a pull request. And we’ll take a look at it. You might have a few suggestions on how the plugin could be improved or made better. But you are on your way to having that be a part of our open source application.

Noah Crowley 00:41:58.881 So that’s it for me today. You can follow me on Twitter, @noahcrowley. You can shoot me an email, [email protected], if you have any questions. I think we have a bit of time now for those. I don’t know, Chris, if—

Chris Churilo 00:42:09.597 Yeah. So I made a mistake on slide 25, so should we go back there and just show them how to add the properties?

Noah Crowley 00:42:17.679 Sure. All right. So this is adding properties to the struct. Basically, what we need to do—let me see if I can just actually get the code open for this plugin. So again, this is the Telegraf repo. Definitely recommend checking it out. We’ve got a great contribution guide. But if you’re interested in the specifics, we’ll go into plugins, inputs. And we’ll take a look at out Trig plugin, which is actually already part of the repo.

[silence]

Noah Crowley 00:43:17.501 So what we’ve done here is, we’ve added a few properties to our struct. This is important because this is the data that is being used by our plugin. If it wasn’t added as properties to our struct, then we would have nowhere to store it; we have no information about it. So we’re adding two properties here. The first property is just X. That’s what I mentioned earlier, which is going to be used to record the state of the wave an any given point in time. And the second property called amplitude. So amplitude is going to be set using the data from the configuration variable. And the X is going to be set, initially, within our init function. And we’ll set it to 0 within the init function. And every time the Gather function runs, what we do is we increment that variable by 1. So again, if this were reading from disk, or something like that; if it were accessing resource over the network, you’d want to add any pertinent information variable to properties within the struct, and the interact with those properties from the Gather method, from the init method, or from wherever it’s appropriate.

Chris Churilo 00:44:47.295 Cool. You do have one question in the Q&A. You want to go ahead and read it out loud?

Noah Crowley 00:44:51.941 Sure. Would you mind reading it, because I’m not sure where it is?

Chris Churilo 00:45:05.045 Yeah. No problem. It’s no problem. So Davie Telesco asks: What would be the suggestion for getting data out of Node.js service?

Noah Crowley 00:45:15.740 So if that’s a service that you’ve written yourself, the best way to do it is actually to instrument the application as a developer. I’m not intimately familiar with Node.js. But with other web frameworks, and things like that—if you look at Flask, for example, Flask is a Python framework. It’s built on top of WSGI. And the best way to get metrics out of that is actually to instrument your application directly. So with the Flask framework, you can go in, you can add to middleware. And the middleware can start recording whatever information you feel is most valuable about the application. So if it’s a web application that you’re writing, you can start recording things like response times for endpoints, error rates, information like that. You can instrument it directly within the application, and then you can either send it to Telegraf via UDP, or you can expose it within your application as a Prometheus/metrics endpoint, and the Telegraf can read from, or something like that. So I believe Node is very similar in that way, and that you have middleware, and you can add your own code to the middleware. There might be some packages out there, particularly for Express, or something like that, for actually instrumenting the application. But once you have instrumented your application, and you have some code there to start generating the metrics that you want, you can do a whole bunch of things to get them in Telegraf: you can write them to a file, you can send them out via UDP, or you can expose them as a metrics endpoint that Telegraf can then go an check.

Noah Crowley 00:46:58.992 I think in terms of speed and performance, UDP is probably going to be the most performing way to get your data from that Node.js application to Telegraf. But it really depends on your environment, and how you’re working, as to whether or not that’s the best approach. But I’d say, generally, you want to instrument your application first, and then depending on your setup, you want to get those metrics from your instrumentation into Telegraf either via UDP or by writing to a structured log file, or by creating a metrics endpoint than exposing that.

Chris Churilo 00:47:37.672 So looks like you answered that problem perfect. So if there are any other questions, feel free to throw them in the Q&A chatter. Otherwise, if you have questions after, which often is the case, feel free to go to our community site, and Noah will be there with the rest of the team and be more than happy to answer any of your questions. Now you’re fairly new to the project. When you first approached Telegraf, based on that experience, do you have any tips that you might want to share with the people on this webinar? Things that, maybe, you did that you with you didn’t? Or maybe there would have been more efficient approaches?

Noah Crowley 00:48:21.789 I think the easiest thing to do with Telegraf is definitely to make sure you understand how your applications work, and to take a look and see whether there are plugins out there that are already configured for them. I definitely tried to go the hard route and start reading from some log files, initially, when there was actually already a plugin prepared for my application that I could just easily drop in and start getting metrics out immediately. I think in general, collection agents like this are pretty easy to get started with and get started using. So my recommendation is, really, don’t wait; put this into your system as quickly as possible. Drop in some plugins, start getting some data out of it, and the begin taking actions based on that data. I think it’s really hard to understand the systems that you’re using without actionable information about those systems. And once you start gathering data, you’ll get a better feel for how your systems work; you’ll get a better understanding of what you might need additionally, that you’re not currently collecting; and you can start making data-driven decisions about how you want to engineer your system in general.

Chris Churilo 00:49:46.222 I think that’s really good advice. You’re right, the more you can dig into it, then the better you will understand things. I think the other bit of advice that I would offer to our attendees today is that, don’t be shy about putting in your own contributions. And one of the things that we often find is, people will fault the project and make some changes, fix some bugs, then kinda just leave it at that. And it often turns out that there’s actually someone else that had already done that. So you know, it’s always good to bring back those changes to the main branch and share the work amongst everyone else. I can’t tell you how many times that I’ve introduced people to each other that have done the same changes. You get a bit of a chuckle, but then you realize, “It could have been a lot more efficient if we just all put them into the main product.” And as Noah has mentioned, it’s a lot of fun. It’s a great way to get started. So we encourage you guys to get started. If you do get started and you do run into some issues, Noah is actually going to be holding workshops at InfluxDays in New York. So if you happen to be in town, I would recommend that you pop over and get to know him face-to-face. And then really try to stamp him with some really hard questions [laughter]. I’d really love to do that for a change. All right. We’ll wait just for a few more minutes. If there are any questions, put them in the chatter Q&A. Just as a reminder, this session was recorded. So we’ll post it later on today. And if you happen to have a really cool project that, maybe, you want to even share with us, let us know. Noah is always looking for people to join his meetup in New York City, so we’d love to ask you to come in and present your solution as well.

Noah Crowley 00:51:36.462 Yeah. Absolutely. Oh, sorry, I didn’t—

Chris Churilo 00:51:40.208 No, no. We’ll just wait just a couple of seconds, maybe two more minutes.

Noah Crowley 00:51:42.919 Sure. Yeah. I wanted to add to what Chris said about contributing and getting any changes that you make back into the open source project. I ran into a small bug while I was working with one of various parsers that we have out there. And my first reaction was to, sort of, reach out to the team and say, “Hey, there’s a bug,” and then kinda walk away. And then the response that I got from them was, “Hey, we’d really appreciate a pull request.” And so I started digging into the code and found that I was able to come up with a fix pretty easily, and put it out there, and submit a pull request. And it got reviewed by our engineers, who actually gave me a bunch of really good advice about how I can make it better. And then got the code merged into the master branch. So that was really fulfilling. It was a great learning experience. I got some advice from them about how to write my code and how to make it better. And I got to make a contribution to the product and make it work better than it worked before. So definitely, don’t stop at just making your changes and keeping them to yourself; engage with us, engage with our team. And I think you’ll find that it’s a really worthwhile experience.

Chris Churilo 00:53:04.540 That is a great story, great experience that you shared, Noah. And I really appreciate it. Fantastic job on the training, Noah. I think everybody, including myself, learned a bunch. I apologize about the little mistake that I made [laughter], the properties page; I have fixed it. I will also ask Noah, if you don’t mind, looking at the deck one more time. Once I get your blessing, I will upload it to SlideShare for you guys to take another look at, as well as the link to the recorded webinar for today’s training. So thanks, again, for joining us today. And we look forward to meeting you again at our training next week, next Thursday, or at one of our many events. Thanks, everyone, and have a great day.