3. Installation of InfluxEnterprise (cluster)
In this webinar, you will learn how to install an InfluxEnterprise cluster in your own environment.
Watch the Webinar
Watch the webinar “Installation of InfluxEnterprise (cluster)” by clicking on the download button on the right. This will open the recording.
Transcript
Here is an unedited transcript of the webinar “Installation of InfluxEnterprise (cluster).” This is provided for those who prefer to read than watch the webinar. Please note that the transcript is raw. We apologize for any transcribing errors.
Speakers:
• Chris Churilo: Director Product Marketing, InfluxData
• Michael DeSa: Software Engineer, InfluxData
Michael DeSa 00:01.781 All right. Thank you very much, Chris. So as Chris mentioned, today we’re going to go over the TICK stack and kind of InfluxEnterprise. So we’re going to talk about what InfluxEnterprise is, what InfluxCloud is, what the TICK stack is, how you determine which one of them is going to suit your needs. And then we’re going to get into what end InfluxDB cluster is. So the clustering is a part of InfluxEnterprise, like stationed between the two. In the process, we’ll talk a little bit about meta nodes, and data nodes, and how all these things kind of come together. We’ll go briefly over what kind of hardware you should actually run a cluster on, and some minimum requirements that we set there. And then, finally, we’ll do a short demo of an installation of InfluxEnterprise. So just in case that you don’t know what is InfluxData, InfluxData is a modern engine for metrics and events, and this kind of came down, or it came out of some new workload requirements that we started noticing, which is, there’s sensors on everything, we want to monitor everything, and so we need kind of a new system to deal with these high-volume writes. We wanted something that could handle both regular and irregular data, so events versus metrics. And we wanted a lot of logic about data retention, particularly based off of time.
Michael DeSa 01:38.044 We also wanted to have time-based series, so giving the average over the last day, things like this. And then, we wanted something that was scalable and available. So available in some sort of clustered fashion, not just sort of federated thing, where you have to do a bunch of work to have some high availability. InfluxData has three product offerings. There is InfluxCloud, which is a hosted version of InfluxEnterprise. So the main idea here is, it’s all the Enterprise features, and we’ll talk about what those are in just a moment, except we will host the instance for you. There’s InfluxEnterprise if you want to run that, what we have in our Cloud in your own hardware, on premises. And then, finally, if you’re sort of just getting started, or you’ve already have something built out, you can use just the TICK Stack, which is 100% open source. So this brings me to the question here which is, what is InfluxEnterprise? So InfluxEnterprise is the full sort of InfluxData TICK stack. So you get Telegraf, you get InfluxDB, you get Chronograf and you get Kapacitor but whether some additional features. Additional features namely that you get a high availability of each of the components of the TICK Stack. So you can get a highly available version of InfluxDB or a Kapacitor, Chronograf soon enough. So you get kind of this whole high availability thing and then kind of possible to have something that is a little bit more sought of scale out performance. You get more sought of cluster management tools that come along with Chronograf. So Chronograf can know if you are using InfluxEnterprise, and we can see that this feature is available, and you kind of just get to use them out of the box. We have more advance of backup and control scenarios or procedures, and you get sought of access control so you can do things like restrict what users can see, which types of information, make read or write only type users to the thing from the database.
Michael DeSa 04:08.251 So those are sort of the distinction between Open-source and Enterprise. Enterprise is really just clustering so high availability and scalability, and advanced sort of management features that are sought of nice in some sort of enterprise content. Now InfluxCloud is the entire InfluxEnterprise feature but packaged together as a service that you can purchase that you don’t have to host yourself, it’s just an end-point that you call out to. So just to give you a little bit sought of different representation of what it is that we offer, just kind of the way we think about what InfluxData and the TICK Stack is a way you can accumulate data, analyze data and take actions on data. So there’s not plenty of things you can get in there, you get data normalization, correlation of various data points, pattern matching, time series functions over some time ranges, an anomaly that will really allow you to sort of take massive amounts of data and sort of do something with them, or visualize it, or act on it. And just to give the sort of final, sort of, distinction between what is open source and what is closed source. Open-source core is Telegraf, InfluxDB, Chronograf, and Kapacitor, they’re sort of base-line versions of them. And then you have the Enterprise features which are clustering, access control, high availability and some other security features.
Michael DeSa 05:45.357 So now we are going to talk a little bit about the InfluxEnterprise Architecture, an InfluxEnterprise installation consists of three separate software sort of processes. You can also throw a Kapacitor in here as well. Although, the Enterprise version of Kapacitor is still kind of in its works at the moment. But to have an Enterprise installation, what you would do is get some meta nodes—or sorry, data nodes, which are nodes that store the actual time-series data. You have meta nodes, that keep kind of some—stay consistent across the cluster. And then, you have Chronograf, which you use kind of as the UI layer to your cluster. And so to run an InfluxDB cluster, all you really need is the meta and data nodes, and that way you wouldn’t have the entire sort of feature suite necessarily available to you, just the scaling and clustering.
Michael DeSa 06:42.941 So meta nodes are nodes that keep meta-state consistent across the cluster. And so things that we store in a meta-state are things like the users, the databases, the continuous queries, the retention policies, the shard locations, servers, and whatnot. So the important thing to make a note of here is that meta nodes do not actually store any time-series data. They just store the extra information that needs to be in a strongly consistent state across the cluster. So what do I mean by strongly consistent? I mean that you could query any one of the meta nodes in your cluster, and you would get the exact same response. So if I say, “Show users,” or, “Show databases,” or, “Show continuous queries,” every single meta node would give you the exact same answer. The same thing would not be true for the data nodes. The data nodes are an eventually consistent state. So it is possible to query two separate data nodes and get slightly different answers. In order to have a highly available cluster, you must have at least three meta nodes. And then a small little note that I make here is, meta nodes don’t typically need a large number of resources. We’ll talk about what kind of resources you can anticipate in more detail in just a moment.
Michael DeSa 08:19.256 So the next type of nodes that there are, are data nodes. Data nodes store the actual time-series data, so the measurements, and fields, and tags, all of that is actually stored on the data nodes. And data nodes are actually the nodes that respond to and receive queries. And so they don’t participate in sort of the strong consensus, or things like users, and continuous queries. If they ever have to interact with that kind of information, they’ll call out to the meta-service. In order to have a highly available cluster, you must have at least two data nodes. And typically, data nodes do need a large number of rate sources. So this is where you want to put your thousand IOP SSDs, and your 30 gigs of ram, or 128 gigs of RAM 16 cores, these are the nodes you really want to put that type of processing onto. Finally, the last piece that sort of comes into play here is Chronograf. And Chronograf is used to do, kind of user management, it also has a number of pre-built dashboards. You can do some custom dashboarding you can do database management and retention policy management, all the kind of UI features that you would want out of a cluster, you can do all of that through Chronograf. And as mentioned, Chronograf can detect if you’re using an Enterprise version of the software and appropriately unlock a certain number of features that may not have been previously been available to you.
Michael DeSa 10:03.171 So the final step in a sort of complete Enterprise, InfluxEnterprise installation is an InfluxEnterprise monitor or something that is actually monitoring your InfluxEnterprise cluster. And so we have this little bit of a visualization off to the right here, where you have your meta nodes, your data nodes, you have Telegraf on each of those nodes and then a separate InfluxDB instance that is consuming that data. And maybe a Chronograf or a Telegraf, or a Kapacitor sort of hooked up to that, alerting you to let you know if there’s anything wrong with the cluster itself. So the idea here is that you have a separate InfluxDB instance, Telegraf are reporting to each of the nodes in your cluster, all of them reporting to that InfluxDB instance, and then maybe Chronograf to do some visualization or dashboarding of that instance. And this is something that we strongly recommend for any Enterprise, InfluxEnterprise installation, just so you have something monitor the thing that is monitoring your cluster.
Michael DeSa 11:16.869 So we’ve got a question and a Q&A, which is: “Can you give a quick explanation of Telegraf?” Yeah, so Telegraf is our metrics collector so it’s the way that you generate data. So there’s two main fashions that you can operate Telegraf in, is you can put it on every node in your cluster, or every node in your server farm and have it report things like the amount of memory that’s being used, the amount of discs that’s being used, the amount of CPU that is being used. It will report all of that information back to your cluster where you can sort of monitor each of the nodes in the cluster independently. And that’s one way you can use it, the other way you can use it is to monitor things like monitor your MongoDB instance, or your Postgres instance. And have it collect some statistics about that and write it into InfluxDB. So like a system monitoring agent, exactly. So it’s much like Collectd—is a similar kind of analogy that it can draw to. So some general cluster advice is you want to put a load bouncer in front of your data nodes so that queries and writes are spread across each of the nodes in the cluster. And we’re going to talk about higher replication factors and how that comes into play in just a moment.
Michael DeSa 12:43.696 And as I was saying, there’s some general hardware recommendations that we can have. Just high-level recommendations is if you are going to be operating an InfluxEnterprise cluster you want to have a good experience. Your data nodes should really have a minimum of four CPU cores, preferably eight, but four is probably sufficient. And you want to have a minimum of 16 gigabytes of RAM. And the reason for that is you just want to give yourself enough operating capacity to really grow into your cluster. You could probably operate with less RAM. It’s just in the past that we’ve observed that things typically work the best when you have a little bit more RAM. And then for meta nodes you really don’t need that many resources. In fact, you can just have one CPU core and two gigs of RAM is more than sufficient. And the important thing to note here is that you need to have an odd number of meta nodes. So the particular consensus algorithm that the meta nodes use is Raft. And Raft relies on their being an odd number of Raft peers. And so another thing to make a note of here is—at InfluxData, we don’t charge—or the pricing is not reflected on the number of meta nodes you have but rather the number of data nodes that you have, and sort of how large those data nodes are. So if you do have more questions about data nodes and pricing, please reach out to our sales team. I’m sure they’d be happy to answer any questions that you have.
Michael DeSa 14:28.997 So the next thing I’d like to talk about very briefly is I’m going to be trying to work to replication factors, or how many copies of data exist in a cluster. But to get there I want to talk briefly about shards, technically shard groups. So in InfluxDB all data in the database is sharded by time. So the reason why we do this is that we can very easily expire blocks of time, it’s just removing a file on discs. So if you want to, say, get rid of any data that’s older than a month, it’s very easy. So all of the data is sharded up into time. And we kind of get to this replication factor, which is how many copies of a particular shard should exist in a cluster? So if you want to have some sort of resiliency to, say, a node going down, you want to have a very high replication factor. So your replication factor can never be higher than the number of data nodes in the cluster. So if I have a two-data node cluster like I do in this example here, the highest replication factor I can have is two. And having a replication factor of two simply means that there are two copies of that shard in the cluster. So in this example here, we can see that shard of one exists on both data node one and data two. Now this is the contrast this with I can have a replication factor of one, in which case, there would only ever be one copy of a shard in a cluster. To make things a little bit more resilient, we actually do fragment shards up a bit more. So if you happen to lose a node, you don’t lose all of the data that is for a particular time range. You just lose some of it just based off of how we split things up in a cluster.
Michael DeSa 16:32.762 So there is some things to kind of keep in mind about replication factors. And let me try—I have a slide. I do have a slide about that. So how do you utilize replication factors? How do I monitor or create a retention policy that has a replication factor? So you can do it in kind of two ways. One, you can create a new database. You can say create database my DB with replication. And you get the replication that you’d like to have. Or alternatively, you can say create a retention policy my RP on my DB with some duration, and the replication that you’d like to have. You could actually get a little bit more—if suppose you already had a database and you wanted to modify their replication factor, you can do that as well by issuing an alter retention policy command. If that interests you, I’d recommend checking out the documentation.
Michael DeSa 17:29.859 So one thing to keep in mind is that higher replication factors will result in lower query latency but higher write latency. So just to give you a little bit of an idea and sort of reason about this is when I’m writing a data—when I’m writing data, I have a high replication factor meaning that all nodes must receive all of the data. You’re basically doing write application, right? It’s write to one location, it has the write to the other nodes in the cluster. And so this kind of just increases the amount of time that any one write will take place. There’s some things you can do to control the level of guarantee of the requests. So if I’d like to have it verified that that data point was written to each of the data nodes, I can issue a write query with a consistency all. And that will make sure that each node has accepted the write before returning an accepted write to the client. Or if I can be a little bit more lax with things, there’s things like consistency any of consistency one, or a quorum where maybe a majority of the nodes needs to acknowledge a write before it’s accepted. So typically that’s why having a high replication factor will probably decrease the write throughput of your cluster. So if you do have very, very performance-critical instance, having a higher replication factor will lower your write latency. And what do I mean by lower? It is definitely not in order of magnitude, I would say. So an OSS instance could probably take on the order of 700,000 writes per second. If you had a replication factor of two in a cluster with two data nodes, I would say that you might see that drop to something you may say like 450,000 rather than that 700,000 that I previously stated whereas if you had singularly replicated, it would probably be around 750,000, possibly even higher.
Michael DeSa 19:45.306 So to contrast that, to know why would anybody ever want to do that then, well, you get higher resiliency but you also get the added benefit of more data nodes in the cluster can respond to a query. So if I have, say, 100 users, that all querying the database, being able to spread that load across two data nodes or three data nodes or four, whatever it be, and have each of them respond to queries independently—that’s super beneficial just because it doesn’t need to call out to the other system and it can handle all of the load that’s coming from—it just sort of—you have more resources to fulfill queries. So a higher [inaudible] replication factors lower query leniency but increase write leniency, it’s the sort of the moral of the story here. So here we are going to have sort of a quick walk through of how to set up a two data-node cluster, just going to do a high-level sort of overview. And then I’m going to switch into a screen share and just kind of go through the process of setting up one myself. My hope is that nothing will go wrong, I’ve kind of staged some incidences little bit but I haven’t actually gone through and run things to make sure they work correctly. Things typically work correctly but whenever you’re doing things on the fly there’s always a possibility that some thing could go wrong. So we’re going to hope that nothing goes wrong today.
Michael DeSa 21:19.469 So the process that you go through to set up a two data-node cluster is to first and foremost go get a license key. So license key you can get a two week, I believe, free trial which we can send based off of circumstances where you can test out a cluster and get a feel for the experience and work with the tool before, say, moving on further in the sales process. So you can do that by going through our portal and we’ll kind of walk you through the process of getting your license key. Once you’ve got a license key you can—you want to go through it and start five machines. So two meta instances and two—or three meta instances and two data instances. And the process for the meta instances very straightforward is, download the meta instances, download the package. configure the node. Start the node, join the node. And then similarly for the data, it’s download the package, configure the nodes, start the nodes, add the nodes. And so we’ll see what that looks like in just a moment here. Obviously, if you’re using things like Kubernetes or Docker Swarm or any number of other solutions, this process is a little bit easier. And I believe we even have int scripts that you can just use out of the box to set things up, though don’t quote me on that. I’m not entirely certain. So I’m going to start sharing my screen and we’ll go through that process. And share screen. All right, so here I have my instances. I’ve got three data—or three meta, so the three over here. Let me increase my font just so everybody can see. All right.
Chris Churilo 23:25.484 Yeah, you might want to do it a little bit more.
Michael DeSa 23:26.947 Little bit more? All right, how’s that Chris?
Chris Churilo 23:37.013 It looks tiny for me so I’m going to ask someone else to verify.
Michael DeSa 23:44.257 How about now?
Chris Churilo 23:45.790 Yeah, that’s much better.
Michael DeSa 23:46.661 All right, so I’ll go through the process for the most part on this screen and I’ll switch over to the other one. So the first thing that we do is to download the package. You can get those available—links for that available on our website. And as I mentioned you want to start the service. So pseudo-service, you want to configure the nodes beforehand so we’re going to go into Etsy, InfluxDB, InfluxDB meta conf. And so the main thing here is you just want to assign the host name. You don’t have to do this, you can just use the node’s IP address if you please. Every node in the cluster must obviously be addressable by all of the other nodes. And you want to have a couple ports open, namely 8091, and 8088 on the meta nodes, and I think there’s 8089. You want to have all of those available on the meta nodes or addressable by each of the other meta nodes in the cluster. And then you want to set the host name, and then you want to set the license key. So, in this case, I have an empty license key but I happen to have a special version of the software that doesn’t require me to have the license key. And so once we configure our instance—and again the things you only really need to do are set the host name and the license key or the license path if you want to have a file. And then you kind of just move on from there. Important thing to keep in mind here is you really don’t need a whole lot of configuration at this point.
Michael DeSa 25:32.509 So first thing I’ll do is pseudo service InfluxDB meta and start. And that’s been happening to me all day but I can verify that things are indeed running by checking some logs. There we go. And we can see things appear to mostly be working and executing, but there’s no leader and so we’re going to join the node to the cluster. So to do that we say, influxd, influxd, ctl, join. Failed to join cluster, so we’re going to say, H. And we’re going to say—we’re going to do this one here. All right, so if we do—so what I’ve done here is I’ve added the meta node to the cluster. And then, I can do an influxd, ctl, show, to see the various meta nodes of cluster. So what I’m going to do next is start each of the other services. So pseudo service, start, InfluxDB meta, start, and pseudo service, InfluxDB, and hit start. And I’m going to do the same exact—I’ll give it a moment to do it for the rest of the meta nodes or data nodes. So going to add the other data nodes here—my apologies, for meta nodes. And then, I can reissue the influxd, ctl, show command. And I can see that all the meta nodes here are part of the cluster, or at least this node believes they’re all part of the cluster with the other nodes. Also believe that they are part of the cluster. I can see the version of the software and I can currently see there’s nodes in and nodes running.
Michael DeSa 27:46.378 So now we’re going to move on to a data node. And we’re going to do—make it bigger again so everyone can see. And we’re going to do vim, etc, influxdb. All right. And so again, the only thing we really need to touch here is the host name. So I just fill it out with something that is addressable from the other nodes. So you want to have supports 8088, and 8086 available. And then, the next thing you’re going to want to do is get a license key and fill this in. And you can do it through an environment variable or by putting it into the configuration file, whichever is easier for you. Once you’ve done that, we want to start the service, service InfluxDB, start. And we’ll do the same one here, pseudo service, InfluxDB, start. And then, we’re going to go over here. And we’re going to add data on 8088. Oops. We want to add data. So what we’re going to say is Influxd, ctl, add-data, data-1, 8088. And then, we’re going to do the same thing for data 22. And now we can issue an influxd, ctl, show. And we can see our entire sort of InfluxDB cluster. So this is great.
Michael DeSa 29:24.627 The next thing we can do is we want to actually start Chronograf. So we’re going to go to pseudo service, influxd, chronograf, start, and then I’m going to go actually look at that node in my browser. Whoops. And Chronograf by default runs on 8888. Here we’re going to connect to the thing that is actual on local host so that’s there, and we’ll call this Influx one. And there’s actually not currently a Telegraf running on there, but sure, that’s fine. So from here I can create some databases. So if I want to—let’s see, do some administration I can—cannot communicate with the server. Let’s see. Oh, it’s says that it’s fine. All right. So I just made a database called X. If I want to view that I can do show databases. And I can see that database. Let’s say, “Use X.” I want to say, “Insert my measurement.” And say, “No, value equals 10.” And then I can start doing things like, “Show shards.” And we can start to see some information here. Particularly we can see that the database X has two owners so that’s reflected in its retention policy. So if I say, “Show retention policies on X.” Retention. That’s just because I can’t spell or type. If I say, “Show retention policies on X, X being my database, I can see that there is a retention policy called auto gen that has a duration of zero which is just an alias for infinity. That has a shard group duration of, I want to say, seven days, and it has a replication of two, and is currently the default retention policy. So I can see that the replication by looking here at the number of owners which is the number of owners that I have of that shard in my cluster. So as I mentioned previously we can also do something where we had a singularly replicated database. So what we can do in that case is we’re going to issue a create database MyDB with replication of one. MyDB. I, for the life of me, cannot type and I apologize for that.
Michael DeSa 33:04.357 So it’s create database MyDB with replication one. And so, now, if we say use MyDB, and we’re going to insert some data, let’s insert MyMeasurement. And then let’s issue that show retention policies on MyDB, and we can see that it’s got a replication of one. And now if I issue a show shards, we can see from MyDB, there’s just a single owner. Which means that that individual shard had—there’s one copy of the individual shard in the cluster whereas in this case here, there’s the shard ID-2. And it has two copies of it in the cluster, one living in each of owners four and five. If I want to get in and do some other things, I can do som user monogram; I can do things like create user, say, MyUser with password, and I can ascribe various rules and permissions. Oops. I wonder why that keeps coming up? It appears that there’s something going wrong, I think that maybe a connection issue. Former issue for doing these type of things going live. Self-connected. So that’s sort of a number of the features that we have in the clustering. You can obviously do a lot more. You can do things like as create dashboards, certain roles, to certain users and describe the best permissions that they have which tags and fields they are allowed to see. For the time being now, I think now that mostly covers a sort of base demo of InfluxEnterprise and so I am going to close up on my screen share and then answer any questions that may have come up in the chat. So thank you for your time and I’ll spend some time going through the questions. It looks like it says here, “Can you talk a bit more about the name resolution requirements host files or DNS, Docks require a hostfile entities.” DNS is fine, you don’t need host files. The reason why I’ve done it with Etsy host is just to make things a little bit simple when you’re doing things by hand. And just so you don’t have to paste IP addresses everywhere.
Michael DeSa 35:42.300 So the next question is, does each of the meta nodes not require the IP and/or host name of the other nodes added to the configuration file? You don’t need to do that, yeah. So in fact, we actually—a long time ago we used to have something similar, where you needed to have every other node in there, but you don’t need that in the configuration file at all. All you do is—there’s sort of two commands in influxd, ctl, one of them is influxd, ctl, join. And the other one is add. If you are on a node and you want to join another cluster, you can call out to say, “I’d like to join this cluster,” and give any node in the clusters, address and it will figure everything out from there. Alternatively, you can make a request to one event, the meta nodes and tell it to add a new node to cluster. So you do not have to have a complete list of everything you ever remember in a conflict file anyway.
Michael DeSa 36:58.090 In addition to the meta and data nodes, there’s a web node, can this be considered an optional component?” Yes, it’s 100% an optional component. So the main reason why we see people use Chronograf is, it just gives you kind of a nice UI layer for doing things like managing users and say permissions of various things that go on in the database. So it just really—the way we think about it is you have the InfluxDB cluster which kind of stands on its own, but there’s a number of operations that are a little bit sophisticated to get into, managing users, enrolls and permissions, and you can do all of that very easily through Chronograf for instance, what node that we’re talking about. So it’s 100% optional, one of the benefits though is, you can get some sort of dashboards right out of the bat for free. But if you’re actually using your Grafana or anything of that sort, you can kind of bypass that.
[silence]
Michael DeSa 38:15.166 “During your download, you installed starting Chronograf as a separate component. During test installation, it was included in the web council. Any advantages to a separate install?” Not entirely sure I understand the question during—so we previously had another thing called the enterprise web or enterprise web UI. Due to some sort of issues that we’ve had with it, we’ve moved away from it. We haven’t 100% retired it yet, but we’re very close to it. And we built the exact same functionality into Chronograf that you can use, to sort of do the same kind of user management and operations. They’re going to be tested, and installation is included in the web console. Oh, perfect. Glad to hear it. So the next question is, “How do you add a load balancer for the data nodes?” Really, any load balancer will do. I have seen people roll their own, sort of, very small reverse proxy in front of the data nodes. Hot proxy, or J proxy, or engine X. Really, any load balancer is sufficient. And typically what we do in our cloud instance is we have a service that’s pinging each of the data nodes. And if it notices that one of them isn’t responding to a ping it’ll actually remove it from the load balancer. So it’s kind of—it’s opaque to you whether or not—all of your requests should mostly be successful. Hopefully, you can get in, in case that one node happens to go down.
Michael DeSa 40:10.843 “So which cloud provider is your main service with?” We currently work on AWS, is what we’re built on for our cloud service. “You spoke about using Telegraf to monitor your cluster, are there key metrics that should be used to track cluster or node health?” Great question. If you use Chronograf and you have Telegraf on each of those nodes and you enable the InfluxDB plugin, there’s sort of pre-built dashboards that will come out of the box, that will track all of that information for you. So you don’t need to figure out which ones are off. If there’s anything sort of custom that you’d like to add in, you can obviously do that. And we’re currently working on if you want to thread your capacitor into the mix, you can even get alerts if one of your nodes happens to be going down. But, that’s still to come. “Are there supported Docker containers for meta and data nodes?” There should be. I would be surprised if there are not. Let me check in more detail though. I actually don’t know that answer—I am fairly certain there are Docker containers for the meta and data nodes. At the very least we run our cloud instances, our cloud-hosted versions of it, in Docker containers and we are using it. So at the very least they must exist somewhere.
Michael DeSa 41:42.847 They probably would not be in the docks. I would highly recommend reaching out to our sales team and they’ll be able to assist you with that process of where they are. I myself actually don’t know where they are. So if you send us an email, we’ll put you in touch with your sales team. As part of your trial, we can get you a docking container.
Chris Churilo 42:07.263 Yeah, Mark, I’ll go ahead and email you and Lauren to get you guys connected so we can get that moving. I also encourage everyone to put their questions into Community. One of our big contributors at Community is Jack Zampolin, and he’s a huge Docker fan and he might actually know that answer.
Michael DeSa 42:28.176 Yeah, Jack would very likely know that answer.
[silence]
Michael DeSa 42:45.626 All right, any more questions? “Where are a cluster of log files and what additional information is available comparing to single instance logs?” Not entirely sure I understand the question. So log files depending on what version of Ubuntu or Red Hat or whatever OS that you’re on, you can pull it from the journal—sorry, my apologies, throat’s acting up here a little bit. You can pull them from Journal D or I think there’s an option where you can specify a log file that you’d like it to be written to by default. That is, var, log, InfluxDB and then the name of the service that is running. “The thing I don’t understand is what additional information is available comparing to single instance logs.” I don’t know. What do you mean by single instance logs?
[silence]
Michael DeSa 44:12.387 Yeah, great question is, “Can you speak to the detail about disk requirements, shared storage and local storage subject for data nodes?” Yes, so typically we—sorry. We prefer local storage if possible. You want to have about 500 to 1,000 IOPS on your disk depending on what kind of workflow you do. You could end up using a lot less disc, you can end up using a lot more. We usually don’t see things go above a 1,000 IOPS. We’ve seen people try to do network-attached storage before and typically that ends with a decent amount of sadness. But if you want to run EBS volumes, that’s usually fine, we don’t see any issues there provided that the IOPS are in some kind of reasonable range. Does that answer your question sufficiently?
[silence]
Michael DeSa 45:18.130 To get maybe even a little bit more detail of how big should the disc be? Will surges an idea for production, yeah. Yeah. So of course, then have an EBS volume for sans, yeah. Best practices for sans. What do you mean by best practices? Maybe I’m not familiar enough to speak intelligently about this.
[silence]
Michael DeSa 46:19.919 Do you need high read/writes? So typically we see about 500 IOPS is where things hang out. We really don’t do any testing on specific types of disk formats, though we’ve sort of been moving in that direction. Typically, you have the initial, sort of, pend only that goes on to a wall and that should be read-optimized—I’m sorry, write-optimized. And then things are becoming queries, that is the date directory, you want to do something that is read optimized. We haven’t done specific things around what would be best. If you email in, we can get you a bit more information somebody that knows a bit more about and service and disc things. This is really outside of my realm of expertise. So definitely email, and I’ll ask around the Community we can get you a better answer for that. Yeah. I agree. The main reason why it’s probably been relatively light is we haven’t done a whole lot of intensive testing around and that ourselves. Perfect. So the other question is how are CQ workloads distributed among the data nodes. So this is a great question. At any one time, one data node will be answering all of the continuous queries. This is not necessarily ideal. So if you have a very high continuous query workload, it is a little bit—it can put a lot excess load on an instance. So if you do have a high CQ workload, you have a lot of data on sampling, we highly recommend using Kapacitor. So Kapacitor, you can translate continuous queries directly to Kapacitor tasks. You can do that in either a batch or a stream mode. So if you do a stream, it will take even more load off of InfluxDB. If you’re doing batch, it will still place that load on InfluxDB, but you can use it kind of—the load will be spread across the nodes a little bit more. Hopefully that is a sufficient answer.
Michael DeSa 48:47.631 So another question is, “Can nodes/clusters be configured to send alerts once something is wrong, like low disc space or cluster error, etc.?” Yes, they can. Currently, that would require a little bit more external tooling. You could use Kapacitor, and that separate monitoring cluster, or monitoring instance that we talked about to do it. So that is all you could do. We’re working on trying to get to a place where you can just get that out of the box without having to do a bunch of extra configuration and do work yourself.
[silence]
Michael DeSa 49:47.260 So the final question is, “Do you guys offer training courses on Influx, in general?” Yes, we do. So there’s sort of two ways that we can go about that. We have these webinars or our trainings—I believe, every week, maybe even twice a week—where we cover some various topics. But if you’d like more in-depth, more focused to you, or more interaction, we also do professional services, training engagements, where, more likely than not, I will be hanging out with you for a number of hours, and we’ll kind of go through all of the components of InfluxData, or whichever ones are particularly interesting to you. And we’ll cover sort of everything about Time Series. So we have the ones that we offer for free, and then we have professional services and engagements that we can do either in-person or online, whichever is more preferable to you. I’m sure Chris can even speak to this more than I can.
Chris Churilo 50:50.374 Yeah. Mark, great question. So we just pulled the link on our website. So we have a Getting Started series, which is seven webinars that we’d put on. And as—let me just pull this up for you guys—and it starts just right from the beginning; What is Time Series, all the way to a lot more advanced topics. And what we do is we try to rotate through the Getting Started series of—it’s about for the course of a couple of months. And then we insert some newer topics that people ask us to cover. So we get into some really advanced details about Kapacitor; we just did one on writing a Telegraf plugin. And all the videos are listed at that URL that I just gave to everybody. And if there are other topics that you want, just shoot me an email, and we’re always happy to add those in there.
Chris Churilo 51:56.996 All right. Great session. We really love it when people have a lot of questions. If we didn’t get your question answered properly, please go into our Community site, and post your questions there. And there’s a whole host of people that will be sure to answer questions, especially about the questions about Docker, and also storage. We can probably get those answered, even though we don’t have the appropriate documentation just yet. I will post this webinar before the end of the day. And join us again next week. And we always have webinar on Tuesday, and a training on Thursday. And then I always post them on our website for everyone to take another listen to, at their convenience. So thanks again, Michael. It’s always awesome to have you. And thanks everybody for your really great questions. And we will see you again soon. Bye-bye.