InfluxData Blog - Adam Anthony

Welcoming InfluxData's 2020 Interns

Adam Anthony (InfluxData) — Tue, 02 Jun 2020 07:00:00 -0700

InfluxData has a history of hiring talented students into summer internships. We started hiring interns for software development in 2016 when we decided to hire some very talented students who were referred by full-time team members. At the time, we had just two interns but in sequential years, we gradually increased the number to 5 in 2019. We also took the leap into hiring and managing remote interns. You can read more about this in my previous blog series here. While that previous series focused primarily on how to create a valuable experience for individual hires, this post focuses on macro concepts: how to start and fulfill a formal internship hiring program.

InfluxData crossed a total employee count of 200 in early 2020, versus being just over 100 in 2019. Accompanying this rapid growth, we decided to change our mindset around summer internships. Instead of taking the approach of “we hire interns,” we have shifted our mindset to “we have an internship program.” This subtle difference was important to us, because we see many benefits to this formality. First, a formal program is…formal. This means it has clear processes and expectations. How did the interns perform? Will any of them convert to full-time hires? How did we perform last year compared to this year? A well-designed internship program should offer us answers to these questions, as well as many other tangible benefits.

For example, the program can also be designed — from the start — for growth. We implemented a loosely-coupled internship program in which 5 engineering managers, working closely with our recruiting manager, collaborated to draft a shared job description and hiring plan. The job description was generalized to briefly identify each hiring team, and the hiring plan was a truncated version of our plan for full-time engineers.

Once established, each manager ran their own pipeline to hire 2 interns for their team for a total of 10 hires into a single internship cohort. In the future, each manager could hire more than 2 interns if management capacity and budgets allow it. More practically, we plan to pull more teams into the program in the future to grow the program at a manageable pace.

The program was also designed for consistency. By sharing a hiring plan, we had some assurance that the standard for hiring interns was similar across teams even though we had different technical needs (e.g. front-end developers vs. platform engineers). By establishing a separate-but-together organization for hiring, we also established the idea that managers should be collaborating centrally for some outcomes. For example, we are now collaborating on onboarding plans so that each intern has a consistent and positive first impression of our company.

A well-designed program also gives us strength in numbers when solving problems. With the obvious challenges of the COVID-19 pandemic, we have been faced with some unexpected planning for interns. Firstly, some of our teams were planning to colocate their interns in our SFO headquarters. With our offices likely to be closed for most or all of the summer months, it was valuable for those managers to have a channel open with peers that were already planning to run a remote team. We were able to quickly communicate and adapt our plans to their team with the outcome that we could confidently assure our interns that we were prepared to adapt to the social changes and had every intention of running the internship program as planned. We are also working together to brainstorm different ways of creating shared social experiences for interns since meeting peers, making friends, and expanding social networks is an important part of the intern’s experience.

Finally, an intentionally designed internship program has clear, measurable goals. The core principles of our internship program are learning and growth. This applies to both the interns as well as the full-time employees involved in the program. We aim to put our interns — as well as their managers, mentors and coworkers — in a position to safely take risks, learn as much as they can, and contribute value where they are able.

For example, as a recently promoted hiring manager, I was offered the opportunity to be the lead manager on the hiring plan in order to gain experience in team-building. Because our goals are centered on learning and growth, I was given ample autonomy to try and fail a few times with minimal risk of consequences. In the end, what I was able to accomplish was what I’ve described in this post. And in the future, when I have an opportunity to expand our full-time ranks, I will be able to leverage what I learned this year for building out a talented full-time team. I am excited for our interns’ arrival in June, when I can enjoy the privilege of being a part of their own career growth.

Lessons Learned from Our Engineering Internships Program: Part 3

Adam Anthony (InfluxData) — Fri, 08 Nov 2019 07:00:45 -0700

In 2019, InfluxData hired its largest-ever class of interns in the engineering department, with a total of 5 hires into our storage and query language teams. This series of blog posts discusses our experience gained from building this program. (Click the links to read Part 1 and Part 2 of this series.)

In Part 3 of this blog series, we share advice from our remote interns.

Advice from former interns

At the conclusion of our 2019 summer internship program, I conducted a brief exit interview with each intern. One of the questions I asked was: “What advice would you give to next summer’s interns?” In this post, I quote their responses and discuss what we learned from them to help further improve Influx Interns’ experience in the future.

Above all: Ask questions!

“Make sure you know who/when/where/how to ask questions.”

“Make sure you don’t feel bad about asking questions.”

“Don’t pick your own equipment. Ask for whatever equipment your team recommends.”

“Be proactive in approaching engineers for help! They are willing to help, but can’t predict when that will be. Just ask.”

A common dilemma for any intern or junior engineer is determining when and how much to ask of senior team members. Part of their onboarding included a discussion about how they are expected to ask questions and they should not hesitate to do so. And yet, all the interns I interviewed gave some type of advice that involved asking questions, especially early in the internship.

Some of our interns work remotely, and some in an office. But all of them have mentors and teammates that are not co-located. This means that without intentional action taken to connect and talk, many questions will go unanswered. An interesting insight from the 2019 class was that they saw it as a personal responsibility to stay engaged and ask questions. We also expect managers to perform daily check-ins to create opportunities to communicate, but ultimately only the interns will know when they need help the most. It’s part of their job to reach out when it’s necessary.

Share experiences

“Talk to each other. Benefit from a shared experience.”

“Help each other (other interns) first.”

“Don’t be isolated.”

Interns should stick together. It is valuable to be a part of a group that is sharing the same experience. Software engineering is hard. There are days that this difficulty leads to a task completed and a wonderful sense of accomplishment. Other days, the difficulty leads to a task failed and a terrible sense of inability. Talking to each other, and realizing that this is totally normal and expected can help you work through those difficult times.

Talking to each other also helps the rest of the team. Maybe someone ran into a build failure, and had to get a full time engineer to help fix it. Later, another person runs into the same failure. If you are talking to each other about your experiences and struggles, then one intern will be able to assist the other through these recurring issues.

Learn to work independently

“Get used to working independently.”

“Be a user of the software before you become a developer.”

At InfluxData, we find it rewarding to both team and interns alike to fully integrate them into team processes. Remote teams tend to manage workloads by doing lots of independent work. While this seems to be in contrast to the “Above all: Ask questions!” principle, it actually is complementary. In the first several weeks of an internship, we expect to be front-loaded with lots of questions. Our expectation is that we are teaching you how to analyze a problem and come up with a solution and get you to the point where you feel like you can work independently. What I have found is that interns usually progress in steps: a week or two of questions, followed by a week or two of independent work. Then, when they reach the next level of capability, new questions come up about the more complex tasks they are attempting.

Lessons Learned from Our Engineering Internships Program: Part 2

Adam Anthony (InfluxData) — Thu, 07 Nov 2019 07:00:16 -0700

In Part 2 of this blog series, we share some tips on how to succeed with remote interns.

Succeeding with remote interns

A common trend in hiring remote engineers is that they must have some seniority. After all, the ability to work independently and with non-constant guidance can be challenging to junior-level engineers. InfluxData has a remote-first culture for engineering where there is little difference in the contribution one can make regardless of whether they work in our San Francisco headquarters, or in any number of cities around the world. When we decided to add an internship program to enhance our remote team, we knew that we would have to plan ahead to make sure that we were prepared to properly support a team of remote interns.

How we view interns

The first way we view interns is as open source community contributors. With limited experience and a clear mind, a skilled intern gives us an excellent insight into the structuring of our open source projects, from code structure to tooling to build processes, to quality assurance. We also get daily feedback from them about whether a particular library or tool is hard to work with. Oftentimes, in working through a difficult process, we find that there are better ways to explain it. Other times, interns’ work can validate our assumptions about how community members will work with our code base.

The second way that we view interns is as internal team members. Within reasonable expectations about their experience level, we do expect interns to be picking up issues off of our backlog and submitting pull requests to merge solutions into our production process. Now, this all occurs with some safeguards, due to their limited and temporary status with the company. First, they are granted fewer git privileges to prevent accidental damage to our repository history. Second, interns do not do any on-call or operational work to support our products. Finally, interns must always have their code reviewed by a full-time engineer.

We also view our interns as students. We value the opportunity to teach them the elements of style and craft that go into high-quality software engineering. We provide extra meetings and support to ensure that they have the guidance needed to act with confidence. And if they act confidently and fail — that’s OK. According to our core values we embrace failure. Successful interns should not be placed in a situation where success is defined on whether or not code gets merged. Instead, we aim to provide work items that are a benefit if they are finished, but not a blocker or loss to the team if they are not. Even the best students may have times when they struggle with a task and it must be abandoned. It’s important for team culture, and the well-being of the intern, to understand that learning from the experience is valuable in and of itself.

Tips for a successful remote internship program

At the completion of our 2019 summer internship program, we conducted a series of exit interviews and retrospectives to find out what worked and what didn’t. Here we will discuss our findings and offer some techniques that we believe can be reproduced to help other remote teams successfully integrate interns.

Above all else: Prevent isolation

Isolation causes a number of problems for any remote employee. It’s in our most isolated moments that a small voice inside can be heard: you can’t do this; everyone else knows what they are doing; if they wanted to know what you thought they would ask. Experienced remote employees know how to avoid these feelings, and we all have techniques that work for us personally. Interns, in their inexperience can lose weeks of productive work to feelings of deep isolation. Therefore, most of the advice in this section involves a single principle: prevent isolation.

Team organization principles

Hire interns in cohorts of at least two. Providing them a peer group to relate with will reap dividends in terms of confidence and motivation.
Assign each intern an individual mentor. This is a person who the intern understands will be available for all questions. They should feel permitted to interrupt this person at will and trust that proper feedback will be provided if the questioning is too much.
Clarify that the whole team is responsible for helping all the interns. Mentors take vacations, get sick etc.
Develop new managers. If one of your engineers has expressed or shown an aptitude for a leadership role, assign them to manage the whole group of interns. This is in addition to individual mentors. The intern manager will lead a separate planning/scrum meeting every day to supplement the full-time engineers' meetings where decisions may go by too quickly for the interns to fully grasp what's going on. The extra meeting gives them a chance to ask clarifying questions that may have been disruptive to the flow of the regular meeting. It also gives the manager a chance to be sure that each intern has formed a work plan for the day/week.

Team work principles

Don't waste too much time in training. They need to know what they are doing well enough to keep them from doing harm to your repositories, but otherwise, let them learn on the job. They are eager to write code that's meaningful. You'll see such a huge difference in an intern's productivity after they merge their first pull request. Get them to that point as quickly as possible!
For their second or third week together, assign a large epic filled with small tasks to the group. This helps them in a few ways. First, they have a fast way to find appropriate work items. Second, they are, if you choose good issues, working on productive contributions. Finally, as they see the long list of issues whittle down and finally close the epic, they will form a bond as a team around the sense of collective accomplishment. Good sets of work include writing tests, low-priority features, tech debt that isn't too complex to fix, or even just a collection of unrelated "intern-friendly" tagged issues.
Mentors should do a daily check-in. Yes, we tell interns "you can ask me questions any time," but sometimes, they don't. Particularly with remote interns, we don't have cues like body language or laptops being thrown in a fit of rage to let us know someone is frustrated. The check-in does not have to take any time. Send a message on Slack: "How's it going? Any questions?" Sometimes they don't have anything, and that's great. Sometimes they do. After a few weeks, they will feel like their working relationship requires frequent, open questions and you'll find that an intentional check-in becomes less necessary.
Code reviews must have two reviewers. These are a full-time engineer and an intern.
Occasionally, do a live review. This is where the full-time engineer will do the code review as they normally would but will think out loud during a video conference so that the interns get a first-hand experience of the team's expectations for coding quality and standards.
Have a mid-season gathering at a central location. If your company has a headquarters, set up a week to bring everyone in for a physical meetup. There are several choices we made in planning our meetup that ended up being highly successful:
- Plan for the majority of time to be a hackathon event where interns will form larger groups across teams and work on significant challenges.
- Don't schedule it too early: give them a few weeks to learn the platform of code so that they can feel competent to generate new, useful ideas.
- Don't schedule it too late: we observed that the experience was a great team-building and friendship-building event. Make sure they can enjoy the benefits of the week for some time afterward.
Give interns time towards the end to explore new things. Within 2 weeks of their end date, stop assigning issues and encourage them to just go try whatever they want. They've worked hard for you, and they've earned some time to pursue an interest.

Lessons Learned from Our Engineering Internships Program: Part 1

Adam Anthony (InfluxData) — Wed, 06 Nov 2019 07:00:40 -0700

In Part 1 of this blog series, we share the “why” of our internships program.

Why we hire interns

Historically, InfluxData has always had 1 or 2 interns, most of whom came to us as a direct referral from a full-time team member. Last year, we decided to grow our internship hiring into a larger program with the hope of developing an entry-level hiring pipeline. This was a large undertaking with many expected outcomes. As with many important decisions we make at Influx, we were guided by our core values:

We value each other: We value the presence of interns at our company, for how they help us enhance the team. Hiring a large number of interns infuses energy and new ideas into our teams. It creates a culture of mentorship that helps us to develop new leaders and help engineers reach new career levels.
We get stuff done: We hire our interns to work on our products. With patience, training and mentorship, we've seen 100% of our interns successfully merge pull requests into production.
We believe humility drives learning: As stated above, we value the opportunity for mentorship that an internship program creates. Our engineers can learn much about working with others, but also more deeply about their software craftwork, by teaching it to interns. We also value that interns can join our teams and be immersed in learning agile software development principles as integrated team members.
We embrace failure: Put simply, we expect interns to gain experience by trying things out on their own, and sometimes failing. They learn that this is an acceptable use of time, so long as they learn from the experience and show improvement.
We are committed to open source: During on-boarding, interns must, if they haven't already, create a personal GitHub ID that is not directly affiliated with Influx. They will work on well-groomed issues in our open source products and ultimately merge them back via pull request review. This experience benefits the interns, most of whom have never contributed to an open source project. It also benefits the company, as we get first-hand knowledge of how easy/hard/productive it is for community members to make contributions to the code base.

Internship outcomes

Overall, we have found that our 2019 internship class exceeded our expectations based on our core values. Our full-time engineers gained valuable mentoring experience and our interns learned many new skills while pushing many commits into production. As proof, look at all of the open source contributions made by interns in 2019 — ewendai, roshie548, sauren-khosla, adamperlin, and maxunt.

New Features in InfluxDB Open Source Backup and Restore

Adam Anthony (InfluxData) — Thu, 15 Mar 2018 13:00:24 -0700

New in version 1.5, the open source backup utility provides the option to run both backup and restore functions on a live database. It also provides features to backup and restore single or multiple databases, along with optional filtering based on data point timestamps. Finally, the restore tool supports import of data from an InfluxDB Enterprise cluster, while the backup tool can now generate backup files that may be imported into an Enterprise database. The offline backup/restore functions provided in InfluxDB versions 1.4 and earlier are retained in version 1.5 without change, and are detailed in the open source documentation.

About File Formats

Prior to version 1.5, the open source backup tool created a different backup file format than the Enterprise version of InfluxDB. This legacy format remains fully supported, and in some cases may even be used as input to the new online restore functionality. For new users, we recommend using the new portable backup format, which uses less disk space and also provides a clear transfer path for data between the Enterprise and open source versions of InfluxDB.

Backup

The improved backup command is similar to previous versions of InfluxDB, except that it can optionally generate backups in a portable format and has some new filtering options to constrain the range of data points that are exported to the backup. It is invoked by the influxd binary using the -portable flag:

influxd backup -portable [options] <path-to-backup>

Backup Options

-host <host:port> - The host to connect to and perform a snapshot of. Defaults to 127.0.0.1:8088.
-database <name> - The database to backup. Optional. If not given, all databases are backed up.
-retention <name> - The retention policy to backup. Optional.
-shard <id> - The shard id to backup. Optional. If specified, -retention is required.
-since <2015-12-24T08:12:13Z> - Do a file-level backup since the given time. The time needs to be in the RFC3339 format. Optional.
-start <2015-12-24T08:12:23Z> - All points earlier than this timestamp will be excluded from the export. Not compatible with -since.
-end <2015-12-24T08:12:23Z> - All points later than this time stamp will be excluded from the export. Not compatible with -since.
-portable - Generate backup files in the format used for InfluxDB Enterprise.

Example:

To backup all databases in an existing system:

[/tmp/backup_ex]$ influxd backup -portable /tmp/backup_ex
2018/03/15 12:55:26 backing up metastore to /tmp/backup_ex/meta.00
2018/03/15 12:55:26 No database, retention policy or shard ID given. Full meta store backed up.
2018/03/15 12:55:26 Backing up all databases in portable format
2018/03/15 12:55:26 backing up db=
2018/03/15 12:55:26 backing up db=collectd_db rp=autogen shard=1 to /tmp/backup_ex/collectd_db.autogen.00001.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=2 to /tmp/backup_ex/telegraf.autogen.00002.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=3 to /tmp/backup_ex/telegraf.autogen.00003.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=4 to /tmp/backup_ex/telegraf.autogen.00004.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=5 to /tmp/backup_ex/telegraf.autogen.00005.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=6 to /tmp/backup_ex/telegraf.autogen.00006.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=7 to /tmp/backup_ex/telegraf.autogen.00007.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=telegraf rp=autogen shard=8 to /tmp/backup_ex/telegraf.autogen.00008.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=mydb rp=forever shard=9 to /tmp/backup_ex/mydb.forever.00009.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=mydb rp=forever shard=10 to /tmp/backup_ex/mydb.forever.00010.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=tmp3 rp=autogen shard=11 to /tmp/backup_ex/tmp3.autogen.00011.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=tmp3 rp=autogen shard=12 to /tmp/backup_ex/tmp3.autogen.00012.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=tmp3 rp=autogen shard=13 to /tmp/backup_ex/tmp3.autogen.00013.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=tmp3 rp=autogen shard=14 to /tmp/backup_ex/tmp3.autogen.00014.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backing up db=_internal rp=monitor shard=15 to /tmp/backup_ex/_internal.monitor.00015.00 since 0001-01-01T00:00:00Z
2018/03/15 12:55:26 backup complete:
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.meta
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s1.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s2.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s3.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s4.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s5.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s6.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s7.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s8.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s9.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s10.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s11.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s12.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s13.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s14.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.s15.tar.gz
2018/03/15 12:55:26 	/tmp/backup_ex/20180315T165526Z.manifest

Restore

The restore function is improved in two important ways. First, the restore process no longer requires system down time. Second, a data restore no longer erases the data currently on the target system. A brief technical explanation is that the old restore process deactivated the system and replaced the data folder with the backup data, while the new online process imports data through a streaming API provided by the influxd program.

Regardless of whether you have existing backup automation that supports the legacy format, or you are a new user, you may wish to test the new online feature for legacy to gain the advantages described above. It is activated by using either the -portable or -online flags. The flags indicate that the input is in either the new portable backup format (which is the same format that Enterprise InfluxDB uses), or the legacy backup format, respectively. It has the following options:

-host <host:port> - The host to connect to and perform a snapshot of. Defaults to 127.0.0.1:8088.
-db <name> - Identifies the database from the backup that will be restored.
-newdb <name> - The name of the database into which the archived data will be imported on the target system. If not given, then the value of -db is used. The new database name must be unique to the target system.
-rp <name> - Identifies the retention policy from the backup that will be restored. Requires that -db is set.
-newrp <name> - The name of the retention policy that will be created on the target system. Requires that -rp is set. If not given, the value of -rp is used.
-shard <id> - Optional. If given, -db and -rp are required. Will restore the single shard's data.

Example:

To restore the backup taken above to a new, empty instance of influxdb (Note: the _internal database is skipped by default. Though it would be uncommon to do so, it may be imported explicitly using the -db parameter as described above):

[/tmp/backup_ex]$ influxd restore -portable /tmp/backup_ex
2018/03/15 13:01:45 Restoring shard 13 live from backup 20180315T165526Z.s13.tar.gz
2018/03/15 13:01:45 Restoring shard 14 live from backup 20180315T165526Z.s14.tar.gz
2018/03/15 13:01:45 Restoring shard 5 live from backup 20180315T165526Z.s5.tar.gz
2018/03/15 13:01:45 Restoring shard 6 live from backup 20180315T165526Z.s6.tar.gz
2018/03/15 13:01:45 Restoring shard 11 live from backup 20180315T165526Z.s11.tar.gz
2018/03/15 13:01:45 Restoring shard 3 live from backup 20180315T165526Z.s3.tar.gz
2018/03/15 13:01:45 Restoring shard 10 live from backup 20180315T165526Z.s10.tar.gz
2018/03/15 13:01:45 Restoring shard 12 live from backup 20180315T165526Z.s12.tar.gz
2018/03/15 13:01:45 Meta info not found for shard 15 on database _internal. Skipping shard file 20180315T165526Z.s15.tar.gz
2018/03/15 13:01:45 Restoring shard 7 live from backup 20180315T165526Z.s7.tar.gz
2018/03/15 13:01:45 Restoring shard 8 live from backup 20180315T165526Z.s8.tar.gz
2018/03/15 13:01:46 Restoring shard 9 live from backup 20180315T165526Z.s9.tar.gz
2018/03/15 13:01:46 Restoring shard 1 live from backup 20180315T165526Z.s1.tar.gz
2018/03/15 13:01:46 Restoring shard 2 live from backup 20180315T165526Z.s2.tar.gz
2018/03/15 13:01:46 Restoring shard 4 live from backup 20180315T165526Z.s4.tar.gz