ChatOps: Let’s Talk Plugins

By Eric Sigler

This is the second post in our series on ChatOps

In our last post, you learned what ChatOps is, how it works and the ways in which it can transform your organization. By promoting collaboration, communication and rapid learning, ChatOps presents some serious benefits not only for engineers, but also non-technical teams too.

We want to take that a bit farther with our next two posts. Once you've chosen a framework and kicked the tires a bit, you'll probably want to add features to your bot. The plugins detailed below can be easily added and will help out with previously manual tasks you may have had to do daily.

At PagerDuty, we use Ruby-based Lita and our chat bot, Officer URL, with a variety of plugins, so those are the examples we'll roll with below.  But, many of these plugins have alternatives in other frameworks so you can continue using what works best within your team.

lita-dig & lita-whois

"dig" and "whois" are simple plugins to look up DNS & whois records.  They're easy to drop in, and quick to get started with.  These are good examples for applying ChatOps to your day-to-day workflow - with them you can now ask in-line questions about things like record changes.  They also tie in really well with lita-enhance, which we'll go into more detail about below.

lita-locker

lita-locker allows you to define resources (such as a server, or Git repo), and labels (such as "production").

When a PagerDuty engineer wants to take certain actions on an environment, they can signal to everyone else that environment is unavailable for other changes by "taking a lock".  Since lita-locker has the idea of both resources and labels, it can prevent accidental collisions if you have a single resource and two different labels that contain that resource.  (For example, a database server could be a resource to both the "request-backend" and "job-worker" labels.)

lita-github

We use Github at PagerDuty, so it's really useful to be able to ask in a chat room for a code review, and have the merge in-line (especially along with the deployment and environment locks while the changes are being tested out).  If you have a company Github account, and you need to create a new repo within it, this plugin can be extremely helpful - now you don't have to hunt down someone with administrator rights on the account.

lita-wtf

Internally, we have a lot of projects, and a lot of service names.  As mentioned in our last post, ChatOps can help with onboarding new engineers and employees. When new people are starting up (or a new project is introduced and isn't well known yet), they don't have to feel silly asking a lot of "what is X?" questions. Everyone has access to a self-service dictionary right within your chat client.

lita-enhance

Often times log snippets get pasted into a chat room, but they contain opaque identifiers such as IP addresses or AWS instance ID's.  lita-enhance takes that text, and replaces it with more helpful information about parts of your infrastructure.  MAC addresses, IP addresses, short hostnames & DNS names, all can be turned into something more useful.  Combined with plugins such as lita-dig, you can get workflow like:

lita-status

Many sites have internal status pages of one kind or another.  And when there's a problem, that's often the first place people have to check out what's going on.  The status plugin helps us keep that information in-line within our chat client. You can avoid the "repeating what the computer said" effect on phone bridges, Google Hangouts or whatever it may be during an incident or outage.  It also keeps a convenient log for postmortem review. (Note, this plugin is not yet open-sourced, but will be shortly.)

Implementing these plugins (and many more) at PagerDuty has really helped us as we've scaled our team and infrastructure over the last year.  We value things that help make engineers more productive, and want to be able to share those things with other engineers and businesses. Because of that idea, we work to open source many of the things we build internally.

In part 3 of our series on ChatOps, we'll go over how we do deployments and schema migrations with Igor, our in-house deployment tool.