As you might expect, we rely heavily on Sifter on our own process. In fact, we route virtually everything we do through Sifter. Whenever we begin a significant new feature or change to Sifter, it goes through a pretty consistent process. This process has evolved organically as we’ve grown, and it’s slowly becoming second nature to help us ship faster and better.
1. Ideas and Discussion
Almost every idea begins as a result of support requests. With Help Scout, we also tag and archive all of our old support requests. That way, once we start development on a particular feature, we can go back through old support requests to get insight into the underlying customer problems that originally sparked the idea.
With almost any interesting idea, we’ll create an issue in our “Ideas” project regardless of whether we think it really has a chance. The issue will generally live in there for some time while we discuss it and work on other features. In the ideas project, we explicitly don’t use milestones. That lack of milestones combined with the fact that it’s separate from our main development project, we’re able to stay more focused on today’s work rather than what we might be working on in the future. It also helps us treat the ideas project as more of a blue sky project without any obligations.
Depending on the idea, we’ll often create a branch at this point and just start prototyping and playing with the idea to see how we like it. Usually this just helps us fill out the idea and we add thoughts back to the issues. Every now and then, though, the idea will turn into an unstoppable force and we’ll just build it.
2. Development
As we complete work on one thing, we start thinking about what we could work on next. We usually have a good idea of what that will be, and it’s almost always already heavily documented in our ideas project. We’ll generally take that issue, create a milestone in our main “Sifter” project, and begin logging issues against that milestone based on the discussion in the original issue.
Once we’re satisfied with the work laid out in the milestone, we’ll create a Git branch with the same name as the milestone for handling the development work without affecting our master branch. Once we start writing code, Travis and Code Climate are constantly monitoring our work to make sure that we’re writing high quality code and not introducing any new bugs. If anything goes wrong, they’ll send us an email and post a message to Campfire.
Once the branch and milestone are up and running, we simply start iterating. As we’re writing new code and resolving/closing issues, we’ll occasionally release the work in progress to our staging environment so that other team members can see what’s happening and share their feedback without having to do the work and run the branch locally.
3. Review, Internal Testing, and Dark Launch
As this process continues we eventually get to a point where virtually all of the issues are closed and only a few superficial items are left. Then the lead on that milestone will open a pull request on GitHub.
This pull request signifies that while the code may not be 100% ready to ship, it won’t be changing drastically, and other team members can start reviewing the code and sharing their thoughts. This helps us catch any oversights in logic or performance and gives us all a chance to familiarize ourselves with everyone else’s code.
The pull request also signifies the point where we know we can start exhaustively testing the new code on staging and logging any final bugs in Sifter. Sometimes we won’t find anything, and other times, we’ll uncover a handful of bugs. Either way, we’ll get to work closing out the last few issues, and once all of the kinks are worked out, the pull request is ready to go.
4. Merging and Releasing
As soon as all of the issues are closed in Sifter, and Travis green lights the pull request, we’ll do one last release to staging and run through the relevant functionality just to make sure no new bugs slipped in at the last second.
After that, we’ll go ahead and merge the new branch into master via the pull request and push it to production with a final production smoke test to make sure that everything is running smoothly. We borrowed Nathaniel Talbott’s post-deploy smoke tests to add simple automated smoke tests to our release process, but we almost always do manual spot checks as well. It’s rare that we find anything, but when we do, we’re always glad that we were able to catch it sooner.
5. Monitoring and Analytics
Of course, once new code is in production, the development process isn’t over. No matter how careful a team is, some bugs won’t show up until your code is exposed to real world usage. For this, we rely heavily on New Relic. If a new release has an usually high error rate or slower performance, we’ll be able to quickly identify the problem and take steps to address it.
Simplifying and Automating the Development Process
We didn’t implement this process overnight. It has slowly evolved along with the tools that we use and our team size. Outside of me, nobody else works on Sifter full-time, so our communication is incredibly asynchronous. Even Sifter’s role in this process has evolved over time. For instance, separating our active development from our backlog of ideas is a recent development in the last year or so. Similarly, our usage of GitHub pull requests has evolved over time as more people have contributed to Sifter.
There’s no doubt that having a consistent process helps, but we’ve found that it’s best to let processes evolve organically as the team grows or as your technology and infrastructure evolve.