As a video hosting platform trusted by thousands of businesses, large and small, across the globe (over 75 countries), vzaar is big on quality assurance. They believe in everyone on support and really push quality assurance to make sure their video player is working flawlessly in different browsers, operating systems, and devices.
At Vzaar, QA and bug tracking is a huge part of what we do. Ensuring that our video player works flawlessly across the variety of desktop web browsers, iOS, Android, and even some of the older Blackberrys takes extensive effort. Sifter helps us do that and more. For us, Sifter is a hub for all of our development activity. We maintain separate Sifter projects for development, bugs, design, devops, marketing, change requests, and customer support.
Our QA process is constantly evolving and improving as we grow. We now manage separate development, staging, quality assurance, and production environments. Staging provides a development environment that mirrors our production cloud infrastructure so that we can work out any kinks prior to releasing. Then the quality assurance environment gives a secure production site that is only accessible internally for end to end testing prior to a full production deployment.
Our development style is fairly flexible. Each new feature release can involve anywhere from a single developer to a combination of developers, designers, copywriters, and devops. Our developer gets to work and does cursory testing throughout development. Once a feature is ready, we’ll release it to QA where the developer tests the code, pulls in specialists to test as needed, and files all of the bugs in Sifter. Everyone across the company tests the platform; we don’t run a dedicated test team.
Once we’re satisfied with the quality of the new feature, we perform some user acceptance testing on behalf of our customers to make sure everything is tuned perfectly. We organize our work into four major components which also helps us prioritize any work.
- Playback. There’s no room for error here. Nothing matters more to us than making sure our customers never have to worry about the reliability of video playback.
- Application. We never want downtime, and our core web application is second only to playback in our priorities.
- Marketing Site. Of course, we don’t want our marketing site to be unavailable, but if it does go down, our customers are unaffected. So it gets third billing.
- Background Jobs. Losing background jobs for a bit will surely slow our customers down, but it’s resilient by nature. So it’s the lowest of our incredibly high priorities.
In particular, any changes or improvements to our video player must be tested exhaustively across multiple environments and numerous devices. Even the tiniest device conflict could have a terrible impact on our customers’ playback. So we maintain a test lab with all versions of iOS since iOS 5 (on both phones and tablets), multiple Android devices, as well as some Blackberries and Windows Phones to help ensure that we’re covering the spectrum. Testing our video player is a neverending quest for perfection.
Once we begin to wrap up testing for a release, we begin to make a plan for communicating the changes to customers. With some releases, this is a key part of our quality assurance process. For instance, a player change is rolled out to different sets of users at a time with the user base divided up into smaller groups. This helps minimize the number of customers affected if a problem does slip through.
Once a feature is released, we use a combination of email notifications, announcements via a dashboard widget, and release notes to help make sure that our customers know about, and can begin benefiting from, the improvements. Whether we’re dealing with bugs, design, devops, marketing, support, or good old-fashioned change requests, Sifter helps us stay organized.
By the Numbers
- Founded In 2007
- Sifting Since 2009
- Company Size 16
- Average Team Size 2-5
- Releases Per Week about 6
- Average Release Length "When it's done"
Sifter is the glue that keeps us organized across all of our development work, and in our experience, the quality and responsiveness of their support is second only to our own.Adrian Sevitz, CTO