There are roughly 18,700 unique Android devices on the market. There are somewhere around 14 browsers, powered by several different rendering engines available for each of those devices. Then there’s iOS, Windows Mobile and scores of other platforms to consider.
You can’t test every configuration. And you shouldn’t try.
The good news is that you don’t need to worry about testing every device on the web. Even better, though, you don’t need to own a plethora of devices for manual testing. We’ve got some tips to help you test more effectively. You don’t need a pile of devices that each need to be checked. One. At. A. Time. And you can narrow down your testing to focus on the configurations that matter with just a little research.
Enter BrowserStack
BrowserStack is a browser-based virtualization service that puts nearly every operating system and browser combination under the sun at your fingertips. 1
With BrowserStack, you can quickly grab a handful of screenshots for different browsers or spin up a virtual machine in your browser and test a different browser live. BrowserStack also offers mobile emulators for testing almost every version of iOS, Android and Opera Mobile (Sadly, at the time of writing, there are no Windows Mobile emulators available on BrowserStack).
You can also choose the screen resolution you’d like to test at, which makes possible to test resolution-based CSS media queries if you’re using them. BrowserStack also has a dedicated Responsive Design testing service. 2
We’ve been using BrowserStack to test Sifter for some time now, and it’s enabled us to have better breadth of coverage as well as streamline our process so that it’s quicker and easier to locate problem areas. Here’s a peek behind the curtain on the workflow that we’ve found helps us do more with less.
Our Browser Testing Workflow
We’ve divided our workflow into three steps to help make sure that we’re testing the most important configurations for our visitors, finding significant problems across that spectrum of configurations, and finally focusing on exhaustive testing and debugging of particularly challenging configurations.
1. Narrow the Field
Before you dive into testing, consult your analytics and narrow the field based on what you know about your traffic. Focus on the platform/browsers that make up the top 80 percent of your visitors. Then, once you’re confident you’ve got a great experience for the majority of your visitors, you can address the more obscure cases.
If you’re launching something new and don’t have existing data to consult you’ll need to do some research to uncover which devices your target audience favors. That way you’ll know, for example, that your target market tends to favor iOS devices and you’ll want to spend some extra time testing on various iPhone/iPad configurations.
Even after you’ve limited your scope to a handful of relative devices and browsers, you’ll still have your work cut out for you. You can’t efficiently test each configuration manually, and you certainly don’t want to maintain a collection of virtual machines or a pile of devices on your desk that each has to be powered or charged. Enter hosted virtualized testing. Thankfully, it’s come a long ways.
2. Run a Broad Smoke Test
BrowserStack’s screenshot service can help you quickly test a single page across a diverse set of browsers as a way to identify the browsers that need more extensive testing. This way, you get a good overall feel for where your browser and device compatibility stands without spinning up dozens of individual virtual machines.
You’ll always want to complement your screenshot testing with some manual testing against key browsers and devicdes to weed out JavaScript or other interactivity bugs, but the screenshots help you keep that to a minimum.
3. Zero in on Specific Configurations
BrowserStack is not just a screenshot service. It launches a fully interactive virtual machine in your browser window. That means you can interact with your site just like you would if you were using a “real” virtual machine or had the actual device in your hand. Once you know which browsers are most important and which are most problematic, you can do more extensive testing with the virtual browsers.
You pick an OS/browser combination you want to test, enter your URL and start up your virtual machine. 3 BrowserStack will then launch a virtual machine in your browser window. Now you have a real virtual machine running, in this case IE 11 on Windows 8.1. 4
But wait, it get’s better. Since development sites are usually local to your machine, BrowserStack makes it easy to test local sites as well.
Local Testing with BrowserStack
If you’re building a brand new site or app, chances are you’ll want to do your testing before everything is available on a public server. If you have a staging server you could point BrowserStack to that URL, but having to re-release your site to a staging server to test changes can slow you down. That’s where BrowserStack’s local testing comes in handy–just point BrowserStack to your local server or directory of files on your computer. 5
To do this BrowserStack will need to install a browser plugin, but once that’s ready to go, testing a local site is as easy as testing any other URL.
Start by clicking the “Setup Local Testing” button in the sidebar at the left side of the screen. This will present you with a choice to use either a local server or a local folder.
If you’ve got a dynamic app, pick the local server option and point BrowserStack to your local URL. Alternately, just point BrowserStack to a folder of files and it will serve them up for you.
That’s it! Now you can edit files locally, make your changes and refresh BrowserStack’s virtual machine to test across platforms without ever making your site public.
Debugging with Developer Tools
Another major benefit of virtual machines over a simple screenshot service is that it also means you can use the virtual browser’s developer tools to debug any problems you encounter. This means you can view source, inspect elements, and even use the console when something’s not quite right.
Capture and Annotate Screenshots
Another really handy feature is the ability to easily capture and annotate screenshots of cosmetic problems. You just click the little gear icon in the top right corner of your browser window, which will reveal a camera icon for capturing screenshots. 6
Click the camera and BrowserStack will generate and present a screenshot that you can annotate and share with your team. 7 Then you can download it and add it to the relevant issue in your favorite bug tracker.
Beyond the Basics
This is only the tip of the iceberg. There’s also a good bit more than can be covered in a short review like this, including automated functional testing, a responsive design testing service that can show your site on multiple devices and browsers, an automated screenshot service and more. You can even integrate it with Visual Studio.
The various types of services each have different pricing and are sold separately which can make it a little confusing at first, but once you see that it’s really three different services that happen to be sold under the same product name, it’s not too bad. A free trial account will get you 30 minutes of virtual machine time, which you can use for testing. If you decide it’s right for you there are a variety of reasonably priced plans starting at $39/month or $19/month for limited single user lite plans.