Quality assurance doesn’t end when you ship. In some ways, you could argue that it’s only beginning. The organizations that enable their teams to quickly respond to problems in production code create the highest quality software. Lingering bugs have an incredible cost, if not in terms of revenue, then in terms of lost credibility and trust from customers.

The good news is that responding quickly and attentively to customer problems can ultimately increase customer satisfaction. Barring constant reoccurring problems, most customers are surprisingly tolerant of bugs if they are addressed quickly and the customer knows that the bug is being addressed.

Throughout software development and testing, the goal is to prevent issues from affecting your customers. However, your ability to receive feedback and react efficiently to handle customer problems after shipping is just as important. You can’t prevent every bug, but you can make sure that your team is able to react and fix a problem when one is present.

Tools & Processes

Some bugs simply won’t be found in even the best test environments because nothing uncovers edge cases like real people using your software in real world scenarios. Due to the increase in the number of parties involved and the fact that these bugs are implicitly sneakier because they weren’t found during testing, these situations can quickly become very involved to both track down and fix. So having the right tools in place is all that much more important.

Performance Monitoring

Without monitoring tools in place, performance is an almost invisible problem. Customers use a wide variety of systems with varying degrees of power and internet connectivity. Even if you test locally, it’s impossible to test against all of the different setups that your customers will be using. So performance monitoring is a must.

Jakob Nielsen describes the three main performance limits for web page response times at 0.1 second, 1.0 second, and 10 seconds. At each of these points the user’s attention and feeling of response dips significantly. Nielsen also addresses the fact that a snappy user experience beats a glamorous one.

Exception Monitoring

Customers can never be as good at reporting bugs as testers are. As a result, customer bug reports will frequently be missing relevant information for tracking down problems. The best way to address this is by using exception monitoring to automatically capture technical details about production bugs. This way, the technical information can be referred to when troubleshooting customers problems. Good exception monitoring not only helps developers troubleshoot more effectively, but it also minimizes the need to further inconvenience customers with requests for additional information.

Help Desk Software

While you may be able to get by in the short-run without help desk software, in the long-term, it’s absolutely critical. However, the wrong help desk software can often make matters worse rather than better. If you’re serious about quality when evaluating help desk software, nothing should matter more than your customer’s experience. If your help desk software is difficult or frustrating for your customers, the consequences will be costly.

To begin with, if a customer is contacting support, the chances are slim that it’s because their thrilled with every aspect of your software. It’s much more likely that they’re already confused, frustrated, or worse. If your help desk software makes it difficult for them to submit an issue or followup on it, it will only make matters worse. When customer need help, the last thing you want to do is pile additional frustration onto the experience. Forcing customers to deal with even more bad software is adding insult to injury.

Moreover, if your customers have negative experiences with your support software, the likelihood of them reporting future issues decreases dramatically, and you can’t fix a problem that nobody told you about. If customers dread contacting support, you’ll never have an opportunity to make it right or improve quality. Some companies view support as a cost center and intentionally make it difficult for customers to contact support in order to lower costs. Those aren’t the companies that truly value quality.

Support Team

So many opportunities to improve quality boil down to empathetic humans, and nowhere is this more important than with your support team. Their primary goal is of course to help customers, but their secondary goal is to help the organization improve the software so that customers don’t need support in the first place.

Beyond needing your support team to be friendly, understanding, and empathetic, you also want them to be proactive. In order for customer support to help improve the quality of your product, they need to be able to offer feedback and suggestions to developers instead of spending all of their time putting out fires. Your customer support team needs access to the proper channels so they can escalate and address reoccurring problems.

One of the best ways to encourage your support team to be proactive is to make sure that the development team is responsive when they escalate a problem. If management doesn’t make time to listen to the requests of the support team, it will erode morale. Eventually, the support team feels that it’s hopeless to raise issues, and they’ll stop.

Feedback from Real World Usage

While usability or productivity impact isn’t as easily quantifiable as a bug, it’s just as important to customers. The support team is on the front-line of hearing customer requests, and it’s imperative that they’re able to escalate and share this information with the product folks. Unfortunately, given the frequently subjective nature or the distance between support and development, it’s often easy for this kind of feedback to be ignored or filed away without serious consideration.

Not all customer feedback will align with your team’s vision, but the more exposure that you have to customer requests and ideas, the more you’ll begin to understand their needs and build the right product to make them more successful.

Business Analytics

Changes in your software can affect usage, adoption, and more. For instance, if your payment form is broken when testing against your production gateway rather than your test gateway, your customers will likely notice before you do. Monitoring these things is imperative at maintaining high quality. If somebody visits your site and your registration page is broken, they probably won’t take the time to let you know. They’ll just leave.

Similarly, there are countless variables that can affect customer behavior and decision-making that they aren’t even aware of. So even if they wanted to tell you that something was off, they couldn’t. Business analytics shouldn’t limited only to conversion and sales, though. The best companies are also closely monitoring and understanding how customers use different features, which features are most popular, and which features might need to be improved or even removed.

Execution & Communication

Few things are more frustrating as a customer than taking the time to report a problem and feeling like the company doesn’t care. And these kinds of problems aren’t always as obvious or simple as the customer being blown off by the support team. If a customer takes the time to let you know that something is broken and feels like the report isn’t taken seriously, they’re not only going to be unhappy with support, but they won’t report issues in the future.

The best way to address this is to ensure that each and every bug reported by a customer gets top billing. If it’s unreproducible, let them know that you spent time looking into it and are having a hard time but have taken steps to keep a close eye on it. You can set up additional logging or exception handling around it to help you track down the problem.

The good news is that being responsive to customer issues is a virtuous cycle. When customers know that you’re serious about fixing bugs quickly, they’re much more likely to continue to report issues in the future, helping you uncover and fix more problems.

Release Management

A subtle but key part of execution and communication around customer-discovered bugs is being able to get the fixes live as soon as possible. Even if you’re able to fix and test quickly and communicate well with your customers, the speed of fixing it doesn’t matter if you can’t release it for a week or two.

If you’re running a web application, you should be able to release without fear at any time. Responding to bugs in hours is amazing, same day is impressive, same week is tolerable, but waiting longer for a scheduled release will leave customers frustrated because they have to deal with the issue in the mean time.

Some of the most important bug reports and feedback will come from the front lines. While most teams do eventually take this information into account, few make it a true priority. Responding to customer-facing problems can be a virtuous cycle of improving quality or a downward spiral of indifference. There’s no doubt that these are the most costly types of issues to address, but the benefits of addressing them promptly and effectively can be huge. Alternatively, the consequences of being slow or unresponsive can elevate problems from being slight inconveniences to incredibly costly bugs.

It’s often death by a thousand cuts. No one problem alone will destroy the team’s ability to deliver high quality software, but if left unattended, problems discovered in production will erode customer confidence, increase customer frustration, and ultimately affect the bottom line. If you’re serious about shipping high quality software, you’ll want to pay just as close of attention after shipping as you do before shipping.

Now that you have some context about the various components of delivering quality, we'll explore the steps that you can take to increase the quality of your software. While we've touched on some tips and advice throughout the course, we'll start with an overview and summary of everything your team needs to do to create a culture of quality.