Quality is cultural. It permeates design, development, business, sales, marketing, legal, and everything that touches the product. The way that those different groups interact, communicate, and value the others’ contributions will have a higher impact on quality than any tool you can buy or process that you can adopt.

Issue tracking and other tools and processes for improving quality aren’t magical. For them to work, everyone has to understand and appreciate what they do. Quality is important, but it’s a useless metric outside of the context of business constraints. A team where quality always wins will never ship and ultimately go out of business. A team where business always wins and quality suffers will decay over time affecting morale, development speed, and eventually leading to lost customers.

Good tools amplify effectiveness. If your testing effectiveness is negative, adding tools will only amplify the negativity. Gerald Weinberg ¹

Quality requires a holistic view of building software by everyone on the team. Choosing the right tools for your team and structure is important, but if your team isn’t completely sold on quality and care, tools won’t be enough.

Ed Catmull, one of the founders of Pixar, recently wrote a book, Creativity, Inc., about lessons learned with Pixar and Disney. The majority of the book talks about the culture of Pixar and how that culture enables Pixar’s high quality. Of course, Pixar makes movies, but the idea holds true for any team that’s creating new things…

Quality is not a consequence of following some prescribed set of behaviors. It is a mindset you must have before you decide what you are setting out to do. You can say you are going to be a company that never settles, but saying it isn’t enough: You must live and breathe it.Ed Catmull, President of Pixar Animation and Disney Animation

Over our years of developing software, we’ve observed healthy approaches to quality as well as toxic approaches, and we’ve pulled together a variety of tips to help encourage a focus on quality as well as cues that may signal that quality is slipping. The short of it is that adopting tools or processes won’t solve any problems with quality unless everyone on the team is truly committed to higher quality. If they don’t buy into the premise, the tools won’t help.

People & Processes

The two key components to a culture of quality are people and processes. People are, without a doubt, the most important component. Instilling a eye for detail and a commitment to doing things well will do more than any tool you can buy or process you can adopt. The processes help enable your team to really do their best work.

Both people and processes have intangibles that can poison the well and make quality an unrealistic goal. Even though these variables can’t easily be quantified and measured, they can affect the team dynamic if ignored.

People

Participation. Participation. Participation.

Whether it’s increasing adoption of the tools you’ve chosen or getting everyone to understand the value of quality, participation is where it starts. Focusing on quality is hard work. There are dozens of competing forces to encourage short cuts, and everyone has their own metrics and priorities to manage. If everyone isn’t on board with quality it will slide.

Communication.

Quality communication creates quality results. Tracking and resolving issues and fixing bugs will frequently require many team members to participate. The better they’re able to understand each other and their expectations, the better the results will be. This requires clear communication. A recent study of issue tracking by Microsoft confirmed that due to the diversity of roles involved, communication and collaboration are key components of successful issue tracking.

Because of the large number and broad range of stakeholders involved in this form of group work, communication and collaboration become an integral part of the process, even within small, collocated teams. ²

An issue tracker isn’t just about bug tracking, and it’s definitely not only for developers. It’s a platform for developers, testers, stakeholders, project managers, and others outside of development to collaborate and contribute to the process and ultimately create the knowledge that will guide the course of the project. That study by Microsoft summarizes it well…

We found that an issue tracker is not just a database for tracking bugs, features, and inquiries, but also a focal point for communication and coordination for many stakeholders within and beyond the software team. Customers, project managers, quality assurance personnel, and programmers all contribute to the shared knowledge and persistent communication that exists within the issue tracking system. ³

Some smaller teams get away with using email, sticky notes, spreadsheets, or todo lists for a little while, but those don’t foster centrally available hubs that are designed for both communication and collaboration. Each of them brings weaknesses that lead to a breakdown in communication in one form or another.

One team.

Organizations which design systems…are constrained to produce designs which are copies of the communication structures of these organizations. Conway's Law

Depending on the size of your team or company, it’s likely that teams are compartmentalized. Different departments may have competing agendas, and their ability to communicate seamlessly as well as value the contributions of the other teams and departments is vital for quality to survive. A departmental organization with politics will create worse software than a unified organization where everyone is on the same page.

Andrew Hunt and David Thomas address this eloquently in The Pragmatic Programmer…

It’s a mistake to think that the activities of a project—analysis, design, coding, and testing—can happen in isolation They can’t. These are different views of the same problem, and artificially separating them can cause a boatload of trouble. ⁴

Similarly, teams shouldn’t feel that they have turf to protect. Bug tracking and quality is everyone’s job, and everyone should be able to contribute and participate in that process. It’s all too easy for bug trackers to become the domain of the developers. Complex tools that require training can easily alienate clients or non-technical team members, and with small teams, these are often the most important team members for recognizing if something is correct or not. It’s important to ensure that non-technical team members feel welcome and valued when they report issues.

In particular, while testers should be separate from developers, both groups should think nothing of working together. You want to avoid any divisions, physical or conceptual, that might prevent open communication between the two groups. No matter what, it’s imperative to avoid the creation of an “us vs. them” mentality between testers and developers.

Avoid blame.

The more time and effort someone spends looking for someone else to blame for a problem, the less the chance of solving the problem.Gerald Weinberg ⁵

Accountability is a central part of issue tracking. More often than not, the combination of issue tracking and source control make it fairly easy to know who’s at fault for a given problem. This can be either helpful or toxic. When something goes wrong, there’s no value in placing blame. The best companies focus on fixing problems rather than wasting effort tracking down someone to pin the blame on.

Unforeseen, random events happen. And when they do, don’t waste time playing the blame game. To think one can control or prevent problems or guard against randomness by making an example of someone is naïve and wrongheaded. Instead, empower employees at every level to own the problems and give them the freedom to fix them without asking permission. Ed Catmull, President of Pixar Animation and Disney Animation

Excessive blame can quickly lead to a culture of fear and hesitation with the end result being a breakdown of communication or team members being too cautious with new ideas or code. You want people to take pride in the work, but you don’t want them to constantly fear the repercussions of raising or admitting to issues.

Don’t take it personally.

Testers aren’t creating the bugs, they’re only exposing the presence of bugs. Bugs aren’t personal and good testers aren’t vindictive. Testing and the associated processes are only striving to uncover existing problems so developers can fix them. Like a parent checking your homework when you’re younger or an editor reviewing a book, their goal is to help, not hinder.

As an extension of placing blame, team members have to view bugs as constructive feedback rather than criticism of their work. If someone repeatedly has significantly higher bug counts than other team members, sure, sit down and have a chat. But in general, don’t worry about it. Bugs happen. That’s what testing is for. Just like an editor wouldn’t chew out a writer for every single typo, there’s no need to view discovered bugs as scarlet letters.

This is incredibly challenging for some teams because people need to learn to criticize ideas, not people. As soon as criticisms focus on the people, it’s nearly impossible not to take feedback personally.

Processes

Avoid broken windows.

One of the most difficult challenges of maintaining high quality is that quality doesn’t compromise. While it’s understandable to make the occasional short-term tradeoff in quality in the name of some other key business metric, you have to stay ahead of your issue backlog. Quality is a long-term commitment that requires ongoing discipline. Any organization that ships new features before fixing existing issues isn’t committed to high quality. ⁶

Your team has to be intolerant of even small mistakes. Once an area of a codebase begins the downward spiral, future additions to that codebase are lackluster, and even the most disciplined developers can lose hope. Some teams treat bug and issue trackers as a place to log issues and then assume that someone else will handle them. Reporting issues is only half the battle, and it’s the easy half. The real work is committing the time to fix even minor bugs before your list becomes overwhelmingly large.

When issue-tracking tools are forced to track thousands of issues , then the word “tracking” loses meaning. They become “issue dumps” (and often smell like a dump too). Robert C. Martin ⁷

If your issue backlog is particularly overwhelming, it can hurt morale. When this happens, the best approach is to slow down on testing until the team can catch up with the existing issues. Since fixing bugs generally requires more time than reporting them, it’s often easy for testers to overwhelm developers with bugs at the beginning of a testing period. In the absolute worst case where the issue backlog could be years old and out of control, you may even want to hit reset and start over. Instead of validating legacy issues, starting fresh will make the issue backlog seem more realistic. ⁸

Focus on value, not effort.

When faced with any decision about quality, it’s tempting for management to view the solution to a problem in terms of how much effort will be required. However, effort is only important in relation to value. Some high-effort bugs are low value because they’re optimizing for incredibly rare edge cases. Other easy-to-fix bugs are incredibly high value.

With issue tracking, everything is a tradeoff. You can only work on one issue at a time, and the goal should always be to fix the one that brings the most significant increase in quality (and revenue) relative to the amount of effort.

The goal is greater than the process.

It’s necessary to set up a good process, but the process should always come second to the ultimate goal of increased quality. It’s possible, even likely, that your team can become dogmatic about process and following the rules to the point that the quality actually suffers. It’s important to be aware of this and keep an eye out for steps in your process that may need to adapt to work for your team.

For instance, if you have a small team and adopt a rigid and complex workflow used by teams of fifty or more people, you’ll inevitably find that you have too much process, and it will slow you down. I often talk to folks that work on small teams and try to adopt detailed practices that they used at previous jobs for huge projects. In most cases, they’re so fixated on following the process, they end up over-engineering their workflow and being counter-productive.

Of course, you don’t want to constantly modify your process or switch tools. You just want to be careful to ensure that you’re not too attached to certain tools or methodologies if they aren’t actively helping you improve the quality of your product. The tools and process are only a means to an end. Quality is what really matters.

Automate but verify.

Automation is one of the most powerful testing tools available, but it’s not perfect. You should always seek to leverage automation, but human testing will always be needed to find the problems that the automated tests overlook or to find the subjective issues that a computer would never catch.

It’s important that the team recognize both the strengths and weaknesses of both automated and manual testing. They complement each other, and one isn’t any more or less important than the other. To achieve the best results, both needed to be valued.

Be ready to recognize tradeoffs.

At every step of creating something, there are countless opportunities to neglect quality. Some of these opportunities present valid cases where subjective quality can justifiably be sacrificed to some small extent. The catch is that the majority of bugs and technical issues are not subjective. They are known problems in a system, and they will create costs down the line if they affect customers.

Ship dates must eventually be met, and there are cases where hitting a ship date matters more than a minor bug that would only affect 0.001% of customers. The challenge will be in objectively evaluating the two and recognizing that if quality always loses the battle in the face of other metrics, it’s not saving time or money, only deferring the costs of that time and money to a future date with interest.

It’s not the quality or lack of quality that’s the problem. Comparing a ship date to a batch of bugs is focusing on the wrong things. The real tradeoff to consider is the cost of missing a ship date (or working overtime) to the cost of the impact those bugs will have on customers, support, and development down the line.

Slow and steady wins the race.

There are many components of improving quality, and trying to implement all of them cold turkey will be nearly impossible. The best approach is to ease into them. Have a plan for adding the additional tools and processes to improve quality, and pay close attention to whether they truly improve your process and quality.

Similarly, if you’re working with a legacy codebase with less-than-perfect quality, you may constantly be tempted to scrap the existing code and start fresh with a ground-up rewrite. This is usually a terrible idea. Large rewrites rarely work because they’re attempting to solve a cultural problem with a technical solution. With rewrites, the assumption is that the existing codebase is the only impediment, but poor quality code is a cultural problem. If the developers or managers that created the codebase are still there, the same problems will happen.

Creating an emphasis on quality throughout a team can be exponentially more valuable than just formalizing processes and paying for some tools. If your team cares about quality, they'll naturally gravitate towards the tools that will help them do their job. Keeping these things in mind as you create your own testing strategy will make the journey easier and the results better. So, with no further ado, let's dive into designing your process.

1. Gerald Weinberg, Perfect Software And Other Illusions About Testing (Dorset House Pub., 2011), 161. ↩
2. Dane Bertram, et. al., "Communication, Collaboration, and Bugs: The Social Nature of Issue Tracking in Small, Collocated Teams" (2010), 1. ↩
3. Bertram, et. al., "Communication, Collaboration, and Bugs: The Social Nature of Issue Tracking in Small, Collocated Teams", 1. ↩
4. Andrew Hunt and David Thomas, The Pragmatic Programmer (Addison-Wesley, 2000), 227. ↩
5. Weinberg, Perfect Software And Other Illusions About Testing, 80. ↩
6. Hunt and Thomas, The Pragmatic Programmer, 5. ↩
7. Robert C. Martin, The Clean Coder: A Code of Conduct for Professional Programmers (Pearson Education, 2011), 290. ↩
8. Weinberg, Perfect Software And Other Illusions About Testing, 51. ↩