Sifter was and never will be designed to support bug tracking at the level of billion dollar software companies, but there’s a lot to be learned by looking at the challenges one faces tracking bugs on that scale. The iOS developer community has exploded with the success of iPhones and iPads, but Apple hasn’t done much to make it easier for developers to submit bug reports. In fact, it’s bad enough that there’s a site dedicated to asking Apple to do something about it.
Important: While Apple is the target of this letter, the supporting points are by no means unique to Apple’s bug reporting tool. Let’s remove the Apple brand from this scenario and just look at the complaints in the context of people using a bug tracker. Some of these problems are with the tool, but some are simply cultural. Either way, they aren’t good practices. We should also bear in mind that this is all from the perspective of the bug reporters. There’s certainly value in perspectives of the developers that receive and interpret the bug reports, but we’ll save that for another day.
From the open letter…
Radar is also a black hole. We file radars and we’re lucky to hear back about them. The majority of radars are either left untouched or marked as duplicates of other radars we cannot see. We may get a request for more information from engineering, but sometimes it is for irrelevant information or information already given in the original report. All this makes us feel like our radars make little difference. And this is important as our time is valuable.
There are a couple of valuable tidbits here to keep in mind with bug trackers…
- Radar is a black hole. This is always a bad sign. Bug trackers have to be two-way, and it has to be easy for that two-way communication to happen. Developers need to be able to request more information, and reporters need to be able to easily provide that information. Bug reporters also need to know that the bug reports are being heard.
- The majority of radars are either left untouched or marked as duplicates of other radars we cannot see. At a large scale, tracking duplicates is important. However, this point addresses something more significant, and that’s visibility. While software at this scale could justify making some bugs private for security reasons, bug trackers are, at their core, about collaborating to build higher quality software. Hiding things by default isn’t collaborating.
- We may get a request for more information from engineering, but sometimes it is for irrelevant information or information already given in the original report. This is less a function of the software and more a function of developers taking care to read reports and communicate with the reporters rather than go through the motions of requesting standard information, especially when it has already been provided. Treat each bug and reporters as an individual, not another tally mark.
- All this makes us feel like our radars make little difference. This is a dangerous place to be. If people begin to question the value of the bugs and feedback they’re submitting, they eventually stop submitting it. The only things worse than a bug without enough information is not having the bug filed at all.
By making radars so hard and painful to file, most developers end up not filing them. For every radar that is filed, there are many more that developers would file but don’t consider it a big enough issue to be worth the time. It may be a small bug or feature request, or it may be a common issue that we figure someone else has already filed so there’s no point wasting our time telling you about it.
- By making radars so hard and painful to file, most developers end up not filing them. On top of giving reporters reason to question the value of filing the bug reports, they’re difficult to file. These little reasons all eventually add up to bug reports simply not being filed at all.
- …it may be a common issue that we figure someone else has already filed so there’s no point wasting our time telling you about it. That about sums it up. With bugs, or any feedback, duplicates are rarely pure duplicates. Every duplicate tells you more about an issue. Even if it only helps you understand the frequency of the problem, that’s valuable. In the best cases, though, duplicates can actually provide a different insight to the problem, and often they provide the insight that helps you get to the bottom of things. The open letter goes on to make specific suggestions about how some of these problems can be remedied, but the suggestions boil down to making it easier to file bug reports and making it easier to provide additional information. Bug and issue tracking is a two-way street. The people reporting the issues are just as important as the people reading and fixing them. Marco Arment adds some anecdotal insight that validates these concerns…
Despite reassurances from Apple people to the contrary at WWDC, it sure looks to us outsiders that most of our bug reports go unread or skimmed, filed away, and ignored. But it takes a lot of time to file a good bug report, since the filer should take reasonable measures to ensure that it’s truly an Apple bug and not something else in their code.
- …it sure looks to us outsiders that most of our bug reports go unread or skimmed, filed away, and ignored. File this one under “perception is reality”. If people don’t file bug reports, then bugs don’t get fixed. Feeling like the bug reports don’t matter is the first step in giving up. That’s a dangerous feeling.
- But it takes a lot of time to file a good bug report, since the filer should take reasonable measures to ensure that it’s truly an Apple bug and not something else in their code. This is an often over-looked aspect of bug reporting, but even once you factor out the interface for the reporting the bug, the process of isolating and documenting the bug is already time-consuming. That means that good bug reporters are investing their time to tell you how to improve your software. Making it difficult to report the bug on top of the effort they’re already making is like a slap in the face to the people taking the time to create detailed bug reports.
The recurring story behind all of this is how it makes people feel about reporting bugs. Regardless of the company behind the system, there’s a lot we can learn from these sentiments. Bug tracking is a decidedly technical process, but there’s an incredibly important human component as well. The people taking their time to report bugs are priceless, and if they feel like that contribution doesn’t matter, they will stop contributing. If that happens, the tool itself becomes irrelevant.