The Culture of Reporting Bugs

Did you know that Plone has a QA team? It does! The QA team is just getting off the ground, so it's a great time to talk about the culture of quality assurance. Since I don't use enough old english in my posts, I want to kick-off the topic with a couple of QA Commandments that when recognized by all members of the community ensure a yummy culture of quality. The first part involves bug reporting, and they go something like this:


1.Thou shalt report every bug, big and small, content, documentation, infrastructure, and beyond

Is it a spelling mistake? Report it. Is the interface about as useful as a bag of sand? Report it. Is it impossible to find the documentation you are looking for? Report it!

Will your bug get fixed in a timely manner? Probably not. But that's ok. The point is that you noticed something that was below perfection, and you cared enough to document it by writing a bug report.

I worked on a QA team right out of college and I am proud to say we worked just as hard (if not more!) as the developers. There was constant arguing between the teams but together we delivered a super stable end product. A key component of our strategy was competition within the QA team itself over who could report the most bugs. There was no prize, just pride. And it worked! Every nook and cranny of that system that wasn't perfect was documented.

2. Thou shalt not bitch about bug reports

This follows from commandment 1. Good QA people care even more than the end user about a perfect experience, and caring takes a lot of effort. The thing that happens when developers stop complaining about bug reports is that it actually encourages people to report more bugs - imagine that!

A good set of bug reports should be treated like problem documentation. The better documented the faults are the easier it is to fix and the easier it is to provide support until that fix goes through (since it may be a while). 

The best part is that it's easy to encourage this culture. The next time you close a bug, big or small, thank the original reporter. 

3. Thou shalt make it easy for everyone to report bugs 

Yes you! If you want to find lots of bugs you need to make it easy to report them. This makes me nuts with just about every software product known to man. One click to report a bug. No screens, no searching, just get the info and get them outta there. And yes, it's on my list of things to change with Plone.org.

Unfortunately, I think Plone unnecessarily suffers from the bug collecting of zope and zodb. Searching for a zope bug collector without the exact keywords in google returns two bug collectors, one from 2003 and one from 2005. Both abandoned. And if you report in the wrong one they just close the ticket and that's it. Screw you end user for not being able to differentiate between a plone, zope, or zodb bug. Boo.

So I am putting a list of collectors here for my reference as well as yours. Go ahead, report that bug you've been sitting on for a while. 

• Plone bug
• Plone.org bug
• Plone Documentation Bug
• Zope2 Bug
• PAS
• CMF Bug
• ZODB Bug
• Zope3 -> Bluebream

Now, if a bug gets marked invalid because it's someone else's problem you can just move on to the next tracker and follow through there. Did I miss any? 

4. Thou shalt stand by that bug and not be bullied by developers

This is the hardest part of any QA effort. It's easy to give up when a developer spouts some mojo that you just don't understand. Nonsense poopy pants! Good QA people are so annoying developers wish they would go bury themselves in a hole in a forrest. And you know what? They are often wrong. But a lot of the time they are right, and a HUGE bug manifests from what seems like a template error. That's when it pays off to be a PIA.

So stick to your guns QA jedi's and do what's right for the end user. If you always keep them in mind the end result will always be what's best for the product and the community.