Buggy reporting talks about how buggy real bug reports can be. It bugs me a lot to read buggy bug reports which hardly talk about bugs in a understandable way. As testers, we all know that bug reports are important for providing information to the stakeholders. Still, we don’t take any effort to review our own bug reports. If we ourselves are not interested in reading our own bug reports for clarity, how can we expect programmers and other stakeholders to not just read them, but also decide whether to fix problems reported in those bugs? Height of Optimism!
Awesome Titles
“Script warning”“Spelling mistake”
“file not loading” and
My personal favorite “Thomas is a fool!”
“Thomas is a fool” was a real bug in one of my previous organizations. As part of learning a complex component, I was reviewing all the older bugs that were reported and fixed within that component. That’s when I chanced upon this “Antique Piece". Thomas was one of the managers in the organization who may or may not have done anything good to one of the employees who literally reported this bug on last day of his employment. Quite a cool bug, I never know if Thomas was a bug or the guy who called Thomas a bug was a bug. Nevertheless, If you plan to report such a bug, please don’t quote me on that!
Jokes apart, let’s change roles for a moment as we become programmers. If you are entrusted with the responsibility of fixing bugs with bug summaries as above, what would your first thought be? Scratch your head or Trash them? Once, twice, thrice. If I were a programmer, I would trash not just the bugs, but even folks who write such bugs. And yes I am harsh, because every important bug that fails to catch stakeholders attention due to bad reporting is as good as a dead one.
Some of us might wonder “Oh! The product is so buggy that I have to report at least 20 bugs a day. With whatever little time I have, I can only make my bug reports shorter so I can write more bugs in a day”. My question to such testers is “How much are you bothered or not bothered if none of the 20 bugs reported this way are fixed”. Don’t tell me “That’s not my problem”. Agreed that we are information providers and not decision makers. But are we really providing useful information? Just a thought.
Detailed Descriptions
“Why I need to login?”“search for item, I got script error”
“when my transaction failed, link to database cannot establish”
“in payment by debit card the bak name is choosen from the drop down menu , after that there is no requirement in the gateway page to ask the name of the ‘Issuing Bank’”
Don’t be surprised. These are real descriptions of real bugs. In other words, "Steps to reproduce" if that is the word your organization uses. At first sight, it might look like I am making fun of all those non-English speaking testers who confess openly about their lack of good English writing skills. Bug reporting is much more than knowing English well. Can’t we just look back and read/re-read the bug before hitting the submit button? Is that so hard?
“What’s the fun in detailed reporting? Am I so jobless to type tons of information for every bug I report? Can’t programmers figure out such a simple thing? Won’t my expertise go away if I gave all this information out for free? Let’s be honest - I struggled a lot to get here and find this bug, why should I give away information just like that. Let people come back to me, I’ll tell them and I’ll tell them only if I like them”
Some descriptions are so painless. A simple copy/paste of the bug summary. It’s as though the bug reporter was held at gunpoint and asked to report a bug one last time before giving away his life forever and ever. Or maybe his bowel movements didn’t permit him to do so in the time available for him to rush from his desk to the washroom. No offense meant – frankly it’s that annoying to read bad bugs to figure out what’s going on. Total waste of time.
Attachments
We have many free tools to take screenshots, many more to capture videos and above that, we have the freedom to attach logs, xmls, data files, test files and lot more options to help with bug investigation and fixing. Again, the same old question – who has the time? We don’t want to provide anything until the bug goes to “Needs more Information” state. And we would be offended if it’s moved to “Invalid” state, all this for no mistake of folks who read our dead bug reports.Reputation Points
1. Bug reporting skills
2. Investigation skills
3. Ability to provide detailed information so that programmers don’t float between the bug and the bug reporter to get additional information
4. How many bugs are valid versus invalid?
5. Questioning skills (No, not the “why I need to login” type of bugs)
6. How many of the bugs reported did the tester get fixed finally?
All that mattered to the management was Rajesh has reported the highest number of bugs and he should be given the ‘Star Tester of the Quarter’ award in Q4 2010.
What a fishy method to earn reputation? Just because you are testing a buggy product or you find a bug every minute doesn’t mean you have to report bugs in a lousy way. Just because you want to beat your counterpart in the number of bugs reported as part of the bet you had with him or your manager rates you this way, doesn’t mean that you can stoop to any level of lack of quality in your own bugs.
If you are a tester, go back and read your bug reports as if you did not report it at all. Identify the gaps and work on those gaps by practicising writing good bug reports. If you are programmer who is annoyed with your tester’s bug reporting skills, tell them politely about what basic information you expect instead of running to his desk every 5 minutes which wastes not just his time, but yours too!
I recently met up with a CTO of one company whose product was tested by a bunch of testers outside his organization. When I questioned him about what he thought about the bugs reported by independent testers, he just said one thing, “Each time I read the bugs reported by these testers, I often wondered - why aren’t our in-house testers reporting bugs this way…….. " And he kept scratching his head for the next 30 seconds after uttering this. I felt a lot of pain in what he said. I wish he goes back to them and politely tells them that they need to improve in their bug reporting skills.
Happy Bug Reporting,
Regards,
Parimala Shankaraiah
I worked at a company, where the bug situation got that worse, that we printed out some of the bug reports, just to collect them.
ReplyDeleteMy personal favourite came from one our operations guys. The title was:
"please open bug"
and came as a request initially from one of our customers. The bug description was a full copy & paste of the complaints (two lines of text), and I was hopelessly lost what this bug was all about. Since then I haven't crossed a worse bug, though, but I also complaint about this. "Please open bug".
Parimala nice post and you bought up almost every point.
ReplyDeleteI wanted to bring up one point regarding tools section you talked about. I have been in the situation where there are lot of bugs in product and its really difficult to capture everything. Most of the time I have thought of writing tools which will automatically fetch information from logs and other such files while I am testing. This can be one thing which saves time in investigation.
Other important thing testers miss out is looking for duplicates. It may be hard to find if duplicates in a big database, but most don't put effort even to do a quick search to figure out the duplicates. For a real passionate tester total number of bugs logged doesnt matter it's all about how many are valid and how many are fixed that helps to improve the quality of product and also that of testers.
ReplyDeleteHere is my favorite lesson from Lesson learned from software testing book Lesson 101(I like the number as well :-) ) “ When You Dedice to fight,decide to win!”
For that you should also be aware of few other lessons Lesson 55 : You are what you write , Lesson 58 : Your bug report is your representative Lesson 83 : Summary line is the most important line in the bug report.
-Dhanasekar S
PS : I wanted to comment in one line but I could'nt ;-)
@Markus Gartner
ReplyDeleteMy personal favourite came from one our operations guys. The title was: "please open bug"
Often times, its the frustration that takes the shape of such bugs. And behind these frustrations is a serious problem that needs to be addressed.
@Nitin Purswani
Most of the time I have thought of writing tools which will automatically fetch information from logs and other such files while I am testing. This can be one thing which saves time in investigation
Strongly Agree with you. I wrote a simple log parser script that would filter all HTTP errors generated on our server. It was such a new lease of life when compared to scanning a 2 GB logfile that was nothing less than painful. I would love to hear from you about the tools that you have developed!
@Testing Ideas
For a real passionate tester total number of bugs logged doesnt matter. it's all about how many are valid and how many are fixed that helps to improve the quality of product and also that of testers
Spot on! I missed this point though we discussed it the other day ;-). I think this calls for a separate blog post. BTW, I loved your one liner blog post ;)
Regards,
Parimala Shankaraiah
@Parimala I also could achieve much on this front due to time constraint. Parsing errors/exception from log files is all I have done .
ReplyDeleteI have few ideas but could not convert much in to actual tools. They are
-writing a file watcher which reports when any file changes on workarea I am working on.
-Reporting CPU spikes,Memory usage automatically and correlating that with log files
Sorry for typo, I wanted to write "I could not achieve much on this front", but I missed "not" .
ReplyDeleteGood Article Parimala.Even some of the good bug reports also need clarification .Bcos even we post well some of the developers unable to understand due lack of communication skills to them.We post technical terms like popups,roomful,validation msg,dojo etc..but what can we do even they get confused with their learning terminologies.
ReplyDeleteLast time when I posted bug like "am being displayed session expire msg each time when clicked on submit button". ...concerned module developer came to me and asked me: how can u display that msg? As I wrote am being displayed...
prabhakarvuram.blogspot.com
Good post Parimala. I would like to add one more thing. A buggy reporting has increase the cost of the product and time because of communication/discussion between the dev and the tester.
ReplyDeleteNice post, Parimala :)
ReplyDeleteThe problem about testers being judged on number of bugs hits too close to home. I would add one more problem here. In one of the orgs I worked at, the programmers could never understand one of the testers' bugs that they had to discuss almost every bug with him, while another tester's bugs were reported neatly and clearly with all the apt screenshots and logs. But when the "judgment time" came, the former tester was considered to have better communication skills and a better team player than the latter.
Such problems are all too common I feel. Are there any solutions to these?
Looks like poor bug reporting got the better of you. Well I agree that language is not a constraint for a "sensible" tester to convey what he/she has found. And it's a sad reality especially in large Services companies that testers are evaluated based on number of bugs, test cases etc and not on quality of work. Resolving this requires mindset change and great deal of testing evangelism and we're getting there slowly. Keep your posts coming !
ReplyDeleteReally nice post, not just to read only but to go and implement/correct somethings in bug reporting for many of the testers like me :). Especially this will be good information for new babies and I would consider bug reporting should be added as one of the induction topics in every organization which I don't think many are following.
ReplyDelete