15th August 2009, 9.00 PM sharp, I am told to download, install and test WireMaster v0.46. Mission: To report bugs in the product in 1 hour(Attack In and Out!)
Product Overview:
WireMaster is a user interface (UI) tool for creating wireframes in an open, XML based format. A wireframe is a basic visual guide used in web design to suggest the structure of a website and relationships between it's pages.
Schedule:
9.00 PM - Gather on gmail chat, download and install the application. Discussions/queries on how to start testing(if any) and advice to the newbies in BWT-3 on how to proceed.
9.30 PM to 10.30 PM - Break off from chat(stay online though) and Start attacking WireMaster.
10.30 PM to 11.30 PM - Discussion of bugs in a Round Robin fashion where each tester reports one bug, next tester reports one of his/her bug and so on until all testers are done with discussing all of their bugs (Originally, the time for discussion of bugs was 30 minutes, but there were about 97+ defects found and needed 1 hour).
Testers:
Karthikeyan, Bhargavi, Parimala Shankaraiah, Jassi, Shruti Gudi, Ajay Balamurugadas, Manoj Nair, Pradeep Soundararajan and Rajesh
Final Reports:
1. My Report
2. Consolidated Report
What were the Challenges?
Challenge 1
Learning and Understanding the product a few minutes before starting on testing the product
How to overcome this Challenge
This is an ongoing universal challenge which every tester is going to face in his/her day to day life as a tester. One way I see to overcome this challenge is to enhance the knowledge base as a tester and keep on learning by testing more and more diversified set of products. I would like to quote James Bach's personal vision of Testing Expertise as follows: 'I can test anything under any conditions in any timeframe relative to my standing in my local community, how hard I try and my technical insight'.
Challenge 2
Confused about testing a particular feature in depth versus testing all features at a high level
How to overcome this Challenge
A very critical challenge. Critical because I personally think handling this challenge differentiates a good tester from a not so good tester. As a not so good tester, if I just touch upon all the features without testing indepth(I call this cursory testing), I may find more bugs which are very direct and visible in nature. But what I would miss is a heap of critical defects that are hidden within the features which cannot be found without testing the feature indepth. As a good tester, even if I am testing just one feature in depth, I would atleast uncover all the defects in that feature instead of just touching upon all features. I find testing in depth as a better approach over testing all features using a cursory approach.
Challenge 3
Pressure to file more number of bugs compared to others in a multiple tester setup (Bugs Hungry??)
How to overcome this Challenge
I am very proud to claim that I am a good functional tester(self certified). This claim eludes me in finding defects in other areas like Security, Performance and several other areas. This claim also obstructs my ability to find defects in other areas which are more critical in terms of the entire product itself. I need to improve in my overall cognition of the product or system as a whole instead of focusing just on functionality. I believe that by testing more and more products(mostly open source) catering to different customers, I should be able to overcome this challenge.
Challenge 4
Time Pressure - Just Do It in 1 hour syndrome
How to overcome this Challenge
I have just 1 hour to test a product and I am asked to give all the quality related information I can about the product at the end of that 1 hour. Assuming I am the only tester testing the product, it becomes very difficult to find and report every defect observed - atleast critical defects if not all. Even If I am asked to test this way, I still may not be able to find all the defects that a group of testers would have found (this thought haunts me to an extent as I should be capable of finding atleast the most critical defects!). One way to overcome this challenge is to handle it better both as an individual and in a group. Improving on Testing skills helps here. Like Pradeep Soundararajan says 'Testing is a Skillset and a Mindset'.
Challenge 5
Poor Reporting
How to overcome this Challenge
In addition to testing, it is important to jot down investigation notes, bugs, screenshots/audio/video of the bugs, prepare a report in a formal/semi-formal readable format. Where is the time? There is and there is lot of time sufficient enough to do all this. I need to practise this better. Like I mentioned earlier, in my hunger to find more bugs, I failed on good reporting. Ultimately, if bugs are found and not reported and advocated to the developers, the whole idea of testing the product becomes a waste of precious time and valuable effort. I need to work on better bug reporting and bug advocacy.
What I Learnt?
1. I learned that I overlooked some bugs (Inattentional blindness)
2. I learned that I missed some bugs because I thought they were not bugs
3. I learned that it is not the number of bugs that is important, but the value that each bug brings along that is more important
4. I learned that I need to test whatever I test IN and OUT. Just doing plain running across the features will not uncover some hidden defects which could threaten the quality of the product
5. I learned that time is not a constraint to do anything - it is about how effective one can be when given a time constraint of 1 hour.
6. I learned how poor my bug reporting can be - I need to work on it. Testing and Reporting have to go hand in hand. I should not be spending additional 1 hour after testing to beautify the bug reports.
7. Killer bugs are still eluding me at this point - I need to improve on finding the Killer bugs and finding them quicker.
Takeaways from BWT-3
1. Testing becomes very effective when people with different perspectives, cultural backgrounds and thinking get together and test (I mean attack the product from all sides :-))
2. Though the testers in BWT-3 did not explicity discuss who had to focus on which areas, most of the defects in most of the areas(functional,usability,GUI,Integration testing etc) were uncovered(97+ defects found in 1 hour)
3. Testers in BWT-3 were very encouraging and supportive of other fellow testers' by defects appreciating good defects and discussing non-defects offline (and offline discussions continue...)
4. Great way to test in a virtual group setting and learn to test better
Do you want to be a part of BWT - 4?
BWT-3 Testers enjoyed a lot in terms of testing the product and reporting bugs in a virtual group setting(in the comforts of their own homes). This virtual group setting did not have the hassles of a team lead or a manager sitting on the top of tester's heads pressurizing for finding bugs. Like one tester mentioned 'I learnt how comfortable it is to work without a formal dress :)'. Apart from all the hype that BWT-3 will create eventually, I would like to highlight the fact that this is a group of testers who is very passionate and serious about testing not just individually, but in a group setting and see the advantages that it brings along.
Are you interested? - Send an email to weekendtesting@gmail.com. You are just an email away.
Happy Testing,
Parimala Shankaraiah
http://curioustester.blogspot.com
No comments:
Post a Comment