22 August, 2009

Bangalore Weekend Testing 4 (BWT 4)

22nd August 2009, 3.00 PM sharp, BWT testers assemble (on Google chat). The product was Free Mind 0.9.0 RC5. Mission: The Mission is to test the entire product and find bugs.

Product Overview
FreeMind is a premier free mind-mapping software written in Java. The recent development has hopefully turned it into high productivity tool. We are proud that the operation and navigation of FreeMind is faster than that of MindManager because of one-click "fold / unfold" and "follow link" operations.

3.00 PM - Gather on gmail chat, download and install the application FreeMind.
3.30 PM to 4.30 PM - Testing FreeMind.
4.30 PM to 5.30 PM - Discussion of Test Strategies used for Testing.

Ajay Balamurugadas, Anil Sonune, Parimala Shankaraiah, Rajesh Iyer, Ravisuriya, Vivek Bhat, Jaswinder Kaur Nagi

Final Reports
1. My Report
2. Consolidated Report

In BWT 4, I wanted to overcome the challenges I faced in BWT 3. Though the mission was to find bugs in the product, I was concentrating more on overcoming my shortcomings from previous BWT session (a slight shift in the mission). I was able to shift my mindset from finding more bugs to finding quality bugs. Instead of rushing for bugs, I focused mainly on Installation testing, Basic functionality and Help for FreeMind. I was able to overcome my pressure to perform/find more bugs in a short period of time. Like it or not, there is always going to be time pressure whether it is in hours, days, months or even years in the worst case. The only way to handle it is testing the critical features in the product and uncovering bugs, then move on to next critical feature etc. My reporting capability has improved quite a lot.

I still had the challenge of learning/understanding the product in a short period of time and test. I am yet to figure out the following based on a question Ajay asked me during BWT 4:

1. Should a product be learned/understood for testing or just find defects without learning the product?
2. Does understanding the product better help uncover more bugs while testing?
3. Should learning and testing the product be a simultaneous activity?

I am still thinking about these questions. Any thoughts are welcome.

Happy Testing,
Parimala Shankaraiah

21 August, 2009

Are you an Ostrich sticking your head in the sand?

One particular discussion in TestRepublic ‘ISTQB and CSTE' caught my attention. There is an interesting trend in how students approach testing certifications.

Different ways of clearing testing certifications:
1. A student studies and practices in and out of the subject by referring to at least some of the reference books, gains tremendous knowledge and attempts the certification exam.
2. A student studies all the model question papers, previous years’ question papers, training material given by some institutes which train students to pass such certifications etc. Such a student would ideally just prepare for the exam by rote and dump it onto the answer sheets.
3. A student attempts all the questions through wild guessing and depends on luck(I had read on one of the ISTQB discussion forums that for most of the questions, if a student marks (B) or (C), there is a high probability to pass the exam easily)
4. A student does a bit of (1) and (2) above.

In each of the above cases, based on the type of questions that appeared in the exam and based on whether the student had prepared for those specific questions, he/she will answer accordingly.

Format of Testing Certifications
The question paper format for most of the testing certifications is objective in nature. Irrespective of whether you have knowledge or not, if you know what is the question and what is the answer and you can match them correctly, Bingo, you pass the exams! It is as simple as ‘Match the Following’ exercises given to 3rd/4th/5th grade students.

Students run behind certifications because:
1. Better Future Job Prospects(the so called hype around it)
2. Reputation as a Skilled employee (person A has passed X, Y, Z certifications and hence highly skilled!)
3. Better raises and bonuses (person A has passed X, Y, Z certifications while person B has not passed any. Hence person A deserves a better raise)
4. A lot of clients especially in service oriented companies emphasize on certifications and standards
5. A non-computer science graduate thinks that doing certifications is one of the best ways to get into such fields and testing is the easiest option. Isn’t it?
And the list goes on and on.

What do the students gain?
When the so called X, Y, Z certified testers get back to their offices, the only change they carry with them is the new certification.
1. Change the Email Signature to ISTQB/CSTE/CSQA/CSTP certified engineer
2. Put up a photocopy of the certificate in the workstation
3. Add the certification logo to the resume to attract interview calls
4. And brag about the certification to fellow colleagues who have not even heard of it.

What did the students gain? – Loss of MONEY, TIME and GETTING FOOLED

What do the Testing Boards/Companies gain?
The Testing Boards or the companies which are offering the testing certifications at exhorbitant prices are making hell lot of money at the cost of fooling the students. Let us take a look at the profits they make: ISTQB Foundation Level exam costs around Rs. 4,000/- per person. If there are around 300 students taking this exam every month, an estimated 3600 students take up this exam in a single year. Profit: 300*12*4000 = Rs. 1,44,00,000/- per year.

Dear students, forget about the money part for a while. Have you gained any knowledge by clearing the certification? Have you learned any new skills? Have you been able to make a single good change at your workplace(apart from mouthing a few testing definitions which you would have learned by rote).

What did the Testing Boards/Companies gain? – CRAZY MONEY

Are you an Ostrich sticking your head in the sand? You Decide!

Happy Testing,
Parimala Shankaraiah,

18 August, 2009

An Ignorant Tester’s Story

Once upon a time, there was an ignorant tester who followed test scripts(written by someone else) while testing a product. This tester ran tests over tests, updated the test case status to Pass/Fail/Blocked/Not Executed and this went on. At the end of 27 months, the tester got bored because not only did this tester execute test scripts written by others, but the tester never wrote the test cases for any of the new products( Now that is what a Test Lead is meant to do in most companies right?). This tester moved on to another job which required a different skillset. The tester got excited and started learning new things at the new job. At the end of 27 months, yet again the tester got bored because this tester executed test scripts written by others. Apart from learning the new technologies, new products and processes, there was nothing worthwhile that there was to learn in the 2nd job as well. So the tester wanted to move on yet another time.

Its time to prepare for the interviews and the search for readymade test cases in Google. Test cases to test an ATM, test cases to test a marker pen, test cases to test a chair etc. Google for it and you get it! Now, this tester never used the ‘Best Tool in the World – the BRAIN’ and instead opted for Google (Google can be a spoilsport at times! I do not blame it though). Google ‘Test Cases for ATM ‘. Wow. Hundreds of results. The tester thought there is no way to get rejected in the interview if this material is referred to. This tester then goes to one particular link that contains test cases for testing a Notepad and starts reading it. End Result - this tester was Re-born. ‘Why didn’t I use my own brain all this while? Why did I depend on someone else’s test cases? Why did I wait for someone else’s ideas to test? Why Why Why?’ These questions haunted the tester for a long time after that. The next five days just passed by in digesting every little word that was written on http://testertested.blogspot.com/ and http://shrinik.blogspot.com/. Was the tester Enlightened? May be. May be not.

Who was this ignorant tester? I do not want to tell you that this Ignorant tester was a friend of mine or a friends’ friend. I want to tell you that this ignorant tester was ME and this is MY story! I started off as a hard core scripted tester who was dedicated to testing day in and day out running about 5000 test cases for every release! How did I feel at the end of 5 releases in 27 months? Emptyness and Boredom. I did learn a lot of tricks in the trade, the functional knowledge, the technologies, a few scripting languages(for the little API testing that I did) etc etc. Great, Then, why did I get bored? I think that scripted testing is highly automated compared to any other automation done while testing. Open the script, read the test case, execute the test case, record the result as pass/fail/blocked/not executed, move on to the next script and keep doing this until you realize that you did not get promoted though u executed 4000 test cases compared to the guy who got promoted by executing just 2000 testcases! And then start looking for another fancy job which you think might not bore you after 27 months.

Can you sense the idea of getting bored every 2-3 years and start looking for a new job. Can you think about making this shift every two years? What do you gain? What do you lose? Can you dare to answer these questions by ignoring the money part for a while? Can you answer these questions by considering the money it brings along? Are you bored at your current job? Were you bored the same way in your last job? Can you answer these questions to yourself if not others?

P.S: All said and done, I think that searching for ‘Test Cases for testing a Notepad’ has been the greatest mistake of my life which in turn became the greatest learning in my life. It does help at times to be ignorant, but that should not go on for long or else it gets dangerous and/or dirty! This post could appear to be an attack on scripted testers, but it is not. It is an attack on the traditional processes we have been following without asking ‘WHY’. This one question can change your life. Ask ‘WHY’ for your own Good.

Happy Questioning,
Parimala Shankaraiah,

17 August, 2009

Bangalore Weekend Testing 3 (BWT-3)

15th August 2009, 9.00 PM sharp, I am told to download, install and test WireMaster v0.46. Mission: To report bugs in the product in 1 hour(Attack In and Out!)

Product Overview:
WireMaster is a user interface (UI) tool for creating wireframes in an open, XML based format. A wireframe is a basic visual guide used in web design to suggest the structure of a website and relationships between it's pages.

9.00 PM - Gather on gmail chat, download and install the application. Discussions/queries on how to start testing(if any) and advice to the newbies in BWT-3 on how to proceed.
9.30 PM to 10.30 PM - Break off from chat(stay online though) and Start attacking WireMaster.
10.30 PM to 11.30 PM - Discussion of bugs in a Round Robin fashion where each tester reports one bug, next tester reports one of his/her bug and so on until all testers are done with discussing all of their bugs (Originally, the time for discussion of bugs was 30 minutes, but there were about 97+ defects found and needed 1 hour).

Karthikeyan, Bhargavi, Parimala Shankaraiah, Jassi, Shruti Gudi, Ajay Balamurugadas, Manoj Nair, Pradeep Soundararajan and Rajesh

Final Reports:
1. My Report
2. Consolidated Report

What were the Challenges?

Challenge 1
Learning and Understanding the product a few minutes before starting on testing the product

How to overcome this Challenge
This is an ongoing universal challenge which every tester is going to face in his/her day to day life as a tester. One way I see to overcome this challenge is to enhance the knowledge base as a tester and keep on learning by testing more and more diversified set of products. I would like to quote James Bach's personal vision of Testing Expertise as follows: 'I can test anything under any conditions in any timeframe relative to my standing in my local community, how hard I try and my technical insight'.

Challenge 2
Confused about testing a particular feature in depth versus testing all features at a high level

How to overcome this Challenge
A very critical challenge. Critical because I personally think handling this challenge differentiates a good tester from a not so good tester. As a not so good tester, if I just touch upon all the features without testing indepth(I call this cursory testing), I may find more bugs which are very direct and visible in nature. But what I would miss is a heap of critical defects that are hidden within the features which cannot be found without testing the feature indepth. As a good tester, even if I am testing just one feature in depth, I would atleast uncover all the defects in that feature instead of just touching upon all features. I find testing in depth as a better approach over testing all features using a cursory approach.

Challenge 3
Pressure to file more number of bugs compared to others in a multiple tester setup (Bugs Hungry??)

How to overcome this Challenge
I am very proud to claim that I am a good functional tester(self certified). This claim eludes me in finding defects in other areas like Security, Performance and several other areas. This claim also obstructs my ability to find defects in other areas which are more critical in terms of the entire product itself. I need to improve in my overall cognition of the product or system as a whole instead of focusing just on functionality. I believe that by testing more and more products(mostly open source) catering to different customers, I should be able to overcome this challenge.

Challenge 4
Time Pressure - Just Do It in 1 hour syndrome

How to overcome this Challenge
I have just 1 hour to test a product and I am asked to give all the quality related information I can about the product at the end of that 1 hour. Assuming I am the only tester testing the product, it becomes very difficult to find and report every defect observed - atleast critical defects if not all. Even If I am asked to test this way, I still may not be able to find all the defects that a group of testers would have found (this thought haunts me to an extent as I should be capable of finding atleast the most critical defects!). One way to overcome this challenge is to handle it better both as an individual and in a group. Improving on Testing skills helps here. Like Pradeep Soundararajan says 'Testing is a Skillset and a Mindset'.

Challenge 5
Poor Reporting

How to overcome this Challenge
In addition to testing, it is important to jot down investigation notes, bugs, screenshots/audio/video of the bugs, prepare a report in a formal/semi-formal readable format. Where is the time? There is and there is lot of time sufficient enough to do all this. I need to practise this better. Like I mentioned earlier, in my hunger to find more bugs, I failed on good reporting. Ultimately, if bugs are found and not reported and advocated to the developers, the whole idea of testing the product becomes a waste of precious time and valuable effort. I need to work on better bug reporting and bug advocacy.

What I Learnt?

1. I learned that I overlooked some bugs (Inattentional blindness)
2. I learned that I missed some bugs because I thought they were not bugs
3. I learned that it is not the number of bugs that is important, but the value that each bug brings along that is more important
4. I learned that I need to test whatever I test IN and OUT. Just doing plain running across the features will not uncover some hidden defects which could threaten the quality of the product
5. I learned that time is not a constraint to do anything - it is about how effective one can be when given a time constraint of 1 hour.
6. I learned how poor my bug reporting can be - I need to work on it. Testing and Reporting have to go hand in hand. I should not be spending additional 1 hour after testing to beautify the bug reports.
7. Killer bugs are still eluding me at this point - I need to improve on finding the Killer bugs and finding them quicker.

Takeaways from BWT-3
1. Testing becomes very effective when people with different perspectives, cultural backgrounds and thinking get together and test (I mean attack the product from all sides :-))
2. Though the testers in BWT-3 did not explicity discuss who had to focus on which areas, most of the defects in most of the areas(functional,usability,GUI,Integration testing etc) were uncovered(97+ defects found in 1 hour)
3. Testers in BWT-3 were very encouraging and supportive of other fellow testers' by defects appreciating good defects and discussing non-defects offline (and offline discussions continue...)
4. Great way to test in a virtual group setting and learn to test better

Do you want to be a part of BWT - 4?
BWT-3 Testers enjoyed a lot in terms of testing the product and reporting bugs in a virtual group setting(in the comforts of their own homes). This virtual group setting did not have the hassles of a team lead or a manager sitting on the top of tester's heads pressurizing for finding bugs. Like one tester mentioned 'I learnt how comfortable it is to work without a formal dress :)'. Apart from all the hype that BWT-3 will create eventually, I would like to highlight the fact that this is a group of testers who is very passionate and serious about testing not just individually, but in a group setting and see the advantages that it brings along.

Are you interested? - Send an email to weekendtesting@gmail.com. You are just an email away.

Happy Testing,
Parimala Shankaraiah

14 August, 2009

BWT - Do you wanna Rock?

A strong passion to test within a group of tester friends ignited an idea of testing for a couple of hours over the weekends very far away from the pressures of a regular full time job. It started off with myself and Ajay testing a website and posting results of our testing efforts HERE. The subsequent weekend had Ajay, Sharath and Manoj testing another website. View the Experience Report HERE. This gave birth to 'BANGALORE WEEKEND TESTERS'. It is a group of people who will pick any random open source project and test alongwith other testers all in a virtual setup, and publish Experience Reports.

When is the next BANGALORE WEEKEND TESTERS Session?

Date: Aug 15th 2009
Time: 09.30pm IST

If you want to join the session or need more details, Send an email to weekendtesting@gmail.com

It is Indian Independence Day on 15th August. Lets fight for Indian Testing Independence. Lets work towards better testing community. Come join hands with Us. You can be from any part of the world, not just Bangalore.

We need more and more BAD Testers (Bringing Awesome Defects Testers).

Happy Testing,
Parimala Shankaraiah

P.S: BAD Testers(Bringing Awesome Defects Testers) is a term coined by one of my tester friend RaviSuriya.

06 August, 2009

The Silent Spectator – Please Speak Up

Until recently, I used to ghost follow several blogs. By this, I simply mean that I read a lot of blogs, absorbed them if I liked them or ignored if I didn’t. While reading one particular blog post, I realized that I had faced a similar situation in the past and I felt that I had to share that story. I even thought that my story was better than the story of that blog post. I made up my mind to write my story in the comments sections. Hurray! The blog owner was amused and got interested in knowing more about it and hence started our journey of ‘Learning from each other’.

How many of us add comments on the blog posts we read? I am not saying that the blog reader should just add some random comments like ‘Great Post’, ‘Nice Read’ etc etc(I have done this in the past too). Though such comments do appreciate the blog owner, it neither helps the blog reader nor the blog owner in terms of learning. All I am saying is that if you have a better story to share and you think that it will do good to a larger audience, do share that story. Who knows 'yours could be the best story ever thought or heard of'. You could be sharing it on your own blog or on different forums where people are involved in discussions or by adding comments to the blog posts that you read on different blogs.

As a blog reader myself, I used to think what do I gain by sharing my side of the story on someone else’s blog? Trust Me, Its human to think this way. I realized slowly that by doing this, I was waking up myself from the close knit shell that I had limited myself to. I started asking a lot of questions to myself. I started questioning other testers/bloggers. I started challenging what I already know. I started challenging others about what they know. And finally, I started Thinking in the right direction(As a human, I could still err. but that would be fine as long as I learn from them).

There could be Deep Thinkers who do not blog. There could be bloggers who may not be thinking effectively(if that is the right word to use). But all this is just part of the Learning Curve. Sooner or Later, everybody is going to learn from their experiences and mistakes, the easy way or the hard way. This is my side of the Story? What's Yours?

Happy Testing,

01 August, 2009

Paired Testing at a Distance - Part 1

Yet another TE(Thriller Experience) in Testing. Myself and my friend Ajay teamed up on Saturday night at 8.15pm to test a website. Website testing - A lot of us would have tested several websites both big and small. Now the website we thought of testing was http://www.vischeck.com/vischeck/vischeckURL.php (It was Ajay’s idea to test this website).

What is Vischeck?
Vischeck is a way of showing you what things look like to someone who is color blind. The first thought that came to my mind was I am not color blind! As a person who is not color blind, testing a website which is caters to color blind people seemed a little unjustified. However, it challenged me to figure out what a non-color blind person like me could infer from such a website.

Summary of Paired Testing
Myself and Ajay agreed upon a particular page http://www.vischeck.com/vischeck/vischeckURL.php for testing. The Duration was 30 minutes for testing and 30 minutes for discussing the issues we found. It was a challenge to interact with each other as we were not face to face, but on Google Chat! Start 8.36 pm and we got disconnected from each other and started testing. We decided to regroup(via chat) at 9.06 pm and discuss the issues. At 9.06 pm, we stopped our testing effort and started discussing the issues. Please find my experience report HERE. You can find Ajay’s experience report HERE.

What went right?
1. Good number of defects in 30 minutes time
2. Good quality of defects in 30 minutes time

What went wrong?
1. Limited knowledge of the website which caters to color blind people
2. Limited knowledge about the expectations of the color blind people
3. Limited time. I think 30 minutes was too short for me to absorb information about the website and test it at the same time
4. Did not test with multiple browsers
5. Was suffering with severe backache(probably slightly reduced productivity)

What could have been better?
1. Should have brainstormed about the product before testing
2. Documentation could have been better for the defects
3. Screenshots were not captured to detect more defects (Hunger for more defects stopped me from taking screenshots of the issues)
4. Wasted some time in the 30 minutes of testing getting some queries answered and doubts clarified(Unclear mission). I would have loved the idea of being seated next to Ajay and validate each defect as we find them, but in this case we were interacting on chat which ate up a little time exchanging information during testing.

To summarise, it was encouraging to see how the defect count can go up while two or more testers pair up to find issues at the same time. For myself and Ajay, it was Different perspectives, Two People and One Mission. This worked really well, Looking forward to a lot more testing challenges as part of Paired Testing.

I would like to thank Ajay for his Valuable Time and Passion :-)

Pair up while testing - it ROCKS!
Happy Testing,
Pari – http://curioustester.blogspot.com