Bangalore Weekend Testing 8 (BWT – 8)

Date and Time
20th September 2009, 3.00 PM, BWT testers meet up on Google Chat.

Mission
Choose one of the quality criteria out of the following –
Installability / Usability / Performance / Reliability / Compatibility /Testability.

Product Overview
Areca-Backup is a file backup software that supports incremental, image and delta backup on local drives or FTP servers.

Testers
Ajay Balamurugadas, Vasupratha, Bhargavi, Sudhakar, Gunjan, Parimala Shankaraiah.

Reports
My Report
Ajay's Report

Approach
The beauty of BWT 8 lay in the fact that there was a flexibility to choose the mission by selecting any one of the Quality Criteria listed: Installability / Usability / Performance / Reliability / Compatibility / Testability. Previously, I was fascinated by the results of testing Nokia 1650 Insert Options by breaking down the feature sets and choosing one small testing chunk to test. I wanted to learn to better my previous testing performance. I chose Installability as the quality criteria - something that I had not tested earlier in session based format. I set out to test and loved the entire experience.

One thing I noticed in this session was that a few testers chose the much known, much easier quality criteria. In short, they chose to be in their comfort zones. Please note that if any product is given to test with a vague mission like find bugs in the product, we as humans have a normal tendency to choose areas that we are comfortable with. For eg. I love functional and usability testing and I would just jump at any opportunity to do just this . Over a period of time, I realized that other areas like performance, claims, reliability, compatibility etc are important as well based on what the stakeholder demands. Hence, as a tester, it is good to have basic skills in different test techniques if not great skills. This in turn can be improved over a period of time by practicing more and more.

The best way to learn is to push yourself out of your comfort zone. Test what you are not sure about. Test what you do not know about. Test what you have not heard about. Ask questions. Find answers. Question your answers. And ask more questions. The final answer could be well within your reach. You be your first critic.

Advantages of testing smaller chunks of the product
1. Brainstorming one feature/task results in an umpteen number of test ideas
2. Improved Focus on a single feature/task
3. Adequate testing can be done in a 90 – 120 minutes session
4. Finding lot more valuable and hidden bugs (bugs hungry? – Naa!)
5. Highly buggy features can be retested in future sessions possibly in short session say 60 minutes

If you have tried testing by breaking testing tasks into smaller chunks, feel free to add to the above list.

Get Uncomfortable Today,
Parimala Shankaraiah
http://curioustester.blogspot.com

Session Based Testing – Nokia 1650

I have not explicitly tested mobile devices though I have found issues on different types of cell phones that I have used in the past. This gave me an idea to test Nokia 1650 model cell phone that I own currently. I chose to test the 'Insert Options' feature in Create Message functionality. I quickly glanced through the Insert Options feature at a high level, broke it down into different types of Insert options available (smileys, words, numbers, symbols and templates), how different these options are when the dictionary is on and the language chosen is English or Hindi, how different is the behavior if the dictionary is turned off, what are the effects of changing the text format to different sentence cases like uppercase, lowercase and title case. The testing checklist was ready. Please find it HERE.

As the session unfolded, I tested dictionary on (English), dictionary on(Hindi) and dictionary off sub-features. I referred to the checklist from time to time to ensure that I was focusing on the charter and not deviating from it at any point. For eg. When I was testing the Insert Options with dictionary On(Hindi), I found that the Create Message > Options feature has 4 additional options like Text: A B C, Text: a b c, Numbers: 1 2 3 and Insert Symbol. I was surprised because these were already available in the Insert Options. Redundant feature? May be. This was a guaranteed deviation from my current charter and hence I noted it down as an opportunity and continued testing. Please find the Session Based Test Report is HERE.

My Learnings:
1. It is a good idea to break the charter of the session to the smallest possible task(chunk) so as to focus and test it adequately(Note that I am not using the word completely).

2. I can get rid of a separate testing checklist document by adding the checklist items in the Task Breakdown section of the Session Based Test report.

3. Any activity related to this testing session should be done within the duration that was originally planned.

4. Opportunities play a spoilsport by deviating the tester from the charter unless the tester gives in to it resulting in boondoggle.

5. Opportunities in current session (outside of the current charter) can become the charters for future sessions.

6. Test Notes section should have information about observations and inferences made while exploring, understanding and testing the product. Any bug should get to the Bugs sections and queries if any should get to the Issues sections.

7. Add Risk field to talk about the risk associated with each bug(the risk of not getting the bug fixed) and the Customer Impact field to talk about any possible impact to the customer. In general, It would be nice to add Risk, Customer Impact and Oracle/Heuristic fields to indicate why and how the bug was found (OK, I am tweaking the original template suggested by Jonathan Bach).

8. Issues in the Issues section should be followed up without fail with developers and product management teams for further clarity and follow up testing in a separate session.

Ever since I read Jonathan Bach’s article on Session Based Test Management, I have been thinking how it can be adopted in any company which takes the conventional route to testing. Can you dare to tell your manager that you will no longer write test cases? Can you tell your manager that he/she cannot generate any reports automatically anymore? Can you tell them that you test some areas here and there(explore), but nothing in particular? Can you just convince your manager about your work just by showing the bugs you have filed? What if you did not find any bugs? According to the conventional testers, the main problems with Exploratory testing is that it is not measurable, auditable and manageable. If you read Jonathan Bach’s article above, you will come to know that these ‘so called’ problems are not problems at all. I will work on answering the above questions by writing a follow up post to this in the near future. In the meantime, if you want to answer them, please feel free to add them in the comments section.

Happy Testing,
Parimala Shankaraiah,
http://curioustester.blogspot.com

Never Give Up – Atleast I did not!

Recently, RaviSuriya who is one of my blog readers reported two observations that he had encountered on my blog which are listed below:

1.‘Subscription Links' gadget not displaying all the 6 available options to Subscribe for Posts and Comments
2.Date field was missing for one of the posts on the blog

There were a few email exchanges between me and Ravisuriya during which I gathered information about his observations. I was very glad when he sent over a detailed investigation report for the first issue.

Whew! I had got some food for my brain. I started off on my second attempt at Session Based testing. This time around, I was very firm on working in isolation so that I would not be interrupted for any reason whatsoever.

Day 1 – Just played around the blogspot.com features on my blog as well as by creating a brand new dummy blog and noted some basic observations using OFAT (testing One Factor at a Time)*

Day 2 – Started testing blogspot.com with regard to the above 2 behaviours, this time using MFAT(testing Multiple Factors at a Time). Spent around 5 hours testing different scenarios and each time I discovered something new about the behaviors. It was taking a lot of time to pinpoint when exactly the above behaviors were encountered. The report of this amazing testing experience is HERE.

*Factors in this case would refer to Widgets on blogspot.com.

Token of Thanks: I thank Ravisuriya for reporting the above 2 behaviours. In general, I am amazed by the openness of the Context Driven Testing Community to take time to read fellow testers’ blogs, share their personal opinions/views, provide live examples to learn, and in the meantime encourage countless upcoming testers to pursue testing with utmost passion. A Big Thank You to all the Context Driven Testers who are selflessly helping a lot of fellow testers knowingly and/or unknowingly. I foresee a wonderful Testing World already of which we all are part of or going to be part of.

"There is only one way to learn, It's through action. Everything you need to know you have learned through your journey” – from the book ‘The Alchemist’ by Paulo Coelho.

Keep your feedback pouring in here :-)
Happy Testing,
Pari - http://curioustester.blogspot.com/

I failed miserably!

Like I mentioned in my previous post, I wanted to practice what I learnt from the ET Workshop that I had attended. I picked a feature in the product that was not tested by me in the past.

Here goes my report:
1. Played around with the feature for better understanding
2. Brainstormed a few test ideas for testing that feature
3. Shortlisted a few areas/sub-features to be tested(used an Excel Checklist)
4. Created a text file similar to the one used in Session Based Test Management report which included Mission, Date and Time, Observations and Issues sections (rough draft at this point)
5. Started testing the feature. It was just 1 minute later when my manager informed me of an urgent unscheduled meeting. 30 minutes gone! Back from the meeting and moved the mouse over the feature and 2 developers standing next to me expecting me to help them understand a trivial UI issue which I had explained in a clear and precise manner in my defect report. 5 minutes gone.
6. Got back to testing and an urgent request for reviewing the product guide!
7. Stopped testing without any actual testing and ended up with an empty report

Doesn’t it look like our traditional test case procedure executed pointlessly and without ample reasoning. I was ignorant of the several interruptions that could happen while I Time Box myself for testing a small feature. I failed miserably at my attempt to do Exploratory Testing in a session based manner the first time. Each time I fail, I am happy to believe that it is an opportunity to learn. The faster you fail, the quicker you succeed.

Watch this space for more is coming :-)
Happy Testing,
Pari