15 September, 2009

Session Based Testing – Nokia 1650

I have not explicitly tested mobile devices though I have found issues on different types of cell phones that I have used in the past. This gave me an idea to test Nokia 1650 model cell phone that I own currently. I chose to test the 'Insert Options' feature in Create Message functionality. I quickly glanced through the Insert Options feature at a high level, broke it down into different types of Insert options available (smileys, words, numbers, symbols and templates), how different these options are when the dictionary is on and the language chosen is English or Hindi, how different is the behavior if the dictionary is turned off, what are the effects of changing the text format to different sentence cases like uppercase, lowercase and title case. The testing checklist was ready. Please find it HERE.

As the session unfolded, I tested dictionary on (English), dictionary on(Hindi) and dictionary off sub-features. I referred to the checklist from time to time to ensure that I was focusing on the charter and not deviating from it at any point. For eg. When I was testing the Insert Options with dictionary On(Hindi), I found that the Create Message > Options feature has 4 additional options like Text: A B C, Text: a b c, Numbers: 1 2 3 and Insert Symbol. I was surprised because these were already available in the Insert Options. Redundant feature? May be. This was a guaranteed deviation from my current charter and hence I noted it down as an opportunity and continued testing. Please find the Session Based Test Report is HERE.

My Learnings:
1. It is a good idea to break the charter of the session to the smallest possible task(chunk) so as to focus and test it adequately(Note that I am not using the word completely).

2. I can get rid of a separate testing checklist document by adding the checklist items in the Task Breakdown section of the Session Based Test report.

3. Any activity related to this testing session should be done within the duration that was originally planned.

4. Opportunities play a spoilsport by deviating the tester from the charter unless the tester gives in to it resulting in boondoggle.

5. Opportunities in current session (outside of the current charter) can become the charters for future sessions.

6. Test Notes section should have information about observations and inferences made while exploring, understanding and testing the product. Any bug should get to the Bugs sections and queries if any should get to the Issues sections.

7. Add Risk field to talk about the risk associated with each bug(the risk of not getting the bug fixed) and the Customer Impact field to talk about any possible impact to the customer. In general, It would be nice to add Risk, Customer Impact and Oracle/Heuristic fields to indicate why and how the bug was found (OK, I am tweaking the original template suggested by Jonathan Bach).

8. Issues in the Issues section should be followed up without fail with developers and product management teams for further clarity and follow up testing in a separate session.

Ever since I read Jonathan Bach’s article on Session Based Test Management, I have been thinking how it can be adopted in any company which takes the conventional route to testing. Can you dare to tell your manager that you will no longer write test cases? Can you tell your manager that he/she cannot generate any reports automatically anymore? Can you tell them that you test some areas here and there(explore), but nothing in particular? Can you just convince your manager about your work just by showing the bugs you have filed? What if you did not find any bugs? According to the conventional testers, the main problems with Exploratory testing is that it is not measurable, auditable and manageable. If you read Jonathan Bach’s article above, you will come to know that these ‘so called’ problems are not problems at all. I will work on answering the above questions by writing a follow up post to this in the near future. In the meantime, if you want to answer them, please feel free to add them in the comments section.

Happy Testing,
Parimala Shankaraiah,