I have not explicitly tested mobile devices though I have found issues on different types of cell phones that I have used in the past. This gave me an idea to test Nokia 1650 model cell phone that I own currently. I chose to test the 'Insert Options' feature in Create Message functionality. I quickly glanced through the Insert Options feature at a high level, broke it down into different types of Insert options available (smileys, words, numbers, symbols and templates), how different these options are when the dictionary is on and the language chosen is English or Hindi, how different is the behavior if the dictionary is turned off, what are the effects of changing the text format to different sentence cases like uppercase, lowercase and title case. The testing checklist was ready. Please find it HERE.
As the session unfolded, I tested dictionary on (English), dictionary on(Hindi) and dictionary off sub-features. I referred to the checklist from time to time to ensure that I was focusing on the charter and not deviating from it at any point. For eg. When I was testing the Insert Options with dictionary On(Hindi), I found that the Create Message > Options feature has 4 additional options like Text: A B C, Text: a b c, Numbers: 1 2 3 and Insert Symbol. I was surprised because these were already available in the Insert Options. Redundant feature? May be. This was a guaranteed deviation from my current charter and hence I noted it down as an opportunity and continued testing. Please find the Session Based Test Report is HERE.
My Learnings:
1. It is a good idea to break the charter of the session to the smallest possible task(chunk) so as to focus and test it adequately(Note that I am not using the word completely).
2. I can get rid of a separate testing checklist document by adding the checklist items in the Task Breakdown section of the Session Based Test report.
3. Any activity related to this testing session should be done within the duration that was originally planned.
4. Opportunities play a spoilsport by deviating the tester from the charter unless the tester gives in to it resulting in boondoggle.
5. Opportunities in current session (outside of the current charter) can become the charters for future sessions.
6. Test Notes section should have information about observations and inferences made while exploring, understanding and testing the product. Any bug should get to the Bugs sections and queries if any should get to the Issues sections.
7. Add Risk field to talk about the risk associated with each bug(the risk of not getting the bug fixed) and the Customer Impact field to talk about any possible impact to the customer. In general, It would be nice to add Risk, Customer Impact and Oracle/Heuristic fields to indicate why and how the bug was found (OK, I am tweaking the original template suggested by Jonathan Bach).
8. Issues in the Issues section should be followed up without fail with developers and product management teams for further clarity and follow up testing in a separate session.
Ever since I read Jonathan Bach’s article on Session Based Test Management, I have been thinking how it can be adopted in any company which takes the conventional route to testing. Can you dare to tell your manager that you will no longer write test cases? Can you tell your manager that he/she cannot generate any reports automatically anymore? Can you tell them that you test some areas here and there(explore), but nothing in particular? Can you just convince your manager about your work just by showing the bugs you have filed? What if you did not find any bugs? According to the conventional testers, the main problems with Exploratory testing is that it is not measurable, auditable and manageable. If you read Jonathan Bach’s article above, you will come to know that these ‘so called’ problems are not problems at all. I will work on answering the above questions by writing a follow up post to this in the near future. In the meantime, if you want to answer them, please feel free to add them in the comments section.
Happy Testing,
Parimala Shankaraiah,
http://curioustester.blogspot.com
This remembers me the exercises which I practiced and practicing, when I was said "you don't domain knowledge".
ReplyDelete@Ravisuriya
ReplyDeleteThis remembers me the exercises which I practiced and practicing, when I was said "you don't domain knowledge"
To quote Pradeep Soundararajan 'There is just one domain, the software domain'
Parimala Shankaraiah
http://curioustester.blogspot.com
My typo mistake corrected:
ReplyDelete... ,when I was said "you don't have domain knowledge".
Welcome to the community, Pari. See my blog.
ReplyDelete@James
ReplyDeleteWelcome to the community, Pari. See my blog.
Thank You very much for your support and encouragement James. It is a great honour for me.
Parimala Shankaraiah
http://curioustester.blogspot.com
@Sunjeet
ReplyDeleteI appreciate your questions and your time.
DOes that mean if any unplanned but related activity/test comes up in the mind during that ET testing ...should not we extend the session ?
Any unplanned activity or related activity outside of the charter is a Big No. You should be noting this activity in the Opportunity section of
the SBTM Report. This may potentially become the charter for your next test session.
Does not that effectively mean shackling the tester to the session's limit ?
It is not a hard core rule that if you specify the session limit as 60 minutes, it should not go beyond it. After all, we are humans and we may
err. But, the main ideas are 1)Not to deviate too much from the charter and 2)Do not lose focus beyond the charter in any particular session.
At this point, I am sensing that if you start doing more and more such sessions, you will find less and less of this problem.
How is this different form the draconian "traditional" testing then !?
When I say traditional/conventional, I simply mean executing test scripts in a mechanized way without giving much work to the brain. This
session is different in many ways: 1)Breaking you testing task to smaller manageable chunks 2)Focusing on each feature at a time and yet have
the Big Picture of the feature in the entire product and many more. I will be writing a follow up post to this post. Hopefully, your question
will be better answered there.
Opportunities play a spoilsport by deviating the tester from the charter unless the tester gives in to it resulting in boondoggle.
Can you please clarify this a bit more..sorry could not understand this completely ?
I mean that if you see that by slightly deviating from the charter, you can find a lot of bugs(which is cool), you may get tempted to do that.
But by doing that, you will deviate from the charter which means that you will not test adequately with respect to the charter. One way is to
note this down in the Opportunities section and consider it while framing a future charter. Another way is to jot down your test ideas in the
Notes section if you fear that you will lose your test ideas by then.
What do oppurtunities mean here? any new test ideas (directly related or closely related ) to the sesion's tesitng mission ?
Good Question. An opportunity is any test idea that may help you find a bug if tested in an area of the product which is outside of the
charter. Now, if you put in more time into an opportunity, you may lose time on your current session.
If yes, are you suggesting that we should only stick to the ideas thought out up-front and any oppurtunities of new test ideas during the
session be ignored !? doies not that beat down the very essence of ET ?
Yes. You should stick to testing the charter upfront, but not limiting your test ideas. If your test ideas are part of the current charter, you
would test them right away. If the test ideas belong outside of it, then you note them as opportunities so you do not actually lose them. This
way, the essence of ET is not lost. Please let me know if you think otherwise. Good Question though.
IN my expreiecne with ET ...i have seen on numerous numerous occasions that just pushing this small window of related "oppurtunity" I have found many critical bugs :)!
I have seen this in my experience too. In addition to that, I have also missed bugs in the current session because I wondered to test outside of the charter in my thirst for bugs. As testers, it is a great challenge to come out of the mindset of finding more bugs versus providing more and more quality related information. There is a difference between the two.
Regards,
Parimala Shankaraiah
http://curioustester.blogspot.com
Hi Pari,
ReplyDeleteI have been reading your blog from a few days now. Thank you so much for a lot of amazing articles. Until now I have been a silent reader of your blog posts. First time I wanted to contribute to your efforts by commenting on the ISSUES part of your Session based test report. Your astounding observation on things amazes me.
I noticed that you mentioned that *,+,p,w should be removed from the "Insert Number" option. However, I think + sign is used commonly before a phone number to specify country code. For example +1, +91, +46 etc. So when you want to type a phone number in the message you might want to use this convention. It can be convenient for user instead of switching to "Insert Symbol" and then switching to "Insert Number" feature.
I think the * sign used before some pre-computed numbers to dial for a specific purpose that cell phone service provider might offer. For Example, if you are a AT&T user then you can dial *66 to reach the customer care directly. However, I do not see its importance in creating a message. Assumption: The application might be using the same part of the code for saving a number (address book) in which case * can be helpful. Although I won't be sure until I test it myself.
Most often a 'p' is used before a number to cite a book page in messages. For Example p.112 can be used to cite a quotation or phrase from page 112.
I am not sure why 'w' option is available in "Insert Number" feature.
These are just my assumptions and I am not sure if this really is the application requirement.
Please let me know your inputs on this.
Thanks,
Haritha
@Haritha
ReplyDeleteFirst time I wanted to contribute to your efforts by commenting on the ISSUES part of your Session based test report. Your astounding observation on things amazes me
Thank you Haritha. And your response to this post amazes me. Taking time to read an old blog post, analyzing it in depth and posting a summary of your understanding does require patience and passion and you seem to have both of these.
I noticed that you mentioned that *,+,p,w should be removed from the "Insert Number" option. However, I think + sign is used commonly before a phone number to specify country code. For example +1, +91, +46 etc. So when you want to type a phone number in the message you might want to use this convention. It can be convenient for user instead of switching to "Insert Symbol" and then switching to "Insert Number" feature
Wonderful observation. I had not thought about it at that time. I am not sure about the 'p' and 'w' options at this point. Perhaps, its time to test my brand new Nokia phone for this ;-)
These are just my assumptions and I am not sure if this really is the application requirement
These are inference you have made with valid reasons as to why you think they are valid. And I must say, you are good at it.
Thanks for your time and questions,
Regards,
Parimala Shankaraiah