Serve your Stakeholder

Dr. Cem Kaner describes stakeholder as someone who has a vested interest in the success of the testing effort and/or in the success of the product. Stakeholders could be project managers, marketing managers, product managers, programmers, technical support representative, sales representatives, customers and many others in many different roles. Testing is done on behalf of the stakeholders. Here is a question for you. Have you ever asked who your stakeholder is before you start to test?

On a balmy Monday morning, I visited Weekend Testing site to read previous weekend's session reports. The battle of the URL shorteners lured me.

Mission
You are contracted by three stakeholders to evaluate URL shortening services - http://tinyurl.com, http://bit.ly, http://3.ly, http://is.gd, http://tr.im, http://stnx.at. The stakeholders are:
1. a heavy twitter user tweeting a lot of internet references over the day
2. a university student wanting to share stuff from his homework/courses over the internet with his fellow students
3. an online magazine publisher for referencing further readings on the internet

Experience Report
This time around, I did not jump right away to test and find bugs. This was partly because I did not know what information to provide and how. I explored all 6 URL shorteners and played around with features for a while. I was specifically looking at evaluating URL Shorteners on behalf of the stakeholders. Stakeholders were of 3 types – heavy twitter users, university student and an online magazine publisher. I went about with a small feature list of what I thought was important to the stakeholders. I chose URL Validation, Default Length, Auto-copy to clipboard, Proceed with URL shortening on same page, Custom Names & Validations and Preview. I identified just these as important features and ignored the others. This is where many of us commit mistakes [Remember, Assuming that we are the final stakeholders and claiming that all we say is right is dangerous].

Now that I had played with URL shorteners a while ago, I started exploring each feature and prepared an excel with details as to which shorteners support above features and which do not. In the end, I looked for the URL shorteners which supported most of the features above and concluded that 3.ly and Tr.im were the good ones.

You can read my summarized report HERE. You can also learn from few more reports at Weekend Testing EWT 04 session.

What went right?
1. Exploring URL shorteners one by one without being lost in finding bugs
2. Came up with ideas to test for each stakeholder requirement separately. I would say this was a good start
3. Came up with a feature list to compare the URL shorteners against each other and evaluate. This approach was good
4. Captured test results to the best of my knowledge and provided good enough information to stakeholders or so I thought.

What went wrong?
1. I did not use Google effectively. I could have googled about URL shorteners in general and fetched more information about other important features
2. I did not think deeply about the stakeholders categories. For e.g. what type of users are heavy twitter users – what are the important features that matter to them? What do online magazine publishers emphasize on while using URL shorteners? What features would be helpful to university students? As a result, my test coverage was very restricted and narrow. Rather, my test coverage was restricted and narrow, hence the result. Do you see the difference here?
3. I failed to summarize my findings with respect to each stakeholder. All I did was to prepare a generic report with my findings. I had started out with an intention of testing for each stakeholder differently and separately. Though the approach was good, I got lost somewhere and failed to identify many important features like Twitter integration, Shelf life of generated URLs and others.
4. Poor reporting yet again – one of my most common mistakes in uncommon situations. I did not present the report in a way that each stakeholder would be able to fish out information from. A generic report like this may not even be worth a glance. In short, it did not show anything useful for the stakeholders.
5. If only I had tried to ask each stakeholder what they wanted directly [assuming they would answer back for their own good], it would have been a lot more easier. In this case, asking questions meant I should have listed down what was more important to each stakeholder and tweaked my findings accordingly in the summarized report.
6. My conclusion that 3.ly and Tr.im were good was completely baseless. The conclusion varied for each stakeholder based on what was important to them which I hardly focused on.

Identify your stakeholders
We often ask our immediate bosses, colleagues and other team members about the stakeholders directly or indirectly. Some times, they don't know. Sometimes, they do. Irrespective of this, it is good to identify the targeted stakeholders for the products/applications being tested and come up with features that matter to them the most. After all, we need to find important bugs quickly contrary to finding all bugs which hardly matter to the stakeholders. Ask who the stakeholder is, what is the information they are looking for and what are the risks that they want to mitigate at any given point of time? (taken from Dr.Cem Kaner's article which escapes my memory right now).

Dr.Cem Kaner opines about understanding stakeholders in Weekend Testing session 20 as follows:

To test any product related to drug submission to FDA, SOME TESTER ON THE TEAM SHOULD mandatorily should have knowledge on the drug submission process. No one is responsible for my ignorance of quantitive trading systems but me. In the United States, over half of the stock trades are now driven by automated trading systems rather than by humans. If I want to understand how the market works, I have to understand the trading
system. If I want to test models of the market, I have to understand the range of tradings systems. If I want to test the models underlying a specific trading system (one of the things that distinguishes a high quality trading system from a low quality one is whether it makes money for the people who use it, yes? So assessing the quality of the system requires assessment of its probable value to users, which is often done in highly-detailed simulations)--If I want to understand these, I have to educate myself. If I am unwilling or unable to educate myself about these, I should test something else. If I am more generally unwilling or unable to educate myself about a product domain, its risks, and how to assess a product in that domain, I should give up on testing and do something easier that requires less learning.

Serve your stakeholder diligently,
Parimala Shankaraiah

P.S.
Ever since I created Testing Fiesta and Defects Fiesta sections on this blog, I have only dreamt about growing these 2 gadgets and hence accelerate my learning. My 7 day work weeks screwed it up and continue to do so for many more weeks. If at all, I do get time, I run back home to play with my 3 yo. If at all, my 3 yo sleeps, I go to sleep as well thinking it’s high time I take some rest. All lame excuses! I’ll start blogging regularly from now on. Keep visiting. Thank You.

Dear Readers - I owe it all to YOU

Curious Tester was born exactly 1 year ago on this day. I had created a blog page around February 2009 and did not know how to start and where to start. I was pondering over this for quite some time until I came across a blog post 5 Questions for Pradeep Soundararajan on Philk's blog. A few hours later, I got an email from Pradeep as follows:

Hi Parimala,
I found your blog with no posts. No matter how ever short it is, I would want you to consider posting your experiences as a software tester.

One of the important things to consider when you start blogging is - first do it for yourself and if others are reading it, fine.

Pradeep Soundararajan

At the time, Pradeep hardly knew me or so I thought. I was following his blog as a silent reader. I was literally in love with his blog posts. However, I did not know how he got to know about my blog – an empty blog, a blog with no posts on it.

Foray into Blogosphere
I can’t quite describe my experiences in last one year in a single post. I started blogging. I was quite a failure in writing, I must say. However, I wanted to try. I wanted to try because I didn’t want to regret about not trying. I started with a ‘1 post per month’ goal. I did not want to post theory from testing books. I did not want to copy someone. I did not know how to be authentic and original. As long as I liked a blog post, I posted it though it made me very nervous many a times. I even joked to myself 'Pari, nobody is going to read your blog anyway'. This went on for a few months and here I am in front of you.

Side Effects of blogging
Blogging is contagious. I attended Pradeep’s and Michael’s workshops, made it to STC 2009 conference, networked with many testers who were as passionate as me about testing, co-founded Weekend Testing with a group of friends, attended BBST foundations course, got an article published in print for the first time etc. This is what you can see from my blog. A lot happened behind the screens. I realized how poor I was in testing, how pathetic I was in technical knowledge and skills, how hopeless I was in many other skills related to testing (Lateral/Logical thinking, Investigation, Observation, Speaking, Note taking, and Presentation etc).

At times, it was painful to see that my passion for testing was not directly proportional to the quality of testing that I used to do. The good thing is I now know where I am failing (at least some of them). Another good thing is I can work on those areas. All along this journey, there was one person who made the greatest difference to me and my blog. You may know this person for sure. It's YOU!

Token of Thanks
I am really glad that YOU – My Dear Reader made a very
big difference in my journey of learning to be a better tester. Thank you very much
for reading my blog, supporting my work, commenting on my work, criticizing me
and helping me grow. It feels so great to know you care for me, my work and my
writing. Once Again, Thank you very much. I owe it all to YOU.

First Anniversary Gift
Trust me. Blogging is hard. Keeping oneself up to date is hard. Learning is hard. But then, what is easy in life? Take it from me - Nothing comes on a platter. You’ll have to work hard for whatever you want to be in life. You'll also have to work hard for whatever you don’t want to be in life. Think about it.

As a person who is facing many challenges in writing as a blogger and an aspiring writer hoping to write about testing, I am taking every opportunity to write better and express myself. I also want to help fellow colleagues like you to blog (if you are not blogging already). Here I am with a little gift for you – The Fieldstone Method.

If you are a tester cum aspiring blogger, feel free to reach out to me HERE. I am sure you have it in you to make it big in this world – in your own great way.


Happy International Women’s Day,
Thanking You,
Parimala Shankaraiah

30 Minute Challenge

A few weeks ago, my colleague Ajay Balamurugadas and I went books shopping together. Jerry Weinberg’s books were on the top of my list. I wanted to buy Wienberg on Writing while Ajay had 3 choices: The Black Swan and a couple other books related to Perl. What happens when two thirsty intellectuals go book hunting? We had every reason to buy books spanning Psychology, Management, Philosophy, Computer Science, Problem Solving, Writing and many more. Luckily, Weinberg on Writing was available. Ajay got The Black Swan.

While I was getting a glimpse of Perfect Software … , Ajay asked the storekeeper for Introduction to General Systems thinking. After some initial search, the storekeeper replied politely, “Sir, our inventory shows that one last copy is available. However, it appears to be misplaced. If you come back at a later time, we can get one from the warehouse for you”. Ajay nodded his head disappointedly. I was wondering if he would really buy so many books. I already carried 5 books for both of us though we planned to buy just 2. The sumptuous lunch party at my mom’s place was calling me home!

Conversation between us
Ajay: Are you game for a 30 minute challenge?
Pari: What?
Ajay: Let’s hunt down Introduction to General Systems thinking on our own. Let’s see if we’ll find it
Pari: OK. I’ll search the Management section. You start off from the Mathematics section (the other end of the floor). If we complete these sections, we can start looking in neighboring sections.
[This is the usual me jumping off without any preparation or strategy - Trap Gods must be happy]

Ajay: We don’t know how the book looks like. That might help us find the book faster. Let me google for the cover page of the book.
[Smart move by Ajay. This guy always springs new surprises. You'll know the reason behind it as you read on]

Pari: That is a very good idea.
Ajay: This is the cover page. It’s Red in color and if placed vertically in the book stand the side might look like this which is also in red color tilting his cell phone as though the image would tilt: D.
Pari: This is a good start for us.
Ajay: Ok. I’ll see you with the book!
Pari: Wow! Bye.

I headed to the Management section which was at the other end of that floor farther away from the Mathematics section. I chose to start my search from Management section for 2 reasons: 1) Weinberg on Writing and Perfect Software.. came from that section. I had seen the storekeeper picking them from there. So I knew there was a high possibility of finding Introduction to General Systems thinking there. 2) I thought that if I don’t find the book in the management section, I can move on to neighboring Computer Science and Investments sections while Ajay could move on to Philosophy and Psychology sections if he didn’t find the book in the Mathematics section. You may ask ‘Jerry’s book in Mathematics section?’ Well, if the book has been misplaced, then it could be in any of these sections. As a matter of factly, it could be on any of the other 3 floors of the bookstore as well. Whoa! My head rolled as I thought about the coverage for this mission.

My Strategy
Ajay’s idea of looking at the book cover was a good start. I started looking for a red book. There were so many reds. I looked at Weinberg on Writing and Perfect Software.. which I already had. I remembered that some of Jerry Weinberg’s books are mostly from Dorset House. Dorset House logo is a ‘DH’ with an inverted V over the letters DH. Now, I assumed that Introduction to General Systems thinking was from Dorset House (If I were in front of the computer, I would have googled to confirm this).

I used the following strategy: First level – look for a red book. Second level - look for Dorset House logo. Third level – check if it’s written by Jerry. Fourth level – Check if it’s Introduction to General Systems thinking. Fifth level – Success. Except the first level, every other level is an elimination level. If my findings don’t match my expectations at any level, I'll find the next red book and re-apply this strategy.

Many dread Red. Though I did not assume anything about red colored books, I was astonished to see a huge number of books in red. Each time I saw a red book, I moved on to the second level looking for the logo. If DH logo was absent, I backtracked to the first level looking for the next red book. Again, remember there are different flavors of red. I wondered if the picture we saw on the cell phone was red. It could have been maroon or magenta or any other color as well displayed dimly. So I looked not just for red books, but also maroon, magenta and others which appeared close to red color.

As I completed my search in one bookstand, I observed that most bookstands had enough space to put in an extra row of books I.e., one row of books are in the front directly visible to the onlookers. There was another row hidden behind this row. Maybe, the storekeepers stored multiple copies of books displayed in the front. Maybe, some of those books were not displayed in the front at all and hence escaped the onlookers. This could be one way in which Introduction to General Systems thinking escaped the storekeeper’s eyes.

I had two options to redesign my strategy: 1) Continue searching the front row alone 2) Search both front and back rows. In the interest of time, I chose option 1. The plan was to move to option 2 if option 1 did not help me find the book. I searched in the management section and did not find any book. Instead of switching to option 2, I moved on to the next section. I said to myself, “Let me use option 1 for other sections as well. If I don’t find the book, I’ll switch over to option 2”. I looked into a couple other sections. Ajay had moved to the computer science section already though he started from the other end of the floor. I was stunned “This guy is so fast. How could he do that?”. I stood and watched him for a while if I could get any clues to optimize my search [pair testing?]. He was not only looking at the front row, but also at the back row in each bookstand. And yet quicker than me. He had picked a few more books too.

Ajay’s Strategy
What I saw here was Ajay used option 2. He searched not only the first row, but also the back rows. He simply removed a few books in the front row and peeped into the back row if there were any red books. I was amazed to see how quick he was in spite of looking for many more books. He was not just looking for Introduction to General Systems thinking, but also was open to finding other books of interest. As he ended the search in his section where he agreed to search, he started searching in my section. ‘What? Doesn’t he believe me as I just finished searching?’, I asked myself. ‘If he finds the book in my section which I missed, then there is a lesson for me to learn’ I thought and moved to psychology section where I thought there was a possibility to find the book. I gave up at the end of 20 minutes to read the first chapter of Perfect Software … while Ajay stuck to it for the entire 30 minutes.

Let’s understand Ajay’s strategy in his own words:
“The feeling of happiness when I’d get the book and smile fully look at the storekeeper encouraged me. Once the mission was decided, there was no looking back. There was no use wasting time on things not under my control. Instead, I focused on the thought: ‘The book I’d touch now could be the book I am looking for’. I ran my fingers on each book to skim the words on the title. Quick Reading I learnt at School by Dr. Raj Bapna’s Memory Techniques accelerated my search. My prior experience of helping a library teacher in arranging books helps me fish out the book quickly. Having close to 700 books at home helped. Basically, I’m used to such book hunting exercises and hence the speed. I scanned the books for the words ‘Introduction’, ‘Thinking’, ‘General’ and any title which occupied considerable space in terms of words. Though I was aware that the book was RED in color, I had the belief that the entire list can be covered easily in 30 minutes even if you select each and every book”

Did we find the book?
No. What we learned is how common this challenge is to the daily challenges we face as testers. When we get a new build for testing, We:
1. Hit the Tests right away. No planning. No preparation. No strategy. Just jump at it. This is so bad an idea given that it hardly gives a chance to our thought process.
2. Assume many things without questioning anything or anyone – We don’t ask for more information. We think that it is the programmer’s responsibility to share. How would they know what you want to know?
3. Don't Observe enough. Red is Red. But, What is red to you may not be red to your friend. It’s about perception. My 3 yo thinks all her teeth are yellow. Being a good observant helps. Observe from your eyes. Observe from the other testers eyes. Observe by thinking about the big picture.
4. Fear Multi-Storied problem. One key issue in our search was that the bookstore had several floors and given 2 people and 30 minutes, it was hard to cover all the floors. Suppose, we had found the book in 2nd floor itself, what is the guarantee that there was no book in the other floors. There could be a possibility of finding a few more misplaced books which even the inventory [read as automation] missed. Maybe. Maybe not. You never know. Now equate this book for a bug and re-read point #4. There you go. Repeat after me..... "A bug can hide wherever it wants to"
5. Don't Brainstorm. If Ajay and I brainstormed about the strategy before setting out on our own, we could have devised better ways of looking for the book. Pairing helps and speeds up testing as well. 2 pairs of eyes observe many different things at the same time.
6. Don't Share Knowledge. From Ajay's description of his experience, it is quite clear that he had prior experience of handling similar situations and dealing with them. Talking it out with him would have definitely helped us strategize this challenge better.

What I suggest above are methods that we used. These methods are fallible. When these methods work, we call it ‘Luck’. Well, it really isn’t about luck. It’s about using the right strategy. It's about how effectively we use our prior experiences and knowledge to the challenge at hand.

At the end of 30 minutes, Ajay continued to look for more books while I signed off from the bookstore. It was a day of astonishing revelation to me. It also reinforces my belief in team work and diversity of the team members when it comes to solving problems collectively.

I feel apologetic for lengthy posts at times keeping my readers' time in mind. However, some posts appear so incomplete without being lengthy. This prevents me from chopping off any content in the review phase. This is one such post which I didn't want to miss sharing with you.

Addendum on 6th March 2010
I quoted the name of the book as Perfect Illusions instead of Perfect Software. The complete name of the book is Perfect Software And Other Illusions about Testing. This is now corrected. Thanks to Fiona Charles for correcting me.

Happy Problem Solving,
Parimala Shankaraiah