30 March, 2010

Serve your Stakeholder

Dr. Cem Kaner describes stakeholder as someone who has a vested interest in the success of the testing effort and/or in the success of the product. Stakeholders could be project managers, marketing managers, product managers, programmers, technical support representative, sales representatives, customers and many others in many different roles. Testing is done on behalf of the stakeholders. Here is a question for you. Have you ever asked who your stakeholder is before you start to test?

On a balmy Monday morning, I visited Weekend Testing site to read previous weekend's session reports. The battle of the URL shorteners lured me.

Mission
You are contracted by three stakeholders to evaluate URL shortening services - http://tinyurl.com, http://bit.ly, http://3.ly, http://is.gd, http://tr.im, http://stnx.at. The stakeholders are:
1. a heavy twitter user tweeting a lot of internet references over the day
2. a university student wanting to share stuff from his homework/courses over the internet with his fellow students
3. an online magazine publisher for referencing further readings on the internet

Experience Report
This time around, I did not jump right away to test and find bugs. This was partly because I did not know what information to provide and how. I explored all 6 URL shorteners and played around with features for a while. I was specifically looking at evaluating URL Shorteners on behalf of the stakeholders. Stakeholders were of 3 types – heavy twitter users, university student and an online magazine publisher. I went about with a small feature list of what I thought was important to the stakeholders. I chose URL Validation, Default Length, Auto-copy to clipboard, Proceed with URL shortening on same page, Custom Names & Validations and Preview. I identified just these as important features and ignored the others. This is where many of us commit mistakes [Remember, Assuming that we are the final stakeholders and claiming that all we say is right is dangerous].

Now that I had played with URL shorteners a while ago, I started exploring each feature and prepared an excel with details as to which shorteners support above features and which do not. In the end, I looked for the URL shorteners which supported most of the features above and concluded that 3.ly and Tr.im were the good ones.

You can read my summarized report HERE. You can also learn from few more reports at Weekend Testing EWT 04 session.

What went right?
1. Exploring URL shorteners one by one without being lost in finding bugs
2. Came up with ideas to test for each stakeholder requirement separately. I would say this was a good start
3. Came up with a feature list to compare the URL shorteners against each other and evaluate. This approach was good
4. Captured test results to the best of my knowledge and provided good enough information to stakeholders or so I thought.

What went wrong?
1. I did not use Google effectively. I could have googled about URL shorteners in general and fetched more information about other important features
2. I did not think deeply about the stakeholders categories. For e.g. what type of users are heavy twitter users – what are the important features that matter to them? What do online magazine publishers emphasize on while using URL shorteners? What features would be helpful to university students? As a result, my test coverage was very restricted and narrow. Rather, my test coverage was restricted and narrow, hence the result. Do you see the difference here?
3. I failed to summarize my findings with respect to each stakeholder. All I did was to prepare a generic report with my findings. I had started out with an intention of testing for each stakeholder differently and separately. Though the approach was good, I got lost somewhere and failed to identify many important features like Twitter integration, Shelf life of generated URLs and others.
4. Poor reporting yet again – one of my most common mistakes in uncommon situations. I did not present the report in a way that each stakeholder would be able to fish out information from. A generic report like this may not even be worth a glance. In short, it did not show anything useful for the stakeholders.
5. If only I had tried to ask each stakeholder what they wanted directly [assuming they would answer back for their own good], it would have been a lot more easier. In this case, asking questions meant I should have listed down what was more important to each stakeholder and tweaked my findings accordingly in the summarized report.
6. My conclusion that 3.ly and Tr.im were good was completely baseless. The conclusion varied for each stakeholder based on what was important to them which I hardly focused on.

Identify your stakeholders
We often ask our immediate bosses, colleagues and other team members about the stakeholders directly or indirectly. Some times, they don't know. Sometimes, they do. Irrespective of this, it is good to identify the targeted stakeholders for the products/applications being tested and come up with features that matter to them the most. After all, we need to find important bugs quickly contrary to finding all bugs which hardly matter to the stakeholders. Ask who the stakeholder is, what is the information they are looking for and what are the risks that they want to mitigate at any given point of time? (taken from Dr.Cem Kaner's article which escapes my memory right now).

Dr.Cem Kaner opines about understanding stakeholders in Weekend Testing session 20 as follows:
To test any product related to drug submission to FDA, SOME TESTER ON THE TEAM SHOULD mandatorily should have knowledge on the drug submission process. No one is responsible for my ignorance of quantitive trading systems but me. In the United States, over half of the stock trades are now driven by automated trading systems rather than by humans. If I want to understand how the market works, I have to understand the trading
system. If I want to test models of the market, I have to understand the range of tradings systems. If I want to test the models underlying a specific trading system (one of the things that distinguishes a high quality trading system from a low quality one is whether it makes money for the people who use it, yes? So assessing the quality of the system requires assessment of its probable value to users, which is often done in highly-detailed simulations)--If I want to understand these, I have to educate myself. If I am unwilling or unable to educate myself about these, I should test something else. If I am more generally unwilling or unable to educate myself about a product domain, its risks, and how to assess a product in that domain, I should give up on testing and do something easier that requires less learning.

Serve your stakeholder diligently,
Parimala Shankaraiah

P.S.
Ever since I created Testing Fiesta and Defects Fiesta sections on this blog, I have only dreamt about growing these 2 gadgets and hence accelerate my learning. My 7 day work weeks screwed it up and continue to do so for many more weeks. If at all, I do get time, I run back home to play with my 3 yo. If at all, my 3 yo sleeps, I go to sleep as well thinking it’s high time I take some rest. All lame excuses! I’ll start blogging regularly from now on. Keep visiting. Thank You.

10 comments:

  1. Your report wa very helpful

    ReplyDelete
  2. @ZXC8

    Your report was very helpful


    Wow. You seem to be another URL shortener company. Isn't it?

    I plan to do a Part II of this exercise if that helps you :)

    Thank you for your feedback because you are one of those rare souls who read my report and even acknowledged it :)

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  3. No doubt, 3 Yo takes the highest priority,BTW how about 30 yo ;)
    --Dhanasekar S

    ReplyDelete
  4. Hi Pari!
    Allthough users are one type,of stakeholders,and I liked to read this terminology in the mission statement, most the stakeholders which we as testers interface with are usualy not the users themselves, but people which part of their role is to represent the users towards the development and test team.
    Helping the stakehpolders within the team to understand the User profiles is one one the things that defines a good tester, and I think you gave a great demonstartion of that.

    P.S. I'll better got back to spend some time with my 3,5 & 7 YO :-)

    ReplyDelete
  5. @Dhanasekar S
    No doubt, 3 Yo takes the highest priority,BTW how about 30 yo ;)


    Both are doing great. Thanks for asking :)


    @Issi Hazan-Fuchs
    most the stakeholders which we as testers interface with are usually not the users themselves, but people which part of their role is to represent the users towards the development and test team


    True. I see this as one of the key problems as well. We should involve the 'real' stakeholders early and more often to get a clear picture. To some extent, What I do is to try and get different team members to brief exact customer flows and requirements based on prior implementations of the product. I don't really know how to get this done in any organization reasonably well. It would be good to listen to some stakeholder success stories.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  6. Hi Parimala,

    I have been following your blogs from couple of weeks and must admit they are brilliant. It is by nature and also as a tester by profession that we tend to think about a mission by putting ourselves in the epicenter of that mission and thinking you would had done this and that and finding the same in your report makes yourself proud. I have just registered on Weekend Testing and would love to be part of it soon. And yes I am planning to start a blog of my own and would really help in getting some tips from you experts.

    Cheerz,
    Deven B.
    UK.

    ReplyDelete
  7. @Deven

    Thank you for your kind words.

    I have just registered on Weekend Testing and would love to be part of it soon

    Europe chapter might work well for your timezone. Looking forward to your participation.

    And yes I am planning to start a blog of my own and would really help in getting some tips from you experts

    I am not an expert. It's a long way to go. It's infact a journey rather than a destination.

    Good to know you are about to blog. I would be glad to help. Do write in to me.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  8. Hi


    Stakeholders could be project managers, marketing managers, product managers, programmers, technical support representative, sales representatives, customers and many others in many different roles. Testing is done on behalf of the stakeholders


    In my opinion we cannot group all such groups of people into one large category as Stakeholders. As the groups that are listed above may not have the same kind of interest. Few that I could think are
    • Stakeholder – Owns the program/project/product and have financial/non-financial impacts of the fallouts
    • Facilitators – Help and support but do not own it
    • Consumers – May or may not own it but certainly consume the benefit
    Serving the users may not always be possible but what is more important is to know the domain/nature of the software supposed to serve (as quoted by Cem). This can sound synonym to stakeholder/user but in my opinion these are two different. One is the *concept* and the other is the *consumer*. Understanding one or the other is key to the success of the project. Knowledge of the *concept* and interaction with *consumer* is ideal

    In a service based/oriented software development process, we speak to stakeholders (in my experience 99.9 times out of 100) who represent the users; a program manager who runs the software development show, a business analyst who tries to decipher the user’s mind and so on.

    A project where public website to be built for the potential investors across the world for an investment bank: Here the *real* users range from the potential investor to casual browsers for information (of different age/country/background (knowledge level)) and I don’t think they have any interest in the software that’s being developed at all
    In this case, we as tester will never get to hear the *real* users until the software is out in the street. Here the stakeholders presume what a user would want to see (information) and navigate. Then we mix the assumption with some standards of a web development like usability factor, application performance factor (which are assertions).

    A typical product development process (say a trading system): As designers/developers/testers (stakeholders), we have to understand the way market works and trades are generated (for all possible instruments and algorithms). Then this is mixed with the technical feasibilities/available trading algorithms, and some standards (again assertions) about the performance, usability.

    So here the design, development and so the testing is done not after talking to users but people who are user-like, then the product goes to market.

    I do agree to an extend that as testers we should think like users, but it’s not just the users we should try to think-like, rather all the different groups playing different roles. I think that’s the most handy contribution as a tester we make to the entire project/product development process, ie “Role Play”

    ReplyDelete
  9. @Parthi

    Help me to understand 'stakeholders' and who are they.


    In my opinion we cannot group all such groups of people into one large category as Stakeholders. As the groups that are listed above may not have the same kind of interest. Few that I could think are
    • Stakeholder – Owns the program/project/product and have financial/non-financial impacts of the fallouts
    • Facilitators – Help and support but do not own it
    • Consumers – May or may not own it but certainly consume the benefit
    Serving the users may not always be possible but what is more important is to know the domain/nature of the software supposed to serve (as quoted by Cem). This can sound synonym to stakeholder/user but in my opinion these are two different. One is the *concept* and the other is the *consumer*. Understanding one or the other is key to the success of the project. Knowledge of the *concept* and interaction with *consumer* is ideal


    1. What does 'stake' mean?
    2. What does 'stakeholders' mean?
    3. When can we see or group them as or they appear as stakeholders?
    4. When we cannot group as or cannot see them as or they do not appear as stakeholders?
    5. What we understand for word 'stakeholder' in context software project?
    6. Who is a stakeholder in context of software project? How can we conclude they are stakeholders and they are not stakeholders? How far the conclusion made is accepted and not accepted and why?


    Scenario: A farmer cultivates paddy. Now who consumes this paddy as a rice? Who will work with farmer to cultivate the crop? Who assisted farmer to do agriculture and how? Where, when, why and how did they assist farmer? Are they using rice of this farmer? The other people who farmer does not know and who did not assist her or him directly or indirectly, are they using rice of this farmer? Does farmer will use her or his own rice? Now who is a stakeholder here?


    How can we relate above scenario to software development which includes testing too? Who is a stakeholder here?

    -
    Ravisuriya

    ReplyDelete
  10. @Parthi

    In my opinion we cannot group all such groups of people into one large category as Stakeholders. As the groups that are listed above may not have the same kind of interest. Few that I could think are
    • Stakeholder – Owns the program/project/product and have financial/non-financial impacts of the fallouts
    • Facilitators – Help and support but do not own it
    • Consumers – May or may not own it but certainly consume the benefit


    As per Dr.Cem’s definition, stakeholder is someone who has a vested interest in the success of the testing effort and/or in the success of the product. Facilitators and Consumers belong to the latter category of success of the product. Anybody who has some stake with the product directly or indirectly is a stakeholder in some form.

    Serving the users may not always be possible but what is more important is to know the domain/nature of the software supposed to serve (as quoted by Cem)

    We try hard to solve problems around serving the users better. Isn’t it? Just to paraphrase, you say that domain knowledge of the software is very important for the tester which is important.

    In a service based/oriented software development process, we speak to stakeholders (in my experience 99.9 times out of 100) who represent the users

    Speaking to stakeholders is a first good step. However, do you think you would have spoken to all the stakeholders of the software? You take a small sample among the stakeholders to do this exercise. This will be very useful, but definitely not good enough to find all defects in the software [Complete Testing is impossible].

    a program manager who runs the software development show, a business analyst who tries to decipher the user’s mind and so on

    Depending upon the program manager or business analyst’s caliber and skill set, we could try to decipher the user’s mind. The more the number of people who attempt this, the more shocking the revelations can be. It would not be good to assume that they are the best folks. They might be good enough.

    In this case, we as tester will never get to hear the *real* users until the software is out in the street

    I disagree. I would replace ‘will never get to hear’ with ‘may not get to hear’. It is however, possible to know provided the tester is interested to find out.

    I do agree to an extend that as testers we should think like users, but it’s not just the users we should try to think-like, rather all the different groups playing different roles. I think that’s the handiest contribution as a tester we make to the entire project/product development process, i.e. “Role Play”

    Well said. Testers need to play diversified roles to be able to provide quality related information better.


    Thanks for your time in detailing out your thoughts,
    Regards,
    Parimala Shankaraiah

    ReplyDelete