03 March, 2010

30 Minute Challenge

A few weeks ago, my colleague Ajay Balamurugadas and I went books shopping together. Jerry Weinberg’s books were on the top of my list. I wanted to buy Wienberg on Writing while Ajay had 3 choices: The Black Swan and a couple other books related to Perl. What happens when two thirsty intellectuals go book hunting? We had every reason to buy books spanning Psychology, Management, Philosophy, Computer Science, Problem Solving, Writing and many more. Luckily, Weinberg on Writing was available. Ajay got The Black Swan.

While I was getting a glimpse of Perfect Software … , Ajay asked the storekeeper for Introduction to General Systems thinking. After some initial search, the storekeeper replied politely, “Sir, our inventory shows that one last copy is available. However, it appears to be misplaced. If you come back at a later time, we can get one from the warehouse for you”. Ajay nodded his head disappointedly. I was wondering if he would really buy so many books. I already carried 5 books for both of us though we planned to buy just 2. The sumptuous lunch party at my mom’s place was calling me home!

Conversation between us
Ajay: Are you game for a 30 minute challenge?
Pari: What?
Ajay: Let’s hunt down Introduction to General Systems thinking on our own. Let’s see if we’ll find it
Pari: OK. I’ll search the Management section. You start off from the Mathematics section (the other end of the floor). If we complete these sections, we can start looking in neighboring sections.
[This is the usual me jumping off without any preparation or strategy - Trap Gods must be happy]

Ajay: We don’t know how the book looks like. That might help us find the book faster. Let me google for the cover page of the book.
[Smart move by Ajay. This guy always springs new surprises. You'll know the reason behind it as you read on]

Pari: That is a very good idea.
Ajay: This is the cover page. It’s Red in color and if placed vertically in the book stand the side might look like this which is also in red color tilting his cell phone as though the image would tilt: D.
Pari: This is a good start for us.
Ajay: Ok. I’ll see you with the book!
Pari: Wow! Bye.

I headed to the Management section which was at the other end of that floor farther away from the Mathematics section. I chose to start my search from Management section for 2 reasons: 1) Weinberg on Writing and Perfect Software.. came from that section. I had seen the storekeeper picking them from there. So I knew there was a high possibility of finding Introduction to General Systems thinking there. 2) I thought that if I don’t find the book in the management section, I can move on to neighboring Computer Science and Investments sections while Ajay could move on to Philosophy and Psychology sections if he didn’t find the book in the Mathematics section. You may ask ‘Jerry’s book in Mathematics section?’ Well, if the book has been misplaced, then it could be in any of these sections. As a matter of factly, it could be on any of the other 3 floors of the bookstore as well. Whoa! My head rolled as I thought about the coverage for this mission.

My Strategy
Ajay’s idea of looking at the book cover was a good start. I started looking for a red book. There were so many reds. I looked at Weinberg on Writing and Perfect Software.. which I already had. I remembered that some of Jerry Weinberg’s books are mostly from Dorset House. Dorset House logo is a ‘DH’ with an inverted V over the letters DH. Now, I assumed that Introduction to General Systems thinking was from Dorset House (If I were in front of the computer, I would have googled to confirm this).

I used the following strategy: First level – look for a red book. Second level - look for Dorset House logo. Third level – check if it’s written by Jerry. Fourth level – Check if it’s Introduction to General Systems thinking. Fifth level – Success. Except the first level, every other level is an elimination level. If my findings don’t match my expectations at any level, I'll find the next red book and re-apply this strategy.

Many dread Red. Though I did not assume anything about red colored books, I was astonished to see a huge number of books in red. Each time I saw a red book, I moved on to the second level looking for the logo. If DH logo was absent, I backtracked to the first level looking for the next red book. Again, remember there are different flavors of red. I wondered if the picture we saw on the cell phone was red. It could have been maroon or magenta or any other color as well displayed dimly. So I looked not just for red books, but also maroon, magenta and others which appeared close to red color.

As I completed my search in one bookstand, I observed that most bookstands had enough space to put in an extra row of books I.e., one row of books are in the front directly visible to the onlookers. There was another row hidden behind this row. Maybe, the storekeepers stored multiple copies of books displayed in the front. Maybe, some of those books were not displayed in the front at all and hence escaped the onlookers. This could be one way in which Introduction to General Systems thinking escaped the storekeeper’s eyes.

I had two options to redesign my strategy: 1) Continue searching the front row alone 2) Search both front and back rows. In the interest of time, I chose option 1. The plan was to move to option 2 if option 1 did not help me find the book. I searched in the management section and did not find any book. Instead of switching to option 2, I moved on to the next section. I said to myself, “Let me use option 1 for other sections as well. If I don’t find the book, I’ll switch over to option 2”. I looked into a couple other sections. Ajay had moved to the computer science section already though he started from the other end of the floor. I was stunned “This guy is so fast. How could he do that?”. I stood and watched him for a while if I could get any clues to optimize my search [pair testing?]. He was not only looking at the front row, but also at the back row in each bookstand. And yet quicker than me. He had picked a few more books too.

Ajay’s Strategy
What I saw here was Ajay used option 2. He searched not only the first row, but also the back rows. He simply removed a few books in the front row and peeped into the back row if there were any red books. I was amazed to see how quick he was in spite of looking for many more books. He was not just looking for Introduction to General Systems thinking, but also was open to finding other books of interest. As he ended the search in his section where he agreed to search, he started searching in my section. ‘What? Doesn’t he believe me as I just finished searching?’, I asked myself. ‘If he finds the book in my section which I missed, then there is a lesson for me to learn’ I thought and moved to psychology section where I thought there was a possibility to find the book. I gave up at the end of 20 minutes to read the first chapter of Perfect Software … while Ajay stuck to it for the entire 30 minutes.

Let’s understand Ajay’s strategy in his own words:
“The feeling of happiness when I’d get the book and smile fully look at the storekeeper encouraged me. Once the mission was decided, there was no looking back. There was no use wasting time on things not under my control. Instead, I focused on the thought: ‘The book I’d touch now could be the book I am looking for’. I ran my fingers on each book to skim the words on the title. Quick Reading I learnt at School by Dr. Raj Bapna’s Memory Techniques accelerated my search. My prior experience of helping a library teacher in arranging books helps me fish out the book quickly. Having close to 700 books at home helped. Basically, I’m used to such book hunting exercises and hence the speed. I scanned the books for the words ‘Introduction’, ‘Thinking’, ‘General’ and any title which occupied considerable space in terms of words. Though I was aware that the book was RED in color, I had the belief that the entire list can be covered easily in 30 minutes even if you select each and every book”


Did we find the book?
No. What we learned is how common this challenge is to the daily challenges we face as testers. When we get a new build for testing, We:
1. Hit the Tests right away. No planning. No preparation. No strategy. Just jump at it. This is so bad an idea given that it hardly gives a chance to our thought process.
2. Assume many things without questioning anything or anyone – We don’t ask for more information. We think that it is the programmer’s responsibility to share. How would they know what you want to know?
3. Don't Observe enough. Red is Red. But, What is red to you may not be red to your friend. It’s about perception. My 3 yo thinks all her teeth are yellow. Being a good observant helps. Observe from your eyes. Observe from the other testers eyes. Observe by thinking about the big picture.
4. Fear Multi-Storied problem. One key issue in our search was that the bookstore had several floors and given 2 people and 30 minutes, it was hard to cover all the floors. Suppose, we had found the book in 2nd floor itself, what is the guarantee that there was no book in the other floors. There could be a possibility of finding a few more misplaced books which even the inventory [read as automation] missed. Maybe. Maybe not. You never know. Now equate this book for a bug and re-read point #4. There you go. Repeat after me..... "A bug can hide wherever it wants to"
5. Don't Brainstorm. If Ajay and I brainstormed about the strategy before setting out on our own, we could have devised better ways of looking for the book. Pairing helps and speeds up testing as well. 2 pairs of eyes observe many different things at the same time.
6. Don't Share Knowledge. From Ajay's description of his experience, it is quite clear that he had prior experience of handling similar situations and dealing with them. Talking it out with him would have definitely helped us strategize this challenge better.

What I suggest above are methods that we used. These methods are fallible. When these methods work, we call it ‘Luck’. Well, it really isn’t about luck. It’s about using the right strategy. It's about how effectively we use our prior experiences and knowledge to the challenge at hand.

At the end of 30 minutes, Ajay continued to look for more books while I signed off from the bookstore. It was a day of astonishing revelation to me. It also reinforces my belief in team work and diversity of the team members when it comes to solving problems collectively.

I feel apologetic for lengthy posts at times keeping my readers' time in mind. However, some posts appear so incomplete without being lengthy. This prevents me from chopping off any content in the review phase. This is one such post which I didn't want to miss sharing with you.

Addendum on 6th March 2010
I quoted the name of the book as Perfect Illusions instead of Perfect Software. The complete name of the book is Perfect Software And Other Illusions about Testing. This is now corrected. Thanks to Fiona Charles for correcting me.

Happy Problem Solving,
Parimala Shankaraiah

31 comments:

  1. This is great display of "walking testing". An example that demonstrates that if you are passionate about something like testing, how get immersed in it and internalize it.

    Testing as problem solving is a something many don't get at all. Testing is typically seen as discovering problems, investigating them, reporting them in way that gets them the right audience... where do you fit solving problems using testing?

    Other day when I was talking to a colleage who mentioned "testers should provide/suggest solutions to problems like"

    1.Making it cheaper, quicker
    2.Making it highly repeatable.
    3.Make it process dependent
    4.Problem of testing estimation
    5.Helping development so that testing becomes unnecessary (after all if you build the product correct first time, who needs testing)...

    Some of these problems are tricky ones ...as there are traps in them.

    Good post ...

    Shrini

    ReplyDelete
  2. Awesome!! :) I really liked the way you put forward the idea of planning and strategy. It was fun reading the post (an edge of the seat thriller :P)

    Off Topic: You never know; may be there is one superb script writer hiding within you!! :)

    And thanks for sharing it in length.. :)

    ReplyDelete
  3. This is great display of "walking testing". An example that demonstrates that if you are passionate about something like testing, how get immersed in it and internalize it.

    Shrini. Thank you for commenting. If I remember correctly, this is your first comment on my blog post. Its really special for me.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  4. @Nandagopal
    Off Topic: You never know; may be there is one superb script writer hiding within you!! :)


    Wow. I am blushing. I wish you could see that :)

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  5. @ Hit the Tests right away. No planning. No preparation. No strategy. Just jump at it. This is so bad an idea given that it hardly gives a chance to our thought process.

    >> Very true. A better plan and a strategy increases productivity. But it won't assure you the success in your mission. Some times we miss few details(seen/unseen or hidden/unhidden) to be observed.

    By the way, did you take care of smoke test approach in this mission? No where i see you mentioning something like "We observed books of certain topic were misplaced under different catogory/topic section". I think that should have been the first step in your approach which would have reduced the time/resources spent on this misson. You might have observed this but you forgot to mention in this post i believe (Just by going through the details of your testing approach). Wonderful display of mission description, approach and observations. Overall a nice post :-)

    Cheers,
    Vijay...

    ReplyDelete
  6. @Vijay
    "We observed books of certain topic were misplaced under different catogory/topic section".


    Good Question. I think looking for other books which might have been misplaced is a deviation from the mission that we set for ourselves. Clearly, we focused on the book we wanted and not books that we didn't want.

    Does this clarify your question?

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  7. @Pari,

    Wow, may be you got it wrong. What i meant was, did you observe books of certain topic were misplaced under different section? If yes, it should have prompted you to search in that fashion. until and Unless you know or have a clue that books were placed randomly or misplaced few diff books under diff section, your mission and approach doesn't makes any sense in here. Mean,
    > What if the book store was managed in such a way that there is a no scope of misplacing the books under diff section in that store?

    Does this clarify my point?

    Cheers,
    Vijay...

    ReplyDelete
  8. Hi Parimala,

    You write really wonderfully and interestingly. Even a boring stuff can be interesting when you write.

    :-)
    Prashanti

    ReplyDelete
  9. @Vijay
    What i meant was, did you observe books of certain topic were misplaced under different section? If yes, it should have prompted you to search in that fashion.


    Yes. Weinberg's 2 books were misplaced in the Management section as I mentioned above. This was the assumption I started with. Again, I try to ensure that I make Assumptions. I also ensure to break my assumptions. Irrespective of whether the assumptions are true or false, the resulting information helps my mission :-)

    This way I try not to get trapped.

    Thank You,
    Parimala Shankaraiah

    ReplyDelete
  10. @Prashanti
    Even a boring stuff can be interesting when you write.


    That is a huge compliment. Thank you Prashanti :-). Yet another silent reader speaks up. Yeh!

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  11. Beautiful post! I liked the way you wrote it. Well organized and well explained :)

    “The more intensely we feel about an idea or a goal, the more assuredly the idea, buried deep in our subconscious, will direct us along the path to its fulfillment.” (Saying by Earl Nightingale )

    Move one with this "Never Give Up!" attitude. Wish you Good Luck :)

    -Bhargavi M

    ReplyDelete
  12. Hi Pari,
    This post is simply awesome.

    The way you related your challenge to hunt down the book, to some super learnings we can use in testing, is very good.

    This post helped me learn the impact and importance of planning. preparation. strategy, false assumption, good observation, brainstorming and sharing knowledge for a good testing effort through your lovely story. The story was very well told and am not gonna forget this story and the related testing lessons from it ever :-)

    Thanks for making the learning simple and effective.

    Thanks a lot for helping me learn these lessons through your lovely story.

    Regards,
    Krishnaveni

    http://passionatetester.wordpress.com

    ReplyDelete
  13. Well, I guess I have to congratulate you twice, Parimala!

    Firstly, for having such an exquisite taste in books, of course :-). And also for the nice story and the lessons learned. Pairing is fun, but it takes a while to perfect the practice - I found that out the hard way while pairing with Ajay during a weekend testing session.
    I also recognize the fact that you immediately jumped into action without really talking about a strategy or doing a short brainstorm. That's what time pressure does with us, I guess... We donj't want to waste time talking, we want to ACT. Very hard to resist that urge!

    Regards,
    Zeger
    http://testsidestory.wordpress.com

    ReplyDelete
  14. Great post, Pari!

    What I found most interesting -- apart from your "post-test" analysis, which is really valuable -- is the fact that you had a conscious search strategy in your Option 1. Knowing that, it was much more likely that you'd be able to vary your strategy for solving the problem if you had persisted. You could say, "Okay. This particular technique didn't get what I wanted. Now what would I happen if I set that aside and did X. --or broadened the current strategy by adding a different technique?"

    If we aren't conscious we're flailing around in the dark, and apt to repeat our mistakes.

    ReplyDelete
  15. Many years ago, a year or two after "An Intro to General Systems came out, I was visiting London and went to a very large bookstore to see if they had the book there, and how it was selling. The clerk said they had five copies, but we couldn't find any--even after a search much like yours.

    "Well," said the clerk, "It's one of our most popular books, and people seem to like to steal it. We can't keep it in stock."

    So, perhaps it wasn't there. Looking for a needle in a haystack is one kind of search. Looking for a needle in a haystack--if there is a needle--is a different kind of search altogether.

    Thanks for the informative story--and for reading my books. - Jerry Weinberg

    ReplyDelete
  16. Awesome post Parimala,

    Thanks for posting about the exercise as a blog post.

    Lots of learning for me :)
    1. Reading the post, the lessons.
    2. Reading the comments.

    And yes, waiting for the next (PAIRED) challenge :)

    Regards,
    Ajay Balamurugadas
    Tweets @ajay184f

    ReplyDelete
  17. Parimala,

    I was with you at every step of this "walkling test", feeling the same exhilaration when reporing a bug.

    This is a masterful account of our craft, with rich storytelling I do not often find on tech blogs.

    It should be required reading for testers everywhere, no matter the range of their experience. It is honest, compelling, and fun.

    Well done!

    ReplyDelete
  18. fantastic post, Parimala. Agreed opportunity for learning everywhere. thank you for capturing this into a blog story.

    ReplyDelete
  19. Nice one !! thoroughly enjoyed ....and actually I could visualize how both of you would be reacting to the whole problem .....

    And this was so good as compared to something I reviewed from you 'ages back' ;)

    Keep it up Pari !!

    ReplyDelete
  20. I like the way that you talk about brainstorming and sharing knowledge before we begin testing a build. Good things to keep in mind before starting to test the build...

    ReplyDelete
  21. @Bhargavi
    Beautiful post! I liked the way you wrote it. Well organized and well explained :)


    Thank you Bhargavi


    @Krishnaveni
    This post helped me learn the impact and importance of planning. preparation. strategy, false assumption, good observation, brainstorming and sharing knowledge for a good testing effort through your lovely story.


    Thank you Krishna. I am glad you took away so many lessons from this story. I hope you will practise these regularly towards becoming a better tester.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  22. @Zeger Van Hese
    Firstly, for having such an exquisite taste in books


    Thank you. I simply made up this list of books from Jerry, James, Michael Bolton, Dr.Cem Kaner and Pradeep's suggested reading list :-)

    Pairing is fun, but it takes a while to perfect the practice - I found that out the hard way while pairing with Ajay during a weekend testing session

    I read that report about your pairing attempt with Ajay. I must say it was a good first attempt. As you practise many pairing session with different testers, you will learn many more lessons to succeed.

    That's what time pressure does with us, I guess... We donj't want to waste time talking, we want to ACT. Very hard to resist that urge!

    Jon Bach talks about resisting this urge very often in the form of overcoming traps in testing. This is one of the major traps that I fall into very often. However, I am slowing down and resisting the urge to an extent.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  23. @Fiona Charles
    Great post, Pari!


    Thank you very much Fiona. I am glad to know you read my post :)

    Knowing that, it was much more likely that you'd be able to vary your strategy for solving the problem if you had persisted. You could say, "Okay. This particular technique didn't get what I wanted. Now what would I happen if I set that aside and did X -- or broadened the current strategy by adding a different technique?"

    I agree with you Fiona. I was joking the other day to a friend that this post is half complete. If I persisted, I am sure, I could have succeeded. At one point, the thought of doing this operation on all the floors (Coverage) scared me and I did give up in a way. When it comes to testing, this is a must know lesson for each one of us not to give up assuming that all is well.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  24. @Gerald Weinberg
    Many years ago, a year or two after "An Intro to General Systems came out, I was visiting London and went to a very large bookstore to see if they had the book there, and how it was selling. The clerk said they had five copies, but we couldn't find any--even after a search much like yours.

    "Well," said the clerk, "It's one of our most popular books, and people seem to like to steal it. We can't keep it in stock."


    Hi Jerry, Thank you very much for taking time to comment on my blog. I am in love with what you write and the way you write. I am literally going through every blog post of yours on both of your blogs. You are helping me be a better tester in many many ways.

    Thank you for sharing the story.

    So, perhaps it wasn't there. Looking for a needle in a haystack is one kind of search. Looking for a needle in a haystack--if there is a needle--is a different kind of search altogether.

    Actually, I did not quite think about this scenario wherein the book is not there, but the storekeeper thought so. He trusted his inventory system and I trusted him - maybe not a good sign for a tester.

    Thanks for the informative story--and for reading my books. - Jerry Weinberg

    Pleasure is all mine!

    Thank you once again,
    Parimala Shankaraiah

    ReplyDelete
  25. @Ajay Balamurugadas
    Thanks for posting about the exercise as a blog post


    Thanks to you too. This would have been incomplete without discussion the strategy you used. I was thrilled to know about your prior experience with books.


    And yes, waiting for the next (PAIRED) challenge :)
    Yes. let's set up one shortly.

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  26. @Jon Bach
    I was with you at every step of this "walkling test", feeling the same exhilaration when reporing a bug


    Thank you very much Jon. Your first comment on my blog. I am excited :)


    This is a masterful account of our craft, with rich storytelling I do not often find on tech blogs

    Thank you once again. I am glad you liked it. Like Fiona mentioned above, I did not pursue this challenge to my fullest potential. When it comes to testing products, I think this attitude of giving up early may not work at all. However, It is good that I learnt this lesson after posting this story.

    It should be required reading for testers everywhere, no matter the range of their experience. It is honest, compelling, and fun. Well done!

    Wow! I am so thrilled!

    Thank You,
    Parimala Shankaraiah

    ReplyDelete
  27. @Ram
    fantastic post, Parimala. Agreed opportunity for learning everywhere. thank you for capturing this into a blog story


    Thank you very much Ram. I am glad you liked it.

    @Meeta Prakash
    I could visualize how both of you would be reacting to the whole problem


    Wow! Thank you Meeta Didi.

    And this was so good as compared to something I reviewed from you 'ages back' ;)

    I am very glad I got it reviewed by you. I still remember those tips while I write anything ever since.


    @Fake Software Tester
    Good things to keep in mind before starting to test the build...


    I liked your name :). That is how a small part of our world considers testers. Maybe a little irony involved. Thank you!

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  28. Hi Pari!
    I realy could not wait till you get Introduction to General Systems thinking in your hands, and to read your thoughts about it.

    For me it was much easier to get it (I used the more traditional method of ordering from AMAZON and filling expense report for my manager...), but no new ideas (or bugs)came from the ordering process in this way :-).

    About the book - I especialy got inspired from following the example about black box observation. I think that sometime we should build the machine he describes in his exercise and use it for testers training [you might read and understand this comment when you actualy read the book...].

    ReplyDelete
  29. @Issi Hazan-Fuchs
    I realy could not wait till you get Introduction to General Systems thinking in your hands, and to read your thoughts about it.


    Sure. Will read and post my thoughts in near future. I am yet to buy the book. I think Jerry Weinberg's books are total Paisa Vasool. Every new idea in his book is priceless!

    Regards,
    Parimala Shankaraiah

    ReplyDelete
  30. it was interesting reading. keep your humurous out outlook towards life and be the wife of the 30 yo kid and the mother of the 3 yo man always.

    ReplyDelete
  31. @Vijayakumar

    Thank you Vijayakumar.

    Regards,
    Parimala Shankaraiah

    ReplyDelete