A few weeks ago, my colleague Ajay Balamurugadas and I went books shopping together. Jerry Weinberg’s books were on the top of my list. I wanted to buy Wienberg on Writing while Ajay had 3 choices: The Black Swan and a couple other books related to Perl. What happens when two thirsty intellectuals go book hunting? We had every reason to buy books spanning Psychology, Management, Philosophy, Computer Science, Problem Solving, Writing and many more. Luckily, Weinberg on Writing was available. Ajay got The Black Swan.
While I was getting a glimpse of Perfect Software … , Ajay asked the storekeeper for Introduction to General Systems thinking. After some initial search, the storekeeper replied politely, “Sir, our inventory shows that one last copy is available. However, it appears to be misplaced. If you come back at a later time, we can get one from the warehouse for you”. Ajay nodded his head disappointedly. I was wondering if he would really buy so many books. I already carried 5 books for both of us though we planned to buy just 2. The sumptuous lunch party at my mom’s place was calling me home!
Conversation between us
Ajay: Are you game for a 30 minute challenge?
Pari: What?
Ajay: Let’s hunt down Introduction to General Systems thinking on our own. Let’s see if we’ll find it
Pari: OK. I’ll search the Management section. You start off from the Mathematics section (the other end of the floor). If we complete these sections, we can start looking in neighboring sections.
[This is the usual me jumping off without any preparation or strategy - Trap Gods must be happy]
Ajay: We don’t know how the book looks like. That might help us find the book faster. Let me google for the cover page of the book.
[Smart move by Ajay. This guy always springs new surprises. You'll know the reason behind it as you read on]
Pari: That is a very good idea.
Ajay: This is the cover page. It’s Red in color and if placed vertically in the book stand the side might look like this which is also in red color tilting his cell phone as though the image would tilt: D.
Pari: This is a good start for us.
Ajay: Ok. I’ll see you with the book!
Pari: Wow! Bye.
I headed to the Management section which was at the other end of that floor farther away from the Mathematics section. I chose to start my search from Management section for 2 reasons: 1) Weinberg on Writing and Perfect Software.. came from that section. I had seen the storekeeper picking them from there. So I knew there was a high possibility of finding Introduction to General Systems thinking there. 2) I thought that if I don’t find the book in the management section, I can move on to neighboring Computer Science and Investments sections while Ajay could move on to Philosophy and Psychology sections if he didn’t find the book in the Mathematics section. You may ask ‘Jerry’s book in Mathematics section?’ Well, if the book has been misplaced, then it could be in any of these sections. As a matter of factly, it could be on any of the other 3 floors of the bookstore as well. Whoa! My head rolled as I thought about the coverage for this mission.
My Strategy
Ajay’s idea of looking at the book cover was a good start. I started looking for a red book. There were so many reds. I looked at Weinberg on Writing and Perfect Software.. which I already had. I remembered that some of Jerry Weinberg’s books are mostly from Dorset House. Dorset House logo is a ‘DH’ with an inverted V over the letters DH. Now, I assumed that Introduction to General Systems thinking was from Dorset House (If I were in front of the computer, I would have googled to confirm this).
I used the following strategy: First level – look for a red book. Second level - look for Dorset House logo. Third level – check if it’s written by Jerry. Fourth level – Check if it’s Introduction to General Systems thinking. Fifth level – Success. Except the first level, every other level is an elimination level. If my findings don’t match my expectations at any level, I'll find the next red book and re-apply this strategy.
Many dread Red. Though I did not assume anything about red colored books, I was astonished to see a huge number of books in red. Each time I saw a red book, I moved on to the second level looking for the logo. If DH logo was absent, I backtracked to the first level looking for the next red book. Again, remember there are different flavors of red. I wondered if the picture we saw on the cell phone was red. It could have been maroon or magenta or any other color as well displayed dimly. So I looked not just for red books, but also maroon, magenta and others which appeared close to red color.
As I completed my search in one bookstand, I observed that most bookstands had enough space to put in an extra row of books I.e., one row of books are in the front directly visible to the onlookers. There was another row hidden behind this row. Maybe, the storekeepers stored multiple copies of books displayed in the front. Maybe, some of those books were not displayed in the front at all and hence escaped the onlookers. This could be one way in which Introduction to General Systems thinking escaped the storekeeper’s eyes.
I had two options to redesign my strategy: 1) Continue searching the front row alone 2) Search both front and back rows. In the interest of time, I chose option 1. The plan was to move to option 2 if option 1 did not help me find the book. I searched in the management section and did not find any book. Instead of switching to option 2, I moved on to the next section. I said to myself, “Let me use option 1 for other sections as well. If I don’t find the book, I’ll switch over to option 2”. I looked into a couple other sections. Ajay had moved to the computer science section already though he started from the other end of the floor. I was stunned “This guy is so fast. How could he do that?”. I stood and watched him for a while if I could get any clues to optimize my search [pair testing?]. He was not only looking at the front row, but also at the back row in each bookstand. And yet quicker than me. He had picked a few more books too.
Ajay’s Strategy
What I saw here was Ajay used option 2. He searched not only the first row, but also the back rows. He simply removed a few books in the front row and peeped into the back row if there were any red books. I was amazed to see how quick he was in spite of looking for many more books. He was not just looking for Introduction to General Systems thinking, but also was open to finding other books of interest. As he ended the search in his section where he agreed to search, he started searching in my section. ‘What? Doesn’t he believe me as I just finished searching?’, I asked myself. ‘If he finds the book in my section which I missed, then there is a lesson for me to learn’ I thought and moved to psychology section where I thought there was a possibility to find the book. I gave up at the end of 20 minutes to read the first chapter of Perfect Software … while Ajay stuck to it for the entire 30 minutes.
Let’s understand Ajay’s strategy in his own words:
“The feeling of happiness when I’d get the book and smile fully look at the storekeeper encouraged me. Once the mission was decided, there was no looking back. There was no use wasting time on things not under my control. Instead, I focused on the thought: ‘The book I’d touch now could be the book I am looking for’. I ran my fingers on each book to skim the words on the title. Quick Reading I learnt at School by Dr. Raj Bapna’s Memory Techniques accelerated my search. My prior experience of helping a library teacher in arranging books helps me fish out the book quickly. Having close to 700 books at home helped. Basically, I’m used to such book hunting exercises and hence the speed. I scanned the books for the words ‘Introduction’, ‘Thinking’, ‘General’ and any title which occupied considerable space in terms of words. Though I was aware that the book was RED in color, I had the belief that the entire list can be covered easily in 30 minutes even if you select each and every book”
Did we find the book?
No. What we learned is how common this challenge is to the daily challenges we face as testers. When we get a new build for testing, We:
1. Hit the Tests right away. No planning. No preparation. No strategy. Just jump at it. This is so bad an idea given that it hardly gives a chance to our thought process.
2. Assume many things without questioning anything or anyone – We don’t ask for more information. We think that it is the programmer’s responsibility to share. How would they know what you want to know?
3. Don't Observe enough. Red is Red. But, What is red to you may not be red to your friend. It’s about perception. My 3 yo thinks all her teeth are yellow. Being a good observant helps. Observe from your eyes. Observe from the other testers eyes. Observe by thinking about the big picture.
4. Fear Multi-Storied problem. One key issue in our search was that the bookstore had several floors and given 2 people and 30 minutes, it was hard to cover all the floors. Suppose, we had found the book in 2nd floor itself, what is the guarantee that there was no book in the other floors. There could be a possibility of finding a few more misplaced books which even the inventory [read as automation] missed. Maybe. Maybe not. You never know. Now equate this book for a bug and re-read point #4. There you go. Repeat after me..... "A bug can hide wherever it wants to"
5. Don't Brainstorm. If Ajay and I brainstormed about the strategy before setting out on our own, we could have devised better ways of looking for the book. Pairing helps and speeds up testing as well. 2 pairs of eyes observe many different things at the same time.
6. Don't Share Knowledge. From Ajay's description of his experience, it is quite clear that he had prior experience of handling similar situations and dealing with them. Talking it out with him would have definitely helped us strategize this challenge better.
What I suggest above are methods that we used. These methods are fallible. When these methods work, we call it ‘Luck’. Well, it really isn’t about luck. It’s about using the right strategy. It's about how effectively we use our prior experiences and knowledge to the challenge at hand.
At the end of 30 minutes, Ajay continued to look for more books while I signed off from the bookstore. It was a day of astonishing revelation to me. It also reinforces my belief in team work and diversity of the team members when it comes to solving problems collectively.
I feel apologetic for lengthy posts at times keeping my readers' time in mind. However, some posts appear so incomplete without being lengthy. This prevents me from chopping off any content in the review phase. This is one such post which I didn't want to miss sharing with you.
Addendum on 6th March 2010
I quoted the name of the book as Perfect Illusions instead of Perfect Software. The complete name of the book is Perfect Software And Other Illusions about Testing. This is now corrected. Thanks to Fiona Charles for correcting me.
Happy Problem Solving,
Parimala Shankaraiah
03 March, 2010
20 February, 2010
Puzzle Personified
SCENE I
A little over two months ago, Pradeep Soundararajan published a blog post about the importance of Testers learning to code. As a Perl enthusiast himself, Pradeep developed a puzzle Finding Nemo. He not only released it to the testing world, but also challenged testers to find the logic of the puzzle using Reverse Engineering.
Pradeep’s Nemo survived wear and tear and adverse climatic conditions while I carried it on my To Do list (a sheet of paper) for close to 3 months now. Time was ripe. Today was the D-day to solve the puzzle. Read on.
Mary had a little Lamb
The specification in the puzzle provided a gist of what Nemo does, figure out actual positions and logic behind Nemo’s changing positions. Re-reading the specification (4 times) gave me a new idea each time. First time, I got an idea about Nemo’s positioning by default. Second time, I got a feeler about how Nemo moves within the array. Third time, I was cognizant of the fact that other fishes move to different positions every time the position of Nemo is guessed. Fourth time, I was happy to know that the order of all the fishes will be revealed if Nemo’s position is guessed correctly. ‘Mary had a little Lamb’ heuristic from the book Exploring Requirements was of great help.
My Approach
The minimum number of attempts to guess the position of Nemo is set to 10. At first glance, I thought 7 was good enough as there were 7 fishes. 10 might be a trap as it includes 3 attempts more. I accepted the default setting as it was simple and would help me track Nemo’s progress with respect to his friends:
1. First guess - 1 i.e., Nemo would be in the first position. Bingo! I found Nemo in 1st position. Now, I was more interested in the order of the fishes – Nemo, Gill, Bubbles, SharkTooth, Flow, Phamplet and Stinger. I noted down the order in MS Excel.
2. Second guess - 2 assuming that Nemo would move linearly to the next position. I was wrong. What next?
3. I tried a monkey trick. Instead of guessing Nemo’s positions in the next 8 attempts, I closed the program. I ran the program again and entered 1 again. I found Nemo again. You may ask ‘Not a big deal, you found this in Step 1 above’. You are wrong. It was big deal for me. Note that every time you run this program, Nemo’s position is intact. This is unlike the Minesweeper game where the mines are placed differently each time the user clicks on the smiley button or the game is re-started.
4. Firstly, I got the order of the fishes in Step 1 above. Now I know that Nemo’s position doesn’t change if I restart the program. I thought ‘This puzzle is not as difficult as I thought it was’! Excitement? Yes. Over-confidence? No.
5. As a follow up to Step 3, my second guess was 1. Maybe Nemo was too sleepy to move. You never know. I was wrong again.
6. Closed and restarted the program. First guess – 1, Second guess – 3. Boom! Nemo had jumped to the third position. Third guess – 1. I decided that going forward, I’ll guess the positions one by one without mixing up too many things in my mind. My note taking skills were of good help here. Closed and re-started the program.
7. First guess – 1, Second guess – 3, Third guess – 2. FAIL
8. First guess – 1, Second guess – 3, Third guess – 3. FAIL
9. First guess – 1, Second guess – 3, Third guess – 4. FAIL
10. First guess – 1, Second guess – 3, Third guess – 5. PASS
11. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – start from 1 to 7 until I find Nemo’s position
12. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess - 3 . PASS. What? Nemo jumped backwards? How could he do that? Cheater! Oh No. Nemo can move in any direction he wants. It’s after all a Perl array (a big sigh). Closed and re-started the program.
13. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – 3, Fifth guess - start from 1 to 7 until I find Nemo’s position.
14. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – 3, Fifth guess – 2. Closed and re-started the program.
15. ………
16. …………
17. ………
18. Refer to the excel for successful guesses of Nemo’s position HERE
Analysis
Was I done? Do you remember the mission? The mission was to guess the position of Nemo in the array and reverse engineer the logic behind Nemo’s movements. I was only 50% done. Some of us would have forgotten the second part of the mission by now. Always take note of the mission at the beginning of the session. This helps to focus on the mission and avoid traps.
I knew how Nemo moved from time to time as summarized in the Excel. I kept staring at Nemo’s position again and again. I did not see anything prominent. Let’s use some colors and beautify the report with some greens and reds and yellows. This feeling pepped me up. I highlighted Nemo’s position in green. I was looking for a pattern. I didn’t see one. What the heck? I have pretty much guessed the puzzle. Isn't it?
I grabbed a fresh cup of coffee to celebrate achieving the first part of the mission. After all, the first part wasn’t that hard. The aroma coming from the fresh coffee spruced me up. I kept staring at the excel, this time focusing more on the green bars where Nemo was positioned. I then looked at the order in which Nemo moved – 1, 3, 5, 3, 2, 7, 3, 4, 7, and 4. Does the logic have something to do with arithmetic progression?. I am not good at Math, but sufficiently good enough to remember harmonic progression from my 8th grade. I did some calculations to see if there was any relation between this and the logic. No luck on the logic. No luck with the coffee as well. It was as cold as my mood.
Let’s be practical. Nemo can’t be best friends with all 6 fishes. Some were his best friends, some good and some not so good. So he would obviously hang out with friends who he was most comfortable with (fishes have a human side too and humans have a fishy side! Haven’t you watched the movie yet?). Enough of staring at the Excel. It was time for the Blink Test. After all, I had to put to use, my little knowledge gained reading Malcolm Gladwell’s book Blink – The Power of Thinking without Thinking. I took a quick look (blink) at Nemo’s positions and realized that Nemo hangs out with Phamplet, Gill and Bubbles most of the times. Aha! Now that's a cool finding. On careful observation, I saw another surprising behavior. On the 1st, 3rd, 5th, 7th and 9th attempts, Nemo positioned himself just before Gill(who was followed by Bubbles). On 2nd, 4th, 6th, 8th and 10th attempts, Nemo positioned himself before Phamplet who was followed by Gill and Bubbles. So, Nemo found comfort in the company of Phamplet and Gill most of the time and hence followed them who in turn had their own comforting friends.
Michael Bolton in his RST workshop talks about the importance of sneaking in, breaking the rules, watching the states carefully, monitoring the change of states, using visual/auditory/tactile skills and making snap judgments. Cognition to these has helped me learn to test better day by day. Break the rules. Sneak it. It’s ok for Testing’s sake.
Did I get it right?
Is this the right logic? Was I good enough in solving the puzzle? My answer is “I don’t know”. All I know is I thoroughly enjoyed solving the puzzle and travelling through an amazing journey of sneaking in backdoor, applying harmonic progressions, recognizing Nemo’s comfort zone and last but not the least blinking. I would still not mind if I am wrong for 1 reason: I tried and that makes a big difference to me. Come – Join – Make a Difference – NOW.
Do tell me what you think about my answer while I rush to Pradeep’s blog to read what Shmuel and William have to say.
SCENE II
I just read what Shmuel Gorshon and William Fisher had to say about the puzzle. I am so hurt about my problem solving skills!
9 days later…………
After feeling terribly embarrassed and hopeless about my effort at solving a puzzle (Don’t forget I have been poor in solving puzzles in general – looks like a red flag for me), I decided to learn how Shmuel and William nailed the problem.
I was kind of close to Shmuel in that my initial approach was good. What I lost on the way was to figure out if my solution worked for attempts greater than 10. At this point, I am thinking that playing around the puzzle with a few different attempts would have thrown some light on my understanding.
One important thing is Shmuel used many tools to aid and speed up the testing process. He starts off with Note Taking. Shmuel smartly used some automation tests to really try out umpteen attempts. “Don’t be afraid of automation. It can help you talk with the software in different ways and interact with it differently. It helps a lot with data generation and logging, too. Never let the automation take control of the testing. It is there to help you, not to drive the effort.” hints Shmuel. This is exactly how I feel about automation – Afraid.
The excel spreadsheet that he built is a mind boggling oracle to compare the puzzle against. This is one of the coolest manifestations of oracle I have witnessed. And I loved his storytelling report. Awesome appears like a very small word to commend what he has done with the puzzle. Well Done Shmuel. Great Work!
I liked William’s approach as well. His constant search for the right tools made me appear so dumb for not even thinking about the tools. I was unable to view the video, so I can’t talk much about his strategy yet.
Oh! The log file. The first time I was upset was when I saw that I did not do as well as Shmuel did. The second time was when I learned there was a log file that I missed. If the log file were a human, I might have assaulted it yelling ‘Why didn’t you show up when I needed you the most?”. Jokes apart, I did not look for it until Pradeep enlightened me about it.
Many of us grow up with a “You cannot blame me for things I don’t know. That is not fair” attitude. If only we kept our egos aside and be more open to learn, then we could become the most intelligent humans our near and dear ones would ever witness.
Striking the Right Balance
One video brought me back to my senses. Click HERE.
It’s Better to Try and Fail Than to Have Never Tried At All
Regards,
Parimala Shankaraiah
A little over two months ago, Pradeep Soundararajan published a blog post about the importance of Testers learning to code. As a Perl enthusiast himself, Pradeep developed a puzzle Finding Nemo. He not only released it to the testing world, but also challenged testers to find the logic of the puzzle using Reverse Engineering.
Pradeep’s Nemo survived wear and tear and adverse climatic conditions while I carried it on my To Do list (a sheet of paper) for close to 3 months now. Time was ripe. Today was the D-day to solve the puzzle. Read on.
Mary had a little Lamb
The specification in the puzzle provided a gist of what Nemo does, figure out actual positions and logic behind Nemo’s changing positions. Re-reading the specification (4 times) gave me a new idea each time. First time, I got an idea about Nemo’s positioning by default. Second time, I got a feeler about how Nemo moves within the array. Third time, I was cognizant of the fact that other fishes move to different positions every time the position of Nemo is guessed. Fourth time, I was happy to know that the order of all the fishes will be revealed if Nemo’s position is guessed correctly. ‘Mary had a little Lamb’ heuristic from the book Exploring Requirements was of great help.
My Approach
The minimum number of attempts to guess the position of Nemo is set to 10. At first glance, I thought 7 was good enough as there were 7 fishes. 10 might be a trap as it includes 3 attempts more. I accepted the default setting as it was simple and would help me track Nemo’s progress with respect to his friends:
1. First guess - 1 i.e., Nemo would be in the first position. Bingo! I found Nemo in 1st position. Now, I was more interested in the order of the fishes – Nemo, Gill, Bubbles, SharkTooth, Flow, Phamplet and Stinger. I noted down the order in MS Excel.
2. Second guess - 2 assuming that Nemo would move linearly to the next position. I was wrong. What next?
3. I tried a monkey trick. Instead of guessing Nemo’s positions in the next 8 attempts, I closed the program. I ran the program again and entered 1 again. I found Nemo again. You may ask ‘Not a big deal, you found this in Step 1 above’. You are wrong. It was big deal for me. Note that every time you run this program, Nemo’s position is intact. This is unlike the Minesweeper game where the mines are placed differently each time the user clicks on the smiley button or the game is re-started.
4. Firstly, I got the order of the fishes in Step 1 above. Now I know that Nemo’s position doesn’t change if I restart the program. I thought ‘This puzzle is not as difficult as I thought it was’! Excitement? Yes. Over-confidence? No.
5. As a follow up to Step 3, my second guess was 1. Maybe Nemo was too sleepy to move. You never know. I was wrong again.
6. Closed and restarted the program. First guess – 1, Second guess – 3. Boom! Nemo had jumped to the third position. Third guess – 1. I decided that going forward, I’ll guess the positions one by one without mixing up too many things in my mind. My note taking skills were of good help here. Closed and re-started the program.
7. First guess – 1, Second guess – 3, Third guess – 2. FAIL
8. First guess – 1, Second guess – 3, Third guess – 3. FAIL
9. First guess – 1, Second guess – 3, Third guess – 4. FAIL
10. First guess – 1, Second guess – 3, Third guess – 5. PASS
11. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – start from 1 to 7 until I find Nemo’s position
12. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess - 3 . PASS. What? Nemo jumped backwards? How could he do that? Cheater! Oh No. Nemo can move in any direction he wants. It’s after all a Perl array (a big sigh). Closed and re-started the program.
13. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – 3, Fifth guess - start from 1 to 7 until I find Nemo’s position.
14. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – 3, Fifth guess – 2. Closed and re-started the program.
15. ………
16. …………
17. ………
18. Refer to the excel for successful guesses of Nemo’s position HERE
Analysis
Was I done? Do you remember the mission? The mission was to guess the position of Nemo in the array and reverse engineer the logic behind Nemo’s movements. I was only 50% done. Some of us would have forgotten the second part of the mission by now. Always take note of the mission at the beginning of the session. This helps to focus on the mission and avoid traps.
I knew how Nemo moved from time to time as summarized in the Excel. I kept staring at Nemo’s position again and again. I did not see anything prominent. Let’s use some colors and beautify the report with some greens and reds and yellows. This feeling pepped me up. I highlighted Nemo’s position in green. I was looking for a pattern. I didn’t see one. What the heck? I have pretty much guessed the puzzle. Isn't it?
I grabbed a fresh cup of coffee to celebrate achieving the first part of the mission. After all, the first part wasn’t that hard. The aroma coming from the fresh coffee spruced me up. I kept staring at the excel, this time focusing more on the green bars where Nemo was positioned. I then looked at the order in which Nemo moved – 1, 3, 5, 3, 2, 7, 3, 4, 7, and 4. Does the logic have something to do with arithmetic progression?. I am not good at Math, but sufficiently good enough to remember harmonic progression from my 8th grade. I did some calculations to see if there was any relation between this and the logic. No luck on the logic. No luck with the coffee as well. It was as cold as my mood.
Let’s be practical. Nemo can’t be best friends with all 6 fishes. Some were his best friends, some good and some not so good. So he would obviously hang out with friends who he was most comfortable with (fishes have a human side too and humans have a fishy side! Haven’t you watched the movie yet?). Enough of staring at the Excel. It was time for the Blink Test. After all, I had to put to use, my little knowledge gained reading Malcolm Gladwell’s book Blink – The Power of Thinking without Thinking. I took a quick look (blink) at Nemo’s positions and realized that Nemo hangs out with Phamplet, Gill and Bubbles most of the times. Aha! Now that's a cool finding. On careful observation, I saw another surprising behavior. On the 1st, 3rd, 5th, 7th and 9th attempts, Nemo positioned himself just before Gill(who was followed by Bubbles). On 2nd, 4th, 6th, 8th and 10th attempts, Nemo positioned himself before Phamplet who was followed by Gill and Bubbles. So, Nemo found comfort in the company of Phamplet and Gill most of the time and hence followed them who in turn had their own comforting friends.
Michael Bolton in his RST workshop talks about the importance of sneaking in, breaking the rules, watching the states carefully, monitoring the change of states, using visual/auditory/tactile skills and making snap judgments. Cognition to these has helped me learn to test better day by day. Break the rules. Sneak it. It’s ok for Testing’s sake.
Did I get it right?
Is this the right logic? Was I good enough in solving the puzzle? My answer is “I don’t know”. All I know is I thoroughly enjoyed solving the puzzle and travelling through an amazing journey of sneaking in backdoor, applying harmonic progressions, recognizing Nemo’s comfort zone and last but not the least blinking. I would still not mind if I am wrong for 1 reason: I tried and that makes a big difference to me. Come – Join – Make a Difference – NOW.
Do tell me what you think about my answer while I rush to Pradeep’s blog to read what Shmuel and William have to say.
SCENE II
I just read what Shmuel Gorshon and William Fisher had to say about the puzzle. I am so hurt about my problem solving skills!
9 days later…………
After feeling terribly embarrassed and hopeless about my effort at solving a puzzle (Don’t forget I have been poor in solving puzzles in general – looks like a red flag for me), I decided to learn how Shmuel and William nailed the problem.
I was kind of close to Shmuel in that my initial approach was good. What I lost on the way was to figure out if my solution worked for attempts greater than 10. At this point, I am thinking that playing around the puzzle with a few different attempts would have thrown some light on my understanding.
One important thing is Shmuel used many tools to aid and speed up the testing process. He starts off with Note Taking. Shmuel smartly used some automation tests to really try out umpteen attempts. “Don’t be afraid of automation. It can help you talk with the software in different ways and interact with it differently. It helps a lot with data generation and logging, too. Never let the automation take control of the testing. It is there to help you, not to drive the effort.” hints Shmuel. This is exactly how I feel about automation – Afraid.
The excel spreadsheet that he built is a mind boggling oracle to compare the puzzle against. This is one of the coolest manifestations of oracle I have witnessed. And I loved his storytelling report. Awesome appears like a very small word to commend what he has done with the puzzle. Well Done Shmuel. Great Work!
I liked William’s approach as well. His constant search for the right tools made me appear so dumb for not even thinking about the tools. I was unable to view the video, so I can’t talk much about his strategy yet.
Oh! The log file. The first time I was upset was when I saw that I did not do as well as Shmuel did. The second time was when I learned there was a log file that I missed. If the log file were a human, I might have assaulted it yelling ‘Why didn’t you show up when I needed you the most?”. Jokes apart, I did not look for it until Pradeep enlightened me about it.
Many of us grow up with a “You cannot blame me for things I don’t know. That is not fair” attitude. If only we kept our egos aside and be more open to learn, then we could become the most intelligent humans our near and dear ones would ever witness.
Striking the Right Balance
One video brought me back to my senses. Click HERE.
It’s Better to Try and Fail Than to Have Never Tried At All
Regards,
Parimala Shankaraiah
12 February, 2010
A bugging story on Testers’ Credibility
A little over a year ago, I earned a reputation of reporting maximum number of severity 3 and 4 defects in the team. This appeared bad at the time as compared to other testers who reported severity 1 and 2 defects. Unlike these testers, I reported all the defects because I did not want anyone questioning my credibility as to why I did not find/report a defect. Ok, agreed that I was high on ego.
I often questioned myself “Am I really as bad as my colleagues think I am?”. I filed as many critical issues as other testers did. Ah! Turns out I wasn’t the crash specialist in the team. And we go gaga over crash specialists and worship them like Gods at times. Don’t we? Naturally, I received feedback that I need to focus on finding more defects of high importance and less of low importance (which wasn’t as direct as I wrote here).
Here I was, a tester who strongly believed that higher the number of defects I reported, the better I was as a tester. And now, people tell me I am not a good tester. I was devastated. I cried. I was angry. A day later when emotions settled down, I started to think about the feedback practically. I relooked at the summary and category of defects that I had reported until then. A total of 575 defects! Categories varied from functionality to usability to performance to user experience to cosmetic errors.
What did I do? I…………
1. Reported trivial issues with higher severity at a time when there were limitations on time and resources.
2. Ate up significant time of programmers on defects irrelevant at the time by coaxing them to take a look at the reports immediately after I reported them.
3. Reported low severity defects by assigning high severity consciously. I remembered Dr. Cem Kaner say "The best tester isn’t the one who finds the most bugs or who embarrasses the most programmers. The best tester is the one who gets the most bugs fixed". I miss-interpreted what Cem said of a good tester in this context.
4. Happily explained every unimportant defect in the bug scrub meeting.
5. Did not know that this wasn’t a workable approach. By the way, no one told me until the next appraisal cycle.
What did I change? I now………
1. Report important defects quickly by informing programmers informally. While they provide buddy fixes or potential fixes, I report the defects and close them after verification.
2. Don’t pester programmers to fix unimportant defects when there are important ones to be cared about.
3. Report trivial issues by assigning exact severities instead of pushing them one level up :D.
4. Create a separate backlog milestone or queue for trivial defects to be looked at a later point of time. But report them anyways.
5. If programmers work in the same office building as mine, I walk up to them and discuss the defect and share my findings. This helps me learn new things about the feature or the defect and also sets the context for the programmer. This also helps the programmer understand my concerns with regard to the defects.
Magic Trick
MIP heuristic helped me get over my defect mania. Mention in passing (aka MIP) is to refer to someone or something briefly while talking about something else. Eg. During the interview, Kavita mentioned in passing that her dad had also been involved in publishing. I MIP bugs that don't matter as much as critical bugs do. I MIP trivial bugs to programmers who crib 'No user would do that' often. I MIP bugs that are of lesser importance at the time, but may be important a little later, maybe customer asks for it after sometime, but all that he is bothered about now is to get rid of the crashes or blockers.
I struggle with defects that are of low severity, yet important enough to be fixed. I MIP such defects. I walk over to the programmer, talk a little about that feature and say ‘Hey, you know what? I just saw this behavior while testing this feature. What do you think of this?’. The programmer agrees most of the time. When you give control to the programmers and ask them what they think of a defect, they will be open about it. If testers are going to push the programmers by nudging or embarrassing the product (which is their baby), they will hate you. 'How dare you tell me that my baby is born with deformations? I wish I could kill you right now right here'. I do believe that in addition to mipping, reporting defects in defect tracking system for internal tracking and closure is important. You never know when your manager will ask ‘how could you miss such a simple defect?’. Most managers don’t realize that complete testing is not possible. Mip your defects, yet report them all.
Happy Mipping,
Parimala Shankaraiah
I often questioned myself “Am I really as bad as my colleagues think I am?”. I filed as many critical issues as other testers did. Ah! Turns out I wasn’t the crash specialist in the team. And we go gaga over crash specialists and worship them like Gods at times. Don’t we? Naturally, I received feedback that I need to focus on finding more defects of high importance and less of low importance (which wasn’t as direct as I wrote here).
Here I was, a tester who strongly believed that higher the number of defects I reported, the better I was as a tester. And now, people tell me I am not a good tester. I was devastated. I cried. I was angry. A day later when emotions settled down, I started to think about the feedback practically. I relooked at the summary and category of defects that I had reported until then. A total of 575 defects! Categories varied from functionality to usability to performance to user experience to cosmetic errors.
What did I do? I…………
1. Reported trivial issues with higher severity at a time when there were limitations on time and resources.
2. Ate up significant time of programmers on defects irrelevant at the time by coaxing them to take a look at the reports immediately after I reported them.
3. Reported low severity defects by assigning high severity consciously. I remembered Dr. Cem Kaner say "The best tester isn’t the one who finds the most bugs or who embarrasses the most programmers. The best tester is the one who gets the most bugs fixed". I miss-interpreted what Cem said of a good tester in this context.
4. Happily explained every unimportant defect in the bug scrub meeting.
5. Did not know that this wasn’t a workable approach. By the way, no one told me until the next appraisal cycle.
What did I change? I now………
1. Report important defects quickly by informing programmers informally. While they provide buddy fixes or potential fixes, I report the defects and close them after verification.
2. Don’t pester programmers to fix unimportant defects when there are important ones to be cared about.
3. Report trivial issues by assigning exact severities instead of pushing them one level up :D.
4. Create a separate backlog milestone or queue for trivial defects to be looked at a later point of time. But report them anyways.
5. If programmers work in the same office building as mine, I walk up to them and discuss the defect and share my findings. This helps me learn new things about the feature or the defect and also sets the context for the programmer. This also helps the programmer understand my concerns with regard to the defects.
Magic Trick
MIP heuristic helped me get over my defect mania. Mention in passing (aka MIP) is to refer to someone or something briefly while talking about something else. Eg. During the interview, Kavita mentioned in passing that her dad had also been involved in publishing. I MIP bugs that don't matter as much as critical bugs do. I MIP trivial bugs to programmers who crib 'No user would do that' often. I MIP bugs that are of lesser importance at the time, but may be important a little later, maybe customer asks for it after sometime, but all that he is bothered about now is to get rid of the crashes or blockers.
I struggle with defects that are of low severity, yet important enough to be fixed. I MIP such defects. I walk over to the programmer, talk a little about that feature and say ‘Hey, you know what? I just saw this behavior while testing this feature. What do you think of this?’. The programmer agrees most of the time. When you give control to the programmers and ask them what they think of a defect, they will be open about it. If testers are going to push the programmers by nudging or embarrassing the product (which is their baby), they will hate you. 'How dare you tell me that my baby is born with deformations? I wish I could kill you right now right here'. I do believe that in addition to mipping, reporting defects in defect tracking system for internal tracking and closure is important. You never know when your manager will ask ‘how could you miss such a simple defect?’. Most managers don’t realize that complete testing is not possible. Mip your defects, yet report them all.
Happy Mipping,
Parimala Shankaraiah
04 February, 2010
What type of team are you? – Part II
Rahul Gupta asked me some interesting questions in my previous post. He also suggested that I do a Part II of the post. Here I am with part II. Part I is HERE.
Rob the Star Performer
Rob managed a group of system administrators in his previous job. He joined current organization as a QA Engineer. He tested an enterprise product and reported defects. A year later, he became a Project Lead. In subsequent year, he got promoted as a QA Manager and headed a different team (now Team B). In addition to managing team B, he also volunteered to own a couple other projects.
A QA engineer rising to the ranks of a QA Manager is not an easy thing (or so it is believed to be). Rob had achieved this in 3 years time. Many people in the organization had to believe that he was a star performer. Incidentally, Rob actually tested for less than year during his 7 years tenure.
Management Style
Rob rewarded employees who worked 60 hr work week; he reprimanded employees who worked for 8 hours a day telling them that they aren’t working hard enough. What amuses me about Rob till this moment is that he did not have the common sense to think that some employees are smart. Perhaps, he is pretending so. The smart employees stayed for more than 10 hrs a day. Some even were in office for 14 hrs a day. They did not work for the full 10 hrs or 14 hrs that they stayed at office. They were not productive enough. They were not good at time management. They were there to impress the manager, not to do good work.
On the contrary, some employees worked for exactly 8 hours a day excluding the lunch hour. They worked sincerely for 8 hours between 9 AM – 6 PM. Rob started assigning critical tasks like customer escalations to such employees. Some started staying late for fear of bad appraisals. Some wondered why these tasks are assigned to them when there are few employees who usually work late into the night. Some didn’t budge. They declined to work beyond 6 PM citing personal reasons. Rob did not like these people. He thought they were not dedicated and committed to their work and the organization.
Over a period of time, people who pretended to work for more hours by staying late in the office simply passing on some metrics to please Rob. People who believed this was not right or were incapable of doing this either moved to different teams (if Rob did not goof up their internal transfer) or eventually quit the organization.
Why wasn’t Rob stopped?
Why wasn’t Rob stopped? Why was the 360 degree feedback about Rob thrashed year after year? Why wasn’t the management interested in educating Rob about his management style? Why wasn’t HR doing anything about Rob though they had got a feeler about him from the employees who quit his team and/or the organization? Only the management can answer these questions.
You know what? The management liked Rob. The management was like Rob. It’s only obvious that they liked Rob. The organization claimed to follow a very aggressive business strategy and all it needed was many more managers like Rob who could treat employees like factory workers, make them work for several hours on weekdays and weekends, pay less and yet tell them they got the best raise in the entire team.
What impressed the management team most was the crazy metrics and the sexy graphs that Rob managed to send on a daily basis to the management. They were least bothered to know about how he was getting those metrics, what he did to get them and how he managed many more green lines compared to the red ones that other managers produced at the time. Worse, they did not even know what the green and red lines meant. All they were bothered about was more and more green lines which could be passed on to their bosses who in turn would pass on to their bosses. This passing became a routine until those metrics get projected as the ‘truth’ in the organization.
What about Scapegoats?
Some employees in Rob’s team moved to a different team thanks to a new agile team that was started. Their joy knew no bounds. Most of the sincere employees upon whom he inflicted problems quit over a period of time. Rob’s so called hard working employees threatened to quit the team after which some received a 50%-100% raise. They eventually quit after taking the raise!
When will this stop?
Some day, one smart person will walk into the organization with the courage to question and the power to bring a change. He’ll call Rob into his cabin and ask ‘Hello buddy, can you tell me what these green lines mean?’. Until then, this will not stop. You can be that smart person. Will you be one?
With a very heavy heart, let me say this once and get it off my chest to never mention it again - Rob is the fore runner for the post of the Vice President this year. God Bless!
PS: The characters in this post are true and NOT fictional except the names :D. Any resemblance to you or your friends is deeply regretted. I have no offense to Rob, his so called hard working employees or the scapegoats who gave up fighting similar situations. These are just my observations of one such environment where I FAILED to succeed, yet learned the most about my profession. I EXCELLED in many ways to say the least. My belief in team collaboration only grew stronger. In fact, I cherish working with Rob the most in my career because I now realize the value of being treated with respect and treating fellow colleagues with respect.
Let's be human first - Team Players will grow within,
Regards,
Parimala Shankaraiah
Rob the Star Performer
Rob managed a group of system administrators in his previous job. He joined current organization as a QA Engineer. He tested an enterprise product and reported defects. A year later, he became a Project Lead. In subsequent year, he got promoted as a QA Manager and headed a different team (now Team B). In addition to managing team B, he also volunteered to own a couple other projects.
A QA engineer rising to the ranks of a QA Manager is not an easy thing (or so it is believed to be). Rob had achieved this in 3 years time. Many people in the organization had to believe that he was a star performer. Incidentally, Rob actually tested for less than year during his 7 years tenure.
Management Style
Rob rewarded employees who worked 60 hr work week; he reprimanded employees who worked for 8 hours a day telling them that they aren’t working hard enough. What amuses me about Rob till this moment is that he did not have the common sense to think that some employees are smart. Perhaps, he is pretending so. The smart employees stayed for more than 10 hrs a day. Some even were in office for 14 hrs a day. They did not work for the full 10 hrs or 14 hrs that they stayed at office. They were not productive enough. They were not good at time management. They were there to impress the manager, not to do good work.
On the contrary, some employees worked for exactly 8 hours a day excluding the lunch hour. They worked sincerely for 8 hours between 9 AM – 6 PM. Rob started assigning critical tasks like customer escalations to such employees. Some started staying late for fear of bad appraisals. Some wondered why these tasks are assigned to them when there are few employees who usually work late into the night. Some didn’t budge. They declined to work beyond 6 PM citing personal reasons. Rob did not like these people. He thought they were not dedicated and committed to their work and the organization.
Over a period of time, people who pretended to work for more hours by staying late in the office simply passing on some metrics to please Rob. People who believed this was not right or were incapable of doing this either moved to different teams (if Rob did not goof up their internal transfer) or eventually quit the organization.
Why wasn’t Rob stopped?
Why wasn’t Rob stopped? Why was the 360 degree feedback about Rob thrashed year after year? Why wasn’t the management interested in educating Rob about his management style? Why wasn’t HR doing anything about Rob though they had got a feeler about him from the employees who quit his team and/or the organization? Only the management can answer these questions.
You know what? The management liked Rob. The management was like Rob. It’s only obvious that they liked Rob. The organization claimed to follow a very aggressive business strategy and all it needed was many more managers like Rob who could treat employees like factory workers, make them work for several hours on weekdays and weekends, pay less and yet tell them they got the best raise in the entire team.
What impressed the management team most was the crazy metrics and the sexy graphs that Rob managed to send on a daily basis to the management. They were least bothered to know about how he was getting those metrics, what he did to get them and how he managed many more green lines compared to the red ones that other managers produced at the time. Worse, they did not even know what the green and red lines meant. All they were bothered about was more and more green lines which could be passed on to their bosses who in turn would pass on to their bosses. This passing became a routine until those metrics get projected as the ‘truth’ in the organization.
What about Scapegoats?
Some employees in Rob’s team moved to a different team thanks to a new agile team that was started. Their joy knew no bounds. Most of the sincere employees upon whom he inflicted problems quit over a period of time. Rob’s so called hard working employees threatened to quit the team after which some received a 50%-100% raise. They eventually quit after taking the raise!
When will this stop?
Some day, one smart person will walk into the organization with the courage to question and the power to bring a change. He’ll call Rob into his cabin and ask ‘Hello buddy, can you tell me what these green lines mean?’. Until then, this will not stop. You can be that smart person. Will you be one?
With a very heavy heart, let me say this once and get it off my chest to never mention it again - Rob is the fore runner for the post of the Vice President this year. God Bless!
PS: The characters in this post are true and NOT fictional except the names :D. Any resemblance to you or your friends is deeply regretted. I have no offense to Rob, his so called hard working employees or the scapegoats who gave up fighting similar situations. These are just my observations of one such environment where I FAILED to succeed, yet learned the most about my profession. I EXCELLED in many ways to say the least. My belief in team collaboration only grew stronger. In fact, I cherish working with Rob the most in my career because I now realize the value of being treated with respect and treating fellow colleagues with respect.
Let's be human first - Team Players will grow within,
Regards,
Parimala Shankaraiah
23 January, 2010
What type of team are you? - Part I
In one my previous organizations’, I worked for 2 different teams during my ~3 years tenure. Let’s call them Team A and Team B. Team A was a vibrant team with highly energetic and motivated people with diversified experience and skill-sets. The team members included fresh college grads, people with couple of years of experience and a handful with good knowledge of customers’ usage of the product we tested.
Team A was led by a manager we’ll call Larry. What struck most about Larry was that he had faith in his team members (note that I don’t use the word ‘team’ here). He would talk to every team member at a regular basis about each one’s work, if there were any obstacles, if there was anything he could help with and so on. He organized team lunches/picnics/movies once in 2 months. He did all this while we worked a 70 hr work week. He came up with ideas to work differently to test the product better, but did not mandate to use his ideas alone. He was fine with people coming up with their own ideas as well. He encouraged people to have Disposable Time during office hours. He led by example.
Good Things don’t last forever. Isn’t it? I was moved to Team B for which I was originally hired. Team B was opposite of Team A. Team B consisted of testers who had been with the organization for 4+ years. People kept to themselves, some of them were short tempered and inaccessible. Some would even yell in public if you went with a question which they found to be silly. Some would never answer though they knew how to deal with it, but always pointed to a person who hardly knew anything about it.
Team B was headed by a manager named Rob who always carried a fake smile. I often wondered if his jaws didn’t hurt at all. I had a tough time getting information about the product in this team. Other than my reporting team lead, no one was willing to help. About 90% of the team members were unhappy with their raises just 1 month before I joined. They were obviously unhappy. What a bad timing it was for me?
To add to these woes, Rob walked around people’s cubicles without saying a ‘Hi’. His eyes were always on peoples’ monitors. Who is browsing what, who is not at their desk, how much time do they take during the allotted 1 hr lunch break, do they answer calls at their desk phone by their colleagues or not and many more.
It was appraisal time again. Cafetaria and the Table Tennis area are the best places to hangout post the performance evaluation phase. One gets to hear so many important things about the organization and its processes. I overheard an employee grumbling ‘My manager says on 11th Jan 2005, he came to my desk three times between 2pm and 3pm, but he didn’t find me at desk. He also says that I have been going for long lunches beyond the stipulated 30 minutes (btw lunch time lasts 1 hour in the employee handbook). He concluded that I am not being productive enough’.
The other employee replied reassuringly, ‘You know what? Your manager is still better. My manager says ‘I called you 21 times between 11am and 11.30am to your desk phone and you did not pick up the phone. Where on earth were you? He has my mobile number. I am surprised why he didn’t call to find out? I was at the hospital tending to my wife who suffered a miscarriage that day’. This is just 2 of the many conversations I have overheard or heard directly from the employees. 1 year later, the rate at which people joined team B was way too less compared to the rate at which people quit that team(and the organization) thanks to Rob.
If you are a team member, which team would you love to be in? A or B? If you are a manager, how do you manage your team? As Larry or as Rob?
Happy Working,
Parimala Shankaraiah
Team A was led by a manager we’ll call Larry. What struck most about Larry was that he had faith in his team members (note that I don’t use the word ‘team’ here). He would talk to every team member at a regular basis about each one’s work, if there were any obstacles, if there was anything he could help with and so on. He organized team lunches/picnics/movies once in 2 months. He did all this while we worked a 70 hr work week. He came up with ideas to work differently to test the product better, but did not mandate to use his ideas alone. He was fine with people coming up with their own ideas as well. He encouraged people to have Disposable Time during office hours. He led by example.
Good Things don’t last forever. Isn’t it? I was moved to Team B for which I was originally hired. Team B was opposite of Team A. Team B consisted of testers who had been with the organization for 4+ years. People kept to themselves, some of them were short tempered and inaccessible. Some would even yell in public if you went with a question which they found to be silly. Some would never answer though they knew how to deal with it, but always pointed to a person who hardly knew anything about it.
Team B was headed by a manager named Rob who always carried a fake smile. I often wondered if his jaws didn’t hurt at all. I had a tough time getting information about the product in this team. Other than my reporting team lead, no one was willing to help. About 90% of the team members were unhappy with their raises just 1 month before I joined. They were obviously unhappy. What a bad timing it was for me?
To add to these woes, Rob walked around people’s cubicles without saying a ‘Hi’. His eyes were always on peoples’ monitors. Who is browsing what, who is not at their desk, how much time do they take during the allotted 1 hr lunch break, do they answer calls at their desk phone by their colleagues or not and many more.
It was appraisal time again. Cafetaria and the Table Tennis area are the best places to hangout post the performance evaluation phase. One gets to hear so many important things about the organization and its processes. I overheard an employee grumbling ‘My manager says on 11th Jan 2005, he came to my desk three times between 2pm and 3pm, but he didn’t find me at desk. He also says that I have been going for long lunches beyond the stipulated 30 minutes (btw lunch time lasts 1 hour in the employee handbook). He concluded that I am not being productive enough’.
The other employee replied reassuringly, ‘You know what? Your manager is still better. My manager says ‘I called you 21 times between 11am and 11.30am to your desk phone and you did not pick up the phone. Where on earth were you? He has my mobile number. I am surprised why he didn’t call to find out? I was at the hospital tending to my wife who suffered a miscarriage that day’. This is just 2 of the many conversations I have overheard or heard directly from the employees. 1 year later, the rate at which people joined team B was way too less compared to the rate at which people quit that team(and the organization) thanks to Rob.
If you are a team member, which team would you love to be in? A or B? If you are a manager, how do you manage your team? As Larry or as Rob?
Happy Working,
Parimala Shankaraiah
13 January, 2010
Book Review: Lessons Learned in Software Testing
About 5 years ago, I bought William Perry’s ‘Effective Methods for Software Testing’ book. I struggled for 2 months to get myself to read that book as it appeared to be too much of theory for me at the time (my first testing book). At the end of 2 months, I gave up. I gave up reading not just this book, but my reading habit itself.
In July 2009, I came across The Secrets of a Buccaneer Scholar by James Bach. I had just started blogging a few months before that and was following James’ blog for a while. There was a free offer to read his book within a timeline which I did. In five days, I had finished reading the book. I enjoyed reading very much after a long time. It was fun. I realized what I had lost in last 5 years.
Lessons Learned in Software Testing is a book that every tester should read to get their basics right. A couple of my friends suggested that this is a book mostly for freshers and not targeted towards experienced testers. I digressed and read the book. I was right. This is a great book meant for every tester.
This book depicts finer nuances of many testing challenges we face on a daily basis. Another good part of this book is that every chapter addresses a topic independent of other chapters. So you could choose to read few chapters earlier than others. If you are a tester, you must read this book.
I bought this book 2 months ago and often stared at it (placed on the dining table) cribbing that I did not have enough time to read it. Around the same time, I came across Anuj Magazine’s post about Reading Skills. I started reading for 30 minutes everyday and it has worked. Who knows, it may work for you too. All you need is to TRY.
Happy Reading,
Parimala Shankaraiah
In July 2009, I came across The Secrets of a Buccaneer Scholar by James Bach. I had just started blogging a few months before that and was following James’ blog for a while. There was a free offer to read his book within a timeline which I did. In five days, I had finished reading the book. I enjoyed reading very much after a long time. It was fun. I realized what I had lost in last 5 years.
Lessons Learned in Software Testing is a book that every tester should read to get their basics right. A couple of my friends suggested that this is a book mostly for freshers and not targeted towards experienced testers. I digressed and read the book. I was right. This is a great book meant for every tester.
This book depicts finer nuances of many testing challenges we face on a daily basis. Another good part of this book is that every chapter addresses a topic independent of other chapters. So you could choose to read few chapters earlier than others. If you are a tester, you must read this book.
I bought this book 2 months ago and often stared at it (placed on the dining table) cribbing that I did not have enough time to read it. Around the same time, I came across Anuj Magazine’s post about Reading Skills. I started reading for 30 minutes everyday and it has worked. Who knows, it may work for you too. All you need is to TRY.
Happy Reading,
Parimala Shankaraiah
Subscribe to:
Posts (Atom)