Puzzle Personified

SCENE I
A little over two months ago, Pradeep Soundararajan published a blog post about the importance of Testers learning to code. As a Perl enthusiast himself, Pradeep developed a puzzle Finding Nemo. He not only released it to the testing world, but also challenged testers to find the logic of the puzzle using Reverse Engineering.

Pradeep’s Nemo survived wear and tear and adverse climatic conditions while I carried it on my To Do list (a sheet of paper) for close to 3 months now. Time was ripe. Today was the D-day to solve the puzzle. Read on.

Mary had a little Lamb
The specification in the puzzle provided a gist of what Nemo does, figure out actual positions and logic behind Nemo’s changing positions. Re-reading the specification (4 times) gave me a new idea each time. First time, I got an idea about Nemo’s positioning by default. Second time, I got a feeler about how Nemo moves within the array. Third time, I was cognizant of the fact that other fishes move to different positions every time the position of Nemo is guessed. Fourth time, I was happy to know that the order of all the fishes will be revealed if Nemo’s position is guessed correctly. ‘Mary had a little Lamb’ heuristic from the book Exploring Requirements was of great help.

My Approach
The minimum number of attempts to guess the position of Nemo is set to 10. At first glance, I thought 7 was good enough as there were 7 fishes. 10 might be a trap as it includes 3 attempts more. I accepted the default setting as it was simple and would help me track Nemo’s progress with respect to his friends:

1. First guess - 1 i.e., Nemo would be in the first position. Bingo! I found Nemo in 1st position. Now, I was more interested in the order of the fishes – Nemo, Gill, Bubbles, SharkTooth, Flow, Phamplet and Stinger. I noted down the order in MS Excel.
2. Second guess - 2 assuming that Nemo would move linearly to the next position. I was wrong. What next?
3. I tried a monkey trick. Instead of guessing Nemo’s positions in the next 8 attempts, I closed the program. I ran the program again and entered 1 again. I found Nemo again. You may ask ‘Not a big deal, you found this in Step 1 above’. You are wrong. It was big deal for me. Note that every time you run this program, Nemo’s position is intact. This is unlike the Minesweeper game where the mines are placed differently each time the user clicks on the smiley button or the game is re-started.
4. Firstly, I got the order of the fishes in Step 1 above. Now I know that Nemo’s position doesn’t change if I restart the program. I thought ‘This puzzle is not as difficult as I thought it was’! Excitement? Yes. Over-confidence? No.
5. As a follow up to Step 3, my second guess was 1. Maybe Nemo was too sleepy to move. You never know. I was wrong again.
6. Closed and restarted the program. First guess – 1, Second guess – 3. Boom! Nemo had jumped to the third position. Third guess – 1. I decided that going forward, I’ll guess the positions one by one without mixing up too many things in my mind. My note taking skills were of good help here. Closed and re-started the program.
7. First guess – 1, Second guess – 3, Third guess – 2. FAIL
8. First guess – 1, Second guess – 3, Third guess – 3. FAIL
9. First guess – 1, Second guess – 3, Third guess – 4. FAIL
10. First guess – 1, Second guess – 3, Third guess – 5. PASS
11. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – start from 1 to 7 until I find Nemo’s position
12. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess - 3 . PASS. What? Nemo jumped backwards? How could he do that? Cheater! Oh No. Nemo can move in any direction he wants. It’s after all a Perl array (a big sigh). Closed and re-started the program.
13. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – 3, Fifth guess - start from 1 to 7 until I find Nemo’s position.
14. First guess – 1, Second guess – 3, Third guess – 5, Fourth guess – 3, Fifth guess – 2. Closed and re-started the program.
15. ………
16. …………
17. ………
18. Refer to the excel for successful guesses of Nemo’s position HERE

Analysis
Was I done? Do you remember the mission? The mission was to guess the position of Nemo in the array and reverse engineer the logic behind Nemo’s movements. I was only 50% done. Some of us would have forgotten the second part of the mission by now. Always take note of the mission at the beginning of the session. This helps to focus on the mission and avoid traps.

I knew how Nemo moved from time to time as summarized in the Excel. I kept staring at Nemo’s position again and again. I did not see anything prominent. Let’s use some colors and beautify the report with some greens and reds and yellows. This feeling pepped me up. I highlighted Nemo’s position in green. I was looking for a pattern. I didn’t see one. What the heck? I have pretty much guessed the puzzle. Isn't it?

I grabbed a fresh cup of coffee to celebrate achieving the first part of the mission. After all, the first part wasn’t that hard. The aroma coming from the fresh coffee spruced me up. I kept staring at the excel, this time focusing more on the green bars where Nemo was positioned. I then looked at the order in which Nemo moved – 1, 3, 5, 3, 2, 7, 3, 4, 7, and 4. Does the logic have something to do with arithmetic progression?. I am not good at Math, but sufficiently good enough to remember harmonic progression from my 8th grade. I did some calculations to see if there was any relation between this and the logic. No luck on the logic. No luck with the coffee as well. It was as cold as my mood.

Let’s be practical. Nemo can’t be best friends with all 6 fishes. Some were his best friends, some good and some not so good. So he would obviously hang out with friends who he was most comfortable with (fishes have a human side too and humans have a fishy side! Haven’t you watched the movie yet?). Enough of staring at the Excel. It was time for the Blink Test. After all, I had to put to use, my little knowledge gained reading Malcolm Gladwell’s book Blink – The Power of Thinking without Thinking. I took a quick look (blink) at Nemo’s positions and realized that Nemo hangs out with Phamplet, Gill and Bubbles most of the times. Aha! Now that's a cool finding. On careful observation, I saw another surprising behavior. On the 1st, 3rd, 5th, 7th and 9th attempts, Nemo positioned himself just before Gill(who was followed by Bubbles). On 2nd, 4th, 6th, 8th and 10th attempts, Nemo positioned himself before Phamplet who was followed by Gill and Bubbles. So, Nemo found comfort in the company of Phamplet and Gill most of the time and hence followed them who in turn had their own comforting friends.

Michael Bolton in his RST workshop talks about the importance of sneaking in, breaking the rules, watching the states carefully, monitoring the change of states, using visual/auditory/tactile skills and making snap judgments. Cognition to these has helped me learn to test better day by day. Break the rules. Sneak it. It’s ok for Testing’s sake.

Did I get it right?
Is this the right logic? Was I good enough in solving the puzzle? My answer is “I don’t know”. All I know is I thoroughly enjoyed solving the puzzle and travelling through an amazing journey of sneaking in backdoor, applying harmonic progressions, recognizing Nemo’s comfort zone and last but not the least blinking. I would still not mind if I am wrong for 1 reason: I tried and that makes a big difference to me. Come – Join – Make a Difference – NOW.

Do tell me what you think about my answer while I rush to Pradeep’s blog to read what Shmuel and William have to say.

SCENE II
I just read what Shmuel Gorshon and William Fisher had to say about the puzzle. I am so hurt about my problem solving skills!

9 days later…………

After feeling terribly embarrassed and hopeless about my effort at solving a puzzle (Don’t forget I have been poor in solving puzzles in general – looks like a red flag for me), I decided to learn how Shmuel and William nailed the problem.

I was kind of close to Shmuel in that my initial approach was good. What I lost on the way was to figure out if my solution worked for attempts greater than 10. At this point, I am thinking that playing around the puzzle with a few different attempts would have thrown some light on my understanding.

One important thing is Shmuel used many tools to aid and speed up the testing process. He starts off with Note Taking. Shmuel smartly used some automation tests to really try out umpteen attempts. “Don’t be afraid of automation. It can help you talk with the software in different ways and interact with it differently. It helps a lot with data generation and logging, too. Never let the automation take control of the testing. It is there to help you, not to drive the effort.” hints Shmuel. This is exactly how I feel about automation – Afraid.

The excel spreadsheet that he built is a mind boggling oracle to compare the puzzle against. This is one of the coolest manifestations of oracle I have witnessed. And I loved his storytelling report. Awesome appears like a very small word to commend what he has done with the puzzle. Well Done Shmuel. Great Work!

I liked William’s approach as well. His constant search for the right tools made me appear so dumb for not even thinking about the tools. I was unable to view the video, so I can’t talk much about his strategy yet.

Oh! The log file. The first time I was upset was when I saw that I did not do as well as Shmuel did. The second time was when I learned there was a log file that I missed. If the log file were a human, I might have assaulted it yelling ‘Why didn’t you show up when I needed you the most?”. Jokes apart, I did not look for it until Pradeep enlightened me about it.

Many of us grow up with a “You cannot blame me for things I don’t know. That is not fair” attitude. If only we kept our egos aside and be more open to learn, then we could become the most intelligent humans our near and dear ones would ever witness.

Striking the Right Balance
One video brought me back to my senses. Click HERE.

It’s Better to Try and Fail Than to Have Never Tried At All

Regards,
Parimala Shankaraiah

A bugging story on Testers’ Credibility

A little over a year ago, I earned a reputation of reporting maximum number of severity 3 and 4 defects in the team. This appeared bad at the time as compared to other testers who reported severity 1 and 2 defects. Unlike these testers, I reported all the defects because I did not want anyone questioning my credibility as to why I did not find/report a defect. Ok, agreed that I was high on ego.

I often questioned myself “Am I really as bad as my colleagues think I am?”. I filed as many critical issues as other testers did. Ah! Turns out I wasn’t the crash specialist in the team. And we go gaga over crash specialists and worship them like Gods at times. Don’t we? Naturally, I received feedback that I need to focus on finding more defects of high importance and less of low importance (which wasn’t as direct as I wrote here).

Here I was, a tester who strongly believed that higher the number of defects I reported, the better I was as a tester. And now, people tell me I am not a good tester. I was devastated. I cried. I was angry. A day later when emotions settled down, I started to think about the feedback practically. I relooked at the summary and category of defects that I had reported until then. A total of 575 defects! Categories varied from functionality to usability to performance to user experience to cosmetic errors.

What did I do? I…………
1. Reported trivial issues with higher severity at a time when there were limitations on time and resources.
2. Ate up significant time of programmers on defects irrelevant at the time by coaxing them to take a look at the reports immediately after I reported them.
3. Reported low severity defects by assigning high severity consciously. I remembered Dr. Cem Kaner say "The best tester isn’t the one who finds the most bugs or who embarrasses the most programmers. The best tester is the one who gets the most bugs fixed". I miss-interpreted what Cem said of a good tester in this context.
4. Happily explained every unimportant defect in the bug scrub meeting.
5. Did not know that this wasn’t a workable approach. By the way, no one told me until the next appraisal cycle.

What did I change? I now………
1. Report important defects quickly by informing programmers informally. While they provide buddy fixes or potential fixes, I report the defects and close them after verification.
2. Don’t pester programmers to fix unimportant defects when there are important ones to be cared about.
3. Report trivial issues by assigning exact severities instead of pushing them one level up :D.
4. Create a separate backlog milestone or queue for trivial defects to be looked at a later point of time. But report them anyways.
5. If programmers work in the same office building as mine, I walk up to them and discuss the defect and share my findings. This helps me learn new things about the feature or the defect and also sets the context for the programmer. This also helps the programmer understand my concerns with regard to the defects.

Magic Trick
MIP heuristic helped me get over my defect mania. Mention in passing (aka MIP) is to refer to someone or something briefly while talking about something else. Eg. During the interview, Kavita mentioned in passing that her dad had also been involved in publishing. I MIP bugs that don't matter as much as critical bugs do. I MIP trivial bugs to programmers who crib 'No user would do that' often. I MIP bugs that are of lesser importance at the time, but may be important a little later, maybe customer asks for it after sometime, but all that he is bothered about now is to get rid of the crashes or blockers.

I struggle with defects that are of low severity, yet important enough to be fixed. I MIP such defects. I walk over to the programmer, talk a little about that feature and say ‘Hey, you know what? I just saw this behavior while testing this feature. What do you think of this?’. The programmer agrees most of the time. When you give control to the programmers and ask them what they think of a defect, they will be open about it. If testers are going to push the programmers by nudging or embarrassing the product (which is their baby), they will hate you. 'How dare you tell me that my baby is born with deformations? I wish I could kill you right now right here'. I do believe that in addition to mipping, reporting defects in defect tracking system for internal tracking and closure is important. You never know when your manager will ask ‘how could you miss such a simple defect?’. Most managers don’t realize that complete testing is not possible. Mip your defects, yet report them all.

Happy Mipping,
Parimala Shankaraiah

What type of team are you? – Part II

Rahul Gupta asked me some interesting questions in my previous post. He also suggested that I do a Part II of the post. Here I am with part II. Part I is HERE.

Rob the Star Performer
Rob managed a group of system administrators in his previous job. He joined current organization as a QA Engineer. He tested an enterprise product and reported defects. A year later, he became a Project Lead. In subsequent year, he got promoted as a QA Manager and headed a different team (now Team B). In addition to managing team B, he also volunteered to own a couple other projects.

A QA engineer rising to the ranks of a QA Manager is not an easy thing (or so it is believed to be). Rob had achieved this in 3 years time. Many people in the organization had to believe that he was a star performer. Incidentally, Rob actually tested for less than year during his 7 years tenure.

Management Style
Rob rewarded employees who worked 60 hr work week; he reprimanded employees who worked for 8 hours a day telling them that they aren’t working hard enough. What amuses me about Rob till this moment is that he did not have the common sense to think that some employees are smart. Perhaps, he is pretending so. The smart employees stayed for more than 10 hrs a day. Some even were in office for 14 hrs a day. They did not work for the full 10 hrs or 14 hrs that they stayed at office. They were not productive enough. They were not good at time management. They were there to impress the manager, not to do good work.

On the contrary, some employees worked for exactly 8 hours a day excluding the lunch hour. They worked sincerely for 8 hours between 9 AM – 6 PM. Rob started assigning critical tasks like customer escalations to such employees. Some started staying late for fear of bad appraisals. Some wondered why these tasks are assigned to them when there are few employees who usually work late into the night. Some didn’t budge. They declined to work beyond 6 PM citing personal reasons. Rob did not like these people. He thought they were not dedicated and committed to their work and the organization.

Over a period of time, people who pretended to work for more hours by staying late in the office simply passing on some metrics to please Rob. People who believed this was not right or were incapable of doing this either moved to different teams (if Rob did not goof up their internal transfer) or eventually quit the organization.

Why wasn’t Rob stopped?
Why wasn’t Rob stopped? Why was the 360 degree feedback about Rob thrashed year after year? Why wasn’t the management interested in educating Rob about his management style? Why wasn’t HR doing anything about Rob though they had got a feeler about him from the employees who quit his team and/or the organization? Only the management can answer these questions.

You know what? The management liked Rob. The management was like Rob. It’s only obvious that they liked Rob. The organization claimed to follow a very aggressive business strategy and all it needed was many more managers like Rob who could treat employees like factory workers, make them work for several hours on weekdays and weekends, pay less and yet tell them they got the best raise in the entire team.
What impressed the management team most was the crazy metrics and the sexy graphs that Rob managed to send on a daily basis to the management. They were least bothered to know about how he was getting those metrics, what he did to get them and how he managed many more green lines compared to the red ones that other managers produced at the time. Worse, they did not even know what the green and red lines meant. All they were bothered about was more and more green lines which could be passed on to their bosses who in turn would pass on to their bosses. This passing became a routine until those metrics get projected as the ‘truth’ in the organization.

What about Scapegoats?
Some employees in Rob’s team moved to a different team thanks to a new agile team that was started. Their joy knew no bounds. Most of the sincere employees upon whom he inflicted problems quit over a period of time. Rob’s so called hard working employees threatened to quit the team after which some received a 50%-100% raise. They eventually quit after taking the raise!

When will this stop?
Some day, one smart person will walk into the organization with the courage to question and the power to bring a change. He’ll call Rob into his cabin and ask ‘Hello buddy, can you tell me what these green lines mean?’. Until then, this will not stop. You can be that smart person. Will you be one?

With a very heavy heart, let me say this once and get it off my chest to never mention it again - Rob is the fore runner for the post of the Vice President this year. God Bless!

PS: The characters in this post are true and NOT fictional except the names :D. Any resemblance to you or your friends is deeply regretted. I have no offense to Rob, his so called hard working employees or the scapegoats who gave up fighting similar situations. These are just my observations of one such environment where I FAILED to succeed, yet learned the most about my profession. I EXCELLED in many ways to say the least. My belief in team collaboration only grew stronger. In fact, I cherish working with Rob the most in my career because I now realize the value of being treated with respect and treating fellow colleagues with respect.

Let's be human first - Team Players will grow within,

Regards,
Parimala Shankaraiah