My interview is published in Testing Circus magazine, May 2011 Issue. You could read it HERE. Let me know what you think.
Testing Circus is a free monthly magazine read by testers across the world. Testers worldwide contribute content to this magazine which is free of cost at this point in time. You can also read previous issues on the websites by downloading or reading the pdf online. Recommended for Testers and the rest alike - It's one of the best testing magazines for coffee time reading. Did I tell you they accept articles from "not so expert people" like us too. This could be your first shot at public writing about testing.
We're all busy people. It's with great difficulty that we find time to read blogs, like you're reading mine. Where is the time to read magazines?
How about carrying it to the toilet? How about glancing it while you walk in your balcony (avoid high rise buildings please!)? How about reading as you travel, unless you are driving? How about reading it as you wait for mustard seeds to splutter as you cook? How about spending a few minutes just before you hit the bed? How about letting your 6 yr old to read it for you as you get household chores done? How about trying at least one of the above? Now.
There is no dearth of time Friends; Einstein had the same number of hours in a day that we do.
'Come on Pari, everyone can't be like Einstein'. You can't. But, You can be YOURSELF!
Happy Reading,
Regards,
Parimala Shankaraiah
24 May, 2011
27 April, 2011
Co-operative Unit
Rapid Fire Question - What comes to your mind when I say “Co-operative unit”? I was curious to know what people would come up with and tweeted this question. Below are the only 3 responses I got:
1. A team located in the same premises, cooperative bank, a team cooperative with each other (vipsgupta)
2. Football (lnsimha)
3. Amul (sdhanasekar)
I was expecting “Team” as an answer.
Over the years, I watched many teams perform. Some excelled, some merely existed while the rest fell asleep. What is a Great Team made up of? Unity? Collaboration? Strict Manager? A Great Team is made up of Great Individuals who believe in Team Work.
Couple of months ago, I was chatting with a friend who said something like this: “I want to hire a great team” My response was: “You can’t hire a great team, but you can build one”. Building teams is serious business today. We may solve millions of business problems, but if you leave one people problem unsolved, that is good enough to bring down entire business. Here’s a sneak preview into few good/bad/ugly teams.
Real life example: “Solo Ownership”
Solo Ownership team consisted of team members with sole ownership for anything and everything
Key features of the test team
a. Each tester owned a module based on his experience and knowledge of products
b. Not a single tester looked into someone else’s module [even if his module was tightly integrated with some other module ]
c. If a tester found a bug in a module other than his own, he would not report it. He would not even mention it to the original owner [The attitude was “it’s not my problem”]
d. If a reasonably responsible tester found a bug in a module other than his own and reported it, he would be reprimanded. His reporting manager would ask, “Why are you spending time on someone else’s module?” The other module’s manager would ask, “Why are you testing our module. We have our own test team to do that”.
e. Testing happened in isolation most of the time
f. Integration/System testing happened at the end of the cycle during which severity 1 bugs poured left, right and centre.
Real life example: “Young Entrepreneurs”
Each one in this team felt young and believed in entrepreneurial thinking. It comprised a mature set of individuals who thought not just about getting their work done, but helping others to get their work done too.
Key features of the test team
a. Every tester was a primary owner for one module and secondary owner for another module.
b. Every tester reported bugs in any module of his choice as long as he completed testing his module within stipulated time.
c. Testing happened over the shelf – people walked to almost anyone in the team or outside the team if they had questions or concerns
d. Integration/System testing started pretty early in the testing cycle.
e. Job oriented trainings - mostly conducted by members internal to the team to facilitate quick learning curve for new hires in the team.
Real life example: “Mind your own Business”
This team comprised mostly of senior testers with several years of testing experience in their kitty, yet behaved like a bunch of egoistic people. There were a few good people, but tried hard to appear bad because good deeds weren't respected. In fact, good people would often be taken for a ride.
Key features of test team
a. Each tester would do what he liked – mostly after threatening his reporting manager to quit if things don’t improve [by the way, this happened just before the end of each quarter during which bonus numbers would be decided].
b. If critical bugs were found early in the test cycle by a colleague, some team members would say “Dude! It’s no big deal. You are doing your job. After all, that is why you were hired”
c. If a cosmetic bug was missed and customer reported it, team members crucified the corresponding tester and ask, “Were you sleeping? How did you miss such a simple bug? You should get your eyes checked” in a mocking tone.
d. Any delay in dropping builds to testers was OK, but deadline was NOT negotiable.
e. It was mandatory to work extra hours every day. Anyone who consistently spent more than 14 hrs a day in office was a rock star tester and a dedicated team member [Notice the word “spent”]
f. Every problem reported by the customer in any tester’s module meant his bonus would be reduced by 20%. Five such problems and zero bonus. Simple Math.
g. Testers attend trainings so they can sleep for a while and occasionally shout from the top of their voices to show off that they are more intelligent than the rest.
h. And there is no end to this story. It goes on and on …..
Real life example: “We are Family”
Truly a family. There is no distinction between programmers, testers, managers, technical writers and other immediate stakeholders within the organization. Everyone was important and was encouraged to make some unique contribution. Each team member worked closely with rest of the team to meet commonly set objectives. These objectives were directly aligned with organizational objectives.
Key features of the team
a. There was no single rock star team member. All were star performers and pushed each other to excel and perform better.
b. Every complex activity happened in pairs or in groups – which encouraged group learning and on the job training.
c. Team members met at regular intervals to discuss the state of the project they were working on to ensure everyone is on the same page.
d. Programmers highlighted risks pretty early in the life cycle and testers would test early builds to provide immediate information to them.
e. Product owners wrote/re-wrote/reviewed requirements in detail at the beginning of any project and reviewed/modified them periodically with customers and engineering teams.
f. Customers were given monthly demos of working features to solicit early feedback.
Closing comments
While writing this blog post, I wrote a little section called Outcome for each team and how those teams excelled or disintegrated. During review, I realized everyone who hears about these teams knows the outcome. Why bother calling names? Let's bother about building teams. Eh?
My ex-manager Dipankar Roy had tremendous influence on me with regard to team work. His belief in team work helped him build not just extraordinary teams, but amazing professionals as well. He once said “Any team is only as strong as its weakest link”. It doesn’t matter how many are star performers, how many won quarterly awards or how many excelled. Even if one team member is slow or not on par with rest of their crew, the entire team slows down and heads downhill. The key to any strong team is to empower ‘weaklings’ to become star performers. Remember, these so called ‘weaklings’ are not on par with the rest simply because they don’t know that they don’t know. You read it right. They don’t know that they don’t know (MOSTLY). The moment, they are nurtured and valued, they’ll outgrow the best of star performers. Empower people to transform as stars. Whether they shine or become dust is their choice.
Regards,
Parimala Shankaraiah
P.S.
Weakling is an offensive word. The reason I used it here is to emphasize how people who are slow/passive in the team are considered almost worthless by their immediate leads/managers. These very same weaklings can deliver beautifully if mentored by true leaders who believe in people power. Some leaders create magic with people. They value each person’s interests, ambitions, goals and help align team goals such that every team member feels respected and contributes to the team. Only true leaders can do that.
1. A team located in the same premises, cooperative bank, a team cooperative with each other (vipsgupta)
2. Football (lnsimha)
3. Amul (sdhanasekar)
I was expecting “Team” as an answer.
Over the years, I watched many teams perform. Some excelled, some merely existed while the rest fell asleep. What is a Great Team made up of? Unity? Collaboration? Strict Manager? A Great Team is made up of Great Individuals who believe in Team Work.
Couple of months ago, I was chatting with a friend who said something like this: “I want to hire a great team” My response was: “You can’t hire a great team, but you can build one”. Building teams is serious business today. We may solve millions of business problems, but if you leave one people problem unsolved, that is good enough to bring down entire business. Here’s a sneak preview into few good/bad/ugly teams.
Real life example: “Solo Ownership”
Solo Ownership team consisted of team members with sole ownership for anything and everything
Key features of the test team
a. Each tester owned a module based on his experience and knowledge of products
b. Not a single tester looked into someone else’s module [even if his module was tightly integrated with some other module ]
c. If a tester found a bug in a module other than his own, he would not report it. He would not even mention it to the original owner [The attitude was “it’s not my problem”]
d. If a reasonably responsible tester found a bug in a module other than his own and reported it, he would be reprimanded. His reporting manager would ask, “Why are you spending time on someone else’s module?” The other module’s manager would ask, “Why are you testing our module. We have our own test team to do that”.
e. Testing happened in isolation most of the time
f. Integration/System testing happened at the end of the cycle during which severity 1 bugs poured left, right and centre.
Real life example: “Young Entrepreneurs”
Each one in this team felt young and believed in entrepreneurial thinking. It comprised a mature set of individuals who thought not just about getting their work done, but helping others to get their work done too.
Key features of the test team
a. Every tester was a primary owner for one module and secondary owner for another module.
b. Every tester reported bugs in any module of his choice as long as he completed testing his module within stipulated time.
c. Testing happened over the shelf – people walked to almost anyone in the team or outside the team if they had questions or concerns
d. Integration/System testing started pretty early in the testing cycle.
e. Job oriented trainings - mostly conducted by members internal to the team to facilitate quick learning curve for new hires in the team.
Real life example: “Mind your own Business”
This team comprised mostly of senior testers with several years of testing experience in their kitty, yet behaved like a bunch of egoistic people. There were a few good people, but tried hard to appear bad because good deeds weren't respected. In fact, good people would often be taken for a ride.
Key features of test team
a. Each tester would do what he liked – mostly after threatening his reporting manager to quit if things don’t improve [by the way, this happened just before the end of each quarter during which bonus numbers would be decided].
b. If critical bugs were found early in the test cycle by a colleague, some team members would say “Dude! It’s no big deal. You are doing your job. After all, that is why you were hired”
c. If a cosmetic bug was missed and customer reported it, team members crucified the corresponding tester and ask, “Were you sleeping? How did you miss such a simple bug? You should get your eyes checked” in a mocking tone.
d. Any delay in dropping builds to testers was OK, but deadline was NOT negotiable.
e. It was mandatory to work extra hours every day. Anyone who consistently spent more than 14 hrs a day in office was a rock star tester and a dedicated team member [Notice the word “spent”]
f. Every problem reported by the customer in any tester’s module meant his bonus would be reduced by 20%. Five such problems and zero bonus. Simple Math.
g. Testers attend trainings so they can sleep for a while and occasionally shout from the top of their voices to show off that they are more intelligent than the rest.
h. And there is no end to this story. It goes on and on …..
Real life example: “We are Family”
Truly a family. There is no distinction between programmers, testers, managers, technical writers and other immediate stakeholders within the organization. Everyone was important and was encouraged to make some unique contribution. Each team member worked closely with rest of the team to meet commonly set objectives. These objectives were directly aligned with organizational objectives.
Key features of the team
a. There was no single rock star team member. All were star performers and pushed each other to excel and perform better.
b. Every complex activity happened in pairs or in groups – which encouraged group learning and on the job training.
c. Team members met at regular intervals to discuss the state of the project they were working on to ensure everyone is on the same page.
d. Programmers highlighted risks pretty early in the life cycle and testers would test early builds to provide immediate information to them.
e. Product owners wrote/re-wrote/reviewed requirements in detail at the beginning of any project and reviewed/modified them periodically with customers and engineering teams.
f. Customers were given monthly demos of working features to solicit early feedback.
Closing comments
While writing this blog post, I wrote a little section called Outcome for each team and how those teams excelled or disintegrated. During review, I realized everyone who hears about these teams knows the outcome. Why bother calling names? Let's bother about building teams. Eh?
My ex-manager Dipankar Roy had tremendous influence on me with regard to team work. His belief in team work helped him build not just extraordinary teams, but amazing professionals as well. He once said “Any team is only as strong as its weakest link”. It doesn’t matter how many are star performers, how many won quarterly awards or how many excelled. Even if one team member is slow or not on par with rest of their crew, the entire team slows down and heads downhill. The key to any strong team is to empower ‘weaklings’ to become star performers. Remember, these so called ‘weaklings’ are not on par with the rest simply because they don’t know that they don’t know. You read it right. They don’t know that they don’t know (MOSTLY). The moment, they are nurtured and valued, they’ll outgrow the best of star performers. Empower people to transform as stars. Whether they shine or become dust is their choice.
Regards,
Parimala Shankaraiah
P.S.
Weakling is an offensive word. The reason I used it here is to emphasize how people who are slow/passive in the team are considered almost worthless by their immediate leads/managers. These very same weaklings can deliver beautifully if mentored by true leaders who believe in people power. Some leaders create magic with people. They value each person’s interests, ambitions, goals and help align team goals such that every team member feels respected and contributes to the team. Only true leaders can do that.
17 April, 2011
Belling the Cat - Learnings from Testing Contests
There is a huge rise in testing contests in some parts of the world. Many testing conferences today facilitate testing contests, quiz competitions, puzzles solving sessions and many more to draw people’s attention at conferences. Several crowdsourcing companies today offer short term testing projects as contests and award testers appropriately for reported bugs. If you are a tester, you love what you do and you have disposable time with you and willing to make some quick bucks, why not participate?
I have participated in many testing contests in the last one year. It’s been quite an experience. I would like to share my learnings here as it might help some of you who follow suit.
What do testing contests offer?
Quick Learning
Learning a product quickly and reporting problems within stipulated time is a challenge. Testing contests offer such an environment. You are expected to learn as much as possible and test the product. Zero Documentation available. Some of us feel miserable. Zero Documentation? It’s impossible to test. If you are working for a large organization, you must be familiar with Feasibility analysis, requirements specification, functional specification, technical specification, test design specification and technical documentation. If you are working for a tiny startup, all you would have is a bundle of code which looks like a 2 month old foetus; If you are a little fortunate, you might be blessed with a developer whom you can talk to; if you are a little more fortunate, you may have that guy next to your cubicle. Your luck factor ends right there. What if you neither have documents nor people to tell about the product. Self Learning. This is one of the most wonderful ways of learning till date. Self Learning nurtures creativity, exploration, investigation and critical thinking. Testing contests encourage self learning to a good extent.
Time Pressure
There’s not a single tester who wouldn’t crib about lack of enough time to test [except testers who have learned the art of coverage and risk mitigation]. Testing contests offer this pressure too. BUT, participants hardly take pressure. Its just a game for them. They neither have fear of missing a bug nor do they worry about getting laid off for lousy work. At the most, they won’t win the contest. Outcome: Calm and Relaxed Mind. This helps them think critically about how to test the product. Contests offer a lot of time for testers to think without worrying about time. Weekend Testing participants can tell you better how enjoyable testing experience is when testers test without anyone sitting on their heads for status or numbers.
Fault-Safe Environment
Contests offer a fault-safe setup where one can exercise individual skills without rebuke. Success in contests is not about winning alone, but the journey one goes through to reach the destination. Testers should cash in on their knowledge and enhance their skills on such platforms. One of my ex-ceo Josh Pickus often said, “Never be afraid to fail. Whenever you fail, don’t fail to learn from your mistakes”.
Exposure to variety of products
Where can you get exposure to different products at your fingertips? Softwares used in medical devices, online library management, applications on mobile devices, banking - anything that you would not have tested before. At workplace, many testers end up testing same product for 1-3 years, sometimes maintaining them for many more years. There’s nothing wrong with that, but learning about products addressing different customer needs is interesting and fun. When such an opportunity strikes, one should not let go of it.
Collaboration with other testers
Some products in some contests require collaboration from multiple users. For e.g. if there is a chat feature, unless the same user is logged in on multiple browsers/machines, it may not be possible to have real time chat sessions. Such scenarios facilitate collaboration with other testers who not just help themselves, but help others too. I collaborated with Bharath in one such competition and tested some shared features together. This virtual pair testing took us through a journey of loopholes in the product. We found many interesting bugs and each one different at my end and at his end. Both were happy not just because we found more information about the product,but because we unearthed them in such a short time as we worked closely with each other. That was a great lesson on collaboration.
Credibility
Win or lose - Testers build credibility as they take part in more contests. Such credibility gives them not just more practice, but more and more contests go their way. If that works out, they might even end up getting some free lance projects to test - in their spare time. What more do you want? Wake up folks.
Contests are open for everyone alike. This throws you open into a group of diverse people whose brains are wired differently. This facilitates a one of the kind experience by allowing many less experienced testers or students to take guidance from experienced testers. This also provides a great opportunity to mentor other testers who need help.
Consistency in quality of bugs
Contests don’t accept your bugs as they are. Each bug is validated, prioritized and weighed against the criticality of fixing it. That brings us to valid and invalid bugs. Its hurtful when bugs get rejected. However, each rejected bug teaches a lesson, not of ridicule, but of realization. Taking part in contests can help you by allowing you to think about the validity of each bug before reporting it. This way, testers can learn to weed out invalid bugs from their queue. This helps get rid of ‘report anything and everything different as a bug’ thinking.
Varied Test Combinations
Unless you have worked on One Man Army projects where you are the only tester, you wouldn’t know that testing under different configurations (operating systems, browsers etc) can reveal amazing things about these products. Contests usually offer a choice to choose from different test configurations where you can learn and experiment based on your skillset/expertise. You can also choose to test using a completely new approach and get bowled by your performance.
Test ideas and bug ideas exhaustion
I happened to take part in a few contests where scores of bugs were already reported. My first thought was, “Oh! Most of the bugs are fished out already. I don’t think I am needed here”. I took these opportunities as challenges and said to myself, “Let’s see if I can find anything new here”. I was surprised by the variety of choices I had to test. Sometimes, I chose different test techniques. Other times, I tested on different operating systems and browsers. In some cases, I tested for different language support options, search engine optimization support, looking for broken links etc. At the end of this exercise, I was happy as I found many problems in the product which I would not find if I was one of the earliest testers on the contest. Filing straight forward bugs is a cakewalk. Filing hard to notice bugs is a uphill task.
And don't forget the test ideas you might get for future tests when you read through existing bug reports of other testers. That is a gold mine of knowledge as well. Are you game for it?
Challenges in a testing contest
Filing bugs - left, right and center
Contest instructs your brain more often to Win. To win, you can do anything. If you can win more by cheating instead of playing fair, why not do that. Many testers get lost in suchemotions and report anything that they think is a deviation from the product. This trend makes programmers life difficult which many testers fail to understand or in simple terms ignore. Another common problem is poor bug reporting skills. Some testers produce lousy test reports in a hurry to find more bugs. Sure. find more bugs and report more. But work on your writing skills. Don't degrade them further. Let me tell you, bug reporting is an art in itself.
Severity vs. Priority
This is another common problem where testers report low severity bugs as higher ones. In cases, where testers are not aware of whether a particular type of bug is severe or not, assigning a severity based on their experience and belief is fine. In situations where testers are instructed to find only certain types of bugs, if other types of defects are reported and at high severity, it would annoy the product management team for sure.
Not sticking to the mission
When there is a mission set for any contest, testers should stick to their mission. They shouldn’t deviate from the mission. Though deviations are unavoidable, sticking to the mission and executing tests within the mission gives more focus to that contest can be more productive. Testers often miss out on these things. In most cases, they tend to ignore which is not a good approach to contests.
Winning streak
Increasing bug count for winning the contest than providing quality information about product is a wrong doing. Though testers can win many contests this way, not reporting quality bugs can eat into our own credibility. Again, reporting symptoms as bug clusters may hamper credibility. Success is not just about winning, so its high time testers take contests in the right competitive spirit and put up a fair fight with the rest of the crew.
Duplicates
Testers file duplicates without looking into similar bugs reported already. How much time does it take to quickly look at defects before reporting one especially in a crowdsourcing environment. Many testers say that they get bored to browse through already reported bugs. Imagine the plight of programmers who have to go through every bug 2-3 times assuming at least 3 testers are testing one product and most of the bugs are duplicates.
Stake holders late entry
Bringing in stake holders very late into the project who then instruct testers to shift focus to other testing approaches or test techniques do more harm to the contest especially if there was no mission set at the begginning. It is a good idea to throw open stake holders and testers into an open room where they discuss the requirements, expectations and risks involved in the products. This will bring more success to the contests in long run.
Testing Contests I know of
If you have come this far, you must be wondering where you can find paid/unpaid testing contests to nurture your testing soul and also make some extra money. Some of these include 99Tests, uTest Bug Battles and Software Testing Club (Flash Mob Testing). These days, many testers on twitter offer free testing challenges as well.
Next time, you come across a contest, please give it a try. If you have already participated in some of these, I would love to hear about your experiences. Let's talk.
Regards,
Parimala Shankaraiah
I have participated in many testing contests in the last one year. It’s been quite an experience. I would like to share my learnings here as it might help some of you who follow suit.
What do testing contests offer?
Quick Learning
Learning a product quickly and reporting problems within stipulated time is a challenge. Testing contests offer such an environment. You are expected to learn as much as possible and test the product. Zero Documentation available. Some of us feel miserable. Zero Documentation? It’s impossible to test. If you are working for a large organization, you must be familiar with Feasibility analysis, requirements specification, functional specification, technical specification, test design specification and technical documentation. If you are working for a tiny startup, all you would have is a bundle of code which looks like a 2 month old foetus; If you are a little fortunate, you might be blessed with a developer whom you can talk to; if you are a little more fortunate, you may have that guy next to your cubicle. Your luck factor ends right there. What if you neither have documents nor people to tell about the product. Self Learning. This is one of the most wonderful ways of learning till date. Self Learning nurtures creativity, exploration, investigation and critical thinking. Testing contests encourage self learning to a good extent.
Time Pressure
There’s not a single tester who wouldn’t crib about lack of enough time to test [except testers who have learned the art of coverage and risk mitigation]. Testing contests offer this pressure too. BUT, participants hardly take pressure. Its just a game for them. They neither have fear of missing a bug nor do they worry about getting laid off for lousy work. At the most, they won’t win the contest. Outcome: Calm and Relaxed Mind. This helps them think critically about how to test the product. Contests offer a lot of time for testers to think without worrying about time. Weekend Testing participants can tell you better how enjoyable testing experience is when testers test without anyone sitting on their heads for status or numbers.
Fault-Safe Environment
Contests offer a fault-safe setup where one can exercise individual skills without rebuke. Success in contests is not about winning alone, but the journey one goes through to reach the destination. Testers should cash in on their knowledge and enhance their skills on such platforms. One of my ex-ceo Josh Pickus often said, “Never be afraid to fail. Whenever you fail, don’t fail to learn from your mistakes”.
Exposure to variety of products
Where can you get exposure to different products at your fingertips? Softwares used in medical devices, online library management, applications on mobile devices, banking - anything that you would not have tested before. At workplace, many testers end up testing same product for 1-3 years, sometimes maintaining them for many more years. There’s nothing wrong with that, but learning about products addressing different customer needs is interesting and fun. When such an opportunity strikes, one should not let go of it.
Collaboration with other testers
Some products in some contests require collaboration from multiple users. For e.g. if there is a chat feature, unless the same user is logged in on multiple browsers/machines, it may not be possible to have real time chat sessions. Such scenarios facilitate collaboration with other testers who not just help themselves, but help others too. I collaborated with Bharath in one such competition and tested some shared features together. This virtual pair testing took us through a journey of loopholes in the product. We found many interesting bugs and each one different at my end and at his end. Both were happy not just because we found more information about the product,but because we unearthed them in such a short time as we worked closely with each other. That was a great lesson on collaboration.
Credibility
Win or lose - Testers build credibility as they take part in more contests. Such credibility gives them not just more practice, but more and more contests go their way. If that works out, they might even end up getting some free lance projects to test - in their spare time. What more do you want? Wake up folks.
Contests are open for everyone alike. This throws you open into a group of diverse people whose brains are wired differently. This facilitates a one of the kind experience by allowing many less experienced testers or students to take guidance from experienced testers. This also provides a great opportunity to mentor other testers who need help.
Consistency in quality of bugs
Contests don’t accept your bugs as they are. Each bug is validated, prioritized and weighed against the criticality of fixing it. That brings us to valid and invalid bugs. Its hurtful when bugs get rejected. However, each rejected bug teaches a lesson, not of ridicule, but of realization. Taking part in contests can help you by allowing you to think about the validity of each bug before reporting it. This way, testers can learn to weed out invalid bugs from their queue. This helps get rid of ‘report anything and everything different as a bug’ thinking.
Varied Test Combinations
Unless you have worked on One Man Army projects where you are the only tester, you wouldn’t know that testing under different configurations (operating systems, browsers etc) can reveal amazing things about these products. Contests usually offer a choice to choose from different test configurations where you can learn and experiment based on your skillset/expertise. You can also choose to test using a completely new approach and get bowled by your performance.
Test ideas and bug ideas exhaustion
I happened to take part in a few contests where scores of bugs were already reported. My first thought was, “Oh! Most of the bugs are fished out already. I don’t think I am needed here”. I took these opportunities as challenges and said to myself, “Let’s see if I can find anything new here”. I was surprised by the variety of choices I had to test. Sometimes, I chose different test techniques. Other times, I tested on different operating systems and browsers. In some cases, I tested for different language support options, search engine optimization support, looking for broken links etc. At the end of this exercise, I was happy as I found many problems in the product which I would not find if I was one of the earliest testers on the contest. Filing straight forward bugs is a cakewalk. Filing hard to notice bugs is a uphill task.
And don't forget the test ideas you might get for future tests when you read through existing bug reports of other testers. That is a gold mine of knowledge as well. Are you game for it?
Challenges in a testing contest
Filing bugs - left, right and center
Contest instructs your brain more often to Win. To win, you can do anything. If you can win more by cheating instead of playing fair, why not do that. Many testers get lost in suchemotions and report anything that they think is a deviation from the product. This trend makes programmers life difficult which many testers fail to understand or in simple terms ignore. Another common problem is poor bug reporting skills. Some testers produce lousy test reports in a hurry to find more bugs. Sure. find more bugs and report more. But work on your writing skills. Don't degrade them further. Let me tell you, bug reporting is an art in itself.
Severity vs. Priority
This is another common problem where testers report low severity bugs as higher ones. In cases, where testers are not aware of whether a particular type of bug is severe or not, assigning a severity based on their experience and belief is fine. In situations where testers are instructed to find only certain types of bugs, if other types of defects are reported and at high severity, it would annoy the product management team for sure.
Not sticking to the mission
When there is a mission set for any contest, testers should stick to their mission. They shouldn’t deviate from the mission. Though deviations are unavoidable, sticking to the mission and executing tests within the mission gives more focus to that contest can be more productive. Testers often miss out on these things. In most cases, they tend to ignore which is not a good approach to contests.
Winning streak
Increasing bug count for winning the contest than providing quality information about product is a wrong doing. Though testers can win many contests this way, not reporting quality bugs can eat into our own credibility. Again, reporting symptoms as bug clusters may hamper credibility. Success is not just about winning, so its high time testers take contests in the right competitive spirit and put up a fair fight with the rest of the crew.
Duplicates
Testers file duplicates without looking into similar bugs reported already. How much time does it take to quickly look at defects before reporting one especially in a crowdsourcing environment. Many testers say that they get bored to browse through already reported bugs. Imagine the plight of programmers who have to go through every bug 2-3 times assuming at least 3 testers are testing one product and most of the bugs are duplicates.
Stake holders late entry
Bringing in stake holders very late into the project who then instruct testers to shift focus to other testing approaches or test techniques do more harm to the contest especially if there was no mission set at the begginning. It is a good idea to throw open stake holders and testers into an open room where they discuss the requirements, expectations and risks involved in the products. This will bring more success to the contests in long run.
Testing Contests I know of
If you have come this far, you must be wondering where you can find paid/unpaid testing contests to nurture your testing soul and also make some extra money. Some of these include 99Tests, uTest Bug Battles and Software Testing Club (Flash Mob Testing). These days, many testers on twitter offer free testing challenges as well.
Next time, you come across a contest, please give it a try. If you have already participated in some of these, I would love to hear about your experiences. Let's talk.
Regards,
Parimala Shankaraiah
25 March, 2011
Where’s Curious Tester been?
Ever since I started blogging, I kept bragging how Curious Tester was my second child and how much I enjoyed spending most of my online time here. When my second child arrived in REAL 3 months ago, I realized how this blog became a step child almost instantly. It hurt because for two years, I nurtured it as if it was a baby and all of a sudden I had orphaned it. As the proverb goes “No pain, no gain”. I had to prioritize and my little one won over unanimously. As she grappled with the outside world, I grappled with my learnings, shortcomings and also came up with some cool stuff.
So what did I do all this while? Thanks to many of those wonderful sleepless nights, I was still on twitter and my RSS feeds entertained me while I was awake at odd hours. I read several books spanning testing, technology, parenting, relationships, motivation and many more. Now, don’t ask me for the list of books, I seriously don’t remember and I am too lazy to look up the bookshelf to recreate the list here J. I also happened to test many websites and applications on my Nokia E63 cell phone which was a great boon this time around. My reactions varied from ‘Oh, this site is so lousy on the phone’ to ‘What more can be done?’ to 'What value does testing bring in here' to many many more. There were many interesting observations which I wouldn't have found out if I had to test on a computer. Again, don’t ask me for those experience reports J
For a while, I often wondered if my readers would ever miss my writings ( this little human thing in me!). Thankfully, many did and I am grateful to all of them who messaged to find out if all was well with me. And yes, I am back! Watch this space!
New Beginnings!
Regards,
Parimala Shankaraiah
17 December, 2010
Buggy Reporting
What’s buggy reporting? You might ask. No, No. It’s not yet another new term. I don’t want to add newer terms to the technical jargon wars going on in different parts of the world already.
Buggy reporting talks about how buggy real bug reports can be. It bugs me a lot to read buggy bug reports which hardly talk about bugs in a understandable way. As testers, we all know that bug reports are important for providing information to the stakeholders. Still, we don’t take any effort to review our own bug reports. If we ourselves are not interested in reading our own bug reports for clarity, how can we expect programmers and other stakeholders to not just read them, but also decide whether to fix problems reported in those bugs? Height of Optimism!
“Spelling mistake”
“file not loading” and
My personal favorite “Thomas is a fool!”
“Thomas is a fool” was a real bug in one of my previous organizations. As part of learning a complex component, I was reviewing all the older bugs that were reported and fixed within that component. That’s when I chanced upon this “Antique Piece". Thomas was one of the managers in the organization who may or may not have done anything good to one of the employees who literally reported this bug on last day of his employment. Quite a cool bug, I never know if Thomas was a bug or the guy who called Thomas a bug was a bug. Nevertheless, If you plan to report such a bug, please don’t quote me on that!
Jokes apart, let’s change roles for a moment as we become programmers. If you are entrusted with the responsibility of fixing bugs with bug summaries as above, what would your first thought be? Scratch your head or Trash them? Once, twice, thrice. If I were a programmer, I would trash not just the bugs, but even folks who write such bugs. And yes I am harsh, because every important bug that fails to catch stakeholders attention due to bad reporting is as good as a dead one.
Some of us might wonder “Oh! The product is so buggy that I have to report at least 20 bugs a day. With whatever little time I have, I can only make my bug reports shorter so I can write more bugs in a day”. My question to such testers is “How much are you bothered or not bothered if none of the 20 bugs reported this way are fixed”. Don’t tell me “That’s not my problem”. Agreed that we are information providers and not decision makers. But are we really providing useful information? Just a thought.
“search for item, I got script error”
“when my transaction failed, link to database cannot establish”
“in payment by debit card the bak name is choosen from the drop down menu , after that there is no requirement in the gateway page to ask the name of the ‘Issuing Bank’”
Don’t be surprised. These are real descriptions of real bugs. In other words, "Steps to reproduce" if that is the word your organization uses. At first sight, it might look like I am making fun of all those non-English speaking testers who confess openly about their lack of good English writing skills. Bug reporting is much more than knowing English well. Can’t we just look back and read/re-read the bug before hitting the submit button? Is that so hard?
“What’s the fun in detailed reporting? Am I so jobless to type tons of information for every bug I report? Can’t programmers figure out such a simple thing? Won’t my expertise go away if I gave all this information out for free? Let’s be honest - I struggled a lot to get here and find this bug, why should I give away information just like that. Let people come back to me, I’ll tell them and I’ll tell them only if I like them”
Some descriptions are so painless. A simple copy/paste of the bug summary. It’s as though the bug reporter was held at gunpoint and asked to report a bug one last time before giving away his life forever and ever. Or maybe his bowel movements didn’t permit him to do so in the time available for him to rush from his desk to the washroom. No offense meant – frankly it’s that annoying to read bad bugs to figure out what’s going on. Total waste of time.
Reputation PointsRajesh filed about 22 bugs in 2 days timeframe while Anil who was testing the same module reported only 4 bugs in same time. Both have 6.5 years experience, both are from computer science background, both should be filing at least reasonably equal number of bugs. Hence, Rajesh is a better tester. Till now, not a single soul would have looked at the following factors:
1. Bug reporting skills
2. Investigation skills
3. Ability to provide detailed information so that programmers don’t float between the bug and the bug reporter to get additional information
4. How many bugs are valid versus invalid?
5. Questioning skills (No, not the “why I need to login” type of bugs)
6. How many of the bugs reported did the tester get fixed finally?
All that mattered to the management was Rajesh has reported the highest number of bugs and he should be given the ‘Star Tester of the Quarter’ award in Q4 2010.
What a fishy method to earn reputation? Just because you are testing a buggy product or you find a bug every minute doesn’t mean you have to report bugs in a lousy way. Just because you want to beat your counterpart in the number of bugs reported as part of the bet you had with him or your manager rates you this way, doesn’t mean that you can stoop to any level of lack of quality in your own bugs.
If you are a tester, go back and read your bug reports as if you did not report it at all. Identify the gaps and work on those gaps by practicising writing good bug reports. If you are programmer who is annoyed with your tester’s bug reporting skills, tell them politely about what basic information you expect instead of running to his desk every 5 minutes which wastes not just his time, but yours too!
I recently met up with a CTO of one company whose product was tested by a bunch of testers outside his organization. When I questioned him about what he thought about the bugs reported by independent testers, he just said one thing, “Each time I read the bugs reported by these testers, I often wondered - why aren’t our in-house testers reporting bugs this way…….. " And he kept scratching his head for the next 30 seconds after uttering this. I felt a lot of pain in what he said. I wish he goes back to them and politely tells them that they need to improve in their bug reporting skills.
Happy Bug Reporting,
Regards,
Parimala Shankaraiah
Buggy reporting talks about how buggy real bug reports can be. It bugs me a lot to read buggy bug reports which hardly talk about bugs in a understandable way. As testers, we all know that bug reports are important for providing information to the stakeholders. Still, we don’t take any effort to review our own bug reports. If we ourselves are not interested in reading our own bug reports for clarity, how can we expect programmers and other stakeholders to not just read them, but also decide whether to fix problems reported in those bugs? Height of Optimism!
Awesome Titles
“Script warning”“Spelling mistake”
“file not loading” and
My personal favorite “Thomas is a fool!”
“Thomas is a fool” was a real bug in one of my previous organizations. As part of learning a complex component, I was reviewing all the older bugs that were reported and fixed within that component. That’s when I chanced upon this “Antique Piece". Thomas was one of the managers in the organization who may or may not have done anything good to one of the employees who literally reported this bug on last day of his employment. Quite a cool bug, I never know if Thomas was a bug or the guy who called Thomas a bug was a bug. Nevertheless, If you plan to report such a bug, please don’t quote me on that!
Jokes apart, let’s change roles for a moment as we become programmers. If you are entrusted with the responsibility of fixing bugs with bug summaries as above, what would your first thought be? Scratch your head or Trash them? Once, twice, thrice. If I were a programmer, I would trash not just the bugs, but even folks who write such bugs. And yes I am harsh, because every important bug that fails to catch stakeholders attention due to bad reporting is as good as a dead one.
Some of us might wonder “Oh! The product is so buggy that I have to report at least 20 bugs a day. With whatever little time I have, I can only make my bug reports shorter so I can write more bugs in a day”. My question to such testers is “How much are you bothered or not bothered if none of the 20 bugs reported this way are fixed”. Don’t tell me “That’s not my problem”. Agreed that we are information providers and not decision makers. But are we really providing useful information? Just a thought.
Detailed Descriptions
“Why I need to login?”“search for item, I got script error”
“when my transaction failed, link to database cannot establish”
“in payment by debit card the bak name is choosen from the drop down menu , after that there is no requirement in the gateway page to ask the name of the ‘Issuing Bank’”
Don’t be surprised. These are real descriptions of real bugs. In other words, "Steps to reproduce" if that is the word your organization uses. At first sight, it might look like I am making fun of all those non-English speaking testers who confess openly about their lack of good English writing skills. Bug reporting is much more than knowing English well. Can’t we just look back and read/re-read the bug before hitting the submit button? Is that so hard?
“What’s the fun in detailed reporting? Am I so jobless to type tons of information for every bug I report? Can’t programmers figure out such a simple thing? Won’t my expertise go away if I gave all this information out for free? Let’s be honest - I struggled a lot to get here and find this bug, why should I give away information just like that. Let people come back to me, I’ll tell them and I’ll tell them only if I like them”
Some descriptions are so painless. A simple copy/paste of the bug summary. It’s as though the bug reporter was held at gunpoint and asked to report a bug one last time before giving away his life forever and ever. Or maybe his bowel movements didn’t permit him to do so in the time available for him to rush from his desk to the washroom. No offense meant – frankly it’s that annoying to read bad bugs to figure out what’s going on. Total waste of time.
Attachments
We have many free tools to take screenshots, many more to capture videos and above that, we have the freedom to attach logs, xmls, data files, test files and lot more options to help with bug investigation and fixing. Again, the same old question – who has the time? We don’t want to provide anything until the bug goes to “Needs more Information” state. And we would be offended if it’s moved to “Invalid” state, all this for no mistake of folks who read our dead bug reports.Reputation Points
1. Bug reporting skills
2. Investigation skills
3. Ability to provide detailed information so that programmers don’t float between the bug and the bug reporter to get additional information
4. How many bugs are valid versus invalid?
5. Questioning skills (No, not the “why I need to login” type of bugs)
6. How many of the bugs reported did the tester get fixed finally?
All that mattered to the management was Rajesh has reported the highest number of bugs and he should be given the ‘Star Tester of the Quarter’ award in Q4 2010.
What a fishy method to earn reputation? Just because you are testing a buggy product or you find a bug every minute doesn’t mean you have to report bugs in a lousy way. Just because you want to beat your counterpart in the number of bugs reported as part of the bet you had with him or your manager rates you this way, doesn’t mean that you can stoop to any level of lack of quality in your own bugs.
If you are a tester, go back and read your bug reports as if you did not report it at all. Identify the gaps and work on those gaps by practicising writing good bug reports. If you are programmer who is annoyed with your tester’s bug reporting skills, tell them politely about what basic information you expect instead of running to his desk every 5 minutes which wastes not just his time, but yours too!
I recently met up with a CTO of one company whose product was tested by a bunch of testers outside his organization. When I questioned him about what he thought about the bugs reported by independent testers, he just said one thing, “Each time I read the bugs reported by these testers, I often wondered - why aren’t our in-house testers reporting bugs this way…….. " And he kept scratching his head for the next 30 seconds after uttering this. I felt a lot of pain in what he said. I wish he goes back to them and politely tells them that they need to improve in their bug reporting skills.
Happy Bug Reporting,
Regards,
Parimala Shankaraiah
11 November, 2010
What does your Customer want?
Product Conclave 2010
I had the privilege of attending Product Conclave 2010 conference organized by Nasscom recently thanks to my organization’s sponsorship. Apart from the fact that I was nominated to learn about Cloud based services, all that the agenda said was that it was targeted towards entrepreneurs. And it feels good to think that you are an entrepreneur, isn’t it? At least, an entrepreneur in the future?
One really nice thing about this conference was the wide array of panel discussions and workshops spread over two days. I happened to attend one such workshop on “Building Great Products with Inspiration and Rigor”. What struck me at the end of the workshop was how stupidly we as engineers build products by thinking less about the customer and more about the profits we need to make.
Goal of the Workshop
You need to design and build the best pet possible in a competitive market. Your goal is to make more net revenue in units than your competitors. The team that has made the most units, not necessarily sold the most animals, will win the competition. Legos will be provided to build the pets.
Strategy
Our team was a complete set of strangers including a senior manager, two entrepreneurs, one marketing person, two directors of a startup, a developer and a tester (Me!). For a moment, I felt like the worst nerd in the team and being the worst helped me as you’ll find out in a while.
The senior manager and the developer suggested that we break into two teams: Core pet building team and the revenue team. Pet building team started building the pet based on the specifications given to them while the revenue strategy team started working out the pricing detail. We would have 3 iterations to build the pet. At the end of each iteration, a record number of 24 customers would walk by for a demo after which they would decide to buy or reject the pet.
Lost Once, Lost Forever
At the end of first iteration, we were ready with a sexy pet we called ‘Pet-O-Smart’ – an alien dog which is not just a caring pet, but also acts as a robot to entertain the pet owner [This is just a simulation, remember?]. When the customers came by, our team collectively decided that we would not sell the pet yet as we thought it wasn’t fully ready. Our pet needed some more development while the revenue team was busy working out the pricing based on the market research we bought during first iteration. Some customers found our partially developed pet very attractive to buy, but were disappointed with our decision not to sell the pet yet.
Lesson: Customers who visited us at the end of 1st iteration were not only interested in our pet, but were cash rich. There was a high possibility that they would have bought many pets at the price we quoted. We didn’t intend to sell at this point and we prided in that decision as we heard cheers from other teams as they made their sales. It was a pretty bad decision for a reason. Even if we did not want to sell it, we could have demoed our pet to many customers (Campaign Management) and develop some leads (potential customers) for subsequent iterations. We lost some customers some of whom never came back because they were offended that we drove them away.
Conservative – May Buy
At the end of second iteration, Pet-O-Smart was fully ready for demo and sale. We priced it at 61 units per pet. First customer came in and all that we did as a team was surround like ants around a sugar cube and confuse him with all sorts of data of what it doesn’t do. We highlighted how our pet does bite now and then, but it’s harmless. Customer got angry. Another customer was driven away knowing our pet barked.
After a little retrospection, I informed the team how we were talking less about the positives and more about the negatives of the pet which forced the customer not to buy our pet. I took the lead in demoing to the customers henceforth. The approach I used was simple: Focus and highlight all the features and advantages in the pet. Don’t even mention the negatives. It went well. Some customers were smart. They asked ‘Does your pet bite?’, ‘Does it bark?’, ‘How aggressive is your pet?’ In addition to answering these direct questions, I also highlighted how our pet was harmless and could be child friendly for kids to play with. Impressive. We sold about 5 pets after 2nd iteration.
Lesson: We thought we were smart. Some customers ended up asking key questions like ‘How many legs does your pet have?’ When our answer did not meet their requirement, they simply walked away yelling ‘Oh! This is not what I am looking for’. For some customers, their requirements did not match ours.
To spice up the whole workshop, we were not told about what the customers expected from the pet. All we were told was we had to answer customer’s questions and based on that we had to build/re-build our pet. How insane? We don’t get to talk to our customer about a product they want? Sounded foolish. But what about customers whom you don’t even get to know about in your lifetime, but still may be potential users of the product.
Lesson: At the end of third iteration, we met up with many customers who were interested to buy our pet, but were falling short of money. This is when we realized that we shouldn’t have driven away customers after first iteration when they had the most money. We did not adapt quickly. We took time not just to build the product, but to market it as well. It cost us a few more sales deals.
Our team stood 4th in the total number of sales made, but that’s not the point. The point is where we faltered. The whole idea of the workshop was to identify those failure points and learn from them. And we did.
What does a Customer want anyway?
Customers are of prime importance. Engineers have talent, Customers have money. Engineers build products; Customers are ready to pay for them. Some customers have rich cash flow, some are conservative in spending, but they don’t mind investing once in a while on products which will profit them and some customers are desperate to buy your product, but they just don’t have the money. There are all types of customers and it's important to study them and the markets they cater to in order to build products that are suitable for them.
Customers doesn’t need an infinite number of features in the product to make them look super cool in a room of a hundred people at a conference. They’ll be happy with a single feature that can touch lakhs of people’s lives and make a difference and of course, fetches a reasonable profit in turn for their hard work. Is that so hard?
Regards,
Parimala Shankaraiah
Regards,
Parimala Shankaraiah
Subscribe to:
Posts (Atom)