I took part in James Bach's Testing Challenge at Weekend Testing recently. Pradeep Soundararajan reviewed my report and spoke about the gaps in my testing effort as well as the report. As always, I revisited my report to figure out areas that need improvement(read dismal performance report). One of the key problems with my report was that - I did not question the mission that was given to me. I did skype James before the testing session with a few questions that were either too broad or too open ended. James did help me by rephrasing a few questions. However, I did not use this opportunity to its fullest potential.
A little while later, Jonathan Bach & James Bach tweeted on Traps in testing
JBTestPilot: The #1 trap that testers fall into when I ask them to test something: They start testing'.
CuriousTester: I am trapped! din't see testing without questioning as a trap! Traps work that way. This is more often than I would wish to get trapped!
JBTestPilot: No worries... there are many ways out of traps. The trick is, knowing you're IN one.
JamesMarcusBach: Good testing is a questioning process. All tests are questions. But some things are better asked of humans.
Here I am inflicted with lack of good questioning skills. I did not worry much about it until it started affecting my ability to test better. Now you ask ‘Do you mean you never questioned till date?’. Of course, I did. They did not seem to add value to what I already do. They did not appear to bring in new ideas. In short, I am not skilled at questioning yet.
[ Tester ] - [ Questioning Skills ] = [ Unskilled Testing ]
I am wondering why questioning is not in my DNA. Is it really in my DNA and I am unaware? I grew up as an obedient child. I don’t remember saying ‘No’ to my parents, no matter what I was told to do - a very big claim indeed (grins). When I say as a child, I mean the period from which I remember the events of my life to a considerable extent. Being obedient, not breaking the rules, not being demanding and many more blocked my ability to question since childhood.
In general, questions are not encouraged in India (I think so). They are seen as a symbol of revolt. Questions from elders are reprimanded and questions from children are snubbed. Children are taught to be obedient without questioning. If they break the rules, they will be punished. Yes, child beating is quite normal and not violent. Actually, its not even violent as it sounds to you as you read this. A 911 type helpline may flop here. Growing amidst a culture where questioning is a taboo could be one reason why I am not good at it.
Every question adds a new dimension to better understanding through answers. Yet, we don’t question. The sad part is most of us don’t question because we don’t know that we can question. We have the liberty to question, but we don’t.
Question what? Question your product, question your testing mission, question your test strategies question your deliverables. Question anything that you think will make a difference to what you do. Some people are good at questioning and a few others are not so good. The good news is that this skill can be inculcated with constant practice by one and all. It can be honed over a period of time.
Practice questioning skills with your programmers/team members. Play games that help improve your questioning skills. Play Twenty questions game. Playing Dumb Charades might be of great help too. Ask the right questions. To ask the right questions, you will need to ask many wrong questions and learn from them. Open ended questions may be too broad. Close ended questions may be too vague. Probing questions focus on specific areas (can be open or close ended) and are very useful. A good mix of above three types of questions will improve questioning skills. Note taking if coupled with Questioning is a powerful tool for testing.
Wishing You a Prosperous & Happy New Year 2010!
Addendum on 30th April 2010
One way to practice questioning skills is using Q-Patterns by Vipul Kocher.
P.S: The title of this post is inspired by Rahul Mirakhur who made a mention of 'Questioning in your DNA' during one of our informal talks.
Happy Questioning,
Parimala Shankaraiah
30 December, 2009
19 December, 2009
Easter Eggs in Software
Easter eggs are specially decorated eggs given to celebrate the Easter holiday. These eggs are often hidden, allegedly by the Easter Bunny, for children to find on Easter morning as per Wikipedia.
Easter Eggs in software refer to hidden features in the program that are intentionally injected into the product by the programmer. These may include hidden commands, animations, funny jokes, games, credits for programmers who built the product etc. The programmers of easter eggs do it for their own personal reasons or for entertainment and are not meant to be harmful. If the programmer intentionally added malicious code to behave as a virus/spam/trojan, it may not be an easter egg. However, if the programmer added some code to behave as an easter egg that accidentally turned out to be harmful, it is still an easter egg.
How to unearth easter eggs?
Programmers Names
Search for the names of programmers who built the product. These are a very common form of easter eggs. These references mean that a few programmers wanted to take credit for the work done by them and inserted their names into the code without the knowledge of the management. More often, programmers resort to this method due to lack of appreciation for their work by the management.
Guidewords
Looking for swear words or expletives helps discover easter eggs. Dissatisfied/disgruntled/quitting employees may add these words in the code to take revenge on people they disliked or hated. I once saw a line reading ‘Thomas is a fool’ as part of a code snippet in one of the products that I tested a few years ago (No offense to Thomas here!).
Personalized images/animations
A few programmers embed their favorite images or animation in the code as they find themselves emotionally bound to the images/animations as much as the code that was written. For eg, a programmer might include icon images to include one of his favorite picture or a scenery.
Magical Keystrokes
A few easter eggs can be found by pressing a magical key combination/shortcut keys on the keyboard. These may be discovered accidentally or based on the knowledge of similar easter eggs in previous versions of the product.
Clusters of Easter Eggs
If an easter egg of any one type as described above was found, there is a high possibility of many more being found in close proximity to the already found egg. Pattern matching with previously found easter eggs helps find a few more surrounding them.
Read the code
Reading the code completely is one of the ways of looking for any unusual code that does not fall in line with the actual code in the product. Any code that is not in sync with the rest of the code may be easter eggs.
Ask the programmers
One of the fastest ways of finding most easter eggs is to ask the programmers themselves about what kind of easter eggs have been injected into the code. Not all programmers would reveal about all the easter eggs. A few of them might reveal theirs, but may not know about other programmers easter eggs. Some of them might be tightlipped to talk about easter eggs for fear of being fired or disciplined by the management.
Easter eggs by definition cause no harm by hiding inside the product. However, If the customers find easter eggs in the production environment, it could raise doubts about the kind of testing done on the product by the testers. The companies become liable for any offensive or illegal content that is shipped in the product and may face legal consequences. It is therefore very important to test for easter eggs as part of the normal testing process to avoid embarrassing situations after the product is shipped to the customer.
Start looking for Easter Eggs – in your software,
Happy Testing,
Parimala Shankaraiah
Easter Eggs in software refer to hidden features in the program that are intentionally injected into the product by the programmer. These may include hidden commands, animations, funny jokes, games, credits for programmers who built the product etc. The programmers of easter eggs do it for their own personal reasons or for entertainment and are not meant to be harmful. If the programmer intentionally added malicious code to behave as a virus/spam/trojan, it may not be an easter egg. However, if the programmer added some code to behave as an easter egg that accidentally turned out to be harmful, it is still an easter egg.
How to unearth easter eggs?
Programmers Names
Search for the names of programmers who built the product. These are a very common form of easter eggs. These references mean that a few programmers wanted to take credit for the work done by them and inserted their names into the code without the knowledge of the management. More often, programmers resort to this method due to lack of appreciation for their work by the management.
Guidewords
Looking for swear words or expletives helps discover easter eggs. Dissatisfied/disgruntled/quitting employees may add these words in the code to take revenge on people they disliked or hated. I once saw a line reading ‘Thomas is a fool’ as part of a code snippet in one of the products that I tested a few years ago (No offense to Thomas here!).
Personalized images/animations
A few programmers embed their favorite images or animation in the code as they find themselves emotionally bound to the images/animations as much as the code that was written. For eg, a programmer might include icon images to include one of his favorite picture or a scenery.
Magical Keystrokes
A few easter eggs can be found by pressing a magical key combination/shortcut keys on the keyboard. These may be discovered accidentally or based on the knowledge of similar easter eggs in previous versions of the product.
Clusters of Easter Eggs
If an easter egg of any one type as described above was found, there is a high possibility of many more being found in close proximity to the already found egg. Pattern matching with previously found easter eggs helps find a few more surrounding them.
Read the code
Reading the code completely is one of the ways of looking for any unusual code that does not fall in line with the actual code in the product. Any code that is not in sync with the rest of the code may be easter eggs.
Ask the programmers
One of the fastest ways of finding most easter eggs is to ask the programmers themselves about what kind of easter eggs have been injected into the code. Not all programmers would reveal about all the easter eggs. A few of them might reveal theirs, but may not know about other programmers easter eggs. Some of them might be tightlipped to talk about easter eggs for fear of being fired or disciplined by the management.
Easter eggs by definition cause no harm by hiding inside the product. However, If the customers find easter eggs in the production environment, it could raise doubts about the kind of testing done on the product by the testers. The companies become liable for any offensive or illegal content that is shipped in the product and may face legal consequences. It is therefore very important to test for easter eggs as part of the normal testing process to avoid embarrassing situations after the product is shipped to the customer.
Start looking for Easter Eggs – in your software,
Happy Testing,
Parimala Shankaraiah
07 December, 2009
Book Review: I am a Bug
I am a Bug is a book written by Robert Sabourin. This is a short book illustrated by his daughter Catherine which explains the software development process using cartoons and live examples from real time projects.
Little Nuggets from the book
1. When a bug is found, we do not ask if the bug is good or bad. We decide if it is important. We guess how much damage it could cause.
2. When we rush too much making new software, our offices and programs can get pretty messed up. We know there will be a lot of bugs!
3. Some bugs have many brothers and sisters (Bug clusters).
4. Bugs much worse than buffer overflows would just walk right by! (Inattentinal Blindness).
5. Some bugs are good friends! (Deferred bugs). A friendly bug is a bug that does not stop users from enjoying and benefiting from the software.
6. If we use the program in a different way, we will avoid some bugs completely (Change in perception).
7. Bugs can be fixed, avoided, or ignored. If a bug won't be fixed, users can be encouraged to use the software in a way that avoids the bug (Workarounds).
8. Sometimes we find bugs a long time after people start to use the program (Sleeping bugs).
9. Salespeople from tool vendors sometimes misrepresent the costs and benefits of tools and their ease of use. For example, they may claim that a test automation tool that can capture actions and play them back will "solve all your problems". They'll say, "Take this tool, and you can write a program that will find all your bugs."
Testers vs. Frogs
1. Frogs like bugs! People, at work, think positively about bugs. We like to find bugs!
2. Frogs eat bugs!
a. When we find bugs we really, really, really like to fix them!!
b. Whether or not a given bug should be fixed depends upon the business context.
c. Just as the frog's stomach can only contain a finite amount of bugs, a development team doesn't usually have the time, effort, and money to fix all of the bugs found in a program.
d. Bug priority and severity need to be established based on a cost/benefit analysis to figure out which bugs are worth fixing. The costs of not fixing a bug can be the damage to a user's computer, the number of support calls generated by a bug, damage to a company's reputation, and so on.
e. Just a few of the potential drawbacks of fixing a bug are the danger of injecting or uncovering more bugs, the developer time and effort spent, and the chance of decreasing adherence to quality goals for availability, performance, etc.
3. Bugs try to avoid frogs - Naturally, when we try to fix a bug, it tries to avoid us!
4. Getting lots of frogs (Crowdsourcing) can help get rid of lots of bugs. Once we decided to get lots of people to help us find and correct all the bugs. but too many frogs in the same place are a big problem too!!! But that didn't help because we got mixed up and confused.
5. How did we catch it? Is it important? Can it hurt us?
6. Quick patches don't solve problems. Fix it right the first time!
7. Think about how the program will be used. The exact same program can be used by many different people in many different ways, and a bug that is a dealbreaker for one user community may be inconsequential to another.
a. This is why I want everyone on a project or specific task within a project to have a common notion about what the answer to the fundamental question is. This way everyone knows what her role in the completion of the task is.
As testers, we are emotionally bound to every single bug that we report to the development. We expect that every bug be fixed in the immediate next build. In reality, it is very hard to fix all the bugs. Any business context demands that a product be released in the quickest time possible, within budget and on schedule. As a result, it is important to realize which bug is important and which is not. Testers are not decision makers, they are information providers. Let the decision makers decide which bug to fix. This book highlights the importance of bugs and the prioritization of fixes for the important bugs. I have documented important points from the book in the ‘Pearls of Wisdom’ section. Do take a look if that helps. Rob Sabourin has been generous to provide a FREE online version of this book on his website Amibug.com.
Happy Reading,
Parimala Shankaraiah
Little Nuggets from the book
1. When a bug is found, we do not ask if the bug is good or bad. We decide if it is important. We guess how much damage it could cause.
2. When we rush too much making new software, our offices and programs can get pretty messed up. We know there will be a lot of bugs!
3. Some bugs have many brothers and sisters (Bug clusters).
4. Bugs much worse than buffer overflows would just walk right by! (Inattentinal Blindness).
5. Some bugs are good friends! (Deferred bugs). A friendly bug is a bug that does not stop users from enjoying and benefiting from the software.
6. If we use the program in a different way, we will avoid some bugs completely (Change in perception).
7. Bugs can be fixed, avoided, or ignored. If a bug won't be fixed, users can be encouraged to use the software in a way that avoids the bug (Workarounds).
8. Sometimes we find bugs a long time after people start to use the program (Sleeping bugs).
9. Salespeople from tool vendors sometimes misrepresent the costs and benefits of tools and their ease of use. For example, they may claim that a test automation tool that can capture actions and play them back will "solve all your problems". They'll say, "Take this tool, and you can write a program that will find all your bugs."
Testers vs. Frogs
1. Frogs like bugs! People, at work, think positively about bugs. We like to find bugs!
2. Frogs eat bugs!
a. When we find bugs we really, really, really like to fix them!!
b. Whether or not a given bug should be fixed depends upon the business context.
c. Just as the frog's stomach can only contain a finite amount of bugs, a development team doesn't usually have the time, effort, and money to fix all of the bugs found in a program.
d. Bug priority and severity need to be established based on a cost/benefit analysis to figure out which bugs are worth fixing. The costs of not fixing a bug can be the damage to a user's computer, the number of support calls generated by a bug, damage to a company's reputation, and so on.
e. Just a few of the potential drawbacks of fixing a bug are the danger of injecting or uncovering more bugs, the developer time and effort spent, and the chance of decreasing adherence to quality goals for availability, performance, etc.
3. Bugs try to avoid frogs - Naturally, when we try to fix a bug, it tries to avoid us!
4. Getting lots of frogs (Crowdsourcing) can help get rid of lots of bugs. Once we decided to get lots of people to help us find and correct all the bugs. but too many frogs in the same place are a big problem too!!! But that didn't help because we got mixed up and confused.
5. How did we catch it? Is it important? Can it hurt us?
6. Quick patches don't solve problems. Fix it right the first time!
7. Think about how the program will be used. The exact same program can be used by many different people in many different ways, and a bug that is a dealbreaker for one user community may be inconsequential to another.
a. This is why I want everyone on a project or specific task within a project to have a common notion about what the answer to the fundamental question is. This way everyone knows what her role in the completion of the task is.
As testers, we are emotionally bound to every single bug that we report to the development. We expect that every bug be fixed in the immediate next build. In reality, it is very hard to fix all the bugs. Any business context demands that a product be released in the quickest time possible, within budget and on schedule. As a result, it is important to realize which bug is important and which is not. Testers are not decision makers, they are information providers. Let the decision makers decide which bug to fix. This book highlights the importance of bugs and the prioritization of fixes for the important bugs. I have documented important points from the book in the ‘Pearls of Wisdom’ section. Do take a look if that helps. Rob Sabourin has been generous to provide a FREE online version of this book on his website Amibug.com.
Happy Reading,
Parimala Shankaraiah
03 December, 2009
Claims Testing
What is a Claim?
A claim can be anything that someone in the company says about the product. It could be requirements, online help, sales/marketing material, Advertisements on different media or even word of mouth blabbering along the hallway by a project manager to the stakeholder.
Claims can be classified into two categories. ‘Explicit’ claims are the claims that are explicitly stated. ‘Implicit’ claims are not stated explicitly, but imply something. It is important to analyze all the individual claims and clarify vague claims. Every claim about the product needs to be verified. In reality, most of the time goes in testing the product that there is hardly any time left to test the claims. I am forced to think that most projects do not see the light of the day when it comes to claims testing, usability testing and performance testing. If the context does not demand these, so be it. Is the customer Ok with this is the question?
Application to Test
Tux Paint is a free, award-winning drawing program for children ages 3 to 12 (for example, preschool and K-6). It combines an easy-to-use interface, fun sound effects, and an encouraging cartoon mascot who guides children as they use the program. Kids are presented with a blank canvas and a variety of drawing tools to help them be creative.
Mission
Test the claims made in Tux Paint application
Sue-ability
In my experience so far, the Sales/Marketing professionals handling product demos make a few fake promises to the customers about some features and then rush back to the engineering team to create that feature from scratch. So these folks claimed that the feature that the customer is interested in is indeed present in the product while it is not! Wow. What a Claim! I have seen this happen in a few projects where the sales guys come running to the Product Management team t(y)elling ‘I have told the customer that this is supported. You develop it now’ kind of a statement. What if the customer sues the organization on this basis?
There are hardly any projects where claims made by the product/people are tested. Moreover, it is hard to track what each and every person in the project is claiming about the product. Hence, there is a strong need to educate the entire project team to the pitfalls of false claims within and outside the organization. If the organization does not educate you, you educate yourself! Claim the Truth.
Click HERE for the SBMT Report on Tux Paint.
Happy Claiming,
Parimala Shankaraiah
Addendum on 18th November 2011
Mindmap for Claims Testing is found HERE
A claim can be anything that someone in the company says about the product. It could be requirements, online help, sales/marketing material, Advertisements on different media or even word of mouth blabbering along the hallway by a project manager to the stakeholder.
Claims can be classified into two categories. ‘Explicit’ claims are the claims that are explicitly stated. ‘Implicit’ claims are not stated explicitly, but imply something. It is important to analyze all the individual claims and clarify vague claims. Every claim about the product needs to be verified. In reality, most of the time goes in testing the product that there is hardly any time left to test the claims. I am forced to think that most projects do not see the light of the day when it comes to claims testing, usability testing and performance testing. If the context does not demand these, so be it. Is the customer Ok with this is the question?
Application to Test
Tux Paint is a free, award-winning drawing program for children ages 3 to 12 (for example, preschool and K-6). It combines an easy-to-use interface, fun sound effects, and an encouraging cartoon mascot who guides children as they use the program. Kids are presented with a blank canvas and a variety of drawing tools to help them be creative.
Mission
Test the claims made in Tux Paint application
Sue-ability
In my experience so far, the Sales/Marketing professionals handling product demos make a few fake promises to the customers about some features and then rush back to the engineering team to create that feature from scratch. So these folks claimed that the feature that the customer is interested in is indeed present in the product while it is not! Wow. What a Claim! I have seen this happen in a few projects where the sales guys come running to the Product Management team t(y)elling ‘I have told the customer that this is supported. You develop it now’ kind of a statement. What if the customer sues the organization on this basis?
There are hardly any projects where claims made by the product/people are tested. Moreover, it is hard to track what each and every person in the project is claiming about the product. Hence, there is a strong need to educate the entire project team to the pitfalls of false claims within and outside the organization. If the organization does not educate you, you educate yourself! Claim the Truth.
Click HERE for the SBMT Report on Tux Paint.
Happy Claiming,
Parimala Shankaraiah
Addendum on 18th November 2011
Mindmap for Claims Testing is found HERE
Subscribe to:
Posts (Atom)