Once upon a time, there lived a beautiful Princess. The Supreme Commander of the kingdom lusted on the Princess, while she fell in love with one of the greatest warriors of her kingdom who trained soldiers for war. She admired him for his bravery and courage. This resulted in a battle between the commander and the warrior on a tall mountain which is 10 times bigger than Mount Everest. After a futile attempt to kill the warrior, the commander took revenge by piercing a dagger in the Princess’s heart. She lay motionless at the edge of that mountain while the warrior killed the commander. The warrior runs towards the Princess to say that he loves her…. something that he never confessed to her until that time though he loved her. And the Princess dies and slips from the edge of the mountain. The warrior sees the Princess falling down, runs back a few steps and jumps from the mountain hoping to reach out to her. He somehow manages to go close to the falling Princess, but not close enough to catch hold of her.
I was fully involved watching this scene in a multiplex theatre when my niece shook me up with a question. ‘Did you know that the warrior can never reach out to the Princess if he jumps with an initial velocity 0?' I was like 'What? Why do you think so?' She said ‘The dead Princess slipped off with an initial velocity 0 (inertia of rest). However, the warrior went running and jumped off the mountain with a certain velocity say ‘X’ (inertia of motion)'.
She continued ‘If the warrior jumped with initial velocity 0, then the distance between him and the dead Princess would be same until the Princess reaches the ground. If the warrior jumped with an initial velocity ‘X’, then the distance changes and he will be able to get closer and even reach the Princess’. It was a truly ‘WOW’ moment. I am pretty bad at physics thanks to my teachers who only forced and tortured me and my class mates to learn formulae by rote and vomit it in exams. It is also my mistake that I never questioned what I was taught (shame on me!).
Imagination, Analogy, Practical thinking is infinite. This experience left me thinking for quite some time. As a tester, I set a mission for myself to ‘Test whether the warrior catches hold of the Princess’s hand and report my observations’. Just like any movie buff, I got busy watching the warrior reach out to the Princess. What if I did not know anything about initial velocity? What if I did not know under what circumstances the Princess fell off the mountain? What if the Princess jumped at her own will? What if the warrior suffered a shock, died and then slipped off the mountain. I would end up watching them falling until they reach the ground without focusing on my mission.
It is important to understand the context be it testing or problem solving or even a crime scene investigation.
1. What problem is the application intended to solve?
2. How will the application be used in a real world?
3. Who is the target audience for this application?
4. What is the environment/platform in which the application will be used?
5. Where does the application stand with regard to competitor products catering to a similar target audience?
6. What is the expected scalability for the application in the near future?
7. What kind of performance is expected of the application?
Without knowing answers to many more questions like these, testers can only do superficial testing leaving out critical defects in the application intact. Imagine the plight of the customer who buys this application?
We talk about getting laid off in our organizations. We do not talk about why the organization laid off or even shut down the entire office. We ask about why the organization did not retain people with the profits it made the last to last before year(correct me if this phrase is wrong). We do not ask why the customers did not buy the application. We need to provide the application the way the customer wants it, not the way we want it to be. If we do provide the way we want it to be, it will be with us. It will never go to the customer and by god's grace(sales/marketing) even if it does, it will come back to us. Do we want the application to come back or the appreciation?
Get your Context Right,
Parimala Shankaraiah
http://curioustester.blogspot.com
08 October, 2009
06 October, 2009
The Power of Mnemonics
I always wondered how a few testers catch killer bugs so quickly. How is their intuition so sharp compared to others? Practice? In addition to practice, these testers find easy ways to remember important stuff that matters to them in different contexts. Mnemonics can be very useful to remember heuristics and oracles. I compiled the below list for me and you to learn and practice. Of course, with due credit to the creators of these heuristics and oracles.
Michael D Kelly’s Touring Heuristics – FCC CUTS VIDS
Feature tour, Complexity tour, Claims tour, Configuration tour, User tour, Testability tour, Scenario tour, Variability tour, Interoperability tour, Data tour, Structure tour
Michael D Kelly’s Test Reporting Heuristics – MCOASTER
Mission, Coverage, Obstacles, Audience, Status, Techniques, Environment, Risk
Ben Simo’s Error handling Heuristics – FAILURE
Functional, Appropriate, Impact, Log, UI, Recovery, Emotions
Adam Goucher’s ordering of Testing Tasks heuristics – SLIME
Security, Languages, requIrements, Measurement, Existing
Scott Barber’s Performance Testing heuristics – FIBLOTS
Frequent, Intensive, Business Critical, Legal, Obvious, Technically risky, Stakeholder Mandated
James Bach’s Test Strategy Heuristics – SFDPOT (San Francisco Depot)
Structure, Function, Data, Platform, Operations, Time
James Bach’s Project Environment Heuristics – CIDTESTD (Kid Tested)
Customers, Information, Developer relations, Team, Equipment & tools, Schedule, Test Items, Deliverables
James Bach’s Quality Criteria heuristics – CRUSSPIC STMPL
A. Operational Criteria - CRUSSPIC
Capability, Reliability, Usability, Security, Scalability, Performance, Installability, Compatibility
B. Development Criteria - STMPL
Supportability, Testability, Maintainability, Portability, Localizability
James Bach’s Test Techniques Heuristics – DUFFSSCRA(FDSFSCURA)
Domain, User, Function, Flow, Stress, Scenario, Claims, Risk, Automatic
James Bach’s Test Oracles – HICCUPPSF
History, Image, Comparable product, Claims, User Expectation, Product, Purpose, Standards and Statutes, Familiar Problems
James Bach’s Learning Heuristics – SACKED SCOWS
Scouting Obsessively, Authentic Problems, Cognitive Savvy, Knowledge attracts Knowledge, Experimentation, Disposable Time, Stories(Contrasting Ideas, Skepticism, Critical thinking, Lateral thinking), Other Minds, Words and Pictures, Systems Thinking
Most of us probably read and learn about a lot about heuristics and oracles. While we continue to use them, we blindly stick to them forever without discovering our own heuristics or oracles that work best for us. Please note I have a lot of respect for the existing heuristics and oracles and the people who came up with them. All I want to convey here is for us to think and experiment more to see if we can come up with our own heuristics and oracles which makes better sense to the testing that we do.
Points to Note
1. I have intentionally not detailed the above heuristics because I want you to go read those wonderful articles/blog posts where these heuristics are as is (if you are not already familiar). Summon google for help.
2. James Bach's book 'The Secrets of a Buccaneer Scholar' has a lot more learning heuristics - Will update this list once I buy the book and read it a second time.
3. If you know of any heuristics/oracles or you have come up with on your own, please do let me know.
4. There is a bug in here. No clues. Lemme see if you can find it.
Addendum on 16th Oct 09
In Michael Bolton's words: "To HICCUPP, you can now add S, for Standards and Statutes--relevant laws or industry standards. F, for Familiar Problems. All the prior oracle heuristics are consistency heuristics; we suspect that everything's okay if the product is consistent with them, and it's not okay if we see an inconsistency. The Familiar Problems oracle is the opposite; we want the product to be inconsistent with patterns of programs that we've seen before, and here if we see a consistency we suspect that there's a problem."
Happy Discovering,
Parimala Shankaraiah
http://curioustester.blogspot.com
Michael D Kelly’s Touring Heuristics – FCC CUTS VIDS
Feature tour, Complexity tour, Claims tour, Configuration tour, User tour, Testability tour, Scenario tour, Variability tour, Interoperability tour, Data tour, Structure tour
Michael D Kelly’s Test Reporting Heuristics – MCOASTER
Mission, Coverage, Obstacles, Audience, Status, Techniques, Environment, Risk
Ben Simo’s Error handling Heuristics – FAILURE
Functional, Appropriate, Impact, Log, UI, Recovery, Emotions
Adam Goucher’s ordering of Testing Tasks heuristics – SLIME
Security, Languages, requIrements, Measurement, Existing
Scott Barber’s Performance Testing heuristics – FIBLOTS
Frequent, Intensive, Business Critical, Legal, Obvious, Technically risky, Stakeholder Mandated
James Bach’s Test Strategy Heuristics – SFDPOT (San Francisco Depot)
Structure, Function, Data, Platform, Operations, Time
James Bach’s Project Environment Heuristics – CIDTESTD (Kid Tested)
Customers, Information, Developer relations, Team, Equipment & tools, Schedule, Test Items, Deliverables
James Bach’s Quality Criteria heuristics – CRUSSPIC STMPL
A. Operational Criteria - CRUSSPIC
Capability, Reliability, Usability, Security, Scalability, Performance, Installability, Compatibility
B. Development Criteria - STMPL
Supportability, Testability, Maintainability, Portability, Localizability
James Bach’s Test Techniques Heuristics – DUFFSSCRA(FDSFSCURA)
Domain, User, Function, Flow, Stress, Scenario, Claims, Risk, Automatic
James Bach’s Test Oracles – HICCUPPSF
History, Image, Comparable product, Claims, User Expectation, Product, Purpose, Standards and Statutes, Familiar Problems
James Bach’s Learning Heuristics – SACKED SCOWS
Scouting Obsessively, Authentic Problems, Cognitive Savvy, Knowledge attracts Knowledge, Experimentation, Disposable Time, Stories(Contrasting Ideas, Skepticism, Critical thinking, Lateral thinking), Other Minds, Words and Pictures, Systems Thinking
Most of us probably read and learn about a lot about heuristics and oracles. While we continue to use them, we blindly stick to them forever without discovering our own heuristics or oracles that work best for us. Please note I have a lot of respect for the existing heuristics and oracles and the people who came up with them. All I want to convey here is for us to think and experiment more to see if we can come up with our own heuristics and oracles which makes better sense to the testing that we do.
Points to Note
1. I have intentionally not detailed the above heuristics because I want you to go read those wonderful articles/blog posts where these heuristics are as is (if you are not already familiar). Summon google for help.
2. James Bach's book 'The Secrets of a Buccaneer Scholar' has a lot more learning heuristics - Will update this list once I buy the book and read it a second time.
3. If you know of any heuristics/oracles or you have come up with on your own, please do let me know.
4. There is a bug in here. No clues. Lemme see if you can find it.
Addendum on 16th Oct 09
In Michael Bolton's words: "To HICCUPP, you can now add S, for Standards and Statutes--relevant laws or industry standards. F, for Familiar Problems. All the prior oracle heuristics are consistency heuristics; we suspect that everything's okay if the product is consistent with them, and it's not okay if we see an inconsistency. The Familiar Problems oracle is the opposite; we want the product to be inconsistent with patterns of programs that we've seen before, and here if we see a consistency we suspect that there's a problem."
Happy Discovering,
Parimala Shankaraiah
http://curioustester.blogspot.com
04 October, 2009
Testability – What art Thou?
Software testability is the degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context. Take a moment to read about Testability Heuristics from James Bach.
Background
I have not tested for Testability. I did not know what is testability until I heard it as a quality criteria in Weekend Testing 8. When Pradeep Soundararajan pinged me for a testing session on 3rd Oct 2009, I jumped out of my seat. Sharath Byregowda was keen to join us too. The Three Musketeers. I was asked to come up with an application to test and the mission.
Application to Test
Microsoft Narrator is a text-to-speech utility for users who are blind or have impaired vision. Narrator reads what is displayed on your screen: the contents of the active window, menu options, or the text you have typed. I have been amused by its functionality for quite some time and this was the opportunity.
Mission
To test the Testability of the Narrator application.
Experience Report
The three of us set out to test narrator. I started off with Google looking for information on testability. Read up a little on Wikipedia and another article on stickyminds (I found James Bach's article while composing this post). Identified a set of criteria to look out for while testing for testability of the Narrator. I had no clue if I was right or wrong. Nevertheless, it was exciting because this was my first foray into testability. It was very thrilling though I knew I had the potential to do a lot more better. But then, I always discount myself the first time I don’t do anything correctly. This way, I don’t get stressed to run the rat race nor do I have the pressure to learn something just because I am left behind compared to other good testers. Testing was stopped after 1 hour and we had a small discussion. Sharath had concerns with using the product as a whole. I had concerns about not knowing testability enough. Pradeep simply shared his report and headed for lunch without speaking a word. He probably wanted us to see what was in there! Please find my report HERE.
Challenges
1. New Product as usual
2. New mission – Testability
3. I am neither blind nor do I have an impaired vision
4. In any testing activity, whining within or outside of the testing window is an utter waste of time. It is better to find ways to know how to do things and do them better
5. Once out of the testing window, it is better to go back and learn more on unknown areas – in this case ‘learning more on testability’ because this is not the end of it
6. One of the biggest gift we humans can give to ourselves is to identify what we do not know and find ways to know what we do not know. The challenge here is to know what you do not know
Do you know what you do not know?
Addendum(5th Oct 09)
Michael Bolton's take on Testability.
Happy Knowing Thyself,
Parimala Shankaraiah
http://curioustester.blogspot.com
Background
I have not tested for Testability. I did not know what is testability until I heard it as a quality criteria in Weekend Testing 8. When Pradeep Soundararajan pinged me for a testing session on 3rd Oct 2009, I jumped out of my seat. Sharath Byregowda was keen to join us too. The Three Musketeers. I was asked to come up with an application to test and the mission.
Application to Test
Microsoft Narrator is a text-to-speech utility for users who are blind or have impaired vision. Narrator reads what is displayed on your screen: the contents of the active window, menu options, or the text you have typed. I have been amused by its functionality for quite some time and this was the opportunity.
Mission
To test the Testability of the Narrator application.
Experience Report
The three of us set out to test narrator. I started off with Google looking for information on testability. Read up a little on Wikipedia and another article on stickyminds (I found James Bach's article while composing this post). Identified a set of criteria to look out for while testing for testability of the Narrator. I had no clue if I was right or wrong. Nevertheless, it was exciting because this was my first foray into testability. It was very thrilling though I knew I had the potential to do a lot more better. But then, I always discount myself the first time I don’t do anything correctly. This way, I don’t get stressed to run the rat race nor do I have the pressure to learn something just because I am left behind compared to other good testers. Testing was stopped after 1 hour and we had a small discussion. Sharath had concerns with using the product as a whole. I had concerns about not knowing testability enough. Pradeep simply shared his report and headed for lunch without speaking a word. He probably wanted us to see what was in there! Please find my report HERE.
Challenges
1. New Product as usual
2. New mission – Testability
3. I am neither blind nor do I have an impaired vision
4. In any testing activity, whining within or outside of the testing window is an utter waste of time. It is better to find ways to know how to do things and do them better
5. Once out of the testing window, it is better to go back and learn more on unknown areas – in this case ‘learning more on testability’ because this is not the end of it
6. One of the biggest gift we humans can give to ourselves is to identify what we do not know and find ways to know what we do not know. The challenge here is to know what you do not know
Do you know what you do not know?
Addendum(5th Oct 09)
Michael Bolton's take on Testability.
Happy Knowing Thyself,
Parimala Shankaraiah
http://curioustester.blogspot.com
Testing out of Comfort Zone
The last time I mentioned about Getting Out of Comfort Zone, I anticipated great learning experience, but I wasn’t ready for surprises! I wanted to participate in the Weekend Testers (Chennai Chapter) and test a CSS Menu Builder application.
Charter
Cross Browser compatibility testing of IzzyMenu using IE 8.0 and FF 3.0 browsers.
Experience Report
IzzyMenu did not have a direct hyperlink which could be tested using a cross browser compatibility testing tool for IE 8.0 and FF 3.0. I as a user had to navigate to the menu page explicitly. I tested for alignment, layout, font size, color, positioning of tabs in the layout, hyperlinks, look and feel of the entire application, print, print preview option and a lot more. Tested the same set of test ideas in FF 3.0. Ok, finding usability and functional bugs while I look for compatibility bugs– Murphy’s Law? Found about 3 compatibility bugs, but that is not important for me right now. What is important is what is new to me in this testing exercise and what I learned while testing IzzyMenu.
Learnings
1. Though I had performed compatibility testing previously, while testing IzzyMenu, I had a mental block testing it. I had not performed compatibility testing on websites for a start. I have to take care of this in future.
2. When learning to test with a charter that I have not done before, it invites some prior learning on how to do. When I jumped at IzzyMenu without deciding what to test, there was a block. I did not see anything reasonable to test. My approach was wrong here and I had to change it.
3. Do not confine yourself to 90 minutes Sessions while you are learning to test out of your comfort zone. It more or less extends beyond 90 minutes timeframe. Ideally, its better not to have any time limit while testing this way.
4. Note down all the bottlenecks or choking points that you face while testing and sort it out yourself. This way, you will slowly realize that what you saw as a bottleneck is not a bottleneck at all. And that will be your Wow moment.
5. You may not find bugs all the time. Absence of bugs or lack of skills to find bugs(which tricks you to think that bugs are absent) does not mean you are a bad tester. You probably did not look at the right places and this can be improved with continuous practice.
6. From time to time, we should get together with people from different perspectives and watch them test. Trust Me, the first time you do this, you may think you are a horrible tester. You may be one too. The next time when you watch them test, it will be one of the coolest ways of learning and testing and you would do more of it more often.
Happy Testing,
Parimala Shankaraiah
http://curioustester.blogspot.com
Charter
Cross Browser compatibility testing of IzzyMenu using IE 8.0 and FF 3.0 browsers.
Experience Report
IzzyMenu did not have a direct hyperlink which could be tested using a cross browser compatibility testing tool for IE 8.0 and FF 3.0. I as a user had to navigate to the menu page explicitly. I tested for alignment, layout, font size, color, positioning of tabs in the layout, hyperlinks, look and feel of the entire application, print, print preview option and a lot more. Tested the same set of test ideas in FF 3.0. Ok, finding usability and functional bugs while I look for compatibility bugs– Murphy’s Law? Found about 3 compatibility bugs, but that is not important for me right now. What is important is what is new to me in this testing exercise and what I learned while testing IzzyMenu.
Learnings
1. Though I had performed compatibility testing previously, while testing IzzyMenu, I had a mental block testing it. I had not performed compatibility testing on websites for a start. I have to take care of this in future.
2. When learning to test with a charter that I have not done before, it invites some prior learning on how to do. When I jumped at IzzyMenu without deciding what to test, there was a block. I did not see anything reasonable to test. My approach was wrong here and I had to change it.
3. Do not confine yourself to 90 minutes Sessions while you are learning to test out of your comfort zone. It more or less extends beyond 90 minutes timeframe. Ideally, its better not to have any time limit while testing this way.
4. Note down all the bottlenecks or choking points that you face while testing and sort it out yourself. This way, you will slowly realize that what you saw as a bottleneck is not a bottleneck at all. And that will be your Wow moment.
5. You may not find bugs all the time. Absence of bugs or lack of skills to find bugs(which tricks you to think that bugs are absent) does not mean you are a bad tester. You probably did not look at the right places and this can be improved with continuous practice.
6. From time to time, we should get together with people from different perspectives and watch them test. Trust Me, the first time you do this, you may think you are a horrible tester. You may be one too. The next time when you watch them test, it will be one of the coolest ways of learning and testing and you would do more of it more often.
Happy Testing,
Parimala Shankaraiah
http://curioustester.blogspot.com
02 October, 2009
Do you heed to bugs in blogger anymore?
I know how pathetic blogger has been. You readers who use blogger or read blogs on blogger would know how pathetic it is. At one time, I was so frustrated with blogger that I decided to move to Wordpress, but then, I read a bad experience report on Wordpress and decided to wait for a better one. While I continue to wait (or decide to buy my own domain), I found an interesting observation on blogger today (while you say No, Not Again).
Typing any URL say http://curioustester.blogspot.com/2009/10/interesting-bug-in-yahoo-messenger.html in the body of the blog post does not convert it to a hyperlink automatically(this happens automatically in word, excel and powerpoint to name a few). So blogger expects you to use the 'Insert Link' option available in Blogger editor to convert it to a hyperlink.
Do you think this is normal?
Do you have a blogger bug story to share?
Do you think there is a better blogging platform you know of?
Happy Sharing,
Parimala Shankaraiah
http://curioustester.blogspot.com
Typing any URL say http://curioustester.blogspot.com/2009/10/interesting-bug-in-yahoo-messenger.html in the body of the blog post does not convert it to a hyperlink automatically(this happens automatically in word, excel and powerpoint to name a few). So blogger expects you to use the 'Insert Link' option available in Blogger editor to convert it to a hyperlink.
Do you think this is normal?
Do you have a blogger bug story to share?
Do you think there is a better blogging platform you know of?
Happy Sharing,
Parimala Shankaraiah
http://curioustester.blogspot.com
01 October, 2009
Interesting Bug in Yahoo Messenger
I have been busy in an internal product training for last 3 days along with my colleague cum mentee Narain Mittal. Today afternoon, product trainer Ravi messaged me on Yahoo Messenger that the training is postponed to 3.30 PM. He requested me to inform Narain as well.
Chat text on my chat window
Ravi: we will have KT at 3:30pm
Pari: sure.
I copied Ravi’s chat message and pasted it onto Narain’s chat window as below:
Chat text on Narain’s chat window
Ravi: we will have KT at 3:30pm
Ravi is typing a message.
Pari: sure.
If you observe the chat text on Narain’s chat window, 'Ravi is typing a message.' is an additional message here. Interestingly, this is not visible in my chat window.
Inference
Additional message is visible only when I copy the chat text from my chat window to narain’s chat window. It looks to me that the text is plain hidden on my chat window. However, explicitly copying the text using Ctrl-C or Copy options will copy the hidden text as well. This happens to another message ‘Last message received on 10/1/2009 at 3:08 PM’ as well. I checked this on Gmail chat and this bug is not prevalent on gmail chat.
Any thoughts on this bug?
Happy Investigating,
Parimala Shankaraiah
http://curioustester.blogspot.com
Chat text on my chat window
Ravi: we will have KT at 3:30pm
Pari: sure.
I copied Ravi’s chat message and pasted it onto Narain’s chat window as below:
Chat text on Narain’s chat window
Ravi: we will have KT at 3:30pm
Ravi is typing a message.
Pari: sure.
If you observe the chat text on Narain’s chat window, 'Ravi is typing a message.' is an additional message here. Interestingly, this is not visible in my chat window.
Inference
Additional message is visible only when I copy the chat text from my chat window to narain’s chat window. It looks to me that the text is plain hidden on my chat window. However, explicitly copying the text using Ctrl-C or Copy options will copy the hidden text as well. This happens to another message ‘Last message received on 10/1/2009 at 3:08 PM’ as well. I checked this on Gmail chat and this bug is not prevalent on gmail chat.
Any thoughts on this bug?
Happy Investigating,
Parimala Shankaraiah
http://curioustester.blogspot.com
Subscribe to:
Posts (Atom)