Q-Patterns by Vipul Kocher

When I talked about Questioning skills a few months ago, I wasn't very clear on which tools to use to practice questioning [Of course, in addition to my brain]. Plenty Questions and Dumb Charades took me a step forward. I saw a downside to it. Some questions were lost, some were forgotten and others not important enough. This is where my Note-taking skill came in handy. As I made notes of different questions and grouped them under specific categories, I got many more which were interrelated or newer. How to manage this ocean of questions? How do I structure them well enough to solve the problem at hand? How many questions are good enough? This is when I remembered Q-Patterns that I had read about a few months ago.

Questioning Pattern (also known as Q-Pattern) is a set of interrelated QUESTIONS grouped together to:
         • Relate to some aspect of user or software requirements
         • Provide various alternatives to arrive at a solution

Vipul Kocher, Co-President of Pure Testing is the creator of Questioning Patterns. In his own words, "Q-Patterns is a great tool for communication of domain specific knowledge across people and continuous skill enhancement. It is also a tool for requirement elicitation and defining specifications".

Structure of a Q-Pattern

• Name of the Q-Pattern
• Intent/Explanation/Definition
• Classification
• Metadata
• Template Version
• Q-Pattern Version
• Author
• Author Contact information
• Keywords
• Questions on
• Usage
• User Interface
• Administration
• Performance
• Security
• Internationalization
• Localization
• Examples
• Associated Q-Patterns
• Specialization

Vipul’s Example of a Q-Pattern
Name
Password Management

Intent
The most general and common approach to authenticate a system or user is asking for a Password. Password authentication can be at different levels like user level, group level etc or at different stages like Operating System authentication, Application authentication etc.

Questions
If you are using Password authentication anywhere in your spec/design/code/test you may ask following questions:

Administration
1. Can administrator reset the password?
2. Can administrator's password be reset?
3. What happens If the administrator forgets his password (any default password is given or reinstallation would take place)?
4. Can administrator set the default password?
5. Can another administrator reset an administrator's password?
6. Can an administrator read the password of a user?

Usage
1. What's the maximum and minimum length of password?
2. Can we enter numbers in password?
3. Can blank password be used?
4. Where are passwords stored?
5. What is the default password (If any)?
6. Can one customize the default password?
7. Can Special characters (like #,$) and accented characters be used in password?
8. How is password change affected? Is original password required before change password is allowed?
9. Is Confirm password used?
10. Is `Save Password' facility there on the screen (so that user may not need to enter password every time she logs in)?

UI
1. Is password shown as stars (at the time of entering the password, at the time of changing or resetting the password etc.)?
2. How many stars are shown for a password
       a. When it is being entered?
       b. When it is to be changed? (Note: Do not show same number of stars as the number of characters.)

Security
1. How are passwords stored? Are they encrypted before storing? If yes what is the encryption algorithm used?
2. Whether the password is case sensitive or not?
3. Whether the password can be cut and pasted?
4. Can a previously used password be used again? If `Yes' then after how many changes?
5. Is there any expiry time for the password? What happens after the date if user does not change the password during that period?
6. Is there any policy to count the number of password validations in succession (e.g.. If user enters wrong password 4 times, then she is not able to enter password again in succession).
7. If application creates logs of all activities, then the logs of password are created or not?
8. If logs of password are being made then the password is stored in encrypted form or not?

Performance
1. Whether password is made up of single-byte characters (even if multi-byte character set is being used in the application).
2. How much time will it take to authenticate the user after the submission of password?
3. What is the maximum space required to store a password? Will all the passwords require same space irrespective of size?
4. If wrong password is given, how much time will it take to give the error message?
5. How many users can be authenticated at the same time?

Example
Various login screens and mechanisms (web based mail systems, console based login etc.)

Associated patterns
Access Rights, Error messages

Specialization
Say, login for any particular web based mail system.

Q-Patterns and Test Oracles
We find problems in software because somehow, the software doesn’t look quite right. How? Based on feelings, emotions, prior experience? with similar or different products? We build our knowledge base from our past knowledge and perceptions. Sometimes, our perceptions about the software could be wrong. Sometimes, how we desire the software should behave could be wrong. What to do next? Change the software OR the desire about software? OR change the perception about how software is used in general.

As I see it, Q-Patterns are based upon prior experiences and perceptions of people. It helps build a knowledge repository by asking appropriate questions. And this knowledge repository keeps growing as we learn more about the product every single day.

Q-Patterns and Testing Checklists
Once the Knowledge Repository (or the Question Bank) is ready, one can notice that answers to these questions transform into a Testing Checklist for that feature. Voila! I have interacted with few people who find it hard to shift from test cases to test checklists. I have faced this in the past myself. After using Q-Patterns for some time, I now see it as a tool to help me build my Testing checklist by answering the questions in my Q-Pattern.

Agreed, that it depends on the person's skills and talent at questioning. Questions may not make a robust list for a start. But, they can be evolved over time. One way to come up with more is to brainstorm questions with different teams and people. Another way is to get your questions reviewed by your peers or friends asking for feedback.

Q-Patterns and Test Coverage
The Questions section in the Q-Pattern template addresses what areas we are willing to address. The ones mentioned above like Administration, Usage, UI etc are just examples. We could add whatever suites the feature we have selected and storm ourselves with questions. Suppose, I am testing a call center flow of a CRM application, I would add Usability, Number of clicks to perform an operation and keyboard shortcuts as additional section [These in turn may/may not get covered as part of different Test Techniques if any]. There is no one Q-Pattern fits all solution. We need to come up with some basic patterns and build them over time. If we address all the features in the system using Q-Patterns and asking questions about each feature, there is a possibility to have covered at least important test techniques to test the software.

References
http://www.whatistesting.com/qpatterns.htm

Happy Questioning,
Parimala Shankaraiah

Is Questioning in your DNA?

I took part in James Bach's Testing Challenge at Weekend Testing recently. Pradeep Soundararajan reviewed my report and spoke about the gaps in my testing effort as well as the report. As always, I revisited my report to figure out areas that need improvement(read dismal performance report). One of the key problems with my report was that - I did not question the mission that was given to me. I did skype James before the testing session with a few questions that were either too broad or too open ended. James did help me by rephrasing a few questions. However, I did not use this opportunity to its fullest potential.


A little while later, Jonathan Bach & James Bach tweeted on Traps in testing

JBTestPilot: The #1 trap that testers fall into when I ask them to test something: They start testing'.

CuriousTester: I am trapped! din't see testing without questioning as a trap! Traps work that way. This is more often than I would wish to get trapped!

JBTestPilot: No worries... there are many ways out of traps. The trick is, knowing you're IN one.

JamesMarcusBach: Good testing is a questioning process. All tests are questions. But some things are better asked of humans.

Here I am inflicted with lack of good questioning skills. I did not worry much about it until it started affecting my ability to test better. Now you ask ‘Do you mean you never questioned till date?’. Of course, I did. They did not seem to add value to what I already do. They did not appear to bring in new ideas. In short, I am not skilled at questioning yet.

[ Tester ] - [ Questioning Skills ] = [ Unskilled Testing ]

I am wondering why questioning is not in my DNA. Is it really in my DNA and I am unaware? I grew up as an obedient child. I don’t remember saying ‘No’ to my parents, no matter what I was told to do - a very big claim indeed (grins). When I say as a child, I mean the period from which I remember the events of my life to a considerable extent. Being obedient, not breaking the rules, not being demanding and many more blocked my ability to question since childhood.

In general, questions are not encouraged in India (I think so). They are seen as a symbol of revolt. Questions from elders are reprimanded and questions from children are snubbed. Children are taught to be obedient without questioning. If they break the rules, they will be punished. Yes, child beating is quite normal and not violent. Actually, its not even violent as it sounds to you as you read this. A 911 type helpline may flop here. Growing amidst a culture where questioning is a taboo could be one reason why I am not good at it.

Every question adds a new dimension to better understanding through answers. Yet, we don’t question. The sad part is most of us don’t question because we don’t know that we can question. We have the liberty to question, but we don’t.

Question what? Question your product, question your testing mission, question your test strategies question your deliverables. Question anything that you think will make a difference to what you do. Some people are good at questioning and a few others are not so good. The good news is that this skill can be inculcated with constant practice by one and all. It can be honed over a period of time.

Practice questioning skills with your programmers/team members. Play games that help improve your questioning skills. Play Twenty questions game. Playing Dumb Charades might be of great help too. Ask the right questions. To ask the right questions, you will need to ask many wrong questions and learn from them. Open ended questions may be too broad. Close ended questions may be too vague. Probing questions focus on specific areas (can be open or close ended) and are very useful. A good mix of above three types of questions will improve questioning skills. Note taking if coupled with Questioning is a powerful tool for testing.

Wishing You a Prosperous & Happy New Year 2010!

Addendum on 30th April 2010
One way to practice questioning skills is using Q-Patterns by Vipul Kocher.

P.S: The title of this post is inspired by Rahul Mirakhur who made a mention of 'Questioning in your DNA' during one of our informal talks.
Happy Questioning,
Parimala Shankaraiah