13 April, 2010

Q-Patterns by Vipul Kocher

When I talked about Questioning skills a few months ago, I wasn't very clear on which tools to use to practice questioning [Of course, in addition to my brain]. Plenty Questions and Dumb Charades took me a step forward. I saw a downside to it. Some questions were lost, some were forgotten and others not important enough. This is where my Note-taking skill came in handy. As I made notes of different questions and grouped them under specific categories, I got many more which were interrelated or newer. How to manage this ocean of questions? How do I structure them well enough to solve the problem at hand? How many questions are good enough? This is when I remembered Q-Patterns that I had read about a few months ago.

Questioning Pattern (also known as Q-Pattern) is a set of interrelated QUESTIONS grouped together to:
         • Relate to some aspect of user or software requirements
         • Provide various alternatives to arrive at a solution

Vipul Kocher, Co-President of Pure Testing is the creator of Questioning Patterns. In his own words, "Q-Patterns is a great tool for communication of domain specific knowledge across people and continuous skill enhancement. It is also a tool for requirement elicitation and defining specifications".

Structure of a Q-Pattern
  • Name of the Q-Pattern
  • Intent/Explanation/Definition
  • Classification
  • Metadata
    • Template Version
    • Q-Pattern Version
    • Author
    • Author Contact information
    • Keywords
  • Questions on
    • Usage
    • User Interface
    • Administration
    • Performance
    • Security
    • Internationalization
    • Localization
  • Examples
  • Associated Q-Patterns
  • Specialization

Vipul’s Example of a Q-Pattern
Name
Password Management

Intent
The most general and common approach to authenticate a system or user is asking for a Password. Password authentication can be at different levels like user level, group level etc or at different stages like Operating System authentication, Application authentication etc.

Questions
If you are using Password authentication anywhere in your spec/design/code/test you may ask following questions:

Administration
1. Can administrator reset the password?
2. Can administrator's password be reset?
3. What happens If the administrator forgets his password (any default password is given or reinstallation would take place)?
4. Can administrator set the default password?
5. Can another administrator reset an administrator's password?
6. Can an administrator read the password of a user?

Usage
1. What's the maximum and minimum length of password?
2. Can we enter numbers in password?
3. Can blank password be used?
4. Where are passwords stored?
5. What is the default password (If any)?
6. Can one customize the default password?
7. Can Special characters (like #,$) and accented characters be used in password?
8. How is password change affected? Is original password required before change password is allowed?
9. Is Confirm password used?
10. Is `Save Password' facility there on the screen (so that user may not need to enter password every time she logs in)?

UI
1. Is password shown as stars (at the time of entering the password, at the time of changing or resetting the password etc.)?
2. How many stars are shown for a password
       a. When it is being entered?
       b. When it is to be changed? (Note: Do not show same number of stars as the number of characters.)

Security
1. How are passwords stored? Are they encrypted before storing? If yes what is the encryption algorithm used?
2. Whether the password is case sensitive or not?
3. Whether the password can be cut and pasted?
4. Can a previously used password be used again? If `Yes' then after how many changes?
5. Is there any expiry time for the password? What happens after the date if user does not change the password during that period?
6. Is there any policy to count the number of password validations in succession (e.g.. If user enters wrong password 4 times, then she is not able to enter password again in succession).
7. If application creates logs of all activities, then the logs of password are created or not?
8. If logs of password are being made then the password is stored in encrypted form or not?

Performance
1. Whether password is made up of single-byte characters (even if multi-byte character set is being used in the application).
2. How much time will it take to authenticate the user after the submission of password?
3. What is the maximum space required to store a password? Will all the passwords require same space irrespective of size?
4. If wrong password is given, how much time will it take to give the error message?
5. How many users can be authenticated at the same time?

Example
Various login screens and mechanisms (web based mail systems, console based login etc.)

Associated patterns
Access Rights, Error messages

Specialization
Say, login for any particular web based mail system.

Q-Patterns and Test Oracles
We find problems in software because somehow, the software doesn’t look quite right. How? Based on feelings, emotions, prior experience? with similar or different products? We build our knowledge base from our past knowledge and perceptions. Sometimes, our perceptions about the software could be wrong. Sometimes, how we desire the software should behave could be wrong. What to do next? Change the software OR the desire about software? OR change the perception about how software is used in general.

As I see it, Q-Patterns are based upon prior experiences and perceptions of people. It helps build a knowledge repository by asking appropriate questions. And this knowledge repository keeps growing as we learn more about the product every single day.

Q-Patterns and Testing Checklists
Once the Knowledge Repository (or the Question Bank) is ready, one can notice that answers to these questions transform into a Testing Checklist for that feature. Voila! I have interacted with few people who find it hard to shift from test cases to test checklists. I have faced this in the past myself. After using Q-Patterns for some time, I now see it as a tool to help me build my Testing checklist by answering the questions in my Q-Pattern.

Agreed, that it depends on the person's skills and talent at questioning. Questions may not make a robust list for a start. But, they can be evolved over time. One way to come up with more is to brainstorm questions with different teams and people. Another way is to get your questions reviewed by your peers or friends asking for feedback.

Q-Patterns and Test Coverage
The Questions section in the Q-Pattern template addresses what areas we are willing to address. The ones mentioned above like Administration, Usage, UI etc are just examples. We could add whatever suites the feature we have selected and storm ourselves with questions. Suppose, I am testing a call center flow of a CRM application, I would add Usability, Number of clicks to perform an operation and keyboard shortcuts as additional section [These in turn may/may not get covered as part of different Test Techniques if any]. There is no one Q-Pattern fits all solution. We need to come up with some basic patterns and build them over time. If we address all the features in the system using Q-Patterns and asking questions about each feature, there is a possibility to have covered at least important test techniques to test the software.

References
http://www.whatistesting.com/qpatterns.htm

Happy Questioning,
Parimala Shankaraiah