16 April, 2010

BWST 2 - Great Learning Experience

Theme: Cutting (c)trap and getting good things done

Event Sponsor: Vipul Kocher

Venue: Shilton Royal, Bangalore

Co-organizers: Pradeep Soundararajan and Santhosh Tuppad

Co-facilitators: Pradeep Soundararajan and Parimala Shankaraiah

Image Courtesy: Rahul Mirakhur

Presenters: Sharath Byregowda, Selim Mia, Ashok T, Rahul Verma, Vipul Kocher, Dr.Meeta Prakash, Sukanta Bhatt.

Attendees: Swetha Ghorpade, Senthilnathan C S, Allmas Mullah, Eshwar Kumar, LakshmiNarasimha, Gokul Srivathsan, Dhanasekar S, Rayanagouda Patil, Vasu Swami, Rahul Mirakhur, Mandeep Singh, Ajay Balamurugadas, Ravisuriya, Yeshwanth Rao, Sai Divya, Chandrasekhar, Pradeep Soundararajan, Parimala Shankaraiah, Santhosh Tuppad, Sharath Byregowda, Selim Mia, Ashok T, Rahul Verma, Vipul Kocher, Dr.Meeta Prakash, Sukanta Bhatt

Peer Workshops

When I attended STC Conference a few months ago, I had mixed feelings about it with regard to the topics being presented, the length of the presentations, time allotted for questions and discussions post presentation, the speakers, the sponsors and many others. At that time, all I thought was this: "for hands-on testers like me, it would be too much business and very little testing stuff". Although it is good to know about the business aspect of testing, I was not interested to an extent that there was not much related to testing mostly. I had heard about peer testing workshops, read about LAWST, TWST, BWST and AYE in bits and pieces. However, I did not realize that it was in my good taste until I attended BWST 2 held on 3rd April 2010 at Hotel Shilton Royale, Bangalore.

How to Kill a Tester?

Sharath Byregowda, Co-founder of Weekend Testing wanted to present "How to Kill a Tester?" With an extremely hectic work schedule, he hardly had time to attend the workshop, forget about presenting at the workshop. He however made it to the workshop by clearing crappy traps. It was only ideal to get him to talk about his traps and the crap that followed it. It was only hilarious to see that he wanted to talk about How to Kill a Tester? and described how he almost got killed in the process! He talked about unrealistic schedules, short test cycles and subsequent stretching by testers to make ends meet. He also said he learned negotiation skills and being able to convey his concerns to the management assertively. Sharath also hit upon certifications and its associated challenges.

Sharath presented a good experience report in truest sense. He is one no nonsense guy when it comes to presenting. He was his usual self speaking truth to power!

Dangerous decision making - Assumption traps

Ashok T, Founder and CEO at Stag Software Private Limited talked about making decision based on meaningless assumptions. I was floored by the classic examples he provided to convey the message. At one point, he compared fungal cells versus cancerous cells in the body with defect density in the product. He asked ‘Is 33 diseases in a human body good enough to be healthy?’ with respect to metrics.

Ashok spoke in detail on the assumptions in test design, execution and assessment phases of testing and how decisions based on these assumptions were dangerous for the overall health of the product. He emphasized on the need for outcome/goal focused rather than activity focused results.

Benefits of applying Scripted & Exploratory approaches and Problems in Test Management

Selim Mia, a Test Manager came all the way from Bangladesh to attend BWST 2. He was the cynosure of all eyes at BWST 2. Selim started his presentation with how he used scripted and exploratory approaches in a complimentary way to test better. A lot of questions were thrown open to the audience to discuss. It turned out to be a little war of words with different people telling different things based on their own personal experiences. One of the participants said ‘Every scripted activity is exploratory in nature. Only freedom varies’. Selim asked key questions on test estimation, production environment, client specific issues and training challenges etc. He smartly figured out what were doing and comparing against what he does as a Test Manager himself.

Someone from the audience asked why the client did not do anything about certain problems to which Selim replied emphathetically "The client is blinded!" How true!

The Good, The Crap, The Plain Bull

Rahul Verma, Senior Technical QA Lead with a reputed MNC discussed the importance of critical thinking and evaluation for accepting/rejecting new ideas He talked how we testers victimize ourselves and project that the world around us is bad and ugly. He explained how what is good to us can be utter crap to others, ‘Your good can be crap or plain bull to others’ he said. ‘Tell your crap to yourself before others come and tell you’, he went on.

Another key thing was about Ideas. He openly announced how blindly we accept ideas without evaluating them. It’s like the sheep herd wherein we follow whichever sheep in front of us heads towards. We fail to evaluate out of blind trust or laziness. He said it’s ok to not have an opinion about something. It’s ok not to take sides when you are not sure about an idea.

All along Rahul’s session, I felt as if he had a hammer in his hand and kept on hitting on my head by sharing bitter facts about the way I am and the way I work. My head continues to ache! I need to change myself………Right Now!

Noun and Verb technique

Vipul Kocher, Founder and CEO at PureTesting presented Noun and Verb technique designed by Elizabeth Hendrickson. I had heard about this technique somewhere, but my lizard brain took over my humble goal of learning more about it. He talked about what this technique does, how it helps in coming up with test ideas for testing and also to deal with related biases while using this technique. Many participants had not heard of this technique and were interested in knowing more about it. Rahul Verma especially commented on how this technique is very helpful to him in reviewing requirement documents. Vipul added more by saying any technique should elicit more questions as it helps testers in testing the products better.

Yet another meeting ....Yet another talk !

Dr.Meeta Prakash, Project Manager at Infosys talked about crappy meetings and how valuable time at office goes down the drain. She classified meetings into many categories and went into details about how content discussed in meetings could be misunderstood or misinterpreted. She also emphasized on how paraphrasing helps in meetings. She briefed a bit about how meetings can be effective, how to learn to say no to wasteful meetings and learn the art of effective meetings.

Lessons Learned and Best Practices in Healthcare Imaging Platform Testing

Dr.Sukanta Bhatt presented a brand new viewpoint to Healthcare testing. He gave wonderful insight into day to day challenges in testing healthcare products and the traps in testing them. He gave great examples of how a missed bug can kill a person and how important it is to find every critical bug. Not to mention his hilarious style of quoting real life examples, he kept the participants in splits most of the time.

Sukanta elaborated on how UI design can be challenging for healthcare products. He emphasized on the importance of sociology and anthropology and their respective value systems. He also briefed about ‘Go to Gemba’ which means we need to get in touch with the real stakeholders to get products tested based on their needs.

Why only Bangalore?

The other day, Allmas mulled over the event saying 'You guys in Bangalore have all the fun'. Of course, we chose to have it. We created such forums for ourselves and it wasn't as hard as you may think.

We are hoping that there are some passionate people outside Bangalore to be able to create, foster & have consistent meetings of any kind. Learning is your responsibility and hence be responsible".

Want help in starting one of such groups, we shall be glad to help you help yourselves.

Token of Thanks

In the interest of time, I won’t go into the nitty gritties of each participant’s questions or the discussions that followed them. In short, it was one of the most useful workshops I have ever attended in testing so far. And yes, I mean it! The participants were the joyous lot who got a feeler of diverse topics on testing in one day. As Sukanta Bhatt summarised it "Years of Experience in hours"!

Pradeep Soundararajan's report can be read HERE

Thank You BWST 2,

Parimala Shankaraiah

13 April, 2010

Q-Patterns by Vipul Kocher

When I talked about Questioning skills a few months ago, I wasn't very clear on which tools to use to practice questioning [Of course, in addition to my brain]. Plenty Questions and Dumb Charades took me a step forward. I saw a downside to it. Some questions were lost, some were forgotten and others not important enough. This is where my Note-taking skill came in handy. As I made notes of different questions and grouped them under specific categories, I got many more which were interrelated or newer. How to manage this ocean of questions? How do I structure them well enough to solve the problem at hand? How many questions are good enough? This is when I remembered Q-Patterns that I had read about a few months ago.

Questioning Pattern (also known as Q-Pattern) is a set of interrelated QUESTIONS grouped together to:
         • Relate to some aspect of user or software requirements
         • Provide various alternatives to arrive at a solution

Vipul Kocher, Co-President of Pure Testing is the creator of Questioning Patterns. In his own words, "Q-Patterns is a great tool for communication of domain specific knowledge across people and continuous skill enhancement. It is also a tool for requirement elicitation and defining specifications".

Structure of a Q-Pattern
  • Name of the Q-Pattern
  • Intent/Explanation/Definition
  • Classification
  • Metadata
    • Template Version
    • Q-Pattern Version
    • Author
    • Author Contact information
    • Keywords
  • Questions on
    • Usage
    • User Interface
    • Administration
    • Performance
    • Security
    • Internationalization
    • Localization
  • Examples
  • Associated Q-Patterns
  • Specialization

Vipul’s Example of a Q-Pattern
Password Management

The most general and common approach to authenticate a system or user is asking for a Password. Password authentication can be at different levels like user level, group level etc or at different stages like Operating System authentication, Application authentication etc.

If you are using Password authentication anywhere in your spec/design/code/test you may ask following questions:

1. Can administrator reset the password?
2. Can administrator's password be reset?
3. What happens If the administrator forgets his password (any default password is given or reinstallation would take place)?
4. Can administrator set the default password?
5. Can another administrator reset an administrator's password?
6. Can an administrator read the password of a user?

1. What's the maximum and minimum length of password?
2. Can we enter numbers in password?
3. Can blank password be used?
4. Where are passwords stored?
5. What is the default password (If any)?
6. Can one customize the default password?
7. Can Special characters (like #,$) and accented characters be used in password?
8. How is password change affected? Is original password required before change password is allowed?
9. Is Confirm password used?
10. Is `Save Password' facility there on the screen (so that user may not need to enter password every time she logs in)?

1. Is password shown as stars (at the time of entering the password, at the time of changing or resetting the password etc.)?
2. How many stars are shown for a password
       a. When it is being entered?
       b. When it is to be changed? (Note: Do not show same number of stars as the number of characters.)

1. How are passwords stored? Are they encrypted before storing? If yes what is the encryption algorithm used?
2. Whether the password is case sensitive or not?
3. Whether the password can be cut and pasted?
4. Can a previously used password be used again? If `Yes' then after how many changes?
5. Is there any expiry time for the password? What happens after the date if user does not change the password during that period?
6. Is there any policy to count the number of password validations in succession (e.g.. If user enters wrong password 4 times, then she is not able to enter password again in succession).
7. If application creates logs of all activities, then the logs of password are created or not?
8. If logs of password are being made then the password is stored in encrypted form or not?

1. Whether password is made up of single-byte characters (even if multi-byte character set is being used in the application).
2. How much time will it take to authenticate the user after the submission of password?
3. What is the maximum space required to store a password? Will all the passwords require same space irrespective of size?
4. If wrong password is given, how much time will it take to give the error message?
5. How many users can be authenticated at the same time?

Various login screens and mechanisms (web based mail systems, console based login etc.)

Associated patterns
Access Rights, Error messages

Say, login for any particular web based mail system.

Q-Patterns and Test Oracles
We find problems in software because somehow, the software doesn’t look quite right. How? Based on feelings, emotions, prior experience? with similar or different products? We build our knowledge base from our past knowledge and perceptions. Sometimes, our perceptions about the software could be wrong. Sometimes, how we desire the software should behave could be wrong. What to do next? Change the software OR the desire about software? OR change the perception about how software is used in general.

As I see it, Q-Patterns are based upon prior experiences and perceptions of people. It helps build a knowledge repository by asking appropriate questions. And this knowledge repository keeps growing as we learn more about the product every single day.

Q-Patterns and Testing Checklists
Once the Knowledge Repository (or the Question Bank) is ready, one can notice that answers to these questions transform into a Testing Checklist for that feature. Voila! I have interacted with few people who find it hard to shift from test cases to test checklists. I have faced this in the past myself. After using Q-Patterns for some time, I now see it as a tool to help me build my Testing checklist by answering the questions in my Q-Pattern.

Agreed, that it depends on the person's skills and talent at questioning. Questions may not make a robust list for a start. But, they can be evolved over time. One way to come up with more is to brainstorm questions with different teams and people. Another way is to get your questions reviewed by your peers or friends asking for feedback.

Q-Patterns and Test Coverage
The Questions section in the Q-Pattern template addresses what areas we are willing to address. The ones mentioned above like Administration, Usage, UI etc are just examples. We could add whatever suites the feature we have selected and storm ourselves with questions. Suppose, I am testing a call center flow of a CRM application, I would add Usability, Number of clicks to perform an operation and keyboard shortcuts as additional section [These in turn may/may not get covered as part of different Test Techniques if any]. There is no one Q-Pattern fits all solution. We need to come up with some basic patterns and build them over time. If we address all the features in the system using Q-Patterns and asking questions about each feature, there is a possibility to have covered at least important test techniques to test the software.


Happy Questioning,
Parimala Shankaraiah