Software testability is the degree to which a software artifact (i.e. a software system, software module, requirements- or design document) supports testing in a given test context. Take a moment to read about Testability Heuristics from James Bach.
Background
I have not tested for Testability. I did not know what is testability until I heard it as a quality criteria in Weekend Testing 8. When Pradeep Soundararajan pinged me for a testing session on 3rd Oct 2009, I jumped out of my seat. Sharath Byregowda was keen to join us too. The Three Musketeers. I was asked to come up with an application to test and the mission.
Application to Test
Microsoft Narrator is a text-to-speech utility for users who are blind or have impaired vision. Narrator reads what is displayed on your screen: the contents of the active window, menu options, or the text you have typed. I have been amused by its functionality for quite some time and this was the opportunity.
Mission
To test the Testability of the Narrator application.
Experience Report
The three of us set out to test narrator. I started off with Google looking for information on testability. Read up a little on Wikipedia and another article on stickyminds (I found James Bach's article while composing this post). Identified a set of criteria to look out for while testing for testability of the Narrator. I had no clue if I was right or wrong. Nevertheless, it was exciting because this was my first foray into testability. It was very thrilling though I knew I had the potential to do a lot more better. But then, I always discount myself the first time I don’t do anything correctly. This way, I don’t get stressed to run the rat race nor do I have the pressure to learn something just because I am left behind compared to other good testers. Testing was stopped after 1 hour and we had a small discussion. Sharath had concerns with using the product as a whole. I had concerns about not knowing testability enough. Pradeep simply shared his report and headed for lunch without speaking a word. He probably wanted us to see what was in there! Please find my report HERE.
Challenges
1. New Product as usual
2. New mission – Testability
3. I am neither blind nor do I have an impaired vision
4. In any testing activity, whining within or outside of the testing window is an utter waste of time. It is better to find ways to know how to do things and do them better
5. Once out of the testing window, it is better to go back and learn more on unknown areas – in this case ‘learning more on testability’ because this is not the end of it
6. One of the biggest gift we humans can give to ourselves is to identify what we do not know and find ways to know what we do not know. The challenge here is to know what you do not know
Do you know what you do not know?
Addendum(5th Oct 09)
Michael Bolton's take on Testability.
Happy Knowing Thyself,
Parimala Shankaraiah
http://curioustester.blogspot.com
An interesting application to test :) i would also like to see the other 2 musketeers report :) if possiblel.
ReplyDeleteHi,
ReplyDeleteTestability of an application can often be determined by static testing of requirements. An ambiguity review is an excellent way of doing this as it helps establish that all of the items to be tested have definite outcomes. I would strongly recommend Tom Gilbs writing on Software engineering, particularly on the concept of goals, attributes and measures. It is particularly good in helping to define non-functional testing. I would also look at Booch et al writing on use cases for UML which defines "shall statements" as a way of removing ambiguity from a requirement. One of the things that can be thought of in this discussion and may inspire a curious tester to look further is, have you thought about test capability? I did research sometime ago and came up with the concept, a template and heuristics to measure the capability of an organisation or project to actually enable and support testing. For example, you can have the best testers in the world, but what if your change and release management, version control etc is rubbish? the testers will struggle. I would maintain that the capability of an organisation or project to support testing is a far more pertinent question than the testability of the software. Frankly, I think good testers can test anything given a fair chance at it! Peter. e-test@hotmail.com
PS you are a breath of fresh air!
Hi,
ReplyDeleteI am just curious to know that, did you guys make any preparation before testing the application? Like, did you meet the end customers? Did you understood about the scenarios what may come in real world and how the application might be used by visually impaired persons? Because somewhere i read that physically challenged people will be having something extraordinary skill in them to overcome their disability. I have seen a live example of a blind person who is a kool businessman and people say that he never got confused/cheated with notes. He recognise them very well. Also, he knows where the particuar item is located in his store and when people working with him asks about a particular item, he tells them exactly in which rack to look for it. That's something amazing. So when we test such applications, some times it's important to understand and know the customers behaviour patterns. It provides a crucial data, which can make a product a hit or flop. All this i said because, what you might feel as a problem may not be a problem or what you missedout might be the important data, right?
Cheers,
Vijay...