15 September, 2010

Wherez my Cappuccino?

I am Back!
I am back or should I say tentatively. I have been focusing on my personal priorities and a couple of releases both at work and home  :-). Hence hectic. As I type this, I know that it’s all crap! The reality is I haven’t been able to focus for a while on the stuff that I am passionate about. And consequently, several excuses follow. Finally, I thought today was the day to start over again.

Morning Cappuccino
The first thing I do in the morning when I get to office is to head to the pantry to get a fresh cup of coffee. It’s not necessarily ‘Fresh’ as it comes out of a vending machine. However, machine coffee is better than no coffee especially during rainy and winter seasons.

Given a small organization like ours, there are just a handful of employees who come in early to work and are potential coffee drinkers - one of my colleagues, Director, VP and myself. Among the four of us, my director and I have coffee pretty much first thing in the morning even before checking emails. Why should you know about who drinks coffee or not?

Here’s the story. On some days, I reach office before my director does, hence being the first one to use the coffee machine. When I select "Cappuccino" on the coffee vending machine, milk pours over and coffee follows. That is the expected result. Isn't it? On some days, I prefer tea. On all days that I select Cappuccino the first time, I see that only milk pours over, but coffee doesn’t. If you were me, you would end up feeling guilty for wasting one full glass of milk which is a potential savior for under nourished kids in several parts of the world. Keeping emotions aside about what happened to the milk which failed to transform to Cappuccino, if I selected Cappuccino second time, I would get coffee – milk followed by coffee. Hurray! I would rush back to my seat to check my emails as I sip coffee with a lovely view of busy morning traffic of which I am not a victim as I walk to my office.

One fine day, my colleague Selva ended up early in office. Incidentally, Selva opted for coffee. He was frustrated when he got milk inspite of selecting Cappuccino. I was like “Oh, Selva. If you didn’t know, you don’t get coffee the first time. You get it the second time”. Now, Selva had already gone nuts because I had demonstrated the elevator bug a few days ago. He replied with his trademark smile “Are you kidding me?”. “Ok, check it yourself”, I replied as I washed my mug.

Every human being is a tester by birth. How skilled he/she is varies. Selva went on and tested the flow himself and figured out he got cappuccino the second time. Selva burst out laughing as we discussed about the fun involved in finding bugs on live systems be it the elevator, coffee machine, music player on the cell phone or even the appliances we use on a daily basis.

Background

Cappuccino doesn’t like first time user [Assumption 1]

What happened when I selected cappuccino the first time ?
How did it work second time?
Does it work the third time?or
Does it alternate its behavior?

Questions. Being able to ask questions. A true boon to testers.

With good intentions to save electricity, the housekeeping guys switch off all the appliances in the night once all the employees leave office. This includes the refridgerator, microwave, coffee vending machine, hand dryers and lights. Every day, milk, butter, cheese and other supplies that need refridgeration are bought typically lasting for not more than a day. At the end of the day, most of it is over if not all and hence refridgerator can be switched off.

The coffee vending machine is cleared of the contents as well [Assumption 2]. The milk container is emptied, washed and dried. Water dispenser is disconnected and the machine is switched off. Coffee beans continue to remain in the coffee dispenser. Next day morning, fresh milk is filled in, water dispenser reconnected and the machine is switched on.

When a first time user (on that particular day) selects Cappucino, it takes a while for the system to start up Since the user won’t wait that long and selects Cappuccino, as the milk flows, the coffee beans container is not yet ready to supply coffee beans [ Analogous to logging into a machine where the system starts applying basic settings for that user]. End result – milks flows out minus the coffee. By the time user is ready to select second time, coffee beans would be ready to disperse. So, the moment cappuccino is selected, milk flows, followed by coffee beans drilling down from the container eventually followed by coffee.

What happens third time? Once cappuccino flows out second time, it will continue to work consistently without any hassles. Now, what if the machine is left unused for a couple of hours contrary to overnight time of 8 hrs)? Does leaving the coffee vending machine idle and unused render the same state similar to switching off the machine overnight? May be. May not.

Find the culprit - He must be elsewhere too!
If a typical night break between switching off the machine to switching it on is say ~8 hrs, we can try to reproduce the issue without switching off the machine as well.

Keep the machine switched on, but leave it unused starting with 5 minutes
Select Cappuccino and check if coffee arrives
Keep trying every 5 minutes until you hit closer to 8 hrs of idling to reproduce the problem without switching off the system.
Stop when you are able to reproduce the issue

The above approach would tell me if similar behavior is observed in other situations as well

Nail the Problem
Now that the problem is reproducible consistently by first time users everyday, it’s important to isolate the problem and investigate further.

States of the Coffee Beans Dispenser
Initial State after the machine is switched on
State when the machine is on, but has been idle for ‘X’ units of time
State when user presses/holds down Cappucino option
State when user releases Cappucino option
State after the user releases Cappucino option

Dismantling the coffee vending machine and watching over different aspects of its functionality would help me determine above listed states and corresponding behavior of the coffee beans dispenser. This would have provided additional clue as to why the beans were not getting pushed to the grinder interface on first attempt. How I wish this machine was at home? Not to mention the cell phone charger I recently damaged in the name of testing and troubleshooting.

Making and breaking assumptions

[Assumption 1] – Cappuccino doesn’t like first time user
I do not really know if anyone other than IT employees like housekeeping guys, receptionists use the coffee machine before me or some other first time user. It’s an assumption I made because on days when I have come earlier than the housekeeping guys, I have switched on the appliances myself though I would wait for them to fill up milk and other supplies.

[Assumption 2] - The coffee vending machine is cleared of the contents as well
Above illustration of cleaning a coffee vending machine is my view of how it is cleaned and may not be the professionally correct way of cleaning it. This is again based on my observations of how they are cleaned when I worked on a couple of weekends.

[Assumption 3] – The problem lies with Cappuccino option alone
I did try a few dry runs while reproducing the above problem with other options like tea , milk, espresso, latte, water and others. I used MFAT (Multiple Factors at a Time) heuristic and got a lead that this happens for Cappuccino only. This was an assumption because in depth testing of other options might have given me more hints about the problem behavior.

Listing down my assumptions about the system, environment and the type of users (myself, Selva and Giri) helped me to evaluate my understanding of many more parameters within the system. If I proved or disproved my assumptions based on the facts/information I was collecting about the system, I would get closer to the root cause of the problem.

Yet another First Time User
The other day Giri, another colleague of mine came early to office. As I entered pantry, I heard Selva talking about the "Cappucino bug" to Giri. At least, Selva was glad that he was not the first user that day. It was Giri’s turn to crib. And he did

Giri gave a quick fix for this problem. He looked at me and said “Parimala, I’ll fix this bug. Look. If I shake the coffee bean dispenser” moving his hands harshly over the dispenser, “I’ll get coffee”. I asked “Giri, have you unit tested it?”. He said “Yes. I am doing it right now”. As he selected Cappuccino, only half the amount of milk poured out and coffee never arrived. We had another round of laughter when I said “I reject your quick fix. Your fix not just fails to fix the problem, but has introduced a newer problem”. Giri’s fix failed buddy testing and consequent regression as well. He has promised to work on the fix someday. In the meantime, I was telling Giri and Selva how much fun it is to play with live systems and investigate problems. If you have not forgotten, the coffee machine is not owned by me. I would have loved to dismantle the entire system to see how each component works - be it milk container, milk pipe, coffee beans dispenser, grinder inside the vending machine, water flow and many others. But, if it stops working completely after my real time investigation, I might have to refund the cost for replacing it. Do you see a connection here? How often do we stop investigating bugs to avoid witnessing further failures?

I’ll end the investigation halfway while I wait for the monthly maintenance engineer to arrive so I can pair with him on this problem while he opens up the vending machine. For now, we continue to use the same coffee vending machine while many more testers struggle to understand how their Cappuccino disappeared.

Regards,
Parimala Shankaraiah

8 comments:

James Marcus Bach said...

Hi Pari,

Perhaps the system goes into a stand-by mode after a certain period of inactivity, and the bug is related to that. Maybe if the first user asks only for coffee, or only for milk, the next user will get cappuccino as normal.

-- james

Parimala Shankaraiah said...

@James

I did try asking for only milk and only water in some tests. Still, it didn't appear to rectify the problem.

Still, running each of the other options and observing carefully will surely help. I was wondering why I did not use the word 'stand-by' anywhere in the post :)

Thanks for your time,

Regards,
Parimala Shankaraiah

Anonymous said...

hi Parimala

I was eagerly waiting for your next post .. and here it is .

Simply Awesome !!


Hiral

Ram said...

Hi Pari,
What is the make of that coffee maker machine?Is this issue happening in that specific hardware of that make or more than one such hardware of that same manufacturer.Is this type of issue a common one in other makes(or brands from various manufacturers)?.May be an competitor analysis would give more such interesting information ?

Thanks
Ram

Ravisuriya said...

I witnessed conducts which seems quite similar to you said from coffee vending machine. Each time I got which I did not want from coffee vending machine, it was joy and fun for me to test this automated machine. Do not know whether all coffee vending machines will have this behavior. Studying further came to know all coffee vending machines in facilities where I work for now, seems to have one such similar problem.


Analysis of tests yielded few inferences:

* coffee powder was getting struck as coffee bean powder grinder container overflowed. And it jammed in its outlet where it is about to combine with hot water and later with milk;
* time interval set in processor of coffee vending machine in such context for initiating each subsequent action and corresponding reactive action on failure;
* processor unable to look for coffee bean grinder container for overflowing or jam;
* power interruption (main supply and UPS, while coffee bean was being powdered);
* cleaning of steam, milk and coffee been powder channels for each new request did not happen from processor or was not considered in this machine;
* unexpected number of users using coffee vending machine in given interval of time which manufacturer of vending machine did not expect; or with no or minimal latency which was programmed may have any influence?
* may be the heat generated in machine for steam and grinding coffee bean powder had influence on this behavior to get stronger each time;
* may be there was presence of moisture in coffee bean grinder due to hot water and steam;
* weakened contact of press buttons on coffee vending machine;
* delay in passing signals or interruption between events/threads, by programmed processor;
* did any other request like hot water or milk or steam alone had any influences;
* registry contents of processor are not cleared off when vending machine switched off and switched on.

H!M@N$HU said...

Hi Parimala,

I am a regular reader of your blog but this is the first time I am commenting.

Yet another good post as expected.

Thanks for sharing these stories which helps us to understand the kind of details that we need to take care while testing applications.

Thanks,

Anonymous said...

Hi, Just thought to send you our team's appreciation for your posts. We are a SaaS test management service providers, and we all like to read your posts, written as stories but are professional and show creative (curious?) thinking. Keep them coming.
Testuff team
www.testuff.com

Karthik said...

Amazing how analogous a coffee machine is in the testers circle. We had a different problem with our coffee machine. Most people would come to the machine at the same time and queue up. When in the queue, I once commented that what this machine lacks is a "multi-threading concept". That way, tea and coffee drinkers could parallelly fill their cups. It was well received by the other queue members. I am a tester who tests from a product management point of view and it showed in a coffee queue too.