Priceless bugreports
You go to the kitchen and turn on the light to see what's so nice has your darling husband been cooking for
you. But something's broken and the light won't turn on. How are you going to tell him that?
«Sweetheart, please replace the bubble in the kitchen» or «Darling, it's dark there»? At this very moment
your darling is having a hard time with SVN trying to merge two repositories and he couldn't care less for your light bulb.
So how to tell him? And not just shout out the words in the air. How to tell him so that he will listen to you
and understand what is the problem?
This is called a "bug report". A description of something that is gone wrong. This came from software industry
but in general it has something to do with us human and how are we talking to each other. Not that it really
matters, but it's good to know how to report problems and issues to collegues and friends. And not just report
— we want our mates to clearly get the message. And more, we want them to start looking for the
solution immeditely, even if that happens at the background of their mind.
No matter how blondish the bugreport is ("This thing is kinda like broken")
you will still have to reduce it to something clear. And if the original bugreport is fuzzy, you will
spend time asking stupid questions ("Which thing? Why broken?") and will end
up with stress ("I've told you already, it's not working!!!").
Programmers have used to think things clearly (okay, I'm trying to be positive). So, during years of growth of
IT industry they have invented a nice and simple way to describe bugs.
Priceless bugreport
The priceless bugreport has three important items:
What have you done?
(Steps required to reproduce the problem)
What have you got?
(Actual results)
What did you expect?
(Expected results)
(Steps required to reproduce the problem)
What have you got?
(Actual results)
What did you expect?
(Expected results)
Also you've to tell where exactly the defect has been found, in which environment and also give the bug a short title.
«Dear, I have tried to turn on the light in
the kitched over the oven to see what have you cooked, but the
light don't come up. Can you please take a look what's wrong?»
What have you done. What actions have you taken to get this result, step by step.
What have you got. After you have executed all the steps mentioned above. This is the actual defect.
What did you expected. What you think must happen after the same steps are executed in the correct system.
Also:
Where happened. This info must be also present in the bugreport. The reader must have an immediate understanding of the scope of the problem i.e. which part of the system could be affected. You don't have to write this down as an additional item. You just have to mention it. You may also tell this in the "What have I done" clause because travelling thru the system to the buggy form could be considered actions as well.
Environment. Everything that is not an action but what is important to know. Say, for web application, you must note the browser/OS combination.
Title. This is the shortest bug description possible. You will use it later in bugs lists, hallway discussions, etc.
This is a very nice form:
- It's straightforward. The bugreporter can't really populate it with useless noise, unless he will try hard to ignore the rules;
- Verifiable. It means that by reproducing the written steps of instruction you will get the very same result and confirm that the bug truly exists. Or, you will end up with a different result so that you could fill another bugreport for another topic by yourself. Or, you will get the expected result and reject the bugreport;
- Self-confirmed. You can immediately tell if that is a bug or not by looking at the "Expected results" clause. What if it's not a bug at all? There is no light on top of the oven installed at all? And the light switch is an orphan piece of hardware mounted by manufacturer simply because the chineese engineer failed to merge two repositories?
- This form eliminates the stressful stupid communication, those boring what-did-you-mean-by-that question;
- It's easy to teach regular people to report according to this form. Only two-three days of cry and tears in your office — and your collegues will learn to report clearly;
- By expicitly telling you what is the expected results the user will confirm that he indeed knows what the system is supposed to do;
- This kind of bugreport doesn't motivate you to skip on it and pretend it never existed;
Examples
A bug that was noticed by the user:
Can't login
What I did:
1. Opened http://www.something.com/
2. Clicked on "login", saw the login form
3. Entered "egor" in the login field and correct password in the password field, clicked on "enter" button
What I have got:
White page, URL in the address bar is http://www.something.com/checklogin
What I have expected:
1. Login form with "incorrect password" or
2. Main page of the member area
Environment:
MSIE 9.5b2/Windows7
What I did:
1. Opened http://www.something.com/
2. Clicked on "login", saw the login form
3. Entered "egor" in the login field and correct password in the password field, clicked on "enter" button
What I have got:
White page, URL in the address bar is http://www.something.com/checklogin
What I have expected:
1. Login form with "incorrect password" or
2. Main page of the member area
Environment:
MSIE 9.5b2/Windows7
A bug that was noticed by the system administrator:
Exim died
What I did:
Ran /etc/init.d/exim4 restart on server lopata.something.com
What I have got:
touch: `/var/lib/exim4′: directory not found
What I have expected:
exim restarted, standard success message on stdout
What I did:
Ran /etc/init.d/exim4 restart on server lopata.something.com
What I have got:
touch: `/var/lib/exim4′: directory not found
What I have expected:
exim restarted, standard success message on stdout
A bug that was noticed by a programmer:
dbPeerAccess->version() got broken
What I did:
What I have got:
What I have expected:
Environment:
What I did:
use dbPeerAccessor;
use DBI;
my $dbh = DBI->connect("dbi:mysql:telme", "telme", "telme");
my $dbAccess = $dbPeerAccessor->new($dbh);
What I have got:
$dbAccess->version() == undef
What I have expected:
$dbAccess->version() == "1.3.0″;
Environment:
trust:htdocs egor$ mysql -utelme -ptelme telme
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 302
Server version: 5.0.51 Source distribution
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer.
mysql>
BTW, this kind of bugreports (on the code) you can report as a single piece of code (a test case), just like that:
use dbPeerAccessor;
use DBI;
my $dbh = DBI->connect("dbi:mysql:telme", "telme", "telme");
my $dbAccess = $dbPeerAccessor->new($dbh);
printf("dbPeerInstance is %s\n",
defined $dbAccess->version() ? "defined" : "undefined");
A bug that was noticed by the project manager:
"Forgot password" must work in a different way
What I did:
1. Opened http://www.something.com/
2. Clicked into "forgot password"
3. I see a form with the email field
4. I typed in my email that is existent in the database and which belongs to my username
5. Clicked "submit"
What I have got:
1. "The new password has been emailed to you"
2. A letter came in which, truly, the new password was delivered in plain text
3. This new password indeed works, the old password doesn't let me in anymore
What I have expected:
According to our specs, the email must contain a link to the "generate new password" page and not the newly generated password.
What I did:
1. Opened http://www.something.com/
2. Clicked into "forgot password"
3. I see a form with the email field
4. I typed in my email that is existent in the database and which belongs to my username
5. Clicked "submit"
What I have got:
1. "The new password has been emailed to you"
2. A letter came in which, truly, the new password was delivered in plain text
3. This new password indeed works, the old password doesn't let me in anymore
What I have expected:
According to our specs, the email must contain a link to the "generate new password" page and not the newly generated password.
A bug that was noticed by the CFO:
Why such expensive chairs?
In the report dated 01.04.2008 I see the "Furniture" clause and there the "Chairs for new programmers" item. 2 pcs bought at $1,400 each. Well, we assumed that Carl will order chairs priced no more than $200/pcs for his department, and definitely not a fourteen hundreds bucks, no? Please argument such an expense. Please do not referer to the April 1st.
In the report dated 01.04.2008 I see the "Furniture" clause and there the "Chairs for new programmers" item. 2 pcs bought at $1,400 each. Well, we assumed that Carl will order chairs priced no more than $200/pcs for his department, and definitely not a fourteen hundreds bucks, no? Please argument such an expense. Please do not referer to the April 1st.
As you see, it's not required that the text must look exactly as the form. Is it important that all the information which is required by the form is there. The programmer's and CTO's bugs are pefrect examples of how you can report the bug in different ways still with the comply to the form.
No pain — no gain
There is one more thing that you have to be warned of before you establish this bug report form (which is in fact a business process) in your team. You see, nobody wants to feel himself stupid; moreover, the person who discovered the bug will immediately feel himself smarter than the programmer who let this bug happen. Even unintentionally. So being in context of the newly discovered bug, the user will feel absolutely comfortable filling a bugreport like «this piece of shit doesn't work, period».
This is why enforcing this form in a team will be painful step. The pain relief will come quickly in an software development team, because software developers tend to be highly rational people; not so in other industries. Those kind of bugreports bring transparency to the working process and transparency require additional efforts which nobody wants to put in. And more, not all people can really understand what is the context thing anyway and why his mate doesn't have a clue what's he talking about, see, the program just doesn't work, why don't you see it, look, it's so simple.
Once I had to introduce this business process in a company where only a single manager was allowed to talk to the customers. She collected feedback and requests from users and filled up bug reports for the software team. As most of the non-technical personnel, her mind process wasn't so straightforward like we programmers have. So when the bugreport form was enforced, she cried, she screamed, she promised to kill me and all my people, she took cold showers, then she cried, she cried and then she cried. I was so sorry for her! She was frank in her tears. She really felt betraded. But still I was heartless and reject bugreports one after another until she learned this simple form. It took her three days to calm down. Two more days and she was comfortable submitting perfect bugreports. And yet two more days and she didn't turn back to her past practices. She became a wholehearted evangelist of this approach in the company. Her productivity raised up considerally and customers's issues were resolved quicker from that moment.
So to conclude I suggest you the following. Introduction of this bug reporting approach in the company must be enforced. It must be enforced by the team leader and only when the team leader truly knows why he wants this approach and what are it's benefits for the business.
About the author
My name is Egor Egorov. I am a project manager, a business analyst and a programmer with over 15 years of experience. I specialise in backend development of mass serving systems.
My name is Egor Egorov. I am a project manager, a business analyst and a programmer with over 15 years of experience. I specialise in backend development of mass serving systems.
