The changing nature of bug reports

[Henrik Nilsen Omma]

Hi,

Just a point for discussion.

With the new automated bug reporting tools like apport we are seeing a 
new breed of bug report[er] from what has been typical before. We should 
be aware that this is happening and adjust the way we deal with bugs 
accordingly. This should include an active approach to educating users 
and we might also consider splitting bug reports into two types: with 
high and low user interaction.

In the free software world there is an unwritten agreement between users 
and developers WRT bug reports. The user will submit a useful report 
detailing the steps taken, expected and actual result, etc. and will 
follow up with more information as needed. The developer will in turn do 
his best to fix the problem, and provide the fix for free.

We are now seeing a number bug reports where this agreement is clearly 
not in the mind of the reporter. In some sense they are more like 
support requests with statements of the form "I did X which broke Y. 
Please help me fix it." However with an accompanying stacktrace from a 
crash it is obviously a bug that should be followed up. A bug triager 
then comes along asking for some log files with the rather terse "Please 
attach /var/log/foo. Thanks." The reporter may then ask for help on how 
to find that log and then how to attach it, by which point the bug may 
be getting old. We also see "I've fixed it now [with a new hard drive], 
[using thunderbird instead], [gone back to Dapper/XP], [whatever]. I'm 
OK now, thanks guys!", implying that the subscriber has no intention of 
providing more information once his problem is solved. But for the 
developer it may still be an interesting bug that should be followed up 
so it is left open.

I'm sure these reports have always been around but with a growing user 
base they will become more common and in the past one could brush these 
off as the user was not playing by the rules. However with Ubuntu we are 
appealing to a more casual group of computer users and we are actively 
encouraging them to submit bugs via dialogs that appear on their 
desktop. We then feed them into a bug tracking system that still implies 
the unwritten agreement in many ways, rounded a bit at the edges by the CoC.

When you experience a crash in Windows, which is what most new users 
will be used to, a dialog also pops up and you are given the choice of 
reporting the crash or not. If you accept  some logs are uploaded and 
you're done. In our case you are taken to a page on launchpad where you 
first have to register by giving your email address and then write a 
description of what happened. Some users may give up at that point, 
which means we are loosing reports. Others will continue without quite 
knowing how it works. We've seen "I'm here because the crash dialog told 
me to go here, but I don't know what I'm supposed to do." Such users may 
also not be prepared for the bugmail they have just signed up for (esp. 
if the bug gets duped) and may form a bad impression of Ubuntu based on 
that.

So what should we do? Getting more info gathered automatically with 
apport will of course also help, but we should also try to improve the 
quality of the written reports and the user-triager/developer interaction.

I think this proposed feature: 
https://launchpad.net/products/malone/+bug/43893 will probably help, 
where we can give specific instructions about how to file a good bug for 
a given package. So we can explain (with screenshots!) how to upload 
/var/log/foo right from the start, saving the round trips.

I think we should go further than that too though. If the user reporting 
the bug has no Launchpad account arrives at a new bug page for a 
product, she should be presented with two choices: "Take my automated 
report and leave me alone." or "Yes, I want to be involved in following 
this up further."

Or IOW: "The data from your crash has been added your our database as an 
anonymous entry. If you would like to add further information about the 
circumstances of the crash and follow the work to solve the issue (via 
email reports) please click here to register for a Launchpad account."

The link would take the user to a page with some information about the 
bug flow process, what feedback is expected, some guidance of how to 
upload attachments, etc.

We would then get two types of bug reports, one with just log and trace 
data which we could run some batch analysis on and one set of of higher 
quality reports where the triagers and developers can get into a 
dialogue with the user with less risk of that user just walking off and 
both parties getting annoyed.

Henrik