Proposed changes to workflow bug management

[Brian Murray]

On Mon, May 26, 2008 at 04:56:46PM -0700, Jordan Mantha wrote:
> On Mon, May 26, 2008 at 12:45 PM, Scott Kitterman <ubuntu at kitterman.com>
> wrote:
> <snip>
> 
> > Problem: A number of development processes use bugs to track work flow.
> >  This
> > class of bugs has unique rules and bug status has different meanings (even
> > perhaps different meanings depending on the phase of the development
> > cycle).
> > This class of bugs rarely benefits from work by bugsquad (non-developer)
> > triage efforts.  In some cases changes may actually be harmful.
> >
> 
> While this is true, workflow bugs are pretty much the easiest to work with
> because they have well-defined subscriptions, statuses, etc. do to being
> used in our processes.
> 
> 
> >
> > After some review, it appears that there is no easy way for bugsquad
> > members
> > to reliably identify such bugs that is consistent with their normal work
> > flow.  The most reliable method to identify them by team subscription (e.g.
> > ubuntu-archive, ubuntu-mir, ubuntun-universe-sponsors, etc.), but this is
> > not
> > something generally used in the triage process.
> >
> 
> I'm very curious as to why that is. It seems much easier to me to check bug
> subscribers (for archive, sru, or release bugs) or titles
> (merges/syncs/removals) than making sure a bug is reported against the right
> package and has all the information needed for developers to start working
> (typical triaging task, no?).

There are a great number of things a new Launchpad user needs to learn
to be able to work with bug reports and we have not been teaching about
the subscriber's portlet or some of the others.  Also trying to checking
if one of approximately six special teams is subscribed seems
suboptimal.  Additionally, the subscribers portlet is not expanded by
default, as far as I can tell.  

However, I'm under the impression that the visibility of subscribers
will change in Launchpad sometime soon.  So we could try having bug
triagers parse the subscriber list and see how well that works.

<snip> 
> > Advantages:  No bugsquad process changes required (the current "Special
> > types
> > of bugs" section would be changed to indicate ubuntu-bugcontrol should be
> > asked to see if the bug needs to be added to the workflow bug class if a
> > triager believes they have found one not appropriately marked).  The risk
> > of
> > inadvertent/incorrect changes by people not involved in the development
> > process would be greatly reduced.  Noise level changes appearing in the
> > bugmail stream would also be reduced.
> >
> 
> Do we particularily know that this is the case? I'm not necessarily
> rejecting it, but I've not seen an analysis that shows how often we have
> incorrect changes and what teams "offenders" belong to. Do we know that it's
> consistently triagers who are not in ubuntu-bugcontrol?

I too was curious about the quantities of workflow reports and the
frequency of incorrect changes.  Subsequently, I took the ubuntu-bugs
mailing list archive for the past couple of months and ran some queries
on it.  I came up with the following:

2008-05 (ending 14 May)
3557 New bug reports
  76 Merge requests
  81 New package requests (excessive due to one user)
 159 Sync requests
   3 Removal requests
 319 Total workflow reports

Of those 319 I was able to find 7 that were mishandled and 4 of those
were by the same person.

2008-04 
8833 New bug reports
   2 Merge requests
  23 New package requests
  77 Sync requests
   4 Removal requests
   1 Promotion requests
 107 Total workflow reports

Of those 107 I was unable to find any that were incorrectly handled.

I'm happy to perform further research and publish the results if anyone
is interested.

<snip> 
> >
> > ubuntu-bugcontrol would have access to these bugs (ubuntu-dev is a member
> > of
> > this team, so it gives all developers access).  universe-contributors
> > should
> > also be a member.  One open question is, should there be an open team that
> > has access to these bugs?
> 
> 
> The ubuntu-bugcontrol team has something like 200 members, a number of which
> are triagers as far as I can tell. It's not nearly as big as ubuntu-bugsquad
> (1100 members!) but I see it as primarily as an advanced triaging team. Is
> the idea here to keep the workflow bugs from inexperienced triagers but
> allow the more experienced ones to "figure it out". It would seem to me to
> be rather confusing and counter-productive to have different people in the
> triaging team seeing different bugs in search results, etc.
> 
> Overall, this feels like an attempt to solve a problem with the
> presupposition that the triaging team can't change anything. It also feels
> like a classic case of solving social problems with technical solutions. The
> cases I've seen primarily boiled down to triagers taking advice badly and
> developers being unforgiving towards people who don't know better. Is it
> really so inconceivable to to have clear and concise documentation that
> helps new people figure out what workflow bugs look like? Is it really so
> unreasonable to expect people to be forgiving toward new people who make
> mistakes and people to actually learn from mistakes.

Given the scope of the problem in combination with the challenges in
determining whether a bug has special subscribers or not, I think the
best way to proceed is documenting and teaching how to identify these
bug reports.  However, in the event that mistakes are made I think
forgiveness and tact are the key.
 
-- 
Brian Murray                                                 @ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20080527/a9183f18/attachment-0001.pgp