Herd 3 ISO testing post-mortem

[Matt Zimmerman]

On Sun, Feb 04, 2007 at 07:05:07PM +0100, Henrik Nilsen Omma wrote:
> Feedback:
> 
> * It should be easy to see what tests have been performed already, 
> ideally on a single page as was possible with the wiki. This can 
> probably be done reasonably well using the status field on each test 
> report and filtering with tags (so those wishing to track Kubuntu 
> results can filter on that tag)

This is definitely important, as we need to be able to get an overview of
whether we have sufficient test coverage to say that a test cycle is
complete.

> * A dedicated mailing list used for announcements and ISO testing bug 
> mail -- Those subscribed would get messages such as 'Start testing now' 
> and 'The amd64 images have been rebuilt. Test again.' without having to 
> check the web page. The list traffic could also be greped for things 
> like bug numbers.

Why a dedicated list, rather than one of the existing lists?

> * Users inevitably confuse these tracker bugs with real distro bugs and 
> file real bugs for ubuntu in the ubuntu-iso-tests product. That's not a 
> major concern as long s the number is small and they can be reassigned 
> to distro tasks.

I believe there's a Launchpad wishlist item to have product-specific
information displayed when reporting a bug; if that gets implemented, we
could presumably use it to display a prominent warning about this.

I agree that this is a minor issue so long as the volume remains low.  As
testers join the team, they should become better informed about this.  I
assume it's already documented?

-- 
 - mdz