New queue tool, usable by non-Canonical archive admins

[Colin Watson]

I've written a new queue API client in lp:ubuntu-archive-tools which
replaces the old queue script on cocoplum with immediate effect, and
which can be used without ssh access.  It's been around for a week or
so, so some people have already started using it, but as of this
morning's Launchpad deployment and ubuntu-archive-tools r525 I now
consider it feature-complete.

The old queue script on cocoplum is now deprecated, and I'll be removing
it once we've shaken out any issues with the new code, so please stop
using it and switch to the new API client.  I've updated all the
documentation in ArchiveAdministration accordingly.  If you see any
timeouts or other bugs, please tell me, as once I remove the queue
script we won't have an easy workaround for those, so I need to know
about them.

The new client should behave largely identically to the old script for
the moment, but I want to note a few changes:

 * I've simplified the "override" subcommand, so it no longer requires
   explicitly saying "source" or "binary" (I don't know about anyone
   else but I could never get the old syntax right first time), and will
   automatically apply overrides to whatever queue items match.  If you
   have some reason to limit to source or binary uploads, then there are
   corresponding --source and --binary options.

 * To complement "fetch", there's a new "show-urls" subcommand, which
   just prints the URLs needed to download all the files in a queue
   item.  This is useful if the system where you have launchpadlib
   authentication set up is on a relatively slow network connection, but
   you have ssh access to a faster one where you can easily run wget and
   review source packages.

 * "fetch" now skips syncs rather than failing.

 * "fetch" now overwrites existing files rather than failing.  (I
   suspect that it would be better for it to ask for confirmation, so
   don't rely on this behaviour.)

 * The short option name for --dry-run is now -n, not -N.

 * There is no longer a -M/--no-mail option.  The API doesn't offer
   this, and Launchpad should handle all of the situations where it used
   to be needed sensibly nowadays: for example, language pack uploads
   have automatically had their notifications suppressed for some time.

 * I didn't bother implementing the -f/--file and -i/--ignore-errors
   options.  From my perspective these were really just a workaround for
   queue being very slow to start up, which is much less of a problem
   now (the typical trade-off between native Launchpad scripts and API
   clients is that native scripts are slow to start up but then get to
   access the database directly so incremental performance is better,
   while API clients are fast to start up but each query is slower).  If
   you have a realistic use case for this where just running queue
   multiple times is insufficient for some reason, then let me know or
   feel free to implement these options yourself.

There were three scripts on cocoplum that wrapped queue.  Of these, I
converted new-binary-debian-universe and kernel-overrides to the API and
added those to ubuntu-archive-tools.  I removed new-remove-duplicates on
the basis that it's much less useful with the new auto-sync system, and
that when duplicate manual uploads are involved we should be checking
all of them and picking the best rather than arbitrarily rejecting all
but the most recent (although in future Launchpad ought to reject
duplicates of new/unapproved uploads at upload time with a useful
message).

Please *do not* depend on the output format of queue in any scripts: it
is intended only for human consumption, and since it has some
infelicities (particularly truncating package names in some places, and
output that's not as copy-and-paste friendly as it could be) I'd like to
retain the option to change it in future.  If you need to do any
complicated automatic processing then you should use the API instead.
See new-binary-debian-universe and kernel-overrides or indeed queue
itself for examples, or ask me and I'll be happy to help.

Cheers,

-- 
Colin Watson                                       [cjwatson at ubuntu.com]