PPA update process (was Re: No bazaar 2.1.0b1 release in beta paa)

[Martin Pool]

2009/11/26 Ian Clatworthy <ian.clatworthy at canonical.com>:
> Martin Pool wrote:
>
>> I think the ideal would be that packages are uploaded as soon as
>> possibly after the release, but that there's also a ppa that holds a
>> consistent set.  Possibly we could have a bot that waits for a staging
>> ppa to get to a consistent state, and then copies everything into the
>> ppa people are actually meant to use?
>
> I can't comment on the details but that sounds the right approach
> conceptually.
>
> IMO, users have a strong expectation that using one of our PPAs will
> give them a *consistent* set of plugins. If they simply wanted the
> latest stuff without the consistency debian packaging provides, they'd
> use 'bzr multi-pull' across their plugins directory each morning.

I looked into this a bit this afternoon.  There are a few things we could do:

0- I think the main thing is to get this a bit away from being a
single person's laborious job to something that anyone can pick up if
it breaks, and that

1- Building a bot that checks whether a set of updates will leave the
archive self-consistent and installable is totally possible, but not
trivial, for somewhat silly reasons.

(Silly reasons include: apt itself doesn't let you talk about "the
version of bzr _from this archive_ and nor does python-apt; you can
check that the set of overall available packages is consistent but
that's not the same thing.  Bugs like
<https://bugs.edge.launchpad.net/ubuntu/+source/devscripts/+bug/434987>.
 There are tools like dak that have some of the logic we need, but
none obviously allow it to be pulled out and reused.  For some reason
setting APT::Dir::RootDir doesn't stop it seeing the system-wide
packages, so you seem to need a real chroot for each environment you
want to even think about installation.)

These could be overcome, but I didn't succeed this afternoon.  So that
made me wonder whether this was really necessary.

2- Maybe it's better all around to get a consistent archive not by
building archive-specific tools, but by trying to keep the source
packages compatible.  After all, that will also help with packaging
for all other platforms, and people installing directly from the
branches.

We already have one important brick, which is a stable branch, where
there should be no API breakage.  On the beta branch, perhaps we can
encourage plugins to just check for 2.1.x and not the particular beta
(perhaps they do already?) and to assume that things will stay
compatible?  Then there won't be any breakage unless there really is a
bug because of changes in bzr.  If we do see that bug then perhaps we
can jointly get an update into the package and get it rolled out
again.

Concretely, the policy would be: it's always ok for packagers to
upload a later bzr into the archive: if it's in the stable series it
shouldn't break anything; and if it's in the beta series it shouldn't
conflict with plugins.  It may discover bugs in them, but that's kind
of the point.

3- I looked into automatically copying packages across distroseries,
and there is a snag there: you can only copy binaries, but for things
with library dependencies rather than being just pure python the
binaries may not be safe to copy.  That's a bit annoying.  This
restriction is (I think) because the whole ppa shares one package
pool, so you can have two distros pointing to one binary, but each
version name must be unique to a binary.

One solution is to continue doing what we do now, and make multiple
uploads tagged with the distroseries (<bzrtools -
2.0.1-1~bazaar1~karmic>) into a single archive.  This can potentially
be automated but it's a bit messy.

There is another option which is to have multiple archives, one per
distroseries.  We could then have bzrtools-2.0.1~bazaar1 be copied as
source into each of these, then rebuilt appropriately.  This could be
scripted fairly straightforwardly through an lp api client.  This
actually seems superior except that it clashes a bit with the api that
launchpad presents for ppas, which is that they have multiple
distroseries within them.  It would be a change in ppa name to which
users would have to adjust.

4- A babune-like warning system for problems with either installation
or selftest of software in the ppa still seems useful - there are more
possible combinations than are likely to be covered by core developer
use of PPAs, because many of us run from trunk or are on non-Ubuntu
platforms.  That's what I was originally setting out for with the
smoketest script - installing into a snapshot of a fresh OS install in
a chroot.  However, while this is useful, it's perhaps not really
going to unblock developers, and we may get similar feedback from
people just using the ppa.

-- 
Martin <http://launchpad.net/~mbp/>