[Bug 1990856] Re: Does not consider all versions in Packages files

[Dan Watkins]

I've created a reproducer (attached, also at
https://pastebin.ubuntu.com/p/H6dTzGBTGp/).  This is a small Flask app
which wraps archive.ubuntu.com, and modifies the focal main Packages
file to include the e2fsprogs and libext2fs2 packages currently in
focal-updates (as well as rewriting other metadata as required).  It can
be run like this (assuming it's written to repro.py):

FLASK_APP=repro.py flask run

Once it's running, the reported debootstrap error can be reproduced by
attempting to bootstrap from this "mirror" (without GPG verification:
that obviously won't work!):

debootstrap --keep-debootstrap-dir --variant minbase --no-check-gpg
focal bootstrap http://127.0.0.1:5000/ubuntu/

You should see this fail with:

W: Failure trying to run: chroot "/root/bootstrap" dpkg --force-
overwrite --force-confold --skip-same-version --install

This command should have a package name appended to it.  debootstrap has
installed the older version of libext2fs2 into the chroot (because it
failed to find the newer one), but it has written
`/var/lib/dpkg/available` such that `dpkg --predep-package` does detect
the newer version and returns "libext2fs2" as a package that needs
installing to satisfy Pre-Depends.  debootstrap filters out any already-
installed packages from that list, and that's why we don't have any
package name appended here.

Fixing the determination logic in pkgdetails_field so that debootstrap
installs the most recent packages initially fixes this problem: the Pre-
Depends on the newer package is satisfied, so `dpkg --predep-package`
doesn't return it.

** Attachment added: "repro.py"
   https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/1990856/+attachment/5619235/+files/repro.py

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to debootstrap in Ubuntu.
https://bugs.launchpad.net/bugs/1990856

Title:
  Does not consider all versions in Packages files

Status in debootstrap package in Ubuntu:
  New

Bug description:
  Some implementations of apt mirror metadata generation generate
  Packages files which are not alpha-sorted by package name.  apt and
  britney2 handle these files without issue, but debootstrap does not:
  it will only consider the first contiguous run of stanzas for a
  package (taking the last stanza as the latest).

  When running debootstrap against a mirror with such Packages files,
  debootstrap can fail to resolve versioned dependencies which _are_
  present in the Packages file, if the satisfying package version isn't
  within the first contiguous run of stanzas for that package.  This
  leads to avoidable bootstrap failures.  (The specific case we hit:
  each e2fsprogs package Pre-Depends on the libext2fs2 package with the
  same version: `dpkg --predep-package` finds the newer e2fsprogs
  version and reports that libext2fs2 needs installing, but debootstrap
  has already installed (the old) libext2fs2 so errors out.)

  The problem lies in the pkgdetails_field function (which is
  implemented in Perl):
  https://git.launchpad.net/ubuntu/+source/debootstrap/tree/functions#n1439.
  It takes a list of expected packages and, once it has found a version
  for all of them, exits (even if there is more of the Packages file to
  process).  It does already have special-casing for multiple versions
  of the same package, but only if they are a contiguous run.  (That
  behaviour was introduced by cjwatson in 1.0.38[0] to fix
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=649319: that bug
  report also has useful background info on why multiple versions of a
  package in metadata should be supported).

  [0] https://salsa.debian.org/installer-
  team/debootstrap/-/commit/0fbf86aa8fbcd06cb62fddddcfd4605fef2ee8b2

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/1990856/+subscriptions