[Bug 1998265] Re: OverFlow error when using cnf-extractor

[Brian Murray]

I removed the try / except OverflowError handling in the production
command-not-found-extractor and was able to run the job successfully
with the new version of python-apt.

ubuntu at juju-8cacac-prod-cnf-extractor-0:/srv/cnf-extractor$ apt-cache policy python3-apt
python3-apt:
  Installed: 1.6.6
  Candidate: 1.6.6
  Version table:
 *** 1.6.6 500
        500 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     1.6.5ubuntu0.7 500
        500 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     1.6.5ubuntu0.5 500
        500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
     1.6.0 500
        500 http://prodstack-zone-1.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages


** Tags removed: verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to python-apt in Ubuntu.
https://bugs.launchpad.net/bugs/1998265

Title:
  OverFlow error when using cnf-extractor

Status in python-apt package in Ubuntu:
  In Progress
Status in python-apt source package in Bionic:
  Fix Committed
Status in python-apt source package in Focal:
  Fix Committed
Status in python-apt source package in Jammy:
  Fix Committed
Status in python-apt source package in Kinetic:
  Fix Committed

Bug description:
  [Impact]
  Large files can't be downloaded as we use a 32-bit integer in AcquireFile constructor, this breaks e.g. command-not-found generator:

  I'm trying to update the command-not-found indexes (bin/cnf-extract.py
  $MIRROR $suite $component $arch) on the production cnf extractor
  system (running bionic) and have encountered Tracebacks when
  generating them for Jammy and Lunar. This is causing the command not
  found indexes to be out of date for all(?) releases of Ubuntu.

  The lunar failure:

  Get:1 dotnet-sdk-6.0_6.0.111-0ubuntu3_amd64.deb [79.1 MB]
  Fetched 79.1 MB in 0s (0 B/s)
  Traceback (most recent call last):64... 7%
    File "./bin/cnf-extract.py", line 54, in <module>
      mirror_root, suite, component, arch)
    File "/srv/cnf-extractor/extractor/extractor.py", line 174, in command_not_found_extract
      debpath = pkg.candidate.fetch_binary(destdir=tmpdir)
    File "/usr/lib/python3/dist-packages/apt/package.py", line 883, in fetch_binary
      self.size, base, destfile=destfile)
  OverflowError: signed integer is greater than maximum

  I'll get more context for the Jammy failure but it wouldn't surprise
  me if it was the same package.

  [Test plan]
  I have included a test for the AcquireFile constructor to accept a large file size, and then a test that ensures it can read the file size.

  Run the autopkgtest suite.

  [Where problems could occur]
  It could fail in other places now.

  For kinetic, the package was copied in from lunar without changes.
  This upload is actually built in kinetic and hence there is additional
  regression potential from toolchain changes.

  [Other info]
  Also including some auxiliary changes as we move back to proper upstream releases to enable testing, mostly around

  d/gbp.conf, .gitlab-ci.yml - being updated to point at right places
  d/t/control - adding a missing test dependency
  data/ - updated mirror lists

  The mirror lists are standard updates; the debian/tests/control change
  is needed to enable testing on more minimal autopkgtest images (our images
  accidentally include binutils); the other changes have zero impact on the Ubuntu infra or end user side as they only run during development.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-apt/+bug/1998265/+subscriptions