[Bug 1991606] Re: Invalid PEP440 package version breaking setuptools >= 66

Fri Jun 30 00:09:06 UTC 2023

Sorry, I'm having a hard time understanding how the dput check shows
that distro-info and python-debian are fixed. I'm guessing dput ends up
importing the other two?

Shouldn't python-debian be a block-proposed-kinetic update, since it's
currently unaffected due to its version being still fine regarding
PEP440 rules??

In any case, I did a no-change rebuild locally just to add the "build1"
suffix to the version and break the PEP440 check, and I think this is a
simpler test case:

$ virtualenv  --system-site-packages myenv
$ myenv/bin/python -m pip install -U setuptools
$ myenv/bin/python -c "import pkg_resources; pkg_resources.require('@BROKEN-PACKAGE@')"

For example, with my rebuild python-debian:
$ myenv/bin/python -c "import pkg_resources; pkg_resources.require('python-debian')"
<string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/ubuntu/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 968, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/ubuntu/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 829, in resolve
    dist = self._resolve_dist(
  File "/home/ubuntu/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 854, in _resolve_dist
    if dist is None or (dist not in req and replace_conflicting):
  File "/home/ubuntu/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3205, in __contains__
    return self.specifier.contains(item, prereleases=True)
  File "/home/ubuntu/myenv/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/specifiers.py", line 905, in contains
    item = Version(item)
  File "/home/ubuntu/myenv/lib/python3.10/site-packages/pkg_resources/_vendor/packaging/version.py", line 198, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '0.1.46build1'

And with your package in proposed, the crash does not happen:

ubuntu at k-pep440:~$ myenv/bin/python -c "import pkg_resources; pkg_resources.require('python-debian')"
<string>:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

ubuntu at k-pep440:~$ dpkg -s python3-debian | grep ^Version
Version: 0.1.46ubuntu1

The dep8 migration-reference/0 runs for devscripts did fail as expected, but the Pending SRU report hasn't picked that up yet, so I'm no keen on releasing python-debian for kinetic just yet. And for jammy and earlier, we currently have this issue with britney[1] that needs resolving first.

1. https://lists.ubuntu.com/archives/ubuntu-release/2023-June/005670.html

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to gpgme1.0 in Ubuntu.
https://bugs.launchpad.net/bugs/1991606

Title:
  Invalid PEP440 package version breaking setuptools >= 66

Status in devscripts package in Ubuntu:
  Fix Released
Status in distro-info package in Ubuntu:
  Fix Released
Status in dput package in Ubuntu:
  New
Status in drslib package in Ubuntu:
  New
Status in duecredit package in Ubuntu:
  Fix Released
Status in gpgme1.0 package in Ubuntu:
  Fix Released
Status in python-debian package in Ubuntu:
  Fix Released
Status in reportbug package in Ubuntu:
  Fix Released
Status in ubuntu-dev-tools package in Ubuntu:
  Fix Released
Status in update-manager package in Ubuntu:
  Fix Released
Status in devscripts source package in Bionic:
  New
Status in distro-info source package in Bionic:
  New
Status in dput source package in Bionic:
  New
Status in drslib source package in Bionic:
  Invalid
Status in duecredit source package in Bionic:
  New
Status in gpgme1.0 source package in Bionic:
  New
Status in python-debian source package in Bionic:
  Invalid
Status in reportbug source package in Bionic:
  New
Status in ubuntu-dev-tools source package in Bionic:
  New
Status in update-manager source package in Bionic:
  New
Status in devscripts source package in Focal:
  New
Status in distro-info source package in Focal:
  New
Status in dput source package in Focal:
  New
Status in drslib source package in Focal:
  New
Status in duecredit source package in Focal:
  New
Status in gpgme1.0 source package in Focal:
  New
Status in python-debian source package in Focal:
  Fix Committed
Status in reportbug source package in Focal:
  New
Status in ubuntu-dev-tools source package in Focal:
  Fix Released
Status in update-manager source package in Focal:
  New
Status in devscripts source package in Jammy:
  New
Status in distro-info source package in Jammy:
  Fix Committed
Status in dput source package in Jammy:
  New
Status in drslib source package in Jammy:
  New
Status in duecredit source package in Jammy:
  New
Status in gpgme1.0 source package in Jammy:
  New
Status in python-debian source package in Jammy:
  Fix Committed
Status in reportbug source package in Jammy:
  New
Status in ubuntu-dev-tools source package in Jammy:
  Invalid
Status in update-manager source package in Jammy:
  New
Status in devscripts source package in Kinetic:
  New
Status in distro-info source package in Kinetic:
  Fix Released
Status in dput source package in Kinetic:
  New
Status in drslib source package in Kinetic:
  New
Status in duecredit source package in Kinetic:
  New
Status in gpgme1.0 source package in Kinetic:
  Fix Released
Status in python-debian source package in Kinetic:
  Fix Committed
Status in reportbug source package in Kinetic:
  New
Status in ubuntu-dev-tools source package in Kinetic:
  Invalid
Status in update-manager source package in Kinetic:
  New

Bug description:
  [ Impact ]

  With setuptools 66, the versions of all packages visible in the Python
  environment *must* obey PEP440 <https://peps.python.org/pep-0440/>.
  Otherwise, attempts to use pip to install a package with a setup.py-
  based build system, or other attempts to use the `pkg-resources`
  module, can produce errors like this:

    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 844, in _resolve_dist
      env = Environment(self.entries)
    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1044, in __init__
      self.scan(search_path)
    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1077, in scan
      self.add(dist)
    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1096, in add
      dists.sort(key=operator.attrgetter('hashcmp'), reverse=True)
    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2631, in hashcmp
      self.parsed_version,
    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2678, in parsed_version
      self._parsed_version = parse_version(self.version)
    File "/builds/databiosphere/toil/venv/lib/python3.9/site-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__
      raise InvalidVersion(f"Invalid version: '{version}'")
  pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '0.23ubuntu1'

  The official opinion of the setuptools maintainers seems to be that
  version strings of this form haven't *really* been allowed since about
  2014, and distributions need to change their package version naming
  scheme for Python packages they install, so that the resulting version
  strings obey PEP440. See for example
  <https://github.com/pypa/setuptools/issues/3772#issuecomment-1384342813>.

  suffix 1build1 is invalid.

  Some python building tools, that verifies if version strings are
  compatible with PEP440, are failing.

  Example: python poetry: Invalid PEP 440 version: '1.1build1'

  [ Test Plan ]
  1.
  $ mkdir /tmp/lp.1991606
  $ export HOME=/tmp/lp.1991606
  $ sudo apt install dput python3-pip
  $ pip3 install -U setuptools
  $ sudo apt install <broken package>
  $ dput

  This will produce a backtrace.

  2. Enable -proposed
  3. Upgrade <broken package> from proposed
  4. dput

  This will succeed.

  [ Where problems could occur ]

  The fix touches only setup.py which is only used for installing the
  package. So there should be no effect on the installed package (except
  for exposing a different version). But there could be indirect effects
  (hidden bugs that trigger on package upgrade, etc).

  [ Other Info ]

  The related upstream bug report is
  https://github.com/pypa/setuptools/issues/3772. As of 2023-04-25 it
  has 85 comments and 108 thumbs ups.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/devscripts/+bug/1991606/+subscriptions