[Bug 2136906] Re: python3-urllib3 in 24.04 is now incompatible with shipped python3-zstandard

[Hlib Korzhynskyy]

Sorry for the inconvenience once again. This issue should now be
properly fixed in version 2.0.7-1ubuntu0.6. Let me know if you are still
encountering issues.

I have tried to reproduce the issue using urlwatch in a virtual machine
but everything seemed to be working normally on my end. If you keep
encountering issues even after this update, please send me the steps
that you are performing so that I can properly debug it. Thanks!

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to python-urllib3 in Ubuntu.
https://bugs.launchpad.net/bugs/2136906

Title:
  python3-urllib3 in 24.04 is now incompatible with shipped
  python3-zstandard

Status in python-urllib3 package in Ubuntu:
  Fix Released

Bug description:
  This seems to be a direct result of a backport of upstream fixes per
  2.0.7-1ubuntu0.3:

    * SECURITY UPDATE: Denial of service due to decompression bomb.
      - debian/patches/CVE-2025-66471.patch: Fix decompression bomb in
        src/urllib3/response.py. Add tests in test/test_response.py.
      - debian/patches/CVE-2025-66471-post1.patch: Remove brotli version warning
        due to intrusive backport for brotli fixes and upstream version warning
        not being appropriate for distro backporting.
      - CVE-2025-66471

  Specifically the patch debian/patches/CVE-2025-66471.patch.

  When urllib3 attempts to decode a zstd response from a remote server,
  the following stack trace is now seen (this is from 'urlwatch', which
  utilizes urllib3). This is because urllib3 is expecting 2 things from
  python3-zstandard which don't exist in 0.22.0-1build1.

  1. decompress() taking an optional max_length parameter; it expects only 1.
  2. The decoder object having a "needs_input" parameter.

  Traceback (most recent call last):
    File "/usr/bin/urlwatch", line 33, in <module>
      sys.exit(load_entry_point('urlwatch==2.28', 'console_scripts', 'urlwatch')())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/urlwatch/cli.py", line 109, in main
      urlwatch_command.run()
    File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 458, in run
      self.handle_actions()
    File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 256, in handle_actions
      sys.exit(self.test_filter(self.urlwatch_config.test_filter))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/urlwatch/command.py", line 139, in test_filter
      raise job_state.exception
    File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 113, in process
      data = self.job.retrieve(self)
             ^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/urlwatch/jobs.py", line 327, in retrieve
      response = requests.request(url=self.url,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request
      return session.request(method=method, url=url, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/requests/sessions.py", line 589, in request
      resp = self.send(prep, **send_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/requests/sessions.py", line 747, in send
      r.content
    File "/usr/lib/python3/dist-packages/requests/models.py", line 899, in content
      self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/requests/models.py", line 816, in generate
      yield from self.raw.stream(chunk_size, decode_content=True)
    File "/usr/lib/python3/dist-packages/urllib3/response.py", line 1123, in stream
      yield from self.read_chunked(amt, decode_content=decode_content)
    File "/usr/lib/python3/dist-packages/urllib3/response.py", line 1271, in read_chunked
      decoded = self._decode(
                ^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/urllib3/response.py", line 613, in _decode
      data = self._decoder.decompress(data, max_length=max_length)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib/python3/dist-packages/urllib3/response.py", line 267, in decompress
      part = self._obj.decompress(data, max_length=max_length)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  TypeError: decompress() takes at most 1 argument (2 given)

  
  I was able to workaround the issue by making the following change (which disables the "fix"):

  --- /storage/root/_btrbk/@.20251219//usr/lib/python3/dist-packages/urllib3/response.py	2025-12-10 14:26:11.000000000 -0500
  +++ /usr/lib/python3/dist-packages/urllib3/response.py	2025-12-19 22:02:38.609540775 -0500
  @@ -264,7 +264,7 @@
               if self._obj.eof:
                   data = self._obj.unused_data + data
                   self._obj = zstd.ZstdDecompressor().decompressobj()
  -            part = self._obj.decompress(data, max_length=max_length)
  +            part = self._obj.decompress(data)
               length = len(part)
               data_parts = [part]
               # Every loop iteration is supposed to read data from a separate frame.
  @@ -294,7 +294,7 @@

           @property
           def has_unconsumed_tail(self) -> bool:
  -            return not (self._obj.needs_input or self._obj.eof) or bool(
  +            return not self._obj.eof or bool(
                   self._obj.unused_data
               )

  Some info:
  Ubuntu 24.04.3 amd64
  python3-urllib3 2.0.7-1ubuntu0.3
  python3-zstandard 0.22.0-1build1

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-urllib3/+bug/2136906/+subscriptions