[Bug 1950095] Re: [github] 20.04: Apt fails to download URLs with non-encoded querystrings

[David Kalnischkies]

"minimal potential for causing regressions" is a big claim given I had
to fix regressions in later commits like
149b23c2b9697bc262c0af1934c7a3f6114d903f and
2b0369a5d1673d9e40f2af4db7677b040a26ee58. There might be more, that is
just what I remember directly. It is certainly not the most complicated
code in the world, but it's quite a bit of it as I was not trying for
minimal, but instead maximized for forward and backward compat.

(Disclaimer: I am the upstream author of the patch set in question. Not
involved enough with Ubuntu to know and/or predict if this qualifies or
not for backport, so not commenting on that part. Pretty sure Debian
would refuse if we tried including that in a stable update through).

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1950095

Title:
  [github] 20.04: Apt fails to download URLs with non-encoded
  querystrings

Status in apt package in Ubuntu:
  Confirmed

Bug description:
  I've just helped a group of Ubuntu 20.04 users with Microsoft Surface
  devices. They rely upon a github repository releases pocket as the apt
  archive. Those users recently hit a bug "Ubuntu: Apt update fails with
  Error 401 Unauthorized" [1]

  The initial simple Github URL gets a Location: redirect to a complex
  URL with un-escaped query-string. The resulting complex URL causes apt
  to fail to fetch the resource.

  $ sudo apt upgrade -y
  Err:1 https://pkg.surfacelinux.com/debian release/main amd64 libwacom-surface amd64 1.12-2
    401  Unauthorized [IP: 185.199.110.133 443]

  The URL can be manually corrected. One of the Surface users provided
  this example:

  bad:  https://objects.githubusercontent.com/github-production-release-
  asset-2e65be/139604852/86019e52-7bfa-4bc6-8cc1-52147027aee6?X-Amz-
  Algorithm=AWS4-HMAC-SHA256&X-Amz-
  Credential=AKIAIWNJYAX4CSVEH53A/20211105/us-
  east-1/s3/aws4_request&X-Amz-Date=20211105T161053Z&X-Amz-
  Expires=300&X-Amz-
  Signature=2bc0c28946db539ada250b1030c37249dae909d73a68c90b5e7bfe7fecd5d347&X-Amz-
  SignedHeaders=host&actor_id=0&key_id=0&repo_id=139604852&response-
  content-disposition=attachment; filename=libwacom-
  surface_1.12-2_amd64.deb&response-content-type=application/octet-
  stream

  good: https://objects.githubusercontent.com/github-production-release-
  asset-2e65be/139604852/86019e52-7bfa-4bc6-8cc1-52147027aee6?X-Amz-
  Algorithm=AWS4-HMAC-SHA256&X-Amz-
  Credential=AKIAIWNJYAX4CSVEH53A%2F20211105%2Fus-
  east-1%2Fs3%2Faws4_request&X-Amz-Date=20211105T160935Z&X-Amz-
  Expires=300&X-Amz-
  Signature=44d9307e66dfb5b3672ee0082b8801ad2532ac4b6be61c3442fb265ffce72852&X-Amz-
  SignedHeaders=host&actor_id=0&key_id=0&repo_id=139604852&response-
  content-disposition=attachment%3B%20filename%3Dlibwacom-
  surface_1.12-2_amd64.deb&response-content-type=application%2Foctet-
  stream

  I found this also affects a github repository I recently added for
  Zotero on 20.04.

  The problem is fixed in later versions of apt upstream, in Debian and
  Ubuntu releases. I cherry-picked the 4 commits [3] and provided a
  package for Focal in my PPA [2] which multiple users have reported (in
  [1]) solves the issue.

  It would be really good to get those patches included in 20.04 as an
  SRU.

  As the code is in later versions of apt and is focused on the URL
  encoding only it has minimal potential for causing regressions.

  [1] https://github.com/linux-surface/linux-surface/issues/625

  [2] https://launchpad.net/~tj/+archive/ubuntu/bugfixes

  [3] https://salsa.debian.org/apt-
  team/apt/-/commit/06ec0067057e0578f3bc515f6a97d6a9d70824f6

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1950095/+subscriptions