[Bug 1925833] [NEW] apt-transport-https reparses URLs, breaking CloudFront signing

[Danny Sauer]

Public bug reported:

CloudFront URL signing requires that the the whole URL matches the signature, as compared to S3 URLs which prune the querystring before validating the signature. The URL re-parsing which I'm pretty sure was introduced in resolving #1651923 decodes some HTML entities which were encoded before signing. That invalidates the signature and results in permission denied errors. In my case, the equal and semicolon chars from the return-content-disposition header are being decoded, but it applies to any entity which is "optionally" encoded.
   
Personally, I think that the URL returned in a Location: redirect header should be handled as-is, and if some piece of garbage http server is generating redirects which contain spaces or are otherwise invalid, the problem lies with the web server generating invalid redirects and not in apt failing to follow broken URLs.  I say that knowing full well that the http spec says those things should be identical, so CloudFront also falls into the general "garbage" category by breaking the spec by requiring a specific format for identical characters.

Either way, things are what they are.  Rather than unnecessarily fully
reencoding the URL, I'd suggest that apt-transport-https (and presumably
the http transport as well, but I don't know) should at most just
replace the spaces with plusses or %20s to keep the original bug
resolved without breaking other stuff. :)

PS: I tried to report this against the apt-transport-https package, but
the bug tool says that doesn't exist in Ubuntu.  That's weird,
considering it's definitely a package
(https://packages.ubuntu.com/groovy/apt-transport-https), so I assume
the apt maintainer will know how to triage. Sorry.

** Affects: apt (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1925833

Title:
  apt-transport-https reparses URLs, breaking CloudFront signing

Status in apt package in Ubuntu:
  New

Bug description:
  CloudFront URL signing requires that the the whole URL matches the signature, as compared to S3 URLs which prune the querystring before validating the signature. The URL re-parsing which I'm pretty sure was introduced in resolving #1651923 decodes some HTML entities which were encoded before signing. That invalidates the signature and results in permission denied errors. In my case, the equal and semicolon chars from the return-content-disposition header are being decoded, but it applies to any entity which is "optionally" encoded.
     
  Personally, I think that the URL returned in a Location: redirect header should be handled as-is, and if some piece of garbage http server is generating redirects which contain spaces or are otherwise invalid, the problem lies with the web server generating invalid redirects and not in apt failing to follow broken URLs.  I say that knowing full well that the http spec says those things should be identical, so CloudFront also falls into the general "garbage" category by breaking the spec by requiring a specific format for identical characters.

  Either way, things are what they are.  Rather than unnecessarily fully
  reencoding the URL, I'd suggest that apt-transport-https (and
  presumably the http transport as well, but I don't know) should at
  most just replace the spaces with plusses or %20s to keep the original
  bug resolved without breaking other stuff. :)

  PS: I tried to report this against the apt-transport-https package,
  but the bug tool says that doesn't exist in Ubuntu.  That's weird,
  considering it's definitely a package
  (https://packages.ubuntu.com/groovy/apt-transport-https), so I assume
  the apt maintainer will know how to triage. Sorry.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1925833/+subscriptions