[Bug 1651923] Re: apt https method decodes redirect locations and sends them to the destination undecoded.
Elvis Stansvik
elvstone at gmail.com
Wed Jan 18 18:32:39 UTC 2017
"And please note: You need to upgrade apt-transport-https for testing,
just apt does nothing."
Aha, I might have forgotten to upgrade apt-transport-https... But, I
just tested, and now I'm getting:
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Err:1 http://downloads.sourceforge.net/corefonts/andale32.exe
The HTTP server sent an invalid Content-Range header
E: Failed to fetch https://netassist.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe The HTTP server sent an invalid Content-Range header
E: Download Failed
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Err:1 http://downloads.sourceforge.net/corefonts/andale32.exe
The HTTP server sent an invalid Content-Range header
E: Failed to fetch https://netcologne.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe The HTTP server sent an invalid Content-Range header
E: Download Failed
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Err:1 http://downloads.sourceforge.net/corefonts/andale32.exe
The HTTP server sent an invalid Content-Range header
E: Failed to fetch https://netcologne.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe The HTTP server sent an invalid Content-Range header
E: Download Failed
estan at newton:~$ apt-cache policy apt apt-transport-https
apt:
Installed: 1.2.19
Candidate: 1.2.19
Version table:
*** 1.2.19 500
500 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
100 /var/lib/dpkg/status
1.2.18 500
500 http://se.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
1.2.15ubuntu0.2 500
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
1.2.10ubuntu1 500
500 http://se.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
apt-transport-https:
Installed: 1.2.19
Candidate: 1.2.19
Version table:
*** 1.2.19 500
500 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
100 /var/lib/dpkg/status
1.2.18 500
500 http://se.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
1.2.15ubuntu0.2 500
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
1.2.10ubuntu1 500
500 http://se.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
estan at newton:~$
With debug output, I first got:
estan at newton:~$ /usr/lib/apt/apt-helper -o debug::acquire::https=1 -o debug::acquire::http=1 download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
0% [Working]GET /corefonts/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/corefonts/andale32.exe
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 18 Jan 2017 18:30:14 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Location: http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 178
0% [Working]GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/project/corefonts/the fonts/final/andale32.exe
HTTP/1.1 302 Found
Server: nginx
Date: Wed, 18 Jan 2017 18:30:14 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
content-disposition: attachment; filename="andale32.exe"
Set-Cookie: sf_mirror_attempt="corefonts:vorboss:the%20fonts/final/andale32.exe"; expires=120; Path=/
Location: https://vorboss.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 163
0% [Working]* Trying 5.10.152.194...
* Connected to vorboss.dl.sourceforge.net (5.10.152.194) port 443 (#0)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* found 708 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
0% [Working]* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: vorboss.dl.sourceforge.net (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=vorboss.dl.sourceforge.net
* start date: Mon, 31 Oct 2016 19:27:00 GMT
* expire date: Sun, 29 Jan 2017 19:27:00 GMT
* issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: vorboss.dl.sourceforge.net
User-Agent: Debian APT-CURL/1.0 (1.2.19)
Accept: */*
Cache-Control: max-age=0
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
< HTTP/1.1 302 Found
< Date: Wed, 18 Jan 2017 18:30:24 GMT
< Server: Apache/2.2.22 (Debian)
< Location: http://downloads.sourceforge.net/mirrorproblem?failedmirror=vorboss.dl.sourceforge.net
< Vary: Accept-Encoding
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=iso-8859-1
<
* Closing connection 0
* Issue another request to this URL: 'http://downloads.sourceforge.net/mirrorproblem?failedmirror=vorboss.dl.sourceforge.net'
* Protocol "http" not supported or disabled in libcurl
* Closing connection -1
Err:1 http://downloads.sourceforge.net/corefonts/andale32.exe
Protocol "http" not supported or disabled in libcurl
E: Failed to fetch https://vorboss.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe Protocol "http" not supported or disabled in libcurl
E: Download Failed
estan at newton:~$
But then I tried again and got:
estan at newton:~$ /usr/lib/apt/apt-helper -o debug::acquire::https=1 -o debug::acquire::http=1 download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
0% [Working]GET /corefonts/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/corefonts/andale32.exe
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 18 Jan 2017 18:30:54 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Location: http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 178
0% [Working]GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/project/corefonts/the fonts/final/andale32.exe
HTTP/1.1 302 Found
Server: nginx
Date: Wed, 18 Jan 2017 18:30:54 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
content-disposition: attachment; filename="andale32.exe"
Set-Cookie: sf_mirror_attempt="corefonts:netassist:the%20fonts/final/andale32.exe"; expires=120; Path=/
Location: https://netassist.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 165
0% [Working]* Trying 62.205.134.42...
* Connected to netassist.dl.sourceforge.net (62.205.134.42) port 443 (#0)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* found 708 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
0% [Working]* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: netassist.dl.sourceforge.net (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=netassist.dl.sourceforge.net
* start date: Thu, 05 Jan 2017 23:28:00 GMT
* expire date: Wed, 05 Apr 2017 23:28:00 GMT
* issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: netassist.dl.sourceforge.net
User-Agent: Debian APT-CURL/1.0 (1.2.19)
Accept: */*
Cache-Control: max-age=0
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
0% [Working]< HTTP/1.1 302 Moved Temporarily
< Server: nginx
< Date: Wed, 18 Jan 2017 18:50:58 GMT
< Content-Type: text/html
< Content-Length: 154
< Connection: close
< Content-Range: bytes */198384
* Failed writing header
* Closing connection 0
Err:1 http://downloads.sourceforge.net/corefonts/andale32.exe
The HTTP server sent an invalid Content-Range header
E: Failed to fetch https://netassist.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe The HTTP server sent an invalid Content-Range header
E: Download Failed
estan at newton:~$
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1651923
Title:
apt https method decodes redirect locations and sends them to the
destination undecoded.
Status in apt package in Ubuntu:
Fix Released
Status in apt source package in Xenial:
Fix Committed
Status in apt source package in Yakkety:
Fix Committed
Bug description:
[Impact]
Downloads via HTTPS fail if the URL contains a space (before yakkety only if there is no redirect from a previous space-free https URL). This breaks packages like ttf-mscorefonts-installer and various third party hosters.
[Test case]
Install/Upgrade apt-transport-https, that's where the fix is.
Check that
/usr/lib/apt/apt-helper download-file
http://kxstudio.linuxaudio.org/repo/pool/free/ardour4_4.7.0-1kxstudio1_i386.deb
test.deb
can successfully download the file (or at least start downloading it)
and does not fail early with a 505 HTTP version not supported error
message.
This problem does not occur with that file on xenial, as it first
redirects to an https URI without a space which then redirects to an
HTTPS uri with a space (http w/o space -> https w/o space -> https w/
space). In xenial, https->https redirects where handled internally by
curl.
Another test (applicable to xenial) is to install ttf-mscorefonts-
installer.
[Regression potential]
The added code is:
Uri.Path = QuoteString(Uri.Path, "+~ ");
Some servers might not like + or ~ being quoted. We use the same
quoting call for the http method too, though, so it seems highly
unlikely to cause an issue.
[Original bug report]
Distributor ID: Ubuntu
Description: Ubuntu 16.10
Release: 16.10
Codename: yakkety
apt version 1.3.3 (also tried 1.4-beta2 .deb, same results)
When trying to install a package hosted on s3 from the kxstudio repo,
the download fails with an HTTP error:
nico at nico-lenovo-ubuntu:~/Downloads$ sudo apt-get install wineasio-amd64
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
wine1.6-amd64
The following NEW packages will be installed
wine1.6-amd64 wineasio-amd64
0 to upgrade, 2 to newly install, 0 to remove and 1 not to upgrade.
Need to get 30.9 kB/32.6 kB of archives.
After this operation, 184 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Err:1 http://kxstudio.linuxaudio.org/repo stable/free amd64 wineasio-amd64 amd64 0.9.0+git20110613-2kxstudio3
505 HTTP Version not supported
E: Failed to fetch https://github-cloud.s3.amazonaws.com/releases/39372848/0f048802-2fb5-11e5-9d8c-907ec7b97c46.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ/20161222/us-east-1/s3/aws4_request&X-Amz-Date=20161222T022041Z&X-Amz-Expires=300&X-Amz-Signature=750f9b2ee076dcb8ae6992cae911f43208b3eec41976362cebf694e3c72b7aef&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment; filename=wineasio-amd64_0.9.0.git20110613-2kxstudio3_amd64.deb&response-content-type=application/octet-stream 505 HTTP Version not supported
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Error allegedly not present in Ubuntu 14.04 and 16.04
More details in these forum posts:
https://github.com/KXStudio/Repository/issues/73#issuecomment-268649503
https://www.linuxmusicians.com/viewtopic.php?t=16056
https://www.drupal.org/node/2324991 (clues on root cause)
ProblemType: Bug
DistroRelease: Ubuntu 16.10
Package: apt 1.3.3
ProcVersionSignature: Ubuntu 4.8.0-30.32-lowlatency 4.8.6
Uname: Linux 4.8.0-30-lowlatency x86_64
ApportVersion: 2.20.3-0ubuntu8.2
Architecture: amd64
CurrentDesktop: X-Cinnamon
Date: Thu Dec 22 02:31:47 2016
InstallationDate: Installed on 2016-10-20 (62 days ago)
InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1651923/+subscriptions
More information about the foundations-bugs
mailing list