[Bug 1651923] Re: apt https method decodes redirect locations and sends them to the destination undecoded.
Elvis Stansvik
elvstone at gmail.com
Wed Jan 18 18:18:08 UTC 2017
I'm afraid the 1.2.19 in xenial-proposed does not seem to solve the
problem here:
estan at newton:~$ apt-cache policy apt
apt:
Installerad: 1.2.19
Kandidat: 1.2.19
Versionstabell:
*** 1.2.19 500
500 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
100 /var/lib/dpkg/status
1.2.18 500
500 http://se.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
1.2.15ubuntu0.2 500
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
1.2.10ubuntu1 500
500 http://se.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Fel:1 http://downloads.sourceforge.net/corefonts/andale32.exe
404 Not Found
E: Misslyckades med att hämta https://vorboss.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe 404 Not Found
E: Hämtning misslyckades
estan at newton:~$ wget http://downloads.sourceforge.net/corefonts/andale32.exe
--2017-01-18 19:10:15-- http://downloads.sourceforge.net/corefonts/andale32.exe
Slår upp downloads.sourceforge.net (downloads.sourceforge.net)... 216.34.181.59
Ansluter till downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:80... ansluten.
HTTP-begäran skickad, väntar på svar... 301 Moved Permanently
Adress: http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe [följer]
--2017-01-18 19:10:15-- http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Ansluter till downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:80... ansluten.
HTTP-begäran skickad, väntar på svar... 302 Found
Adress: https://netcologne.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe [följer]
--2017-01-18 19:10:16-- https://netcologne.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Slår upp netcologne.dl.sourceforge.net (netcologne.dl.sourceforge.net)... 78.35.24.46, 2001:4dd0:1234:6::5f
Ansluter till netcologne.dl.sourceforge.net (netcologne.dl.sourceforge.net)|78.35.24.46|:443... ansluten.
HTTP-begäran skickad, väntar på svar... 200 OK
Längd: 198384 (194K) [application/octet-stream]
Sparar till: "andale32.exe"
andale32.exe
100%[========================================================================================================================================>]
193,73K 1,25MB/s in 0,2s
2017-01-18 19:10:16 (1,25 MB/s) - "andale32.exe" sparades
[198384/198384]
estan at newton:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
estan at newton:~$
And I'm afraid it's not deterministic; this is the output from three
runs:
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Fel:1 http://downloads.sourceforge.net/corefonts/andale32.exe
404 Not Found
E: Misslyckades med att hämta https://netix.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe 404 Not Found
E: Hämtning misslyckades
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Fel:1 http://downloads.sourceforge.net/corefonts/andale32.exe
404 Not Found
E: Misslyckades med att hämta https://netix.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe 404 Not Found
E: Hämtning misslyckades
estan at newton:~$ /usr/lib/apt/apt-helper download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
Läs:1 http://downloads.sourceforge.net/corefonts/andale32.exe [198 kB]
Hämtade 198 kB på 1s (142 kB/s)
estan at newton:~$
The two first ones failed while the last one succeeded.
Here's the output with -o debug::acquire::https=1 -o
debug::acquire::http=1 for a successful run:
estan at newton:~$ /usr/lib/apt/apt-helper -o debug::acquire::https=1 -o debug::acquire::http=1 download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
0% [Arbetar]GET /corefonts/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/corefonts/andale32.exe
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 18 Jan 2017 18:13:17 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Location: http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 178
0% [Arbetar]GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/project/corefonts/the fonts/final/andale32.exe
HTTP/1.1 302 Found
Server: nginx
Date: Wed, 18 Jan 2017 18:13:18 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
content-disposition: attachment; filename="andale32.exe"
Set-Cookie: sf_mirror_attempt="corefonts:netassist:the%20fonts/final/andale32.exe"; expires=120; Path=/
Location: https://netassist.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 165
0% [Arbetar]* Trying 62.205.134.42...
* Connected to netassist.dl.sourceforge.net (62.205.134.42) port 443 (#0)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* found 708 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
0% [Arbetar]* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: netassist.dl.sourceforge.net (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=netassist.dl.sourceforge.net
* start date: Thu, 05 Jan 2017 23:28:00 GMT
* expire date: Wed, 05 Apr 2017 23:28:00 GMT
* issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /project/corefonts/the fonts/final/andale32.exe HTTP/1.1
Host: netassist.dl.sourceforge.net
User-Agent: Debian APT-CURL/1.0 (1.2.15)
Accept: */*
Cache-Control: max-age=0
0% [Arbetar]< HTTP/1.1 200 OK
< Server: nginx
< Date: Wed, 18 Jan 2017 18:32:55 GMT
< Content-Type: application/octet-stream
< Content-Length: 198384
< Last-Modified: Thu, 15 Aug 2002 14:33:49 GMT
< Connection: close
< ETag: "3d5bbbcd-306f0"
< Accept-Ranges: bytes
<
Läs:1 http://downloads.sourceforge.net/corefonts/andale32.exe [198 kB]
53% [1 test.exe 131 kB/198 kB 66%]* Closing connection 0
Hämtade 198 kB på 16s (12,0 kB/s)
estan at newton:~$
And here's for a failed run:
estan at newton:~$ /usr/lib/apt/apt-helper -o debug::acquire::https=1 -o debug::acquire::http=1 download-file http://downloads.sourceforge.net/corefonts/andale32.exe test.exe
0% [Arbetar]GET /corefonts/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/corefonts/andale32.exe
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 18 Jan 2017 18:17:19 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Location: http://downloads.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 178
0% [Arbetar]GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
GET /project/corefonts/the%20fonts/final/andale32.exe HTTP/1.1
Host: downloads.sourceforge.net
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
User-Agent: Debian APT-HTTP/1.3 (1.2.19)
Answer for: http://downloads.sourceforge.net/project/corefonts/the fonts/final/andale32.exe
HTTP/1.1 302 Found
Server: nginx
Date: Wed, 18 Jan 2017 18:17:20 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
content-disposition: attachment; filename="andale32.exe"
Set-Cookie: sf_mirror_attempt="corefonts:freefr:the%20fonts/final/andale32.exe"; expires=120; Path=/
Location: https://freefr.dl.sourceforge.net/project/corefonts/the%20fonts/final/andale32.exe
Content-Length: 162
0% [Arbetar]* Trying 88.191.250.136...
* Connected to freefr.dl.sourceforge.net (88.191.250.136) port 443 (#0)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* found 708 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
0% [Arbetar]* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: freefr.dl.sourceforge.net (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=freefr.dl.sourceforge.net
* start date: Fri, 30 Dec 2016 23:34:00 GMT
* expire date: Thu, 30 Mar 2017 23:34:00 GMT
* issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* compression: NULL
* ALPN, server accepted to use http/1.1
> GET /project/corefonts/the fonts/final/andale32.exe HTTP/1.1
Host: freefr.dl.sourceforge.net
User-Agent: Debian APT-CURL/1.0 (1.2.15)
Accept: */*
Cache-Control: max-age=0
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
0% [Arbetar]< HTTP/1.1 302 Found
< Date: Wed, 18 Jan 2017 18:17:20 GMT
< Server: Apache
< Location: https://downloads.sourceforge.net/project/corefonts/the?download&failedmirror=freefr.dl.sourceforge.net
< Content-Length: 291
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
<
* Closing connection 0
* Issue another request to this URL: 'https://downloads.sourceforge.net/project/corefonts/the?download&failedmirror=freefr.dl.sourceforge.net'
* Trying 216.34.181.59...
* Connected to downloads.sourceforge.net (216.34.181.59) port 443 (#1)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* found 708 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
0% [Arbetar]* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_CBC_SHA1
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: *.sourceforge.net (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: C=US,ST=California,L=San Diego,O=SourceForge Media\, LLC,OU=sourceforge.net,CN=*.sourceforge.net
* start date: Tue, 13 Dec 2016 00:00:00 GMT
* expire date: Fri, 12 Jan 2018 23:59:59 GMT
* issuer: C=US,O=GeoTrust Inc.,CN=GeoTrust SSL CA - G3
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /project/corefonts/the?download&failedmirror=freefr.dl.sourceforge.net HTTP/1.1
Host: downloads.sourceforge.net
User-Agent: Debian APT-CURL/1.0 (1.2.15)
Accept: */*
Cache-Control: max-age=0
Range: bytes=198384-
If-Range: Thu, 15 Aug 2002 14:33:49 GMT
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Wed, 18 Jan 2017 18:17:22 GMT
< Content-Type: text/html; charset=UTF-8
< Connection: close
< Content-Length: 154
<
* Closing connection 1
Fel:1 http://downloads.sourceforge.net/corefonts/andale32.exe
404 Not Found
E: Misslyckades med att hämta https://freefr.dl.sourceforge.net/project/corefonts/the fonts/final/andale32.exe 404 Not Found
E: Hämtning misslyckades
estan at newton:~$
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1651923
Title:
apt https method decodes redirect locations and sends them to the
destination undecoded.
Status in apt package in Ubuntu:
Fix Released
Status in apt source package in Xenial:
Fix Committed
Status in apt source package in Yakkety:
Fix Committed
Bug description:
[Impact]
Downloads via HTTPS fail if the URL contains a space. This breaks packages like ttf-mscorefonts-installer and various third party hosters.
[Test case]
Check that
/usr/lib/apt/apt-helper download-file
http://kxstudio.linuxaudio.org/repo/pool/free/ardour4_4.7.0-1kxstudio1_i386.deb
test.deb
can successfully download the file (or at least start downloading it)
and does not fail early with a 505 HTTP version not supported error
message.
This problem does not occur with that file on xenial, as it first
redirects to an https URI without a space which then redirects to an
HTTPS uri with a space (http w/o space -> https w/o space -> https w/
space). In xenial, https->https redirects where handled internally by
curl.
Another test (applicable to xenial) is to install ttf-mscorefonts-
installer.
[Regression potential]
The added code is:
Uri.Path = QuoteString(Uri.Path, "+~ ");
Some servers might not like + or ~ being quoted. We use the same
quoting call for the http method too, though, so it seems highly
unlikely to cause an issue.
[Original bug report]
Distributor ID: Ubuntu
Description: Ubuntu 16.10
Release: 16.10
Codename: yakkety
apt version 1.3.3 (also tried 1.4-beta2 .deb, same results)
When trying to install a package hosted on s3 from the kxstudio repo,
the download fails with an HTTP error:
nico at nico-lenovo-ubuntu:~/Downloads$ sudo apt-get install wineasio-amd64
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
wine1.6-amd64
The following NEW packages will be installed
wine1.6-amd64 wineasio-amd64
0 to upgrade, 2 to newly install, 0 to remove and 1 not to upgrade.
Need to get 30.9 kB/32.6 kB of archives.
After this operation, 184 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Err:1 http://kxstudio.linuxaudio.org/repo stable/free amd64 wineasio-amd64 amd64 0.9.0+git20110613-2kxstudio3
505 HTTP Version not supported
E: Failed to fetch https://github-cloud.s3.amazonaws.com/releases/39372848/0f048802-2fb5-11e5-9d8c-907ec7b97c46.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ/20161222/us-east-1/s3/aws4_request&X-Amz-Date=20161222T022041Z&X-Amz-Expires=300&X-Amz-Signature=750f9b2ee076dcb8ae6992cae911f43208b3eec41976362cebf694e3c72b7aef&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment; filename=wineasio-amd64_0.9.0.git20110613-2kxstudio3_amd64.deb&response-content-type=application/octet-stream 505 HTTP Version not supported
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Error allegedly not present in Ubuntu 14.04 and 16.04
More details in these forum posts:
https://github.com/KXStudio/Repository/issues/73#issuecomment-268649503
https://www.linuxmusicians.com/viewtopic.php?t=16056
https://www.drupal.org/node/2324991 (clues on root cause)
ProblemType: Bug
DistroRelease: Ubuntu 16.10
Package: apt 1.3.3
ProcVersionSignature: Ubuntu 4.8.0-30.32-lowlatency 4.8.6
Uname: Linux 4.8.0-30-lowlatency x86_64
ApportVersion: 2.20.3-0ubuntu8.2
Architecture: amd64
CurrentDesktop: X-Cinnamon
Date: Thu Dec 22 02:31:47 2016
InstallationDate: Installed on 2016-10-20 (62 days ago)
InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
SourcePackage: apt
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1651923/+subscriptions
More information about the foundations-bugs
mailing list