[Bug 1651923] Re: apt https method decodes redirect locations and sends them to the destination undecoded.

Tue Jan 10 23:29:14 UTC 2017

I just uploaded 1.4~rc3ubuntu1 to zesty. It's building now, and should
hit zesty-proposed soon, and hopefully migrate without issues to zesty
release repository.

This is the included workaround for now:

commit 12d5863a6ecd358db5645a4c1ca75576ef3c6232
Author: Julian Andres Klode <jak at debian.org>
Date:   Thu Dec 29 14:16:07 2016 +0100

    https: Quote path in URL before passing it to curl

    Curl requires URLs to be urlencoded. We are however giving it
    undecoded URLs. This causes it go completely nuts if there is
    a space in the URI, producing requests like:

        GET /a file HTTP/1.1

    which the servers then interpret as a GET request for "/a" with
    HTTP version "file" or some other non-sense.

    This works around the issue by encoding the path component of
    the URL. I'm not sure if we should encode other parts of the URL
    as well, this one seems to do the trick for the actual issue at
    hand.

    A more correct fix is to avoid the dequoting and (re-)quoting
    of URLs when a redirect occurs / a new request is sent. That's
    been on the radar for probably a year or two now, but nobody
    bothered implementing that yet.

    LP: #1651923

diff --git a/methods/https.cc b/methods/https.cc
index d60bc6fbc..d71ef0bf0 100644
--- a/methods/https.cc
+++ b/methods/https.cc
@@ -275,6 +275,10 @@ bool HttpsMethod::Fetch(FetchItem *Itm)
    if (Server == nullptr || Server->Comp(Itm->Uri) == false)
       Server = CreateServerState(Itm->Uri);

+   // The "+" is encoded as a workaround for a amazon S3 bug
+   // see LP bugs #1003633 and #1086997. (taken from http method)
+   Uri.Path = QuoteString(Uri.Path, "+~ ");
+
    FetchResult Res;
    RequestState Req(this, Server.get());
    CURLUserPointer userp(this, &Res, Itm, &Req);


If that works fine for everyone, I can cherry-pick this into xenial and yakkety. The SRU procedure takes a few weeks though - in fact, many of the current changes pending for xenial are waiting since 2 months already.

Trusty also has an issue with the downloads, BTW, but it just hangs
completely, so there's probably a different issue there.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/1651923

Title:
  apt https method decodes redirect locations and sends them to the
  destination undecoded.

Status in apt package in Ubuntu:
  Fix Committed
Status in apt source package in Xenial:
  Triaged
Status in apt source package in Yakkety:
  Triaged

Bug description:
  Distributor ID:	Ubuntu
  Description:	Ubuntu 16.10
  Release:	16.10
  Codename:	yakkety

  apt version 1.3.3 (also tried 1.4-beta2 .deb, same results)

  When trying to install a package hosted on s3 from the kxstudio repo,
  the download fails with an HTTP error:

  nico at nico-lenovo-ubuntu:~/Downloads$ sudo apt-get install wineasio-amd64
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  The following additional packages will be installed:
    wine1.6-amd64
  The following NEW packages will be installed
    wine1.6-amd64 wineasio-amd64
  0 to upgrade, 2 to newly install, 0 to remove and 1 not to upgrade.
  Need to get 30.9 kB/32.6 kB of archives.
  After this operation, 184 kB of additional disk space will be used.
  Do you want to continue? [Y/n] y
  Err:1 http://kxstudio.linuxaudio.org/repo stable/free amd64 wineasio-amd64 amd64 0.9.0+git20110613-2kxstudio3
    505  HTTP Version not supported
  E: Failed to fetch https://github-cloud.s3.amazonaws.com/releases/39372848/0f048802-2fb5-11e5-9d8c-907ec7b97c46.deb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ/20161222/us-east-1/s3/aws4_request&X-Amz-Date=20161222T022041Z&X-Amz-Expires=300&X-Amz-Signature=750f9b2ee076dcb8ae6992cae911f43208b3eec41976362cebf694e3c72b7aef&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment; filename=wineasio-amd64_0.9.0.git20110613-2kxstudio3_amd64.deb&response-content-type=application/octet-stream  505  HTTP Version not supported
  E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

  Error allegedly not present in Ubuntu 14.04 and 16.04

  More details in these forum posts:

  https://github.com/KXStudio/Repository/issues/73#issuecomment-268649503

  https://www.linuxmusicians.com/viewtopic.php?t=16056

  https://www.drupal.org/node/2324991 (clues on root cause)

  ProblemType: Bug
  DistroRelease: Ubuntu 16.10
  Package: apt 1.3.3
  ProcVersionSignature: Ubuntu 4.8.0-30.32-lowlatency 4.8.6
  Uname: Linux 4.8.0-30-lowlatency x86_64
  ApportVersion: 2.20.3-0ubuntu8.2
  Architecture: amd64
  CurrentDesktop: X-Cinnamon
  Date: Thu Dec 22 02:31:47 2016
  InstallationDate: Installed on 2016-10-20 (62 days ago)
  InstallationMedia: Ubuntu 16.10 "Yakkety Yak" - Release amd64 (20161012.2)
  SourcePackage: apt
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1651923/+subscriptions

