[Bug 996151] Re: disable apt http pipelining in quantal

[Jason Gunthorpe]

Wow, I'm really surprised you guys have decided to turn pipe-lining off.
That is crazy. Pipelining has been in APT since day one (ie since 1997,
wow!), and I personally worked with a number of web server developers to
make sure their severs worked properly, according to the RFC.

Squid has *always* had varying levels of breakage when working with
pipelining, but I also extensively tested APT's HTTP method with squid
and ensured it worked for many years. It looks to me like someone must
have tried to 'improve' things in squid (probably tried to support
HTTP/1.1 keep-alive) and broke it even more..

The thought that pipe lining is inherently broken is ridiculous. The
behaviour of requesters, proxies and completer's is very well defined,
and if you follow the damn spec you don't create any problems, security,
correctness, or otherwise.

And yes, it makes an huge, obvious, night and day difference:

$ time sudo apt-get update -o Acquire::http::Pipeline-Depth=10
real    0m9.090s                                                                                                                          
$ time sudo apt-get update -o Acquire::http::Pipeline-Depth=0
real    0m19.700s

A much better suggestion would be to detect a proxy during install (most
proxies add headers to their reply)  and drop a pipeline depth config
into /etc/apt/apt.conf.d/ ... Or perhaps not pipeline the first request
and look for a proxy in the reply, then turn it on.

It is completely mind blowing that squid has been broken since 1997,
even to the extent that the breakage created a whole new class of proxy
vulnerabilities (request smuggling) and nobody has fixed it, or even
really cared to notice..

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to apt in Ubuntu.
https://bugs.launchpad.net/bugs/996151

Title:
  disable apt http pipelining in quantal

Status in “apt” package in Ubuntu:
  Fix Released
Status in “apt” package in Debian:
  New

Bug description:
  Per UDS session on Apt improvements, it has been proposed to remove
  apt http pipelining

  The reasons:
  1. HTTP Pipelining has issue with certain proxy implementation
  2. Some new object stores, like S3, or Google's APT repositories have problems with HTTP Pipelining

  Running a test shows that disabling apt-pipelining has no perceptable
  diffferenvce, and disabling apt pipeling actually performed slightly
  better with an average of 31.899s versuses 32.456s. I tested an "apt-
  get -y update" with and without apt HTTP pipelining turned on.

  For more information on apt-pipelining, here are 2 threads to read:
   http://old.nabble.com/APT-do-not-work-with-Squid-as-a-proxy-because-of-pipelining-default-td28579596.html
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565555

  
  Pipelining on (apt-get -y upgrade):
  33.92
  31.37
  31.64
  31.63
  33.29
  33.08
  32.92
  32.88
  31.73
  31.98
  32.01
  32.96
  31.51
  32.68
  33.25

  Pipelining off (apt-get -o Acquire::http::Pipeline-Depth="0" -y upgrade):
  31.66
  31.59
  31.24
  31.30
  31.29
  32.85
  32.75
  31.50
  31.18
  32.26
  31.43
  33.28
  31.67
  32.45
  32.04

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt/+bug/996151/+subscriptions