twisted much?

[Martin Pool]

(Maybe some of the Canonical people have an opinion on this.)

To get really good HTTP download performance it seems that we need to
do parallel/overlapped downloads.

One way to do that would be to use Twisted Python's http client
library.  Some people like it very much and some strongly dislike it.
It would introduce a sizeable dependency.

Another way is to do hand-carved select() or thread based clients.
While I would normally hesitate to introduce threads into a program
that doesn't strongly need them I think it might be OK here, if
they're contained behind an API firewall.

-- 
Martin