[MERGE] Sort knit fetches

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Arbash Meinel wrote:
...

> This caused "bzr branch --stacked
> http://people.ubuntu.org/~jameinel/python/trunk" to be cut in half
> (9m40s => 4m40s), and make much better use of my network bandwidth.

...

> 
> It isn't the whole fix, as the rest of the fix was to change
> "get_record_stream()" code to request all file content at once, which
> has negative impact on memory. But this is a prerequisite which
> shouldn't be controversial.


So building on the earlier patch, this now attempts to batch requests up
into approx 5MB compressed content chunks.

The code defers to the Index for the final determination of size,
because stuff like GroupCompress is going to hold very different
information in the index, not to mention have different grouping
properties. (gc does batching differently anyway, which has not really
been tuned for network fetches.)

I haven't timed this for 'bzr branch --stacked' yet, as I did the final
work offline. But I'll run it when I get the chance.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmkft8ACgkQJdeBCYSNAAOmmQCfU3rS8HrQPEzh2mQ5koeQ4FuE
s88AnjjL1juerx6yGIauZVIHT88ZIorh
=LbHO
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sort_knit_fetch_grouped.patch
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20090224/3f51b297/attachment-0001.diff