bzr log http+urllib does not work, http+pycurl is too slow

[Vincent Ladeuil]

>>>>> "bialix" == Alexander Belchenko <bialix at ukr.net> writes:

    bialix> I have Trac running as separate python application on separate
    bialix> win32 machine.
    bialix> I put pack shared repo in the Trac's htdocs directory, so my branches are
    bialix> available via http protocol (I use custom port 8000).
    bialix> Well, at least `bzr revno http://host:8000/chrome/site/branches/Logic` works
    bialix> as expected. But not log command.

    bialix> On my client machine I use custom bzr.exe 1.0rc3 compiled without pycurl, so
    bialix> I encounter this bug.

<snip/>

    bialix> http readv of 930af4714be1e33318bb630b8d626540.pack  offsets =>

Using -Dhttp will give us the true ranges requested.

    bialix> 14 collapsed 5
    bialix> Traceback (most recent call last):
    bialix>   File "bzrlib\commands.pyc", line 802, in run_bzr_catch_errors
    bialix>   File "bzrlib\commands.pyc", line 758, in run_bzr
    bialix>   File "bzrlib\commands.pyc", line 492, in run_argv_aliases
    bialix>   File "bzrlib\commands.pyc", line 768, in ignore_pipe
    bialix>   File "bzrlib\builtins.pyc", line 1750, in run
    bialix>   File "bzrlib\log.pyc", line 189, in show_log
    bialix>   File "bzrlib\log.pyc", line 303, in _show_log
    bialix>   File "bzrlib\log.pyc", line 290, in iter_revisions
    bialix>   File "bzrlib\decorators.pyc", line 127, in read_locked
    bialix>   File "bzrlib\repository.pyc", line 1003, in get_revisions
    bialix>   File "bzrlib\decorators.pyc", line 127, in read_locked
    bialix>   File "bzrlib\repository.pyc", line 1012, in _get_revisions
    bialix>   File "bzrlib\store\revision\knit.pyc", line 88, in get_revisions
    bialix>   File "bzrlib\store\revision\knit.pyc", line 104, in
    bialix> _get_serialized_revisions
    bialix>   File "bzrlib\knit.pyc", line 1062, in get_texts
    bialix>   File "bzrlib\knit.pyc", line 1068, in get_line_list
    bialix>   File "bzrlib\knit.pyc", line 1086, in _get_content_maps
    bialix>   File "bzrlib\knit.pyc", line 1051, in _get_record_map
    bialix>   File "bzrlib\knit.pyc", line 2426, in read_records_iter
    bialix>   File "bzrlib\knit.pyc", line 2033, in get_raw_records
    bialix>   File "bzrlib\pack.pyc", line 253, in iter_records
    bialix>   File "bzrlib\pack.pyc", line 294, in _read_format
    bialix>   File "bzrlib\pack.pyc", line 221, in _read_line
    bialix>   File "bzrlib\pack.pyc", line 185, in readline
    bialix>   File "bzrlib\pack.pyc", line 172, in _next
    bialix>   File "bzrlib\transport\http\__init__.pyc", line 236, in _readv
    bialix>   File "bzrlib\transport\http\__init__.pyc", line 318, in _coalesce_readv
    bialix>   File "bzrlib\transport\http\__init__.pyc", line 281, in get_and_yield
    bialix>   File "bzrlib\transport\http\_urllib.pyc", line 135, in _get
    bialix>   File "bzrlib\transport\http\_urllib.pyc", line 75, in _perform
    bialix>   File "bzrlib\transport\http\_urllib2_wrappers.pyc", line 170,
    bialix> in cleanup_pipe

Aaargh. Since we use a persistent connection we have to read
unprocessed bytes left on the socket.

Can you try this patch:

=== modified file 'bzrlib/transport/http/_urllib2_wrappers.py'
--- bzrlib/transport/http/_urllib2_wrappers.py	2007-12-06 22:46:16 +0000
+++ bzrlib/transport/http/_urllib2_wrappers.py	2007-12-11 20:18:53 +0000
@@ -134,7 +134,8 @@
         """
         if not self.isclosed():
             # Make sure nothing was left to be read on the socket
-            data = self.read(self.length)
+            while self.length and self.length > 1024:
+                data = self.read(self.length)
             self.close()
 
 
If it works, try tweaking the 1024 until it breaks.  That will
give us an idea of how much window can buffer for a socket.

Additional bonus if you can also check how much bytes are left
unprocessed.

        Vincent