patch: more sftp unit tests

[Robert Collins]

On Mon, 2005-12-19 at 21:04 -0800, Robey Pointer wrote:
> On 14 Dec 2005, at 18:10, Robert Collins wrote:
> 
> > On Mon, 2005-12-12 at 20:33 -0800, Robey Pointer wrote:
> >>
> >> I think this is the real problem.  SSH negotiation is hard, and
> >> (currently) for each unit test, we do each side of the negotiation,
> >> sync up, do the test, then do a clean sync'd shutdown.
> >
> > Can we profile this? I think connection sharing during the tests is
> > quite ugly for us - and I'd rather have a slightly slow (but not
> > hugely!) test suite that is clean. My main concern is that sftp
> > *appears* much slower than HTTP, and if that is the case, then its
> > concerning independently of the test suite : optimising the test suite
> > is a bandaid.
> 
> I had some free time this weekend, so I played around with lsprof.   
> The result of running
> 
> 	$ ./bzr --lsprof selftest test_no_working_tree
> 
> (which runs a single sftp unit test in 2.5 seconds on my iBook) is  
> posted here:
> 
> 	http://www.lag.net/~robey/sftp_test.html
> 
> One surprising thing to me was that the branch's put_controlfiles()  
> accounted for nearly an entire second.  I guess the branch  
> initialization is doing a handful of sync requests as part of each  
> test's setup.  So that's at least one thing I've learned. :)
> 
> I wrote a little script to generate HTML files out of lsprof output,  
> so I can click around in the output and "follow the slow".  Since  
> others may find it useful too, it's attached.  To make it work you  
> also need a small patch to bzr, to make it generate pickled lsprof  
> files instead of dumping text.  Also attached.


Interesting. +1 on the patch if altered to still pprint the output for
casual users.

As for the put_controlfiles thing it looks like nearly half that second
is taken up by 'open exclusive' and 'rename' operations: which on a
local connection with encryption established seems remarkably long.

bzrlib.transport.sftp:789(_sftp_open_exclusive) 9 0 226.2940 4.3180
<.../paramiko/sftp_client.py>:229(rename) 18 0 399.9710 0.3690

This seems to be the primary culprit there:

<.../paramiko/packet.py>:207(write_all) 54 0 522.3890 522.3890 
54 calls for 522 ms: 10ms per call - slow.


Rob


-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051220/d7ab7350/attachment.pgp