svn-import performance analysis

[Jelmer Vernooij]

On Wed, 2008-12-03 at 05:44 +0100, Jelmer Vernooij wrote:
> On Tue, 2008-12-02 at 18:03 -0600, John Arbash Meinel wrote:
> > > I'm at a bit of a loss as to how I can optimize this further. Does
> > > anybody have any ideas? Also, I would expect CHK inventories to be of
> > > help here - is that correct?
> > CHK inventories will be much more capable of "partial update without
> > full serialization/deserialization", as it is designed in from the
> > beginning.
> > 
> > One hack I did a long time ago was to have each InventoryEntry remember
> > its serialized form (along with some other info like what serializer
> > created it, etc.) and then have it throw away that value when it was
> > modified. Though it really needs the calling code to be careful about
> > modifications, because InventoryEntries are just plain ol' data
> > structures, without a way to know if they have been made "dirty".
> > 
> > So... CHK should be a big help here, and without doing some dirty hacks
> > in bzrlib, I'm not sure if there is much else to be done.
> Cool, I'm looking forward to them landing then :-) At least the code in
> bzr-svn is ready for that now.
> 
> When using brisbane-core with 1.9-rich-root, the time spent seems to've
> shifted significantly. Only 17% of the time is now spent in
> Repository.add_inventory_delta(). Instead, more time is now spent in:
> 
>  * Knit.add_lines() (30.11%)
>  * Knit.get_record_stream() (18.78%)
>  * Repository.get_inventory() (5.22%)
Argh, stupid me had forgotten to build the pyrex extensions... Here are
the updated numbers (different svn repository):

* Repository.add_inventory_delta() (28.69%)
* VersionedFile.get_record_stream() (16.11%)
* Knit.add_lines() (24.01%)

Cheers,

Jelmer