(fwd) reducing/measuring python memory usage

[Robert Collins]

On Wed, 2007-11-28 at 15:51 -0800, Wichmann, Mats D wrote:
> bazaar-bounces at lists.canonical.com wrote:
> > I was thinking of mixing http://wingolog.org/pub/mem_usage.py into
> > bzr as a plugin. 
> > 
> > -Rob
> > --
> > GPG key available at: <http://www.robertcollins.net/keys.txt>.
> 
> It would be interesting to see.  I've reported on my performance
> issues and suspect a combination of a lot of memory usage computing
> on branches with a lot of history/files plus the need to access
> lots of files in the metadata (with buffer cache "cold") collectively
> putting too much pressure on the vm subsystem, since the two share
> (compete for) memory. The exact same operations are  relatively 
> crisp on a different system with lots of ram.  I'm not sure there's
> any tool that can tell us about the effects of the second part of
> the combination, but I can grub around a little.
> 
> Do we know if other currently interesting systems (git, hg, svn,
> etc.) have to look at as many files during operations like commit?

Have you tried packs yet? They access:
your working files (stat, or stat + read + sha1)
your pack indices (4 files per pack, maintained in logarthimically
increasing sizes)
enough data from your .pack files to generate text parents when
committing, to generate deltas.

-Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20071129/890e0ca7/attachment.pgp