B+tree discussions

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
| On Thu, 2008-07-03 at 11:15 -0500, John Arbash Meinel wrote:
|
|
|> I'm counting about 204 bytes per row, and we know that we get ~90 rows
|> into a page. Or 18K per page unpacked in memory.
|>
|> So the 1000 node cache is actually an 18MB memory cache.
|
| in two-tuple indices, we should have very high locality of reference for
| the first element of the tuple (file ids should often be the same for an
| entire page, for both keys and references).
|
| That would allow saving some memory.
|
| -Rob

yeah. I was thinking about passing the parsers a cache object that could
be used for the lifetime of some portion of a request.

So that rather than interning all strings and tuples in some enormous
dict, you would instead pass in a

_LeafNode(bytes, key_length, ref_list_length, obj_cache)

And then the 'obj_cache' lifetime is determined up at the say...
'iter_entries' level. It doesn't have to last forever to give us some
memory savings. And that probably improves the "dict the size of all
possible strings" issue with intern().

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhtPmwACgkQJdeBCYSNAAPGpQCgmfuI9gcaOhuzfHyir1mvvQii
uMMAn0FqV5rVYpiXbYdlN8d39IDTKO8C
=dcmx
-----END PGP SIGNATURE-----