Rev 4793: More notes on memory consumption. in http://bazaar.launchpad.net/~jameinel/bzr/chk-index

[John Arbash Meinel]

At http://bazaar.launchpad.net/~jameinel/bzr/chk-index

------------------------------------------------------------
revno: 4793
revision-id: john at arbash-meinel.com-20091028170002-sob4l165skyupg3h
parent: john at arbash-meinel.com-20091028165347-px6xmyjssmblo1om
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: chk-index
timestamp: Wed 2009-10-28 12:00:02 -0500
message:
  More notes on memory consumption.
-------------- next part --------------
=== modified file 'bzrlib/chk_index.py'
--- a/bzrlib/chk_index.py	2009-10-28 16:53:47 +0000
+++ b/bzrlib/chk_index.py	2009-10-28 17:00:02 +0000
@@ -525,6 +525,11 @@
         :return: A file handle for a temporary file containing the nodes added
             to the index.
         """
+        # Note: We *could* buffer all of this to a temp file like btree_index
+        #       does. However, by my count, the memory overhead of finalizing
+        #       these strings is << the memory overhead of the data structure.
+        #       We are talking something like 80+MB for the CHKIndexBuilder,
+        #       versus 12MB for the actual content.
         header = self._build_header()
         output_bytes = []
         header_bytes = header.serialize()