Rev 3652: (robertc) Permit filtering of log output by plugins. (Robert Collins) in file:///home/pqm/archives/thelove/bzr/%2Btrunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Thu Aug 28 03:30:35 BST 2008
At file:///home/pqm/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 3652
revision-id: pqm at pqm.ubuntu.com-20080828023029-r2qwgt7zu9udtla3
parent: pqm at pqm.ubuntu.com-20080827044137-4ox67ehr4bxtj7b0
parent: robertc at robertcollins.net-20080828015504-19de8uda9gpi4x51
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Thu 2008-08-28 03:30:29 +0100
message:
(robertc) Permit filtering of log output by plugins. (Robert Collins)
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
bzrlib/tests/test_log.py testlog.py-20050728115707-1a514809d7d49309
------------------------------------------------------------
revno: 3642.1.8
revision-id: robertc at robertcollins.net-20080828015504-19de8uda9gpi4x51
parent: robertc at robertcollins.net-20080828012759-4m8ng541gnk2d6b9
parent: pqm at pqm.ubuntu.com-20080827044137-4ox67ehr4bxtj7b0
committer: Robert Collins <robertc at robertcollins.net>
branch nick: integration
timestamp: Thu 2008-08-28 11:55:04 +1000
message:
Resolve conflicts in NEWS.
added:
bzrlib/_btree_serializer_c.pyx _parse_btree_c.pyx-20080703034413-3q25bklkenti3p8p-2
bzrlib/_btree_serializer_py.py _parse_btree_py.py-20080703034413-3q25bklkenti3p8p-3
bzrlib/btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
bzrlib/chunk_writer.py chunk_writer.py-20080630234519-6ggn4id17nipovny-1
bzrlib/tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
bzrlib/tests/test_chunk_writer.py test_chunk_writer.py-20080630234519-6ggn4id17nipovny-2
modified:
.bzrignore bzrignore-20050311232317-81f7b71efa2db11a
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/_patiencediff_c.c _patiencediff_c.c-20070721205602-q3imkipwlgagp3cy-1
bzrlib/branch.py branch.py-20050309040759-e4baf4e0d046576e
bzrlib/builtins.py builtins.py-20050830033751-fc01482b9ca23183
bzrlib/bzrdir.py bzrdir.py-20060131065624-156dfea39c4387cb
bzrlib/config.py config.py-20051011043216-070c74f4e9e338e8
bzrlib/mail_client.py mail_client.py-20070809192806-vuxt3t19srtpjpdn-1
bzrlib/tests/__init__.py selftest.py-20050531073622-8d0e3c8845c97a64
bzrlib/tests/blackbox/test_merge.py test_merge.py-20060323225809-9bc0459c19917f41
bzrlib/tests/blackbox/test_missing.py test_missing.py-20051211212735-a2cf4c1840bb84c4
bzrlib/tests/blackbox/test_non_ascii.py test_non_ascii.py-20060105214030-68010be784a5d854
bzrlib/tests/blackbox/test_push.py test_push.py-20060329002750-929af230d5d22663
bzrlib/tests/blackbox/test_send.py test_bundle.py-20060616222707-c21c8b7ea5ef57b1
bzrlib/tests/branch_implementations/test_permissions.py test_permissions.py-20060210110243-245c01403bf0fde6
bzrlib/tests/test_diff.py testdiff.py-20050727164403-d1a3496ebb12e339
bzrlib/workingtree.py workingtree.py-20050511021032-29b6ec0a681e02e3
doc/developers/ppa.txt ppa.txt-20080722055539-606u7t2z32t3ae4w-1
doc/en/mini-tutorial/index.txt index.txt-20070813141352-2u64ooqzo0or4hss-2
doc/es/mini-tutorial/index.txt index.txt-20080504182136-wmoc35u2t6kom8ca-1
setup.py setup.py-20050314065409-02f8a0a6e3f9bc70
------------------------------------------------------------
revno: 3642.1.7
revision-id: robertc at robertcollins.net-20080828012759-4m8ng541gnk2d6b9
parent: robertc at robertcollins.net-20080821042736-742hdcpes9e8p5b5
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-28 11:27:59 +1000
message:
Review feedback.
modified:
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
------------------------------------------------------------
revno: 3642.1.6
revision-id: robertc at robertcollins.net-20080821042736-742hdcpes9e8p5b5
parent: robertc at robertcollins.net-20080821041202-typ3c7u8e67xlgst
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-21 14:27:36 +1000
message:
Make log revision filtering pluggable.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
------------------------------------------------------------
revno: 3642.1.5
revision-id: robertc at robertcollins.net-20080821041202-typ3c7u8e67xlgst
parent: robertc at robertcollins.net-20080821024338-palsst8qcdzzdob0
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-21 14:12:02 +1000
message:
Separate out batching of revisions.
modified:
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
bzrlib/tests/test_log.py testlog.py-20050728115707-1a514809d7d49309
------------------------------------------------------------
revno: 3642.1.4
revision-id: robertc at robertcollins.net-20080821024338-palsst8qcdzzdob0
parent: robertc at robertcollins.net-20080821023225-4vhy9yq7884myw3c
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-21 12:43:38 +1000
message:
Factor out revision object extraction from revision batching.
modified:
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
------------------------------------------------------------
revno: 3642.1.3
revision-id: robertc at robertcollins.net-20080821023225-4vhy9yq7884myw3c
parent: robertc at robertcollins.net-20080821020100-hnqht1b4wt0ixqqp
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-21 12:32:25 +1000
message:
Split out delta generation from revision content reading, and structure it after message evaluation, increasing performance of log -v -m.
modified:
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
------------------------------------------------------------
revno: 3642.1.2
revision-id: robertc at robertcollins.net-20080821020100-hnqht1b4wt0ixqqp
parent: robertc at robertcollins.net-20080821014143-0gabznzvn10vv3sl
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-21 12:01:00 +1000
message:
Setup a log iterator that more closely matches what the code tries to do with repository operations.
modified:
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
------------------------------------------------------------
revno: 3642.1.1
revision-id: robertc at robertcollins.net-20080821014143-0gabznzvn10vv3sl
parent: pqm at pqm.ubuntu.com-20080820164550-e4vt9gdxv8hlic7n
committer: Robert Collins <robertc at robertcollins.net>
branch nick: log
timestamp: Thu 2008-08-21 11:41:43 +1000
message:
Refactoring in log towards more pluggable revision selection.
modified:
bzrlib/log.py log.py-20050505065812-c40ce11702fe5fb1
=== modified file 'NEWS'
--- a/NEWS 2008-08-25 17:49:41 +0000
+++ b/NEWS 2008-08-28 01:55:04 +0000
@@ -70,6 +70,10 @@
INTERNALS:
+ * A new plugin interface, ``bzrlib.log.log_adapters``, has been added.
+ This allows dynamic log output filtering by plugins.
+ (Robert Collins)
+
* ``bzrlib.btree_index`` is now available, providing a b-tree index
layer. The design is memory conservative (limited memory cache),
faster to seek (approx 100 nodes per page, gives 100-way fan out),
=== modified file 'bzrlib/log.py'
--- a/bzrlib/log.py 2008-07-18 03:07:07 +0000
+++ b/bzrlib/log.py 2008-08-28 01:27:59 +0000
@@ -210,11 +210,6 @@
specific_fileid,
generate_merge_revisions,
allow_single_merge_revision)
- if search is not None:
- searchRE = re.compile(search, re.IGNORECASE)
- else:
- searchRE = None
-
rev_tag_dict = {}
generate_tags = getattr(lf, 'supports_tags', False)
if generate_tags:
@@ -225,19 +220,17 @@
# now we just print all the revisions
log_count = 0
- for (rev_id, revno, merge_depth), rev, delta in _iter_revisions(
- branch.repository, view_revisions, generate_delta):
- if searchRE:
- if not searchRE.search(rev.message):
- continue
-
- lr = LogRevision(rev, revno, merge_depth, delta,
- rev_tag_dict.get(rev_id))
- lf.log_revision(lr)
- if limit:
- log_count += 1
- if log_count >= limit:
- break
+ revision_iterator = make_log_rev_iterator(branch, view_revisions,
+ generate_delta, search)
+ for revs in revision_iterator:
+ for (rev_id, revno, merge_depth), rev, delta in revs:
+ lr = LogRevision(rev, revno, merge_depth, delta,
+ rev_tag_dict.get(rev_id))
+ lf.log_revision(lr)
+ if limit:
+ log_count += 1
+ if log_count >= limit:
+ break
def calculate_view_revisions(branch, start_revision, end_revision, direction,
@@ -295,24 +288,127 @@
yield revision_id, str(start_revno - num), 0
-def _iter_revisions(repository, view_revisions, generate_delta):
+def make_log_rev_iterator(branch, view_revisions, generate_delta, search):
+ """Create a revision iterator for log.
+
+ :param branch: The branch being logged.
+ :param view_revisions: The revisions being viewed.
+ :param generate_delta: Whether to generate a delta for each revision.
+ :param search: A user text search string.
+ :return: An iterator over lists of ((rev_id, revno, merge_depth), rev,
+ delta).
+ """
+ # Convert view_revisions into (view, None, None) groups to fit with
+ # the standard interface here.
+ if type(view_revisions) == list:
+ # A single batch conversion is faster than many incremental ones.
+ # As we have all the data, do a batch conversion.
+ nones = [None] * len(view_revisions)
+ log_rev_iterator = iter([zip(view_revisions, nones, nones)])
+ else:
+ def _convert():
+ for view in view_revisions:
+ yield (view, None, None)
+ log_rev_iterator = iter([_convert()])
+ for adapter in log_adapters:
+ log_rev_iterator = adapter(branch, generate_delta, search,
+ log_rev_iterator)
+ return log_rev_iterator
+
+
+def _make_search_filter(branch, generate_delta, search, log_rev_iterator):
+ """Create a filtered iterator of log_rev_iterator matching on a regex.
+
+ :param branch: The branch being logged.
+ :param generate_delta: Whether to generate a delta for each revision.
+ :param search: A user text search string.
+ :param log_rev_iterator: An input iterator containing all revisions that
+ could be displayed, in lists.
+ :return: An iterator over lists of ((rev_id, revno, merge_depth), rev,
+ delta).
+ """
+ if search is None:
+ return log_rev_iterator
+ # Compile the search now to get early errors.
+ searchRE = re.compile(search, re.IGNORECASE)
+ return _filter_message_re(searchRE, log_rev_iterator)
+
+
+def _filter_message_re(searchRE, log_rev_iterator):
+ for revs in log_rev_iterator:
+ new_revs = []
+ for (rev_id, revno, merge_depth), rev, delta in revs:
+ if searchRE.search(rev.message):
+ new_revs.append(((rev_id, revno, merge_depth), rev, delta))
+ yield new_revs
+
+
+def _make_delta_filter(branch, generate_delta, search, log_rev_iterator):
+ """Add revision deltas to a log iterator if needed.
+
+ :param branch: The branch being logged.
+ :param generate_delta: Whether to generate a delta for each revision.
+ :param search: A user text search string.
+ :param log_rev_iterator: An input iterator containing all revisions that
+ could be displayed, in lists.
+ :return: An iterator over lists of ((rev_id, revno, merge_depth), rev,
+ delta).
+ """
+ if not generate_delta:
+ return log_rev_iterator
+ return _generate_deltas(branch.repository, log_rev_iterator)
+
+
+def _generate_deltas(repository, log_rev_iterator):
+ """Create deltas for each batch of revisions in log_rev_iterator."""
+ for revs in log_rev_iterator:
+ revisions = [rev[1] for rev in revs]
+ deltas = repository.get_deltas_for_revisions(revisions)
+ revs = [(rev[0], rev[1], delta) for rev, delta in izip(revs, deltas)]
+ yield revs
+
+
+def _make_revision_objects(branch, generate_delta, search, log_rev_iterator):
+ """Extract revision objects from the repository
+
+ :param branch: The branch being logged.
+ :param generate_delta: Whether to generate a delta for each revision.
+ :param search: A user text search string.
+ :param log_rev_iterator: An input iterator containing all revisions that
+ could be displayed, in lists.
+ :return: An iterator over lists of ((rev_id, revno, merge_depth), rev,
+ delta).
+ """
+ repository = branch.repository
+ for revs in log_rev_iterator:
+ # r = revision_id, n = revno, d = merge depth
+ revision_ids = [view[0] for view, _, _ in revs]
+ revisions = repository.get_revisions(revision_ids)
+ revs = [(rev[0], revision, rev[2]) for rev, revision in
+ izip(revs, revisions)]
+ yield revs
+
+
+def _make_batch_filter(branch, generate_delta, search, log_rev_iterator):
+ """Group up a single large batch into smaller ones.
+
+ :param branch: The branch being logged.
+ :param generate_delta: Whether to generate a delta for each revision.
+ :param search: A user text search string.
+ :param log_rev_iterator: An input iterator containing all revisions that
+ could be displayed, in lists.
+ :return: An iterator over lists of ((rev_id, revno, merge_depth), rev, delta).
+ """
+ repository = branch.repository
num = 9
- view_revisions = iter(view_revisions)
- while True:
- cur_view_revisions = [d for x, d in zip(range(num), view_revisions)]
- if len(cur_view_revisions) == 0:
- break
- cur_deltas = {}
- # r = revision, n = revno, d = merge depth
- revision_ids = [r for (r, n, d) in cur_view_revisions]
- revisions = repository.get_revisions(revision_ids)
- if generate_delta:
- deltas = repository.get_deltas_for_revisions(revisions)
- cur_deltas = dict(izip((r.revision_id for r in revisions),
- deltas))
- for view_data, revision in izip(cur_view_revisions, revisions):
- yield view_data, revision, cur_deltas.get(revision.revision_id)
- num = min(int(num * 1.5), 200)
+ for batch in log_rev_iterator:
+ batch = iter(batch)
+ while True:
+ step = [detail for _, detail in zip(range(num), batch)]
+ if len(step) == 0:
+ break
+ yield step
+ num = min(int(num * 1.5), 200)
def _get_mainline_revs(branch, start_revision, end_revision):
@@ -894,3 +990,20 @@
properties_handler_registry = registry.Registry()
+
+# adapters which revision ids to log are filtered. When log is called, the
+# log_rev_iterator is adapted through each of these factory methods.
+# Plugins are welcome to mutate this list in any way they like - as long
+# as the overall behaviour is preserved. At this point there is no extensible
+# mechanism for getting parameters to each factory method, and until there is
+# this won't be considered a stable api.
+log_adapters = [
+ # core log logic
+ _make_batch_filter,
+ # read revision objects
+ _make_revision_objects,
+ # filter on log messages
+ _make_search_filter,
+ # generate deltas for things we will show
+ _make_delta_filter
+ ]
=== modified file 'bzrlib/tests/test_log.py'
--- a/bzrlib/tests/test_log.py 2008-07-09 20:15:29 +0000
+++ b/bzrlib/tests/test_log.py 2008-08-21 04:12:02 +0000
@@ -692,8 +692,8 @@
self.build_tree(['a'])
wt.add('a')
b.nick = 'test-line-log'
- wt.commit(message='add a',
- timestamp=1132711707,
+ wt.commit(message='add a',
+ timestamp=1132711707,
timezone=36000,
committer='Line-Log-Formatter Tester <test at line.log>')
logfile = file('out.tmp', 'w+')
More information about the bazaar-commits
mailing list