Rev 3184: (robertc) Provide a smart server verb for get_parent_map against a in file:///home/pqm/archives/thelove/bzr/%2Btrunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Tue Jan 15 14:19:44 GMT 2008
At file:///home/pqm/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 3184
revision-id:pqm at pqm.ubuntu.com-20080115141934-3vujw0up5rc8e0gn
parent: pqm at pqm.ubuntu.com-20080115074456-0v1w54h783oq7n48
parent: robertc at robertcollins.net-20080115124543-v6sp6i2n3um710ec
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Tue 2008-01-15 14:19:34 +0000
message:
(robertc) Provide a smart server verb for get_parent_map against a
RemoteRepository, (Robert Collins)
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/smart/repository.py repository.py-20061128022038-vr5wy5bubyb8xttk-1
bzrlib/smart/request.py request.py-20061108095550-gunadhxmzkdjfeek-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
bzrlib/tests/test_smart.py test_smart.py-20061122024551-ol0l0o0oofsu9b3t-2
------------------------------------------------------------
revno: 3172.1.1.1.9
revision-id:robertc at robertcollins.net-20080115124543-v6sp6i2n3um710ec
parent: robertc at robertcollins.net-20080114224807-zhx2hiy2902zr7aw
parent: pqm at pqm.ubuntu.com-20080115074456-0v1w54h783oq7n48
committer: Robert Collins <robertc at robertcollins.net>
branch nick: integration
timestamp: Tue 2008-01-15 23:45:43 +1100
message:
Merge bzr.dev.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/graph.py graph_walker.py-20070525030359-y852guab65d4wtn0-1
bzrlib/smart/client.py client.py-20061116014825-2k6ada6xgulslami-1
bzrlib/smart/medium.py medium.py-20061103051856-rgu2huy59fkz902q-1
bzrlib/smart/protocol.py protocol.py-20061108035435-ot0lstk2590yqhzr-1
bzrlib/tests/test_graph.py test_graph_walker.py-20070525030405-enq4r60hhi9xrujc-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
bzrlib/tests/test_smart_transport.py test_ssh_transport.py-20060608202016-c25gvf1ob7ypbus6-2
bzrlib/transport/__init__.py transport.py-20050711165921-4978aa7ce1285ad5
bzrlib/transport/remote.py ssh.py-20060608202016-c25gvf1ob7ypbus6-1
doc/en/tutorials/using_bazaar_with_launchpad.txt using_bazaar_with_lp-20071211073140-7msh8uf9a9h4y9hb-1
doc/en/user-guide/controlling_registration.txt controlling_registra-20071121073725-0corxykv5irjal00-3
------------------------------------------------------------
revno: 3172.1.1.1.8
revision-id:robertc at robertcollins.net-20080114224807-zhx2hiy2902zr7aw
parent: robertc at robertcollins.net-20080114051607-8icqda4bvrc2u1xe
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Tue 2008-01-15 09:48:07 +1100
message:
Review feedback.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/smart/repository.py repository.py-20061128022038-vr5wy5bubyb8xttk-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
------------------------------------------------------------
revno: 3172.1.1.1.7
revision-id:robertc at robertcollins.net-20080114051607-8icqda4bvrc2u1xe
parent: robertc at robertcollins.net-20080114044608-bmse3mmsnp1663rf
parent: robertc at robertcollins.net-20080114030008-xdf5xvub5prv2zal
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Mon 2008-01-14 16:16:07 +1100
message:
Merge more-find-ghosts newer logic.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/debug.py debug.py-20061102062349-vdhrw9qdpck8cl35-1
bzrlib/fetch.py fetch.py-20050818234941-26fea6105696365d
bzrlib/graph.py graph_walker.py-20070525030359-y852guab65d4wtn0-1
bzrlib/help_topics/__init__.py help_topics.py-20060920210027-rnim90q9e0bwxvy4-1
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
bzrlib/reconfigure.py reconfigure.py-20070908040425-6ykgo7escxhyrg9p-1
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/tests/test_graph.py test_graph_walker.py-20070525030405-enq4r60hhi9xrujc-1
bzrlib/tests/test_osutils.py test_osutils.py-20051201224856-e48ee24c12182989
bzrlib/tests/test_reconfigure.py test_reconfigure.py-20070908040425-6ykgo7escxhyrg9p-2
------------------------------------------------------------
revno: 3172.1.1.1.6
revision-id:robertc at robertcollins.net-20080114044608-bmse3mmsnp1663rf
parent: robertc at robertcollins.net-20080111071215-n51r2omrs3e7216m
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Mon 2008-01-14 15:46:08 +1100
message:
Create new smart server verb Repository.get_parent_map.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/smart/repository.py repository.py-20061128022038-vr5wy5bubyb8xttk-1
bzrlib/smart/request.py request.py-20061108095550-gunadhxmzkdjfeek-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
bzrlib/tests/test_smart.py test_smart.py-20061122024551-ol0l0o0oofsu9b3t-2
------------------------------------------------------------
revno: 3172.1.1.1.5
revision-id:robertc at robertcollins.net-20080111071215-n51r2omrs3e7216m
parent: robertc at robertcollins.net-20080111055647-tdn8bkdri6q5i39w
parent: robertc at robertcollins.net-20080111043302-0pi5csqyr1ugry00
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Fri 2008-01-11 18:12:15 +1100
message:
Merge more-find-ghosts.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/fetch.py fetch.py-20050818234941-26fea6105696365d
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
------------------------------------------------------------
revno: 3172.1.1.1.4
revision-id:robertc at robertcollins.net-20080111055647-tdn8bkdri6q5i39w
parent: robertc at robertcollins.net-20080111053222-dxepbgap3vobkvqd
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Fri 2008-01-11 16:56:47 +1100
message:
Implement get_parent_map for RemoteRepository with caching, based on get_revision_graph.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
------------------------------------------------------------
revno: 3172.1.1.1.3
revision-id:robertc at robertcollins.net-20080111053222-dxepbgap3vobkvqd
parent: robertc at robertcollins.net-20080111044231-jxb35emhdivufnzg
parent: robertc at robertcollins.net-20080111052846-oxt52a13l5mu73wj
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Fri 2008-01-11 16:32:22 +1100
message:
Merge has_revisions and bzr.dev.
added:
bzrlib/tests/repository_implementations/test_has_revisions.py test_has_revisions.p-20080111035443-xaupgdsx5fw1q54b-1
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/repofmt/knitrepo.py knitrepo.py-20070206081537-pyy4a00xdas0j4pf-1
bzrlib/repofmt/pack_repo.py pack_repo.py-20070813041115-gjv5ma7ktfqwsjgn-1
bzrlib/repofmt/weaverepo.py presplitout.py-20070125045333-wfav3tsh73oxu3zk-1
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/smart/client.py client.py-20061116014825-2k6ada6xgulslami-1
bzrlib/smart/repository.py repository.py-20061128022038-vr5wy5bubyb8xttk-1
bzrlib/smart/request.py request.py-20061108095550-gunadhxmzkdjfeek-1
bzrlib/tests/repository_implementations/__init__.py __init__.py-20060131092037-9564957a7d4a841b
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
bzrlib/tests/test_smart.py test_smart.py-20061122024551-ol0l0o0oofsu9b3t-2
bzrlib/tests/test_smart_transport.py test_ssh_transport.py-20060608202016-c25gvf1ob7ypbus6-2
------------------------------------------------------------
revno: 3172.1.1.1.2
revision-id:robertc at robertcollins.net-20080111044231-jxb35emhdivufnzg
parent: robertc at robertcollins.net-20080110223405-uuaxxfwv44b3ptp0
parent: pqm at pqm.ubuntu.com-20080111025153-mqrn9i5o3hqezw75
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Fri 2008-01-11 15:42:31 +1100
message:
Merge get_parents full deprecation.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/graph.py graph_walker.py-20070525030359-y852guab65d4wtn0-1
bzrlib/tests/test_graph.py test_graph_walker.py-20070525030405-enq4r60hhi9xrujc-1
------------------------------------------------------------
revno: 3172.1.1.1.1
revision-id:robertc at robertcollins.net-20080110223405-uuaxxfwv44b3ptp0
parent: robertc at robertcollins.net-20080110220839-ze9uhnjp04h7y9mp
committer: Robert Collins <robertc at robertcollins.net>
branch nick: remote.graph
timestamp: Fri 2008-01-11 09:34:05 +1100
message:
Create a RemoteRepository get_graph implementation and delegate get_parents_map to the real repository.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
=== modified file 'bzrlib/remote.py'
--- a/bzrlib/remote.py 2008-01-11 05:28:46 +0000
+++ b/bzrlib/remote.py 2008-01-14 22:48:07 +0000
@@ -22,6 +22,7 @@
from bzrlib import (
branch,
errors,
+ graph,
lockdir,
repository,
revision,
@@ -264,6 +265,8 @@
self._lock_token = None
self._lock_count = 0
self._leave_lock = False
+ # A cache of looked up revision parent data; reset at unlock time.
+ self._parents_map = None
# For tests:
# These depend on the actual remote format, so force them off for
# maximum compatibility. XXX: In future these should depend on the
@@ -390,8 +393,13 @@
def get_graph(self, other_repository=None):
"""Return the graph for this repository format"""
- self._ensure_real()
- return self._real_repository.get_graph(other_repository)
+ parents_provider = self
+ if (other_repository is not None and
+ other_repository.bzrdir.transport.base !=
+ self.bzrdir.transport.base):
+ parents_provider = graph._StackedParentsProvider(
+ [parents_provider, other_repository._make_parents_provider()])
+ return graph.Graph(parents_provider)
def gather_stats(self, revid=None, committers=None):
"""See Repository.gather_stats()."""
@@ -462,6 +470,7 @@
if not self._lock_mode:
self._lock_mode = 'r'
self._lock_count = 1
+ self._parents_map = {}
if self._real_repository is not None:
self._real_repository.lock_read()
else:
@@ -499,6 +508,7 @@
self._leave_lock = False
self._lock_mode = 'w'
self._lock_count = 1
+ self._parents_map = {}
elif self._lock_mode == 'r':
raise errors.ReadOnlyError(self)
else:
@@ -558,6 +568,7 @@
self._lock_count -= 1
if self._lock_count > 0:
return
+ self._parents_map = None
old_mode = self._lock_mode
self._lock_mode = None
try:
@@ -726,6 +737,75 @@
self._ensure_real()
return self._real_repository.iter_files_bytes(desired_files)
+ def get_parent_map(self, keys):
+ """See bzrlib.Graph.get_parent_map()."""
+ # Hack to build up the caching logic.
+ ancestry = self._parents_map
+ missing_revisions = set(key for key in keys if key not in ancestry)
+ if missing_revisions:
+ self._parents_map.update(self._get_parent_map(missing_revisions))
+ return dict((k, ancestry[k]) for k in keys if k in ancestry)
+
+ def _response_is_unknown_method(self, response, verb):
+ """Return True if response is an unknonwn method response to verb.
+
+ :param response: The response from a smart client call_expecting_body
+ call.
+ :param verb: The verb used in that call.
+ :return: True if an unknown method was encountered.
+ """
+ # This might live better on
+ # bzrlib.smart.protocol.SmartClientRequestProtocolOne
+ if (response[0] == ('error', "Generic bzr smart protocol error: "
+ "bad request '%s'" % verb) or
+ response[0] == ('error', "Generic bzr smart protocol error: "
+ "bad request u'%s'" % verb)):
+ response[1].cancel_read_body()
+ return True
+ return False
+
+ def _get_parent_map(self, keys):
+ """Helper for get_parent_map that performs the RPC."""
+ keys = set(keys)
+ if NULL_REVISION in keys:
+ keys.discard(NULL_REVISION)
+ found_parents = {NULL_REVISION:()}
+ if not keys:
+ return found_parents
+ else:
+ found_parents = {}
+ path = self.bzrdir._path_for_remote_call(self._client)
+ for key in keys:
+ assert type(key) is str
+ verb = 'Repository.get_parent_map'
+ response = self._client.call_expecting_body(
+ verb, path, *keys)
+ if self._response_is_unknown_method(response, verb):
+ # Server that does not support this method, get the whole graph.
+ response = self._client.call_expecting_body(
+ 'Repository.get_revision_graph', path, '')
+ if response[0][0] not in ['ok', 'nosuchrevision']:
+ reponse[1].cancel_read_body()
+ raise errors.UnexpectedSmartServerResponse(response[0])
+ elif response[0][0] not in ['ok']:
+ reponse[1].cancel_read_body()
+ raise errors.UnexpectedSmartServerResponse(response[0])
+ if response[0][0] == 'ok':
+ coded = response[1].read_body_bytes()
+ if coded == '':
+ # no revisions found
+ return {}
+ lines = coded.split('\n')
+ revision_graph = {}
+ for line in lines:
+ d = tuple(line.split())
+ if len(d) > 1:
+ revision_graph[d[0]] = d[1:]
+ else:
+ # No parents - so give the Graph result (NULL_REVISION,).
+ revision_graph[d[0]] = (NULL_REVISION,)
+ return revision_graph
+
@needs_read_lock
def get_signature_text(self, revision_id):
self._ensure_real()
@@ -918,8 +998,7 @@
return self._real_repository._check_for_inconsistent_revision_parents()
def _make_parents_provider(self):
- self._ensure_real()
- return self._real_repository._make_parents_provider()
+ return self
class RemoteBranchLockableFiles(LockableFiles):
=== modified file 'bzrlib/smart/repository.py'
--- a/bzrlib/smart/repository.py 2008-01-02 08:09:32 +0000
+++ b/bzrlib/smart/repository.py 2008-01-14 22:48:07 +0000
@@ -30,6 +30,7 @@
SmartServerRequest,
SuccessfulSmartServerResponse,
)
+from bzrlib import revision as _mod_revision
class SmartServerRepositoryRequest(SmartServerRequest):
@@ -51,6 +52,64 @@
return self.do_repository_request(repository, *args)
+class SmartServerRepositoryGetParentMap(SmartServerRepositoryRequest):
+
+ def do_repository_request(self, repository, *revision_ids):
+ repository.lock_read()
+ try:
+ return self._do_repository_request(repository, revision_ids)
+ finally:
+ repository.unlock()
+
+ def _do_repository_request(self, repository, revision_ids):
+ """Get parent details for some revisions.
+
+ All the parents for revision_ids are returned. Additionally up to 64KB
+ of additional parent data found by performing a breadth first search
+ from revision_ids is returned.
+
+ :param repository: The repository to query in.
+ :param revision_ids: The utf8 encoded revision_id to answer for.
+ :return: A smart server response where the body contains an utf8
+ encoded flattened list of the parents of the revisions, (the same
+ format as Repository.get_revision_graph).
+ """
+ lines = []
+ repo_graph = repository.get_graph()
+ result = {}
+ queried_revs = set()
+ size_so_far = 0
+ next_revs = revision_ids
+ first_loop_done = False
+ while next_revs:
+ queried_revs.update(next_revs)
+ parent_map = repo_graph.get_parent_map(next_revs)
+ next_revs = set()
+ for revision_id, parents in parent_map.iteritems():
+ # adjust for the wire
+ if parents == (_mod_revision.NULL_REVISION,):
+ parents = ()
+ # add parents to the result
+ result[revision_id] = parents
+ # prepare the next query
+ next_revs.update(parents)
+ # Approximate the serialized cost of this revision_id.
+ size_so_far += 2 + len(revision_id) + sum(map(len, parents))
+ # get all the directly asked for parents, and then flesh out to
+ # 64K or so.
+ if first_loop_done and size_so_far > 65000:
+ next_revs = set()
+ break
+ # don't query things we've already queried
+ next_revs.difference_update(queried_revs)
+ first_loop_done = True
+
+ for revision, parents in result.items():
+ lines.append(' '.join((revision, ) + tuple(parents)))
+
+ return SuccessfulSmartServerResponse(('ok', ), '\n'.join(lines))
+
+
class SmartServerRepositoryGetRevisionGraph(SmartServerRepositoryRequest):
def do_repository_request(self, repository, revision_id):
=== modified file 'bzrlib/smart/request.py'
--- a/bzrlib/smart/request.py 2008-01-02 07:58:37 +0000
+++ b/bzrlib/smart/request.py 2008-01-14 04:46:08 +0000
@@ -318,6 +318,9 @@
request_handlers.register_lazy('Repository.gather_stats',
'bzrlib.smart.repository',
'SmartServerRepositoryGatherStats')
+request_handlers.register_lazy('Repository.get_parent_map',
+ 'bzrlib.smart.repository',
+ 'SmartServerRepositoryGetParentMap')
request_handlers.register_lazy(
'Repository.stream_knit_data_for_revisions',
'bzrlib.smart.repository',
=== modified file 'bzrlib/tests/test_remote.py'
--- a/bzrlib/tests/test_remote.py 2008-01-14 22:45:15 +0000
+++ b/bzrlib/tests/test_remote.py 2008-01-15 12:45:43 +0000
@@ -554,6 +554,62 @@
result)
+class TestRepositoryGetGraph(TestRemoteRepository):
+
+ def test_get_graph(self):
+ # get_graph returns a graph with the repository as the
+ # parents_provider.
+ responses = []
+ transport_path = 'quack'
+ repo, client = self.setup_fake_client_and_repository(
+ responses, transport_path)
+ graph = repo.get_graph()
+ self.assertEqual(graph._parents_provider, repo)
+
+
+class TestRepositoryGetParentMap(TestRemoteRepository):
+
+ def test_get_parent_map_caching(self):
+ # get_parent_map returns from cache until unlock()
+ # setup a reponse with two revisions
+ r1 = u'\u0e33'.encode('utf8')
+ r2 = u'\u0dab'.encode('utf8')
+ lines = [' '.join([r2, r1]), r1]
+ encoded_body = '\n'.join(lines)
+ responses = [(('ok', ), encoded_body), (('ok', ), encoded_body)]
+
+ transport_path = 'quack'
+ repo, client = self.setup_fake_client_and_repository(
+ responses, transport_path)
+ repo.lock_read()
+ graph = repo.get_graph()
+ parents = graph.get_parent_map([r2])
+ self.assertEqual({r2: (r1,)}, parents)
+ # locking and unlocking deeper should not reset
+ repo.lock_read()
+ repo.unlock()
+ parents = graph.get_parent_map([r1])
+ self.assertEqual({r1: (NULL_REVISION,)}, parents)
+ self.assertEqual(
+ [('call_expecting_body', 'Repository.get_parent_map',
+ ('quack/', r2))],
+ client._calls)
+ repo.unlock()
+ # now we call again, and it should use the second response.
+ repo.lock_read()
+ graph = repo.get_graph()
+ parents = graph.get_parent_map([r1])
+ self.assertEqual({r1: (NULL_REVISION,)}, parents)
+ self.assertEqual(
+ [('call_expecting_body', 'Repository.get_parent_map',
+ ('quack/', r2)),
+ ('call_expecting_body', 'Repository.get_parent_map',
+ ('quack/', r1))
+ ],
+ client._calls)
+ repo.unlock()
+
+
class TestRepositoryGetRevisionGraph(TestRemoteRepository):
def test_null_revision(self):
=== modified file 'bzrlib/tests/test_smart.py'
--- a/bzrlib/tests/test_smart.py 2008-01-02 07:58:37 +0000
+++ b/bzrlib/tests/test_smart.py 2008-01-14 04:46:08 +0000
@@ -919,6 +919,9 @@
smart.request.request_handlers.get('Repository.gather_stats'),
smart.repository.SmartServerRepositoryGatherStats)
self.assertEqual(
+ smart.request.request_handlers.get('Repository.get_parent_map'),
+ smart.repository.SmartServerRepositoryGetParentMap)
+ self.assertEqual(
smart.request.request_handlers.get(
'Repository.get_revision_graph'),
smart.repository.SmartServerRepositoryGetRevisionGraph)
More information about the bazaar-commits
mailing list