Rev 2634: (robertc) Add FileNames a managed collection of names to get file-discovery without transport listing. in file:///home/pqm/archives/thelove/bzr/%2Btrunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Fri Jul 20 02:53:49 BST 2007
At file:///home/pqm/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 2634
revision-id: pqm at pqm.ubuntu.com-20070720015347-eaeqmggngaemmbde
parent: pqm at pqm.ubuntu.com-20070719224856-pv0v2n7tmgpypfmf
parent: robertc at robertcollins.net-20070720005841-xnu6um6vx0n41h0k
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Fri 2007-07-20 02:53:47 +0100
message:
(robertc) Add FileNames a managed collection of names to get file-discovery without transport listing.
added:
bzrlib/file_names.py file_collection.py-20070714100753-j2zz4ahtk331k5zm-1
bzrlib/tests/test_file_names.py test_file_collection-20070714093417-5gc9d821to85zo4t-1
bzrlib/transport/unlistable.py unlistable.py-20070714093417-5gc9d821to85zo4t-2
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/tests/__init__.py selftest.py-20050531073622-8d0e3c8845c97a64
bzrlib/transport/__init__.py transport.py-20050711165921-4978aa7ce1285ad5
------------------------------------------------------------
revno: 2617.4.8
merged: robertc at robertcollins.net-20070720005841-xnu6um6vx0n41h0k
parent: robertc at robertcollins.net-20070720005743-1s083l5cvp9823nj
parent: pqm at pqm.ubuntu.com-20070719224856-pv0v2n7tmgpypfmf
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Fri 2007-07-20 10:58:41 +1000
message:
Merge bzr.dev.
------------------------------------------------------------
revno: 2617.4.7
merged: robertc at robertcollins.net-20070720005743-1s083l5cvp9823nj
parent: robertc at robertcollins.net-20070719064046-3mbyfeb6ioaw4htk
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Fri 2007-07-20 10:57:43 +1000
message:
Tweak docstrings after over-aggressive search n replace.
------------------------------------------------------------
revno: 2617.4.6
merged: robertc at robertcollins.net-20070719064046-3mbyfeb6ioaw4htk
parent: robertc at robertcollins.net-20070719063913-l1afgl876zpaysf4
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Thu 2007-07-19 16:40:46 +1000
message:
self._names.discard wins over conditional remove - thanks Aaron.
------------------------------------------------------------
revno: 2617.4.5
merged: robertc at robertcollins.net-20070719063913-l1afgl876zpaysf4
parent: robertc at robertcollins.net-20070719033717-2ajc3s1ua9wq2a9k
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Thu 2007-07-19 16:39:13 +1000
message:
Reinstate the fakenfs+ decorator registration.
------------------------------------------------------------
revno: 2617.4.4
merged: robertc at robertcollins.net-20070719033717-2ajc3s1ua9wq2a9k
parent: robertc at robertcollins.net-20070714101338-eu0cblg5ds2q2him
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Thu 2007-07-19 13:37:17 +1000
message:
Rename to FileNames as per review.
------------------------------------------------------------
revno: 2617.4.3
merged: robertc at robertcollins.net-20070714101338-eu0cblg5ds2q2him
parent: robertc at robertcollins.net-20070714100801-bo0gezkinkusmtr3
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Sat 2007-07-14 20:13:38 +1000
message:
Check loading empty collections and support removing items.
------------------------------------------------------------
revno: 2617.4.2
merged: robertc at robertcollins.net-20070714100801-bo0gezkinkusmtr3
parent: robertc at robertcollins.net-20070714093531-n1kt1ch73qxhfw8i
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Sat 2007-07-14 20:08:01 +1000
message:
Add FileCollection support class.
------------------------------------------------------------
revno: 2617.4.1
merged: robertc at robertcollins.net-20070714093531-n1kt1ch73qxhfw8i
parent: pqm at pqm.ubuntu.com-20070713074627-93zxs9uh528y0fki
committer: Robert Collins <robertc at robertcollins.net>
branch nick: file-collection
timestamp: Sat 2007-07-14 19:35:31 +1000
message:
Add a new transport decorator unlistable+ for testing.
=== added file 'bzrlib/file_names.py'
--- a/bzrlib/file_names.py 1970-01-01 00:00:00 +0000
+++ b/bzrlib/file_names.py 2007-07-20 00:57:43 +0000
@@ -0,0 +1,81 @@
+# Copyright (C) 2007 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+"""A collection of file names which is persisted on a transport."""
+
+from bzrlib.lazy_import import lazy_import
+lazy_import(globals(), """
+from bzrlib import (
+ errors,
+ )
+""")
+
+
+class FileNames(object):
+ """A collection of file names.
+
+ The file names are persisted to a file on a transport, and cand be
+ added, removed and initialised.
+
+ The set of names are stored in a flat file, one name per line.
+ New names are allocated sequentially.
+ Initialisation creates an empty file.
+
+ This is intended to support management of modest numbers of files in
+ write-locked environments which may be read from unlistable transports.
+
+ The save method must be called to cause the state to be saved to the
+ transport.
+
+ Finally, load is used to obtain a previously saved set.
+ """
+
+ def __init__(self, transport, index_name):
+ """Create a collection on transport called index_name."""
+ self._transport = transport
+ self._index_name = index_name
+ self._names = None
+ self._cap = 10000
+
+ def allocate(self):
+ for number in xrange(self._cap):
+ if str(number) not in self._names:
+ self._names.add(str(number))
+ return str(number)
+ raise errors.BzrError('too many files')
+
+ def initialise(self):
+ """Initialise the collection."""
+ self._names = set()
+
+ def load(self):
+ """Load the names from the transport."""
+ self._names = set(self._transport.get_bytes(
+ self._index_name).split('\n'))
+ self._names.discard('')
+
+ def names(self):
+ """What are the names in this collection?"""
+ return frozenset(self._names)
+
+ def remove(self, name):
+ """Remove name from the collection."""
+ self._names.remove(name)
+
+ def save(self):
+ """Save the set of names."""
+ self._transport.put_bytes(self._index_name, '\n'.join(self._names))
+
=== added file 'bzrlib/tests/test_file_names.py'
--- a/bzrlib/tests/test_file_names.py 1970-01-01 00:00:00 +0000
+++ b/bzrlib/tests/test_file_names.py 2007-07-19 03:37:17 +0000
@@ -0,0 +1,103 @@
+# Copyright (C) 2007 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+"""Tests for the FileNames class."""
+
+from bzrlib import errors
+from bzrlib.file_names import FileNames
+from bzrlib.tests import TestCaseWithMemoryTransport
+from bzrlib.transport import get_transport
+
+
+class TestFileNames(TestCaseWithMemoryTransport):
+
+ def test_initialise(self):
+ t = self.get_transport()
+ for name in ('index', '00index'):
+ names = FileNames(t, name)
+ names.initialise()
+ self.assertFalse(t.has(name))
+ names.save()
+ self.assertEqual('', t.get_bytes(name))
+
+ def test_allocate_trivial(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ name = names.allocate()
+ self.assertEqual('0', name)
+ self.assertFalse(t.has('index'))
+ name = names.allocate()
+ self.assertEqual('1', name)
+ self.assertFalse(t.has('index'))
+
+ def test_allocate_overrun(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ names._cap = 5
+ for number in xrange(5):
+ name = names.allocate()
+ self.assertRaises(errors.BzrError, names.allocate)
+
+ def test_load(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ names.allocate()
+ names.allocate()
+ names.save()
+ names = FileNames(t, 'index')
+ names.load()
+ self.assertEqual(set(['0', '1']), names.names())
+
+ def test_load_empty(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ names.save()
+ names = FileNames(t, 'index')
+ names.load()
+ self.assertEqual(set(), names.names())
+
+ def test_names(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ names.allocate()
+ names.allocate()
+ self.assertEqual(set(['0', '1']), names.names())
+
+ def test_names_on_unlistable_works(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ names.allocate()
+ names.allocate()
+ names.save()
+ names = FileNames(
+ get_transport('unlistable+' + self.get_url()), 'index')
+ names.load()
+ self.assertEqual(set(['0', '1']), names.names())
+
+ def test_remove(self):
+ t = self.get_transport()
+ names = FileNames(t, 'index')
+ names.initialise()
+ name1 = names.allocate()
+ name2 = names.allocate()
+ names.remove(name1)
+ self.assertEqual(set([name2]), names.names())
=== added file 'bzrlib/transport/unlistable.py'
--- a/bzrlib/transport/unlistable.py 1970-01-01 00:00:00 +0000
+++ b/bzrlib/transport/unlistable.py 2007-07-14 09:35:31 +0000
@@ -0,0 +1,52 @@
+# Copyright (C) 2005, 2006 Canonical Ltd
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+"""Transport implementation that disables listing to simulate HTTP cheaply."""
+
+import bzrlib.errors as errors
+from bzrlib.transport import Transport
+from bzrlib.transport.decorator import TransportDecorator, DecoratorServer
+
+
+class UnlistableTransportDecorator(TransportDecorator):
+ """A transport that disables file listing for testing."""
+
+ @classmethod
+ def _get_url_prefix(self):
+ """Unlistable transports are identified by 'unlistable+'"""
+ return 'unlistable+'
+
+ def iter_files_recursive(self):
+ Transport.iter_files_recursive(self)
+
+ def listable(self):
+ return False
+
+ def list_dir(self, relpath):
+ Transport.list_dir(self, relpath)
+
+
+class UnlistableServer(DecoratorServer):
+ """Server for the UnlistableTransportDecorator for testing with."""
+
+ def get_decorator_class(self):
+ return UnlistableTransportDecorator
+
+
+def get_test_permutations():
+ """Return the permutations to be used in testing."""
+ return [(UnlistableTransportDecorator, UnlistableServer),
+ ]
=== modified file 'NEWS'
--- a/NEWS 2007-07-19 13:13:06 +0000
+++ b/NEWS 2007-07-20 00:58:41 +0000
@@ -69,6 +69,12 @@
index performance is not optimised, however the API is stable to allow
development on top of the index. (Robert Collins)
+ * New transport decorator 'unlistable+' which disables the list_dir
+ functionality for testing.
+
+ * New ``file_names.FileNames`` support class which mananges names
+ for unlistable transport situations. (Robert Collins)
+
TESTING:
* Remove selftest ``--clean-output``, ``--numbered-dirs`` and
=== modified file 'bzrlib/tests/__init__.py'
--- a/bzrlib/tests/__init__.py 2007-07-19 13:13:06 +0000
+++ b/bzrlib/tests/__init__.py 2007-07-20 00:58:41 +0000
@@ -2262,7 +2262,6 @@
'bzrlib.tests.test_commit_merge',
'bzrlib.tests.test_config',
'bzrlib.tests.test_conflicts',
- 'bzrlib.tests.test_pack',
'bzrlib.tests.test_counted_lock',
'bzrlib.tests.test_decorators',
'bzrlib.tests.test_delta',
@@ -2273,6 +2272,7 @@
'bzrlib.tests.test_escaped_store',
'bzrlib.tests.test_extract',
'bzrlib.tests.test_fetch',
+ 'bzrlib.tests.test_file_names',
'bzrlib.tests.test_ftp_transport',
'bzrlib.tests.test_generate_docs',
'bzrlib.tests.test_generate_ids',
@@ -2309,6 +2309,7 @@
'bzrlib.tests.test_options',
'bzrlib.tests.test_osutils',
'bzrlib.tests.test_osutils_encodings',
+ 'bzrlib.tests.test_pack',
'bzrlib.tests.test_patch',
'bzrlib.tests.test_patches',
'bzrlib.tests.test_permissions',
=== modified file 'bzrlib/transport/__init__.py'
--- a/bzrlib/transport/__init__.py 2007-07-13 02:23:34 +0000
+++ b/bzrlib/transport/__init__.py 2007-07-19 06:39:13 +0000
@@ -1298,6 +1298,9 @@
register_transport_proto('fakenfs+')
register_lazy_transport('fakenfs+', 'bzrlib.transport.fakenfs', 'FakeNFSTransportDecorator')
+register_transport_proto('unlistable+')
+register_lazy_transport('unlistable+', 'bzrlib.transport.unlistable', 'UnlistableTransportDecorator')
+
register_transport_proto('brokenrename+')
register_lazy_transport('brokenrename+', 'bzrlib.transport.brokenrename',
'BrokenRenameTransportDecorator')
More information about the bazaar-commits
mailing list