New benchmakt tests

[Robert Collins]

On Mon, 2006-07-24 at 09:28 +0200, Jan Balster wrote:
> Hi all,
> 
> as part of the performance enhancements efforts there are to be
> a lot of new benchmark tests.  I went ahead and tackled a few
> in a simple way.  It is based on bzr.dev and i am happy for
> comments and review:
> 
>     http://codespeak.net/bzr/branches/bzr.usecases_bench/
> 
> The whole benchmark run takes some time, but
> there is potential to make it faster.

Hi, this looks ok in broad terms, but there are some things that concern
me.

There are a couple of typos and funny formatting - just read through
your diff and you should spot them easily ;). (or I'll do that once the
bigger issue below is addressed)

More importantly though is the BenchmarkConfig changes - what are these
in aid of ? 

And finally, the bundle tests are all not-quite there in two respects.
Firstly they are testing the end user api - and we want bundles tested
as a library tool, because the smart server will be using them in that
manner. Secondly, they are testing the creation of a bundle with only
one revision - what we need is creating a bundle for some data points in
a space which there are currently a number of interesting dimensions:
Number of files in the bundle x number of files in the tree x number of
revisions in the bundle x number of revisions in the tree.

Of those four dimensions, the total number of revisions in the tree is
probably not all that interesting *at the moment* - but is useful to
benchmark because its a likely scaling dimension.

So to parameterise this:
file count in bundle:
 bundle with few files (say 5), bundle with a number of files (say 100),
bundle with many files (10K) 

file count in tree:
small tree (say 5 files)
moderate tree( say 1000 files)
huge tree (10K files)

number of revisions in bundle:
 1 revision
 500 revisions
 10K revisions

which is 27 (3*3*3) combinations. I think all the combinations are all
interesting, even though creating a tree with 10K revisions and 10K
files will take some time.


Rob


-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060727/426d63f3/attachment.pgp