A simpler test framework?

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> I thought of one specific thing we could do, which is to support
> doctest-like black box tests, so you could write literally:
> 
>   % echo foo > foo
>   % bzr add
>   added foo
> 
> This would be pretty approachable for new people adding tests and also
> makes it very clear what's happening.  It could potentially be
> copy-and-pasted from a bug report with just a little cleanup.

I think we would want to avoid repeated invocation of bzr, so "% bzr
add" would need to do run_bzr under the hood, and there's a lot of magic
that way.  We'd need to implement a pseudo-shell, actually, so that it
would work on Windows.

> We could alternatively support this kind of thing not based on doctest
> but our own code, maybe providing an extended blackbox routine which
> takes a triplequoted string containing commands and output and
> evaluates it.  This could fit within regular tests, could be
> parameterized and could be automatically cleaned up.

There is also a new doctest-like thing: "manuel"
http://packages.python.org/manuel/

It seems to focus on extensibility.

> I do think, looking at for example Aaron's 'merge -i' proposal, it
> would have been nice to see some screenshot examples of it in use.

Fully agreed.  Testing this interactive stuff is pretty hard with our
current framework, but I'm not sure this would help that case.  Shelver
hides its prompts after they're answered, so its output is something like:

"Apply change? [yNfq?]y\r                     Apply 1 change(s)y\r
                "

etc.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpclsAACgkQ0F+nu1YWqI27wwCdGaZuwNlewy4td2PM1wzHyuzt
ZocAniR+LBbJxsEO5T8Br3ISz3Z36Pe7
=Kt60
-----END PGP SIGNATURE-----