[storm] Select results randomisation

James Henstridge james at jamesh.id.au
Mon Feb 11 14:40:35 GMT 2008


Hi everyone,

I've put together an in-progress branch to add select results
randomisation to Storm:
    https://bugs.launchpad.net/storm/+bug/94983

The patch adds a Store.set_randomise_order(value) method that can be
used to turn on the feature.  When turned on, the Select() expressions
generated by Store.find() will include RANDOM() as the last item (or
RAND() on MySQL), to randomise the order of the results (modulo any
requested ordering constraints).

This is essentially a port of an SQLObject feature we wrote to help
test Launchpad.  The problem is that when creating data for a test,
the data is generally inserted in a fixed order.  When running select
queries without any ORDER BY clause on the table, you will generally
get the results back in this same order.  This can easily lead to bugs
related to insufficient result sorting, since production data rarely
has such a nice natural ordering.

By adding RANDOM() to the end of the ORDER BY clause, we can make sure
that the tests do not depend on this natural ordering, so must perform
any required sorting.

As it stands, the branch works for simple queries.  I need to add a
few more tests for a few more cases:
 * interaction with set operations
 * distinct queries
 * aggregates

As a quick test, I tried turning randomisation on for the whole Storm
test suite.  As expected, it showed problems with some of the above
classes of queries, but it also showed up a number of tests that
currently depend on natural result ordering.  This should probably be
fixed ...

Comments are welcome (especially suggestions of better names for the API).

James.



More information about the storm mailing list