[storm] Getting changes applied to an object to build an history

[James Henstridge]

On Thu, Oct 27, 2011 at 8:23 PM, Vincent Alquier
<vincent.alquier at gmail.com> wrote:
> Thanks for your answer, and thanks for pointing out the problems I
> could meet recording changes with Storm.
>
> 2011/10/27 James Henstridge <james at jamesh.id.au>:
>> On Fri, Oct 21, 2011 at 1:25 AM, Vincent Alquier
>> <vincent.alquier at gmail.com> wrote:
>>> First of all, sorry for my approximate english...
>>>
>>> For a project I am currently working on, I am trying to migrate from a
>>> homemade ORM to storm. One of my needs is to get all changes applied
>>> to an object on flush. The goal is to fill an history table with any
>>> changes.
>>>
>>> Is it safe to call Store._get_changes_map into the __storm_pre_flush__
>>> hook ? And then, to save the changes into the __storm_flushed__ hook ?
>>
>> The underscore at the start of the _get_changes_map() method indicates
>> that the method is private, so it would be better to find some other
>> method to do what you want.
>
> I understand this is not recommended at all to use a private method
> outside its context. What I did is a bit better, but not recommended
> neither : I did override the _get_changes_map method to notify me of
> changes. This solution is very vulnerable to changes in Storm.
>
> class MyOwnStore(storm.store.Store):
>    [...]
>
>    def _get_changes_map(self, obj_info, adding=False):
>        changes = super(Store, self)._get_changes_map(obj_info, adding)
>        self.changeManager.notify(obj_info, changes)
>        return changes

I would strongly suggest against using this method, as mentioned earlier.

>> One thing to keep in mind is that it is possible to make changes in
>> Storm that won't trigger these flush hooks, such as through the
>> ResultSet.set() method (which generally corresponds to an UPDATE
>> statement acting on rows that may or may not have Python
>> representations), so if you require all changes to be recorded, you
>> might be best off going with database level triggers.
>
> Database triggers are massively used in my project, but can't cover my
> needs on this specific case.

Fair enough.


>> If you can't use triggers, and are not going to use the bulk update
>> operations Storm provides, then perhaps the validator feature might
>> cover your needs?  If you attach a validator to a property it will be
>> called each time there is a change, so you could use that to record
>> which fields have been changed.
>
> I didn't look at the validator feature. Thanks for the tip, it looks promising !

Here is a quick run-down:

1. a validator function is passed three arguments: a storm object, the
property name and the new value.
2. if the validator raises an exception, then the property will not be changed.
3. the validator returns the value that the property will be set to.
This lets the validator alter the value if necessary.

So it would be pretty easy to write a single validator function that
could record changes to arbitrary properties on a class, and then
apply it to the properties you want to capture changes to.


> Another point,
> I am a bit confused with the event mechanism. It looks like it have
> been thought for internal Storm needs, more than as a user feature.
> Some examples :
>  - store._event is private but not the obj_info.event.

The ObjectInfo structures are internal to Storm, rather than being a
public part of the API.  Since it is an internal API, there is less
reason to hide it (also, that attribute makes up part of the interface
of the ObjectInfo class).

If you are writing new variable classes there is a slight chance you
might end up dealing with the event system, but in many cases you
won't even run into it then.

>  - An event is emitted on "global store flush start", then on any "one
> flushed". Why not on "one flush start" and on "global store flushed" ?

Most property types have immutable values (e.g. strings, integers,
etc).  The only way to change such properties is to assign to them,
which is easy to track.  However, we also support some property types
with mutable values.  Since it is possible to change these values
without assigning to the property, those values need to be manually
inspected for changes when it comes time to flush changes to the
database.  The global pre-flush event is used to trigger such checks.

Some values in objects (such as database assigned object IDs) will not
be visible until the object is flushed.  The individual object
post-flush event is used to handle this case (e.g. a Reference object
can fill in a property on a related object once the object is
flushed).


>  - The __storm_flushed__ hook looks redundant with the "flushed" event.

The __storm_flushed__() method is a public API of Storm.  The event
system is not.

James.