[storm] recovering from a transactionrollbackerror

[Stuart Bishop]

On Fri, May 15, 2009 at 2:44 PM, Martin DeMello <martindemello at gmail.com> wrote:

> try:
>  do non-db stuff
>  if success:
>    try:
>      do db stuff
>    except retry_exceptions:
>      redo db stuff
> except:
>  cleanup
>
> Since the non-db stuff is often not revertible if it has succeeded.
> Does python offer anything to help with this pattern? e.g. in ruby I
> could define an ensure_commit method and then pass it the db changes I
> want to make:
>
> def do_stuff
>  begin
>    do non_db stuff
>    ensure_commit {
>       do db_stuff
>    }
>  rescue
>    cleanup non_db stuff
>  end
> end
>
> Where ensure_commit would once and for all capture the pattern of
> retrying db commits till they succeeded or passed the retry cutoff and
> logged a failure.

You can do the same thing in Python, either as a normal function or as a function decorator.


def ensure_commit(func, *args, **kw):
    while True:
        try:
            rv = func(*args, **kw)
            store.commit()
            return rv
        except RETRY_EXCEPTIONS:
            pass

result = ensure_commit(do_stuff, argument1, argument2, keyword_arg1='boo')


def ensure_commit(func):
    "ensure_commit decorator"
    def do_ensure_commit(*args, **kw):
        while True:
            try:
                rv = func(*args, **kw)
                store.commit()
                return rv
            except RETRY_EXCEPTIONS:
                pass
    return do_ensure_commit

@ensure_commit
def do_stuff(foo):
    [...]

result = do_stuff(42)


If you are having trouble rolling back non-db stuff on failure, you might consider doing the database operations first, then the non-db-stuff, then the commit. You can even use two-phase commit to ensure the commit will succeed, but few people bother.


> Incidentally, to get the app working here-and-now while we make the
> more sweeping changes, I've taken to committing as often as possible,
> before and after every block of code that writes to the db, to keep
> transactions small. It seems to be working, though of course it's not
> the 100% reliable way to do it, and we will need to do the rewrite.

Its good to commit often if you are not risking your data integrity - long running transactions cause pain.


-- 
Stuart Bishop <stuart at stuartbishop.net>
http://www.stuartbishop.net/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/storm/attachments/20090515/6a2a9946/attachment.pgp