[storm] closing connections

[Jamu Kakar]

Hi Jason,

On Tue, Apr 22, 2014 at 4:19 PM, Jason Novotny <jason.novotny at gmail.com> wrote:
> I'm trying to wrap Storm in a class called Repository that is so far pretty
> simple:
>
> class Repository(object):
>
>     def __init__(self, url):
>         database = create_database(url)
>         self.store = Store(database)
>
>     def update(self, domain):
>         self.store.add(domain)
>         self.store.commit()
>
>     def findById(self, klasses, id):
>         conds = [k.id == id for k in klasses]
>         result = self.store.find(tuple(klasses), *conds)
>         klass_dict = dict((r.__class__, r) for r in result.one())
>         return klass_dict
>
> But I'm wondering where is the best place in the application to close a
> connection, can I use a destructor or something? Note, I'm not building a
> webapp, in fact I'm just writing python "tasks" that will execute and
> complete before another task gets launched so looks like closing any db
> connections is pretty critical.

Will you have more than one instance of Repository in your
application?  Generally speaking, you should call create_database()
once in your application and then use one Store per thread.  It's
pretty common to use ZStorm (even if you don't use Zope) to manage
stores.  You can do something like:

  from storm.zope.interfaces import IZStorm
  from storm.zope.zstorm import ZStorm
  from zope.component import getUtility, provideUtility

  def setup_store():
      zstorm = ZStorm()
      provideUtility(zstorm)
      url = "mysql://..."
      zstorm.set_default_uri('main', url)

  def get_store():
      zstorm = getUtility(IZStorm)
      return zstorm.get('main')

If you need to talk to different databases you can register them with
different names.  'main' is just an example of using a single
database.  Anyway, if you have something like that you can call
get_store() in a thread and the right thing will happen.  You'll want
to make sure you deal with transactions carefully, which will look
something like:

  try:
      store = get_store()
      # Do work
  except:
      store.rollback()
      raise
  else:
      store.commit()

If you want to close a Store's connection you can call Store.close,
but generally speaking you want to always use a store instance in the
same thread (don't share them between threads) and manage the number
of threads you have to control the number of open connections you
have.  You probably don't want to be closing the database connections
regularly, because opening them is expensive.

Hope that helps!