More ideas about re-designing config files

[vila]

>>>>> Martin Pool <mbp at canonical.com> writes:

    > On 31 March 2011 18:02, vila <v.ladeuil+lp at free.fr> wrote:
    >>    > In pseudocode, it's
    >> 
    >>    >   get_config_stack(url=lockdir_url).get_option_as_boolean('bzr.auto_break_dead_locks')
    >> 
    >>    > is that enough?
    >> 
    >>        config.regsitry.get('bazaar', location).get('bzr.auto_break_dead_lock', as=bool)
    >> 
    >> is roughly the target.
    >> 
    >> Except that Global(location) is probably wrong. While 'bazaar.conf' (the
    >> Store) can provide default values for various contexts, these contexts
    >> are probably *also* getting options from different stores.
    >> 
    >> There is certainly an already known object that could provide the
    >> location, let say the working tree:
    >> 
    >>        wt.config.get('bzr.auto_break_dead_lock', as=bool)
    >> 
    >> knowing that at lock time (hu ho, breaking lock here remember, you
    >> really had to start with such a nasty example... ;) the working tree can
    >> provide the location itself to build a config stack as:
    >> 
    >> * command-line options
    >> * os.environ
    >> * a stack for matching sections in locations.conf
    >> * a stack for matching sections in tree.conf
    >> * a stack for matching sections in bazaar.conf
    >> 
    >> So basically the new design provides the configs applying to a given
    >> context when the objects themselves are built (working trees, branches,
    >> repos, etc).

    > Having a config stack available on key objects (branch, etc) that
    > looks up the right configuration for that object sounds good; it
    > should also give the right cache life time.  We'll have to make sure
    > that just building the stack does not eagerly load all the config
    > files because it's probably quite possible to use an object without
    > needing its config.

+1

Getting an option should be lazy.

The tricky part is asserting that the option exist or not, so the first
check will trigger getting the config file, but from there, the proposed
design *allows* to guarantee that it will occur only once whether the
actual one just re-read the file.

    > If we create the config stack when the object is created we can
    > put in just one place the policy about which files are consulted.

/me nods

    > Let's call it .config_stack, to make it clear that
    > branch.config_stack is everything that applies to the branch, not
    > just the branch.conf.  That would be a good thing to put in the
    > developer guide that we have this pattern.

/me nods

But I'll go further and just call it .config because that's the only
relevant config that the *code* should care about.

Likewise, the user should always be consistently be presented the full
config stack when dealing with a specified branch (when using 'bzr
config').

>From there, the modifications to the 'branch.conf' file itself should be
unambiguous.

    > Some objects like LockDir will want to be configurable but don't
    > really have an opinion about where their configuration should be
    > stored.  Perhaps when eg the Branch constructs one it can tell the
    > lockdir where to gets its configuration.  Perhaps following that
    > pattern of passing things down into lower-level objects will be
    > enough to avoid ever needing actual globals for it.

This is my feeling too. The config options for lock themselves are
trickier as config files need a lock to be accessed so we may not be
able to provide as much flexibility there.

At worst, we'll end up with a specific stack:

* command-line options

* os.environ

* bzrlib (or option registry) default values

    > It seems like we should then also be able to do away with eg
    > BranchConfig and just have it be an instance of a Store that's
    > addressing a particular file.

yup.

    > Using the registry of options to set the default type interpretation
    > would be good.

yup, I didn't elaborate on that but a few ideas sound promising:

* I want a bool from this option and give me 'True' if it's not defined.

* I want a bool but if the option is not defined or its value is 'ask',
  the user should be prompted.

* I want a dict for all values starting with this prefix. This is
  requested by bialix for qbzr. I'm not fully convinced yet unless a lot
  of constraints are added: a single level dict, all values are strings,
  etc. And even there deciding if a dict can be built from options
  coming from different sections (to provide default values) still
  sounds complicated.

Defining such conversions at the option level instead of the config
level (as it is today, get_editor, get_mail_client, etc) is probably the
main design shift I'm proposing here.

    > The registry of options needs to be done with care for lazy
    > loading.

hehe, exactly :) I'm close to think that the registration should
*always* be lazy (jelmer already addressed the requested changes there
as far as registry features are concerned with his recent
modifications).

Thanks for the positive feedback, it seems we're indeed on the same
page, that's nice to hear :-)

As mentioned on IRC, but maybe worth repeating there:

This proposal is a long term goal for the config options, some features
won't be implemented yet, mostly:

* minimizing IOs (this requires more analysis about *when* we really
  care about possible conflicts between concurrent modifications), so
  far the known case is two processes updating the same config file and
  this has been fixed in the current implementation by using
  locks. That's good enough for now and for the foreseeable future. But
  once we start to aggressively reduce the IOs, this may raise *some*
  issues ;)

* the option registry doesn't exist yet and registering options
  shouldn't be mandatory *anyway*. The expected features there are
  mainly to generate help and centralize type interpretation. We've been
  fine without that so far so this isn't a requirement for a first
  implementation.

So the minimal implementation plan I'm targeting is:
* sections (read-only and mutable)
* Abstract Store
* Stores for existing configs (global, location, branch)
* Stacks for existing configs (global, location, branch)

And even that can start with just global.

That should give us enough dog-fooding to work on the deployment and
migration issues.

    Vincent