Relation ordering: Does it matter?

[Gustavo Niemeyer]

Thanks for the debate Clint. Honestly appreciating it.

On Tue, Feb 7, 2012 at 06:15, Clint Byrum <clint at ubuntu.com> wrote:
> keystone<->nova: joined+changed
>  keystone has no db, sets nothing
> nova<->keystone: joined+changed
>  nothing happens
>
> keystone<->mysql: joined+changed
>  keystone configures database, starts keystone
>    for server in relation-list --name=nova

Right, something resembling this:

provides:
    keystonerelation:
        interface: keystoneiface

for r in $(relation-list --name=keystonerelation; do
    relation-set --relation=$r ready=true
done

> This seems like it would work and solve the cited problem.  I think this
> would make charms a little harder to write than ordering, but perhaps
> thats the price for complex relationships. Another thought is to make
> this simpler with a helper, something like 'relation-defer' to cause
> the hook to be retried after the next succesful relation establishment.

Retrying blindly just because a relation changed is a very ad-hoc
behavior, and will encourage people to not think through what they're
doing.

What is being suggested here is a very simple protocol: when I have a
database, I can allow clients to connect.

> Indeed, I hope we can tumble down a little bit, as not having to do the
> whole dance in charms would be quite welcome... when I say that a package
> Depends on something, I can expect that it is there and working when
> the postinst runs. This makes postinst maintainer scripts much easier
> to write... though I do understand that doing things on one system is a
> lot more straightforward than coordinating them across many.

Exactly.

>> - Relations are optional, so they may become steady simply because the
>>    admin hasn't finished the add-relation commands yet.
>
> I'm still very confused why we call them 'requires' if they are optional.
> I have always assumed that eventually we'd start enforcing these a
> bit more, and making use of a "consumes" section, or "optional: true"
> attribute.

It's an unfortunate historical fact. They were going to be strict
dependencies, and we were going to have optional: true, but by the
time we've put our heads up, it was too late. Today, we already have
relations, and they already work the way they do. We can't invert the
meaning without breaking the world. We'll most likely have actually
required relations at some point, and for them we'll need a flag such
as "enforce: true".

> Anyway, a relationship that does not exist would not seem to me to be
> "steady", and therefore, would not be considered ready.  So relating
> anything to that service's provided interfaces would result in that
> relationship waiting for the user to 'add-relation', perhaps in a new
> state something like 'wait-for-requires'.

That's not an optional relation, and thus we can't have such behavior
today because all of our relations are optional.

>> - Another consequence of optional relations is that services generally
>>    will work without the requirements alive, in many cases. We shouldn't
>>    make them unavailable just because a given relation isn't around.
>>
>
> Mediawiki serves a wiki w/o memcached, this is true. However, its
> sessions are stored on disk without memcached, and so, an add-unit
> produces a broken 'website' relation to haproxy, since it is no longer
> stateless. If we had this ordering in place, the http relation would
> never be established unless the charm author marked memcached optional
> or the admin established the required relationship to memcached.

Again, all of our relations are optional today. We can brainstorm
about what that feature might look like, but this doesn't work with
what we have in place now, and we'll have to specify all the edges of
required relations themselves before we get to the ordering aspect.

> So I think it would make sense to have some way to say "this relation
> cannot be satisfied without that required relation". Perhaps we can
> be more direct and make it
>
>
> provides:
>  website:
>    interface: http
>    depends: [ memcached, mysql ]

This looks like a more realistic and quite interesting idea. I'd be
happy to see a more detailed specification about it, including details
about the life cycle of one relation according to changes that happen
in the others.

>> - Machines stop/die and get restarted. No matter what order we
>>    establish relations, the software has to know how to reestablish
>>    itself to a live instance on the other side.
>>
>
> If the database is dead, and I start trying to relate things to keystone,
> I'd expect the hooks to error, as keystone could not satisfy the needs. An
> admin would expect to have to use resolved --retry to fix problems caused
> by a dead machine.

If the topology is fully established and a machine dies, the relation
must be departed with the specific service unit that is running on
that machine. This is not a hook error, and resolved --retry is not
necessary.

> Upon recovery, right now with juju, there'd be no event to actually inform
> keystone that the database is back and you can run relation-set again. So

relation-joined is the event that should be called in this scenario
again. The behavior may be a bit rough still since we're still
polishing watches, but that's what we should be walking towards, and I
believe we are already close to that.

> My thinking is purely that we can eliminate complexity from charms by
> doing some work for the user inside juju.

Entirely agreed. I'm all for making charmers' life a sweet one.

>> - It's easy to find different use cases that require a different
>>    notion of ordering requirements.
>>
>> - Loops may easily happen depending on the topology
>
> Indeed, circular dependencies are the devil... however, we have the
> dependency graph available to us, since the user has built it for us with
> add-relation. So we can pinpoint the very moment at which the circle is
> completed, and note this to the user.. "Circular Dependency detected."

The point is that a circular dependency is not a problem today.

A silly example:

    logger => mongodb (for storing consolidated data)
    mongodb <= logger (for gathering logs of mongodb itself)

> Perhaps the state would be changed to something like 'circular-depends'
> and resolved would be used to kick these off in the necessary order.

resolved is about fixing hooks whose execution failed. Nothing we've
been discussing so far should require it.

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I'm not absolutely sure of anything.