Ensemble and configuration

[Shawn Edmondson]

On Jun 10, 2011, at 11:13 AM, Gustavo Niemeyer wrote:

> Hello Shawn,
> 
> It's interesting that every once in a while we are in touch.  I recall
> talking to some of the developers of RPath back in the days I worked
> on RPM and APT-RPM.  More than half a decade later, and we have moved
> up the stack.  Good to be in touch again.
> 
>> Hi folks, I work on product management and strategy here at rPath.  We
>> are committed to adding model-driven orchestration to our system
>> management offering, and would love to find a compatible open-source
>> project to contribute to rather than build something new, so I've been
>> looking at Ensemble after hearing about it from Ben Saller.
> 
> That's great news, and thanks for reaching out.  I hope we can find
> ways to collaborate around the project.
> 

Same here, looking forward to it.  Glad you remember us!

> (...)
>> 1. Does it/will it require one instance of the ensemble manager VM per
> 
>>   environment?  If I'm managing many environments (such as at an
>>   ASP/SaaS vendor with many instances of a single-tenant app), I'd
>>   expect one manager to handle multiple environments.
> 
> Yes, we'll require an ensemble manager per environment, but there
> might be a couple of interesting points around this to clarify:
> 
> First, it won't need its own VM.  We'll certainly have single-machine
> deployments, for instance, and that means the manager has to be
> lightweight. The current per-VM approach was just a quick way for us
> to get going, but there are no technical walls that enforce that model
> in the system.
> 
> Then, and perhaps that's the most interesting point for your case, you
> can have several deployments of the same single-tenant app on the same
> environment.  You can even have several disconnected network graphs
> within the same deployment, with overlapping formulas (potentially
> partially overlapping).
> 

Sounds good.

>> 2. For the wordpress+mysql example in the doc, you start up each
>>   service independently, then add the relation.  Before you add the
>>   relation, w hat logical state is the wordpress service in -- is it
>>   something like "can't really start yet because I don't have a DB?"
> 
> Yes, that's right.  Ultimately, it's up to the formula author to
> decide what's the appropriate time to bring up the formula.
> Internally, Ensemble recognizes and leaves the service unit (the
> formula deployment) aware that the relation hasn't been established.
> The desired behavior on this case is really application-based, though.
> A web-server could politely come up online and let the user know that
> the setup is still going on, for instance, or even that it's an
> unplanned down time (the database might have gone away for whatever
> reason).
> 

Makes sense.  I like how it doesn't impose too much behavior on the
formula author.

>> 3. Looks like formulas are models for individual services, and
>>   formulas can refer to others as provides/requires, but there isn't
>>   an enclosing model for a multi-service application.  I expected to
>>   see some sort of overarching composition model for (say) a full
>>   wordpress deployment, where that model refers to the mysql service
>>   and the wordpress service.  Is that something planned, or seen as
>>   unnecessary?
> 
> That's very much necessary, and we believe we have a good plan for
> them.  We don't yet have documentation available, but you'll see us
> talking about these as "stacks".
> 
> Also note that, in this specific example, we'll have something else
> too: automatic dependency resolution.  We're postponing that feature
> only because we want to develop it in light of the remote formula
> repository mechanism working, so that we consider the full problem of
> non-universal knowledge while designing the algorithm.
> 

Excellent, good to hear.

Sounds like the project's vision is significantly larger than it appears
on the website to a casual browser.  An architecture preso or whitepaper
would be really valuable (and of course expensive!)

>> 4. In the example, wordpress/0 didn't automatically become related to
>>   mysql/0, but wordpress/1 did.  Why did those two behave
>>   differently?
> 
> We'd have to investigate what happened, but your assumption is correct
> about what should happen.  All the service units within a deployed
> service must respect the same relations, whether you deploy the
> service unit before the relation was established, or afterwards.
> 
>> 5. Are the project developers OK making it widely cross-platform over
>>   time (for not only other Linuxes, but also Windows)?
> 
> We're certainly not against the idea of porting it to other platforms,
> and actually I don't think today we have any hard dependencies on
> Ubuntu.  That said, our focus is naturally on providing users a very
> smooth experience on Ubuntu.  Those things don't have to conflict,
> though.

Makes sense.

> 
>> 6. You have a draft concept of config properties for services, which I
>>   really like.  Right now it is for what I call "write only" config
>>   -- the sysadmin specifies property values that get passed down to
>>   hooks for implementation.  (rPath config is similar today.)  I'd
> 
> I don't think that's accurate. See below.
> 
>>   like to see "read/write" config properties, plus config
>>   dependencies, that would let you solve this point problem:
>> 
>>     * Each application constellation needs an app server and a DB
>>       server.
>> 
>>     * The app server needs a JDBC string for the corresponding DB.
>> 
>>     * The DB gets its IP from DHCP.
>> 
>>     * How do I automatically configure each app+DB combo correctly?
> 
> I'm not sure I understand what the problem you see in that scenario
> is.  This sounds like a trivial relation:
> 
> myapp requires an interface jdbc named db
> mydb provides an interface jdbc named app
> 
> mydb has an app-relation-joined hook with:
> 
>    relation-set JDBCAddr=$ADDR
> 
> myapp has a db-relation-changed hook with:
> 
>    ADDR=$(relation-get JDBCAddr)
> 
> Is that all you need, or are you seeing an additional issue that I miss?
> 

That makes sense -- I think I was staring at the Service Configuration draft
(https://ensemble.ubuntu.com/docs/drafts/service-config.html) and missed 
relation values, which do solve that problem.

>> We're leaning toward building something like that, but of course would
>> rather see it within an existing open-source project if possible.
> 
> It looks like we already cover it, and we are open to collaboration for sure.
> 
> Thanks again for getting in touch, Shawn.
> 

Thanks for the patient answers!  I'll start playing with a build, look forward
to getting deeper into this.

Shawn

> -- 
> Gustavo Niemeyer
> http://niemeyer.net
> http://niemeyer.net/blog
> http://niemeyer.net/twitter