machine placement spec

Thu Nov 17 03:53:07 UTC 2011

> ...and here's the latest version, incorporating the recent discussions
> (I think; let me know if I've missed anything).

Great stuff. I'll highlight just a few last points, but overall we're
almost ready.

> Machine Placement
> =================
>
>
> Motivation
> ----------
>
> To fulfil a popular set of use cases, juju needs a mechanism to control
> placement of service units; to fulfil the simplest possible version of
> this story, we consider two possible cases:
>
> * User needs to specify minimum hardware requirements
> * User needs to specify other, context-dependent, hardware attributes
>
> This feature allows us to remove the temporary EC2 environment
> properties `default-image-id` and `default-machine-type`, and makes
> obsolete the requirement to interact with cobbler out-of-band when
> placing services and units in an orchestra environment.
>
>
> Constraints
> -----------
>
> We introduce "machine constraints", which are used to encode an
> administrator's hardware requirements. Constraints can be set for
> environments and services, with lookups for each key falling back from
> more specific to more general settings, with default values set by juju
> when otherwise unspecified [0]_. Changes to constraints do not affect
> any unit that's already been placed on a machine.
>
> Environment constraints, like other environment settings that are not
> related to access, will be controlled with a new command, `juju env-set`
> [1]_; the existing `juju set` command will be used for service
> constraints. Any number of constraints can be set in a single
> `--constraints` option, as a space-separate list of `key=value` pairs.
> An empty `value` is treated as "the juju default", allowing users to
> ignore environment settings at the service level without having to
> explicitly remember and use the juju default value.

The API for the change still feels a bit fuzzy. "juju set" today is
really about configuration. We're giving it a new meaning that doesn't
feel entirely clean.

Let's take this as an example:

    juju set --constraint ec2-type=small postgres username=joe

It just doesn't feel correct to have both of these in the same
namespace. The configuration options are entirely unrelated to the
constraints.

As a solution to avoid this, I suggest we create a new command called
set-constraints that handles that, specifically, with a syntax like
this:

    juju set-constraints ec2-type=m1.small ec2-zone=eu-west-1

The above changes the default for the environment. For changing a
specific service, the following would be used:

    juju set-constraints --service=postgres ec2-type=m1.large

And perhaps at some point:

    juju set-constraints --unit=postgres/0 ec2-type=m1.large

How does that feel?

> We also extend the syntax for `juju deploy`, such that `--constraints`
> is understood as above; these constraints will be set on the service
> before the first unit is deployed.

+1 on that.

> The constraints fall into two categories:
>
> * Generic constraints: based on the minimal properties exposed by every
> provider. These are:
>
>  * `cpu`: The minimum processing power of the machine, measured in `ECU
> http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud#Elastic_compute_units`_, defaulting to 1; any real number >= 0 is valid.
>  * `ram`: The minimum memory for the machine, defaulting to 512MB; any
> real number >= 0 and suffixed with M, G or T is valid.

Let's please call it "mem" as we've been discussing since that's how
most people relate to this concept, including the well known
providers.

>  In each case above, the value `0` is treated specially, and used to
> denote that the value doesn't matter at all. EC2 could, for example,
> special-case `cpu=0` to allow for deploying to a t1.micro without naming
> it explicitly, but this is not strictly necessary; in the general case,
> `<key>=0` simply means "do not filter on `<key>`.
>
>  When interpreting generic constraints, juju is expected to deploy
> machines with as little wastage as possible: that is to say, given a
> `ram=3G` constraint and two available machines with 4G and 6G
> respectively, juju would deploy to the 4G machine.
>
> * Provider constraints: based on the additional properties exposed by a
> given provider, which are not expected to be directly portable to other
> providers:
>
>  * For EC2, we expect to expose:
>
>    * `ec2-region`: the region in which to deploy new instances.
> Defaults to "us-east-1"; valid values are the current set of suported
> EC2 regions.

s/supo/suppo/

>    * `ec2-zone`: availability zone within the region. Defaults to "a";
> valid values depend on the current value of `ec2-region`.

It should default to unset, allowing the use of any zone. Considering
that Amazon shuffles zone names either way, we can't really use that
reliably.

>    * `ec2-instance-type`: instance type. Unset by default (so we fall
> back to the juju-level `ram` and `cpu` requirements); valid values are
> those machine types that EC2 has available in the given region.
>
>  * For orchestra, we expect to expose:
>
>    * `orchestra-name`: to allow admins to specify a specific instance
> by name, if required. Unset by default; the valid values are the set of
> instance names exposed by cobbler.
>    * `orchestra-classes`: a comma-separated list of cobbler
> mgmt-classes to which the instance must belong. Empty by default; the
> valid values are the existing cobbler mgmt-classes, excluding the
> current values of `available-mgmt-class` and `acquired-mgmt-class` (from
> the orchestra environment settings).
>
>  Provider constraints are only effective when used with the appropriate
> provider, and are silently ignored when specified with a different
> provider.
>
>  Note that provider constraints can overlap with generic constraints:
> it would not be possible to deploy a machine with the constraints
> `ram=8G` and `ec2-instance-type=m1.small`, because the requirements are
> contradictory.

This feels like a bad idea. We said we'd be ignoring provider-specific
constraints so the charms are portable to different providers. When
that's the case, the generic constraints will be useful, so it sounds
very reasonable to say that the most specific constraints take
precedence.

> Finally, please note that for the local LXC provider, given its nature
> and its intended use as a development tool, no constraints have any
> effect whatsoever.

s/local LXC/local/.. it's really about local rather than LXC.

> Issues
> ------
>
> Please note that this spec depends on features of juju and orchestra
> that are not present at the time of writing:
>
> * We need orchestra to expose `cpu` and `ram`; as long as the numbers
> are available, the orchestra team is free to expose them in whatever way
> is convenient; but the feature will be severaly hobbled if we don't have
> access at all.
>
> * Environment settings will need to be rearranged, and our handling of
> `environments.yaml` will need to change, so that we can separate an
> environment's access information from the runtime properties contolled
> by `juju env-set`.
>
>
> Potential Enhancements
> ----------------------
>
> The above spec is intended to be small and focused; several enhancements
> are possible and may be desirable in the future. They include:
>
>  * Unit-level constraints, specified at `juju add-unit` time, and
> inheriting naturally from service, environment, and juju.
>
>  * An additional generic constraint, `storage`: The minimum persistent
> disk space for the machine, defaulting to 4GB. Valid inputs as for
> `ram`.
>
> * Max constraints: allow generic constraints to also take the value
> `max`, meaning "the best available". (If you specify both `cpu=max` and
> `ram=max`, the constraints cannot be satisfied unless there is an
> available machine with the (equal) greatest amount of RAM and also the
> (equal) most processing power; in general, we would expect people to
> only set one constraint to `max`.)

Please capture also the concerns exposes during discussion about the
conflicts, or drop the details.

> * Additional provider constraints, such as:
>
>    * `orchestra-dns-name`: potentially useful for sysadmins who don't
> primarily think of their systems by `orchestra-name`.
>
> * Co-location constraints: for determining machine placement in terms of
> existing juju components. Possibilities include:
>
>  * `place-in=<machine-id>`: On a separate container in the machine with
> juju id `machine-id`.
>
>  * `place-with=<service-name>`: On a separate container in *any*
> machine with a unit of `service-name` deployed.
>
>  * `scale-with=<service-name>`: On a separate container in *every*
> machine running a unit of `service-name`; henceforth every add or remove
> of a unit of `service-name` will lead to the addition or removal of the
> corresponding unit of the requested service.
>
>  Note that it is not advisable to implement co-location constraints
> without having the ability to deploy units in isolated containers on
> single machines; doing this is somewhat tricky.
>
> * Provider->generic constraint translation: In the event of our
> implementing stacks, it may be useful to be able to convert
> provider-specific constraints into generic ones (where possible) to
> facilitate creation of provider-independent stacks.
>
> * Named groups of constraints, provisionally named "roles": Useful for
> OAOO-ness and reduced typing when scripting or running from the command
> line; also useful for recording intended machine characteristics when
> not otherwise translatable (for example, when serialising a deployment
> as a stack, it would be thoughtful to include a `fast-network` role to
> hold the `orchestra-classes=rack-c` constraint which *you* know means
> "next to the switch" but is meaningless to people outside your
> datacentre).

If we keep this in the spec, we should include as well a note that
there are known problems with this approach that may undermine other
features along the lines of what we discussed, otherwise we're saving
only the pros and forgetting the cons that the brainstorm has exposed.

> .. [0] So: if an environment has specified `ec2-zone=a ram=1G` and a
> service has specified `ram=2G`, instances of that service in that
> environment will inherit the `ec2-zone` setting of "a", and the juju
> default `storage` of "4G", while using the locally-specified `ram` of
> "2G".
>
> .. [1] `juju env-set` is intended to take on reponsibility for a number
> settings previously only exposed via `environments.yaml`; these
> environment settings will henceforth be exposed via `juju set-env`, and
> can also be passed into `juju bootstrap`, using the same syntax.

This still feels like a good idea, but just not for the constraints.

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I'm not absolutely sure of anything.