lxd and constraints
Aaron Bentley
aaron.bentley at canonical.com
Mon Jan 16 17:28:54 UTC 2017
ISTM that
- constraints are used to ensure that a workload runs well. Minimum
constraints serve this, and maximum constraints do not. (Maximum
constraints may be useful to ensure that a workload does not swamp
processes outside its container.)
- Juju cannot enforce a minimum constraint. LXD could potentially add
support for this, and then Juju would be able to leverage it.
- Given that Juju cannot enforce a minimum constraint on LXD at this
time, it would make sense to emit a warning that it is ignoring the
constraint. This would retain the portability of bundles that use
constraints while keeping the user informed.
On 2017-01-13 01:14 PM, Nate Finch wrote:
> I just feel like we're entering a minefield that our application and CLI
> aren't really built to handle. I think we *should* handle it, but it
> needs to be well planned out, instead of just doing a tiny piece at a
> time and only figuring out later if we did the right thing.
>
> There's a few problems I can see:
>
> 1.) you can have 10 lxd containers with memory limit of 2GB on a machine
> with 4GB of RAM. Deploying 10 applications to those containers that
> each have a constraint of mem=2GB will not work as you expect. We could
> add extra bookkeeping for this, and warn you that you appear to be
> oversubscribing the host, but that's more work.
>
> 2.) What happens if you try to deploy a container without a memory limit
> on a host that already has a container on it?
>
> For example:
> 4GB host
> 2GB lxd container
> try to deploy a new service in a container on this machine.
> Do we warn? We have no clue how much RAM the service will use. Maybe
> it'll be fine, maybe it won't.
>
> 3.) Our CLI doesn't really work well with constraints on containers:
>
> juju deploy mysql --constraints mem=2G --to lxd
>
> Does this deploy a machine with 2GB of ram and a container with a 2GB
> ram limit on it? Or does it deploy a machine with 2GB of ram and a
> container with no limit on it? It has to be one or the other, and
> currently we have no way of indicating which we want to do, and no way
> to do the other one without using multiple commands.
>
> This is a more likely use case, creating a bigger machine that can hold
> multiple containers:
> juju add-machine --constraints mem=4GB
> // adds machine, let's say 5
> // create a container on machine 5 with 2GB memory limit
> juju deploy mysql --constraints mem=2GB --to lxd:5
>
> At least in this case, the deploy command is clear, there's only one
> thing they can possibly mean. Usually, the placement directive would
> override the constraint, but in this case, it does what you would
> want... but it is a littler weird that --to lxd:5 uses the constraint,
> but --to 5 ignores it.
>
> Note that you can't just write a simple script to do the above, because
> the machine number is variable, so you have to parse our output and then
> use that for the next command. It's still scriptable, obviously, but
> it's more complicated script than just two lines of bash.
>
> Also note that using this second method, you can't deploy more than one
> unit at a time, unless you want multiple units on containers on the same
> machine (which I think would be pretty odd).
>
>
>
> On Fri, Jan 13, 2017 at 3:48 AM Rick Harding <rick.harding at canonical.com
> <mailto:rick.harding at canonical.com>> wrote:
>
> In the end, you say you want an instance with 2gb of ram and if the
> cloud has an instance with that exact limit it is in fact an exact
> limit. The key things here is the clouds don't have infinite
> malleable instance types like containers (this works for kvm and for
> lxd). So I'm not sure the mis-match is as far apart as it seems.
> root disk means give me a disk this big, if you ask for 2 core as
> long as you can match an instance type with 2 cores it's exactly the
> max you get.
>
> It seems part of this can be more adjusting the language from
> "minimum" to something closer to "requested X" where request gives
> it more of a "I want X" without the min/max boundaries.
>
>
>
> On Fri, Jan 13, 2017 at 3:14 AM John Meinel <john at arbash-meinel.com
> <mailto:john at arbash-meinel.com>> wrote:
>
> So we could make it so that constraints are actually 'exactly'
> for LXD, which would then conform to both minimum and maximum,
> and would still be actually useful for people deploying to
> containers. We could certainly probe the host machine and say
> "you asked for 48 cores, and the host machine doesn't have it".
>
> However, note that explicit placement also takes precedence over
> constraints anyway. If you do:
> juju deploy mysql --constraints mem=4G
> today, and then do:
> juju add-unit --to 2
> We don't apply the constraint limitations to that specific unit.
> Arguably we should at *least* be warning that the constraints
> for the overall application don't appear to be valid for this
> instance.
>
> I guess I'd rather see constraints still set limits for
> containers, because people really want that functionality, and
> that we warn any time you do a direct placement and the
> constraints aren't satisfied. (but warn isn't failing the attempt)
>
> John
> =:->
>
> On Fri, Jan 13, 2017 at 10:09 AM, Stuart Bishop
> <stuart.bishop at canonical.com
> <mailto:stuart.bishop at canonical.com>> wrote:
>
> On 13 January 2017 at 02:20, Nate Finch
> <nate.finch at canonical.com <mailto:nate.finch at canonical.com>>
> wrote:
>
> I'm implementing constraints for lxd containers and
> provider... and stumbled on an impedance mismatch that I
> don't know how to handle.
>
>
>
> I'm not really sure how to resolve this problem. Maybe
> it's not a problem. Maybe constraints just have a
> different meaning for containers? You have to specify
> the machine number you're deploying to for any
> deployment past the first anyway, so you're already
> manually choosing the machine, at which point,
> constraints don't really make sense anyway.
>
>
> I don't think Juju can handle this. Either constraints have
> different meanings with different cloud providers, or lxd
> needs to accept minimum constraints (along with any other
> cloud providers with this behavior).
>
> If you decide constraints need to consistently mean minimum,
> then I'd argue it is best to not pass them to current-gen
> lxd at all. Enforcing that containers are restricted to the
> minimum viable resources declared in a bundle does not seem
> helpful, and Juju does not have enough information to choose
> suitable maximums (and if it did, would not know if they
> would remain suitable tomorrow).
>
> --
> Stuart Bishop <stuart.bishop at canonical.com
> <mailto:stuart.bishop at canonical.com>>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com <mailto:Juju-dev at lists.ubuntu.com>
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20170116/e87003e0/attachment.pgp>
More information about the Juju-dev
mailing list