Stalled deployment while using --edit-placement

[Jeff McLamb]

OK so I started over, this time making sure only to assign services to
LXC containers or bare metal nodes that were in the “Ready” state.
There are 4 of these and they are physical servers.

This time everything looked good at first. All machines and services
came up, I got a green check mark next to everything, and then at the
bottom it showed the login IP and credentials for both Horizon and the
Juju GUI.

I tried to login to Horizon and got an error occurred authenticating.
Please try again later.

I looked back at the openstack-installer results and realized that
Keystone had never been deployed! It was listed as chosen when I went
to add services. I had remembered to put it in an LXC container like
the rest. And there were no more required services to deploy.

So I re-added it in the (A)dd Services section, and it was already
selected to be deployed in a container to the proper host. I clicked
Deploy and it did add Keystone into the list of services, now the only
one pending.

However, it chose to deploy it to an LXC container on “Machine 5”. It
failed because there were no more new machines to deploy. I had
specifically chosen it to go in an LXC container on a different host
though.

Here is commands.log:

http://paste.ubuntu.com/12068748/

And here is placements.yaml:

http://paste.ubuntu.com/12068766/

Any ideas?

Getting really close now…

Thanks,

Jeff


On Wed, Aug 12, 2015 at 4:35 PM, Mike McCracken
<mike.mccracken at canonical.com> wrote:
> ok, yeah the juju status is telling us what happened, the installer tried to
> add the machine with the maas ID node-ac253c98-3547-11e5-921d-a0481ca53e55,
> which I'm guessing is double-juice, to juju. It uses the maas ID as a tag to
> identify it.
> Juju couldn't add the machine, most likely because it wasn't available
> because it was already deployed to be used as the juju bootstrap node.
>
> The solution is not to try to deploy to that node, but really the installer
> shouldn't have shown it as available.
> I'll look into that.
>
> Unfortunately there isn't a way to just tell it to skip the current charm's
> deployment. In our experience, there are relatively few errors that can be
> safely ignored in this process and still end up with a working cloud, so we
> generally try to just surface errors as clearly as possible and stop. I'll
> also look into how to make this a more clear error, since it's definitely
> fatal and shouldn't just hang.
>
> I recommend just trying again either without juju gui or placing it
> differently. Stop the deployment and use openstack-install -u to clear out
> the juju environment and release the machines from maas.
>
> -mike
>
> On Wed, Aug 12, 2015 at 1:20 PM, Jeff McLamb <mclamb at gmail.com> wrote:
>>
>> At this point it would just be nice to kill the stalled attempt at
>> deploying Juju GUI and go on to the next deployment. Is there any way
>> to do that? The (A)dd Services menu does not seem to respond to
>> removals.
>>
>> Jeff
>>
>> On Wed, Aug 12, 2015 at 4:20 PM, Jeff McLamb <mclamb at gmail.com> wrote:
>> > Hey Mike -
>> >
>> > Here is the whole thing:
>> >
>> > http://paste.ubuntu.com/12065381/
>> >
>> > Here is juju status:
>> >
>> > http://paste.ubuntu.com/12065399/
>> >
>> > Thanks!
>> >
>> > Jeff
>> >
>> > On Wed, Aug 12, 2015 at 2:59 PM, Mike McCracken
>> > <mike.mccracken at canonical.com> wrote:
>> >> Hi Jeff, that error from the log is telling you that it tried to find a
>> >> machine in the juju environment that matched the MAAS ID for the
>> >> machine you
>> >> chose for juju-gui, and it couldn't.
>> >> The installer should take care of adding machines to juju that you
>> >> choose in
>> >> the placement UI, so this should only happen if there was an error in
>> >> that
>> >> process, which will be slightly earlier on in the log.
>> >>
>> >> If you'd rather not share the whole log, grep for a log line with
>> >> "calling
>> >> add_machines with params", and see if there are errors after that. But
>> >> if I
>> >> could see the whole log, it might be faster. You can mail it to me off
>> >> list
>> >> if you like.
>> >>
>> >> The output of 'juju status' would be useful here too, to see what it
>> >> knows
>> >> about double-juice
>> >>
>> >> Thanks,
>> >> -mike
>> >>
>> >> On Wed, Aug 12, 2015 at 10:33 AM, Jeff McLamb <mclamb at gmail.com> wrote:
>> >>>
>> >>> Hey Mike -
>> >>>
>> >>> My juju deployment host is trusty and it is commissioning trusty
>> >>> nodes. I am using openstack kilo via the experimental ppa. Juju is
>> >>> version 1.24.5.
>> >>>
>> >>> The installation line is:
>> >>>
>> >>> $ sudo openstack-install --edit-placement --http-proxy <PROXY>
>> >>> --https-proxy <PROXY> --debug
>> >>>
>> >>> Here is my placements.yaml:
>> >>>
>> >>> http://paste.ubuntu.com/12063466/
>> >>>
>> >>> Here is the relevant portion of commands.log, indicating that the node
>> >>> I selected for juju gui is unsuitable I guess? Like I said, I guess I
>> >>> can understand this, because the machine double-juice (which is a KVM
>> >>> instance that I manually deployed via MAAS) is already deployed (it is
>> >>> the juju deployment machine) and not in a Ready state like all the
>> >>> other machines:
>> >>>
>> >>> http://paste.ubuntu.com/12063514/
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Jeff
>> >>>
>> >>>
>> >>> On Wed, Aug 12, 2015 at 1:12 PM, Mike McCracken
>> >>> <mike.mccracken at canonical.com> wrote:
>> >>> > Hi Jeff, I'll try to answer inline.
>> >>> >
>> >>> > In general, it's helpful to know which ubuntu release you're working
>> >>> > with
>> >>> > and what openstack release you're trying to install, as the
>> >>> > installer
>> >>> > does
>> >>> > try to support a few combinations.
>> >>> > If you don't specify a release, the current default is kilo.
>> >>> >
>> >>> > Also, are you using the installer from the 'testing' PPA or the
>> >>> > newer
>> >>> > 'experimental' one?
>> >>> > If you are using the testing PPA, then switching to the newer one
>> >>> > may
>> >>> > fix
>> >>> > your bug:
>> >>> >
>> >>> > $ sudo apt-add-repository ppa:cloud-installer/experimental
>> >>> > $ sudo apt-get update
>> >>> > $ sudo apt-get install openstack
>> >>> >
>> >>> >
>> >>> > On Wed, Aug 12, 2015 at 9:37 AM, Jeff McLamb <mclamb at gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> Hi everyone,
>> >>> >>
>> >>> >> I am playing around with openstack-installer using --edit-placement
>> >>> >> to
>> >>> >> manually assign services to my 4 physical nodes.
>> >>> >>
>> >>> >> Due to having only 4 nodes, I have to use several servers as
>> >>> >> multi-use. My first question is, are there any functional
>> >>> >> limitations
>> >>> >> on using nodes with multiple services? Will openstack-installer
>> >>> >> detect
>> >>> >> this if there are? I realize that for performance reasons it might
>> >>> >> not
>> >>> >> make sense, but right now I just want to get a working setup with
>> >>> >> what
>> >>> >> I have.
>> >>> >
>> >>> >
>> >>> > There are some limitations, and the installer does try not to let
>> >>> > you do
>> >>> > some things that will break, but it can't guarantee a working setup.
>> >>> > In
>> >>> > particular, nova-compute and quantum/neutron need to be deployed
>> >>> > onto
>> >>> > bare
>> >>> > metal or KVM, not LXC, IIRC because of how they interact with the
>> >>> > host
>> >>> > kernel.
>> >>> >
>> >>> >>
>> >>> >> I am trying to deploy 2 bare metal compute nodes, 2 bare metal Ceph
>> >>> >> nodes (shared across compute), 1 bare metal Ceph OSD (shared on one
>> >>> >> of
>> >>> >> the compute nodes). Most of the required controller-like services
>> >>> >> are
>> >>> >> in LXC containers on a single node. Neutron services are in LXC
>> >>> >> containers on another node.
>> >>> >
>> >>> >
>> >>> > The installer shouldn't allow placing neutron in a container. Can
>> >>> > you
>> >>> > send
>> >>> > along the contents of ~/.cloud-install/placements.yaml so we can see
>> >>> > how
>> >>> > the
>> >>> > services are laid out?
>> >>> >
>> >>> > Also, if you could include the log at ~/.cloud-install/commands.log,
>> >>> > we
>> >>> > can
>> >>> > look there for errors explaining why it's hanging at juju gui.
>> >>> >
>> >>> > In general if you don't want the juju gui, you can skip it, either
>> >>> > just
>> >>> > not
>> >>> > placing it or removing it from the default placement.
>> >>> > It's not necessary for the operation of any other services.
>> >>> >
>> >>> > Thanks,
>> >>> > -mike
>> >>> >
>> >>> >>
>> >>> >> I also chose to install Juju-GUI in a container on the Juju
>> >>> >> deployment
>> >>> >> node, which itself is a manually deployed KVM node hosted on the
>> >>> >> MAAS
>> >>> >> server, managed by MAAS.
>> >>> >>
>> >>> >> Using the latest installer, deployment has stalled during Juju GUI.
>> >>> >> It
>> >>> >> successfully deployed Keystone, MySQL, NTP to containers, but has
>> >>> >> been
>> >>> >> sitting at Juju GUI. I am suspicious that this is because the
>> >>> >> chosen
>> >>> >> node is one that Juju is not deploying itself?
>> >>> >>
>> >>> >> Is there any way to (A)dd Services and remove Juju GUI during this
>> >>> >> stage? I tried to do that, but Remove does not seem to register
>> >>> >> anything. And juju-gui does not yet appear in `juju status` so I
>> >>> >> can't
>> >>> >> remove it that way either.
>> >>> >>
>> >>> >> Basically, I'd like to move along with the deployment and forgo
>> >>> >> Juju
>> >>> >> GUI for now, but I can't figure out how to tell it to skip over
>> >>> >> this
>> >>> >> part of the deployment.
>> >>> >>
>> >>> >> Thanks,
>> >>> >>
>> >>> >> Jeff
>> >>> >>
>> >>> >> --
>> >>> >> ubuntu-openstack-installer mailing list
>> >>> >> ubuntu-openstack-installer at lists.ubuntu.com
>> >>> >> Modify settings or unsubscribe at:
>> >>> >>
>> >>> >> https://lists.ubuntu.com/mailman/listinfo/ubuntu-openstack-installer
>> >>> >
>> >>> >
>> >>
>> >>
>
>