Re: Multi install with existing MAAS starts all services except for “IP Pending” on Glance Simplestreams Image Sync
Jeff McLamb
mclamb at gmail.com
Wed Jul 29 17:21:05 UTC 2015
This bug report seems to be related, if not the same thing:
https://bugs.launchpad.net/landscape/+bug/1451385
On Wed, Jul 29, 2015 at 1:10 PM, Jeff McLamb <mclamb at gmail.com> wrote:
> Here is the output from trying with --series vivid:
>
> http://paste.ubuntu.com/11960777/
>
> I also did a re-install with --series trusty (really no different from
> the first time) and everything turned out the same, except the order
> in which containers came up and what got assigned to which container.
>
> However, logging into the physical node (machine 3) that hosts the
> containers, I found the console log of the actual vivid template lxc
> when it is in the RUNNING state and the glance sync container is the
> only one left to provision.
>
> The errors below seem to make sense with respect to the complaint in
> `juju status` that says it could not clone properly because the
> template container did not terminate:
>
> ubuntu at cooperative-crowd:~$ sudo cat
> /var/lib/juju/containers/juju-vivid-lxc-template/console.log
> Failed to mount cgroup at /sys/fs/cgroup/systemd: Permission denied
> systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA
> +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL
> +XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN)
> Detected virtualization lxc.
> Detected architecture x86-64.
>
> Welcome to Ubuntu 15.04!
>
> Set hostname to <juju-vivid-lxc-template>.
> Initializing machine ID from random generator.
> Failed to install release agent, ignoring: No such file or directory
> Failed to create root cgroup hierarchy: No such file or directory
> Failed to allocate manager object: No such file or directory
> [!!!!!!] Failed to allocate manager object, freezing.
>
> On Wed, Jul 29, 2015 at 12:35 PM, Mike McCracken
> <mike.mccracken at canonical.com> wrote:
>> Can you share what goes wrong when you tried --series vivid ?
>> It's always supported specifying whatever you like there, so it should work,
>> and if something complained, that's a bug.
>>
>> Thanks,
>> -mike
>>
>> On Wed, Jul 29, 2015 at 9:20 AM, Jeff McLamb <mclamb at gmail.com> wrote:
>>>
>>> It seems as though vivid is not a valid --series name in my version of
>>> openstack-installer 0.99.19
>>>
>>> I can run it with trusty just fine, but I suspect that that will lead
>>> to the same issue where glance is installed as vivid and the rest are
>>> installed as trusty?
>>>
>>> Thanks,
>>>
>>> Jeff
>>>
>>>
>>> On Wed, Jul 29, 2015 at 11:48 AM, Mike McCracken
>>> <mike.mccracken at canonical.com> wrote:
>>> > If you're able, I think the easiest way is to tear it down and
>>> > re-deploy.
>>> > Anything else would be a lot of manual steps. Juju has some upgrade
>>> > capabilities, but the openstack installer pre-dates them, and so you'd
>>> > have
>>> > to hand-edit some config files, etc.
>>> >
>>> > -mike
>>> >
>>> > On Wed, Jul 29, 2015 at 8:44 AM, Jeff McLamb <mclamb at gmail.com> wrote:
>>> >>
>>> >> Cool, thanks for the tip, Mike.
>>> >>
>>> >> Yes the host is vivid running juju 1.24.3-vivid-amd64
>>> >>
>>> >> Given my current state, should I kill the environment and start again,
>>> >> or is there a way I can deploy or update what's currently there
>>> >> without having to start over?
>>> >>
>>> >> Thanks,
>>> >>
>>> >> Jeff
>>> >>
>>> >>
>>> >> On Wed, Jul 29, 2015 at 11:39 AM, Mike McCracken
>>> >> <mike.mccracken at canonical.com> wrote:
>>> >> > Hi Jeff, so it looks like you're running vivid on the host, is that
>>> >> > right?
>>> >> > Your point that the sync charm is the only one deployed as vivid is a
>>> >> > good
>>> >> > catch - while this should still work, it might be causing problems
>>> >> > for
>>> >> > juju's local provider.
>>> >> > I will fix the deploy code for the sync charm to use the same series
>>> >> > as
>>> >> > the
>>> >> > other charms, but one thing you can try in the meantime is use
>>> >> > --series
>>> >> > vivid to just deploy the whole thing as vivid.
>>> >> > This has been tested and should work.
>>> >> >
>>> >> > -mike
>>> >> >
>>> >> > On Wed, Jul 29, 2015 at 8:25 AM, Jeff McLamb <mclamb at gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Hey Adam, Mike -
>>> >> >>
>>> >> >> The environment is up, but the container that should contain
>>> >> >> glance-simplestreams is not up at all. Here is the lxc-ls output
>>> >> >> from
>>> >> >> the node that contains all of the infrastructure containers:
>>> >> >>
>>> >> >> ubuntu at cooperative-crowd:~$ sudo lxc-ls --fancy
>>> >> >> NAME STATE IPV4 IPV6 AUTOSTART
>>> >> >> ----------------------------------------------------------------
>>> >> >> juju-machine-3-lxc-0 RUNNING 192.168.1.17 - YES
>>> >> >> juju-machine-3-lxc-1 RUNNING 192.168.1.18 - YES
>>> >> >> juju-machine-3-lxc-2 RUNNING 192.168.1.19 - YES
>>> >> >> juju-machine-3-lxc-3 RUNNING 192.168.1.20 - YES
>>> >> >> juju-machine-3-lxc-4 RUNNING 192.168.1.21 - YES
>>> >> >> juju-machine-3-lxc-6 RUNNING 192.168.1.22 - YES
>>> >> >> juju-machine-3-lxc-7 RUNNING 192.168.1.23 - YES
>>> >> >> juju-trusty-lxc-template STOPPED - - NO
>>> >> >> juju-vivid-lxc-template STOPPED - - NO
>>> >> >> ubuntu at cooperative-crowd:~$
>>> >> >>
>>> >> >> As you can see, the lxc-5 machine that should be running is not.
>>> >> >>
>>> >> >> When I look at openstack-status all of the services have a green
>>> >> >> checkmark and are [started] except for Glance Simplestreams which is
>>> >> >> in the [pending] stage.
>>> >> >>
>>> >> >> In the place of an IP address it says IP Pending and "waiting for
>>> >> >> initial status".
>>> >> >>
>>> >> >> I was wondering if I was running out of IP addresses in that range
>>> >> >> on
>>> >> >> the MAAS DHCP server, but there are only 8 containers and there are
>>> >> >> plenty of addresses left in that range.
>>> >> >>
>>> >> >> Here is the output of juju status where it is revealed that:
>>> >> >>
>>> >> >> 3/lxc/5:
>>> >> >> agent-state-info: 'failed to retrieve the template to clone:
>>> >> >> template container
>>> >> >> "juju-vivid-lxc-template" did not stop'
>>> >> >> instance-id: pending
>>> >> >> series: vivid
>>> >> >>
>>> >> >> For some reason this is the only container that it tries to install
>>> >> >> using the vivid series. All other containers use trusty.
>>> >> >>
>>> >> >> Here is the full juju status output:
>>> >> >>
>>> >> >> http://paste.ubuntu.com/11960193/
>>> >> >>
>>> >> >> Thanks!
>>> >> >>
>>> >> >> Jeff
>>> >> >>
>>> >> >> On Wed, Jul 29, 2015 at 11:15 AM, Adam Stokes
>>> >> >> <adam.stokes at canonical.com>
>>> >> >> wrote:
>>> >> >> > Jeff,
>>> >> >> >
>>> >> >> > Is the environment still up? Can you `juju ssh
>>> >> >> > glance-simplestreams-sync/0`
>>> >> >> > and grab the unit logs from `/var/log/juju/`.
>>> >> >> >
>>> >> >> > CCing Mike for further investigation into that service
>>> >> >> >
>>> >> >> > On Wed, Jul 29, 2015 at 10:25 AM, Jeff McLamb <mclamb at gmail.com>
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> Hey Adam -
>>> >> >> >>
>>> >> >> >> Sure, here it is:
>>> >> >> >>
>>> >> >> >> http://paste.ubuntu.com/11959888/
>>> >> >> >>
>>> >> >> >> Thanks!
>>> >> >> >>
>>> >> >> >> Jeff
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> On Wed, Jul 29, 2015 at 10:07 AM, Adam Stokes
>>> >> >> >> <adam.stokes at canonical.com>
>>> >> >> >> wrote:
>>> >> >> >> > Hey Jeff,
>>> >> >> >> >
>>> >> >> >> > Can you paste.ubuntu.com your ~/.cloud-install/commands.log?
>>> >> >>
>>> >> >> >>
>>> >> >> >> >
>>> >> >> >> > On Wed, Jul 29, 2015 at 1:38 AM, Jeff McLamb <mclamb at gmail.com>
>>>
>>> >>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >> Hello -
>>> >> >> >> >>
>>> >> >> >> >> Due to Adam’s suggestion in a previous thread about Autopilot
>>> >> >> >> >> not
>>> >> >> >> >> working properly with a proxy running (e.g. masa-proxy) on the
>>> >> >> >> >> MAAS
>>> >> >> >> >> server, I killed the environment and started over with the
>>> >> >> >> >> Multi
>>> >> >> >> >> option.
>>> >> >> >> >>
>>> >> >> >> >> Interesting aside: you must openstack-install -k and
>>> >> >> >> >> openstack-install
>>> >> >> >> >> -u or otherwise remove ~/.cloud-install entirely because if
>>> >> >> >> >> you
>>> >> >> >> >> just
>>> >> >> >> >> do a -k and do not remove the landscape related files, it will
>>> >> >> >> >> still
>>> >> >> >> >> try to install an Autopilot-based setup even if you choose
>>> >> >> >> >> Multi.
>>> >> >> >> >>
>>> >> >> >> >> Anyway, the Multi install progressed well, bootstrapping juju
>>> >> >> >> >> with
>>> >> >> >> >> the
>>> >> >> >> >> VM node, followed by deploying services across 3 other
>>> >> >> >> >> physical
>>> >> >> >> >> nodes
>>> >> >> >> >> that are all big RAM and big core.
>>> >> >> >> >>
>>> >> >> >> >> One physical node was chosen as dedicated compute, one as
>>> >> >> >> >> dedicated
>>> >> >> >> >> Neutron, and the third was chosen for infrastructure,
>>> >> >> >> >> deploying
>>> >> >> >> >> what
>>> >> >> >> >> should be 8 containers.
>>> >> >> >> >>
>>> >> >> >> >> In the openstack-status view I see that all services have
>>> >> >> >> >> “started”
>>> >> >> >> >> except for the Glance - Simplestreams Image Sync service which
>>> >> >> >> >> is
>>> >> >> >> >> to
>>> >> >> >> >> be deployed as Container 5 on the infrastructure machine
>>> >> >> >> >> (Machine
>>> >> >> >> >> 3
>>> >> >> >> >> in
>>> >> >> >> >> my case). It is stuck in “IP Pending” state.
>>> >> >> >> >>
>>> >> >> >> >> I have rebooted the infrastructure node and all lxc containers
>>> >> >> >> >> came
>>> >> >> >> >> back up, except a lxc-ls —fancy does not show the existence of
>>> >> >> >> >> juju-machine-3-lxc-5. It does have STOPPED
>>> >> >> >> >> juju-trusty-lxc-template
>>> >> >> >> >> and juju-vivid-lxc-template. The vivid one was actually
>>> >> >> >> >> RUNNING
>>> >> >> >> >> before
>>> >> >> >> >> I rebooted the system, but now it is STOPPED.
>>> >> >> >> >>
>>> >> >> >> >> I can get to the horizon dashboard GUI but I cannot login with
>>> >> >> >> >> a
>>> >> >> >> >> “An
>>> >> >> >> >> error occurred authenticating. Please try again later.”
>>> >> >> >> >>
>>> >> >> >> >> Any ideas how to get that last service running so I can
>>> >> >> >> >> hopefully
>>> >> >> >> >> log
>>> >> >> >> >> into the horizon GUI for the first time?
>>> >> >> >> >>
>>> >> >> >> >> Thanks,
>>> >> >> >> >>
>>> >> >> >> >> Jeff
>>> >> >> >> >>
>>> >> >> >> >> --
>>> >> >> >> >> ubuntu-openstack-installer mailing list
>>> >> >> >> >> ubuntu-openstack-installer at lists.ubuntu.com
>>> >> >> >> >> Modify settings or unsubscribe at:
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> https://lists.ubuntu.com/mailman/listinfo/ubuntu-openstack-installer
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >
>>> >> >
>>> >
>>> >
>>
>>
More information about the ubuntu-openstack-installer
mailing list