[Bug 1874147] Re: virt-install of focal-server-cloudimg-amd64.img panics on first boot, breaking cloud-init
David Baird
1874147 at bugs.launchpad.net
Tue May 12 16:17:06 UTC 2020
** Summary changed:
- focal-server-cloudimg-amd64.img panics on first boot, cannot open root device
+ virt-install of focal-server-cloudimg-amd64.img panics on first boot, breaking cloud-init
** Description changed:
- Various Focal cloudimg builds fail on first time ever booting (e.g.
- https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-
- amd64.img). Here is the failed output:
+ Various Focal cloudimg builds panic on first time ever booting with
+ "Cannot open root device" when booting with virt-install (libvirt).
+
+ E.g.
+ https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img
+
+ Here is the failed output:
[ 0.000000] Linux version 5.4.0-24-generic (buildd at lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1
...
[ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6
[ 1.045238] Please append a correct "root=" boot option; here are the available partitions:
[ 1.046921] 0b00 1048575 sr0
- [ 1.046922] driver: sr
+ [ 1.046922] driver: sr
[ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu
[ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[ 1.052904] Call Trace:
[ 1.053459] dump_stack+0x6d/0x9a
[ 1.054158] panic+0x101/0x2e3
[ 1.054812] mount_block_root+0x23f/0x2e8
- [ 1.055645] mount_root+0x38/0x3a
+ [ 1.055645] mount_root+0x38/0x3a
[ 1.056319] prepare_namespace+0x13f/0x194
[ 1.057236] kernel_init_freeable+0x231/0x255
[ 1.058131] ? rest_init+0xb0/0xb0
[ 1.058834] kernel_init+0xe/0x100
[ 1.059535] ret_from_fork+0x35/0x40
[ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
When booted, the second time, it works!
The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to
"initrdfail." On first ever boot, grub will try to boot *without* an
initrd, which (apparently) makes the kernel unable to mount root.
Relevant section of /boot/grub/grub.cfg:
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {
recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
else
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
fi
if [ "${initrdfail}" = 1 ]; then
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
initrd /boot/initrd.img-5.4.0-24-generic
else
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0 panic=-1
fi
initrdfail
}
** Description changed:
Various Focal cloudimg builds panic on first time ever booting with
"Cannot open root device" when booting with virt-install (libvirt).
+ After panicing, it reboots, and the second boot succeeds (using initrd
+ on the second try). However, it needs to succeed on the first try
+ without a panic. This breaks the cloud-init NoCloud process which
+ expects the first time booting to succeed.
E.g.
https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img
Here is the failed output:
[ 0.000000] Linux version 5.4.0-24-generic (buildd at lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1
...
[ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6
[ 1.045238] Please append a correct "root=" boot option; here are the available partitions:
[ 1.046921] 0b00 1048575 sr0
[ 1.046922] driver: sr
[ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu
[ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[ 1.052904] Call Trace:
[ 1.053459] dump_stack+0x6d/0x9a
[ 1.054158] panic+0x101/0x2e3
[ 1.054812] mount_block_root+0x23f/0x2e8
[ 1.055645] mount_root+0x38/0x3a
[ 1.056319] prepare_namespace+0x13f/0x194
[ 1.057236] kernel_init_freeable+0x231/0x255
[ 1.058131] ? rest_init+0xb0/0xb0
[ 1.058834] kernel_init+0xe/0x100
[ 1.059535] ret_from_fork+0x35/0x40
[ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
When booted, the second time, it works!
The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to
"initrdfail." On first ever boot, grub will try to boot *without* an
initrd, which (apparently) makes the kernel unable to mount root.
Relevant section of /boot/grub/grub.cfg:
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {
recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
else
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
fi
if [ "${initrdfail}" = 1 ]; then
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
initrd /boot/initrd.img-5.4.0-24-generic
else
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0 panic=-1
fi
initrdfail
}
** Also affects: virt-manager (Ubuntu)
Importance: Undecided
Status: New
** Description changed:
Various Focal cloudimg builds panic on first time ever booting with
"Cannot open root device" when booting with virt-install (libvirt).
After panicing, it reboots, and the second boot succeeds (using initrd
on the second try). However, it needs to succeed on the first try
without a panic. This breaks the cloud-init NoCloud process which
expects the first time booting to succeed.
E.g.
https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img
Here is the failed output:
[ 0.000000] Linux version 5.4.0-24-generic (buildd at lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1
...
[ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6
[ 1.045238] Please append a correct "root=" boot option; here are the available partitions:
[ 1.046921] 0b00 1048575 sr0
[ 1.046922] driver: sr
[ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu
[ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[ 1.052904] Call Trace:
[ 1.053459] dump_stack+0x6d/0x9a
[ 1.054158] panic+0x101/0x2e3
[ 1.054812] mount_block_root+0x23f/0x2e8
[ 1.055645] mount_root+0x38/0x3a
[ 1.056319] prepare_namespace+0x13f/0x194
[ 1.057236] kernel_init_freeable+0x231/0x255
[ 1.058131] ? rest_init+0xb0/0xb0
[ 1.058834] kernel_init+0xe/0x100
[ 1.059535] ret_from_fork+0x35/0x40
[ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
When booted, the second time, it works!
- The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to
- "initrdfail." On first ever boot, grub will try to boot *without* an
- initrd, which (apparently) makes the kernel unable to mount root.
+ The issue appears to be that cloudimg's kernel doesn't come with the
+ necessary drivers pre-compiled into the kernel on a KVM/QEMU system.
+ And, combined with a recent (2019) change to /boot/grub/grub.cfg which
+ doesn't load initrd on the first boot, this ensures that the first time
+ booting will fail.
Relevant section of /boot/grub/grub.cfg:
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {
recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
else
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
fi
if [ "${initrdfail}" = 1 ]; then
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
initrd /boot/initrd.img-5.4.0-24-generic
else
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0 panic=-1
fi
initrdfail
}
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to grub2 in Ubuntu.
https://bugs.launchpad.net/bugs/1874147
Title:
virt-install of focal-server-cloudimg-amd64.img panics on first boot,
breaking cloud-init
Status in VMBuilder:
New
Status in grub2 package in Ubuntu:
New
Status in virt-manager package in Ubuntu:
New
Bug description:
Various Focal cloudimg builds panic on first time ever booting with
"Cannot open root device" when booting with virt-install (libvirt).
After panicing, it reboots, and the second boot succeeds (using initrd
on the second try). However, it needs to succeed on the first try
without a panic. This breaks the cloud-init NoCloud process which
expects the first time booting to succeed.
E.g.
https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img
Here is the failed output:
[ 0.000000] Linux version 5.4.0-24-generic (buildd at lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1
...
[ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6
[ 1.045238] Please append a correct "root=" boot option; here are the available partitions:
[ 1.046921] 0b00 1048575 sr0
[ 1.046922] driver: sr
[ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu
[ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014
[ 1.052904] Call Trace:
[ 1.053459] dump_stack+0x6d/0x9a
[ 1.054158] panic+0x101/0x2e3
[ 1.054812] mount_block_root+0x23f/0x2e8
[ 1.055645] mount_root+0x38/0x3a
[ 1.056319] prepare_namespace+0x13f/0x194
[ 1.057236] kernel_init_freeable+0x231/0x255
[ 1.058131] ? rest_init+0xb0/0xb0
[ 1.058834] kernel_init+0xe/0x100
[ 1.059535] ret_from_fork+0x35/0x40
[ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
When booted, the second time, it works!
The issue appears to be that cloudimg's kernel doesn't come with the
necessary drivers pre-compiled into the kernel on a KVM/QEMU system.
And, combined with a recent (2019) change to /boot/grub/grub.cfg which
doesn't load initrd on the first boot, this ensures that the first
time booting will fail.
Relevant section of /boot/grub/grub.cfg:
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {
recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
else
search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e
fi
if [ "${initrdfail}" = 1 ]; then
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
initrd /boot/initrd.img-5.4.0-24-generic
else
linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0 panic=-1
fi
initrdfail
}
To manage notifications about this bug go to:
https://bugs.launchpad.net/vmbuilder/+bug/1874147/+subscriptions
More information about the foundations-bugs
mailing list