[Bug 1969365] Re: focal: backport kexec fallback patch

[Dan Watkins]

Apologies, I completely missed your comment, Nick!  I was just able to
reproduce this using uvtool.

To launch the VM (and monitor the console output):

uvt-simplestreams-libvirt sync release=focal arch=amd64
uvt-kvm create firsttest release=focal
virsh console firsttest

Then, within the instance via `uvt-kvm ssh firsttest`:

sudo apt update
sudo apt install kexec-tools
sudo reboot  # I wanted to check `virsh console` was getting output

After reboot and reconnecting:

cd /boot
sudo kexec -l vmlinuz --initrd initrd.img --append 'BOOT_IMAGE=/boot/vmlinuz-5.4.0-167-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0'
sudo reboot

I then observed the following in `virsh console`'s output (note the
"Unmount All Filesystems", which is what I think is causing the
problem):

[  OK  ] Reached target Unmount All Filesystems.
         Stopping Monitoring of LVM…meventd or progress polling...
         Stopping Device-Mapper Multipath Device Controller...
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  OK  ] Stopped Create System Users.
[  OK  ] Stopped Remount Root and Kernel File Systems.
[  OK  ] Stopped File System Check on Root Device.
[  OK  ] Stopped Device-Mapper Multipath Device Controller.
[  OK  ] Stopped Monitoring of LVM2… dmeventd or progress polling.
[  OK  ] Reached target Shutdown.
[  OK  ] Reached target Final Step.
         Starting Reboot via kexec...
[   53.030829] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [960] did not accept message, killing the worker: Connection refused
[   53.032915] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [954] did not accept message, killing the worker: Connection refused
[   53.034697] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [953] did not accept message, killing the worker: Connection refused
[   53.036756] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [955] did not accept message, killing the worker: Connection refused
[   53.039144] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [956] did not accept message, killing the worker: Connection refused
[   53.041049] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [958] did not accept message, killing the worker: Connection refused
[   53.042673] systemd-udevd[372]: proc_inode_cache(1258:kexec.service): Worker [961] did not accept message, killing the worker: Connection refused
[   53.123845] shutdown[1]: (sd-kexec) failed with exit status 1.
[   53.138542] reboot: Restarting system
[    0.000000] Linux version 5.4.0-167-generic (buildd at lcy02-amd64-010) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.2)) #184-Ubuntu SMP Tue Oct 31 09:21:49 UTC 2023 (Ubuntu 5.4.0-167.184-generic 5.4.252)

(I initially tried to re-reproduce using an Incus VM (and so a community
image rather than a cloud-images one) and could not do so.  Executing
the above `kexec` command produced "ima: impossible to appraise a kernel
image without a file descriptor; try using kexec_file_load syscall." in
the console.  Adding `-s`/`--kexec-file-syscall` to the kexec command
caused it to exit zero, and the VM did then successfully kexec on
`reboot`.)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1969365

Title:
  focal: backport kexec fallback patch

Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Focal:
  Fix Committed

Bug description:
  It would be great if focal's systemd could have
  https://github.com/systemd/systemd/commit/71180f8e57f8fbb55978b00a13990c79093ff7b3
  backported to it.

  [Impact]

  We have observed that kexec'ing to another kernel will fail as the
  drive containing the `kexec` binary has been unmounted by the time
  systemd attempts to do so, indicated in the console:

           Starting Reboot via kexec...
  [  163.960938] shutdown[1]: (sd-kexec) failed with exit status 1.
  [  163.963463] reboot: Restarting system

  [Test Plan]

  1) Launch a 20.04 instance
  2) `apt-get install kexec-tools`
  3) In `/boot`, filling in whatever <cmdline> needed in your environment:

  kexec -l vmlinuz --initrd initrd.img --append '<cmdline>'

  4) `reboot`

  (I have reproduced this in a single-disk VM, so I assume it reproduces
  ~everywhere: if not, `apt-get remove kexec-tools` before the `reboot`
  could be used to emulate the unmounting.)

  [Where problems could occur]

  Users could inadvertently be relying on the current behaviour: if they
  have configured their systems to kexec, they currently will be
  rebooting normally, and this patch would cause them to start actually
  kexec'ing.

  [Other info]

  We're currently maintaining a systemd tree with only this patch added
  to focal's tree: this patch has received a bunch of testing from us in
  focal.

  This patch landed in v246, so it's already present in supported
  releases later than focal.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1969365/+subscriptions