[Bug 1832859] Re: during shutdown libvirt-guests gets stopped after file system unmount

[Christian Ehrhardt ]

yeah I think I found the issue in your log.
Consider [1] again and look at
  /lib/systemd/system/blk-availability.service

This is what calls blkdeactivate on shutdown.

It only has a
  WantedBy=sysinit.target
But no ordering Before anything.
It even has
  DefaultDependencies=no
which will make it start early only waiting for the listed
After=lvm2-activation.service lvm2-lvmetad.service iscsi-shutdown.service iscsi.service iscsid.service fcoe.service

But in reverse that also means it has to wait on NOTHING to start
"stopping" which is the call to blkdeactivate.

IMHO that is a bug in lvm2, that should ensure that this probably should be ordered
somewhere:
- closely after local-fs.target (on shutdown); I mean FS should be gone before deactivating devices right?
- the startup ordering isn't important as it is calling just /bin/true

My (uneducated) suggestion would be to add
  Before=local-fs.target
to
 /lib/systemd/system/blk-availability.service
Then run 
 $ systemctl daemon-reload

(maybe needs another reboot then, not sure).
>From there I'd assume that the shutdown ordering (beig the inverse) would order it AFTER local-fs which IMHO is what we'd want.

Please give this a try and I'll add an LVM task here.

[1]: https://www.freedesktop.org/software/systemd/man/bootup.html

** Also affects: lvm2 (Ubuntu)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to lvm2 in Ubuntu.
https://bugs.launchpad.net/bugs/1832859

Title:
  during shutdown libvirt-guests gets stopped after file system unmount

Status in libvirt package in Ubuntu:
  Incomplete
Status in lvm2 package in Ubuntu:
  New

Bug description:
  When using automatic suspend at reboot/shutdown, it makes sense to
  store the suspend data on a separate partition to ensure there is
  always enough available space. However this does not work, as the
  partition gets unmounted before or during libvirt suspend.

  Steps to reproduce:

  1. Use Ubuntu 18.04.02 LTS
  2. Install libvirt + qemu-kvm
  3. Start a guest
  4. Set libvirt-guests to suspend at shutdown/reboot by editing /etc/default/libvirt-guests
  5. Create a fstab entry to mount a separate partition to mount point /var/lib/libvirt/qemu/save. Then run sudo mount /var/lib/libvirt/qemu/save to mount the partition.
  6. Reboot

  Expected result:
  The guest suspend data would be written to the /var/lib/libvirt/qemu/save, resulting in the data being stored at the partition specified in fstab. At boot, this partition would be mounted as specified in fstab and libvirt-guest would be able to read the data and restore the guests.

  Actual result:
  The partition gets unmounted before libvirt-guests suspends the guests, resulting in the data being stored on the partition containing the root file system. During boot, the empty partition gets mounted over the non-empty /var/lib/libvirt/qemu/save directory, resulting in libvirt-guests being unable to read the saved data.

  As a side effect, the saved data is using up space on the root
  partition even if the directory appears empty.

  Here is some of the relevant lines from the journal:

  Jun 14 00:00:04 libvirt-host blkdeactivate[4343]: Deactivating block devices:
  Jun 14 00:00:04 libvirt-host systemd[1]: Unmounted /var/lib/libvirt/qemu/save.
  Jun 14 00:00:04 libvirt-host blkdeactivate[4343]:   [UMOUNT]: unmounting libvirt_lvm-suspenddata (dm-3) mounted on /var/lib/libvirt/qemu/save... done

  Jun 14 00:00:04 libvirt-host libvirt-guests.sh[4349]: Running guests on default URI: vps1, vps2, vps3
  Jun 14 00:00:04 libvirt-host blkdeactivate[4343]:   [MD]: deactivating raid1 device md1... done
  Jun 14 00:00:05 libvirt-host libvirt-guests.sh[4349]: Suspending guests on default URI...
  Jun 14 00:00:05 libvirt-host libvirt-guests.sh[4349]: Suspending vps1: ...
  Jun 14 00:00:05 libvirt-host blkdeactivate[4343]:   [LVM]: deactivating Volume Group libvirt_lvm... skipping

  Jun 14 00:00:10 libvirt-host libvirt-guests.sh[4349]: Suspending vps1: 5.989 GiB
  Jun 14 00:00:15 libvirt-host libvirt-guests.sh[4349]: Suspending vps1: ...
  Jun 14 00:00:20 libvirt-host libvirt-guests.sh[4349]: Suspending vps1: ...

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1832859/+subscriptions