[Bug 1654600] Re: unattended-upgrade-shutdown hangs when /var is a separate filesystem

[Louis Bouchard]

Ok, here is how I understand the situation.

According to the systemd documentation about [Unit] Before= :

"Given two units with any ordering dependency between them, if one unit
is shut down and the other is started up, the shutdown is ordered before
the start-up. It doesn't matter if the ordering dependency is After= or
Before=."

To me this means that network.service & local-fs.service will be
shutdown _BEFORE_ unattended-upgrades-shutdown runs, hence /var var will
be unmounted when it runs.

My current solution is to turn the unattended-upgrades.service
ExecStart= into an ExecStop= so the unit will run as a shutdown instead
of a start when the system shuts down :

[Unit]
Description=Unattended Upgrades Shutdown
DefaultDependencies=no
Before=shutdown.target reboot.target halt.target network.target local-fs.target
Documentation=man:unattended-upgrade(8)

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStop=/usr/share/unattended-upgrades/unattended-upgrade-shutdown --debug
TimeoutStopSec=900

[Install]
WantedBy=shutdown.target

Preliminary tests seem to run fine but I want to get confirmation on
such a change by someone more expert with systemd that I am.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1654600

Title:
  unattended-upgrade-shutdown hangs when /var is a separate filesystem

Status in unattended-upgrades package in Ubuntu:
  Confirmed
Status in unattended-upgrades source package in Xenial:
  Confirmed
Status in unattended-upgrades source package in Yakkety:
  New

Bug description:
  The systemd unit file unattended-upgrades.service is used to stop a running unattended-upgrade
  process during shutdown. This unit file is running together with all filesystem
  unmount services.

  The unattended-upgrades service checks if the lockfile for unattended-upgrade
  (in /var/run) exists, and if it does, there is an unattended-upgrade in progress
  and the service will wait until it finishes (and therefore automatically wait at
  shutdown).

  However, if /var is a separate filesystem, it will get unmounted even though /var/run
  is a tmpfs that's still mounted on top of the /var/run directory in the /var filesystem.
  The unattended-upgrade script will fail to find lockfile, sleeps for 5 seconds, and
  tries to check the lockfile again. After 10 minutes (the default timeout), it will finally
  exit and the system will continue shutdown.

  The problem is the error handling in /usr/share/unattended-upgrades/unattended-upgrade-shutdown
  where it tries to lock itself:

      while True:
          res = apt_pkg.get_lock(options.lock_file)
          logging.debug("get_lock returned %i" % res)
          # exit here if there is no lock
          if res > 0:
              logging.debug("lock not taken")
              break
          lock_was_taken = True

  The function apt_pkg.get_lock() either returns a file descriptor, or -1 on an error.
  File descriptors are just C file descriptors, so they are always positive integers.
  The code should check the result to be negative, not positive. I have attached a patch
  to reverse the logic.

  Additional information:

  1)
  Description:	Ubuntu 16.04.1 LTS
  Release:	16.04

  2)
  unattended-upgrades:
    Installed: 0.90ubuntu0.3
    Candidate: 0.90ubuntu0.3
    Version table:
   *** 0.90ubuntu0.3 500
          500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
          500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages
          100 /var/lib/dpkg/status
       0.90 500
          500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
          500 http://nl.archive.ubuntu.com/ubuntu xenial/main i386 Packages
  3)
  Fast reboot
  4)
  Very slow reboot (after a 10 minutes timeout)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1654600/+subscriptions