[Bug 1654600] Re: unattended-upgrade-shutdown hangs when /var is a separate filesystem

[kay]

Tested the fix. It works. Thanks!

quick fix:

sed -i "s#ExecStart=#RemainAfterExit=yes\nExecStop=#;"
/lib/systemd/system/unattended-upgrades.service

But now it seems that it doesn't run it on shutdown. I used the
following debug service:

ExecStop=/bin/bash -c 'echo -e "\nhello######################\n" >
/dev/ttyS0'

And it doesn't print message in serial console. But when you start and
stop this unit manually - it prints.

[  OK  ] Stopped target Graphical Interface.
         Stopping Accounts Service...
         Stopping User Manager for UID 1000...
         Stopping Session 1 of user ubuntu.
         Stopping ACPI event daemon...
[  OK  ] Stopped target Cloud-init target.
[  OK  ] Stopped Execute cloud user/final scripts.
[  OK  ] Stopped target Multi-User System.
         Stopping Deferred execution scheduler...
         Stopping LXD - container startup/shutdown...
         Stopping D-Bus System Message Bus...
         Stopping LSB: daemon to balance interrupts for SMP systems...
         Stopping LSB: Set the CPU Frequency Scaling governor to "ondemand"...
         Stopping LSB: MD monitoring daemon...
         Stopping Regular background program processing daemon...
         Stopping FUSE filesystem for LXC...
         Stopping OpenBSD Secure Shell server...
         Stopping LSB: Record successful boot for GRUB...
[  OK  ] Stopped target Timers.
[  OK  ] Stopped Timer to automatically refresh installed snaps.
[  OK  ] Stopped Daily apt activities.
[  OK  ] Stopped Daily Cleanup of Temporary Directories.
[  OK  ] Stopped target Login Prompts.
         Stopping Getty on tty1...
         Stopping Serial Getty on ttyS0...
[  OK  ] Stopped Apply the settings specified in cloud-config.
[  OK  ] Stopped target Cloud-config availability.
         Stopping Snappy daemon...
         Stopping System Logging Service...
[  OK  ] Stopped target System Time Synchronized.
         Stopping LSB: automatic crash report generation...
[  OK  ] Closed Load/Save RF Kill Switch Status /dev/rfkill Watch.
         Stopping Authenticate and Authorize Users to Run Privileged Tasks...
[  OK  ] Unmounted /var/lib/lxcfs.
[  OK  ] Stopped System Logging Service.
[  OK  ] Stopped Deferred execution scheduler.
[  OK  ] Stopped OpenBSD Secure Shell server.
[  OK  ] Stopped Accounts Service.
[  OK  ] Stopped Snappy daemon.
[  OK  ] Stopped ACPI event daemon.
[  OK  ] Stopped Authenticate and Authorize Users to Run Privileged Tasks.
[  OK  ] Stopped Serial Getty on ttyS0.
[  OK  ] Stopped Regular background program processing daemon.
[  OK  ] Stopped Getty on tty1.
[  OK  ] Stopped User Manager for UID 1000.
[  OK  ] Stopped Session 1 of user ubuntu.
[  OK  ] Stopped D-Bus System Message Bus.
[  OK  ] Stopped FUSE filesystem for LXC.
[  OK  ] Stopped LXD - container startup/shutdown.
[  OK  ] Stopped LSB: MD monitoring daemon.
[  OK  ] Stopped LSB: Record successful boot for GRUB.
[  OK  ] Stopped LSB: daemon to balance interrupts for SMP systems.
[  OK  ] Stopped LSB: automatic crash report generation.
[  OK  ] Stopped LSB: Set the CPU Frequency Scaling governor to "ondemand".
[  OK  ] Stopped User Manager for UID 1000.
[  OK  ] Removed slice User Slice of ubuntu.
         Stopping Login Service...
[  OK  ] Removed slice system-getty.slice.
[  OK  ] Removed slice system-serial\x2dgetty.slice.
[  OK  ] Stopped /etc/rc.local Compatibility.
         Stopping Permit User Sessions...
[  OK  ] Stopped Login Service.
[  OK  ] Stopped Permit User Sessions.
[  OK  ] Stopped target Remote File Systems.
[  OK  ] Stopped target Remote File Systems (Pre).
         Stopping Login to default iSCSI targets...
[  OK  ] Stopped target User and Group Name Lookups.
[  OK  ] Stopped target Basic System.
[  OK  ] Stopped target Paths.
[  OK  ] Stopped Forward Password Requests to Wall Directory Watch.
[  OK  ] Stopped Trigger resolvconf update for networkd DNS.
[  OK  ] Stopped Dispatch Password Requests to Console Directory Watch.
[  OK  ] Stopped ACPI Events Check.
[  OK  ] Stopped target Slices.
[  OK  ] Removed slice User and Session Slice.
[  OK  ] Stopped target Sockets.
[  OK  ] Closed ACPID Listen Socket.
[  OK  ] Closed UUID daemon activation socket.
[  OK  ] Closed LXD - unix socket.
[  OK  ] Closed Syslog Socket.
[  OK  ] Closed D-Bus System Message Bus Socket.
[  OK  ] Closed Socket activation for snappy daemon.
[  OK  ] Stopped target System Initialization.
         Stopping Network Time Synchronization...
[  OK  ] Stopped target Encrypted Volumes.
         Stopping Load/Save Random Seed...
[  OK  ] Stopped target Swap.
[  OK  ] Stopped Network Time Synchronization.
[  OK  ] Stopped Load/Save Random Seed.
[  OK  ] Stopped Create Volatile Files and Directories.
[  OK  ] Unmounted /home.
[  OK  ] Unmounted /boot.
[  OK  ] Stopped Login to default iSCSI targets.
         Stopping iSCSI initiator daemon (iscsid)...
[  OK  ] Stopped iSCSI initiator daemon (iscsid).
[  OK  ] Stopped target Network is Online.
[  OK  ] Stopped target Network.
[  OK  ] Stopped Initial cloud-init job (metadata service crawler).
         Stopping Raise network interfaces...
[  OK  ] Stopped Raise network interfaces.
[  OK  ] Stopped target Network (Pre).
[  OK  ] Stopped Initial cloud-init job (pre-networking).
[  OK  ] Stopped target Local File Systems.
[  OK  ] Started Unattended Upgrades Shutdown.
         Unmounting /run/user/1000...
         Unmounting /tmp...
[  OK  ] Stopped Apply Kernel Variables.
[  OK  ] Stopped Load Kernel Modules.
[  OK  ] Unmounted /run/user/1000.
[  OK  ] Unmounted /tmp.
[  OK  ] Reached target Unmount All Filesystems.
[  OK  ] Stopped target Local File Systems (Pre).
         Stopping Monitoring of LVM2 mirrors... dmeventd or progress polling...
[  OK  ] Stopped Remount Root and Kernel File Systems.
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  OK  ] Reached target Shutdown.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1654600

Title:
  unattended-upgrade-shutdown hangs when /var is a separate filesystem

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  Confirmed
Status in unattended-upgrades source package in Yakkety:
  Confirmed
Status in unattended-upgrades package in Debian:
  New

Bug description:
  The systemd unit file unattended-upgrades.service is used to stop a running unattended-upgrade
  process during shutdown. This unit file is running together with all filesystem
  unmount services.

  The unattended-upgrades service checks if the lockfile for unattended-upgrade
  (in /var/run) exists, and if it does, there is an unattended-upgrade in progress
  and the service will wait until it finishes (and therefore automatically wait at
  shutdown).

  However, if /var is a separate filesystem, it will get unmounted even though /var/run
  is a tmpfs that's still mounted on top of the /var/run directory in the /var filesystem.
  The unattended-upgrade script will fail to find lockfile, sleeps for 5 seconds, and
  tries to check the lockfile again. After 10 minutes (the default timeout), it will finally
  exit and the system will continue shutdown.

  The problem is the error handling in /usr/share/unattended-upgrades/unattended-upgrade-shutdown
  where it tries to lock itself:

      while True:
          res = apt_pkg.get_lock(options.lock_file)
          logging.debug("get_lock returned %i" % res)
          # exit here if there is no lock
          if res > 0:
              logging.debug("lock not taken")
              break
          lock_was_taken = True

  The function apt_pkg.get_lock() either returns a file descriptor, or -1 on an error.
  File descriptors are just C file descriptors, so they are always positive integers.
  The code should check the result to be negative, not positive. I have attached a patch
  to reverse the logic.

  Additional information:

  1)
  Description:	Ubuntu 16.04.1 LTS
  Release:	16.04

  2)
  unattended-upgrades:
    Installed: 0.90ubuntu0.3
    Candidate: 0.90ubuntu0.3
    Version table:
   *** 0.90ubuntu0.3 500
          500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
          500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages
          100 /var/lib/dpkg/status
       0.90 500
          500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
          500 http://nl.archive.ubuntu.com/ubuntu xenial/main i386 Packages
  3)
  Fast reboot
  4)
  Very slow reboot (after a 10 minutes timeout)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1654600/+subscriptions