[Bug 1709384] Re: failed to unmount filesystems during shutdown, reboot hangs

[ben thielsen]

i've figured out at least part of this issue.

the system now no longer hangs at shutdown, but there are still
filesystems it complains it is unable to unmount, and the system still
hangs if it unable to unmount /home, which happens to be an nfs share
[i'm uncertain if this is of particular relevance, but it is a
characteristic unique among this system's filesystems].

the root cause was pam_systemd inadvertently excluded from the pam
session config, following customization.  this was causing another
problem [ssh sessions hanging during shutdown instead of being cleanly
terminated], and it was during troubleshooting of the ssh issue when it
hit me that there was a relationship between the two.

during shutdown, systemd brings down the network connection before ssh
processes are gracefully disconnected/terminated.  this leaves the
client unaware, thus the hanging symptom.

additionally, this appears to also leave processes active or files open,
etc., in various locations, and thus affected filesystems are still in
use and cannot be unmounted.  to make matters worse, when the filesystem
in use happens to be an nfs share [in this case,
/home/foo.example.com/], the system hangs at shutdown.

with pam_systemd properly configured in the pam session config, ssh
processes/sessions get to gracefully close, /home/foo.example.com/ and
/tmp/ are cleanly/successfully unmounted, and since the nfs share gets
unmounted, the system does not hang.  however, /var/ and /var/log/ still
fail to be unmounted:

Aug 30 03:27:09 template systemd[1]: Failed unmounting /var/log.
Aug 30 03:27:09 template systemd[1]: Failed unmounting /var.

this is still a problem which shouldn't be happening, but fortunately,
since it's only the nfs unmount fail that makes the system hang, reboots
aren't disastrous.

there are still issues here that need to be addressed:

1] under nominal conditions, filesystems should cleanly unmount.  unmounting should not fail.
2] if a filesystem fails to unmount [nfs or otherwise], the system should not hang during shutdown.

this can be recreated by doing the following:

• configure the system to mount an nfs share at boot [i used fstab].  it may also be enough to just mount an nfs share manually after boot
• disable pam_systemd in the pam session config
• ssh to the computer
• change directory into the nfs share, so the filesystem is in use
• reboot

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1709384

Title:
  failed to unmount filesystems during shutdown, reboot hangs

Status in systemd package in Ubuntu:
  New

Bug description:
  i'd done a new install of 17.04, and this had been working ok.  this
  morning, i did an update, and now the system complains about being
  unable to unmount certain filesystems, as well as hanging during the
  shutdown process upon displaying "[ ok ] reached target shutdown"

  Aug  8 16:11:48 template systemd[1]: Failed unmounting /home/foo.example.com.
  Aug  8 16:11:49 template systemd[1]: Failed unmounting /var/log.
  Aug  8 16:11:49 template systemd[1]: Failed unmounting /tmp.
  Aug  8 16:11:49 template systemd[1]: Failed unmounting /home.
  Aug  8 16:11:49 template systemd[1]: Failed unmounting /var.

  1] >lsb_release -rd
  Description:	Ubuntu 17.04
  Release:	17.04

  2] >apt-cache policy systemd
  systemd:
    Installed: 232-21ubuntu5
    Candidate: 232-21ubuntu5
    Version table:
   *** 232-21ubuntu5 500
          500 http://us.archive.ubuntu.com/ubuntu zesty-updates/main amd64 Packages
          500 http://security.ubuntu.com/ubuntu zesty-security/main amd64 Packages
          100 /var/lib/dpkg/status
       232-21ubuntu2 500
          500 http://us.archive.ubuntu.com/ubuntu zesty/main amd64 Packages

  3] i expected the system to not fail when unmounting filesystems during shutdown
  4] it did

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1709384/+subscriptions