[Bug 1958284] Re: shutdown hangs at "Waiting for process: ..." for 90s, ignoring DefaultTimeoutStopSec

[Lukas Märdian]

It has recently been picked up by Foundations, and we should have the
capacity to start working on this next week.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1958284

Title:
  shutdown hangs at "Waiting for process: ..." for 90s, ignoring
  DefaultTimeoutStopSec

Status in systemd package in Ubuntu:
  Confirmed
Status in systemd source package in Focal:
  Confirmed

Bug description:
  With systemd v245 as shipped with 20.04, the shutdown sequence does
  not use the value of `DefaultTimeoutStopSec` to wait for remaining
  processes, it instead uses the compiled in default of 90s.

  This is most visible with services that use `KillMode=process`
  (docker, k8s, k3s, etc...), especially if the remaining processes do
  not handle `SIGTERM` or choose to ignore it.

  For example:
  ```
  [ OK ] Finished Reboot.
  [ OK ] Reached target Reboot.
  [ 243.652848 ] systemd-shutdown[1]: Waiting for process: containerd-shim, containerd-shim, containerd-shim, fluent-bit

  --- hangs here for 90s even if DefaultTimeoutStopSec is set to a lower
  value ---

  ```

  The bug has been fixed upstream here:
  https://github.com/systemd/systemd/commit/7d9eea2bd3d4f83668c7a78754d201b22

  Marc was kind enough to package the patch for 20.04 so I could test it
  (https://launchpad.net/~mdeslaur/+archive/ubuntu/testing/+sourcepub/13210617/+listing-
  archive-extra) and with that package, I can confirm that it indeed
  fixes the issue.

  Here's a few github issues I stumbled upon while trying to debug this,
  along with a short writeup of the workaround I ended up using:

  - https://github.com/moby/moby/issues/41831
  - https://github.com/k3s-io/k3s/issues/2400
  - https://github.com/systemd/systemd/issues/16991
  - https://raby.sh/debugging-90s-hangs-during-shutdown-on-ubuntu-2004.html

  Of course, it would be much better if all the processes would properly
  handle `SIGTERM`, but having a way to enforce a maximum wait time at
  shutdown is a decent workaround.

  Given that the patch is relatively simple, would it be possible to add
  it the package for 20.04?

  Thanks

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1958284/+subscriptions