[Bug 1839290] Re: systemd doesn't restart a service after crashes

Wed May 6 11:26:30 UTC 2020

bionic:

ubuntu at lp1839290-b:~$ dpkg -l systemd|grep systemd
ii  systemd        237-3ubuntu10.39 amd64        system and service manager
ubuntu at lp1839290-b:~$ systemctl cat foo
# /etc/systemd/system/foo.service
[Unit]
PartOf=bar.service
[Service]
ExecStart=/bin/sleep infinity
RestartSec=0s
Restart=always

ubuntu at lp1839290-b:~$ systemctl cat bar
# /etc/systemd/system/bar.service
[Unit]
BindsTo=foo.service
#Same with or without After=, the window is very small to make a difference.
After=foo.service
[Service]
ExecStart=/bin/sleep infinity

ubuntu at lp1839290-b:~$ sudo systemctl start foo
ubuntu at lp1839290-b:~$ sudo systemctl start bar
ubuntu at lp1839290-b:~$ systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:20:31 UTC; 4s ago
 Main PID: 8560 (sleep)
    Tasks: 1 (limit: 4702)
   CGroup: /system.slice/foo.service
           └─8560 /bin/sleep infinity

May 06 11:20:31 lp1839290-b systemd[1]: Started foo.service.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:20:33 UTC; 3s ago
 Main PID: 8570 (sleep)
    Tasks: 1 (limit: 4702)
   CGroup: /system.slice/bar.service
           └─8570 /bin/sleep infinity

May 06 11:20:33 lp1839290-b systemd[1]: Started bar.service.

ubuntu at lp1839290-b:~$ sudo kill -6 8560
ubuntu at lp1839290-b:~$ systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: failed (Result: core-dump) since Wed 2020-05-06 11:21:08 UTC; 1s ago
  Process: 8560 ExecStart=/bin/sleep infinity (code=dumped, signal=ABRT)
 Main PID: 8560 (code=dumped, signal=ABRT)

May 06 11:21:08 lp1839290-b systemd[1]: foo.service: Service has no hold-off time, scheduling restart.
May 06 11:21:08 lp1839290-b systemd[1]: foo.service: Failed to schedule restart job: Transaction is destructive.
May 06 11:21:08 lp1839290-b systemd[1]: foo.service: Failed with result 'core-dump'.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: inactive (dead) since Wed 2020-05-06 11:21:08 UTC; 1s ago
  Process: 8570 ExecStart=/bin/sleep infinity (code=killed, signal=TERM)
 Main PID: 8570 (code=killed, signal=TERM)

May 06 11:20:33 lp1839290-b systemd[1]: Started bar.service.
May 06 11:21:08 lp1839290-b systemd[1]: Stopping bar.service...
May 06 11:21:08 lp1839290-b systemd[1]: Stopped bar.service.

ubuntu at lp1839290-b:~$ dpkg -l systemd|grep systemd
ii  systemd        237-3ubuntu10.40 amd64        system and service manager
ubuntu at lp1839290-b:~$ sudo systemctl start bar
ubuntu at lp1839290-b:~$ sudo systemctl start foo
ubuntu at lp1839290-b:~$ sudo systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:24:57 UTC; 10s ago
 Main PID: 1256 (sleep)
    Tasks: 1 (limit: 4702)
   CGroup: /system.slice/foo.service
           └─1256 /bin/sleep infinity

May 06 11:24:57 lp1839290-b systemd[1]: Started foo.service.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:24:57 UTC; 10s ago
 Main PID: 1258 (sleep)
    Tasks: 1 (limit: 4702)
   CGroup: /system.slice/bar.service
           └─1258 /bin/sleep infinity

May 06 11:24:57 lp1839290-b systemd[1]: Started bar.service.
ubuntu at lp1839290-b:~$ sudo kill -6 1256
ubuntu at lp1839290-b:~$ sudo systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:25:19 UTC; 2s ago
 Main PID: 1276 (sleep)
    Tasks: 1 (limit: 4702)
   CGroup: /system.slice/foo.service
           └─1276 /bin/sleep infinity

May 06 11:25:19 lp1839290-b systemd[1]: Started foo.service.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:25:19 UTC; 2s ago
 Main PID: 1277 (sleep)
    Tasks: 1 (limit: 4702)
   CGroup: /system.slice/bar.service
           └─1277 /bin/sleep infinity

May 06 11:25:19 lp1839290-b systemd[1]: Started bar.service.

ubuntu at lp1839290-b:~$ journalctl -b -u foo
-- Logs begin at Wed 2020-05-06 11:11:19 UTC, end at Wed 2020-05-06 11:25:22 UTC. --
May 06 11:24:57 lp1839290-b systemd[1]: Started foo.service.
May 06 11:25:19 lp1839290-b systemd[1]: foo.service: Main process exited, code=dumped, status=6/ABRT
May 06 11:25:19 lp1839290-b systemd[1]: foo.service: Failed with result 'core-dump'.
May 06 11:25:19 lp1839290-b systemd[1]: foo.service: Service has no hold-off time, scheduling restart.
May 06 11:25:19 lp1839290-b systemd[1]: foo.service: Scheduled restart job, restart counter is at 1.
May 06 11:25:19 lp1839290-b systemd[1]: Stopped foo.service.
May 06 11:25:19 lp1839290-b systemd[1]: Started foo.service.
ubuntu at lp1839290-b:~$ journalctl -b -u bar
-- Logs begin at Wed 2020-05-06 11:11:19 UTC, end at Wed 2020-05-06 11:25:57 UTC. --
May 06 11:24:57 lp1839290-b systemd[1]: Started bar.service.
May 06 11:25:19 lp1839290-b systemd[1]: Stopping bar.service...
May 06 11:25:19 lp1839290-b systemd[1]: Stopped bar.service.
May 06 11:25:19 lp1839290-b systemd[1]: Started bar.service.

** Tags removed: verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1839290

Title:
  systemd doesn't restart a service after crashes

Status in systemd:
  Fix Released
Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Xenial:
  Fix Committed
Status in systemd source package in Bionic:
  Fix Committed
Status in systemd source package in Disco:
  Fix Released

Bug description:
  [impact]

  when a service configured to restart crashes, under certain
  configurations, that service is not correctly restarted.

  [test case]

  see comment 8

  [regression potential]

  this changes the job mode of manager-triggered restarts to 'replace'
  any existing queued job(s), instead of failing if there are queued
  job(s).  thus any regressions would occur when a service fails, that
  is configured to restart on failure.

  [scope]

  This is needed only for Xenial and Bionic.

  this is fixed with commit 03ff2dc71ecb09272d728d458498b44f7f132f51
  which is included already in Eoan.

  Fixed in Disco with the backported v240 patch from
  https://github.com/systemd/systemd/commit/677b4cc753f183731fc54fcb68ad46f806c394bc

  [other info]

  original description:
  ---

  Affected versions of OS and systemd:
  $ cat /etc/issue
  Ubuntu 16.04.6 LTS \n \l
  $ systemd --version
  systemd 229
  +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN

  Affected packages:
  systemd 229-4ubuntu21.22 and previous versions.

  Expected behaviour you didn't see:
  Scheduling restart of failed service.
  A process crashed by sigabrt and didn't restart.

  Description:
  The bug was reported to a systemd upstream repository: https://github.com/systemd/systemd/issues/11456
  The bug was fixed and accepted to the master branch: https://github.com/systemd/systemd/pull/11467/files

  Action:
  Include this patch to Ubuntu 16.04 and other version of Ubuntu which are supported.

To manage notifications about this bug go to:
https://bugs.launchpad.net/systemd/+bug/1839290/+subscriptions