[Bug 1795658] Re: xenial systemd reports 'inactive' instead of 'failed' for service units that repeatedly failed to restart / failed permanently

[Robie Basak]

Hello Mauricio, or anyone else affected,

Accepted systemd into xenial-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/systemd/229-4ubuntu21.5 in a few
hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested and change the tag from
verification-needed-xenial to verification-done-xenial. If it does not
fix the bug for you, please add a comment stating that, and change the
tag to verification-failed-xenial. In either case, details of your
testing will help us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: systemd (Ubuntu Xenial)
       Status: In Progress => Fix Committed

** Tags added: verification-needed verification-needed-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1795658

Title:
  xenial systemd reports 'inactive' instead of 'failed' for service
  units that repeatedly failed to restart / failed permanently

Status in systemd package in Ubuntu:
  Invalid
Status in systemd source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

   * In case a service unit has repeatedly failed to restart, it should be
     reported as 'failed' permanently, but currently it's instead reported
     as 'inactive'.

   * System monitoring tools that evaluate the status of systemd service units
     and act upon it (for example: restart service, report permanent failure)
     are currently misled by information in 'systemctl status <unit>.service'.

   * System management tools based on such information may take wrong and/or
     sub-optimal actions in the managed systems regarding such service units.

   * This systemd patch [1] directly addresses this issue (see systemd github
     PR #3166 [2]), and its code is still effectice in upstream systemd today,
     without further fixes/changes (the only changes were in doc text and the
     busname files that were removed, but still without further fixes to this).

  [Test Case]

   * This is copied from systemd PR #3166 [2].

   * This has been tested by a customer as well, and with its system monitoring
     and management solution, for interoperability verification.

      $ cat <<EOF | sudo tee /etc/systemd/system/fail-on-restart.service
      [Service]
      ExecStart=/bin/false
      Restart=always
      EOF

      $ sudo systemctl daemon-reload
      $ sudo systemctl start fail-on-restart

      Before) "Active: inactive (dead)"

      $ systemctl status -n0 fail-on-restart
      fail-on-restart.service
         Loaded: loaded (/etc/systemd/system/fail-on-restart.service; static; vendor preset: enabled)
         Active: inactive (dead)

      After) "Active: failed (Result: start-limit-hit)"
   
      $ systemctl status -n0 fail-on-restart
      fail-on-restart.service
         Loaded: loaded (/etc/systemd/system/fail-on-restart.service; static; vendor preset: enabled)
         Active: failed (Result: start-limit-hit) since Sat 2018-09-29 11:01:34 UTC; 4s ago
        Process: 7066 ExecStart=/bin/false (code=exited, status=1/FAILURE)
       Main PID: 7066 (code=exited, status=1/FAILURE)

  [Regression Potential]

   * This code changes at which point the check for the number of (re)start
     attempts are made, so regressions to (re)start units are theoretically
     possible.

   * However, this code actually reverts a change that caused a regression,
     so it goes back to the code that was known to work correctly before ..

   * .. and it is still in this form in upstream systemd nowadays, 
     without further fixes/changes (see comment in the Impact section).

  [Other Info]

   * Test package was built on Launchpad PPA for all architectures,
     with dependencies from Proposed enabled (more up-to-date for SRU).

   * The testsuite (in package build time; blocks the package build result)
     has identical results to that in buildlog of current xenial-updates.

      ============================================================================
      Testsuite summary for systemd 229
      ============================================================================
      # TOTAL: 128
      # PASS:  109
      # SKIP:  19
      # XFAIL: 0
      # FAIL:  0
      # XPASS: 0
      # ERROR: 0
      ============================================================================

  [Links]
   
  [1] https://github.com/systemd/systemd/commit/072993504e3e4206ae1019f5461a0372f7d82ddf
  [2] https://github.com/systemd/systemd/issues/3166
  [3] https://launchpad.net/~mfo/+archive/ubuntu/sf199312

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1795658/+subscriptions