[Bug 1928259] Re: Package upgrade won't restart services
Andreas Hasenack
1928259 at bugs.launchpad.net
Mon Jun 7 19:50:43 UTC 2021
** Description changed:
[Impact]
+ In order to get the fixes provided by a package update, the affected services shipped in it need to be restarted. When that restart does not happen, the system remains running the old binaries with the bug(s).
- * An explanation of the effects of the bug on users and
-
- * justification for backporting the fix to the stable release.
-
- * In addition, it is helpful, but not required, to include an
- explanation of how the upload fixes this bug.
+ This bug was found while testing the fix for #1927745, which affected
+ rpc.gssd, one of the services shipped in nfs-common. Without the
+ restart, systems that installed the update are still affected by the
+ bug.
[Test Plan]
+ To make the test simple, we are not going to mount an NFSv4 share using kerberos. We are just going to have a minimal configuration that gets rpc.gssd running to demonstrate the before and after of this bug.
- * detailed instructions how to reproduce the bug
+ # create a VM for the affected ubuntu release under test, login and run:
+ sudo touch /etc/krb5.keytab
+ sudo chmod 0600 /etc/krb5.keytab
- * these should allow someone who is not familiar with the affected
- package to reproduce the bug and verify that the updated package fixes
- the problem.
+ # install nfs-common
+ sudo apt install nfs-common -y
- * if other testing is appropriate to perform before landing this update,
- this should also be described here.
+ # note message about nfs-utils.service being disabled/static:
+ nfs-utils.service is a disabled or a static unit, not starting it.
+
+ # Manually start rpc-gssd. It will start, but since we have an empty
+ # krb5.keytab file, it won't work. That's ok, we are not actually going to
+ # mount nfsv4
+ systemctl start rpc-gssd.service
+
+ # Check it's running, and make note of its pid:
+ pidof rpc.gssd
+ 2994
+
+ # reinstall nfs-common
+ sudo apt install --reinstall nfs-common
+
+ # note rpc-gssd wasn't restarted
+ pidof rpc.gssd
+ 2994
+
+ # install the fixed nfs-common package:
+ sudo apt install nfs-commont
+
+ # this time, rpc.gssd is restarted
+ pidof rpc.gssd
+ 5000
+
[Where problems could occur]
- * Think about what the upload changes in the software. Imagine the change is
- wrong or breaks something else: how would this show up?
+ * Think about what the upload changes in the software. Imagine the change is
+ wrong or breaks something else: how would this show up?
- * It is assumed that any SRU candidate patch is well-tested before
- upload and has a low overall risk of regression, but it's important
- to make the effort to think about what ''could'' happen in the
- event of a regression.
+ * It is assumed that any SRU candidate patch is well-tested before
+ upload and has a low overall risk of regression, but it's important
+ to make the effort to think about what ''could'' happen in the
+ event of a regression.
- * This must '''never''' be "None" or "Low", or entirely an argument as to why
- your upload is low risk.
+ * This must '''never''' be "None" or "Low", or entirely an argument as to why
+ your upload is low risk.
- * This both shows the SRU team that the risks have been considered,
- and provides guidance to testers in regression-testing the SRU.
+ * This both shows the SRU team that the risks have been considered,
+ and provides guidance to testers in regression-testing the SRU.
[Other Info]
-
- * Anything else you think is useful to include
- * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
- * and address these questions in advance
+ * Anything else you think is useful to include
+ * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
+ * and address these questions in advance
[Original Description]
Upgrading the nfs-common debian package will not restart its services.
Specifically, the package tries to restart "nfs-utils.service", which is a "fake" service meant to coordinate all the other daemons that make up a modern NFS server. This service, however, as it is, cannot be enabled:
$ sudo systemctl enable nfs-utils.service
The unit files have no installation config (WantedBy, RequiredBy, Also, Alias
settings in the [Install] section, and DefaultInstance for template units).
This means they are not meant to be enabled using systemctl.
Possible reasons for having this kind of units are:
1) A unit may be statically enabled by being symlinked from another unit's
.wants/ or .requires/ directory.
2) A unit's purpose may be to act as a helper for some other unit which has
a requirement dependency on it.
3) A unit may be started when needed via activation (socket, path, timer,
D-Bus, udev, scripted systemctl call, ...).
4) In case of template units, the unit is meant to be enabled with some
instance name specified
Granted, d/rules of the nfs-utils package doesn't even try:
dh_systemd_enable -p nfs-common nfs-client.target
dh_systemd_enable -p nfs-kernel-server nfs-server.service
dh_installinit -pnfs-common -R
dh_systemd_start -p nfs-common --restart-after-upgrade nfs-utils.service
dh_systemd_start -p nfs-kernel-server --restart-after-upgrade nfs-server.service
We can see it tries to start and restart it, but that won't work on disabled or non-started services: deb-systemd-invoke won't do it:
# If the job is disabled and is not currently running, the job is not started or restarted.
# However, if the job is disabled but has been forced into the running state, we *do* stop
# and restart it since this is expected behaviour for the admin who forced the start.
# We don't autostart static units either.
The above can be seen while attempting a fresh install (or even upgrade) of nfs-common:
(...)
Setting up nfs-common (1:1.3.4-2.5ubuntu6) ...
Creating config file /etc/idmapd.conf with new version
Adding system user `statd' (UID 113) ...
Adding new user `statd' (UID 113) with group `nogroup' ...
Not creating home directory `/var/lib/nfs'.
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-client.target → /lib/systemd/system/nfs-client.target.
Created symlink /etc/systemd/system/remote-fs.target.wants/nfs-client.target → /lib/systemd/system/nfs-client.target.
nfs-utils.service is a disabled or a static unit, not starting it.
^^^^^^^^^^^^^^^^^
$ systemctl status nfs-utils.service
● nfs-utils.service - NFS server and client services
Loaded: loaded (/lib/systemd/system/nfs-utils.service; static)
Active: inactive (dead)
This was found while testing the fix for bug #1927745. In that bug, the
affected service is rpc.gssd and it's critical that it be restarted, but
it's not happening. It will only be restarted if nfs-utils.service is
already "started".
I'm marking this bug as "high" because it prevents valid fixes from
being deployed after just upgrading a package.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to nfs-utils in Ubuntu.
https://bugs.launchpad.net/bugs/1928259
Title:
Package upgrade won't restart services
Status in nfs-utils package in Ubuntu:
In Progress
Status in nfs-utils source package in Bionic:
In Progress
Status in nfs-utils source package in Focal:
In Progress
Status in nfs-utils source package in Groovy:
In Progress
Status in nfs-utils source package in Hirsute:
In Progress
Status in nfs-utils package in Debian:
New
Status in nfs-utils package in Fedora:
Confirmed
Bug description:
[Impact]
In order to get the fixes provided by a package update, the affected services shipped in it need to be restarted. When that restart does not happen, the system remains running the old binaries with the bug(s).
This bug was found while testing the fix for #1927745, which affected
rpc.gssd, one of the services shipped in nfs-common. Without the
restart, systems that installed the update are still affected by the
bug.
[Test Plan]
To make the test simple, we are not going to mount an NFSv4 share using kerberos. We are just going to have a minimal configuration that gets rpc.gssd running to demonstrate the before and after of this bug.
# create a VM for the affected ubuntu release under test, login and run:
sudo touch /etc/krb5.keytab
sudo chmod 0600 /etc/krb5.keytab
# install nfs-common
sudo apt install nfs-common -y
# note message about nfs-utils.service being disabled/static:
nfs-utils.service is a disabled or a static unit, not starting it.
# Manually start rpc-gssd. It will start, but since we have an empty
# krb5.keytab file, it won't work. That's ok, we are not actually going to
# mount nfsv4
systemctl start rpc-gssd.service
# Check it's running, and make note of its pid:
pidof rpc.gssd
2994
# reinstall nfs-common
sudo apt install --reinstall nfs-common
# note rpc-gssd wasn't restarted
pidof rpc.gssd
2994
# install the fixed nfs-common package:
sudo apt install nfs-commont
# this time, rpc.gssd is restarted
pidof rpc.gssd
5000
[Where problems could occur]
* Think about what the upload changes in the software. Imagine the change is
wrong or breaks something else: how would this show up?
* It is assumed that any SRU candidate patch is well-tested before
upload and has a low overall risk of regression, but it's important
to make the effort to think about what ''could'' happen in the
event of a regression.
* This must '''never''' be "None" or "Low", or entirely an argument as to why
your upload is low risk.
* This both shows the SRU team that the risks have been considered,
and provides guidance to testers in regression-testing the SRU.
[Other Info]
* Anything else you think is useful to include
* Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
* and address these questions in advance
[Original Description]
Upgrading the nfs-common debian package will not restart its services.
Specifically, the package tries to restart "nfs-utils.service", which is a "fake" service meant to coordinate all the other daemons that make up a modern NFS server. This service, however, as it is, cannot be enabled:
$ sudo systemctl enable nfs-utils.service
The unit files have no installation config (WantedBy, RequiredBy, Also, Alias
settings in the [Install] section, and DefaultInstance for template units).
This means they are not meant to be enabled using systemctl.
Possible reasons for having this kind of units are:
1) A unit may be statically enabled by being symlinked from another unit's
.wants/ or .requires/ directory.
2) A unit's purpose may be to act as a helper for some other unit which has
a requirement dependency on it.
3) A unit may be started when needed via activation (socket, path, timer,
D-Bus, udev, scripted systemctl call, ...).
4) In case of template units, the unit is meant to be enabled with some
instance name specified
Granted, d/rules of the nfs-utils package doesn't even try:
dh_systemd_enable -p nfs-common nfs-client.target
dh_systemd_enable -p nfs-kernel-server nfs-server.service
dh_installinit -pnfs-common -R
dh_systemd_start -p nfs-common --restart-after-upgrade nfs-utils.service
dh_systemd_start -p nfs-kernel-server --restart-after-upgrade nfs-server.service
We can see it tries to start and restart it, but that won't work on disabled or non-started services: deb-systemd-invoke won't do it:
# If the job is disabled and is not currently running, the job is not started or restarted.
# However, if the job is disabled but has been forced into the running state, we *do* stop
# and restart it since this is expected behaviour for the admin who forced the start.
# We don't autostart static units either.
The above can be seen while attempting a fresh install (or even upgrade) of nfs-common:
(...)
Setting up nfs-common (1:1.3.4-2.5ubuntu6) ...
Creating config file /etc/idmapd.conf with new version
Adding system user `statd' (UID 113) ...
Adding new user `statd' (UID 113) with group `nogroup' ...
Not creating home directory `/var/lib/nfs'.
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-client.target → /lib/systemd/system/nfs-client.target.
Created symlink /etc/systemd/system/remote-fs.target.wants/nfs-client.target → /lib/systemd/system/nfs-client.target.
nfs-utils.service is a disabled or a static unit, not starting it.
^^^^^^^^^^^^^^^^^
$ systemctl status nfs-utils.service
● nfs-utils.service - NFS server and client services
Loaded: loaded (/lib/systemd/system/nfs-utils.service; static)
Active: inactive (dead)
This was found while testing the fix for bug #1927745. In that bug,
the affected service is rpc.gssd and it's critical that it be
restarted, but it's not happening. It will only be restarted if nfs-
utils.service is already "started".
I'm marking this bug as "high" because it prevents valid fixes from
being deployed after just upgrading a package.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1928259/+subscriptions
More information about the foundations-bugs
mailing list