[Bug 2051766] Re: needrestart should priorize and wait for network and DNS when restarting services

Simon Chopin 2051766 at bugs.launchpad.net
Mon Jun 3 13:01:15 UTC 2024 

I'm guessing the glibc security update is the one that caused all those
services to restart.

Is that something you can reproduce reliably, e.g. using `apt reinstall libc6`?
I might have an idea that could solve this problem without having to special-case networking service, but that's a theory and I'd need to actually test it out.

The gist of it would be to replace the single `systemctl restart` call
by multiple ones. My underlying assumption is that systemd will batch up
all those services for restart in parallel (unless there is an explicit
ordering in the system files), whereas using dedicated restart calls
means that, assuming the services are properly written to signal
themselves as ready when they actually are, any service should always
have a stable environment to use during its startup.

** Also affects: needrestart (Ubuntu Noble)
   Importance: Undecided
       Status: New

** Also affects: needrestart (Ubuntu Oracular)
   Importance: Undecided
       Status: New

** Changed in: needrestart (Ubuntu Noble)
     Assignee: (unassigned) => Simon Chopin (schopin)

** Changed in: needrestart (Ubuntu Oracular)
     Assignee: (unassigned) => Simon Chopin (schopin)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to needrestart in Ubuntu.
https://bugs.launchpad.net/bugs/2051766

Title:
  needrestart should priorize and wait for network and DNS when
  restarting services

Status in needrestart package in Ubuntu:
  New
Status in needrestart source package in Noble:
  New
Status in needrestart source package in Oracular:
  New

Bug description:
  I got today the same problem in 2 servers.

  In one of they the problem was nginx and on the other server was
  proftpd.

  I didn't touched any config on both servers, I simply tried to start
  the services another time and it worked, so I think it's a network or
  DNS related problem.

  I think needrestart should have a list of processes (we will call they
  'first to restart' services) which can affect the restart of other
  services. Priorize the restart of those 'first to restart services'
  and wait for network and DNS before restarting other services which
  need to be restarted.

  IMHO those 'first to restart' services should be, on this report
  scenario, the services systemd-networkd.service and systemd-
  resolved.service.

  
  Let's check the logs:

  Those are the services needrestart restarted and I got one error on
  nginx.

  ---------------------------------
  Restarting services...
   /etc/needrestart/restart.d/systemd-manager
   systemctl restart nginx.service packagekit.service ssh.service systemd-journald.service systemd-networkd.service systemd-resolved.service systemd-udevd.service udisks2.service
  Job for nginx.service failed because the control process exited with error code.
  See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.
  Service restarts being deferred:
   systemctl restart systemd-logind.service
   systemctl restart user at 1000.service
  ---------------------------------

  Checking the log I see the failure when needrestart tried to do this
  job, and the next block of the log shows my manual start of the
  service just before the failure.

  ---------------------------------
  Jan 31 08:12:39 ip-172-31-14-249 systemd[1]: Failed to start A high performance web server and a reverse proxy server.
  ░░ Subject: A start job for unit nginx.service has failed
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ A start job for unit nginx.service has finished with a failure.
  ░░
  ░░ The job identifier is 9587 and the job result is failed.

  Jan 31 08:12:50 ip-172-31-14-249 systemd[1]: Starting A high performance web server and a reverse proxy server...
  ░░ Subject: A start job for unit nginx.service has begun execution
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ A start job for unit nginx.service has begun execution.
  ---------------------------------

  
  These are the logs for the other server when I got the problem when restarting proftpd. 08:09:06=failure, 08:09:33=ok when restarted manually

  ---------------------------------
  Jan 31 08:09:06 ip-172-31-22-67 systemd[1]: Stopping ProFTPD FTP Server...
  ░░ Subject: A stop job for unit proftpd.service has begun execution
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ A stop job for unit proftpd.service has begun execution.
  ░░
  ░░ The job identifier is 26554.
  Jan 31 08:09:06 ip-172-31-22-67 proftpd[62832]: Checking syntax of configuration file
  Jan 31 08:09:06 ip-172-31-22-67 proftpd[62832]: 2024-01-31 08:09:06,971 ip-172-31-22-67 proftpd[62832]: mod_memcache/0.1: compiled using libmemcached-1.0.18 headers, but linke>
  Jan 31 08:09:06 ip-172-31-22-67 proftpd[62832]: 2024-01-31 08:09:06,986 ip-172-31-22-67 proftpd[62832]: warning: unable to determine IP address of 'ip-172-31-22-67'
  Jan 31 08:09:06 ip-172-31-22-67 proftpd[62832]: 2024-01-31 08:09:06,986 ip-172-31-22-67 proftpd[62832]: error: no valid servers configured
  Jan 31 08:09:06 ip-172-31-22-67 proftpd[62832]: 2024-01-31 08:09:06,986 ip-172-31-22-67 proftpd[62832]: fatal: error processing configuration file '/etc/proftpd/proftpd.conf'
  Jan 31 08:09:07 ip-172-31-22-67 systemd[1]: proftpd.service: Control process exited, code=exited, status=1/FAILURE
  ░░ Subject: Unit process exited
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ An ExecStartPre= process belonging to unit proftpd.service has exited.
  ░░
  ░░ The process' exit code is 'exited' and its exit status is 1.
  Jan 31 08:09:07 ip-172-31-22-67 systemd[1]: proftpd.service: Failed with result 'exit-code'.
  ░░ Subject: Unit failed
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ The unit proftpd.service has entered the 'failed' state with result 'exit-code'.
  Jan 31 08:09:07 ip-172-31-22-67 systemd[1]: Failed to start ProFTPD FTP Server.
  ░░ Subject: A start job for unit proftpd.service has failed
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ A start job for unit proftpd.service has finished with a failure.
  ░░
  ░░ The job identifier is 26554 and the job result is failed.
  Jan 31 08:09:33 ip-172-31-22-67 systemd[1]: Starting ProFTPD FTP Server...
  ░░ Subject: A start job for unit proftpd.service has begun execution
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ A start job for unit proftpd.service has begun execution.
  ░░
  ░░ The job identifier is 26875.
  Jan 31 08:09:33 ip-172-31-22-67 proftpd[63122]: Checking syntax of configuration file
  Jan 31 08:09:33 ip-172-31-22-67 proftpd[63122]: 2024-01-31 08:09:33,532 ip-172-31-22-67 proftpd[63122]: mod_memcache/0.1: compiled using libmemcached-1.0.18 headers, but linke>
  Jan 31 08:09:33 ip-172-31-22-67 proftpd[63122]: 2024-01-31 08:09:33,541 ip-172-31-22-67 proftpd[63122] ip-172-31-22-67.eu-west-1.compute.internal: 172.31.22.67:21 masquerading>
  Jan 31 08:09:33 ip-172-31-22-67 proftpd[63123]: 2024-01-31 08:09:33,591 ip-172-31-22-67 proftpd[63123]: mod_memcache/0.1: compiled using libmemcached-1.0.18 headers, but linke>
  Jan 31 08:09:33 ip-172-31-22-67 proftpd[63123]: 2024-01-31 08:09:33,595 ip-172-31-22-67 proftpd[63123] ip-172-31-22-67.eu-west-1.compute.internal: 172.31.22.67:21 masquerading>
  Jan 31 08:09:33 ip-172-31-22-67 systemd[1]: Started ProFTPD FTP Server.
  ░░ Subject: A start job for unit proftpd.service has finished successfully
  ░░ Defined-By: systemd
  ░░ Support: http://www.ubuntu.com/support
  ░░
  ░░ A start job for unit proftpd.service has finished successfully.
  ░░
  ░░ The job identifier is 26875.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/needrestart/+bug/2051766/+subscriptions

More information about the foundations-bugs mailing list