[Bug 1982218] Re: wait-online does not correctly identify managed links

[Mike Kasick]

Apologies for the late response:

Yes, we have a Ubuntu 22.04 LTS image that we use across a set of
machines, some of which have multiple Ethernet ports, but for which we
only connect a single port.  For this image we use the following Netplan
configuration (/etc/netplan/ethernet.yaml):

network:
  ethernets:
    en:
      match:
        name: en*
      dhcp4: true
  version: 2

Which places _all_ Ethernet interfaces under systemd-networkd
management, allowing us to bring up at least one interface via DHCP
without needing to know the exact interface name (which depends on the
exact hardware) and enables us to use any of the available Ethernet
ports on the machine.

As part of this image we also have a boot-time systemd service that
requires functional network connectivity in order to successfully start.
The unit file for this service depends on the network-online target
("Wants=network-online.target").  For example purposes you could replace
this service with a remote NFS mount specified in /etc/fstab, since
boot-time NFS mounts implicitly depend on the network-online target as
well.

The default behavior of systemd-networkd-wait-online is to wait until
all networkd-managed interfaces are routable before the network-online
target is started, but since we only ever connect a single Ethernet
interface to the network, we override its behavior here in our image
(/etc/systemd/system/systemd-networkd-wait-online.service.d/any.conf):

[Service]
ExecStart=
ExecStart=/usr/local/lib/systemd-networkd-wait-online --any --timeout=0

Now, the problem introduced by the UBUNTU-wait-online-exit-if-no-links-
are-managed patch is that it places us in a race against our DHCP
server.  During a successful boot, where networkd is able to obtain an
address via DHCP _before_ the wait-online service is reached, networkctl
reports this status:

$ sudo networkctl
IDX LINK   TYPE     OPERATIONAL SETUP      
  1 lo     loopback carrier     unmanaged
  2 enp0s1 ether    routable    configured 
  3 enp0s2 ether    no-carrier  configuring

Since the second interface (enp0s1) is in the configured state and is
routable, systemd-networkd-wait-online behaves correctly by exiting
immediately.  However, during an unsuccessful boot, where networkd has
yet to obtain an address via DHCP at the time the wait-online service is
reached, networkctl reports this status:

$ sudo networkctl
IDX LINK   TYPE     OPERATIONAL SETUP      
  1 lo     loopback carrier     unmanaged
  2 enp0s1 ether    degraded    configuring
  3 enp0s2 ether    no-carrier  configuring

Here, both of the managed Ethernet interfaces are still in the
configuring state.  Since UBUNTU-wait-online-exit-if-no-links-are-
managed doesn't check if managed interfaces are in this state, only
configured, it assumes those links are unmanaged and systemd-networkd-
wait-online also exits immediately.  This results in failure of our
network-dependent service to start (or equivalently, failure of the
remote NFS volume to mount).

With the provided patch, systemd-networkd-wait-online identifies two
managed interfaces in the configuring state and waits until at least one
of them becomes routable, and thus waits until the network is functional
before continuing to boot.

I hope this example better illustrates the issue.  Thanks!

** Changed in: systemd (Ubuntu)
       Status: Expired => New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1982218

Title:
  wait-online does not correctly identify managed links

Status in systemd package in Ubuntu:
  New

Bug description:
  Ubuntu's packaging of systemd-networkd-wait-online includes a patch
  (UBUNTU-wait-online-exit-if-no-links-are-managed) intended to exit if
  none of the present links are managed by systemd-networkd.  However,
  the patch fails to identify links in the "configuring" state, which
  implies they are managed, but not yet fully online.  This undermines
  the purpose of systemd-networkd-wait-online with the --any option,
  since at the time the systemd-networkd-wait-online service is started,
  managed links are _likely_ in a configuring, but-not-yet-configured
  state.

  Please see the attached patch.  Thanks!

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1982218/+subscriptions