[Bug 2036358] Re: systemd wait-online now times out after jammy and lunar upgrade

[Dave Jones]

> If you have a bunch of unmanaged interfaces, then you should use a
> drop-in configuration to pass the --any flag to
> systemd-networkd-wait-online:
> 
> cat > /etc/systemd/system/systemd-networkd-wait-online.service.d/any.conf << EOF
> [Service]
> ExecStart=
> ExecStart=/lib/systemd/systemd-networkd-wait-online --any
> EOF
> 
> This makes it so that systemd-networkd-wait-online will return success once at least one interface becomes configured.

Unfortunately this doesn't appear to work. I've tried this on the
current noble dailies (which also suffer the 2 minute boot delay on
the server images due to this issue). The default netplan config is
simply:

  network:
    version: 2
    ethernets:
      eth0:
        dhcp4: true
        optional: true

This displays the issue, even with --any presumably because wlan0 is
unmanaged so networkctl reports the online state is "unknown" (even
when eth0 is happily configured and routable).

I attempting making wlan0 "managed" but optional. This requires a
mandatory access point to be defined to conform to the YAML schema, so
I added a fake one (I should open a separate issue for this because I
shouldn't have to make up access points just to mark an interface
optional):

  network:
    version: 2
    ethernets:
      eth0:
        dhcp4: true
        optional: true
    wifis:
      wlan0:
        dhcp4: true
        optional: true
        access-points:
          foo:
            password: foobarbaz

Unfortunately this also displays the issue. This time wlan0 is
managed, but "configuring" and the online state is still "unknown". So
far as I can tell, the --any option doesn't match the behaviour
claimed in the man-page (i.e. "exits with success when at least one
interface becomes online").

> There is no workaround for this with the stock netplan config on
> raspberry pi, which trips this bug. But you can customize
> /etc/netplan/ on your system to mark your expected/required network
> interfaces as "optional: false" (which is the default if
> unspecified); AIUI that would avoid this issue with
> systemd-networkd.

This *does* work ... assuming the Pi actually has a network interface.
If the Pi is networkless, it's unfortunately doomed to the 2 minute
delay still. And obviously I don't want to make eth0 mandatory by
default in the images (which would break the 3A+ and Zero 2W, which
lack eth0 entirely).

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/2036358

Title:
  systemd wait-online now times out after jammy and lunar upgrade

Status in systemd package in Ubuntu:
  Invalid
Status in systemd source package in Jammy:
  Fix Released
Status in systemd source package in Lunar:
  Fix Released

Bug description:
  [NOTE]

  If you are running a desktop system and you see this issue, you should
  run:

  $ systemctl disable --now systemd-networkd.service

  This will disable systemd-networkd and associated units, including
  systemd-networkd-wait-online.service. NetworkManager and systemd-
  networkd should not be running at the same time. On desktop,
  NetworkManager is the default network stack.

  [Impact]

  When all interfaces are "not required for online", e.g. when they are
  marked "optional: true" in netplan, systemd-networkd-wait-online will
  timeout. Or, in other words, systemd-networkd-wait-online will timeout
  even though all interfaces are ignored, hence none of them will ever
  be marked as "ready." Depending on what units depend on network-
  online.target, this can delay boot by 120 seconds (the default timeout
  for systemd-networkd-wait-online).

  [Test Plan]

  1. Create a new LXD container. These instructions assume jammy is the
  release, but the same can be done for lunar.

  $ lxc launch ubuntu-daily:jammy jammy
  $ lxc exec jammy bash

  2. Once in the container, modify the default /etc/netplan/10-lxc.yaml
  so that eth0 is configured with "optional: true":

  $ vi /etc/netplan/50-cloud-init.yaml # Use whatever editor you like
  $ cat /etc/netplan/50-cloud-init.yaml
  network:
    version: 2
    ethernets:
      eth0:
        dhcp4: true
        dhcp-identifier: mac
        optional: true

  3. Re-generate and apply the netplan configuration.

  $ netplan generate
  $ netplan apply

  4. Manually run systemd-networkd-wait-online, and observe that all
  links are ignored, and the command times out:

  $ SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-networkd-wait-online --timeout=10
  Found link lo(1)
  Found link eth0(19)
  lo: link is ignored
  eth0: link is ignored
  Timeout occurred while waiting for network connectivity.

  [Where problems could occur]

  This patch partially re-instates a patch remove in bug 1982218.
  However, instead of exiting if all links are unmanaged, we exit if all
  links are ignored in manager_configured(). If the patch was wrong, we
  may re-introduce bug 1982218, so as part of this SRU verification,
  that bug should be tested too. Any other regressions would also be
  related to systemd-networkd-wait-online behavior.

  [Original Description]

  On Ubuntu 22.04 desktop system using network-manager and upgrading to
  systemd 249.11-0ubuntu3.10, wait-online now times out which prevents
  logins (GDM, ssh, console) until it does time out. This seems to be
  introduced by the change for
  https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1982218.

  https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1982218/comments/21
  also mentioned the problem on Lunar.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2036358/+subscriptions