[Bug 1649931] Re: systemd-networkd needs to ensure DNS is up before network-online.target

Thu Apr 20 07:48:45 UTC 2017

Ok, I've succeeded in verifying this by adding an artificial
ExecStartPre=/bin/sleep 10 to both old and new resolvconf.service.  Old:

# journalctl -o short-precise --unit resolvconf.service --unit network-online.target --unit systemd-networkd-wait-online.service --unit systemd-networkd.service  --no-pager
-- Logs begin at Thu 2017-04-20 07:36:15 UTC, end at Thu 2017-04-20 07:36:37 UTC. --
Apr 20 07:36:17.265818 ubuntu systemd[1]: Starting Network Service...
Apr 20 07:36:17.430595 ubuntu systemd-networkd[905]: Enumeration completed
Apr 20 07:36:17.437376 ubuntu systemd[1]: Started Network Service.
Apr 20 07:36:17.440060 ubuntu systemd[1]: Starting Wait for Network to be Configured...
Apr 20 07:36:17.460259 ubuntu systemd-networkd[905]: ens3: IPv6 enabled for interface: Success
Apr 20 07:36:17.473412 ubuntu systemd-networkd[905]: ens3: Gained carrier
Apr 20 07:36:17.499888 ubuntu systemd-networkd[905]: ens3: DHCPv4 address 192.168.122.138/24 via 192.168.122.1
Apr 20 07:36:18.934247 ubuntu systemd-networkd[905]: ens3: Gained IPv6LL
Apr 20 07:36:25.978756 ubuntu systemd[1]: Started Nameserver information manager.
Apr 20 07:36:31.522527 ubuntu systemd-networkd[905]: ens3: Configured
Apr 20 07:36:31.531655 ubuntu systemd[1]: Started Wait for Network to be Configured.
Apr 20 07:36:31.533566 ubuntu systemd[1]: Reached target Network is Online.

New:
# journalctl -o short-precise --unit resolvconf.service --unit network-online.target --unit systemd-networkd-wait-online.service --unit systemd-networkd.service  --no-pager
-- Logs begin at Thu 2017-04-20 07:38:31 UTC, end at Thu 2017-04-20 07:39:01 UTC. --
Apr 20 07:38:41.872259 ubuntu systemd[1]: Started Nameserver information manager.
Apr 20 07:38:41.890403 ubuntu systemd[1]: Starting Network Service...
Apr 20 07:38:41.930968 ubuntu systemd-networkd[987]: Enumeration completed
Apr 20 07:38:41.931769 ubuntu systemd[1]: Started Network Service.
Apr 20 07:38:41.934063 ubuntu systemd[1]: Starting Wait for Network to be Configured...
Apr 20 07:38:41.944754 ubuntu systemd-networkd[987]: ens3: IPv6 enabled for interface: Success
Apr 20 07:38:41.950236 ubuntu systemd-networkd[987]: ens3: Gained carrier
Apr 20 07:38:41.957193 ubuntu systemd-networkd[987]: ens3: DHCPv4 address 192.168.122.138/24 via 192.168.122.1
Apr 20 07:38:43.444868 ubuntu systemd-networkd[987]: ens3: Gained IPv6LL
Apr 20 07:38:56.057060 ubuntu systemd-networkd[987]: ens3: Configured
Apr 20 07:38:56.060665 ubuntu systemd-networkd-wait-online[1010]: ignoring: lo
Apr 20 07:38:56.069850 ubuntu systemd[1]: Started Wait for Network to be Configured.
Apr 20 07:38:56.076092 ubuntu systemd[1]: Reached target Network is Online.

In neither case did I end up with an unconfigured /etc/resolv.conf , but
this does at least demonstrate the capacity for the race (and its
resolution).  Marking verified.

** Tags added: verification-done-yakkety

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to resolvconf in Ubuntu.
https://bugs.launchpad.net/bugs/1649931

Title:
  systemd-networkd needs to ensure DNS is up before network-
  online.target

Status in resolvconf package in Ubuntu:
  Fix Released
Status in systemd package in Ubuntu:
  Fix Released
Status in resolvconf source package in Xenial:
  Fix Released
Status in systemd source package in Xenial:
  Fix Committed
Status in resolvconf source package in Yakkety:
  Fix Committed
Status in resolvconf package in Debian:
  Fix Committed

Bug description:
  Currently resolvconf and systemd-networkd don't ensure DNS has been
  configured before allowing network-online.target to be reached.

  This was discussed in https://launchpad.net/bugs/1636912 however it
  was not a regression since there aren't any users of networkd + DNS
  early in boot at this time, it was requested that we move this DNS
  issue to a separate bug.

  [SRU]
  Fix: switch resolvconf.service to run Before=network-pre.target and add Wants=network-pre.target.  Add a Before=network-online.target to systemd-networkd-resolvconf-update.service to ensure we update /etc/resolv.conf with DNS config prior to reaching network-online.target.

  Regression potential: Low. networkd is not widely being used outside
  of netplan/snappy in xenial.

  Test Case:
    lxc launch ubuntu-daily:xenial x1
    lxc exec x1 /bin/bash

    # make sure you're on systemd-229-4ubuntu17
    apt update && apt install -y systemd

    # enable networkd and netplan
    apt install -y nplan
  cat <<EOF > /etc/netplan/nplan.yaml
  network:
      version: 2
      ethernets:
          all-en:
              match:
                  name: "en*"
              dhcp4: true
          all-eth:
              match:
                  name: "eth*"
              dhcp4: true
  EOF
    sed -i.orig -e 's/^source/# source/' /etc/network/interfaces

    netplan generate

    # make sure cloud-init.service uses networkd
    sed -i.orig -e '/After=networking.service/a After=systemd-networkd-wait-online.service' /lib/systemd/system/cloud-init.service

    reboot

    # check that the order of execution with:
    journalctl -o short-precise --unit resolvconf.service --unit network-online.target --unit systemd-networkd-wait-online.service --unit systemd-networkd-resolvconf-update.service

    # the order should be:
      1. resolvconf:  systemd[1]: Started Nameserver information manager.
      2. systemd-networkd-wait-online.service:  systemd[1]: Starting Wait for Network to be Configured...
      3. systemd-networkd-resolvconf-update.service: systemd[1]: Started Update resolvconf for networkd DNS.
      4. network-online.target: systemd[1]: Reached target Network is Online.

  === BAD OUTPUT ===
  On a failing system, Reached target Network is Online occurs before (1, 2, or 3) above, like this output:

  Dec 15 19:18:15.233443 x4 systemd[1]: Started Nameserver information manager.
  Dec 15 19:18:15.797857 x4 systemd[1]: Starting Wait for Network to be Configured...
  Dec 15 19:18:15.799573 x4 systemd-networkd-wait-online[145]: ignoring: lo
  Dec 15 19:18:15.804949 x4 systemd-networkd-wait-online[145]: ignoring: lo
  Dec 15 19:18:15.805079 x4 systemd-networkd-wait-online[145]: ignoring: lo
  Dec 15 19:18:29.100305 x4 systemd[1]: Starting Update resolvconf for networkd DNS...
  Dec 15 19:18:29.101870 x4 systemd[1]: Started Wait for Network to be Configured.
  Dec 15 19:18:29.102144 x4 systemd[1]: Reached target Network is Online.
  Dec 15 19:18:29.212842 x4 systemd[1]: Started Update resolvconf for networkd DNS.

  === GOOD OUTPUT ===
  On a passing system, Reached target Network is Online occurs after 1, 2, and 3.

  Dec 15 19:28:42.548545 x4 systemd[1]: Started Nameserver information manager.
  Dec 15 19:28:43.144389 x4 systemd[1]: Starting Wait for Network to be Configured...
  Dec 15 19:28:43.146155 x4 systemd-networkd-wait-online[145]: ignoring: lo
  Dec 15 19:28:56.081487 x4 systemd[1]: Started Wait for Network to be Configured.
  Dec 15 19:28:56.100353 x4 systemd[1]: Starting Update resolvconf for networkd DNS...
  Dec 15 19:28:56.124005 x4 systemd[1]: Started Update resolvconf for networkd DNS.
  Dec 15 19:28:56.124555 x4 systemd[1]: Reached target Network is Online.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/resolvconf/+bug/1649931/+subscriptions