[Bug 1588477] [NEW] regression-update: systemd init behavior change

[Anthony Howe]

Public bug reported:

regression-update for 1575572.

After the time 1575572 was checked in we observed a behavior change in
the systemd init.

We have been able to repro the issue and can observe that dpkg seems to
start docker on “16.04.201605310 Daily”, but not “16.04.201605280
Daily”.

The problem seems to be that in 16.04.201605310.Daily after  “dcos-
docker-install.service” runs “Job for docker.service” can be seen trying
to start docker.  On 16.04.0-LTS, it does not try to start docker while
running “dcos-docker-install.service”.  Why is the binary being started
now and not before?

Below are the Contents of the cloud-init, with the 3 docker installation
files that fail:

Contents of Cloud-Init, show relationship between Systemd files
#cloud-config

write_files:
- path: /etc/systemd/system/dcos-docker-install.service
  permissions: "0644"
  content: |
    [Unit]
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=oneshot
    Environment=DEBIAN_FRONTEND=noninteractive
    StandardOutput=journal+console
    StandardError=journal+console
    ExecStartPre=/usr/bin/curl -fLsSv --retry 20 -Y 100000 -y 60 -o /tmp/d.deb https://az837203.vo.msecnd.net/dcos-deps/docker-engine_1.11.0-0~xenial_amd64.deb
    ExecStart=/usr/bin/bash -c "try=1;until dpkg -D3 -i /tmp/d.deb || ((try > 5));do echo retry $((try++));sleep 120;done;systemctl --now start docker"
- path: /etc/systemd/system/docker.service.d/execstart.conf
  permissions: "0644"
  content: |
    [Service]
    Restart=always
    StartLimitInterval=0
    RestartqSec=15
    ExecStart=
    ExecStart=/usr/bin/docker daemon -H fd:// --storage-driver=overlay
- path: /etc/systemd/system/docker.socket
  permissions: "0644"
  content: |
    [Unit]
    PartOf=docker.service
    [Socket]
    ListenStream=/var/run/docker.sock
    SocketMode=0660
    SocketUser=root
    SocketGroup=docker
    ListenStream=2375
    BindIPv6Only=both
    [Install]
    WantedBy=sockets.target
runcmd:
- [ ln, -s, /bin/rm, /usr/bin/rm ]
- [ ln, -s, /bin/mkdir, /usr/bin/mkdir ]
- [ ln, -s, /bin/tar, /usr/bin/tar ]
- [ ln, -s, /bin/ln, /usr/bin/ln ]
- [ ln, -s, /bin/cp, /usr/bin/cp ]
- [ ln, -s, /bin/systemctl, /usr/bin/systemctl ]
- [ ln, -s, /bin/mount, /usr/bin/mount ]
- [ ln, -s, /bin/bash, /usr/bin/bash ]
- [ systemctl, stop, resolvconf.service ]
- [ systemctl, disable, resolvconf.service ]
- [ systemctl, stop, lxc-net.service ]
- [ systemctl, disable, lxc-net.service ]
- [ systemctl, mask, lxc-net.service ]
- [ systemctl, start, dcos-docker-install.service ]

** Affects: init-system-helpers (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: 1575572 regression-update

** Description changed:

  regression-update for 1575572.
  
- After 1575572 was checked in we observed.
+ After the time 1575572 was checked in we observed a behavior change in
+ the systemd init.
  
  We have been able to repro the issue and can observe that dpkg seems to
  start docker on “16.04.201605310 Daily”, but not “16.04.201605280
  Daily”.
  
  The problem seems to be that in 16.04.201605310.Daily after  “dcos-
  docker-install.service” runs “Job for docker.service” can be seen trying
  to start docker.  On 16.04.0-LTS, it does not try to start docker while
  running “dcos-docker-install.service”.  Why is the binary being started
  now and not before?
  
- 
  Below are the Contents of the cloud-init, with the 3 docker installation
  files that fail:
  
  Contents of Cloud-Init, show relationship between Systemd files
  #cloud-config
  
  write_files:
  - path: /etc/systemd/system/dcos-docker-install.service
-   permissions: "0644"
-   content: |
-     [Unit]
-     After=network-online.target
-     Wants=network-online.target
-     [Service]
-     Type=oneshot
-     Environment=DEBIAN_FRONTEND=noninteractive
-     StandardOutput=journal+console
-     StandardError=journal+console
-     ExecStartPre=/usr/bin/curl -fLsSv --retry 20 -Y 100000 -y 60 -o /tmp/d.deb https://az837203.vo.msecnd.net/dcos-deps/docker-engine_1.11.0-0~xenial_amd64.deb
-     ExecStart=/usr/bin/bash -c "try=1;until dpkg -D3 -i /tmp/d.deb || ((try > 5));do echo retry $((try++));sleep 120;done;systemctl --now start docker"
+   permissions: "0644"
+   content: |
+     [Unit]
+     After=network-online.target
+     Wants=network-online.target
+     [Service]
+     Type=oneshot
+     Environment=DEBIAN_FRONTEND=noninteractive
+     StandardOutput=journal+console
+     StandardError=journal+console
+     ExecStartPre=/usr/bin/curl -fLsSv --retry 20 -Y 100000 -y 60 -o /tmp/d.deb https://az837203.vo.msecnd.net/dcos-deps/docker-engine_1.11.0-0~xenial_amd64.deb
+     ExecStart=/usr/bin/bash -c "try=1;until dpkg -D3 -i /tmp/d.deb || ((try > 5));do echo retry $((try++));sleep 120;done;systemctl --now start docker"
  - path: /etc/systemd/system/docker.service.d/execstart.conf
-   permissions: "0644"
-   content: |
-     [Service]
-     Restart=always
-     StartLimitInterval=0
-     RestartqSec=15
-     ExecStart=
-     ExecStart=/usr/bin/docker daemon -H fd:// --storage-driver=overlay
+   permissions: "0644"
+   content: |
+     [Service]
+     Restart=always
+     StartLimitInterval=0
+     RestartqSec=15
+     ExecStart=
+     ExecStart=/usr/bin/docker daemon -H fd:// --storage-driver=overlay
  - path: /etc/systemd/system/docker.socket
-   permissions: "0644"
-   content: |
-     [Unit]
-     PartOf=docker.service
-     [Socket]
-     ListenStream=/var/run/docker.sock
-     SocketMode=0660
-     SocketUser=root
-     SocketGroup=docker
-     ListenStream=2375
-     BindIPv6Only=both
-     [Install]
-     WantedBy=sockets.target
+   permissions: "0644"
+   content: |
+     [Unit]
+     PartOf=docker.service
+     [Socket]
+     ListenStream=/var/run/docker.sock
+     SocketMode=0660
+     SocketUser=root
+     SocketGroup=docker
+     ListenStream=2375
+     BindIPv6Only=both
+     [Install]
+     WantedBy=sockets.target
  runcmd:
  - [ ln, -s, /bin/rm, /usr/bin/rm ]
  - [ ln, -s, /bin/mkdir, /usr/bin/mkdir ]
  - [ ln, -s, /bin/tar, /usr/bin/tar ]
  - [ ln, -s, /bin/ln, /usr/bin/ln ]
  - [ ln, -s, /bin/cp, /usr/bin/cp ]
  - [ ln, -s, /bin/systemctl, /usr/bin/systemctl ]
  - [ ln, -s, /bin/mount, /usr/bin/mount ]
  - [ ln, -s, /bin/bash, /usr/bin/bash ]
  - [ systemctl, stop, resolvconf.service ]
  - [ systemctl, disable, resolvconf.service ]
  - [ systemctl, stop, lxc-net.service ]
  - [ systemctl, disable, lxc-net.service ]
  - [ systemctl, mask, lxc-net.service ]
  - [ systemctl, start, dcos-docker-install.service ]

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to init-system-helpers in Ubuntu.
https://bugs.launchpad.net/bugs/1588477

Title:
  regression-update: systemd init behavior change

Status in init-system-helpers package in Ubuntu:
  New

Bug description:
  regression-update for 1575572.

  After the time 1575572 was checked in we observed a behavior change in
  the systemd init.

  We have been able to repro the issue and can observe that dpkg seems
  to start docker on “16.04.201605310 Daily”, but not “16.04.201605280
  Daily”.

  The problem seems to be that in 16.04.201605310.Daily after  “dcos-
  docker-install.service” runs “Job for docker.service” can be seen
  trying to start docker.  On 16.04.0-LTS, it does not try to start
  docker while running “dcos-docker-install.service”.  Why is the binary
  being started now and not before?

  Below are the Contents of the cloud-init, with the 3 docker
  installation files that fail:

  Contents of Cloud-Init, show relationship between Systemd files
  #cloud-config

  write_files:
  - path: /etc/systemd/system/dcos-docker-install.service
    permissions: "0644"
    content: |
      [Unit]
      After=network-online.target
      Wants=network-online.target
      [Service]
      Type=oneshot
      Environment=DEBIAN_FRONTEND=noninteractive
      StandardOutput=journal+console
      StandardError=journal+console
      ExecStartPre=/usr/bin/curl -fLsSv --retry 20 -Y 100000 -y 60 -o /tmp/d.deb https://az837203.vo.msecnd.net/dcos-deps/docker-engine_1.11.0-0~xenial_amd64.deb
      ExecStart=/usr/bin/bash -c "try=1;until dpkg -D3 -i /tmp/d.deb || ((try > 5));do echo retry $((try++));sleep 120;done;systemctl --now start docker"
  - path: /etc/systemd/system/docker.service.d/execstart.conf
    permissions: "0644"
    content: |
      [Service]
      Restart=always
      StartLimitInterval=0
      RestartqSec=15
      ExecStart=
      ExecStart=/usr/bin/docker daemon -H fd:// --storage-driver=overlay
  - path: /etc/systemd/system/docker.socket
    permissions: "0644"
    content: |
      [Unit]
      PartOf=docker.service
      [Socket]
      ListenStream=/var/run/docker.sock
      SocketMode=0660
      SocketUser=root
      SocketGroup=docker
      ListenStream=2375
      BindIPv6Only=both
      [Install]
      WantedBy=sockets.target
  runcmd:
  - [ ln, -s, /bin/rm, /usr/bin/rm ]
  - [ ln, -s, /bin/mkdir, /usr/bin/mkdir ]
  - [ ln, -s, /bin/tar, /usr/bin/tar ]
  - [ ln, -s, /bin/ln, /usr/bin/ln ]
  - [ ln, -s, /bin/cp, /usr/bin/cp ]
  - [ ln, -s, /bin/systemctl, /usr/bin/systemctl ]
  - [ ln, -s, /bin/mount, /usr/bin/mount ]
  - [ ln, -s, /bin/bash, /usr/bin/bash ]
  - [ systemctl, stop, resolvconf.service ]
  - [ systemctl, disable, resolvconf.service ]
  - [ systemctl, stop, lxc-net.service ]
  - [ systemctl, disable, lxc-net.service ]
  - [ systemctl, mask, lxc-net.service ]
  - [ systemctl, start, dcos-docker-install.service ]

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/init-system-helpers/+bug/1588477/+subscriptions