[Bug 1983588] [NEW] update-motd-fsck-at-reboot stalls ssh logins for minutes if there is a slow/busy USB disk attached

[Peter Maydell]

Public bug reported:

I found recently that some ssh logins to my machine could stall for
minutes, and tracked this down to the update-motd-fsck-at-reboot hook.

Here's the relevant bit of 'ps wafux' output:

root     3659433  0.0  0.0  56920 14836 ?        Ss   14:29   0:00  \_ sshd: petmay01 [priv]
root     3659435  0.0  0.0   2608   592 ?        S    14:29   0:00      \_ sh -c /usr/bin/env -i PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin run-parts --lsbsysinit /etc/update-motd.d > /run/motd.dynamic.new
root     3659437  0.0  0.0   2496  1400 ?        S    14:29   0:00          \_ run-parts --lsbsysinit /etc/update-motd.d
root     3659488  0.0  0.0   2608  1824 ?        S    14:29   0:00              \_ /bin/sh /usr/lib/update-notifier/update-motd-fsck-at-reboot
root     3659535  0.0  0.0   3480   896 ?        D    14:29   0:00                  \_ dumpe2fs -h /dev/mapper/backup2

It's currently 14:40 and the 'dumpe2fs -h' has been stalled in the D
state for 10 minutes; the ssh login is blocked by this. (In fact, as I
was writing up this bug report the dumpe2fs finished at 14:42 and the
ssh login completed successfully.)

The 'backup2' disk, as the name suggests, is a backup disk, and it's an
external USB disk. The backups are currently running, so the disk is
currently busy doing a lot of I/O, which is why the dumpe2fs command
blocks for ages.

I think that anything like this that gets run during logins should not
be performing operations which could potentially take a long time to
complete. I don't know if there's some mechanism for identifying "fast
disks" and only looking at those.

The workaround is to add an "exit 0" to "/etc/update-motd.d/98-fsck-at-
reboot", disabling the hook.

This is with Ubuntu 20.04, and update-notifier-common version
3.192.30.11.

** Affects: update-notifier (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to update-notifier in Ubuntu.
https://bugs.launchpad.net/bugs/1983588

Title:
  update-motd-fsck-at-reboot stalls ssh logins for minutes if there is a
  slow/busy USB disk attached

Status in update-notifier package in Ubuntu:
  New

Bug description:
  I found recently that some ssh logins to my machine could stall for
  minutes, and tracked this down to the update-motd-fsck-at-reboot hook.

  Here's the relevant bit of 'ps wafux' output:

  root     3659433  0.0  0.0  56920 14836 ?        Ss   14:29   0:00  \_ sshd: petmay01 [priv]
  root     3659435  0.0  0.0   2608   592 ?        S    14:29   0:00      \_ sh -c /usr/bin/env -i PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin run-parts --lsbsysinit /etc/update-motd.d > /run/motd.dynamic.new
  root     3659437  0.0  0.0   2496  1400 ?        S    14:29   0:00          \_ run-parts --lsbsysinit /etc/update-motd.d
  root     3659488  0.0  0.0   2608  1824 ?        S    14:29   0:00              \_ /bin/sh /usr/lib/update-notifier/update-motd-fsck-at-reboot
  root     3659535  0.0  0.0   3480   896 ?        D    14:29   0:00                  \_ dumpe2fs -h /dev/mapper/backup2

  It's currently 14:40 and the 'dumpe2fs -h' has been stalled in the D
  state for 10 minutes; the ssh login is blocked by this. (In fact, as I
  was writing up this bug report the dumpe2fs finished at 14:42 and the
  ssh login completed successfully.)

  The 'backup2' disk, as the name suggests, is a backup disk, and it's
  an external USB disk. The backups are currently running, so the disk
  is currently busy doing a lot of I/O, which is why the dumpe2fs
  command blocks for ages.

  I think that anything like this that gets run during logins should not
  be performing operations which could potentially take a long time to
  complete. I don't know if there's some mechanism for identifying "fast
  disks" and only looking at those.

  The workaround is to add an "exit 0" to "/etc/update-motd.d/98-fsck-
  at-reboot", disabling the hook.

  This is with Ubuntu 20.04, and update-notifier-common version
  3.192.30.11.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/1983588/+subscriptions