[Bug 1852571] [NEW] degraded raid must assemble for boot, workaround included

Thu Nov 14 12:08:28 UTC 2019

Public bug reported:

The first time after pulling one of two disks in my RAID 1,
ubuntu studio 18.04 drops to the Busybox prompt.

I can manually type vgchange -a y and exit and the boot succeeds.
Thereafter, subsequent boots finish successfully.

mdadm 4.1~rc1-3-ubuntu18.04.2
lvm2  2.02.176-4.1ubuntu3.18.04.1

I traced this issue and saw in initramfs-tools/scripts/local-block/mdadm
the code that tries mdadm --run /dev/md?* seems to be what is needed
to "assemble" the single-disk arrays.  I have md1 thru md4, /boot on /md1.

The thing is, I read in http://manpages.ubuntu.com/manpages/xenial/man8
/initramfs-tools.8.html

              local-top  OR  nfs-top After these scripts have been executed, the root device node
              is expected to be present (local) or the network interface is expected to be usable
              (NFS).

              local-block  These scripts are called with the name of a local block device.  After
              these scripts have been executed, that device  node  should  be  present.   If  the
              local-top  or local-block scripts fail to create the wanted device node, the local-
              block scripts will be called periodically to try again.

So the degraded arrays need to be assembled before the lvm2 script in local-top, which does have
an mdadm prereq listed.  Still in my distro, the local-top directory is free of any mdadm script.
I also had a hint that the volume groups would need to be brought online also, by reading the last comment here: https://serverfault.com/questions/688207/how-to-auto-start-degraded-software-raid1-under-debian-8-0-0-on-boot .

Bottom line: I fixed my setup by adding scripts/local-top/mdadm and running update-initramfs -u.
It may be that the mdadm --run should remain and the vgchange -a y move to local-top/lvm2,
so maybe both packages should be adjusted.  For now I am just filing this report to mdadm figuring you will decide if a report into lvm is also appropriate. 

Gratitude. Here is my workaround fix:

#!/bin/sh
#		/usr/share/initramfs-tools/scripts/local-top/mdadm
#
# code--1299-f680-1193d4d7-dc94-9bc93e882a4d--13.0.6.17.18-t13--6.94.71--5.03--gn4q1rUW2gMkV25LiROpfo
#
# This script enables boot immediately after removing a RAID drive.
#
# The very first time a boot cycle happens after removing a drive,
# the standard mdadm --assemble needs some help to spin up the array.
#
# gratitude https://alioth-lists-archive.debian.net/pipermail/pkg-mdadm-commits/2013-August/002401.html
# gratitude https://serverfault.com/questions/688207/how-to-auto-start-degraded-software-raid1-under-debian-8-0-0-on-boot
# gratitude /usr/share/initramfs-tools/scripts/local-block/mdadm
# gratitude from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830770
# Also:
# gratitude https://arstechnica.com/civis/viewtopic.php?f=16&t=1271945
# gratitude https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/573477
# gratitude https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814036
# gratitude http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html
# gratitude https://wiki.debian.org/InitramfsDebug#Saving_debug_information

PREREQS="udev"

prereqs() { echo "$PREREQS"; }

case "$1" in
    prereqs) prereqs; exit 0 ;;
esac

. /scripts/functions

# cat /proc/mdstat >>/dev/local-top.mdadm.out || true   # use /dev for
persistent log from initramfs scripts

mdadm --run /dev/md?* || true   # spin up the array

# cat /proc/mdstat >>/dev/local-top.mdadm.out || true   # use /dev for
persistent log from initramfs scripts

vgchange -a y || true           # activate the lvm volume groups

exit 0

** Affects: mdadm (Ubuntu)
     Importance: Undecided
         Status: New

** Package changed: ubiquity (Ubuntu) => mdadm (Ubuntu)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubiquity in Ubuntu.
https://bugs.launchpad.net/bugs/1852571

Title:
  degraded raid must assemble for boot, workaround included

Status in mdadm package in Ubuntu:
  New

Bug description:
  The first time after pulling one of two disks in my RAID 1,
  ubuntu studio 18.04 drops to the Busybox prompt.

  I can manually type vgchange -a y and exit and the boot succeeds.
  Thereafter, subsequent boots finish successfully.

  mdadm 4.1~rc1-3-ubuntu18.04.2
  lvm2  2.02.176-4.1ubuntu3.18.04.1

  I traced this issue and saw in initramfs-tools/scripts/local-block/mdadm
  the code that tries mdadm --run /dev/md?* seems to be what is needed
  to "assemble" the single-disk arrays.  I have md1 thru md4, /boot on /md1.

  The thing is, I read in
  http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html

                local-top  OR  nfs-top After these scripts have been executed, the root device node
                is expected to be present (local) or the network interface is expected to be usable
                (NFS).

                local-block  These scripts are called with the name of a local block device.  After
                these scripts have been executed, that device  node  should  be  present.   If  the
                local-top  or local-block scripts fail to create the wanted device node, the local-
                block scripts will be called periodically to try again.

  So the degraded arrays need to be assembled before the lvm2 script in local-top, which does have
  an mdadm prereq listed.  Still in my distro, the local-top directory is free of any mdadm script.
  I also had a hint that the volume groups would need to be brought online also, by reading the last comment here: https://serverfault.com/questions/688207/how-to-auto-start-degraded-software-raid1-under-debian-8-0-0-on-boot .

  Bottom line: I fixed my setup by adding scripts/local-top/mdadm and running update-initramfs -u.
  It may be that the mdadm --run should remain and the vgchange -a y move to local-top/lvm2,
  so maybe both packages should be adjusted.  For now I am just filing this report to mdadm figuring you will decide if a report into lvm is also appropriate. 

  Gratitude. Here is my workaround fix:

  #!/bin/sh
  #		/usr/share/initramfs-tools/scripts/local-top/mdadm
  #
  # code--1299-f680-1193d4d7-dc94-9bc93e882a4d--13.0.6.17.18-t13--6.94.71--5.03--gn4q1rUW2gMkV25LiROpfo
  #
  # This script enables boot immediately after removing a RAID drive.
  #
  # The very first time a boot cycle happens after removing a drive,
  # the standard mdadm --assemble needs some help to spin up the array.
  #
  # gratitude https://alioth-lists-archive.debian.net/pipermail/pkg-mdadm-commits/2013-August/002401.html
  # gratitude https://serverfault.com/questions/688207/how-to-auto-start-degraded-software-raid1-under-debian-8-0-0-on-boot
  # gratitude /usr/share/initramfs-tools/scripts/local-block/mdadm
  # gratitude from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830770
  # Also:
  # gratitude https://arstechnica.com/civis/viewtopic.php?f=16&t=1271945
  # gratitude https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/573477
  # gratitude https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814036
  # gratitude http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html
  # gratitude https://wiki.debian.org/InitramfsDebug#Saving_debug_information

  PREREQS="udev"

  prereqs() { echo "$PREREQS"; }

  case "$1" in
      prereqs) prereqs; exit 0 ;;
  esac

  . /scripts/functions

  # cat /proc/mdstat >>/dev/local-top.mdadm.out || true   # use /dev for
  persistent log from initramfs scripts

  mdadm --run /dev/md?* || true   # spin up the array

  # cat /proc/mdstat >>/dev/local-top.mdadm.out || true   # use /dev for
  persistent log from initramfs scripts

  vgchange -a y || true           # activate the lvm volume groups

  exit 0

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1852571/+subscriptions