[Bug 1891680] Re: grub-pc needs to detect when debconf points to invalid drive and stop in preinst, before unpacking files, and also treat this as a failure in postinst
Dimitri John Ledkov
1891680 at bugs.launchpad.net
Tue Aug 18 17:47:21 UTC 2020
** Description changed:
[Impact]
* grub-pc currently installs new core to MBR and installs new modules
to /boot in an unsafe manner, which may lead to incompatible combination
of MBR and modules resulting in failure to boot.
[Test Case]
* Install using old point media, of an old release. I.e. 16.04.(p-1)
for testing upgrades to 18.04 sru, in bios mode.
* backup the contents of /boot
-
- * Configure invalid grub-pc/install_devices to a non existing device
-
- * Upgrade to the package from next series-proposed, non-interactively
-
- * Observe the package installation has failed, the grub-pc package is
- in a broken state.
-
- * Compare the backup of /boot with current /boot, it should have
- remained the same, and is different to modules in /usr/lib/grub/i386-pc
-
- * Reboot, reboot should be successful
* install /etc/apparmor.d/usr.sbin.grub-install profile
"/usr/sbin/grub-install" {
capability,
mount,
ptrace,
signal,
unix,
file,
deny /dev/vda w,
}
and load it with
sudo apparmor_parser -r usr.sbin.grub-install
- * Set grub-pc/install_devices to the correct existing device
- * Attempt non-interactive configuration of the grub-pc package
+ * Upgrade to the package from next series-proposed, non-interactively
* Observe the package installation has failed, the grub-pc package is
in a broken state.
* Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in /usr/lib/grub/i386-pc
* Remove the apparmor profile /etc/apparmor.d/usr.sbin.grub-install
- * Reboot, reboot should be successful
+ * Reboot, reboot should be successful. If possible observe the version
+ number in the grub menu, it should still be old.
- * Set grub-pc/install_devices to the non-existing device
+ * Configure invalid grub-pc/install_devices to a non existing device
+ (e.g. /dev/sdk)
+
+ * Attempt non-interactive configuration of the grub-pc package
+
+ * Observe the package fails, and the grub-pc package remains in a
+ broken state.
+
+ * Compare the backup of /boot with current /boot, it should have
+ remained the same, and is different to modules in /usr/lib/grub/i386-pc
+
+ * Reboot, reboot should be successful. If possible observe the version
+ number in the grub menu, it should still be old.
* Try to configure all the packages, interactively (i.e. using $ sudo
dpkg --configure -a or by using $ sudo apt install -f) and ensure to
select the right drive for grub installation offer
* Observe that now /boot matches /usr/lib/grub/i386-pc contents, and is
different from the backup taken at the start.
+ * Reboot should be successful, and grub menu should have the new
+ version number finally
- [Regression Potential]
+ [Regression Potential]
- * discussion of how regressions are most likely to manifest as a result
- of this change.
+ * Existing call to grub-install, is now split into two. And when any
+ devices fail to configure, non-interactively error is reported just
+ like it was already done with the interactive case.
- * It is assumed that any SRU candidate patch is well-tested before
- upload and has a low overall risk of regression, but it's important
- to make the effort to think about what ''could'' happen in the
- event of a regression.
+ It means, it will fail configuration of the package, where
+ previously it would report success. However, it is now safer and
+ keeps the system bootable, whilst having unconfigured
+ packages. This mostly affects non-interactive upgrades, as the
+ interactive ones have always shown critical errors trying to
+ correct grub-pc installation problems.
- * This both shows the SRU team that the risks have been considered,
- and provides guidance to testers in regression-testing the SRU.
+ The first stage of grub-install only tries to update the MBR,
+ whilst utilizing tmpdirectory to create the core image. This is a
+ slight increase in disk space usage, as previously core was created
+ in-pace in /boot. Then whilst tmpdir is still populated, /boot
+ modules and core are upgraded.
+
+ These changes do not address multi-mbr systems, or cases where
+ updating modules fails. For example, it is possible that MBR update
+ is successful, yet writting updated modules fails, in such scenario
+ MBR is not rolled back to previous one. Or a case where MBR updates
+ have succeeded, but only on some devices. A choice has been made to
+ update modules in /boot, if at least one device has a successful
+ MBR update. No backup, or rollback of MBR is performed if module
+ updates fail. This is tricky to do, as it is uncertain if current
+ MBR matches the core.img & boot.img from /boot, or if some other
+ bootsectors code was in use before. Ideally in the future,
+ grub-install itself will be able to stage module updates, and
+ commit/rollback them upon successful MBR update.
[Other Info]
- * Anything else you think is useful to include
- * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
- * and address these questions in advance
+ * Original bug report description
-
-
- Currently on upgrade if the debconf variable for the drive to install grub-pc to point to a non-existent drive, the grub package will nevertheless happily carry on and the postinst will exit 0 - as a result leaving the /boot/grub contents and the MBR in an inconsistent state, which due to recent ABI changes will leave the system unbootable on reboot.
+ Currently on upgrade if the debconf variable for the drive to install
+ grub-pc to point to a non-existent drive, the grub package will
+ nevertheless happily carry on and the postinst will exit 0 - as a result
+ leaving the /boot/grub contents and the MBR in an inconsistent state,
+ which due to recent ABI changes will leave the system unbootable on
+ reboot.
Three changes required in order to make grub upgrades more resilient:
- exit non-zero from the postinst when the drive targets are invalid, so that we signal to the user that there is a problem BEFORE they reboot and give them the opportunity to deal with it. This is addressed by https://code.launchpad.net/~xnox/grub/+git/grub/+merge/388383
- include a check for target drive validity in the grub preinst, not just in the postinst, so that we avoid unpacking boot assets onto disk that might be incorrectly used by another package (despite grub-pc being in an unconfigured state) and still render the system unbootable; this will in general break release upgrades for affected users, but a failing postinst would do the same anyway, and failing early should leave the package manager in a more consistent state overall. This is addressed by https://code.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/+merge/388423
- modify grub-install so that it handles the flaky part of the install - updating the BIOS disks - FIRST, and aborts if this fails; instead of the current behavior, which is that /boot/grub is updated on disk first, then it attempts to install to the BIOS disk, and if this part fails, no rollback of the contents of /boot/grub is possible.
** Description changed:
[Impact]
- * grub-pc currently installs new core to MBR and installs new modules
+ * grub-pc currently installs new core to MBR and installs new modules
to /boot in an unsafe manner, which may lead to incompatible combination
of MBR and modules resulting in failure to boot.
[Test Case]
- * Install using old point media, of an old release. I.e. 16.04.(p-1)
+ * Install using old point media, of an old release. I.e. 16.04.(p-1)
for testing upgrades to 18.04 sru, in bios mode.
- * backup the contents of /boot
+ * backup the contents of /boot
- * install /etc/apparmor.d/usr.sbin.grub-install profile
+ * install /etc/apparmor.d/usr.sbin.grub-install profile
"/usr/sbin/grub-install" {
- capability,
- mount,
- ptrace,
- signal,
- unix,
- file,
- deny /dev/vda w,
+ capability,
+ mount,
+ ptrace,
+ signal,
+ unix,
+ file,
+ deny /dev/* w,
}
- and load it with
+ and load it with
- sudo apparmor_parser -r usr.sbin.grub-install
+ sudo apparmor_parser -r usr.sbin.grub-install
+ * Upgrade to the package from next series-proposed, non-interactively
- * Upgrade to the package from next series-proposed, non-interactively
-
- * Observe the package installation has failed, the grub-pc package is
+ * Observe the package installation has failed, the grub-pc package is
in a broken state.
- * Compare the backup of /boot with current /boot, it should have
+ * Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in /usr/lib/grub/i386-pc
- * Remove the apparmor profile /etc/apparmor.d/usr.sbin.grub-install
+ * Remove the apparmor profile /etc/apparmor.d/usr.sbin.grub-install
- * Reboot, reboot should be successful. If possible observe the version
+ * Reboot, reboot should be successful. If possible observe the version
number in the grub menu, it should still be old.
- * Configure invalid grub-pc/install_devices to a non existing device
+ * Configure invalid grub-pc/install_devices to a non existing device
(e.g. /dev/sdk)
- * Attempt non-interactive configuration of the grub-pc package
+ * Attempt non-interactive configuration of the grub-pc package
- * Observe the package fails, and the grub-pc package remains in a
+ * Observe the package fails, and the grub-pc package remains in a
broken state.
- * Compare the backup of /boot with current /boot, it should have
+ * Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in /usr/lib/grub/i386-pc
- * Reboot, reboot should be successful. If possible observe the version
+ * Reboot, reboot should be successful. If possible observe the version
number in the grub menu, it should still be old.
- * Try to configure all the packages, interactively (i.e. using $ sudo
+ * Try to configure all the packages, interactively (i.e. using $ sudo
dpkg --configure -a or by using $ sudo apt install -f) and ensure to
select the right drive for grub installation offer
- * Observe that now /boot matches /usr/lib/grub/i386-pc contents, and is
+ * Observe that now /boot matches /usr/lib/grub/i386-pc contents, and is
different from the backup taken at the start.
- * Reboot should be successful, and grub menu should have the new
+ * Reboot should be successful, and grub menu should have the new
version number finally
[Regression Potential]
- * Existing call to grub-install, is now split into two. And when any
- devices fail to configure, non-interactively error is reported just
- like it was already done with the interactive case.
+ * Existing call to grub-install, is now split into two. And when any
+ devices fail to configure, non-interactively error is reported just
+ like it was already done with the interactive case.
- It means, it will fail configuration of the package, where
- previously it would report success. However, it is now safer and
- keeps the system bootable, whilst having unconfigured
- packages. This mostly affects non-interactive upgrades, as the
- interactive ones have always shown critical errors trying to
- correct grub-pc installation problems.
+ It means, it will fail configuration of the package, where
+ previously it would report success. However, it is now safer and
+ keeps the system bootable, whilst having unconfigured
+ packages. This mostly affects non-interactive upgrades, as the
+ interactive ones have always shown critical errors trying to
+ correct grub-pc installation problems.
- The first stage of grub-install only tries to update the MBR,
- whilst utilizing tmpdirectory to create the core image. This is a
- slight increase in disk space usage, as previously core was created
- in-pace in /boot. Then whilst tmpdir is still populated, /boot
- modules and core are upgraded.
+ The first stage of grub-install only tries to update the MBR,
+ whilst utilizing tmpdirectory to create the core image. This is a
+ slight increase in disk space usage, as previously core was created
+ in-pace in /boot. Then whilst tmpdir is still populated, /boot
+ modules and core are upgraded.
- These changes do not address multi-mbr systems, or cases where
- updating modules fails. For example, it is possible that MBR update
- is successful, yet writting updated modules fails, in such scenario
- MBR is not rolled back to previous one. Or a case where MBR updates
- have succeeded, but only on some devices. A choice has been made to
- update modules in /boot, if at least one device has a successful
- MBR update. No backup, or rollback of MBR is performed if module
- updates fail. This is tricky to do, as it is uncertain if current
- MBR matches the core.img & boot.img from /boot, or if some other
- bootsectors code was in use before. Ideally in the future,
- grub-install itself will be able to stage module updates, and
- commit/rollback them upon successful MBR update.
+ These changes do not address multi-mbr systems, or cases where
+ updating modules fails. For example, it is possible that MBR update
+ is successful, yet writting updated modules fails, in such scenario
+ MBR is not rolled back to previous one. Or a case where MBR updates
+ have succeeded, but only on some devices. A choice has been made to
+ update modules in /boot, if at least one device has a successful
+ MBR update. No backup, or rollback of MBR is performed if module
+ updates fail. This is tricky to do, as it is uncertain if current
+ MBR matches the core.img & boot.img from /boot, or if some other
+ bootsectors code was in use before. Ideally in the future,
+ grub-install itself will be able to stage module updates, and
+ commit/rollback them upon successful MBR update.
[Other Info]
-
- * Original bug report description
+
+ * Original bug report description
Currently on upgrade if the debconf variable for the drive to install
grub-pc to point to a non-existent drive, the grub package will
nevertheless happily carry on and the postinst will exit 0 - as a result
leaving the /boot/grub contents and the MBR in an inconsistent state,
which due to recent ABI changes will leave the system unbootable on
reboot.
Three changes required in order to make grub upgrades more resilient:
- exit non-zero from the postinst when the drive targets are invalid, so that we signal to the user that there is a problem BEFORE they reboot and give them the opportunity to deal with it. This is addressed by https://code.launchpad.net/~xnox/grub/+git/grub/+merge/388383
- include a check for target drive validity in the grub preinst, not just in the postinst, so that we avoid unpacking boot assets onto disk that might be incorrectly used by another package (despite grub-pc being in an unconfigured state) and still render the system unbootable; this will in general break release upgrades for affected users, but a failing postinst would do the same anyway, and failing early should leave the package manager in a more consistent state overall. This is addressed by https://code.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/+merge/388423
- modify grub-install so that it handles the flaky part of the install - updating the BIOS disks - FIRST, and aborts if this fails; instead of the current behavior, which is that /boot/grub is updated on disk first, then it attempts to install to the BIOS disk, and if this part fails, no rollback of the contents of /boot/grub is possible.
** Description changed:
[Impact]
* grub-pc currently installs new core to MBR and installs new modules
to /boot in an unsafe manner, which may lead to incompatible combination
of MBR and modules resulting in failure to boot.
[Test Case]
* Install using old point media, of an old release. I.e. 16.04.(p-1)
for testing upgrades to 18.04 sru, in bios mode.
* backup the contents of /boot
* install /etc/apparmor.d/usr.sbin.grub-install profile
"/usr/sbin/grub-install" {
capability,
mount,
ptrace,
signal,
unix,
file,
deny /dev/* w,
}
and load it with
sudo apparmor_parser -r usr.sbin.grub-install
* Upgrade to the package from next series-proposed, non-interactively
* Observe the package installation has failed, the grub-pc package is
in a broken state.
* Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in /usr/lib/grub/i386-pc
* Remove the apparmor profile /etc/apparmor.d/usr.sbin.grub-install
* Reboot, reboot should be successful. If possible observe the version
number in the grub menu, it should still be old.
* Configure invalid grub-pc/install_devices to a non existing device
(e.g. /dev/sdk)
* Attempt non-interactive configuration of the grub-pc package
* Observe the package fails, and the grub-pc package remains in a
broken state.
* Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in /usr/lib/grub/i386-pc
* Reboot, reboot should be successful. If possible observe the version
number in the grub menu, it should still be old.
* Try to configure all the packages, interactively (i.e. using $ sudo
dpkg --configure -a or by using $ sudo apt install -f) and ensure to
select the right drive for grub installation offer
* Observe that now /boot matches /usr/lib/grub/i386-pc contents, and is
different from the backup taken at the start.
* Reboot should be successful, and grub menu should have the new
version number finally
[Regression Potential]
* Existing call to grub-install, is now split into two. And when any
devices fail to configure, non-interactively error is reported just
like it was already done with the interactive case.
It means, it will fail configuration of the package, where
previously it would report success. However, it is now safer and
keeps the system bootable, whilst having unconfigured
packages. This mostly affects non-interactive upgrades, as the
interactive ones have always shown critical errors trying to
correct grub-pc installation problems.
The first stage of grub-install only tries to update the MBR,
whilst utilizing tmpdirectory to create the core image. This is a
slight increase in disk space usage, as previously core was created
in-pace in /boot. Then whilst tmpdir is still populated, /boot
modules and core are upgraded.
These changes do not address multi-mbr systems, or cases where
updating modules fails. For example, it is possible that MBR update
- is successful, yet writting updated modules fails, in such scenario
- MBR is not rolled back to previous one. Or a case where MBR updates
- have succeeded, but only on some devices. A choice has been made to
- update modules in /boot, if at least one device has a successful
- MBR update. No backup, or rollback of MBR is performed if module
- updates fail. This is tricky to do, as it is uncertain if current
- MBR matches the core.img & boot.img from /boot, or if some other
- bootsectors code was in use before. Ideally in the future,
- grub-install itself will be able to stage module updates, and
- commit/rollback them upon successful MBR update.
+ is successful, yet writting updated modules fails (out of disk space),
+ in such scenario MBR is not rolled back to previous one. Or a case
+ where MBR updates have succeeded, but only on some devices.
+ A choice has been made to update modules in /boot, if at least one
+ device has a successful MBR update. No backup, or rollback of MBR is
+ performed if module updates fail. This is tricky to do, as it is
+ uncertain if current MBR matches the core.img & boot.img from /boot, or
+ if some other bootsectors code was in use before. Ideally in the
+ future, grub-install itself will be able to stage module updates, and
+ commit/rollback them upon successful MBR update.
[Other Info]
* Original bug report description
Currently on upgrade if the debconf variable for the drive to install
grub-pc to point to a non-existent drive, the grub package will
nevertheless happily carry on and the postinst will exit 0 - as a result
leaving the /boot/grub contents and the MBR in an inconsistent state,
which due to recent ABI changes will leave the system unbootable on
reboot.
Three changes required in order to make grub upgrades more resilient:
- exit non-zero from the postinst when the drive targets are invalid, so that we signal to the user that there is a problem BEFORE they reboot and give them the opportunity to deal with it. This is addressed by https://code.launchpad.net/~xnox/grub/+git/grub/+merge/388383
- include a check for target drive validity in the grub preinst, not just in the postinst, so that we avoid unpacking boot assets onto disk that might be incorrectly used by another package (despite grub-pc being in an unconfigured state) and still render the system unbootable; this will in general break release upgrades for affected users, but a failing postinst would do the same anyway, and failing early should leave the package manager in a more consistent state overall. This is addressed by https://code.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/+merge/388423
- modify grub-install so that it handles the flaky part of the install - updating the BIOS disks - FIRST, and aborts if this fails; instead of the current behavior, which is that /boot/grub is updated on disk first, then it attempts to install to the BIOS disk, and if this part fails, no rollback of the contents of /boot/grub is possible.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to grub2 in Ubuntu.
https://bugs.launchpad.net/bugs/1891680
Title:
grub-pc needs to detect when debconf points to invalid drive and stop
in preinst, before unpacking files, and also treat this as a failure
in postinst
Status in grub2 package in Ubuntu:
In Progress
Status in grub2 source package in Xenial:
Confirmed
Status in grub2 source package in Bionic:
Confirmed
Status in grub2 source package in Focal:
Confirmed
Status in grub2 source package in Groovy:
In Progress
Bug description:
[Impact]
* grub-pc currently installs new core to MBR and installs new modules
to /boot in an unsafe manner, which may lead to incompatible
combination of MBR and modules resulting in failure to boot.
[Test Case]
* Install using old point media, of an old release. I.e. 16.04.(p-1)
for testing upgrades to 18.04 sru, in bios mode.
* backup the contents of /boot
* First we will test a case where target boot device exists, yet
writes to it are denied, thus one can update modules, but cannot
update the MBR.
* install /etc/apparmor.d/usr.sbin.grub-install profile
"/usr/sbin/grub-install" {
capability,
mount,
ptrace,
signal,
unix,
file,
deny /dev/* w,
}
and load it with
sudo apparmor_parser -r usr.sbin.grub-install
* Upgrade to the package from next series-proposed, non-interactively
* Observe the package installation has failed, the grub-pc package is
in a broken state.
* Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in
/usr/lib/grub/i386-pc
* Remove the apparmor profile /etc/apparmor.d/usr.sbin.grub-install
* Reboot, reboot should be successful. If possible observe the
version number in the grub menu, it should still be old.
* Now we will test a case where a non-existing device ended up being
configured in debconf. For example, due to old buggy cloud-init having
been used during first boot, or because the VM got migrated from one
hardware configuration to another (i.e. offline switch from SCSI sda,
to VIRTIO vda).
* Configure invalid grub-pc/install_devices to a non existing device
(e.g. /dev/sdk)
* Attempt non-interactive configuration of the grub-pc package
* Observe the package fails, and the grub-pc package remains in a
broken state.
* Compare the backup of /boot with current /boot, it should have
remained the same, and is different to modules in
/usr/lib/grub/i386-pc
* Reboot, reboot should be successful. If possible observe the
version number in the grub menu, it should still be old.
* Try to configure all the packages, interactively (i.e. using $ sudo
dpkg --configure -a or by using $ sudo apt install -f) and ensure to
select the right drive for grub installation offer
* Observe that now /boot matches /usr/lib/grub/i386-pc contents, and
is different from the backup taken at the start.
* Reboot should be successful, and grub menu should have the new
version number finally
[Regression Potential]
* Existing call to grub-install, is now split into two. And when any
devices fail to configure, non-interactively error is reported just
like it was already done with the interactive case.
It means, it will fail configuration of the package, where
previously it would report success. However, it is now safer and
keeps the system bootable, whilst having unconfigured
packages. This mostly affects non-interactive upgrades, as the
interactive ones have always shown critical errors trying to
correct grub-pc installation problems.
The first stage of grub-install only tries to update the MBR,
whilst utilizing tmpdirectory to create the core image. This is a
slight increase in disk space usage, as previously core was created
in-pace in /boot. Then whilst tmpdir is still populated, /boot
modules and core are upgraded.
These changes do not address multi-mbr systems, or cases where
updating modules fails. For example, it is possible that MBR update
is successful, yet writting updated modules fails (out of disk space),
in such scenario MBR is not rolled back to previous one. Or a case
where MBR updates have succeeded, but only on some devices.
A choice has been made to update modules in /boot, if at least one
device has a successful MBR update. No backup, or rollback of MBR is
performed if module updates fail. This is tricky to do, as it is
uncertain if current MBR matches the core.img & boot.img from /boot, or
if some other bootsectors code was in use before. Ideally in the
future, grub-install itself will be able to stage module updates, and
commit/rollback them upon successful MBR update.
[Other Info]
* Original bug report description
Currently on upgrade if the debconf variable for the drive to install
grub-pc to point to a non-existent drive, the grub package will
nevertheless happily carry on and the postinst will exit 0 - as a
result leaving the /boot/grub contents and the MBR in an inconsistent
state, which due to recent ABI changes will leave the system
unbootable on reboot.
Three changes required in order to make grub upgrades more resilient:
- exit non-zero from the postinst when the drive targets are invalid, so that we signal to the user that there is a problem BEFORE they reboot and give them the opportunity to deal with it. This is addressed by https://code.launchpad.net/~xnox/grub/+git/grub/+merge/388383
- include a check for target drive validity in the grub preinst, not just in the postinst, so that we avoid unpacking boot assets onto disk that might be incorrectly used by another package (despite grub-pc being in an unconfigured state) and still render the system unbootable; this will in general break release upgrades for affected users, but a failing postinst would do the same anyway, and failing early should leave the package manager in a more consistent state overall. This is addressed by https://code.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/+merge/388423
- modify grub-install so that it handles the flaky part of the install - updating the BIOS disks - FIRST, and aborts if this fails; instead of the current behavior, which is that /boot/grub is updated on disk first, then it attempts to install to the BIOS disk, and if this part fails, no rollback of the contents of /boot/grub is possible.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1891680/+subscriptions
More information about the foundations-bugs
mailing list