[Bug 2051999] Re: Grub2 2.06 has upstream bug that results in Non-booting with ZFS after snapshot of bpool.

Mon Feb 5 16:53:23 UTC 2024

@mkukri --- 
>>>
The question is, are we going to backport the fix to 2.06 and push that to the updates channel, or can this wait for the 2.12 SRU sometimes after April?
>>>
What does that mean?

Option #1 -- The cherry picked Grub2 2.12 ZFS related commit code gets
applied to Grub2 2.06 as a patch?

Option #2 -- After April, when Grub2 2.12 gets published and released as
a "stable release update," that it also gets released to Jammy and
Mantic?

But you said it has the potential to break Jammy? I'm not seeing that in
my tests yet, but I believe you.

****
Here is how I see the logic of this, but it means nothing, as I am not in the decision process, but:

For option 1, that would take time anyways, and the Noble April Release
is less than 2 months. So option #2 would be about the same wait with a
whole lot less work involved by everyone.

***
Until then, I have three working work-arounds for this non-booting error: 

Work-around #1-- The above work-around, which you say may break some
things.

Work-around #2-- I do have another work-around, that involves booting from an Installer LiveUSB. Backing up the content of bpool to a created backup directory "somehwere" (size needed is less than 2GB), saving the old UUID that was used in the old dataset name to a file. I used another USB Flash drive. Destroy the old bpool pool. Create the new bpool with these explicit creation options:
>>>
zpool create \
    -o ashift=12 \
    -o autotrim=on \
    -o cachefile=/etc/zfs/zpool.cache \
    -o feature at async_destroy=enabled \
    -o feature at empty_bpobj=active \
    -o feature at lz4_compress=active \
    -o feature at multi_vdev_crash_dump=disabled \
    -o feature at spacemap_histogram=active \
    -o feature at enabled_txg=active \
    -o feature at hole_birth=active \
    -o feature at extensible_dataset=disabled \
    -o feature at embedded_data=active \
    -o feature at bookmarks=disabled \
    -o feature at filesystem_limits=disabled \
    -o feature at large_blocks=disabled \
    -o feature at large_dnode=disabled \
    -o feature at sha512=disabled \
    -o feature at skein=disabled \
    -o feature at edonr=disabled \
    -o feature at userobj_accounting=disabled \
    -o feature at encryption=disabled \
    -o feature at project_quota=disabled \
    -o feature at device_removal=disabled \
    -o feature at obsolete_counts=disabled \
    -o feature at zpool_checkpoint=disabled \
    -o feature at spacemap_v2=disabled \
    -o feature at allocation_classes=disabled \
    -o feature at resilver_defer=disabled \
    -o feature at bookmark_v2=disabled \
    -o feature at redaction_bookmarks=disabled \
    -o feature at redacted_datasets=disabled \
    -o feature at bookmark_written=disabled \
    -o feature at log_spacemap=disabled \
    -o feature at livelist=disabled \
    -o feature at device_rebuild=disabled \
    -o feature at zstd_compress=disabled \
    -o feature at draid=disabled \
    -o feature at zilsaxattr=disabled \
    -o feature at head_errlog=disabled \
    -o feature at blake3=disabled \
    -o feature at block_cloning=disabled \
    -o feature at vdev_zaps_v2=disabled \
    -o compatibility=grub2,ubuntu-22.04 \
    -O devices=off \
    -O acltype=posixacl \
    -O xattr=sa \
    -O compression=lz4 \
    -O normalization=formD \
    -O relatime=on \
    -O canmount=off -O mountpoint=/boot -R /mnt\
    bpool /dev/disk/by-id/ata-QEMU_HARDDISK_QM00003-part3
>>>

Set the UUID VAR from the old UIUID stored in the file, then recreate the bpool filesystem datasets
>>>
zfs create -o canmount=off -o mountpoint=none bpool/BOOT bpool/BOOT
zfs create -o mountpoint=/boot bpool/BOOT/ubuntu_$UUID
>>>
rsync the old bpool contents back into the new bpool... 

Export the new bpool and reboot.

Work-around #3-- Is to back up and reinstall, telling the user not to do
any snapshots of bpool until this problem is resolved.

Work-arounds #1 & #3 make no changes to the pool options. Work-around #3
has the risk of losing the user data. Work-around #2 is a whole lot of
work, and has the highest risk factor of them all... And the options
used are different than was installed... But it works with Grub2 2.06.

@Mate: What do you think on this? What should we tell Users until April?

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to grub2-unsigned in Ubuntu.
https://bugs.launchpad.net/bugs/2051999

Title:
  Grub2 2.06 has upstream bug that results in Non-booting with ZFS after
  snapshot of bpool.

Status in grub2 package in Ubuntu:
  Confirmed
Status in grub2-unsigned package in Ubuntu:
  Confirmed
Status in zfs-linux package in Ubuntu:
  New

Bug description:
  There is an upstream Bug with Grub where if you create snapshots of bpool, it results in a non-booting System. The problem was found to be an upstream Bug with Grub2:
  https://savannah.gnu.org/bugs/index.php?64297

  Multiple Ubuntu 22.04.3 Users Affected:
  https://ubuntuforums.org/showthread.php?t=2494397&highlight=zfs+grub+bug
  https://ubuntuforums.org/showthread.php?t=2494957

  Brought up as an issue at OpenZFS:
  https://github.com/openzfs/zfs/issues/13873

  If you look at this comment (https://github.com/openzfs/zfs/issues/13873#issuecomment-1892911836), if was found the Savanaugh at GNU released a fix for it in Grub2 2.12, here:
  https://git.savannah.gnu.org/cgit/grub.git/log/grub-core/fs/zfs/zfs.c

  Ubuntu Jammy 22.04.3 is Grub2 2.06. We need to backported this patch
  to Grub2 2.06 so that Users are not caught of in this bug for or
  currently supported LTS Release.

  ProblemType: Bug
  DistroRelease: Ubuntu 22.04
  Package: grub-efi-amd64 2.06-2ubuntu14.4
  ProcVersionSignature: Ubuntu 6.2.0-39.40~22.04.1-generic 6.2.16
  Uname: Linux 6.2.0-39-generic x86_64
  NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair nvidia_modeset nvidia
  ApportVersion: 2.20.11-0ubuntu82.5
  Architecture: amd64
  CasperMD5CheckResult: unknown
  CurrentDesktop: GNOME
  Date: Thu Feb  1 16:40:28 2024
  InstallationDate: Installed on 2021-09-23 (861 days ago)
  InstallationMedia: Ubuntu 20.04.3 LTS "Focal Fossa" - Release amd64 (20210819)
  SourcePackage: grub2-unsigned
  UpgradeStatus: Upgraded to jammy on 2022-08-17 (533 days ago)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/2051999/+subscriptions