[Bug 2081738] Re: [SRU] Guided disk reformat for disk with fs signature fails with "Partition ... is not in list"

[Olivier Gayot]

** Changed in: subiquity
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to subiquity in Ubuntu.
https://bugs.launchpad.net/bugs/2081738

Title:
  [SRU] Guided disk reformat for disk with fs signature fails with
  "Partition ... is not in list"

Status in subiquity:
  Fix Released
Status in ubuntu-desktop-provision:
  New
Status in subiquity package in Ubuntu:
  Confirmed
Status in subiquity source package in Noble:
  Confirmed

Bug description:
  [ Impact ]

   * When a disk has partitions + a filesystem signature (e.g., it used to be a ZFS member and then ended up partitioned), using it in a guided reformat scenario results in a failed installation. Subiquity logs show "Partition xxx is not in list".
   * Having Subiquity advertise guided scenarios that do not work gives a bad impression on users.

  [ Test Plan ]

   * Boot the 24.04.1 desktop ISO (can be a VM).
   * If testing the fix, refresh the `ubuntu-desktop-bootstrap` snap to a patched version.
   * Open a terminal and add a FS signature + partitions to a disk (here /dev/vda)
      # dd if=/dev/zero of=/root/test.img count=100 bs=1M
      # zpool create testpool /root/test.img
      # zpool export testpool
      # dd if=/root/test.img of=/dev/vda
      # fdisk /dev/vda --wipe=never
       * create a GPT partition table using "g"
       * create partitions using "n" and follow instructions
       * write to the disk using "w"
   * Go through the desktop installer screens normally until "Disk setup"
   * Select "Erase disk and install Ubuntu"
   * Go through the next screens. At some point the installer will show a "Something went wrong" screen.
   * Check for the presence of the "is not in list" error:
   * $ sudo grep -F "is not in list" /var/log/installer/subiquity-server-debug.log

  
  [ Where problems could occur ]

   * Currently, the bug makes the installation fail quite early ; before
  the installer starts wiping the disk; so there should be no risk of
  data loss for users running into it.

  * We think that the risk of a regression caused by the patch is low.
  Fixing this bug will lower the number of failed installations and the
  number of bug reports we get, which is great. However, it also means
  that more installations will go past the disk wipe operation, so we
  might end up with a slightly higher count of bug reports for other
  unrelated issues (which are typically more severe if they occur past
  the disk operation point).

  [ Other Info ]

   * The fix to backport is
  https://github.com/canonical/subiquity/pull/2092

  [ Original description ]

  We have many private reports where the user selects a
  GuidedTargetReformat ; which fails with "Partition ... is not in
  list".

  Example:

  2024-04-26 19:39:43,220 DEBUG subiquity.server.controllers.filesystem:1182 GuidedChoiceV2(target=GuidedStorageTargetReformat(disk_id='disk-sda', allowed=[<GuidedCapability.DIRECT: 2>, <GuidedCapability.LVM: 3>, <GuidedCapability.LVM_LUKS: 4>, <GuidedCapability.ZFS: 5>, <GuidedCapability.ZFS_LUKS_KEYSTORE: 6>], disallowed=[GuidedDisallowedCapability(capability=<GuidedCapability.CORE_BOOT_ENCRYPTED: 7>, reason=<GuidedDisallowedCapabilityReason.CORE_BOOT_ENCRYPTION_UNAVAILABLE: 2>, message='not encrypting device storage as checking TPM gave: the TPM is in DA lockout mode'), GuidedDisallowedCapability(capability=<GuidedCapability.CORE_BOOT_UNENCRYPTED: 8>, reason=<GuidedDisallowedCapabilityReason.THIRD_PARTY_DRIVERS: 4>, message='Enhanced secure boot options cannot currently install third party drivers.')]), capability=<GuidedCapability.DIRECT: 2>, recovery_key=None, sizing_policy=<SizingPolicy.ALL: 2>, reset_partition=False, reset_partition_size=None)
  2024-04-26 19:39:43,221 DEBUG root:38 finish: subiquity/Filesystem/v2_guided_POST: SUCCESS: 500 Traceback (most recent call last):
    File "/snap/ubuntu-desktop-bootstrap/171...
  2024-04-26 19:39:43,221 DEBUG subiquity.server.server:541 request to /storage/v2/guided crashed
  Traceback (most recent call last):
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/api/server.py", line 164, in handler
      result = await implementation(**args)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py", line 1184, in v2_guided_POST
      await self.guided(data)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py", line 726, in guided
      gap = self.start_guided(choice.target, disk)
    File "/snap/ubuntu-desktop-bootstrap/171/usr/lib/python3.10/functools.py", line 926, in _method
      return method.__get__(obj, cls)(*args, **kwargs)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py", line 645, in start_guided_reformat
      self.reformat(disk, wipe="superblock-recursive")
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/manipulator.py", line 257, in reformat
      self.delete_partition(p, True)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/manipulator.py", line 120, in delete_partition
      self.model.remove_partition(part)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/models/filesystem.py", line 2110, in remove_partition
      for p2 in movable_trailing_partitions_and_gap_size(part)[0]:
    File "/snap/ubuntu-desktop-bootstrap/171/usr/lib/python3.10/functools.py", line 889, in wrapper
      return dispatch(args[0].__class__)(*args, **kw)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/gaps.py", line 281, in _movable_trailing_partitions_and_gap_size_partition
      part_idx = pgs.index(partition)
  ValueError: Partition(device=disk-sda, size=6109091840, wipe='superblock', number=1, preserve=True, offset=32768, partition_type='xxxxxxxx', path='/dev/sda1', uuid='xxxxxxxx', id='partition-sda1', type='partition') is not in list

To manage notifications about this bug go to:
https://bugs.launchpad.net/subiquity/+bug/2081738/+subscriptions