[Bug 2081738] Re: [SRU] Guided disk reformat for disk with fs signature fails with "Partition ... is not in list"

Germaine Windley 2081738 at bugs.launchpad.net
Fri Jan 10 20:28:47 UTC 2025 

I’m sorry I know I might have started this bug report but I believe I just
reinstalled it and it happened again a few times but eventually it
installed

On Fri, Jan 10, 2025 at 6:00 AM Olivier Gayot <2081738 at bugs.launchpad.net>
wrote:

> Subscribing ubuntu-sru.
>
> SRU team, what do you think about having us (the Subiquity team)
> subscribe you to the bugs we want to get fixed in stable LTS releases?
> We're trying to meet the expectations from the new policy regarding
> third-party software sources.
>
> Obviously there is no archive-related action involved because subiquity
> is a snap. Testing will also be done differently.
>
> Any feedback is appreciated.
>
> Thanks,
> Olivier
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (2090802).
> https://bugs.launchpad.net/bugs/2081738
>
> Title:
>   [SRU] Guided disk reformat for disk with fs signature fails with
>   "Partition ... is not in list"
>
> Status in subiquity:
>   Fix Committed
> Status in ubuntu-desktop-provision:
>   New
> Status in subiquity package in Ubuntu:
>   Confirmed
> Status in subiquity source package in Noble:
>   Confirmed
>
> Bug description:
>   [ Impact ]
>
>    * When a disk has partitions + a filesystem signature (e.g., it used to
> be a ZFS member and then ended up partitioned), using it in a guided
> reformat scenario results in a failed installation. Subiquity logs show
> "Partition xxx is not in list".
>    * Having Subiquity advertise guided scenarios that do not work gives a
> bad impression on users.
>
>   [ Test Plan ]
>
>    * Boot the 24.04.1 desktop ISO (can be a VM).
>    * If testing the fix, refresh the `ubuntu-desktop-bootstrap` snap to a
> patched version.
>    * Open a terminal and add a FS signature + partitions to a disk (here
> /dev/vda)
>       # dd if=/dev/zero of=/root/test.img count=100 bs=1M
>       # zpool create testpool /root/test.img
>       # zpool export testpool
>       # dd if=/root/test.img of=/dev/vda
>       # fdisk /dev/vda --wipe=never
>        * create a GPT partition table using "g"
>        * create partitions using "n" and follow instructions
>        * write to the disk using "w"
>    * Go through the desktop installer screens normally until "Disk setup"
>    * Select "Erase disk and install Ubuntu"
>    * Go through the next screens. At some point the installer will show a
> "Something went wrong" screen.
>    * Check for the presence of the "is not in list" error:
>    * $ sudo grep -F "is not in list"
> /var/log/installer/subiquity-server-debug.log
>
>
>   [ Where problems could occur ]
>
>    * Currently, the bug makes the installation fail quite early ; before
>   the installer starts wiping the disk; so there should be no risk of
>   data loss for users running into it.
>
>   * We think that the risk of a regression caused by the patch is low.
>   Fixing this bug will lower the number of failed installations and the
>   number of bug reports we get, which is great. However, it also means
>   that more installations will go past the disk wipe operation, so we
>   might end up with a slightly higher count of bug reports for other
>   unrelated issues (which are typically more severe if they occur past
>   the disk operation point).
>
>   [ Other Info ]
>
>    * The fix to backport is
>   https://github.com/canonical/subiquity/pull/2092
>
>   [ Original description ]
>
>   We have many private reports where the user selects a
>   GuidedTargetReformat ; which fails with "Partition ... is not in
>   list".
>
>   Example:
>
>   2024-04-26 19:39:43,220 DEBUG
> subiquity.server.controllers.filesystem:1182
> GuidedChoiceV2(target=GuidedStorageTargetReformat(disk_id='disk-sda',
> allowed=[<GuidedCapability.DIRECT: 2>, <GuidedCapability.LVM: 3>,
> <GuidedCapability.LVM_LUKS: 4>, <GuidedCapability.ZFS: 5>,
> <GuidedCapability.ZFS_LUKS_KEYSTORE: 6>],
> disallowed=[GuidedDisallowedCapability(capability=<GuidedCapability.CORE_BOOT_ENCRYPTED:
> 7>,
> reason=<GuidedDisallowedCapabilityReason.CORE_BOOT_ENCRYPTION_UNAVAILABLE:
> 2>, message='not encrypting device storage as checking TPM gave: the TPM is
> in DA lockout mode'),
> GuidedDisallowedCapability(capability=<GuidedCapability.CORE_BOOT_UNENCRYPTED:
> 8>, reason=<GuidedDisallowedCapabilityReason.THIRD_PARTY_DRIVERS: 4>,
> message='Enhanced secure boot options cannot currently install third party
> drivers.')]), capability=<GuidedCapability.DIRECT: 2>, recovery_key=None,
> sizing_policy=<SizingPolicy.ALL: 2>, reset_partition=False,
> reset_partition_size=None)
>   2024-04-26 19:39:43,221 DEBUG root:38 finish:
> subiquity/Filesystem/v2_guided_POST: SUCCESS: 500 Traceback (most recent
> call last):
>     File "/snap/ubuntu-desktop-bootstrap/171...
>   2024-04-26 19:39:43,221 DEBUG subiquity.server.server:541 request to
> /storage/v2/guided crashed
>   Traceback (most recent call last):
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/api/server.py",
> line 164, in handler
>       result = await implementation(**args)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py",
> line 1184, in v2_guided_POST
>       await self.guided(data)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py",
> line 726, in guided
>       gap = self.start_guided(choice.target, disk)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/usr/lib/python3.10/functools.py", line
> 926, in _method
>       return method.__get__(obj, cls)(*args, **kwargs)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py",
> line 645, in start_guided_reformat
>       self.reformat(disk, wipe="superblock-recursive")
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/manipulator.py",
> line 257, in reformat
>       self.delete_partition(p, True)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/manipulator.py",
> line 120, in delete_partition
>       self.model.remove_partition(part)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/models/filesystem.py",
> line 2110, in remove_partition
>       for p2 in movable_trailing_partitions_and_gap_size(part)[0]:
>     File
> "/snap/ubuntu-desktop-bootstrap/171/usr/lib/python3.10/functools.py", line
> 889, in wrapper
>       return dispatch(args[0].__class__)(*args, **kw)
>     File
> "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/gaps.py",
> line 281, in _movable_trailing_partitions_and_gap_size_partition
>       part_idx = pgs.index(partition)
>   ValueError: Partition(device=disk-sda, size=6109091840,
> wipe='superblock', number=1, preserve=True, offset=32768,
> partition_type='xxxxxxxx', path='/dev/sda1', uuid='xxxxxxxx',
> id='partition-sda1', type='partition') is not in list
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/subiquity/+bug/2081738/+subscriptions
>
>

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to subiquity in Ubuntu.
https://bugs.launchpad.net/bugs/2081738

Title:
  [SRU] Guided disk reformat for disk with fs signature fails with
  "Partition ... is not in list"

Status in subiquity:
  Fix Committed
Status in ubuntu-desktop-provision:
  New
Status in subiquity package in Ubuntu:
  Confirmed
Status in subiquity source package in Noble:
  Confirmed

Bug description:
  [ Impact ]

   * When a disk has partitions + a filesystem signature (e.g., it used to be a ZFS member and then ended up partitioned), using it in a guided reformat scenario results in a failed installation. Subiquity logs show "Partition xxx is not in list".
   * Having Subiquity advertise guided scenarios that do not work gives a bad impression on users.

  [ Test Plan ]

   * Boot the 24.04.1 desktop ISO (can be a VM).
   * If testing the fix, refresh the `ubuntu-desktop-bootstrap` snap to a patched version.
   * Open a terminal and add a FS signature + partitions to a disk (here /dev/vda)
      # dd if=/dev/zero of=/root/test.img count=100 bs=1M
      # zpool create testpool /root/test.img
      # zpool export testpool
      # dd if=/root/test.img of=/dev/vda
      # fdisk /dev/vda --wipe=never
       * create a GPT partition table using "g"
       * create partitions using "n" and follow instructions
       * write to the disk using "w"
   * Go through the desktop installer screens normally until "Disk setup"
   * Select "Erase disk and install Ubuntu"
   * Go through the next screens. At some point the installer will show a "Something went wrong" screen.
   * Check for the presence of the "is not in list" error:
   * $ sudo grep -F "is not in list" /var/log/installer/subiquity-server-debug.log

  
  [ Where problems could occur ]

   * Currently, the bug makes the installation fail quite early ; before
  the installer starts wiping the disk; so there should be no risk of
  data loss for users running into it.

  * We think that the risk of a regression caused by the patch is low.
  Fixing this bug will lower the number of failed installations and the
  number of bug reports we get, which is great. However, it also means
  that more installations will go past the disk wipe operation, so we
  might end up with a slightly higher count of bug reports for other
  unrelated issues (which are typically more severe if they occur past
  the disk operation point).

  [ Other Info ]

   * The fix to backport is
  https://github.com/canonical/subiquity/pull/2092

  [ Original description ]

  We have many private reports where the user selects a
  GuidedTargetReformat ; which fails with "Partition ... is not in
  list".

  Example:

  2024-04-26 19:39:43,220 DEBUG subiquity.server.controllers.filesystem:1182 GuidedChoiceV2(target=GuidedStorageTargetReformat(disk_id='disk-sda', allowed=[<GuidedCapability.DIRECT: 2>, <GuidedCapability.LVM: 3>, <GuidedCapability.LVM_LUKS: 4>, <GuidedCapability.ZFS: 5>, <GuidedCapability.ZFS_LUKS_KEYSTORE: 6>], disallowed=[GuidedDisallowedCapability(capability=<GuidedCapability.CORE_BOOT_ENCRYPTED: 7>, reason=<GuidedDisallowedCapabilityReason.CORE_BOOT_ENCRYPTION_UNAVAILABLE: 2>, message='not encrypting device storage as checking TPM gave: the TPM is in DA lockout mode'), GuidedDisallowedCapability(capability=<GuidedCapability.CORE_BOOT_UNENCRYPTED: 8>, reason=<GuidedDisallowedCapabilityReason.THIRD_PARTY_DRIVERS: 4>, message='Enhanced secure boot options cannot currently install third party drivers.')]), capability=<GuidedCapability.DIRECT: 2>, recovery_key=None, sizing_policy=<SizingPolicy.ALL: 2>, reset_partition=False, reset_partition_size=None)
  2024-04-26 19:39:43,221 DEBUG root:38 finish: subiquity/Filesystem/v2_guided_POST: SUCCESS: 500 Traceback (most recent call last):
    File "/snap/ubuntu-desktop-bootstrap/171...
  2024-04-26 19:39:43,221 DEBUG subiquity.server.server:541 request to /storage/v2/guided crashed
  Traceback (most recent call last):
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/api/server.py", line 164, in handler
      result = await implementation(**args)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py", line 1184, in v2_guided_POST
      await self.guided(data)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py", line 726, in guided
      gap = self.start_guided(choice.target, disk)
    File "/snap/ubuntu-desktop-bootstrap/171/usr/lib/python3.10/functools.py", line 926, in _method
      return method.__get__(obj, cls)(*args, **kwargs)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/server/controllers/filesystem.py", line 645, in start_guided_reformat
      self.reformat(disk, wipe="superblock-recursive")
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/manipulator.py", line 257, in reformat
      self.delete_partition(p, True)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/manipulator.py", line 120, in delete_partition
      self.model.remove_partition(part)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/models/filesystem.py", line 2110, in remove_partition
      for p2 in movable_trailing_partitions_and_gap_size(part)[0]:
    File "/snap/ubuntu-desktop-bootstrap/171/usr/lib/python3.10/functools.py", line 889, in wrapper
      return dispatch(args[0].__class__)(*args, **kw)
    File "/snap/ubuntu-desktop-bootstrap/171/bin/subiquity/subiquity/common/filesystem/gaps.py", line 281, in _movable_trailing_partitions_and_gap_size_partition
      part_idx = pgs.index(partition)
  ValueError: Partition(device=disk-sda, size=6109091840, wipe='superblock', number=1, preserve=True, offset=32768, partition_type='xxxxxxxx', path='/dev/sda1', uuid='xxxxxxxx', id='partition-sda1', type='partition') is not in list

To manage notifications about this bug go to:
https://bugs.launchpad.net/subiquity/+bug/2081738/+subscriptions

More information about the foundations-bugs mailing list