[Bug 2051299] Re: nvme-cli: fguid is printed as binary data and causes MAAS to fail erasing NVME disks

[Timo Aaltonen]

Hello maasuser1, or anyone else affected,

Accepted nvme-cli into jammy-proposed. The package will build now and be
available at https://launchpad.net/ubuntu/+source/nvme-
cli/1.16-3ubuntu0.2 in a few hours, and then in the -proposed
repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
jammy to verification-done-jammy. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-jammy. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: nvme-cli (Ubuntu Jammy)
       Status: In Progress => Fix Committed

** Tags added: verification-needed verification-needed-jammy

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to nvme-cli in Ubuntu.
https://bugs.launchpad.net/bugs/2051299

Title:
  nvme-cli: fguid is printed as binary data and causes MAAS to fail
  erasing NVME disks

Status in MAAS:
  Triaged
Status in nvme-cli package in Ubuntu:
  Fix Released
Status in nvme-cli source package in Jammy:
  Fix Committed

Bug description:
  [Impact]

  When a user tries to release a system deployed with MAAS, that has
  erase disks on release set, erasing NVME disks fails on Jammy.

  Traceback (most recent call last):
  File "/tmp/user_data.sh.jNE4lC/bin/maas-wipe", line 542, in <module>
  main()
  File "/tmp/user_data.sh.jNE4lC/bin/maas-wipe", line 522, in main
  disk_info = get_disk_info()
  File "/tmp/user_data.sh.jNE4lC/bin/maas-wipe", line 165, in get_disk_info
  return {kname: get_disk_security_info(kname) for kname in list_disks()}
  File "/tmp/user_data.sh.jNE4lC/bin/maas-wipe", line 165, in <dictcomp>
  return {kname: get_disk_security_info(kname) for kname in list_disks()}
  File "/tmp/user_data.sh.jNE4lC/bin/maas-wipe", line 158, in get_disk_security_info
  return get_nvme_security_info(disk)
  File "/tmp/user_data.sh.jNE4lC/bin/maas-wipe", line 64, in get_nvme_security_info
  output = output.decode()
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 385: invalid start byte

  This is due to maas_wipe.py running "nvme id-ctrl <device>" and
  parsing the results. This should be human readable data, in string
  format, so utf-8 should be appropriate for MAAS to use.

  Instead, the "fguid" field is being printed as binary data, and is not
  parsable as utf-8.

  e.g. From comment #8.

  The user sees:

  `fguid : 2.`

  on closer inspection, the hex is:

  x32,0x89,0x82,0x2E

  Note it is cut off early, likely because the next byte would be 0x00,
  and is being interprested as a null byte.

  Fix nvme-cli such that we print out the fguid as a correct utf-8
  string, so MAAS works as intended.

  [Testcase]

  Deploy Jammy onto a system that has a NVME device.

  $ sudo apt install nvme-cli

  Run the 'id-ctrl' command and look at the fguid entry:

  $ sudo nvme id-ctrl /dev/nvme1n1 | grep fguid
  fguid     :

  Due to the UUID being all zeros, this was interpreted as a null byte,
  and the UUID was not printed correctly.

  There is a test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf387274-test

  If you install the test package, the fguid will be printed as a proper
  string:

  $ sudo nvme id-ctrl /dev/nvme1n1 | grep fguid
  fguid     : 00000000-0000-0000-0000-000000000000

  Also check that json output works as expected:

  $ sudo nvme id-ctrl -o json /dev/nvme1n1 | grep fguid
    "fguid" : "00000000-0000-0000-0000-000000000000",

  Additionally, also test that the new package allows a MAAS deployed system to
  be released correctly with the erase option enabled, as maas_wipe.py should now
  complete successfully.

  [Where problems could occur]

  We are changing the output of the 'id-ctrl' subcommand. No other
  subcommands are changed. Users who for some reason rely on broken,
  incomplete binary data that is printed might be impacted. For users
  doing a hard diff of the command output, the output will now change to
  reflect the actual fguid, and might need a change. The fguid is now
  supplied in json output for 'id-ctrl', and might change programs
  parsing the json object.

  There are no workarounds, and if a regression were to occur, it would
  only affect the 'id-ctrl' subcommand, and not change anything else.

  [Other info]

  Upstream bug:
  https://github.com/linux-nvme/nvme-cli/issues/1653

  This was fixed in the below commit in version 2.2, found in mantic and
  later:

  commit 78b7ad235507ddd59c75c7fcc74fc6c927811f87
  From: Pierre Labat <plabat at micron.com>
  Date: Fri, 26 Aug 2022 17:02:08 -0500
  Subject: nvme-print: Print fguid as a UUID
  Link: https://github.com/linux-nvme/nvme-cli/commit/78b7ad235507ddd59c75c7fcc74fc6c927811f87

  The commit required a minor backport. In later versions, a major
  refactor occurred that changed nvme_uuid_to_string() among numerous
  other functions, that is not appropriate to backport. Instead, just
  take the current implementation of nvme_uuid_to_string() and move it
  like the patch suggests, so json output works correctly.

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/2051299/+subscriptions