[Bug 2054390] Re: Refine proc mounts entries traversal

[Chengen Du]

** Description changed:

  [Impact]
  The shell's read builtin iterates through /proc/mounts one line at a time. This becomes problematic when LDAP automount maps generate a large number of entries in /proc/mounts. It can lead to timeout issues, especially when iterating through the entries twice in the cryptroot hook.
  
  [Fix]
  Applying the following upstream commit [1] can resolve this issue.
  
  95fd4be9b4c6 d/functions: get_mnt_devno(): Speed up execution time on large /proc/mounts.
      Use awk rather than a `while read; do done` loop here as the /proc/mounts
      pseudo-file can be many thousands lines long and the shell's `read` builtin
      traverses it one read(2) at the time which cruelly slows down execution time.
  
      See https://salsa.debian.org/cryptsetup-
  team/cryptsetup/-/merge_requests/36 .
  
- [1] https://salsa.debian.org/cryptsetup-
- team/cryptsetup/-/commit/95fd4be9b4c6471e94c418101e7acfae7e1aa4fc
- 
  [Test Plan]
  1. Prepare a VM with its root partition encrypted using LUKS.
  2. Save the content of /etc/crypttab for reference.
  3. Install the patched package and execute the binary located at /usr/share/initramfs-tools/hooks/cryptroot.
  4. Verify that the /etc/crypttab content remains unchanged and ensure the output does not include the following warning message:
  cryptsetup: WARNING: Couldn't determine root device
  
+ For ZFS, which does not have a major/minor device number, the hook function will skip it.
+ To reproduce this scenario, it is necessary to configure the root partition on ZFS.
+ You can refer to the documentation [2] for the setup process.
+ Alternatively, we can prepare a simple script to manually trigger this scenario for testing purposes.
+ ===
+ #!/bin/sh
+ 
+ . /lib/cryptsetup/functions
+ 
+ if devnos="$(get_mnt_devno $1)"; then
+ 	echo "devnos: ${devnos}"
+ else
+ 	echo "WARNING: Couldn't determine device"
+ fi
+ ===
+ 
+ The logic can be easily verified by following these steps:
+ # mount | grep zfs
+ /zfs on /zfs type zfs (rw,xattr,noacl)
+ zfs/dataset on /mnt type zfs (rw,xattr,noacl)
+ # ./test /mnt
+ devnos:
+ # echo $?
+ 0
+ 
+ The devnos should be empty, and no errors should occur.
+ 
  The performance improvement test is outlined as follows:
  root at jammy-ptp:~# mkdir src dst
  root at jammy-ptp:~# for i in {1..5000}; do touch src/test_${i} dst/test_${i}; mount --bind src/test_${i} dst/test_${i}; done
- root at jammy-ptp:~# wc -l /proc/mounts 
+ root at jammy-ptp:~# wc -l /proc/mounts
  5028 /proc/mounts
  
  [Before]
  root at jammy-ptp:~# time /usr/share/initramfs-tools/hooks/cryptroot
  real	0m1.415s
  user	0m0.975s
  sys	0m0.529s
  
  [After]
  root at jammy-ptp:~/cryptsetup# time /usr/share/initramfs-tools/hooks/cryptroot
  real	0m0.129s
  user	0m0.098s
  sys	0m0.037s
  
  [Where problems could occur]
  The patch exclusively modifies the method of extracting information without altering the underlying hook logic.
  It's crucial to note that the successful generation of the crypttab is contingent upon the accuracy of the information provided by the patch.
  Any inaccuracies may impede the crypttab generation process.
  
  [Other Info]
  The proposed change [1] is already applied in Oracular and Plucky.
+ 
+ [1] https://salsa.debian.org/cryptsetup-team/cryptsetup/-/commit/95fd4be9b4c6471e94c418101e7acfae7e1aa4fc
+ [2] https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2022.04%20Root%20on%20ZFS.html

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to cryptsetup in Ubuntu.
https://bugs.launchpad.net/bugs/2054390

Title:
  Refine proc mounts entries traversal

Status in cryptsetup package in Ubuntu:
  Fix Released
Status in cryptsetup source package in Focal:
  Incomplete
Status in cryptsetup source package in Jammy:
  In Progress
Status in cryptsetup source package in Mantic:
  Won't Fix
Status in cryptsetup source package in Noble:
  In Progress
Status in cryptsetup source package in Oracular:
  Fix Released
Status in cryptsetup source package in Plucky:
  Fix Released

Bug description:
  [Impact]
  The shell's read builtin iterates through /proc/mounts one line at a time. This becomes problematic when LDAP automount maps generate a large number of entries in /proc/mounts. It can lead to timeout issues, especially when iterating through the entries twice in the cryptroot hook.

  [Fix]
  Applying the following upstream commit [1] can resolve this issue.

  95fd4be9b4c6 d/functions: get_mnt_devno(): Speed up execution time on large /proc/mounts.
      Use awk rather than a `while read; do done` loop here as the /proc/mounts
      pseudo-file can be many thousands lines long and the shell's `read` builtin
      traverses it one read(2) at the time which cruelly slows down execution time.

      See https://salsa.debian.org/cryptsetup-
  team/cryptsetup/-/merge_requests/36 .

  [Test Plan]
  1. Prepare a VM with its root partition encrypted using LUKS.
  2. Save the content of /etc/crypttab for reference.
  3. Install the patched package and execute the binary located at /usr/share/initramfs-tools/hooks/cryptroot.
  4. Verify that the /etc/crypttab content remains unchanged and ensure the output does not include the following warning message:
  cryptsetup: WARNING: Couldn't determine root device

  For ZFS, which does not have a major/minor device number, the hook function will skip it.
  To reproduce this scenario, it is necessary to configure the root partition on ZFS.
  You can refer to the documentation [2] for the setup process.
  Alternatively, we can prepare a simple script to manually trigger this scenario for testing purposes.
  ===
  #!/bin/sh

  . /lib/cryptsetup/functions

  if devnos="$(get_mnt_devno $1)"; then
  	echo "devnos: ${devnos}"
  else
  	echo "WARNING: Couldn't determine device"
  fi
  ===

  The logic can be easily verified by following these steps:
  # mount | grep zfs
  /zfs on /zfs type zfs (rw,xattr,noacl)
  zfs/dataset on /mnt type zfs (rw,xattr,noacl)
  # ./test /mnt
  devnos:
  # echo $?
  0

  The devnos should be empty, and no errors should occur.

  The performance improvement test is outlined as follows:
  root at jammy-ptp:~# mkdir src dst
  root at jammy-ptp:~# for i in {1..5000}; do touch src/test_${i} dst/test_${i}; mount --bind src/test_${i} dst/test_${i}; done
  root at jammy-ptp:~# wc -l /proc/mounts
  5028 /proc/mounts

  [Before]
  root at jammy-ptp:~# time /usr/share/initramfs-tools/hooks/cryptroot
  real	0m1.415s
  user	0m0.975s
  sys	0m0.529s

  [After]
  root at jammy-ptp:~/cryptsetup# time /usr/share/initramfs-tools/hooks/cryptroot
  real	0m0.129s
  user	0m0.098s
  sys	0m0.037s

  [Where problems could occur]
  The patch exclusively modifies the method of extracting information without altering the underlying hook logic.
  It's crucial to note that the successful generation of the crypttab is contingent upon the accuracy of the information provided by the patch.
  Any inaccuracies may impede the crypttab generation process.

  [Other Info]
  The proposed change [1] is already applied in Oracular and Plucky.

  [1] https://salsa.debian.org/cryptsetup-team/cryptsetup/-/commit/95fd4be9b4c6471e94c418101e7acfae7e1aa4fc
  [2] https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2022.04%20Root%20on%20ZFS.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/2054390/+subscriptions