[Bug 2059976] Re: unmkinitramfs is very slow on initrd from ubuntu 23.10+, solution proposed

[Benjamin Drung]

My mid-term plan is to add cpio creation support to 3cpio
(https://github.com/bdrung/3cpio/issues/2). Then change initramfs-tools
to use 3cpio instead of cpio. Then the cpio dependency can be replaced
by 3cpio.

** Bug watch added: github.com/bdrung/3cpio/issues #2
   https://github.com/bdrung/3cpio/issues/2

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to initramfs-tools in Ubuntu.
https://bugs.launchpad.net/bugs/2059976

Title:
  unmkinitramfs is very slow on initrd from ubuntu 23.10+, solution
  proposed

Status in initramfs-tools package in Ubuntu:
  Triaged

Bug description:
  Since 23.10 the initrd consists of 4 embedded cpio:
  - amd microcode
  - intel microcode
  - firmware + kernel modules
  - rest (compressed)

  Previously it was:
  - amd microcode
  - intel microcode
  - rest (compressed)

  There is no archive size header for cpio, so unmkinitramfs has to
  parse headers of each entry in uncompressed archives.

  > time unmkinitramfs 23.04/{initrd,unpack}

  real    0m3.625s
  user    0m1.595s
  sys     0m3.467s

  > time unmkinitramfs 23.10/{initrd,unpack}

  real    0m15.982s
  user    0m20.174s
  sys     0m6.995s

  > for dir in 23.04/unpack/early*; do printf '%-20s %s\n' "$dir" "$(find "$dir" | wc -l)"; done
  23.04/unpack/early   5
  23.04/unpack/early2  6

  > for dir in 23.10/unpack/early*; do printf '%-20s %s\n' "$dir" "$(find "$dir" | wc -l)"; done
  23.10/unpack/early   5
  23.10/unpack/early2  6
  23.10/unpack/early3  2874

  
  Here https://github.com/slowpeek/unmkinitramfs-turbo I've made it a lot faster with two changes:
  - bash instead of sh for bash's "read -N"
  - xxd dump of initrd instead of raw binary data

  
  I wonder, if the developers are interested in accepting my improvements into the code base?

  
  > time unmkinitramfs-classic-turbo 23.10/{initrd,unpack}

  real    0m2.209s
  user    0m2.161s
  sys     0m1.341s

  > time unmkinitramfs-turbo 23.10/{initrd,unpack}
   early .. +
  early2 .. +
  early3 .. +
    main .. +

  real    0m1.973s
  user    0m2.094s
  sys     0m1.244s

  
  All tools in the repo, even the classic one, feature -s/--scan option. With it, the tools only dump offset + size for embedded archives, nothing is unpacked. It can be used to measure parsing speed.

  In the repo there are three tools:
  - unmkinitramfs-classic: the original tool
  - unmkinitramfs-classic-turbo: the improved tool with minimal changes
  - unmkinitramfs-turbo: the improved tool with many changes to my tastes, extra dependency on "file" and some additional features (-ss option, decompression with 7za)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/2059976/+subscriptions