NACK: [PATCH 0/1][SRU][U/P/O/N/J/F] distribute sources list generation in parallel

[Mehmet Basaran]

Nacked-by: Mehmet Basaran <mehmet.basaran at canonical.com>

There is a v2 in the mailing list.

-------------- next part --------------
You-Sheng Yang <vicamo.yang at canonical.com> writes:

> BugLink: https://bugs.launchpad.net/bugs/2104911
>
> SRU Justification:
>
> [ Impact ]
>
> In debian/rules.d/2-binary-arch.mk, stamp-build-% targets, we have:
> ```
> # Collect the list of kernel source files used for this build. Need to do this early
> # before modules are stripped. Fail if the resulting file is empty.
> find $(build_dir) \( -name vmlinux -o -name \*.ko \) -exec dwarfdump -i {} \; | \
>         grep -E 'DW_AT_(call|decl)_file' | sed -n 's|.*\s/|/|p' | sort -u > \
>         $(build_dir)/sources.list
> test -s $(build_dir)/sources.list
> ```
>
> The process is single threaded and may takes up to half an hour to complete:
> ```
> $ time ...
>
> real 28m15.167s
> user 54m32.917s
> sys 4m32.531s
> ```
>
> The best case (don't sort, ignore output overlapping from find) takes about 4
> minutes.
>
> For the solution, let `find` output only kernel module paths and use
> `xargs -P $(nproc)` to invoke dwarfdump pipelines in parallel. This shorten the
> time for the whole process to 13 minutes.
>
> Then pre-sort output from each of the dwarfdump pipeline, this shorten the time
> to 6 and half minutes.
>
> Finally, use `flock` to guard output control.
>
> [ Test Plan ]
>
> 1. Add `time` at the beginning of the `find ...` line to measure time spent.
>    Also dump the checksum of the generated sources.list file for later
>    comparison.
>
> 2. Rebuild kernel, check the time spent for generating sources list
>
> 3. Compare the checksum of sources.list generated.
>
> [ Where problems could occur ]
>
> Build recipt changes only.
>
> [ Other Info ]
>
> This affects only series with dwarfdump usage. Nominate for U/P/O/N/J/F.
>
> You-Sheng Yang (1):
>   UBUNTU: [Packaging] distribute sources list generation in parallel
>
>  debian/rules.d/2-binary-arch.mk | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> --
> 2.48.1
>
>
> --
> kernel-team mailing list
> kernel-team at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 873 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/kernel-team/attachments/20250331/db379bf8/attachment.sig>