[PATCH 0/1][SRU][U/P/O/N/J/F] distribute sources list generation in parallel
You-Sheng Yang
vicamo.yang at canonical.com
Fri Mar 28 18:48:04 UTC 2025
BugLink: https://bugs.launchpad.net/bugs/2104911
SRU Justification:
[ Impact ]
In debian/rules.d/2-binary-arch.mk, stamp-build-% targets, we have:
```
# Collect the list of kernel source files used for this build. Need to do this early
# before modules are stripped. Fail if the resulting file is empty.
find $(build_dir) \( -name vmlinux -o -name \*.ko \) -exec dwarfdump -i {} \; | \
grep -E 'DW_AT_(call|decl)_file' | sed -n 's|.*\s/|/|p' | sort -u > \
$(build_dir)/sources.list
test -s $(build_dir)/sources.list
```
The process is single threaded and may takes up to half an hour to complete:
```
$ time ...
real 28m15.167s
user 54m32.917s
sys 4m32.531s
```
The best case (don't sort, ignore output overlapping from find) takes about 4
minutes.
For the solution, let `find` output only kernel module paths and use
`xargs -P $(nproc)` to invoke dwarfdump pipelines in parallel. This shorten the
time for the whole process to 13 minutes.
Then pre-sort output from each of the dwarfdump pipeline, this shorten the time
to 6 and half minutes.
Finally, use `flock` to guard output control.
[ Test Plan ]
1. Add `time` at the beginning of the `find ...` line to measure time spent.
Also dump the checksum of the generated sources.list file for later
comparison.
2. Rebuild kernel, check the time spent for generating sources list
3. Compare the checksum of sources.list generated.
[ Where problems could occur ]
Build recipt changes only.
[ Other Info ]
This affects only series with dwarfdump usage. Nominate for U/P/O/N/J/F.
You-Sheng Yang (1):
UBUNTU: [Packaging] distribute sources list generation in parallel
debian/rules.d/2-binary-arch.mk | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
--
2.48.1
More information about the kernel-team
mailing list