APPLIED: [Bionic][SRU][PATCH 0/4] Handle the skip return code in kernel_selftests on Bionic

[Khaled Elmously]

On 2019-11-06 18:29:02 , Po-Hsu Lin wrote:
> BugLink: https://bugs.launchpad.net/bugs/1812352
> 
> == Justification ==
> In the ubuntu_kernel_selftests, the skipped test will uses return code 4
> (KSFT_SKIP).
> 
> However the code to handle this non-zero return code was not implemented
> in the kselftest framework. And this will generate some false-positive
> results as those skipped tests were treated as a failure.
> 
> For example the raw_skew test in timers on 4.15:
>     1..0 # Skipped: The clock was adjusted externally. Shutdown NTPd or other time sync daemons
>     not ok 1..7 selftests: raw_skew [FAIL]
> 
> And the test_user_copy.sh test in user (failure could be found in KVM kernels):
>     user: module test_user_copy is not found [SKIP]
>     not ok 1..1 selftests: test_user_copy.sh [FAIL]
> 
> Xenial is affected by this issue as well. Considering the fact that this
> issue in Bionic affects more tests / kernel variants, and it's way
> easier to fix this in Bionic, I decided to go for Bionic first.
> 
> == Fix ==
> 771cbc3b (selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers)
> a3355440 (selftests: Fix lib.mk run_tests target shell script)
> 42b44c34 (selftests: lib.mk: cleanup RUN_TESTS define and make it readable)
> 3f4435b5 (selftests: lib.mk: add SKIP handling to RUN_TESTS define)
> 
> All these can be cherry-picked into Bionic.
> 
> The crucial patch is commit 3f4435b5, the other three patches are just
> some minor fixes that pave the road for it to be cherry-picked.
> 
> == Test ==
> Tested on a Bionic amd64 node, the result shows positive.
> 
> A test summary before the patch:
> END GOOD ubuntu_kernel_selftests.setup
> END GOOD ubuntu_kernel_selftests.breakpoints
> END GOOD ubuntu_kernel_selftests.cpu-hotplug
> END GOOD ubuntu_kernel_selftests.efivarfs
> END GOOD ubuntu_kernel_selftests.memfd
> END GOOD ubuntu_kernel_selftests.memory-hotplug
> END GOOD ubuntu_kernel_selftests.mount
> END GOOD ubuntu_kernel_selftests.net
> END GOOD ubuntu_kernel_selftests.ptrace
> END GOOD ubuntu_kernel_selftests.seccomp
> END ERROR ubuntu_kernel_selftests.timers
> END GOOD ubuntu_kernel_selftests.powerpc
> END GOOD ubuntu_kernel_selftests.user
> END GOOD ubuntu_kernel_selftests.ftrace
> 
> A test summary after the patch:
> END GOOD ubuntu_kernel_selftests.setup
> END GOOD ubuntu_kernel_selftests.breakpoints
> END GOOD ubuntu_kernel_selftests.cpu-hotplug
> END GOOD ubuntu_kernel_selftests.efivarfs
> END GOOD ubuntu_kernel_selftests.memfd
> END GOOD ubuntu_kernel_selftests.memory-hotplug
> END GOOD ubuntu_kernel_selftests.mount
> END GOOD ubuntu_kernel_selftests.net
> END GOOD ubuntu_kernel_selftests.ptrace
> END GOOD ubuntu_kernel_selftests.seccomp
> END GOOD ubuntu_kernel_selftests.timers
> END GOOD ubuntu_kernel_selftests.powerpc
> END GOOD ubuntu_kernel_selftests.user
> END GOOD ubuntu_kernel_selftests.ftrac
> 
> No tests were skipped unintentionally. And the raw_skew test in timers
> is now marked as [SKIPPED] and no longer causing failure.
> 
> A complete test report could be found in the bug report attachment.
> 
> == Regression Potential ==
> Low, this patch set just added support to handle the skip return code
> and changes are limited to the kselftest framework itself, no actual
> impact to normal users.
> 
> Mathieu Desnoyers (1):
>   selftests: Fix lib.mk run_tests target shell script
> 
> Shuah Khan (1):
>   selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers
> 
> Shuah Khan (Samsung OSG) (2):
>   selftests: lib.mk: cleanup RUN_TESTS define and make it readable
>   selftests: lib.mk: add SKIP handling to RUN_TESTS define
> 
>  tools/testing/selftests/lib.mk | 26 ++++++++++++++++++++------
>  1 file changed, 20 insertions(+), 6 deletions(-)
> 
> -- 
> 2.7.4
> 
> 
> -- 
> kernel-team mailing list
> kernel-team at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team