Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30)

[Andrey Grodzovsky]

Dear Upstream Livepatch  team and Ubuntu Kernel team - I included you both in this since the issue lies on the boundary between Ubuntu kernel and upstream. 

According to official kernel documentation  - https://docs.kernel.org/livepatch/livepatch.html#livepatch, section 7, Limitations - 
1 - Kretprobes using the ftrace framework conflict with the patched functions.
2 - Kprobes in the original function are ignored when the code is redirected to the new implementation.

Yet, when testing on my Ubuntu 5.15.0-1005.7-aws (based on 5.15.30 stable kernel) machine, I have no problem applying Livepatch and then setting krpobes and kretprobes on a patched function using bpftrace (or directly by coding a BPF program with kprobe/kretprobe attachment)and can confirm both execute without issues. Also the opposite works fine, running my krpobe and kretprobe hooks doesn't prevent from livepatch to be applied successfully. 

fentry/fexit probes do fail in in both directions - but this is expected according to my understanding as coexistence of livepatching and ftrace based BPF hooks are mutually exclusive until 6.0 based kernels 

libbpf: prog 'begin_new_exec_fentry': failed to attach: Device or resource busy
libbpf: prog 'begin_new_exec_fentry': failed to auto-attach: -16


Please help me understand this contradiction about kprobes - is this because the KPROBES are FTRACE based  or any other reason ?


Bellow a few prints to prove my point - the kernel function i am using to test is begin_new_exec

ubuntu at ip-xxxxx:~$ sudo canonical-livepatch status
last check: 19 minutes ago
kernel: 5.15.0-1005.7-aws
server check-in: succeeded
kernel state: ✗ Livepatch coverage ended 2023-05-20; please upgrade the kernel and reboot
patch state: ✓ all applicable livepatch modules inserted
patch version: 94.1
tier: stable
machine id: xxxxx

ubuntu at ip-xxxxxx:~$ sudo cat /proc/kallsyms | grep begin_new_exec
ffffffff8af7bdc0 T begin_new_exec
ffffffff8c335694 r __ksymtab_begin_new_exec
ffffffff8c369830 r __kstrtabns_begin_new_exec
ffffffff8c36cf3d r __kstrtab_begin_new_exec
ffffffffc28c1650 t begin_new_exec	[lkp_Ubuntu_5_15_0_1005_7_aws_94]
ffffffffc1a9d388 t bpf_prog_211a1dc71de113a4_begin_new_exec_kprobe	[bpf]
ffffffffc1a9f588 t bpf_prog_5522bacfbb7628bd_begin_new_exec_kretprobe	[bpf]

ubuntu at ip-xxxxx:~$ sudo cat /sys/kernel/debug/kprobes/list  | grep begin_new_exec
ffffffff8af7bdc0  r  begin_new_exec+0x0    [FTRACE]
ffffffff8af7bdc0  k  begin_new_exec+0x0    [FTRACE]


root at ip-xxxxx:~# echo 102400 > /sys/kernel/debug/tracing/buffer_size_kb && echo 1 > /sys/kernel/debug/tracing/events/bpf_trace/enable && echo 1 > /sys/kernel/debug/tracing/tracing_on && cat 
/sys/kernel/debug/tracing/trace_pipe
             cat-10406   [000] d...1 433341.003035: bpf_trace_printk: kprobe: begin_new_exec called with bprm=000000005304a36e
             cat-10406   [000] d...1 433341.003163: bpf_trace_printk: kretprobe: begin_new_exec returned 0