[External] Re: Question - Livepatch/Kprobe Coexistence on Ftrace-enabled Functions (Ubuntu kernel based on Linux stable 5.15.30)

[Song Liu]

On Mon, Oct 20, 2025 at 9:45 AM Andrey Grodzovsky
<andrey.grodzovsky at crowdstrike.com> wrote:
>
> On 10/20/25 12:03, Song Liu wrote:
> > On Mon, Oct 20, 2025 at 7:56 AM Andrey Grodzovsky
> > <andrey.grodzovsky at crowdstrike.com> wrote:
> > [...]
> >>> If you build the kernel from source code, there are some samples in
> >>> samples/livepatch that you can use for testing. PS: You need to enable
> >>>
> >>>     CONFIG_SAMPLE_LIVEPATCH=m
> >>>
> >>> I hope this helps.
> >> Thanks Song, working on repro, kernel rebuilt, test module is loading
> >> but, bpftrace is refusing to attach now to fentries/fexits claiming the
> >> costum kernel is not supporting it. It did
> >> attach in the case of stock AWS kernel i copied the .config from. So
> >> just trying to figure out now if some Kcofnig flags are missing or
> >> different . Let me know in case you manage to confirm yourself in the
> >> meanwhile the fix works for
> >> you.
> > Yes, it worked in my tests.
> >
> > [root@(none) /]# kpatch load linux/samples/livepatch/livepatch-sample.ko
> > loading patch module: linux/samples/livepatch/livepatch-sample.ko
> > [root@(none) /]# bpftrace.real -e 'fexit:cmdline_proc_show
> > {printf("fexit\n");}' &
> > [1] 388
> > [root@(none) /]# Attached 1 probe
> > [root@(none) /]# bpftrace.real -e 'fentry:cmdline_proc_show
> > {printf("fentry\n");}' &
> > [2] 397
> > [root@(none) /]# Attached 1 probe
> >
> > [root@(none) /]# cat /proc/cmdline
> > this has been live patched
> > fentry
> > fexit
> >
> > Thanks,
> > Song
> >
>
> Verified the failures I observe when trying to attach with BPF trace are
> only in presence of patch you provided.
> Please see attached dmesg for failures. Initial warning on boot.
> Subsequebt warnings and errors at the point i try to run
> sudo bpftrace -e "fexit:cmdline_proc_show { printf(\"fexit hit\\n\");
> exit(); }"
>
> sudo: unable to resolve host ip-10-10-115-238: Temporary failure in name
> resolution
> stdin:1:1-25: ERROR: kfunc/kretfunc not available for your kernel version.
>
> ubuntu at ip-10-10-115-238:~/linux-6.8.1$ sudo cat
> /sys/kernel/debug/tracing/available_filter_functions | grep
> cmdline_proc_show
> sudo: unable to resolve host ip-10-10-115-238: Temporary failure in name
> resolution
> cat: /sys/kernel/debug/tracing/available_filter_functions: No such device
>
> After reboot and before trying to attacg with bpftrace,
> /sys/kernel/debug/tracing/available_filter_functions is available and
> shows all function.
>
> Using stable kernel from
> https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.8.1.tar.gz for build.
> FTRACE related KCONFIGs bellow

I can see the similar issue with the upstream kernel. I was testing on
stable 6.17 before just know because of another issue with upstream
kernel, and somehow 6.17 kernel doesn't seem to have the issue.

To fix this, I think we should land a fix similar to the earlier diff:

diff --git i/kernel/trace/ftrace.c w/kernel/trace/ftrace.c
index 42bd2ba68a82..8f320df0ac52 100644
--- i/kernel/trace/ftrace.c
+++ w/kernel/trace/ftrace.c
@@ -6049,6 +6049,9 @@ int register_ftrace_direct(struct ftrace_ops
*ops, unsigned long addr)

        err = register_ftrace_function_nolock(ops);

+       if (err)
+               remove_direct_functions_hash(hash, addr);
+
  out_unlock:
        mutex_unlock(&direct_mutex);


Steven,

Does this change look good to you?

Thanks,
Song