panic: Taint kernel if fault injection has been used
Commit Message
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Since the function error injection framework in the fault injection
subsystem can change the function code flow forcibly, it may cause
unexpected behavior (and that is the purpose of this feature) even
if it is applied to the ALLOW_ERROR_INJECTION functions.
So this feature must be used only for debugging or testing purpose.
To identify this in the kernel oops message, add a new taint flag
for the fault injection. This taint flag will be set by either
function error injection is used or the BPF use the kprobe_override
on error injectable functions (identified by ALLOW_ERROR_INJECTION).
Link: https://lore.kernel.org/all/20221121104403.1545f9b5@gandalf.local.home/T/#u
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Documentation/admin-guide/tainted-kernels.rst | 5 +++++
include/linux/panic.h | 3 ++-
kernel/fail_function.c | 2 ++
kernel/panic.c | 1 +
kernel/trace/bpf_trace.c | 2 ++
5 files changed, 12 insertions(+), 1 deletion(-)
Comments
On Fri, Dec 02, 2022 at 01:45:59PM +0900, Masami Hiramatsu (Google) wrote:
> From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
>
> Since the function error injection framework in the fault injection
> subsystem can change the function code flow forcibly, it may cause
> unexpected behavior (and that is the purpose of this feature) even
> if it is applied to the ALLOW_ERROR_INJECTION functions.
> So this feature must be used only for debugging or testing purpose.
>
> To identify this in the kernel oops message, add a new taint flag
> for the fault injection. This taint flag will be set by either
> function error injection is used or the BPF use the kprobe_override
> on error injectable functions (identified by ALLOW_ERROR_INJECTION).
>
> Link: https://lore.kernel.org/all/20221121104403.1545f9b5@gandalf.local.home/T/#u
>
> Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> ---
> Documentation/admin-guide/tainted-kernels.rst | 5 +++++
> include/linux/panic.h | 3 ++-
> kernel/fail_function.c | 2 ++
> kernel/panic.c | 1 +
> kernel/trace/bpf_trace.c | 2 ++
> 5 files changed, 12 insertions(+), 1 deletion(-)
I think you forgot to also update tools/debugging/kernel-chktaint with
this new entry :(
On Fri, 2 Dec 2022 07:33:11 +0100
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> On Fri, Dec 02, 2022 at 01:45:59PM +0900, Masami Hiramatsu (Google) wrote:
> > From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> >
> > Since the function error injection framework in the fault injection
> > subsystem can change the function code flow forcibly, it may cause
> > unexpected behavior (and that is the purpose of this feature) even
> > if it is applied to the ALLOW_ERROR_INJECTION functions.
> > So this feature must be used only for debugging or testing purpose.
> >
> > To identify this in the kernel oops message, add a new taint flag
> > for the fault injection. This taint flag will be set by either
> > function error injection is used or the BPF use the kprobe_override
> > on error injectable functions (identified by ALLOW_ERROR_INJECTION).
> >
> > Link: https://lore.kernel.org/all/20221121104403.1545f9b5@gandalf.local.home/T/#u
> >
> > Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> > ---
> > Documentation/admin-guide/tainted-kernels.rst | 5 +++++
> > include/linux/panic.h | 3 ++-
> > kernel/fail_function.c | 2 ++
> > kernel/panic.c | 1 +
> > kernel/trace/bpf_trace.c | 2 ++
> > 5 files changed, 12 insertions(+), 1 deletion(-)
>
> I think you forgot to also update tools/debugging/kernel-chktaint with
> this new entry :(
Oops, thanks for pointing!
Let me update the patch.
@@ -101,6 +101,7 @@ Bit Log Number Reason that got the kernel tainted
16 _/X 65536 auxiliary taint, defined for and used by distros
17 _/T 131072 kernel was built with the struct randomization plugin
18 _/N 262144 an in-kernel test has been run
+ 19 _/J 524288 a function-level error has been injected
=== === ====== ========================================================
Note: The character ``_`` is representing a blank in this table to make reading
@@ -182,3 +183,7 @@ More detailed explanation for tainting
produce extremely unusual kernel structure layouts (even performance
pathological ones), which is important to know when debugging. Set at
build time.
+
+ 19) ``J`` if a function-level error has been injected and the code path was
+ forcibly changed by either function error injection framework or BPF's
+ function override feature.
@@ -69,7 +69,8 @@ static inline void set_arch_panic_timeout(int timeout, int arch_default_timeout)
#define TAINT_AUX 16
#define TAINT_RANDSTRUCT 17
#define TAINT_TEST 18
-#define TAINT_FLAGS_COUNT 19
+#define TAINT_FAULT_INJECTED 19
+#define TAINT_FLAGS_COUNT 20
#define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1)
struct taint_flag {
@@ -9,6 +9,7 @@
#include <linux/kprobes.h>
#include <linux/module.h>
#include <linux/mutex.h>
+#include <linux/panic.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
@@ -298,6 +299,7 @@ static ssize_t fei_write(struct file *file, const char __user *buffer,
fei_attr_free(attr);
goto out;
}
+ add_taint(TAINT_FAULT_INJECTED, LOCKDEP_NOW_UNRELIABLE);
fei_debugfs_add_attr(attr);
list_add_tail(&attr->list, &fei_attr_list);
ret = count;
@@ -426,6 +426,7 @@ const struct taint_flag taint_flags[TAINT_FLAGS_COUNT] = {
[ TAINT_AUX ] = { 'X', ' ', true },
[ TAINT_RANDSTRUCT ] = { 'T', ' ', true },
[ TAINT_TEST ] = { 'N', ' ', true },
+ [ TAINT_FAULT_INJECTED ] = { 'J', ' ', false },
};
/**
@@ -2137,6 +2137,8 @@ int perf_event_attach_bpf_prog(struct perf_event *event,
goto unlock;
/* set the new array to event->tp_event and set event->prog */
+ if (prog->kprobe_override)
+ add_taint(TAINT_FAULT_INJECTED, LOCKDEP_NOW_UNRELIABLE);
event->prog = prog;
event->bpf_cookie = bpf_cookie;
rcu_assign_pointer(event->tp_event->prog_array, new_array);