From patchwork Wed Nov 8 14:28:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 163054 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:aa0b:0:b0:403:3b70:6f57 with SMTP id k11csp950384vqo; Wed, 8 Nov 2023 06:29:27 -0800 (PST) X-Google-Smtp-Source: AGHT+IHaJ3md3IQjEqR1nAsU8mi5F3yGyAvl62m592PXKJ4eNdD6CevveCeNDXIpPkRsoLrtqVEv X-Received: by 2002:a17:90b:1b44:b0:262:f449:4497 with SMTP id nv4-20020a17090b1b4400b00262f4494497mr1972707pjb.2.1699453767079; Wed, 08 Nov 2023 06:29:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699453767; cv=none; d=google.com; s=arc-20160816; b=d8M5yqCaeQCpInvT+rYMaYO9fLSsjjXluuqBHwRYPf/UpyEfazVDRIOKBV9nDV7K8Y saC3QuQk4+8a30N5FvO0P49yVDjZtyD1hIcUTekX6wFXNzMjQoyqTNh/H8ATuG4qqK+h OKbWt9G/t4Uc7VSWRRoV5kZCEHA5Fhcw9Ug5PZhSDWt74ZKYQ2ch3NufNzF6N/GwbC4l M/ZvYETSblUqoerW/boEQ4be65PttWjCUVvpiA5L8YHOlmlznkcSf+klmS2VG+QOJpfk vwVeT4QYsEHxmw05hEGGtrGjZ/wQP2T24P87iQaV4PIX4nkUqM+eP3QfhQRaq2i6lChk qwGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=rrtSSQc8lFAUiHrU80jSk8mdrwzknQ3Xs+6Pz5doVig=; fh=SIgps5XdV0XNwjZfT2uAI7g3mrspDldK9Qs8qQAfoa4=; b=g9tlea/3/hIj/D46/NitSYPTESQLK3PGZ8RL47gwMTU/LbVcGHEMioznh1WGPHRkBp lKGZI/gfMdA9YzECoADSQfRNil9beSEeAXEP5oaUP9XV76VeJvqhOhGRdk2FaTmc910+ O65OcmyBP1yEAtNM8fCDbfmC+PeyU8Nz4D9aulWYW17Z8yfU/7oqChJRYzJegQlp1UIg pQEoJyq0geQBXZDnTNgWiHKNa79qAlzMZMqVr9vkInnFyeVPi5yuaYkzF4hR5jYVEKOi eMcF0C6EubPweCucGr0K7VIzXiIUdb8cPfXmfVuZG0/DHcuLtlxyeKMSv2oBXkH0Y7/1 95NA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="W/5RorDe"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id x9-20020a17090aca0900b0026b502223cbsi2318373pjt.10.2023.11.08.06.29.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Nov 2023 06:29:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="W/5RorDe"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id E64EB8184A90; Wed, 8 Nov 2023 06:29:16 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235709AbjKHO2u (ORCPT + 32 others); Wed, 8 Nov 2023 09:28:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343927AbjKHO23 (ORCPT ); Wed, 8 Nov 2023 09:28:29 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E9172115 for ; Wed, 8 Nov 2023 06:28:18 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED501C433C7; Wed, 8 Nov 2023 14:28:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699453697; bh=GvAYVpCT1/lgHCFucnCN7PpXdofclOcOgUgdSy5xBGY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W/5RorDeBfZBaXbKZNPVGHxh23sviEFSwiyoeQH4ndthykVmK/IQec5l7UfGdXFIa m7uyKZrsOkARY60LvDpXPUkRGpRqLsC3pP0ElIaLagD88ZIR3SZDRw7WWr4WVoWx5W xB8sNjNSoNLzpW85nk6XAzA3CoO+EwOIDNMoQmtszsWk3G28O9ond6lIyh+Uk2i9LI wl4r+ybeJ48FmuI1KnQ0xxuRyQg9E0t8cWaPs9VeJKjzFR4ctvptCUBDVlgaGGjtb8 Efq9gChOFHFi3MyPxW5cU0OeQOBCWg59N1l542jApzXGjV3eyHxEVBPsSlvuj+J3cT RAi8kNwpUd8mQ== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner , Guo Ren Subject: [RFC PATCH v2 20/31] function_graph: Add a new exit handler with parent_ip and ftrace_regs Date: Wed, 8 Nov 2023 23:28:11 +0900 Message-Id: <169945369135.55307.16963023663631933656.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169945345785.55307.5003201137843449313.stgit@devnote2> References: <169945345785.55307.5003201137843449313.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Wed, 08 Nov 2023 06:29:17 -0800 (PST) X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782006432847334022 X-GMAIL-MSGID: 1782006432847334022 From: Masami Hiramatsu (Google) Add a new return handler to fgraph_ops as 'retregfunc' which takes parent_ip and ftrace_regs instead of ftrace_graph_ret. This handler is available only if the arch support CONFIG_HAVE_FUNCTION_GRAPH_FREGS. Note that the 'retfunc' and 'reregfunc' are mutual exclusive. You can set only one of them. Signed-off-by: Masami Hiramatsu (Google) --- arch/x86/include/asm/ftrace.h | 2 + include/linux/ftrace.h | 10 +++++- kernel/trace/Kconfig | 5 ++- kernel/trace/fgraph.c | 74 +++++++++++++++++++++++++++-------------- 4 files changed, 63 insertions(+), 28 deletions(-) diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h index fc60974a1d89..4701d009c215 100644 --- a/arch/x86/include/asm/ftrace.h +++ b/arch/x86/include/asm/ftrace.h @@ -72,6 +72,8 @@ arch_ftrace_get_regs(struct ftrace_regs *fregs) override_function_with_return(&(fregs)->regs) #define ftrace_regs_query_register_offset(name) \ regs_query_register_offset(name) +#define ftrace_regs_get_frame_pointer(fregs) \ + frame_pointer(&(fregs)->regs) struct ftrace_ops; #define ftrace_graph_func ftrace_graph_func diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index c91b234949d5..8efccd911d41 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -43,7 +43,9 @@ struct dyn_ftrace; char *arch_ftrace_match_adjust(char *str, const char *search); -#ifdef CONFIG_HAVE_FUNCTION_GRAPH_RETVAL +#ifdef CONFIG_HAVE_FUNCTION_GRAPH_FREGS +unsigned long ftrace_return_to_handler(struct ftrace_regs *fregs); +#elif defined(CONFIG_HAVE_FUNCTION_GRAPH_RETVAL) struct fgraph_ret_regs; unsigned long ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs); #else @@ -156,6 +158,7 @@ struct ftrace_regs { #define ftrace_regs_set_instruction_pointer(fregs, ip) do { } while (0) #endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ + static __always_inline struct pt_regs *ftrace_get_regs(struct ftrace_regs *fregs) { if (!fregs) @@ -1066,6 +1069,10 @@ typedef int (*trace_func_graph_regs_ent_t)(unsigned long func, unsigned long parent_ip, struct ftrace_regs *fregs, struct fgraph_ops *); /* entry w/ regs */ +typedef void (*trace_func_graph_regs_ret_t)(unsigned long func, + unsigned long parent_ip, + struct ftrace_regs *, + struct fgraph_ops *); /* return w/ regs */ extern int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace, struct fgraph_ops *gops); @@ -1075,6 +1082,7 @@ struct fgraph_ops { trace_func_graph_ent_t entryfunc; trace_func_graph_ret_t retfunc; trace_func_graph_regs_ent_t entryregfunc; + trace_func_graph_regs_ret_t retregfunc; struct ftrace_ops ops; /* for the hash lists */ void *private; int idx; diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 61c541c36596..308b3bec01b1 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -34,6 +34,9 @@ config HAVE_FUNCTION_GRAPH_TRACER config HAVE_FUNCTION_GRAPH_RETVAL bool +config HAVE_FUNCTION_GRAPH_FREGS + bool + config HAVE_DYNAMIC_FTRACE bool help @@ -232,7 +235,7 @@ config FUNCTION_GRAPH_TRACER config FUNCTION_GRAPH_RETVAL bool "Kernel Function Graph Return Value" - depends on HAVE_FUNCTION_GRAPH_RETVAL + depends on HAVE_FUNCTION_GRAPH_RETVAL || HAVE_FUNCTION_GRAPH_FREGS depends on FUNCTION_GRAPH_TRACER default n help diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c index 6567b18c6c54..cf240914ef9b 100644 --- a/kernel/trace/fgraph.c +++ b/kernel/trace/fgraph.c @@ -623,8 +623,7 @@ int function_graph_enter_regs(unsigned long ret, unsigned long func, /* Retrieve a function return address to the trace stack on thread info.*/ static struct ftrace_ret_stack * -ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret, - unsigned long frame_pointer) +ftrace_pop_return_trace(unsigned long *ret, unsigned long frame_pointer) { struct ftrace_ret_stack *ret_stack; int index; @@ -669,10 +668,6 @@ ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret, #endif *ret = ret_stack->ret; - trace->func = ret_stack->func; - trace->calltime = ret_stack->calltime; - trace->overrun = atomic_read(¤t->trace_overrun); - trace->depth = current->curr_ret_depth; /* * We still want to trace interrupts coming in if * max_depth is set to 1. Make sure the decrement is @@ -711,22 +706,43 @@ static struct notifier_block ftrace_suspend_notifier = { /* fgraph_ret_regs is not defined without CONFIG_FUNCTION_GRAPH_RETVAL */ struct fgraph_ret_regs; +static void fgraph_call_retfunc(struct ftrace_regs *fregs, + struct fgraph_ret_regs *ret_regs, + struct ftrace_ret_stack *ret_stack, + struct fgraph_ops *gops) +{ + struct ftrace_graph_ret trace; + + trace.func = ret_stack->func; + trace.calltime = ret_stack->calltime; + trace.overrun = atomic_read(¤t->trace_overrun); + trace.depth = current->curr_ret_depth; + trace.rettime = trace_clock_local(); +#ifdef CONFIG_FUNCTION_GRAPH_RETVAL + if (fregs) + trace.retval = ftrace_regs_return_value(fregs); + else + trace.retval = fgraph_ret_regs_return_value(ret_regs); +#endif + gops->retfunc(&trace, gops); +} + /* * Send the trace to the ring-buffer. * @return the original return address. */ -static unsigned long __ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs, +static unsigned long __ftrace_return_to_handler(struct ftrace_regs *fregs, + struct fgraph_ret_regs *ret_regs, unsigned long frame_pointer) { struct ftrace_ret_stack *ret_stack; - struct ftrace_graph_ret trace; - unsigned long ret; + struct fgraph_ops *gops; int curr_ret_stack; + unsigned long ret; int stop_at; int index; - int idx; - ret_stack = ftrace_pop_return_trace(&trace, &ret, frame_pointer); + ret_stack = ftrace_pop_return_trace(&ret, frame_pointer); if (unlikely(!ret_stack)) { ftrace_graph_stop(); @@ -735,11 +751,6 @@ static unsigned long __ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs return (unsigned long)panic; } - trace.rettime = trace_clock_local(); -#ifdef CONFIG_FUNCTION_GRAPH_RETVAL - trace.retval = fgraph_ret_regs_return_value(ret_regs); -#endif - curr_ret_stack = current->curr_ret_stack; index = get_ret_stack_index(current, curr_ret_stack - 1); @@ -752,8 +763,11 @@ static unsigned long __ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs val = current->ret_stack[curr_ret_stack - 1]; switch (__get_type(val)) { case FGRAPH_TYPE_ARRAY: - idx = __get_array(val); - fgraph_array[idx]->retfunc(&trace, fgraph_array[idx]); + gops = fgraph_array[__get_array(val)]; + if (gops->retregfunc) + gops->retregfunc(ret_stack->func, ret, fregs, gops); + else + fgraph_call_retfunc(fregs, ret_regs, ret_stack, gops); curr_ret_stack -= __get_index(val); break; case FGRAPH_TYPE_RESERVED: @@ -777,20 +791,22 @@ static unsigned long __ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs return ret; } -/* - * After all architecures have selected HAVE_FUNCTION_GRAPH_RETVAL, we can - * leave only ftrace_return_to_handler(ret_regs). - */ -#ifdef CONFIG_HAVE_FUNCTION_GRAPH_RETVAL +#ifdef CONFIG_HAVE_FUNCTION_GRAPH_FREGS +unsigned long ftrace_return_to_handler(struct ftrace_regs *fregs) +{ + return __ftrace_return_to_handler(fregs, NULL, + ftrace_regs_get_frame_pointer(fregs)); +} +#elif defined(CONFIG_HAVE_FUNCTION_GRAPH_RETVAL) unsigned long ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs) { - return __ftrace_return_to_handler(ret_regs, + return __ftrace_return_to_handler(NULL, ret_regs, fgraph_ret_regs_frame_pointer(ret_regs)); } #else unsigned long ftrace_return_to_handler(unsigned long frame_pointer) { - return __ftrace_return_to_handler(NULL, frame_pointer); + return __ftrace_return_to_handler(NULL, NULL, frame_pointer); } #endif @@ -1131,9 +1147,15 @@ int register_ftrace_graph(struct fgraph_ops *gops) int ret = 0; int i; - if (gops->entryfunc && gops->entryregfunc) + if ((gops->entryfunc && gops->entryregfunc) || + (gops->retfunc && gops->retregfunc)) return -EINVAL; +#ifndef CONFIG_HAVE_FUNCTION_GRAPH_FREGS + if (gops->retregfunc) + return -EOPNOTSUPP; +#endif + mutex_lock(&ftrace_lock); if (!gops->ops.func) {