From patchwork Tue Feb 6 15:08:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 197517 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp1623395dyb; Tue, 6 Feb 2024 07:36:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IGfsEnSAuwUzhA3D8sJQtWB3A1chg7Lfc107QxBa2bmGwASuuycRa7lBFQR7Jee/BkyYd9U X-Received: by 2002:a17:902:ec82:b0:1d9:bfde:d635 with SMTP id x2-20020a170902ec8200b001d9bfded635mr2484518plg.18.1707233811435; Tue, 06 Feb 2024 07:36:51 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707233811; cv=pass; d=google.com; s=arc-20160816; b=jF3yF61xXt6zKfJOk2+KkIXnyjBcqrIgHyZTxmi6qFemPaEbvbDft6cEWuzdE9QYSI DoJTMBL0bf8oKv8El1KyKj/1YCKZK/RMqfkVgrL9e5Sby0Z5W0Jo0UlDha95quGo2Puz ISm0AmkrdrshAV7hpIo/6RC2+h3xFtzBUldu9Q/5eAWW1De0vH2PmdtACbH7zJ+SROQa LAdkgPK7p4vlLAPz6MWSV/bz/Tp+VNci2tySUXysmfy6W+AGpnWKlmFrr/XqJROkd5wJ vQpCSqu9Mb+K/Of3SrYlY9v553DCLcWDA3vt4w/9C7Ex3R9OYLnRYKTzTt6fvqPQHP7a v/eg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:user-agent:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=fs82T8hV0vQbKPNq2BQvLSid2V3dkDLdahl1iTwKtcg=; fh=h8I8bfuvAZAX9JKag/b2vUScFi3hBOlyiMGXhZJucv8=; b=wMZdnn0oGuJGXOS8kH8UuykLvfycJ7FhgTRksfLCf4uwQSm8Ytqlj0KQBXjLUK8lv0 rzBUBSx8twYifuIL46ZbuoUPxYZUmVZ/Jid1y8SnpyDpCrMMME0OMRqkwAWLIQdVbMQi EB12ela5vhNC5rRflGuN7RzGYHxKEmQw3JIuClixbMqdSjIF3mSiwQFWib8+sWwretc8 vTSH6LDaanjvNxdde1fLWgNIn8C3Rk6ok8cClheahuLjNCuA6acYETA+bLGGoRlcDK/T wdtf6WqdhpGaePNOInJmJjuBtQrN4p91xe02FUrnMaRXihJfeXLqs8lnWjaw6p+GiqTY Ayjw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PfQDgU2z; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-55182-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-55182-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCWZXB6yOtjTxpshl9kI4yqESqjvjiSA+zSg0CsV5Ukzs62opWH3WB3SnhAM3KSEqWVEhJg0FIFKd/sqz7/nIPucUsPe/w== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id h1-20020a170902eec100b001d8f7d2a3fbsi1743202plb.635.2024.02.06.07.36.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 07:36:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-55182-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PfQDgU2z; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-55182-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-55182-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 5E973B2E0CE for ; Tue, 6 Feb 2024 15:10:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id F22CD13340C; Tue, 6 Feb 2024 15:08:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PfQDgU2z" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD2981E866; Tue, 6 Feb 2024 15:08:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707232134; cv=none; b=uaxMXKwDQlUpYgpMbPGNWD9/kHgHkMWO3ohz0bLI0Kt1D7jv3E51bXRfQnIh3qohx40KIGDi3jvkyL44Ag9or0+W4DJhLPo2eNiSZH1vugHMF6zz99bqmQcoc2R4vlN2abtSNaNlhgJAwTyLAiGqCNWKGsLEx0+CcMAMcT/gWBo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707232134; c=relaxed/simple; bh=tA+xME0cqxvdm/SL6/3HnWW/4VMwGO83nJFglJrGK1Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=FXDQiQ6L7DRo6zYiEEv5j4WMrdJmk+mvNRVuHZFVxI77rr7gyIqKCo34i6edfRltiBo8xmjXtHew9o/Shgl7LHxmOFsiZ5ITZR0JjRnKnmXFNh7wBFxOT3857Trd0MGh8T3ihLXj5JGPPZbXdRbYt7FGePX9Oni23MWltCWltCA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PfQDgU2z; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9206DC433F1; Tue, 6 Feb 2024 15:08:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707232134; bh=tA+xME0cqxvdm/SL6/3HnWW/4VMwGO83nJFglJrGK1Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PfQDgU2zs3j1o4+AE+6YK5rzCbJuIiEoN9VVdriOGJ6sckJ15hwOUqUFH9NspK135 bVquqFtn2cXt3nPGo06vmKW4LkCAfSOwf2IxLgzS7TSP2dCG+ZKgQl6gBEaZRdwzZB eKI/LNC/IQbGKYEVIg5jVKriIEnf47vgOa83hXqiyzcp+iwQTLFp6/jKW8Nj/7A5ml XZEc+HYjL2FA9uYk4Hs5WIK+f328h+Zvrs9xh6QZ4IaLrsa9BiQXBD3/yZuFk2SUyi YklEQ/s1EUJZjCi/Gg6mFcTKE47wr8VbWynG/+1nRLo/+7T1KBr40e5ZiINy/U6fjN 6C9SL5oYrrfaw== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner , Guo Ren Subject: [PATCH v7 07/36] function_graph: Add an array structure that will allow multiple callbacks Date: Wed, 7 Feb 2024 00:08:47 +0900 Message-Id: <170723212744.502590.17637620718033431909.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <170723204881.502590.11906735097521170661.stgit@devnote2> References: <170723204881.502590.11906735097521170661.stgit@devnote2> User-Agent: StGit/0.19 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790164401032888721 X-GMAIL-MSGID: 1790164401032888721 From: Steven Rostedt (VMware) Add an array structure that will eventually allow the function graph tracer to have up to 16 simultaneous callbacks attached. It's an array of 16 fgraph_ops pointers, that is assigned when one is registered. On entry of a function the entry of the first item in the array is called, and if it returns zero, then the callback returns non zero if it wants the return callback to be called on exit of the function. The array will simplify the process of having more than one callback attached to the same function, as its index into the array can be stored on the shadow stack. We need to only save the index, because this will allow the fgraph_ops to be freed before the function returns (which may happen if the function call schedule for a long time). Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Masami Hiramatsu (Google) --- Changes in v2: - Remove unneeded brace. --- kernel/trace/fgraph.c | 114 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 81 insertions(+), 33 deletions(-) diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c index 6f8d36370994..3f9dd213e7d8 100644 --- a/kernel/trace/fgraph.c +++ b/kernel/trace/fgraph.c @@ -39,6 +39,11 @@ DEFINE_STATIC_KEY_FALSE(kill_ftrace_graph); int ftrace_graph_active; +static int fgraph_array_cnt; +#define FGRAPH_ARRAY_SIZE 16 + +static struct fgraph_ops *fgraph_array[FGRAPH_ARRAY_SIZE]; + /* Both enabled by default (can be cleared by function_graph tracer flags */ static bool fgraph_sleep_time = true; @@ -62,6 +67,20 @@ int __weak ftrace_disable_ftrace_graph_caller(void) } #endif +int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace) +{ + return 0; +} + +static void ftrace_graph_ret_stub(struct ftrace_graph_ret *trace) +{ +} + +static struct fgraph_ops fgraph_stub = { + .entryfunc = ftrace_graph_entry_stub, + .retfunc = ftrace_graph_ret_stub, +}; + /** * ftrace_graph_stop - set to permanently disable function graph tracing * @@ -159,7 +178,7 @@ int function_graph_enter(unsigned long ret, unsigned long func, goto out; /* Only trace if the calling function expects to */ - if (!ftrace_graph_entry(&trace)) + if (!fgraph_array[0]->entryfunc(&trace)) goto out_ret; return 0; @@ -274,7 +293,7 @@ static unsigned long __ftrace_return_to_handler(struct fgraph_ret_regs *ret_regs trace.retval = fgraph_ret_regs_return_value(ret_regs); #endif trace.rettime = trace_clock_local(); - ftrace_graph_return(&trace); + fgraph_array[0]->retfunc(&trace); /* * The ftrace_graph_return() may still access the current * ret_stack structure, we need to make sure the update of @@ -410,11 +429,6 @@ void ftrace_graph_sleep_time_control(bool enable) fgraph_sleep_time = enable; } -int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace) -{ - return 0; -} - /* * Simply points to ftrace_stub, but with the proper protocol. * Defined by the linker script in linux/vmlinux.lds.h @@ -652,37 +666,54 @@ static int start_graph_tracing(void) int register_ftrace_graph(struct fgraph_ops *gops) { int ret = 0; + int i; mutex_lock(&ftrace_lock); - /* we currently allow only one tracer registered at a time */ - if (ftrace_graph_active) { + if (!fgraph_array[0]) { + /* The array must always have real data on it */ + for (i = 0; i < FGRAPH_ARRAY_SIZE; i++) + fgraph_array[i] = &fgraph_stub; + } + + /* Look for an available spot */ + for (i = 0; i < FGRAPH_ARRAY_SIZE; i++) { + if (fgraph_array[i] == &fgraph_stub) + break; + } + if (i >= FGRAPH_ARRAY_SIZE) { ret = -EBUSY; goto out; } - register_pm_notifier(&ftrace_suspend_notifier); + fgraph_array[i] = gops; + if (i + 1 > fgraph_array_cnt) + fgraph_array_cnt = i + 1; ftrace_graph_active++; - ret = start_graph_tracing(); - if (ret) { - ftrace_graph_active--; - goto out; - } - ftrace_graph_return = gops->retfunc; + if (ftrace_graph_active == 1) { + register_pm_notifier(&ftrace_suspend_notifier); + ret = start_graph_tracing(); + if (ret) { + ftrace_graph_active--; + goto out; + } + + ftrace_graph_return = gops->retfunc; - /* - * Update the indirect function to the entryfunc, and the - * function that gets called to the entry_test first. Then - * call the update fgraph entry function to determine if - * the entryfunc should be called directly or not. - */ - __ftrace_graph_entry = gops->entryfunc; - ftrace_graph_entry = ftrace_graph_entry_test; - update_function_graph_func(); + /* + * Update the indirect function to the entryfunc, and the + * function that gets called to the entry_test first. Then + * call the update fgraph entry function to determine if + * the entryfunc should be called directly or not. + */ + __ftrace_graph_entry = gops->entryfunc; + ftrace_graph_entry = ftrace_graph_entry_test; + update_function_graph_func(); - ret = ftrace_startup(&graph_ops, FTRACE_START_FUNC_RET); + ret = ftrace_startup(&graph_ops, FTRACE_START_FUNC_RET); + } out: mutex_unlock(&ftrace_lock); return ret; @@ -690,19 +721,36 @@ int register_ftrace_graph(struct fgraph_ops *gops) void unregister_ftrace_graph(struct fgraph_ops *gops) { + int i; + mutex_lock(&ftrace_lock); if (unlikely(!ftrace_graph_active)) goto out; - ftrace_graph_active--; - ftrace_graph_return = ftrace_stub_graph; - ftrace_graph_entry = ftrace_graph_entry_stub; - __ftrace_graph_entry = ftrace_graph_entry_stub; - ftrace_shutdown(&graph_ops, FTRACE_STOP_FUNC_RET); - unregister_pm_notifier(&ftrace_suspend_notifier); - unregister_trace_sched_switch(ftrace_graph_probe_sched_switch, NULL); + for (i = 0; i < fgraph_array_cnt; i++) + if (gops == fgraph_array[i]) + break; + if (i >= fgraph_array_cnt) + goto out; + fgraph_array[i] = &fgraph_stub; + if (i + 1 == fgraph_array_cnt) { + for (; i >= 0; i--) + if (fgraph_array[i] != &fgraph_stub) + break; + fgraph_array_cnt = i + 1; + } + + ftrace_graph_active--; + if (!ftrace_graph_active) { + ftrace_graph_return = ftrace_stub_graph; + ftrace_graph_entry = ftrace_graph_entry_stub; + __ftrace_graph_entry = ftrace_graph_entry_stub; + ftrace_shutdown(&graph_ops, FTRACE_STOP_FUNC_RET); + unregister_pm_notifier(&ftrace_suspend_notifier); + unregister_trace_sched_switch(ftrace_graph_probe_sched_switch, NULL); + } out: mutex_unlock(&ftrace_lock); }