From patchwork Fri Dec 8 10:24:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 17749 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp5361927vqy; Fri, 8 Dec 2023 02:24:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IEzdstYapkEejp9e9lzvhoDzsxI7+q8ow34dWCgPOO6ccw5brJTb3F3JtqZSGre8Ym/qxEa X-Received: by 2002:a17:902:f686:b0:1d0:ba41:1ec with SMTP id l6-20020a170902f68600b001d0ba4101ecmr4773973plg.49.1702031071813; Fri, 08 Dec 2023 02:24:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702031071; cv=none; d=google.com; s=arc-20160816; b=TKgyiaTVZ7cx0FGJVaQQt9Lyth5FCBxYw/HUGPYDZ/cq8RgLmjp6LGW16Ghb7ftIik vOAJ3O7gzranlYvI3z5jBWOwJs8K1bkHC3FLcHhIVk9a0XqUJGOcF14lkoYeNoaArfCq 7rRu/u4x/J82pvdIKX16HenDbMnazFMbnkLQW2a3CAwnZkTVBJPeU8+yFyRm+c75HT6K u4CgjyWsnV4VrEOpLuemAx+3HjpsuVCh9VwK0GoNJfZxlbjIlcOMOkRTbiog0bAIFkQ9 HJ5InR67NlDgY24/boEc7mI2dRkjdZUBxUn7Cum10HsxeRrR3GZHZ4LQV4IQjA6JCJwM 1t+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:message-id:date:subject:cc:to:from:dkim-signature; bh=ekxfMaYehKVMcWHL8iQqMN5t1QyoQWbKkHHxvJMnITs=; fh=SIgps5XdV0XNwjZfT2uAI7g3mrspDldK9Qs8qQAfoa4=; b=IpL4AHIkfGQ+hZ8kbODAB47h6T1a4C1ltTyeGfMC+ylzQX35es9FRInHTVY0UwpVMn RerO5UjSFbeAR0M9Y/gmQTSlvNEfx0wnuIIG1q/+zxhLVXw7Kxc9rcltrXtN+cEWGgeG 3mnErFIWr5mXHAI2cthe33RLpk/rZ7JnBmKa42XId5vXuQtevZcdmBESbRnUHB+VVYzx aiSxdBIn9xxcNE4C2Nq+eBdvDvbiAkhZvstyFdCgpzi+U9MeqvRK82g+7mM4cgnUDVik B6hgfd1SnZMtyFST1R/6/I3Thax55tz5qpA4IYmn67H8Q0ZoHi+5YK9j3Z1dsV0YD8qj L2Rw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qHfBMWiZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id v12-20020a170902b7cc00b001d08d307f89si1352512plz.496.2023.12.08.02.24.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Dec 2023 02:24:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qHfBMWiZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 6B05D80F66A1; Fri, 8 Dec 2023 02:24:28 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1573490AbjLHKYT (ORCPT + 99 others); Fri, 8 Dec 2023 05:24:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1573453AbjLHKYQ (ORCPT ); Fri, 8 Dec 2023 05:24:16 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B51611708 for ; Fri, 8 Dec 2023 02:24:22 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 61317C433C8; Fri, 8 Dec 2023 10:24:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702031062; bh=AbXWhwaHZjGBblilfPN9RBN+R9mqoytPii3caQypaKE=; h=From:To:Cc:Subject:Date:From; b=qHfBMWiZMLkYYyGnpR0FFdkm3BSPPqJkmoPstC2j0s8l+dKu/cAWLWWxIwGeX1EHQ LcVwZN6f5PG5UosfYN3YCDxJihWTptVO5zoYD2qkU/+t+OP/JUE3ilGVTpXL1478ZX EcdLPREFZFq19W7Ym+a5rHj8g80WdOsR/MAW8yDfLCCeQ049U30oGaVGqwAhHqJKTL Xeq6NZVgT/FoIXSxPC9qUtnry4qdq1RTjG3m4dKTnD/a9674YlvHnLO5aAD5W4PeZG Bp2uBB+4NJMV1DK9G+ErwwmQ0EBpB9EAzG5dFy7ebpEPcPZBe4PTgRXghScZELpweH 6YM54n/RLAQJw== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner , Guo Ren Subject: [PATCH v4 00/33] tracing: fprobe: function_graph: Multi-function graph and fprobe on fgraph Date: Fri, 8 Dec 2023 19:24:15 +0900 Message-Id: <170203105427.579004.8033550792660734570.stgit@devnote2> X-Mailer: git-send-email 2.34.1 User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 08 Dec 2023 02:24:28 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784708933091030781 X-GMAIL-MSGID: 1784708933091030781 Hi, Here is the 4th version of the series to re-implement the fprobe on function-graph tracer. The previous version is; https://lore.kernel.org/all/170109317214.343914.4784420430328654397.stgit@devnote2/ This version is rebased on "trace-v6.7-rc4" tag on linux-trace tree and fix some issues on the new multiple function graph tracer implementation[11/33]. I also add a simple LRU so that the recently released fgraph_array index is not reused soon [12/33]. Overview -------- This series does major 2 changes, enable multiple function-graphs on the ftrace (e.g. allow function-graph on sub instances) and rewrite the fprobe on this function-graph. The former changes had been sent from Steven Rostedt 4 years ago (*), which allows users to set different setting function-graph tracer (and other tracers based on function-graph) in each trace-instances at the same time. (*) https://lore.kernel.org/all/20190525031633.811342628@goodmis.org/ The purpose of latter change are; 1) Remove dependency of the rethook from fprobe so that we can reduce the return hook code and shadow stack. 2) Make 'ftrace_regs' the common trace interface for the function boundary. 1) Currently we have 2(or 3) different function return hook codes, the function-graph tracer and rethook (and legacy kretprobe). But since this is redundant and needs double maintenance cost, I would like to unify those. From the user's viewpoint, function- graph tracer is very useful to grasp the execution path. For this purpose, it is hard to use the rethook in the function-graph tracer, but the opposite is possible. (Strictly speaking, kretprobe can not use it because it requires 'pt_regs' for historical reasons.) 2) Now the fprobe provides the 'pt_regs' for its handler, but that is wrong for the function entry and exit. Moreover, depending on the architecture, there is no way to accurately reproduce 'pt_regs' outside of interrupt or exception handlers. This means fprobe should not use 'pt_regs' because it does not use such exceptions. (Conversely, kprobe should use 'pt_regs' because it is an abstract interface of the software breakpoint exception.) This series changes fprobe to use function-graph tracer for tracing function entry and exit, instead of mixture of ftrace and rethook. Unlike the rethook which is a per-task list of system-wide allocated nodes, the function graph's ret_stack is a per-task shadow stack. Thus it does not need to set 'nr_maxactive' (which is the number of pre-allocated nodes). Also the handlers will get the 'ftrace_regs' instead of 'pt_regs'. Since eBPF mulit_kprobe/multi_kretprobe events still use 'pt_regs' as their register interface, this changes it to convert 'ftrace_regs' to 'pt_regs'. Of course this conversion makes an incomplete 'pt_regs', so users must access only registers for function parameters or return value. Design ------ Instead of using ftrace's function entry hook directly, the new fprobe is built on top of the function-graph's entry and return callbacks with 'ftrace_regs'. Since the fprobe requires access to 'ftrace_regs', the architecture must support CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS, which enables to call function-graph entry callback with 'ftrace_regs', and also CONFIG_HAVE_FUNCTION_GRAPH_FREGS, which passes the ftrace_regs to return_to_handler. All fprobes share a single function-graph ops (means shares a common ftrace filter) similar to the kprobe-on-ftrace. This needs another layer to find corresponding fprobe in the common function-graph callbacks, but has much better scalability, since the number of registered function-graph ops is limited. In the entry callback, the fprobe runs its entry_handler and saves the address of 'fprobe' on the function-graph's shadow stack as data. The return callback decodes the data to get the 'fprobe' address, and runs the exit_handler. The fprobe introduces two hash-tables, one is for entry callback which searches fprobes related to the given function address passed by entry callback. The other is for a return callback which checks if the given 'fprobe' data structure pointer is still valid. Note that it is possible to unregister fprobe before the return callback runs. Thus the address validation must be done before using it in the return callback. Series ------ - Patch [1/33] and [2/33] are adding a comment for ftrace_regs. - Patch [3/33] to [18/33] are the multiple function-graph support. - Patch [19/33] and [20/33] adds new function-graph callbacks with ftrace_regs and x86-64 implementation. - Patch [21/33] to [27/33] are preparation (adding util functions) of the new fprobe and its user. - Patch [28/33] to [32/33] rewrites fprobes and updates its users. - Patch [33/33] is a documentation update. This series can be applied against the trace-v6.7-rc4 on linux-trace tree. This series can also be found below branch. https://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git/log/?h=topic/fprobe-on-fgraph Thank you, --- Masami Hiramatsu (Google) (18): tracing: Add a comment about ftrace_regs definition x86: tracing: Add ftrace_regs definition in the header function_graph: Use a simple LRU for fgraph_array index number function_graph: Add a new entry handler with parent_ip and ftrace_regs function_graph: Add a new exit handler with parent_ip and ftrace_regs x86/ftrace: Enable HAVE_FUNCTION_GRAPH_FREGS tracing: Rename ftrace_regs_return_value to ftrace_regs_get_return_value arm64: ftrace: Enable HAVE_FUNCTION_GRAPH_FREGS fprobe: Use ftrace_regs in fprobe entry handler fprobe: Use ftrace_regs in fprobe exit handler tracing: Add ftrace_partial_regs() for converting ftrace_regs to pt_regs tracing: Add ftrace_fill_perf_regs() for perf event fprobe: Rewrite fprobe on function-graph tracer tracing/fprobe: Remove nr_maxactive from fprobe tracing/fprobe: Enable fprobe events with CONFIG_DYNAMIC_FTRACE_WITH_ARGS bpf: Enable kprobe_multi feature if CONFIG_FPROBE is enabled selftests: ftrace: Remove obsolate maxactive syntax check Documentation: probes: Update fprobe on function-graph tracer Steven Rostedt (VMware) (15): function_graph: Convert ret_stack to a series of longs fgraph: Use BUILD_BUG_ON() to make sure we have structures divisible by long function_graph: Add an array structure that will allow multiple callbacks function_graph: Allow multiple users to attach to function graph function_graph: Remove logic around ftrace_graph_entry and return ftrace/function_graph: Pass fgraph_ops to function graph callbacks ftrace: Allow function_graph tracer to be enabled in instances ftrace: Allow ftrace startup flags exist without dynamic ftrace function_graph: Have the instances use their own ftrace_ops for filtering function_graph: Add "task variables" per task for fgraph_ops function_graph: Move set_graph_function tests to shadow stack global var function_graph: Move graph depth stored data to shadow stack global var function_graph: Move graph notrace bit to shadow stack global var function_graph: Implement fgraph_reserve_data() and fgraph_retrieve_data() function_graph: Add selftest for passing local variables Documentation/trace/fprobe.rst | 42 + arch/arm64/Kconfig | 2 arch/arm64/include/asm/ftrace.h | 24 arch/arm64/kernel/entry-ftrace.S | 28 + arch/arm64/kernel/ftrace.c | 19 arch/loongarch/Kconfig | 1 arch/loongarch/include/asm/ftrace.h | 2 arch/loongarch/kernel/ftrace_dyn.c | 6 arch/powerpc/include/asm/ftrace.h | 9 arch/powerpc/kernel/trace/ftrace.c | 2 arch/powerpc/kernel/trace/ftrace_64_pg.c | 10 arch/s390/Kconfig | 1 arch/s390/include/asm/ftrace.h | 7 arch/x86/Kconfig | 4 arch/x86/include/asm/ftrace.h | 17 arch/x86/kernel/ftrace.c | 51 + arch/x86/kernel/ftrace_64.S | 37 + include/linux/fprobe.h | 58 + include/linux/ftrace.h | 167 +++ include/linux/sched.h | 2 include/linux/trace_recursion.h | 39 - kernel/trace/Kconfig | 19 kernel/trace/bpf_trace.c | 14 kernel/trace/fgraph.c | 1003 ++++++++++++++++---- kernel/trace/fprobe.c | 631 +++++++++---- kernel/trace/ftrace.c | 13 kernel/trace/ftrace_internal.h | 2 kernel/trace/trace.h | 94 ++ kernel/trace/trace_fprobe.c | 114 +- kernel/trace/trace_functions.c | 8 kernel/trace/trace_functions_graph.c | 96 +- kernel/trace/trace_irqsoff.c | 10 kernel/trace/trace_probe_tmpl.h | 2 kernel/trace/trace_sched_wakeup.c | 10 kernel/trace/trace_selftest.c | 178 +++- lib/test_fprobe.c | 51 - samples/fprobe/fprobe_example.c | 4 .../ftrace/test.d/dynevent/fprobe_syntax_errors.tc | 4 38 files changed, 2082 insertions(+), 699 deletions(-) -- Masami Hiramatsu (Google)