From patchwork Sat Aug 12 05:36:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134816 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1536803vqi; Fri, 11 Aug 2023 22:47:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGe3RZVzRo1JBu1V+7Irpqvfqc7KW6oHqpc16uOEg2hw/PvWWFOSxQq7ID11TiLJIW/hkq9 X-Received: by 2002:a17:907:b1a:b0:99c:e38d:a824 with SMTP id h26-20020a1709070b1a00b0099ce38da824mr3082723ejl.37.1691819256570; Fri, 11 Aug 2023 22:47:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691819256; cv=none; d=google.com; s=arc-20160816; b=flaeTXug7uqS4AESto/zoBRRsIEMEZIoVri1IOtxUxO3Q4NRJaz7ZMpIB/97V3/lym cyeGlx90Qbbj1zpI6/JXDa/L32I9A5v7hQjSTTOmBJcEd0IO1RyfmnyaKaIrYCRRylJC wWSuzAdDlKoAa7rHcILc6Y8cQ6iQ1+U1VFVNukjJAuccoMyw2XoU6N9AGIrFwxHHoLN1 7jZXSjSorXBpgk2ugmnIwx3/vgP1WgYgwvbk7dSn6D0pcisoYIj1q11BU+jXJZpsrD86 +hur94lCbLe6liYGBur/eEj0O6H4JcQZW8PIg2wjjitVkW15LzsbcsWYQ8beagiPQy3J Ujqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=wUhRWi46TgpGHrD08iF4tDbJawrcCfYsFt4z2PuaRXY=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=PUwvmqO3RjEsvrOgks9rFQIBEwldEwWU72pPkiIlc8/lrmEqhM4AZ2uW8tY5iXtemg iUOniTWElmhLnxr/lyTtbmrWENbPYOGvhAMXq1U0hu5Aez+B9TplHlAEnKH/gwYZyEb1 DW1Dytv9mW/4SffOShmzFFplNQGLANdfKKC/yTxcs+gV9hCpk6Lqm8kxTT4QNECGAtHa LJuY5swajVra3V+clLFQYhsC5RjhKUCvo8p0qnkWU7GL62le27SodRC+7YoZVtW0bKwh 4q+gcgfP5AHtm0eOMUdwFTbuD2LLgCUnt+RS3vNEZeDY3ovzU82R/Yya2BGk3imgzhRf UjNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=GrCcvdTy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xa1-20020a170907b9c100b009930e13beb8si4773848ejc.952.2023.08.11.22.47.12; Fri, 11 Aug 2023 22:47:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=GrCcvdTy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235898AbjHLFg7 (ORCPT + 99 others); Sat, 12 Aug 2023 01:36:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235893AbjHLFg4 (ORCPT ); Sat, 12 Aug 2023 01:36:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C111E30C1; Fri, 11 Aug 2023 22:36:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 544AC63FC8; Sat, 12 Aug 2023 05:36:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 337D9C433C8; Sat, 12 Aug 2023 05:36:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818613; bh=blEGHkYpfxZ/1E7WBPNHsUXM+7p3REUp7I8zSsGXqWg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GrCcvdTy2YsEo6/vYNC3BwLrflolqioYyx+EYQOJp/1Pz/a5YY3sXMpalYrFs0R0j JUIQSw6pgmUv7Tfe9wyv74ePTOsgjbJ3zwxZV2V0HsR2T0KIDU+TKt4kyMg3cTp+X5 sgQY9Q3NR8p/0zQgnqUQ2qwfpMYUr4xvNEMNKfoliizh9wDQVmxeP/xLrtDBOiwR8v C8h96HzSoorF4DAxA+c4aPR2TZCDrAJ++rqFMe+xEY2Sb4pKSIXPsm1i1atzc2+VcK ie96vOscAPAGsPQPlipmD8M4ucicOAATf3OSw7n6GFcAI1LuilADu3ZHxycQ/at05l HJLuWZRu2LpQA== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 1/8] Documentation: probes: Add a new ret_ip callback parameter Date: Sat, 12 Aug 2023 14:36:47 +0900 Message-Id: <169181860742.505132.14215380532909911090.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774001069016413374 X-GMAIL-MSGID: 1774001069016413374 From: Masami Hiramatsu (Google) Add a new ret_ip callback parameter description. Fixes: cb16330d1274 ("fprobe: Pass return address to the handlers") Signed-off-by: Masami Hiramatsu (Google) --- Documentation/trace/fprobe.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/Documentation/trace/fprobe.rst b/Documentation/trace/fprobe.rst index 40dd2fbce861..a6d682478147 100644 --- a/Documentation/trace/fprobe.rst +++ b/Documentation/trace/fprobe.rst @@ -91,9 +91,9 @@ The prototype of the entry/exit callback function are as follows: .. code-block:: c - int entry_callback(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs, void *entry_data); + int entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, void *entry_data); - void exit_callback(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs, void *entry_data); + void exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, void *entry_data); Note that the @entry_ip is saved at function entry and passed to exit handler. If the entry callback function returns !0, the corresponding exit callback will be cancelled. @@ -108,6 +108,10 @@ If the entry callback function returns !0, the corresponding exit callback will Note that this may not be the actual entry address of the function but the address where the ftrace is instrumented. +@ret_ip + This is the return address of the traced function. This can be used + at both entry and exit. + @regs This is the `pt_regs` data structure at the entry and exit. Note that the instruction pointer of @regs may be different from the @entry_ip From patchwork Sat Aug 12 05:36:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134821 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1545477vqi; Fri, 11 Aug 2023 23:12:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IENCEfpHh00BXDamYptsZlv7flBzwuALFnQY9D9TqF26QtGONewDYZSv1MobX71Zn+L7f9D X-Received: by 2002:a05:6808:14d1:b0:3a7:1478:50a0 with SMTP id f17-20020a05680814d100b003a7147850a0mr5661138oiw.46.1691820754295; Fri, 11 Aug 2023 23:12:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691820754; cv=none; d=google.com; s=arc-20160816; b=J2l3ibPiL+E+AKJSrTy+r1d8acKPKSQ3Pk/asJcG5GqIjrfWN2aC9cMJMa+hFb3F0e AyzJ9ai8+c3dAJdIolkdMDZKKlZFC5GXjxihmnZwNdJ+CLEOdsOA0X+HncCejiYTEks/ K7+tMPji5y5ake5lhpmj8E9i1BY0fSsHGK5YuSG5YmPT4UKGX6q/ZkSaF0X114uawCFZ IOH8TunRLauX1GhDEPfkS6wm4LRrz5zshysHmLaLoNaoN4hErqL0WF0eY3VynGDid2VX uu0LBhPWiTAVRF48jV3x4EURCLTY773D77x8mbfCozQdIPOG2kh9cRY0YlhTahZ83u4T vtEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=vLgwA3+a6UykiZqoZYNHBahXu8Iv1addzFJOLNXn++k=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=UED3aypnia+kscw9xhABNwjiYwP6UEm3WbDdt7Q7RK33s9DuTcKAp7Mxj/9mqH/mtV zTjySd6pfgGUQiVCwa8+YFyUdz7lks0eq7UTT5r5j/urdsaLBic6BxFPd1/cb/EK0Jpn AACAs+MHELQ5edisWNoNGQQVMz72yqSfba+evKHErkHKGy1vx+A/QQst/enaLCXIntv5 3Nq/kMQfGNbruW3UNP2/2Ti0laHz+6S9GQEJJ4PlK//07wpuPMDwXm76VTmB9B0IeXNc c8MiQEo/I/sS+OYl+lLjP2KyB0Kz+l9Z7sFaVnxwk8BkDN4A/HFj7qRPv2hSpo80od5n dr2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Uq0c7qn1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bg26-20020a056a02011a00b0051b90567f99si4462678pgb.690.2023.08.11.23.12.21; Fri, 11 Aug 2023 23:12:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Uq0c7qn1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235273AbjHLFhJ (ORCPT + 99 others); Sat, 12 Aug 2023 01:37:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230072AbjHLFhH (ORCPT ); Sat, 12 Aug 2023 01:37:07 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7CB62D52; Fri, 11 Aug 2023 22:37:06 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6753663FDA; Sat, 12 Aug 2023 05:37:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E98FC433C7; Sat, 12 Aug 2023 05:37:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818625; bh=4qPUZlUTFHSQi/UelkxI/UnG2AeW5ahcJbFTYeUciiM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Uq0c7qn155A+dPZs9LWUvRezDgvEEvskGy2s1eGHvyaC+UIrZMrKtrfGEtz+r9C9g Qcgr5QypW/GoRWya2vyZpaRaenSD5iD8TN36lQO7urxCoLAWGH+jZB/c2Vjz3Rv9rn GcI0DbwKKkwzo5IQ6YPGU4gXorXG45Ci9CEqggSjB0ubmJI8HirXSJjfR4uyV/pEa2 ITTnC6U0IRcKONVZAyQS/OfXQ+iqW+IhnPuMtj/yIspHqhBU2ZWFSKfq6SbAGjF6be I8IMksN/A813oOqLpI/YtIrn0HJGP23rEVkH4lN0MYPgoKCPsnDX2Cj0i4oxIYIF9a fs/ihMbkwST4g== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 2/8] fprobe: Use fprobe_regs in fprobe entry handler Date: Sat, 12 Aug 2023 14:36:59 +0900 Message-Id: <169181861911.505132.8322840504168319403.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774002639096500158 X-GMAIL-MSGID: 1774002639096500158 From: Masami Hiramatsu (Google) This allows fprobes to be available with CONFIG_DYNAMIC_FTRACE_WITH_ARGS instead of CONFIG_DYNAMIC_FTRACE_WITH_REGS, then we can enable fprobe on arm64. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v3: - Use FTRACE_OPS_FL_SAVE_ARGS instead of FTRACE_OPS_FL_SAVE_REGS. --- include/linux/fprobe.h | 2 +- kernel/trace/Kconfig | 3 ++- kernel/trace/bpf_trace.c | 10 +++++++--- kernel/trace/fprobe.c | 4 ++-- kernel/trace/trace_fprobe.c | 6 +++++- lib/test_fprobe.c | 4 ++-- samples/fprobe/fprobe_example.c | 2 +- 7 files changed, 20 insertions(+), 11 deletions(-) diff --git a/include/linux/fprobe.h b/include/linux/fprobe.h index 3e03758151f4..36c0595f7b93 100644 --- a/include/linux/fprobe.h +++ b/include/linux/fprobe.h @@ -35,7 +35,7 @@ struct fprobe { int nr_maxactive; int (*entry_handler)(struct fprobe *fp, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *regs, void *entry_data); void (*exit_handler)(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 61c541c36596..976fd594b446 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -287,7 +287,7 @@ config DYNAMIC_FTRACE_WITH_ARGS config FPROBE bool "Kernel Function Probe (fprobe)" depends on FUNCTION_TRACER - depends on DYNAMIC_FTRACE_WITH_REGS + depends on DYNAMIC_FTRACE_WITH_REGS || DYNAMIC_FTRACE_WITH_ARGS depends on HAVE_RETHOOK select RETHOOK default n @@ -672,6 +672,7 @@ config FPROBE_EVENTS select TRACING select PROBE_EVENTS select DYNAMIC_EVENTS + depends on DYNAMIC_FTRACE_WITH_REGS default y help This allows user to add tracing events on the function entry and diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index bd1a42b23f3f..126acbfdb26f 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2467,7 +2467,7 @@ static int __init bpf_event_init(void) fs_initcall(bpf_event_init); #endif /* CONFIG_MODULES */ -#ifdef CONFIG_FPROBE +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS struct bpf_kprobe_multi_link { struct bpf_link link; struct fprobe fp; @@ -2659,10 +2659,14 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, static int kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *data) { struct bpf_kprobe_multi_link *link; + struct pt_regs *regs = ftrace_get_regs(fregs); + + if (!regs) + return 0; link = container_of(fp, struct bpf_kprobe_multi_link, fp); kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs); @@ -2917,7 +2921,7 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr kvfree(cookies); return err; } -#else /* !CONFIG_FPROBE */ +#else /* !CONFIG_DYNAMIC_FTRACE_WITH_REGS */ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { return -EOPNOTSUPP; diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index 3b21f4063258..07deb52df44a 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -46,7 +46,7 @@ static inline void __fprobe_handler(unsigned long ip, unsigned long parent_ip, } if (fp->entry_handler) - ret = fp->entry_handler(fp, ip, parent_ip, ftrace_get_regs(fregs), entry_data); + ret = fp->entry_handler(fp, ip, parent_ip, fregs, entry_data); /* If entry_handler returns !0, nmissed is not counted. */ if (rh) { @@ -182,7 +182,7 @@ static void fprobe_init(struct fprobe *fp) fp->ops.func = fprobe_kprobe_handler; else fp->ops.func = fprobe_handler; - fp->ops.flags |= FTRACE_OPS_FL_SAVE_REGS; + fp->ops.flags |= FTRACE_OPS_FL_SAVE_ARGS; } static int fprobe_init_rethook(struct fprobe *fp, int num) diff --git a/kernel/trace/trace_fprobe.c b/kernel/trace/trace_fprobe.c index dfe2e546acdc..4d3ae79f036e 100644 --- a/kernel/trace/trace_fprobe.c +++ b/kernel/trace/trace_fprobe.c @@ -320,12 +320,16 @@ NOKPROBE_SYMBOL(fexit_perf_func); #endif /* CONFIG_PERF_EVENTS */ static int fentry_dispatcher(struct fprobe *fp, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data) { struct trace_fprobe *tf = container_of(fp, struct trace_fprobe, fp); + struct pt_regs *regs = ftrace_get_regs(fregs); int ret = 0; + if (!regs) + return 0; + if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) fentry_trace_func(tf, entry_ip, regs); #ifdef CONFIG_PERF_EVENTS diff --git a/lib/test_fprobe.c b/lib/test_fprobe.c index 24de0e5ff859..ff607babba18 100644 --- a/lib/test_fprobe.c +++ b/lib/test_fprobe.c @@ -40,7 +40,7 @@ static noinline u32 fprobe_selftest_nest_target(u32 value, u32 (*nest)(u32)) static notrace int fp_entry_handler(struct fprobe *fp, unsigned long ip, unsigned long ret_ip, - struct pt_regs *regs, void *data) + struct ftrace_regs *fregs, void *data) { KUNIT_EXPECT_FALSE(current_test, preemptible()); /* This can be called on the fprobe_selftest_target and the fprobe_selftest_target2 */ @@ -81,7 +81,7 @@ static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, static notrace int nest_entry_handler(struct fprobe *fp, unsigned long ip, unsigned long ret_ip, - struct pt_regs *regs, void *data) + struct ftrace_regs *fregs, void *data) { KUNIT_EXPECT_FALSE(current_test, preemptible()); return 0; diff --git a/samples/fprobe/fprobe_example.c b/samples/fprobe/fprobe_example.c index 64e715e7ed11..1545a1aac616 100644 --- a/samples/fprobe/fprobe_example.c +++ b/samples/fprobe/fprobe_example.c @@ -50,7 +50,7 @@ static void show_backtrace(void) static int sample_entry_handler(struct fprobe *fp, unsigned long ip, unsigned long ret_ip, - struct pt_regs *regs, void *data) + struct ftrace_regs *fregs, void *data) { if (use_trace) /* From patchwork Sat Aug 12 05:37:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134823 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1546795vqi; Fri, 11 Aug 2023 23:16:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHJ6zKXDz1aA3B21VfCGM9QxWwKUyG+ErhpxSsReF9MYTDgeX9JbBbmR11HQhTNNdFqnr8y X-Received: by 2002:a05:6e02:13c5:b0:348:dd48:e9d4 with SMTP id v5-20020a056e0213c500b00348dd48e9d4mr4824817ilj.1.1691820985613; Fri, 11 Aug 2023 23:16:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691820985; cv=none; d=google.com; s=arc-20160816; b=y7k4rH8+RTpy5sX62/9kQ8x0Y22Fb1oHqqAR9U6XtaYLV4P5e8xFcDw5F2Jtw+8s2F 8iu1Ho9sC14aY/EUALSnky1d6VMiSi+VRK5oM7WcXNJMsl/1BakFDd5JI5hS9C5bB6IH BVeF6dQ2kgB1AfFZZsb7kXeTqXoyFeXkwJHwE+okDFlZr4CUDg6wOhijlTMrIw0Aut6B ZIhavjihqIp3eIQiSZRGyyT9fZRuLdm6S96/71wX6K9xgYEBSZrWrA55Q5QtxJJOxX6o ApI1tLuVHQI6zFKu6dolp9u0iuWdkirKYkvpl8/lR7bnERV/TwasLrXHP18DTBgwI5DK +qhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=cIBQpU2HjHQ7YOB7onuhJu81hNk4igKzwKYv2Jv8MaA=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=q8TtxVLVIl4z2Oral08flGZMhnHIKOe9TJuApJfL2PR82u6XVKgEzlFD/VqxI29ID8 ocXHGwkVHi5iJTdUe5XJ1xFPPBTF+l8R6fWA80oT0i5VRAix6KDv/9KFkpCFMwcNtC1/ qwsywSq9S+DguimOMFXe/5Kse5Ct+9V4CAEnoxpWhh8z0oAwEpSMIICxKZ92uYZxZUTY DyyneQsjIaAJLnAaFoW7+9dQDWOpYMDAWuhm+gW9deeJ8O9xmM/VmiSuoXoEotaLUp68 l4daeLkzqzYZu2KoRBG17wlo7KbMSbLaz5wpotNkjAR4jcpcDF0+6wSjWrKwm+QIf0Jr Gv0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=T8QpAqBU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 185-20020a6300c2000000b005648fc8632asi4399646pga.432.2023.08.11.23.16.12; Fri, 11 Aug 2023 23:16:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=T8QpAqBU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235902AbjHLFhU (ORCPT + 99 others); Sat, 12 Aug 2023 01:37:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230072AbjHLFhT (ORCPT ); Sat, 12 Aug 2023 01:37:19 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E41A42D5B; Fri, 11 Aug 2023 22:37:18 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7B5D96405A; Sat, 12 Aug 2023 05:37:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11119C433C7; Sat, 12 Aug 2023 05:37:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818637; bh=idoIszjUW0P8tm6vGK/1+jb2ZV7Bg8zd/p0KkM3vUyo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T8QpAqBUkuiM8RM10CjO0bt6F+snrisBlXE96ECG4iI7Sklb0ivWpTnhtT34R4+mJ O9GxVbEbXVSDQQnr7kMWSewhwrvuBaNn3hM4qhlsIScmNC9B8hkH3X9N3OrVcFQ4aS PcJTEabk6rqtNWjvZMDa42EQ+NiKF3C+YPrZlYKaxx1zWWvZTE6nbYg43ULi5GOgQB mFUMdD94N66E7kutMiEt4xzugICbCWFsE5r32PgDcj0p88k7EUIdFP6/BWEd0stHvB m6BcyjWwM3EUT70bOuy0v4Ir1zV2pBKGpL6NLmxOzbFVkEL4E3BOZYreaVyBUm2PyR ZA0NJRoP2bkNw== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 3/8] tracing: Expose ftrace_regs regardless of CONFIG_FUNCTION_TRACER Date: Sat, 12 Aug 2023 14:37:11 +0900 Message-Id: <169181863118.505132.13233554057378608176.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774002881672730412 X-GMAIL-MSGID: 1774002881672730412 From: Masami Hiramatsu (Google) In order to be able to use ftrace_regs even from features unrelated to function tracer (e.g. kretprobe), expose ftrace_regs structures and APIs even if the CONFIG_FUNCTION_TRACER=n. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v3: - arch/s390/include/asm/ftrace.h: hide ftrace_regs parts if CONFIG_FUNCTION_TRACER=n to avoid conflict. - Fixed typo. - Define accessor macros only if HAVE_REGS_AND_STACK_ACCESS_API=y --- arch/s390/include/asm/ftrace.h | 4 +++ include/linux/ftrace.h | 51 +++++++++++++++++++++++----------------- 2 files changed, 33 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm/ftrace.h b/arch/s390/include/asm/ftrace.h index e5c5cb1207e2..74cd2e8bf660 100644 --- a/arch/s390/include/asm/ftrace.h +++ b/arch/s390/include/asm/ftrace.h @@ -41,6 +41,8 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr) return addr; } +#ifdef CONFIG_FUNCTION_TRACER + struct ftrace_regs { struct pt_regs regs; }; @@ -80,6 +82,8 @@ ftrace_regs_set_instruction_pointer(struct ftrace_regs *fregs, #define ftrace_regs_query_register_offset(name) \ regs_query_register_offset(name) +#endif /* CONFIG_FUNCTION_TRACER */ + #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS /* * When an ftrace registered caller is tracing a function that is diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index aad9cf8876b5..fe335d861f08 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -112,24 +112,44 @@ static inline int ftrace_mod_get_kallsym(unsigned int symnum, unsigned long *val } #endif -#ifdef CONFIG_FUNCTION_TRACER - -extern int ftrace_enabled; - -#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS +/* + * If the architecture doesn't support FTRACE_WITH_ARGS or disables function + * tracer, define the default(pt_regs compatible) ftrace_regs. + */ +#if !defined(CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS) || !defined(CONFIG_FUNCTION_TRACER) struct ftrace_regs { struct pt_regs regs; }; #define arch_ftrace_get_regs(fregs) (&(fregs)->regs) +#ifdef CONFIG_HAVE_REGS_AND_STACK_ACCESS_API + /* * ftrace_regs_set_instruction_pointer() is to be defined by the architecture * if to allow setting of the instruction pointer from the ftrace_regs when * HAVE_DYNAMIC_FTRACE_WITH_ARGS is set and it supports live kernel patching. */ #define ftrace_regs_set_instruction_pointer(fregs, ip) do { } while (0) -#endif /* CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS */ + +#define ftrace_regs_get_instruction_pointer(fregs) \ + instruction_pointer(ftrace_get_regs(fregs)) +#define ftrace_regs_get_argument(fregs, n) \ + regs_get_kernel_argument(ftrace_get_regs(fregs), n) +#define ftrace_regs_get_stack_pointer(fregs) \ + kernel_stack_pointer(ftrace_get_regs(fregs)) +#define ftrace_regs_return_value(fregs) \ + regs_return_value(ftrace_get_regs(fregs)) +#define ftrace_regs_set_return_value(fregs, ret) \ + regs_set_return_value(ftrace_get_regs(fregs), ret) +#define ftrace_override_function_with_return(fregs) \ + override_function_with_return(ftrace_get_regs(fregs)) +#define ftrace_regs_query_register_offset(name) \ + regs_query_register_offset(name) + +#endif /* CONFIG_HAVE_REGS_AND_STACK_ACCESS_API */ + +#endif /* !CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS || !CONFIG_FUNCTION_TRACER */ static __always_inline struct pt_regs *ftrace_get_regs(struct ftrace_regs *fregs) { @@ -151,22 +171,9 @@ static __always_inline bool ftrace_regs_has_args(struct ftrace_regs *fregs) return ftrace_get_regs(fregs) != NULL; } -#ifndef CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS -#define ftrace_regs_get_instruction_pointer(fregs) \ - instruction_pointer(ftrace_get_regs(fregs)) -#define ftrace_regs_get_argument(fregs, n) \ - regs_get_kernel_argument(ftrace_get_regs(fregs), n) -#define ftrace_regs_get_stack_pointer(fregs) \ - kernel_stack_pointer(ftrace_get_regs(fregs)) -#define ftrace_regs_return_value(fregs) \ - regs_return_value(ftrace_get_regs(fregs)) -#define ftrace_regs_set_return_value(fregs, ret) \ - regs_set_return_value(ftrace_get_regs(fregs), ret) -#define ftrace_override_function_with_return(fregs) \ - override_function_with_return(ftrace_get_regs(fregs)) -#define ftrace_regs_query_register_offset(name) \ - regs_query_register_offset(name) -#endif +#ifdef CONFIG_FUNCTION_TRACER + +extern int ftrace_enabled; typedef void (*ftrace_func_t)(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *op, struct ftrace_regs *fregs); From patchwork Sat Aug 12 05:37:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134822 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1546550vqi; Fri, 11 Aug 2023 23:15:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHF6UF3J9PnHGJdHz+KyC4ng20aKjH3PXMZ7rnDYP6LetAZZfq+1k7B42rAFBl2O8ak8loE X-Received: by 2002:a05:6358:9041:b0:139:9fc3:eade with SMTP id f1-20020a056358904100b001399fc3eademr5262891rwf.29.1691820949405; Fri, 11 Aug 2023 23:15:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691820949; cv=none; d=google.com; s=arc-20160816; b=buSj53SeDgeetE+Bo4ywI4f4YfGUtRp+3btMr2ecYZxzYbqbpG1z2WmIGDQSEs9OYm GiN+S65lbQs/6KTMehiuAbQnI9lEm8mNY71hki8pzJP4yFn5XwnP6xw+Qkt7fFiFebs9 Jh4CLo+vL0QLZbvyxAKXMWpQBIwSoK4f5hPjPcFAq0n0p5jgFm0qcb1xx9uPw1Ef2xmo l7Sh6Wp7pDoBvpAbtY3IPcPEK56R44qsIIB7lsId1JKtwUVh4+jock7B4APEMZNaSLyR jMYeurvDUg2vyhpq5DS1dYMRKV46Ft/edHtrr6XcVLCWNZ/HR0Qvz/6Tm2Sf4IOUYkco gpdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=uuTH/Sq6dJAVns3w/My5VByeQjL5nPaoYAkFGY5+hBY=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=vc9sbr4usbe6LzSTWmKF2DxvA82Fqb/k4H8KAiPAaG7ezX0WEDdEKAmyatWn6XMaXT fIpr9oqQm1LOExV7OHjZDG45aQzZfQhwcTpSa6VNq0opGHLS1WljkSXXmHNjh4W7VFiz YOAHHd8mpyonsTONAwjxcRz7Xz9wxgmTxtELOW3HCulsUpcT7t5zjA18hdXgvC1bmEjA x74EGgNbOX+DQbKEhrJRHvSrRi6irnKmywiFH6vZm2O9kuOBqg+NW30aCkz5qLO1pIL0 TuxQef2TXtZd8fVh+PPBJcCm6hZ/wIkw7A6amOCUSejiTlVzFsXWwaLZrIXh9xOE190L ZhoA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=nw9fkB2l; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l13-20020a170903244d00b001b9fef6bf58si4562474pls.294.2023.08.11.23.15.29; Fri, 11 Aug 2023 23:15:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=nw9fkB2l; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234317AbjHLFhe (ORCPT + 99 others); Sat, 12 Aug 2023 01:37:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230072AbjHLFhd (ORCPT ); Sat, 12 Aug 2023 01:37:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FDC22D5B; Fri, 11 Aug 2023 22:37:30 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2757964003; Sat, 12 Aug 2023 05:37:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25AC1C433C8; Sat, 12 Aug 2023 05:37:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818649; bh=c68NALMJak4Co/BDbjDneSEJf3H9w9lqyA/S/zyJbNo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nw9fkB2lgF3HDU8Jdn6kPbtINgw6PxjKZvLjVkWD6T7VEV6/LxgTgFetb/c6OX5gw Y6TvPFwVc28S+JfpLb86v5d86XgpvqWx8bV4/sDFERwgJkgti6xZG5NPwy05mKh8GW 3QEH0PfDL0fFXIYNKYuhCRGJ6/zpOQdhn/VYEcL3G3mNUD8YrGkerdw5QVMS6fytSj FWWBoNbcsr2W7+T4AQOqysupF10x2b7YwjZpT3SnonKZmH37lQbHR64fqWMj6xpsVQ mcle2lExnGUD8vDcE0rkJNkQmz/aF31XD6NkIuvtTf3l07D40mRm8DTRuvIY+0kTLp g52BuXTV3YTBw== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 4/8] fprobe: rethook: Use ftrace_regs in fprobe exit handler and rethook Date: Sat, 12 Aug 2023 14:37:23 +0900 Message-Id: <169181864347.505132.7098838654251139622.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774002843637874961 X-GMAIL-MSGID: 1774002843637874961 From: Masami Hiramatsu (Google) Change the fprobe exit handler and rethook to use ftrace_regs data structure instead of pt_regs. This also introduce HAVE_PT_REGS_TO_FTRACE_REGS_CAST which means the ftrace_regs's memory layout is equal to the pt_regs so that those are able to cast. Only if it is enabled, kretprobe will use rethook since kretprobe requires pt_regs for backward compatibility. This means the archs which currently implement rethook for kretprobes needs to set that flag and it must ensure struct ftrace_regs is same as pt_regs. If not, it must be either disabling kretprobe or implementing kretprobe trampoline separately from rethook trampoline. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v3: - Config rename to HAVE_PT_REGS_TO_FTRACE_REGS_CAST - Use ftrace_regs_* APIs instead of ftrace_get_regs(). --- arch/Kconfig | 1 + arch/loongarch/Kconfig | 1 + arch/s390/Kconfig | 1 + arch/x86/Kconfig | 1 + arch/x86/kernel/rethook.c | 13 +++++++------ include/linux/fprobe.h | 2 +- include/linux/rethook.h | 11 ++++++----- kernel/kprobes.c | 10 ++++++++-- kernel/trace/Kconfig | 7 +++++++ kernel/trace/bpf_trace.c | 6 +++++- kernel/trace/fprobe.c | 6 +++--- kernel/trace/rethook.c | 16 ++++++++-------- kernel/trace/trace_fprobe.c | 6 +++++- lib/test_fprobe.c | 6 +++--- samples/fprobe/fprobe_example.c | 2 +- 15 files changed, 58 insertions(+), 31 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index aff2746c8af2..e41a270c30bb 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -201,6 +201,7 @@ config KRETPROBE_ON_RETHOOK def_bool y depends on HAVE_RETHOOK depends on KRETPROBES + depends on HAVE_PT_REGS_TO_FTRACE_REGS_CAST || !HAVE_DYNAMIC_FTRACE_WITH_ARGS select RETHOOK config USER_RETURN_NOTIFIER diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index e71d5bf2cee0..33c3a4598ae0 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -103,6 +103,7 @@ config LOONGARCH select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_ARGS + select HAVE_PT_REGS_TO_FTRACE_REGS_CAST select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_EBPF_JIT diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index 5b39918b7042..ef06c3c2b06d 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -165,6 +165,7 @@ config S390 select HAVE_DMA_CONTIGUOUS select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_ARGS + select HAVE_PT_REGS_TO_FTRACE_REGS_CAST select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_EBPF_JIT if HAVE_MARCH_Z196_FEATURES diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7422db409770..15fb8e8bf064 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -207,6 +207,7 @@ config X86 select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_REGS select HAVE_DYNAMIC_FTRACE_WITH_ARGS if X86_64 + select HAVE_PT_REGS_TO_FTRACE_REGS_CAST if X86_64 select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS select HAVE_SAMPLE_FTRACE_DIRECT if X86_64 select HAVE_SAMPLE_FTRACE_DIRECT_MULTI if X86_64 diff --git a/arch/x86/kernel/rethook.c b/arch/x86/kernel/rethook.c index 8a1c0111ae79..d714d0276c93 100644 --- a/arch/x86/kernel/rethook.c +++ b/arch/x86/kernel/rethook.c @@ -83,7 +83,8 @@ __used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs) * arch_rethook_fixup_return() which called from this * rethook_trampoline_handler(). */ - rethook_trampoline_handler(regs, (unsigned long)frame_pointer); + rethook_trampoline_handler((struct ftrace_regs *)regs, + (unsigned long)frame_pointer); /* * Copy FLAGS to 'pt_regs::ss' so that arch_rethook_trapmoline() @@ -104,22 +105,22 @@ NOKPROBE_SYMBOL(arch_rethook_trampoline_callback); STACK_FRAME_NON_STANDARD_FP(arch_rethook_trampoline); /* This is called from rethook_trampoline_handler(). */ -void arch_rethook_fixup_return(struct pt_regs *regs, +void arch_rethook_fixup_return(struct ftrace_regs *fregs, unsigned long correct_ret_addr) { - unsigned long *frame_pointer = (void *)(regs + 1); + unsigned long *frame_pointer = (void *)(fregs + 1); /* Replace fake return address with real one. */ *frame_pointer = correct_ret_addr; } NOKPROBE_SYMBOL(arch_rethook_fixup_return); -void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) +void arch_rethook_prepare(struct rethook_node *rh, struct ftrace_regs *fregs, bool mcount) { - unsigned long *stack = (unsigned long *)regs->sp; + unsigned long *stack = (unsigned long *)ftrace_regs_get_stack_pointer(fregs); rh->ret_addr = stack[0]; - rh->frame = regs->sp; + rh->frame = (unsigned long)stack; /* Replace the return addr with trampoline addr */ stack[0] = (unsigned long) arch_rethook_trampoline; diff --git a/include/linux/fprobe.h b/include/linux/fprobe.h index 36c0595f7b93..b9c0c216dedb 100644 --- a/include/linux/fprobe.h +++ b/include/linux/fprobe.h @@ -38,7 +38,7 @@ struct fprobe { unsigned long ret_ip, struct ftrace_regs *regs, void *entry_data); void (*exit_handler)(struct fprobe *fp, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *regs, void *entry_data); }; diff --git a/include/linux/rethook.h b/include/linux/rethook.h index 26b6f3c81a76..138d64c8b67b 100644 --- a/include/linux/rethook.h +++ b/include/linux/rethook.h @@ -7,6 +7,7 @@ #include #include +#include #include #include #include @@ -14,7 +15,7 @@ struct rethook_node; -typedef void (*rethook_handler_t) (struct rethook_node *, void *, unsigned long, struct pt_regs *); +typedef void (*rethook_handler_t) (struct rethook_node *, void *, unsigned long, struct ftrace_regs *); /** * struct rethook - The rethook management data structure. @@ -64,12 +65,12 @@ void rethook_free(struct rethook *rh); void rethook_add_node(struct rethook *rh, struct rethook_node *node); struct rethook_node *rethook_try_get(struct rethook *rh); void rethook_recycle(struct rethook_node *node); -void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount); +void rethook_hook(struct rethook_node *node, struct ftrace_regs *regs, bool mcount); unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame, struct llist_node **cur); /* Arch dependent code must implement arch_* and trampoline code */ -void arch_rethook_prepare(struct rethook_node *node, struct pt_regs *regs, bool mcount); +void arch_rethook_prepare(struct rethook_node *node, struct ftrace_regs *regs, bool mcount); void arch_rethook_trampoline(void); /** @@ -84,11 +85,11 @@ static inline bool is_rethook_trampoline(unsigned long addr) } /* If the architecture needs to fixup the return address, implement it. */ -void arch_rethook_fixup_return(struct pt_regs *regs, +void arch_rethook_fixup_return(struct ftrace_regs *regs, unsigned long correct_ret_addr); /* Generic trampoline handler, arch code must prepare asm stub */ -unsigned long rethook_trampoline_handler(struct pt_regs *regs, +unsigned long rethook_trampoline_handler(struct ftrace_regs *regs, unsigned long frame); #ifdef CONFIG_RETHOOK diff --git a/kernel/kprobes.c b/kernel/kprobes.c index ca385b61d546..8199d881168a 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -2132,7 +2132,11 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs) if (rp->entry_handler && rp->entry_handler(ri, regs)) rethook_recycle(rhn); else - rethook_hook(rhn, regs, kprobe_ftrace(p)); + /* + * We can cast pt_regs to ftrace_regs because this depends on + * HAVE_PT_REGS_TO_FTRACE_REGS_CAST. + */ + rethook_hook(rhn, (struct ftrace_regs *)regs, kprobe_ftrace(p)); return 0; } @@ -2140,9 +2144,11 @@ NOKPROBE_SYMBOL(pre_handler_kretprobe); static void kretprobe_rethook_handler(struct rethook_node *rh, void *data, unsigned long ret_addr, - struct pt_regs *regs) + struct ftrace_regs *fregs) { struct kretprobe *rp = (struct kretprobe *)data; + /* Ditto, this depends on HAVE_PT_REGS_TO_FTRACE_REGS_CAST. */ + struct pt_regs *regs = (struct pt_regs *)fregs; struct kretprobe_instance *ri; struct kprobe_ctlblk *kcb; diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 976fd594b446..d56304276318 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -57,6 +57,13 @@ config HAVE_DYNAMIC_FTRACE_WITH_ARGS This allows for use of ftrace_regs_get_argument() and ftrace_regs_get_stack_pointer(). +config HAVE_PT_REGS_TO_FTRACE_REGS_CAST + bool + help + If this is set, the memory layout of the ftrace_regs data structure + is the same as the pt_regs. So the pt_regs is possible to be casted + to ftrace_regs. + config HAVE_DYNAMIC_FTRACE_NO_PATCHABLE bool help diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 126acbfdb26f..31f2707f8df7 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2675,10 +2675,14 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, static void kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *data) { struct bpf_kprobe_multi_link *link; + struct pt_regs *regs = ftrace_get_regs(fregs); + + if (!regs) + return; link = container_of(fp, struct bpf_kprobe_multi_link, fp); kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs); diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index 07deb52df44a..dfddc7e8424e 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -53,7 +53,7 @@ static inline void __fprobe_handler(unsigned long ip, unsigned long parent_ip, if (ret) rethook_recycle(rh); else - rethook_hook(rh, ftrace_get_regs(fregs), true); + rethook_hook(rh, fregs, true); } } @@ -120,7 +120,7 @@ static void fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip, } static void fprobe_exit_handler(struct rethook_node *rh, void *data, - unsigned long ret_ip, struct pt_regs *regs) + unsigned long ret_ip, struct ftrace_regs *fregs) { struct fprobe *fp = (struct fprobe *)data; struct fprobe_rethook_node *fpr; @@ -141,7 +141,7 @@ static void fprobe_exit_handler(struct rethook_node *rh, void *data, return; } - fp->exit_handler(fp, fpr->entry_ip, ret_ip, regs, + fp->exit_handler(fp, fpr->entry_ip, ret_ip, fregs, fp->entry_data_size ? (void *)fpr->data : NULL); ftrace_test_recursion_unlock(bit); } diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c index 5eb9b598f4e9..7c5cf9d5910c 100644 --- a/kernel/trace/rethook.c +++ b/kernel/trace/rethook.c @@ -189,7 +189,7 @@ NOKPROBE_SYMBOL(rethook_try_get); /** * rethook_hook() - Hook the current function return. * @node: The struct rethook node to hook the function return. - * @regs: The struct pt_regs for the function entry. + * @fregs: The struct ftrace_regs for the function entry. * @mcount: True if this is called from mcount(ftrace) context. * * Hook the current running function return. This must be called when the @@ -199,9 +199,9 @@ NOKPROBE_SYMBOL(rethook_try_get); * from the real function entry (e.g. kprobes) @mcount must be set false. * This is because the way to hook the function return depends on the context. */ -void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount) +void rethook_hook(struct rethook_node *node, struct ftrace_regs *fregs, bool mcount) { - arch_rethook_prepare(node, regs, mcount); + arch_rethook_prepare(node, fregs, mcount); __llist_add(&node->llist, ¤t->rethooks); } NOKPROBE_SYMBOL(rethook_hook); @@ -269,7 +269,7 @@ unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame } NOKPROBE_SYMBOL(rethook_find_ret_addr); -void __weak arch_rethook_fixup_return(struct pt_regs *regs, +void __weak arch_rethook_fixup_return(struct ftrace_regs *fregs, unsigned long correct_ret_addr) { /* @@ -281,7 +281,7 @@ void __weak arch_rethook_fixup_return(struct pt_regs *regs, } /* This function will be called from each arch-defined trampoline. */ -unsigned long rethook_trampoline_handler(struct pt_regs *regs, +unsigned long rethook_trampoline_handler(struct ftrace_regs *fregs, unsigned long frame) { struct llist_node *first, *node = NULL; @@ -295,7 +295,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, BUG_ON(1); } - instruction_pointer_set(regs, correct_ret_addr); + ftrace_regs_set_instruction_pointer(fregs, correct_ret_addr); /* * These loops must be protected from rethook_free_rcu() because those @@ -315,7 +315,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, handler = READ_ONCE(rhn->rethook->handler); if (handler) handler(rhn, rhn->rethook->data, - correct_ret_addr, regs); + correct_ret_addr, fregs); if (first == node) break; @@ -323,7 +323,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, } /* Fixup registers for returning to correct address. */ - arch_rethook_fixup_return(regs, correct_ret_addr); + arch_rethook_fixup_return(fregs, correct_ret_addr); /* Unlink used shadow stack */ first = current->rethooks.first; diff --git a/kernel/trace/trace_fprobe.c b/kernel/trace/trace_fprobe.c index 4d3ae79f036e..f440c97e050f 100644 --- a/kernel/trace/trace_fprobe.c +++ b/kernel/trace/trace_fprobe.c @@ -341,10 +341,14 @@ static int fentry_dispatcher(struct fprobe *fp, unsigned long entry_ip, NOKPROBE_SYMBOL(fentry_dispatcher); static void fexit_dispatcher(struct fprobe *fp, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data) { struct trace_fprobe *tf = container_of(fp, struct trace_fprobe, fp); + struct pt_regs *regs = ftrace_get_regs(fregs); + + if (!regs) + return; if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) fexit_trace_func(tf, entry_ip, ret_ip, regs); diff --git a/lib/test_fprobe.c b/lib/test_fprobe.c index ff607babba18..d1e80653bf0c 100644 --- a/lib/test_fprobe.c +++ b/lib/test_fprobe.c @@ -59,9 +59,9 @@ static notrace int fp_entry_handler(struct fprobe *fp, unsigned long ip, static notrace void fp_exit_handler(struct fprobe *fp, unsigned long ip, unsigned long ret_ip, - struct pt_regs *regs, void *data) + struct ftrace_regs *fregs, void *data) { - unsigned long ret = regs_return_value(regs); + unsigned long ret = ftrace_regs_return_value(fregs); KUNIT_EXPECT_FALSE(current_test, preemptible()); if (ip != target_ip) { @@ -89,7 +89,7 @@ static notrace int nest_entry_handler(struct fprobe *fp, unsigned long ip, static notrace void nest_exit_handler(struct fprobe *fp, unsigned long ip, unsigned long ret_ip, - struct pt_regs *regs, void *data) + struct ftrace_regs *fregs, void *data) { KUNIT_EXPECT_FALSE(current_test, preemptible()); KUNIT_EXPECT_EQ(current_test, ip, target_nest_ip); diff --git a/samples/fprobe/fprobe_example.c b/samples/fprobe/fprobe_example.c index 1545a1aac616..d476d1f07538 100644 --- a/samples/fprobe/fprobe_example.c +++ b/samples/fprobe/fprobe_example.c @@ -67,7 +67,7 @@ static int sample_entry_handler(struct fprobe *fp, unsigned long ip, } static void sample_exit_handler(struct fprobe *fp, unsigned long ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *regs, void *data) { unsigned long rip = ret_ip; From patchwork Sat Aug 12 05:37:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134819 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1539800vqi; Fri, 11 Aug 2023 22:58:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGy+Xk43mi6QDl+nN7BdpWBFkWXRNqVSJeu+pHmcE6fHmsC3n895jJt2QnnhTqOxvS6/Ap3 X-Received: by 2002:a05:6a21:6811:b0:13e:90aa:8c8b with SMTP id wr17-20020a056a21681100b0013e90aa8c8bmr3711824pzb.4.1691819898265; Fri, 11 Aug 2023 22:58:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691819898; cv=none; d=google.com; s=arc-20160816; b=AhhIno7skyblRnvqhZawabn0Mh9VECk1URiPAVUJickGLQgPECRSIW1XwOdmCRUq1O vZtcPNZVnj5hHYKfXYNp+BpOFy9jvOkWv/lD7NyV8ewTa3WpfMWvt9bmCmVq9t++BpXj /DxR9eaeq5EskbOeGiav3MZoxw1jpEoOgHiqER8yUwSCXH3rlm/GqeAFWQuVFQvTbXfw DYFgQoI9p7XwBvrwfFHdKj7PT5KQqFoWrpAEQ9HU5Gsg+2ooy/0pKOLPxBE55lPxNXSc q5E47BlvkdWU82XokMsE8rOznB+fKl5e+TBzuQPNBzJDrOUHeoN/HAVb01kZXuwQOL9W UIdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=CfgIcdJs/Q2cxxhnYh2sCq/vyKFr4U83tdqie83Yl24=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=OFZwtbBJsDbx+OblQWjhQiKoSKzK1WRI6syameQEFqwuSivF1ovHtmXXWTpz5EvvL/ 0bcVWkCbY7iqGbMh9daPmayTRdvuJniXMnljChehjpPqcG8Im2ZuyyMIx8ldgDXQ+YwV bGTlPOQL0zygfoyWl8a31ftlPeATTlz25ca/525UYVJkIT4mwYVVUPv+azWtKWsrMVMC HCbIostFbQt1sr1nUObiMa+qNfLn6c0fEP8zXqv2AJoiH3lW9PX8PP6Fu+mfvDoC49ZF 6ZI4wmbpEQDQh8JOg85rc4bm3MYdz/h2OT2WMlpov5oFmj1BO1qsFJjMCZxFBX6cy3Ik DPZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=a7DKH26z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s7-20020a634507000000b0056546b5fef6si4431413pga.231.2023.08.11.22.58.05; Fri, 11 Aug 2023 22:58:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=a7DKH26z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236153AbjHLFhp (ORCPT + 99 others); Sat, 12 Aug 2023 01:37:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232008AbjHLFhm (ORCPT ); Sat, 12 Aug 2023 01:37:42 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D3FC2D69; Fri, 11 Aug 2023 22:37:42 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D106864275; Sat, 12 Aug 2023 05:37:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 978FDC433C9; Sat, 12 Aug 2023 05:37:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818661; bh=BigcOhEYrGI5HEgUFTsUs1DBiFzGT40U2roXl0B5ffQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a7DKH26z8ejfMsQoZpZmG3lu8MBlbJSuW2fPpRRKv5cBkLISYwxjOn4Ij7c8sRxyJ 2VzA3ZQrIYzV/ByqFsfow1FJTBGDrc1unc12Mn6MHrl50fi6htDRiStGSr1Xd9+lWx Z11rBYW9zyzxfVbAdWbbtZpJ4AHnluZ46u5YRJm6vI9p8WPaYP3UbRItf8xLSFQbFj MifItv+AVGmlegBCPeCCPhKD2KPQWW98lvrJ0b2aUG16CK5OQZvGjWTLW4vd2Sh7NW Y+0/PFILHCEGdWF0wZ4vMQAKaTkc35zNM0z06CrayR8kU/U5gPWVr1rvTySyp/LSPn ax/SvXRn4sxYA== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 5/8] tracing/fprobe: Enable fprobe events with CONFIG_DYNAMIC_FTRACE_WITH_ARGS Date: Sat, 12 Aug 2023 14:37:35 +0900 Message-Id: <169181865486.505132.6447946094827872988.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774001741360731572 X-GMAIL-MSGID: 1774001741360731572 From: Masami Hiramatsu (Google) Allow fprobe events to be enabled with CONFIG_DYNAMIC_FTRACE_WITH_ARGS. With this change, fprobe events mostly use ftrace_regs instead of pt_regs. Note that if the arch doesn't enable HAVE_PT_REGS_COMPAT_FTRACE_REGS, fprobe events will not be able to be used from perf. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v3: - introduce ftrace_regs_get_kernel_stack_nth(). - fix typo. --- include/linux/ftrace.h | 15 +++++++++ kernel/trace/Kconfig | 1 - kernel/trace/trace_fprobe.c | 65 ++++++++++++++++++++------------------- kernel/trace/trace_probe_tmpl.h | 2 + 4 files changed, 50 insertions(+), 33 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index fe335d861f08..3b3a0b1f592f 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -171,6 +171,21 @@ static __always_inline bool ftrace_regs_has_args(struct ftrace_regs *fregs) return ftrace_get_regs(fregs) != NULL; } +#ifdef CONFIG_HAVE_REGS_AND_STACK_ACCESS_API +static __always_inline unsigned long +ftrace_regs_get_kernel_stack_nth(struct ftrace_regs *fregs, unsigned int nth) +{ + unsigned long *stackp; + + stackp = (unsigned long *)ftrace_regs_get_stack_pointer(fregs); + if (((unsigned long)(stackp + nth) & ~(THREAD_SIZE - 1)) == + ((unsigned long)stackp & ~(THREAD_SIZE - 1))) + return *(stackp + nth); + + return 0; +} +#endif /* CONFIG_HAVE_REGS_AND_STACK_ACCESS_API */ + #ifdef CONFIG_FUNCTION_TRACER extern int ftrace_enabled; diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index d56304276318..6fb4ecf8767d 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -679,7 +679,6 @@ config FPROBE_EVENTS select TRACING select PROBE_EVENTS select DYNAMIC_EVENTS - depends on DYNAMIC_FTRACE_WITH_REGS default y help This allows user to add tracing events on the function entry and diff --git a/kernel/trace/trace_fprobe.c b/kernel/trace/trace_fprobe.c index f440c97e050f..94c01dc061ec 100644 --- a/kernel/trace/trace_fprobe.c +++ b/kernel/trace/trace_fprobe.c @@ -132,7 +132,7 @@ static int process_fetch_insn(struct fetch_insn *code, void *rec, void *dest, void *base) { - struct pt_regs *regs = rec; + struct ftrace_regs *fregs = rec; unsigned long val; int ret; @@ -140,17 +140,17 @@ process_fetch_insn(struct fetch_insn *code, void *rec, void *dest, /* 1st stage: get value from context */ switch (code->op) { case FETCH_OP_STACK: - val = regs_get_kernel_stack_nth(regs, code->param); + val = ftrace_regs_get_kernel_stack_nth(fregs, code->param); break; case FETCH_OP_STACKP: - val = kernel_stack_pointer(regs); + val = ftrace_regs_get_stack_pointer(fregs); break; case FETCH_OP_RETVAL: - val = regs_return_value(regs); + val = ftrace_regs_return_value(fregs); break; #ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API case FETCH_OP_ARG: - val = regs_get_kernel_argument(regs, code->param); + val = ftrace_regs_get_argument(fregs, code->param); break; #endif case FETCH_NOP_SYMBOL: /* Ignore a place holder */ @@ -170,7 +170,7 @@ NOKPROBE_SYMBOL(process_fetch_insn) /* function entry handler */ static nokprobe_inline void __fentry_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - struct pt_regs *regs, + struct ftrace_regs *fregs, struct trace_event_file *trace_file) { struct fentry_trace_entry_head *entry; @@ -184,36 +184,36 @@ __fentry_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, if (trace_trigger_soft_disabled(trace_file)) return; - dsize = __get_data_size(&tf->tp, regs); + dsize = __get_data_size(&tf->tp, fregs); entry = trace_event_buffer_reserve(&fbuffer, trace_file, sizeof(*entry) + tf->tp.size + dsize); if (!entry) return; - fbuffer.regs = regs; + fbuffer.regs = ftrace_get_regs(fregs); entry = fbuffer.entry = ring_buffer_event_data(fbuffer.event); entry->ip = entry_ip; - store_trace_args(&entry[1], &tf->tp, regs, sizeof(*entry), dsize); + store_trace_args(&entry[1], &tf->tp, fregs, sizeof(*entry), dsize); trace_event_buffer_commit(&fbuffer); } static void fentry_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - struct pt_regs *regs) + struct ftrace_regs *fregs) { struct event_file_link *link; trace_probe_for_each_link_rcu(link, &tf->tp) - __fentry_trace_func(tf, entry_ip, regs, link->file); + __fentry_trace_func(tf, entry_ip, fregs, link->file); } NOKPROBE_SYMBOL(fentry_trace_func); /* Kretprobe handler */ static nokprobe_inline void __fexit_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs, + unsigned long ret_ip, struct ftrace_regs *fregs, struct trace_event_file *trace_file) { struct fexit_trace_entry_head *entry; @@ -227,37 +227,37 @@ __fexit_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, if (trace_trigger_soft_disabled(trace_file)) return; - dsize = __get_data_size(&tf->tp, regs); + dsize = __get_data_size(&tf->tp, fregs); entry = trace_event_buffer_reserve(&fbuffer, trace_file, sizeof(*entry) + tf->tp.size + dsize); if (!entry) return; - fbuffer.regs = regs; + fbuffer.regs = ftrace_get_regs(fregs); entry = fbuffer.entry = ring_buffer_event_data(fbuffer.event); entry->func = entry_ip; entry->ret_ip = ret_ip; - store_trace_args(&entry[1], &tf->tp, regs, sizeof(*entry), dsize); + store_trace_args(&entry[1], &tf->tp, fregs, sizeof(*entry), dsize); trace_event_buffer_commit(&fbuffer); } static void fexit_trace_func(struct trace_fprobe *tf, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs) + unsigned long ret_ip, struct ftrace_regs *fregs) { struct event_file_link *link; trace_probe_for_each_link_rcu(link, &tf->tp) - __fexit_trace_func(tf, entry_ip, ret_ip, regs, link->file); + __fexit_trace_func(tf, entry_ip, ret_ip, fregs, link->file); } NOKPROBE_SYMBOL(fexit_trace_func); #ifdef CONFIG_PERF_EVENTS static int fentry_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, - struct pt_regs *regs) + struct ftrace_regs *fregs, struct pt_regs *regs) { struct trace_event_call *call = trace_probe_event_call(&tf->tp); struct fentry_trace_entry_head *entry; @@ -269,7 +269,7 @@ static int fentry_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, if (hlist_empty(head)) return 0; - dsize = __get_data_size(&tf->tp, regs); + dsize = __get_data_size(&tf->tp, fregs); __size = sizeof(*entry) + tf->tp.size + dsize; size = ALIGN(__size + sizeof(u32), sizeof(u64)); size -= sizeof(u32); @@ -280,7 +280,7 @@ static int fentry_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, entry->ip = entry_ip; memset(&entry[1], 0, dsize); - store_trace_args(&entry[1], &tf->tp, regs, sizeof(*entry), dsize); + store_trace_args(&entry[1], &tf->tp, fregs, sizeof(*entry), dsize); perf_trace_buf_submit(entry, size, rctx, call->event.type, 1, regs, head, NULL); return 0; @@ -289,7 +289,8 @@ NOKPROBE_SYMBOL(fentry_perf_func); static void fexit_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, - unsigned long ret_ip, struct pt_regs *regs) + unsigned long ret_ip, struct ftrace_regs *fregs, + struct pt_regs *regs) { struct trace_event_call *call = trace_probe_event_call(&tf->tp); struct fexit_trace_entry_head *entry; @@ -301,7 +302,7 @@ fexit_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, if (hlist_empty(head)) return; - dsize = __get_data_size(&tf->tp, regs); + dsize = __get_data_size(&tf->tp, fregs); __size = sizeof(*entry) + tf->tp.size + dsize; size = ALIGN(__size + sizeof(u32), sizeof(u64)); size -= sizeof(u32); @@ -312,7 +313,7 @@ fexit_perf_func(struct trace_fprobe *tf, unsigned long entry_ip, entry->func = entry_ip; entry->ret_ip = ret_ip; - store_trace_args(&entry[1], &tf->tp, regs, sizeof(*entry), dsize); + store_trace_args(&entry[1], &tf->tp, fregs, sizeof(*entry), dsize); perf_trace_buf_submit(entry, size, rctx, call->event.type, 1, regs, head, NULL); } @@ -327,14 +328,15 @@ static int fentry_dispatcher(struct fprobe *fp, unsigned long entry_ip, struct pt_regs *regs = ftrace_get_regs(fregs); int ret = 0; + if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) + fentry_trace_func(tf, entry_ip, fregs); + +#ifdef CONFIG_PERF_EVENTS if (!regs) return 0; - if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) - fentry_trace_func(tf, entry_ip, regs); -#ifdef CONFIG_PERF_EVENTS if (trace_probe_test_flag(&tf->tp, TP_FLAG_PROFILE)) - ret = fentry_perf_func(tf, entry_ip, regs); + ret = fentry_perf_func(tf, entry_ip, fregs, regs); #endif return ret; } @@ -347,14 +349,15 @@ static void fexit_dispatcher(struct fprobe *fp, unsigned long entry_ip, struct trace_fprobe *tf = container_of(fp, struct trace_fprobe, fp); struct pt_regs *regs = ftrace_get_regs(fregs); + if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) + fexit_trace_func(tf, entry_ip, ret_ip, fregs); + +#ifdef CONFIG_PERF_EVENTS if (!regs) return; - if (trace_probe_test_flag(&tf->tp, TP_FLAG_TRACE)) - fexit_trace_func(tf, entry_ip, ret_ip, regs); -#ifdef CONFIG_PERF_EVENTS if (trace_probe_test_flag(&tf->tp, TP_FLAG_PROFILE)) - fexit_perf_func(tf, entry_ip, ret_ip, regs); + fexit_perf_func(tf, entry_ip, ret_ip, fregs, regs); #endif } NOKPROBE_SYMBOL(fexit_dispatcher); diff --git a/kernel/trace/trace_probe_tmpl.h b/kernel/trace/trace_probe_tmpl.h index 3935b347f874..05445a745a07 100644 --- a/kernel/trace/trace_probe_tmpl.h +++ b/kernel/trace/trace_probe_tmpl.h @@ -232,7 +232,7 @@ process_fetch_insn_bottom(struct fetch_insn *code, unsigned long val, /* Sum up total data length for dynamic arrays (strings) */ static nokprobe_inline int -__get_data_size(struct trace_probe *tp, struct pt_regs *regs) +__get_data_size(struct trace_probe *tp, void *regs) { struct probe_arg *arg; int i, len, ret = 0; From patchwork Sat Aug 12 05:37:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134824 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1547288vqi; Fri, 11 Aug 2023 23:17:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHkETgLnemrjH1tYEuIb8dMhCLTZ0ma8+oTUlK34z9YVpuYxBR3ENwO65SlKWTsLu0dQmcW X-Received: by 2002:a05:6a00:2489:b0:63a:ea82:b7b7 with SMTP id c9-20020a056a00248900b0063aea82b7b7mr5642563pfv.28.1691821064873; Fri, 11 Aug 2023 23:17:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691821064; cv=none; d=google.com; s=arc-20160816; b=GwQ7wJTLnQtkZK588Yf/pDWZfbmhGGDjnfrpSJgRkZouewSQanNfXTBJLQ+Ln90vJ8 ezwypuS9IRta3HaqnyL+xYuDWKtilYFr1i5rlj5HNuWrYLPXlF7IBX89tZnSSocUtw7x NiCgdQeOEL7YMD2M52BeLwOjnWf83UBQMKqHnoB5Y81pwwscofPKe3tx+wHeE+ot1+f+ jt9rsTLnyLELCyGHPE1sGgY7DdCXBOrijvio4gqWHxv+ZG4lzQnhvw6HLREQ6cbdcUls bAwW9gZS9OcEaEh46aNIkCWOJ0J/FmHTWfDQGuj5k4e264u4XEwFAPNEF4805OhAb4li /ycA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=6u/SP7jCKr+kuRCoOC3/fX5RXNtkBk+5cCuCHqpDV+k=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=KfjV6d3XQ62QuYrD6f5vrNJEbnjf7C4/DErMUD3+cbtzzc34HIWpMwgQYBss5qKApS tBXyPromk+0C9Vo+ePyETbEvz5G6vSUQYgNStjmLkZBtshcpr37TEv4zEU7A9TEUAfc5 y11bLie2wMD5F6heH3/7xbn/m3k+kK/2loz+u3ydTLgaoGLMokOXj8de/OJ+DtnEn8DC KBuCanqYdFxkHDzoj8SCz03DUcJXgf1204gkIcfxX3H+4NyylXQxZBezvI2iZICmS3l0 pUmk/csgA7Uflm6RwR252ViUediMM3IWLbFnwmof20FA7lFYG1XYlwt9dTaF84e4X8NQ sHDQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="r/iaF88f"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q18-20020a056a00089200b006878da95c2csi4606994pfj.81.2023.08.11.23.17.31; Fri, 11 Aug 2023 23:17:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="r/iaF88f"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236302AbjHLFh5 (ORCPT + 99 others); Sat, 12 Aug 2023 01:37:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236221AbjHLFh4 (ORCPT ); Sat, 12 Aug 2023 01:37:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CE442D69; Fri, 11 Aug 2023 22:37:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 96F736405A; Sat, 12 Aug 2023 05:37:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 61953C433C7; Sat, 12 Aug 2023 05:37:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818673; bh=CEHOSZbi9FJS29a7d+B13NM/f0jx4Qwp0DObzr3ONVU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r/iaF88fGLKh04qQMuviOSGCmeO0+C/yJ/IuYPOnZ1aZjTtaGYr/0fT4Naxj6G4l8 ODCGhNnht3p0WT9yBFiUv52pxY+XlNp2MkbJITBeyfVB0j+e5N+jhx6Eoq0FngXY2r SFy0kEaClON9tPBYCF9VlVg9uRvhD+lNyC2QOvpOymtcYrUlsN6pT4BlisW5bxTMZF w++4XPuY2INCKYyuwkgAwWw9DLVAUAQJss/tbbICqbJw5nSHDklCJBfeVGQlEa6ABJ YtBz7MmfsTSqagEdShULsPM4BNrltoEzzOBuPh2S21f+MEtF0qwKECWKalmn5N93iu r5bqR5KjNRSdQ== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 6/8] ftrace: Add ftrace_partial_regs() for converting ftrace_regs to pt_regs Date: Sat, 12 Aug 2023 14:37:46 +0900 Message-Id: <169181866661.505132.3229847361646568033.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774002965224184990 X-GMAIL-MSGID: 1774002965224184990 From: Masami Hiramatsu (Google) Add ftrace_partial_regs() which converts the ftrace_regs to pt_regs. If the architecture defines its own ftrace_regs, this copies partial registers to pt_regs and returns it. If not, ftrace_regs is the same as pt_regs and ftrace_partial_regs() will return ftrace_regs::regs. Signed-off-by: Masami Hiramatsu (Google) --- Changes in v3: - Fix to use pt_regs::regs instead of x. - Return ftrace_regs::regs forcibly if HAVE_PT_REGS_COMPAT_FTRACE_REGS=y. - Fix typo. - Fix to copy correct registers to the pt_regs on arm64. --- arch/arm64/include/asm/ftrace.h | 11 +++++++++++ include/linux/ftrace.h | 17 +++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h index ab158196480c..5ad24f315d52 100644 --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -137,6 +137,17 @@ ftrace_override_function_with_return(struct ftrace_regs *fregs) fregs->pc = fregs->lr; } +static __always_inline struct pt_regs * +ftrace_partial_regs(const struct ftrace_regs *fregs, struct pt_regs *regs) +{ + memcpy(regs->regs, fregs->regs, sizeof(u64) * 9); + regs->sp = fregs->sp; + regs->pc = fregs->pc; + regs->regs[29] = fregs->fp; + regs->regs[30] = fregs->lr; + return regs; +} + int ftrace_regs_query_register_offset(const char *name); int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec); diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 3b3a0b1f592f..dcb8ee38d78a 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -159,6 +159,23 @@ static __always_inline struct pt_regs *ftrace_get_regs(struct ftrace_regs *fregs return arch_ftrace_get_regs(fregs); } +#if !defined(CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS) || \ + defined(CONFIG_HAVE_PT_REGS_TO_FTRACE_REGS_CAST) + +static __always_inline struct pt_regs * +ftrace_partial_regs(struct ftrace_regs *fregs, struct pt_regs *regs) +{ + /* + * If CONFIG_HAVE_PT_REGS_TO_FTRACE_REGS_CAST=y, ftrace_regs memory + * layout is the same as pt_regs. So always returns that address. + * Since arch_ftrace_get_regs() will check some members and may return + * NULL, we can not use it. + */ + return &fregs->regs; +} + +#endif /* !CONFIG_HAVE_DYNAMIC_FTRACE_WITH_ARGS || CONFIG_HAVE_PT_REGS_TO_FTRACE_REGS_CAST */ + /* * When true, the ftrace_regs_{get,set}_*() functions may be used on fregs. * Note: this can be true even when ftrace_get_regs() cannot provide a pt_regs. From patchwork Sat Aug 12 05:37:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134817 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1539047vqi; Fri, 11 Aug 2023 22:55:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGXRNuxVG6WkyRr5ko+i60Ra0zFSUA1aT+QxQLp0XreDmteQbQJAHcbG7pITfr0Dgj1ZjSL X-Received: by 2002:a05:6870:c086:b0:1be:9b41:a144 with SMTP id c6-20020a056870c08600b001be9b41a144mr4816149oad.37.1691819727598; Fri, 11 Aug 2023 22:55:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691819727; cv=none; d=google.com; s=arc-20160816; b=HtAwPB1716z2Gv64bMreoPdtzp/lXdhbFBISMxzJlIUCPdayl2DvQLjQYQPRfmOcO0 rnGJb6J1gdrwWu2wpwdAKd1+Rt1KVUU43trBjZX+7eSeHvec1Xcb80PMC8WZ+0dzZuhw z1JUMZ6RDPtUC+5O2BI5WkfUJKMclpS29K1ZEsfBonG59PCVM/oA8+JdSMhKgBGsHxvf B2AMb1+5G0bBv4aS7VG8nzXcV4U8nscl7gVmR0lbto3E58RIZQFa+i7WBlGCrXfZLqBC j6oYGkgrOMhjN+oRRZE7OYgQJ8j9r3/FS70Bzx5ufveVO9jhzkGnBvaduYIPluBYcji1 GZuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=L1sPBgZBqD/VPBRS76BWRdViHfXvJfXvFcWJan54Adw=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=EDzUyNiIl2gSXNEzPSI9/+mSEh+BWLrog4TdRmzlrG5MRQUpHBsFk2FYZjV9oN25Jn QUQcFAtMhTloC2bYvURVuf2xCQ2F0NldL5hk801uOGN7xCkWIOUhw5JNf2LX0snvT2xm 6HrqMhMrriqsyjudOjShU7/yZuoCQV+YUKRHQik68YwCOPpiqpba6k1SJsAcHveZIsJr 8obbrVUtxt88VdG8V1YIuaiHG9rQgQYxkGZ96Y7jOutdeMFByB1edpe2aUYtf9ZaOHWf 9EJxed2rNF5SSFFMkY3jOCqLIWraFIYY8f+3wXSGXNFwdwEILV75nenKcRbfo75LoiS0 5aIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jfaDa+GX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z19-20020a170903409300b001bb42c27496si4281962plc.233.2023.08.11.22.55.11; Fri, 11 Aug 2023 22:55:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jfaDa+GX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236370AbjHLFiL (ORCPT + 99 others); Sat, 12 Aug 2023 01:38:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236294AbjHLFiK (ORCPT ); Sat, 12 Aug 2023 01:38:10 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A60993594; Fri, 11 Aug 2023 22:38:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4746064275; Sat, 12 Aug 2023 05:38:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11FC0C433C8; Sat, 12 Aug 2023 05:38:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818684; bh=btg0nVFd43rpoQn9jPOpj+aatNUMeTGQfbwvThvH8GU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jfaDa+GXfxN3i5eFggQmuF2xK/ICqBU1egNhAgRLxMg/cu90+CeybZV8YhC8o3U4s TOsFDSt56jt481duvUK/MB7pxcxY15E1wXtzjzraIBAKSe9akH/flMiG/sDLA6uiWL s1a13lCN54U5hRt4XQMj1/+hgZN0JR99CeyEB8r+zpywxB+d6HB00Es1FTRQnkBrH+ zeTh0I29M9Dym4RMWzTnVfF94C8ytztViXqUYzbqJeMxxplxEO3E0UhsChayH/7CEt RU7l5ASb0IwRO4ulWNEDb0I9ds5JLFGLbjgMZYT4K9EFHQzLSHZ0JAXtE6dkIcl0e3 H5sVvQj4gWcVg== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 7/8] bpf: Enable kprobe_multi feature if CONFIG_FPROBE is enabled Date: Sat, 12 Aug 2023 14:37:58 +0900 Message-Id: <169181867839.505132.10717747708330472036.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774001562522418931 X-GMAIL-MSGID: 1774001562522418931 From: Masami Hiramatsu (Google) Enable kprobe_multi feature if CONFIG_FPROBE is enabled. The pt_regs is converted from ftrace_regs by ftrace_partial_regs(), thus some registers may always returns 0. But it should be enough for function entry (access arguments) and exit (access return value). Signed-off-by: Masami Hiramatsu (Google) --- kernel/trace/bpf_trace.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 31f2707f8df7..ac9bd5e7ec27 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -2467,7 +2467,7 @@ static int __init bpf_event_init(void) fs_initcall(bpf_event_init); #endif /* CONFIG_MODULES */ -#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS +#ifdef CONFIG_FPROBE struct bpf_kprobe_multi_link { struct bpf_link link; struct fprobe fp; @@ -2489,6 +2489,8 @@ struct user_syms { char *buf; }; +static DEFINE_PER_CPU(struct pt_regs, bpf_kprobe_multi_pt_regs); + static int copy_user_syms(struct user_syms *us, unsigned long __user *usyms, u32 cnt) { unsigned long __user usymbol; @@ -2630,13 +2632,14 @@ static u64 bpf_kprobe_multi_entry_ip(struct bpf_run_ctx *ctx) static int kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, - unsigned long entry_ip, struct pt_regs *regs) + unsigned long entry_ip, struct ftrace_regs *fregs) { struct bpf_kprobe_multi_run_ctx run_ctx = { .link = link, .entry_ip = entry_ip, }; struct bpf_run_ctx *old_run_ctx; + struct pt_regs *regs; int err; if (unlikely(__this_cpu_inc_return(bpf_prog_active) != 1)) { @@ -2646,6 +2649,7 @@ kprobe_multi_link_prog_run(struct bpf_kprobe_multi_link *link, migrate_disable(); rcu_read_lock(); + regs = ftrace_partial_regs(fregs, this_cpu_ptr(&bpf_kprobe_multi_pt_regs)); old_run_ctx = bpf_set_run_ctx(&run_ctx.run_ctx); err = bpf_prog_run(link->link.prog, regs); bpf_reset_run_ctx(old_run_ctx); @@ -2663,13 +2667,9 @@ kprobe_multi_link_handler(struct fprobe *fp, unsigned long fentry_ip, void *data) { struct bpf_kprobe_multi_link *link; - struct pt_regs *regs = ftrace_get_regs(fregs); - - if (!regs) - return 0; link = container_of(fp, struct bpf_kprobe_multi_link, fp); - kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs); + kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), fregs); return 0; } @@ -2679,13 +2679,9 @@ kprobe_multi_link_exit_handler(struct fprobe *fp, unsigned long fentry_ip, void *data) { struct bpf_kprobe_multi_link *link; - struct pt_regs *regs = ftrace_get_regs(fregs); - - if (!regs) - return; link = container_of(fp, struct bpf_kprobe_multi_link, fp); - kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), regs); + kprobe_multi_link_prog_run(link, get_entry_ip(fentry_ip), fregs); } static int symbols_cmp_r(const void *a, const void *b, const void *priv) @@ -2925,7 +2921,7 @@ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *pr kvfree(cookies); return err; } -#else /* !CONFIG_DYNAMIC_FTRACE_WITH_REGS */ +#else /* !CONFIG_FPROBE */ int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { return -EOPNOTSUPP; From patchwork Sat Aug 12 05:38:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Masami Hiramatsu (Google)" X-Patchwork-Id: 134815 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp1536497vqi; Fri, 11 Aug 2023 22:46:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFY18GG4tHKNifInGgXtMlXD8igB3U63scnsZ65hF6WC5bmjXcdEQNjb5tmMpzRKrv0OEPm X-Received: by 2002:a17:907:7886:b0:993:d7ff:afe7 with SMTP id ku6-20020a170907788600b00993d7ffafe7mr3531644ejc.13.1691819183517; Fri, 11 Aug 2023 22:46:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691819183; cv=none; d=google.com; s=arc-20160816; b=JWjH6GeVpI2xU3hT5w7x1LMs6620YbVKcSWFlnSSpqo94Xu1zuBp0Rli3i3zqmTcmr pEFxyJwdwb0psLHipKnPB0WToJ9w2a4y3VJ/DvFFvK9o3HhDy1XFidSKlVqiRkSIRKpP LRnQsNM8MYlyDS0sH8TNGkI1333Vxw7JQacByLOkJFH15dT45zNXVp3qFvYSF35cMVu3 dg9uO/dMkosThGJONRJWEg6jA+cSTKmCgua5PBSEEam0xi22dHnyj1BcSZkSFWwcIkB1 YHOrR1dKAAx2fzZ3muJ7D90UGI/TlBBm2Xsy7Or/U+nyn7Wtj/pL1a0hdb5fuEH8Hn+W yBlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=VhTQqWfpeulyuK/3Nd3H8+KRgbXglpWC9QFhAsbHfTY=; fh=nNLeqPmm07c4DrdDej5z+HdJpmx5SqFFnpeka2nrMVM=; b=e4SHyDXceuBjYmD912t+vJnnJVMlZOeRkZeTuq5r7RYEx19koI1oqcFiWWG747qLPz n4lJO/YqwalqJpQeDCPO1r943MTHjDBPSrce4Zh96InK2bUHBny4i59bBmrh8U0/cQOL ODC2Nra0hjsUUTyGsRRoon4PL4CC/gyKRgm5nT2aPoN+9G7w/F/s992k2Hc5/WkI5wA2 QVzUnq13MM6etBDzL6SBcydi2FLnEcbqFUw/fPrg0H3V6BFUOhEdmGLRqDJKcToJBofj uJW81AkLKd0T+OJbJ57RL6xM6GyMgUnSR+REjgKbIqwLjSK2vzpdxxoOFX3UYmBLIYBu OxcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=vIyseh5+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a2-20020a17090640c200b0099ce392aec9si4451511ejk.803.2023.08.11.22.45.57; Fri, 11 Aug 2023 22:46:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=vIyseh5+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234859AbjHLFiU (ORCPT + 99 others); Sat, 12 Aug 2023 01:38:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229912AbjHLFiS (ORCPT ); Sat, 12 Aug 2023 01:38:18 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 570A41712; Fri, 11 Aug 2023 22:38:17 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E24486430F; Sat, 12 Aug 2023 05:38:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8381C433C8; Sat, 12 Aug 2023 05:38:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1691818696; bh=nAPla5bM8WSnDqOSNz1wS0JqUKaETcUf6cGd53WZP3U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vIyseh5+5Iingx8OnLvN6kaJ7wrl4oBRajbwJKbZugplFLx7LonBLHuGJaNAidhln Qj456vP1yYN9RTIN1KuRcN1+7BD2igrXkl02vy+9zwYogTG1qgfV9HKHQjlGtrmyRk 7F3XE+M124rLYmw3dNQ+yLvjS+4/VIV0NVZmZYodOkndN7GRRi1pjqquI0UCwfICEA oMtY+DIpevdQU/qN/LntUD3J3ZAHzGhhrRsKDfHB2WpivcowDRu1nwtpEhigKlCUxd VbiOauyMb0KjEINJ82tB7NfWPNNOduLNqjb/NTr8azbs51dA96LmNmw/uL73Jlv6Lt WbQlYsCxikycQ== From: "Masami Hiramatsu (Google)" To: Alexei Starovoitov , Steven Rostedt , Florent Revest Cc: linux-trace-kernel@vger.kernel.org, LKML , Martin KaFai Lau , bpf , Sven Schnelle , Alexei Starovoitov , Jiri Olsa , Arnaldo Carvalho de Melo , Daniel Borkmann , Alan Maguire , Mark Rutland , Peter Zijlstra , Thomas Gleixner Subject: [PATCH v3 8/8] Documentations: probes: Update fprobe document to use ftrace_regs Date: Sat, 12 Aug 2023 14:38:10 +0900 Message-Id: <169181869006.505132.4695602314698748304.stgit@devnote2> X-Mailer: git-send-email 2.34.1 In-Reply-To: <169181859570.505132.10136520092011157898.stgit@devnote2> References: <169181859570.505132.10136520092011157898.stgit@devnote2> User-Agent: StGit/0.19 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774000992230935830 X-GMAIL-MSGID: 1774000992230935830 From: Masami Hiramatsu (Google) Update fprobe document so that the entry/exit handler uses ftrace_regs instead of pt_regs. Signed-off-by: Masami Hiramatsu (Google) --- Documentation/trace/fprobe.rst | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/Documentation/trace/fprobe.rst b/Documentation/trace/fprobe.rst index a6d682478147..8eee88eda4e6 100644 --- a/Documentation/trace/fprobe.rst +++ b/Documentation/trace/fprobe.rst @@ -91,9 +91,9 @@ The prototype of the entry/exit callback function are as follows: .. code-block:: c - int entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, void *entry_data); + int entry_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data); - void exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct pt_regs *regs, void *entry_data); + void exit_callback(struct fprobe *fp, unsigned long entry_ip, unsigned long ret_ip, struct ftrace_regs *fregs, void *entry_data); Note that the @entry_ip is saved at function entry and passed to exit handler. If the entry callback function returns !0, the corresponding exit callback will be cancelled. @@ -112,12 +112,10 @@ If the entry callback function returns !0, the corresponding exit callback will This is the return address of the traced function. This can be used at both entry and exit. -@regs - This is the `pt_regs` data structure at the entry and exit. Note that - the instruction pointer of @regs may be different from the @entry_ip - in the entry_handler. If you need traced instruction pointer, you need - to use @entry_ip. On the other hand, in the exit_handler, the instruction - pointer of @regs is set to the currect return address. +@fregs + This is the `ftrace_regs` data structure at the entry and exit. Note that + the instruction pointer of @fregs may be incorrect in entry handler and + exit handler, so you have to use @entry_ip and @ret_ip instead. @entry_data This is a local storage to share the data between entry and exit handlers.