From patchwork Mon May 15 03:13:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93833 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6632114vqo; Sun, 14 May 2023 20:18:01 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6xpUubzKelDG641vtfzfHdMMzC2ecTX6rL+VqWDCKw670ann/srFQ6pRnl44Kgnt6XMVT0 X-Received: by 2002:a05:6a00:1681:b0:639:c88b:c3e0 with SMTP id k1-20020a056a00168100b00639c88bc3e0mr44987542pfc.22.1684120681135; Sun, 14 May 2023 20:18:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684120681; cv=none; d=google.com; s=arc-20160816; b=Ryhhfq5thFoFUEYRee2CKp84dnwaBIW/THgZ/i6t4S7wqpsFxO8ZxgoCmFY4WziKqx iBCWt2lOB+H2VeFy5nZ5ARi6FJlPTUYxkSxLi1hCb/Gl/xRYFPwufD2uuN6vDySgyRlE dpfVJAumwfhxbSKWZQAz5f5ngo2M5AvPaSAgKNINo8aY6z/UCuLhsSuHVPDFVEXl3+Yz b9hJYXPQQyFnvW2vVGfGk4gyRDk4og6RZKRca4oCJFAQS5v0dqzP80PxiWRla8bzT+wK AMAcly+sAjGwa+DLggyx2CTkvvKJ4YVHS7zLvHVq/959VpLGHHqJKOSVTHhRazeYPcHj 2zrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HfO0FQe7LL8tpwg2PdMURaHU+jV7nrgpfKB4jccXJDQ=; b=SJetWWRjKiyik+RxFH1d/UheWmEAnezRnzpiBTSKmdKU59zcWASsG/GH/lVfiT2arY 03M8kFOHOX/vCBn38VbyB+uv8y054EeO5rxTSVu0yMGmblp2w7Ih9DJYqKQX/GsJhF3K 5YZqawaR65DbctJanFgbX7tL0zJWKeP9BduqFMCg53i+hK4Eov0qmwV1jz1BdejBDXFx A1/2hcWBSn9//MHTOetD3vz8wn6lY2zlsI1tsjvMgMIcVVZvkbSlgo97N8fkrGbyieKz Ieh+wBB3K4i4ZgugzgoR5n3iDQeagpS1cL07oOQmffuZlO3WgpstG/wVFA64eqbKYe/E MZ1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=AriHxAGw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i6-20020a056a00004600b00643bac6f81esi16326874pfk.16.2023.05.14.20.17.46; Sun, 14 May 2023 20:18:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=AriHxAGw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235564AbjEODNq (ORCPT + 99 others); Sun, 14 May 2023 23:13:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58256 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229701AbjEODNn (ORCPT ); Sun, 14 May 2023 23:13:43 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28E3B10F8; Sun, 14 May 2023 20:13:42 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-52160f75920so8354957a12.2; Sun, 14 May 2023 20:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684120421; x=1686712421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HfO0FQe7LL8tpwg2PdMURaHU+jV7nrgpfKB4jccXJDQ=; b=AriHxAGwHUCx9Thh3FsC2fX+7ryTYGbNbpqTWfPEJAGknhGeexqWbFK1+boA7T7l2i cvZsPOsPIDZabmoL1UYaQ+W0PbauJN8DEj9jsFE1MONzFGabg2F0r698TyuuiWsH50oP WNj1umDfsuCg8kJggoxamsUhZs+OZeDDFlixPn8g1MsXajPaIUXOBnIqcNt2Mm8I/+uO laHDPUW5/MBzYBYJarVwqxAoi6OOwB5Zw0ic+dsbDsopHMdUSJJd1/BaMGf/WU2MJrLG hSjR4vV3n0yt9gYg/ctMJrrFHZGySk92o/C+2L1l6BXChx8u64b+cCyyN4cYN2OMU/2i V6tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684120421; x=1686712421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HfO0FQe7LL8tpwg2PdMURaHU+jV7nrgpfKB4jccXJDQ=; b=TURNCvXJlwtci6WNG/Yc5w/HZs2jo5stgqkBGTAhDwmfWctkltqwLN/lgh4/ajzLwB wmMsmMP3RoyMofXnfXoXGtUyuMZDgL+1LggNAzpETQCBehDOQWQ/DSbMYYLwWJi/dg1W sTB1Z4xy8SrtFW0I1eD03FWe1xypv2OJ7hnE0CAjdgIUiSLrtosGP8gvf6BCC3KZZlod NuCg3WhBjomfCw77IQSNiTc8a5WMAO3pPDKXlMKA4SzgSzNCktmFveDHnCc5mEowpKEw 18MHP2VE1DWaU5hCmWJx6clViXpgbC6N0wop4SCNbsz170p8vb9U4CGA1WikHwwuI1eY E2sg== X-Gm-Message-State: AC+VfDzswF2ciQESfRUy8HROF/Qd5oCakPz6AU2iX1DBxOGD3OXElmV7 S1lEQI6mVElU6zXaeYO7160= X-Received: by 2002:a05:6a20:3d1a:b0:104:d028:ffe8 with SMTP id y26-20020a056a203d1a00b00104d028ffe8mr10782506pzi.45.1684120421467; Sun, 14 May 2023 20:13:41 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id o4-20020a17090a744400b00252b3328ad8sm4943577pjk.0.2023.05.14.20.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:13:41 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 1/4] rethook: use preempt_{disable, enable}_notrace in rethook_trampoline_handler Date: Mon, 15 May 2023 11:13:10 +0800 Message-Id: <20230515031314.7836-2-zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230515031314.7836-1-zegao@tencent.com> References: <20230515031314.7836-1-zegao@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765928527487585819?= X-GMAIL-MSGID: =?utf-8?q?1765928527487585819?= This patch replace preempt_{disable, enable} with its corresponding notrace version in rethook_trampoline_handler so no worries about stack recursion or overflow introduced by preempt_count_{add, sub} under fprobe + rethook context. Signed-off-by: Ze Gao --- kernel/trace/rethook.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c index 32c3dfdb4d6a..60f6cb2b486b 100644 --- a/kernel/trace/rethook.c +++ b/kernel/trace/rethook.c @@ -288,7 +288,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, * These loops must be protected from rethook_free_rcu() because those * are accessing 'rhn->rethook'. */ - preempt_disable(); + preempt_disable_notrace(); /* * Run the handler on the shadow stack. Do not unlink the list here because @@ -321,7 +321,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, first = first->next; rethook_recycle(rhn); } - preempt_enable(); + preempt_enable_notrace(); return correct_ret_addr; } From patchwork Mon May 15 03:13:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93831 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6631793vqo; Sun, 14 May 2023 20:17:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4SJe97KtxyWtuKjMj8ikG5y0GbH84qgo0oPcSnIZqVzP91Zrjr8yr200Dkle3vpTw/IOkh X-Received: by 2002:a05:6a00:150b:b0:645:c730:f826 with SMTP id q11-20020a056a00150b00b00645c730f826mr37593516pfu.24.1684120632406; Sun, 14 May 2023 20:17:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684120632; cv=none; d=google.com; s=arc-20160816; b=WU14J1G5X6m39ZunFwALQbGTe/F4kRRmU4BCMwn+HjuWB/otT3/P99YYb3agKcbX2N VB1hxYZltMKhAeeTCFQ/fRe6Fk/pupJ2e+aVJLpOXDfKkofJ9TYDvlUMZF60sYaAeUGg Egmfkey3CakXNofZrqMR2OnBEuYBNd2MruXPlJ7AStQQOJFN0DpeH2lDCVjOM4cWYJT/ YiTbe/VAzCSZtqOJres04tcnc1plLtMICBFvqyesCtVvkgVnIHHaah/w79bds9kau45a kh0kuV4sal/vGB1dLdS20QvVPAtDGs0KF5WVs9X3ruSNxlGnzyEl9sU66v5W5/2JJWME huJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LiiaLoe2FHTpcejFRmixK8pQOtJa58cAbjMGJsmgl/s=; b=BY1WhlAAxtdMwqXagyFJEI0mS/iW9BhDspUKmeRzN3zPiGhx7v0tZul8sKT9lP99fM 2Fj6UNAx7PrtqvSCKfvoiXwEbKg33se0ZkPBRXD7eLmQvFgJipCn87bnBmWN9qc8O+Ff 9jlD2umy773SvQCrEy3o2pDNo56Q7rUvOSNDqrsO3euVRvCWtivo2F98ZN08byCpmv/2 wJ296EMnvwC1J4oSovqWbZyomIzumwng+rRvA1ozCgHga4/J2VwyxVQkl24d9i7x31SG aaYuqyPLDHpmCmsWLqmSp1dbxzRGhS0UB9unLGzz8KcDJ/4/2UsDreIkG2LGZez0Vuxj owhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=J1BfnOlc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b123-20020a62cf81000000b0063b72afb4cdsi16082652pfg.5.2023.05.14.20.16.57; Sun, 14 May 2023 20:17:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=J1BfnOlc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238157AbjEODN4 (ORCPT + 99 others); Sun, 14 May 2023 23:13:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232572AbjEODNt (ORCPT ); Sun, 14 May 2023 23:13:49 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66AF5172C; Sun, 14 May 2023 20:13:47 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-250252e4113so8231306a91.0; Sun, 14 May 2023 20:13:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684120427; x=1686712427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LiiaLoe2FHTpcejFRmixK8pQOtJa58cAbjMGJsmgl/s=; b=J1BfnOlcIOIhgtjq+rMKJpJhtUDWEYN1i5pZ9WI5qw7lQuiyE9W5ziwS9tHiXxr0j8 kHM6HMERPCSvGOdHc+OjUornJZDjV9AYJgg0tiPvQvdzlf0jgyzuUOHg7IFdOuLf4MO0 DR6nLPR0awPaWx9lkWZXsRH1s0dXeaGY5lnKKCu5/49ULoRSChhv75hW/SUQafxI1mo4 b7sJT1eqK/UnkuwQoVFZUrZ3NUZcpbqZw5fxtjodfvEyoQ3WIwl2BG51jz9Jk1G7o45g 3Kv7palxqk2ajwcWYsxS2aG2HGmEKuTUsaxz3QTAGjlYGnSynojEFf8XshUnSyqea3VR Zgxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684120427; x=1686712427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LiiaLoe2FHTpcejFRmixK8pQOtJa58cAbjMGJsmgl/s=; b=PW4fF3u5NJyBxcUfMOE+O1PhZ+5pgd5iksIJB8S/d3RiEDLFZevsUrFcX9QDmyMfVl dm/vA0hmrUqB9qPzZXp+ARiDnxS294y/x5/wIX93ii+hfSxC70gv6iz6uGFtTlO+Ef8B YpJvzm9Yyc2sBv9CC93Dnzkg/ByvMTJD0LkMV11SjGxnJ6m20o3JnAxFa7seOb6UQtIt HAeq9XD/BTlyyyTpv+gDe8CiBaGMC2JINW24aTN4AM/H9iE5hory4xl3LTr2TC7bNYjl J29360Vx4cmGJ+rVr9gfjVTZmqDGbbR/vJd/T0oAcmgfT2FA4kjzrbbY6ssapJfSQk/k J++Q== X-Gm-Message-State: AC+VfDy5xqOh9JZnMGa9JFMNx5YaIko5jq7MK0sYBJQmcIGNTcwia83f rAtBs8XaE+jcE5Vzo2qmKWU= X-Received: by 2002:a17:90a:6a8e:b0:234:ba34:71bf with SMTP id u14-20020a17090a6a8e00b00234ba3471bfmr33197027pjj.1.1684120426786; Sun, 14 May 2023 20:13:46 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id o4-20020a17090a744400b00252b3328ad8sm4943577pjk.0.2023.05.14.20.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:13:46 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 2/4] fprobe: make fprobe_kprobe_handler recursion free Date: Mon, 15 May 2023 11:13:11 +0800 Message-Id: <20230515031314.7836-3-zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230515031314.7836-1-zegao@tencent.com> References: <20230515031314.7836-1-zegao@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765928476462145137?= X-GMAIL-MSGID: =?utf-8?q?1765928476462145137?= Current implementation calls kprobe related functions before doing ftrace recursion check in fprobe_kprobe_handler, which opens door to kernel crash due to stack recursion if preempt_count_{add, sub} is traceable. Refactor the common part out of fprobe_kprobe_handler and fprobe_ handler and call ftrace recursion detection at the very beginning, and also mark these functions notrace so that the whole fprobe_k- probe_handler is free from recusion. And Signed-off-by: Ze Gao --- kernel/trace/fprobe.c | 61 +++++++++++++++++++++++++++++++------------ 1 file changed, 45 insertions(+), 16 deletions(-) diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index 9abb3905bc8e..ad9a36c87ad9 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -20,30 +20,22 @@ struct fprobe_rethook_node { char data[]; }; -static void fprobe_handler(unsigned long ip, unsigned long parent_ip, - struct ftrace_ops *ops, struct ftrace_regs *fregs) +static inline notrace void __fprobe_handler(unsigned long ip, unsigned long + parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { struct fprobe_rethook_node *fpr; struct rethook_node *rh = NULL; struct fprobe *fp; void *entry_data = NULL; - int bit, ret; + int ret; fp = container_of(ops, struct fprobe, ops); - if (fprobe_disabled(fp)) - return; - - bit = ftrace_test_recursion_trylock(ip, parent_ip); - if (bit < 0) { - fp->nmissed++; - return; - } if (fp->exit_handler) { rh = rethook_try_get(fp->rethook); if (!rh) { fp->nmissed++; - goto out; + return; } fpr = container_of(rh, struct fprobe_rethook_node, node); fpr->entry_ip = ip; @@ -61,23 +53,60 @@ static void fprobe_handler(unsigned long ip, unsigned long parent_ip, else rethook_hook(rh, ftrace_get_regs(fregs), true); } -out: +} + +static void notrace fprobe_handler(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *ops, struct ftrace_regs *fregs) +{ + struct fprobe *fp; + int bit; + + fp = container_of(ops, struct fprobe, ops); + if (fprobe_disabled(fp)) + return; + + /* recursion detection has to go before any traceable function and + * all functions before this point should be marked as notrace + */ + bit = ftrace_test_recursion_trylock(ip, parent_ip); + if (bit < 0) { + fp->nmissed++; + return; + } + __fprobe_handler(ip, parent_ip, ops, fregs); ftrace_test_recursion_unlock(bit); + } NOKPROBE_SYMBOL(fprobe_handler); -static void fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip, +static void notrace fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - struct fprobe *fp = container_of(ops, struct fprobe, ops); + struct fprobe *fp; + int bit; + + fp = container_of(ops, struct fprobe, ops); + if (fprobe_disabled(fp)) + return; + + /* recursion detection has to go before any traceable function and + * all functions called before this point should be marked as notrace + */ + bit = ftrace_test_recursion_trylock(ip, parent_ip); + if (bit < 0) { + fp->nmissed++; + return; + } if (unlikely(kprobe_running())) { fp->nmissed++; return; } + kprobe_busy_begin(); - fprobe_handler(ip, parent_ip, ops, fregs); + __fprobe_handler(ip, parent_ip, ops, fregs); kprobe_busy_end(); + ftrace_test_recursion_unlock(bit); } static void fprobe_exit_handler(struct rethook_node *rh, void *data, From patchwork Mon May 15 03:13:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93834 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6632652vqo; Sun, 14 May 2023 20:19:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4b6wjICq0w/jkXLyLg+zysn00bGW1cfhTypHLgd8oCwaPcC7Hx54QkNpuiz1GvJvOUwgmw X-Received: by 2002:a17:90b:190f:b0:24e:16ae:61ca with SMTP id mp15-20020a17090b190f00b0024e16ae61camr30750844pjb.34.1684120764626; Sun, 14 May 2023 20:19:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684120764; cv=none; d=google.com; s=arc-20160816; b=Hf3Rz5FLHT3VgV/Zk5CAK0oerkwNPPMovm+yMSw1S2LSUSa3RuKBtcxh6bt1NOJVMA sR5o0BCqaSulodCbN3sDxQWAhvWqH8pqNLRCCw1GV8gWFwOVYHtbX0nuUwyrkpx/kWV/ U72okPNfOkwsZ+m2R8Fhp5f4e/aIDu0T4dC1NwynZj1yrHARoeqj/weR58VzcAJt02hm Y6ozIKv7v9J3rUGYazKx4CNHUxwTKo9QTO3uEa5q/4mgWvfd0ydai7euEZs1F59eVtRC pTBOIxIxzxPoQwBcsZ647hFFhu/3q2JlzxIJqsc8TuiRUv6zRCAKZmcPrZSu9kbRDc63 7jSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JzkH4lQfb5ZOUjwqO+AdNbxgdgLDuSyyuCXzioTUQhE=; b=qcGplJ5jQZUo4EfWiqwpdVC3jrNGlHk/21DFxVAYR+Op4c9jYzbYlCYBWIm2sfxxvp 04AFTnrdtgEAoVZH8e9fLofzSn7mmFy0qKy1cZ3FvMQWb+u13TaZImlOvRDrV6ZHxR65 gibAVTThyB9ZDbn7JUuqkL6H1O9XvKhhWZlu99iHFXn7sGNJVVhSMOf/JrlELiypIBXo 4Hn2CNnAqzLO0Tk2kkiHeHoS3kE2rQ892CPTbsznT8tVU4aPdfg6JLhiK8LkjtSKGrek zV0vtY5A4R4OUtUTKVrNKTOoymT18AQFDWBHf6UTAi+tQU/5U/LcNCPrXhE38vwAnq1o 5zwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=enBaw82M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l10-20020a17090a850a00b0024715c041c3si28655532pjn.137.2023.05.14.20.19.10; Sun, 14 May 2023 20:19:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=enBaw82M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229942AbjEODOB (ORCPT + 99 others); Sun, 14 May 2023 23:14:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238101AbjEODNx (ORCPT ); Sun, 14 May 2023 23:13:53 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 405E719AE; Sun, 14 May 2023 20:13:52 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-24df6bbf765so10416294a91.0; Sun, 14 May 2023 20:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684120431; x=1686712431; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JzkH4lQfb5ZOUjwqO+AdNbxgdgLDuSyyuCXzioTUQhE=; b=enBaw82MMhF8WgmXnqu36qFSVY3hKbpo+op2O5MAGLA+GpkIJEl3vC1lBt3rERLk8L gRdcppCEuq/5D+3OQ0RfxKTGM6PKQGMJw3kTzsQvcBmKRjF/5dCFVVY86/f8EwCVFMFh gfFgegsBJXoMf5eyTbJWd7LqflGxj2HgBaapTA0wMctFtIPZ9gwkztpfL71uZ5QezH2t yllnfUII7mWYm0JTwyV8wKKJ2C6HQuf/+vk3uizyMMJPyIZ1t8I2aht/uDigW0kAQp42 1vdzBOYtzxIjhyWUSIjfXUopS486Ag1SwrAzvHCXAmDBigR9h3snNa4b3n+aaqAMps2u RWVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684120431; x=1686712431; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JzkH4lQfb5ZOUjwqO+AdNbxgdgLDuSyyuCXzioTUQhE=; b=It18kmxqVFfNTfoCapw1cBcHpuCB8TAdDRZ0dpn40lXD7OrcesZgFFfeNDZr9O7ZtQ cb7N78rgCWv02pnyrFNJ2sDceOWKCoGOU5GKJZBqHv3DDodVE23XgckQq88KOGBHPWlC kXZ12S9ms09Q61YxM/uIZxWv49n5qigS47Iz5MAeNwjcsb593UCgA5ga0CNqgiqNRE5U d//F93GbYCpRiCfnU0IxB0Inqf1MyF0SFh6bO4an1lFouCHxsmj/fyzZwc7o+SQWPp5d l9ZnYEteyBV5m2dwUie2hTj0OJPtQ1eWE/7NWuLmaSngjUbzssacQJKsoycG/XW5PuXO /Iqw== X-Gm-Message-State: AC+VfDxWzA7M6pQf1sJIKbejixWcxrIjThDdSzmwtCvPpWZwlalYfYiX U7m9ard1MkKj0sGEdmIK55s= X-Received: by 2002:a17:90b:a58:b0:23f:81c0:eadd with SMTP id gw24-20020a17090b0a5800b0023f81c0eaddmr31427720pjb.47.1684120431582; Sun, 14 May 2023 20:13:51 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id o4-20020a17090a744400b00252b3328ad8sm4943577pjk.0.2023.05.14.20.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:13:51 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 3/4] fprobe: add recursion detection in fprobe_exit_handler Date: Mon, 15 May 2023 11:13:12 +0800 Message-Id: <20230515031314.7836-4-zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230515031314.7836-1-zegao@tencent.com> References: <20230515031314.7836-1-zegao@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765928614684177805?= X-GMAIL-MSGID: =?utf-8?q?1765928614684177805?= fprobe_hander and fprobe_kprobe_handler has guarded ftrace recusion detection but fprobe_exit_handler has not, which possibly introduce recurive calls if the fprobe exit callback calls any traceable functions. Checking in fprobe_hander or fprobe_kprobe_handler is not enough and misses this case. So add recusion free guard the same way as fprobe_hander and also mark fprobe_exit_handler notrace. Since ftrace recursion check does not employ ips, so here use entry_ip and entry_parent_ip the same as fprobe_handler. Signed-off-by: Ze Gao --- kernel/trace/fprobe.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index ad9a36c87ad9..cf982d4ab142 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -17,6 +17,7 @@ struct fprobe_rethook_node { struct rethook_node node; unsigned long entry_ip; + unsigned long entry_parent_ip; char data[]; }; @@ -39,6 +40,7 @@ static inline notrace void __fprobe_handler(unsigned long ip, unsigned long } fpr = container_of(rh, struct fprobe_rethook_node, node); fpr->entry_ip = ip; + fpr->entry_parent_ip = parent_ip; if (fp->entry_data_size) entry_data = fpr->data; } @@ -109,19 +111,30 @@ static void notrace fprobe_kprobe_handler(unsigned long ip, unsigned long parent ftrace_test_recursion_unlock(bit); } -static void fprobe_exit_handler(struct rethook_node *rh, void *data, +static void notrace fprobe_exit_handler(struct rethook_node *rh, void *data, struct pt_regs *regs) { struct fprobe *fp = (struct fprobe *)data; struct fprobe_rethook_node *fpr; + int bit; if (!fp || fprobe_disabled(fp)) return; fpr = container_of(rh, struct fprobe_rethook_node, node); + /* we need to assure no calls to traceable functions in-between the + * end of fprobe_handler and the beginning of fprobe_exit_handler. + */ + bit = ftrace_test_recursion_trylock(fpr->entry_ip, fpr->entry_parent_ip); + if (bit < 0) { + fp->nmissed++; + return; + } + fp->exit_handler(fp, fpr->entry_ip, regs, fp->entry_data_size ? (void *)fpr->data : NULL); + ftrace_test_recursion_unlock(bit); } NOKPROBE_SYMBOL(fprobe_exit_handler); From patchwork Mon May 15 03:13:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93835 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6635007vqo; Sun, 14 May 2023 20:25:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5eeSYekDpvFcvjeuCUI1CoB9t81OXDsylGOhfEgfDLk0kVidrHlYsBqWpAsbBQ0Jh3FEKc X-Received: by 2002:a05:6a00:18a8:b0:646:6d5e:1d25 with SMTP id x40-20020a056a0018a800b006466d5e1d25mr33522137pfh.16.1684121136165; Sun, 14 May 2023 20:25:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684121136; cv=none; d=google.com; s=arc-20160816; b=PwzFV7m44LbmNNd/Eh1n9R/q6PUxjKNMO2QofkDgpJbkYQdOnRV73oMGvLpsFguz8f SGP1rk5DLfMRkiU1kSUk1KmChw3qDcGXjb7b6s3JZVZcrOQUm7AdJIpz1q8ZcWS2Tfkh /+XY6dzN1GeRZNyc0CTq+yMr0810LadOzE+luUE9yZtOE2wL897PkCgBPT04sA/4y9RV cYpjHBld624kSXr4Zca5nztjq7HwnnlyJCCY/BZTkZBiVzyDnhPsPdJbv8JBVTGwIF7a kKisu8DhPhv1afw71G89hoKGwGEfRNAijyEsbFhKmKrGBW1e9344WGMSJFjxRCMwWQQ4 ESZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Misk8RzcAuhujPj5CN4/zB0H7gvh8yXqb60mLrr5oOQ=; b=kqXz0Io//ywkwYZZAcvUoXn7Q60MjDiJh+AyUvQpq2XPfQ3VmOdCQ09Geh09hkBjn9 DLpXha88Hef6Yz3qZqQM1JTKAXm0jsrkgKd7IWDJ4U38Ba9Cr5NgsJFrHGs2AmG99RKp NJDqPqKmWEO4WJtKm+PSwNovML17OHFgM8AsOvp64T8QH/YAAxNIqw2PjwnBLjbAmNJ/ Jn35R1pkls2tGAv1KN36t5qyb+7V/PZ2ZFVTaO8LT0gYUjmC0XtLP2y4lk+ejqapkfQA wCknHZKRVEoDxtnAeu4G4SOKHvqeOELvJ2Kac2eBFazuCWJOHtgS9kulDjJRLWkZ5VnP JVag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=mxRmqh7R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x24-20020a63db58000000b004fc274006ddsi15185649pgi.670.2023.05.14.20.25.21; Sun, 14 May 2023 20:25:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=mxRmqh7R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238201AbjEODOW (ORCPT + 99 others); Sun, 14 May 2023 23:14:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238361AbjEODOM (ORCPT ); Sun, 14 May 2023 23:14:12 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C85EF19AD; Sun, 14 May 2023 20:14:01 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-24dec03ad8fso8716354a91.1; Sun, 14 May 2023 20:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684120441; x=1686712441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Misk8RzcAuhujPj5CN4/zB0H7gvh8yXqb60mLrr5oOQ=; b=mxRmqh7RrZKI4U323j4QPj9REZOu20NRL2ypQKyhgtAio9Dgz6PqIAtGGj+I/S8d9D JcFiLL86AFLBMrrEBHaYu+c9PBRjeT8ZWxVH7oreEdaW4DovxDL4y/mvq8U3TAp5nHCH SIQhw3ZhUh+IOveWMBg7nw/IhDKU35QJhZLqglLbqU4vxdnznenShULKTrw1SEFnkBEN Q3ZDXR6cmTt4/+py+wnXuC0illIP/GUnZfy/Yt2ef6O5LVrZ+KEcfYOyRvpeSemZA6pQ EmGuZmoAuNIC2ysE9lW8qMU1QpQraTgl16W0ygFIQcEFvIIeLx9HAwNIrWZAs4E20LSx e1vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684120441; x=1686712441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Misk8RzcAuhujPj5CN4/zB0H7gvh8yXqb60mLrr5oOQ=; b=JVjrt342shrV30xUK75ut1phkdRcTB6rQ7CNHFDySFDaM52jdogJijlfBpvRKMV0gp DVqoiShXb5w+SpoGnOrgjtY4JlR2y/8dtDXj6ee4flsyjrlC9xFTpQ/JAoHfRqJGcnC9 ASAb6vZo6Mi02TAGDMwmgt/9lLX5skURdWtWUJPsRmt1tL+oZgEUyuF2KP8p8P2I06Ca 95Eu4W9ct3OcZQAl/zsKXSpjpemSacsxZZgswmYnqumS4FzILBGu9qHwEmRjLB9AiWjX 9eaek5Hn1UhjSuWCDYD95zTTj6TW6ba9b5OWrpIjZa9xdbi05Q6yNOO9KA+/IKorvhB4 3Azg== X-Gm-Message-State: AC+VfDza2G+6VevKEBgMiOeXZpIYbGa0DsQXkM4fJV3juCyDRSvVRQu1 FGDssciIVRgmmnx7YjGqAZg= X-Received: by 2002:a17:90b:3881:b0:246:c097:6a17 with SMTP id mu1-20020a17090b388100b00246c0976a17mr33470341pjb.24.1684120440707; Sun, 14 May 2023 20:14:00 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id o4-20020a17090a744400b00252b3328ad8sm4943577pjk.0.2023.05.14.20.13.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:14:00 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 4/4] rehook, fprobe: mark rethook related functions notrace Date: Mon, 15 May 2023 11:13:13 +0800 Message-Id: <20230515031314.7836-5-zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230515031314.7836-1-zegao@tencent.com> References: <20230515031314.7836-1-zegao@tencent.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765929004577204770?= X-GMAIL-MSGID: =?utf-8?q?1765929004577204770?= These functions are already marked as NOKPROBE to prevent recusion and we have the same reason to blacklist them if rethook is used with fprobe, since they are beyond the recursion-free region ftrace can guard. Signed-off-by: Ze Gao --- arch/riscv/kernel/probes/rethook.c | 4 ++-- arch/s390/kernel/rethook.c | 6 +++--- arch/x86/kernel/rethook.c | 8 +++++--- kernel/trace/rethook.c | 8 ++++---- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/riscv/kernel/probes/rethook.c b/arch/riscv/kernel/probes/rethook.c index 5c27c1f50989..803c412a1bea 100644 --- a/arch/riscv/kernel/probes/rethook.c +++ b/arch/riscv/kernel/probes/rethook.c @@ -8,14 +8,14 @@ #include "rethook.h" /* This is called from arch_rethook_trampoline() */ -unsigned long __used arch_rethook_trampoline_callback(struct pt_regs *regs) +unsigned long __used notrace arch_rethook_trampoline_callback(struct pt_regs *regs) { return rethook_trampoline_handler(regs, regs->s0); } NOKPROBE_SYMBOL(arch_rethook_trampoline_callback); -void arch_rethook_prepare(struct rethook_node *rhn, struct pt_regs *regs, bool mcount) +void notrace arch_rethook_prepare(struct rethook_node *rhn, struct pt_regs *regs, bool mcount) { rhn->ret_addr = regs->ra; rhn->frame = regs->s0; diff --git a/arch/s390/kernel/rethook.c b/arch/s390/kernel/rethook.c index af10e6bdd34e..ad52119826c1 100644 --- a/arch/s390/kernel/rethook.c +++ b/arch/s390/kernel/rethook.c @@ -3,7 +3,7 @@ #include #include "rethook.h" -void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) +void notrace arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) { rh->ret_addr = regs->gprs[14]; rh->frame = regs->gprs[15]; @@ -13,7 +13,7 @@ void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mc } NOKPROBE_SYMBOL(arch_rethook_prepare); -void arch_rethook_fixup_return(struct pt_regs *regs, +void notrace arch_rethook_fixup_return(struct pt_regs *regs, unsigned long correct_ret_addr) { /* Replace fake return address with real one. */ @@ -24,7 +24,7 @@ NOKPROBE_SYMBOL(arch_rethook_fixup_return); /* * Called from arch_rethook_trampoline */ -unsigned long arch_rethook_trampoline_callback(struct pt_regs *regs) +unsigned long notrace arch_rethook_trampoline_callback(struct pt_regs *regs) { return rethook_trampoline_handler(regs, regs->gprs[15]); } diff --git a/arch/x86/kernel/rethook.c b/arch/x86/kernel/rethook.c index 8a1c0111ae79..1f7cef86f73d 100644 --- a/arch/x86/kernel/rethook.c +++ b/arch/x86/kernel/rethook.c @@ -64,7 +64,8 @@ NOKPROBE_SYMBOL(arch_rethook_trampoline); /* * Called from arch_rethook_trampoline */ -__used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs) +__used __visible void notrace arch_rethook_trampoline_callback(struct pt_regs + *regs) { unsigned long *frame_pointer; @@ -104,7 +105,7 @@ NOKPROBE_SYMBOL(arch_rethook_trampoline_callback); STACK_FRAME_NON_STANDARD_FP(arch_rethook_trampoline); /* This is called from rethook_trampoline_handler(). */ -void arch_rethook_fixup_return(struct pt_regs *regs, +void notrace arch_rethook_fixup_return(struct pt_regs *regs, unsigned long correct_ret_addr) { unsigned long *frame_pointer = (void *)(regs + 1); @@ -114,7 +115,8 @@ void arch_rethook_fixup_return(struct pt_regs *regs, } NOKPROBE_SYMBOL(arch_rethook_fixup_return); -void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) +void notrace arch_rethook_prepare(struct rethook_node *rh, struct pt_regs + *regs, bool mcount) { unsigned long *stack = (unsigned long *)regs->sp; diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c index 60f6cb2b486b..e551e86d3927 100644 --- a/kernel/trace/rethook.c +++ b/kernel/trace/rethook.c @@ -127,7 +127,7 @@ static void free_rethook_node_rcu(struct rcu_head *head) * Return back the @node to @node::rethook. If the @node::rethook is already * marked as freed, this will free the @node. */ -void rethook_recycle(struct rethook_node *node) +void notrace rethook_recycle(struct rethook_node *node) { lockdep_assert_preemption_disabled(); @@ -194,7 +194,7 @@ void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount) NOKPROBE_SYMBOL(rethook_hook); /* This assumes the 'tsk' is the current task or is not running. */ -static unsigned long __rethook_find_ret_addr(struct task_struct *tsk, +static unsigned long notrace __rethook_find_ret_addr(struct task_struct *tsk, struct llist_node **cur) { struct rethook_node *rh = NULL; @@ -256,7 +256,7 @@ unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame } NOKPROBE_SYMBOL(rethook_find_ret_addr); -void __weak arch_rethook_fixup_return(struct pt_regs *regs, +void __weak notrace arch_rethook_fixup_return(struct pt_regs *regs, unsigned long correct_ret_addr) { /* @@ -268,7 +268,7 @@ void __weak arch_rethook_fixup_return(struct pt_regs *regs, } /* This function will be called from each arch-defined trampoline. */ -unsigned long rethook_trampoline_handler(struct pt_regs *regs, +unsigned long notrace rethook_trampoline_handler(struct pt_regs *regs, unsigned long frame) { struct llist_node *first, *node = NULL;