From patchwork Mon May 15 03:26:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93850 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6653116vqo; Sun, 14 May 2023 21:15:08 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6NY2UXVBBiA5N8PtP2NZRBcHrwaESoodusWPbiy5WwW2x2EG4tPASTBiG/hqnzcL18MIsU X-Received: by 2002:a17:902:bc83:b0:1ad:fcdc:2a9f with SMTP id bb3-20020a170902bc8300b001adfcdc2a9fmr6034717plb.51.1684124108100; Sun, 14 May 2023 21:15:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684124108; cv=none; d=google.com; s=arc-20160816; b=EYvYpiNhF8EciBgR+gVFGVW26rw+u0oR5UM24Fn+WvA8RjxowU9WVwTaI6Yr7uA26b ZbECCoor8vHwNcHEdU9g8NkosGRzAoUssjnKxBFJe77COhFPhmTocFCBYkerCq+ItmDp Rz7REDi5hNybRlWWf1Y2Ne78bq9uLQxSlN00ogiUc6DOPC2I0JLnmElRnLXOCXzgB21C RZAz2gVKnrkz2jKUasV0G2XrrZXU8co/8c5MwdzyH5RljmTnLJ1E2sKHSWAU492PDQAK PjMxsWd6Xz7jNAQmQOzLKvmakU7vc748oSo9E/H0cRi7Bo9NxNU8tVI4Q84+ZL5uQcIL fuLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HfO0FQe7LL8tpwg2PdMURaHU+jV7nrgpfKB4jccXJDQ=; b=j2mcAurm/gphpqv8WBM18nNkA72yR9rx/mHBet57VLWear/TcynWR8jJmzQ9PPW81a rzQnlhSm01enciPMdeoomS87ik9MNTTDgVBHPNWZp0ZL2fEzDxbTiciDe3/+VQLF4SEi swEYjU0TncOEQTqmBKSbZBcA0nDqVltnUXlk1N6JEakkoJ/zv4XMtSUvQGl3zQLwGXPQ 976H+Zb8Ct8KuVTOu3BwE8l7y4YeyT+HfdkO2EssvQXDKioGeoS16lUMaonsrgGB5/Sj bgSBaYluxeo+dBrdAIMnwwvscXLwy2jXibWSjlkpQzUdP1q/EOFxV7VrplMd1GC7V+J4 SCLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=WVB3C+5o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x13-20020a170902ec8d00b001a1ad6fabdesi16377557plg.228.2023.05.14.21.14.53; Sun, 14 May 2023 21:15:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=WVB3C+5o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238097AbjEOD1I (ORCPT + 99 others); Sun, 14 May 2023 23:27:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232128AbjEOD1F (ORCPT ); Sun, 14 May 2023 23:27:05 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C7F01997; Sun, 14 May 2023 20:27:04 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-6434e263962so8954157b3a.2; Sun, 14 May 2023 20:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684121224; x=1686713224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HfO0FQe7LL8tpwg2PdMURaHU+jV7nrgpfKB4jccXJDQ=; b=WVB3C+5of4uMP7ScO4Yvn60WFDYoZNDWLC01FoZNuZamSGzinFIhL1w/yn5LeSaHsf 2mLOOVY3Edk9fekV4079ljH4DU4Iqdw7lhLf6R6cmjsLGHOhoRbBglRCi3DFwZ5jTP7C GrbVATSZ3VdzrORZcPMoHtC94RWmOjbHrb/jRlj2Urt07DFAmnUl+GeWurCUYg/tdbU0 b9M7fiY9FzWmDsK2/OAEWmMQAeuftDNcl7Y2GYAIR/iUnLRUriLPTEoDCeCwgT7m3iwY KG4kbm1vSDeHkz/PUOiXUGZJsTlGVSQwjg0AdRpxH/Baih5krASD17n4VZxKuzWlL0rz E4kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684121224; x=1686713224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HfO0FQe7LL8tpwg2PdMURaHU+jV7nrgpfKB4jccXJDQ=; b=VI8pyGag1JXpGFppzjhRBEsy8D9nctXuUW1cncuwLrskLgYn+AqhFUxfYDXkfKpFMp aVPm25iOBdY4MaynVxE+AXGHLCr2noMClUbNPj9u7dCLRVujw3f2pNRVbguXBtJEcpjd 2fREDSya3zsx/l5s3Gfg+chof0irn1e6ZmhybviQhAc2wchXgSXoRKQFx2jP9RwS3lP7 MZZ+Sfnkt1bF95xRR8TxlUbEZ/aIfVA+Hbe5L7dOHAluBT94t67AKJtPym07ML56j67K WYQS38NJbW+LSuO078wAKHjJO1INNJFDiwIDR+1sQXdEKN5Gff0GSmGSewZCYdzjBxyj QKXw== X-Gm-Message-State: AC+VfDyJGD03mtPjFNLBT0iDUoiQDos8TbOkKt97bg/UHUFZZqQbQFwr hfz/tl95rrh/QZtnH9+n8Vg= X-Received: by 2002:a05:6a20:4291:b0:100:efa1:eab0 with SMTP id o17-20020a056a20429100b00100efa1eab0mr31177918pzj.2.1684121223711; Sun, 14 May 2023 20:27:03 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id s6-20020a632146000000b0051416609fb7sm10363825pgm.61.2023.05.14.20.27.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:27:03 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 1/4] rethook: use preempt_{disable, enable}_notrace in rethook_trampoline_handler Date: Mon, 15 May 2023 11:26:38 +0800 Message-Id: X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765932120842597319?= X-GMAIL-MSGID: =?utf-8?q?1765932120842597319?= This patch replace preempt_{disable, enable} with its corresponding notrace version in rethook_trampoline_handler so no worries about stack recursion or overflow introduced by preempt_count_{add, sub} under fprobe + rethook context. Signed-off-by: Ze Gao Acked-by: Masami Hiramatsu (Google) --- kernel/trace/rethook.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c index 32c3dfdb4d6a..60f6cb2b486b 100644 --- a/kernel/trace/rethook.c +++ b/kernel/trace/rethook.c @@ -288,7 +288,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, * These loops must be protected from rethook_free_rcu() because those * are accessing 'rhn->rethook'. */ - preempt_disable(); + preempt_disable_notrace(); /* * Run the handler on the shadow stack. Do not unlink the list here because @@ -321,7 +321,7 @@ unsigned long rethook_trampoline_handler(struct pt_regs *regs, first = first->next; rethook_recycle(rhn); } - preempt_enable(); + preempt_enable_notrace(); return correct_ret_addr; } From patchwork Mon May 15 03:26:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93846 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6640914vqo; Sun, 14 May 2023 20:41:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5HnWOIV+LCg7Bkc/4hfJgdJiOgPOPqzRxMzb/fvB798afmSYlKGm6eWVZtm1O40KHGAxa/ X-Received: by 2002:a05:6a20:394d:b0:104:8824:74eb with SMTP id r13-20020a056a20394d00b00104882474ebmr11665916pzg.51.1684122119138; Sun, 14 May 2023 20:41:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684122119; cv=none; d=google.com; s=arc-20160816; b=fvVgoJGXQcTjeSAD644KOdVsFbT2Zk1G6bKZCpdrK/boxaMtKtPERLvJH/iLjQzUnF qQHRwcAX2E20mMo44uxX/i2DPoNwdRPQ2klgDiDgp/yKAnWr/P78w87LP/bq+WQkGRMV zBJNDY0Ah3bMNr1oIjdbFX/sto9iCFCgjVYsx8XJ00dorKChu5V+c4x62ZdAlUhKwST1 gFYGwt3ypwdaPJJqK+69SmF3AlrtNyhgb2szyH/pc2Ntxt1S8qXrdhMd26ez0svHAPkI bwkFuSGMaTL7xH6elNBd2G1UJEymt6rWeg7cyMkIVgdLfhEqBx9HaUW0NRGMH9hM0aD2 N/kA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LiiaLoe2FHTpcejFRmixK8pQOtJa58cAbjMGJsmgl/s=; b=tAEPEuF6rcMyCpsPX8wyMGe866adfJTgF6X8exC4VQC0oRJABzm92zVTbxr7pC7dxB d9CmhdBKn05HGG/YShI62JMx+nAy0QEbpRSIyxJIoRwNKW+b3c/3JfNtuHNo3JiQoAQl EH2PBhJ0FN2KVun8B3BPujmPZVb/MI7hICFcpRUqB8d9kjfue3k6EchHgN/IYL9tJDj5 gBoyb92YuYkz7K3mB4P3lvCp6VMJeWVr5t+YNYEj7chzqyJoSTwr8PbbSXiCN4OcomPb lWAWkQq2yOpvfNZ6uj5hBmIDof+wUJmF+m0xR/Q6DVDw88hN7Ssjnz6Jr8pkekjQpx02 1WPg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=rr80hkcj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bs130-20020a632888000000b0051b9a71329fsi14218201pgb.360.2023.05.14.20.41.47; Sun, 14 May 2023 20:41:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=rr80hkcj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238294AbjEOD1M (ORCPT + 99 others); Sun, 14 May 2023 23:27:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237723AbjEOD1I (ORCPT ); Sun, 14 May 2023 23:27:08 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B8361997; Sun, 14 May 2023 20:27:07 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-64395e741fcso12688226b3a.2; Sun, 14 May 2023 20:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684121226; x=1686713226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LiiaLoe2FHTpcejFRmixK8pQOtJa58cAbjMGJsmgl/s=; b=rr80hkcjv0wdh0TWDxyo+jfz7J/dCaLoIrPfhOEfAZsmxb2C9sfdYQBYLt27Wpycly hAAJWHk5GTOFY06VkHnCW3Y9W9unD/aJmf+gZ1CadLq0VSE+DcGoZlPHZB0yURl9Oy0p zTUzWL7T8lPZx2ODFhnOAHV+sYGMR+WsYHHI/GHJMLrkMW6zaZbFOsBULsPZWekocpR6 LamOP/q7Ux1HCELYDetYBQL0V8zN9r8oTu4qGLtKzzDv8zHhlYGvkjqqLxqKjG6gKvdP fjnhdZcY5ZbPSMmLjtrBgbl+3S4/FA3L1zKoeCZ4I36ljlPbdczKtaVSdL2VMHBptwGv naQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684121226; x=1686713226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LiiaLoe2FHTpcejFRmixK8pQOtJa58cAbjMGJsmgl/s=; b=Ufm0CxBFYO4/ucbeBQXrkYGE9Vvu2oMN65zhdWBY+VDHeTcCOd2nz/SKdqbZ3EvS5X UXbjAPUg9Lm1pRqZVQzuMHkYTFNdBOY7MZ2yossp3zOB6dzKXFxRQDMdwBA7Y7ijXHQ7 3RYrcUB6HiJI8lInndgwbwEs/DHd2SKUQBjk8ZJyCuIsG10+4y9OtGBCMFmL2KZK8Cv8 Hq96aicVEjPhpKh+FAKp2ATUo3JEv1/DbvN+mW2vnrLdk+LFa2xFrr6bJ9U5YiPehYxM XYIeroZTK/7u1LLXiCYpt3bnb3wMQN1QrGud6Mug6RkAitkog8ArBQNfdsIblOh+LvL9 984w== X-Gm-Message-State: AC+VfDzKFttLt7SPpZ3jPi4Od8xDSDaM8OLuKXhHDNxyuSIKaOgb+8c+ O4hF3SAa7GfCZLt0Vc1eO1A= X-Received: by 2002:a05:6a00:168b:b0:63d:2d7d:b6f2 with SMTP id k11-20020a056a00168b00b0063d2d7db6f2mr43084736pfc.4.1684121226572; Sun, 14 May 2023 20:27:06 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id s6-20020a632146000000b0051416609fb7sm10363825pgm.61.2023.05.14.20.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:27:06 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 2/4] fprobe: make fprobe_kprobe_handler recursion free Date: Mon, 15 May 2023 11:26:39 +0800 Message-Id: <6c428b9f74090b2780b67ca498c45aa5e8c38c44.1684120990.git.zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765930035150107469?= X-GMAIL-MSGID: =?utf-8?q?1765930035150107469?= Current implementation calls kprobe related functions before doing ftrace recursion check in fprobe_kprobe_handler, which opens door to kernel crash due to stack recursion if preempt_count_{add, sub} is traceable. Refactor the common part out of fprobe_kprobe_handler and fprobe_ handler and call ftrace recursion detection at the very beginning, and also mark these functions notrace so that the whole fprobe_k- probe_handler is free from recusion. And Signed-off-by: Ze Gao --- kernel/trace/fprobe.c | 61 +++++++++++++++++++++++++++++++------------ 1 file changed, 45 insertions(+), 16 deletions(-) diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index 9abb3905bc8e..ad9a36c87ad9 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -20,30 +20,22 @@ struct fprobe_rethook_node { char data[]; }; -static void fprobe_handler(unsigned long ip, unsigned long parent_ip, - struct ftrace_ops *ops, struct ftrace_regs *fregs) +static inline notrace void __fprobe_handler(unsigned long ip, unsigned long + parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { struct fprobe_rethook_node *fpr; struct rethook_node *rh = NULL; struct fprobe *fp; void *entry_data = NULL; - int bit, ret; + int ret; fp = container_of(ops, struct fprobe, ops); - if (fprobe_disabled(fp)) - return; - - bit = ftrace_test_recursion_trylock(ip, parent_ip); - if (bit < 0) { - fp->nmissed++; - return; - } if (fp->exit_handler) { rh = rethook_try_get(fp->rethook); if (!rh) { fp->nmissed++; - goto out; + return; } fpr = container_of(rh, struct fprobe_rethook_node, node); fpr->entry_ip = ip; @@ -61,23 +53,60 @@ static void fprobe_handler(unsigned long ip, unsigned long parent_ip, else rethook_hook(rh, ftrace_get_regs(fregs), true); } -out: +} + +static void notrace fprobe_handler(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *ops, struct ftrace_regs *fregs) +{ + struct fprobe *fp; + int bit; + + fp = container_of(ops, struct fprobe, ops); + if (fprobe_disabled(fp)) + return; + + /* recursion detection has to go before any traceable function and + * all functions before this point should be marked as notrace + */ + bit = ftrace_test_recursion_trylock(ip, parent_ip); + if (bit < 0) { + fp->nmissed++; + return; + } + __fprobe_handler(ip, parent_ip, ops, fregs); ftrace_test_recursion_unlock(bit); + } NOKPROBE_SYMBOL(fprobe_handler); -static void fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip, +static void notrace fprobe_kprobe_handler(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - struct fprobe *fp = container_of(ops, struct fprobe, ops); + struct fprobe *fp; + int bit; + + fp = container_of(ops, struct fprobe, ops); + if (fprobe_disabled(fp)) + return; + + /* recursion detection has to go before any traceable function and + * all functions called before this point should be marked as notrace + */ + bit = ftrace_test_recursion_trylock(ip, parent_ip); + if (bit < 0) { + fp->nmissed++; + return; + } if (unlikely(kprobe_running())) { fp->nmissed++; return; } + kprobe_busy_begin(); - fprobe_handler(ip, parent_ip, ops, fregs); + __fprobe_handler(ip, parent_ip, ops, fregs); kprobe_busy_end(); + ftrace_test_recursion_unlock(bit); } static void fprobe_exit_handler(struct rethook_node *rh, void *data, From patchwork Mon May 15 03:26:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93848 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6652300vqo; Sun, 14 May 2023 21:12:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6Xkk08ALwaH3K6o5E7iY9SuvItJ/7Wfz6q4yaZpBxVaqtuX6ZsZw/TFeRo87YLJssIWy02 X-Received: by 2002:a17:90a:6089:b0:24e:3a0a:9b48 with SMTP id z9-20020a17090a608900b0024e3a0a9b48mr32675732pji.21.1684123977587; Sun, 14 May 2023 21:12:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684123977; cv=none; d=google.com; s=arc-20160816; b=Je4LmOwizzxfGmdofGjS0IK5avdmNIMlyGndOq9ASTN4K+dkj9PjjTH6ai93L7GbtM Hm0K9/XJiw4JvPZ5KVDHZQnuxptxPWc5WQxDpRNO5bp2HFC29AHlRjdCrVJbWEHjLxTa ExiBeWcNaXA1fbHd2zzZie2hpViD+54zbX8EB2Yyc5oraUuTLu+KxrBCuMvuv1Ml90IO IdfMrHgxAirNps/wn5xc8xA3WKxewWanzaE9NEmHvTpG86d6WwSOfyox7B4UBlH33cqX 4JlMn+0V9TJw50etyH9A9UU8Pr9SO+GbNkplb3FE9RyvhpA20vsxn3xwEDktfoLtmiwC nohQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JzkH4lQfb5ZOUjwqO+AdNbxgdgLDuSyyuCXzioTUQhE=; b=0f9x1Dk/cfHW3blxJhShtrm1AB0yRuYcOeZhmIyzLQ9leACZLu3CC7tKfMuCKZD+pD IoSNowpfmCQgJ/t2bibV3J3S1rpgmFexDUXozv/6eKlVla2qB6fqzx4tSBzLJrECTehp 72FDiJJGilGGXr6IDVEUSoZIAxwCUNot1o0MVc/bx09rlGX0oPH+PjOx9QCh/LcFHw0F OdjGHy7J4JDskKBP7I3v7AD9BTnQUOgHn0RcTwxsH01+sMNQ3L8LTFxmMTkXewkBnbvG p8YcugmTct2mda4tNVGLBhIGOxgswQ6/OUJp+VL2pezX4IgUMn/IozY26vQCpWUiNX3b uGWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=aq2446Ts; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i8-20020a639d08000000b0051ba2ec8320si15175151pgd.353.2023.05.14.21.12.41; Sun, 14 May 2023 21:12:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=aq2446Ts; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238343AbjEOD1P (ORCPT + 99 others); Sun, 14 May 2023 23:27:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229701AbjEOD1L (ORCPT ); Sun, 14 May 2023 23:27:11 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E898819BE; Sun, 14 May 2023 20:27:09 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-24dea6d5ce8so11363511a91.2; Sun, 14 May 2023 20:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684121229; x=1686713229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JzkH4lQfb5ZOUjwqO+AdNbxgdgLDuSyyuCXzioTUQhE=; b=aq2446TsX1Ld9+p66vC4YJ1ME7o1BG1TAYj2c/1PsW+2SjkODlqrO+giTcyvd1epTe KogHvLh033R2cIiXHqpGnJ9IltV/aXGYGkO0v1lc6aai2yysg/1r3sYX0tftiNlYKQja KFxC1LbVFDp6Q/K3rZovzXTg5P+fS4iI7ilPipDoid9MLdSQn5Ux8RJJZ/CpiN1mLQNW py4pZ++8XB7NDzxEPxHocZCz3Opi59bIo6w/gEf3KVUMRdYk+zSgkmFMX9RRpvDnocmv Mcib1fWSXc04nH6Qt8+3kF0chdPUsJI4uwOf5LEApurHqKMViXvQmHu4izxq5xutVvAN /kxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684121229; x=1686713229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JzkH4lQfb5ZOUjwqO+AdNbxgdgLDuSyyuCXzioTUQhE=; b=Z1jO0nZr8M1gkmzm2DV+D+BKbqMMO0sMtLnXtR9Z+DHzDQ8UHRfq5owJWfGcod67Qy 5rNbXoYtsRRZ0IOd+o6SnRXJ2uS8hCptfC7dBZi/yddxmCaQk76aNpcYde9GgBZTkjp1 0ZyfUjoead1Kes6CH/Aw+lrVaFvzVjlp7Hu/BA9W1teN6q6r0mwM1buPcada/f+OF6X9 sRHP0xLP2pWr1syYQJ38s7uMSx+PR1K1ee4DnK4gvvRUKuCDUZA37FsbO8a3zHj42GnJ hsrDlwwAlkrMAP3yeS0htC4pTtXO3CSgJ5s6wkHiyn5wVMBG09SxYGz0RZxhBSyYPlcH vdkA== X-Gm-Message-State: AC+VfDzOt8G+CD3jPvhLbDBIGUpWlxzuvGLOezVzhLAF0JNBRIi+SxKs FR2ZajioY/uKrqDmAYNklbM= X-Received: by 2002:a17:90a:ea82:b0:22b:b832:d32 with SMTP id h2-20020a17090aea8200b0022bb8320d32mr33674815pjz.9.1684121229377; Sun, 14 May 2023 20:27:09 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id s6-20020a632146000000b0051416609fb7sm10363825pgm.61.2023.05.14.20.27.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:27:09 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 3/4] fprobe: add recursion detection in fprobe_exit_handler Date: Mon, 15 May 2023 11:26:40 +0800 Message-Id: <5f8081030f6f5d5af56c93fff95f0a7fadde04ad.1684120990.git.zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765931983679448795?= X-GMAIL-MSGID: =?utf-8?q?1765931983679448795?= fprobe_hander and fprobe_kprobe_handler has guarded ftrace recusion detection but fprobe_exit_handler has not, which possibly introduce recurive calls if the fprobe exit callback calls any traceable functions. Checking in fprobe_hander or fprobe_kprobe_handler is not enough and misses this case. So add recusion free guard the same way as fprobe_hander and also mark fprobe_exit_handler notrace. Since ftrace recursion check does not employ ips, so here use entry_ip and entry_parent_ip the same as fprobe_handler. Signed-off-by: Ze Gao Acked-by: Masami Hiramatsu (Google) --- kernel/trace/fprobe.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/kernel/trace/fprobe.c b/kernel/trace/fprobe.c index ad9a36c87ad9..cf982d4ab142 100644 --- a/kernel/trace/fprobe.c +++ b/kernel/trace/fprobe.c @@ -17,6 +17,7 @@ struct fprobe_rethook_node { struct rethook_node node; unsigned long entry_ip; + unsigned long entry_parent_ip; char data[]; }; @@ -39,6 +40,7 @@ static inline notrace void __fprobe_handler(unsigned long ip, unsigned long } fpr = container_of(rh, struct fprobe_rethook_node, node); fpr->entry_ip = ip; + fpr->entry_parent_ip = parent_ip; if (fp->entry_data_size) entry_data = fpr->data; } @@ -109,19 +111,30 @@ static void notrace fprobe_kprobe_handler(unsigned long ip, unsigned long parent ftrace_test_recursion_unlock(bit); } -static void fprobe_exit_handler(struct rethook_node *rh, void *data, +static void notrace fprobe_exit_handler(struct rethook_node *rh, void *data, struct pt_regs *regs) { struct fprobe *fp = (struct fprobe *)data; struct fprobe_rethook_node *fpr; + int bit; if (!fp || fprobe_disabled(fp)) return; fpr = container_of(rh, struct fprobe_rethook_node, node); + /* we need to assure no calls to traceable functions in-between the + * end of fprobe_handler and the beginning of fprobe_exit_handler. + */ + bit = ftrace_test_recursion_trylock(fpr->entry_ip, fpr->entry_parent_ip); + if (bit < 0) { + fp->nmissed++; + return; + } + fp->exit_handler(fp, fpr->entry_ip, regs, fp->entry_data_size ? (void *)fpr->data : NULL); + ftrace_test_recursion_unlock(bit); } NOKPROBE_SYMBOL(fprobe_exit_handler); From patchwork Mon May 15 03:26:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ze Gao X-Patchwork-Id: 93844 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp6638110vqo; Sun, 14 May 2023 20:33:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5I/uwi1EzCwtJ0b3V05q/H7RZJgvdge3zbykxbhw/npKaOCunRtJgJwoiVxUvUmshvo9HM X-Received: by 2002:a17:902:f68a:b0:1ac:788c:872a with SMTP id l10-20020a170902f68a00b001ac788c872amr31627815plg.1.1684121636716; Sun, 14 May 2023 20:33:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684121636; cv=none; d=google.com; s=arc-20160816; b=kGxFS3VZqer1bFifuctnKyznwM470+9n19EZ1Xz/RF4FLbyPrHpjgL5RnRPS3WbK/s s2u/5psOOmHTnx0kJ4MBQyJeLBKX2QuEJc6SEZMF11O9qvkhjhcrF0t6Qq2fe7vRz3O6 xgdCwFCWLHsemjEvCRugjf1/xbsH3C2uetEPN620iodDR1e5RqsLVnOU65SVLSZ8HtSg fc5lBGr0d7Ey4En7OmPMRBs+8emC33ZHeqpz+MLM2hqaBvYGjNRyI5+68cB5xCXCfcbi VDvjWg2mVFrhJIxPGvzWbuXGPSUTq+4pnFZlB77c9aeh3Y3mM1bmT/TGlN044mkVppBP 85sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Misk8RzcAuhujPj5CN4/zB0H7gvh8yXqb60mLrr5oOQ=; b=nQcspn7n9OtNzOTBQdH5nS8lWW3rflw7j51i2gdGLkXuiIp4Op7kAyNREW/FvtVfC/ FF4MaHwZhzp4DAQjwMepQYW4lZXRMNB8zbGioTzS6CUbR8fRUxAtOcWj3mJilBv0JxSM AlfDmYqrJUNp13z2ZEDm82y3zVRDmRkypIVCgfPDjanfDXffmE3upnPXkalLnNaXflmA Dj1DFQbSV5wH0YbazHZ++Y57+dk8vscwGnE03COAYiVr75/tv/vrz+YIF5LY25RSjcDF UGqJuDQ/ehDKcNYndWqZPjfJ/0UyhfGP/9F59LSZv//W6tWThtaPN/WwwlzbjLcQkjPR svVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=SUTYpi9m; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c15-20020a170902d48f00b001aaecc8a84dsi15704003plg.645.2023.05.14.20.33.42; Sun, 14 May 2023 20:33:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=SUTYpi9m; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238567AbjEOD13 (ORCPT + 99 others); Sun, 14 May 2023 23:27:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237723AbjEOD1X (ORCPT ); Sun, 14 May 2023 23:27:23 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF96019B2; Sun, 14 May 2023 20:27:15 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-64ab2a37812so16612128b3a.1; Sun, 14 May 2023 20:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684121235; x=1686713235; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Misk8RzcAuhujPj5CN4/zB0H7gvh8yXqb60mLrr5oOQ=; b=SUTYpi9m7sfB03krQSEVvZC8BAKoBAFKxMDYCydpITO/I8zUiiyk5Kzo+jNSg95b6v RhIw64DX8qMnXWvKy9BNDKbpwJyFCs1eZwm3295yCaUvKJgzf+2aYHJ8skA18ZY7Y9NG mvxd5TH9uewZGkdyp6C+yQy21hckWwkYwB+4u3eU94xPVquK6qutXALcwB/fmAZ6sIPJ KecxVHrKXwVRQjtpPenLWz035ZHCciWD97jD3PVH2NdwOESiJDOUOtONJGRC4osglUR1 0+ZJgF8NZxTf74J2fP3+p4HBsS5RTDF7qMqjhz6gE6p19cjT7K+BAyvT1rMEnas+TqeB bdQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684121235; x=1686713235; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Misk8RzcAuhujPj5CN4/zB0H7gvh8yXqb60mLrr5oOQ=; b=Erv9su7KgkzKQuU2TRIUd4vCrq/geQ9fqcCuFamqtBwnAYwbiN81pdaKfL4TE4D02X i8ekJLo/qIiawkGTr6RbwsQZPVHJmrk3gfP4wt0hr+Df8hQOkhs3DqPsi9bxmvkG4PxV WAezEXcZm3Oj+eYhLbM/Y9jGzrCCSfJo+Or8LtR1AtuaqMp0pSjIvximZZMYDZ01zd0Y YDinLX51kOY/O+tECwVoDYU/WeVMMYTXjh1ZiZV08FWx+heFEODeCFdRe3dby615Uso6 FnXycTBspsO0Tw6RZYBI/dhrjJMc7Ov6kMnSVxVKCXrgvqtooXGBk8uzsSdijwkcFCLE lGQA== X-Gm-Message-State: AC+VfDxtypHArS/ttUgNDzfXMVE1qJnlrc/cNsx58uQdjNFFAmfwU86S 0oN+/ZI6EydgIu+NzWpmBW0= X-Received: by 2002:a17:90a:ad09:b0:244:d441:8f68 with SMTP id r9-20020a17090aad0900b00244d4418f68mr44442897pjq.16.1684121235252; Sun, 14 May 2023 20:27:15 -0700 (PDT) Received: from localhost.localdomain ([43.132.98.42]) by smtp.googlemail.com with ESMTPSA id s6-20020a632146000000b0051416609fb7sm10363825pgm.61.2023.05.14.20.27.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 20:27:14 -0700 (PDT) From: Ze Gao X-Google-Original-From: Ze Gao To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu Cc: Ze Gao , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 4/4] rehook, fprobe: mark rethook related functions notrace Date: Mon, 15 May 2023 11:26:41 +0800 Message-Id: <238bad4335d029072ca6000fb404f47376197f39.1684120990.git.zegao@tencent.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765929529090233736?= X-GMAIL-MSGID: =?utf-8?q?1765929529090233736?= These functions are already marked as NOKPROBE to prevent recusion and we have the same reason to blacklist them if rethook is used with fprobe, since they are beyond the recursion-free region ftrace can guard. Signed-off-by: Ze Gao --- arch/riscv/kernel/probes/rethook.c | 4 ++-- arch/s390/kernel/rethook.c | 6 +++--- arch/x86/kernel/rethook.c | 8 +++++--- kernel/trace/rethook.c | 8 ++++---- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/riscv/kernel/probes/rethook.c b/arch/riscv/kernel/probes/rethook.c index 5c27c1f50989..803c412a1bea 100644 --- a/arch/riscv/kernel/probes/rethook.c +++ b/arch/riscv/kernel/probes/rethook.c @@ -8,14 +8,14 @@ #include "rethook.h" /* This is called from arch_rethook_trampoline() */ -unsigned long __used arch_rethook_trampoline_callback(struct pt_regs *regs) +unsigned long __used notrace arch_rethook_trampoline_callback(struct pt_regs *regs) { return rethook_trampoline_handler(regs, regs->s0); } NOKPROBE_SYMBOL(arch_rethook_trampoline_callback); -void arch_rethook_prepare(struct rethook_node *rhn, struct pt_regs *regs, bool mcount) +void notrace arch_rethook_prepare(struct rethook_node *rhn, struct pt_regs *regs, bool mcount) { rhn->ret_addr = regs->ra; rhn->frame = regs->s0; diff --git a/arch/s390/kernel/rethook.c b/arch/s390/kernel/rethook.c index af10e6bdd34e..ad52119826c1 100644 --- a/arch/s390/kernel/rethook.c +++ b/arch/s390/kernel/rethook.c @@ -3,7 +3,7 @@ #include #include "rethook.h" -void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) +void notrace arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) { rh->ret_addr = regs->gprs[14]; rh->frame = regs->gprs[15]; @@ -13,7 +13,7 @@ void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mc } NOKPROBE_SYMBOL(arch_rethook_prepare); -void arch_rethook_fixup_return(struct pt_regs *regs, +void notrace arch_rethook_fixup_return(struct pt_regs *regs, unsigned long correct_ret_addr) { /* Replace fake return address with real one. */ @@ -24,7 +24,7 @@ NOKPROBE_SYMBOL(arch_rethook_fixup_return); /* * Called from arch_rethook_trampoline */ -unsigned long arch_rethook_trampoline_callback(struct pt_regs *regs) +unsigned long notrace arch_rethook_trampoline_callback(struct pt_regs *regs) { return rethook_trampoline_handler(regs, regs->gprs[15]); } diff --git a/arch/x86/kernel/rethook.c b/arch/x86/kernel/rethook.c index 8a1c0111ae79..1f7cef86f73d 100644 --- a/arch/x86/kernel/rethook.c +++ b/arch/x86/kernel/rethook.c @@ -64,7 +64,8 @@ NOKPROBE_SYMBOL(arch_rethook_trampoline); /* * Called from arch_rethook_trampoline */ -__used __visible void arch_rethook_trampoline_callback(struct pt_regs *regs) +__used __visible void notrace arch_rethook_trampoline_callback(struct pt_regs + *regs) { unsigned long *frame_pointer; @@ -104,7 +105,7 @@ NOKPROBE_SYMBOL(arch_rethook_trampoline_callback); STACK_FRAME_NON_STANDARD_FP(arch_rethook_trampoline); /* This is called from rethook_trampoline_handler(). */ -void arch_rethook_fixup_return(struct pt_regs *regs, +void notrace arch_rethook_fixup_return(struct pt_regs *regs, unsigned long correct_ret_addr) { unsigned long *frame_pointer = (void *)(regs + 1); @@ -114,7 +115,8 @@ void arch_rethook_fixup_return(struct pt_regs *regs, } NOKPROBE_SYMBOL(arch_rethook_fixup_return); -void arch_rethook_prepare(struct rethook_node *rh, struct pt_regs *regs, bool mcount) +void notrace arch_rethook_prepare(struct rethook_node *rh, struct pt_regs + *regs, bool mcount) { unsigned long *stack = (unsigned long *)regs->sp; diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c index 60f6cb2b486b..e551e86d3927 100644 --- a/kernel/trace/rethook.c +++ b/kernel/trace/rethook.c @@ -127,7 +127,7 @@ static void free_rethook_node_rcu(struct rcu_head *head) * Return back the @node to @node::rethook. If the @node::rethook is already * marked as freed, this will free the @node. */ -void rethook_recycle(struct rethook_node *node) +void notrace rethook_recycle(struct rethook_node *node) { lockdep_assert_preemption_disabled(); @@ -194,7 +194,7 @@ void rethook_hook(struct rethook_node *node, struct pt_regs *regs, bool mcount) NOKPROBE_SYMBOL(rethook_hook); /* This assumes the 'tsk' is the current task or is not running. */ -static unsigned long __rethook_find_ret_addr(struct task_struct *tsk, +static unsigned long notrace __rethook_find_ret_addr(struct task_struct *tsk, struct llist_node **cur) { struct rethook_node *rh = NULL; @@ -256,7 +256,7 @@ unsigned long rethook_find_ret_addr(struct task_struct *tsk, unsigned long frame } NOKPROBE_SYMBOL(rethook_find_ret_addr); -void __weak arch_rethook_fixup_return(struct pt_regs *regs, +void __weak notrace arch_rethook_fixup_return(struct pt_regs *regs, unsigned long correct_ret_addr) { /* @@ -268,7 +268,7 @@ void __weak arch_rethook_fixup_return(struct pt_regs *regs, } /* This function will be called from each arch-defined trampoline. */ -unsigned long rethook_trampoline_handler(struct pt_regs *regs, +unsigned long notrace rethook_trampoline_handler(struct pt_regs *regs, unsigned long frame) { struct llist_node *first, *node = NULL;