From patchwork Tue Feb 7 18:21:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 54032 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp3005156wrn; Tue, 7 Feb 2023 10:23:49 -0800 (PST) X-Google-Smtp-Source: AK7set+LqO2oYGAle4Yh7pFLBzeYHu29zl2S6qVT3XCJ7aREODec1c3thJYnInN1zmWnMSPmXE6p X-Received: by 2002:a17:906:8055:b0:884:d15e:10f8 with SMTP id x21-20020a170906805500b00884d15e10f8mr4468970ejw.51.1675794229221; Tue, 07 Feb 2023 10:23:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675794229; cv=none; d=google.com; s=arc-20160816; b=xDvp67A+4jHPqxYed7Gzb/TfS5kiVzb372PXLm5D8dJXBavUnGUKOiXXCOrrh2kN7h lizurED4IK4/3NIGiiOdmL7nqTVAwK1s39SUVnW6UBI2zkXXjjXzQIsVYNjAgrDKcqHU /xKJehezzt60nRblZiYFJ/I/C6RRg9Y6BCY1euxUunpCjUyp9NUYPmw4UrJ1Cj+X7qra 8jv6IIL/yHYmMwgTtzdCk3iFIpZS2Qvzdeccypr5julbR5278I2it2MdWrUb87uOZ3Q4 UcfGtyBrODPxLStCfJNywIws+SfetuKdqskJWhGQV5h1VHaBEPGZQLTdZmXROlZHAu6d OvuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=J5NwLdTe9+0vZLgVBry/0OWW8ooKZkaJGOHBRX4ufN8=; b=VgCS4Vu3EXkKtvJecSpo7F+aDoqU9ZhiyYNkFpAw/q/XdHQeAN2tZhrrjgDqOa7Q3X zcCPG8d2TqdBRFssnoJCU4DYFeaPgnBwvJeCWH9SS293pdcHBon3NE6d1g1Eu+ez1qnQ Pd7kuv1S29yVN4x8TXQHeTLYkICCHRtrDRtWtMrjZGnnMoErSGvTd4KODze/GkosWij8 6/L2U72GlowTEYU8Mxkh/Ed6KE2fncZ/5uKvs956MXy5/MfXu38wxjvgsTFZ7fHJk8wx VAQPJhA1mjUV9FKwwlcp5uiSnXHXDO4ezLjJkxp+hNkgpi9LT88dQ3L2JYL/SCBSWFTN R/BQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=fRASL2UG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cp5-20020a17090793c500b0084b9b15e1cfsi15288974ejc.41.2023.02.07.10.23.26; Tue, 07 Feb 2023 10:23:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=fRASL2UG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232516AbjBGSXI (ORCPT + 99 others); Tue, 7 Feb 2023 13:23:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231843AbjBGSWj (ORCPT ); Tue, 7 Feb 2023 13:22:39 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 059AE3FF3B for ; Tue, 7 Feb 2023 10:22:08 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id hn2-20020a05600ca38200b003dc5cb96d46so13809702wmb.4 for ; Tue, 07 Feb 2023 10:22:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J5NwLdTe9+0vZLgVBry/0OWW8ooKZkaJGOHBRX4ufN8=; b=fRASL2UGIcRNBlf/7ae//8RHK3T5XjSoyr3hRCQQ/pNn5tg2MlxAaLTHUyrgXCZpKU AHPIQD8hI/OFXApHVT6ZRDjG9/QWKH7JrARdfDNLfSpOI8BsnkXlG47EYE5JYOHPQGyJ x4yiblaJF2UM7yX6fO6T+866rFk5FSIG2vfvw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J5NwLdTe9+0vZLgVBry/0OWW8ooKZkaJGOHBRX4ufN8=; b=yzOenLsSSwuUMpOIjgAfSRb0N1io8t9JCh+EG63WylS4Vq/E134Y18KqhxyOfRjBVd U0BTc4/f5lF7nkPl/yH191sb99/cBM5tNtTlDhTafZJ91cE4QPDxKr95udKaT/heUH2B gOVqVVYNCh7g7y6YdZq0C1hzENmKdJ/GMlpaKT/ZKH8ni5qFKthabO6vNXHwWqiu0q28 Be6+DYj9/v2AUfeLdNJpQtiF9k2NM42qA5E2i35qqB5vHv/5iqQcKwz3AlVu/aaTUerN +IPY84DO+qve0YackWUfd8g8Kt7YleJf0kere4guwtcXe9CJBwdGlss18iFCFx+2nr58 0pBw== X-Gm-Message-State: AO0yUKXwwLffFpZOSqS7+aIPyzojBXGpJnIlGbu50KT9vNYADlSkpkvH 3h7Fdo6cUPTN2cN43PQET5pydT+CwMiHHnPms1k= X-Received: by 2002:a05:600c:30ca:b0:3df:c284:7e78 with SMTP id h10-20020a05600c30ca00b003dfc2847e78mr4149127wmn.38.1675794126436; Tue, 07 Feb 2023 10:22:06 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:5307:c0c0:ff97:80de]) by smtp.gmail.com with ESMTPSA id n6-20020a05600c4f8600b003daf672a616sm15578369wmq.22.2023.02.07.10.22.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Feb 2023 10:22:06 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, lihuafei1@huawei.com, Florent Revest Subject: [PATCH v2 04/10] ftrace: Store direct called addresses in their ops Date: Tue, 7 Feb 2023 19:21:29 +0100 Message-Id: <20230207182135.2671106-5-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230207182135.2671106-1-revest@chromium.org> References: <20230207182135.2671106-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757197609446887576?= X-GMAIL-MSGID: =?utf-8?q?1757197609446887576?= All direct calls are now registered using the register_ftrace_direct API so each ops can jump to only one direct-called trampoline. By storing the direct called trampoline address directly in the ops we can save one hashmap lookup in the direct call ops and implement arm64 direct calls on top of call ops. Signed-off-by: Florent Revest --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a7dbd307c3a4..84f717f8959e 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -321,6 +321,9 @@ struct ftrace_ops { unsigned long trampoline_size; struct list_head list; ftrace_ops_func_t ops_func; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_call; +#endif #endif }; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index cb77a0a208c7..dfa5f34ec320 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2577,9 +2577,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - unsigned long addr; + unsigned long addr = ops->direct_call; - addr = ftrace_find_rec_direct(ip); if (!addr) return; @@ -5375,6 +5374,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) ops->func = call_direct_funcs; ops->flags = MULTI_FLAGS; ops->trampoline = FTRACE_REGS_ADDR; + ops->direct_call = addr; err = register_ftrace_function_nolock(ops); @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) /* Enable the tmp_ops to have the same functions as the direct ops */ ftrace_ops_init(&tmp_ops); tmp_ops.func_hash = ops->func_hash; + tmp_ops.direct_call = addr; err = register_ftrace_function_nolock(&tmp_ops); if (err) @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) entry->direct = addr; } } + WRITE_ONCE(ops->direct_call, addr); mutex_unlock(&ftrace_lock);