From patchwork Wed Feb 1 16:34:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51469 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp379774wrn; Wed, 1 Feb 2023 08:36:14 -0800 (PST) X-Google-Smtp-Source: AK7set++hhM8/IvAIBgPVO0CWXKYXZ+kx9v4haErBLavW/4zCSOw/XACy4a7Ggso3JTWrTnbOGp0 X-Received: by 2002:a05:6402:500c:b0:49e:4254:60a9 with SMTP id p12-20020a056402500c00b0049e425460a9mr3024170eda.29.1675269374690; Wed, 01 Feb 2023 08:36:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269374; cv=none; d=google.com; s=arc-20160816; b=XF2Vc5wuNcMNyMnT7326AocvlMAWEdciKX+tmXdGhoItykowDGrDpeugOnmKFNfNud 5sGFA+yxGw57i2Pt/bwu+1/J4WL3fq0Y3jbYfVKoqJdlIfoYfAnEFcGZeTdUIsCcxTXZ p4uORub0p3iKpgftEY0R9/FszREfuWv/MMcr1Wbo2/T482LJs20R1owjZKW+S+n9ffWq kGUfqaF4BFi3Zh3vgDnqhducG0PvMB/HP8+vT1rSNUo9PCisrSP9gZhn5wYqpyXtqanE 5s3uI5ypVOgqaeyc3FzpUn0KS2YvNnQaq+GQ5SGli17518RB9KPG18xTejRLCk+Uhbpn Q22w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mWFrI/ZwPkf24GvAi4zS93lAjO18wCo9ugFdCRD9vug=; b=mY6NJIv0Ic9yUWf9UjJVE7T8uzUGM8T31BVvQVEiFP0aZEjO3i3+8EapVTA3imuIAx RTMg/JKvnvwa+geHiPu2Z99eFItq2RCBKUBhSAaocJNZiTpdk6pru8ZhWmH8kmYLvrLI bcOocytsOxCQXkz23syVnNcO7EBWcbSo6oHQX+Ehjv1QN1OG1MtXdA2cqhqu3U0h5adQ 1DEKbMJGOG5LB3eA6a6pZGIqfSm8YNyGbiW8A9v71mhwMdyO+Lw08ew0rp/ZTS1P6Pns ZV0G1SS2ua5hZQT66y/i1aGngaVQMEPHaSgVRkbqqktIxEKcnh2f0H3fiDDsZve9i3/Q ExbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="R/wJRycx"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ev8-20020a056402540800b0049ea0e7ae42si21928807edb.393.2023.02.01.08.35.51; Wed, 01 Feb 2023 08:36:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="R/wJRycx"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232127AbjBAQfh (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232200AbjBAQfd (ORCPT ); Wed, 1 Feb 2023 11:35:33 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 628FFCA07 for ; Wed, 1 Feb 2023 08:35:25 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id j32-20020a05600c1c2000b003dc4fd6e61dso1822128wms.5 for ; Wed, 01 Feb 2023 08:35:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mWFrI/ZwPkf24GvAi4zS93lAjO18wCo9ugFdCRD9vug=; b=R/wJRycxVlXr98cXymYAxYn59y2w9nV7JoCbhkjevWZR1Cd6qPhPtO6JgzG3VxSI9Q GKjV0BjUpT2ySQyMbsatvRUEOMMnI83NorJqYQruLmLn9Dd7v4V8prfj6ZAjCD1ul2/+ TWRYbF22VUKSRARcJ2XvXOHcRQs/Rg82E+MiE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mWFrI/ZwPkf24GvAi4zS93lAjO18wCo9ugFdCRD9vug=; b=RqI51WrNZWMuARjmE7wzLv5/k9V/jfmJNCOuJmFsVOgtxDl44rYRGZu7qHUwcuuutw evI86Pq6jFpkE2snmjNK8SdZt90DcFbUQxMGwRSwNxiSq/9x0E55Piq70DT+ETM2F+d9 mf4pHAMLm2lq48JP/jqY8BboQ3A1pBl9i/tqsirsTtfGehjMrOteiaLqSYn98O8jY44T zt2TesSI6Vn755Pvzpak1U3hBqNwm08TGesIU/Suqy2Ci6Xbf4BwLMpriuFULIwTTbpL GQG8hfsHIocTZOci68diQVT3J8imu/NLvvjY2u+QZXmLE9etlIpq3TI9HCfJ1riZoj2A Pyvw== X-Gm-Message-State: AO0yUKUNgIJnGY3neeiWBzHv6+LpYEhTnQC8Td8Wm3FJgtlyqN8+fPWi sZKqxTgKxYiZc9Kawd6XR6gE+A== X-Received: by 2002:a05:600c:3544:b0:3dd:1c46:b92 with SMTP id i4-20020a05600c354400b003dd1c460b92mr2882937wmq.16.1675269323861; Wed, 01 Feb 2023 08:35:23 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:23 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 1/8] ftrace: Replace uses of _ftrace_direct APIs with _ftrace_direct_multi Date: Wed, 1 Feb 2023 17:34:13 +0100 Message-Id: <20230201163420.1579014-2-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647259865876040?= X-GMAIL-MSGID: =?utf-8?q?1756647259865876040?= The _multi API requires that users keep their own ops and can enforce that an op is only associated to one direct call. Signed-off-by: Florent Revest Acked-by: Mark Rutland Tested-by: Mark Rutland --- kernel/trace/trace_selftest.c | 9 ++++++--- samples/ftrace/ftrace-direct-modify.c | 11 +++++++---- samples/ftrace/ftrace-direct-too.c | 12 +++++++----- samples/ftrace/ftrace-direct.c | 12 +++++++----- 4 files changed, 27 insertions(+), 17 deletions(-) diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index ff0536cea968..9b7f10cbc51d 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -806,6 +806,9 @@ trace_selftest_startup_function_graph(struct tracer *trace, int ret; unsigned long count; char *func_name __maybe_unused; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + struct ftrace_ops direct = {}; +#endif #ifdef CONFIG_DYNAMIC_FTRACE if (ftrace_filter_param) { @@ -870,8 +873,8 @@ trace_selftest_startup_function_graph(struct tracer *trace, * Register direct function together with graph tracer * and make sure we get graph trace. */ - ret = register_ftrace_direct((unsigned long) DYN_FTRACE_TEST_NAME, - (unsigned long) trace_direct_tramp); + ftrace_set_filter_ip(&direct, (unsigned long)DYN_FTRACE_TEST_NAME, 0, 0); + ret = register_ftrace_direct_multi(&direct, (unsigned long)trace_direct_tramp); if (ret) goto out; @@ -891,7 +894,7 @@ trace_selftest_startup_function_graph(struct tracer *trace, unregister_ftrace_graph(&fgraph_ops); - ret = unregister_ftrace_direct((unsigned long) DYN_FTRACE_TEST_NAME, + ret = unregister_ftrace_direct_multi(&direct, (unsigned long) trace_direct_tramp); if (ret) goto out; diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c index de5a0f67f320..e1e6c286970c 100644 --- a/samples/ftrace/ftrace-direct-modify.c +++ b/samples/ftrace/ftrace-direct-modify.c @@ -96,6 +96,8 @@ asm ( #endif /* CONFIG_S390 */ +static struct ftrace_ops direct; + static unsigned long my_tramp = (unsigned long)my_tramp1; static unsigned long tramps[2] = { (unsigned long)my_tramp1, @@ -114,7 +116,7 @@ static int simple_thread(void *arg) if (ret) continue; t ^= 1; - ret = modify_ftrace_direct(my_ip, my_tramp, tramps[t]); + ret = modify_ftrace_direct_multi(&direct, tramps[t]); if (!ret) my_tramp = tramps[t]; WARN_ON_ONCE(ret); @@ -129,7 +131,8 @@ static int __init ftrace_direct_init(void) { int ret; - ret = register_ftrace_direct(my_ip, my_tramp); + ftrace_set_filter_ip(&direct, (unsigned long) my_ip, 0, 0); + ret = register_ftrace_direct_multi(&direct, my_tramp); if (!ret) simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn"); return ret; @@ -138,12 +141,12 @@ static int __init ftrace_direct_init(void) static void __exit ftrace_direct_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct(my_ip, my_tramp); + unregister_ftrace_direct_multi(&direct, my_tramp); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct()"); +MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct_multi()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c index e13fb59a2b47..0e907092e2c0 100644 --- a/samples/ftrace/ftrace-direct-too.c +++ b/samples/ftrace/ftrace-direct-too.c @@ -70,21 +70,23 @@ asm ( #endif /* CONFIG_S390 */ +static struct ftrace_ops direct; + static int __init ftrace_direct_init(void) { - return register_ftrace_direct((unsigned long)handle_mm_fault, - (unsigned long)my_tramp); + ftrace_set_filter_ip(&direct, (unsigned long) handle_mm_fault, 0, 0); + + return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct((unsigned long)handle_mm_fault, - (unsigned long)my_tramp); + unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct()"); +MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct_multi()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c index 1f769d0db20f..e446c38f6b58 100644 --- a/samples/ftrace/ftrace-direct.c +++ b/samples/ftrace/ftrace-direct.c @@ -63,21 +63,23 @@ asm ( #endif /* CONFIG_S390 */ +static struct ftrace_ops direct; + static int __init ftrace_direct_init(void) { - return register_ftrace_direct((unsigned long)wake_up_process, - (unsigned long)my_tramp); + ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); + + return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct((unsigned long)wake_up_process, - (unsigned long)my_tramp); + unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using register_ftrace_direct()"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); MODULE_LICENSE("GPL"); From patchwork Wed Feb 1 16:34:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51470 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp379821wrn; Wed, 1 Feb 2023 08:36:19 -0800 (PST) X-Google-Smtp-Source: AK7set+2cNeKrFJ8XX5BqrQ5yI7CEOhFNYEMu2D0nf7cEQ3nWO67lPJIpvgdX6NKe4xQm5Rp1iVW X-Received: by 2002:a17:906:885:b0:87b:d3fa:79f2 with SMTP id n5-20020a170906088500b0087bd3fa79f2mr3187521eje.19.1675269379194; Wed, 01 Feb 2023 08:36:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269379; cv=none; d=google.com; s=arc-20160816; b=Yzzy7LxGIXzPGr3y/BVoETciUIvGOf1Useq8QEj8FMxcTI01UG4rCgYqN+pYX6NnZP 1DkP1AUxBsIPhoKgitIgt6bdyAxAoVeYCGzCYJL3N+JRpE33JDTnrO774KQY4v6ODN3T WMFTdXBPtLj++hgeLVCmrf6ayLNwqGzFAkEg+EsYVPIkCdH4fIcoRklNX2opN9JZHe/i 8UyszPX0X8DARz38YukXO929c9uSqxlJOEEf+DIEsKANfxpdQbvVj9eAsmIPJI0da18F pBWXUMfCRaS6ffUXM+X06Vbi71gxXdthE4I7iJIS7CPh27AaaRuOYEnUGAPzBlrbvhB8 uO8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=X+Q37Omt/MAOZB85dyYH0VxCAqGyfL1V0CNFe50alWw=; b=dNYwL4JgZeskcwzCf2AUfnfsoCrZeeRgL0maKTYPZ12V7wMVJZ/BIGYBgA00LEPNOL Ss/pUrmm8Dr38vw+kbPeIToC5kBKeuil3OQk24Ho+7+9qcX82eMr6V0zucZnphciaAXU ZAgMmtj7/LE1S7tmic99e6J2hC/05qxo8vNy/bIq5tfrxH7KtXFuTZale/vUZHoe0b9a fR9elI5l7I79KLab22TMkMGd5g/V4ZRrs7libRoyJpc1kCKuL9B7HAdYsg0l4gNhyB0u nBED/2la2bPta4nCNpfEd3+p1W2S/Z+WVCnSWJlPBMhQnyhNpRONyJdfnJLQ1P16nH0G YSow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=PwWcZyLy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h3-20020a170906530300b0088a35d495a2si9616456ejo.293.2023.02.01.08.35.55; Wed, 01 Feb 2023 08:36:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=PwWcZyLy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232409AbjBAQfk (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232305AbjBAQff (ORCPT ); Wed, 1 Feb 2023 11:35:35 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E070F79F05 for ; Wed, 1 Feb 2023 08:35:26 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id c4-20020a1c3504000000b003d9e2f72093so1836917wma.1 for ; Wed, 01 Feb 2023 08:35:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X+Q37Omt/MAOZB85dyYH0VxCAqGyfL1V0CNFe50alWw=; b=PwWcZyLy+Z19hGnMU/rvK3uCeqYMJcoOHacKNeZ2E5eLBXT6GqsH1Mt31hw0qtXIxl aWTFapycW8vcGH3EpuYLmCKLHN5VfiMVOtmfowc1hqteDstL5qRq9q7/8c1pJFIGvahS gvTvMiculTkSVln7xHYS3kSRkQyOfyWWvdZI0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X+Q37Omt/MAOZB85dyYH0VxCAqGyfL1V0CNFe50alWw=; b=At7e7e7PiTeN7h9lgloircRP7Dp6ziKzxm3kMwa43bVuWwCd85d5jKUul9qBzUzmN5 Ug5eDRy0PTQ6fIu6DxEygxTjuKe6/q1GZTLlaN1PD70kQKJdK0gKUzE7yY0eScvJ8JRd gX3eiQf7Up6gBGtka1KFKZY5rx8gvYAizJrppyyzt/P+cEVNP0DdNwsMYtQ5KJaSIEGw jn0+64Gg2+Wws7/bxWTZYFpdjSsgTfg7kdL4l+Gs4U9/Nux1Zy6ZD4unlxt/XDnwPa8T CMTZzZF9nZgzD/U/bwpp/zOZ01c9mH84hsoZgSviuh4Y3eWlKyFqp6OEu5fQbDijPGp4 gr4Q== X-Gm-Message-State: AO0yUKVlJbd0a+QUUqaW4AdmEP6FUyr6gMyReYu/HKJgLv6SPpa02v2u UfUoZHEidLcJEeDtRqTxwLXDMA== X-Received: by 2002:a7b:c85a:0:b0:3d2:392e:905f with SMTP id c26-20020a7bc85a000000b003d2392e905fmr2646774wml.24.1675269325301; Wed, 01 Feb 2023 08:35:25 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:24 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 2/8] ftrace: Remove the legacy _ftrace_direct API Date: Wed, 1 Feb 2023 17:34:14 +0100 Message-Id: <20230201163420.1579014-3-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647264551952184?= X-GMAIL-MSGID: =?utf-8?q?1756647264551952184?= This API relies on a single global ops, used for all direct calls registered with it. However, to implement arm64 direct calls, we need each ops to point to a single direct call trampoline. Signed-off-by: Florent Revest Acked-by: Mark Rutland Tested-by: Mark Rutland --- include/linux/ftrace.h | 32 ---- kernel/trace/ftrace.c | 393 ----------------------------------------- 2 files changed, 425 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 366c730beaa3..2d7c85e47c2d 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -397,14 +397,6 @@ struct ftrace_func_entry { #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS extern int ftrace_direct_func_count; -int register_ftrace_direct(unsigned long ip, unsigned long addr); -int unregister_ftrace_direct(unsigned long ip, unsigned long addr); -int modify_ftrace_direct(unsigned long ip, unsigned long old_addr, unsigned long new_addr); -struct ftrace_direct_func *ftrace_find_direct_func(unsigned long addr); -int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, - struct dyn_ftrace *rec, - unsigned long old_addr, - unsigned long new_addr); unsigned long ftrace_find_rec_direct(unsigned long ip); int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); @@ -414,30 +406,6 @@ int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr #else struct ftrace_ops; # define ftrace_direct_func_count 0 -static inline int register_ftrace_direct(unsigned long ip, unsigned long addr) -{ - return -ENOTSUPP; -} -static inline int unregister_ftrace_direct(unsigned long ip, unsigned long addr) -{ - return -ENOTSUPP; -} -static inline int modify_ftrace_direct(unsigned long ip, - unsigned long old_addr, unsigned long new_addr) -{ - return -ENOTSUPP; -} -static inline struct ftrace_direct_func *ftrace_find_direct_func(unsigned long addr) -{ - return NULL; -} -static inline int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, - struct dyn_ftrace *rec, - unsigned long old_addr, - unsigned long new_addr) -{ - return -ENODEV; -} static inline unsigned long ftrace_find_rec_direct(unsigned long ip) { return 0; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index e634b80f49d1..5efe201428fa 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2585,20 +2585,6 @@ static void call_direct_funcs(unsigned long ip, unsigned long pip, arch_ftrace_set_direct_caller(fregs, addr); } - -struct ftrace_ops direct_ops = { - .func = call_direct_funcs, - .flags = FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS - | FTRACE_OPS_FL_PERMANENT, - /* - * By declaring the main trampoline as this trampoline - * it will never have one allocated for it. Allocated - * trampolines should not call direct functions. - * The direct_ops should only be called by the builtin - * ftrace_regs_caller trampoline. - */ - .trampoline = FTRACE_REGS_ADDR, -}; #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** @@ -5295,387 +5281,8 @@ struct ftrace_direct_func { static LIST_HEAD(ftrace_direct_funcs); -/** - * ftrace_find_direct_func - test an address if it is a registered direct caller - * @addr: The address of a registered direct caller - * - * This searches to see if a ftrace direct caller has been registered - * at a specific address, and if so, it returns a descriptor for it. - * - * This can be used by architecture code to see if an address is - * a direct caller (trampoline) attached to a fentry/mcount location. - * This is useful for the function_graph tracer, as it may need to - * do adjustments if it traced a location that also has a direct - * trampoline attached to it. - */ -struct ftrace_direct_func *ftrace_find_direct_func(unsigned long addr) -{ - struct ftrace_direct_func *entry; - bool found = false; - - /* May be called by fgraph trampoline (protected by rcu tasks) */ - list_for_each_entry_rcu(entry, &ftrace_direct_funcs, next) { - if (entry->addr == addr) { - found = true; - break; - } - } - if (found) - return entry; - - return NULL; -} - -static struct ftrace_direct_func *ftrace_alloc_direct_func(unsigned long addr) -{ - struct ftrace_direct_func *direct; - - direct = kmalloc(sizeof(*direct), GFP_KERNEL); - if (!direct) - return NULL; - direct->addr = addr; - direct->count = 0; - list_add_rcu(&direct->next, &ftrace_direct_funcs); - ftrace_direct_func_count++; - return direct; -} - static int register_ftrace_function_nolock(struct ftrace_ops *ops); -/** - * register_ftrace_direct - Call a custom trampoline directly - * @ip: The address of the nop at the beginning of a function - * @addr: The address of the trampoline to call at @ip - * - * This is used to connect a direct call from the nop location (@ip) - * at the start of ftrace traced functions. The location that it calls - * (@addr) must be able to handle a direct call, and save the parameters - * of the function being traced, and restore them (or inject new ones - * if needed), before returning. - * - * Returns: - * 0 on success - * -EBUSY - Another direct function is already attached (there can be only one) - * -ENODEV - @ip does not point to a ftrace nop location (or not supported) - * -ENOMEM - There was an allocation failure. - */ -int register_ftrace_direct(unsigned long ip, unsigned long addr) -{ - struct ftrace_direct_func *direct; - struct ftrace_func_entry *entry; - struct ftrace_hash *free_hash = NULL; - struct dyn_ftrace *rec; - int ret = -ENODEV; - - mutex_lock(&direct_mutex); - - ip = ftrace_location(ip); - if (!ip) - goto out_unlock; - - /* See if there's a direct function at @ip already */ - ret = -EBUSY; - if (ftrace_find_rec_direct(ip)) - goto out_unlock; - - ret = -ENODEV; - rec = lookup_rec(ip, ip); - if (!rec) - goto out_unlock; - - /* - * Check if the rec says it has a direct call but we didn't - * find one earlier? - */ - if (WARN_ON(rec->flags & FTRACE_FL_DIRECT)) - goto out_unlock; - - /* Make sure the ip points to the exact record */ - if (ip != rec->ip) { - ip = rec->ip; - /* Need to check this ip for a direct. */ - if (ftrace_find_rec_direct(ip)) - goto out_unlock; - } - - ret = -ENOMEM; - direct = ftrace_find_direct_func(addr); - if (!direct) { - direct = ftrace_alloc_direct_func(addr); - if (!direct) - goto out_unlock; - } - - entry = ftrace_add_rec_direct(ip, addr, &free_hash); - if (!entry) - goto out_unlock; - - ret = ftrace_set_filter_ip(&direct_ops, ip, 0, 0); - - if (!ret && !(direct_ops.flags & FTRACE_OPS_FL_ENABLED)) { - ret = register_ftrace_function_nolock(&direct_ops); - if (ret) - ftrace_set_filter_ip(&direct_ops, ip, 1, 0); - } - - if (ret) { - remove_hash_entry(direct_functions, entry); - kfree(entry); - if (!direct->count) { - list_del_rcu(&direct->next); - synchronize_rcu_tasks(); - kfree(direct); - if (free_hash) - free_ftrace_hash(free_hash); - free_hash = NULL; - ftrace_direct_func_count--; - } - } else { - direct->count++; - } - out_unlock: - mutex_unlock(&direct_mutex); - - if (free_hash) { - synchronize_rcu_tasks(); - free_ftrace_hash(free_hash); - } - - return ret; -} -EXPORT_SYMBOL_GPL(register_ftrace_direct); - -static struct ftrace_func_entry *find_direct_entry(unsigned long *ip, - struct dyn_ftrace **recp) -{ - struct ftrace_func_entry *entry; - struct dyn_ftrace *rec; - - rec = lookup_rec(*ip, *ip); - if (!rec) - return NULL; - - entry = __ftrace_lookup_ip(direct_functions, rec->ip); - if (!entry) { - WARN_ON(rec->flags & FTRACE_FL_DIRECT); - return NULL; - } - - WARN_ON(!(rec->flags & FTRACE_FL_DIRECT)); - - /* Passed in ip just needs to be on the call site */ - *ip = rec->ip; - - if (recp) - *recp = rec; - - return entry; -} - -int unregister_ftrace_direct(unsigned long ip, unsigned long addr) -{ - struct ftrace_direct_func *direct; - struct ftrace_func_entry *entry; - struct ftrace_hash *hash; - int ret = -ENODEV; - - mutex_lock(&direct_mutex); - - ip = ftrace_location(ip); - if (!ip) - goto out_unlock; - - entry = find_direct_entry(&ip, NULL); - if (!entry) - goto out_unlock; - - hash = direct_ops.func_hash->filter_hash; - if (hash->count == 1) - unregister_ftrace_function(&direct_ops); - - ret = ftrace_set_filter_ip(&direct_ops, ip, 1, 0); - - WARN_ON(ret); - - remove_hash_entry(direct_functions, entry); - - direct = ftrace_find_direct_func(addr); - if (!WARN_ON(!direct)) { - /* This is the good path (see the ! before WARN) */ - direct->count--; - WARN_ON(direct->count < 0); - if (!direct->count) { - list_del_rcu(&direct->next); - synchronize_rcu_tasks(); - kfree(direct); - kfree(entry); - ftrace_direct_func_count--; - } - } - out_unlock: - mutex_unlock(&direct_mutex); - - return ret; -} -EXPORT_SYMBOL_GPL(unregister_ftrace_direct); - -static struct ftrace_ops stub_ops = { - .func = ftrace_stub, -}; - -/** - * ftrace_modify_direct_caller - modify ftrace nop directly - * @entry: The ftrace hash entry of the direct helper for @rec - * @rec: The record representing the function site to patch - * @old_addr: The location that the site at @rec->ip currently calls - * @new_addr: The location that the site at @rec->ip should call - * - * An architecture may overwrite this function to optimize the - * changing of the direct callback on an ftrace nop location. - * This is called with the ftrace_lock mutex held, and no other - * ftrace callbacks are on the associated record (@rec). Thus, - * it is safe to modify the ftrace record, where it should be - * currently calling @old_addr directly, to call @new_addr. - * - * This is called with direct_mutex locked. - * - * Safety checks should be made to make sure that the code at - * @rec->ip is currently calling @old_addr. And this must - * also update entry->direct to @new_addr. - */ -int __weak ftrace_modify_direct_caller(struct ftrace_func_entry *entry, - struct dyn_ftrace *rec, - unsigned long old_addr, - unsigned long new_addr) -{ - unsigned long ip = rec->ip; - int ret; - - lockdep_assert_held(&direct_mutex); - - /* - * The ftrace_lock was used to determine if the record - * had more than one registered user to it. If it did, - * we needed to prevent that from changing to do the quick - * switch. But if it did not (only a direct caller was attached) - * then this function is called. But this function can deal - * with attached callers to the rec that we care about, and - * since this function uses standard ftrace calls that take - * the ftrace_lock mutex, we need to release it. - */ - mutex_unlock(&ftrace_lock); - - /* - * By setting a stub function at the same address, we force - * the code to call the iterator and the direct_ops helper. - * This means that @ip does not call the direct call, and - * we can simply modify it. - */ - ret = ftrace_set_filter_ip(&stub_ops, ip, 0, 0); - if (ret) - goto out_lock; - - ret = register_ftrace_function_nolock(&stub_ops); - if (ret) { - ftrace_set_filter_ip(&stub_ops, ip, 1, 0); - goto out_lock; - } - - entry->direct = new_addr; - - /* - * By removing the stub, we put back the direct call, calling - * the @new_addr. - */ - unregister_ftrace_function(&stub_ops); - ftrace_set_filter_ip(&stub_ops, ip, 1, 0); - - out_lock: - mutex_lock(&ftrace_lock); - - return ret; -} - -/** - * modify_ftrace_direct - Modify an existing direct call to call something else - * @ip: The instruction pointer to modify - * @old_addr: The address that the current @ip calls directly - * @new_addr: The address that the @ip should call - * - * This modifies a ftrace direct caller at an instruction pointer without - * having to disable it first. The direct call will switch over to the - * @new_addr without missing anything. - * - * Returns: zero on success. Non zero on error, which includes: - * -ENODEV : the @ip given has no direct caller attached - * -EINVAL : the @old_addr does not match the current direct caller - */ -int modify_ftrace_direct(unsigned long ip, - unsigned long old_addr, unsigned long new_addr) -{ - struct ftrace_direct_func *direct, *new_direct = NULL; - struct ftrace_func_entry *entry; - struct dyn_ftrace *rec; - int ret = -ENODEV; - - mutex_lock(&direct_mutex); - - mutex_lock(&ftrace_lock); - - ip = ftrace_location(ip); - if (!ip) - goto out_unlock; - - entry = find_direct_entry(&ip, &rec); - if (!entry) - goto out_unlock; - - ret = -EINVAL; - if (entry->direct != old_addr) - goto out_unlock; - - direct = ftrace_find_direct_func(old_addr); - if (WARN_ON(!direct)) - goto out_unlock; - if (direct->count > 1) { - ret = -ENOMEM; - new_direct = ftrace_alloc_direct_func(new_addr); - if (!new_direct) - goto out_unlock; - direct->count--; - new_direct->count++; - } else { - direct->addr = new_addr; - } - - /* - * If there's no other ftrace callback on the rec->ip location, - * then it can be changed directly by the architecture. - * If there is another caller, then we just need to change the - * direct caller helper to point to @new_addr. - */ - if (ftrace_rec_count(rec) == 1) { - ret = ftrace_modify_direct_caller(entry, rec, old_addr, new_addr); - } else { - entry->direct = new_addr; - ret = 0; - } - - if (unlikely(ret && new_direct)) { - direct->count++; - list_del_rcu(&new_direct->next); - synchronize_rcu_tasks(); - kfree(new_direct); - ftrace_direct_func_count--; - } - - out_unlock: - mutex_unlock(&ftrace_lock); - mutex_unlock(&direct_mutex); - return ret; -} -EXPORT_SYMBOL_GPL(modify_ftrace_direct); - #define MULTI_FLAGS (FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS) static int check_direct_multi(struct ftrace_ops *ops) From patchwork Wed Feb 1 16:34:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51471 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp379872wrn; Wed, 1 Feb 2023 08:36:24 -0800 (PST) X-Google-Smtp-Source: AK7set+9tMhsXr6iLFgyPIsUaZSecfdFtPVjhgv8IseNRBM1+wnOMM56gC3BXcr9JPClamF/hyQo X-Received: by 2002:a05:6402:3219:b0:4a2:223d:4514 with SMTP id g25-20020a056402321900b004a2223d4514mr3006407eda.8.1675269384300; Wed, 01 Feb 2023 08:36:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269384; cv=none; d=google.com; s=arc-20160816; b=jsvhfOckLFb0Pfi8Kl2Wms9ne5Z5cOsIRXPPzhK40QtI5+03kWh5FvwIH0o7l0qtFY yK/C6OTfvrVuGAEK4vRuPZPdJLjhIQqgOJTlYh05KlzD0SA9TOcllr/3obRmQ3flCZgl 8swkQw4mIlW65KHXxlRNSD1HZQ3T55GqdDeDct4aZufhTzHPWonGJho2PufvsfRCufhs eSCeS4fQ02ClmIPoHV6/rPgP6z4cVbIDlDLk9Gkp+cD4UK8O1O0NfYQC+LT7fPS6di13 RXlkfYMF/lteKX0/q6RgREq2pBrA/zWFx81jJvHfZ5UfPegP7wHXE/OQRH62/r8JM/TI 8fOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Xjgzv+CaWDCCmc9m2PtN7hh7GiZmUlZ8epcrKuwyKlk=; b=cYuVBkywSZXTu5jlprOSG0xbSl26Lgwm0PV42I7nDyGOET4Ex5aYvVevNx0X++8vY0 bG6m8d3WkMwFGQs24Lj9MSZRYBlC7Z6iCnqPFRu4sfWlj/F2zuXcpPHn4KScw4HR2fIN AkYopaKT8RJFkXaMK4uVWJPuegn0HRZToRwernGOg2h9l/f5Mel6d44wpMhO1/epliW/ f70B47RB3m2ys859tQJhHtpKg4uwbuSTxyysz1WNOsBnR/Ix4TUo6r34McwQ5yMJyYi6 ehmwzljr/lKgpm/ojmushPbWjBs7mkkQq3Q7/BttkiE/3uk384r0JbCEgQGu4I6OSfUG 3TIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=fB2BygET; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z11-20020a50eb4b000000b0049ec7fd6bbesi21648954edp.371.2023.02.01.08.36.00; Wed, 01 Feb 2023 08:36:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=fB2BygET; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232340AbjBAQfr (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232348AbjBAQfh (ORCPT ); Wed, 1 Feb 2023 11:35:37 -0500 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EEE379F2B for ; Wed, 1 Feb 2023 08:35:28 -0800 (PST) Received: by mail-wm1-x336.google.com with SMTP id bg26so7277055wmb.0 for ; Wed, 01 Feb 2023 08:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Xjgzv+CaWDCCmc9m2PtN7hh7GiZmUlZ8epcrKuwyKlk=; b=fB2BygETm73mrQ4E+jxUvmjud7LCPGfH0ctmgJAjk8dhSSEtRFb5FRPeDawZvISnAX SMWlkq1RxsqY0n9ATCYVe4TmRwqBeWUWhE9DKB1oj8V5RTWazQWdsb6E5fUjUwRH4ayK MiA1JFPW7zxvbGfJUF+AszD+OWqU9CTiC5tOo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xjgzv+CaWDCCmc9m2PtN7hh7GiZmUlZ8epcrKuwyKlk=; b=iV4+WtL2/BxRO++yn5LDeOoRINZQHfJ6a9WX8IyQ2igZkZcMn+oocYVKswysON4mxd G41F923p2bJHEp0sM/enKMSWutALIP+1a8BAOpF4GvOrlPRSTYzBeZV/OxWosfOp6sBV YY0MPEybf7ahLan5j8E2UpuhIpNXfiLFvtbwsdLv9zln1CWgtvSM/brNnN5iMFbpnoVI Oirdbm6YbrCi7IKr2Ui7UsGz/26zAJ8xR3wNCBhGQZPytnYEX/OjDLv/YJ4rDj2plcYQ qL9nTI70nRo8xJqW47AW6uNCi/2YmMaNJOO4yZvK5Mggkqirc2+lUhV52TLtFQi0sXAS FZ8g== X-Gm-Message-State: AO0yUKUfNyWyspn5vMGsMnL5LT+6UI9WOxcWjFL7J1SfszSM1u0bVIQR twJHy9/bNleaYEve0mTiHC3d2Q== X-Received: by 2002:a05:600c:468e:b0:3db:1434:c51a with SMTP id p14-20020a05600c468e00b003db1434c51amr2813766wmo.40.1675269326776; Wed, 01 Feb 2023 08:35:26 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:26 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 3/8] ftrace: Rename _ftrace_direct_multi APIs to _ftrace_direct APIs Date: Wed, 1 Feb 2023 17:34:15 +0100 Message-Id: <20230201163420.1579014-4-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647269841133697?= X-GMAIL-MSGID: =?utf-8?q?1756647269841133697?= Now that the original _ftrace_direct APIs are gone, the "_multi" suffixes only add confusion. Signed-off-by: Florent Revest Acked-by: Mark Rutland Tested-by: Mark Rutland --- include/linux/ftrace.h | 16 +++++------ kernel/bpf/trampoline.c | 14 ++++----- kernel/trace/ftrace.c | 32 ++++++++++----------- kernel/trace/trace_selftest.c | 7 +++-- samples/Kconfig | 2 +- samples/ftrace/ftrace-direct-modify.c | 8 +++--- samples/ftrace/ftrace-direct-multi-modify.c | 8 +++--- samples/ftrace/ftrace-direct-multi.c | 6 ++-- samples/ftrace/ftrace-direct-too.c | 6 ++-- samples/ftrace/ftrace-direct.c | 6 ++-- 10 files changed, 53 insertions(+), 52 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 2d7c85e47c2d..a7dbd307c3a4 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -398,10 +398,10 @@ struct ftrace_func_entry { #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS extern int ftrace_direct_func_count; unsigned long ftrace_find_rec_direct(unsigned long ip); -int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); -int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); -int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); -int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr); +int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr); +int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr); +int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr); +int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr); #else struct ftrace_ops; @@ -410,19 +410,19 @@ static inline unsigned long ftrace_find_rec_direct(unsigned long ip) { return 0; } -static inline int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +static inline int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } -static inline int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +static inline int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } -static inline int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +static inline int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } -static inline int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr) +static inline int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index d0ed7d6f5eec..150b53316df2 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -39,14 +39,14 @@ static int bpf_tramp_ftrace_ops_func(struct ftrace_ops *ops, enum ftrace_ops_cmd int ret = 0; if (cmd == FTRACE_OPS_CMD_ENABLE_SHARE_IPMODIFY_SELF) { - /* This is called inside register_ftrace_direct_multi(), so + /* This is called inside register_ftrace_direct(), so * tr->mutex is already locked. */ lockdep_assert_held_once(&tr->mutex); /* Instead of updating the trampoline here, we propagate - * -EAGAIN to register_ftrace_direct_multi(). Then we can - * retry register_ftrace_direct_multi() after updating the + * -EAGAIN to register_ftrace_direct(). Then we can + * retry register_ftrace_direct() after updating the * trampoline. */ if ((tr->flags & BPF_TRAMP_F_CALL_ORIG) && @@ -198,7 +198,7 @@ static int unregister_fentry(struct bpf_trampoline *tr, void *old_addr) int ret; if (tr->func.ftrace_managed) - ret = unregister_ftrace_direct_multi(tr->fops, (long)old_addr); + ret = unregister_ftrace_direct(tr->fops, (long)old_addr); else ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, old_addr, NULL); @@ -215,9 +215,9 @@ static int modify_fentry(struct bpf_trampoline *tr, void *old_addr, void *new_ad if (tr->func.ftrace_managed) { if (lock_direct_mutex) - ret = modify_ftrace_direct_multi(tr->fops, (long)new_addr); + ret = modify_ftrace_direct(tr->fops, (long)new_addr); else - ret = modify_ftrace_direct_multi_nolock(tr->fops, (long)new_addr); + ret = modify_ftrace_direct_nolock(tr->fops, (long)new_addr); } else { ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, old_addr, new_addr); } @@ -243,7 +243,7 @@ static int register_fentry(struct bpf_trampoline *tr, void *new_addr) if (tr->func.ftrace_managed) { ftrace_set_filter_ip(tr->fops, (unsigned long)ip, 0, 1); - ret = register_ftrace_direct_multi(tr->fops, (long)new_addr); + ret = register_ftrace_direct(tr->fops, (long)new_addr); } else { ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, NULL, new_addr); } diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 5efe201428fa..cb77a0a208c7 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5312,7 +5312,7 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long } /** - * register_ftrace_direct_multi - Call a custom trampoline directly + * register_ftrace_direct - Call a custom trampoline directly * for multiple functions registered in @ops * @ops: The address of the struct ftrace_ops object * @addr: The address of the trampoline to call at @ops functions @@ -5333,7 +5333,7 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long * -ENODEV - @ip does not point to a ftrace nop location (or not supported) * -ENOMEM - There was an allocation failure. */ -int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { struct ftrace_hash *hash, *free_hash = NULL; struct ftrace_func_entry *entry, *new; @@ -5391,11 +5391,11 @@ int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) } return err; } -EXPORT_SYMBOL_GPL(register_ftrace_direct_multi); +EXPORT_SYMBOL_GPL(register_ftrace_direct); /** - * unregister_ftrace_direct_multi - Remove calls to custom trampoline - * previously registered by register_ftrace_direct_multi for @ops object. + * unregister_ftrace_direct - Remove calls to custom trampoline + * previously registered by register_ftrace_direct for @ops object. * @ops: The address of the struct ftrace_ops object * * This is used to remove a direct calls to @addr from the nop locations @@ -5406,7 +5406,7 @@ EXPORT_SYMBOL_GPL(register_ftrace_direct_multi); * 0 on success * -EINVAL - The @ops object was not properly registered. */ -int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { struct ftrace_hash *hash = ops->func_hash->filter_hash; int err; @@ -5426,10 +5426,10 @@ int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) ops->trampoline = 0; return err; } -EXPORT_SYMBOL_GPL(unregister_ftrace_direct_multi); +EXPORT_SYMBOL_GPL(unregister_ftrace_direct); static int -__modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +__modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { struct ftrace_hash *hash; struct ftrace_func_entry *entry, *iter; @@ -5476,7 +5476,7 @@ __modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) } /** - * modify_ftrace_direct_multi_nolock - Modify an existing direct 'multi' call + * modify_ftrace_direct_nolock - Modify an existing direct 'multi' call * to call something else * @ops: The address of the struct ftrace_ops object * @addr: The address of the new trampoline to call at @ops functions @@ -5493,19 +5493,19 @@ __modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) * Returns: zero on success. Non zero on error, which includes: * -EINVAL - The @ops object was not properly registered. */ -int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr) +int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr) { if (check_direct_multi(ops)) return -EINVAL; if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) return -EINVAL; - return __modify_ftrace_direct_multi(ops, addr); + return __modify_ftrace_direct(ops, addr); } -EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi_nolock); +EXPORT_SYMBOL_GPL(modify_ftrace_direct_nolock); /** - * modify_ftrace_direct_multi - Modify an existing direct 'multi' call + * modify_ftrace_direct - Modify an existing direct 'multi' call * to call something else * @ops: The address of the struct ftrace_ops object * @addr: The address of the new trampoline to call at @ops functions @@ -5519,7 +5519,7 @@ EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi_nolock); * Returns: zero on success. Non zero on error, which includes: * -EINVAL - The @ops object was not properly registered. */ -int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { int err; @@ -5529,11 +5529,11 @@ int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) return -EINVAL; mutex_lock(&direct_mutex); - err = __modify_ftrace_direct_multi(ops, addr); + err = __modify_ftrace_direct(ops, addr); mutex_unlock(&direct_mutex); return err; } -EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi); +EXPORT_SYMBOL_GPL(modify_ftrace_direct); #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index 9b7f10cbc51d..06218fc9374b 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -874,7 +874,8 @@ trace_selftest_startup_function_graph(struct tracer *trace, * and make sure we get graph trace. */ ftrace_set_filter_ip(&direct, (unsigned long)DYN_FTRACE_TEST_NAME, 0, 0); - ret = register_ftrace_direct_multi(&direct, (unsigned long)trace_direct_tramp); + ret = register_ftrace_direct(&direct, + (unsigned long)trace_direct_tramp); if (ret) goto out; @@ -894,8 +895,8 @@ trace_selftest_startup_function_graph(struct tracer *trace, unregister_ftrace_graph(&fgraph_ops); - ret = unregister_ftrace_direct_multi(&direct, - (unsigned long) trace_direct_tramp); + ret = unregister_ftrace_direct(&direct, + (unsigned long)trace_direct_tramp); if (ret) goto out; diff --git a/samples/Kconfig b/samples/Kconfig index 0d81c00289ee..e85998ca354d 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -38,7 +38,7 @@ config SAMPLE_FTRACE_DIRECT that hooks to wake_up_process and prints the parameters. config SAMPLE_FTRACE_DIRECT_MULTI - tristate "Build register_ftrace_direct_multi() example" + tristate "Build register_ftrace_direct() on multiple ips example" depends on DYNAMIC_FTRACE_WITH_DIRECT_CALLS && m depends on HAVE_SAMPLE_FTRACE_DIRECT_MULTI help diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c index e1e6c286970c..0f1b8859e1e1 100644 --- a/samples/ftrace/ftrace-direct-modify.c +++ b/samples/ftrace/ftrace-direct-modify.c @@ -116,7 +116,7 @@ static int simple_thread(void *arg) if (ret) continue; t ^= 1; - ret = modify_ftrace_direct_multi(&direct, tramps[t]); + ret = modify_ftrace_direct(&direct, tramps[t]); if (!ret) my_tramp = tramps[t]; WARN_ON_ONCE(ret); @@ -132,7 +132,7 @@ static int __init ftrace_direct_init(void) int ret; ftrace_set_filter_ip(&direct, (unsigned long) my_ip, 0, 0); - ret = register_ftrace_direct_multi(&direct, my_tramp); + ret = register_ftrace_direct(&direct, my_tramp); if (!ret) simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn"); return ret; @@ -141,12 +141,12 @@ static int __init ftrace_direct_init(void) static void __exit ftrace_direct_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct_multi(&direct, my_tramp); + unregister_ftrace_direct(&direct, my_tramp); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c index d52370cad0b6..407c56325e65 100644 --- a/samples/ftrace/ftrace-direct-multi-modify.c +++ b/samples/ftrace/ftrace-direct-multi-modify.c @@ -123,7 +123,7 @@ static int simple_thread(void *arg) if (ret) continue; t ^= 1; - ret = modify_ftrace_direct_multi(&direct, tramps[t]); + ret = modify_ftrace_direct(&direct, tramps[t]); if (!ret) my_tramp = tramps[t]; WARN_ON_ONCE(ret); @@ -141,7 +141,7 @@ static int __init ftrace_direct_multi_init(void) ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); ftrace_set_filter_ip(&direct, (unsigned long) schedule, 0, 0); - ret = register_ftrace_direct_multi(&direct, my_tramp); + ret = register_ftrace_direct(&direct, my_tramp); if (!ret) simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn"); @@ -151,12 +151,12 @@ static int __init ftrace_direct_multi_init(void) static void __exit ftrace_direct_multi_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct_multi(&direct, my_tramp); + unregister_ftrace_direct(&direct, my_tramp); } module_init(ftrace_direct_multi_init); module_exit(ftrace_direct_multi_exit); MODULE_AUTHOR("Jiri Olsa"); -MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c index ec1088922517..42a94806a446 100644 --- a/samples/ftrace/ftrace-direct-multi.c +++ b/samples/ftrace/ftrace-direct-multi.c @@ -73,17 +73,17 @@ static int __init ftrace_direct_multi_init(void) ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); ftrace_set_filter_ip(&direct, (unsigned long) schedule, 0, 0); - return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + return register_ftrace_direct(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_multi_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + unregister_ftrace_direct(&direct, (unsigned long) my_tramp); } module_init(ftrace_direct_multi_init); module_exit(ftrace_direct_multi_exit); MODULE_AUTHOR("Jiri Olsa"); -MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c index 0e907092e2c0..7ee5dd3cc61d 100644 --- a/samples/ftrace/ftrace-direct-too.c +++ b/samples/ftrace/ftrace-direct-too.c @@ -76,17 +76,17 @@ static int __init ftrace_direct_init(void) { ftrace_set_filter_ip(&direct, (unsigned long) handle_mm_fault, 0, 0); - return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + return register_ftrace_direct(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp); + unregister_ftrace_direct(&direct, (unsigned long)my_tramp); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c index e446c38f6b58..5ffce87fa83e 100644 --- a/samples/ftrace/ftrace-direct.c +++ b/samples/ftrace/ftrace-direct.c @@ -69,17 +69,17 @@ static int __init ftrace_direct_init(void) { ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); - return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + return register_ftrace_direct(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp); + unregister_ftrace_direct(&direct, (unsigned long)my_tramp); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct()"); MODULE_LICENSE("GPL"); From patchwork Wed Feb 1 16:34:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51472 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp379923wrn; Wed, 1 Feb 2023 08:36:31 -0800 (PST) X-Google-Smtp-Source: AK7set82cPVnoaKukaF7AmueCTfe3wn6rigzjFv9EyWKcNSkVn1L/ES+vVc0mjeBa4dJ2uA0TzSZ X-Received: by 2002:aa7:d49a:0:b0:49d:a87f:ba78 with SMTP id b26-20020aa7d49a000000b0049da87fba78mr2653053edr.35.1675269391705; Wed, 01 Feb 2023 08:36:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269391; cv=none; d=google.com; s=arc-20160816; b=GFBU+66YIyPyNB7s1LyU+cOO8lp/TMivrkvsuYkGfN3pGDI70zcaPcFsna0LnDawKH L1soQqAuTRCG6LTjaTUGODTENTuTM2KP5v5oGvYxX8lWI6snrpKAoGL5cApzuCZ5Nc5P FMbahHFj9B4DNpp98tbl2YfBBbykTytFHyMbXcubmJU/xuvOPwvajPwaO+mtd/eWHSL1 lMMDnmfJOl/sVsIVe3SWddhjdK90eN362jC9JiOFqrup7ymigJKf31xY9sOZwEtOw9M3 9wTvlTRR2Yu5wrKhdBeWp82PGX9m1TRfgTgyGuwuY7F0o0E+KL1IzQXbyRgy6BNj+I0b 3PQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=p0B1nR0feNzt6iYs1fcCSNwhqATraAQZbyN0DagmRgs=; b=qjOr2tigqxOkdXZHplPbmRxHFFAJHCpt6ZUwJc0nMLtxqB21LU7zC1qOZIqrm0tfm4 NGRmozpFVtvuDgFrBExCrmR/HATXHdfbSkjeh1OFKdk1bkGUcVHIY/FknTZDFP7jAc0P x2aLymRKViDyFaQeqH6FHHOzIpUjcjhRkoQhPpbZlmKcHtdQAyeFMuS8D84ybyrMFkgV mjrewj3zYhO0/YGXPaUicQv/6s80/eJy1BQn23f1oac1VsPR/lLd/KwNttGHCtKOd0gQ oOmE9/IiePT/GX0UFCkR8L1qP05GB28mnWsRzcVpIEYNT1EIGb4yrQ1UTkWXZKaFBQQi zWDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=jcn6cTs4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e13-20020a056402190d00b004a24475fdd4si15516996edz.376.2023.02.01.08.36.07; Wed, 01 Feb 2023 08:36:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=jcn6cTs4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232591AbjBAQft (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232446AbjBAQfq (ORCPT ); Wed, 1 Feb 2023 11:35:46 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF20F7A485 for ; Wed, 1 Feb 2023 08:35:29 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id m16-20020a05600c3b1000b003dc4050c94aso1921069wms.4 for ; Wed, 01 Feb 2023 08:35:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p0B1nR0feNzt6iYs1fcCSNwhqATraAQZbyN0DagmRgs=; b=jcn6cTs4+XY1hUWmIDmmwPG44YIXWbCtGh35J88AOeps79NC6Aub1gKAPPv419PyFm HTPegDj2V+wQXQ7DcQvVkjHumtHWHz7verUSkgH7IJrJQQz1clTf1SwWQ3717K7f1cqs BiPYS75krdieSyLXCxa9QTp4cnoHXv5nKh5HI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p0B1nR0feNzt6iYs1fcCSNwhqATraAQZbyN0DagmRgs=; b=ZnwDw0DXWtGbsGn8aMVkoZDmPM0ZDBO5KkSMjQNsZArLonbREyeUmLSxGSQ+SD1OFX MC4IygBWHWSLTDmHBkOnTXlczdbK+/YAbWfAx48wEOLu8qMidirIcBQ05r2x90WY7RTD 2gGSrqByopZXqm8FMMz5L7Lrs6231XSM7qnmWHEM0LVJCi53MS/tJdYmoGGueK/qIukc UoCzC0xFm01LA+REMmyhBXjTI00ywtZYuTlomXatRrdCHOz1xXJYC9cKKdyPEHZQp+VQ nDqVSFONkw0mGgTBYjWs7pWR0QpLaW11u6M+y193aYNcinh6zwN6X99TNip5yCEsFrPL bHLg== X-Gm-Message-State: AO0yUKXgLEdjJLxM0M+TB7CqPeRj2cxIQJWY4ZClmmFHmTn2ky+AuvDi C1NdrmeHlhFrNFBgS+TtYXhmig== X-Received: by 2002:a05:600c:491c:b0:3d9:ee01:60a4 with SMTP id f28-20020a05600c491c00b003d9ee0160a4mr2827768wmp.1.1675269328196; Wed, 01 Feb 2023 08:35:28 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:27 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 4/8] ftrace: Store direct called addresses in their ops Date: Wed, 1 Feb 2023 17:34:16 +0100 Message-Id: <20230201163420.1579014-5-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647277617931532?= X-GMAIL-MSGID: =?utf-8?q?1756647277617931532?= All direct calls are now registered using the register_ftrace_direct API so each ops can jump to only one direct-called trampoline. By storing the direct called trampoline address directly in the ops we can save one hashmap lookup in the direct call ops and implement arm64 direct calls on top of call ops. Signed-off-by: Florent Revest --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a7dbd307c3a4..84f717f8959e 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -321,6 +321,9 @@ struct ftrace_ops { unsigned long trampoline_size; struct list_head list; ftrace_ops_func_t ops_func; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_call; +#endif #endif }; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index cb77a0a208c7..b0426de11c45 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2577,9 +2577,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - unsigned long addr; + unsigned long addr = ops->direct_call; - addr = ftrace_find_rec_direct(ip); if (!addr) return; @@ -5375,6 +5374,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) ops->func = call_direct_funcs; ops->flags = MULTI_FLAGS; ops->trampoline = FTRACE_REGS_ADDR; + ops->direct_call = addr; err = register_ftrace_function_nolock(ops); @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) /* Enable the tmp_ops to have the same functions as the direct ops */ ftrace_ops_init(&tmp_ops); tmp_ops.func_hash = ops->func_hash; + tmp_ops.direct_call = addr; err = register_ftrace_function_nolock(&tmp_ops); if (err) @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) entry->direct = addr; } } + ops->direct_call = addr; mutex_unlock(&ftrace_lock); From patchwork Wed Feb 1 16:34:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51473 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp381068wrn; Wed, 1 Feb 2023 08:38:59 -0800 (PST) X-Google-Smtp-Source: AK7set/U6dknERCAT9b5Tcqhy7mLaw6Z+m0IRitOAdrPwksFqZ7+SC1yI7StrQMD492hkMRmXqvp X-Received: by 2002:a17:906:5286:b0:881:7e85:915d with SMTP id c6-20020a170906528600b008817e85915dmr2490971ejm.13.1675269539434; Wed, 01 Feb 2023 08:38:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269539; cv=none; d=google.com; s=arc-20160816; b=gq3o+VHybSHcBU/MSRAErglqpQaV2Ue0xMSMrwBAZAhtmBRllAerC6+Iz0B8GwSNdj aeWu8hpi9zSI95qpYTvSCS7lqFuz9GZ6PGDawO5bq/UXM6RnwHR2SSzvayV42G1+f9+7 Vnt0wooOcpAc5HYrMJjgj9wx956E1KEaBg70//Z9MDSJB+MLGpdXrdTpwx7813T2CDxi 7jdkh14xtShrZL9nr9oSjvaO1ZeDkyzH6F1S/PkelNmSz5KkfY9lVvKkYVaJXdk/MLc7 Ll/lQ3rFHdm0LZ7s/yIt2GcseVoD5Pd/2tIZ2/+lpMHV0L13tZPiEvCcbYfYUXP7QxRs UyBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=V920tXDgaE7z8bCeiKLUkBejFiJJs4xj6S06IJdrTDo=; b=XIaJu7QgFa4CvR+XBLDNB2HOuJZie2eIbUtAENnYEiLpgZP8PLlRKay2efd4Ybw8Z2 dQ4ZbEhGVF5cMYQ/CmRx92uL4/u1NngTpE+piUZCaJJm99fPWfwzsyIUZbGx0yBqfkPo 8vLIEOJU7kbKCvMP6UbKZYme6TGYcDPhIDUHQSxHINaVJBOPtT1qaC2VChEPIpnq3B+o xznqtcB5gGuCFwmWj46Z4Ff6cc5JGWv9xluYZ8g33JBftqLdadMlhWzvDyQ+q1kUQuyh PbMtPgBAXLvhFEPAEIqNaPgWnLzCumM8XPNARhOWfRJgJrNBOpOYjYEzIAsnFy5aDtlI wFRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=DYuGfqui; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ge21-20020a170907909500b0088db807b22csi2505246ejb.117.2023.02.01.08.38.35; Wed, 01 Feb 2023 08:38:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=DYuGfqui; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232446AbjBAQfx (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232462AbjBAQfr (ORCPT ); Wed, 1 Feb 2023 11:35:47 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21FA49024 for ; Wed, 1 Feb 2023 08:35:31 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id n13so6114061wmr.4 for ; Wed, 01 Feb 2023 08:35:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V920tXDgaE7z8bCeiKLUkBejFiJJs4xj6S06IJdrTDo=; b=DYuGfqui4JmugKay/2snlgInuN8NoSZVoexURJTLb8Kn8f4JIyvQgTexs4woxHNsnz wO8Z6dafnQ/XbyCE2eZZeApXulDjM44aq8GV5zcg7oNuhHSxUTyM5itzgz5nZpMAO8jf QGTjtSgg4ehtaz3yn7QunU2caI+VvbmJ2wS54= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V920tXDgaE7z8bCeiKLUkBejFiJJs4xj6S06IJdrTDo=; b=G8qd004QECfmIZr2BxyP3UQBxSxQALQpOXJiIWGvqWHyRqi0HthMnyffCBCQ3X6zUS POEH+CgA3KpDdkyVBLURzlzkgzhR9j690hw5LK92x2vV2kXZJMewRMFS7dvxK6w6f0Ww OFvCjbfHEsTITbzM9n5xkg7Ii8F0Ou2HVJ2/z1AmK/6q8ruMQl8Al5Im/X61j8srzJLb q/ESLc+Mgn4LxMRGwXMMAKw/hO8Wbl2vnCAwUbCddaTnz7xVpmu5SDLryH7NApMb38Xt I+FMeuIs6E17UfKxRg64mBAmQVSpDqQ4yaqwBA01nbxfxN/SS87u+VcQIZamPxiN29wZ kWgA== X-Gm-Message-State: AO0yUKUN71CKs4rx8NFX1m6qq9JLI7iil36E0g27FgXc695Ogq8lWByx wFMVdfN8GxPMnbru3ygzQGMYKg== X-Received: by 2002:a05:600c:2e06:b0:3dd:dd46:1274 with SMTP id o6-20020a05600c2e0600b003dddd461274mr2822832wmf.4.1675269329591; Wed, 01 Feb 2023 08:35:29 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:29 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 5/8] ftrace: Make DIRECT_CALLS work WITH_ARGS and !WITH_REGS Date: Wed, 1 Feb 2023 17:34:17 +0100 Message-Id: <20230201163420.1579014-6-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647432980532427?= X-GMAIL-MSGID: =?utf-8?q?1756647432980532427?= Direct called trampolines can be called in two ways: - either from the ftrace callsite. In this case, they do not access any struct ftrace_regs nor pt_regs - Or, if a ftrace ops is also attached, from the end of a ftrace trampoline. In this case, the call_direct_funcs ops is in charge of setting the direct call trampoline's address in a struct ftrace_regs Since "ftrace: pass fregs to arch_ftrace_set_direct_caller()", the later case no longer requires a full pt_regs. It only needs a struct ftrace_regs so DIRECT_CALLS can work with both WITH_ARGS or WITH_REGS. With architectures like arm64 already abandoning WITH_REGS in favor of WITH_ARGS, it's important to have DIRECT_CALLS work WITH_ARGS only. Signed-off-by: Florent Revest --- kernel/trace/Kconfig | 2 +- kernel/trace/ftrace.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 5df427a2321d..4496a7c69810 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -257,7 +257,7 @@ config DYNAMIC_FTRACE_WITH_REGS config DYNAMIC_FTRACE_WITH_DIRECT_CALLS def_bool y - depends on DYNAMIC_FTRACE_WITH_REGS + depends on DYNAMIC_FTRACE_WITH_REGS || DYNAMIC_FTRACE_WITH_ARGS depends on HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS config DYNAMIC_FTRACE_WITH_CALL_OPS diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index b0426de11c45..73b6f6489ba1 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5282,7 +5282,7 @@ static LIST_HEAD(ftrace_direct_funcs); static int register_ftrace_function_nolock(struct ftrace_ops *ops); -#define MULTI_FLAGS (FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS) +#define MULTI_FLAGS (FTRACE_OPS_FL_DIRECT) static int check_direct_multi(struct ftrace_ops *ops) { From patchwork Wed Feb 1 16:34:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51474 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp381148wrn; Wed, 1 Feb 2023 08:39:13 -0800 (PST) X-Google-Smtp-Source: AK7set8793keAv32h3yzkZrjztmsKDAN2WePVjHiuc4h3NXoqDPh+WDbwsXYPprFqiy4hlvvIPMZ X-Received: by 2002:a50:ce47:0:b0:4a2:45e3:ede3 with SMTP id k7-20020a50ce47000000b004a245e3ede3mr2672592edj.14.1675269553698; Wed, 01 Feb 2023 08:39:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269553; cv=none; d=google.com; s=arc-20160816; b=Q2FHEEV9Nhqk+Rq6L/VzwNOMUasalpcY/4lNoXMLewi6KHyfhC5SHkpQKIizFKxCW2 0olISqzTmqU8oESp1R7c/BVXHVnPIHtYDojHLqT04WYtJ/9+0rQaePMhAh647bpKk92G KEd0alKtEwq3DGZsVxXWW+jb4LG4xEwO01h47xD7OA+v7xiQTbjnPHTTgfUUQfqWMYAv CyjF/jE8YTyuX2TGvtrIojJtQz5i/psO5pYP84lBqYBuuEsBLkifC2OuqMWJsXSFqu5s J3JRmyZZOhTZejfLISgvutHBhfEPepyO7fFnEVnn75OlgFDUUVmYetfxUOgVYYrPm4KA Ughw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4h7L/6sUB2KnxbssfPmMIi6qgi9N82FF7O8XsVF6WqE=; b=aQPrFk7cDdUCqNSbgJbyZh49b2v6kIQe6jVgFGS4VhGzQk2FVPy9rKlwQz7WTpcexH z0vPg/Ua8/9LVI/gICHDeGIAbBinpY8O2n190XnHTgNDosriT3jrT7NAbTqZ06EPQpcX nh8EWgvaBKGNv6lldF8lM027L+yT1btnUj636B+6FFdujGqaQ53GPGzzpIuLli0osv8j +wYx+MQE7ckBzUX8DyfdPTRigmTNNY7ESlVAQBB91LZ6qQ2OaOr5JnB9d622GR7NoR3F eP32IU8eotGhmAiHzgeZV94K/EtzkGQ0OLTIC1mn0t+xL6NXPzDtZ6XtV0mPHHOYolP0 7YNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=eO7ef0lb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z16-20020a05640235d000b0049e1782c0edsi27210704edc.296.2023.02.01.08.38.50; Wed, 01 Feb 2023 08:39:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=eO7ef0lb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232755AbjBAQf4 (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232345AbjBAQfr (ORCPT ); Wed, 1 Feb 2023 11:35:47 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCBBB65F2C for ; Wed, 1 Feb 2023 08:35:32 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id hn2-20020a05600ca38200b003dc5cb96d46so1823832wmb.4 for ; Wed, 01 Feb 2023 08:35:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4h7L/6sUB2KnxbssfPmMIi6qgi9N82FF7O8XsVF6WqE=; b=eO7ef0lbntzUy8m51n14OyG3x4x9Mi3sefAioFTJImzU5ZFg5Vs78dtPVoYekbT3pk Cg3uxLoGda0UZ7Xy5jaLG8tClEL1RJYhO9a8twyYmoUmUPMXXAoMJjtxTKVjATmXRm3i jc39euXLJjlYSDkRk5RHgqVAXCCYmDq7Le2k8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4h7L/6sUB2KnxbssfPmMIi6qgi9N82FF7O8XsVF6WqE=; b=XVi9WQx3oO2aaEAi7B56X47ISIW9J3mrqYFEWuQWfRY3bbpg4GWiYoITahop9TvnNS wz5Il9hAJ/6/3YNJ0wJCnaJoyGENz2ArrumYrU/OW2gbqBW9F8gxjlQv4dlD0lRTIke5 F6NHi9x8dgoJZLJh5Qb1CVMxvRe0aLlM19OqyITdPWE0TA4YT6U5XGXBJup2G9cJHu+I /092TSbzdi7cTLUBgy/ZKymee4jQDMaF0LyLZeCeV7NOGQJXHfl6uNq0x9eWhDsXxmlY A81GcTyEVBV9OZGCFmY+jP8dv9yK1dZDT0/C+rAwhg25yHp+Uek2XrCBXkqTKcZHpezt ZSkg== X-Gm-Message-State: AO0yUKVSMgXGS8BIyjEsbQIrTnmrjzajZQKJhygKKHHfA3iaLV27mQZ5 GEKLBkKVMafnPa67k4Qew9qAoA== X-Received: by 2002:a05:600c:4f12:b0:3de:271:7b24 with SMTP id l18-20020a05600c4f1200b003de02717b24mr2802649wmq.16.1675269331250; Wed, 01 Feb 2023 08:35:31 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:30 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Xu Kuohai , Li Huafei , Florent Revest Subject: [PATCH 6/8] ftrace: Fix dead loop caused by direct call in ftrace selftest Date: Wed, 1 Feb 2023 17:34:18 +0100 Message-Id: <20230201163420.1579014-7-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647447894887643?= X-GMAIL-MSGID: =?utf-8?q?1756647447894887643?= From: Xu Kuohai After direct call is enabled for arm64, ftrace selftest enters a dead loop: : 00 bti c 01 mov x9, x30 : 02 bl ----------> ret | lr/x30 is 03, return to 03 | 03 mov w0, #0x0 <-----------------------------| | | | dead loop! | | | 04 ret ---- lr/x30 is still 03, go back to 03 ----| The reason is that when the direct caller trace_direct_tramp() returns to the patched function trace_selftest_dynamic_test_func(), lr is still the address after the instrumented instruction in the patched function, so when the patched function exits, it returns to itself! To fix this issue, we need to restore lr before trace_direct_tramp() exits, so use a dedicated trace_direct_tramp() for arm64. Reported-by: Li Huafei Signed-off-by: Xu Kuohai Acked-by: Steven Rostedt (Google) Signed-off-by: Florent Revest Signed-off-by: Mark Rutland --- arch/arm64/include/asm/ftrace.h | 7 +++++++ arch/arm64/kernel/entry-ftrace.S | 10 ++++++++++ kernel/trace/trace_selftest.c | 2 ++ 3 files changed, 19 insertions(+) diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h index 1c2672bbbf37..cf6d9c42ff36 100644 --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -168,6 +168,13 @@ static inline bool arch_syscall_match_sym_name(const char *sym, */ return !strcmp(sym + 8, name); } + +#if defined(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS) && \ + defined(CONFIG_FTRACE_SELFTEST) +extern void ftrace_dummy_tramp(void); +#define trace_direct_tramp ftrace_dummy_tramp +#endif + #endif /* ifndef __ASSEMBLY__ */ #endif /* __ASM_FTRACE_H */ diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 350ed81324ac..9869debd22fb 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -118,6 +118,16 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) ret x9 SYM_CODE_END(ftrace_caller) +#if defined(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS) && \ + defined(CONFIG_FTRACE_SELFTEST) +SYM_CODE_START(ftrace_dummy_tramp) + bti c + mov x10, x30 + mov x30, x9 + ret x10 +SYM_CODE_END(ftrace_dummy_tramp) +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ + #else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ /* diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index 06218fc9374b..f9f5d4e8ab50 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -789,11 +789,13 @@ static struct fgraph_ops fgraph_ops __initdata = { #define CALL_DEPTH_ACCOUNT "" #endif +#ifndef trace_direct_tramp noinline __noclone static void trace_direct_tramp(void) { asm(CALL_DEPTH_ACCOUNT); } #endif +#endif /* * Pretty much the same than for the function tracer from which the selftest From patchwork Wed Feb 1 16:34:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51475 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp381201wrn; Wed, 1 Feb 2023 08:39:21 -0800 (PST) X-Google-Smtp-Source: AK7set/z+8Dldk/S9hSdPQ9HAgCo1iyD2Dw9QDH5bYKh8uD2+aOEDONm3nC6vmgfLcWTppQ6kUR1 X-Received: by 2002:a05:6402:401a:b0:499:bffb:7e58 with SMTP id d26-20020a056402401a00b00499bffb7e58mr6999076eda.20.1675269561042; Wed, 01 Feb 2023 08:39:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675269561; cv=none; d=google.com; s=arc-20160816; b=pmbW8P26gcvr4DoAA4dHIMXZmDt6Gcfe7WKdpGxgPd8kfztLgfYaLxSFb4D0gCpS64 zBv3GvEBaIEjenmzf0e+VG0YcAo+Y5bWOiJi307DLQ/xygq/3Kz0Jogl3wv6+q3fj4u1 ZaNX/4iZ6zmUN5TwQqWuThsohah9LQ3ucQjzzinON+ZHzyY2BCdcgJrK7irvaoD0hr0V ouRjo66ACti/ggBmgVp/XzehGIVLtdnxzio5pZmytG4ru8gNt+wOkTMVUQIEuqPcv8gR B+w97GdeCzwsTKmnlwboZ+Z9olwcryZoIu7ZClRxMwxGnWBlaehP9ZJdoRFaizoEHkVS 7cLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MbHY+Gmtwa2HHtiXzFaehEcw/QkGNqN5J6j1VumLZ5w=; b=hKk4nAR8qB8YsoRasCKnjI+saZifJUf6hEkhasJZDHR6zQkujCkDYhCucYVsDRj1LK ZnUxvWC8F3FmxIQJM+yg35L071whaHcxTWgPTnE08Q2G1PhyItkEJ0Qx0wR66/18sLX3 morNM28W/1MySTUii4ybK36KLXXWApulrOdR4Gtux2EPubU4QlMOmcH88Dx5dOZ29GWC maBfuME12C66A+McahjDHeOsc/jWwqoq89SBTKH/0rq47NLaO5jLCbp0j6Sghls8avsd t75wnUSVM7s0ge38/LhqwvCloswk73HXEUCEYL5BD1foqhnpbn6+IWq76lJBHblc0q9p efrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=B5Kk5qpW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d12-20020a50ea8c000000b004833b85ba26si22178075edo.403.2023.02.01.08.38.57; Wed, 01 Feb 2023 08:39:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=B5Kk5qpW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232775AbjBAQf6 (ORCPT + 99 others); Wed, 1 Feb 2023 11:35:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232390AbjBAQfr (ORCPT ); Wed, 1 Feb 2023 11:35:47 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33C2A4690 for ; Wed, 1 Feb 2023 08:35:34 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id k16so13131509wms.2 for ; Wed, 01 Feb 2023 08:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MbHY+Gmtwa2HHtiXzFaehEcw/QkGNqN5J6j1VumLZ5w=; b=B5Kk5qpWjq/NWChUoJdLAZ15WLm2v1L//Jj1sE7X8MZC3AVQVVYby77BqsgXYZk/qt b8j//orYYdKgUmjIVviXcRXUnAipPwUDBe7ZWqHGBPP1TTVK+p6qMOTArnZw+wO7ZigT 3gGskC/gE5tLjsWGLwCN3/IVdjIhSxpQLR9bo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MbHY+Gmtwa2HHtiXzFaehEcw/QkGNqN5J6j1VumLZ5w=; b=A+bFnbDoVptgbJOMPVBulMtQ3TC4mEcJm/8ucSI0C0FMEl0Vpui6goJiVelutzxZC0 qv07YKQtak5MelMMymhCpHDieU+bMr+WJYN/VeRa8OJ9VwjM6yEXvvtCLhAYVJhbwtkB jHsur4OeX00v2ww4az/Bhjx9GIKlNRYr9NOXY3ENy8B1vHFgcy6rPpI5jXnNlvGhpR6w uGVLH44aJGfbtb1FK+vfQXFJIDTbTwx8+nWCLNWxUjIrjWkm+Riht7ODMPG9Hbp1MJ3D IIAdnNWKQcHBXtyL76wpgk/uzQJuV9JMMqkrqtRYA/OHy3zlNP5/5lsI6XX8GsS5mUzt ESgw== X-Gm-Message-State: AO0yUKUPXf1RlzOCgu7p7I9dFDA8qNzAX1A5Nf2wCeoKTZxMHSGffkv6 a15trnAoUWv3NAERKd1MNqdbvw== X-Received: by 2002:a7b:c5ca:0:b0:3dd:e38b:a12f with SMTP id n10-20020a7bc5ca000000b003dde38ba12fmr5384458wmk.40.1675269332734; Wed, 01 Feb 2023 08:35:32 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:32 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 7/8] arm64: ftrace: Add direct call support Date: Wed, 1 Feb 2023 17:34:19 +0100 Message-Id: <20230201163420.1579014-8-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756647455151882633?= X-GMAIL-MSGID: =?utf-8?q?1756647455151882633?= This builds up on the CALL_OPS work which extends the ftrace patchsite on arm64 with an ops pointer usable by the ftrace trampoline. This ops pointer is valid at all time. Indeed, it is either pointing to ftrace_list_ops or to the single ops which should be called from that patchsite. There are a few cases to distinguish: - If a direct call ops is the only one tracing a function: - If the direct called trampoline is within the reach of a BL instruction -> the ftrace patchsite jumps to the trampoline - Else -> the ftrace patchsite jumps to the ftrace_caller trampoline which reads the ops pointer in the patchsite and jumps to the direct call address stored in the ops - Else -> the ftrace patchsite jumps to the ftrace_caller trampoline and its ops literal points to ftrace_list_ops so it iterates over all registered ftrace ops, including the direct call ops and calls its call_direct_funcs handler which stores the direct called trampoline's address in the ftrace_regs and the ftrace_caller trampoline will return to that address instead of returning to the traced function Signed-off-by: Florent Revest Suggested-by: Mark Rutland --- arch/arm64/Kconfig | 2 ++ arch/arm64/include/asm/ftrace.h | 17 +++++++++ arch/arm64/kernel/asm-offsets.c | 6 ++++ arch/arm64/kernel/entry-ftrace.S | 60 +++++++++++++++++++++++--------- arch/arm64/kernel/ftrace.c | 36 ++++++++++++++++--- 5 files changed, 100 insertions(+), 21 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 6f6f37161cf6..7deafd653c42 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -188,6 +188,8 @@ config ARM64 select HAVE_DYNAMIC_FTRACE select HAVE_DYNAMIC_FTRACE_WITH_ARGS \ if $(cc-option,-fpatchable-function-entry=2) + select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS \ + if DYNAMIC_FTRACE_WITH_ARGS && DYNAMIC_FTRACE_WITH_CALL_OPS select HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS \ if (DYNAMIC_FTRACE_WITH_ARGS && !CFI_CLANG) select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY \ diff --git a/arch/arm64/include/asm/ftrace.h b/arch/arm64/include/asm/ftrace.h index cf6d9c42ff36..9fb95966b6d5 100644 --- a/arch/arm64/include/asm/ftrace.h +++ b/arch/arm64/include/asm/ftrace.h @@ -80,6 +80,10 @@ struct ftrace_regs { unsigned long sp; unsigned long pc; + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long custom_tramp; +#endif }; static __always_inline unsigned long @@ -136,6 +140,19 @@ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec); void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, struct ftrace_ops *op, struct ftrace_regs *fregs); #define ftrace_graph_func ftrace_graph_func + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS +static inline void arch_ftrace_set_direct_caller(struct ftrace_regs *fregs, + unsigned long addr) +{ + /* + * The ftrace trampoline will return to this address instead of the + * instrumented function. + */ + fregs->custom_tramp = addr; +} +#endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ + #endif #define ftrace_return_address(n) return_address(n) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index ae345b06e9f7..c67f0373b88c 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -93,6 +93,9 @@ int main(void) DEFINE(FREGS_LR, offsetof(struct ftrace_regs, lr)); DEFINE(FREGS_SP, offsetof(struct ftrace_regs, sp)); DEFINE(FREGS_PC, offsetof(struct ftrace_regs, pc)); +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + DEFINE(FREGS_CUSTOM_TRAMP, offsetof(struct ftrace_regs, custom_tramp)); +#endif DEFINE(FREGS_SIZE, sizeof(struct ftrace_regs)); BLANK(); #endif @@ -197,6 +200,9 @@ int main(void) #endif #ifdef CONFIG_FUNCTION_TRACER DEFINE(FTRACE_OPS_FUNC, offsetof(struct ftrace_ops, func)); +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + DEFINE(FTRACE_OPS_DIRECT_CALL, offsetof(struct ftrace_ops, direct_call)); +#endif #endif return 0; } diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 9869debd22fb..0576f38e6362 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -36,6 +36,30 @@ SYM_CODE_START(ftrace_caller) bti c +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS + /* + * The literal pointer to the ops is at an 8-byte aligned boundary + * which is either 12 or 16 bytes before the BL instruction in the call + * site. See ftrace_call_adjust() for details. + * + * Therefore here the LR points at `literal + 16` or `literal + 20`, + * and we can find the address of the literal in either case by + * aligning to an 8-byte boundary and subtracting 16. We do the + * alignment first as this allows us to fold the subtraction into the + * LDR. + */ + bic x11, x30, 0x7 + ldr x11, [x11, #-(4 * AARCH64_INSN_SIZE)] // op + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + /* If the op has a direct call address, return to it */ + ldr x12, [x11, #FTRACE_OPS_DIRECT_CALL] // op->direct_call + cbz x12, 1f + ret x12 +1: +#endif +#endif + /* Save original SP */ mov x10, sp @@ -57,6 +81,11 @@ SYM_CODE_START(ftrace_caller) /* Save the PC after the ftrace callsite */ str x30, [sp, #FREGS_PC] +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + /* Set custom_tramp to zero */ + str xzr, [sp, #FREGS_CUSTOM_TRAMP] +#endif + /* Create a frame record for the callsite above the ftrace regs */ stp x29, x9, [sp, #FREGS_SIZE + 16] add x29, sp, #FREGS_SIZE + 16 @@ -71,20 +100,7 @@ SYM_CODE_START(ftrace_caller) mov x3, sp // regs #ifdef CONFIG_DYNAMIC_FTRACE_WITH_CALL_OPS - /* - * The literal pointer to the ops is at an 8-byte aligned boundary - * which is either 12 or 16 bytes before the BL instruction in the call - * site. See ftrace_call_adjust() for details. - * - * Therefore here the LR points at `literal + 16` or `literal + 20`, - * and we can find the address of the literal in either case by - * aligning to an 8-byte boundary and subtracting 16. We do the - * alignment first as this allows us to fold the subtraction into the - * LDR. - */ - bic x2, x30, 0x7 - ldr x2, [x2, #-16] // op - + mov x2, x11 // op ldr x4, [x2, #FTRACE_OPS_FUNC] // op->func blr x4 // op->func(ip, parent_ip, op, regs) @@ -110,12 +126,24 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) /* Restore the callsite's FP, LR, PC */ ldr x29, [sp, #FREGS_FP] ldr x30, [sp, #FREGS_LR] - ldr x9, [sp, #FREGS_PC] + ldr x10, [sp, #FREGS_PC] + +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + ldr x11, [sp, #FREGS_CUSTOM_TRAMP] + cbz x11, 1f + /* Set x9 to parent ip before jump to custom trampoline */ + mov x9, x30 + /* Set lr to self ip */ + ldr x30, [sp, #FREGS_PC] + /* Set x10 (used for return address) to custom trampoline */ + mov x10, x11 +1: +#endif /* Restore the callsite's SP */ add sp, sp, #FREGS_SIZE + 32 - ret x9 + ret x10 SYM_CODE_END(ftrace_caller) #if defined(CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS) && \ diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index 5545fe1a9012..758436727fba 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -206,6 +206,13 @@ static struct plt_entry *get_ftrace_plt(struct module *mod, unsigned long addr) return NULL; } +static bool reachable_by_bl(unsigned long addr, unsigned long pc) +{ + long offset = (long)addr - (long)pc; + + return offset >= -SZ_128M && offset < SZ_128M; +} + /* * Find the address the callsite must branch to in order to reach '*addr'. * @@ -220,14 +227,21 @@ static bool ftrace_find_callable_addr(struct dyn_ftrace *rec, unsigned long *addr) { unsigned long pc = rec->ip; - long offset = (long)*addr - (long)pc; struct plt_entry *plt; + /* + * If a custom trampoline is unreachable, rely on the ftrace_caller + * trampoline which knows how to indirectly reach that trampoline + * through ops->direct_call. + */ + if (*addr != FTRACE_ADDR && !reachable_by_bl(*addr, pc)) + *addr = FTRACE_ADDR; + /* * When the target is within range of the 'BL' instruction, use 'addr' * as-is and branch to that directly. */ - if (offset >= -SZ_128M && offset < SZ_128M) + if (reachable_by_bl(*addr, pc)) return true; /* @@ -330,12 +344,24 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, unsigned long addr) { - if (WARN_ON_ONCE(old_addr != (unsigned long)ftrace_caller)) + unsigned long pc = rec->ip; + u32 old, new; + int ret; + + ret = ftrace_rec_set_ops(rec, arm64_rec_get_ops(rec)); + if (ret) + return ret; + + if (!ftrace_find_callable_addr(rec, NULL, &old_addr)) return -EINVAL; - if (WARN_ON_ONCE(addr != (unsigned long)ftrace_caller)) + if (!ftrace_find_callable_addr(rec, NULL, &addr)) return -EINVAL; - return ftrace_rec_update_ops(rec); + old = aarch64_insn_gen_branch_imm(pc, old_addr, + AARCH64_INSN_BRANCH_LINK); + new = aarch64_insn_gen_branch_imm(pc, addr, AARCH64_INSN_BRANCH_LINK); + + return ftrace_modify_code(pc, old, new, true); } #endif From patchwork Wed Feb 1 16:34:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 51477 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387253wrn; Wed, 1 Feb 2023 08:52:02 -0800 (PST) X-Google-Smtp-Source: AK7set8yxPLB51MZTWSOMq+BeB/+QAYOlGG1r5OF+esTMsPm7fPMSLSdGYgZgzShZXdSvBqekL0K X-Received: by 2002:a17:906:184a:b0:889:ed81:dff7 with SMTP id w10-20020a170906184a00b00889ed81dff7mr2981231eje.9.1675270322632; Wed, 01 Feb 2023 08:52:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675270322; cv=none; d=google.com; s=arc-20160816; b=VXjo4hb4FecCQXPFf/2sSkClcFNMwgDtAoC+2JtklUP1qGa7OlWwQ4dXzkYCIs5ACO SHWwIJoNlyuvL2WYUYiuHmj1SBDb2J4Yj8dZnqY60Wuv8ZiPieKOrsGlNcXOyEotPAH6 3/cd/zg3cARilulluo50qqEQpg4CDBbFFsTt79gictmJt+aPRltbZx7n6dGHUJYTm+S2 kRM0F5A/w6RydBR/STiRKPeHA6ADh3fG18nbXsOKFXy77/7M9JYxutoml6JcTGgAZFFS rqeLVOiSQfG8xKj7IEPPJsqdX29IdSl1qhwHrYFzPwQyBYke0UQSivnco/0d9D95xUjj lmyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dDqx7ldh5sGOgDsQIvKZBz9tRbOKRXiGSQRucIDUJVg=; b=sjiNKpoNuWQIeXiWZthPiINrL7n5mfVgjeTqmQFMGwi6vCWrSwOVuml03b6RltGwNt 1gzyVJNf+XwAszBima3KBCDX7IAhdJ4yzlu6JpDx6GKkJN1snRJF1qkDWpZHLaj3P5OZ lDdtWeTIpxe8TL/GmiM0MkIRla8g5Ppw/mpnI8RFcpvatiDfCQs0B+xo9aFWPt0P/08G 90SrVr+qn591J7l/jMch8l3VEwGOKh6fiDjWTNHi3aQ4ByuQrjHc8PB8tqQtJuCblXQm oG1q94DlL/UMYbg1uMbgxatCkmMUl/mxmEciEJi5Puxn0Zy2WcGPXvuhEUKT1mB2RZwP nZMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=J54wQ4Ld; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 24-20020a170906009800b008879a229025si11920434ejc.333.2023.02.01.08.51.36; Wed, 01 Feb 2023 08:52:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=J54wQ4Ld; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232491AbjBAQgF (ORCPT + 99 others); Wed, 1 Feb 2023 11:36:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232530AbjBAQfs (ORCPT ); Wed, 1 Feb 2023 11:35:48 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEEAF6A739 for ; Wed, 1 Feb 2023 08:35:34 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id hn2-20020a05600ca38200b003dc5cb96d46so1823966wmb.4 for ; Wed, 01 Feb 2023 08:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dDqx7ldh5sGOgDsQIvKZBz9tRbOKRXiGSQRucIDUJVg=; b=J54wQ4LdJPNiFkmWlYnjOwU/5oAaLycrib7NWTRHHuK6LMI3Oh32ersiqtFKxfcDbf bB6OiooeswDyNYnjN3aY6RBtQR6dJmPIK0aKZ+9s0YM5RuF6lyeC+gyvairavzp9M7YA N5I7wrCbCmBvjNwadTlPK7Y0aM+0Hxi4Dk35E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dDqx7ldh5sGOgDsQIvKZBz9tRbOKRXiGSQRucIDUJVg=; b=q4OLcavR5nE4tZiiTgiTdRgE2pRWrTZybuxPHzikCFGxKNUYPhLP1POXocvyQXCrLU 2mjfnA3B+zMy7VHCjPXJaKyKHy5meATprWHQPe/SFVy0RVwjXeNl9uYnuz40pv9LAUZM bqLQ9zFj2fHez1iE3cSfVkOZn4TDawKc+aGAn17bRDPUW5kqa3mUxgNjWYjdlYyhnspf S8RgH5viM0qZYwj180J6gFmNOjMKvpj0ks3bBDLNqSQWIMcQkzsajAmgIs2QcL9TdA+2 qc8HkKzPxa/cboyJFojee/re5WR3HLOLDliZxy8w9QcBFU4L12EydjeGsxAZx2svTtDh NYEw== X-Gm-Message-State: AO0yUKWu+7dK7/fB3l19x3eHlZksxPIVyXPbGniSNWmwe4hQM27YAoIP qG1Z9+gkaSSkL8mw/weooMRwCA== X-Received: by 2002:a05:600c:4fd2:b0:3dc:523c:22b4 with SMTP id o18-20020a05600c4fd200b003dc523c22b4mr2671866wmq.27.1675269334144; Wed, 01 Feb 2023 08:35:34 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:33 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 8/8] arm64: ftrace: Add direct called trampoline samples support Date: Wed, 1 Feb 2023 17:34:20 +0100 Message-Id: <20230201163420.1579014-9-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756648253604680577?= X-GMAIL-MSGID: =?utf-8?q?1756648253604680577?= The ftrace samples need per-architecture trampoline implementations to save and restore argument registers around the calls to my_direct_func* and to restore polluted registers (eg: x30). These samples also include which does not exist on arm64 and which, on arm64, is not necessary and redefines previously defined macros (resulting in warnings) so these includes are guarded by !CONFIG_ARM64. Signed-off-by: Florent Revest --- arch/arm64/Kconfig | 2 ++ samples/ftrace/ftrace-direct-modify.c | 32 ++++++++++++++++++ samples/ftrace/ftrace-direct-multi-modify.c | 36 +++++++++++++++++++++ samples/ftrace/ftrace-direct-multi.c | 22 +++++++++++++ samples/ftrace/ftrace-direct-too.c | 25 ++++++++++++++ samples/ftrace/ftrace-direct.c | 23 +++++++++++++ 6 files changed, 140 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7deafd653c42..5480ef8eaa2a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -194,6 +194,8 @@ config ARM64 if (DYNAMIC_FTRACE_WITH_ARGS && !CFI_CLANG) select FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY \ if DYNAMIC_FTRACE_WITH_ARGS + select HAVE_SAMPLE_FTRACE_DIRECT + select HAVE_SAMPLE_FTRACE_DIRECT_MULTI select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_FAST_GUP select HAVE_FTRACE_MCOUNT_RECORD diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c index 0f1b8859e1e1..944b327e130c 100644 --- a/samples/ftrace/ftrace-direct-modify.c +++ b/samples/ftrace/ftrace-direct-modify.c @@ -2,8 +2,10 @@ #include #include #include +#ifndef CONFIG_ARM64 #include #include +#endif extern void my_direct_func1(void); extern void my_direct_func2(void); @@ -96,6 +98,36 @@ asm ( #endif /* CONFIG_S390 */ +#ifdef CONFIG_ARM64 + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp1, @function\n" +" .globl my_tramp1\n" +" my_tramp1:" +" sub sp, sp, #16\n" +" stp x9, x30, [sp]\n" +" bl my_direct_func1\n" +" ldp x30, x9, [sp]\n" +" add sp, sp, #16\n" +" ret x9\n" +" .size my_tramp1, .-my_tramp1\n" + +" .type my_tramp2, @function\n" +" .globl my_tramp2\n" +" my_tramp2:" +" sub sp, sp, #16\n" +" stp x9, x30, [sp]\n" +" bl my_direct_func2\n" +" ldp x30, x9, [sp]\n" +" add sp, sp, #16\n" +" ret x9\n" +" .size my_tramp2, .-my_tramp2\n" +" .popsection\n" +); + +#endif /* CONFIG_ARM64 */ + static struct ftrace_ops direct; static unsigned long my_tramp = (unsigned long)my_tramp1; diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c index 407c56325e65..ae1f97271d1a 100644 --- a/samples/ftrace/ftrace-direct-multi-modify.c +++ b/samples/ftrace/ftrace-direct-multi-modify.c @@ -2,8 +2,10 @@ #include #include #include +#ifndef CONFIG_ARM64 #include #include +#endif extern void my_direct_func1(unsigned long ip); extern void my_direct_func2(unsigned long ip); @@ -103,6 +105,40 @@ asm ( #endif /* CONFIG_S390 */ +#ifdef CONFIG_ARM64 + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp1, @function\n" +" .globl my_tramp1\n" +" my_tramp1:" +" sub sp, sp, #32\n" +" stp x9, x30, [sp]\n" +" str x0, [sp, #16]\n" +" bl my_direct_func1\n" +" ldp x30, x9, [sp]\n" +" ldr x0, [sp, #16]\n" +" add sp, sp, #32\n" +" ret x9\n" +" .size my_tramp1, .-my_tramp1\n" + +" .type my_tramp2, @function\n" +" .globl my_tramp2\n" +" my_tramp2:" +" sub sp, sp, #32\n" +" stp x9, x30, [sp]\n" +" str x0, [sp, #16]\n" +" bl my_direct_func2\n" +" ldp x30, x9, [sp]\n" +" ldr x0, [sp, #16]\n" +" add sp, sp, #32\n" +" ret x9\n" +" .size my_tramp2, .-my_tramp2\n" +" .popsection\n" +); + +#endif /* CONFIG_ARM64 */ + static unsigned long my_tramp = (unsigned long)my_tramp1; static unsigned long tramps[2] = { (unsigned long)my_tramp1, diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c index 42a94806a446..84713fe28b74 100644 --- a/samples/ftrace/ftrace-direct-multi.c +++ b/samples/ftrace/ftrace-direct-multi.c @@ -4,8 +4,10 @@ #include /* for handle_mm_fault() */ #include #include +#ifndef CONFIG_ARM64 #include #include +#endif extern void my_direct_func(unsigned long ip); @@ -66,6 +68,26 @@ asm ( #endif /* CONFIG_S390 */ +#ifdef CONFIG_ARM64 + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:" +" sub sp, sp, #32\n" +" stp x9, x30, [sp]\n" +" str x0, [sp, #16]\n" +" bl my_direct_func\n" +" ldp x30, x9, [sp]\n" +" ldr x0, [sp, #16]\n" +" add sp, sp, #32\n" +" ret x9\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +#endif /* CONFIG_ARM64 */ static struct ftrace_ops direct; static int __init ftrace_direct_multi_init(void) diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c index 7ee5dd3cc61d..f46ee08caa2b 100644 --- a/samples/ftrace/ftrace-direct-too.c +++ b/samples/ftrace/ftrace-direct-too.c @@ -3,8 +3,10 @@ #include /* for handle_mm_fault() */ #include +#ifndef CONFIG_ARM64 #include #include +#endif extern void my_direct_func(struct vm_area_struct *vma, unsigned long address, unsigned int flags); @@ -70,6 +72,29 @@ asm ( #endif /* CONFIG_S390 */ +#ifdef CONFIG_ARM64 + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:" +" sub sp, sp, #48\n" +" stp x9, x30, [sp]\n" +" stp x0, x1, [sp, #16]\n" +" str x2, [sp, #32]\n" +" bl my_direct_func\n" +" ldp x30, x9, [sp]\n" +" ldp x0, x1, [sp, #16]\n" +" ldr x2, [sp, #32]\n" +" add sp, sp, #48\n" +" ret x9\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +#endif /* CONFIG_ARM64 */ + static struct ftrace_ops direct; static int __init ftrace_direct_init(void) diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c index 5ffce87fa83e..e37e8d9e855c 100644 --- a/samples/ftrace/ftrace-direct.c +++ b/samples/ftrace/ftrace-direct.c @@ -3,8 +3,10 @@ #include /* for wake_up_process() */ #include +#ifndef CONFIG_ARM64 #include #include +#endif extern void my_direct_func(struct task_struct *p); @@ -63,6 +65,27 @@ asm ( #endif /* CONFIG_S390 */ +#ifdef CONFIG_ARM64 + +asm ( +" .pushsection .text, \"ax\", @progbits\n" +" .type my_tramp, @function\n" +" .globl my_tramp\n" +" my_tramp:" +" sub sp, sp, #32\n" +" stp x9, x30, [sp]\n" +" str x0, [sp, #16]\n" +" bl my_direct_func\n" +" ldp x30, x9, [sp]\n" +" ldr x0, [sp, #16]\n" +" add sp, sp, #32\n" +" ret x9\n" +" .size my_tramp, .-my_tramp\n" +" .popsection\n" +); + +#endif /* CONFIG_ARM64 */ + static struct ftrace_ops direct; static int __init ftrace_direct_init(void)