From patchwork Tue Mar 21 14:04:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72888 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1801482wrt; Tue, 21 Mar 2023 07:13:22 -0700 (PDT) X-Google-Smtp-Source: AK7set+H+VhME7L4MXpluW89epql8wY2U9uDnolG2vJR1xmzeEC+xwXwH1KdG0gjzMivlnMrH7Yf X-Received: by 2002:a05:6602:2bd1:b0:753:87a1:3198 with SMTP id s17-20020a0566022bd100b0075387a13198mr1184822iov.5.1679408002656; Tue, 21 Mar 2023 07:13:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679408002; cv=none; d=google.com; s=arc-20160816; b=EOiD877UzZC6RIYal+zlOgoG/uxnZf9C+YgqxYiseovp6lyUah6hJbCHmiWaj5ukVn ws48SRzYwKOXRBWPQDQogPo/wKC+EQ4aWz/MEcX5T5+IGXWZt7zv215C8AFQHSDcsVBo 4UB5nTlLyQbwqAAKWRb5RvdoQblziq13zAe5Xhbz4DrQweXKPz5nnQGebf8NfX+FZ68F IT65Et2sSpc9JaMP4KrkkUrkLj30OhJvN6ilD2luPJEOp+z6jBWicf4NmenEd7d8aSc7 zIVOyyPLb7okQEdnngjoxWk+mWJoq1OF+oWqUo1Z22SVHncQVssdqB7vq7EihjA+jYQB NSsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9WwprlyMlMMHhcPdyuhJ0L2KuDUXGr9VK74fm/Vyg9g=; b=aHFuJh+o26fk9kRnFDxqPurh/+gJ4z8eT1h37mRVFUdzriZB34v0jH0D50RPLRBrTN 475rGzbkDhb3YRVSH+5wuytpIOxTXgPbQNjwdMisBkeJPQD76Bn/ArS1Fv7ITQ7jajeL uPusx4Xyh199YDcHyXVMAFOTPDBn6k/i5YqZobtm8qCjudS5tPDgvNHRihulS6aoUpDt KU1tHeNQGupOCiIIr5pj6km1WhDEIcMEHljUlBElfvl+8nLGTuhAfRup1fcpGGsj5SGv HFWjNp5RWse2Dl9jBEagnG5IMD7/TSX2PcTdAVnVwQbIowM/z2jTkQG2scuzAs/eNwQp 8UDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=nLWVXysp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o1-20020a027401000000b00406302063bcsi2382981jac.147.2023.03.21.07.13.08; Tue, 21 Mar 2023 07:13:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=nLWVXysp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231361AbjCUOEn (ORCPT + 99 others); Tue, 21 Mar 2023 10:04:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231357AbjCUOEk (ORCPT ); Tue, 21 Mar 2023 10:04:40 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A68A20A3A for ; Tue, 21 Mar 2023 07:04:38 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id v25so10346477wra.12 for ; Tue, 21 Mar 2023 07:04:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9WwprlyMlMMHhcPdyuhJ0L2KuDUXGr9VK74fm/Vyg9g=; b=nLWVXyspYz9peVqzG15vQujgApPLo7W8YAlDMmwXgMiAA8jyvGC+C9kFU5nQxGTPhi zJaGZ+jIs25bcgi3FxZOM/mn7T0TS5RAADcWbBNMT8QkFTxWOR1tGMn9V9jRt+S4/O8J vhouX/tHYrYoPpKhnDRpdau1olcLao3K7xi84= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9WwprlyMlMMHhcPdyuhJ0L2KuDUXGr9VK74fm/Vyg9g=; b=oaDEx9slIkW9/qGSs+7QpKxAe8W8AXtcdBJS8qczfv1y1KQl+ojDENOYFreRRqduU9 32poQOQNyosU0qCRTzU5G9D7acHu1acDRCBv5sivu+YGRFt0zDCRwpnzjWRYTcXSxkOt OYWw1nVPb6c+H2o0nfYn6PDyJSa82y7SFy7sMdTebKEY0EHSzDsNzCCeRL1G2hAvRkZG gH/+tyCIwM5AZCw31sVVM/VSmp1TIM1N0QsS6Jvsm6kMI/6qZO6mY19lq2AzF1k/ZZCX brJsYDJGza5ZpwIfDmVTM6/E8H3iB+lapdpTvEfDgTu5aJuqt5mUEreC7QLtCdiy5fry mBPA== X-Gm-Message-State: AO0yUKU3jJEwA6hhcUzL6bqOi8miiY2CbRrpp1inLjMWDML0nrKdHlru 2OXS6L8xBp/uk2oQcx1J9c/v2la4WNWquritswg= X-Received: by 2002:a05:6000:105:b0:2d9:eb77:90d2 with SMTP id o5-20020a056000010500b002d9eb7790d2mr182791wrx.70.1679407476066; Tue, 21 Mar 2023 07:04:36 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:35 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org Subject: [PATCH v2 1/7] ftrace: Let unregister_ftrace_direct_multi() call ftrace_free_filter() Date: Tue, 21 Mar 2023 15:04:18 +0100 Message-Id: <20230321140424.345218-2-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760986926040261617?= X-GMAIL-MSGID: =?utf-8?q?1760986926040261617?= A common pattern when using the ftrace_direct_multi API is to unregister the ops and also immediately free its filter. We've noticed it's very easy for users to miss calling ftrace_free_filter(). This adds a "free_filters" argument to unregister_ftrace_direct_multi() to both remind the user they should free filters and also to make their life easier. Suggested-by: Steven Rostedt Signed-off-by: Florent Revest Acked-by: Mark Rutland --- include/linux/ftrace.h | 6 ++++-- kernel/bpf/trampoline.c | 2 +- kernel/trace/ftrace.c | 6 +++++- samples/ftrace/ftrace-direct-multi-modify.c | 3 +-- samples/ftrace/ftrace-direct-multi.c | 3 +-- 5 files changed, 12 insertions(+), 8 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 366c730beaa3..5b68ee874bc1 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -407,7 +407,8 @@ int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, unsigned long new_addr); unsigned long ftrace_find_rec_direct(unsigned long ip); int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); -int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); +int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, + bool free_filters); int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr); @@ -446,7 +447,8 @@ static inline int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned { return -ENODEV; } -static inline int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +static inline int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, + bool free_filters) { return -ENODEV; } diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index d0ed7d6f5eec..88bc23f1e10a 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -198,7 +198,7 @@ static int unregister_fentry(struct bpf_trampoline *tr, void *old_addr) int ret; if (tr->func.ftrace_managed) - ret = unregister_ftrace_direct_multi(tr->fops, (long)old_addr); + ret = unregister_ftrace_direct_multi(tr->fops, (long)old_addr, false); else ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, old_addr, NULL); diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 29baa97d0d53..fa379cf91fdb 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5804,7 +5804,8 @@ EXPORT_SYMBOL_GPL(register_ftrace_direct_multi); * 0 on success * -EINVAL - The @ops object was not properly registered. */ -int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, + bool free_filters) { struct ftrace_hash *hash = ops->func_hash->filter_hash; int err; @@ -5822,6 +5823,9 @@ int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) /* cleanup for possible another register call */ ops->func = NULL; ops->trampoline = 0; + + if (free_filters) + ftrace_free_filter(ops); return err; } EXPORT_SYMBOL_GPL(unregister_ftrace_direct_multi); diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c index b58c594efb51..196b43971cb5 100644 --- a/samples/ftrace/ftrace-direct-multi-modify.c +++ b/samples/ftrace/ftrace-direct-multi-modify.c @@ -151,8 +151,7 @@ static int __init ftrace_direct_multi_init(void) static void __exit ftrace_direct_multi_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct_multi(&direct, my_tramp); - ftrace_free_filter(&direct); + unregister_ftrace_direct_multi(&direct, my_tramp, true); } module_init(ftrace_direct_multi_init); diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c index c27cf130c319..ea0e88ee5e43 100644 --- a/samples/ftrace/ftrace-direct-multi.c +++ b/samples/ftrace/ftrace-direct-multi.c @@ -78,8 +78,7 @@ static int __init ftrace_direct_multi_init(void) static void __exit ftrace_direct_multi_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long) my_tramp); - ftrace_free_filter(&direct); + unregister_ftrace_direct_multi(&direct, (unsigned long) my_tramp, true); } module_init(ftrace_direct_multi_init); From patchwork Tue Mar 21 14:04:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72887 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1801037wrt; Tue, 21 Mar 2023 07:12:36 -0700 (PDT) X-Google-Smtp-Source: AK7set/nwCNg9eIFqpVsrjBSRFysS9gQCC5pqAFSiLftue0MS9qtDW4WikV2BGbdM5lLGjNBetTj X-Received: by 2002:aa7:9433:0:b0:625:e77b:93b2 with SMTP id y19-20020aa79433000000b00625e77b93b2mr27799pfo.5.1679407955817; Tue, 21 Mar 2023 07:12:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679407955; cv=none; d=google.com; s=arc-20160816; b=Q0g0ZMVsoXUYrimWG1c0XABXcUWCVHxreIUKxaJ7ObrUz1/zsaNkLYGt1y7sKqwYmW O+KmH+lOYvmhlGs115E8IAUNS03rRGOIx7J68k8ZCPE3d16ZD+Kb9azz4NVDa4w3H8Qx +lSu/0KW89jeDFmkAu2cnMfuckbaGf5GJrW13R5idfLHFuObzmVYdKE1qqi6Q/ueXS7z IIjQsvXXtEuAbO5PHbtQ186w9OFkIp8zkx4kIGGyJQ2AFnHBV1OWiFrFF+WLCYEEAZXv vpiDfzLeS+mFnE4Z5s6AcKFgcZQrDCYeepTU6n99WGDZowntpeO4HBsmdfHefCfdzoP1 IvZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=g31w0jns46SiUigtcWxrUaCJsT0rxK7Gtz3lyL2N27Y=; b=MM9qT3Fg4yF5UzJxkVCH36fVvnJM+K8w8wiIVzTeGeqjd1bA5VkGH10H/vCrWEbGbF xt65ZZKwvb8guOjvHYtUCaN2uRzWdj1TOfb2DJJ8ZLHv9GcqA3Fx1OvvuguUJfgN6ZE+ 9RVynR9tN6jXm7NsR3bR17lS01S/fJInScd9lWZa7ftpsEanbnEj7nsCgDSADvFrDN6B TbyQ6QHyahtaIM4aOCDuZWyvcQVb9YD3Dmrj/NLTe5Re/lOGvA0lSJ5OVsGxoDzVcIxX yhQ8PLdj7vuMmyFQ4ayB9yKQZ3h0cjwgKQqEgNZeuYMmS359DR+WbCmyc8R3fovgl2E/ AjAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=SlWbsOlv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i28-20020a63221c000000b004fb1da0e663si12734743pgi.131.2023.03.21.07.12.20; Tue, 21 Mar 2023 07:12:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=SlWbsOlv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229749AbjCUOEw (ORCPT + 99 others); Tue, 21 Mar 2023 10:04:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229734AbjCUOEk (ORCPT ); Tue, 21 Mar 2023 10:04:40 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 379BC20A00 for ; Tue, 21 Mar 2023 07:04:39 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id p16so9634430wmq.5 for ; Tue, 21 Mar 2023 07:04:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g31w0jns46SiUigtcWxrUaCJsT0rxK7Gtz3lyL2N27Y=; b=SlWbsOlvRH/OBDXJtm2l99IiIspzue7YjW5lYz/5n+wyNAMof4/4QVBivbWpCi+HMh X+5PekxDF4y76jGGFOB4iAqxDrQ9rk30NCoF4y1cd9qgKblEYSsXlE1UT/NZ9Qe7kcu4 i5IzBqIXXDo7JZ5kQaZ3j/wDS4XsI2R5i111c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g31w0jns46SiUigtcWxrUaCJsT0rxK7Gtz3lyL2N27Y=; b=Fctcu1f2z3ZTH6ketwp/FD9Ky+5uQkXN1F12GCrpqxxViM1Fei/pcxw7nULrtVrwvf kLtxOZmuqA2BLCNL7R/V05pdYOEtzrkPTj6WlhIu4X4YK+MntvS+aAUYs6dxaOkUEUaW WG4INa7hzY/MsnfEtNCJ4t/cFhCqH9SvU5LeHkTmLnWEIOUwTVjZomVSe4o3pNlTlKab 92qal6LIJvb4Oy763PmaxFa5sRsrq1BuritIlyvtUaTsS8uodBHR8I/3GMjO9+zGi3Zq 1DCvl+KWozJ9krYhbuvAtPfFG1y9chf00n5Owp9PrZ5erQHiksALn0PednPt5zelS0Z8 I2qQ== X-Gm-Message-State: AO0yUKUG8O0YYX+LxBQ1R4i9P0ti6WnuBXb8U3r8Tw+aVcNrrfF9go4j X8+0JjvpyxaDFMW9pWjlEYUZ+3rC8yr2rd0SaEo= X-Received: by 2002:a1c:ed12:0:b0:3ed:d3a7:7077 with SMTP id l18-20020a1ced12000000b003edd3a77077mr2307161wmh.40.1679407477330; Tue, 21 Mar 2023 07:04:37 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:36 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org Subject: [PATCH v2 2/7] ftrace: Replace uses of _ftrace_direct APIs with _ftrace_direct_multi Date: Tue, 21 Mar 2023 15:04:19 +0100 Message-Id: <20230321140424.345218-3-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760986876311414421?= X-GMAIL-MSGID: =?utf-8?q?1760986876311414421?= The _multi API requires that users keep their own ops but can enforce that an op is only associated to one direct call. Signed-off-by: Florent Revest Acked-by: Mark Rutland Tested-by: Mark Rutland --- kernel/trace/trace_selftest.c | 10 ++++++---- samples/ftrace/ftrace-direct-modify.c | 12 ++++++++---- samples/ftrace/ftrace-direct-too.c | 12 +++++++----- samples/ftrace/ftrace-direct.c | 12 +++++++----- 4 files changed, 28 insertions(+), 18 deletions(-) diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index ff0536cea968..9ce80b3ad06d 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -785,6 +785,7 @@ static struct fgraph_ops fgraph_ops __initdata = { }; #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS +static struct ftrace_ops direct; #ifndef CALL_DEPTH_ACCOUNT #define CALL_DEPTH_ACCOUNT "" #endif @@ -870,8 +871,8 @@ trace_selftest_startup_function_graph(struct tracer *trace, * Register direct function together with graph tracer * and make sure we get graph trace. */ - ret = register_ftrace_direct((unsigned long) DYN_FTRACE_TEST_NAME, - (unsigned long) trace_direct_tramp); + ftrace_set_filter_ip(&direct, (unsigned long)DYN_FTRACE_TEST_NAME, 0, 0); + ret = register_ftrace_direct_multi(&direct, (unsigned long)trace_direct_tramp); if (ret) goto out; @@ -891,8 +892,9 @@ trace_selftest_startup_function_graph(struct tracer *trace, unregister_ftrace_graph(&fgraph_ops); - ret = unregister_ftrace_direct((unsigned long) DYN_FTRACE_TEST_NAME, - (unsigned long) trace_direct_tramp); + ret = unregister_ftrace_direct_multi(&direct, + (unsigned long) trace_direct_tramp, + true); if (ret) goto out; diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c index d93abbcb1f4c..f01ac74bac10 100644 --- a/samples/ftrace/ftrace-direct-modify.c +++ b/samples/ftrace/ftrace-direct-modify.c @@ -96,6 +96,8 @@ asm ( #endif /* CONFIG_S390 */ +static struct ftrace_ops direct; + static unsigned long my_tramp = (unsigned long)my_tramp1; static unsigned long tramps[2] = { (unsigned long)my_tramp1, @@ -114,7 +116,7 @@ static int simple_thread(void *arg) if (ret) continue; t ^= 1; - ret = modify_ftrace_direct(my_ip, my_tramp, tramps[t]); + ret = modify_ftrace_direct_multi(&direct, tramps[t]); if (!ret) my_tramp = tramps[t]; WARN_ON_ONCE(ret); @@ -129,7 +131,9 @@ static int __init ftrace_direct_init(void) { int ret; - ret = register_ftrace_direct(my_ip, my_tramp); + ftrace_set_filter_ip(&direct, (unsigned long) my_ip, 0, 0); + ret = register_ftrace_direct_multi(&direct, my_tramp); + if (!ret) simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn"); return ret; @@ -138,12 +142,12 @@ static int __init ftrace_direct_init(void) static void __exit ftrace_direct_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct(my_ip, my_tramp); + unregister_ftrace_direct_multi(&direct, my_tramp, true); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct()"); +MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct_multi()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c index 8139dce2a31c..05c3585ac15e 100644 --- a/samples/ftrace/ftrace-direct-too.c +++ b/samples/ftrace/ftrace-direct-too.c @@ -70,21 +70,23 @@ asm ( #endif /* CONFIG_S390 */ +static struct ftrace_ops direct; + static int __init ftrace_direct_init(void) { - return register_ftrace_direct((unsigned long)handle_mm_fault, - (unsigned long)my_tramp); + ftrace_set_filter_ip(&direct, (unsigned long) handle_mm_fault, 0, 0); + + return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct((unsigned long)handle_mm_fault, - (unsigned long)my_tramp); + unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp, true); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct()"); +MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct_multi()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c index 1d3d307ca33d..42ec9e39453b 100644 --- a/samples/ftrace/ftrace-direct.c +++ b/samples/ftrace/ftrace-direct.c @@ -63,21 +63,23 @@ asm ( #endif /* CONFIG_S390 */ +static struct ftrace_ops direct; + static int __init ftrace_direct_init(void) { - return register_ftrace_direct((unsigned long)wake_up_process, - (unsigned long)my_tramp); + ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); + + return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct((unsigned long)wake_up_process, - (unsigned long)my_tramp); + unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp, true); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using register_ftrace_direct()"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); MODULE_LICENSE("GPL"); From patchwork Tue Mar 21 14:04:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72896 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1808004wrt; Tue, 21 Mar 2023 07:23:43 -0700 (PDT) X-Google-Smtp-Source: AK7set/ZmJiwvWuUjfjC7e4sz2krQmhtYjr57Dmx3IcZyt0IFOsvk1PNaWuwUSez9NODr7XTyeEt X-Received: by 2002:a17:903:1386:b0:1a1:c7b2:e7c7 with SMTP id jx6-20020a170903138600b001a1c7b2e7c7mr2282956plb.49.1679408623305; Tue, 21 Mar 2023 07:23:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679408623; cv=none; d=google.com; s=arc-20160816; b=yXvTd3fx02BaYhnxDOWn2xXqkWMv/blPy77XoFVgN90TYzHQzvYjMGxE+9d23T6fx+ Wr3QBxdSfNA+g2hhIHaFqxx8qPUYs3oQU3xoVb5UR1+lMtcGI8O+xBRCI3lUtLOZFbWp vLOY+O/10hOxQtSEsERK954QEOalMILLXGPibF/CVc6C8bTWd3eZbhTfRKuvVHttKQua vJKECGXyqevZI62TJe8V1tisQ4FhaeaxIjAbCpMKfb+xLBc3fA2N8gWs8QxUAaQnNWGM hUf/3HEhyKinl3k6Zw2eUc/cFxdUu81Z+VoxU3X5+n9D/nPQnaV5JynXI7a9oOPAWvVg GUtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ykroFktY9fNmY/87dXJCl889ZnSkfkqipjmmGP7vOgc=; b=xZOWkc+dCNaZspgBxzKm8LSZbJntOOwKUUV3Ch2j0o085EbjH+pktEtPalFvLcT+KI Vl7TFTZVX40NLPRIYtvbvh+5CoY5UsHgwEoEoKjMy4744o0Rz8ajkSEbZJDJv0OAU5iP D4Ozxrb9BIiuO22zfRNuqB4y3k/B1fuGHMoZXISIkJNUKI69NR1f+FZgZpyJBaPbRfxf Bf5/Hg7ZDt9GPwDBgFiJcpnhfyo3gA+eOQfISJVCUrHmOQnbqEFzLrd7VHjZiqf57+cS 4fd/FfjlbDAA2o914ioDERD2szdALKZRB0Wgqp1r12XnC4Q40xRofPZ4EimqtMmMl2Yv OpZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=VykILIRu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d17-20020a170902ced100b001a18ec2d863si7084474plg.112.2023.03.21.07.23.27; Tue, 21 Mar 2023 07:23:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=VykILIRu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231394AbjCUOEz (ORCPT + 99 others); Tue, 21 Mar 2023 10:04:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231362AbjCUOEn (ORCPT ); Tue, 21 Mar 2023 10:04:43 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5F9021974 for ; Tue, 21 Mar 2023 07:04:40 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id p34so4617552wms.3 for ; Tue, 21 Mar 2023 07:04:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ykroFktY9fNmY/87dXJCl889ZnSkfkqipjmmGP7vOgc=; b=VykILIRu+PxytE834wAk3AmXUwDqBI+8+wJpR9J4VMKMjyGeevk4UN9dUippTv2V+R T/q6QCPA8DA1GVguLN3nDIBaZpr0tLIn+wKdwVcJj6d0CsEtq1rfD1B2DogWq5SmdAev Qkp2brdHH0Bzy6m62qYGv0GQona9qOeoAf8/E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ykroFktY9fNmY/87dXJCl889ZnSkfkqipjmmGP7vOgc=; b=7EnecRHbaJLadE6jKMGRfAe0FIWjhO6qDlV08mbR0icaLX4Q3rdPkhtcWYSTbUFS0M Ghjn/DyPpO6QIUxZr/oJ5pFFtPs+Zz00i3spufhcJ6U4UwFooqrKi0EleDPMjk6hhd43 js3Jd6Zy53jxv7lTCnOfa+cUN80r5HtNNRBjQOV8sFN3YOyI4G7bk5kxNSicgGeWjz8K +aRMyUF79T+koCXFgvO6sdxmMnpXBZVZH6sWj+Tp+iUHImHfCll+66wbZwmgCHuC0upz zW5aai8yVJBsev4mPVfw1ocGDMq5A/JXvqzmnPrO5ACqC/x2K3DdgPi9J+BBmw+hP7d1 s9dw== X-Gm-Message-State: AO0yUKWTf0+NC8sgiWuS6XVx3u3dc7KX/ljowpW/vemLjmOEAiTbqJxX xfcQysiOidRe+afBWuA6EYV1LH8uIcJz+LOPoFY= X-Received: by 2002:a05:600c:1c1a:b0:3e1:e149:b67b with SMTP id j26-20020a05600c1c1a00b003e1e149b67bmr12660322wms.18.1679407478878; Tue, 21 Mar 2023 07:04:38 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:38 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org Subject: [PATCH v2 3/7] ftrace: Remove the legacy _ftrace_direct API Date: Tue, 21 Mar 2023 15:04:20 +0100 Message-Id: <20230321140424.345218-4-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760987576820147326?= X-GMAIL-MSGID: =?utf-8?q?1760987576820147326?= This API relies on a single global ops, used for all direct calls registered with it. However, to implement arm64 direct calls, we need each ops to point to a single direct call trampoline. Signed-off-by: Florent Revest Acked-by: Mark Rutland Tested-by: Mark Rutland --- include/linux/ftrace.h | 32 ---- kernel/trace/ftrace.c | 393 ----------------------------------------- 2 files changed, 425 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 5b68ee874bc1..2f400c9f0787 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -397,14 +397,6 @@ struct ftrace_func_entry { #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS extern int ftrace_direct_func_count; -int register_ftrace_direct(unsigned long ip, unsigned long addr); -int unregister_ftrace_direct(unsigned long ip, unsigned long addr); -int modify_ftrace_direct(unsigned long ip, unsigned long old_addr, unsigned long new_addr); -struct ftrace_direct_func *ftrace_find_direct_func(unsigned long addr); -int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, - struct dyn_ftrace *rec, - unsigned long old_addr, - unsigned long new_addr); unsigned long ftrace_find_rec_direct(unsigned long ip); int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, @@ -415,30 +407,6 @@ int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr #else struct ftrace_ops; # define ftrace_direct_func_count 0 -static inline int register_ftrace_direct(unsigned long ip, unsigned long addr) -{ - return -ENOTSUPP; -} -static inline int unregister_ftrace_direct(unsigned long ip, unsigned long addr) -{ - return -ENOTSUPP; -} -static inline int modify_ftrace_direct(unsigned long ip, - unsigned long old_addr, unsigned long new_addr) -{ - return -ENOTSUPP; -} -static inline struct ftrace_direct_func *ftrace_find_direct_func(unsigned long addr) -{ - return NULL; -} -static inline int ftrace_modify_direct_caller(struct ftrace_func_entry *entry, - struct dyn_ftrace *rec, - unsigned long old_addr, - unsigned long new_addr) -{ - return -ENODEV; -} static inline unsigned long ftrace_find_rec_direct(unsigned long ip) { return 0; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index fa379cf91fdb..fca478396d31 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2590,20 +2590,6 @@ static void call_direct_funcs(unsigned long ip, unsigned long pip, arch_ftrace_set_direct_caller(fregs, addr); } - -struct ftrace_ops direct_ops = { - .func = call_direct_funcs, - .flags = FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS - | FTRACE_OPS_FL_PERMANENT, - /* - * By declaring the main trampoline as this trampoline - * it will never have one allocated for it. Allocated - * trampolines should not call direct functions. - * The direct_ops should only be called by the builtin - * ftrace_regs_caller trampoline. - */ - .trampoline = FTRACE_REGS_ADDR, -}; #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** @@ -5300,387 +5286,8 @@ struct ftrace_direct_func { static LIST_HEAD(ftrace_direct_funcs); -/** - * ftrace_find_direct_func - test an address if it is a registered direct caller - * @addr: The address of a registered direct caller - * - * This searches to see if a ftrace direct caller has been registered - * at a specific address, and if so, it returns a descriptor for it. - * - * This can be used by architecture code to see if an address is - * a direct caller (trampoline) attached to a fentry/mcount location. - * This is useful for the function_graph tracer, as it may need to - * do adjustments if it traced a location that also has a direct - * trampoline attached to it. - */ -struct ftrace_direct_func *ftrace_find_direct_func(unsigned long addr) -{ - struct ftrace_direct_func *entry; - bool found = false; - - /* May be called by fgraph trampoline (protected by rcu tasks) */ - list_for_each_entry_rcu(entry, &ftrace_direct_funcs, next) { - if (entry->addr == addr) { - found = true; - break; - } - } - if (found) - return entry; - - return NULL; -} - -static struct ftrace_direct_func *ftrace_alloc_direct_func(unsigned long addr) -{ - struct ftrace_direct_func *direct; - - direct = kmalloc(sizeof(*direct), GFP_KERNEL); - if (!direct) - return NULL; - direct->addr = addr; - direct->count = 0; - list_add_rcu(&direct->next, &ftrace_direct_funcs); - ftrace_direct_func_count++; - return direct; -} - static int register_ftrace_function_nolock(struct ftrace_ops *ops); -/** - * register_ftrace_direct - Call a custom trampoline directly - * @ip: The address of the nop at the beginning of a function - * @addr: The address of the trampoline to call at @ip - * - * This is used to connect a direct call from the nop location (@ip) - * at the start of ftrace traced functions. The location that it calls - * (@addr) must be able to handle a direct call, and save the parameters - * of the function being traced, and restore them (or inject new ones - * if needed), before returning. - * - * Returns: - * 0 on success - * -EBUSY - Another direct function is already attached (there can be only one) - * -ENODEV - @ip does not point to a ftrace nop location (or not supported) - * -ENOMEM - There was an allocation failure. - */ -int register_ftrace_direct(unsigned long ip, unsigned long addr) -{ - struct ftrace_direct_func *direct; - struct ftrace_func_entry *entry; - struct ftrace_hash *free_hash = NULL; - struct dyn_ftrace *rec; - int ret = -ENODEV; - - mutex_lock(&direct_mutex); - - ip = ftrace_location(ip); - if (!ip) - goto out_unlock; - - /* See if there's a direct function at @ip already */ - ret = -EBUSY; - if (ftrace_find_rec_direct(ip)) - goto out_unlock; - - ret = -ENODEV; - rec = lookup_rec(ip, ip); - if (!rec) - goto out_unlock; - - /* - * Check if the rec says it has a direct call but we didn't - * find one earlier? - */ - if (WARN_ON(rec->flags & FTRACE_FL_DIRECT)) - goto out_unlock; - - /* Make sure the ip points to the exact record */ - if (ip != rec->ip) { - ip = rec->ip; - /* Need to check this ip for a direct. */ - if (ftrace_find_rec_direct(ip)) - goto out_unlock; - } - - ret = -ENOMEM; - direct = ftrace_find_direct_func(addr); - if (!direct) { - direct = ftrace_alloc_direct_func(addr); - if (!direct) - goto out_unlock; - } - - entry = ftrace_add_rec_direct(ip, addr, &free_hash); - if (!entry) - goto out_unlock; - - ret = ftrace_set_filter_ip(&direct_ops, ip, 0, 0); - - if (!ret && !(direct_ops.flags & FTRACE_OPS_FL_ENABLED)) { - ret = register_ftrace_function_nolock(&direct_ops); - if (ret) - ftrace_set_filter_ip(&direct_ops, ip, 1, 0); - } - - if (ret) { - remove_hash_entry(direct_functions, entry); - kfree(entry); - if (!direct->count) { - list_del_rcu(&direct->next); - synchronize_rcu_tasks(); - kfree(direct); - if (free_hash) - free_ftrace_hash(free_hash); - free_hash = NULL; - ftrace_direct_func_count--; - } - } else { - direct->count++; - } - out_unlock: - mutex_unlock(&direct_mutex); - - if (free_hash) { - synchronize_rcu_tasks(); - free_ftrace_hash(free_hash); - } - - return ret; -} -EXPORT_SYMBOL_GPL(register_ftrace_direct); - -static struct ftrace_func_entry *find_direct_entry(unsigned long *ip, - struct dyn_ftrace **recp) -{ - struct ftrace_func_entry *entry; - struct dyn_ftrace *rec; - - rec = lookup_rec(*ip, *ip); - if (!rec) - return NULL; - - entry = __ftrace_lookup_ip(direct_functions, rec->ip); - if (!entry) { - WARN_ON(rec->flags & FTRACE_FL_DIRECT); - return NULL; - } - - WARN_ON(!(rec->flags & FTRACE_FL_DIRECT)); - - /* Passed in ip just needs to be on the call site */ - *ip = rec->ip; - - if (recp) - *recp = rec; - - return entry; -} - -int unregister_ftrace_direct(unsigned long ip, unsigned long addr) -{ - struct ftrace_direct_func *direct; - struct ftrace_func_entry *entry; - struct ftrace_hash *hash; - int ret = -ENODEV; - - mutex_lock(&direct_mutex); - - ip = ftrace_location(ip); - if (!ip) - goto out_unlock; - - entry = find_direct_entry(&ip, NULL); - if (!entry) - goto out_unlock; - - hash = direct_ops.func_hash->filter_hash; - if (hash->count == 1) - unregister_ftrace_function(&direct_ops); - - ret = ftrace_set_filter_ip(&direct_ops, ip, 1, 0); - - WARN_ON(ret); - - remove_hash_entry(direct_functions, entry); - - direct = ftrace_find_direct_func(addr); - if (!WARN_ON(!direct)) { - /* This is the good path (see the ! before WARN) */ - direct->count--; - WARN_ON(direct->count < 0); - if (!direct->count) { - list_del_rcu(&direct->next); - synchronize_rcu_tasks(); - kfree(direct); - kfree(entry); - ftrace_direct_func_count--; - } - } - out_unlock: - mutex_unlock(&direct_mutex); - - return ret; -} -EXPORT_SYMBOL_GPL(unregister_ftrace_direct); - -static struct ftrace_ops stub_ops = { - .func = ftrace_stub, -}; - -/** - * ftrace_modify_direct_caller - modify ftrace nop directly - * @entry: The ftrace hash entry of the direct helper for @rec - * @rec: The record representing the function site to patch - * @old_addr: The location that the site at @rec->ip currently calls - * @new_addr: The location that the site at @rec->ip should call - * - * An architecture may overwrite this function to optimize the - * changing of the direct callback on an ftrace nop location. - * This is called with the ftrace_lock mutex held, and no other - * ftrace callbacks are on the associated record (@rec). Thus, - * it is safe to modify the ftrace record, where it should be - * currently calling @old_addr directly, to call @new_addr. - * - * This is called with direct_mutex locked. - * - * Safety checks should be made to make sure that the code at - * @rec->ip is currently calling @old_addr. And this must - * also update entry->direct to @new_addr. - */ -int __weak ftrace_modify_direct_caller(struct ftrace_func_entry *entry, - struct dyn_ftrace *rec, - unsigned long old_addr, - unsigned long new_addr) -{ - unsigned long ip = rec->ip; - int ret; - - lockdep_assert_held(&direct_mutex); - - /* - * The ftrace_lock was used to determine if the record - * had more than one registered user to it. If it did, - * we needed to prevent that from changing to do the quick - * switch. But if it did not (only a direct caller was attached) - * then this function is called. But this function can deal - * with attached callers to the rec that we care about, and - * since this function uses standard ftrace calls that take - * the ftrace_lock mutex, we need to release it. - */ - mutex_unlock(&ftrace_lock); - - /* - * By setting a stub function at the same address, we force - * the code to call the iterator and the direct_ops helper. - * This means that @ip does not call the direct call, and - * we can simply modify it. - */ - ret = ftrace_set_filter_ip(&stub_ops, ip, 0, 0); - if (ret) - goto out_lock; - - ret = register_ftrace_function_nolock(&stub_ops); - if (ret) { - ftrace_set_filter_ip(&stub_ops, ip, 1, 0); - goto out_lock; - } - - entry->direct = new_addr; - - /* - * By removing the stub, we put back the direct call, calling - * the @new_addr. - */ - unregister_ftrace_function(&stub_ops); - ftrace_set_filter_ip(&stub_ops, ip, 1, 0); - - out_lock: - mutex_lock(&ftrace_lock); - - return ret; -} - -/** - * modify_ftrace_direct - Modify an existing direct call to call something else - * @ip: The instruction pointer to modify - * @old_addr: The address that the current @ip calls directly - * @new_addr: The address that the @ip should call - * - * This modifies a ftrace direct caller at an instruction pointer without - * having to disable it first. The direct call will switch over to the - * @new_addr without missing anything. - * - * Returns: zero on success. Non zero on error, which includes: - * -ENODEV : the @ip given has no direct caller attached - * -EINVAL : the @old_addr does not match the current direct caller - */ -int modify_ftrace_direct(unsigned long ip, - unsigned long old_addr, unsigned long new_addr) -{ - struct ftrace_direct_func *direct, *new_direct = NULL; - struct ftrace_func_entry *entry; - struct dyn_ftrace *rec; - int ret = -ENODEV; - - mutex_lock(&direct_mutex); - - mutex_lock(&ftrace_lock); - - ip = ftrace_location(ip); - if (!ip) - goto out_unlock; - - entry = find_direct_entry(&ip, &rec); - if (!entry) - goto out_unlock; - - ret = -EINVAL; - if (entry->direct != old_addr) - goto out_unlock; - - direct = ftrace_find_direct_func(old_addr); - if (WARN_ON(!direct)) - goto out_unlock; - if (direct->count > 1) { - ret = -ENOMEM; - new_direct = ftrace_alloc_direct_func(new_addr); - if (!new_direct) - goto out_unlock; - direct->count--; - new_direct->count++; - } else { - direct->addr = new_addr; - } - - /* - * If there's no other ftrace callback on the rec->ip location, - * then it can be changed directly by the architecture. - * If there is another caller, then we just need to change the - * direct caller helper to point to @new_addr. - */ - if (ftrace_rec_count(rec) == 1) { - ret = ftrace_modify_direct_caller(entry, rec, old_addr, new_addr); - } else { - entry->direct = new_addr; - ret = 0; - } - - if (unlikely(ret && new_direct)) { - direct->count++; - list_del_rcu(&new_direct->next); - synchronize_rcu_tasks(); - kfree(new_direct); - ftrace_direct_func_count--; - } - - out_unlock: - mutex_unlock(&ftrace_lock); - mutex_unlock(&direct_mutex); - return ret; -} -EXPORT_SYMBOL_GPL(modify_ftrace_direct); - #define MULTI_FLAGS (FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS) static int check_direct_multi(struct ftrace_ops *ops) From patchwork Tue Mar 21 14:04:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72889 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1801577wrt; Tue, 21 Mar 2023 07:13:24 -0700 (PDT) X-Google-Smtp-Source: AK7set/01pAguG1gTtLxQLnTdA0aQVHfidCvAkABX+lI3oYx9GAc760aQ33fzW80s7flK0zDnrAf X-Received: by 2002:a05:6870:e38b:b0:177:cb10:fb9e with SMTP id x11-20020a056870e38b00b00177cb10fb9emr1344643oad.43.1679408004335; Tue, 21 Mar 2023 07:13:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679408004; cv=none; d=google.com; s=arc-20160816; b=PkLlN6YpNu7SbVO7871lMSypWKpVGKzkYKUnPE31ZeO4fVpnNTEgK3QdcbwXOcKwoF ceyV+fWn9UaBV/3pP+xBuLyvZEOY58/2/PA4qeMda/CcAAMRM1wzu1Exs16cawWSFpH0 mKLPeUrqJA1K86LV6aYrbUtdAZiAWh2Hn5W3hdmCp8thV0dsv6LYipyC+NKGsLk8jB97 gv2n32dI5an0idKdJlgoHnnkkgXlxl779zyeKCkmD94PNlOHJUq+Izam2pJLnvKisVOO AZJMfl7kDTlW9+WWSJw2FkcwjuVggGfpVdFYPYziRCQyWXRMKLTM9m4TrBDduUHc32ET no/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eL/stMoP0oV7dlPY6xNAeWk9gf00tw0yFLNPu+2e0kM=; b=SXUYUXMQNfMB5Y27KYsH1r61uYPawK+g4CBiVmgGfCgIIPFOj8L4gK6V1O+8gpXh14 pXLQdI5tvQm2jWqwm/FwdSJTAqJN6f07kJ5buVI3VVEe+MemGUQXpmmo2ZXtwkn67NWQ qFWo0Vq0kbKqQvQ/0VC1Xp34uLKS8ushNT67/cycAELqbihm8utzFx8SFBR/VdawLFGR X42aJYrd+qbT70388V9NbosQu0OyQycBfwbzDvEoJKnNdGA2dwfQ8xn33xbK5+u1Fe0o n5YHP5skCbv8/haIMALLVVLJTb1u/f+ql068KqkEqFgu8wpQSNkUEKxSvbJjOSrppi+/ 6C1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=AHVl0A8Q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i11-20020a056870344b00b0016de7fb2eecsi10817434oah.335.2023.03.21.07.13.10; Tue, 21 Mar 2023 07:13:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=AHVl0A8Q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230488AbjCUOE6 (ORCPT + 99 others); Tue, 21 Mar 2023 10:04:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231375AbjCUOEp (ORCPT ); Tue, 21 Mar 2023 10:04:45 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71E5D22DF9 for ; Tue, 21 Mar 2023 07:04:42 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id p13-20020a05600c358d00b003ed346d4522so9542724wmq.2 for ; Tue, 21 Mar 2023 07:04:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eL/stMoP0oV7dlPY6xNAeWk9gf00tw0yFLNPu+2e0kM=; b=AHVl0A8QMlRrprODd1h5oIZ3f9ROsEkrzZBorKR8ZrLPOsBjc/TDIHpXn/w4Z7aaXr Jy4AuZ0LkvPHqf86e2rt0mkTXVUq8nEwe5Fdkzbo4t1T3d3owH27updwLu8qTEegG6fM reZXP9KMcxHjbvTYnyx9/MyRqGc8aQVbZKXps= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eL/stMoP0oV7dlPY6xNAeWk9gf00tw0yFLNPu+2e0kM=; b=HUe+WbdNFKufwHNwSkJjME6qgPoOpGwhnuiAp31uRDGFQ6+xUMgavL28+M7KhAT8y7 ZqUz1dacG7fauv5SfzvsLDrbb+KG1UPBWJu8gj9zKCqULRKy9jgvWFgbBq0I+9tFu/g1 qz280CmVdgPffrvMRGsXTzrDJMxk0W0dl+PqBF3CxcTgPz3AvxpPBHNxmhMbbikyimc8 bHq1YdhpfrrPrH0Y92RQA4n/4tmuGUZEIWujpNdPJDZUwuMtsnYYrBIE53gveoUywSVj Wjw3GHn9B96tXbx3xXvxMzooLHdG6LKFc4CyiNkEHIniu4+gbxtLSmP6PIn+xaPVF4E9 cGBQ== X-Gm-Message-State: AO0yUKVw7dSkisv8n9Iue+wGjg7Li7m6WX5n9+DJ4bUx1F5Bx21sM3VK NBv3NVKPFYmyLCEkI4M9/9RDNJTue7Gth65DApk= X-Received: by 2002:a1c:ed15:0:b0:3ee:d7f:6667 with SMTP id l21-20020a1ced15000000b003ee0d7f6667mr2491039wmh.31.1679407480133; Tue, 21 Mar 2023 07:04:40 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:39 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org Subject: [PATCH v2 4/7] ftrace: Rename _ftrace_direct_multi APIs to _ftrace_direct APIs Date: Tue, 21 Mar 2023 15:04:21 +0100 Message-Id: <20230321140424.345218-5-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760986927789213875?= X-GMAIL-MSGID: =?utf-8?q?1760986927789213875?= Now that the original _ftrace_direct APIs are gone, the "_multi" suffixes only add confusion. Signed-off-by: Florent Revest Acked-by: Mark Rutland Tested-by: Mark Rutland --- include/linux/ftrace.h | 20 ++++++------ kernel/bpf/trampoline.c | 12 ++++---- kernel/trace/ftrace.c | 34 ++++++++++----------- kernel/trace/trace_selftest.c | 9 +++--- samples/Kconfig | 2 +- samples/ftrace/ftrace-direct-modify.c | 8 ++--- samples/ftrace/ftrace-direct-multi-modify.c | 8 ++--- samples/ftrace/ftrace-direct-multi.c | 4 +-- samples/ftrace/ftrace-direct-too.c | 6 ++-- samples/ftrace/ftrace-direct.c | 6 ++-- 10 files changed, 55 insertions(+), 54 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 2f400c9f0787..abee60865fc7 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -398,11 +398,11 @@ struct ftrace_func_entry { #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS extern int ftrace_direct_func_count; unsigned long ftrace_find_rec_direct(unsigned long ip); -int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); -int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, - bool free_filters); -int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr); -int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr); +int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr); +int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr, + bool free_filters); +int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr); +int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr); #else struct ftrace_ops; @@ -411,20 +411,20 @@ static inline unsigned long ftrace_find_rec_direct(unsigned long ip) { return 0; } -static inline int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +static inline int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } -static inline int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, - bool free_filters) +static inline int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr, + bool free_filters) { return -ENODEV; } -static inline int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +static inline int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } -static inline int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr) +static inline int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr) { return -ENODEV; } diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c index 88bc23f1e10a..a14d0af534b3 100644 --- a/kernel/bpf/trampoline.c +++ b/kernel/bpf/trampoline.c @@ -45,8 +45,8 @@ static int bpf_tramp_ftrace_ops_func(struct ftrace_ops *ops, enum ftrace_ops_cmd lockdep_assert_held_once(&tr->mutex); /* Instead of updating the trampoline here, we propagate - * -EAGAIN to register_ftrace_direct_multi(). Then we can - * retry register_ftrace_direct_multi() after updating the + * -EAGAIN to register_ftrace_direct(). Then we can + * retry register_ftrace_direct() after updating the * trampoline. */ if ((tr->flags & BPF_TRAMP_F_CALL_ORIG) && @@ -198,7 +198,7 @@ static int unregister_fentry(struct bpf_trampoline *tr, void *old_addr) int ret; if (tr->func.ftrace_managed) - ret = unregister_ftrace_direct_multi(tr->fops, (long)old_addr, false); + ret = unregister_ftrace_direct(tr->fops, (long)old_addr, false); else ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, old_addr, NULL); @@ -215,9 +215,9 @@ static int modify_fentry(struct bpf_trampoline *tr, void *old_addr, void *new_ad if (tr->func.ftrace_managed) { if (lock_direct_mutex) - ret = modify_ftrace_direct_multi(tr->fops, (long)new_addr); + ret = modify_ftrace_direct(tr->fops, (long)new_addr); else - ret = modify_ftrace_direct_multi_nolock(tr->fops, (long)new_addr); + ret = modify_ftrace_direct_nolock(tr->fops, (long)new_addr); } else { ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, old_addr, new_addr); } @@ -243,7 +243,7 @@ static int register_fentry(struct bpf_trampoline *tr, void *new_addr) if (tr->func.ftrace_managed) { ftrace_set_filter_ip(tr->fops, (unsigned long)ip, 0, 1); - ret = register_ftrace_direct_multi(tr->fops, (long)new_addr); + ret = register_ftrace_direct(tr->fops, (long)new_addr); } else { ret = bpf_arch_text_poke(ip, BPF_MOD_CALL, NULL, new_addr); } diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index fca478396d31..33530198d1ca 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5317,7 +5317,7 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long } /** - * register_ftrace_direct_multi - Call a custom trampoline directly + * register_ftrace_direct - Call a custom trampoline directly * for multiple functions registered in @ops * @ops: The address of the struct ftrace_ops object * @addr: The address of the trampoline to call at @ops functions @@ -5338,7 +5338,7 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long * -ENODEV - @ip does not point to a ftrace nop location (or not supported) * -ENOMEM - There was an allocation failure. */ -int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { struct ftrace_hash *hash, *free_hash = NULL; struct ftrace_func_entry *entry, *new; @@ -5396,11 +5396,11 @@ int register_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) } return err; } -EXPORT_SYMBOL_GPL(register_ftrace_direct_multi); +EXPORT_SYMBOL_GPL(register_ftrace_direct); /** - * unregister_ftrace_direct_multi - Remove calls to custom trampoline - * previously registered by register_ftrace_direct_multi for @ops object. + * unregister_ftrace_direct - Remove calls to custom trampoline + * previously registered by register_ftrace_direct for @ops object. * @ops: The address of the struct ftrace_ops object * * This is used to remove a direct calls to @addr from the nop locations @@ -5411,8 +5411,8 @@ EXPORT_SYMBOL_GPL(register_ftrace_direct_multi); * 0 on success * -EINVAL - The @ops object was not properly registered. */ -int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, - bool free_filters) +int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr, + bool free_filters) { struct ftrace_hash *hash = ops->func_hash->filter_hash; int err; @@ -5435,10 +5435,10 @@ int unregister_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr, ftrace_free_filter(ops); return err; } -EXPORT_SYMBOL_GPL(unregister_ftrace_direct_multi); +EXPORT_SYMBOL_GPL(unregister_ftrace_direct); static int -__modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +__modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { struct ftrace_hash *hash; struct ftrace_func_entry *entry, *iter; @@ -5485,7 +5485,7 @@ __modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) } /** - * modify_ftrace_direct_multi_nolock - Modify an existing direct 'multi' call + * modify_ftrace_direct_nolock - Modify an existing direct 'multi' call * to call something else * @ops: The address of the struct ftrace_ops object * @addr: The address of the new trampoline to call at @ops functions @@ -5502,19 +5502,19 @@ __modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) * Returns: zero on success. Non zero on error, which includes: * -EINVAL - The @ops object was not properly registered. */ -int modify_ftrace_direct_multi_nolock(struct ftrace_ops *ops, unsigned long addr) +int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr) { if (check_direct_multi(ops)) return -EINVAL; if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) return -EINVAL; - return __modify_ftrace_direct_multi(ops, addr); + return __modify_ftrace_direct(ops, addr); } -EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi_nolock); +EXPORT_SYMBOL_GPL(modify_ftrace_direct_nolock); /** - * modify_ftrace_direct_multi - Modify an existing direct 'multi' call + * modify_ftrace_direct - Modify an existing direct 'multi' call * to call something else * @ops: The address of the struct ftrace_ops object * @addr: The address of the new trampoline to call at @ops functions @@ -5528,7 +5528,7 @@ EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi_nolock); * Returns: zero on success. Non zero on error, which includes: * -EINVAL - The @ops object was not properly registered. */ -int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) +int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) { int err; @@ -5538,11 +5538,11 @@ int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr) return -EINVAL; mutex_lock(&direct_mutex); - err = __modify_ftrace_direct_multi(ops, addr); + err = __modify_ftrace_direct(ops, addr); mutex_unlock(&direct_mutex); return err; } -EXPORT_SYMBOL_GPL(modify_ftrace_direct_multi); +EXPORT_SYMBOL_GPL(modify_ftrace_direct); #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ /** diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index 9ce80b3ad06d..84cd7ba31d27 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -872,7 +872,8 @@ trace_selftest_startup_function_graph(struct tracer *trace, * and make sure we get graph trace. */ ftrace_set_filter_ip(&direct, (unsigned long)DYN_FTRACE_TEST_NAME, 0, 0); - ret = register_ftrace_direct_multi(&direct, (unsigned long)trace_direct_tramp); + ret = register_ftrace_direct(&direct, + (unsigned long)trace_direct_tramp); if (ret) goto out; @@ -892,9 +893,9 @@ trace_selftest_startup_function_graph(struct tracer *trace, unregister_ftrace_graph(&fgraph_ops); - ret = unregister_ftrace_direct_multi(&direct, - (unsigned long) trace_direct_tramp, - true); + ret = unregister_ftrace_direct(&direct, + (unsigned long) trace_direct_tramp, + true); if (ret) goto out; diff --git a/samples/Kconfig b/samples/Kconfig index 30ef8bd48ba3..fd24daa99f34 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -38,7 +38,7 @@ config SAMPLE_FTRACE_DIRECT that hooks to wake_up_process and prints the parameters. config SAMPLE_FTRACE_DIRECT_MULTI - tristate "Build register_ftrace_direct_multi() example" + tristate "Build register_ftrace_direct() on multiple ips example" depends on DYNAMIC_FTRACE_WITH_DIRECT_CALLS && m depends on HAVE_SAMPLE_FTRACE_DIRECT_MULTI help diff --git a/samples/ftrace/ftrace-direct-modify.c b/samples/ftrace/ftrace-direct-modify.c index f01ac74bac10..25fba66f61c0 100644 --- a/samples/ftrace/ftrace-direct-modify.c +++ b/samples/ftrace/ftrace-direct-modify.c @@ -116,7 +116,7 @@ static int simple_thread(void *arg) if (ret) continue; t ^= 1; - ret = modify_ftrace_direct_multi(&direct, tramps[t]); + ret = modify_ftrace_direct(&direct, tramps[t]); if (!ret) my_tramp = tramps[t]; WARN_ON_ONCE(ret); @@ -132,7 +132,7 @@ static int __init ftrace_direct_init(void) int ret; ftrace_set_filter_ip(&direct, (unsigned long) my_ip, 0, 0); - ret = register_ftrace_direct_multi(&direct, my_tramp); + ret = register_ftrace_direct(&direct, my_tramp); if (!ret) simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn"); @@ -142,12 +142,12 @@ static int __init ftrace_direct_init(void) static void __exit ftrace_direct_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct_multi(&direct, my_tramp, true); + unregister_ftrace_direct(&direct, my_tramp, true); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-multi-modify.c b/samples/ftrace/ftrace-direct-multi-modify.c index 196b43971cb5..f72623899602 100644 --- a/samples/ftrace/ftrace-direct-multi-modify.c +++ b/samples/ftrace/ftrace-direct-multi-modify.c @@ -123,7 +123,7 @@ static int simple_thread(void *arg) if (ret) continue; t ^= 1; - ret = modify_ftrace_direct_multi(&direct, tramps[t]); + ret = modify_ftrace_direct(&direct, tramps[t]); if (!ret) my_tramp = tramps[t]; WARN_ON_ONCE(ret); @@ -141,7 +141,7 @@ static int __init ftrace_direct_multi_init(void) ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); ftrace_set_filter_ip(&direct, (unsigned long) schedule, 0, 0); - ret = register_ftrace_direct_multi(&direct, my_tramp); + ret = register_ftrace_direct(&direct, my_tramp); if (!ret) simple_tsk = kthread_run(simple_thread, NULL, "event-sample-fn"); @@ -151,12 +151,12 @@ static int __init ftrace_direct_multi_init(void) static void __exit ftrace_direct_multi_exit(void) { kthread_stop(simple_tsk); - unregister_ftrace_direct_multi(&direct, my_tramp, true); + unregister_ftrace_direct(&direct, my_tramp, true); } module_init(ftrace_direct_multi_init); module_exit(ftrace_direct_multi_exit); MODULE_AUTHOR("Jiri Olsa"); -MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using modify_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct-multi.c b/samples/ftrace/ftrace-direct-multi.c index ea0e88ee5e43..1547c2c6be02 100644 --- a/samples/ftrace/ftrace-direct-multi.c +++ b/samples/ftrace/ftrace-direct-multi.c @@ -73,12 +73,12 @@ static int __init ftrace_direct_multi_init(void) ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); ftrace_set_filter_ip(&direct, (unsigned long) schedule, 0, 0); - return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + return register_ftrace_direct(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_multi_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long) my_tramp, true); + unregister_ftrace_direct(&direct, (unsigned long) my_tramp, true); } module_init(ftrace_direct_multi_init); diff --git a/samples/ftrace/ftrace-direct-too.c b/samples/ftrace/ftrace-direct-too.c index 05c3585ac15e..f28e7b99840f 100644 --- a/samples/ftrace/ftrace-direct-too.c +++ b/samples/ftrace/ftrace-direct-too.c @@ -76,17 +76,17 @@ static int __init ftrace_direct_init(void) { ftrace_set_filter_ip(&direct, (unsigned long) handle_mm_fault, 0, 0); - return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + return register_ftrace_direct(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp, true); + unregister_ftrace_direct(&direct, (unsigned long)my_tramp, true); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Another example use case of using register_ftrace_direct()"); MODULE_LICENSE("GPL"); diff --git a/samples/ftrace/ftrace-direct.c b/samples/ftrace/ftrace-direct.c index 42ec9e39453b..d81a9473b585 100644 --- a/samples/ftrace/ftrace-direct.c +++ b/samples/ftrace/ftrace-direct.c @@ -69,17 +69,17 @@ static int __init ftrace_direct_init(void) { ftrace_set_filter_ip(&direct, (unsigned long) wake_up_process, 0, 0); - return register_ftrace_direct_multi(&direct, (unsigned long) my_tramp); + return register_ftrace_direct(&direct, (unsigned long) my_tramp); } static void __exit ftrace_direct_exit(void) { - unregister_ftrace_direct_multi(&direct, (unsigned long)my_tramp, true); + unregister_ftrace_direct(&direct, (unsigned long)my_tramp, true); } module_init(ftrace_direct_init); module_exit(ftrace_direct_exit); MODULE_AUTHOR("Steven Rostedt"); -MODULE_DESCRIPTION("Example use case of using register_ftrace_direct_multi()"); +MODULE_DESCRIPTION("Example use case of using register_ftrace_direct()"); MODULE_LICENSE("GPL"); From patchwork Tue Mar 21 14:04:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72890 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1801678wrt; Tue, 21 Mar 2023 07:13:31 -0700 (PDT) X-Google-Smtp-Source: AK7set+mglP6xL4AWXfO7Q+v38LpG8ws2DG31sghFpOyVwoxiT2Qt7k/tUBNd1zh/RZuxLjZBr0h X-Received: by 2002:a17:902:f54b:b0:1a1:b440:377d with SMTP id h11-20020a170902f54b00b001a1b440377dmr3283913plf.2.1679408011239; Tue, 21 Mar 2023 07:13:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679408011; cv=none; d=google.com; s=arc-20160816; b=xIJRwBTQC+M018OaJl6B+hlYdhuMTP/jxjEcYjq3nE5oLviUak0yLsZXbHRqO+tu9a EQWhSFMhQ4ia2Pyps0//ip5gFS9dX9f1sR8aON5wydA2X4mJ2UxaHbjBDcxriC5gQEEI AVrru89WhFjvZqVTmUD+ITGkjF7Ltt+3zhL5juhFmS/3Ow3P46EcCj3DFwPgQWoCX1qR oBPRYdDK5Dc108hIjTnFqP52jlRsPSteyEY0S3e9Vt0Ep9FjAOF+LSMl9AVsi3xgzTOR MFLjHTj36i8HHq0EWOYBjA26qAKX53hUlDGfxks2Z5KuVHlG+N1X212IHbPZk9dAxUoO 230w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=m2fmFB1FVRjrhf8zh2dvknFsm562JmIp8xz3gFT+UVA=; b=iO12taRbQ++8JB911A7qJHrhEn16CPTEZ0hGMBR39k9Tf4siwepwHL0pnYp0w8Nunf CflxdQgmCs64/SieBKscVTe0HJn6y7YvUJWJn2G10ZL3BHFK6xXQyPaApyZqa4LNDxRt /RiUOEDZvS0vOZLMFWR+/RxzQ1zXbQoswHNwFzaPzVjomA0XfgMmFprzsh/UoicU9SUK obJY3uA9UTlgGgttlDSwiIxRUFLZ5X8OAy6dAex6uUfv99asqAflZD0D0n2ti73y3LRx knomat37QteYFliaqVr3awIzwS7JS+2NEu2LXIAmi/p3YtrinXVqhd5QXoPB45IceUBM UKPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="VYFq9Y/i"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m10-20020a170902c44a00b001a198422033si12960641plm.21.2023.03.21.07.13.15; Tue, 21 Mar 2023 07:13:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="VYFq9Y/i"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231426AbjCUOFC (ORCPT + 99 others); Tue, 21 Mar 2023 10:05:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231366AbjCUOEo (ORCPT ); Tue, 21 Mar 2023 10:04:44 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3615321A06 for ; Tue, 21 Mar 2023 07:04:43 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id fm20-20020a05600c0c1400b003ead37e6588so11143396wmb.5 for ; Tue, 21 Mar 2023 07:04:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407481; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m2fmFB1FVRjrhf8zh2dvknFsm562JmIp8xz3gFT+UVA=; b=VYFq9Y/i+3bznsGRQQidOXTXEdh59n3ImD7f48RqiRuS6jw4VQFeV9Fbwg6JlfWcTe QO1ejrsHbmk76zOqsucN8SS14PjoqSzyYKmonbBdC64TObAwdijzLJGlGJXBpsoYghSA uXhrwcxvkOVJayviK6NuPHnfY6AB+TBs5H5Ic= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407481; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m2fmFB1FVRjrhf8zh2dvknFsm562JmIp8xz3gFT+UVA=; b=IOFbsNSnWy/i8NJQ31A5KqN7k8FGlRnAK7d7Fhl5rqc2MqTGsHZLaWiZEVYvwJq7nx 3Fp4/84DTo19Ao1lEZOFJPomPbEdigiLMt/rMRLRO9KoM0Fu+X2gyzUrq2oZrHqRJ1xi sopuLywCaYJQkhQZc5oK1bc+6vzX/7J3hCyemgqC9ZOxUMovoozGjTkvCbT2GAy2Xj/q Pgc7bksvYh0Kwzn3dUaMP1VcBbQdbz9PkPAr845rkChf5y/TpaoYHIGqRe3kOgsPGZzL auOh72FbNuQ1bWII99VYE4hB2mq68LXrmMEsUMqRswgM8KY/66sHA9aEQ6Bexozj5Iez 2T1A== X-Gm-Message-State: AO0yUKVYITuQfMyOlp2zRczoBqzqHzj6zX+RJY15arsDEG17faEDPPxa TFwH2EaLksOFPLK4Xbs/VB6TTMnCPo0jej/4m4c= X-Received: by 2002:a05:600c:2148:b0:3ed:b6ad:54d with SMTP id v8-20020a05600c214800b003edb6ad054dmr2523905wml.18.1679407481466; Tue, 21 Mar 2023 07:04:41 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:41 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org Subject: [PATCH v2 5/7] ftrace: Store direct called addresses in their ops Date: Tue, 21 Mar 2023 15:04:22 +0100 Message-Id: <20230321140424.345218-6-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760986934441794633?= X-GMAIL-MSGID: =?utf-8?q?1760986934441794633?= All direct calls are now registered using the register_ftrace_direct API so each ops can jump to only one direct-called trampoline. By storing the direct called trampoline address directly in the ops we can save one hashmap lookup in the direct call ops and implement arm64 direct calls on top of call ops. Signed-off-by: Florent Revest --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 7 +++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index abee60865fc7..6a532dd6789e 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -321,6 +321,9 @@ struct ftrace_ops { unsigned long trampoline_size; struct list_head list; ftrace_ops_func_t ops_func; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_call; +#endif #endif }; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 33530198d1ca..bf1f857bfe76 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2582,9 +2582,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - unsigned long addr; + unsigned long addr = READ_ONCE(ops->direct_call); - addr = ftrace_find_rec_direct(ip); if (!addr) return; @@ -5380,6 +5379,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) ops->func = call_direct_funcs; ops->flags = MULTI_FLAGS; ops->trampoline = FTRACE_REGS_ADDR; + ops->direct_call = addr; err = register_ftrace_function_nolock(ops); @@ -5454,6 +5454,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) /* Enable the tmp_ops to have the same functions as the direct ops */ ftrace_ops_init(&tmp_ops); tmp_ops.func_hash = ops->func_hash; + tmp_ops.direct_call = addr; err = register_ftrace_function_nolock(&tmp_ops); if (err) @@ -5475,6 +5476,8 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) entry->direct = addr; } } + /* Prevent store tearing if a trampoline concurrently accesses the value */ + WRITE_ONCE(ops->direct_call, addr); mutex_unlock(&ftrace_lock); From patchwork Tue Mar 21 14:04:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72891 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1801804wrt; Tue, 21 Mar 2023 07:13:44 -0700 (PDT) X-Google-Smtp-Source: AK7set/onjlaelw8YhKWFjV6CAWcy3otbRaWNPH4t1W4y/T+KPfrPgjPxqSdMQD63mP/sZygfsmn X-Received: by 2002:a17:903:2154:b0:1a1:85d4:a758 with SMTP id s20-20020a170903215400b001a185d4a758mr1866979ple.12.1679408024110; Tue, 21 Mar 2023 07:13:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679408024; cv=none; d=google.com; s=arc-20160816; b=gQEZMZ5qFotU6cdPc5GP0a/UPxEozsR5P9tdopyj8Llw9IcHHAtYpxgogJy6cyhQ/E /DHk0pt9o4xL/ylS/xoIs0+fNKOKOVnCbyZvMuIETAcNsN9b/H8NIUOznaQkN7myrs6t 0fvhQLYU7PMrq65BfuM+opBp1P5uPDFfqWkAZxm40YHXbc1PMgmpvFRNuYqGdK0NT/P2 ztPz++0g1ci2Kiou5N2K4zPOKZLnrUmfAQj/kCdiPJfzgw7Aon+AK8tHXZLTxpIwTbOZ 78qGFCmDiUcMMNNnpDV0BtIeAovOw5XsJXZxmM7taKLoJmZnnwsUQvWvx8gLPUHZtTux wfOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YIDv0HpfCsRoXPuddKsYpWNPnYmNvE3dFnM8cxo3FIU=; b=loLyiwoqyOE3u9EEHqYIdu2Yp986JuKl0HE7SpxAbRCl6kCPAJZfQEia26l2Emw27S EBa5K+HDQgyJTrZB6X9c6anLvteKRqXciH+HtCsFNxL/cyMgZ67K4EI5ZwatFDnRocf8 1X7lk125B8kPNeML3LWWA1xA/v728DSazjcEWgzb3bVmPR3Cx8k7HW9sk99B2CGzoy2c 9MLITpKvFKx1c3TRX7U7WIU8nnrHnzk809/g2FUL3oIBHymJX8EA1pGO+bP+FxiUiawD jbri7aCNO8vyc+xY2uT6Jj+wHi4zFZ+3z04kSO4DrNaGKpdzgh0L88JezdtzTMSqnc8C KTZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=b2LI4SsG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m3-20020a63fd43000000b00502f49f43easi14306865pgj.338.2023.03.21.07.13.29; Tue, 21 Mar 2023 07:13:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=b2LI4SsG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231434AbjCUOFG (ORCPT + 99 others); Tue, 21 Mar 2023 10:05:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231387AbjCUOEq (ORCPT ); Tue, 21 Mar 2023 10:04:46 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E808221974 for ; Tue, 21 Mar 2023 07:04:44 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id az3-20020a05600c600300b003ed2920d585so11152083wmb.2 for ; Tue, 21 Mar 2023 07:04:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407483; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YIDv0HpfCsRoXPuddKsYpWNPnYmNvE3dFnM8cxo3FIU=; b=b2LI4SsGygoMhWddhT9GHCGiqfGYUGt82i81kxoGuAdMR0Osc8ayZoC6WwLtedJmmd nqdnw277p1/7vV3PSBqQUI/JHUOqBdRXoyRoYSiFKTFhLh7UtKbp3vjCjz/BrX2exXOm lUeIlSGheMnDt/oK6TrjyxUIkaeB4bBptVOV4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407483; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YIDv0HpfCsRoXPuddKsYpWNPnYmNvE3dFnM8cxo3FIU=; b=NmBzy/EkjdbiPwrEl4VpRYKq3DcbRjNzByFT6Nx184Y6f7t9u1MZwBhyfhhCvgKdPc ihdoZgWHZndVROeD3GColjjsJ2XiMpJX5D0f6fhjxAVTMfwRCJpBunN3V29UR1cS7x+E svgkZmwNeQVO7DkOhmLmzLsMALeJZpb6GjkkN7Wr0fFKqbBUe93yX0RftLfQopYjz4hx rWVKj7x65aovg41AMcmUdceYTRGD+gq0F5SXNMgtabXnaa1PdLscmvTyl+Bl62YH+Wck vlcL04mfNmWiDvYPmHyUIKlqK74mv8dlPyhlurC5keTAJqqhVpXJuO+D9bn+Wga63OeL WnfA== X-Gm-Message-State: AO0yUKV+ql4OonLfEClEnQ3rVO5BWdWobwt9SRf7cIfbHhPoXz19rvr5 j2t7o5IGEAkdiEYPZrCUnNpYcRzLSGc3Fs3400s= X-Received: by 2002:a7b:cc8c:0:b0:3ed:8079:27d7 with SMTP id p12-20020a7bcc8c000000b003ed807927d7mr2300416wma.40.1679407482669; Tue, 21 Mar 2023 07:04:42 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:42 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org Subject: [PATCH v2 6/7] ftrace: Make DIRECT_CALLS work WITH_ARGS and !WITH_REGS Date: Tue, 21 Mar 2023 15:04:23 +0100 Message-Id: <20230321140424.345218-7-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760986948315730990?= X-GMAIL-MSGID: =?utf-8?q?1760986948315730990?= Direct called trampolines can be called in two ways: - either from the ftrace callsite. In this case, they do not access any struct ftrace_regs nor pt_regs - Or, if a ftrace ops is also attached, from the end of a ftrace trampoline. In this case, the call_direct_funcs ops is in charge of setting the direct call trampoline's address in a struct ftrace_regs Since: commit 9705bc709604 ("ftrace: pass fregs to arch_ftrace_set_direct_caller()") The later case no longer requires a full pt_regs. It only needs a struct ftrace_regs so DIRECT_CALLS can work with both WITH_ARGS or WITH_REGS. With architectures like arm64 already abandoning WITH_REGS in favor of WITH_ARGS, it's important to have DIRECT_CALLS work WITH_ARGS only. Signed-off-by: Florent Revest Co-developed-by: Mark Rutland Signed-off-by: Mark Rutland --- include/linux/ftrace.h | 6 ++++++ kernel/trace/Kconfig | 2 +- kernel/trace/ftrace.c | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 6a532dd6789e..31f1e1df2af3 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -241,6 +241,12 @@ enum { FTRACE_OPS_FL_DIRECT = BIT(17), }; +#ifndef CONFIG_DYNAMIC_FTRACE_WITH_ARGS +#define FTRACE_OPS_FL_SAVE_ARGS FTRACE_OPS_FL_SAVE_REGS +#else +#define FTRACE_OPS_FL_SAVE_ARGS 0 +#endif + /* * FTRACE_OPS_CMD_* commands allow the ftrace core logic to request changes * to a ftrace_ops. Note, the requests may fail. diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index a856d4a34c67..5b1e7fa41ca8 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -257,7 +257,7 @@ config DYNAMIC_FTRACE_WITH_REGS config DYNAMIC_FTRACE_WITH_DIRECT_CALLS def_bool y - depends on DYNAMIC_FTRACE_WITH_REGS + depends on DYNAMIC_FTRACE_WITH_REGS || DYNAMIC_FTRACE_WITH_ARGS depends on HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS config DYNAMIC_FTRACE_WITH_CALL_OPS diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index bf1f857bfe76..437ae55a9f51 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5287,7 +5287,7 @@ static LIST_HEAD(ftrace_direct_funcs); static int register_ftrace_function_nolock(struct ftrace_ops *ops); -#define MULTI_FLAGS (FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_REGS) +#define MULTI_FLAGS (FTRACE_OPS_FL_DIRECT | FTRACE_OPS_FL_SAVE_ARGS) static int check_direct_multi(struct ftrace_ops *ops) { From patchwork Tue Mar 21 14:04:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 72892 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp1802550wrt; Tue, 21 Mar 2023 07:15:02 -0700 (PDT) X-Google-Smtp-Source: AK7set9f0t5srCFfpIYMRA1cdXTvtWw2F/IkBeX2gbLcf/FSAw/J0NXrkTYIp3YG5qKfp1oTU6RF X-Received: by 2002:a17:902:f9cf:b0:1a0:70e5:280a with SMTP id kz15-20020a170902f9cf00b001a070e5280amr2138766plb.60.1679408101942; Tue, 21 Mar 2023 07:15:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679408101; cv=none; d=google.com; s=arc-20160816; b=iBAiJsOj0a5E/hE5PIFliqZmZQTn8mkVEjVm71T/gu9ZPLkO4+SBuR1nRPW/jaCxfD dX7Gx9U18eYEf6n9nLAeUZv1ZBq9hfxQ8ro0cEb8+up/BE4wkPsEGW7HBTU/SXN6wah6 eIkg2+JdhqSQGeyxMV9tgTSDX62R4qW1c4sbCaMvjCVVPXv7nwjRKu7peDbQdOPi1R8S cedYb5TxecYvlGCurgMyb6i/NFCunxRXgli5c1Nmf1CC93GCtWHEh97b22XdF3XaBtLH U29jcPwb+9lhnyMYPuEUYYRShKNbXp1ME1qIQkzfaKer5Iur90TueVibNuaqebuDYZLv PDmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zMn2X0pVjGcM1Rx/07U3BIPDRphxQJGYTZujBYSKg+w=; b=wtp6uykKE7n6E1iiFHixFl/cN+1eOvJ8diM8PfF++H/tYDjOBURXd4RqTAa9LRLGdT K2ap8XqTr7lFvxYhzYv0c/4aGH3E4QyzIRJU9i4z0cElqXsTeYFNvp82RqlO52fvhKyH FsYZ5xoOWB9CTGh4vgembh7gkI88dVTxHUEN3nGkCyQKKmSwVbNnC7XMrBMiBKM7qUeQ 7kVKP5W8D+KXSVFLLm52tirni/f8TSoiOaMLAEAZpOTg4j3/ahKGHa+bwg1d3vCmmfYC Qi+/GZ3TUNxQ9p2LTHQIAMfnjARkRALBxDTm94WxZOCSjXHlEiAVYZdC9afHmWI4UMBb hfyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HLZ5sQjm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jx8-20020a170903138800b001a1c551bcf9si7775341plb.106.2023.03.21.07.14.46; Tue, 21 Mar 2023 07:15:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HLZ5sQjm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231442AbjCUOFJ (ORCPT + 99 others); Tue, 21 Mar 2023 10:05:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231415AbjCUOEt (ORCPT ); Tue, 21 Mar 2023 10:04:49 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30F4322CA7 for ; Tue, 21 Mar 2023 07:04:46 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id u11-20020a05600c19cb00b003edcc414997so4662551wmq.3 for ; Tue, 21 Mar 2023 07:04:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1679407484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zMn2X0pVjGcM1Rx/07U3BIPDRphxQJGYTZujBYSKg+w=; b=HLZ5sQjmOqrjsnENpiaSjqUEi456nEtduJQ882MMeRNijbPddYhHjR8mqzKm3w9/4f ETn7KJPqPkgdAu2giUIzAR9LjvsmpPzceuWL2W55d6QmZUMAfj2jX/litSDOuQEUDGSI CWO+xbE/oxA1mEiEoT856ysWYIpOe768kHYqA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679407484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zMn2X0pVjGcM1Rx/07U3BIPDRphxQJGYTZujBYSKg+w=; b=Kp0bdlVChTt4qokds1BsfIGIjW3b+N1ShJZODEOVV19keaJ4VMFWfSnpsaUpVCuOpF 9/0pCS/Y7YGfGWgT+8QrurGO4JpPK/rhU6UJwazkHHgBrLWQj4ZufX2MurbLE/EYwO56 t87uXG9l0OplHxOB2rT+/JX/KxXbm5OuMdb/kyaBFKX6mygVjx0dh5iiftB/0+HyxN7y M6GyM/SWspkVcK0W/xDZMy/RaGBLngCjJV4EZfRlxiNvli+73DnSq0E+bAcdRHkMHRlJ k0yTHh2aFnY3l4K5JTacMk1yTw+ZxL3mMRCoKEva2B+ONSFGZvX36mHpARm8iMvgsEmX UJHA== X-Gm-Message-State: AO0yUKWBO5m/4CsQyjxAbO/+krM+c7XcqO5ZsOE6C1kTVmoiy+pAR7HI SLVEr3czFwumtYbnQenTcuFBj/ESYV5V6Xcer2E= X-Received: by 2002:a1c:6a16:0:b0:3ed:a45d:aee9 with SMTP id f22-20020a1c6a16000000b003eda45daee9mr2377902wmc.39.1679407484111; Tue, 21 Mar 2023 07:04:44 -0700 (PDT) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4b8c:8b16:90d:8d9a]) by smtp.gmail.com with ESMTPSA id f12-20020a1cc90c000000b003e20cf0408esm13620896wmb.40.2023.03.21.07.04.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 07:04:43 -0700 (PDT) From: Florent Revest To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, kpsingh@kernel.org, revest@chromium.org, jolsa@kernel.org, Li Huafei , Xu Kuohai Subject: [PATCH v2 7/7] ftrace: selftest: remove broken trace_direct_tramp Date: Tue, 21 Mar 2023 15:04:24 +0100 Message-Id: <20230321140424.345218-8-revest@chromium.org> X-Mailer: git-send-email 2.40.0.rc2.332.ga46443480c-goog In-Reply-To: <20230321140424.345218-1-revest@chromium.org> References: <20230321140424.345218-1-revest@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760987030260557804?= X-GMAIL-MSGID: =?utf-8?q?1760987030260557804?= From: Mark Rutland The ftrace selftest code has a trace_direct_tramp() function which it uses as a direct call trampoline. This happens to work on x86, since the direct call's return address is in the usual place, and can be returned to via a RET, but in general the calling convention for direct calls is different from regular function calls, and requires a trampoline written in assembly. On s390, regular function calls place the return address in %r14, and an ftrace patch-site in an instrumented function places the trampoline's return address (which is within the instrumented function) in %r0, preserving the original %r14 value in-place. As a regular C function will return to the address in %r14, using a C function as the trampoline results in the trampoline returning to the caller of the instrumented function, skipping the body of the instrumented function. Note that the s390 issue is not detcted by the ftrace selftest code, as the instrumented function is trivial, and returning back into the caller happens to be equivalent. On arm64, regular function calls place the return address in x30, and an ftrace patch-site in an instrumented function saves this into r9 and places the trampoline's return address (within the instrumented function) in x30. A regular C function will return to the address in x30, but will not restore x9 into x30. Consequently, using a C function as the trampoline results in returning to the trampoline's return address having corrupted x30, such that when the instrumented function returns, it will return back into itself. To avoid future issues in this area, remove the trace_direct_tramp() function, and require that each architecture with direct calls provides a stub trampoline, named ftrace_stub_direct_tramp. This can be written to handle the architecture's trampoline calling convention, and in future could be used elsewhere (e.g. in the ftrace ops sample, to measure the overhead of direct calls), so we may as well always build it in. Signed-off-by: Mark Rutland Cc: Li Huafei Cc: Xu Kuohai Cc: Steven Rostedt (Google) Cc: Florent Revest Signed-off-by: Florent Revest --- arch/s390/kernel/mcount.S | 5 +++++ arch/x86/kernel/ftrace_32.S | 5 +++++ arch/x86/kernel/ftrace_64.S | 4 ++++ include/linux/ftrace.h | 2 ++ kernel/trace/trace_selftest.c | 12 ++---------- 5 files changed, 18 insertions(+), 10 deletions(-) diff --git a/arch/s390/kernel/mcount.S b/arch/s390/kernel/mcount.S index 43ff91073d2a..6c10da43b538 100644 --- a/arch/s390/kernel/mcount.S +++ b/arch/s390/kernel/mcount.S @@ -32,6 +32,11 @@ ENTRY(ftrace_stub) BR_EX %r14 ENDPROC(ftrace_stub) +SYM_CODE_START(ftrace_stub_direct_tramp) + lgr %r1, %r0 + BR_EX %r1 +SYM_CODE_END(ftrace_stub_direct_tramp) + .macro ftrace_regs_entry, allregs=0 stg %r14,(__SF_GPRS+8*8)(%r15) # save traced function caller diff --git a/arch/x86/kernel/ftrace_32.S b/arch/x86/kernel/ftrace_32.S index a0ed0e4a2c0c..0d9a14528176 100644 --- a/arch/x86/kernel/ftrace_32.S +++ b/arch/x86/kernel/ftrace_32.S @@ -163,6 +163,11 @@ SYM_INNER_LABEL(ftrace_regs_call, SYM_L_GLOBAL) jmp .Lftrace_ret SYM_CODE_END(ftrace_regs_caller) +SYM_FUNC_START(ftrace_stub_direct_tramp) + CALL_DEPTH_ACCOUNT + RET +SYM_FUNC_END(ftrace_stub_direct_tramp) + #ifdef CONFIG_FUNCTION_GRAPH_TRACER SYM_CODE_START(ftrace_graph_caller) pushl %eax diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S index 1265ad519249..8fc77e3e039c 100644 --- a/arch/x86/kernel/ftrace_64.S +++ b/arch/x86/kernel/ftrace_64.S @@ -307,6 +307,10 @@ SYM_INNER_LABEL(ftrace_regs_caller_end, SYM_L_GLOBAL) SYM_FUNC_END(ftrace_regs_caller) STACK_FRAME_NON_STANDARD_FP(ftrace_regs_caller) +SYM_FUNC_START(ftrace_stub_direct_tramp) + CALL_DEPTH_ACCOUNT + RET +SYM_FUNC_END(ftrace_stub_direct_tramp) #else /* ! CONFIG_DYNAMIC_FTRACE */ diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index 31f1e1df2af3..931f3d904529 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -413,6 +413,8 @@ int unregister_ftrace_direct(struct ftrace_ops *ops, unsigned long addr, int modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr); int modify_ftrace_direct_nolock(struct ftrace_ops *ops, unsigned long addr); +void ftrace_stub_direct_tramp(void); + #else struct ftrace_ops; # define ftrace_direct_func_count 0 diff --git a/kernel/trace/trace_selftest.c b/kernel/trace/trace_selftest.c index 84cd7ba31d27..a931d9aaea26 100644 --- a/kernel/trace/trace_selftest.c +++ b/kernel/trace/trace_selftest.c @@ -786,14 +786,6 @@ static struct fgraph_ops fgraph_ops __initdata = { #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS static struct ftrace_ops direct; -#ifndef CALL_DEPTH_ACCOUNT -#define CALL_DEPTH_ACCOUNT "" -#endif - -noinline __noclone static void trace_direct_tramp(void) -{ - asm(CALL_DEPTH_ACCOUNT); -} #endif /* @@ -873,7 +865,7 @@ trace_selftest_startup_function_graph(struct tracer *trace, */ ftrace_set_filter_ip(&direct, (unsigned long)DYN_FTRACE_TEST_NAME, 0, 0); ret = register_ftrace_direct(&direct, - (unsigned long)trace_direct_tramp); + (unsigned long)ftrace_stub_direct_tramp); if (ret) goto out; @@ -894,7 +886,7 @@ trace_selftest_startup_function_graph(struct tracer *trace, unregister_ftrace_graph(&fgraph_ops); ret = unregister_ftrace_direct(&direct, - (unsigned long) trace_direct_tramp, + (unsigned long)ftrace_stub_direct_tramp, true); if (ret) goto out;