From patchwork Tue Jan 24 18:16:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 47805 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2304269wrn; Tue, 24 Jan 2023 10:22:34 -0800 (PST) X-Google-Smtp-Source: AMrXdXvvbpieDLLRW82QLCQ7ye1Ue86uIJN4dyWjHXvV54SJkDe1iIbdkQxzjIjMd55Ohzxd05Y1 X-Received: by 2002:a17:907:7da0:b0:86d:67b0:6292 with SMTP id oz32-20020a1709077da000b0086d67b06292mr44417557ejc.73.1674584553886; Tue, 24 Jan 2023 10:22:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674584553; cv=none; d=google.com; s=arc-20160816; b=XtmMJRbQikhv2h821Cqfvv+OzNTWNcb8Xmln+Vm9tGBzZJQCyneadtL7TwsLJKEvrz pG8x0JjbNU3oQREZBQ3Vi8YVYk898TuLjDbjfogfI6jfE0K1iNZzU0v97qBKNmliOaPJ pOHoN3JpacjjIwx8OQbJuUM7mMOMaRsoID6p+DSQLG6RAcTOa75sk3wJofAI7XoQFLCT l3k69Vl3CdKafVDWB1KAH6RoDnHwXSDthakaDsiz3qKAo7AKMLnvlqrFq1RRuQlWrHJh VLEbVS9xJ4yzgKhcy94fZTSraMnZKitvM5CdmBND8ySOhV+mGxz1DR+tQSmB5PrYsDrM nDpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:subject:cc:to:from:date; bh=UHsbFwB/m9pzKyKGVPuL74bzDWGCrqisTTKFUMZbNqY=; b=gwMgceB48DuZXHtjqdKDfHnZmJ6DQuI65t158P0VMg02WXefDMqOYhNE1+u3hHo+Ot 0n7jlMvbDS9iH+1tZlI78YjyJLv8L43DW7fNOp1Bl5yS7v7nDnYWcrkI3mpoRR4wGuTg de2BRQMEYEKyXBknliEDpd0AtRfpuYiN9KHS6akM1b11mp1PSuMf6njmmqBNEx/wIf+k Ize2csBjuz2FFat0acXkcp403m5OqfZvPN911aOSJ01TCHlVFD053awsRxdykrXUfKP9 py+RBukS9HiTm4HgFhrHc5hfskxe2IO61daSfKoIILRsTf4YZpU+u4FWk96A5kVd6EmF EFZw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l16-20020a170907915000b0086689c8fae9si2912321ejs.829.2023.01.24.10.22.09; Tue, 24 Jan 2023 10:22:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233907AbjAXSQM (ORCPT + 99 others); Tue, 24 Jan 2023 13:16:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232517AbjAXSQL (ORCPT ); Tue, 24 Jan 2023 13:16:11 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BDAD4A210; Tue, 24 Jan 2023 10:16:04 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1799E61312; Tue, 24 Jan 2023 18:16:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8F37C433D2; Tue, 24 Jan 2023 18:16:02 +0000 (UTC) Date: Tue, 24 Jan 2023 13:16:01 -0500 From: Steven Rostedt To: LKML , Linux Trace Kernel Cc: Masami Hiramatsu , Mark Rutland Subject: [PATCH v2] ftrace/scripts: Update the instructions for ftrace-bisect.sh Message-ID: <20230124131601.1506b286@gandalf.local.home> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755929173195901849?= X-GMAIL-MSGID: =?utf-8?q?1755929173195901849?= From: "Steven Rostedt (Google)" The instructions for the ftrace-bisect.sh script, which is used to find what function is being traced that is causing a kernel crash, and possibly a triple fault reboot, uses the old method. In 5.1, a new feature was added that let the user write in the index into available_filter_functions that maps to the function a user wants to set in set_ftrace_filter (or set_ftrace_notrace). This takes O(1) to set, as suppose to writing a function name, which takes O(n) (where n is the number of functions in available_filter_functions). The ftrace-bisect.sh requires setting half of the functions in available_filter_functions, which is O(n^2) using the name method to enable and can take several minutes to complete. The number method is O(n) which takes less than a second to complete. Using the number method for any kernel 5.1 and after is the proper way to do the bisect. Update the usage to reflect the new change, as well as using the /sys/kernel/tracing path instead of the obsolete debugfs path. Cc: stable@vger.kernel.org Cc: Masami Hiramatsu Acked-by: Mark Rutland Fixes: f79b3f338564e ("ftrace: Allow enabling of filters via index of available_filter_functions") Signed-off-by: Steven Rostedt (Google) --- Changes since v1: https://lkml.kernel.org/r/20230123112252.022003dd@gandalf.local.home - Slight rewrote the "Note" section. (Mark Rutland) scripts/tracing/ftrace-bisect.sh | 34 ++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/scripts/tracing/ftrace-bisect.sh b/scripts/tracing/ftrace-bisect.sh index 926701162bc8..bb4f59262bbe 100755 --- a/scripts/tracing/ftrace-bisect.sh +++ b/scripts/tracing/ftrace-bisect.sh @@ -12,7 +12,7 @@ # (note, if this is a problem with function_graph tracing, then simply # replace "function" with "function_graph" in the following steps). # -# # cd /sys/kernel/debug/tracing +# # cd /sys/kernel/tracing # # echo schedule > set_ftrace_filter # # echo function > current_tracer # @@ -20,22 +20,40 @@ # # # echo nop > current_tracer # -# # cat available_filter_functions > ~/full-file +# Starting with v5.1 this can be done with numbers, making it much faster: +# +# The old (slow) way, for kernels before v5.1. +# +# [old-way] # cat available_filter_functions > ~/full-file +# +# [old-way] *** Note *** this process will take several minutes to update the +# [old-way] filters. Setting multiple functions is an O(n^2) operation, and we +# [old-way] are dealing with thousands of functions. So go have coffee, talk +# [old-way] with your coworkers, read facebook. And eventually, this operation +# [old-way] will end. +# +# The new way (using numbers) is an O(n) operation, and usually takes less than a second. +# +# seq `wc -l available_filter_functions | cut -d' ' -f1` > ~/full-file +# +# This will create a sequence of numbers that match the functions in +# available_filter_functions, and when echoing in a number into the +# set_ftrace_filter file, it will enable the corresponding function in +# O(1) time. Making enabling all functions O(n) where n is the number of +# functions to enable. +# +# For either the new or old way, the rest of the operations remain the same. +# # # ftrace-bisect ~/full-file ~/test-file ~/non-test-file # # cat ~/test-file > set_ftrace_filter # -# *** Note *** this will take several minutes. Setting multiple functions is -# an O(n^2) operation, and we are dealing with thousands of functions. So go -# have coffee, talk with your coworkers, read facebook. And eventually, this -# operation will end. -# # # echo function > current_tracer # # If it crashes, we know that ~/test-file has a bad function. # # Reboot back to test kernel. # -# # cd /sys/kernel/debug/tracing +# # cd /sys/kernel/tracing # # mv ~/test-file ~/full-file # # If it didn't crash.