From patchwork Fri Feb 17 00:49:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 58309 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp617266wrn; Thu, 16 Feb 2023 16:55:26 -0800 (PST) X-Google-Smtp-Source: AK7set/eAiTDTvKnMdBV9yc7wnrjgfHluZ4LSDRh5wWAsyiKbWw1XNRfjv+Aw9hSvlauVwlzwUam X-Received: by 2002:aa7:d351:0:b0:4ad:714e:2372 with SMTP id m17-20020aa7d351000000b004ad714e2372mr2424558edr.14.1676595326014; Thu, 16 Feb 2023 16:55:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676595325; cv=none; d=google.com; s=arc-20160816; b=S7vl32r8YKpes7Elb6iUhekrmy8eoVrljYJVofkkMfkWBWc38iv9p7y12y/fg9oPZZ ycR3x46HT+kC8sZBxhBNvnfaeCLQPYF+1UUs5+UdxnKIF2ftYFDG6tVFHIhCxsIKiSjZ G4jg/eOv8xTpu8bznZ/YXRXTD7p3xUY5VUaZeiunggxRYKNyh+HLNtnWv8c74G5bDYV0 GXfO6YhjUsE/Hsaaub6qSiMjqzB1N0cxPekkichnLjC3H5IkDHIeVn8igVgryBg+XM6R Cy5YR8UwI/GWuACAXv+KucDYCpogqcaSSuWaT/0dZK+wKLJh9FzKO8O4FOw3N7Pc+vUi BrGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=jpSNA1xA/JIC9LjnhBw5wmkybHxZwsYs26ZbLaIDwao=; b=L58BfrslJISklcgsYkZ8fV/0m/LDNd6u9aSZH3tYNyqdW2y0W7U7f8ovwoPRMcHQra StqFbfRbVMIbhHC7bmTvp9GxFxXrNo7OUo/65P5cuBn2r+HFtyByKxIgUAzVkdyH5rrf UCZnaNukxSETLBhuKSwOOFadweRLac+mZjmxlYv9kw4C+g7jAz1sa8h4gDAJSJkyM1a+ L7MwGd587VnGq+kdUqA1+dBBQnVTgtJy2n8jA3F7J1DlZ0rq/p1cMtU31UqCKLFKjRR3 Lodx+Z9CswxQfXRFiwXud7Gcy2MrRZqioUy4JXFJDtpMWdFBcNT3c7Jd8uWJ7NmoA72B m17A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f16-20020a056402151000b004aab36d2eb8si3482881edw.62.2023.02.16.16.55.02; Thu, 16 Feb 2023 16:55:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229843AbjBQAvB (ORCPT + 99 others); Thu, 16 Feb 2023 19:51:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230157AbjBQAu6 (ORCPT ); Thu, 16 Feb 2023 19:50:58 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9E4C5775C for ; Thu, 16 Feb 2023 16:50:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2F4B0B82ABC for ; Fri, 17 Feb 2023 00:50:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6733C433EF; Fri, 17 Feb 2023 00:50:39 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pSoxS-000qMe-24; Thu, 16 Feb 2023 19:50:38 -0500 Message-ID: <20230217005038.455296375@goodmis.org> User-Agent: quilt/0.66 Date: Thu, 16 Feb 2023 19:49:55 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi Subject: [for-next][PATCH 1/5] tracing/histogram: Dont use strlen to find length of stacktrace variables References: <20230217004954.719324747@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758037620800888282?= X-GMAIL-MSGID: =?utf-8?q?1758037620800888282?= From: Tom Zanussi Because stacktraces are saved in dynamic strings, trace_event_raw_event_synth() uses strlen to determine the length of the stack. Stacktraces may contain 0-bytes, though, in the saved addresses, so the length found and passed to reserve() will be too small. Fix this by using the first unsigned long in the stack variables to store the actual number of elements in the stack and have trace_event_raw_event_synth() use that to determine the length of the stack. Link: https://lkml.kernel.org/r/1ed6906cd9d6477ef2bd8e63c61de20a9ffe64d7.1676063532.git.zanussi@kernel.org Signed-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_hist.c | 12 ++++++++---- kernel/trace/trace_events_synth.c | 7 ++++++- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index 7f3e6ca6f0fa..f21e42ddba69 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -3137,13 +3137,15 @@ static inline void __update_field_vars(struct tracing_map_elt *elt, size = min(val->size, STR_VAR_LEN_MAX); strscpy(str, val_str, size); } else { + char *stack_start = str + sizeof(unsigned long); int e; - e = stack_trace_save((void *)str, + e = stack_trace_save((void *)stack_start, HIST_STACKTRACE_DEPTH, HIST_STACKTRACE_SKIP); if (e < HIST_STACKTRACE_DEPTH - 1) - ((unsigned long *)str)[e] = 0; + ((unsigned long *)stack_start)[e] = 0; + *((unsigned long *)str) = e; } var_val = (u64)(uintptr_t)str; } @@ -5135,13 +5137,15 @@ static void hist_trigger_elt_update(struct hist_trigger_data *hist_data, size = min(hist_field->size, STR_VAR_LEN_MAX); strscpy(str, val_str, size); } else { + char *stack_start = str + sizeof(unsigned long); int e; - e = stack_trace_save((void *)str, + e = stack_trace_save((void *)stack_start, HIST_STACKTRACE_DEPTH, HIST_STACKTRACE_SKIP); if (e < HIST_STACKTRACE_DEPTH - 1) - ((unsigned long *)str)[e] = 0; + ((unsigned long *)stack_start)[e] = 0; + *((unsigned long *)str) = e; } hist_val = (u64)(uintptr_t)str; } diff --git a/kernel/trace/trace_events_synth.c b/kernel/trace/trace_events_synth.c index 306c89e0ce55..70bddb25d9c0 100644 --- a/kernel/trace/trace_events_synth.c +++ b/kernel/trace/trace_events_synth.c @@ -538,7 +538,12 @@ static notrace void trace_event_raw_event_synth(void *__data, val_idx = var_ref_idx[field_pos]; str_val = (char *)(long)var_ref_vals[val_idx]; - len = kern_fetch_store_strlen((unsigned long)str_val); + if (event->dynamic_fields[i]->is_stack) { + len = *((unsigned long *)str_val); + len *= sizeof(unsigned long); + } else { + len = kern_fetch_store_strlen((unsigned long)str_val); + } fields_size += len; } From patchwork Fri Feb 17 00:49:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 58306 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp616800wrn; Thu, 16 Feb 2023 16:53:57 -0800 (PST) X-Google-Smtp-Source: AK7set+T4AJzbNETb4pP1jJGnaVlX3Zw65ajPYxdUHvzD5ksklst1jcHWe/IBBHepOf913cm8DDD X-Received: by 2002:a17:906:fb8c:b0:8b1:7f2a:b6b7 with SMTP id lr12-20020a170906fb8c00b008b17f2ab6b7mr1098636ejb.41.1676595237040; Thu, 16 Feb 2023 16:53:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676595237; cv=none; d=google.com; s=arc-20160816; b=nx1HMJu9svItk+CmZaQNOerQ7lYAfJzMXT+bKBXMMmBpPUiw/mOJWzezALjAeycj3v BGeF/HiQQqCXVUNHelQT5fJeYOwr+swCyXB9MB1N0K/vU81wysl+/rqvF1A0jPpOewSJ hFKlSv6Bs2dPdswSf9FsquTm4gcW7jQZveB4c/nyWbmy9upAY+Gu3FUyDrP3Cr8QW/pA lFtNOsnzpMMEtKF69CWGEBH4DZM4oGupCwcdAIF3+fsx+12WOTUjmNIohzKHdQzLyIh6 8zRq44imMrzd/gk0mX7DGPvL7JFBfpD6RDsQXZIEIVC1mrqn7aloQl+OKFmsn4zT3JpF MI4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=ngSGB2IFZifWjrfTRQAw+S1Tjtbc3dXOxH31Jf+/M1Y=; b=TEbk2RjCIqBNsnLWq/QBL0hkEzcGLFhbtLRyThoMn1j+fZ+5TRVoi7K5FU3EDPy/4n Ypj+MNDwX+5nz4aI0ZYGZNvq+DknD82jNBruXyqHW8xs1/uZUo40r5+Kvb5GZ6BBrSVO C4NihHNUQZk67DXRRNRP/51dK9KfziRno8aGPONRQh5yI419RFIFaXUPtqFDnDOMNT7L yAEDrMhcrKYtWmPS5wvA9eA40h+NklZy4OWEtvaeYUdhJomVbPYlpvQWZWk/WNnqY4YI tWB3KLcpb56VD/YBqN9/JO8sKV7HPAYrUCqazz4bwD9Cks0DofGMg77IvkI6SpiL4Mho JUpw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mx15-20020a1709065a0f00b008b14781b8f3si2995189ejc.124.2023.02.16.16.53.34; Thu, 16 Feb 2023 16:53:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230103AbjBQAuy (ORCPT + 99 others); Thu, 16 Feb 2023 19:50:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229541AbjBQAuw (ORCPT ); Thu, 16 Feb 2023 19:50:52 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0679D54D2B for ; Thu, 16 Feb 2023 16:50:41 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 95F836120F for ; Fri, 17 Feb 2023 00:50:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8990C4339B; Fri, 17 Feb 2023 00:50:39 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pSoxS-000qNB-2j; Thu, 16 Feb 2023 19:50:38 -0500 Message-ID: <20230217005038.662352137@goodmis.org> User-Agent: quilt/0.66 Date: Thu, 16 Feb 2023 19:49:56 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton Subject: [for-next][PATCH 2/5] tracing: Add BUILD_BUG() to make sure stacktrace fits in strings References: <20230217004954.719324747@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758037527572768433?= X-GMAIL-MSGID: =?utf-8?q?1758037527572768433?= From: "Steven Rostedt (Google)" The max string length for a histogram variable is 256 bytes. The max depth of a stacktrace is 16. With 8byte words, that's 16 * 8 = 128. Which can easily fit in the string variable. The histogram stacktrace is being stored in the string value (with the given max length), with the assumption it will fit. To make sure that this is always the case (in the case that the stack trace depth increases), add a BUILD_BUG_ON() to test this. Link: https://lore.kernel.org/linux-trace-kernel/20230214002418.0103b9e765d3e5c374d2aa7d@kernel.org/ Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_hist.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index f21e42ddba69..6cef1def1da5 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -3119,6 +3119,9 @@ static inline void __update_field_vars(struct tracing_map_elt *elt, unsigned int i, j, var_idx; u64 var_val; + /* Make sure stacktrace can fit in the string variable length */ + BUILD_BUG_ON((HIST_STACKTRACE_DEPTH + 1) * sizeof(long) >= STR_VAR_LEN_MAX); + for (i = 0, j = field_var_str_start; i < n_field_vars; i++) { struct field_var *field_var = field_vars[i]; struct hist_field *var = field_var->var; From patchwork Fri Feb 17 00:49:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 58307 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp617229wrn; Thu, 16 Feb 2023 16:55:15 -0800 (PST) X-Google-Smtp-Source: AK7set9Nii3lB5S4uGT7dt5b2I3zMEbdalIfQa2B21I7C9PvrnwQygv+WPyXj2pzXE+8uy7SfaJq X-Received: by 2002:a17:907:c29:b0:8b1:76ca:f228 with SMTP id ga41-20020a1709070c2900b008b176caf228mr3668031ejc.39.1676595315744; Thu, 16 Feb 2023 16:55:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676595315; cv=none; d=google.com; s=arc-20160816; b=ugoiK94VPSqh2PNhm+FEX1QdxuiKRPBtvrmZDn9U+h8TEMHZAYZzhNL5elFIjcQ0zT 3fCWGlxDw9Z+UOayGazf4t6lhFJ+SJqTnUk62t8qSqhBholDp+jle5Ti639Mt1lA/1NB SpTR9UGm+ckn8lPq0MiscsQai7br0QN3oePuy22cAInFsHK/fa7ZMdvmFw9pUzbBoLFn T2KUc2+MIQ8CZzOpNahp3K4hXP5bz5eo5tfWEqOeJdmTp5MpssP1U2EFH9nnRXa2Mxv1 9cHkGYk0tUcPnc1XYmxBaNNw65uONfcwC8DNz8hA6HrufvXKimmWy8H7xNUWRX7+Lf6M ck8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=tgyRYokC6L6gnhDgJ4tEHsA47evSkWS8PbaDsuiW2iQ=; b=H4kncdWt8WfopZxiUf4RIPfVMYUpwlmY24waMeXVN97hBnhEZ+zWWJZmWsCkIZkD3n 1QieMXct8jno6MaHZejY769Xd4CAGka0haLXi3MpBhpjjn71VbLqxYkzmd+fui/SYb6A ERh15wbmBlgcgZPIelbpveTp+btNK6XLGV8TNEcix0nu6o0dDWMGGOlxc/IXPQiVnUjP Ru2R9128zLoZ9G4aYtCiwffFdjfhMy/V8lqUlrtirdWuDnKm5/MB7zjxx0wM8nz9Ppip 1ZZrt+w09qVUVTxeTy/Hm7J1O6BeLzfiRlumQSpmaAnvZf5W+FXMwRge07NgzAAhpeJn K+Sw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fs35-20020a170907602300b008b14ec686c8si2997389ejc.913.2023.02.16.16.54.52; Thu, 16 Feb 2023 16:55:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230255AbjBQAvI (ORCPT + 99 others); Thu, 16 Feb 2023 19:51:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230207AbjBQAu7 (ORCPT ); Thu, 16 Feb 2023 19:50:59 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD306582AC for ; Thu, 16 Feb 2023 16:50:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5ACE6B82AC1 for ; Fri, 17 Feb 2023 00:50:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC6AAC4339E; Fri, 17 Feb 2023 00:50:39 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pSoxT-000qNi-0A; Thu, 16 Feb 2023 19:50:39 -0500 Message-ID: <20230217005038.870067215@goodmis.org> User-Agent: quilt/0.66 Date: Thu, 16 Feb 2023 19:49:57 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi Subject: [for-next][PATCH 3/5] tracing/histogram: Fix a few problems with stacktrace variable printing References: <20230217004954.719324747@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758037610075154393?= X-GMAIL-MSGID: =?utf-8?q?1758037610075154393?= From: Tom Zanussi Currently, there are a few problems when printing hist triggers and trace output when using stacktrace variables. This fixes the problems seen below: # echo 'hist:keys=delta.buckets=100,stack.stacktrace:sort=delta' > /sys/kernel/debug/tracing/events/synthetic/block_lat/trigger # cat /sys/kernel/debug/tracing/events/synthetic/block_lat/trigger hist:keys=delta.buckets=100,stacktrace:vals=hitcount:sort=delta.buckets=100:size=2048 [active] # echo 'hist:keys=next_pid:ts=common_timestamp.usecs,st=stacktrace if prev_state == 2' >> /sys/kernel/debug/tracing/events/sched/sched_switch/trigger # cat /sys/kernel/debug/tracing/events/sched/sched_switch/trigger hist:keys=next_pid:vals=hitcount:ts=common_timestamp.usecs,st=stacktrace.stacktrace:sort=hitcount:size=2048:clock=global if prev_state == 2 [active] and also in the trace output (should be stack.stacktrace): { delta: ~ 100-199, stacktrace __schedule+0xa19/0x1520 Link: https://lkml.kernel.org/r/60bebd4e546728e012a7a2bcbf58716d48ba6edb.1676063532.git.zanussi@kernel.org Signed-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_hist.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index 6cef1def1da5..c4f1fe985f6f 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -1356,9 +1356,12 @@ static const char *hist_field_name(struct hist_field *field, field_name = field->name; } else if (field->flags & HIST_FIELD_FL_TIMESTAMP) field_name = "common_timestamp"; - else if (field->flags & HIST_FIELD_FL_STACKTRACE) - field_name = "stacktrace"; - else if (field->flags & HIST_FIELD_FL_HITCOUNT) + else if (field->flags & HIST_FIELD_FL_STACKTRACE) { + if (field->field) + field_name = field->field->name; + else + field_name = "stacktrace"; + } else if (field->flags & HIST_FIELD_FL_HITCOUNT) field_name = "hitcount"; if (field_name == NULL) @@ -5339,7 +5342,10 @@ static void hist_trigger_print_key(struct seq_file *m, seq_printf(m, "%s: %-30s[%3llu]", field_name, syscall_name, uval); } else if (key_field->flags & HIST_FIELD_FL_STACKTRACE) { - seq_puts(m, "stacktrace:\n"); + if (key_field->field) + seq_printf(m, "%s.stacktrace", key_field->field->name); + else + seq_puts(m, "stacktrace:\n"); hist_trigger_stacktrace_print(m, key + key_field->offset, HIST_STACKTRACE_DEPTH); @@ -5884,7 +5890,8 @@ static void hist_field_print(struct seq_file *m, struct hist_field *hist_field) if (hist_field->flags) { if (!(hist_field->flags & HIST_FIELD_FL_VAR_REF) && - !(hist_field->flags & HIST_FIELD_FL_EXPR)) { + !(hist_field->flags & HIST_FIELD_FL_EXPR) && + !(hist_field->flags & HIST_FIELD_FL_STACKTRACE)) { const char *flags = get_hist_field_flags(hist_field); if (flags) @@ -5917,9 +5924,12 @@ static int event_hist_trigger_print(struct seq_file *m, if (i > hist_data->n_vals) seq_puts(m, ","); - if (field->flags & HIST_FIELD_FL_STACKTRACE) - seq_puts(m, "stacktrace"); - else + if (field->flags & HIST_FIELD_FL_STACKTRACE) { + if (field->field) + seq_printf(m, "%s.stacktrace", field->field->name); + else + seq_puts(m, "stacktrace"); + } else hist_field_print(m, field); } From patchwork Fri Feb 17 00:49:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 58310 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp617459wrn; Thu, 16 Feb 2023 16:56:09 -0800 (PST) X-Google-Smtp-Source: AK7set+YZyu5DUaaDD+0XUNRglOS9n0b8L4Ou6URY6jXOYyCNCvRTMjbpBo4U61AdUvdKcmhDSOK X-Received: by 2002:a17:906:339b:b0:888:456a:cb76 with SMTP id v27-20020a170906339b00b00888456acb76mr8577710eja.70.1676595369186; Thu, 16 Feb 2023 16:56:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676595369; cv=none; d=google.com; s=arc-20160816; b=f+02rWI1/die+OOuZdy1ogNgnDrgMA9KTNSlDsEIJ8mywHFJCLx840lcqt43xD1v08 Zbcsv4u0NO/VdO1shuZsIcvd8LlmFPSru0wFyXpj/qA8zmwyCIMQlcnpk5s7LECUc5OA ZV4FATpE//6uOBV1m1l54XHAu0W28Q1rXi0yaN4cUTKvmv/ycZ9oElT1GaVnjXhQ41PA bAi5+m9ff2nra/Q5mm+eEfJ+PMXG9lnSrfYIz9KImoKq3dK1WKZKWWkOBR0xz01BvyA4 nWkZrgPe/JRMuCO/mj1iTj9UQPhWdIRPW6/5NNQAKVyDtNV2M/mYoRvw+S4ty0GhbXbH 0fbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=N6YXI313MA37Rj6/gesg1Gye76mSSyzQNfsAFwbsUHI=; b=bYoUzU0kujfUSw9i9Rjd6b62xaoTb89z6V4GdgUGf14sQB4q8u1f7Dtu6j+jHfhbPe oayUa/WXEYFSY2bkMSyE6dISzjCbYcBQkhVQ8PD34/wb8fcNaQAtMzn08yXCSWoD+4vw cyPLpvkxKMabLS0rbVXOkbxODmsOmkgQKxP5ChMbpjr1YpGUCMkkVK8x5dirG0opY0lj mdVH7i+zgwtbwgClHLGx27Gy41uWRIQUZZ858Vd7MCJoG/WCcYqQlG1zbEimQSGYYIo0 8s9AWnedkzB0/HpWYqbiGPWwO3R5N/ewnH2J9/+iZKPBKJFv51UylbVHEsVYMWTp1fBl cQKw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c3-20020aa7c983000000b004acb5db1ff3si3644676edt.576.2023.02.16.16.55.45; Thu, 16 Feb 2023 16:56:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbjBQAvK (ORCPT + 99 others); Thu, 16 Feb 2023 19:51:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230192AbjBQAu7 (ORCPT ); Thu, 16 Feb 2023 19:50:59 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7D7E5776A for ; Thu, 16 Feb 2023 16:50:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 72610B82ABF for ; Fri, 17 Feb 2023 00:50:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32C6AC433AC; Fri, 17 Feb 2023 00:50:40 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pSoxT-000qOJ-0p; Thu, 16 Feb 2023 19:50:39 -0500 Message-ID: <20230217005039.070788344@goodmis.org> User-Agent: quilt/0.66 Date: Thu, 16 Feb 2023 19:49:58 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi Subject: [for-next][PATCH 4/5] tracing/histogram: Fix stacktrace key References: <20230217004954.719324747@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758037666064127854?= X-GMAIL-MSGID: =?utf-8?q?1758037666064127854?= From: Tom Zanussi The current code will always use the current stacktrace as a key even if a stacktrace contained in a specific event field was specified. For example, we expect to use the 'unsigned long[] stack' field in the below event in the histogram: # echo 's:block_lat pid_t pid; u64 delta; unsigned long[] stack;' > /sys/kernel/debug/tracing/dynamic_events # echo 'hist:keys=delta.buckets=100,stack.stacktrace:sort=delta' > /sys/kernel/debug/tracing/events/synthetic/block_lat/trigger But in fact, when we type out the trigger, we see that it's using the plain old global 'stacktrace' as the key, which is just the stacktrace when the event was hit and not the stacktrace contained in the event, which is what we want: # cat /sys/kernel/debug/tracing/events/synthetic/block_lat/trigger hist:keys=delta.buckets=100,stacktrace:vals=hitcount:sort=delta.buckets=100:size=2048 [active] And in fact, there's no code to actually retrieve it from the event, so we need to add HIST_FIELD_FN_STACK and hist_field_stack() to get it and hook it into the trigger code. For now, since the stack is just using dynamic strings, this could just use the dynamic string function, but it seems cleaner to have a dedicated function an be able to tweak independently as necessary. Link: https://lkml.kernel.org/r/11aa614c82976adbfa4ea763dbe885b5fb01d59c.1676063532.git.zanussi@kernel.org Signed-off-by: Tom Zanussi [ Fixed 32bit build warning reported by kernel test robot ] Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_hist.c | 34 +++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index c4f1fe985f6f..89877a18f933 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -135,6 +135,7 @@ enum hist_field_fn { HIST_FIELD_FN_DIV_NOT_POWER2, HIST_FIELD_FN_DIV_MULT_SHIFT, HIST_FIELD_FN_EXECNAME, + HIST_FIELD_FN_STACK, }; /* @@ -1982,7 +1983,10 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, } if (flags & HIST_FIELD_FL_STACKTRACE) { - hist_field->fn_num = HIST_FIELD_FN_NOP; + if (field) + hist_field->fn_num = HIST_FIELD_FN_STACK; + else + hist_field->fn_num = HIST_FIELD_FN_NOP; hist_field->size = HIST_STACKTRACE_SIZE; hist_field->type = kstrdup_const("unsigned long[]", GFP_KERNEL); if (!hist_field->type) @@ -4274,6 +4278,19 @@ static u64 hist_field_execname(struct hist_field *hist_field, return (u64)(unsigned long)(elt_data->comm); } +static u64 hist_field_stack(struct hist_field *hist_field, + struct tracing_map_elt *elt, + struct trace_buffer *buffer, + struct ring_buffer_event *rbe, + void *event) +{ + u32 str_item = *(u32 *)(event + hist_field->field->offset); + int str_loc = str_item & 0xffff; + char *addr = (char *)(event + str_loc); + + return (u64)(unsigned long)addr; +} + static u64 hist_fn_call(struct hist_field *hist_field, struct tracing_map_elt *elt, struct trace_buffer *buffer, @@ -4337,6 +4354,8 @@ static u64 hist_fn_call(struct hist_field *hist_field, return div_by_mult_and_shift(hist_field, elt, buffer, rbe, event); case HIST_FIELD_FN_EXECNAME: return hist_field_execname(hist_field, elt, buffer, rbe, event); + case HIST_FIELD_FN_STACK: + return hist_field_stack(hist_field, elt, buffer, rbe, event); default: return 0; } @@ -5238,8 +5257,17 @@ static void event_hist_trigger(struct event_trigger_data *data, if (key_field->flags & HIST_FIELD_FL_STACKTRACE) { memset(entries, 0, HIST_STACKTRACE_SIZE); - stack_trace_save(entries, HIST_STACKTRACE_DEPTH, - HIST_STACKTRACE_SKIP); + if (key_field->field) { + unsigned long *stack, n_entries; + + field_contents = hist_fn_call(key_field, elt, buffer, rbe, rec); + stack = (unsigned long *)(long)field_contents; + n_entries = *stack; + memcpy(entries, ++stack, n_entries * sizeof(unsigned long)); + } else { + stack_trace_save(entries, HIST_STACKTRACE_DEPTH, + HIST_STACKTRACE_SKIP); + } key = entries; } else { field_contents = hist_fn_call(key_field, elt, buffer, rbe, rec); From patchwork Fri Feb 17 00:49:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 58308 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp617249wrn; Thu, 16 Feb 2023 16:55:21 -0800 (PST) X-Google-Smtp-Source: AK7set81Wqauy3Uz+5FIBFSl+QtYY8Ed0GI5uWIWT947XrCCBSESOcXAOEXkFaoFGDecmH/ijFBq X-Received: by 2002:a17:906:eb8e:b0:881:4d98:fe2e with SMTP id mh14-20020a170906eb8e00b008814d98fe2emr7500953ejb.29.1676595321321; Thu, 16 Feb 2023 16:55:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676595321; cv=none; d=google.com; s=arc-20160816; b=GvZoeiu3flnn2gXQsvNwak7uEE6p5K3VMio36KslEkTiGKdWGRfwSwlfSPOyHD/UsK dL4P1kHeYtRaY4b9lvTJ3rMNgIra6T+38k1kB6Gs+0MLUPFhWbpdoe1aUkeVQ111vQfU HEOBPGoUszKoT11WcPc2n75KWtBnT7/jWQ2z5A8U8HC02uM/VK0ur9lsX4xMTPUmsaEn ctX/CF2aY8e+pM3xxFnmfGOCNpwNNyVZulSBr40Opfwf8Ke9iuR26ku0G8jdLgL9w3Zz hlgtH0ez1/YaAJK+pg/U3hb64yJrqLVwdr1wfSXOkHVIOmeVr4Ih808DNms7MUdgDx8q 3IGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=X9rlD6b1IME6ICj0PyiqoAmqJIxVqqZOUEHvmpigP9A=; b=iXb04FTJGkHJqLkO9s4JGOrtk3KWLoMBihYDKbqdxF4oej2De3kV/dhhLPrlxpMqxM bL6Y0Skk4RHQMPEcVVy/tQ/ENPgI1rWqa9tH5gNMkZ3ZbggWJVlmOxY/eAJkq9n66XND SCu7mFAtXt1/TNY+kGVOlVmtFXJ5hSIMp7KQCkbC3eXUs2IC/w6omoSWhxpN3zGvdu0J m3pQes6R/H3yH6rxyiz8yXNm3Ito8+O8ec0fPCF4meys8LaxUs4H3SZvM9+ZcPRZ838c aPqzWDQDwMdQZU0ZgG3BudE4Qv8fVaiv+fGRhuPCmX4Bq0HUKH+CbXp1Qw2VZ2OO2FKI 8nRQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fs35-20020a170907602300b008aac1899a54si3042910ejc.354.2023.02.16.16.54.58; Thu, 16 Feb 2023 16:55:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230154AbjBQAu6 (ORCPT + 99 others); Thu, 16 Feb 2023 19:50:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230092AbjBQAuz (ORCPT ); Thu, 16 Feb 2023 19:50:55 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B86B55E7D for ; Thu, 16 Feb 2023 16:50:41 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2E1F460C7A for ; Fri, 17 Feb 2023 00:50:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 78DC3C433AF; Fri, 17 Feb 2023 00:50:40 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pSoxT-000qOr-1V; Thu, 16 Feb 2023 19:50:39 -0500 Message-ID: <20230217005039.277248689@goodmis.org> User-Agent: quilt/0.66 Date: Thu, 16 Feb 2023 19:49:59 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi Subject: [for-next][PATCH 5/5] tracing/histogram: Fix stacktrace histogram Documententation References: <20230217004954.719324747@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758037615453467062?= X-GMAIL-MSGID: =?utf-8?q?1758037615453467062?= From: Tom Zanussi Fix a small problem with the histogram specification in the Documentation, and change the example to show output using a stacktrace field rather than the global stacktrace. Link: https://lkml.kernel.org/r/f75f807dd4998249e513515f703a2ff7407605f4.1676063532.git.zanussi@kernel.org Signed-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/histogram.rst | 156 +++++++++++++++++------------- 1 file changed, 90 insertions(+), 66 deletions(-) diff --git a/Documentation/trace/histogram.rst b/Documentation/trace/histogram.rst index 8e95295e39b6..c5cfc6e7baea 100644 --- a/Documentation/trace/histogram.rst +++ b/Documentation/trace/histogram.rst @@ -1993,82 +1993,106 @@ uninterruptible state:: A synthetic event that has a stacktrace field may use it as a key in histogram:: - # echo 'hist:delta.buckets=100,stack.stacktrace:sort=delta' > events/synthetic/block_lat/trigger + # echo 'hist:keys=delta.buckets=100,stack.stacktrace:sort=delta' > events/synthetic/block_lat/trigger # cat events/synthetic/block_lat/hist # event histogram # - # trigger info: hist:keys=delta.buckets=100,stacktrace:vals=hitcount:sort=delta.buckets=100:size=2048 [active] + # trigger info: hist:keys=delta.buckets=100,stack.stacktrace:vals=hitcount:sort=delta.buckets=100:size=2048 [active] # - - { delta: ~ 0-99, stacktrace: - event_hist_trigger+0x464/0x480 - event_triggers_call+0x52/0xe0 - trace_event_buffer_commit+0x193/0x250 - trace_event_raw_event_sched_switch+0xfc/0x150 - __traceiter_sched_switch+0x41/0x60 - __schedule+0x448/0x7b0 - schedule_idle+0x26/0x40 - cpu_startup_entry+0x19/0x20 - start_secondary+0xed/0xf0 - secondary_startup_64_no_verify+0xe0/0xeb - } hitcount: 6 - { delta: ~ 0-99, stacktrace: - event_hist_trigger+0x464/0x480 - event_triggers_call+0x52/0xe0 - trace_event_buffer_commit+0x193/0x250 - trace_event_raw_event_sched_switch+0xfc/0x150 - __traceiter_sched_switch+0x41/0x60 - __schedule+0x448/0x7b0 - schedule_idle+0x26/0x40 - cpu_startup_entry+0x19/0x20 - __pfx_kernel_init+0x0/0x10 - arch_call_rest_init+0xa/0x24 - start_kernel+0x964/0x98d - secondary_startup_64_no_verify+0xe0/0xeb + { delta: ~ 0-99, stack.stacktrace __schedule+0xa19/0x1520 + schedule+0x6b/0x110 + io_schedule+0x46/0x80 + bit_wait_io+0x11/0x80 + __wait_on_bit+0x4e/0x120 + out_of_line_wait_on_bit+0x8d/0xb0 + __wait_on_buffer+0x33/0x40 + jbd2_journal_commit_transaction+0x155a/0x19b0 + kjournald2+0xab/0x270 + kthread+0xfa/0x130 + ret_from_fork+0x29/0x50 + } hitcount: 1 + { delta: ~ 0-99, stack.stacktrace __schedule+0xa19/0x1520 + schedule+0x6b/0x110 + io_schedule+0x46/0x80 + rq_qos_wait+0xd0/0x170 + wbt_wait+0x9e/0xf0 + __rq_qos_throttle+0x25/0x40 + blk_mq_submit_bio+0x2c3/0x5b0 + __submit_bio+0xff/0x190 + submit_bio_noacct_nocheck+0x25b/0x2b0 + submit_bio_noacct+0x20b/0x600 + submit_bio+0x28/0x90 + ext4_bio_write_page+0x1e0/0x8c0 + mpage_submit_page+0x60/0x80 + mpage_process_page_bufs+0x16c/0x180 + mpage_prepare_extent_to_map+0x23f/0x530 + } hitcount: 1 + { delta: ~ 0-99, stack.stacktrace __schedule+0xa19/0x1520 + schedule+0x6b/0x110 + schedule_hrtimeout_range_clock+0x97/0x110 + schedule_hrtimeout_range+0x13/0x20 + usleep_range_state+0x65/0x90 + __intel_wait_for_register+0x1c1/0x230 [i915] + intel_psr_wait_for_idle_locked+0x171/0x2a0 [i915] + intel_pipe_update_start+0x169/0x360 [i915] + intel_update_crtc+0x112/0x490 [i915] + skl_commit_modeset_enables+0x199/0x600 [i915] + intel_atomic_commit_tail+0x7c4/0x1080 [i915] + intel_atomic_commit_work+0x12/0x20 [i915] + process_one_work+0x21c/0x3f0 + worker_thread+0x50/0x3e0 + kthread+0xfa/0x130 } hitcount: 3 - { delta: ~ 0-99, stacktrace: - event_hist_trigger+0x464/0x480 - event_triggers_call+0x52/0xe0 - trace_event_buffer_commit+0x193/0x250 - trace_event_raw_event_sched_switch+0xfc/0x150 - __traceiter_sched_switch+0x41/0x60 - __schedule+0x448/0x7b0 - schedule+0x5a/0xb0 - worker_thread+0xaf/0x380 - kthread+0xe9/0x110 - ret_from_fork+0x2c/0x50 + { delta: ~ 0-99, stack.stacktrace __schedule+0xa19/0x1520 + schedule+0x6b/0x110 + schedule_timeout+0x11e/0x160 + __wait_for_common+0x8f/0x190 + wait_for_completion+0x24/0x30 + __flush_work.isra.0+0x1cc/0x360 + flush_work+0xe/0x20 + drm_mode_rmfb+0x18b/0x1d0 [drm] + drm_mode_rmfb_ioctl+0x10/0x20 [drm] + drm_ioctl_kernel+0xb8/0x150 [drm] + drm_ioctl+0x243/0x560 [drm] + __x64_sys_ioctl+0x92/0xd0 + do_syscall_64+0x59/0x90 + entry_SYSCALL_64_after_hwframe+0x72/0xdc } hitcount: 1 - { delta: ~ 100-199, stacktrace: - event_hist_trigger+0x464/0x480 - event_triggers_call+0x52/0xe0 - trace_event_buffer_commit+0x193/0x250 - trace_event_raw_event_sched_switch+0xfc/0x150 - __traceiter_sched_switch+0x41/0x60 - __schedule+0x448/0x7b0 - schedule_idle+0x26/0x40 - cpu_startup_entry+0x19/0x20 - start_secondary+0xed/0xf0 - secondary_startup_64_no_verify+0xe0/0xeb - } hitcount: 15 - [..] - { delta: ~ 8500-8599, stacktrace: - event_hist_trigger+0x464/0x480 - event_triggers_call+0x52/0xe0 - trace_event_buffer_commit+0x193/0x250 - trace_event_raw_event_sched_switch+0xfc/0x150 - __traceiter_sched_switch+0x41/0x60 - __schedule+0x448/0x7b0 - schedule_idle+0x26/0x40 - cpu_startup_entry+0x19/0x20 - start_secondary+0xed/0xf0 - secondary_startup_64_no_verify+0xe0/0xeb + { delta: ~ 0-99, stack.stacktrace __schedule+0xa19/0x1520 + schedule+0x6b/0x110 + schedule_timeout+0x87/0x160 + __wait_for_common+0x8f/0x190 + wait_for_completion_timeout+0x1d/0x30 + drm_atomic_helper_wait_for_flip_done+0x57/0x90 [drm_kms_helper] + intel_atomic_commit_tail+0x8ce/0x1080 [i915] + intel_atomic_commit_work+0x12/0x20 [i915] + process_one_work+0x21c/0x3f0 + worker_thread+0x50/0x3e0 + kthread+0xfa/0x130 + ret_from_fork+0x29/0x50 + } hitcount: 1 + { delta: ~ 100-199, stack.stacktrace __schedule+0xa19/0x1520 + schedule+0x6b/0x110 + schedule_hrtimeout_range_clock+0x97/0x110 + schedule_hrtimeout_range+0x13/0x20 + usleep_range_state+0x65/0x90 + pci_set_low_power_state+0x17f/0x1f0 + pci_set_power_state+0x49/0x250 + pci_finish_runtime_suspend+0x4a/0x90 + pci_pm_runtime_suspend+0xcb/0x1b0 + __rpm_callback+0x48/0x120 + rpm_callback+0x67/0x70 + rpm_suspend+0x167/0x780 + rpm_idle+0x25a/0x380 + pm_runtime_work+0x93/0xc0 + process_one_work+0x21c/0x3f0 } hitcount: 1 Totals: - Hits: 89 - Entries: 11 - Dropped: 0 + Hits: 10 + Entries: 7 + Dropped: 0 2.2.3 Hist trigger 'handlers' and 'actions' -------------------------------------------