From patchwork Wed Jan 25 17:12:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48226 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387405wrn; Wed, 25 Jan 2023 09:14:34 -0800 (PST) X-Google-Smtp-Source: AMrXdXurUJG7+zyyvPSh6DJCJ9VXgl0vTPAM9ggMv69VcDRXgEhPxj+ZH0JRJDIbfe9Q1p3EeiwW X-Received: by 2002:a05:6402:5412:b0:49d:f4de:1988 with SMTP id ev18-20020a056402541200b0049df4de1988mr36697983edb.42.1674666874391; Wed, 25 Jan 2023 09:14:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666874; cv=none; d=google.com; s=arc-20160816; b=wniSJJPSrY9FMuoJHuNSXkXILTAwG5wvLYf0feS6LO1dyZpsxNXnbtutkJBZAnnrTa Oo6WQ6MnpG2qlwdqepwVI8T/mjOBG9togp4LdLeNiwQTOV3Ihs43ZJTeRwz9Eb82yoPV nP0kw6R1GIAYyFpAK0YpeN3Zgs7+E/U9qbKsG0laOVre/siQ3JxQLIiGImffl5k2WlAV coFxOv3JcZmOPsah2eXOZaO7fJue1tD3KVDu1No7hqlDW1TMDxIX690eUq6rXE2e7IMp LuV/9NBJ8rM4ByQwyj5A4O+9bUDOIsSYLCRnCSkr4u3JLO5zG5ZTOqQWfpQsjENWnh5s IYKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=QfHF0ga3AK6HOBbOT1VjnQ/kHDORRb59Pxjdov6tV6o=; b=LGSHZNcrxEZveVaYWUkpwRWseZlrCgf2RxZ5WJMMvvpUQYcs+QDWbNCS9ziuHfRrk2 bv/si106jHL81+ivsvw3u7HR3IHCzXFd78ALkYHZfZNuY48WwfY6ArUoJrOXkbf4CkDr lqC0PmQw9V4X56fUF0YwUKiiwnd3o6T+4gGFibCx69BeX6jZaNFSVPeUlQx2qCwvyDmT owoqCTGOa6BfBofAQwunJpA5aGgg/OzSvWL6gC2bTOBkSZTtgCXJafpCQttMk7thljq0 tKs5zEi3c1KMFlinOOHqqZJZnxmCnqvJczTJw5TMYQJeb+iAvTqOXHFSAyyeuDp65Wqb 2Bfg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i8-20020a0564020f0800b0049e0440d49esi7576835eda.29.2023.01.25.09.14.10; Wed, 25 Jan 2023 09:14:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235331AbjAYRNx (ORCPT + 99 others); Wed, 25 Jan 2023 12:13:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235016AbjAYRNw (ORCPT ); Wed, 25 Jan 2023 12:13:52 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8E6346B8 for ; Wed, 25 Jan 2023 09:13:40 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 35D2061590 for ; Wed, 25 Jan 2023 17:13:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95487C433D2; Wed, 25 Jan 2023 17:13:39 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL8-004Nph-24; Wed, 25 Jan 2023 12:13:38 -0500 Message-ID: <20230125171338.453170548@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:53 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Zheng Yejian , Ross Zwisler Subject: [for-next][PATCH 01/12] tracing: Add a way to filter function addresses to function names References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015492694050411?= X-GMAIL-MSGID: =?utf-8?q?1756015492694050411?= From: "Steven Rostedt (Google)" There's been several times where an event records a function address in its field and I needed to filter on that address for a specific function name. It required looking up the function in kallsyms, finding its size, and doing a compare of "field >= function_start && field < function_end". But this would change from boot to boot and is unreliable in scripts. Also, it is useful to have this at boot up, where the addresses will not be known. For example, on the boot command line: trace_trigger="initcall_finish.traceoff if func.function == acpi_init" To implement this, add a ".function" prefix, that will check that the field is of size long, and the only operations allowed (so far) are "==" and "!=". Link: https://lkml.kernel.org/r/20221219183213.916833763@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Zheng Yejian Reviewed-by: Ross Zwisler Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/events.rst | 12 ++++ kernel/trace/trace_events.c | 2 +- kernel/trace/trace_events_filter.c | 93 +++++++++++++++++++++++++++++- 3 files changed, 105 insertions(+), 2 deletions(-) diff --git a/Documentation/trace/events.rst b/Documentation/trace/events.rst index c47f381d0c00..d0fd5c7220b7 100644 --- a/Documentation/trace/events.rst +++ b/Documentation/trace/events.rst @@ -207,6 +207,18 @@ field name:: As the kernel will have to know how to retrieve the memory that the pointer is at from user space. +You can convert any long type to a function address and search by function name:: + + call_site.function == security_prepare_creds + +The above will filter when the field "call_site" falls on the address within +"security_prepare_creds". That is, it will compare the value of "call_site" and +the filter will return true if it is greater than or equal to the start of +the function "security_prepare_creds" and less than the end of that function. + +The ".function" postfix can only be attached to values of size long, and can only +be compared with "==" or "!=". + 5.2 Setting filters ------------------- diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index 33e0b4f8ebe6..b07931ad97de 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -2822,7 +2822,7 @@ static __init int setup_trace_triggers(char *str) if (!trigger) break; bootup_triggers[i].event = strsep(&trigger, "."); - bootup_triggers[i].trigger = strsep(&trigger, "."); + bootup_triggers[i].trigger = trigger; if (!bootup_triggers[i].trigger) break; } diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c index e095c3b3a50d..1dad64267878 100644 --- a/kernel/trace/trace_events_filter.c +++ b/kernel/trace/trace_events_filter.c @@ -64,6 +64,7 @@ enum filter_pred_fn { FILTER_PRED_FN_PCHAR_USER, FILTER_PRED_FN_PCHAR, FILTER_PRED_FN_CPU, + FILTER_PRED_FN_FUNCTION, FILTER_PRED_FN_, FILTER_PRED_TEST_VISITED, }; @@ -71,6 +72,7 @@ enum filter_pred_fn { struct filter_pred { enum filter_pred_fn fn_num; u64 val; + u64 val2; struct regex regex; unsigned short *ops; struct ftrace_event_field *field; @@ -103,6 +105,7 @@ struct filter_pred { C(INVALID_FILTER, "Meaningless filter expression"), \ C(IP_FIELD_ONLY, "Only 'ip' field is supported for function trace"), \ C(INVALID_VALUE, "Invalid value (did you forget quotes)?"), \ + C(NO_FUNCTION, "Function not found"), \ C(ERRNO, "Error"), \ C(NO_FILTER, "No filter found") @@ -876,6 +879,17 @@ static int filter_pred_comm(struct filter_pred *pred, void *event) return cmp ^ pred->not; } +/* Filter predicate for functions. */ +static int filter_pred_function(struct filter_pred *pred, void *event) +{ + unsigned long *addr = (unsigned long *)(event + pred->offset); + unsigned long start = (unsigned long)pred->val; + unsigned long end = (unsigned long)pred->val2; + int ret = *addr >= start && *addr < end; + + return pred->op == OP_EQ ? ret : !ret; +} + /* * regex_match_foo - Basic regex callbacks * @@ -1335,6 +1349,8 @@ static int filter_pred_fn_call(struct filter_pred *pred, void *event) return filter_pred_pchar(pred, event); case FILTER_PRED_FN_CPU: return filter_pred_cpu(pred, event); + case FILTER_PRED_FN_FUNCTION: + return filter_pred_function(pred, event); case FILTER_PRED_TEST_VISITED: return test_pred_visited_fn(pred, event); default: @@ -1350,8 +1366,13 @@ static int parse_pred(const char *str, void *data, struct trace_event_call *call = data; struct ftrace_event_field *field; struct filter_pred *pred = NULL; + unsigned long offset; + unsigned long size; + unsigned long ip; char num_buf[24]; /* Big enough to hold an address */ char *field_name; + char *name; + bool function = false; bool ustring = false; char q; u64 val; @@ -1393,6 +1414,12 @@ static int parse_pred(const char *str, void *data, i += len; } + /* See if the field is a kernel function name */ + if ((len = str_has_prefix(str + i, ".function"))) { + function = true; + i += len; + } + while (isspace(str[i])) i++; @@ -1423,7 +1450,71 @@ static int parse_pred(const char *str, void *data, pred->offset = field->offset; pred->op = op; - if (ftrace_event_is_function(call)) { + if (function) { + /* The field must be the same size as long */ + if (field->size != sizeof(long)) { + parse_error(pe, FILT_ERR_ILLEGAL_FIELD_OP, pos + i); + goto err_free; + } + + /* Function only works with '==' or '!=' and an unquoted string */ + switch (op) { + case OP_NE: + case OP_EQ: + break; + default: + parse_error(pe, FILT_ERR_INVALID_OP, pos + i); + goto err_free; + } + + if (isdigit(str[i])) { + /* We allow 0xDEADBEEF */ + while (isalnum(str[i])) + i++; + + len = i - s; + /* 0xfeedfacedeadbeef is 18 chars max */ + if (len >= sizeof(num_buf)) { + parse_error(pe, FILT_ERR_OPERAND_TOO_LONG, pos + i); + goto err_free; + } + + strncpy(num_buf, str + s, len); + num_buf[len] = 0; + + ret = kstrtoul(num_buf, 0, &ip); + if (ret) { + parse_error(pe, FILT_ERR_INVALID_VALUE, pos + i); + goto err_free; + } + } else { + s = i; + for (; str[i] && !isspace(str[i]); i++) + ; + + len = i - s; + name = kmemdup_nul(str + s, len, GFP_KERNEL); + if (!name) + goto err_mem; + ip = kallsyms_lookup_name(name); + kfree(name); + if (!ip) { + parse_error(pe, FILT_ERR_NO_FUNCTION, pos + i); + goto err_free; + } + } + + /* Now find the function start and end address */ + if (!kallsyms_lookup_size_offset(ip, &size, &offset)) { + parse_error(pe, FILT_ERR_NO_FUNCTION, pos + i); + goto err_free; + } + + pred->fn_num = FILTER_PRED_FN_FUNCTION; + pred->val = ip - offset; + pred->val2 = pred->val + size; + + } else if (ftrace_event_is_function(call)) { /* * Perf does things different with function events. * It only allows an "ip" field, and expects a string. From patchwork Wed Jan 25 17:12:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48229 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387592wrn; Wed, 25 Jan 2023 09:14:55 -0800 (PST) X-Google-Smtp-Source: AMrXdXsgP1XW92KzlaN3NAE33OtdSR/QZiOF6PttdUAVvV4Upf6Zy87piizXMFhXdR1UYzpFJnrq X-Received: by 2002:a17:907:8a07:b0:7c1:5ee1:4c57 with SMTP id sc7-20020a1709078a0700b007c15ee14c57mr40065726ejc.8.1674666895058; Wed, 25 Jan 2023 09:14:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666895; cv=none; d=google.com; s=arc-20160816; b=ZjcRQoLNDDmBBKYotPb1rhYQrEZbEDblnqnguuZlW1LYRED+kgdh574dqY0wnH6wW9 4ECf9K59zDp/6wM6HG6UjzKp0VRmcJlTNg0xRFMw35odJ26gWLFGtOKWyIDYoGD4al5x xJ5nqXlhn8mNmMu5zmPAfw15K6JzaXFP39h9eadpM4uricnOt7wtRlHbK12TjAy1Ejby JR9M2hiplAaRkq93Zm5b1EoaznUKzz0urzqGczGq+5feSou57G3uTBM2xec+ATvy3Umr Vg45ZXOop4d2Q95//nX7QaHUYsHhxLndGRoTOxGYs82jvNmtcHERNMd5d2cXfWiQES7/ eI6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=Q37Q4NVRknkbxULZToIxRhPhdqLfE6kkiLBWl9KpXS4=; b=cqnmq6aJns7YqQ5W5XbVi3gOEgTPC40ddygyXZeOgSdn/DyRCeK1J3eml18qb4QNAD EhqKrheTQVFd4bkUaepF/puEUdSFoSs9BWay5xiLcjR+xHa01Qbtg6143YoHNesCc+Qj oQIThX0gz01NSchJA1GKQF8bw1SbqHgPCAqkhQHj7m9gboJ6c6tpPCAqq7h2i3tO/sYS vPM0V/nQ8G7obTa9Fim8cJVNk0TZ7XHJ5y08nWsmRcYK0Gdb1f6ftTLj6KPxIJ1K9Cl5 snpqiOpVtaVlEgdUw4seANFTJ2NtziVPryCcqVI4+whfhAYATmPb1ip5aH4zmWHjCfgM xGkw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id iy23-20020a170907819700b0087139670f26si6551905ejc.971.2023.01.25.09.14.30; Wed, 25 Jan 2023 09:14:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236114AbjAYROF (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235395AbjAYRNx (ORCPT ); Wed, 25 Jan 2023 12:13:53 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74781A5C3; Wed, 25 Jan 2023 09:13:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2DEBBB81B4A; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D970EC433A0; Wed, 25 Jan 2023 17:13:39 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL8-004NqF-2j; Wed, 25 Jan 2023 12:13:38 -0500 Message-ID: <20230125171338.659803282@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:54 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Zheng Yejian , linux-kselftest@vger.kernel.org, Ross Zwisler , Shuah Khan Subject: [for-next][PATCH 02/12] tracing/selftests: Add test for event filtering on function name References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015513958271342?= X-GMAIL-MSGID: =?utf-8?q?1756015513958271342?= From: "Steven Rostedt (Google)" With the new filter logic of passing in the name of a function to match an instruction pointer (or the address of the function), add a test to make sure that it is functional. This is also the first test to test plain filtering. The filtering has been tested via the trigger logic, which uses the same code, but there was nothing to test just the event filter, so this test is the first to add such a case. Link: https://lkml.kernel.org/r/20221219183214.075559302@goodmis.org Cc: Andrew Morton Cc: Tom Zanussi Cc: Zheng Yejian Cc: linux-kselftest@vger.kernel.org Suggested-by: Masami Hiramatsu (Google) Reviewed-by: Ross Zwisler Acked-by: Shuah Khan Signed-off-by: Steven Rostedt (Google) --- .../test.d/filter/event-filter-function.tc | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc diff --git a/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc b/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc new file mode 100644 index 000000000000..e2ff3bf4df80 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/filter/event-filter-function.tc @@ -0,0 +1,58 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event filter function - test event filtering on functions +# requires: set_event events/kmem/kmem_cache_free/filter +# flags: instance + +fail() { #msg + echo $1 + exit_fail +} + +echo "Test event filter function name" +echo 0 > tracing_on +echo 0 > events/enable +echo > trace +echo 'call_site.function == exit_mmap' > events/kmem/kmem_cache_free/filter +echo 1 > events/kmem/kmem_cache_free/enable +echo 1 > tracing_on +ls > /dev/null +echo 0 > events/kmem/kmem_cache_free/enable + +hitcnt=`grep kmem_cache_free trace| grep exit_mmap | wc -l` +misscnt=`grep kmem_cache_free trace| grep -v exit_mmap | wc -l` + +if [ $hitcnt -eq 0 ]; then + exit_fail +fi + +if [ $misscnt -gt 0 ]; then + exit_fail +fi + +address=`grep ' exit_mmap$' /proc/kallsyms | cut -d' ' -f1` + +echo "Test event filter function address" +echo 0 > tracing_on +echo 0 > events/enable +echo > trace +echo "call_site.function == 0x$address" > events/kmem/kmem_cache_free/filter +echo 1 > events/kmem/kmem_cache_free/enable +echo 1 > tracing_on +sleep 1 +echo 0 > events/kmem/kmem_cache_free/enable + +hitcnt=`grep kmem_cache_free trace| grep exit_mmap | wc -l` +misscnt=`grep kmem_cache_free trace| grep -v exit_mmap | wc -l` + +if [ $hitcnt -eq 0 ]; then + exit_fail +fi + +if [ $misscnt -gt 0 ]; then + exit_fail +fi + +reset_events_filter + +exit 0 From patchwork Wed Jan 25 17:12:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48235 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp388447wrn; Wed, 25 Jan 2023 09:16:32 -0800 (PST) X-Google-Smtp-Source: AMrXdXuk7B6FTS3ktGyYZ+2ek02m/QoG4BZ5oLb5iIn6iBGpxxrfBlSLfvnUY9bRIJlO7a0ub9pR X-Received: by 2002:a05:6a20:4f30:b0:b6:93b0:f85b with SMTP id gi48-20020a056a204f3000b000b693b0f85bmr34476746pzb.12.1674666991962; Wed, 25 Jan 2023 09:16:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666991; cv=none; d=google.com; s=arc-20160816; b=0540zXwRNYwC81A9wC/7s05FQn7aByzNaPJqyoEC4rXwr+5wvyRFFlfaoHwyrwgiSK ZMzVXyRTZjK846KcLzB9gkJDW6KcSmn0CXexN0JYMygSdwpBC9RYWtEMR1ggmaVyv2A3 vIisNn7+2lI5JiGjq5Cl2OIc1SFYvJE9OKPAVkgPJJfqLaN/1gHFaxvAegtEOCpCczgp pWTjWpQJfr9JC/TIH3lcAwiOi4auphI9oJU4y0aQmpzKgn4xrhjN3BvyL7lt1IN7pfsP QfAspXd1wXTV89lQImLsUOJRff8oGOxFOWcQAYNpx1E/jJ5+HMR3G+NycM+SRnf1r5MD 8EMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=+ynhF1rjyQRDw8sxhrMVeHVIRNC154OHZ2TGvN5PeAg=; b=lO7aZg7D2xMsp26zArcFe/m4+ehkcTdOqj6jsf4N/75E7W9TiAxBneQXyDU/DQLmfs E3MhNar+yS4+h04uQYmTfPu6lYyMV7XEx6aWg5LKB8NEj7N++fYi9IEtWoVMQS++zdG3 SS8QS96Et5Rpfm7hE6BuJkGJBQPJhDlNZcbhjSmZeRaiHzdZrN0vf/rmt9tMYmrypfVt QYU6tB6L/pCzca6JZOjJRGfTt+itckyGahcAmDz2ummTX09G7l5+YIQnd5FIzItPfOYu Fo3tTdc7JwKiQZZoZfUszoPupwXSlO8zeFq/Qq1iQuAMuw0SvqQbAcTYoxEEf7tkBHg2 a9gQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a15-20020aa78e8f000000b0058b98f431f3si6356492pfr.51.2023.01.25.09.16.19; Wed, 25 Jan 2023 09:16:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236169AbjAYROS (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235610AbjAYRNz (ORCPT ); Wed, 25 Jan 2023 12:13:55 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1D31B45D for ; Wed, 25 Jan 2023 09:13:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6BD16B81B4D for ; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0A2E7C433A4; Wed, 25 Jan 2023 17:13:40 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL9-004Nqn-0B; Wed, 25 Jan 2023 12:13:39 -0500 Message-ID: <20230125171338.871383998@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:55 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Florent Revest , Mark Rutland Subject: [for-next][PATCH 03/12] ftrace: Add sample with custom ops References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015615906780743?= X-GMAIL-MSGID: =?utf-8?q?1756015615906780743?= From: Mark Rutland When reworking core ftrace code or architectural ftrace code, it's often necessary to test/analyse/benchmark a number of ftrace_ops configurations. This patch adds a module which can be used to explore some of those configurations. I'm using this to benchmark various options for changing the way trampolines and handling of ftrace_ops work on arm64, and ensuring other architectures aren't adversely affected. For example, in a QEMU+KVM VM running on a 2GHz Xeon E5-2660 workstation, loading the module in various configurations produces: | # insmod ftrace-ops.ko | ftrace_ops: registering: | relevant ops: 1 | tracee: tracee_relevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | irrelevant ops: 0 | tracee: tracee_irrelevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | saving registers: NO | assist recursion: NO | assist RCU: NO | ftrace_ops: Attempted 100000 calls to tracee_relevant [ftrace_ops] in 1681558ns (16ns / call) | # insmod ftrace-ops.ko nr_ops_irrelevant=5 | ftrace_ops: registering: | relevant ops: 1 | tracee: tracee_relevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | irrelevant ops: 5 | tracee: tracee_irrelevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | saving registers: NO | assist recursion: NO | assist RCU: NO | ftrace_ops: Attempted 100000 calls to tracee_relevant [ftrace_ops] in 1693042ns (16ns / call) | # insmod ftrace-ops.ko nr_ops_relevant=2 | ftrace_ops: registering: | relevant ops: 2 | tracee: tracee_relevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | irrelevant ops: 0 | tracee: tracee_irrelevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | saving registers: NO | assist recursion: NO | assist RCU: NO | ftrace_ops: Attempted 100000 calls to tracee_relevant [ftrace_ops] in 11965582ns (119ns / call) | # insmod ftrace-ops.ko save_regs=true | ftrace_ops: registering: | relevant ops: 1 | tracee: tracee_relevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | irrelevant ops: 0 | tracee: tracee_irrelevant [ftrace_ops] | tracer: ops_func_nop [ftrace_ops] | saving registers: YES | assist recursion: NO | assist RCU: NO | ftrace_ops: Attempted 100000 calls to tracee_relevant [ftrace_ops] in 4459624ns (44ns / call) Link: https://lkml.kernel.org/r/20230103124912.2948963-4-mark.rutland@arm.com Cc: Florent Revest Acked-by: Masami Hiramatsu (Google) Signed-off-by: Mark Rutland Signed-off-by: Steven Rostedt (Google) --- samples/Kconfig | 7 + samples/Makefile | 1 + samples/ftrace/Makefile | 1 + samples/ftrace/ftrace-ops.c | 252 ++++++++++++++++++++++++++++++++++++ 4 files changed, 261 insertions(+) create mode 100644 samples/ftrace/ftrace-ops.c diff --git a/samples/Kconfig b/samples/Kconfig index 0d81c00289ee..daf14c35f071 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -46,6 +46,13 @@ config SAMPLE_FTRACE_DIRECT_MULTI that hooks to wake_up_process and schedule, and prints the function addresses. +config SAMPLE_FTRACE_OPS + tristate "Build custom ftrace ops example" + depends on FUNCTION_TRACER + help + This builds an ftrace ops example that hooks two functions and + measures the time taken to invoke one function a number of times. + config SAMPLE_TRACE_ARRAY tristate "Build sample module for kernel access to Ftrace instancess" depends on EVENT_TRACING && m diff --git a/samples/Makefile b/samples/Makefile index 9832ef3f8fcb..7cb632ef88ee 100644 --- a/samples/Makefile +++ b/samples/Makefile @@ -24,6 +24,7 @@ obj-$(CONFIG_SAMPLE_TRACE_CUSTOM_EVENTS) += trace_events/ obj-$(CONFIG_SAMPLE_TRACE_PRINTK) += trace_printk/ obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace/ obj-$(CONFIG_SAMPLE_FTRACE_DIRECT_MULTI) += ftrace/ +obj-$(CONFIG_SAMPLE_FTRACE_OPS) += ftrace/ obj-$(CONFIG_SAMPLE_TRACE_ARRAY) += ftrace/ subdir-$(CONFIG_SAMPLE_UHID) += uhid obj-$(CONFIG_VIDEO_PCI_SKELETON) += v4l/ diff --git a/samples/ftrace/Makefile b/samples/ftrace/Makefile index faf8cdb79c5f..589baf2ec4e3 100644 --- a/samples/ftrace/Makefile +++ b/samples/ftrace/Makefile @@ -5,6 +5,7 @@ obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-too.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace-direct-modify.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT_MULTI) += ftrace-direct-multi.o obj-$(CONFIG_SAMPLE_FTRACE_DIRECT_MULTI) += ftrace-direct-multi-modify.o +obj-$(CONFIG_SAMPLE_FTRACE_OPS) += ftrace-ops.o CFLAGS_sample-trace-array.o := -I$(src) obj-$(CONFIG_SAMPLE_TRACE_ARRAY) += sample-trace-array.o diff --git a/samples/ftrace/ftrace-ops.c b/samples/ftrace/ftrace-ops.c new file mode 100644 index 000000000000..24deb51c7261 --- /dev/null +++ b/samples/ftrace/ftrace-ops.c @@ -0,0 +1,252 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include + +#include + +/* + * Arbitrary large value chosen to be sufficiently large to minimize noise but + * sufficiently small to complete quickly. + */ +unsigned int nr_function_calls = 100000; +module_param(nr_function_calls, uint, 0); +MODULE_PARM_DESC(nr_function_calls, "How many times to call the relevant tracee"); + +/* + * The number of ops associated with a call site affects whether a tracer can + * be called directly or whether it's necessary to go via the list func, which + * can be significantly more expensive. + */ +unsigned int nr_ops_relevant = 1; +module_param(nr_ops_relevant, uint, 0); +MODULE_PARM_DESC(nr_ops_relevant, "How many ftrace_ops to associate with the relevant tracee"); + +/* + * On architectures where all call sites share the same trampoline, having + * tracers enabled for distinct functions can force the use of the list func + * and incur overhead for all call sites. + */ +unsigned int nr_ops_irrelevant = 0; +module_param(nr_ops_irrelevant, uint, 0); +MODULE_PARM_DESC(nr_ops_irrelevant, "How many ftrace_ops to associate with the irrelevant tracee"); + +/* + * On architectures with DYNAMIC_FTRACE_WITH_REGS, saving the full pt_regs can + * be more expensive than only saving the minimal necessary regs. + */ +bool save_regs = false; +module_param(save_regs, bool, 0); +MODULE_PARM_DESC(save_regs, "Register ops with FTRACE_OPS_FL_SAVE_REGS (save all registers in the trampoline)"); + +bool assist_recursion = false; +module_param(assist_recursion, bool, 0); +MODULE_PARM_DESC(assist_reursion, "Register ops with FTRACE_OPS_FL_RECURSION"); + +bool assist_rcu = false; +module_param(assist_rcu, bool, 0); +MODULE_PARM_DESC(assist_reursion, "Register ops with FTRACE_OPS_FL_RCU"); + +/* + * By default, a trivial tracer is used which immediately returns to mimimize + * overhead. Sometimes a consistency check using a more expensive tracer is + * desireable. + */ +bool check_count = false; +module_param(check_count, bool, 0); +MODULE_PARM_DESC(check_count, "Check that tracers are called the expected number of times\n"); + +/* + * Usually it's not interesting to leave the ops registered after the test + * runs, but sometimes it can be useful to leave them registered so that they + * can be inspected through the tracefs 'enabled_functions' file. + */ +bool persist = false; +module_param(persist, bool, 0); +MODULE_PARM_DESC(persist, "Successfully load module and leave ftrace ops registered after test completes\n"); + +/* + * Marked as noinline to ensure that an out-of-line traceable copy is + * generated by the compiler. + * + * The barrier() ensures the compiler won't elide calls by determining there + * are no side-effects. + */ +static noinline void tracee_relevant(void) +{ + barrier(); +} + +/* + * Marked as noinline to ensure that an out-of-line traceable copy is + * generated by the compiler. + * + * The barrier() ensures the compiler won't elide calls by determining there + * are no side-effects. + */ +static noinline void tracee_irrelevant(void) +{ + barrier(); +} + +struct sample_ops { + struct ftrace_ops ops; + unsigned int count; +}; + +static void ops_func_nop(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, + struct ftrace_regs *fregs) +{ + /* do nothing */ +} + +static void ops_func_count(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, + struct ftrace_regs *fregs) +{ + struct sample_ops *self; + + self = container_of(op, struct sample_ops, ops); + self->count++; +} + +struct sample_ops *ops_relevant; +struct sample_ops *ops_irrelevant; + +static struct sample_ops *ops_alloc_init(void *tracee, ftrace_func_t func, + unsigned long flags, int nr) +{ + struct sample_ops *ops; + + ops = kcalloc(nr, sizeof(*ops), GFP_KERNEL); + if (WARN_ON_ONCE(!ops)) + return NULL; + + for (unsigned int i = 0; i < nr; i++) { + ops[i].ops.func = func; + ops[i].ops.flags = flags; + WARN_ON_ONCE(ftrace_set_filter_ip(&ops[i].ops, (unsigned long)tracee, 0, 0)); + WARN_ON_ONCE(register_ftrace_function(&ops[i].ops)); + } + + return ops; +} + +static void ops_destroy(struct sample_ops *ops, int nr) +{ + if (!ops) + return; + + for (unsigned int i = 0; i < nr; i++) { + WARN_ON_ONCE(unregister_ftrace_function(&ops[i].ops)); + ftrace_free_filter(&ops[i].ops); + } + + kfree(ops); +} + +static void ops_check(struct sample_ops *ops, int nr, + unsigned int expected_count) +{ + if (!ops || !check_count) + return; + + for (unsigned int i = 0; i < nr; i++) { + if (ops->count == expected_count) + continue; + pr_warn("Counter called %u times (expected %u)\n", + ops->count, expected_count); + } +} + +ftrace_func_t tracer_relevant = ops_func_nop; +ftrace_func_t tracer_irrelevant = ops_func_nop; + +static int __init ftrace_ops_sample_init(void) +{ + unsigned long flags = 0; + ktime_t start, end; + u64 period; + + if (!IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS) && save_regs) { + pr_info("this kernel does not support saving registers\n"); + save_regs = false; + } else if (save_regs) { + flags |= FTRACE_OPS_FL_SAVE_REGS; + } + + if (assist_recursion) + flags |= FTRACE_OPS_FL_RECURSION; + + if (assist_rcu) + flags |= FTRACE_OPS_FL_RCU; + + if (check_count) { + tracer_relevant = ops_func_count; + tracer_irrelevant = ops_func_count; + } + + pr_info("registering:\n" + " relevant ops: %u\n" + " tracee: %ps\n" + " tracer: %ps\n" + " irrelevant ops: %u\n" + " tracee: %ps\n" + " tracer: %ps\n" + " saving registers: %s\n" + " assist recursion: %s\n" + " assist RCU: %s\n", + nr_ops_relevant, tracee_relevant, tracer_relevant, + nr_ops_irrelevant, tracee_irrelevant, tracer_irrelevant, + save_regs ? "YES" : "NO", + assist_recursion ? "YES" : "NO", + assist_rcu ? "YES" : "NO"); + + ops_relevant = ops_alloc_init(tracee_relevant, tracer_relevant, + flags, nr_ops_relevant); + ops_irrelevant = ops_alloc_init(tracee_irrelevant, tracer_irrelevant, + flags, nr_ops_irrelevant); + + start = ktime_get(); + for (unsigned int i = 0; i < nr_function_calls; i++) + tracee_relevant(); + end = ktime_get(); + + ops_check(ops_relevant, nr_ops_relevant, nr_function_calls); + ops_check(ops_irrelevant, nr_ops_irrelevant, 0); + + period = ktime_to_ns(ktime_sub(end, start)); + + pr_info("Attempted %u calls to %ps in %lluns (%lluns / call)\n", + nr_function_calls, tracee_relevant, + period, period / nr_function_calls); + + if (persist) + return 0; + + ops_destroy(ops_relevant, nr_ops_relevant); + ops_destroy(ops_irrelevant, nr_ops_irrelevant); + + /* + * The benchmark completed sucessfully, but there's no reason to keep + * the module around. Return an error do the user doesn't have to + * manually unload the module. + */ + return -EINVAL; +} +module_init(ftrace_ops_sample_init); + +static void __exit ftrace_ops_sample_exit(void) +{ + ops_destroy(ops_relevant, nr_ops_relevant); + ops_destroy(ops_irrelevant, nr_ops_irrelevant); +} +module_exit(ftrace_ops_sample_exit); + +MODULE_AUTHOR("Mark Rutland"); +MODULE_DESCRIPTION("Example of using custom ftrace_ops"); +MODULE_LICENSE("GPL"); From patchwork Wed Jan 25 17:12:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48234 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp388370wrn; Wed, 25 Jan 2023 09:16:23 -0800 (PST) X-Google-Smtp-Source: AMrXdXsGI0QJhsUW4702ANqXWm2e9Jz3i6ryhcK+Ji1tmNy3V8LLYS/iMsnwIiCnrwdnhrrUUSr6 X-Received: by 2002:a17:90b:3852:b0:228:d550:b04e with SMTP id nl18-20020a17090b385200b00228d550b04emr32822103pjb.49.1674666982741; Wed, 25 Jan 2023 09:16:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666982; cv=none; d=google.com; s=arc-20160816; b=pHJFVNvPBxmR0HAkVf2Fa3FCn2Qz5gtz8NlD7OB/Nz2lS65b0AL5Pt/ECOb+9Kclep h0gIfkhjmW6+6pR29G1/Jv19JbSj2BgwPJPRHmcoPGf3tc/TUV0g8dg7e4QLzxu/7UMs oMKU5cjOpB5U1yIoUCIZUP9C99T6Sm933c2fgOxHgKMO20+ckpa3S6uyfmsig//FRypV zdZPrBxj9fyWv4/kCEysxl2fm+Ik6uqqTCNFIse+oj0T2OmmxaMtCwRTlCRrtfWFzfJX dIYd06z0yysPukf1+Xpf/ymOuVDLMuPvPZqO3If5Kx9/Q7YwIdMURA6HxsbGm+H06xfv JMnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=qYCsWWlWTqSx1mFkFj2xdAMQ3q3E0GZoZIky6LdJiJ0=; b=Mm8kr1Ltd5oQMeZqXHqM64dkTDmGpfStoimSSs2Z/2BEQ1blQTXxmT/2WkPx0t/SIK yRX7zvoqgNHEFYwSsuJLoXCcSm9OxdRNAtKvXO55fj1V+KyVZclHgfFHKW+jkbL2rMw+ MlurXRAZ3mS8CBP0xNsMPE2HdK9H8EEOwdWN5GKESAqLse+rQSe8PSKnckETvspDC9MP 8Rx5juX/y/ukXM6DvH9IxQs4JSqWf5A8wXeBgZ0KDC4o3r2IMbmsZ3JG9sqIhgXGgIVt Vke39a9Nrz/VBRPM/d76hCXrwmz2T32ELeQmXwdRNfQgzJvIgzkI3PIRT0tj67KGas80 XJlg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a2-20020a17090a854200b00229ecc9fabbsi8105pjw.32.2023.01.25.09.16.10; Wed, 25 Jan 2023 09:16:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236166AbjAYROP (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235528AbjAYRNy (ORCPT ); Wed, 25 Jan 2023 12:13:54 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD289C677 for ; Wed, 25 Jan 2023 09:13:43 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 60E3DB81B46 for ; Wed, 25 Jan 2023 17:13:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BA28C433A1; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL9-004NrL-0q; Wed, 25 Jan 2023 12:13:39 -0500 Message-ID: <20230125171339.079529885@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:56 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , TOTE Robot , Jia-Ju Bai Subject: [for-next][PATCH 04/12] tracing: Add NULL checks for buffer in ring_buffer_free_read_page() References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015606217503395?= X-GMAIL-MSGID: =?utf-8?q?1756015606217503395?= From: Jia-Ju Bai In a previous commit 7433632c9ff6, buffer, buffer->buffers and buffer->buffers[cpu] in ring_buffer_wake_waiters() can be NULL, and thus the related checks are added. However, in the same call stack, these variables are also used in ring_buffer_free_read_page(): tracing_buffers_release() ring_buffer_wake_waiters(iter->array_buffer->buffer) cpu_buffer = buffer->buffers[cpu] -> Add checks by previous commit ring_buffer_free_read_page(iter->array_buffer->buffer) cpu_buffer = buffer->buffers[cpu] -> No check Thus, to avod possible null-pointer derefernces, the related checks should be added. These results are reported by a static tool designed by myself. Link: https://lkml.kernel.org/r/20230113125501.760324-1-baijiaju1990@gmail.com Reported-by: TOTE Robot Signed-off-by: Jia-Ju Bai Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index c366a0a9ddba..45d4a23d6044 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5626,11 +5626,16 @@ EXPORT_SYMBOL_GPL(ring_buffer_alloc_read_page); */ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data) { - struct ring_buffer_per_cpu *cpu_buffer = buffer->buffers[cpu]; + struct ring_buffer_per_cpu *cpu_buffer; struct buffer_data_page *bpage = data; struct page *page = virt_to_page(bpage); unsigned long flags; + if (!buffer || !buffer->buffers || !buffer->buffers[cpu]) + return; + + cpu_buffer = buffer->buffers[cpu]; + /* If the page is still in use someplace else, we can't reuse it */ if (page_ref_count(page) > 1) goto out; From patchwork Wed Jan 25 17:12:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48227 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387487wrn; Wed, 25 Jan 2023 09:14:44 -0800 (PST) X-Google-Smtp-Source: AMrXdXuaOe1YEj0gFwIN7AgEqNCEh6IKuUtZUFr24zvKR4wAbVNL77e+kUymwZ9whb4nce+nuLji X-Received: by 2002:a17:907:874e:b0:872:27cb:9430 with SMTP id qo14-20020a170907874e00b0087227cb9430mr39590392ejc.74.1674666884341; Wed, 25 Jan 2023 09:14:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666884; cv=none; d=google.com; s=arc-20160816; b=e2rmOzboYbloZ8OfKyFKyhfKcfrXy5p+kwgaDaFqZPqbZbS56/Q8h9YizpBMIACU00 WzB+YWh249cQd9Q78x8GdLCxP8OlRaL9h2XwM+u5AlvC1s/g18WSlfsLMtvS03z8ntIS S9TPQGf+Ezhs0fHY2Z/I548iGV5YRfEYxBY/QQAR0c2hNayeclPASBd/582X+5MCwQXz enenTxHWmwTIMcA1OB1ptfeS5/HfQHOScXGSDCRstuii4nltnyvr4LpZfhK9vt7VosnT MRCHy33IMa5xQAJ8oyDPLb3+zuISIBk4Q2jLZiYKhqC8gcT6hK/LaH5V1uapDjB/vg7j y5ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=mkQnruujHYke86zr0a96Qu4G6mVJLFpGu9LAwBC22JE=; b=m29dbyniZ6EYT2gNkHn8dxfNQHh7FGTTf6FvCbfBb0lfuh8Vh+gI+7enmoa6Y5AjoI PIw4MH3Gz0QJh3tJaHO+BeuflNHMMTd/2wN6u3lmXna7+7m8p72HD4OP25rFZgnDPxqv WQySHFBLxDtV7uNggu2LICKTre1UAJf0X+fHpVnWJBp/EFAInaPHZvgQ9lZ1a3EcuQNT ybtFrzULUdk4xOu/QAUxTc1ouf1P4pYGvMJ5HPDfFVkdwe4L9ybceGVia3SAjlV5HyBI EFsl5sj6xJ1CSM+uO2cgWy7Vy0Le/DcBtVhhC9X01vquUgqyDDEoxwupbzQMvpCW1JWW ELhw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j24-20020a170906105800b0086e0afb91c8si6058637ejj.514.2023.01.25.09.14.19; Wed, 25 Jan 2023 09:14:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236014AbjAYRN6 (ORCPT + 99 others); Wed, 25 Jan 2023 12:13:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38408 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235195AbjAYRNw (ORCPT ); Wed, 25 Jan 2023 12:13:52 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AA5210CE for ; Wed, 25 Jan 2023 09:13:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B920361591 for ; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26DE2C433AA; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL9-004Nrt-1V; Wed, 25 Jan 2023 12:13:39 -0500 Message-ID: <20230125171339.278637143@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:57 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Ross Zwisler , Ching-lin Yu Subject: [for-next][PATCH 05/12] tracing: Simplify calculating entry size using struct_size() References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015502988976960?= X-GMAIL-MSGID: =?utf-8?q?1756015502988976960?= From: "Steven Rostedt (Google)" When tracing a dynamic string field for a synthetic event, the offset calculation for where to write the next event can use struct_size() to find what the current size of the structure is. This simplifies the code and makes it less error prone. Link: https://lkml.kernel.org/r/20230117152235.698632147@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Ross Zwisler Cc: Ching-lin Yu Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_synth.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/kernel/trace/trace_events_synth.c b/kernel/trace/trace_events_synth.c index 67592eed0be8..9f79cd689b79 100644 --- a/kernel/trace/trace_events_synth.c +++ b/kernel/trace/trace_events_synth.c @@ -416,8 +416,7 @@ static unsigned int trace_string(struct synth_trace_event *entry, if (is_dynamic) { u32 data_offset; - data_offset = offsetof(typeof(*entry), fields); - data_offset += event->n_u64 * sizeof(u64); + data_offset = struct_size(entry, fields, event->n_u64); data_offset += data_size; len = kern_fetch_store_strlen((unsigned long)str_val); From patchwork Wed Jan 25 17:12:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48231 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387854wrn; Wed, 25 Jan 2023 09:15:25 -0800 (PST) X-Google-Smtp-Source: AK7set+awWX74D+mDmUpiISPH+QqGGBnWUZMMKXCBKcQ/Vv1DIl8+Iui+SHVrJyu2O0jIR535jj/ X-Received: by 2002:a17:90b:33d0:b0:22c:8af:2c7 with SMTP id lk16-20020a17090b33d000b0022c08af02c7mr2825224pjb.21.1674666925176; Wed, 25 Jan 2023 09:15:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666925; cv=none; d=google.com; s=arc-20160816; b=D7tAFuL2Pj6gsDU0+3ylceF84itwQOLxgo+7IFZjGzLdw0Iyjkz6+fWpaytgJuqilF RDQSk3YXWrIcWFP2n/JMP4ZHXdFsz2v51yKTSo6/XF8iTzlhkrrxGDyePR0dP0MzKxUk 7+tQSBg6RYTg8WlyuFJWkPJ+QtnS2Esb5/ie/Dhjhg7dHsEbUvTHGtkj3ICaHXN0o6+x NxoU/9qz1OMBBGb5sBAC9Vq2760VAWdQTJytQtcVZ5lyfQ+8KAnjWF+qq3hSHVnWKPt2 8ZvGUpzFfyjxwdNpTNIhV2uGKbWtapSFwwX3KUkiGgLAhct01OYnLP6JBJ5qrlSkJz3E NHZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=nt9ztFKr25e84fk8jBmFKRdAuDFYh0nHTGFdXLcB/8U=; b=ftnFkfyFMkRU+gvTfTYfwvE6qKXQPR5leAzJZe9Mhv0zSujDpbXNTxVfinghwHeBxK Q4IfWlr3BzNym510akp3L1vjztSh8MDU+igluRbI7d//fBBP3AkJJh4GbFFgSqZ0d4mx Ztgn9ss3DtO7sWx/z7vE5QGn9GqouRBvExQOFDnOcKzFgjbb47VJ5/5qIID/bHBXHydk SK72s7ZC9RCj8zK9ALiTykJIZ0bpOSZoK6moyXT9qDa8jQZg3e6N9GwPHhmmeQg6oe/k 0uIXGSjZ+YXHxNRp/2p6ulA/+QjqTHJgnT5R23vcckmk9wn1feiId/dLGSmyKvt+uYqq 3X4g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w6-20020a17090aad4600b0022c0aac2e84si2113099pjv.36.2023.01.25.09.15.11; Wed, 25 Jan 2023 09:15:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236158AbjAYROM (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235591AbjAYRNy (ORCPT ); Wed, 25 Jan 2023 12:13:54 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0470CC03 for ; Wed, 25 Jan 2023 09:13:43 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 77069B81A22 for ; Wed, 25 Jan 2023 17:13:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20BA0C4339C; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL9-004NsR-2B; Wed, 25 Jan 2023 12:13:39 -0500 Message-ID: <20230125171339.487333765@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:58 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Ross Zwisler , Ching-lin Yu Subject: [for-next][PATCH 06/12] tracing: Allow stacktraces to be saved as histogram variables References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015546019727086?= X-GMAIL-MSGID: =?utf-8?q?1756015546019727086?= From: "Steven Rostedt (Google)" Allow to save stacktraces into a histogram variable. This will be used by synthetic events to allow a stacktrace from one event to be passed and displayed by another event. The special keyword "stacktrace" is to be used to trigger a stack trace for the event that the histogram trigger is attached to. echo 'hist:keys=pid:st=stacktrace" > events/sched/sched_waking/trigger Currently nothing can get access to the "$st" variable above that contains the stack trace, but that will soon change. Link: https://lkml.kernel.org/r/20230117152235.856323729@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Ross Zwisler Cc: Ching-lin Yu Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_hist.c | 52 ++++++++++++++++++++++++++------ 1 file changed, 42 insertions(+), 10 deletions(-) diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index 5edbf6b1da3f..f8f67e17898a 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -1360,6 +1360,8 @@ static const char *hist_field_name(struct hist_field *field, field_name = field->name; } else if (field->flags & HIST_FIELD_FL_TIMESTAMP) field_name = "common_timestamp"; + else if (field->flags & HIST_FIELD_FL_STACKTRACE) + field_name = "stacktrace"; else if (field->flags & HIST_FIELD_FL_HITCOUNT) field_name = "hitcount"; @@ -1980,6 +1982,10 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, if (flags & HIST_FIELD_FL_STACKTRACE) { hist_field->fn_num = HIST_FIELD_FN_NOP; + hist_field->size = HIST_STACKTRACE_SIZE; + hist_field->type = kstrdup_const("unsigned long[]", GFP_KERNEL); + if (!hist_field->type) + goto free; goto out; } @@ -2351,6 +2357,8 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file, hist_data->enable_timestamps = true; if (*flags & HIST_FIELD_FL_TIMESTAMP_USECS) hist_data->attrs->ts_in_usecs = true; + } else if (strcmp(field_name, "stacktrace") == 0) { + *flags |= HIST_FIELD_FL_STACKTRACE; } else if (strcmp(field_name, "common_cpu") == 0) *flags |= HIST_FIELD_FL_CPU; else if (strcmp(field_name, "hitcount") == 0) @@ -3119,13 +3127,24 @@ static inline void __update_field_vars(struct tracing_map_elt *elt, var_val = hist_fn_call(val, elt, buffer, rbe, rec); var_idx = var->var.idx; - if (val->flags & HIST_FIELD_FL_STRING) { + if (val->flags & (HIST_FIELD_FL_STRING | + HIST_FIELD_FL_STACKTRACE)) { char *str = elt_data->field_var_str[j++]; char *val_str = (char *)(uintptr_t)var_val; unsigned int size; - size = min(val->size, STR_VAR_LEN_MAX); - strscpy(str, val_str, size); + if (val->flags & HIST_FIELD_FL_STRING) { + size = min(val->size, STR_VAR_LEN_MAX); + strscpy(str, val_str, size); + } else { + int e; + + e = stack_trace_save((void *)str, + HIST_STACKTRACE_DEPTH, + HIST_STACKTRACE_SKIP); + if (e < HIST_STACKTRACE_DEPTH - 1) + ((unsigned long *)str)[e] = 0; + } var_val = (u64)(uintptr_t)str; } tracing_map_set_var(elt, var_idx, var_val); @@ -3824,7 +3843,8 @@ static void save_field_var(struct hist_trigger_data *hist_data, { hist_data->field_vars[hist_data->n_field_vars++] = field_var; - if (field_var->val->flags & HIST_FIELD_FL_STRING) + /* Stack traces are saved in the string storage too */ + if (field_var->val->flags & (HIST_FIELD_FL_STRING | HIST_FIELD_FL_STACKTRACE)) hist_data->n_field_var_str++; } @@ -4103,7 +4123,8 @@ static int action_create(struct hist_trigger_data *hist_data, } hist_data->save_vars[hist_data->n_save_vars++] = field_var; - if (field_var->val->flags & HIST_FIELD_FL_STRING) + if (field_var->val->flags & + (HIST_FIELD_FL_STRING | HIST_FIELD_FL_STACKTRACE)) hist_data->n_save_var_str++; kfree(param); } @@ -4351,7 +4372,8 @@ static int create_var_field(struct hist_trigger_data *hist_data, if (!ret && hist_data->fields[val_idx]->flags & HIST_FIELD_FL_EXECNAME) update_var_execname(hist_data->fields[val_idx]); - if (!ret && hist_data->fields[val_idx]->flags & HIST_FIELD_FL_STRING) + if (!ret && hist_data->fields[val_idx]->flags & + (HIST_FIELD_FL_STRING | HIST_FIELD_FL_STACKTRACE)) hist_data->fields[val_idx]->var_str_idx = hist_data->n_var_str++; return ret; @@ -5092,7 +5114,8 @@ static void hist_trigger_elt_update(struct hist_trigger_data *hist_data, if (hist_field->flags & HIST_FIELD_FL_VAR) { var_idx = hist_field->var.idx; - if (hist_field->flags & HIST_FIELD_FL_STRING) { + if (hist_field->flags & + (HIST_FIELD_FL_STRING | HIST_FIELD_FL_STACKTRACE)) { unsigned int str_start, var_str_idx, idx; char *str, *val_str; unsigned int size; @@ -5105,9 +5128,18 @@ static void hist_trigger_elt_update(struct hist_trigger_data *hist_data, str = elt_data->field_var_str[idx]; val_str = (char *)(uintptr_t)hist_val; - size = min(hist_field->size, STR_VAR_LEN_MAX); - strscpy(str, val_str, size); - + if (hist_field->flags & HIST_FIELD_FL_STRING) { + size = min(hist_field->size, STR_VAR_LEN_MAX); + strscpy(str, val_str, size); + } else { + int e; + + e = stack_trace_save((void *)str, + HIST_STACKTRACE_DEPTH, + HIST_STACKTRACE_SKIP); + if (e < HIST_STACKTRACE_DEPTH - 1) + ((unsigned long *)str)[e] = 0; + } hist_val = (u64)(uintptr_t)str; } tracing_map_set_var(elt, var_idx, hist_val); From patchwork Wed Jan 25 17:12:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48238 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp392301wrn; Wed, 25 Jan 2023 09:24:21 -0800 (PST) X-Google-Smtp-Source: AK7set8Jz1ZaBaN+ntXRZ/68h+nvkpLPQylBF5mCLN2pnJmb0T7VyM/eY4y72+dh8vNWtTOmYFlW X-Received: by 2002:a17:903:120e:b0:196:27be:4f53 with SMTP id l14-20020a170903120e00b0019627be4f53mr3652936plh.13.1674667461402; Wed, 25 Jan 2023 09:24:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674667461; cv=none; d=google.com; s=arc-20160816; b=TW8rjmkDuAqw6zkwyqO+iarHnMbDc1ss+9VnDh5oO53yBJfdxL4dDVUnJzbejzzZYi GeDKr3hlw4IElu94PpzYGU1PTSYF/c4+E0oNn0/8+7tnkZcg9u2l8g8+0Dh6X4SG+ZH3 3C+pYHiRGdPZ5573VBZNlDsUBJKjamIplWWCnGEyf6KM67lSGEOfHQge1kosD/yR4BTh zxW+6pV1TgqoA13S1ipMiPKPelu/EUPuaLhbLYSLuIJuH7b/VDMTO4hGfrVlG4ajhUe6 A6NM2a1u37egnxjZlYV4rLw6F/vm2Augd42e7hV63hxzL9rAZBM1asAJLIobSvTiCdz/ A6ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=4CrAJuNnGqCrx7CH2lJsd8pLmM21sYD1AVRXArbqvb4=; b=Rzt8rT9RpwG3GXHSRtYSVOzGOPzLMRwbnTQvUtJPqjEz3d2EswNbEm52MllUMdtXr8 OTiRELRXGJad7doaxoRbWH0yDzIktbEQkpMPdROz/cSt9ZKhFcrDjYH6fKKacZPMRiDj iAuruVcZm6oMrX1er2ZpLbEoSMpi8o2DVbc7ZihUX01IS3JQOAjuwHFjyKChB5+uEypi sBmwBoWhTsxlv8sqXI/oiKA7yghFk2PCD6K8bdcbtrsaPBVkAgcfXaOIZ5NJlsmpp045 dWp2o7Bgt7qu1vKcn6D6qq9Tog2BhftRJEKvupU5conxTwnxSfGFP6l4VZSwGvzxy1He SxrQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z1-20020a170903018100b00189db2e76f1si7289944plg.389.2023.01.25.09.24.08; Wed, 25 Jan 2023 09:24:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236171AbjAYROc (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236103AbjAYROD (ORCPT ); Wed, 25 Jan 2023 12:14:03 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F169C7289 for ; Wed, 25 Jan 2023 09:13:54 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 84863B81B5B for ; Wed, 25 Jan 2023 17:13:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2442BC4339B; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjL9-004Nt1-2q; Wed, 25 Jan 2023 12:13:39 -0500 Message-ID: <20230125171339.695687314@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:12:59 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Ross Zwisler , Ching-lin Yu Subject: [for-next][PATCH 07/12] tracing: Allow synthetic events to pass around stacktraces References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756016108112921422?= X-GMAIL-MSGID: =?utf-8?q?1756016108112921422?= From: "Steven Rostedt (Google)" Allow a stacktrace from one event to be displayed by the end event of a synthetic event. This is very useful when looking for the longest latency of a sleep or something blocked on I/O. # cd /sys/kernel/tracing/ # echo 's:block_lat pid_t pid; u64 delta; unsigned long[] stack;' > dynamic_events # echo 'hist:keys=next_pid:ts=common_timestamp.usecs,st=stacktrace if prev_state == 1||prev_state == 2' > events/sched/sched_switch/trigger # echo 'hist:keys=prev_pid:delta=common_timestamp.usecs-$ts,s=$st:onmax($delta).trace(block_lat,prev_pid,$delta,$s)' >> events/sched/sched_switch/trigger The above creates a "block_lat" synthetic event that take the stacktrace of when a task schedules out in either the interruptible or uninterruptible states, and on a new per process max $delta (the time it was scheduled out), will print the process id and the stacktrace. # echo 1 > events/synthetic/block_lat/enable # cat trace # TASK-PID CPU# ||||| TIMESTAMP FUNCTION # | | | ||||| | | kworker/u16:0-767 [006] d..4. 560.645045: block_lat: pid=767 delta=66 stack=STACK: => __schedule => schedule => pipe_read => vfs_read => ksys_read => do_syscall_64 => 0x966000aa -0 [003] d..4. 561.132117: block_lat: pid=0 delta=413787 stack=STACK: => __schedule => schedule => schedule_hrtimeout_range_clock => do_sys_poll => __x64_sys_poll => do_syscall_64 => 0x966000aa <...>-153 [006] d..4. 562.068407: block_lat: pid=153 delta=54 stack=STACK: => __schedule => schedule => io_schedule => rq_qos_wait => wbt_wait => __rq_qos_throttle => blk_mq_submit_bio => submit_bio_noacct_nocheck => ext4_bio_write_page => mpage_submit_page => mpage_process_page_bufs => mpage_prepare_extent_to_map => ext4_do_writepages => ext4_writepages => do_writepages => __writeback_single_inode Link: https://lkml.kernel.org/r/20230117152236.010941267@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Ross Zwisler Cc: Ching-lin Yu Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.h | 4 ++ kernel/trace/trace_events_hist.c | 7 ++- kernel/trace/trace_events_synth.c | 80 ++++++++++++++++++++++++++++++- kernel/trace/trace_synth.h | 1 + 4 files changed, 87 insertions(+), 5 deletions(-) diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 4eb6d6b97a9f..d16929dd0f08 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -113,6 +113,10 @@ enum trace_type { #define MEM_FAIL(condition, fmt, ...) \ DO_ONCE_LITE_IF(condition, pr_err, "ERROR: " fmt, ##__VA_ARGS__) +#define HIST_STACKTRACE_DEPTH 16 +#define HIST_STACKTRACE_SIZE (HIST_STACKTRACE_DEPTH * sizeof(unsigned long)) +#define HIST_STACKTRACE_SKIP 5 + /* * syscalls are special, and need special handling, this is why * they are not included in trace_entries.h diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index f8f67e17898a..2c0837ff2299 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -480,10 +480,6 @@ DEFINE_HIST_FIELD_FN(u8); #define for_each_hist_key_field(i, hist_data) \ for ((i) = (hist_data)->n_vals; (i) < (hist_data)->n_fields; (i)++) -#define HIST_STACKTRACE_DEPTH 16 -#define HIST_STACKTRACE_SIZE (HIST_STACKTRACE_DEPTH * sizeof(unsigned long)) -#define HIST_STACKTRACE_SKIP 5 - #define HITCOUNT_IDX 0 #define HIST_KEY_SIZE_MAX (MAX_FILTER_STR_VAL + HIST_STACKTRACE_SIZE) @@ -3869,6 +3865,9 @@ static int check_synth_field(struct synth_event *event, && field->is_dynamic) return 0; + if (strstr(hist_field->type, "long[") && field->is_stack) + return 0; + if (strcmp(field->type, hist_field->type) != 0) { if (field->size != hist_field->size || (!field->is_string && field->is_signed != hist_field->is_signed)) diff --git a/kernel/trace/trace_events_synth.c b/kernel/trace/trace_events_synth.c index 9f79cd689b79..adb630633f31 100644 --- a/kernel/trace/trace_events_synth.c +++ b/kernel/trace/trace_events_synth.c @@ -173,6 +173,14 @@ static int synth_field_is_string(char *type) return false; } +static int synth_field_is_stack(char *type) +{ + if (strstr(type, "long[") != NULL) + return true; + + return false; +} + static int synth_field_string_size(char *type) { char buf[4], *end, *start; @@ -248,6 +256,8 @@ static int synth_field_size(char *type) size = sizeof(gfp_t); else if (synth_field_is_string(type)) size = synth_field_string_size(type); + else if (synth_field_is_stack(type)) + size = 0; return size; } @@ -292,6 +302,8 @@ static const char *synth_field_fmt(char *type) fmt = "%x"; else if (synth_field_is_string(type)) fmt = "%.*s"; + else if (synth_field_is_stack(type)) + fmt = "%s"; return fmt; } @@ -371,6 +383,23 @@ static enum print_line_t print_synth_event(struct trace_iterator *iter, i == se->n_fields - 1 ? "" : " "); n_u64 += STR_VAR_LEN_MAX / sizeof(u64); } + } else if (se->fields[i]->is_stack) { + u32 offset, data_offset, len; + unsigned long *p, *end; + + offset = (u32)entry->fields[n_u64]; + data_offset = offset & 0xffff; + len = offset >> 16; + + p = (void *)entry + data_offset; + end = (void *)p + len - (sizeof(long) - 1); + + trace_seq_printf(s, "%s=STACK:\n", se->fields[i]->name); + + for (; *p && p < end; p++) + trace_seq_printf(s, "=> %pS\n", (void *)*p); + n_u64++; + } else { struct trace_print_flags __flags[] = { __def_gfpflag_names, {-1, NULL} }; @@ -446,6 +475,43 @@ static unsigned int trace_string(struct synth_trace_event *entry, return len; } +static unsigned int trace_stack(struct synth_trace_event *entry, + struct synth_event *event, + long *stack, + unsigned int data_size, + unsigned int *n_u64) +{ + unsigned int len; + u32 data_offset; + void *data_loc; + + data_offset = struct_size(entry, fields, event->n_u64); + data_offset += data_size; + + for (len = 0; len < HIST_STACKTRACE_DEPTH; len++) { + if (!stack[len]) + break; + } + + /* Include the zero'd element if it fits */ + if (len < HIST_STACKTRACE_DEPTH) + len++; + + len *= sizeof(long); + + /* Find the dynamic section to copy the stack into. */ + data_loc = (void *)entry + data_offset; + memcpy(data_loc, stack, len); + + /* Fill in the field that holds the offset/len combo */ + data_offset |= len << 16; + *(u32 *)&entry->fields[*n_u64] = data_offset; + + (*n_u64)++; + + return len; +} + static notrace void trace_event_raw_event_synth(void *__data, u64 *var_ref_vals, unsigned int *var_ref_idx) @@ -498,6 +564,12 @@ static notrace void trace_event_raw_event_synth(void *__data, event->fields[i]->is_dynamic, data_size, &n_u64); data_size += len; /* only dynamic string increments */ + } if (event->fields[i]->is_stack) { + long *stack = (long *)(long)var_ref_vals[val_idx]; + + len = trace_stack(entry, event, stack, + data_size, &n_u64); + data_size += len; } else { struct synth_field *field = event->fields[i]; u64 val = var_ref_vals[val_idx]; @@ -560,6 +632,9 @@ static int __set_synth_event_print_fmt(struct synth_event *event, event->fields[i]->is_dynamic) pos += snprintf(buf + pos, LEN_OR_ZERO, ", __get_str(%s)", event->fields[i]->name); + else if (event->fields[i]->is_stack) + pos += snprintf(buf + pos, LEN_OR_ZERO, + ", __get_stacktrace(%s)", event->fields[i]->name); else pos += snprintf(buf + pos, LEN_OR_ZERO, ", REC->%s", event->fields[i]->name); @@ -696,7 +771,8 @@ static struct synth_field *parse_synth_field(int argc, char **argv, ret = -EINVAL; goto free; } else if (size == 0) { - if (synth_field_is_string(field->type)) { + if (synth_field_is_string(field->type) || + synth_field_is_stack(field->type)) { char *type; len = sizeof("__data_loc ") + strlen(field->type) + 1; @@ -727,6 +803,8 @@ static struct synth_field *parse_synth_field(int argc, char **argv, if (synth_field_is_string(field->type)) field->is_string = true; + else if (synth_field_is_stack(field->type)) + field->is_stack = true; field->is_signed = synth_field_signed(field->type); out: diff --git a/kernel/trace/trace_synth.h b/kernel/trace/trace_synth.h index b29595fe3ac5..43f6fb6078db 100644 --- a/kernel/trace/trace_synth.h +++ b/kernel/trace/trace_synth.h @@ -18,6 +18,7 @@ struct synth_field { bool is_signed; bool is_string; bool is_dynamic; + bool is_stack; }; struct synth_event { From patchwork Wed Jan 25 17:13:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48228 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387503wrn; Wed, 25 Jan 2023 09:14:45 -0800 (PST) X-Google-Smtp-Source: AMrXdXtreoUYUj77HgoHcRCVmOGClTNpKPpwMTHiTRpwbbihrOAX8dF/SbKloaRNnAeA2727PH7u X-Received: by 2002:a17:906:4b53:b0:873:3806:be87 with SMTP id j19-20020a1709064b5300b008733806be87mr35068686ejv.71.1674666885524; Wed, 25 Jan 2023 09:14:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666885; cv=none; d=google.com; s=arc-20160816; b=PK0a9VBIfsV5zFkEPwioS6kKai7JiGew8RwD5QxKHCSBOwcUdR3mvZJheSXqjQ0SQV essdcXIu38Co3bdNjKBbhn9j9ei2tYAJuj2W3zwdTpVqrjfbXSTAzFxf10K1bmvaKIzZ I5LEyN3GGXJr0ykYmFnZGRmZ/xKo5JHjPRCsb7mY9cYkXUrVdPYpAK9cnUxmkN1U/bMg QDQWnjGNOk0fZdkc0hdAALg0zt6DAzJ/l62qpN8poojVqRvWKLEPtHB05K0SDcQ6sKDi nwBAIeMOLB+dNffAkfvGtRYL5qEJxryQQeDeOmkZaSDNp2LGbHYWG5JFsUmp0Hh76ziF Trwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=utEPrX6XnRI+DgSIP+yzmy4EHZ/1jC665oTrM6/HemY=; b=xgWeDmMJ8Yuhr/TH+i/dHH5qiJ1YTRvjOl9ZgA88clHflEStQ1E6VH7LdDsP/Gibuq 0Hn0Y4KTgd+mbI3AfImCoBpxKw7yR1CukuDsdORKBbrDp62fQ+ZYWQxqzaSJLr/NR28B HUv7WVmB53ejk8S2TbtLGu3gLTJPXiUT/FbtnK6SmUccqMTT28xxY05mJmsONIgi4sKM oxX9s/pYla82imZAKifP9mS/s0fWTdj8CKKW2GYTn+mY8bMgDdxn1UCugMC5L7KeM6r9 Vo7EhHR3LQqk/YvuB2uoJwbQd2nxyha5yUcEywYuQeGQXYwxcxYYUNBjiVAF8Zwklp4t +2KQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id wl1-20020a170907310100b00877611b71e1si5941634ejb.622.2023.01.25.09.14.21; Wed, 25 Jan 2023 09:14:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235923AbjAYROC (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235009AbjAYRNw (ORCPT ); Wed, 25 Jan 2023 12:13:52 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03E575FF0 for ; Wed, 25 Jan 2023 09:13:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9D8A061599 for ; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1535EC433EF; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjLA-004Nta-0G; Wed, 25 Jan 2023 12:13:40 -0500 Message-ID: <20230125171339.900965795@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:13:00 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Ross Zwisler , Ching-lin Yu Subject: [for-next][PATCH 08/12] tracing/histogram: Add stacktrace type References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015504171160820?= X-GMAIL-MSGID: =?utf-8?q?1756015504171160820?= From: "Steven Rostedt (Google)" Now that stacktraces can be part of synthetic events, allow a key to be typed as a stacktrace. # cd /sys/kernel/tracing # echo 's:block_lat u64 delta; unsigned long stack[];' >> dynamic_events # echo 'hist:keys=next_pid:ts=common_timestamp.usecs,st=stacktrace if prev_state == 2' >> events/sched/sched_switch/trigger # echo 'hist:keys=prev_pid:delta=common_timestamp.usecs-$ts,st2=$st:onmatch(sched.sched_switch).trace(block_lat,$delta,$st2)' >> events/sched/sched_switch/trigger # echo 'hist:keys=delta.buckets=100,stack.stacktrace:sort=delta' > events/synthetic/block_lat/trigger # cat events/synthetic/block_lat/hist # event histogram # # trigger info: hist:keys=delta.buckets=100,stacktrace:vals=hitcount:sort=delta.buckets=100:size=2048 [active] # { delta: ~ 0-99, stacktrace: event_hist_trigger+0x464/0x480 event_triggers_call+0x52/0xe0 trace_event_buffer_commit+0x193/0x250 trace_event_raw_event_sched_switch+0xfc/0x150 __traceiter_sched_switch+0x41/0x60 __schedule+0x448/0x7b0 schedule_idle+0x26/0x40 cpu_startup_entry+0x19/0x20 start_secondary+0xed/0xf0 secondary_startup_64_no_verify+0xe0/0xeb } hitcount: 6 { delta: ~ 0-99, stacktrace: event_hist_trigger+0x464/0x480 event_triggers_call+0x52/0xe0 trace_event_buffer_commit+0x193/0x250 trace_event_raw_event_sched_switch+0xfc/0x150 __traceiter_sched_switch+0x41/0x60 __schedule+0x448/0x7b0 schedule_idle+0x26/0x40 cpu_startup_entry+0x19/0x20 __pfx_kernel_init+0x0/0x10 arch_call_rest_init+0xa/0x24 start_kernel+0x964/0x98d secondary_startup_64_no_verify+0xe0/0xeb } hitcount: 3 { delta: ~ 0-99, stacktrace: event_hist_trigger+0x464/0x480 event_triggers_call+0x52/0xe0 trace_event_buffer_commit+0x193/0x250 trace_event_raw_event_sched_switch+0xfc/0x150 __traceiter_sched_switch+0x41/0x60 __schedule+0x448/0x7b0 schedule+0x5a/0xb0 worker_thread+0xaf/0x380 kthread+0xe9/0x110 ret_from_fork+0x2c/0x50 } hitcount: 1 { delta: ~ 100-199, stacktrace: event_hist_trigger+0x464/0x480 event_triggers_call+0x52/0xe0 trace_event_buffer_commit+0x193/0x250 trace_event_raw_event_sched_switch+0xfc/0x150 __traceiter_sched_switch+0x41/0x60 __schedule+0x448/0x7b0 schedule_idle+0x26/0x40 cpu_startup_entry+0x19/0x20 start_secondary+0xed/0xf0 secondary_startup_64_no_verify+0xe0/0xeb } hitcount: 15 [..] { delta: ~ 8500-8599, stacktrace: event_hist_trigger+0x464/0x480 event_triggers_call+0x52/0xe0 trace_event_buffer_commit+0x193/0x250 trace_event_raw_event_sched_switch+0xfc/0x150 __traceiter_sched_switch+0x41/0x60 __schedule+0x448/0x7b0 schedule_idle+0x26/0x40 cpu_startup_entry+0x19/0x20 start_secondary+0xed/0xf0 secondary_startup_64_no_verify+0xe0/0xeb } hitcount: 1 Totals: Hits: 89 Entries: 11 Dropped: 0 Link: https://lkml.kernel.org/r/20230117152236.167046397@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Ross Zwisler Cc: Ching-lin Yu Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_events_hist.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/trace/trace_events_hist.c b/kernel/trace/trace_events_hist.c index 2c0837ff2299..7f3e6ca6f0fa 100644 --- a/kernel/trace/trace_events_hist.c +++ b/kernel/trace/trace_events_hist.c @@ -1716,6 +1716,8 @@ static const char *get_hist_field_flags(struct hist_field *hist_field) flags_str = "percent"; else if (hist_field->flags & HIST_FIELD_FL_GRAPH) flags_str = "graph"; + else if (hist_field->flags & HIST_FIELD_FL_STACKTRACE) + flags_str = "stacktrace"; return flags_str; } @@ -2314,6 +2316,8 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file, *flags |= HIST_FIELD_FL_EXECNAME; else if (strcmp(modifier, "syscall") == 0) *flags |= HIST_FIELD_FL_SYSCALL; + else if (strcmp(modifier, "stacktrace") == 0) + *flags |= HIST_FIELD_FL_STACKTRACE; else if (strcmp(modifier, "log2") == 0) *flags |= HIST_FIELD_FL_LOG2; else if (strcmp(modifier, "usecs") == 0) From patchwork Wed Jan 25 17:13:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48237 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp388820wrn; Wed, 25 Jan 2023 09:17:18 -0800 (PST) X-Google-Smtp-Source: AMrXdXtalYgqO+Dpd3QKU166VyVikkzcnFQH7J6oxyeuuqoWonL1NwgAOr8XBHjQAI4lPiQ7xWXQ X-Received: by 2002:a17:90a:1c88:b0:226:ae11:b55b with SMTP id t8-20020a17090a1c8800b00226ae11b55bmr34610663pjt.11.1674667038508; Wed, 25 Jan 2023 09:17:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674667038; cv=none; d=google.com; s=arc-20160816; b=w8/QBh/YAirBJ6PKHaiw2MYBNI0dP1UoMx/ctO6O+Wog8M56VSnIraQzRIV2aT+xHD HN1yOHFH0cD0T1YYwU8qYoRIKTIsFdXoJpidmpQTnyzOxvIvHcXrzqn6aXEMz/B/8cNQ zvoLXMnXJ/BxgOAs3N/8Ndxyya7eyMHV76+53ZqyEpfhSGhmU0F9opv4J7qLdgbBRcXN E/gZd7tQW5t9W1kLevZ8iCSFWsRv13AEMFyNOKfYk82RK+NQvEhm0n5qOlakoNnBk6Dm ra8sbRlWAOmailVMZPjvAj3uEXKJNbZCAGVefVdcKnJB4wyGHeYcf1yVq8U//TYd2fHW /gSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=ihUwFfBh0+N/hvLA+ow9WtgByKcMWrjFCl3ajHBkZiE=; b=kT8bhAoq06lW5V0ESbepAaHFk3uPR9RuJ+QXTe5DCX7jGzSdS2alwr27OHUQyJ4Reb rWn4DRhzR2DrvJszNqo5EW7ONDvA3LXh7EATExAcigaxLaJaFGBr1mJ1eRockQX2ueO6 oHMJAYMsapkBX9Dj9UMIOkOFTfGbGhVtyw6bC6a/jcQq949YIg7p5dWbHmA/kjY7Tit7 DmCXHIDhYIygI6KkLwNxjd/hTjsuufJ1jvv4VFzU8w2JocS7UtIoqo/PWjA9xxv+mfBp eBmo7dc6DpOBMpWKxKcY1ZnPiFOwBpMVxVaQZyyEaAr684c+vw5H0Bsb5362GquKGZ60 KMMw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k63-20020a636f42000000b004bbfa29fb4asi6361929pgc.563.2023.01.25.09.17.05; Wed, 25 Jan 2023 09:17:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236182AbjAYROW (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236084AbjAYRN7 (ORCPT ); Wed, 25 Jan 2023 12:13:59 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC355EB59 for ; Wed, 25 Jan 2023 09:13:54 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B0A0BB81B60 for ; Wed, 25 Jan 2023 17:13:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 366E7C433B4; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjLA-004Nu9-0u; Wed, 25 Jan 2023 12:13:40 -0500 Message-ID: <20230125171340.102429648@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:13:01 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Ross Zwisler , Ching-lin Yu Subject: [for-next][PATCH 09/12] tracing/histogram: Document variable stacktrace References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015664620490105?= X-GMAIL-MSGID: =?utf-8?q?1756015664620490105?= From: "Steven Rostedt (Google)" Add a little documentation (and a useful example) of how a stacktrace can be used within a histogram variable and synthetic event. Link: https://lkml.kernel.org/r/20230117152236.320181354@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Ross Zwisler Cc: Ching-lin Yu Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/histogram.rst | 129 ++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) diff --git a/Documentation/trace/histogram.rst b/Documentation/trace/histogram.rst index f95459aa984f..5c391328b9bb 100644 --- a/Documentation/trace/histogram.rst +++ b/Documentation/trace/histogram.rst @@ -81,6 +81,7 @@ Documentation written by Tom Zanussi .usecs display a common_timestamp in microseconds .percent display a number of percentage value .graph display a bar-graph of a value + .stacktrace display as a stacktrace (must by a long[] type) ============= ================================================= Note that in general the semantics of a given field aren't @@ -1786,6 +1787,8 @@ or assigned to a variable and referenced in a subsequent expression:: # echo 'hist:keys=next_pid:us_per_sec=1000000 ...' >> event/trigger # echo 'hist:keys=next_pid:timestamp_secs=common_timestamp/$us_per_sec ...' >> event/trigger +Variables can even hold stacktraces, which are useful with synthetic events. + 2.2.2 Synthetic Events ---------------------- @@ -1940,6 +1943,132 @@ the ".buckets" modifier and specify a size (in this case groups of 10). Entries: 16 Dropped: 0 +To save stacktraces, create a synthetic event with a field of type "unsigned long[]" +or even just "long[]". For example, to see how long a task is blocked in an +uninterruptible state: + + # cd /sys/kernel/tracing + # echo 's:block_lat pid_t pid; u64 delta; unsigned long[] stack;' > dynamic_events + # echo 'hist:keys=next_pid:ts=common_timestamp.usecs,st=stacktrace if prev_state == 2' >> events/sched/sched_switch/trigger + # echo 'hist:keys=prev_pid:delta=common_timestamp.usecs-$ts,s=$st:onmax($delta).trace(block_lat,prev_pid,$delta,$s)' >> events/sched/sched_switch/trigger + # echo 1 > events/synthetic/block_lat/enable + # cat trace + + # tracer: nop + # + # entries-in-buffer/entries-written: 2/2 #P:8 + # + # _-----=> irqs-off/BH-disabled + # / _----=> need-resched + # | / _---=> hardirq/softirq + # || / _--=> preempt-depth + # ||| / _-=> migrate-disable + # |||| / delay + # TASK-PID CPU# ||||| TIMESTAMP FUNCTION + # | | | ||||| | | + -0 [005] d..4. 521.164922: block_lat: pid=0 delta=8322 stack=STACK: + => __schedule+0x448/0x7b0 + => schedule+0x5a/0xb0 + => io_schedule+0x42/0x70 + => bit_wait_io+0xd/0x60 + => __wait_on_bit+0x4b/0x140 + => out_of_line_wait_on_bit+0x91/0xb0 + => jbd2_journal_commit_transaction+0x1679/0x1a70 + => kjournald2+0xa9/0x280 + => kthread+0xe9/0x110 + => ret_from_fork+0x2c/0x50 + + <...>-2 [004] d..4. 525.184257: block_lat: pid=2 delta=76 stack=STACK: + => __schedule+0x448/0x7b0 + => schedule+0x5a/0xb0 + => schedule_timeout+0x11a/0x150 + => wait_for_completion_killable+0x144/0x1f0 + => __kthread_create_on_node+0xe7/0x1e0 + => kthread_create_on_node+0x51/0x70 + => create_worker+0xcc/0x1a0 + => worker_thread+0x2ad/0x380 + => kthread+0xe9/0x110 + => ret_from_fork+0x2c/0x50 + +A synthetic event that has a stacktrace field may use it as a key in histogram: + + # echo 'hist:delta.buckets=100,stack.stacktrace:sort=delta' > events/synthetic/block_lat/trigger + # cat events/synthetic/block_lat/hist + + # event histogram + # + # trigger info: hist:keys=delta.buckets=100,stacktrace:vals=hitcount:sort=delta.buckets=100:size=2048 [active] + # + + { delta: ~ 0-99, stacktrace: + event_hist_trigger+0x464/0x480 + event_triggers_call+0x52/0xe0 + trace_event_buffer_commit+0x193/0x250 + trace_event_raw_event_sched_switch+0xfc/0x150 + __traceiter_sched_switch+0x41/0x60 + __schedule+0x448/0x7b0 + schedule_idle+0x26/0x40 + cpu_startup_entry+0x19/0x20 + start_secondary+0xed/0xf0 + secondary_startup_64_no_verify+0xe0/0xeb + } hitcount: 6 + { delta: ~ 0-99, stacktrace: + event_hist_trigger+0x464/0x480 + event_triggers_call+0x52/0xe0 + trace_event_buffer_commit+0x193/0x250 + trace_event_raw_event_sched_switch+0xfc/0x150 + __traceiter_sched_switch+0x41/0x60 + __schedule+0x448/0x7b0 + schedule_idle+0x26/0x40 + cpu_startup_entry+0x19/0x20 + __pfx_kernel_init+0x0/0x10 + arch_call_rest_init+0xa/0x24 + start_kernel+0x964/0x98d + secondary_startup_64_no_verify+0xe0/0xeb + } hitcount: 3 + { delta: ~ 0-99, stacktrace: + event_hist_trigger+0x464/0x480 + event_triggers_call+0x52/0xe0 + trace_event_buffer_commit+0x193/0x250 + trace_event_raw_event_sched_switch+0xfc/0x150 + __traceiter_sched_switch+0x41/0x60 + __schedule+0x448/0x7b0 + schedule+0x5a/0xb0 + worker_thread+0xaf/0x380 + kthread+0xe9/0x110 + ret_from_fork+0x2c/0x50 + } hitcount: 1 + { delta: ~ 100-199, stacktrace: + event_hist_trigger+0x464/0x480 + event_triggers_call+0x52/0xe0 + trace_event_buffer_commit+0x193/0x250 + trace_event_raw_event_sched_switch+0xfc/0x150 + __traceiter_sched_switch+0x41/0x60 + __schedule+0x448/0x7b0 + schedule_idle+0x26/0x40 + cpu_startup_entry+0x19/0x20 + start_secondary+0xed/0xf0 + secondary_startup_64_no_verify+0xe0/0xeb + } hitcount: 15 + [..] + { delta: ~ 8500-8599, stacktrace: + event_hist_trigger+0x464/0x480 + event_triggers_call+0x52/0xe0 + trace_event_buffer_commit+0x193/0x250 + trace_event_raw_event_sched_switch+0xfc/0x150 + __traceiter_sched_switch+0x41/0x60 + __schedule+0x448/0x7b0 + schedule_idle+0x26/0x40 + cpu_startup_entry+0x19/0x20 + start_secondary+0xed/0xf0 + secondary_startup_64_no_verify+0xe0/0xeb + } hitcount: 1 + + Totals: + Hits: 89 + Entries: 11 + Dropped: 0 + 2.2.3 Hist trigger 'handlers' and 'actions' ------------------------------------------- From patchwork Wed Jan 25 17:13:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48230 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp387623wrn; Wed, 25 Jan 2023 09:14:58 -0800 (PST) X-Google-Smtp-Source: AK7set9wpy0kq+up711XaVJ/PSetz67vaxNdEiJgbSfak9tFiUs7QUS0coqpE6e1q9herLsVoV9z X-Received: by 2002:a17:906:fb09:b0:878:4bda:200e with SMTP id lz9-20020a170906fb0900b008784bda200emr807198ejb.4.1674666898193; Wed, 25 Jan 2023 09:14:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666898; cv=none; d=google.com; s=arc-20160816; b=RV/y0itigURx4x1AXFcoJAQYv9oR2zpRUhmB/bF1ZoVob0ZkclFjktPgBc/57tlEMg ovGkAKsIEkKqTtapVKnzuEQbD/MvrJt8GzdJjcqFdd8hlUuOe8I+ewEqnfi+i8yo7z7U Twb+p7fFLXB+fOlBzwRN6zqqlpfV4yDcMx3EN0RV8rXmrqVb34+A5SlrUTn9Cx/c8nTj 2pLgkJ/PNSVQIjb5PuPdl1bVJgn2byEQawudHcxYaXUpG/M3+Xb1KaPE2pCFXsYbvfVr wwA7oeeLYfMJwLWELE8mEnlKbtoi7BJwZY7PjI5z4AEllZIvlaKABQfFz8B2Jk+NJz9H sqQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=Nk3OnkGZR0RmpIeYeKsXNevU51KIiBWzMCX9uFV6enM=; b=Q/KD9NDNFdr1sBOp7Che4aaRzjUkujpWC1BksQsbHPI++JZOPx/GloAEpBn+azJzmF jiDn5xqzk6ISiB6gb6cJ272lW0U3H7kgcaTzGRQbd4EsmJgszsWY5TCyJVUKZNuMjVvM x+i9bkWVdmfLAJT5YAsGyOYFlKgQSmjE1UPmKRjrzqfE29KYSWV6MzpqgqZ24o6cKnuc td77rdtbsJ8TxzFnB7vGrTactwywjTje1SZnM406nD5ZGk6tiuZTEUnftmUKBFb92iof eMDdxw9qITrW9UO8Oc9nu52VnqZYAWtJe+z93mcz2ehcdbVzpHMo5U6hZZnhxMhCXR0u Rxaw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fd15-20020a1709072a0f00b00877e53cad6dsi6126373ejc.251.2023.01.25.09.14.33; Wed, 25 Jan 2023 09:14:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236130AbjAYROI (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235356AbjAYRNx (ORCPT ); Wed, 25 Jan 2023 12:13:53 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66109976C for ; Wed, 25 Jan 2023 09:13:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F3FC16158C for ; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6698CC433AC; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjLA-004Nuh-1a; Wed, 25 Jan 2023 12:13:40 -0500 Message-ID: <20230125171340.301718233@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:13:02 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Tom Zanussi , Ross Zwisler , Ching-lin Yu Subject: [for-next][PATCH 10/12] tracing/histogram: Add simple tests for stacktrace usage of synthetic events References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015517539527881?= X-GMAIL-MSGID: =?utf-8?q?1756015517539527881?= From: "Steven Rostedt (Google)" Update the selftests to include a test of passing a stacktrace between the events of a synthetic event. Link: https://lkml.kernel.org/r/20230117152236.475439286@goodmis.org Cc: Masami Hiramatsu Cc: Andrew Morton Cc: Tom Zanussi Cc: Ross Zwisler Cc: Ching-lin Yu Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 2 +- .../trigger-synthetic-event-stack.tc | 24 +++++++++++++++++++ .../trigger-synthetic-event-syntax.tc | 6 +++++ 3 files changed, 31 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-stack.tc diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 78ed5f1baa8c..b90eecd27dfc 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -5757,7 +5757,7 @@ static const char readme_msg[] = #ifdef CONFIG_SYNTH_EVENTS " events/synthetic_events\t- Create/append/remove/show synthetic events\n" "\t Write into this file to define/undefine new synthetic events.\n" - "\t example: echo 'myevent u64 lat; char name[]' >> synthetic_events\n" + "\t example: echo 'myevent u64 lat; char name[]; long[] stack' >> synthetic_events\n" #endif #endif ; diff --git a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-stack.tc b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-stack.tc new file mode 100644 index 000000000000..755dbe94ccf4 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-stack.tc @@ -0,0 +1,24 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: event trigger - test inter-event histogram trigger trace action with dynamic string param +# requires: set_event synthetic_events events/sched/sched_process_exec/hist "long[]' >> synthetic_events":README + +fail() { #msg + echo $1 + exit_fail +} + +echo "Test create synthetic event with stack" + + +echo 's:wake_lat pid_t pid; u64 delta; unsigned long[] stack;' > dynamic_events +echo 'hist:keys=next_pid:ts=common_timestamp.usecs,st=stacktrace if prev_state == 1||prev_state == 2' >> events/sched/sched_switch/trigger +echo 'hist:keys=prev_pid:delta=common_timestamp.usecs-$ts,s=$st:onmax($delta).trace(wake_lat,prev_pid,$delta,$s)' >> events/sched/sched_switch/trigger +echo 1 > events/synthetic/wake_lat/enable +sleep 1 + +if ! grep -q "=>.*sched" trace; then + fail "Failed to create synthetic event with stack" +fi + +exit 0 diff --git a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-syntax.tc b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-syntax.tc index 2968cdc7df30..366f1f3ad906 100644 --- a/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-syntax.tc +++ b/tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-syntax.tc @@ -70,6 +70,12 @@ grep "myevent[[:space:]]unsigned long var" synthetic_events echo "myevent char var[10]" > synthetic_events grep "myevent[[:space:]]char\[10\] var" synthetic_events +if grep -q 'long\[\]' README; then + # test stacktrace type + echo "myevent unsigned long[] var" > synthetic_events + grep "myevent[[:space:]]unsigned long\[\] var" synthetic_events +fi + do_reset exit 0 From patchwork Wed Jan 25 17:13:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48233 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp388127wrn; Wed, 25 Jan 2023 09:15:54 -0800 (PST) X-Google-Smtp-Source: AMrXdXuWLsvKfa6RJvebC3/Twe8GCi56TpKkddELMv2M9jW7Nr/+iUQ0p4kUxcsblPuW6BewC9rU X-Received: by 2002:a05:6a20:a9a0:b0:b5:fc85:d864 with SMTP id cc32-20020a056a20a9a000b000b5fc85d864mr54226771pzb.32.1674666953851; Wed, 25 Jan 2023 09:15:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674666953; cv=none; d=google.com; s=arc-20160816; b=DQLyc3a53KLh3G6hplLNt8lj7xpVUqUYJ2CagXiBBUilC7/YkqpMVdeqn9R/lEh3xt lPzpD9GqJHyt6EbzsAV5rkSlTcxOOcZUyRjyI7jfu0ofj0PdAVtpcauLf8l19XJ9+zdr nMFAwDCCLO/+HsTNt6/MOkkHThFl2H1n/88XMG30niKI8kJU+4hBz4GdbFne4BRvfxPm daCL4QcKXjDbjGuvEYogymMYwMvUiIpfztgg59yAkYhPSVbVKd1yerFYL+kWm54gUluD OKCPnQD7K+AOys4M8204Mpf3oR/SkQ/Mv6sdFyzsyvgolh593x8NG9WO994MdTEBYzOz VdbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=MQGaiQMCRHx4WBKXsdRaaKANcIyUeNBbgZZAp/lKsR8=; b=RfchEJs79jBajpIx8wsPjfW/yLbGKaZCK0W3BKowBiMLmBOtswf57VLqj0L9GYA42A YRZviYhj4sQb66u2G5oxso/OK359ea6kAqbBLYSI0fGTCcX8xntzIPIOIbjOciTlJvoD O3n/3xS09mxIow6N04IZFSk/e7jWn022RfWErVIJuPccCrvKfMaQRuZw7v7y6twmNSl8 DukaCQLiKZcrS6dexYboc7fctKqsgASVen/DGloa4HLaL/YuYg+M/yoplyZtwbUgh7Es X7N4eD46GvCeJwkzpW8C7TVZesmJQF1Z3ctED6w6yfwpAZXpVJZPsJlKyTBEhG/bBsVf 7DUw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u15-20020a63b54f000000b004a173aaa75esi5674039pgo.548.2023.01.25.09.15.41; Wed, 25 Jan 2023 09:15:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235644AbjAYRNz (ORCPT + 99 others); Wed, 25 Jan 2023 12:13:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235172AbjAYRNw (ORCPT ); Wed, 25 Jan 2023 12:13:52 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E92565BE for ; Wed, 25 Jan 2023 09:13:42 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CD24461586 for ; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 92397C43443; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjLA-004NvF-2F; Wed, 25 Jan 2023 12:13:40 -0500 Message-ID: <20230125171340.510349637@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:13:03 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , Frederic Weisbecker , Arnaldo Carvalho de Melo , "Peter Zijlstra (Intel)" , Linyu Yuan Subject: [for-next][PATCH 11/12] perf/tracing: Use stage6 of tracing to not duplicate macros References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015576005340657?= X-GMAIL-MSGID: =?utf-8?q?1756015576005340657?= From: "Steven Rostedt (Google)" The perf events are created by the same macro magic as tracefs trace events are. But to hook into perf, it has its own code. It duplicates many of the same macros as the tracefs macros and this is an issue because it misses bug fixes as well as any new enhancements that come with the other trace macros. As the trace macros have been put into their own staging files, have perf take advantage of this and use the tracefs stage 6 macros that the "fast assign" portion of the trace event macro uses. Link: https://lkml.kernel.org/r/20230124202515.716458410@goodmis.org Link: https://lore.kernel.org/lkml/1671181385-5719-1-git-send-email-quic_linyyuan@quicinc.com/ Cc: Frederic Weisbecker Cc: Arnaldo Carvalho de Melo Acked-by: Peter Zijlstra (Intel) Reported-by: Linyu Yuan Signed-off-by: Steven Rostedt (Google) --- include/trace/perf.h | 46 +------------------- include/trace/stages/stage6_event_callback.h | 3 ++ 2 files changed, 4 insertions(+), 45 deletions(-) diff --git a/include/trace/perf.h b/include/trace/perf.h index 8f3bf1e17707..2c11181c82e0 100644 --- a/include/trace/perf.h +++ b/include/trace/perf.h @@ -4,51 +4,7 @@ #ifdef CONFIG_PERF_EVENTS -#undef __entry -#define __entry entry - -#undef __get_dynamic_array -#define __get_dynamic_array(field) \ - ((void *)__entry + (__entry->__data_loc_##field & 0xffff)) - -#undef __get_dynamic_array_len -#define __get_dynamic_array_len(field) \ - ((__entry->__data_loc_##field >> 16) & 0xffff) - -#undef __get_str -#define __get_str(field) ((char *)__get_dynamic_array(field)) - -#undef __get_bitmask -#define __get_bitmask(field) (char *)__get_dynamic_array(field) - -#undef __get_cpumask -#define __get_cpumask(field) (char *)__get_dynamic_array(field) - -#undef __get_sockaddr -#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field)) - -#undef __get_rel_dynamic_array -#define __get_rel_dynamic_array(field) \ - ((void *)__entry + \ - offsetof(typeof(*__entry), __rel_loc_##field) + \ - sizeof(__entry->__rel_loc_##field) + \ - (__entry->__rel_loc_##field & 0xffff)) - -#undef __get_rel_dynamic_array_len -#define __get_rel_dynamic_array_len(field) \ - ((__entry->__rel_loc_##field >> 16) & 0xffff) - -#undef __get_rel_str -#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field)) - -#undef __get_rel_bitmask -#define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field) - -#undef __get_rel_cpumask -#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field) - -#undef __get_rel_sockaddr -#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field)) +#include "stages/stage6_event_callback.h" #undef __perf_count #define __perf_count(c) (__count = (c)) diff --git a/include/trace/stages/stage6_event_callback.h b/include/trace/stages/stage6_event_callback.h index 49c32394b53f..919b1a4da980 100644 --- a/include/trace/stages/stage6_event_callback.h +++ b/include/trace/stages/stage6_event_callback.h @@ -2,6 +2,9 @@ /* Stage 6 definitions for creating trace events */ +/* Reuse some of the stage 3 macros */ +#include "stage3_trace_output.h" + #undef __entry #define __entry entry From patchwork Wed Jan 25 17:13:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 48236 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp388634wrn; Wed, 25 Jan 2023 09:16:57 -0800 (PST) X-Google-Smtp-Source: AMrXdXsklcBvSDOhPRXktHYw9vXbCdtn1UDxRjH1wwOYZyF7dZOLz9qQjPqXV3J7+9dvasEgcIXZ X-Received: by 2002:a62:61c4:0:b0:581:a8dc:8f95 with SMTP id v187-20020a6261c4000000b00581a8dc8f95mr31705653pfb.12.1674667016866; Wed, 25 Jan 2023 09:16:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674667016; cv=none; d=google.com; s=arc-20160816; b=nCPalAmSYyFOF5bMRqZhE7uVjKz/CG5ZZV7674L0lHrpmI5wcnbnwNkf1tXf+CfZbV TLlFBnIDJ5zssdeR8H29DyWqo921sHaG7NlKeMh92O3EIN6lO+G3fuFMiRcVKj96io3d c7EZiPsfTvP5vaVSGlzUVrGexrqrLjmfy6P+hm/5xwUbPQfuBN2Mwzcj04Sxtor9YTUi qvijHpGBdWpJ+2oWEqKoJrjW/c2VYSkbFTWCj5pmkaSOT4OkCxT1cvk7e2QvrRpGxqRw vcS/Jnjd7gslTpxBfjj7T0THGF4tKtNQMD6lrH7L2UQZw0QNlYVAAyPrVgAKmmJEnXQ1 SUug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=PdsvhDYehupsoqKuXfnv31Nriyc2Sqdvz4DL7X6EFhE=; b=Sm1i6CO4eV3I6BP3ETokLytfYDqOxgiCqkSPmw4r1E5hcTpMC0WfYJxJJIOndbToXq TMt3J1c2kxeuZx3i5oyp3tlo+DjfDutpfV8YPVW1rXJpAKfQBN8Y+fkCDJxjKfJXQwZz zEq81iYZCp/CHhMM/QiqhfgeWAtdzDypnrl9aPiup3pGfDP+Nq79gDn+D5vN7GdDMudX JcZ3h9XzIUW38JKPVFo7p5bjMgkJ+jf9twMJaSveACR26SvlNRURxmMB58erDqCuSVjS tLs1pW28ENy4ajEkdBMuDderPXhAV4HWW8KKvzZ6/Nupc3yRZO4oaBFI4saLRIuYA7BG Xmcg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i14-20020aa78b4e000000b0058e1b16c4dcsi5888253pfd.142.2023.01.25.09.16.44; Wed, 25 Jan 2023 09:16:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236177AbjAYROV (ORCPT + 99 others); Wed, 25 Jan 2023 12:14:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236059AbjAYRN6 (ORCPT ); Wed, 25 Jan 2023 12:13:58 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86B27D7; Wed, 25 Jan 2023 09:13:52 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1486E6159C; Wed, 25 Jan 2023 17:13:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DEDCCC433EF; Wed, 25 Jan 2023 17:13:41 +0000 (UTC) Received: from rostedt by gandalf.local.home with local (Exim 4.96) (envelope-from ) id 1pKjLA-004Nvn-2v; Wed, 25 Jan 2023 12:13:40 -0500 Message-ID: <20230125171340.715046734@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 25 Jan 2023 12:13:04 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Andrew Morton , bpf@vger.kernel.org, Peter Zijlstra , Alexei Starovoitov , Daniel Borkmann , Linyu Yuan Subject: [for-next][PATCH 12/12] bpf/tracing: Use stage6 of tracing to not duplicate macros References: <20230125171252.431857411@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756015641944146754?= X-GMAIL-MSGID: =?utf-8?q?1756015641944146754?= From: "Steven Rostedt (Google)" The bpf events are created by the same macro magic as tracefs trace events are. But to hook into bpf, it has its own code. It duplicates many of the same macros as the tracefs macros and this is an issue because it misses bug fixes as well as any new enhancements that come with the other trace macros. As the trace macros have been put into their own staging files, have bpf take advantage of this and use the tracefs stage 6 macros that the "fast ssign" portion of the trace event macro uses. Link: https://lkml.kernel.org/r/20230124202515.873075730@goodmis.org Link: https://lore.kernel.org/lkml/1671181385-5719-1-git-send-email-quic_linyyuan@quicinc.com/ Cc: bpf@vger.kernel.org Cc: Peter Zijlstra Cc: Alexei Starovoitov Cc: Daniel Borkmann Acked-by: Alexei Starovoitov Reported-by: Linyu Yuan Signed-off-by: Steven Rostedt (Google) --- include/trace/bpf_probe.h | 45 +-------------------------------------- 1 file changed, 1 insertion(+), 44 deletions(-) diff --git a/include/trace/bpf_probe.h b/include/trace/bpf_probe.h index 155c495b89ea..1f7fc1fc590c 100644 --- a/include/trace/bpf_probe.h +++ b/include/trace/bpf_probe.h @@ -4,50 +4,7 @@ #ifdef CONFIG_BPF_EVENTS -#undef __entry -#define __entry entry - -#undef __get_dynamic_array -#define __get_dynamic_array(field) \ - ((void *)__entry + (__entry->__data_loc_##field & 0xffff)) - -#undef __get_dynamic_array_len -#define __get_dynamic_array_len(field) \ - ((__entry->__data_loc_##field >> 16) & 0xffff) - -#undef __get_str -#define __get_str(field) ((char *)__get_dynamic_array(field)) - -#undef __get_bitmask -#define __get_bitmask(field) (char *)__get_dynamic_array(field) - -#undef __get_cpumask -#define __get_cpumask(field) (char *)__get_dynamic_array(field) - -#undef __get_sockaddr -#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field)) - -#undef __get_rel_dynamic_array -#define __get_rel_dynamic_array(field) \ - ((void *)(&__entry->__rel_loc_##field) + \ - sizeof(__entry->__rel_loc_##field) + \ - (__entry->__rel_loc_##field & 0xffff)) - -#undef __get_rel_dynamic_array_len -#define __get_rel_dynamic_array_len(field) \ - ((__entry->__rel_loc_##field >> 16) & 0xffff) - -#undef __get_rel_str -#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field)) - -#undef __get_rel_bitmask -#define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field) - -#undef __get_rel_cpumask -#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field) - -#undef __get_rel_sockaddr -#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field)) +#include "stages/stage6_event_callback.h" #undef __perf_count #define __perf_count(c) (c)