From patchwork Fri Dec 9 22:05:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Bristot de Oliveira X-Patchwork-Id: 31940 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1014514wrr; Fri, 9 Dec 2022 14:10:36 -0800 (PST) X-Google-Smtp-Source: AA0mqf6GeH7rp+qhaTzIWvQoe2ChnVQE+9N0gwXhB4MiMD/UzLyGgff5iFMR015TySvHk6jG0klA X-Received: by 2002:a17:902:7c98:b0:188:5255:66b9 with SMTP id y24-20020a1709027c9800b00188525566b9mr7842062pll.15.1670623836554; Fri, 09 Dec 2022 14:10:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670623836; cv=none; d=google.com; s=arc-20160816; b=spSc1+SBzSTYfeZr63W+w02Y0rbd2b5Ic0HqJWv7LdYQ0VFSjl4h72ulwJm7jJFzKp 2G3J5OM6dkEVOXNrrcNtRW+a//Kg6tsu1/+yYLTpT4RBWwGxyqvyauvh9ACOWo6fVmKq zNmXXBpdaAn4uR7/VQYfa0fCkJhzaL2wR+Wzdo5MZiuKyZzpjsppsUjGXC6dmtdCgM2I nSNq50O7NSVYX4w9Bkqa48JYOoEv0gjcS/hDYcSMdTei1xihPyC9pEJBPgqO9RavNzX8 F/UPumn+WZCNP1GLjVBXE9iFMRIxCJfwtRIKYMqsnXEkMR9+JLZzOKikqcBTsnfo+uew 1qYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aWg9iCmVtcgU19BjwQr7qHc7qmpPSCGL2/gExDLvHZM=; b=AM3j6c2lQLW+Ug7Y9Z/BqmL7I0huyBa01BSaeFl7D9hlOUEcczvKEz1BwR77VhnW2C uLigLGkOvsjDSIdDcyi7ekL4ZTQnJll7ck8JJy5td8X97NncJt5TEgSw+tINdxkvVneE BhHgTACbMytD0uZ9mS2sKJ498tdTxc/9P65ltj7tXyD5MbAeDiPYwkjdHUvrCSvcToMe SXt2GQxNTl3Hr9dcB5hzeUyLZkcLjqBp8yJRqY6Rb+MTkfpWMgPbRwXmugqwrTgKPEu6 /lulKtR/hL5Eij73cO1IJ7nMRgLdfwxnjJPIwkiR/n0aSL4XeT21N8GLnwqz76JfKJT0 4J0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ObcHETnV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y10-20020a17090322ca00b00188fead2329si2903506plg.135.2022.12.09.14.10.23; Fri, 09 Dec 2022 14:10:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ObcHETnV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229733AbiLIWHD (ORCPT + 99 others); Fri, 9 Dec 2022 17:07:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229805AbiLIWGi (ORCPT ); Fri, 9 Dec 2022 17:06:38 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A80AA5DE8; Fri, 9 Dec 2022 14:06:07 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2024FB8293F; Fri, 9 Dec 2022 22:06:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E5C5C433F0; Fri, 9 Dec 2022 22:06:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1670623564; bh=UofQ2Om5TKKPZw19SG+8wKH8TntN+ojXWrOiFhpkAis=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ObcHETnV404P8FdK6dcZ3k8cwnJHWj53iKrb8qBFv+KV9vFx/MNCY2VlnIl83fRKX GkKtSBij7bMDEosyNGwu6dQD9rFbWG8/zfZ9xnYxm3uSF+DjZ1AimX5Pf1ABAConF9 uoaqG0+vlQqK37Vmywu34eMmUPsWdgcX6EBf3ZK/DWcZu1niXdTyKMeSB6pkrCsbNj yC2M4FAI3ea5riv5qgcHam52q6lcjNFL0KfpZ8zTUH9mzcz5duwFKhmmkj7nIorpZL O/rDF/8H8ALoiCLsi17zMbOPm/R9IV5fXqIOu11020UPUk6hylNouVg7cHsdsLim3t 0yLa1Eot8ov1w== From: Daniel Bristot de Oliveira To: Daniel Bristot de Oliveira , Steven Rostedt Cc: Masami Hiramatsu , Jonathan Corbet , Juri Lelli , Clark Williams , Bagas Sanjaya , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V5 1/3] tracing/osnoise: Add PANIC_ON_STOP option Date: Fri, 9 Dec 2022 23:05:52 +0100 Message-Id: <249ce4287c6725543e6db845a6e0df621dc67db5.1670623111.git.bristot@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751776059958962096?= X-GMAIL-MSGID: =?utf-8?q?1751776059958962096?= Often the latency observed in a CPU is not caused by the work being done in the CPU itself, but by work done on another CPU that causes the hardware to stall all CPUs. In this case, it is interesting to know what is happening on ALL CPUs, and the best way to do this is via crash dump analysis. Add the PANIC_ON_STOP option to osnoise/timerlat tracers. The default behavior is having this option off. When enabled by the user, the system will panic after hitting a stop tracing condition. This option was motivated by a real scenario that Juri Lelli and I were debugging. Cc: Juri Lelli Cc: Daniel Bristot de Oliveira Cc: Steven Rostedt Cc: Masami Hiramatsu Cc: Jonathan Corbet Signed-off-by: Daniel Bristot de Oliveira --- kernel/trace/trace_osnoise.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index 3f10dd1f2f1c..801eba0b5cf8 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -54,10 +54,11 @@ enum osnoise_options_index { OSN_DEFAULTS = 0, OSN_WORKLOAD, + OSN_PANIC_ON_STOP, OSN_MAX }; -static const char * const osnoise_options_str[OSN_MAX] = { "DEFAULTS", "OSNOISE_WORKLOAD" }; +static const char * const osnoise_options_str[OSN_MAX] = { "DEFAULTS", "OSNOISE_WORKLOAD", "PANIC_ON_STOP" }; #define OSN_DEFAULT_OPTIONS 0x2 unsigned long osnoise_options = OSN_DEFAULT_OPTIONS; @@ -1270,6 +1271,9 @@ static __always_inline void osnoise_stop_tracing(void) trace_array_printk_buf(tr->array_buffer.buffer, _THIS_IP_, "stop tracing hit on cpu %d\n", smp_processor_id()); + if (test_bit(OSN_PANIC_ON_STOP, &osnoise_options)) + panic("tracer hit stop condition on CPU %d\n", smp_processor_id()); + tracer_tracing_off(tr); } rcu_read_unlock();