From patchwork Sat Aug 5 03:38:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zheng Yejian X-Patchwork-Id: 131438 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp269004vqr; Fri, 4 Aug 2023 21:57:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG/dAxupR5vBSW4ZQpRh/jmHZYUMt7k/QxUiO+CRWRkmHbi77GdexmDtbDROkT5J5JGLiyN X-Received: by 2002:a17:90a:bb8d:b0:268:7ec:51ae with SMTP id v13-20020a17090abb8d00b0026807ec51aemr3104466pjr.41.1691211471906; Fri, 04 Aug 2023 21:57:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691211471; cv=none; d=google.com; s=arc-20160816; b=dl6oAxplOaDUZD5ppdd17DRWp/6u98RC6szL7g7SsfyKHDhnLDfmIIGAm3vnvcZjW8 dpGdcN+Srq1UipE9O4f2vHt/37CJ83rHu8LXuFPpyYubBc7rrHaQg968PZH1AqFFleXr Zxgm66oHWWdOrJY47lmVbZTHNGmaxXhtvJZTBvJtw+hEAiux9ySKhPoI+OWzZcVwc/A5 l6pAh5Jxl5tvgaMSf1Bxazn4EG15LTUq/X+5EOi87T7juQ5olgveX9tf2LWqDNv6C1E9 S/1ObLkdtlQSvHTGC/4jwTg9m7r0apwGxZ8H0vi3GeQZ5l3itNIDho1P+WJRxE0vCxgI O5WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=GWP+Vo3r9gxtoyG1kaY32pxBNjRVbO3+kSQtaxDt5Xc=; fh=6BCBHd+wd9sM5I7f3nZwXI5uSly3s62mQle808OwsEg=; b=DeEOZbzr3lQUhX3zoZAm7jIbAmt/qEzdQe+4v+LZ8VKsgJuC2Cw54ivG5HYF0kDNBv UdWSkXaAMt6HFNMiDvSCJIglgBDeSMfmM+dNhMK3gJDHVcS29Pg8b2fVnzEdviCC7Blk bEZqvnTLDiM91HKnW2LLoRV05Xc0j+bs+XdjaT+Bu8gp7ijdj9HsuGW255AOpWciaq0d 6ROyWxDvSntcZw55XXcRJ2hRQg/ewJXCrBO3We3Dghuhrgi0zR1j6GlRgY96Dvv6i+j8 VN+Hm5WuxlqqSFeod+/dkiGzw3KtNllf7OKIXKnpmF/pGmjQh/bECi4dMFYCjScELAJs 79aA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b3-20020a17090acc0300b0024e47fae466si2834236pju.180.2023.08.04.21.57.37; Fri, 04 Aug 2023 21:57:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229717AbjHEDjF (ORCPT + 99 others); Fri, 4 Aug 2023 23:39:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229652AbjHEDi4 (ORCPT ); Fri, 4 Aug 2023 23:38:56 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE1F24ED8; Fri, 4 Aug 2023 20:38:54 -0700 (PDT) Received: from dggpeml500012.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RHpDH703GzVk4V; Sat, 5 Aug 2023 11:37:03 +0800 (CST) Received: from localhost.localdomain (10.67.175.61) by dggpeml500012.china.huawei.com (7.185.36.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Sat, 5 Aug 2023 11:38:52 +0800 From: Zheng Yejian To: , , , CC: , , Subject: [PATCH v2 1/2] tracing: Fix cpu buffers unavailable due to 'record_disabled' messed Date: Sat, 5 Aug 2023 11:38:15 +0800 Message-ID: <20230805033816.3284594-2-zhengyejian1@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230805033816.3284594-1-zhengyejian1@huawei.com> References: <20230805033816.3284594-1-zhengyejian1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.175.61] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500012.china.huawei.com (7.185.36.15) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773363760430203606 X-GMAIL-MSGID: 1773363760430203606 Trace ring buffer can no longer record anything after executing following commands at the shell prompt: # cd /sys/kernel/tracing # cat tracing_cpumask fff # echo 0 > tracing_cpumask # echo 1 > snapshot # echo fff > tracing_cpumask # echo 1 > tracing_on # echo "hello world" > trace_marker -bash: echo: write error: Bad file descriptor The root cause is that: 1. After `echo 0 > tracing_cpumask`, 'record_disabled' of cpu buffers in 'tr->array_buffer.buffer' became 1 (see tracing_set_cpumask()); 2. After `echo 1 > snapshot`, 'tr->array_buffer.buffer' is swapped with 'tr->max_buffer.buffer', then the 'record_disabled' became 0 (see update_max_tr()); 3. After `echo fff > tracing_cpumask`, the 'record_disabled' become -1; Then array_buffer and max_buffer are both unavailable due to value of 'record_disabled' is not 0. To fix it, enable or disable both array_buffer and max_buffer at the same time in tracing_set_cpumask(). Fixes: 71babb2705e2 ("tracing: change CPU ring buffer state from tracing_cpumask") Signed-off-by: Zheng Yejian --- kernel/trace/trace.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index b8870078ef58..b0e8eb6ea8ac 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -5277,11 +5277,17 @@ int tracing_set_cpumask(struct trace_array *tr, !cpumask_test_cpu(cpu, tracing_cpumask_new)) { atomic_inc(&per_cpu_ptr(tr->array_buffer.data, cpu)->disabled); ring_buffer_record_disable_cpu(tr->array_buffer.buffer, cpu); +#ifdef CONFIG_TRACER_MAX_TRACE + ring_buffer_record_disable_cpu(tr->max_buffer.buffer, cpu); +#endif } if (!cpumask_test_cpu(cpu, tr->tracing_cpumask) && cpumask_test_cpu(cpu, tracing_cpumask_new)) { atomic_dec(&per_cpu_ptr(tr->array_buffer.data, cpu)->disabled); ring_buffer_record_enable_cpu(tr->array_buffer.buffer, cpu); +#ifdef CONFIG_TRACER_MAX_TRACE + ring_buffer_record_enable_cpu(tr->max_buffer.buffer, cpu); +#endif } } arch_spin_unlock(&tr->max_lock); From patchwork Sat Aug 5 03:38:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zheng Yejian X-Patchwork-Id: 131435 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp265978vqr; Fri, 4 Aug 2023 21:46:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEv5ERgNUpfDUcIf3+mJIiFTQEtiPk9tF0nVEVz5jr6dk1NqCVYHYpb89B4hIC0d9Eo9eCf X-Received: by 2002:a05:6a20:4287:b0:134:1011:8582 with SMTP id o7-20020a056a20428700b0013410118582mr4786362pzj.47.1691210800529; Fri, 04 Aug 2023 21:46:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691210800; cv=none; d=google.com; s=arc-20160816; b=i7c9TBDwI8/LDrJMuMbipTGANNshIBn/tZc8+KVQdBvvC7p3J1qzPZobUnAipyTV2P 4w3u9cXd3rGQIMKWpc9BaGgPpKhymRSk8ypf8DZa6vS/byXc5tD0ceQIw8yzcHr+NiT5 F0jbdYGaU2Rmhpz/au9I4hQ1+49pmgKuXXNIpnd5HCnfJXux3AgWExWeKP753mhmryiA 8kd4oq9yTnVmV7DMZLR9NtFgAgtEGxZs6xagSeNMu7HmMoy05F7LmpSkoKD2Sxs7BHHH RcIULQR3lEglN+8h0mE2vXyxCFpho78HgZ+VqMvzb2GofwXDms1jeBYIQDo6CgSDF/FZ QSgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=7mzhH9CcDQt1KrmN2/0Wykq86AG8JhhNqlJJhaigJTc=; fh=6BCBHd+wd9sM5I7f3nZwXI5uSly3s62mQle808OwsEg=; b=hH/8KLR4L5JpxxahK9OUmGr8lVr1cPlvId/4YWbWrYA4fvPY968OFjreA0qGzrc1F8 DZ2SEkwdEld68a8CThoYugFLSMuPAoy2uxV7BlfTTkHaCK0jjzyo6ipKHbCuXRuCvIIp R7gQJ7HYcRZQvbSlLOUpSqTEjT3GkDJTIp7qJ27VVkF4pSudWkKo/632glNeLK/pCMm/ kDAnPIDuQt+ZMyKy0hsSrZzvQKPZSowxyTw/qv0vWv9Hl6SL0xbMd6llJD1Da1+SIkyX JrFJGGIVJlTeysOjjU1Ifl+v8HJC4nidN8mes5vQ8buAS4iQI1J7fsyLkLVziJIW5q3l vVyw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m189-20020a633fc6000000b005641bbe7841si2581737pga.524.2023.08.04.21.46.27; Fri, 04 Aug 2023 21:46:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229688AbjHEDjB (ORCPT + 99 others); Fri, 4 Aug 2023 23:39:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229379AbjHEDi4 (ORCPT ); Fri, 4 Aug 2023 23:38:56 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BDB594ED7; Fri, 4 Aug 2023 20:38:54 -0700 (PDT) Received: from dggpeml500012.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RHpF63703zrS5n; Sat, 5 Aug 2023 11:37:46 +0800 (CST) Received: from localhost.localdomain (10.67.175.61) by dggpeml500012.china.huawei.com (7.185.36.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Sat, 5 Aug 2023 11:38:52 +0800 From: Zheng Yejian To: , , , CC: , , Subject: [PATCH v2 2/2] selftests/ftrace: Add a basic testcase for snapshot Date: Sat, 5 Aug 2023 11:38:16 +0800 Message-ID: <20230805033816.3284594-3-zhengyejian1@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230805033816.3284594-1-zhengyejian1@huawei.com> References: <20230805033816.3284594-1-zhengyejian1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.175.61] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpeml500012.china.huawei.com (7.185.36.15) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773363056248792311 X-GMAIL-MSGID: 1773363056248792311 This testcase is constrived to reproduce a problem that the cpu buffers become unavailable which is due to 'record_disabled' of array_buffer and max_buffer being messed up. Local test result after bugfix: # ./ftracetest test.d/00basic/snapshot1.tc === Ftrace unit tests === [1] Snapshot and tracing_cpumask [PASS] [2] (instance) Snapshot and tracing_cpumask [PASS] # of passed: 2 # of failed: 0 # of unresolved: 0 # of untested: 0 # of unsupported: 0 # of xfailed: 0 # of undefined(test bug): 0 Signed-off-by: Zheng Yejian --- .../ftrace/test.d/00basic/snapshot1.tc | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/snapshot1.tc diff --git a/tools/testing/selftests/ftrace/test.d/00basic/snapshot1.tc b/tools/testing/selftests/ftrace/test.d/00basic/snapshot1.tc new file mode 100644 index 000000000000..63b76cf2a360 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/00basic/snapshot1.tc @@ -0,0 +1,31 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Snapshot and tracing_cpumask +# requires: trace_marker tracing_cpumask snapshot +# flags: instance + +# This testcase is constrived to reproduce a problem that the cpu buffers +# become unavailable which is due to 'record_disabled' of array_buffer and +# max_buffer being messed up. + +# Store origin cpumask +ORIG_CPUMASK=`cat tracing_cpumask` + +# Stop tracing all cpu +echo 0 > tracing_cpumask + +# Take a snapshot of the main buffer +echo 1 > snapshot + +# Restore origin cpumask, note that there should be some cpus being traced +echo ${ORIG_CPUMASK} > tracing_cpumask + +# Set tracing on +echo 1 > tracing_on + +# Write a log into buffer +echo "test input 1" > trace_marker + +# Ensure the log writed so that cpu buffers are still available +grep -q "test input 1" trace +exit 0