From patchwork Tue Oct 10 11:59:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 150700 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp140380vqb; Tue, 10 Oct 2023 05:01:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEInvS27dJyjvu+USyBWD0tQJc4LBr7o4qSSAnh6K8eruFfw+bZ70uwk6eToP9j+Kr8W8qS X-Received: by 2002:a17:902:b287:b0:1c9:bd9c:a333 with SMTP id u7-20020a170902b28700b001c9bd9ca333mr792333plr.62.1696939260868; Tue, 10 Oct 2023 05:01:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696939260; cv=none; d=google.com; s=arc-20160816; b=V3FJpR6eJZYljq+lIdi2c3/YWCGYiJVN4cetUE1mJObfhqiVnICyzfd/6tPeZHFqbg xwgCkzdW4Gjem/tqaIKE90lKSxMwn6w00Nw0QQeYvsdu9sIzSJnO2cOCaJyDFnqfbx9o ewXX0ZLlb/QEChE+AQ9jwgTsDK57ehKjaK/spC/dmIf6I7wqsoYJd6mqieeBaRbudUVX ickF1dtGtdVSsaw5v9g8a0T6//HOH+B6mA8at8UEBpP5oGd4do2BmoWME4DvbcDOFr0R WBkkNV2D0SN2xoFYcqAOv70M+ua0MWCM0eYItyMeaPiW3V9BneQdGqliA9Dq+U7HG5Km I10Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eufxTqPv0aYRGI0SsBO+wsk2lWXiUeC7GZBHCWUeUNw=; fh=90z6sIZ9ECjg7ut/lDLrerV/HGE6AhCodghzoTNcnEY=; b=QceBhxT9F4GmiYYoIBkjvEv//6MvCsY9qp4ejteSqqOf2tzr52MauMU5c26InapQ5p D+DKDlOQL7W/KKcB9Z3BV9EakZ36WrM7Sb4iZXtUUxZj6dNyON4JFxmgmkZWZbv9ZUEP M3Xl4Vmk6z6VCDIGOJB+A9n+wJu6ezj0mDYmA3EhpqMswWzJisuin9wuUcoNtaYPLpzi vojdbqN0RAy5MydxtTM8pTYW32Al+W/fdN8YmAa6ST5u5j8kRUs8rINNTRxDYIuRLOD3 prwJd9I6pA77sWLVFrpdKesXMZ9fcx/P3AVAL/na/CIc5RsJUW9nSZ4pgcF5hbOfdybc 0LpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=FXblqAxd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id c2-20020a170902d48200b001c71eb782c0si12156415plg.464.2023.10.10.05.01.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Oct 2023 05:01:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=FXblqAxd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id F4172826EB95; Tue, 10 Oct 2023 05:00:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231853AbjJJMA3 (ORCPT + 20 others); Tue, 10 Oct 2023 08:00:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231660AbjJJMAF (ORCPT ); Tue, 10 Oct 2023 08:00:05 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45FB212E; Tue, 10 Oct 2023 04:59:57 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E2F3C433CC; Tue, 10 Oct 2023 11:59:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696939196; bh=RuBZznbtX/mlc/kkPyF6YV7D+ZVAKONXJI1sntYrV0o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FXblqAxd47mobSEFAPTo0z4btOJ3qSynuzXsvzorz57lSIbHe4gJH74CFiIAFxa0G 3MSiXTKlAG532lq9+hQ1jDk7xQkuFEuBEHyRppgylAfRr370isohoFUpXmLVrpbrNk YCKP6sk5PEyRhD9eQ/FWaH61sjuAWyoQPy0UDGw4q1j4z2gEoU39b25bM55HObUODr ng9aFSTwsiR0bL1MqyEjtTBG3jZ6CvguC3spw/F4Vwwe8S4XQpyMURBEwnq+dBZEus SDCS2eSVWDf3lWBhGgrieAatNwsPe6Q+3hOvkIIaONDY7Orfe+w3XMB/JO1VDl5IVv dqLJXQpvspXJg== From: Frederic Weisbecker To: LKML Cc: "Joel Fernandes (Google)" , Boqun Feng , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , stable@vger.kernel.org, Frederic Weisbecker Subject: [PATCH 08/23] rcutorture: Fix stuttering races and other issues Date: Tue, 10 Oct 2023 13:59:06 +0200 Message-Id: <20231010115921.988766-9-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231010115921.988766-1-frederic@kernel.org> References: <20231010115921.988766-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=2.4 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 10 Oct 2023 05:00:58 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779369782052350900 X-GMAIL-MSGID: 1779369782052350900 From: "Joel Fernandes (Google)" The stuttering code isn't functioning as expected. Ideally, it should pause the torture threads for a designated period before resuming. Yet, it fails to halt the test for the correct duration. Additionally, a race condition exists, potentially causing the stuttering code to pause for an extended period if the 'spt' variable is non-zero due to the stutter orchestration thread's inadequate CPU time. Moreover, over-stuttering can hinder RCU's progress on TREE07 kernels. This happens as the stuttering code may run within a softirq due to RCU callbacks. Consequently, ksoftirqd keeps a CPU busy for several seconds, thus obstructing RCU's progress. This situation triggers a warning message in the logs: [ 2169.481783] rcu_torture_writer: rtort_pipe_count: 9 This warning suggests that an RCU torture object, although invisible to RCU readers, couldn't make it past the pipe array and be freed -- a strong indication that there weren't enough grace periods during the stutter interval. To address these issues, this patch sets the "stutter end" time to an absolute point in the future set by the main stutter thread. This is then used for waiting in stutter_wait(). While the stutter thread still defines this absolute time, the waiters' waiting logic doesn't rely on the stutter thread receiving sufficient CPU time to halt the stuttering as the halting is now self-controlled. Cc: stable@vger.kernel.org Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney Signed-off-by: Frederic Weisbecker --- kernel/torture.c | 45 ++++++++++++--------------------------------- 1 file changed, 12 insertions(+), 33 deletions(-) diff --git a/kernel/torture.c b/kernel/torture.c index 6ba62e5993e7..fd353f98162f 100644 --- a/kernel/torture.c +++ b/kernel/torture.c @@ -720,7 +720,7 @@ static void torture_shutdown_cleanup(void) * suddenly applied to or removed from the system. */ static struct task_struct *stutter_task; -static int stutter_pause_test; +static ktime_t stutter_till_abs_time; static int stutter; static int stutter_gap; @@ -730,30 +730,16 @@ static int stutter_gap; */ bool stutter_wait(const char *title) { - unsigned int i = 0; bool ret = false; - int spt; + ktime_t till_ns; cond_resched_tasks_rcu_qs(); - spt = READ_ONCE(stutter_pause_test); - for (; spt; spt = READ_ONCE(stutter_pause_test)) { - if (!ret && !rt_task(current)) { - sched_set_normal(current, MAX_NICE); - ret = true; - } - if (spt == 1) { - torture_hrtimeout_jiffies(1, NULL); - } else if (spt == 2) { - while (READ_ONCE(stutter_pause_test)) { - if (!(i++ & 0xffff)) - torture_hrtimeout_us(10, 0, NULL); - cond_resched(); - } - } else { - torture_hrtimeout_jiffies(round_jiffies_relative(HZ), NULL); - } - torture_shutdown_absorb(title); + till_ns = READ_ONCE(stutter_till_abs_time); + if (till_ns && ktime_before(ktime_get(), till_ns)) { + torture_hrtimeout_ns(till_ns, 0, HRTIMER_MODE_ABS, NULL); + ret = true; } + torture_shutdown_absorb(title); return ret; } EXPORT_SYMBOL_GPL(stutter_wait); @@ -764,23 +750,16 @@ EXPORT_SYMBOL_GPL(stutter_wait); */ static int torture_stutter(void *arg) { - DEFINE_TORTURE_RANDOM(rand); - int wtime; + ktime_t till_ns; VERBOSE_TOROUT_STRING("torture_stutter task started"); do { if (!torture_must_stop() && stutter > 1) { - wtime = stutter; - if (stutter > 2) { - WRITE_ONCE(stutter_pause_test, 1); - wtime = stutter - 3; - torture_hrtimeout_jiffies(wtime, &rand); - wtime = 2; - } - WRITE_ONCE(stutter_pause_test, 2); - torture_hrtimeout_jiffies(wtime, NULL); + till_ns = ktime_add_ns(ktime_get(), + jiffies_to_nsecs(stutter)); + WRITE_ONCE(stutter_till_abs_time, till_ns); + torture_hrtimeout_jiffies(stutter - 1, NULL); } - WRITE_ONCE(stutter_pause_test, 0); if (!torture_must_stop()) torture_hrtimeout_jiffies(stutter_gap, NULL); torture_shutdown_absorb("torture_stutter");