From patchwork Fri Sep 8 20:35:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137798 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp807262vqo; Fri, 8 Sep 2023 14:46:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFrBZzjlVyCi/weBpBJk1CnlHOCpfpAs5NYL7Ww4+9+qsAzoEkePUVMMXP5L5pL648YYYnx X-Received: by 2002:a17:903:244f:b0:1c1:d730:f0cf with SMTP id l15-20020a170903244f00b001c1d730f0cfmr4204698pls.20.1694209613379; Fri, 08 Sep 2023 14:46:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694209613; cv=none; d=google.com; s=arc-20160816; b=lxNf/YKwKD/bN54Pd27q8S356428Jbom9Di3CV9WlTFUnekMqPGj44qpVVUZhHrh+j PkuhmdnorkzMMjJr4Y3f48T22TeZRyx3N7IyccWOAEkef6E2fPLNZ8xjvHD1ppdlwWGe BN5o/PM8BzSVdzbJIqu652b/6L10UcM2IwkOzAEFKck7qy3yFV9Sbgm3w22gVV3c5meM b6l4+noFsRPD1BlZ7PK6CLyDMx5Mv82nbeesStKMSCxGPA/MAVbzItkzIrNuX3DUnqRj 9LJODgkUgGg8bgKi7w2aAv9Cx2mUuue8If5Rbfnz3YRbwYMFhp2QnHSf2YqzdTjtTANY 7jyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=blMzM4L7KTCta+RIVTUa9dZqKyvOL2v1f6wRkRwQg2E=; fh=PM0GxARnY2fq++lmnFr5zDtgNQoaxnNKf1FrSBSd8Ck=; b=ogAXh6te2C3AQSoIpylAZQXvtRZiPKqQ9JJeF4WFU7JBV0tYWEMJuNyKwPRGNEkcab WPTOdF/bUwne9UOcxEjHGblfUF0oLeBqMm++s7jkBbCZLvyQL5VHSFtdTg913uI+2iza vq/stUtKmv7Ba+Vr7GSXaRCHzK2Ge9pJC54M8q2dMD7g5nEqdiQxXlrf+BK7I/HJm0Rn Auk9gSIwzSQkmX+1r6M7Adt6DszXe8k8YMPBubeudZPIgH0h6UtP4NrTglof14Ym/N3+ 27187q+mWqf56ql+knxVhF8WbUfDC+Yv9kYPjlrV52lU9iZTudeddmoibnoGbs+CXU0h Dqkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Y4r72CYf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ku11-20020a170903288b00b001b8a3a8d707si2114253plb.573.2023.09.08.14.46.41; Fri, 08 Sep 2023 14:46:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Y4r72CYf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343905AbjIHUgW (ORCPT + 40 others); Fri, 8 Sep 2023 16:36:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233048AbjIHUgT (ORCPT ); Fri, 8 Sep 2023 16:36:19 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE9B18E; Fri, 8 Sep 2023 13:36:15 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5FAD2C433C9; Fri, 8 Sep 2023 20:36:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205375; bh=mvgQfp1WaBcpfDTbCSVaFFlPEh1S9we854LIGmy5oWI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y4r72CYfP8k8zha0VpGBOejgR2oZO5edDsBWsrRayIRwLTkKan/Bok+LpSDI/XWhb VwZYpRvFGYHMJKeGSBriUO30SdFeoL/e9fkvpVrR5YJSosGikFTaSK6Y1xocgPdg2R q66JaD+h4tZs+qql4yWrI2R6+L18dtGn6F+fQjNWx6Ko2WGX65fQHv1vetYyDn+mip YhwtStSj0EtCEqjxkN0Bo6nipse8R5mWnImjlpQDw/jYrNFdWyE93nO/sMVqT9Op8/ JXnhXp0iSo1NB/wwLfbkN5yAHL9PQu29JQiZn8lhZTV1lcB2yK66RAzos/j/zJZwOl I78RcKSLfvNkA== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Qiuxu Zhuo , Joel Fernandes Subject: [PATCH 01/10] rcu: Use rcu_segcblist_segempty() instead of open coding it Date: Fri, 8 Sep 2023 22:35:54 +0200 Message-ID: <20230908203603.5865-2-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776507539754101139 X-GMAIL-MSGID: 1776507539754101139 This makes the code more readable. Reviewed-by: Qiuxu Zhuo Reviewed-by: Joel Fernandes (Google) Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- kernel/rcu/rcu_segcblist.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c index f71fac422c8f..1693ea22ef1b 100644 --- a/kernel/rcu/rcu_segcblist.c +++ b/kernel/rcu/rcu_segcblist.c @@ -368,7 +368,7 @@ bool rcu_segcblist_entrain(struct rcu_segcblist *rsclp, smp_mb(); /* Ensure counts are updated before callback is entrained. */ rhp->next = NULL; for (i = RCU_NEXT_TAIL; i > RCU_DONE_TAIL; i--) - if (rsclp->tails[i] != rsclp->tails[i - 1]) + if (!rcu_segcblist_segempty(rsclp, i)) break; rcu_segcblist_inc_seglen(rsclp, i); WRITE_ONCE(*rsclp->tails[i], rhp); @@ -551,7 +551,7 @@ bool rcu_segcblist_accelerate(struct rcu_segcblist *rsclp, unsigned long seq) * as their ->gp_seq[] grace-period completion sequence number. */ for (i = RCU_NEXT_READY_TAIL; i > RCU_DONE_TAIL; i--) - if (rsclp->tails[i] != rsclp->tails[i - 1] && + if (!rcu_segcblist_segempty(rsclp, i) && ULONG_CMP_LT(rsclp->gp_seq[i], seq)) break; From patchwork Fri Sep 8 20:35:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137810 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp834572vqo; Fri, 8 Sep 2023 15:55:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHjsedtQXVWXIl3aSl7sbRG8vVOnTwSWqKzQJASAuzcxdETYGR1gDthhO/qtyuQC8c/c6qc X-Received: by 2002:a05:6808:2815:b0:3a8:6693:135d with SMTP id et21-20020a056808281500b003a86693135dmr3653935oib.49.1694213739536; Fri, 08 Sep 2023 15:55:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694213739; cv=none; d=google.com; s=arc-20160816; b=Xt76ar2s+FVGgJ7KGYpjl0fYNRDshSJrKm9rOzVu8bxcUC5AzldaV7USy5uHlYbHTa VXlRnREtco1ScszSdsATPkupPGoJk5qAbmxWQ/IH8ioGYAZf8t7itclP+CYFPgDD0cYJ tXtv7oxiSRICVERJN+eWTO2x83iMATJ/dAq/gJBTG0igpmcv8++cFZchhX05w3+3R2JL DAWQwcGwkPGa+F2Au9e7IGiBx8M2VCeDwYvrRcU4JE93URwh4rIAJ56GP8p0AFADhVoX HG8z23STAgFfyW4RrGT+0ZUlEspveOOFcmmVLRnO3uyMkyDHshQl/0piUy3AQ4Xb0x4/ mK8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/zlJ7ne+Yi+AZ1wV8mKuWizqNMvyeIKVqhrzo4m1Dhw=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=r16ebK4ZTdZQ1UsddwYLsmux6spfngz+2QIQMVKN37rBHy12YEbaCBCnXxX9iq6Pcr e2dDL2vQtjUeNQS4UGM8Vwv101tj3KTjbjbf8wGzjMdoFq6W64m2Gh3GVuG2HrdXpaHq u6Pg53xPP7C56DRcuIndF4LzN/JaNErKH2AZmcz6LPX6AnnwL+Xnvn6dosYvntTbMFbK JgSknBwORmj5uYIunTRcBCE/8f2so6LyhapATgLOp8qbM/yz0BGeskrJt9vH6ag/y6ID aYqj9GB6x1QwpjjZj1y7GcQn/u3J5CCpLwRlORTomCqEgWl8ctl2mFUFAYbmaXtgbWuV YnOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Ce3J97XH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bg26-20020a056a02011a00b0054fbd904b6dsi2039676pgb.500.2023.09.08.15.55.37; Fri, 08 Sep 2023 15:55:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Ce3J97XH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344078AbjIHUgX (ORCPT + 16 others); Fri, 8 Sep 2023 16:36:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52218 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233048AbjIHUgW (ORCPT ); Fri, 8 Sep 2023 16:36:22 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69601B2; Fri, 8 Sep 2023 13:36:18 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 136E9C433CC; Fri, 8 Sep 2023 20:36:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205378; bh=Zu1FseTcB8ucaQhbROvFuH9zDir/2M+hapV0YqGSd20=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ce3J97XH0oLc7QOib+7DQ/wHo3Jhw1lL1t8k3AbgKGJ/Yb61Sjq+agUCLWVvGPJm5 Ccc8O2a2Kn+X4nH9CL9PvG/sZTKgC87JXPjazn0+nobFMOuJ4e/nxG4/ZRoGXbhmUc gXmofbYu25ZhA50QS964E7BRuGe1vMmHX8TTA7JJsRDd8GzFafPWMUF90WVflxyhQZ ayMv51DVu1PkhwW9PbuOOnukUxvb+UAy0JiDhwTZCfp1oc9U/XdeQEjBt1ktM0qdlz rIfip7CC6D4LsLLV+tczwSUW2yZnPzAjUvc/6EX9JcejTY/K0zUf3f3l8jGnmHSraw myNgxLgUlj2xg== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 02/10] rcu: Rename jiffies_till_flush to jiffies_lazy_flush Date: Fri, 8 Sep 2023 22:35:55 +0200 Message-ID: <20230908203603.5865-3-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776511865711864500 X-GMAIL-MSGID: 1776511865711864500 The variable name jiffies_till_flush is too generic and therefore: * It may shadow a global variable * It doesn't tell on what it operates Make the name more precise, along with the related APIs. Signed-off-by: Frederic Weisbecker --- kernel/rcu/rcu.h | 8 ++++---- kernel/rcu/rcuscale.c | 6 +++--- kernel/rcu/tree_nocb.h | 20 ++++++++++---------- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index 56a8466a11a2..4ac4daae9917 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -541,11 +541,11 @@ enum rcutorture_type { }; #if defined(CONFIG_RCU_LAZY) -unsigned long rcu_lazy_get_jiffies_till_flush(void); -void rcu_lazy_set_jiffies_till_flush(unsigned long j); +unsigned long rcu_get_jiffies_lazy_flush(void); +void rcu_set_jiffies_lazy_flush(unsigned long j); #else -static inline unsigned long rcu_lazy_get_jiffies_till_flush(void) { return 0; } -static inline void rcu_lazy_set_jiffies_till_flush(unsigned long j) { } +static inline unsigned long rcu_get_jiffies_lazy_flush(void) { return 0; } +static inline void rcu_set_jiffies_lazy_flush(unsigned long j) { } #endif #if defined(CONFIG_TREE_RCU) diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c index ffdb30495e3c..8db4fedaaa1e 100644 --- a/kernel/rcu/rcuscale.c +++ b/kernel/rcu/rcuscale.c @@ -764,9 +764,9 @@ kfree_scale_init(void) if (kfree_by_call_rcu) { /* do a test to check the timeout. */ - orig_jif = rcu_lazy_get_jiffies_till_flush(); + orig_jif = rcu_get_jiffies_lazy_flush(); - rcu_lazy_set_jiffies_till_flush(2 * HZ); + rcu_set_jiffies_lazy_flush(2 * HZ); rcu_barrier(); jif_start = jiffies; @@ -775,7 +775,7 @@ kfree_scale_init(void) smp_cond_load_relaxed(&rcu_lazy_test1_cb_called, VAL == 1); - rcu_lazy_set_jiffies_till_flush(orig_jif); + rcu_set_jiffies_lazy_flush(orig_jif); if (WARN_ON_ONCE(jiffies_at_lazy_cb - jif_start < 2 * HZ)) { pr_alert("ERROR: call_rcu() CBs are not being lazy as expected!\n"); diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 5598212d1f27..b9eab359c597 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -264,21 +264,21 @@ static bool wake_nocb_gp(struct rcu_data *rdp, bool force) * left unsubmitted to RCU after those many jiffies. */ #define LAZY_FLUSH_JIFFIES (10 * HZ) -static unsigned long jiffies_till_flush = LAZY_FLUSH_JIFFIES; +static unsigned long jiffies_lazy_flush = LAZY_FLUSH_JIFFIES; #ifdef CONFIG_RCU_LAZY // To be called only from test code. -void rcu_lazy_set_jiffies_till_flush(unsigned long jif) +void rcu_set_jiffies_lazy_flush(unsigned long jif) { - jiffies_till_flush = jif; + jiffies_lazy_flush = jif; } -EXPORT_SYMBOL(rcu_lazy_set_jiffies_till_flush); +EXPORT_SYMBOL(rcu_set_jiffies_lazy_flush); -unsigned long rcu_lazy_get_jiffies_till_flush(void) +unsigned long rcu_get_jiffies_lazy_flush(void) { - return jiffies_till_flush; + return jiffies_lazy_flush; } -EXPORT_SYMBOL(rcu_lazy_get_jiffies_till_flush); +EXPORT_SYMBOL(rcu_get_jiffies_lazy_flush); #endif /* @@ -299,7 +299,7 @@ static void wake_nocb_gp_defer(struct rcu_data *rdp, int waketype, */ if (waketype == RCU_NOCB_WAKE_LAZY && rdp->nocb_defer_wakeup == RCU_NOCB_WAKE_NOT) { - mod_timer(&rdp_gp->nocb_timer, jiffies + jiffies_till_flush); + mod_timer(&rdp_gp->nocb_timer, jiffies + jiffies_lazy_flush); WRITE_ONCE(rdp_gp->nocb_defer_wakeup, waketype); } else if (waketype == RCU_NOCB_WAKE_BYPASS) { mod_timer(&rdp_gp->nocb_timer, jiffies + 2); @@ -482,7 +482,7 @@ static bool rcu_nocb_try_bypass(struct rcu_data *rdp, struct rcu_head *rhp, // flush ->nocb_bypass to ->cblist. if ((ncbs && !bypass_is_lazy && j != READ_ONCE(rdp->nocb_bypass_first)) || (ncbs && bypass_is_lazy && - (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_till_flush))) || + (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_lazy_flush))) || ncbs >= qhimark) { rcu_nocb_lock(rdp); *was_alldone = !rcu_segcblist_pend_cbs(&rdp->cblist); @@ -723,7 +723,7 @@ static void nocb_gp_wait(struct rcu_data *my_rdp) lazy_ncbs = READ_ONCE(rdp->lazy_len); if (bypass_ncbs && (lazy_ncbs == bypass_ncbs) && - (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_till_flush) || + (time_after(j, READ_ONCE(rdp->nocb_bypass_first) + jiffies_lazy_flush) || bypass_ncbs > 2 * qhimark)) { flush_bypass = true; } else if (bypass_ncbs && (lazy_ncbs != bypass_ncbs) && From patchwork Fri Sep 8 20:35:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137809 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp834424vqo; Fri, 8 Sep 2023 15:55:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHeeo/j85sy2Aoo9L5ftcyE19gGEtQ0xXpyeQ2XOWmsk3X8GpSZq2SX7cMO0F7WnyuPxCAs X-Received: by 2002:a05:6a20:939c:b0:14c:7b5d:1fe0 with SMTP id x28-20020a056a20939c00b0014c7b5d1fe0mr4526168pzh.3.1694213719480; Fri, 08 Sep 2023 15:55:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694213719; cv=none; d=google.com; s=arc-20160816; b=povQ+fYmGmu4e4uoaEZgRaLcDG33RxNuQcEqAq/HYtBdJpaSu8+HKb3AMpZLRjNNjI S+mOVVf5R+PuMWUEhcGdYMnaxWKHttsG8LSB59BaCNSlaOZOVIeLLSCcLG3OSvSdndS1 Ahd5dH4wSlxqgY3aeuTjK0bs8H6wvcNqUd9VdcgCg4d5nT8TJApFAJfiE3/Qq9FnTYuu T+M7Nz5/AK+qejqGdSfzqSNxiTpcQ242I9TBYNderJRz0/7S5jtFkbXXa/g5ECWW/iRC PM52SacRK6EFUGlpkskqzrbDqAmlzMykPzpEzE5h0Gu6BhF8POjbrd7nAJrrF5MNu5oW rfcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=GW498Ik83ReUsJQ0RNq1Wh6hoBZ6ReVeS8KUN0B24Zc=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=jbiVJpSktPjpH/1sU2kcnlyIvr3W1eOGP3dOqK27vKk6XHL6wbHGJd7uWc+ZURZRUE 9LkU4LSOnscJg/rut7PTZA7hBkJQ1kYB6gKPfL13w6U8ZN3dUQowGkngKuhZKSXK0EUy rNph0SO1GaDFFlXON/0Qp2Ihg0totAyynsQM5uMf9Xxr6SEyxjrTceoNpBMlUx0G7dgL aTR93buHl/IZw8xTgs0pjyg3x1BcDgzKCcyZIXRCgqSwCkxngdyU1LmqS5fbmSTVqji8 QTPBWARJwENZ3dpAbDzDYxHxB3dFTiytw/Y5m8hk01SIVz2rC03+WYp7DvsOtkTcb0oF /lwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=If6qvcxV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h7-20020a636c07000000b00553ce421356si2072839pgc.56.2023.09.08.15.55.17; Fri, 08 Sep 2023 15:55:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=If6qvcxV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240020AbjIHUg3 (ORCPT + 16 others); Fri, 8 Sep 2023 16:36:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344101AbjIHUgZ (ORCPT ); Fri, 8 Sep 2023 16:36:25 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCF4A12C; Fri, 8 Sep 2023 13:36:20 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 806C2C433CB; Fri, 8 Sep 2023 20:36:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205380; bh=GOeZfICz6IgN8EOrRlzyyiYJ3WMWQY2axoWb8S5uGRk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=If6qvcxVcxQ4YjCjO7D3jckmaba8n+kof+yqc7NEmlYUHjV/VWZ/w40/6+4jmB9Ri Y+vccZ1BH+/FvRY3YUZyqG482iO5pCVNNQ9EJrPXnt0n6eAzlbWJWS/u4yuxMmi/B3 PsE7hF3WqQAYGEATCBBL2slDPGqkZTQ5pVDr8kOO2ryW37WQvlXcAs0h38/4BBoVTf w7K6tb/HDGgG3PTnsdLCFaSFzpq5gukKxldj02ATIkQ1czkCw80lva05aDoJZ/wVYt TMkG5IiGVMSNuAV2tToOnB4e790Z1OEi8ISCToMU/SULY+V2VuI/O3TqDPG9D3e/nt jU1ZWT9H33nPQ== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 03/10] rcu/nocb: Remove needless LOAD-ACQUIRE Date: Fri, 8 Sep 2023 22:35:56 +0200 Message-ID: <20230908203603.5865-4-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776511844758967095 X-GMAIL-MSGID: 1776511844758967095 The LOAD-ACQUIRE access performed on rdp->nocb_cb_sleep advertizes ordering callback execution against grace period completion. However this is contradicted by the following: * This LOAD-ACQUIRE doesn't pair with anything. The only counterpart barrier that can be found is the smp_mb() placed after callbacks advancing in nocb_gp_wait(). However the barrier is placed _after_ ->nocb_cb_sleep write. * Callbacks can be concurrently advanced between the LOAD-ACQUIRE on ->nocb_cb_sleep and the call to rcu_segcblist_extract_done_cbs() in rcu_do_batch(), making any ordering based on ->nocb_cb_sleep broken. * Both rcu_segcblist_extract_done_cbs() and rcu_advance_cbs() are called under the nocb_lock, the latter hereby providing already the desired ACQUIRE semantics. Therefore it is safe to access ->nocb_cb_sleep with a simple compiler barrier. Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree_nocb.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index b9eab359c597..6e63ba4788e1 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -933,8 +933,7 @@ static void nocb_cb_wait(struct rcu_data *rdp) swait_event_interruptible_exclusive(rdp->nocb_cb_wq, nocb_cb_wait_cond(rdp)); - // VVV Ensure CB invocation follows _sleep test. - if (smp_load_acquire(&rdp->nocb_cb_sleep)) { // ^^^ + if (READ_ONCE(rdp->nocb_cb_sleep)) { WARN_ON(signal_pending(current)); trace_rcu_nocb_wake(rcu_state.name, rdp->cpu, TPS("WokeEmpty")); } From patchwork Fri Sep 8 20:35:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137800 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp808815vqo; Fri, 8 Sep 2023 14:51:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFux12xZ7yvNW9N7iWuFzovFrvxVDmfA9LNn5lsHUBtqwgdU1P6ayqNRPj0WjMNwDRFGW2H X-Received: by 2002:a17:903:1c1:b0:1bd:fa80:103d with SMTP id e1-20020a17090301c100b001bdfa80103dmr4556453plh.25.1694209864594; Fri, 08 Sep 2023 14:51:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694209864; cv=none; d=google.com; s=arc-20160816; b=YhVVMa3D2JtuIPOFR4j/MCFxndkqkzbN9jevGqDt67Zs7/Hvz3kN/EZKxCMmIGzJA9 EP79TomYtRUndnTwob+qMLvWGQt3geNgclJoiqVjoXK7TME4ypkGKJWIYzKAbh8Y9alm 5chCOT2OAsyXauEfOGUtZLjlnKfFiNaWV4gy7Pn81g/1kxkhqmFluyrklFJtFMi/SECm w+UQc/qBw7DNFZXg6FILqOveJl2Y/d+A2XC+9Ux0clPuZWrDK7nzi/eE8oCPRtI4/fCu zRrwDUXSn9PQ+7BmadsFc6Vl1CQohU5kwFBs9YZtgZQRIwXTAjhCDW1nDdtQiSEjc3UJ 4wxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Lw7nh+LXgSMi1dNo/nbt7E096s4QKdiF2fxwdKN2QZ4=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=Qy1VUy+5ZcP8uoe20W9B58Ffd+8fgftoGbgozIH/54FzpuY7SO2T3WPLDRVBdrMpYH IoAN4Xn6TjJnjD4pdhXpry16Bk2Gecesqy//d14OQc0xLvReHlVl4nK7LlyR2F2cYHJ0 n25doaX4MryDRb3Q1/Z2qnkroV4P0y3gfj5fszQgmGYl/UC0FM5exmzh9t1Yi/sxkc6+ 624a/5NTngXHW3KI5gx6O49hR5OGhyrjkClrfUXx5s4rzlyUOImFO+T074xjtonOqNcS w+F//0BV1Gf0CKi6TJ9P9h4g8iSKcqoctw27D8jInvh5DUWgXRt06cTs63excNmQhggU aUQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=cCmajFEM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u2-20020a17090282c200b001b67bdc438csi2137358plz.376.2023.09.08.14.50.57; Fri, 08 Sep 2023 14:51:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=cCmajFEM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344101AbjIHUgb (ORCPT + 39 others); Fri, 8 Sep 2023 16:36:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344135AbjIHUg1 (ORCPT ); Fri, 8 Sep 2023 16:36:27 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 598E3C4; Fri, 8 Sep 2023 13:36:23 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18509C433B9; Fri, 8 Sep 2023 20:36:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205383; bh=GUT92MKTD1zv1ELbTZztdEp1nZcVqJTBRlzdjoDETLs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cCmajFEMvoh8fTOLErYAwt+nF4hfVHNs6QQNNM/2Y8GtU1lnueDle9kgxUkMdxyby DOv1MXqFeu+ZgrAlsLMXbEGswqAyH9ONebKnlQ7L48/FEeGD5EctI+M7bZsHCWRPXy RRuE5dO1nT0G/fxa4WuI2+XtD1nMAWDglSuzgODLlxHwQ7p6eCQA1J9WQ1pZfz4ZUq vh1loZHLvNHYNxGyO8d1W4S7G0Z+GKq21flczenkRJZyt0obo0vypJ2mL6wtMRAYBx /sSjXr+DxA0M9ffpOqnb70Tuj01OKYMRPqqJw8j1dACnMf+MIF4fhnVttQVqFemWqP X+Y93EB2Ad+Eg== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 04/10] rcu/nocb: Remove needless full barrier after callback advancing Date: Fri, 8 Sep 2023 22:35:57 +0200 Message-ID: <20230908203603.5865-5-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776507803041482964 X-GMAIL-MSGID: 1776507803041482964 A full barrier is issued from nocb_gp_wait() upon callbacks advancing to order grace period completion with callbacks execution. However these two events are already ordered by the smp_mb__after_unlock_lock() barrier within the call to raw_spin_lock_rcu_node() that is necessary for callbacks advancing to happen. The following litmus test shows the kind of guarantee that this barrier provides: C smp_mb__after_unlock_lock {} // rcu_gp_cleanup() P0(spinlock_t *rnp_lock, int *gpnum) { // Grace period cleanup increase gp sequence number spin_lock(rnp_lock); WRITE_ONCE(*gpnum, 1); spin_unlock(rnp_lock); } // nocb_gp_wait() P1(spinlock_t *rnp_lock, spinlock_t *nocb_lock, int *gpnum, int *cb_ready) { int r1; // Call rcu_advance_cbs() from nocb_gp_wait() spin_lock(nocb_lock); spin_lock(rnp_lock); smp_mb__after_unlock_lock(); r1 = READ_ONCE(*gpnum); WRITE_ONCE(*cb_ready, 1); spin_unlock(rnp_lock); spin_unlock(nocb_lock); } // nocb_cb_wait() P2(spinlock_t *nocb_lock, int *cb_ready, int *cb_executed) { int r2; // rcu_do_batch() -> rcu_segcblist_extract_done_cbs() spin_lock(nocb_lock); r2 = READ_ONCE(*cb_ready); spin_unlock(nocb_lock); // Actual callback execution WRITE_ONCE(*cb_executed, 1); } P3(int *cb_executed, int *gpnum) { int r3; WRITE_ONCE(*cb_executed, 2); smp_mb(); r3 = READ_ONCE(*gpnum); } exists (1:r1=1 /\ 2:r2=1 /\ cb_executed=2 /\ 3:r3=0) (* Bad outcome. *) Here the bad outcome only occurs if the smp_mb__after_unlock_lock() is removed. This barrier orders the grace period completion against callbacks advancing and even later callbacks invocation, thanks to the opportunistic propagation via the ->nocb_lock to nocb_cb_wait(). Therefore the smp_mb() placed after callbacks advancing can be safely removed. Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree_nocb.h | 1 - 1 file changed, 1 deletion(-) diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h index 6e63ba4788e1..2dc76f5e6e78 100644 --- a/kernel/rcu/tree_nocb.h +++ b/kernel/rcu/tree_nocb.h @@ -779,7 +779,6 @@ static void nocb_gp_wait(struct rcu_data *my_rdp) if (rcu_segcblist_ready_cbs(&rdp->cblist)) { needwake = rdp->nocb_cb_sleep; WRITE_ONCE(rdp->nocb_cb_sleep, false); - smp_mb(); /* CB invocation -after- GP end. */ } else { needwake = false; } From patchwork Fri Sep 8 20:35:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137807 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp823952vqo; Fri, 8 Sep 2023 15:27:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE0FT6jc6ZMmcvME9sfVu2eFVkWVILkFlxTRPDuBHJETqg4zrtFH4o9mWD6jMdLhgwNzusa X-Received: by 2002:a05:6808:1908:b0:3a7:f153:b5e5 with SMTP id bf8-20020a056808190800b003a7f153b5e5mr4600988oib.29.1694212025037; Fri, 08 Sep 2023 15:27:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694212025; cv=none; d=google.com; s=arc-20160816; b=iDUBgjpIKPByimcIYcmSYdNrv7t8SwrKrkM09PJScSk7L80BecmA9XmqB7p7JlK+q6 c9wznUtDmL/p1aCta5z2ig/f0Etge/tILfVbAr3gqGsE0E5560ASdtmSS5PvS1Klg6lE V7c7pxmUITA+lu8+F0stP6QruxF5blxlxzmjcpCBAbk0qj2qInwSujHhmM3ZhsA5+amH TkDNlqVZrANjtNj6a7A2nKu3rPCnIX4eq9zSxbP3SQidYJlP6PwVkU7NNlofpJATufHe EgYvRHT0mtezqtOKUjAanb8St3p/tA5+Prc9DllfbXySfPaVM8EnC8rXtRrpc+dTWkNi Y9tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yLQ+CqQpaJKz7DH/yVrEy8muqSdGGO8889JEDJZrkxg=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=xOtVkEiKOryKZvwqtPU5mkqVRfvVg/hhVAJq7Nt24Q65Rlr1Omr4BofTrLiQJ6vZ5Z WYIgH24gFxJmYkrVELNsnENwcWsXgOTycSMhqgp3B/MZNmXAxuCC5cNgxGHhDO9reOag eaXPJxLJgTqXdWulY5Mb8h4x42KqPi+Le4jToNN+CqVXaxtf60LVQPeC/yUzOCOmaxXa yEfo+I9772VkmNsEnpuq9L63xw7/Bv+nrfqOC/aHZzGlKJNzFpSkRc9kGLHFNx4It9Of LVqnqm+PTUJ2txrAv/k+Kxm7vEJqTaRpA9hrtlJ84GD5g7xvnOzJYvitSfunksySUFaO uCfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="XPpmn/Dm"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m187-20020a6326c4000000b00573fcfe9f19si2117473pgm.283.2023.09.08.15.26.57; Fri, 08 Sep 2023 15:27:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="XPpmn/Dm"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344384AbjIHUgc (ORCPT + 39 others); Fri, 8 Sep 2023 16:36:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344240AbjIHUgb (ORCPT ); Fri, 8 Sep 2023 16:36:31 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC3A012B; Fri, 8 Sep 2023 13:36:25 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76BFAC433CB; Fri, 8 Sep 2023 20:36:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205385; bh=jDNkBDS/qfTFPfcD21qgFaMBxLKgVcToMO54awbM3I4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XPpmn/DmCtFxr+xOeItYC9TcDzjbNDWMr9tkF5K68mh93CaOUVS5q+SXn34MRwg9p GJjYK8PJkQIAboGENjtWkb7uQcNgNSgj45hYcaFXtwRD3k3VO2CB4bwW6QvcPlHZfk tMhmVecVYjMgKebTkthKhWjWVslirLZDQlzxQP5vyJdoQZOcge6ViG7K0ZZ0ZDF25T u/J7PusNlWTRFA4XibyAsebL4M5iBbucAdfutwlqOmPIh9dR3okAfeyi0m2tMon0be IAun0m6SUAEBCBgT9QvIOzRCIrTjBkwuTjBU4tj+WZlB+Slygkaefigt4U7PvOjgNH 6KuRh5JvkTvlg== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 05/10] rcu: Assume IRQS disabled from rcu_report_dead() Date: Fri, 8 Sep 2023 22:35:58 +0200 Message-ID: <20230908203603.5865-6-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776510068199808218 X-GMAIL-MSGID: 1776510068199808218 rcu_report_dead() is the last RCU word from the CPU down through the hotplug path. It is called in the idle loop right before the CPU shuts down for good. Because it removes the CPU from the grace period state machine and reports an ultimate quiescent state if necessary, no further use of RCU is allowed. Therefore it is expected that IRQs are disabled upon calling this function and are not to be re-enabled again until the CPU shuts down. Remove the IRQs disablement from that function and verify instead that it is actually called with IRQs disabled as it is expected at that special point in the idle path. Reviewed-by: Joel Fernandes (Google) Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- kernel/rcu/tree.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index a83ecab77917..8b5ebef32e17 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4553,11 +4553,16 @@ void rcu_cpu_starting(unsigned int cpu) */ void rcu_report_dead(unsigned int cpu) { - unsigned long flags, seq_flags; + unsigned long flags; unsigned long mask; struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ + /* + * IRQS must be disabled from now on and until the CPU dies, or an interrupt + * may introduce a new READ-side while it is actually off the QS masks. + */ + lockdep_assert_irqs_disabled(); // Do any dangling deferred wakeups. do_nocb_deferred_wakeup(rdp); @@ -4565,7 +4570,6 @@ void rcu_report_dead(unsigned int cpu) /* Remove outgoing CPU from mask in the leaf rcu_node structure. */ mask = rdp->grpmask; - local_irq_save(seq_flags); arch_spin_lock(&rcu_state.ofl_lock); raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */ rdp->rcu_ofl_gp_seq = READ_ONCE(rcu_state.gp_seq); @@ -4579,8 +4583,6 @@ void rcu_report_dead(unsigned int cpu) WRITE_ONCE(rnp->qsmaskinitnext, rnp->qsmaskinitnext & ~mask); raw_spin_unlock_irqrestore_rcu_node(rnp, flags); arch_spin_unlock(&rcu_state.ofl_lock); - local_irq_restore(seq_flags); - rdp->cpu_started = false; } From patchwork Fri Sep 8 20:35:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137805 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp818564vqo; Fri, 8 Sep 2023 15:12:42 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGmU+LkQqhi167UMHuItSqsJXOrp3DKct89LX4DGeCN1aD7uWUiNONBnV/+ZIRtOUUqrKBh X-Received: by 2002:a05:6a20:4401:b0:147:ecf6:c4e6 with SMTP id ce1-20020a056a20440100b00147ecf6c4e6mr5179556pzb.0.1694211162206; Fri, 08 Sep 2023 15:12:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694211162; cv=none; d=google.com; s=arc-20160816; b=t21akXupBhEeraToMR4BwxWwjXGD8pXG3O+ghUxYpDPeq7ZcJ9ivKSzSJ27KbMhLZn B3FewkUJidRJHTz9A4UngZGQQiBM9F6TSwE0nx6IQk90zRWk29uSdmRaS8rOyQGACLrp iO4JNV25BFHkeGk8f2vwIp9CZJDlZ1cLV9m0S1awi2pcYhqfA440yZPjsqYD/kHfsxTM 32tdtVRcKfZpzgzzU4pGm28/QAJv6sOtQNPgH4d+72H4iWtXxf3QezYPNIn90m9X2ENe D/sVu3BCkyZy6ugo5z/6F3H6/CeNUZ9++UcTlgh6jfT84VhoZoM8/NXdbUvL8IFGEfz0 JCYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aZFlG0htBrFMQ8FbpoWCt1rpXGn0Qzewb1RH8m7DXXQ=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=a9G9rmEikH4iXmSD/ocQFphO1txbPsiEUf/Crv+da7Mp4I61fPhLbOBxMbY82qDYzJ 9+a+OIZYxHPaXAzxs3w5DeXhy4JE3jMIVPD8znZP2EnzsqsXpP6wYCGGUbG7l4p0Uqmr vtrEh5EqV6EdoD90Au3rHGgNrqJC8MV9SqRKk7Fmv1Q5afBukfL86B8EC4IU34PD0bEQ 1Hkeg1Zcg77rEYna+m+FCa65yrluPZNdidxJPnNX0wuINgUQ3uaYgBKn3m3Kjf+KIEgj 1kqSTrgoztZbYyJRp0hjBOsAJkExCYORPkN3pafaJ0NrYaG/rWzD1GjP9uSyUBUI9gzm MOvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=W0tN5Z7T; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h7-20020a17090acf0700b0026818f6a0c0si2110580pju.86.2023.09.08.15.12.36; Fri, 08 Sep 2023 15:12:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=W0tN5Z7T; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344478AbjIHUgl (ORCPT + 39 others); Fri, 8 Sep 2023 16:36:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344285AbjIHUgg (ORCPT ); Fri, 8 Sep 2023 16:36:36 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 741CDB2; Fri, 8 Sep 2023 13:36:28 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 015D5C433C8; Fri, 8 Sep 2023 20:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205388; bh=6boxuA0q94N/Ed+FRmKuiKbsWIdOVvAtoMhIDXQWL44=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W0tN5Z7ThROb64K9jyX34QkgwAALEzcAns/+NdYr90LdFK4qlApeyiX12brzy/q1Z xtUNO3D1oO7VhYvbSnBBfY90lVEvyjeiSTm1lDZwPEKJ6mEXMibG9Vju2jQ4Klu5A1 MSuWCIbVW2aV1NLRgiCQ+HLQx8CeI65oLeaH13dDy4afo33GwMcoy2U+XSJVdI/Y5r uDdJZlssMSyon4VQFqcMsRSIk/H0CQgYhna8wMQS2FBF43lcQYkqSlo6fjtFBRCWA5 N3HTM5Wwb+V/7ciWUCq7aV7DagAoXGH+e5Vcw+l2/vvO7W6Z/IqJ2XShVJZBa3gahS wY/u8smjc9tUQ== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 06/10] rcu: Assume rcu_report_dead() is always called locally Date: Fri, 8 Sep 2023 22:35:59 +0200 Message-ID: <20230908203603.5865-7-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776509163840110101 X-GMAIL-MSGID: 1776509163840110101 rcu_report_dead() has to be called locally by the CPU that is going to exit the RCU state machine. Passing a cpu argument here is error-prone and leaves the possibility for a racy remote call. Use local access instead. Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- arch/arm64/kernel/smp.c | 2 +- include/linux/rcupdate.h | 2 +- kernel/cpu.c | 2 +- kernel/rcu/tree.c | 4 ++-- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index edd63894d61e..ce672cb69f1c 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -401,7 +401,7 @@ void __noreturn cpu_die_early(void) /* Mark this CPU absent */ set_cpu_present(cpu, 0); - rcu_report_dead(cpu); + rcu_report_dead(); if (IS_ENABLED(CONFIG_HOTPLUG_CPU)) { update_cpu_boot_status(CPU_KILL_ME); diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 5e5f920ade90..aa351ddcbe8d 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -122,7 +122,7 @@ static inline void call_rcu_hurry(struct rcu_head *head, rcu_callback_t func) void rcu_init(void); extern int rcu_scheduler_active; void rcu_sched_clock_irq(int user); -void rcu_report_dead(unsigned int cpu); +void rcu_report_dead(void); void rcutree_migrate_callbacks(int cpu); #ifdef CONFIG_TASKS_RCU_GENERIC diff --git a/kernel/cpu.c b/kernel/cpu.c index 88a7ede322bd..86f08eafbd9f 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1368,7 +1368,7 @@ void cpuhp_report_idle_dead(void) struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); BUG_ON(st->state != CPUHP_AP_OFFLINE); - rcu_report_dead(smp_processor_id()); + rcu_report_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* * We cannot call complete after rcu_report_dead() so we delegate it diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 8b5ebef32e17..289c51417cbc 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4551,11 +4551,11 @@ void rcu_cpu_starting(unsigned int cpu) * from the outgoing CPU rather than from the cpuhp_step mechanism. * This is because this function must be invoked at a precise location. */ -void rcu_report_dead(unsigned int cpu) +void rcu_report_dead(void) { unsigned long flags; unsigned long mask; - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); + struct rcu_data *rdp = this_cpu_ptr(&rcu_data); struct rcu_node *rnp = rdp->mynode; /* Outgoing CPU's rdp & rnp. */ /* From patchwork Fri Sep 8 20:36:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137797 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp806858vqo; Fri, 8 Sep 2023 14:45:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFaOWoJTQlRGFdBMObZ05qB++ktGCULioNH/EOkjPoBDWPRmVTAoOFharuc6DuFGGKDetK0 X-Received: by 2002:a05:6870:2054:b0:1d5:5d57:bde5 with SMTP id l20-20020a056870205400b001d55d57bde5mr4517983oad.56.1694209549082; Fri, 08 Sep 2023 14:45:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694209549; cv=none; d=google.com; s=arc-20160816; b=HaTnZRZrCAce+6uRjbWafHrg2KRgeyf+rcXPnMJanX1FZyF898weeFM5om7otYO6cN 3/Bc2oBU5i5QuNdL/DZZYdsyMeFSqDpkyRp2/lwlGJVUDbPjN0F7q+yIRSdM22JF6ku+ J3NhEeHtktW7dsEgwgUMKZOp6Qta1IOdo30nupUzAycJ5RX/8ZYnza63dXsgcVL2bAWs JmP28DTNQVKbS4ZkCwfUPBH/AwExpRLTSZPt7sy/jvk8S6W+RA00YTMM0KldCnq2kZfP xxhspNjsKzBXAvJFplkHiAk3Br8kiqez55ArREvpAJnfxJg6fx9LtYy/fsc1M+o/6YPz NmPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=m/6gNKGPDKo2Z6ImDZcWAWvJdZY9OAlfNmIdUd7QJ1k=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=mwKa9H5jg7b0edd8/+OoLZVWZaXw2aaW2PIA9w/4yLQbbcv7W2YYZ55fH0MojuLPl7 epSx0ldHQZ9G+znYic7XTDnzSFXEPWZgyq0HCmtYzMPA3aaK2IiwR8tfpmLJXv+zAT34 oKyVMpGVb3Kk9x9y2NQfGMOziavTPdkFiWBIr6Zcf9otOTl5xRbH/uSTZzAgyci9byJz src2fPfkpknyeExk/NdbULUXIi+FhOyn2k1giAgybCL64wnApjK8P1CMh/MPs5MCwShO 9zBFAJocDs9nlmhDRIi2ta74t9iSyIGU4XPMRCoUtmnh8xHS2enECcUBFJAk0nB/UD66 QYDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=NPbjI4+Q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bz16-20020a056a02061000b00573fc1f288csi2137760pgb.569.2023.09.08.14.45.42; Fri, 08 Sep 2023 14:45:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=NPbjI4+Q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344501AbjIHUgm (ORCPT + 40 others); Fri, 8 Sep 2023 16:36:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344410AbjIHUgi (ORCPT ); Fri, 8 Sep 2023 16:36:38 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0713E6F; Fri, 8 Sep 2023 13:36:30 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8A67EC433C7; Fri, 8 Sep 2023 20:36:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205390; bh=Rt+u8fsiOu0KPpZnkh8F2sls3vmHsEdxXPsoe1TZo9g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NPbjI4+QtqWC800HkW6GYwQyoKJ1L/UMbwCYTDdDn11zkS6O35ZmWoMu7QdHEdkg+ fyMCeezOfdYYVA9tRdmNKwnPFeBY8NYQERiVG2zHdKPSorqhK+Z60m37maviO3H5IS TuZ0oJ3nd9V+5IfjQkScC166gyABtTOY8l43s5x2c+z2TyE4DWQa52S7byfYdwJUAc mPJBUyZbzjlhKY1ANPxrUEesAbQES4RgH6vxgoy2YA2BXFnP/hCzzy+O86zi7LzK92 KEOSRhtjXW9KfQLmT+RWkCs3DKB1oURYh/jLgzBSNp7htWs0BTB8Y6+u9RyYn4pBWn oj/NEI1NFTFuA== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 07/10] rcu: Conditionally build CPU-hotplug teardown callbacks Date: Fri, 8 Sep 2023 22:36:00 +0200 Message-ID: <20230908203603.5865-8-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776507471912466243 X-GMAIL-MSGID: 1776507471912466243 Among the three CPU-hotplug teardown RCU callbacks, two of them early exit if CONFIG_HOTPLUG_CPU=n, and one is left unchanged. In any case all of them have an implementation when CONFIG_HOTPLUG_CPU=n. Align instead with the common way to deal with CPU-hotplug teardown callbacks and provide a proper stub when they are not supported. Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- include/linux/rcutree.h | 11 +++- kernel/rcu/tree.c | 114 +++++++++++++++++++--------------------- 2 files changed, 63 insertions(+), 62 deletions(-) diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index af6ddbd291eb..7d75066c72aa 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -109,9 +109,16 @@ void rcu_all_qs(void); /* RCUtree hotplug events */ int rcutree_prepare_cpu(unsigned int cpu); int rcutree_online_cpu(unsigned int cpu); -int rcutree_offline_cpu(unsigned int cpu); +void rcu_cpu_starting(unsigned int cpu); + +#ifdef CONFIG_HOTPLUG_CPU int rcutree_dead_cpu(unsigned int cpu); int rcutree_dying_cpu(unsigned int cpu); -void rcu_cpu_starting(unsigned int cpu); +int rcutree_offline_cpu(unsigned int cpu); +#else +#define rcutree_dead_cpu NULL +#define rcutree_dying_cpu NULL +#define rcutree_offline_cpu NULL +#endif #endif /* __LINUX_RCUTREE_H */ diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 289c51417cbc..875f241db508 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4228,25 +4228,6 @@ static bool rcu_init_invoked(void) return !!rcu_state.n_online_cpus; } -/* - * Near the end of the offline process. Trace the fact that this CPU - * is going offline. - */ -int rcutree_dying_cpu(unsigned int cpu) -{ - bool blkd; - struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); - struct rcu_node *rnp = rdp->mynode; - - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) - return 0; - - blkd = !!(READ_ONCE(rnp->qsmask) & rdp->grpmask); - trace_rcu_grace_period(rcu_state.name, READ_ONCE(rnp->gp_seq), - blkd ? TPS("cpuofl-bgp") : TPS("cpuofl")); - return 0; -} - /* * All CPUs for the specified rcu_node structure have gone offline, * and all tasks that were preempted within an RCU read-side critical @@ -4292,23 +4273,6 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf) } } -/* - * The CPU has been completely removed, and some other CPU is reporting - * this fact from process context. Do the remainder of the cleanup. - * There can only be one CPU hotplug operation at a time, so no need for - * explicit locking. - */ -int rcutree_dead_cpu(unsigned int cpu) -{ - if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) - return 0; - - WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); - // Stop-machine done, so allow nohz_full to disable tick. - tick_dep_clear(TICK_DEP_BIT_RCU); - return 0; -} - /* * Propagate ->qsinitmask bits up the rcu_node tree to account for the * first CPU in a given leaf rcu_node structure coming online. The caller @@ -4461,29 +4425,6 @@ int rcutree_online_cpu(unsigned int cpu) return 0; } -/* - * Near the beginning of the process. The CPU is still very much alive - * with pretty much all services enabled. - */ -int rcutree_offline_cpu(unsigned int cpu) -{ - unsigned long flags; - struct rcu_data *rdp; - struct rcu_node *rnp; - - rdp = per_cpu_ptr(&rcu_data, cpu); - rnp = rdp->mynode; - raw_spin_lock_irqsave_rcu_node(rnp, flags); - rnp->ffmask &= ~rdp->grpmask; - raw_spin_unlock_irqrestore_rcu_node(rnp, flags); - - rcutree_affinity_setting(cpu, cpu); - - // nohz_full CPUs need the tick for stop-machine to work quickly - tick_dep_set(TICK_DEP_BIT_RCU); - return 0; -} - /* * Mark the specified CPU as being online so that subsequent grace periods * (both expedited and normal) will wait on it. Note that this means that @@ -4637,7 +4578,60 @@ void rcutree_migrate_callbacks(int cpu) cpu, rcu_segcblist_n_cbs(&rdp->cblist), rcu_segcblist_first_cb(&rdp->cblist)); } -#endif + +/* + * The CPU has been completely removed, and some other CPU is reporting + * this fact from process context. Do the remainder of the cleanup. + * There can only be one CPU hotplug operation at a time, so no need for + * explicit locking. + */ +int rcutree_dead_cpu(unsigned int cpu) +{ + WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1); + // Stop-machine done, so allow nohz_full to disable tick. + tick_dep_clear(TICK_DEP_BIT_RCU); + return 0; +} + +/* + * Near the end of the offline process. Trace the fact that this CPU + * is going offline. + */ +int rcutree_dying_cpu(unsigned int cpu) +{ + bool blkd; + struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); + struct rcu_node *rnp = rdp->mynode; + + blkd = !!(READ_ONCE(rnp->qsmask) & rdp->grpmask); + trace_rcu_grace_period(rcu_state.name, READ_ONCE(rnp->gp_seq), + blkd ? TPS("cpuofl-bgp") : TPS("cpuofl")); + return 0; +} + +/* + * Near the beginning of the process. The CPU is still very much alive + * with pretty much all services enabled. + */ +int rcutree_offline_cpu(unsigned int cpu) +{ + unsigned long flags; + struct rcu_data *rdp; + struct rcu_node *rnp; + + rdp = per_cpu_ptr(&rcu_data, cpu); + rnp = rdp->mynode; + raw_spin_lock_irqsave_rcu_node(rnp, flags); + rnp->ffmask &= ~rdp->grpmask; + raw_spin_unlock_irqrestore_rcu_node(rnp, flags); + + rcutree_affinity_setting(cpu, cpu); + + // nohz_full CPUs need the tick for stop-machine to work quickly + tick_dep_set(TICK_DEP_BIT_RCU); + return 0; +} +#endif /* #ifdef CONFIG_HOTPLUG_CPU */ /* * On non-huge systems, use expedited RCU grace periods to make suspend From patchwork Fri Sep 8 20:36:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137801 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp809573vqo; Fri, 8 Sep 2023 14:53:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGyIlG2b6RJjbuywl2NRin7uVs5AmwC2WI/mJwsdbU13yxcxzDPArnf9LTdEqufFPPZzYTL X-Received: by 2002:a17:902:dac1:b0:1bc:7e37:e832 with SMTP id q1-20020a170902dac100b001bc7e37e832mr9398326plx.19.1694209993688; Fri, 08 Sep 2023 14:53:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694209993; cv=none; d=google.com; s=arc-20160816; b=o4iOSaiWCXEIIzPAzIudPcLQ7Kn6rHKy+5122k8gp+91LNRCwuLxT8OH8h+ITdYNBx g4vRgzgTAbCpaj3969L4wDQOasm2Dx6D1Q/Uuwi6udr+X7+X57rRkxIkoeoTkDcy48dM WU7IDJtJjFcSxItOmU/T9Rq1DQFNAVtiJmDaxyjl2A4T3ViH21IUWl3C82YILm6kghUn ZfXj4hrrcDuMmokmKgsxnvhasNUr1a/GWnbVTXLk6aXn22XuD/6T36IAGdQHmnmcKyPP dw7LHJIpuMUj2GTR8DmNt/FZDmM51VJtoN483zSXFLwXeXEDhUVhPbLL5+PlCxwr1U7F /0cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=GGQl6tTdTRgQzZXx5HAvE5rwYtyvTClh4YqmZ8TI4e4=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=dAG5LZgUF/djt0p3aiI+jwRIQ9tb4rQ+CyJJv6youD1XBMRWfnJfZNwS6i/Rf4dQIs ppdYN5f8WQ89bsiOLlx7l+lTJ1a11QW/LPD/nP8XSBsynfEiNVyt+JbXBl3GBPumeFou 1+f5pr53jhyTK/aNfpWb2perIHZgS1ZuWFIRSNOatd6ExDCPRvyJJyN2heZkmnEmoUWd aIM39/LGNosKbojYGCFnzPOIYnwUxc36V+vKCSwyONJzMe1mPbjZ3uTXjkot0DR3XRv3 gqfgPfRNg0jbg7qrB1K5tU7drKeYWb6kXt3dRwM+sa472PFSkOzSfuw04LyLtxKx6sTT v8aA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=gHICVJlB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kk16-20020a170903071000b001c0cbaf6970si2094220plb.501.2023.09.08.14.53.07; Fri, 08 Sep 2023 14:53:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=gHICVJlB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234076AbjIHUgt (ORCPT + 39 others); Fri, 8 Sep 2023 16:36:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344410AbjIHUgr (ORCPT ); Fri, 8 Sep 2023 16:36:47 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 901C8B9; Fri, 8 Sep 2023 13:36:33 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E62CCC433CB; Fri, 8 Sep 2023 20:36:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205392; bh=YMCEualW+a2EhVLuJ9eQrxHeX1kAQsg3kYkpfZwDl0E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gHICVJlBIuURbbtxv10oD1LcMfSNiGstakfF2KThTemh7E9PBCN4+i1QJBsWYX171 Z/Fi4EaK68y8Rk2LU/t0WaExP4Bq5W9cAtgW3XAGcuWBm83ZKyamIDG2CogtrzPU7S jnKbIxIekaieBN7Q3Nd3Ha8e7AhIir7lnDjr/G5wvx/jXP54zaqDinqPG9V+kUz/DI SfOGR9TZ+gPUBHVxtqTP76ON1T/2NFMaeYlwCq5h2ZHLm2qKPSDi8/t7A3M8zXLgIY JNerrhgqomgUJCgnXrx0/PsHHV89MCMmmWk1WNT5FPFBxm/2bClArZWYfz9fydDxV2 Q/ioAITsiJJ1g== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 08/10] rcu: Standardize explicit CPU-hotplug calls Date: Fri, 8 Sep 2023 22:36:01 +0200 Message-ID: <20230908203603.5865-9-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776507937916184170 X-GMAIL-MSGID: 1776507937916184170 rcu_report_dead() and rcutree_migrate_callbacks() have their headers in rcupdate.h while those are pure rcutree calls, like the other CPU-hotplug functions. Also rcu_cpu_starting() and rcu_report_dead() have different naming conventions while they mirror each other's effects. Fix the headers and propose a naming that relates both functions and aligns with the prefix of other rcutree CPU-hotplug functions. Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- .../Expedited-Grace-Periods.rst | 2 +- .../RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg | 4 ++-- .../RCU/Design/Memory-Ordering/TreeRCU-gp.svg | 4 ++-- .../RCU/Design/Memory-Ordering/TreeRCU-hotplug.svg | 4 ++-- .../RCU/Design/Requirements/Requirements.rst | 4 ++-- arch/arm64/kernel/smp.c | 4 ++-- arch/powerpc/kernel/smp.c | 2 +- arch/s390/kernel/smp.c | 2 +- arch/x86/kernel/smpboot.c | 2 +- include/linux/interrupt.h | 2 +- include/linux/rcupdate.h | 2 -- include/linux/rcutiny.h | 2 +- include/linux/rcutree.h | 7 ++++++- kernel/cpu.c | 6 +++--- kernel/rcu/tree.c | 12 ++++++++---- 15 files changed, 33 insertions(+), 26 deletions(-) diff --git a/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst b/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst index 93d899d53258..414f8a2012d6 100644 --- a/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst +++ b/Documentation/RCU/Design/Expedited-Grace-Periods/Expedited-Grace-Periods.rst @@ -181,7 +181,7 @@ operations is carried out at several levels: of this wait (or series of waits, as the case may be) is to permit a concurrent CPU-hotplug operation to complete. #. In the case of RCU-sched, one of the last acts of an outgoing CPU is - to invoke ``rcu_report_dead()``, which reports a quiescent state for + to invoke ``rcutree_report_cpu_dead()``, which reports a quiescent state for that CPU. However, this is likely paranoia-induced redundancy. +-----------------------------------------------------------------------+ diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg index 7ddc094d7f28..d82a77d03d8c 100644 --- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg +++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp-fqs.svg @@ -1135,7 +1135,7 @@ font-weight="bold" font-size="192" id="text202-7-5-3-27-6-5" - style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcu_report_dead() + style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_report_cpu_dead() rcu_cpu_starting() + xml:space="preserve">rcutree_report_cpu_starting() rcu_report_dead() + style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_report_cpu_dead() rcu_cpu_starting() + xml:space="preserve">rcutree_report_cpu_starting() rcu_report_dead() + style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_report_cpu_dead() rcu_cpu_starting() + xml:space="preserve">rcutree_report_cpu_starting() setup_cpu) diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c index f9a2b755f510..3e39a5e1bf48 100644 --- a/arch/s390/kernel/smp.c +++ b/arch/s390/kernel/smp.c @@ -894,7 +894,7 @@ static void smp_start_secondary(void *cpuvoid) S390_lowcore.restart_flags = 0; restore_access_regs(S390_lowcore.access_regs_save_area); cpu_init(); - rcu_cpu_starting(cpu); + rcutree_report_cpu_starting(cpu); init_cpu_timer(); vtime_init(); vdso_getcpu_init(); diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index e1aa2cd7734b..d25b952a2b91 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -288,7 +288,7 @@ static void notrace start_secondary(void *unused) cpu_init(); fpu__init_cpu(); - rcu_cpu_starting(raw_smp_processor_id()); + rcutree_report_cpu_starting(raw_smp_processor_id()); x86_cpuinit.early_percpu_clock_init(); ap_starting(); diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index a92bce40b04b..d05e1e9a553c 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -566,7 +566,7 @@ enum * * _ RCU: * 1) rcutree_migrate_callbacks() migrates the queue. - * 2) rcu_report_dead() reports the final quiescent states. + * 2) rcutree_report_cpu_dead() reports the final quiescent states. * * _ IRQ_POLL: irq_poll_cpu_dead() migrates the queue */ diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index aa351ddcbe8d..f7206b2623c9 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -122,8 +122,6 @@ static inline void call_rcu_hurry(struct rcu_head *head, rcu_callback_t func) void rcu_init(void); extern int rcu_scheduler_active; void rcu_sched_clock_irq(int user); -void rcu_report_dead(void); -void rcutree_migrate_callbacks(int cpu); #ifdef CONFIG_TASKS_RCU_GENERIC void rcu_init_tasks_generic(void); diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h index 7f17acf29dda..04889a9602e7 100644 --- a/include/linux/rcutiny.h +++ b/include/linux/rcutiny.h @@ -169,6 +169,6 @@ static inline void rcu_all_qs(void) { barrier(); } #define rcutree_offline_cpu NULL #define rcutree_dead_cpu NULL #define rcutree_dying_cpu NULL -static inline void rcu_cpu_starting(unsigned int cpu) { } +static inline void rcutree_report_cpu_starting(unsigned int cpu) { } #endif /* __LINUX_RCUTINY_H */ diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h index 7d75066c72aa..07d0fc1e0d31 100644 --- a/include/linux/rcutree.h +++ b/include/linux/rcutree.h @@ -109,7 +109,7 @@ void rcu_all_qs(void); /* RCUtree hotplug events */ int rcutree_prepare_cpu(unsigned int cpu); int rcutree_online_cpu(unsigned int cpu); -void rcu_cpu_starting(unsigned int cpu); +void rcutree_report_cpu_starting(unsigned int cpu); #ifdef CONFIG_HOTPLUG_CPU int rcutree_dead_cpu(unsigned int cpu); @@ -121,4 +121,9 @@ int rcutree_offline_cpu(unsigned int cpu); #define rcutree_offline_cpu NULL #endif +void rcutree_migrate_callbacks(int cpu); + +/* Called from hotplug and also arm64 early secondary boot failure */ +void rcutree_report_cpu_dead(void); + #endif /* __LINUX_RCUTREE_H */ diff --git a/kernel/cpu.c b/kernel/cpu.c index 86f08eafbd9f..a41a6fff3c91 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1368,10 +1368,10 @@ void cpuhp_report_idle_dead(void) struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); BUG_ON(st->state != CPUHP_AP_OFFLINE); - rcu_report_dead(); + rcutree_report_cpu_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* - * We cannot call complete after rcu_report_dead() so we delegate it + * We cannot call complete after rcutree_report_cpu_dead() so we delegate it * to an online cpu. */ smp_call_function_single(cpumask_first(cpu_online_mask), @@ -1575,7 +1575,7 @@ void notify_cpu_starting(unsigned int cpu) struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu); enum cpuhp_state target = min((int)st->target, CPUHP_AP_ONLINE); - rcu_cpu_starting(cpu); /* Enables RCU usage on this CPU. */ + rcutree_report_cpu_starting(cpu); /* Enables RCU usage on this CPU. */ cpumask_set_cpu(cpu, &cpus_booted_once_mask); /* diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 875f241db508..5698c3f30b1d 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4207,7 +4207,7 @@ bool rcu_lockdep_current_cpu_online(void) rdp = this_cpu_ptr(&rcu_data); /* * Strictly, we care here about the case where the current CPU is - * in rcu_cpu_starting() and thus has an excuse for rdp->grpmask + * in rcutree_report_cpu_starting() and thus has an excuse for rdp->grpmask * not being up to date. So arch_spin_is_locked() might have a * false positive if it's held by some *other* CPU, but that's * OK because that just means a false *negative* on the warning. @@ -4436,8 +4436,10 @@ int rcutree_online_cpu(unsigned int cpu) * from the incoming CPU rather than from the cpuhp_step mechanism. * This is because this function must be invoked at a precise location. * This incoming CPU must not have enabled interrupts yet. + * + * This mirrors the effects of rcutree_report_cpu_dead(). */ -void rcu_cpu_starting(unsigned int cpu) +void rcutree_report_cpu_starting(unsigned int cpu) { unsigned long mask; struct rcu_data *rdp; @@ -4491,8 +4493,10 @@ void rcu_cpu_starting(unsigned int cpu) * Note that this function is special in that it is invoked directly * from the outgoing CPU rather than from the cpuhp_step mechanism. * This is because this function must be invoked at a precise location. + * + * This mirrors the effect of rcutree_report_cpu_starting(). */ -void rcu_report_dead(void) +void rcutree_report_cpu_dead(void) { unsigned long flags; unsigned long mask; @@ -5063,7 +5067,7 @@ void __init rcu_init(void) pm_notifier(rcu_pm_notify, 0); WARN_ON(num_online_cpus() > 1); // Only one CPU this early in boot. rcutree_prepare_cpu(cpu); - rcu_cpu_starting(cpu); + rcutree_report_cpu_starting(cpu); rcutree_online_cpu(cpu); /* Create workqueue for Tree SRCU and for expedited GPs. */ From patchwork Fri Sep 8 20:36:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 137811 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp834839vqo; Fri, 8 Sep 2023 15:56:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFDBSwpuv4lxQzqMiAwtxRLkggmJer2nSnoMqtJ9LT2wG/Uh9/DJGfxPmcuI2+lKVbS+Mf+ X-Received: by 2002:a17:90a:6bc1:b0:268:3f6d:9751 with SMTP id w59-20020a17090a6bc100b002683f6d9751mr4166718pjj.23.1694213779373; Fri, 08 Sep 2023 15:56:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694213779; cv=none; d=google.com; s=arc-20160816; b=vrFTCyRIfn80JMLKkwLMxeI6/29wQNofvK7+/gGIIVBUHaypIuQjDHos0oZA4Y0RZo mEcpVK8ibp/zZ0NxZTewuAasHkCamO4+HGVtg1/wMtvoITQfeKvg0lfepX0grNWwN3N9 NaXHY8cGNZ+ZJeSuftZULH8YhBViBINpPKfe4G8y1tz6IC34VWxM8D75cUWj0nuHvp4A pvHR3yRW2ljX1V+/UoXQYjd6FHcDKzXyVT2/7cs83OsrjvTGl87xJCYOszhswhxbN7Kf zwgykZV0b+OqgkRXmrM4VP4G77z/jb6x0jHx6fH6lxrvL/v18uKZ7ZysErXeEyOZ7BEb Kwnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xJ3/04UK8LfqsSXZtEwobfxCKeYSE+Rzy2WQtcTPOzc=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=nuTugzPzH06hkcxDbYzxVRgoLDUVd0GpDOXad4JEu14DqZ19I5M0RuYV1vuBKS2WTs mtFsQxZCKOgKQRb5HPlp7Ouqfakvlmu9k+zJFQYFkzIE7wlMw07Ii8ySxoNYOTZeCOWm x4rkLL03AZ8uOwkxzvMNL2N6NqTFmgkTSud88cv3Em1m9JWWfSC465vGzj4Ewh2bj6E+ sG16VWXdLLS31Z47TMAGk+pNTG8enuMd+3kvOvvUCpX2kfmJJZIVGZAzzx9X/fJjeGqk kd1n7kLxWsc4aMGGyB/txO99FLT+v+qKLSoCnQXpKgioVr225TYzvCqotQFoo8WtK0Gz UZhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=rBPRxcpv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l20-20020a17090aaa9400b0027198fe5672si2114492pjq.129.2023.09.08.15.56.16; Fri, 08 Sep 2023 15:56:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=rBPRxcpv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344356AbjIHUgz (ORCPT + 16 others); Fri, 8 Sep 2023 16:36:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242606AbjIHUgx (ORCPT ); Fri, 8 Sep 2023 16:36:53 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A619A12B; Fri, 8 Sep 2023 13:36:35 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 52F32C433CA; Fri, 8 Sep 2023 20:36:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205395; bh=2O4AhK1gRpJtW3osmb4mUbcQUKqDQptCCmdY+N3unFU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rBPRxcpvefJOnN0MiTosiPzzSQ/E6C7aKG8cABnVJcL0onMiVbqj8F0MKP82um8tc nao4dnggpvn43WYZHTiQlmcrB3tXAC00kWGyv1bVyDv2YolPqW9BHpKpvVLGhA8q5a Iv/p2AooAAjFQG10ZjWhGvK7B3sqwNasgRqzr/id/bdkLUAMn69DMMAkWFRb32wqdM xUZ9sxNLxEJwAxeTeRhSVB94vOf8SDfG/Ih0tNNuyw8oc9lYJ2BBuII+zc7w/fi1FL 70n/3tTq7ADZVjqQe0L8ccZ7RJ0Ni1mD+chFcOL3LIxPtIZkrBs9yl+0UeDSMFsFwj woYOwiNFueTAw== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 09/10] rcu: Remove references to rcu_migrate_callbacks() from diagrams Date: Fri, 8 Sep 2023 22:36:02 +0200 Message-ID: <20230908203603.5865-10-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776511907763813430 X-GMAIL-MSGID: 1776511907763813430 This function is gone since: 53b46303da84 (rcu: Remove rsp parameter from rcu_boot_init_percpu_data() and friends) Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- .../Design/Memory-Ordering/TreeRCU-callback-registry.svg | 9 --------- Documentation/RCU/Design/Memory-Ordering/TreeRCU-gp.svg | 9 --------- 2 files changed, 18 deletions(-) diff --git a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-callback-registry.svg b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-callback-registry.svg index 7ac6f9269806..63eff867175a 100644 --- a/Documentation/RCU/Design/Memory-Ordering/TreeRCU-callback-registry.svg +++ b/Documentation/RCU/Design/Memory-Ordering/TreeRCU-callback-registry.svg @@ -564,15 +564,6 @@ font-size="192" id="text202-7-9-6" style="font-size:192px;font-style:normal;font-weight:bold;text-anchor:start;fill:#000000;stroke-width:0.025in;font-family:Courier">rcutree_migrate_callbacks() - rcu_migrate_callbacks() rcutree_migrate_callbacks() - rcu_migrate_callbacks() X-Patchwork-Id: 137816 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp877405vqo; Fri, 8 Sep 2023 17:42:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH63ISChnYfKLDAcXC2UaDJeTkpFbE6mZxjomvO5z+0PyCmbTIfeSo+wb+I0isbI7ZtcDCG X-Received: by 2002:a2e:8ec8:0:b0:2bd:d34:d98a with SMTP id e8-20020a2e8ec8000000b002bd0d34d98amr2922219ljl.44.1694220123629; Fri, 08 Sep 2023 17:42:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694220123; cv=none; d=google.com; s=arc-20160816; b=gJsJLG3Bt0a+BlkzocXDBsWSgZug54cAxXjwSmqF/JYnBDf8EdkBDSX/ieSZtjfS0b krc8yFOHsvUq8j5NDMhMEYNOjretGgsuwGC0Xle+INx3asl5yi4IZL4xjhVY4vHKBZhJ eOLC/6U6g76lK2qc9v30VgeKaU9tME6bRfTQye93A5mp9s6aF5t/Y6I6p5BzJDGcxygl Xc7DwQ/5RjsX/ChEisfLQXDZYdo/Hnh4n7wYSy6SU93ShZw8+fDlPHlF3V8PVX/nJjHh FePaB9jJSw5OL3anS9Mw6XT+KCUr7G81RWAhrdOIYrWbqMjihbqLqZEySxK1AGOLAvlH FUbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yy5UmxkIpfJd1M/aii5v5guWV9r1++Cv7FwGzLWJZU0=; fh=b/0dXqvz0kAo+07ch4KNz5dZ+JhJoOmBzkQIhQsuGPg=; b=JcZiOXxvwEb6hXa0VR6XjJMoKQ410Hw8IMGSzNhQMkCeBNiuOvQDCq7S9TMkKm1L52 o/c0fS4O3eJqQqBBDN5Wiy7P4XxVabT76Dr6h4WVw6iMrnYfVw31tslf5e2CrdlAsXP7 7ORislvlj0Cp49QrYN3RKpGsCEVCCdNmKq5Uochf6eGPYVbPuFypaXSX+2KgsR9VwxoR ZJkKa2ddF5nDZe3hJEgDyVXj7+YnnTp3Q8m1eXPPKFcsiX7CbN+FHYKJmdpJBEhvxXT7 ibqd0DtIyca8qByt2aahaV+N3XeuPP339CmG1ewocEN3xasQQeqJaQZYNXwXGBXFADzw WQ8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=n0sLQb1o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a19-20020a1709064a5300b0098722b28a18si2152411ejv.458.2023.09.08.17.41.59; Fri, 08 Sep 2023 17:42:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=n0sLQb1o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344537AbjIHUhB (ORCPT + 16 others); Fri, 8 Sep 2023 16:37:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236159AbjIHUhA (ORCPT ); Fri, 8 Sep 2023 16:37:00 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80B71E65; Fri, 8 Sep 2023 13:36:38 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BF15FC433CC; Fri, 8 Sep 2023 20:36:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694205397; bh=Myb0x4KbAYJfwZIc0cipS2HFSdskG2s5T9FsIJvmid8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n0sLQb1ovYq4ry65mQo6nbCRm0BjIK14Q2Y2YTtP6N/G9GgJYl8Prsmdnw+BDxezF Suqz6NvTkljdrVQtGPL8+4AfFKJXj8JkWH2EKQSP3VrWg8X/MEx/RF/Kr8qn3IHOph xktc8JtPLSuSDv9HSn3VpgfpKVkyekrfBMUlFL8eCi2MXlBXisR3d7r2jwOUdolRd+ e6irVd/fD8CK/MYu3uyV7sX9ofnl6AWI0WpzEI1ydA3ScvxKGBA7LAvzy2osae359i G8YDLc+OkrB8xt9vsPXMtjdTWF/vTZNg7kIBPM4xwn5k9frlEm5V80/6/0k4bThKf6 6nzjFgjkIwhVw== From: Frederic Weisbecker To: "Paul E . McKenney" Cc: LKML , Frederic Weisbecker , rcu , Uladzislau Rezki , Neeraj Upadhyay , Boqun Feng , Joel Fernandes Subject: [PATCH 10/10] rcu: Comment why callbacks migration can't wait for CPUHP_RCUTREE_PREP Date: Fri, 8 Sep 2023 22:36:03 +0200 Message-ID: <20230908203603.5865-11-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230908203603.5865-1-frederic@kernel.org> References: <20230908203603.5865-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776518560043131719 X-GMAIL-MSGID: 1776518560043131719 The callbacks migration is performed through an explicit call from the hotplug control CPU right after the death of the target CPU and before proceeding with the CPUHP_ teardown functions. This is unusual but necessary and yet uncommented. Summarize the reason as explained in the changelog of: a58163d8ca2c (rcu: Migrate callbacks earlier in the CPU-offline timeline) Signed-off-by: Frederic Weisbecker Reviewed-by: Paul E. McKenney --- kernel/cpu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/cpu.c b/kernel/cpu.c index a41a6fff3c91..b135bb481be1 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1352,7 +1352,14 @@ static int takedown_cpu(unsigned int cpu) cpuhp_bp_sync_dead(cpu); tick_cleanup_dead_cpu(cpu); + + /* + * Callbacks must be re-integrated right away to the RCU state machine. + * Otherwise an RCU callback could block a further teardown function + * waiting for its completion. + */ rcutree_migrate_callbacks(cpu); + return 0; }