From patchwork Fri Oct 27 14:40:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 158974 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp655525vqb; Fri, 27 Oct 2023 07:43:02 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG+sCyAnXhqYGMMUxkRLH7brJZHqiQLArXk8jJA74kXCvKAZkNVJTKO0RXWd/97oToMinhn X-Received: by 2002:a25:e601:0:b0:da0:3922:d7bb with SMTP id d1-20020a25e601000000b00da03922d7bbmr10553146ybh.17.1698417782448; Fri, 27 Oct 2023 07:43:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698417782; cv=none; d=google.com; s=arc-20160816; b=FdpZHGG9F7kKg+HtjYR6tzndNCa/AOoHb31VeYhCTdSOysUgoUi2wvAS0ZmcS9XuIw GV58NpIDilKMRMCrwX4MqrOgTOC/PoLSyonqWSSFSkmoM7lPueZjUSjTMWaAcbq/mRAf 9EvtcZUEki1FFgFdLDoVjn6XJIGYkhRWaZObGOHuFN6kMRPNmOPLZcyQpSgP5apxjaTh piTKviVozA5m7taVrzLajbrByj7kcF+4FWmIpHPuzz2KmNatneZRROZHHXqWLzvMylBo 08WLovvz60vQxwlCxCZjQUpmI1ar5h2zc5C/9Ba+ntNt2Zd9NpjExqjjTs1kRaaeKWsy gZYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=oPJ6O5eps6NDOQjKmjqdygoO2PHIy2OS60u5mwzHI8s=; fh=Z5RfXUdA7RqZB5lwRaxglRkGuBAkH08O6zmnqH6r62g=; b=L6PmwyCOCsLncI7V+c22LF5XEmZs5VVBo7qMop61C+OWjHfP+sswEIhIFni4I58oqq /nujhBBskQNtvFCxvXP8MUFIH797Lv2RNSfXW8JAB1hZQ5MlK2TsTGI9sw+mqL7MJ6O8 Hph6lqt8k9HRg+kYzY4nh6+5XaFXQl0OWFx7UJV3fggaFYI2srWs3MW5q1IStsrf4KRu RnOsDEAtygiE6hCWiANnsAKow5PgjoObS0AhupfIZB7y7UNmCnFQEhuVt/rODcHeHwV9 W8AcvRTgV0pLEW90/bRcN9pIzVtg04kOaBVwe0LoDSyJeKJ8MmTl5rrfRiJJHoqPYyha /xXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=FfBAW1p2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id 204-20020a2503d5000000b00da1c6c86120si1907004ybd.560.2023.10.27.07.42.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Oct 2023 07:43:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=FfBAW1p2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 17472801B335; Fri, 27 Oct 2023 07:41:16 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345914AbjJ0OlI (ORCPT + 25 others); Fri, 27 Oct 2023 10:41:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345868AbjJ0OlF (ORCPT ); Fri, 27 Oct 2023 10:41:05 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C10F0C4; Fri, 27 Oct 2023 07:41:03 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95924C43395; Fri, 27 Oct 2023 14:41:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698417663; bh=quyhYQ3j6IeBsmp/D+pIjVv3njLC2nvqNbVgvWxgsXw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FfBAW1p2HHmIIhCOdMUI7oJ2srPdQzfPdJX8oxj2aZqpMy+fiZbLneWX9qNACJvjl pde4exRsDU0/CsbJjMyjLJM/+HmivjhoJQ/CY3Ti/mGKBN9HwHVkoIa1gPIBw4sQEL QAKlUgdR4LWAOlYyAuCjdLBiyrHI2t6PjVA56/yqzCExU62QzPqFoVPuCU9qLPPc2g tbp4qrhmfu23Mz+TdxTr/psCUXAV7sved5Y82f5Fjoofj+U08uM31AX22BBputrdso FUw1LzcjzzpCVtvMEQN0UtJcZA9kEKqdn8DUPzexWFEK1lvodM4tf6tdTzgv8Ur9nJ vOrbOREDggimQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Zqiang , "Liam R . Howlett" , Peter Zijlstra Subject: [PATCH 1/4] rcu: Introduce rcu_cpu_online() Date: Fri, 27 Oct 2023 16:40:47 +0200 Message-Id: <20231027144050.110601-2-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231027144050.110601-1-frederic@kernel.org> References: <20231027144050.110601-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 27 Oct 2023 07:41:17 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780920124538465025 X-GMAIL-MSGID: 1780920124538465025 Export the RCU point of view as to when a CPU is considered offline (ie: when does RCU consider that a CPU is sufficiently down in the hotplug process to not feature any possible read side). This will be used by RCU-tasks whose vision of an offline CPU should reasonably match the one of RCU core. Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Signed-off-by: Frederic Weisbecker --- kernel/rcu/rcu.h | 2 ++ kernel/rcu/tree.c | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index 0d866eaa4cc8..b531c33e9545 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -500,6 +500,7 @@ static inline void rcu_expedite_gp(void) { } static inline void rcu_unexpedite_gp(void) { } static inline void rcu_async_hurry(void) { } static inline void rcu_async_relax(void) { } +static inline bool rcu_cpu_online(int cpu) { return true; } #else /* #ifdef CONFIG_TINY_RCU */ bool rcu_gp_is_normal(void); /* Internal RCU use. */ bool rcu_gp_is_expedited(void); /* Internal RCU use. */ @@ -509,6 +510,7 @@ void rcu_unexpedite_gp(void); void rcu_async_hurry(void); void rcu_async_relax(void); void rcupdate_announce_bootup_oddness(void); +bool rcu_cpu_online(int cpu); #ifdef CONFIG_TASKS_RCU_GENERIC void show_rcu_tasks_gp_kthreads(void); #else /* #ifdef CONFIG_TASKS_RCU_GENERIC */ diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 700524726079..54f04c1aa371 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4202,6 +4202,13 @@ static bool rcu_rdp_cpu_online(struct rcu_data *rdp) return !!(rdp->grpmask & rcu_rnp_online_cpus(rdp->mynode)); } +bool rcu_cpu_online(int cpu) +{ + struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu); + + return rcu_rdp_cpu_online(rdp); +} + #if defined(CONFIG_PROVE_RCU) && defined(CONFIG_HOTPLUG_CPU) /* From patchwork Fri Oct 27 14:40:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 158972 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp654363vqb; Fri, 27 Oct 2023 07:41:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGQ1yM7NDG28XmZatiSALFpqR4KB8FxPOOdsgn7WEh+uLUCXzwjt+ztgOg12WW921tkbZY+ X-Received: by 2002:a05:6808:28c:b0:3a7:2598:ab2c with SMTP id z12-20020a056808028c00b003a72598ab2cmr2590052oic.7.1698417698597; Fri, 27 Oct 2023 07:41:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698417698; cv=none; d=google.com; s=arc-20160816; b=FGORLn6IA+5EYnfN0iLgDsX+Lw+K12q5tbIiZ831zeDUaTpZbge2TOqljYiYb7dayG esklgygoGyVUrvhpG0t55mZW98e5wqejqVEm78SpQGlRpdVy3cfcykzaONmD5p2eoHee qlc2t3Iq7OtzmtZ84qd1FO3prd80rFzMW2Y3O4JD9PhdncU+HHiMf0uleebpMq/hl2aN rPINlBXGwv2cBBEIjufj0fCRMP1JmN1ZMdWfgt9zIEIbKKc3q/K0laMr7fhdoDGBazTE 4KJ61WFsHnQA0+V5CrO7pfXf2PAHSlDI1S8aTU4LwA6J6DYb4+bj2MV+qAe7kU6m/vcV 3urg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ZdMcBX28/OOJ17CEoIzmn2m4naKz9LL++9bBrVcnuWE=; fh=Z5RfXUdA7RqZB5lwRaxglRkGuBAkH08O6zmnqH6r62g=; b=n6wZToczSJnQv0fLyMN/cvXwUcPNeW4cdjsHv5iYN5x4tgpRmyqw9Qj8Zft09iGay5 gNf5g3aGtGgz7zp1FT7Z0gd5iM0fhem5a5trxUNzglo4A3bkU2689h7Z1jVIAjKdE5gH Iz2vrE+8+q4xc3tTWI3AIY5xVsjwqqrGsyPwiABHQwhOHZJOCqjCFn6ImSfkzOY2AR6r 8nkH3LRM9RhHpUZS7GNYyApmi9XcxKEYohnJ3hUJ5aanvE/ZHHe5sC5Qx4bQ5L5p0spO mHwWrbPibZ81iZ9PBdccX2ANzAubJLiH9v3HCcadzSsjhHN/koVVfXoLJDjPI79X0DiV U1kA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PFegXcS4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id v30-20020a25fc1e000000b00da0c7aa2504si2852244ybd.709.2023.10.27.07.41.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Oct 2023 07:41:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PFegXcS4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 00B308301511; Fri, 27 Oct 2023 07:41:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345981AbjJ0OlL (ORCPT + 25 others); Fri, 27 Oct 2023 10:41:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345868AbjJ0OlJ (ORCPT ); Fri, 27 Oct 2023 10:41:09 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10A3311F; Fri, 27 Oct 2023 07:41:07 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7EA9C433A9; Fri, 27 Oct 2023 14:41:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698417666; bh=aJr8jQIzF20EIQq6xhSUAOn5x5pigArjiu7D5NS7zLY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PFegXcS4zQj7CCraC8MJw6fNVlpUO/oO4mmGf3ki+qHwWjqKacCtoXEDbxT0h9jWX auZ0eB+YbwgXXYRtRHnamL0OFTlXSsi0BhFauWn0EogHhnMV8YRI4WekODYPlOIUA/ 7fNaBqicwOWC6ceKEZ7Jlt5WKRAENgiue7s3kgiu2Zt+L4jZrlOhTyQ7dwl16p8WEC h33z7TzjAt1B2B604vErrCdDh+dgwhpNZQPOfwTQJEbhBBKyHjeueQgfk/UxKDpN8a JvHeqweFw1j5+k9+9ALT+6HqmVNFxoqE2bHT+ae+TyZOLNet6dbUo4x82d+PBL2ja+ dzKBLmIyJUK+w== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Zqiang , "Liam R . Howlett" , Peter Zijlstra Subject: [PATCH 2/4] rcu/tasks: Handle new PF_IDLE semantics Date: Fri, 27 Oct 2023 16:40:48 +0200 Message-Id: <20231027144050.110601-3-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231027144050.110601-1-frederic@kernel.org> References: <20231027144050.110601-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 27 Oct 2023 07:41:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780920036640439132 X-GMAIL-MSGID: 1780920036640439132 The commit: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") has changed the semantics of what is to be considered an idle task in such a way that CPU boot code preceding the actual idle loop is excluded from it. This has however introduced new potential RCU-tasks stalls when either: 1) Grace period is started before init/0 had a chance to set PF_IDLE, keeping it stuck in the holdout list until idle ever schedules. 2) Grace period is started when some possible CPUs have never been online, keeping their idle tasks stuck in the holdout list until the CPU ever boots up. 3) Similar to 1) but with secondary CPUs: Grace period is started concurrently with secondary CPU booting, putting its idle task in the holdout list because PF_IDLE isn't yet observed on it. It stays then stuck in the holdout list until that CPU ever schedules. The effect is mitigated here by the hotplug AP thread that must run to bring the CPU up. Fix this with handling the new semantics of PF_IDLE, keeping in mind that it may or may not be set on an idle task. Take advantage of that to strengthen the coverage of an RCU-tasks quiescent state within an idle task, excluding the CPU boot code from it. Only the code running within the idle loop is now a quiescent state, along with offline CPUs. Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Suggested-by: Joel Fernandes Suggested-by: Paul E . McKenney" Signed-off-by: Frederic Weisbecker --- kernel/rcu/tasks.h | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index bf5f178fe723..a604f59aee0b 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -895,10 +895,36 @@ static void rcu_tasks_pregp_step(struct list_head *hop) synchronize_rcu(); } +/* Check for quiescent states since the pregp's synchronize_rcu() */ +static bool rcu_tasks_is_holdout(struct task_struct *t) +{ + int cpu; + + /* Has the task been seen voluntarily sleeping? */ + if (!READ_ONCE(t->on_rq)) + return false; + + /* + * Idle tasks (or idle injection) within the idle loop are RCU-tasks + * quiescent states. But CPU boot code performed by the idle task + * isn't a quiescent state. + */ + if (is_idle_task(t)) + return false; + + cpu = task_cpu(t); + + /* Idle tasks on offline CPUs are RCU-tasks quiescent states. */ + if (t == idle_task(cpu) && !rcu_cpu_online(cpu)) + return false; + + return true; +} + /* Per-task initial processing. */ static void rcu_tasks_pertask(struct task_struct *t, struct list_head *hop) { - if (t != current && READ_ONCE(t->on_rq) && !is_idle_task(t)) { + if (t != current && rcu_tasks_is_holdout(t)) { get_task_struct(t); t->rcu_tasks_nvcsw = READ_ONCE(t->nvcsw); WRITE_ONCE(t->rcu_tasks_holdout, true); @@ -947,7 +973,7 @@ static void check_holdout_task(struct task_struct *t, if (!READ_ONCE(t->rcu_tasks_holdout) || t->rcu_tasks_nvcsw != READ_ONCE(t->nvcsw) || - !READ_ONCE(t->on_rq) || + !rcu_tasks_is_holdout(t) || (IS_ENABLED(CONFIG_NO_HZ_FULL) && !is_idle_task(t) && READ_ONCE(t->rcu_tasks_idle_cpu) >= 0)) { WRITE_ONCE(t->rcu_tasks_holdout, false); From patchwork Fri Oct 27 14:40:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 158975 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp655528vqb; Fri, 27 Oct 2023 07:43:02 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGtEWtaf21f4e81EcrUTjjXYcp/cE55V8eFKB+sAaYLRdViMsu+fpk78K9T+6oMC6AiVSqM X-Received: by 2002:a81:bd41:0:b0:5a7:b893:6f64 with SMTP id n1-20020a81bd41000000b005a7b8936f64mr2673725ywk.16.1698417782682; Fri, 27 Oct 2023 07:43:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698417782; cv=none; d=google.com; s=arc-20160816; b=VWxJfyWljljIBm8zOYAnuZt+zvuMdiIRRp5UxRm27x6DPOxWVjrasN+Tn6LXl2m8GZ z1Wrd/ZpgoZLpT8M52AoZmTOKjxS/78ns5nV5Ew3m7ls/jhBUnwupGpwFCKw8KRVp2js v5tonEnzNgFLDf5C5xPUNX2dCpBkK6D98s1UTxUCVERb5XKZyv3QjwCdLlTpDrhlxmUB R7L490HQwAUWmqmiO5pLpFIrASdQFterwfJ9EarcXKBpkSF91aE2YhJY+zZR4GG6ksC6 J0uVAsiA8jHA/8KIF2nIEhUAVtkEzs3Z+MxSz/p4JD9c07L7o6qKXMjys/SNpeINSioj RFww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zEXPCBMJ1AEA/oLOYEucr76mHLC2MYFA9+I5Pi5epjw=; fh=iUkoi/K8IT/RJJVBdiQ8XO07tR0wG9xibh5YtxOmKQw=; b=plr6C7NCtas082e1tFJm0XKK2h6YuXxGIm+f2Tc2s73aOerc0VrVIPKLESxXlo9kRy uw8ATRVG2cGSBnn76Z0kZNAeLdQJyQoUZ6p9kXKb9nHI3aZ9qxeC1n/OCCjcege2zHha W39kdi0/QP0IMZ46slAqhp80m0fhvk1ELVisJvnY1y9NW+ZL9UPzEtyhzprBffz5iWEG 56qAFwrjWAzKM99uMJwGqS3YGHK5ALTTpxfh21d/MW/dVSBtgtnPY+UIXrB6i2PBCjgM MPsjUVRc5kAsWeCp/xyqwZjDLPYspb//LSaWcy8Y7WJk0AWCpGnEi7pE8GM02dsG/Q/S GHVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Wf3XrB4l; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id x127-20020a81a085000000b005a7af4e22a8si2607605ywg.29.2023.10.27.07.42.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Oct 2023 07:43:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Wf3XrB4l; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 006E98040EB9; Fri, 27 Oct 2023 07:41:19 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346000AbjJ0OlO (ORCPT + 25 others); Fri, 27 Oct 2023 10:41:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346044AbjJ0OlM (ORCPT ); Fri, 27 Oct 2023 10:41:12 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83935194; Fri, 27 Oct 2023 07:41:10 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1E9E5C433AB; Fri, 27 Oct 2023 14:41:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698417670; bh=1Gy2J4196Ason/4s+Q9DcJjUAQ+dsGogthDv6LJxxNs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wf3XrB4lbonjZ8b33Z/v5oowVFoMi9EgCyHv+7NQ4QwyevXljSj2ViB9KjmbkkCqf K2of/gnVoYyBDuYM1WWpRf+guN9Ka6iJMgrP05eTQ1cT7fhn5jIgCi9owyDdoHGc9V IXEg0Kf6DonMuMOmePd4mCY8YLDYisAYaZ0y57KDLAVzwn7rH9CoCUlyOP1vlqU51C MHF/EBlT15HIyU1fHoALOy0HUfLqC/kOJsgBbQD5yyufZyBHWX8zjez9y9zsQ80y8W QUT5TRyLk5u8+tgcofiL3v9H7CljFt9txqp7JKkeV9wzLXMU7WReeiPzc3h/FaQR+9 zL1XMUmKg0riQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Zqiang , "Liam R . Howlett" , Peter Zijlstra , Naresh Kamboju Subject: [PATCH 3/4] rcu/tasks-trace: Handle new PF_IDLE semantics Date: Fri, 27 Oct 2023 16:40:49 +0200 Message-Id: <20231027144050.110601-4-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231027144050.110601-1-frederic@kernel.org> References: <20231027144050.110601-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 27 Oct 2023 07:41:19 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780920124578160797 X-GMAIL-MSGID: 1780920124578160797 The commit: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") has changed the semantics of what is to be considered an idle task in such a way that the idle task of an offline CPU may not carry the PF_IDLE flag anymore. However RCU-tasks-trace tests the opposite assertion, still assuming that idle tasks carry the PF_IDLE flag during their whole lifecycle. Remove this assumption to avoid spurious warnings but keep the initial test verifying that the idle task is the current task on any offline CPU. Reported-by: Naresh Kamboju Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Suggested-by: Joel Fernandes Suggested-by: Paul E . McKenney" Signed-off-by: Frederic Weisbecker --- kernel/rcu/tasks.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index a604f59aee0b..732ad5b39946 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -1551,7 +1551,7 @@ static int trc_inspect_reader(struct task_struct *t, void *bhp_in) } else { // The task is not running, so C-language access is safe. nesting = t->trc_reader_nesting; - WARN_ON_ONCE(ofl && task_curr(t) && !is_idle_task(t)); + WARN_ON_ONCE(ofl && task_curr(t) && (t != idle_task(task_cpu(t)))); if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB) && ofl) n_heavy_reader_ofl_updates++; } From patchwork Fri Oct 27 14:40:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 158973 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp654427vqb; Fri, 27 Oct 2023 07:41:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IESzHm6NDG9z0LXqKE6WpFud0YI5SDpTLu+/7lle/WdV3AK98fQTO6EOnL37QUoz3T69FIA X-Received: by 2002:a05:6808:20d:b0:3ae:5bf5:4ad1 with SMTP id l13-20020a056808020d00b003ae5bf54ad1mr2686714oie.33.1698417706819; Fri, 27 Oct 2023 07:41:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698417706; cv=none; d=google.com; s=arc-20160816; b=ApcUFA/vtHr38CTWsPPdGC0H0VA9K9jJNSdDGc1FoYqQmQY98ANdwiHgOP5ghfaXwb n6/PUn5bAmYV86P4RvBisLHyDjAdHV5JA0Kmr/zVuDu9bSd3G1hpLBOs+wkGN2AdhV9O XB2EiPxbro8xECt9kBt7ECdV4PvdSZDyHxmHZqx67y9JsJZbHq8Zddls4T8C1swbuuBs ZNM8b1ZNz3w7WbSTQFZZY7lzrYKfW0B/wu2Opbf4kvOJMXtN3+zHEwyu1+ucW2TSc1+3 w+Flv4NpbpNlHPB5oeHYXuV33i38jwOPLDsJSzv/Lftu6pfTotLGF9oADfFaydhc282X FhWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SPDNDE3G1uv6yz5XugN2FxJAra6zoR1QHXnnGzPggtY=; fh=Z5RfXUdA7RqZB5lwRaxglRkGuBAkH08O6zmnqH6r62g=; b=mP/WoOAzpgL1xNT/3GMtE8QdPkGt77Q73XEBi8TvpRJj6OU7H53duz3ZJYJpzjj5bb Ke/77jXBNm+Rb/xwfG/bz/58T6HB6hAyr9XkI43jphrGpaCVQFa2Abg7J+lzOXtZDyfS LNRpAjTpVrf/pzfPokLuA0pdsg785K7SB7P01S0I3AgUXmnWG6wTjwG+aNAF4ELCRDWS O/+IWIIAXW9LlWZkdZ5gxCFh0j9ebIOU0QF6n581RaYUXduKhIO9Ae2/yQI5O6Cab/Ns XFqXKKPd/CnxW6Cj714P0HfQZJi7HF358PV9OnzSyXrIe8KSWTE7jksMguh019ly0cqd lf/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Fu9vUSOw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id 184-20020a2516c1000000b00d7f92144c06si2771292ybw.66.2023.10.27.07.41.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Oct 2023 07:41:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Fu9vUSOw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id DCABB82F7BAC; Fri, 27 Oct 2023 07:41:41 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346111AbjJ0OlY (ORCPT + 25 others); Fri, 27 Oct 2023 10:41:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346133AbjJ0OlU (ORCPT ); Fri, 27 Oct 2023 10:41:20 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A59D31BD; Fri, 27 Oct 2023 07:41:13 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 80D98C433CC; Fri, 27 Oct 2023 14:41:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698417673; bh=yhA516cCxkh8DPn4+h8OsNKdVrTpg3e/ugq1V3UWGyA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Fu9vUSOwcIXxWQg5AG0jUpdX0pWUHbfTHDWhouM4ZnBXmN8yi8Y9li983fAIsOGU0 rDVYLzJOkxHPBjvRliMv3Hn/DcgIFmxpJ/zHNYBn1Gh9/wdzf7ecHL7ywENAh3gzn+ r3kQec48NUbUmUcRFG3ta1x3WG1kYoQ36QIA20/lT5hw/l5Vgs+FdHAO6q6BGlYPF+ 1fvWQtynINo9YWaxL5N1i2OevmmeL4nxkihnKoDCSVXUSKbjsks4iVXlYwjRZNsS77 3pE3MwoHTxLpnL5nPy/+2OyOrElZDzpKdzEJ4H7JvuEEaxC01WSFve5FQpNWsseUWd y6RHTZb+hqL/g== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , rcu , Zqiang , "Liam R . Howlett" , Peter Zijlstra Subject: [PATCH 4/4] sched: Exclude CPU boot code from PF_IDLE area Date: Fri, 27 Oct 2023 16:40:50 +0200 Message-Id: <20231027144050.110601-5-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231027144050.110601-1-frederic@kernel.org> References: <20231027144050.110601-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 27 Oct 2023 07:41:41 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780920045332718757 X-GMAIL-MSGID: 1780920045332718757 The commit: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") has changed the semantics of what is to be considered an idle task in such a way that only the actual idle loop is accounted as PF_IDLE. The intent is to exclude the CPU boot code from that coverage. However this doesn't clear the flag when the CPU goes down. Therefore when the CPU goes up again, its boot code is part of the PF_IDLE zone. Make sure this flag behave consistently and clear the flag when a CPU exits from the idle loop. If anything, RCU-tasks relies on it to exclude CPU boot code from its quiescent states. Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Signed-off-by: Frederic Weisbecker --- kernel/cpu.c | 4 ++++ kernel/sched/idle.c | 1 - 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index 3b9d5c7eb4a2..b24d7937b989 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1394,7 +1394,9 @@ void cpuhp_report_idle_dead(void) { struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); + current->flags &= ~PF_IDLE; BUG_ON(st->state != CPUHP_AP_OFFLINE); + rcutree_report_cpu_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* @@ -1642,6 +1644,8 @@ void cpuhp_online_idle(enum cpuhp_state state) { struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); + current->flags |= PF_IDLE; + /* Happens for the boot cpu */ if (state != CPUHP_AP_ONLINE_IDLE) return; diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 5007b25c5bc6..342f58a329f5 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -373,7 +373,6 @@ EXPORT_SYMBOL_GPL(play_idle_precise); void cpu_startup_entry(enum cpuhp_state state) { - current->flags |= PF_IDLE; arch_cpu_idle_prepare(); cpuhp_online_idle(state); while (1)