From patchwork Tue Oct 24 21:46:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 157708 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp2220062vqx; Tue, 24 Oct 2023 14:47:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFZ52/EibGeFbvFx3NUZXEApEIUKb8UrVt2w8W2DX7E0QlRUMZjlJ+7OS9KgfUfkFbAbBr/ X-Received: by 2002:a17:902:ec8e:b0:1c6:d70:144d with SMTP id x14-20020a170902ec8e00b001c60d70144dmr14875766plg.34.1698184023506; Tue, 24 Oct 2023 14:47:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698184023; cv=none; d=google.com; s=arc-20160816; b=uU1Vs4Vl8t+fBdoup+gZz37mxnqqqbtMtUuH9zc4By/SIgKXvdpsofq6YVr7/7CUYx 0zVcKTFgJfX4lGlnRMy1CGsIhuKBD3T/p26fc0EpNS17qdPiVVb3LwfvOX51/iiyCWFv +mjoTUWl/Otpcb3B4IisCzs0YPWrtZ3ZX5WtAIvtIXGZQec58SI0ybO1Txv0Fz5IPtF2 krezsEKpNDgImDKqA9BFosEK8xDgNGc3zXvkmiEYk+0oIHM3YM7QyIg34toMa73b9Wlg 9AxAITHjpoP4lT5N+SpZabC5coKhJTynNPhHp0tAodZx996dqliINNGe4GNIhfUqxd+2 8IYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=43+qOvO1pSIEkar0nm9UeJWflzC/otSuyoj0sGvmF2U=; fh=vRLsh+mlJ/3erNsj9qJLuZWu4oX6irbICpSWFnY9W1o=; b=dvFmATwS51VnEh1uhsjB9prwx1iX/II1Q5pend6pewqhGgNb5Gt3m9mlc3X76mSu7i epKhD8FDOpkmHRwY7sd3Tgklqkf0UxD7ulbrLGUbxUlZWqCSZlMIifVWLYMNox5ZWE6t fkHP0QET5pFf/GvQEGAJT+oGDtJgXvtb8gKs6PlOZDiRL5Ugw2v/MpgrJAvqJBHtY05m jK/dibWKO0lixOBBUqF6zHlcREronM6RgTBcy89wJjaIY8hAp8nDWuy2EAfLaca+NTGI bPV68NndzC7wO0nnnobd72ZBWqRqbipHiWKroIW76YW+FNzmrRy3sXbHv2reaxNkbG0c DlKA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qJwBiK2z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id w2-20020a1709029a8200b001bb9533b968si8606327plp.408.2023.10.24.14.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 14:47:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qJwBiK2z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 24C9680BD532; Tue, 24 Oct 2023 14:47:01 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344459AbjJXVqm (ORCPT + 27 others); Tue, 24 Oct 2023 17:46:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344454AbjJXVqk (ORCPT ); Tue, 24 Oct 2023 17:46:40 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17440E8; Tue, 24 Oct 2023 14:46:38 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D70CC433CA; Tue, 24 Oct 2023 21:46:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698183997; bh=55r/JWo6Uc73yf5DqaZWWtL9jn0mpU9zwuvJddzRrBI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qJwBiK2zpYQrDryXd6sMwPwBHy1iWVHeuLLm1/SKdspkdlBLaYb177rzprEGFzQFC d/2ylslU8EmC2+4Sxk4BW38uhxydzZeW+CDALQ60QUQXXXMYjKOENLFfZDMsQ7glEN 8TPMKcAiWCjVXy86PTUMPR4BEzvx+20j7GxF0ihu6H1skiQnZQQdkTR+l8JG5Oe8yb qTLqfRwx7ExEu1za8vQIQvLUB4dJIgVHGysKG9CVdjrd+GFhiUBp0cCxXA3ht+hfqb Yy0yzvn+ueDYrOWnH1UYuicR+p70IkpRclVnhpdORftbhzg+rh52jd3Uz8zDi6+0G/ yVG9Kidh8eqhw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , "Liam R . Howlett" , Peter Zijlstra Subject: [PATCH 1/4] rcu: Introduce rcu_cpu_online() Date: Tue, 24 Oct 2023 23:46:22 +0200 Message-ID: <20231024214625.6483-2-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024214625.6483-1-frederic@kernel.org> References: <20231024214625.6483-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 24 Oct 2023 14:47:01 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780675010311545082 X-GMAIL-MSGID: 1780675010311545082 Export the RCU point of view as to when a CPU is considered offline (ie: when does RCU consider that a CPU is sufficiently down in the hotplug process to not feature any possible read side). This will be used by RCU-tasks whose vision of an offline CPU should reasonably match the one of RCU core. Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Signed-off-by: Frederic Weisbecker --- kernel/rcu/rcu.h | 2 ++ kernel/rcu/tree.c | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index 0d866eaa4cc8..b531c33e9545 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -500,6 +500,7 @@ static inline void rcu_expedite_gp(void) { } static inline void rcu_unexpedite_gp(void) { } static inline void rcu_async_hurry(void) { } static inline void rcu_async_relax(void) { } +static inline bool rcu_cpu_online(int cpu) { return true; } #else /* #ifdef CONFIG_TINY_RCU */ bool rcu_gp_is_normal(void); /* Internal RCU use. */ bool rcu_gp_is_expedited(void); /* Internal RCU use. */ @@ -509,6 +510,7 @@ void rcu_unexpedite_gp(void); void rcu_async_hurry(void); void rcu_async_relax(void); void rcupdate_announce_bootup_oddness(void); +bool rcu_cpu_online(int cpu); #ifdef CONFIG_TASKS_RCU_GENERIC void show_rcu_tasks_gp_kthreads(void); #else /* #ifdef CONFIG_TASKS_RCU_GENERIC */ diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 700524726079..fd21c1506092 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -4202,6 +4202,13 @@ static bool rcu_rdp_cpu_online(struct rcu_data *rdp) return !!(rdp->grpmask & rcu_rnp_online_cpus(rdp->mynode)); } +bool rcu_cpu_online(int cpu) +{ + struct rcu_data *rdp = this_cpu_ptr(&rcu_data); + + return rcu_rdp_cpu_online(rdp); +} + #if defined(CONFIG_PROVE_RCU) && defined(CONFIG_HOTPLUG_CPU) /* From patchwork Tue Oct 24 21:46:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 157707 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp2220014vqx; Tue, 24 Oct 2023 14:46:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEv5NUWEbp9Eeios7aBK48R+NEG1rX5uHsX9Zry+kG089T7jNeI5IHKNkIJaiZ1U5Asm8bw X-Received: by 2002:a05:6a20:144c:b0:13d:5b8e:db83 with SMTP id a12-20020a056a20144c00b0013d5b8edb83mr4231976pzi.9.1698184014415; Tue, 24 Oct 2023 14:46:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698184014; cv=none; d=google.com; s=arc-20160816; b=e0G/r5Yj9jaJTYTlI3/m1hW5AUz49gaKuxJwWhhjnVasmWPEQOa+GHqMToc2SO9Rbv 8ScGzuKCuGF0IumV9jKHOfgR8WBwPXLT2iNNcB3FCYiu/mO0i+U32vAknDrJh4oQLcT8 1ErUiA1O2+XqhOV0n46XWNwGJlnV8SB8u2nX9ybHhU4NeZX4hHLbqiXdXXpIKObXJLPo XhkyALS4PyaRwRnKHQyaPBpogfu8QYP/PktJZkBWZb9NICB9jHQgEeLMetb/xliMXd80 ySv2Kbdn0r7mFe9sr/G1HQ+nEArY7h4xR6RBGirZIHtLax54Rk0lLhiclOlsXMQ0IY6A CNCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=v+7GjAyILigS8hAAFW+aJ3436C0aXKncZVkr0C9NJz4=; fh=vRLsh+mlJ/3erNsj9qJLuZWu4oX6irbICpSWFnY9W1o=; b=0ycJQHfueN3OnHgmy0j4ruzThDLWTQDLDbbmEfzELqRVsevbumKjstIvw+2UDweumn Fh2YCacP7Vvi9Gbnhs3SVEyDa+UJHcPiAl/PE1AM2dZXe8TPzZtpK9tzwZhoKb7wV5yQ PktG9Zp3He8V2FAOGV1+Wk/Mll0QCm0cQTeAIslBbm/BJ9bW2lcveHn/GHno+cWDC7JR zEIKcCFFUIv5weul9qdUXmtDZ2lcr7zL8fPhetYbTmkcpElYNUHk90yGD5NRzNXRuzbs GNHdn8HjfQ6/BKelkjnDeFWGBLqNMgupYcD8blMftYRmsSYYftTittc0VDsxk4gh4qno 3wKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="au3/Ymnr"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id j12-20020a056a00174c00b006be55174f3fsi9394762pfc.28.2023.10.24.14.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 14:46:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="au3/Ymnr"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 5F27D802767B; Tue, 24 Oct 2023 14:46:52 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234979AbjJXVqt (ORCPT + 27 others); Tue, 24 Oct 2023 17:46:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234973AbjJXVqq (ORCPT ); Tue, 24 Oct 2023 17:46:46 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A75410D7; Tue, 24 Oct 2023 14:46:42 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F750C433C9; Tue, 24 Oct 2023 21:46:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698184001; bh=VF4rwPu4+YtRb7YG4mHtznxP8H/nXqrrus0i9svAMXk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=au3/YmnrmAJQJG631/cxb4ORAO9pRQTWXnK7H6KnQmVugQQJmTa3DGjZcpQw9ePHJ VdX4Ifg7c8hlUO3kYIPVedfUuNpoouQDRjZIwigFJ5/PmE7rSzXFb9v8iDKOJcliSR HgWUCmh1iEQyjGwkEZxVBM6mSAlAQmfGiMlCD5A/OtOAJPxJyw+h+ApaZRgo0tcomy WmAMclPVf8javsIc5NZGs+kYXXeO8I0+YOzKenD/QkuKkpVtgeK7ciGyjzLbweSBWv 4lZLEwd3THfxpNQ7GMPc5W8yr/npFOeo0iM66/n73ymQi9DO+Zs85gzU/c+TmxleVd 5PiXlKNNqLlqw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , "Liam R . Howlett" , Peter Zijlstra Subject: [PATCH 2/4] rcu/tasks: Handle new PF_IDLE semantics Date: Tue, 24 Oct 2023 23:46:23 +0200 Message-ID: <20231024214625.6483-3-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024214625.6483-1-frederic@kernel.org> References: <20231024214625.6483-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 24 Oct 2023 14:46:52 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780675001053828841 X-GMAIL-MSGID: 1780675001053828841 The commit: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") has changed the semantics of what is to be considered an idle task in such a way that CPU boot code preceding the actual idle loop is excluded from it. This has however introduced new potential RCU-tasks stalls when either: 1) Grace period is started before init/0 had a chance to set PF_IDLE, keeping it stuck in the holdout list until idle ever schedules. 2) Grace period is started when some possible CPUs have never been online, keeping their idle tasks stuck in the holdout list until the CPU ever boots up. 3) Similar to 1) but with secondary CPUs: Grace period is started concurrently with secondary CPU booting, putting its idle task in the holdout list because PF_IDLE isn't yet observed on it. It stays then stuck in the holdout list until that CPU ever schedules. The effect is mitigated here by the hotplug AP thread that must run to bring the CPU up. Fix this with handling the new semantics of PF_IDLE, keeping in mind that it may or may not be set on an idle task. Take advantage of that to strengthen the coverage of an RCU-tasks quiescent state within an idle task, excluding the CPU boot code from it. Only the code running within the idle loop is now a quiescent state, along with offline CPUs. Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Suggested-by: Joel Fernandes Suggested-by: Paul E . McKenney" Signed-off-by: Frederic Weisbecker --- kernel/rcu/tasks.h | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index bf5f178fe723..acf81efe5eff 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -895,10 +895,37 @@ static void rcu_tasks_pregp_step(struct list_head *hop) synchronize_rcu(); } +/* Check for quiescent states since the pregp's synchronize_rcu() */ +static bool rcu_tasks_is_holdout(struct task_struct *t) +{ + int cpu; + + /* Has the task been seen voluntarily sleeping? */ + if (!READ_ONCE(t->on_rq)) + return false; + + cpu = task_cpu(t); + + /* + * Idle tasks within the idle loop or offline CPUs are RCU-tasks + * quiescent states. But CPU boot code performed by the idle task + * isn't a quiescent state. + */ + if (t == idle_task(cpu)) { + if (is_idle_task(t)) + return false; + + if (!rcu_cpu_online(cpu)) + return false; + } + + return true; +} + /* Per-task initial processing. */ static void rcu_tasks_pertask(struct task_struct *t, struct list_head *hop) { - if (t != current && READ_ONCE(t->on_rq) && !is_idle_task(t)) { + if (t != current && rcu_tasks_is_holdout(t)) { get_task_struct(t); t->rcu_tasks_nvcsw = READ_ONCE(t->nvcsw); WRITE_ONCE(t->rcu_tasks_holdout, true); @@ -947,7 +974,7 @@ static void check_holdout_task(struct task_struct *t, if (!READ_ONCE(t->rcu_tasks_holdout) || t->rcu_tasks_nvcsw != READ_ONCE(t->nvcsw) || - !READ_ONCE(t->on_rq) || + !rcu_tasks_is_holdout(t) || (IS_ENABLED(CONFIG_NO_HZ_FULL) && !is_idle_task(t) && READ_ONCE(t->rcu_tasks_idle_cpu) >= 0)) { WRITE_ONCE(t->rcu_tasks_holdout, false); From patchwork Tue Oct 24 21:46:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 157709 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp2220072vqx; Tue, 24 Oct 2023 14:47:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHySQa6T/seXITsCOFajM9D+O+vj4cf2WK4p1ZE1Qu586obxg570P0i9pCdQoE+X97tIsGl X-Received: by 2002:a17:90a:1c8:b0:27d:2109:6279 with SMTP id 8-20020a17090a01c800b0027d21096279mr10538898pjd.12.1698184024847; Tue, 24 Oct 2023 14:47:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698184024; cv=none; d=google.com; s=arc-20160816; b=Rn07XktGT5+UMatP1yOs4ZsM0IYPVF7s6M3WkccoEIrcUvMv4/MJFMO9YqV4MNyxnJ xqslA9+3C80DbTItmJLs1QbPy23D1YboxGtktqdGbWv18eZOTLzllEpNaJNebguyORRW k+z4OHwzGr8HsU/5jF2WeVojq07p+chChWuapdfAGsL0eX9yFYCWZrrj1OD9cYqZQVP5 NejkalR7jucEG+5Va7lndEc8tjv+PGfbZwwPGoiRCYxMG/jM+kIRzTFvTq5iZ9oDU6AF lsWfGO7aQLwPOs8qV2+UjMifyxxhhq4MgQaBpuEu6YK/XxYcqq4Hpfp/5ntCHSpE2ijz g8Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vtgj67I3koTvKTmVNZcijT3znViLr5LAUhFxAxHC3Zs=; fh=kjhz86PwvLs0DCSx3CYZ93RH/zcHDwqZp0T+KAJhehM=; b=UavXLJqOVOGKh4Ety/5Y+4nmFzIcAt95kF840gBxuEnCMbfFS4SDqtYklAH4m9bUbf +89XN3S2PS3V3wbikpGO9Lz4Pt3DhMP8xgmfLyq2saHeGBvXfXalcIHirTXVLmm04qmB syPCRK/67IbefmT6prKzrCJwydeho3NbAz5ktUXKAV8O28zT3/TMn4YwIqHJGGZo7YFy wYVA6I1eGnqFfGODNk+b2ppquixidPDq/oULCLpmHPe96TYj93d9SkQUfnsMRoq24pTm JucAXSXGuv1lxHqmNEKZsml47EbtmRuvxBYRLDYdfqQexACReCotPnN87BJ17PkT2Ysu DmZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=usKGKhB8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id o16-20020a17090aac1000b002773af24826si11617788pjq.123.2023.10.24.14.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 14:47:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=usKGKhB8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id EE911802209E; Tue, 24 Oct 2023 14:47:01 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344466AbjJXVqx (ORCPT + 27 others); Tue, 24 Oct 2023 17:46:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234984AbjJXVqs (ORCPT ); Tue, 24 Oct 2023 17:46:48 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5F73D7F; Tue, 24 Oct 2023 14:46:45 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0FE7FC433C8; Tue, 24 Oct 2023 21:46:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698184005; bh=SWH3H5Xdn3J5BOiYP+tbzdRJbhkTqpBLrBJhKbzm4Fo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=usKGKhB8JfO8caii44hKkyfsLAu0qLEivqPpVb5HoIyzboKeu0t9NOqlr5pq7VhBt QA6eSDiSI63VG1CG6dLZo9glJsKFKPHBIK1xN/syP7JfBPcjUxlBYf1rMg/r+u788I gP/+BztEMPJ5FUutfZ6KmeUL3wrg42cVHmg0igQIQMnK0sI04vsAiAmmnvmziFhbEM zh4nYA71JTFaW6qLDN/S7uwATgtoGBwyhJuex65txdCp8/8IWlstvWppT4gc+fJyB4 gcZ1zbzHYqU7Wy+743hSyHh2TLnFRa0PSdwqvwWaxOwLyC8AK3Db0MGYsps52afPoQ rn+mH6j3kFvkQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , "Liam R . Howlett" , Peter Zijlstra , Naresh Kamboju Subject: [PATCH 3/4] rcu/tasks-trace: Handle new PF_IDLE semantics Date: Tue, 24 Oct 2023 23:46:24 +0200 Message-ID: <20231024214625.6483-4-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024214625.6483-1-frederic@kernel.org> References: <20231024214625.6483-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 24 Oct 2023 14:47:02 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780675012218676664 X-GMAIL-MSGID: 1780675012218676664 The commit: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") has changed the semantics of what is to be considered an idle task in such a way that the idle task of an offline CPU may not carry the PF_IDLE flag anymore. However RCU-tasks-trace tests the opposite assertion, still assuming that idle tasks carry the PF_IDLE flag during their whole lifecycle. Remove this assumption to avoid spurious warnings but keep the initial test verifying that the idle task is the current task on any offline CPU. Reported-by: Naresh Kamboju Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Suggested-by: Joel Fernandes Suggested-by: Paul E . McKenney" Signed-off-by: Frederic Weisbecker --- kernel/rcu/tasks.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/rcu/tasks.h b/kernel/rcu/tasks.h index acf81efe5eff..4dd70f2af4af 100644 --- a/kernel/rcu/tasks.h +++ b/kernel/rcu/tasks.h @@ -1552,7 +1552,7 @@ static int trc_inspect_reader(struct task_struct *t, void *bhp_in) } else { // The task is not running, so C-language access is safe. nesting = t->trc_reader_nesting; - WARN_ON_ONCE(ofl && task_curr(t) && !is_idle_task(t)); + WARN_ON_ONCE(ofl && task_curr(t) && (t != idle_task(task_cpu(t)))); if (IS_ENABLED(CONFIG_TASKS_TRACE_RCU_READ_MB) && ofl) n_heavy_reader_ofl_updates++; } From patchwork Tue Oct 24 21:46:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 157710 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp2220123vqx; Tue, 24 Oct 2023 14:47:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEB9ZTME3Wk8AVUHciEgNVt2xvU7PBwVYWzWb/zLCm+e9fWaV1SHpSwMEUCGg4MciYhK2d3 X-Received: by 2002:a17:90a:e409:b0:27d:1126:1ff6 with SMTP id hv9-20020a17090ae40900b0027d11261ff6mr23520518pjb.8.1698184035781; Tue, 24 Oct 2023 14:47:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698184035; cv=none; d=google.com; s=arc-20160816; b=r+/rGyQEhYWQ+iwolIpqfVYT845C7Ro1qnH+ABiHJ08p5RP8I26g1Smrwm8agMkmWL gx2z4u2cGLJe8hqaQ3uy9WbvO2Pc2DBnNreuASCrz62wpOBrgKlGjJgW1TepbUKi+tpM +pk9GZLdWwkmZC8nLBaEO7IuqWXrjGPq3rcPDKMvBjspV18M34/7rRwnyiicmWnmA9ab JN+Q7LlMrFw0nRyizwTDilu8j5nEkh5vxRb/YVOtvnxwE8N2SIBay0adGKTc3F2kugo5 hCmgYV8SblKKBhSF+ASC0hkGv5f8A96dCpZ+xyl9M4kMalrtTDR6mGR3iF/mwo0tuYVU M+Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=MYiFxL/NR4y9QBhU4XPHSky0NxYDbU+lNetAadSXM1g=; fh=vRLsh+mlJ/3erNsj9qJLuZWu4oX6irbICpSWFnY9W1o=; b=vHN2LNc0FJ0pG84SZGE8SNa1p6aEn310aZr160wUa7QEOg9APLeloRPyRKhR8kuGkz o0T1Nobiq3z0r1fNfuN/Zib2K2ejiGNTqyGySl5gNw+XsB/Yoz1IewUG8NKfWlpEgudW e6Wu0YWDSxKbvmbi+fGs4xjOPRZEfb07Zfce7OxaP4697fq5rESOChJYi0R+OT1dz/DW QqVqXPV1/Z1ISUsPo1fS7+nv9m4eoYCiH06zPjIEqZgkw3xjxZfpTdEZBV9og+iIx82X pTJ8AatcGp1UQ//cwbDuzkmSUEEfmILpPwCwQuMfTcy+RumHA35CD8DySWqn319qJabv CC7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=EDaO3ZmC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id b1-20020a17090a9bc100b0027491203b43si11070689pjw.189.2023.10.24.14.47.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 14:47:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=EDaO3ZmC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 957F380220A6; Tue, 24 Oct 2023 14:47:13 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344499AbjJXVq5 (ORCPT + 27 others); Tue, 24 Oct 2023 17:46:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344477AbjJXVqw (ORCPT ); Tue, 24 Oct 2023 17:46:52 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C101AE8; Tue, 24 Oct 2023 14:46:49 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE75DC433C7; Tue, 24 Oct 2023 21:46:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698184009; bh=AqHnLeB5xem2kiRsFLowge6w/ot0AAmqtV9xGzicirc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EDaO3ZmCJO3fKoF+gIfWqQroBqbgxx5f8hX2+JrCOc4JsdJNI38tcDbzDbxDd9IiD lQQjafqLkkawzo6WRYBYq6iBDRLc6rw/K82bL4QufhCMuuMtuiIDHLj1HTZLkWwhv0 Y7mPAxMD5NRzchUhYgtYdpyVNGyV6qMUsXDOOPc0ShJDQv6YARd0YrIOFvg4uFaCm6 1ngaB9X+/AarJdFS3TRUPmWau++XTxbJE6uZMiH0YYWipraB8z3fgsqMiLJkFXYbza ZaAmHAP183sCvV3XzoyR81WsOfdotWNwXasO1UWQeKF/ElKrOC4OzjJFDnqlnpa/LJ dKFkktu+IKyVw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Josh Triplett , Lai Jiangshan , Mathieu Desnoyers , Neeraj Upadhyay , "Paul E . McKenney" , Steven Rostedt , Uladzislau Rezki , Zqiang , rcu , "Liam R . Howlett" , Peter Zijlstra Subject: [PATCH 4/4] sched: Exclude CPU boot code from PF_IDLE area Date: Tue, 24 Oct 2023 23:46:25 +0200 Message-ID: <20231024214625.6483-5-frederic@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024214625.6483-1-frederic@kernel.org> References: <20231024214625.6483-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 24 Oct 2023 14:47:13 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780675023493096998 X-GMAIL-MSGID: 1780675023493096998 The commit: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") has changed the semantics of what is to be considered an idle task in such a way that only the actual idle loop is accounted as PF_IDLE. The intent is to exclude the CPU boot code from that coverage. However this doesn't clear the flag when the CPU goes down. Therefore when the CPU goes up again, its boot code is part of the PF_IDLE zone. Make sure this flag behave consistently and clear the flag when a CPU exits from the idle loop. If anything, RCU-tasks relies on it to exclude CPU boot code from its quiescent states. Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup") Signed-off-by: Frederic Weisbecker --- include/linux/sched.h | 2 +- kernel/cpu.c | 4 ++++ kernel/sched/idle.c | 1 - 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 8885be2c143e..ad18962b921d 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1945,7 +1945,7 @@ extern struct task_struct *idle_task(int cpu); */ static __always_inline bool is_idle_task(const struct task_struct *p) { - return !!(p->flags & PF_IDLE); + return !!(READ_ONCE(p->flags) & PF_IDLE); } extern struct task_struct *curr_task(int cpu); diff --git a/kernel/cpu.c b/kernel/cpu.c index 3b9d5c7eb4a2..3a1991010f4e 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1394,7 +1394,9 @@ void cpuhp_report_idle_dead(void) { struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); + WRITE_ONCE(current->flags, current->flags & ~PF_IDLE); BUG_ON(st->state != CPUHP_AP_OFFLINE); + rcutree_report_cpu_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* @@ -1642,6 +1644,8 @@ void cpuhp_online_idle(enum cpuhp_state state) { struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); + WRITE_ONCE(current->flags, current->flags | PF_IDLE); + /* Happens for the boot cpu */ if (state != CPUHP_AP_ONLINE_IDLE) return; diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 5007b25c5bc6..342f58a329f5 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -373,7 +373,6 @@ EXPORT_SYMBOL_GPL(play_idle_precise); void cpu_startup_entry(enum cpuhp_state state) { - current->flags |= PF_IDLE; arch_cpu_idle_prepare(); cpuhp_online_idle(state); while (1)