From patchwork Mon Dec 18 23:19:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 180679 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp1594265dyi; Mon, 18 Dec 2023 15:19:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IHXaOwDrX5lI3AQWEFZMNjsAQ4WT+OR6FC3OXG5qshF4or1vfJunLu2b6mvCwtspKBS9HJL X-Received: by 2002:a05:6358:441e:b0:172:d191:aee5 with SMTP id z30-20020a056358441e00b00172d191aee5mr3697150rwc.33.1702941594734; Mon, 18 Dec 2023 15:19:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702941594; cv=none; d=google.com; s=arc-20160816; b=qC/W8WppRm9wUaMl2bMdE6COYtIjGmIgW6Ljmg/Aywfsuhq2fR3R7mt9EuHfMOwSLD Ph1uc8yQ+M3Q/dMqYfgNFXg9OvsL7H6IlInasTkoQTUPAE8FiTUiZoGvKuQIxpbgsSk7 tnG8tqp+tuBopDDli8gUb4DybUyt+8gtXb54v167kN6EukUlIqu9RtevE8l1T4hTvm0B Cdwl29UcliKkoHf/0jaQe2PnaH+T97lBpDoUBYERqf26z9iOKbXwh8zQwNLCg14aAk/g D7vFGBCkaUBvNa8DLOTgrwbCTzueis0D07HirnLH5NPKmNxo4TVLWWeYcL4PH7A7D4TG fO9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Asl6aLsRmwP2sFFs9YzqrnABpsTWBrhHNnBKAX0bLZk=; fh=xvjKaM6hmtHqnOhs670mkmt5SmfrGcW6BwluoXDWXqs=; b=a4WTVskS0Mk77Hv67uCfxmtIfpbc8zqu3wxKwEqfwI43U/VU3QHKaJmXbJNTWl7kQL cF9kny2EWJfYRWW3y/zIonprZGZlzUYmY+wwWril35SFTUb8jAv2NbGPZvfSSzY+ZxYR 69WcHWISrMlJSNq3inTk3yTOE/4LIEJVe/Jv/y6UItpxPFpTk0mwZiup1oKSdDqWa46c j1B8kJ0xQuiz+AcNBrIzmxoHYrtnaPAadl8ORbxZ7udVIsORNVMS9F/xqjPb3LlqjZO4 cxvElgwX7L4CauufPzZjNomMUX6aavhLMDsgj3tDgRMPRGuWiuewCQVNvZ7cQ3wLu6s6 ExIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=i0OedXEp; spf=pass (google.com: domain of linux-kernel+bounces-4476-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-4476-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id ei37-20020a056a0080e500b006ce707e00d3si3521833pfb.241.2023.12.18.15.19.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Dec 2023 15:19:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-4476-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=i0OedXEp; spf=pass (google.com: domain of linux-kernel+bounces-4476-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-4476-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 7C0BE2838A1 for ; Mon, 18 Dec 2023 23:19:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 50CF57994D; Mon, 18 Dec 2023 23:19:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="i0OedXEp" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BBC87690D; Mon, 18 Dec 2023 23:19:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DF7EC433C9; Mon, 18 Dec 2023 23:19:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702941566; bh=Jj6CqcZq/pZ6jGPq5AqwOGCyrcsDTwMv9Bv+MG30lpE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i0OedXEpe7TdHAuEUjoZS031nMA8vIi+C1jB4UJK9EyzWV5mDKvo9vEQwWlnsKFlb VJyx6izFRylZqstyJNMQCUmi5pTjUFVDkRvtyVjkH0r2hVK4KzeVOs/u608jbuGsDY ZMGe1pG4kzPypddBqkW9FweztB4mfeP+m9NrGwisBFfWW/gMoSYDcxRTsNGT7+Bqh8 HmLmbVGdpKOmgzg9+5t2CtLlOwuSksH/9RsknFHUtkyHfOhtM2YK0iWwXLx61Cb/s7 bl5hxyARrPV0l9Dtw7ji0RAJN7ewXdITIXOR5JHYAcS7gEdmSWKCRra67QeVO1NyV3 vEUJp6Ll1YXXw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Neeraj Upadhyay , Uladzislau Rezki , Zqiang , rcu , "Paul E . McKenney" , Thomas Gleixner , Peter Zijlstra Subject: [PATCH 1/3] hrtimer: Report offline hrtimer enqueue Date: Tue, 19 Dec 2023 00:19:14 +0100 Message-ID: <20231218231916.11719-2-frederic@kernel.org> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231218231916.11719-1-frederic@kernel.org> References: <20231218231916.11719-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785663685563743556 X-GMAIL-MSGID: 1785663685563743556 The hrtimers migration on CPU-down hotplug process has been moved earlier, before the CPU actually goes to die. This leaves a small window of opportunity to queue an hrtimer in a blind spot, leaving it ignored. For example a practical case has been reported with RCU waking up a SCHED_FIFO task right before the CPUHP_AP_IDLE_DEAD stage, queuing that way a sched/rt timer to the local offline CPU. Make sure such situations never go unnoticed and warn when that happens. Reported-by: Paul E. McKenney Fixes: 5c0930ccaad5 ("hrtimers: Push pending hrtimers away from outgoing CPU earlier") Signed-off-by: Frederic Weisbecker --- include/linux/hrtimer.h | 3 ++- kernel/time/hrtimer.c | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h index f2044d5a652b..f0204630a443 100644 --- a/include/linux/hrtimer.h +++ b/include/linux/hrtimer.h @@ -219,7 +219,8 @@ struct hrtimer_cpu_base { unsigned int hres_active : 1, in_hrtirq : 1, hang_detected : 1, - softirq_activated : 1; + softirq_activated : 1, + online : 1; #ifdef CONFIG_HIGH_RES_TIMERS unsigned int nr_events; unsigned short nr_retries; diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 760793998cdd..edb0f821dcea 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -1085,6 +1085,7 @@ static int enqueue_hrtimer(struct hrtimer *timer, enum hrtimer_mode mode) { debug_activate(timer, mode); + WARN_ON_ONCE(!base->cpu_base->online); base->cpu_base->active_bases |= 1 << base->index; @@ -2183,6 +2184,7 @@ int hrtimers_prepare_cpu(unsigned int cpu) cpu_base->softirq_next_timer = NULL; cpu_base->expires_next = KTIME_MAX; cpu_base->softirq_expires_next = KTIME_MAX; + cpu_base->online = 1; hrtimer_cpu_base_init_expiry_lock(cpu_base); return 0; } @@ -2250,6 +2252,7 @@ int hrtimers_cpu_dying(unsigned int dying_cpu) smp_call_function_single(ncpu, retrigger_next_event, NULL, 0); raw_spin_unlock(&new_base->lock); + old_base->online = 0; raw_spin_unlock(&old_base->lock); return 0; From patchwork Mon Dec 18 23:19:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 180680 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp1594343dyi; Mon, 18 Dec 2023 15:20:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IFz52DRXSAewbLfrfsgVT5fnRot5Hk60mncE+vhB5mZFidFMFMEU1y1FrvGN5Ibr/xAJogd X-Received: by 2002:a05:6a00:ac7:b0:6d0:aece:cdfc with SMTP id c7-20020a056a000ac700b006d0aececdfcmr5467439pfl.41.1702941607173; Mon, 18 Dec 2023 15:20:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702941607; cv=none; d=google.com; s=arc-20160816; b=uIjP+IFZ7BcTZa/I93D86IC/vYu+2koPiueq47Kit3mUtfZCI7xZ/1YCSwzBCg93T+ IEYE6AmtHx+U2vAJ7p7bcPmRiYSFZh8L+lABpDlZ7pAwuTscvLPiFaMBr2YvjmLmCkRF X7L3NWGggQY01TDtQcsOypy5S/CsX84GHX9tfET3LUzmtOYL4vxiTWIx1oachGZhgxx5 XMpZ5culPBvND9gg5YblyT8QybjWqPc44UIHX+/elwI4zRQc2lxyTnSpFsCtBoFkOQ+M qyPZtirB96QiBSPaNJmuHLBgBFv5qcyUAncRiAPlX1ocORIvjYOyZDhVFbx1mSrKyPv6 qqcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=siIBhypBPJBsU+JxaET//sOJpdfgrnmQnavzlIOImLM=; fh=xvjKaM6hmtHqnOhs670mkmt5SmfrGcW6BwluoXDWXqs=; b=QiTBih7bxvvoUa6kLjlFioIJ4lhE9eqs21/VNxAHmY26J5amztBOCMefprc8iFUEu8 bgm7qoE9AyrkYvC1cgiZQyzKjzkoRhXmeGKElBBLEArf3KTbaN3U9Yj8fcXAk4vYvNJ/ GXtgYpSK3wq2FGvTIV28U+8uSxhQBO5yuvcz1f8rKYbQ31vQXs9v3SJlf9PvQ98dhIK4 MMRgKQ5CGXZ09US1eo8J3TUQ1PXJ5qcNEW6CPc0tmQAmDSTI3zXM2BOinqcBjQ1S8a45 uRIJ8Tz+4D1ErYdUWwKYeuE27XshUeTk0fcUrD2fQ5tB4SFEpxlwDUmB2hLsARPgrpbq xsog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=L8J2LrPG; spf=pass (google.com: domain of linux-kernel+bounces-4477-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-4477-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id w11-20020a056a0014cb00b006d71fc698f8si2622632pfu.30.2023.12.18.15.20.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Dec 2023 15:20:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-4477-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=L8J2LrPG; spf=pass (google.com: domain of linux-kernel+bounces-4477-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-4477-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id BE6A3285424 for ; Mon, 18 Dec 2023 23:20:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B5CEE768ED; Mon, 18 Dec 2023 23:19:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L8J2LrPG" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED61A79944; Mon, 18 Dec 2023 23:19:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 02F3FC433D9; Mon, 18 Dec 2023 23:19:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702941569; bh=FjnYKWmKazFNSVW61ol8hhKh2qDPsvgWNQgyqkISaA8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L8J2LrPGtoGQcZ1ZvOzNwBrc2kpjH28ttt/sL3uw0zBI2iBEVxxQfynVn3lDyDk9N 1t5z5Yw4arepFW3hGmvgFTg3/DZfofyYJ4reEfO/qx9llQctX6HMLKNxOUo+Tf5hyz l7iywMPD6S3TZxGCjZhnRyo/mJbIJCJYS3h2hjbGjTQVS8J0QR0VCKYBx0KqaP3w/G EQiKFdixyevxwCsm85+DjiPdIua6uCPZVT4ucU/Nlw7gw3KdMVbLVIDiCIPBpjaGfw CHlBnagO7M2QWUAPXbzTYEf/wXU17zp1GzVwWal4b6fLKa3CTBU8Aqf9JWqCRdtHT+ uL36thz7eMxvg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Neeraj Upadhyay , Uladzislau Rezki , Zqiang , rcu , "Paul E . McKenney" , Thomas Gleixner , Peter Zijlstra Subject: [PATCH 2/3] rcu: Defer RCU kthreads wakeup when CPU is dying Date: Tue, 19 Dec 2023 00:19:15 +0100 Message-ID: <20231218231916.11719-3-frederic@kernel.org> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231218231916.11719-1-frederic@kernel.org> References: <20231218231916.11719-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785663698605253530 X-GMAIL-MSGID: 1785663698605253530 When the CPU goes idle for the last time during the CPU down hotplug process, RCU reports a final quiescent state for the current CPU. If this quiescent state propagates up to the top, some tasks may then be woken up to complete the grace period: the main grace period kthread and/or the expedited main workqueue (or kworker). If those kthreads have a SCHED_FIFO policy, the wake up can indirectly arm the RT bandwith timer to the local offline CPU. Since this happens after hrtimers have been migrated at CPUHP_AP_HRTIMERS_DYING stage, the timer gets ignored. Therefore if the RCU kthreads are waiting for RT bandwidth to be available, they may never be actually scheduled. This triggers TREE03 rcutorture hangs: rcu: INFO: rcu_preempt self-detected stall on CPU rcu: 4-...!: (1 GPs behind) idle=9874/1/0x4000000000000000 softirq=0/0 fqs=20 rcuc=21071 jiffies(starved) rcu: (t=21035 jiffies g=938281 q=40787 ncpus=6) rcu: rcu_preempt kthread starved for 20964 jiffies! g938281 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:R running task stack:14896 pid:14 tgid:14 ppid:2 flags:0x00004000 Call Trace: __schedule+0x2eb/0xa80 schedule+0x1f/0x90 schedule_timeout+0x163/0x270 ? __pfx_process_timeout+0x10/0x10 rcu_gp_fqs_loop+0x37c/0x5b0 ? __pfx_rcu_gp_kthread+0x10/0x10 rcu_gp_kthread+0x17c/0x200 kthread+0xde/0x110 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2b/0x40 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1b/0x30 The situation can't be solved with just unpinning the timer. The hrtimer infrastructure and the nohz heuristics involved in finding the best remote target for an unpinned timer would then also need to handle enqueues from an offline CPU in the most horrendous way. So fix this on the RCU side instead and defer the wake up to an online CPU if it's too late for the local one. Reported-by: Paul E. McKenney Fixes: 5c0930ccaad5 ("hrtimers: Push pending hrtimers away from outgoing CPU earlier") Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree.c | 34 +++++++++++++++++++++++++++++++++- kernel/rcu/tree_exp.h | 3 +-- 2 files changed, 34 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 3ac3c846105f..157f3ca2a9b5 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1013,6 +1013,38 @@ static bool rcu_future_gp_cleanup(struct rcu_node *rnp) return needmore; } +static void swake_up_one_online_ipi(void *arg) +{ + struct swait_queue_head *wqh = arg; + + swake_up_one(wqh); +} + +static void swake_up_one_online(struct swait_queue_head *wqh) +{ + int cpu = get_cpu(); + + /* + * If called from rcutree_report_cpu_starting(), wake up + * is dangerous that late in the CPU-down hotplug process. The + * scheduler might queue an ignored hrtimer. Defer the wake up + * to an online CPU instead. + */ + if (unlikely(cpu_is_offline(cpu))) { + int target; + + target = cpumask_any_and(housekeeping_cpumask(HK_TYPE_RCU), + cpu_online_mask); + + smp_call_function_single(target, swake_up_one_online_ipi, + wqh, 0); + put_cpu(); + } else { + put_cpu(); + swake_up_one(wqh); + } +} + /* * Awaken the grace-period kthread. Don't do a self-awaken (unless in an * interrupt or softirq handler, in which case we just might immediately @@ -1037,7 +1069,7 @@ static void rcu_gp_kthread_wake(void) return; WRITE_ONCE(rcu_state.gp_wake_time, jiffies); WRITE_ONCE(rcu_state.gp_wake_seq, READ_ONCE(rcu_state.gp_seq)); - swake_up_one(&rcu_state.gp_wq); + swake_up_one_online(&rcu_state.gp_wq); } /* diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h index 6d7cea5d591f..2ac440bc7e10 100644 --- a/kernel/rcu/tree_exp.h +++ b/kernel/rcu/tree_exp.h @@ -173,7 +173,6 @@ static bool sync_rcu_exp_done_unlocked(struct rcu_node *rnp) return ret; } - /* * Report the exit from RCU read-side critical section for the last task * that queued itself during or before the current expedited preemptible-RCU @@ -201,7 +200,7 @@ static void __rcu_report_exp_rnp(struct rcu_node *rnp, raw_spin_unlock_irqrestore_rcu_node(rnp, flags); if (wake) { smp_mb(); /* EGP done before wake_up(). */ - swake_up_one(&rcu_state.expedited_wq); + swake_up_one_online(&rcu_state.expedited_wq); } break; } From patchwork Mon Dec 18 23:19:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 180681 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp1594429dyi; Mon, 18 Dec 2023 15:20:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IHte9E2DIqIQDV+2j+xGDYUL5Rjs6AZ5K2hMb0m8QV35Vre4VsG7qyQm8ddIM1eBv56A4I8 X-Received: by 2002:a17:90b:350:b0:28b:4750:c85a with SMTP id fh16-20020a17090b035000b0028b4750c85amr194830pjb.43.1702941620134; Mon, 18 Dec 2023 15:20:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702941620; cv=none; d=google.com; s=arc-20160816; b=ZPlaHwWU+rIKaoTruzJZpFCtNJgaQgJpHA4/taPn5bBaGJAImKtdkwC1jkuv7rezRu QlvHDCUkMSGCcrJgZoG0nwZKorrPNgJmVavW+r5u2XvKELJLCvs5RnjAyASwhOVWVNOz iBaRCqAlB42cmbCtFbaueRsjapV5gjfdb/vmran9cCDFltixfrQLYfjBH23qFbHcUMEd hRMdP2c7VAs6lCCEEfocS7yZD/c+ADLASQhJ5Nm6bIjYX6eFGFeeLU2o5WkxBfPlXFkW CFbBc+ah3o765jp7vA9R1H0kDSKE2DWV6vM9hPQHbNQkl9kN3gf3yGBS2P4va86rWYUf X4Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=4gRFC094GsoLU4fYXEtJbe2p4KyIYghrmDbh0DBntrE=; fh=xvjKaM6hmtHqnOhs670mkmt5SmfrGcW6BwluoXDWXqs=; b=YuAlycaxw5xVzrkHF4bPcEEF0aV3PmgseOYPHqc9BBKChJCxwOU99PeArVpPp+I/jC 15fGLgDAKfd3WH1p++YCkdVgcLBxL4zOfEOSTYGu30Sv2rDqwZ30upNLgi6ud+NAEneu h7+OUHEx6Fg5WgyCyZwaDer1VgZiJ2t7NH/gn3qfgS8C0xOqYMtAoqGrk/tKR0ce4UPw 3EfkHOP9/aVRpUwIHTuL+k16fMsUg6d0fXggBdU2GhpYxyuD4PlLrLW0xDmrGhjQafMn dSLEkbu80l1fT8WKKYjQAc5i1Im9GK86pJZfM+aGeRpvK74KbL8TtIsTSk+Txa2MYb86 sa2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TuJESOJ2; spf=pass (google.com: domain of linux-kernel+bounces-4478-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-4478-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id j15-20020a170903024f00b001d1db5e39absi18785274plh.320.2023.12.18.15.20.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Dec 2023 15:20:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-4478-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TuJESOJ2; spf=pass (google.com: domain of linux-kernel+bounces-4478-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-4478-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id D759928390E for ; Mon, 18 Dec 2023 23:20:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C6F4879948; Mon, 18 Dec 2023 23:19:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TuJESOJ2" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 149B1768FC; Mon, 18 Dec 2023 23:19:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8BCEC433C8; Mon, 18 Dec 2023 23:19:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702941572; bh=x/5xI2oI/qJM/7tPaUtUa288bDEyvUKnDytjUeL9Kh8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TuJESOJ2ve3eFm0UYRg2rZDQWAy4X4nBL5GMaqztfenpsittW/7VpTa2Fhrd4QSGm +4WtgAd5/4b0ttyCzM/F0tWSKiBtHuIgRSmPJC+MNF4qQK7AzbDFLquasL5HIhBvko 2od7mFYsC3y7Z9lM/13oy3C5Ds0lgasesvO74eNVZbI0UDBt/hrYn+2OcZE1v8lBsQ F1o10ntJRcveoVk9Uqd/OvHDDEpWoLi5bGtodA9+4g8SXSWR7E84jbSCX3xnEqp+ta tuxCuQSKug4vCynxWCzXdsfwjFQS+o/dd0A4bRCEumq3jmNKrGJFoV6WBRWwEugbtK 0X6UvUFT1qdUw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Boqun Feng , Joel Fernandes , Neeraj Upadhyay , Uladzislau Rezki , Zqiang , rcu , "Paul E . McKenney" , Thomas Gleixner , Peter Zijlstra Subject: [PATCH 3/3] rcu/exp: Remove full barrier upon main thread wakeup Date: Tue, 19 Dec 2023 00:19:16 +0100 Message-ID: <20231218231916.11719-4-frederic@kernel.org> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231218231916.11719-1-frederic@kernel.org> References: <20231218231916.11719-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785663712256198253 X-GMAIL-MSGID: 1785663712256198253 When an expedited grace period is ending, care must be taken so that all the quiescent states propagated up to the root are correctly ordered against the wake up of the main expedited grace period workqueue. This ordering is already carried through the root rnp locking augmented by an smp_mb__after_unlock_lock() barrier. Therefore the explicit smp_mb() placed before the wake up is not needed and can be removed. Signed-off-by: Frederic Weisbecker --- kernel/rcu/tree_exp.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h index 2ac440bc7e10..014ddf672165 100644 --- a/kernel/rcu/tree_exp.h +++ b/kernel/rcu/tree_exp.h @@ -198,10 +198,9 @@ static void __rcu_report_exp_rnp(struct rcu_node *rnp, } if (rnp->parent == NULL) { raw_spin_unlock_irqrestore_rcu_node(rnp, flags); - if (wake) { - smp_mb(); /* EGP done before wake_up(). */ + if (wake) swake_up_one_online(&rcu_state.expedited_wq); - } + break; } mask = rnp->grpmask;