From patchwork Wed Jan 31 23:11:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195050 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp89634dyc; Wed, 31 Jan 2024 15:29:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IE+qNwZbTPWieb7afrmKP8PxknfX4iSkBfUQBwLhj2tP7qY2fyYq8xql/eIYZmd2DbjwIKe X-Received: by 2002:a17:906:190a:b0:a36:7608:8f91 with SMTP id a10-20020a170906190a00b00a3676088f91mr2407035eje.6.1706743784461; Wed, 31 Jan 2024 15:29:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743784; cv=pass; d=google.com; s=arc-20160816; b=ifaotssDwhYaZSqt4+0A7tiLOpo4nZ/r2VjVpcAUpP7oS4sBn3jDj3bUzSQOKWNS36 Klaf4KEN4O+EBxkug7wfRboTKbCiDNfoDig/BTfS6yruUOvPfUs402NU5xgL/GABDvJg 9udfpUp6aOcff8/Uj0b+Rj3/EDRgggYrub7mdZYH/mOMxfXRyyvzoOKm8MFYt+C9QZqE qv+RGjuRZr4yq9lsJAZ6KdpPeR4EotIAfpY9POdP2LuaaxPZQnmXUkkeEH6zmvVrlHWC p3uf9xJp2EZpWGaGmmGehPrJbPL5Sq1U9PEB85YPmLRvsw2sg97Ch2yMI7oietJ5cBE1 3WsA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=rikaqKwz5QTTyQ6tTp4fmEVke9sYVhWGQulgxnOCRww=; fh=iQ/AwNXhKvHr/Zkyhc18x2XfjzSP48t+bD8cBfuYRuA=; b=IgdJ5icsZExDOuBDC5aUGAtisxZmkEUT0OeKThlUFxZpCQv9Oh4S7cj/L5EkWLMKQR uHEISfLz9Y2b2EkvYnkXOVV7TCCRWCWGoy0MPmZ9Q+ws93DnkIasNKoGmqj/0oN5IHcL ry19aj/KYUzkhVxbuTkxXK17AC4og7qVBplbIWwpOgUF9mpfs7Mz9YQHeWWjxrYru8nC Cq1aBkNfmM8bHf2VSa54fUlwpkGOJbTyQObvcS3fImmf/Ley3ZTQ48n0jRHV0XpRp5s3 jhcMo+MS7l2O65s1hAlwkdLD9KGtgFSubpIRlzkCWZXr0eYt3O04hj8sbGUGNAv17tdL Gi1Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=uUjLqIMD; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUrq9SbSLrMBZLRj8cGUTx72EV9Xb13tFLthv9VJF7WvjUQnUKjiLpxcOZf/uUpVyvegd91WbajPkUI8Ng8lGoa/M556A== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id y9-20020a170906558900b00a348dd696a1si6133230ejp.971.2024.01.31.15.29.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:29:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=uUjLqIMD; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E17371F224D5 for ; Wed, 31 Jan 2024 23:29:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 54BDE54F87; Wed, 31 Jan 2024 23:11:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uUjLqIMD" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E9394CE17 for ; Wed, 31 Jan 2024 23:11:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742702; cv=none; b=qk10eZy8rYMhxYoYKtt2rHREgWbxSBs3T+GJfFT/fLZ4QWol2Ld6WusAZXl2dplffBtKGJRIymwjqp/eQZXfR1yGfRdGR71IpgsiAYiucuaPROr5aBb9AV9ND1dyKbSJJvMqRVAzuJ//NoG25HlcahtMsHGSjeUHnXuKSTLIePs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742702; c=relaxed/simple; bh=VeKB2CcnfxaMa74jOXPT020xZoApnn0QOwGRqmxYQ/I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HJMCbkL9FcCTBmvLA58S2LSatitbMh1dOiB5iZADHPRtlPritsv3du3BU+/6uy9hecvpPsoooZ413ZJ0k6uCN0Bo0OO8MVUU+6p0jSoMTZRAW68gzuqeh7AWk/0hqAnzrtAVoa+MUGa3sXvjMjd+/sKECS6tBwUaXpkZ1akuE0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uUjLqIMD; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F997C433C7; Wed, 31 Jan 2024 23:11:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742701; bh=VeKB2CcnfxaMa74jOXPT020xZoApnn0QOwGRqmxYQ/I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uUjLqIMDv27lgka4aqV7u90w6slLnq1qvEGzmwp/78x00vv5pGUv+wkUCg/ovObJL jNnRDFadbB1AxT9Cq3QQwX8DbQdu9PPQZckckHOrBwW5kyjnsuf9WYFSRRDGlGxnQJ aU+QgjNU0Cn6ocBCfXCWPDwZOjHa3oilSHSfc27Vdh1iCSb58AgtowMefFEzHIGBJD sdsapMO/D0CXETIVtg6dYzdUJcbRJV7l2qgbrM8OqRzrSopPyudezBxlSBPT83jSEm CXCZfjEi/NxSm4lKi1aSUHAHbzTnYUuNkL9JtyEsMAvSxMz3DWwh5bUEQYUBM7/1Ek sP/6P31JpxDaA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 07/15] tick: Start centralizing tick related CPU hotplug operations Date: Thu, 1 Feb 2024 00:11:12 +0100 Message-ID: <20240131231120.12006-8-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650570197718187 X-GMAIL-MSGID: 1789650570197718187 During the CPU offlining process, the various timer tick features are shut down from scattered places, sometimes from teardown callbacks on stop machine, sometimes through explicit calls, sometimes from the control CPU after the CPU died. The reason why these shutdown operations are spread around is not always clear and it makes the tick lifecycle hard to follow. The tick should be shut down in order from highest to lowest level: On stop machine from the dying CPU (high-level): 1) Hand-over the timekeeping duty (tick_handover_do_timer()) 2) Cancel the tick implementation called by the clockevent callback (tick_cancel_sched_timer()) 3) Shutdown broadcasting (tick_offline_cpu() / tick_broadcast_offline()) On stop machine from the dying CPU (low-level): 4) Shutdown clockevents drivers (CPUHP_AP_*_TIMER_STARTING states) From the control CPU after the CPU died (low-level): 5) Shutdown/unregister/cleanup clockevents for the dead CPU (tick_cleanup_dead_cpu()) Instead the current order is 2, 4 (both from CPU hotplug states), then 1 and 3 through direct calls. This layout and order don't make much sense. The operations 1, 2, 3 should be gathered together and in order. Sort this situation with creating a new TICK shut-down CPU hotplug state and start with introducing the timekeeping duty hand-over there. The state must precede hrtimers migration because the tick hrtimer will be stopped from it in a further patch. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- include/linux/cpuhotplug.h | 1 + include/linux/tick.h | 8 ++++++-- kernel/cpu.c | 8 +++++--- kernel/time/tick-common.c | 17 +++++++++++------ 4 files changed, 23 insertions(+), 11 deletions(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 172d0a743e5d..74fcdd2b82c8 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -184,6 +184,7 @@ enum cpuhp_state { CPUHP_AP_ARM64_ISNDEP_STARTING, CPUHP_AP_SMPCFD_DYING, CPUHP_AP_HRTIMERS_DYING, + CPUHP_AP_TICK_DYING, CPUHP_AP_X86_TBOOT_DYING, CPUHP_AP_ARM_CACHE_B15_RAC_DYING, CPUHP_AP_ONLINE, diff --git a/include/linux/tick.h b/include/linux/tick.h index 716d17f31c45..afff4c207bd8 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -19,16 +19,20 @@ extern void __init tick_init(void); extern void tick_suspend_local(void); /* Should be core only, but XEN resume magic and ARM BL switcher require it */ extern void tick_resume_local(void); -extern void tick_handover_do_timer(void); extern void tick_cleanup_dead_cpu(int cpu); #else /* CONFIG_GENERIC_CLOCKEVENTS */ static inline void tick_init(void) { } static inline void tick_suspend_local(void) { } static inline void tick_resume_local(void) { } -static inline void tick_handover_do_timer(void) { } static inline void tick_cleanup_dead_cpu(int cpu) { } #endif /* !CONFIG_GENERIC_CLOCKEVENTS */ +#if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_HOTPLUG_CPU) +extern int tick_cpu_dying(unsigned int cpu); +#else +#define tick_cpu_dying NULL +#endif + #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_SUSPEND) extern void tick_freeze(void); extern void tick_unfreeze(void); diff --git a/kernel/cpu.c b/kernel/cpu.c index e6ec3ba4950b..263508073da8 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1324,8 +1324,6 @@ static int take_cpu_down(void *_param) */ cpuhp_invoke_callback_range_nofail(false, cpu, st, target); - /* Give up timekeeping duties */ - tick_handover_do_timer(); /* Remove CPU from timer broadcasting */ tick_offline_cpu(cpu); /* Park the stopper thread */ @@ -2205,7 +2203,11 @@ static struct cpuhp_step cpuhp_hp_states[] = { .startup.single = NULL, .teardown.single = hrtimers_cpu_dying, }, - + [CPUHP_AP_TICK_DYING] = { + .name = "tick:dying", + .startup.single = NULL, + .teardown.single = tick_cpu_dying, + }, /* Entry state on starting. Interrupts enabled from here on. Transient * state for synchronsization */ [CPUHP_AP_ONLINE] = { diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 0084e1ae2583..a89ef450fda7 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -397,15 +397,20 @@ EXPORT_SYMBOL_GPL(tick_broadcast_oneshot_control); #ifdef CONFIG_HOTPLUG_CPU /* - * Transfer the do_timer job away from a dying cpu. - * - * Called with interrupts disabled. No locking required. If - * tick_do_timer_cpu is owned by this cpu, nothing can change it. + * Stop the tick and transfer the timekeeping job away from a dying cpu. */ -void tick_handover_do_timer(void) +int tick_cpu_dying(unsigned int dying_cpu) { - if (tick_do_timer_cpu == smp_processor_id()) + /* + * If the current CPU is the timekeeper, it's the only one that + * can safely hand over its duty. Also all online CPUs are in + * stop machine, guaranteed not to be idle, therefore it's safe + * to pick any online successor. + */ + if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); + + return 0; } /*