From patchwork Fri May 12 21:07:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 93395 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5396541vqo; Fri, 12 May 2023 14:19:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6qT0aXkZlZD6ihwj6zHgzH8n47OPvPQ8WxpS8bjn1aUPeT/WKOhb4LtTvX1JJI745T42rn X-Received: by 2002:a17:90a:df07:b0:24d:ecf7:cabb with SMTP id gp7-20020a17090adf0700b0024decf7cabbmr25606993pjb.22.1683926373619; Fri, 12 May 2023 14:19:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683926373; cv=none; d=google.com; s=arc-20160816; b=EmtYbZfDXKVP/b5TwzBxHold3t+MOd6M94+NypEgoAPaWpGUuM6IAdlpUcubpC24xb wQfcXkxTcqYoXnl7SXqk0TCRYp9NkcNSl4F0lFGgS4319Bqw2JOjem+/EiSF51ZdGc0r ypSnmb2CjrkyTPWpJVyhthCSyv/FPT8j9hOpGDdH9tOChQ22mBnSGA1HQsDsvhcFaRfP Uw1NfRGHpmFu+sZQg1mYP5GT3W0UTg9Gr7vKEiYbozYirkTvZ31vYxFrXwUcKQBvVJA3 rqJL1ofZfg53ZiqZp3aaJUiWmLUZK1ivQKHIxMzME8b3X6SAeISRwkRgJoN0JRjzjZxY /ksw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=SxTJFZiyF2WGH/LZ05mC0rxxeGaROu74J7Cy/GPHn70=; b=u5Xs7UF/HsYxLccgvlpLyTRESCnIviCmEOOzUdmfKqhozZErEHCXvzpQ+qdXatEPXm 3CPAufupECfkoVR8aptdWO61QHAjKXIh95n2Doo5sc4Sjxemlsr7xmx6kA/BIq3rjdzF XP01Pn8KTUXRJt2NyLF55WxwB3dkbSO4Yni1Odvw6Zao4lYYQs0jCaOslMgcB2mS4K6i zNRReWzH5gsS5YfxCasNrR4fPAwFzN0grtUBEyGFSIheC+7BzRWa2zsGkIjJQtfCq5tE ROwHrd3V5QipeRdxG17KwUX5dH4yPS6Lh/I/Ebc6TBU7GYahu8RtUxMadV5gqz9qQwyt qRFA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="K/FezNvB"; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f23-20020a17090ac29700b00246f8b06125si19612485pjt.108.2023.05.12.14.19.20; Fri, 12 May 2023 14:19:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="K/FezNvB"; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240535AbjELVKx (ORCPT + 99 others); Fri, 12 May 2023 17:10:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240521AbjELVKV (ORCPT ); Fri, 12 May 2023 17:10:21 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33467A5C2; Fri, 12 May 2023 14:08:51 -0700 (PDT) Message-ID: <20230512205257.080801387@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1683925665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=SxTJFZiyF2WGH/LZ05mC0rxxeGaROu74J7Cy/GPHn70=; b=K/FezNvBOJNnPeMjn0jwZGZLrdjicSHW686SvUJHgCP7i5tNRPZ0+TcBzpR3zsHIAjx7el oZD3T4LSHKe96Gl8Vn9tHLhiugtu0Peaz0vQATG5sk1xtglaKPxkN9q6sYgkMinNckV8/o 10M+IlKGzHanW2pN6YNi9PrY5sZrLsfysPv/Rf/Y1Br/eUV1uOi0RLQrCO075ABqwfNzUn Qw3IiZGS0j3l+s+ARRAaaJktiUQ2fdrXWEZmaXcucSO1rkDizolMPZOl5m8lUPUZsNHIlu X7qCk8reaCGdfX2XcY6t80L+Yvj3odTymg6hokHUnI0YYCr7zk0zNLVIAAqpWw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1683925665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=SxTJFZiyF2WGH/LZ05mC0rxxeGaROu74J7Cy/GPHn70=; b=ZlpauXamVislT6/DN2ZOHNylyjhLP1HBQpResarL3DaTxfAUdyt/zGjPUsAQBjx7GbeYCQ FfQ51EH8ZACSPgDw== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, David Woodhouse , Andrew Cooper , Brian Gerst , Arjan van de Veen , Paolo Bonzini , Paul McKenney , Tom Lendacky , Sean Christopherson , Oleksandr Natalenko , Paul Menzel , "Guilherme G. Piccoli" , Piotr Gorski , Usama Arif , Juergen Gross , Boris Ostrovsky , xen-devel@lists.xenproject.org, Russell King , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Will Deacon , Guo Ren , linux-csky@vger.kernel.org, Thomas Bogendoerfer , linux-mips@vger.kernel.org, "James E.J. Bottomley" , Helge Deller , linux-parisc@vger.kernel.org, Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, Mark Rutland , Sabin Rapan , "Michael Kelley (LINUX)" , Ross Philipson Subject: [patch V4 30/37] cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism References: <20230512203426.452963764@linutronix.de> MIME-Version: 1.0 Date: Fri, 12 May 2023 23:07:45 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765724781120036198?= X-GMAIL-MSGID: =?utf-8?q?1765724781120036198?= From: Thomas Gleixner The bring up logic of a to be onlined CPU consists of several parts, which are considered to be a single hotplug state: 1) Control CPU issues the wake-up 2) To be onlined CPU starts up, does the minimal initialization, reports to be alive and waits for release into the complete bring-up. 3) Control CPU waits for the alive report and releases the upcoming CPU for the complete bring-up. Allow to split this into two states: 1) Control CPU issues the wake-up After that the to be onlined CPU starts up, does the minimal initialization, reports to be alive and waits for release into the full bring-up. As this can run after the control CPU dropped the hotplug locks the code which is executed on the AP before it reports alive has to be carefully audited to not violate any of the hotplug constraints, especially not modifying any of the various cpumasks. This is really only meant to avoid waiting for the AP to react on the wake-up. Of course an architecture can move strict CPU related setup functionality, e.g. microcode loading, with care before the synchronization point to save further pointless waiting time. 2) Control CPU waits for the alive report and releases the upcoming CPU for the complete bring-up. This allows that the two states can be split up to run all to be onlined CPUs up to state #1 on the control CPU and then at a later point run state #2. This spares some of the latencies of the full serialized per CPU bringup by avoiding the per CPU wakeup/wait serialization. The assumption is that the first AP already waits when the last AP has been woken up. This obvioulsy depends on the hardware latencies and depending on the timings this might still not completely eliminate all wait scenarios. This split is just a preparatory step for enabling the parallel bringup later. The boot time bringup is still fully serialized. It has a separate config switch so that architectures which want to support parallel bringup can test the split of the CPUHP_BRINGUG step separately. To enable this the architecture must support the CPU hotplug core sync mechanism and has to be audited that there are no implicit hotplug state dependencies which require a fully serialized bringup. Signed-off-by: Thomas Gleixner Tested-by: Michael Kelley --- arch/Kconfig | 4 ++ include/linux/cpuhotplug.h | 4 ++ kernel/cpu.c | 70 +++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 76 insertions(+), 2 deletions(-) --- a/arch/Kconfig +++ b/arch/Kconfig @@ -49,6 +49,10 @@ config HOTPLUG_CORE_SYNC_FULL select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU select HOTPLUG_CORE_SYNC +config HOTPLUG_SPLIT_STARTUP + bool + select HOTPLUG_CORE_SYNC_FULL + config GENERIC_ENTRY bool --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -133,6 +133,7 @@ enum cpuhp_state { CPUHP_MIPS_SOC_PREPARE, CPUHP_BP_PREPARE_DYN, CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20, + CPUHP_BP_KICK_AP, CPUHP_BRINGUP_CPU, /* @@ -517,9 +518,12 @@ void cpuhp_online_idle(enum cpuhp_state static inline void cpuhp_online_idle(enum cpuhp_state state) { } #endif +struct task_struct; + void cpuhp_ap_sync_alive(void); void arch_cpuhp_sync_state_poll(void); void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu); +int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle); #ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD void cpuhp_ap_report_dead(void); --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -761,6 +761,47 @@ static int bringup_wait_for_ap_online(un return 0; } +#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP +static int cpuhp_kick_ap_alive(unsigned int cpu) +{ + if (!cpuhp_can_boot_ap(cpu)) + return -EAGAIN; + + return arch_cpuhp_kick_ap_alive(cpu, idle_thread_get(cpu)); +} + +static int cpuhp_bringup_ap(unsigned int cpu) +{ + struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu); + int ret; + + /* + * Some architectures have to walk the irq descriptors to + * setup the vector space for the cpu which comes online. + * Prevent irq alloc/free across the bringup. + */ + irq_lock_sparse(); + + ret = cpuhp_bp_sync_alive(cpu); + if (ret) + goto out_unlock; + + ret = bringup_wait_for_ap_online(cpu); + if (ret) + goto out_unlock; + + irq_unlock_sparse(); + + if (st->target <= CPUHP_AP_ONLINE_IDLE) + return 0; + + return cpuhp_kick_ap(cpu, st, st->target); + +out_unlock: + irq_unlock_sparse(); + return ret; +} +#else static int bringup_cpu(unsigned int cpu) { struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu); @@ -781,7 +822,6 @@ static int bringup_cpu(unsigned int cpu) */ irq_lock_sparse(); - /* Arch-specific enabling code. */ ret = __cpu_up(cpu, idle); if (ret) goto out_unlock; @@ -805,6 +845,7 @@ static int bringup_cpu(unsigned int cpu) irq_unlock_sparse(); return ret; } +#endif static int finish_cpu(unsigned int cpu) { @@ -1944,13 +1985,38 @@ static struct cpuhp_step cpuhp_hp_states .startup.single = timers_prepare_cpu, .teardown.single = timers_dead_cpu, }, - /* Kicks the plugged cpu into life */ + +#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP + /* + * Kicks the AP alive. AP will wait in cpuhp_ap_sync_alive() until + * the next step will release it. + */ + [CPUHP_BP_KICK_AP] = { + .name = "cpu:kick_ap", + .startup.single = cpuhp_kick_ap_alive, + }, + + /* + * Waits for the AP to reach cpuhp_ap_sync_alive() and then + * releases it for the complete bringup. + */ + [CPUHP_BRINGUP_CPU] = { + .name = "cpu:bringup", + .startup.single = cpuhp_bringup_ap, + .teardown.single = finish_cpu, + .cant_stop = true, + }, +#else + /* + * All-in-one CPU bringup state which includes the kick alive. + */ [CPUHP_BRINGUP_CPU] = { .name = "cpu:bringup", .startup.single = bringup_cpu, .teardown.single = finish_cpu, .cant_stop = true, }, +#endif /* Final state before CPU kills itself */ [CPUHP_AP_IDLE_DEAD] = { .name = "idle:dead",