From patchwork Fri May 12 21:07:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 93385 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5395994vqo; Fri, 12 May 2023 14:18:17 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ63QaXCOd+bxL3Z30psOZlB5WNDfPaRrAvQR1XWHn58PYRyqeSlgC2Uqrj7+zEz89nE7dsr X-Received: by 2002:a05:6a20:9183:b0:fc:8dfb:318b with SMTP id v3-20020a056a20918300b000fc8dfb318bmr34210369pzd.0.1683926297234; Fri, 12 May 2023 14:18:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683926297; cv=none; d=google.com; s=arc-20160816; b=noBFTtSl8DRCCqbNhux9/LQmtdHEoLE5LqUvEIrtcEcZSiHogVOo+J8CJ6o/zvc8iv twNhJequySntdeAzFB+Tai/368duEfPHD/8F+itjlILEg3gHXAguYOQAHlBtLSvJB/AH nbLeHleOSa4ldDIefPILGyi2POKKp8gEfdz8KuqMFZNHNhPp478QMSrGmQ0teg1Wo9ID A2byewLMHGKzlpgvn0AdV5dCRRP2cs41wBg4Edk2Dv7jEtJkhxzQRTAUeAtYcL4HZQdr lb5oNH65rd8dtzNRbTF0R4/f9D2xNHKjDhH61YKfmqO6D0odzoAxZwm9KQRWL69MEhMC UdjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=Vdgu0US5isMOkXvsbtEKbpQbMn5jiJ+2zBLMqmM0YL8=; b=A4EZQbY/zOMx7UMig1DfzD9PGLU3du5TDSfZA2M/wD0AVSSPwpSwzfNbd8EGTDphKr IxGj1u/YwnkbxYV/yOFggHHh5ldxTD0i2Z18MZO+6v319bI7i7uOmJnv8kwOzuUoesce fsYP8Qai/tLRrcm5vtHv7Y6vW/O5bLlMnfakjs6EKshQ0mHlGPuhTC4Hohz9DrRepbNb birJTdXC6ywFb4Fa1KJXb0QYar71gjZY0AD6BR75NBpfasdawsWszReFo1wAUeuZgrig S+fqKDyJ1DPBoBUH+6gp2jJO58N2CmWUWUnnjGjl0eNRoL74YDFWpfWPtjR3ls4/l8AD H/6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=rHjvfgvb; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=Nt8LRINm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j184-20020a6380c1000000b0050f736a8d3dsi11190819pgd.64.2023.05.12.14.18.05; Fri, 12 May 2023 14:18:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=rHjvfgvb; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=Nt8LRINm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239763AbjELVHd (ORCPT + 99 others); Fri, 12 May 2023 17:07:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239649AbjELVHP (ORCPT ); Fri, 12 May 2023 17:07:15 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CCE2CE; Fri, 12 May 2023 14:07:08 -0700 (PDT) Message-ID: <20230512205255.768845190@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1683925627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Vdgu0US5isMOkXvsbtEKbpQbMn5jiJ+2zBLMqmM0YL8=; b=rHjvfgvb2+9T3PZ89upCVf/htUJYRgaiE4wYsm0JoNYDceADItU6dQkHNUxMVT0mAKxQCQ g1/OIsowQWZ6Bng8y3rPxKD6j9JG6nO9AqcKXUIfNqadj2wqAP10ZTZqf/wQ8bZuweIxU8 FbGJYBtbB0RLpj8XodKxfM0T9yYAZ8rC5lIdf6QnDplFrmHRxCno1cLmIswNLEF/jQuqCF ufPsxRi6VSkjYg9pqBKmigegl+aCRSB9QunUmA1+oo3u2YiKlvmXRZ/YpzUK4x67YwhumP TeVJDM0kZMKjpua2HnPGv2PoAp8D/THStbkBRKgjlP6r9216eUdapX+gssKumg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1683925627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Vdgu0US5isMOkXvsbtEKbpQbMn5jiJ+2zBLMqmM0YL8=; b=Nt8LRINmVeDr769RErpnLaIeXfPELF9qhbpYFR7qzL7qqBvkabcnCfZ6XufKRvpBRaDcqh 7k1uPZJfBkBIsNBA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, David Woodhouse , Andrew Cooper , Brian Gerst , Arjan van de Veen , Paolo Bonzini , Paul McKenney , Tom Lendacky , Sean Christopherson , Oleksandr Natalenko , Paul Menzel , "Guilherme G. Piccoli" , Piotr Gorski , Usama Arif , Juergen Gross , Boris Ostrovsky , xen-devel@lists.xenproject.org, Russell King , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Will Deacon , Guo Ren , linux-csky@vger.kernel.org, Thomas Bogendoerfer , linux-mips@vger.kernel.org, "James E.J. Bottomley" , Helge Deller , linux-parisc@vger.kernel.org, Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, Mark Rutland , Sabin Rapan , "Michael Kelley (LINUX)" , Ross Philipson Subject: [patch V4 06/37] x86/smpboot: Remove the CPU0 hotplug kludge References: <20230512203426.452963764@linutronix.de> MIME-Version: 1.0 Date: Fri, 12 May 2023 23:07:06 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765724701427215204?= X-GMAIL-MSGID: =?utf-8?q?1765724701427215204?= From: Thomas Gleixner This was introduced with commit e1c467e69040 ("x86, hotplug: Wake up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support physical hotplug of CPU0: "We'll change this code in the future to wake up hard offlined CPU0 if real platform and request are available." 11 years later this has not happened and physical hotplug is not officially supported. Remove the cruft. Signed-off-by: Thomas Gleixner Tested-by: Michael Kelley --- arch/x86/include/asm/apic.h | 1 arch/x86/include/asm/smp.h | 1 arch/x86/kernel/smpboot.c | 170 +++--------------------------------------- drivers/acpi/processor_idle.c | 4 4 files changed, 14 insertions(+), 162 deletions(-) --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -377,7 +377,6 @@ extern struct apic *__apicdrivers[], *__ * APIC functionality to boot other CPUs - only used on SMP: */ #ifdef CONFIG_SMP -extern int wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip); extern int lapic_can_unplug_cpu(void); #endif --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -130,7 +130,6 @@ void native_play_dead(void); void play_dead_common(void); void wbinvd_on_cpu(int cpu); int wbinvd_on_all_cpus(void); -void cond_wakeup_cpu0(void); void native_smp_send_reschedule(int cpu); void native_send_call_func_ipi(const struct cpumask *mask); --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -216,9 +216,6 @@ static void ap_calibrate_delay(void) cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy; } -static int cpu0_logical_apicid; -static int enable_start_cpu0; - /* * Activate a secondary processor. */ @@ -241,8 +238,6 @@ static void notrace start_secondary(void x86_cpuinit.early_percpu_clock_init(); smp_callin(); - enable_start_cpu0 = 0; - /* otherwise gcc will move up smp_processor_id before the cpu_init */ barrier(); /* Check TSC synchronization with the control CPU: */ @@ -410,7 +405,7 @@ void smp_store_cpu_info(int id) c->cpu_index = id; /* * During boot time, CPU0 has this setup already. Save the info when - * bringing up AP or offlined CPU0. + * bringing up an AP. */ identify_secondary_cpu(c); c->initialized = true; @@ -807,51 +802,14 @@ static void __init smp_quirk_init_udelay } /* - * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal - * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this - * won't ... remember to clear down the APIC, etc later. - */ -int -wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip) -{ - u32 dm = apic->dest_mode_logical ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL; - unsigned long send_status, accept_status = 0; - int maxlvt; - - /* Target chip */ - /* Boot on the stack */ - /* Kick the second */ - apic_icr_write(APIC_DM_NMI | dm, apicid); - - pr_debug("Waiting for send to finish...\n"); - send_status = safe_apic_wait_icr_idle(); - - /* - * Give the other CPU some time to accept the IPI. - */ - udelay(200); - if (APIC_INTEGRATED(boot_cpu_apic_version)) { - maxlvt = lapic_get_maxlvt(); - if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */ - apic_write(APIC_ESR, 0); - accept_status = (apic_read(APIC_ESR) & 0xEF); - } - pr_debug("NMI sent\n"); - - if (send_status) - pr_err("APIC never delivered???\n"); - if (accept_status) - pr_err("APIC delivery error (%lx)\n", accept_status); - - return (send_status | accept_status); -} - -static int -wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) + * Wake up AP by INIT, INIT, STARTUP sequence. + */ +static int wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip) { unsigned long send_status = 0, accept_status = 0; int maxlvt, num_starts, j; + preempt_disable(); maxlvt = lapic_get_maxlvt(); /* @@ -957,6 +915,7 @@ wakeup_secondary_cpu_via_init(int phys_a if (accept_status) pr_err("APIC delivery error (%lx)\n", accept_status); + preempt_enable(); return (send_status | accept_status); } @@ -997,67 +956,6 @@ static void announce_cpu(int cpu, int ap node, cpu, apicid); } -static int wakeup_cpu0_nmi(unsigned int cmd, struct pt_regs *regs) -{ - int cpu; - - cpu = smp_processor_id(); - if (cpu == 0 && !cpu_online(cpu) && enable_start_cpu0) - return NMI_HANDLED; - - return NMI_DONE; -} - -/* - * Wake up AP by INIT, INIT, STARTUP sequence. - * - * Instead of waiting for STARTUP after INITs, BSP will execute the BIOS - * boot-strap code which is not a desired behavior for waking up BSP. To - * void the boot-strap code, wake up CPU0 by NMI instead. - * - * This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined - * (i.e. physically hot removed and then hot added), NMI won't wake it up. - * We'll change this code in the future to wake up hard offlined CPU0 if - * real platform and request are available. - */ -static int -wakeup_cpu_via_init_nmi(int cpu, unsigned long start_ip, int apicid, - int *cpu0_nmi_registered) -{ - int id; - int boot_error; - - preempt_disable(); - - /* - * Wake up AP by INIT, INIT, STARTUP sequence. - */ - if (cpu) { - boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip); - goto out; - } - - /* - * Wake up BSP by nmi. - * - * Register a NMI handler to help wake up CPU0. - */ - boot_error = register_nmi_handler(NMI_LOCAL, - wakeup_cpu0_nmi, 0, "wake_cpu0"); - - if (!boot_error) { - enable_start_cpu0 = 1; - *cpu0_nmi_registered = 1; - id = apic->dest_mode_logical ? cpu0_logical_apicid : apicid; - boot_error = wakeup_secondary_cpu_via_nmi(id, start_ip); - } - -out: - preempt_enable(); - - return boot_error; -} - int common_cpu_up(unsigned int cpu, struct task_struct *idle) { int ret; @@ -1086,8 +984,7 @@ int common_cpu_up(unsigned int cpu, stru * Returns zero if CPU booted OK, else error code from * ->wakeup_secondary_cpu. */ -static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, - int *cpu0_nmi_registered) +static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle) { /* start_ip had better be page-aligned! */ unsigned long start_ip = real_mode_header->trampoline_start; @@ -1120,7 +1017,6 @@ static int do_boot_cpu(int apicid, int c * This grunge runs the startup process for * the targeted processor. */ - if (x86_platform.legacy.warm_reset) { pr_debug("Setting warm reset code and vector.\n"); @@ -1149,15 +1045,14 @@ static int do_boot_cpu(int apicid, int c * - Use a method from the APIC driver if one defined, with wakeup * straight to 64-bit mode preferred over wakeup to RM. * Otherwise, - * - Use an INIT boot APIC message for APs or NMI for BSP. + * - Use an INIT boot APIC message */ if (apic->wakeup_secondary_cpu_64) boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip); else if (apic->wakeup_secondary_cpu) boot_error = apic->wakeup_secondary_cpu(apicid, start_ip); else - boot_error = wakeup_cpu_via_init_nmi(cpu, start_ip, apicid, - cpu0_nmi_registered); + boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip); if (!boot_error) { /* @@ -1206,9 +1101,8 @@ static int do_boot_cpu(int apicid, int c int native_cpu_up(unsigned int cpu, struct task_struct *tidle) { int apicid = apic->cpu_present_to_apicid(cpu); - int cpu0_nmi_registered = 0; unsigned long flags; - int err, ret = 0; + int err; lockdep_assert_irqs_enabled(); @@ -1247,11 +1141,10 @@ int native_cpu_up(unsigned int cpu, stru if (err) return err; - err = do_boot_cpu(apicid, cpu, tidle, &cpu0_nmi_registered); + err = do_boot_cpu(apicid, cpu, tidle); if (err) { pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu); - ret = -EIO; - goto unreg_nmi; + return err; } /* @@ -1267,15 +1160,7 @@ int native_cpu_up(unsigned int cpu, stru touch_nmi_watchdog(); } -unreg_nmi: - /* - * Clean up the nmi handler. Do this after the callin and callout sync - * to avoid impact of possible long unregister time. - */ - if (cpu0_nmi_registered) - unregister_nmi_handler(NMI_LOCAL, "wake_cpu0"); - - return ret; + return 0; } /** @@ -1373,14 +1258,6 @@ static void __init smp_cpu_index_default } } -static void __init smp_get_logical_apicid(void) -{ - if (x2apic_mode) - cpu0_logical_apicid = apic_read(APIC_LDR); - else - cpu0_logical_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR)); -} - void __init smp_prepare_cpus_common(void) { unsigned int i; @@ -1443,8 +1320,6 @@ void __init native_smp_prepare_cpus(unsi /* Setup local timer */ x86_init.timers.setup_percpu_clockev(); - smp_get_logical_apicid(); - pr_info("CPU0: "); print_cpu_info(&cpu_data(0)); @@ -1752,18 +1627,6 @@ void play_dead_common(void) local_irq_disable(); } -/** - * cond_wakeup_cpu0 - Wake up CPU0 if needed. - * - * If NMI wants to wake up CPU0, start CPU0. - */ -void cond_wakeup_cpu0(void) -{ - if (smp_processor_id() == 0 && enable_start_cpu0) - start_cpu0(); -} -EXPORT_SYMBOL_GPL(cond_wakeup_cpu0); - /* * We need to flush the caches before going to sleep, lest we have * dirty data in our caches when we come back up. @@ -1831,8 +1694,6 @@ static inline void mwait_play_dead(void) __monitor(mwait_ptr, 0, 0); mb(); __mwait(eax, 0); - - cond_wakeup_cpu0(); } } @@ -1841,11 +1702,8 @@ void __noreturn hlt_play_dead(void) if (__this_cpu_read(cpu_info.x86) >= 4) wbinvd(); - while (1) { + while (1) native_halt(); - - cond_wakeup_cpu0(); - } } void native_play_dead(void) --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -597,10 +597,6 @@ static int acpi_idle_play_dead(struct cp io_idle(cx->address); } else return -ENODEV; - -#if defined(CONFIG_X86) && defined(CONFIG_HOTPLUG_CPU) - cond_wakeup_cpu0(); -#endif } /* Never reached */