Message ID | 20230414232310.569498144@linutronix.de |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp736010vqo; Fri, 14 Apr 2023 17:07:46 -0700 (PDT) X-Google-Smtp-Source: AKy350bkLxkE7Lkt0526AFjXot7s9UzsMgGRF5EOCMGAz4FxSi2R2ZJifrG6iNisEem9WgYxfrUE X-Received: by 2002:a05:6a20:cb43:b0:da:368e:7c73 with SMTP id hd3-20020a056a20cb4300b000da368e7c73mr6551600pzb.37.1681517266007; Fri, 14 Apr 2023 17:07:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681517265; cv=none; d=google.com; s=arc-20160816; b=Fbv16ZLnnL4fW341yBaGk61srSLofKOXhxiSKmuTx/haQxM08n7h//xfWlGK1+U6F5 g2/uJAK0B4Ha4oLRduSsrvunzFNYzfxPe2vUDAGgnU1K/5fm2kBiXZVeV8O32GzDrk0t l/IqeXjV8RnBv/ww3ap6uGTxBqQVjFCUYas7Sv4A2dgTMhSo0LbtpfOug2Uiy1gsfsRl MxDRFa6S9nIFuvtwH5LxXG5wpFW+pxZh+kjdAi4qAIpBsH8zOT1j+0mRA+CRiiJ7NmYq /KLQ/3DR/UbrmgXMHV+kl7xZF8OP1mr5d4RCRadU3l9w8DGIUdBRBSxfFPwYWeB6Ee/k xIhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=dAqk1yNFRcZF7qilzpcfVR/70NZ8+F/SNrVuS7Hd0dk=; b=EXywY4kJoZ+R/FLTihGBP/JijTnMujwQ8MIXjBC6ZPGQHldRupdsNC2CNJx/kY7wKZ udtUQ2Or9JAy6qzqozdnE33zbHi8YbU9TX1Rr95Mumn8P7K3RO1zpjAsV+oL95NosWtC 3O1ZVtA1nXIX9w8Ca+YggJeu3SAF2QwO4fld4qXJcaGRIIbg5m3a2i/ktQb4W750n2Cy zpaNxd+UgVp3iajgvQ6lXzwaq/7UV1BIL0Mv5BPWVI/EwBD3TPtky+XUq/p2RddTkfkg ZjOitpWKAWqOpclpBq+/JdzCJNi99izUeckWAm0MM1UEQQAD+kYKHuR9Qevf6B2en6H0 i79g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=AU6aWn3l; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e26-20020a63501a000000b0051b31338860si5847090pgb.542.2023.04.14.17.07.28; Fri, 14 Apr 2023 17:07:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=AU6aWn3l; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230523AbjDNXrF (ORCPT <rfc822;leviz.kernel.dev@gmail.com> + 99 others); Fri, 14 Apr 2023 19:47:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230230AbjDNXql (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Fri, 14 Apr 2023 19:46:41 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FE70B464; Fri, 14 Apr 2023 16:45:41 -0700 (PDT) Message-ID: <20230414232310.569498144@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1681515889; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=dAqk1yNFRcZF7qilzpcfVR/70NZ8+F/SNrVuS7Hd0dk=; b=AU6aWn3llfgN87vGRPuEWQIrjtTFwK3yn2f2vw94yybrTzt5H4LRbc91tm1Xw3BwSnq5hP rQmL01eHYwGuD/tJ0f+L2+Oe406bYjAoY3D+heXxkuTKO7KuUfo+Vs6B1IlkwX6ZMSgfKv MeC7l/ITcGbKIYw5eRyObkCJegLxrMXZzPk/rNGf+zutst2sEf6AX47fGrPZphUN9CnuIP 6Hqq0/37hmlZ5Bbfm3XWsFI3yH4n+lXK8uvrCVgYW4m/9hcfQmz9XLNpsQ2ZNbshSo8dWB 0ORFcD1Xi54QY+d+vfpgXIdhI1e6dB7Jux3a2oBfTKMsXcUuUQI04NVVQTd0zA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1681515889; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=dAqk1yNFRcZF7qilzpcfVR/70NZ8+F/SNrVuS7Hd0dk=; b=9EnFpcJNs8EwCrNKZip93QB/LFNFkvbQArfrS6NvwMyIg/IEdfp5AE2HOIUr5Sp+QKyDnB /vmf1BkRatgvPdDA== From: Thomas Gleixner <tglx@linutronix.de> To: LKML <linux-kernel@vger.kernel.org> Cc: x86@kernel.org, David Woodhouse <dwmw@infradead.org>, Andrew Cooper <andrew.cooper3@citrix.com>, Brian Gerst <brgerst@gmail.com>, "Arjan van de Veen" <arjan@linux.intel.com>, Paolo Bonzini <pbonzini@redhat.com>, Paul McKenney <paulmck@kernel.org>, Tom Lendacky <thomas.lendacky@amd.com>, Sean Christopherson <seanjc@google.com>, Oleksandr Natalenko <oleksandr@natalenko.name>, Paul Menzel <pmenzel@molgen.mpg.de>, "Guilherme G. Piccoli" <gpiccoli@igalia.com>, Piotr Gorski <lucjan.lucjanov@gmail.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, linux-arm-kernel@lists.infradead.org, David Woodhouse <dwmw@amazon.co.uk>, Usama Arif <usama.arif@bytedance.com>, Juergen Gross <jgross@suse.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>, xen-devel@lists.xenproject.org, Russell King <linux@armlinux.org.uk>, Arnd Bergmann <arnd@arndb.de>, Guo Ren <guoren@kernel.org>, linux-csky@vger.kernel.org, Thomas Bogendoerfer <tsbogend@alpha.franken.de>, linux-mips@vger.kernel.org, "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>, Helge Deller <deller@gmx.de>, linux-parisc@vger.kernel.org, Paul Walmsley <paul.walmsley@sifive.com>, Palmer Dabbelt <palmer@dabbelt.com>, linux-riscv@lists.infradead.org, Mark Rutland <mark.rutland@arm.com>, Sabin Rapan <sabrapan@amazon.com> Subject: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization References: <20230414225551.858160935@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Date: Sat, 15 Apr 2023 01:44:49 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763198648967252718?= X-GMAIL-MSGID: =?utf-8?q?1763198648967252718?= |
Series |
cpu/hotplug, x86: Reworked parallel CPU bringup
|
|
Commit Message
Thomas Gleixner
April 14, 2023, 11:44 p.m. UTC
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/smp.h | 2 +-
arch/arm64/kernel/smp.c | 14 +++++---------
3 files changed, 7 insertions(+), 10 deletions(-)
Comments
On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote: > Switch to the CPU hotplug core state tracking and synchronization > mechanim. No functional change intended. > > Signed-off-by: Thomas Gleixner <tglx@linutronix.de> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will@kernel.org> > Cc: linux-arm-kernel@lists.infradead.org I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to work fine with a bunch of vCPUs being hotplugged off and on again randomly. FWIW: Tested-by: Mark Rutland <mark.rutland@arm.com> I also hacked the code to have the dying CPU spin forever before the call to cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per /sys/devices/system/cpu/$N/online). As a tangent/aside, we might need to improve that for confidential compute architectures, and we might want to generically track cpus which might still be using kernel text/data. On arm64 we ensure that via our cpu_kill() callback (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP have a similar mechanism. Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things to blow up. Thanks, Mark. > --- > arch/arm64/Kconfig | 1 + > arch/arm64/include/asm/smp.h | 2 +- > arch/arm64/kernel/smp.c | 14 +++++--------- > 3 files changed, 7 insertions(+), 10 deletions(-) > > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -216,6 +216,7 @@ config ARM64 > select HAVE_KPROBES > select HAVE_KRETPROBES > select HAVE_GENERIC_VDSO > + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU > select IRQ_DOMAIN > select IRQ_FORCED_THREADING > select KASAN_VMALLOC if KASAN > --- a/arch/arm64/include/asm/smp.h > +++ b/arch/arm64/include/asm/smp.h > @@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_ > > extern int __cpu_disable(void); > > -extern void __cpu_die(unsigned int cpu); > +static inline void __cpu_die(unsigned int cpu) { } > extern void cpu_die(void); > extern void cpu_die_early(void); > > --- a/arch/arm64/kernel/smp.c > +++ b/arch/arm64/kernel/smp.c > @@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu) > } > > /* > - * called on the thread which is asking for a CPU to be shutdown - > - * waits until shutdown has completed, or it is timed out. > + * Called on the thread which is asking for a CPU to be shutdown after the > + * shutdown completed. > */ > -void __cpu_die(unsigned int cpu) > +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) > { > int err; > > - if (!cpu_wait_death(cpu, 5)) { > - pr_crit("CPU%u: cpu didn't die\n", cpu); > - return; > - } > pr_debug("CPU%u: shutdown\n", cpu); > > /* > @@ -370,8 +366,8 @@ void cpu_die(void) > > local_daif_mask(); > > - /* Tell __cpu_die() that this CPU is now safe to dispose of */ > - (void)cpu_report_death(); > + /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */ > + cpuhp_ap_report_dead(); > > /* > * Actually shutdown the CPU. This must never fail. The specific hotplug >
On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote: > On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote: > I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to > work fine with a bunch of vCPUs being hotplugged off and on again randomly. > > FWIW: > > Tested-by: Mark Rutland <mark.rutland@arm.com> > > I also hacked the code to have the dying CPU spin forever before the call to > cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call > arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per > /sys/devices/system/cpu/$N/online). Nice! > As a tangent/aside, we might need to improve that for confidential compute > architectures, and we might want to generically track cpus which might still be > using kernel text/data. On arm64 we ensure that via our cpu_kill() callback > (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP > have a similar mechanism. > > Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the > kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait > for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things > to blow up. There are a gazillion ways for a malicious hypervisor to blow up a 'squint enough to be confident' guest. The real question is whether it can utilize such a blow up to extract confidential information from the guest. If not then it's just yet another way of DoS which is an "acceptable" attack as it only affects availability but not confidentiality. Thanks, tglx
On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote: > On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote: > > As a tangent/aside, we might need to improve that for confidential compute > > architectures, and we might want to generically track cpus which might still be > > using kernel text/data. On arm64 we ensure that via our cpu_kill() callback > > (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP > > have a similar mechanism. > > > > Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the > > kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait > > for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things > > to blow up. > > There are a gazillion ways for a malicious hypervisor to blow up a > 'squint enough to be confident' guest. > > The real question is whether it can utilize such a blow up to extract > confidential information from the guest. > > If not then it's just yet another way of DoS which is an "acceptable" > attack as it only affects availability but not confidentiality. Sure. My thinking is that this is an attack against the *integrity* of the guest (since the vCPU that gets unpasued may write to memory), and so it's potentially more than just a DoS. I only mention this because I'd like to account for that on arm64, and if other architectures also wanted to handle that it might make sense to have some common infrastructure to track whether CPUs are potentially still within the kernel. Thanks, Mark.
On Wed, Apr 26 2023 at 08:59, Mark Rutland wrote: > On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote: >> If not then it's just yet another way of DoS which is an "acceptable" >> attack as it only affects availability but not confidentiality. > > Sure. > > My thinking is that this is an attack against the *integrity* of the guest > (since the vCPU that gets unpasued may write to memory), and so it's > potentially more than just a DoS. > > I only mention this because I'd like to account for that on arm64, and if other > architectures also wanted to handle that it might make sense to have some > common infrastructure to track whether CPUs are potentially still within the > kernel. Fair enough.
--- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -216,6 +216,7 @@ config ARM64 select HAVE_KPROBES select HAVE_KRETPROBES select HAVE_GENERIC_VDSO + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU select IRQ_DOMAIN select IRQ_FORCED_THREADING select KASAN_VMALLOC if KASAN --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_ extern int __cpu_disable(void); -extern void __cpu_die(unsigned int cpu); +static inline void __cpu_die(unsigned int cpu) { } extern void cpu_die(void); extern void cpu_die_early(void); --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu) } /* - * called on the thread which is asking for a CPU to be shutdown - - * waits until shutdown has completed, or it is timed out. + * Called on the thread which is asking for a CPU to be shutdown after the + * shutdown completed. */ -void __cpu_die(unsigned int cpu) +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { int err; - if (!cpu_wait_death(cpu, 5)) { - pr_crit("CPU%u: cpu didn't die\n", cpu); - return; - } pr_debug("CPU%u: shutdown\n", cpu); /* @@ -370,8 +366,8 @@ void cpu_die(void) local_daif_mask(); - /* Tell __cpu_die() that this CPU is now safe to dispose of */ - (void)cpu_report_death(); + /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */ + cpuhp_ap_report_dead(); /* * Actually shutdown the CPU. This must never fail. The specific hotplug