From patchwork Thu May 4 19:02:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 90206 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:6358:3046:b0:115:7a1d:dabb with SMTP id p6csp517567rwl; Thu, 4 May 2023 12:14:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ68CbPID1jPyibcMVgtPHQQKaRW4dCnfpIBiL3oNlUlbMyDoScjWTfQyaGl31ZtF0KYCsO7 X-Received: by 2002:a17:902:e810:b0:1a6:7ed0:147e with SMTP id u16-20020a170902e81000b001a67ed0147emr5897108plg.33.1683227646461; Thu, 04 May 2023 12:14:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683227646; cv=none; d=google.com; s=arc-20160816; b=eu5YlagrJNYdf/kqm+EapE3f/XbbL6n4uPHen4V3en+fVbdAZ2mDT0Y+E35soz46U0 J3klvyhLqWh/GToWC6wzzYO01BYTZK659zD3V28aIvhdax3QN9avjlU1FAIoXnbfncrw YgiiO6l4CL5VFaDNDmjJRiVf/6b1IioNPP1RV6MlJ6AIo+3bsrgU67sXQ67pxz75mOOY ADao8qC2voOPQRXgTPavKt3x1/xbdiqV6Pmc+wlieZfbptCbKR5U4a4thIm1pile7uI5 OBJsZB3DWO2TyVgO0IcRTeHSFoYYO6+y/b/NYGy19KJlsfXsMcIMgGLc4SDP03YdpGD6 SQkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=PV6WWoQh1vSNv/X00PyVAYzPvMrMTPPv2Nyz2I8t2to=; b=ZEpGXGl9a+6H+/dnhapHkLAf6WcBL+fMviGUpvgFl990teMcsZDNb069mgg6DQ9TKV LlU9A91oUmWwuzBe30Txg5Jo9UKX9uw+1TEaOS/n5gPXxkVxw3q82rv5Pdf+Y/7O8HxU LAIGaGZ5N+NyWSFbLfHBTA5hOV4I5j2+xWU26tvu/BAaamhnhIz9wKbMEIj52ZoVbvlf 4Uj2vrFm0QT4kvefZoNOSxIp/nPcy42LJXr5WI4sPNBFrBD95tj9cwqONVa3saAr1toi 0n3XQvpjLdlQhKLTyZgMdDauRep5mslt3dFIlvc5z+ak+36kJcxzT4UwrJ7/KvbmH74E sLBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="ngwo/uGw"; dkim=neutral (no key) header.i=@linutronix.de header.b=JP8BLAG9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b1-20020a170902d50100b001a9930c626dsi25076165plg.273.2023.05.04.12.13.53; Thu, 04 May 2023 12:14:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="ngwo/uGw"; dkim=neutral (no key) header.i=@linutronix.de header.b=JP8BLAG9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230435AbjEDTGE (ORCPT + 99 others); Thu, 4 May 2023 15:06:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230236AbjEDTFf (ORCPT ); Thu, 4 May 2023 15:05:35 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7A39A265; Thu, 4 May 2023 12:03:49 -0700 (PDT) Message-ID: <20230504185938.285676159@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1683226977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=PV6WWoQh1vSNv/X00PyVAYzPvMrMTPPv2Nyz2I8t2to=; b=ngwo/uGwLLzUyff7hlx9g/RxH5SaJg/lwXSYHXpqdS6uHQZR+3fUUQfO5oVYKvfxde06o+ +RefzZ3wy9tsLi/Hy8zZ6JDATGhXPUq/QZGaWvxRx4XQWUnhfUzD50bvR2y7xGwKDKLwGm X2C2HqxX/WfTRVLQ/0g6RaEnIOW09xcuy0eKhcHRqbtoKQXKdt82wFhNqtvbm9wFJV5YkB OALYtVjQEZsIWpsJDbJIBCW6VkLxJKc0h6Dc19Ep7b3dc7MH5JO49SLLpNpVovgHkV/qQu V+Pl5jVc8Rxfg5JATIDLK+m3hZbXbKpQITDQu4OVIIYiQysd2QZB5OGLMeokNg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1683226977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=PV6WWoQh1vSNv/X00PyVAYzPvMrMTPPv2Nyz2I8t2to=; b=JP8BLAG9V88lZ5GHmeL8/xfbmLZJ07n8AFIskCcZbg4rYh3fxmVO9mzY0ohdj0bEvJKj2e XTgJfhIo5hUgrgBA== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, David Woodhouse , Andrew Cooper , Brian Gerst , Arjan van de Veen , Paolo Bonzini , Paul McKenney , Tom Lendacky , Sean Christopherson , Oleksandr Natalenko , Paul Menzel , "Guilherme G. Piccoli" , Piotr Gorski , Usama Arif , Juergen Gross , Boris Ostrovsky , xen-devel@lists.xenproject.org, Russell King , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Will Deacon , Guo Ren , linux-csky@vger.kernel.org, Thomas Bogendoerfer , linux-mips@vger.kernel.org, "James E.J. Bottomley" , Helge Deller , linux-parisc@vger.kernel.org, Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, Mark Rutland , Sabin Rapan , "Michael Kelley (LINUX)" , David Woodhouse Subject: [patch V2 36/38] x86/smpboot: Implement a bit spinlock to protect the realmode stack References: <20230504185733.126511787@linutronix.de> MIME-Version: 1.0 Date: Thu, 4 May 2023 21:02:56 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764992112297203506?= X-GMAIL-MSGID: =?utf-8?q?1764992112297203506?= From: Thomas Gleixner Parallel AP bringup requires that the APs can run fully parallel through the early startup code including the real mode trampoline. To prepare for this implement a bit-spinlock to serialize access to the real mode stack so that parallel upcoming APs are not going to corrupt each others stack while going through the real mode startup code. Co-developed-by: David Woodhouse Signed-off-by: David Woodhouse Signed-off-by: Thomas Gleixner --- arch/x86/include/asm/realmode.h | 3 +++ arch/x86/kernel/head_64.S | 13 +++++++++++++ arch/x86/realmode/init.c | 3 +++ arch/x86/realmode/rm/trampoline_64.S | 27 ++++++++++++++++++++++----- 4 files changed, 41 insertions(+), 5 deletions(-) --- --- a/arch/x86/include/asm/realmode.h +++ b/arch/x86/include/asm/realmode.h @@ -52,6 +52,7 @@ struct trampoline_header { u64 efer; u32 cr4; u32 flags; + u32 lock; #endif }; @@ -64,6 +65,8 @@ extern unsigned long initial_stack; extern unsigned long initial_vc_handler; #endif +extern u32 *trampoline_lock; + extern unsigned char real_mode_blob[]; extern unsigned char real_mode_relocs[]; --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -252,6 +252,17 @@ SYM_INNER_LABEL(secondary_startup_64_no_ movq TASK_threadsp(%rax), %rsp /* + * Now that this CPU is running on its own stack, drop the realmode + * protection. For the boot CPU the pointer is NULL! + */ + movq trampoline_lock(%rip), %rax + testq %rax, %rax + jz .Lsetup_gdt + lock + btrl $0, (%rax) + +.Lsetup_gdt: + /* * We must switch to a new descriptor in kernel space for the GDT * because soon the kernel won't have access anymore to the userspace * addresses where we're currently running on. We have to do that here @@ -433,6 +444,8 @@ SYM_DATA(initial_code, .quad x86_64_star #ifdef CONFIG_AMD_MEM_ENCRYPT SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb) #endif + +SYM_DATA(trampoline_lock, .quad 0); __FINITDATA __INIT --- a/arch/x86/realmode/init.c +++ b/arch/x86/realmode/init.c @@ -154,6 +154,9 @@ static void __init setup_real_mode(void) trampoline_header->flags = 0; + trampoline_lock = &trampoline_header->lock; + *trampoline_lock = 0; + trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd); /* Map the real mode stub as virtual == physical */ --- a/arch/x86/realmode/rm/trampoline_64.S +++ b/arch/x86/realmode/rm/trampoline_64.S @@ -37,6 +37,24 @@ .text .code16 +.macro LOAD_REALMODE_ESP + /* + * Make sure only one CPU fiddles with the realmode stack + */ +.Llock_rm\@: + btl $0, tr_lock + jnc 2f + pause + jmp .Llock_rm\@ +2: + lock + btsl $0, tr_lock + jc .Llock_rm\@ + + # Setup stack + movl $rm_stack_end, %esp +.endm + .balign PAGE_SIZE SYM_CODE_START(trampoline_start) cli # We should be safe anyway @@ -49,8 +67,7 @@ SYM_CODE_START(trampoline_start) mov %ax, %es mov %ax, %ss - # Setup stack - movl $rm_stack_end, %esp + LOAD_REALMODE_ESP call verify_cpu # Verify the cpu supports long mode testl %eax, %eax # Check for return code @@ -93,8 +110,7 @@ SYM_CODE_START(sev_es_trampoline_start) mov %ax, %es mov %ax, %ss - # Setup stack - movl $rm_stack_end, %esp + LOAD_REALMODE_ESP jmp .Lswitch_to_protected SYM_CODE_END(sev_es_trampoline_start) @@ -177,7 +193,7 @@ SYM_CODE_START(pa_trampoline_compat) * In compatibility mode. Prep ESP and DX for startup_32, then disable * paging and complete the switch to legacy 32-bit mode. */ - movl $rm_stack_end, %esp + LOAD_REALMODE_ESP movw $__KERNEL_DS, %dx movl $(CR0_STATE & ~X86_CR0_PG), %eax @@ -241,6 +257,7 @@ SYM_DATA_START(trampoline_header) SYM_DATA(tr_efer, .space 8) SYM_DATA(tr_cr4, .space 4) SYM_DATA(tr_flags, .space 4) + SYM_DATA(tr_lock, .space 4) SYM_DATA_END(trampoline_header) #include "trampoline_common.S"