From patchwork Tue May 16 09:09:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 94582 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp288027vqo; Tue, 16 May 2023 02:21:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Mz3LIC3/F35HnwQXXISrEAoumy0Y9QA8LsrfU5HU2FTOVeETWGlIvmZdtT/0eh5adsOsr X-Received: by 2002:a17:90b:1d0c:b0:247:1997:6a1f with SMTP id on12-20020a17090b1d0c00b0024719976a1fmr37338766pjb.12.1684228879901; Tue, 16 May 2023 02:21:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684228879; cv=none; d=google.com; s=arc-20160816; b=SBqcXoeeig2cm0SKujp5chBEAUFo3YbYP6AhcrngnZ054rHX5QQmzTUYXPbDQYvy6y oE3iwpO17KaDBjTRS3KxCAMCamDErpKLy5FQ8tp/oUr02umDv2lvDvxF9MsfCwyd9FwD Baw8ijpwNeBxx4K2ngn0n5YoLqp1MnqStsM3oa7O4XxoFEPjEAWDCcTltXEODLs0dAg/ 97aQlkKVKMNSAFO7ffN9f4bUZyAF4JTWzVxkMB5POq5OYTivowr9O+XKT7Z2aAh2IY3s cqgJGxUH0vr2x3n4GTjxV1rCb9iCeokwsfO0beFS2MVg8kjVjnxkE1pjtbjIFbyPMNaJ uxfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=kmb7GmwcHkt8LlypsC9tX9bRJEK96DDA+0p2OMOgRGo=; b=qBUC6fsfX4L4QbxoTPzvAyBdorXpKU/fPmOZdSzyh4ngrSTibQSQogjZr5RaDrk1ki nF0rGFxBrLXE3ux1zsHlt8uN+1zwc1AhIaEBxbggSuB2qEUG3ckB8Vp/t07dUe+Bn9ph /S27IOGfeCt79kIauuPYKIY8hrnXaUV7RgNbDXAvB1v3t0tjV1rF4NyFjKXl7s9gwKPS 1w4rGrwz/Zk/orJGKX/9h0h8EDbux6ZAxOw4xlqnmUGmEBBnnmFRMDh4QARvSmUQ41vT amvWiarlGwp9Qu1OOBenTArkvohOIcaG9A0QEbgEdQwMrAXhy8fnWc2sv1i5RUIZp8DI aJNA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=LY4aWGco; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o1-20020a17090a420100b00246fe4e326dsi1356931pjg.81.2023.05.16.02.21.07; Tue, 16 May 2023 02:21:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=LY4aWGco; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231869AbjEPJKS (ORCPT + 99 others); Tue, 16 May 2023 05:10:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231707AbjEPJJ7 (ORCPT ); Tue, 16 May 2023 05:09:59 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D8722D55; Tue, 16 May 2023 02:09:56 -0700 (PDT) Date: Tue, 16 May 2023 09:09:54 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1684228194; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kmb7GmwcHkt8LlypsC9tX9bRJEK96DDA+0p2OMOgRGo=; b=LY4aWGcoc7RTEbArVVANGoM0EgN0rxrRdwynspt5eVn+dPrx5gpYlqfTUZDROkqmiZ5vwh tyopAw8O0IGpYEivvcCL6iBKrBzw77YIoNiYAWJ9yMsXI5zX4zaTblvn3OzqLwOwWMe97r nEowt7baksVZlILmVwvLrMCsgaUQbEpP6r5ZJSbUnDRpKX8xCqUoRGhwcjBgae73OrOo8k 6J11VpWtlv/wUSHKUqK5GDMdB3FEw505kMga46OUh2kg/ibyxyWhbduguo64o4LGgfVJxn rginyLKyecRTXVAv7jGE5wzgL/XwkUSWhsYOO7LTm8SqrBFTsl/FwjF4j8znhQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1684228194; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kmb7GmwcHkt8LlypsC9tX9bRJEK96DDA+0p2OMOgRGo=; b=X+avK+pgRfomuhvc1BDXsqohLxWBaOih9sxTTrxaJTnWUXlqUu/5Ugk2X5IyGkJBhPAJ+g u6ZJzBlfpEyifjCA== From: "tip-bot2 for Thomas Gleixner" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: smp/core] x86/smpboot: Implement a bit spinlock to protect the realmode stack Cc: David Woodhouse , Thomas Gleixner , "Peter Zijlstra (Intel)" , Michael Kelley , Oleksandr Natalenko , Helge Deller , "Guilherme G. Piccoli" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20230512205257.355425551@linutronix.de> References: <20230512205257.355425551@linutronix.de> MIME-Version: 1.0 Message-ID: <168422819410.404.5810226958076562803.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765724710862759668?= X-GMAIL-MSGID: =?utf-8?q?1766041982225343732?= The following commit has been merged into the smp/core branch of tip: Commit-ID: f6f1ae9128d2a080ecdd55f85e8a0ca3ed1d58eb Gitweb: https://git.kernel.org/tip/f6f1ae9128d2a080ecdd55f85e8a0ca3ed1d58eb Author: Thomas Gleixner AuthorDate: Fri, 12 May 2023 23:07:53 +02:00 Committer: Peter Zijlstra CommitterDate: Mon, 15 May 2023 13:45:03 +02:00 x86/smpboot: Implement a bit spinlock to protect the realmode stack Parallel AP bringup requires that the APs can run fully parallel through the early startup code including the real mode trampoline. To prepare for this implement a bit-spinlock to serialize access to the real mode stack so that parallel upcoming APs are not going to corrupt each others stack while going through the real mode startup code. Co-developed-by: David Woodhouse Signed-off-by: David Woodhouse Signed-off-by: Thomas Gleixner Signed-off-by: Peter Zijlstra (Intel) Tested-by: Michael Kelley Tested-by: Oleksandr Natalenko Tested-by: Helge Deller # parisc Tested-by: Guilherme G. Piccoli # Steam Deck Link: https://lore.kernel.org/r/20230512205257.355425551@linutronix.de --- arch/x86/include/asm/realmode.h | 3 +++ arch/x86/kernel/head_64.S | 12 ++++++++++++ arch/x86/realmode/init.c | 3 +++ arch/x86/realmode/rm/trampoline_64.S | 23 ++++++++++++++++++----- 4 files changed, 36 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h index f6a1737..87e5482 100644 --- a/arch/x86/include/asm/realmode.h +++ b/arch/x86/include/asm/realmode.h @@ -52,6 +52,7 @@ struct trampoline_header { u64 efer; u32 cr4; u32 flags; + u32 lock; #endif }; @@ -64,6 +65,8 @@ extern unsigned long initial_stack; extern unsigned long initial_vc_handler; #endif +extern u32 *trampoline_lock; + extern unsigned char real_mode_blob[]; extern unsigned char real_mode_relocs[]; diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 8458033..f99e9ab 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -252,6 +252,16 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) movq TASK_threadsp(%rax), %rsp /* + * Now that this CPU is running on its own stack, drop the realmode + * protection. For the boot CPU the pointer is NULL! + */ + movq trampoline_lock(%rip), %rax + testq %rax, %rax + jz .Lsetup_gdt + movl $0, (%rax) + +.Lsetup_gdt: + /* * We must switch to a new descriptor in kernel space for the GDT * because soon the kernel won't have access anymore to the userspace * addresses where we're currently running on. We have to do that here @@ -433,6 +443,8 @@ SYM_DATA(initial_code, .quad x86_64_start_kernel) #ifdef CONFIG_AMD_MEM_ENCRYPT SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb) #endif + +SYM_DATA(trampoline_lock, .quad 0); __FINITDATA __INIT diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c index af56581..788e555 100644 --- a/arch/x86/realmode/init.c +++ b/arch/x86/realmode/init.c @@ -154,6 +154,9 @@ static void __init setup_real_mode(void) trampoline_header->flags = 0; + trampoline_lock = &trampoline_header->lock; + *trampoline_lock = 0; + trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd); /* Map the real mode stub as virtual == physical */ diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S index e38d61d..4822ad2 100644 --- a/arch/x86/realmode/rm/trampoline_64.S +++ b/arch/x86/realmode/rm/trampoline_64.S @@ -37,6 +37,20 @@ .text .code16 +.macro LOAD_REALMODE_ESP + /* + * Make sure only one CPU fiddles with the realmode stack + */ +.Llock_rm\@: + lock btsl $0, tr_lock + jnc 2f + pause + jmp .Llock_rm\@ +2: + # Setup stack + movl $rm_stack_end, %esp +.endm + .balign PAGE_SIZE SYM_CODE_START(trampoline_start) cli # We should be safe anyway @@ -49,8 +63,7 @@ SYM_CODE_START(trampoline_start) mov %ax, %es mov %ax, %ss - # Setup stack - movl $rm_stack_end, %esp + LOAD_REALMODE_ESP call verify_cpu # Verify the cpu supports long mode testl %eax, %eax # Check for return code @@ -93,8 +106,7 @@ SYM_CODE_START(sev_es_trampoline_start) mov %ax, %es mov %ax, %ss - # Setup stack - movl $rm_stack_end, %esp + LOAD_REALMODE_ESP jmp .Lswitch_to_protected SYM_CODE_END(sev_es_trampoline_start) @@ -177,7 +189,7 @@ SYM_CODE_START(pa_trampoline_compat) * In compatibility mode. Prep ESP and DX for startup_32, then disable * paging and complete the switch to legacy 32-bit mode. */ - movl $rm_stack_end, %esp + LOAD_REALMODE_ESP movw $__KERNEL_DS, %dx movl $(CR0_STATE & ~X86_CR0_PG), %eax @@ -241,6 +253,7 @@ SYM_DATA_START(trampoline_header) SYM_DATA(tr_efer, .space 8) SYM_DATA(tr_cr4, .space 4) SYM_DATA(tr_flags, .space 4) + SYM_DATA(tr_lock, .space 4) SYM_DATA_END(trampoline_header) #include "trampoline_common.S"