From patchwork Mon May 22 07:14:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97138 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272804vqo; Mon, 22 May 2023 00:46:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6Lfgi9xxpLufFmcb5UZ8/9xp9q3hf2oGToRmRw7DIWfj2H7xxZYgQwEjUODrpYfrsBgFDO X-Received: by 2002:a17:90b:e8f:b0:24e:3b85:a8a with SMTP id fv15-20020a17090b0e8f00b0024e3b850a8amr10739574pjb.8.1684741562416; Mon, 22 May 2023 00:46:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741562; cv=none; d=google.com; s=arc-20160816; b=WWisiSXV+5ay6XKc2OiTplrnorlUvKVr1k1jyWWXkq/jV8kryLXjz/ATdJOgpc6pvR JE8ivtFwHL/Uu8r6NA171xzv76uxAvwzXUS/y/Or41pnfYO97ha1NIFtL6iZmWWYHa3a 8bMiAwfZez/IuKAVWIjsJZQ6YLF2WT1QMJ7N6CTXc2/uY+KfLlKmq4PDL+iGCuJ5FrqU b4Q8qqYKgoghTK6Y/NlkC7HsmrfKzsmLkHu0zswSWciOejIpuUu5kCRakMe8nKL0ULwz l05k2jY2KNT7OdmyjlqrstL7x6j1HoK2pPp5HgN+iG8TxU1C4tmPdoYYG5rI1TwQ7Jtr 2Xpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2nct0eXrdF4JNt6BmazV/eXEUxn2frpqXHOq85y9xRY=; b=bIE5DkepdhMv3ju4yjpDrnP/0Zt4gSaKi8JVxfxb0FoOmNDwZDlVjv+3nMWk+bauun 6n8z+Xr6Tueiam2wZ/eQXwDyqwPKgSFpw9osISAEvIATKVgLTyR+kH6Iu0TKgC5OImK7 16FRLiESdpp4yLphm1FBW1oqrJkxgtzkXqtHaxX11gFSLv2oddFw5OuJqaanXwM6FLKg wKKFY5L5tnf33qOSZ9+WhWDLc3PDcdr6FJFsz7JXVzKL3LfwYixqqTDOhfgNoimkIPd5 kyrdG8mYuU9XKho1XZrNpG9wHknBu0ueeco7HoiD790/8mr+W+1VQ2AsYc5HWGfOUW+G hEHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LcLHWjFN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t22-20020a17090ad51600b0024e33c63ed5si2477696pju.15.2023.05.22.00.45.50; Mon, 22 May 2023 00:46:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LcLHWjFN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231657AbjEVHPz (ORCPT + 99 others); Mon, 22 May 2023 03:15:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232318AbjEVHPR (ORCPT ); Mon, 22 May 2023 03:15:17 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66E19BA; Mon, 22 May 2023 00:15:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 04100611D8; Mon, 22 May 2023 07:15:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5FB1C4339C; Mon, 22 May 2023 07:15:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739711; bh=opd5H6IVQawxAyaERFSxkNMBcud1PLTZeWa/VKUckUQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LcLHWjFN1y4DO4wP08aTtHVSTOdt6iegTgkMZRlvdenRn54tcbRMqwcmm4s51XJRU xZTkpP6Zm1M6y2Z902fcBOSN2x0ubjIx//SFFBiZeo5Q+wzuTCm6bJpmWKWeBzao9h Jyg3v7Plta9HRrFr1tIn89Pr2CiYZs8/MD0hBsdGPom0aYdk9FEGKT077ychWS8QMD vWLFh0dhr5DxryBz3Oc/4DMQDswdPI99IrdreVCOLdpu6WjKJYdoxFyQgzejE8RlG4 7dyZ7FsJp3fCMSY2KpElpa1/6A7RON61LopYpBVED/ldN9TZVd/P6q5bb+kT2DDfLh mGS8mai9+jYEQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 09/21] x86/decompressor: Avoid the need for a stack in the 32-bit trampoline Date: Mon, 22 May 2023 09:14:03 +0200 Message-Id: <20230522071415.501717-10-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6184; i=ardb@kernel.org; h=from:subject; bh=opd5H6IVQawxAyaERFSxkNMBcud1PLTZeWa/VKUckUQ=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzOLwtagOtjZW63VHePMPTpXZwnphQ/3/P2tkN/wVz GmJ963rKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOJ72Jk+CRg5J8iU7T5gOip usWnl/ldi/z/YQn3xh3Prs1NbJW6c5fhfzjji+smJ5y3Nbsselyb47hLYZLzf489ky5wLQo0ZPV exQsA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579568528595190?= X-GMAIL-MSGID: =?utf-8?q?1766579568528595190?= The 32-bit trampoline no longer uses the stack for anything except performing a far return back to long mode. Currently, this stack is placed in the same page that carries the trampoline code, which means this page must be mapped writable and executable, and the stack is therefore executable as well. Replace the far return with a far jump, so that the return address can be pre-calculated and patched into the code before it is called. This removes the need for a stack entirely, and in a later patch, this will be taken advantage of by removing writable permissions from (and adding executable permissions to) this code page explicitly when booting via the EFI stub. Not touching the stack pointer also makes it more straight-forward to call the trampoline code as an ordinary 64-bit function from C code. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 45 ++++++++------------ arch/x86/boot/compressed/pgtable.h | 6 +-- arch/x86/boot/compressed/pgtable_64.c | 12 +++++- 3 files changed, 32 insertions(+), 31 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index a387cd80964e1a1e..741b4e8fefc915ea 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -449,9 +449,6 @@ SYM_CODE_START(startup_64) leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax call *%rax - /* Restore the stack, the 32-bit trampoline uses its own stack */ - leaq rva(boot_stack_end)(%rbx), %rsp - /* * cleanup_trampoline() would restore trampoline memory. * @@ -537,24 +534,22 @@ SYM_FUNC_END(.Lrelocated) * EDI contains the base address of the trampoline memory. * Non-zero ESI means trampoline needs to enable 5-level paging. */ + .section ".rodata", "a", @progbits SYM_CODE_START(trampoline_32bit_src) - popq %r8 /* Switch to compatibility mode (CS.L = 0 CS.D = 1) via far return */ pushq $__KERNEL32_CS leaq 0f(%rip), %rax pushq %rax lretq + /* + * The 32-bit code below will do a far jump back to long mode and end + * up here after reconfiguring the number of paging levels. + */ +.Lret: retq + .code32 -0: /* Set up data and stack segments */ - movl $__KERNEL_DS, %eax - movl %eax, %ds - movl %eax, %ss - - /* Set up new stack */ - leal TRAMPOLINE_32BIT_STACK_END(%edi), %esp - - /* Disable paging */ +0: /* Disable paging */ movl %cr0, %eax btrl $X86_CR0_PG_BIT, %eax movl %eax, %cr0 @@ -608,26 +603,22 @@ SYM_CODE_START(trampoline_32bit_src) 1: movl %eax, %cr4 - /* Calculate address of paging_enabled() once we are executing in the trampoline */ - leal .Lpaging_enabled - trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_OFFSET(%edi), %eax - - /* Prepare the stack for far return to Long Mode */ - pushl $__KERNEL_CS - pushl %eax - /* Enable paging again. */ movl %cr0, %eax btsl $X86_CR0_PG_BIT, %eax movl %eax, %cr0 - lret + /* + * Return to the 64-bit calling code using LJMP rather than LRET, to + * avoid the need for a 32-bit addressable stack. The destination + * address will be adjusted after the template code is copied into a + * 32-bit addressable buffer. + */ +.Ljmp: ljmpl $__KERNEL_CS, $(.Lret - trampoline_32bit_src) SYM_CODE_END(trampoline_32bit_src) - .code64 -SYM_FUNC_START_LOCAL_NOALIGN(.Lpaging_enabled) - /* Return from the trampoline */ - jmp *%r8 -SYM_FUNC_END(.Lpaging_enabled) +/* keep this right after trampoline_32bit_src() so we can infer its size */ +SYM_DATA(trampoline_ljmp_imm_offset, .word .Ljmp + 1 - trampoline_32bit_src) /* * The trampoline code has a size limit. @@ -636,7 +627,7 @@ SYM_FUNC_END(.Lpaging_enabled) */ .org trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_SIZE - .code32 + .text SYM_FUNC_START_LOCAL_NOALIGN(.Lno_longmode) /* This isn't an x86-64 CPU, so hang intentionally, we cannot continue */ 1: diff --git a/arch/x86/boot/compressed/pgtable.h b/arch/x86/boot/compressed/pgtable.h index 4e8cef135226bcbb..131488f50af55d0a 100644 --- a/arch/x86/boot/compressed/pgtable.h +++ b/arch/x86/boot/compressed/pgtable.h @@ -6,9 +6,7 @@ #define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0 #define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0xA0 - -#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZE +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80 #ifndef __ASSEMBLER__ @@ -16,5 +14,7 @@ extern unsigned long *trampoline_32bit; extern void trampoline_32bit_src(void *trampoline, bool enable_5lvl); +extern const u16 trampoline_ljmp_imm_offset; + #endif /* __ASSEMBLER__ */ #endif /* BOOT_COMPRESSED_PAGETABLE_H */ diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index 2ac12ff4111bf8c0..09fc18180929fab3 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -109,6 +109,7 @@ static unsigned long find_trampoline_placement(void) struct paging_config paging_prepare(void *rmode) { struct paging_config paging_config = {}; + void *tramp_code; /* Initialize boot_params. Required for cmdline_find_option_bool(). */ boot_params = rmode; @@ -143,9 +144,18 @@ struct paging_config paging_prepare(void *rmode) memset(trampoline_32bit, 0, TRAMPOLINE_32BIT_SIZE); /* Copy trampoline code in place */ - memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), + tramp_code = memcpy(trampoline_32bit + + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE); + /* + * Avoid the need for a stack in the 32-bit trampoline code, by using + * LJMP rather than LRET to return back to long mode. LJMP takes an + * immediate absolute address, so we have to adjust that based on the + * placement of the trampoline. + */ + *(u32 *)(tramp_code + trampoline_ljmp_imm_offset) += (unsigned long)tramp_code; + /* * The code below prepares page table in trampoline memory. *