From patchwork Mon May 22 07:13:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97129 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272522vqo; Mon, 22 May 2023 00:45:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7zC7z2yDjpxYzA+C5Uu2bA4Lc7j0otViOzYgu1Cgg+Eb+dxB1+nmB4uUeboaNvTTZ0dG46 X-Received: by 2002:a05:6a20:3c91:b0:100:ed92:7d4d with SMTP id b17-20020a056a203c9100b00100ed927d4dmr11526250pzj.31.1684741526066; Mon, 22 May 2023 00:45:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741526; cv=none; d=google.com; s=arc-20160816; b=IyLn1lFY1rQ7ZOL3CyK6s1DllP+FWG/CSNAPtqqpO0QlbbJlmK6y+lykUGNEBUivUo DZi0X6albSWCG+L0X4DcGcFvBf5x/9zGkjWV6su1gVMHkloosFtvBRBkUvJz5AEq8Bl6 PK4hkrKyH5NIUuY/F6z+EPXVvjW2cbbJYGRaM69sC6hw0G5vrutmimdQk/uemxYhyxeB cswAKhbTq/j+vuwM6aXVkv4d5mmwIY+Z1QSfnHIEQiDpAnqUIAta0uF5GtDlD3rumvoI XU3vH746o72SynVewaxJqoiLUyZMUG31dCtEtO1ob8YoI9gLgMlAWMzAzlpoo46g3ku/ LVbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KeTPHj+I1fYwbE5NTuxo0fAziTIJ76oBXolfyY/XHLA=; b=JoXbh8wUwhvNhi1cbFRwwYacS2Ks9FH9JHTDIOvd58B7npnp9JIzF1Ow5UxlnLgzDi YE4bSJN66I+CgNG//EBvoDMHBRT6a7hY9e66zO2Ft3K1Yn7Bhfe5rn8N4Rdqu+vPAQbY V8PEhU7T4zc4SyBM9G4I3PXAWohRM2/UHB7JKPfBNjjTVI4POaecKgY6ambI3Ptv361/ fOD/LhEAUgTkRNZGuAycNrXzunHshNYje4ZfGVjSfB8BKXomIr8KpwvzIccUojjCbni2 SaCKHI0nO3K+SEm4cifRNQ/P7IALZe3Z5bbUtruw2Gbd9fsd4PJPQBMBNXi9ECfloCBL Wo0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=cbZw27Yc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i1-20020a639d01000000b005348464b81fsi2873837pgd.371.2023.05.22.00.45.13; Mon, 22 May 2023 00:45:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=cbZw27Yc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232427AbjEVHPZ (ORCPT + 99 others); Mon, 22 May 2023 03:15:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232424AbjEVHOn (ORCPT ); Mon, 22 May 2023 03:14:43 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CA6E18C; Mon, 22 May 2023 00:14:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BCE5361DD9; Mon, 22 May 2023 07:14:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73E98C433A0; Mon, 22 May 2023 07:14:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739678; bh=2ugw2aI3WS8XFLpq392iX/qheNdS02iXQMzmuFjsdqI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cbZw27Ycs8S+dDX/2VuheOzDlMUCm1wZOt8RUWc+IP8kxfzR0U8iXHqIVeqhkLUxG Nc0gfiw/AyKvGwZ7GqUR5xZGagAdlZ22F1cnAEMSYtFyDHtm/v7hiiS0iN8un0pdtO HoBM0vU7Bm42jdFt/bhGATlwG4su4IXa1/sOpUtd6v5O1t5hCqc7Ni0dmgQoH2Jw1k iHKngvdziAtkRsHDHKFs8OBG6f+B5juNfs3HASC5cyc0q12eeQbqTVLnFjdgIVY2Kv /s5lMfZ1QfKHUDikhqjLfI5HlsxDlUIWc/nixFPhRgD20b0oiS+FucfaXTc4VLUpV5 6WZ2RRyFlyMAg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 01/21] x86/efistub: Branch straight to kernel entry point from C code Date: Mon, 22 May 2023 09:13:55 +0200 Message-Id: <20230522071415.501717-2-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1778; i=ardb@kernel.org; h=from:subject; bh=2ugw2aI3WS8XFLpq392iX/qheNdS02iXQMzmuFjsdqI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbTO+qocu11eWX0wLvqVVu3fSrSm7+6bl1WwxUFK4GT /zD8+BPRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI/3ZGhq/ibDHmCYkrn+04 sDIi54ll3wfl0upKb/7r/Ld+rZgaXMfIMEH0pLX0TbWQuFm+F65WlXB2dixbmH9l1dPK2S9ZPrE Y8AAA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579530789387476?= X-GMAIL-MSGID: =?utf-8?q?1766579530789387476?= Instead of returning to the calling code in assembler that does nothing more than perform an indirect call with the boot_params pointer in register ESI/RSI, perform the jump directly from the EFI stub C code. This will allow the asm entrypoint code to be dropped entirely in subsequent patches. Signed-off-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/x86-stub.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 8d17cee8b98e1a63..095aaa8b0ee30fb9 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -769,9 +769,17 @@ static efi_status_t exit_boot(struct boot_params *boot_params, void *handle) return EFI_SUCCESS; } +static void __noreturn enter_kernel(unsigned long kernel_addr, + struct boot_params *boot_params) +{ + /* enter decompressed kernel with boot_params pointer in RSI/ESI */ + asm("jmp *%0"::"r"(kernel_addr), "S"(boot_params)); + + unreachable(); +} + /* - * On success, we return the address of startup_32, which has potentially been - * relocated by efi_relocate_kernel. + * On success, we jump to the relocated kernel directly and never return. * On failure, we exit to the firmware via efi_exit instead of returning. */ asmlinkage unsigned long efi_main(efi_handle_t handle, @@ -914,7 +922,11 @@ asmlinkage unsigned long efi_main(efi_handle_t handle, goto fail; } - return bzimage_addr; + if (IS_ENABLED(CONFIG_X86_64)) + /* add offset of startup_64() */ + bzimage_addr += 0x200; + + enter_kernel(bzimage_addr, boot_params); fail: efi_err("efi_main() failed!\n"); From patchwork Mon May 22 07:13:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97144 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272904vqo; Mon, 22 May 2023 00:46:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7g9LvtobK6kSWGipfu6eQsLQla6T3Ev24luL+HdxRAFxlCIOdk3PVUjYl9orFkw0P38O1G X-Received: by 2002:a17:902:c94f:b0:1a9:8ab1:9f3b with SMTP id i15-20020a170902c94f00b001a98ab19f3bmr12428312pla.14.1684741574898; Mon, 22 May 2023 00:46:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741574; cv=none; d=google.com; s=arc-20160816; b=utjmXPfLs8lvEpXUdiw9zvdjUQDyA/1uIYz2ESZmEQReTuWdGH4hL7ia2QR13ZaDEc VA9PsTH5T0nmCxO8hjBGui0hLSXppJlGhSajoQiVAI4IpT3x8tLJlqAViQho+VdfvoTs G0P7GHH0P3Y9CG7egLKqIr2oJfU+0N1zJ2Wn/iUUxtptJnxMpzaBnDFVYSftr86UhqlY HoR9X+BkZhK3Lvia/W8pwD2r8sc91wSGe5gRZFEOozeYx3uktsI0X8Ejeka+VPCJQuSf dfvkdURxvil66cKObAWxYoNlPzGFjkzUQuMLsnAy4waYB62evWcor/kuxxSo4+1Nwna3 +oHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HHl6DaPAZuzuVBaUYnhJ30eQSA1WIWAumx+zdtOJIoE=; b=fXR9UD9NBnfRTtjts5Y9td1aTJhiUo5icpzO1Y9dt15PIyGIAe1jgUFV6klyOehZLX 4xdJ2vAPGMmaGFLvedAbF2uker0q6zIxfRElFYu3dL3d0KBSpBGyeQc3jAzwC3UqDvuE 9g8B/o9cRlfbBuypUzlCCpv+3GZXFeSdIaGne2ZVCFAQqR+5DpR2rxlQjPgbhECPPd4V w8OYmdTyhf1sQfjNyEpxdQljHE6ChoyyGIAG2nYeoRAFr58OoHh7qHXjiLagADQG1KtC k0ewkE2Lf8/4ymnLReRkC+zTBH4j2St5FYIgisOx2WDIsjBRM+ZLd2v5NHD/mni8GVzL 6uow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ZIZ8oOky; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 19-20020a630113000000b0051ba2ec8320si4273044pgb.353.2023.05.22.00.46.02; Mon, 22 May 2023 00:46:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ZIZ8oOky; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232424AbjEVHPa (ORCPT + 99 others); Mon, 22 May 2023 03:15:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232476AbjEVHOq (ORCPT ); Mon, 22 May 2023 03:14:46 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 604E91A4; Mon, 22 May 2023 00:14:43 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E6DD661DDD; Mon, 22 May 2023 07:14:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BDECC4339E; Mon, 22 May 2023 07:14:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739682; bh=6vIoJTudACrMHaUZpEmA9kKzuuAA0lBBUtfzLDgP/Vg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZIZ8oOkyrgL8CeF7w9z7a27wkdrj1AdQ7rNdEm+nMnPCxkpbMJp3dv9SKJSsVZHCe 9Dp9Y85TJ9fhl4i+6uP3Kyfiu1IroKotwfmWNqltXEcrZsHAaUa/rv/BP3gMW9BimW 2v+8PSMXuLm8bykkzdYluJunTzGZk9a1Z6MBkVqYnbQwrL6j4/epy/srH1AcwYjUnr SIQR8RbJ19LebnRxWhJz77XqdVCxmKI//AWIHhG1yZVGQqNe3prgs5jJC36PgB8oK8 M6MOoEZ1m826M4xN/jNc93INx1dqXb1EnGWY75XNHEj1Bq12ZMp3r5Dq2BCXzhWHto 0L/J/fHdWqUJg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 02/21] x86/efistub: Simplify and clean up handover entry code Date: Mon, 22 May 2023 09:13:56 +0200 Message-Id: <20230522071415.501717-3-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7508; i=ardb@kernel.org; h=from:subject; bh=6vIoJTudACrMHaUZpEmA9kKzuuAA0lBBUtfzLDgP/Vg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbTP9P8dlXrbdVNQ6tbFDZuGLnA+uQxpZzGrtOsQrMb T9Tu2xlRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZhIfQwjw8/E2zvPcc9R/crF 8UXA+rp34MHlgVdc755iU7r77OyNKUWMDHc97zc+c3Ga84015suXHx9XMa5zUNrm/Mzvyu9v/+T ZjzADAA== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579581569135123?= X-GMAIL-MSGID: =?utf-8?q?1766579581569135123?= Now that the EFI entry code in assembler is only used by the optional and deprecated EFI handover protocol, and given that the EFI stub C code no longer returns to it, most of it can simply be dropped. While at it, clarify the symbol naming, by merging efi_main() and efi_stub_entry(), making the latter the shared entry point for all different boot modes that enter via the EFI stub. The efi32_stub_entry() and efi64_stub_entry() names are referenced explicitly by the tooling that populates the setup header, so these must be retained, but can be emitted as aliases of efi_stub_entry() where appropriate. Signed-off-by: Ard Biesheuvel --- Documentation/arch/x86/boot.rst | 2 +- arch/x86/boot/compressed/efi_mixed.S | 22 +++++++++++--------- arch/x86/boot/compressed/head_32.S | 11 ---------- arch/x86/boot/compressed/head_64.S | 12 ++--------- drivers/firmware/efi/libstub/x86-stub.c | 20 ++++++++++++++---- 5 files changed, 31 insertions(+), 36 deletions(-) diff --git a/Documentation/arch/x86/boot.rst b/Documentation/arch/x86/boot.rst index 33520ecdb37abfda..cdbca15a4fc23833 100644 --- a/Documentation/arch/x86/boot.rst +++ b/Documentation/arch/x86/boot.rst @@ -1417,7 +1417,7 @@ execution context provided by the EFI firmware. The function prototype for the handover entry point looks like this:: - efi_main(void *handle, efi_system_table_t *table, struct boot_params *bp) + efi_stub_entry(void *handle, efi_system_table_t *table, struct boot_params *bp) 'handle' is the EFI image handle passed to the boot loader by the EFI firmware, 'table' is the EFI system table - these are the first two diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S index 4ca70bf93dc0bdcd..dcc562c8f7f35162 100644 --- a/arch/x86/boot/compressed/efi_mixed.S +++ b/arch/x86/boot/compressed/efi_mixed.S @@ -26,8 +26,8 @@ * When booting in 64-bit mode on 32-bit EFI firmware, startup_64_mixed_mode() * is the first thing that runs after switching to long mode. Depending on * whether the EFI handover protocol or the compat entry point was used to - * enter the kernel, it will either branch to the 64-bit EFI handover - * entrypoint at offset 0x390 in the image, or to the 64-bit EFI PE/COFF + * enter the kernel, it will either branch to the common 64-bit EFI stub + * entrypoint efi_stub_entry() directly, or via the 64-bit EFI PE/COFF * entrypoint efi_pe_entry(). In the former case, the bootloader must provide a * struct bootparams pointer as the third argument, so the presence of such a * pointer is used to disambiguate. @@ -37,21 +37,23 @@ * | efi32_pe_entry |---->| | | +-----------+--+ * +------------------+ | | +------+----------------+ | * | startup_32 |---->| startup_64_mixed_mode | | - * +------------------+ | | +------+----------------+ V - * | efi32_stub_entry |---->| | | +------------------+ - * +------------------+ +------------+ +---->| efi64_stub_entry | - * +-------------+----+ - * +------------+ +----------+ | - * | startup_64 |<----| efi_main |<--------------+ - * +------------+ +----------+ + * +------------------+ | | +------+----------------+ | + * | efi32_stub_entry |---->| | | | + * +------------------+ +------------+ | | + * V | + * +------------+ +----------------+ | + * | startup_64 |<----| efi_stub_entry |<--------+ + * +------------+ +----------------+ */ SYM_FUNC_START(startup_64_mixed_mode) lea efi32_boot_args(%rip), %rdx mov 0(%rdx), %edi mov 4(%rdx), %esi +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL mov 8(%rdx), %edx // saved bootparams pointer test %edx, %edx - jnz efi64_stub_entry + jnz efi_stub_entry +#endif /* * efi_pe_entry uses MS calling convention, which requires 32 bytes of * shadow space on the stack even if all arguments are passed in diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 987ae727cf9f0d04..8876ffe30e9a4819 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -150,17 +150,6 @@ SYM_FUNC_START(startup_32) jmp *%eax SYM_FUNC_END(startup_32) -#ifdef CONFIG_EFI_STUB -SYM_FUNC_START(efi32_stub_entry) - add $0x4, %esp - movl 8(%esp), %esi /* save boot_params pointer */ - call efi_main - /* efi_main returns the possibly relocated address of startup_32 */ - jmp *%eax -SYM_FUNC_END(efi32_stub_entry) -SYM_FUNC_ALIAS(efi_stub_entry, efi32_stub_entry) -#endif - .text SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 03c4328a88cbd5d0..71c1f40a7ac067b9 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -523,19 +523,11 @@ trampoline_return: jmp *%rax SYM_CODE_END(startup_64) -#ifdef CONFIG_EFI_STUB -#ifdef CONFIG_EFI_HANDOVER_PROTOCOL +#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL) .org 0x390 -#endif SYM_FUNC_START(efi64_stub_entry) - and $~0xf, %rsp /* realign the stack */ - movq %rdx, %rbx /* save boot_params pointer */ - call efi_main - movq %rbx,%rsi - leaq rva(startup_64)(%rax), %rax - jmp *%rax + jmp efi_stub_entry SYM_FUNC_END(efi64_stub_entry) -SYM_FUNC_ALIAS(efi_stub_entry, efi64_stub_entry) #endif .text diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 095aaa8b0ee30fb9..d6a376e52cbe1399 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -782,9 +782,9 @@ static void __noreturn enter_kernel(unsigned long kernel_addr, * On success, we jump to the relocated kernel directly and never return. * On failure, we exit to the firmware via efi_exit instead of returning. */ -asmlinkage unsigned long efi_main(efi_handle_t handle, - efi_system_table_t *sys_table_arg, - struct boot_params *boot_params) +void __noreturn efi_stub_entry(efi_handle_t handle, + efi_system_table_t *sys_table_arg, + struct boot_params *boot_params) { unsigned long bzimage_addr = (unsigned long)startup_32; unsigned long buffer_start, buffer_end; @@ -928,7 +928,19 @@ asmlinkage unsigned long efi_main(efi_handle_t handle, enter_kernel(bzimage_addr, boot_params); fail: - efi_err("efi_main() failed!\n"); + efi_err("efi_stub_entry() failed!\n"); efi_exit(handle, status); } + +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL +#ifndef CONFIG_EFI_MIXED +extern __alias(efi_stub_entry) +void efi32_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, + struct boot_params *boot_params); + +extern __alias(efi_stub_entry) +void efi64_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, + struct boot_params *boot_params); +#endif +#endif From patchwork Mon May 22 07:13:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97125 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1270680vqo; Mon, 22 May 2023 00:40:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6XtEC5XVUj7rzU+aoO1CwjYLk3fghpD5alI+qO1avQpmXHsen9ihbxO9yhtML9dYCmaJDs X-Received: by 2002:a05:6a20:3d15:b0:10c:1378:f2ae with SMTP id y21-20020a056a203d1500b0010c1378f2aemr249578pzi.46.1684741210668; Mon, 22 May 2023 00:40:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741210; cv=none; d=google.com; s=arc-20160816; b=aAyEDMta+NT4+LA9TR14T9YTgAFdsL36ip/1Ht8AK65TbXQH7jIH/0nUQO84apOmF+ NiFqON3Ya7tCIwPY5BctXc89aEiCKmcIudSbWxNAcCG8I6Tdn6zRALOKnwQZdGqHF2WZ PkUS4jTovTgl/uAOwLbYnDYRnqTeEqBqUpIYchKKbwqODAf9LFwjRuV7W36N/wC/mf2k PcFrPDQeBHDJQgIvUjP6pz7rTALjz+Sa/r6WMjUj8Ui/eo1HEJoG9x3vMN2AH4Eoyke0 yi3P8WNbFOa28nxBXlKvpOzgrKW9qC156TYMIkfdnZ5q9PQLbfzM7svGb5M7U+SJdd16 o6LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XJ255OBwItkgl/RrDZ4/XIccKLHanS1q8evPlqMyMy0=; b=sUk11Ki5STj9vAv6BTAaew3unuR7JbfDYPMHRfh146Za5ES+va8f0pWFjPqgXZUmS8 GlI9fknIBdbmmD+eQqjRrSHy9gv5KBIvOAcI4ZLiNKXbQcZP4CF7bGFX9rVGLbmJ+BEW SAZYWdhODRPqc0QXKyS9WsA6HdTaLWQ7uNTwDyipf/1EhuB1TQlZS1+4Pf8Jf71cMJl0 BCq1IQqvjZTubr5riO7SyekUr81DM2djzZzKPMEhGN/K8Z7MG3psyTD5QuoxMqdYIanz wOlC9x1rrqUQDxQd0SCaSQtxkVoZGdKCyigeNcSSPz0W6Qxe1de8ie/OulJDBHbZPFlE lvvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="Z/eg7yFJ"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u72-20020a63794b000000b005130f15c5b1si4436071pgc.363.2023.05.22.00.39.58; Mon, 22 May 2023 00:40:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="Z/eg7yFJ"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231961AbjEVHPL (ORCPT + 99 others); Mon, 22 May 2023 03:15:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232509AbjEVHOv (ORCPT ); Mon, 22 May 2023 03:14:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A109EB6; Mon, 22 May 2023 00:14:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 126D561DC7; Mon, 22 May 2023 07:14:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C30D7C4339C; Mon, 22 May 2023 07:14:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739686; bh=+pOeFG4I2M0NxWTqt2DlFeFWBXmc5I0TflwHlC5RHXA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z/eg7yFJCmf7fVb6NYCJcBPl5QfJKS98YfDdyW+b37Bjqr4u27h1NZswcE1+lpjtZ 98Q8gXYSnUv0T90Qg7a67k9AjBjASBXdjmLuEZXlLENvJ9dhKNwXEJqJGKFXiq5fdO bfNk6AWEEv8jkTmLgSkFfINHyiJaD9QpZ3WQ64V5cPga2DTxoMIItrxlDjrCG8kdB/ VojqEKIm64AAJQMR+bJXVjSIVXoPWeq/OryHTSFlScwPRB3rIZaMIf3L+pn5ALC1ee a5QJRsAAGhgv4dIaXf0qfKUrmlcHxxdi0yoeuJhJ7YUmRK5D9mmWaKKRVE1BX0Jzkl wl6eQ8NCMgJzA== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 03/21] x86/decompressor: Avoid magic offsets for EFI handover entrypoint Date: Mon, 22 May 2023 09:13:57 +0200 Message-Id: <20230522071415.501717-4-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3376; i=ardb@kernel.org; h=from:subject; bh=+pOeFG4I2M0NxWTqt2DlFeFWBXmc5I0TflwHlC5RHXA=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzPBNh4/k5gqHKBtjoxNfX9xeE5N2szXBxHFW0OH12 3TCd2h2lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgInwlTP8Yjp4qHd1YnzF+dYc n7YfoUfqrkVsTHGoEBbKf8ayLn23KSPDpchH6dy2bvxPMv8f1vp8QkxnXlL0ulWX2Re4Py9dF7a dDwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579199790544527?= X-GMAIL-MSGID: =?utf-8?q?1766579199790544527?= The native 32-bit or 64-bit EFI handover protocol entrypoint offset relative to the respective startup_32/64 address is described in boot_params as handover_offset, so that the special Linux/x86 aware EFI loader can find it there. When mixed mode is enabled, this single field has to describe this offset for both the 32-bit and 64-bit entrypoints, so their respective relative offsets have to be identical. Given that startup_32 and startup_64 are 0x200 bytes apart, and the EFI handover entrypoint resides at a fixed offset, the 32-bit and 64-bit versions of those entrypoints must be exactly 0x200 bytes apart as well. Currently, hard-coded fixed offsets are used to ensure this, but it is sufficient to emit the 64-bit entrypoint 0x200 bytes after the 32-bit one, wherever it happens to reside. This allows this code (which is now EFI mixed mode specific) to be moved into efi_mixed.S and out of the startup code in head_64.S. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/efi_mixed.S | 20 +++++++++++++++++++- arch/x86/boot/compressed/head_64.S | 18 ------------------ 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S index dcc562c8f7f35162..9308b595f6f0a5de 100644 --- a/arch/x86/boot/compressed/efi_mixed.S +++ b/arch/x86/boot/compressed/efi_mixed.S @@ -140,6 +140,16 @@ SYM_FUNC_START(__efi64_thunk) SYM_FUNC_END(__efi64_thunk) .code32 +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL +SYM_FUNC_START(efi32_stub_entry) + add $0x4, %esp /* Discard return address */ + popl %ecx + popl %edx + popl %esi + jmp efi32_entry +SYM_FUNC_END(efi32_stub_entry) +#endif + /* * EFI service pointer must be in %edi. * @@ -220,7 +230,7 @@ SYM_FUNC_END(efi_enter32) * stub may still exit and return to the firmware using the Exit() EFI boot * service.] */ -SYM_FUNC_START(efi32_entry) +SYM_FUNC_START_LOCAL(efi32_entry) call 1f 1: pop %ebx @@ -320,6 +330,14 @@ SYM_FUNC_START(efi32_pe_entry) RET SYM_FUNC_END(efi32_pe_entry) +#ifdef CONFIG_EFI_HANDOVER_PROTOCOL + .org efi32_stub_entry + 0x200 + .code64 +SYM_FUNC_START_NOALIGN(efi64_stub_entry) + jmp efi_stub_entry +SYM_FUNC_END(efi64_stub_entry) +#endif + .section ".rodata" /* EFI loaded image protocol GUID */ .balign 4 diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 71c1f40a7ac067b9..9f90661744741210 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -294,17 +294,6 @@ SYM_FUNC_START(startup_32) lret SYM_FUNC_END(startup_32) -#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL) - .org 0x190 -SYM_FUNC_START(efi32_stub_entry) - add $0x4, %esp /* Discard return address */ - popl %ecx - popl %edx - popl %esi - jmp efi32_entry -SYM_FUNC_END(efi32_stub_entry) -#endif - .code64 .org 0x200 SYM_CODE_START(startup_64) @@ -523,13 +512,6 @@ trampoline_return: jmp *%rax SYM_CODE_END(startup_64) -#if IS_ENABLED(CONFIG_EFI_MIXED) && IS_ENABLED(CONFIG_EFI_HANDOVER_PROTOCOL) - .org 0x390 -SYM_FUNC_START(efi64_stub_entry) - jmp efi_stub_entry -SYM_FUNC_END(efi64_stub_entry) -#endif - .text SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) From patchwork Mon May 22 07:13:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97142 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272867vqo; Mon, 22 May 2023 00:46:10 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4CnojlPgSsXhxUrA6Ol24SaWglWirknVzOOO/ApHcecHprweAHuof25+P3EVbPYkgfeOM3 X-Received: by 2002:a17:902:e80c:b0:1ad:f407:37d2 with SMTP id u12-20020a170902e80c00b001adf40737d2mr12536803plg.5.1684741570583; Mon, 22 May 2023 00:46:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741570; cv=none; d=google.com; s=arc-20160816; b=lTHlL1hwj3slqBJif5jXMFvn0VzBw/VeSH9fvAwkZ8Q01kvEpYKfYRTUDTBoorhKrP ba209q1Iwnf80vmSe4sCauAf+N09JGqiNTi1sNVu3PYnqiLwE1aVpPaIr6c3iAkMoami Tvt1cSlSaBiw6zAPUfDwX6aS77UbFFgwQhL203bO0sK41oTguc+LAhujPWxR1zBIycaT D9/WWDO9KhneVhIvxj/3BiyVtt3nOTW5X+HMtewv/9aFCBIEoKPEVQVt2ogwelvo5XFc uDQKvl0HxF4YQA7Rpt7LORhOMwiHsgcMCsDFNbacuGcTA2nPpjNDUuR5yYU0u03AQP0d Cy3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KnbIczG5VJn+Jh7FPVtSUai82qqjNe0euxAX6s+PaBo=; b=tCfrLiodLLP7tn3YHnwngbXIFcuPhJOjp0i7ZMcQzRdZ8a8yfrc9k9uszu+SDYZKlR 4BK4BiH5DTrUNukTaZBFSN4l4PDSXUYx0I0ZQKCqDi1Ou80NkgDnaz2IrB4ZQq8tDzLy dWcd/RW0w9VMWWWq/Yhq8kVSA/TdAE5E9gAzVJEA82XM+XYTqJuowOMGzV5KK22jPbtv fRc58ST9N2CGlc7rJ4hjw7XIxOzYpGJ4SfhktF8wiMISTjQ+Jt7DOOOoQODj9qKnliHs c2q/zJbkdiGzBRuQL5lxKX+Kij3Ww4g6GA39zo+b20UOD6HKj3KYJXaOup5xfyYRQKZU oaYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=otLwxEON; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w189-20020a6382c6000000b00530b7eca08dsi4654537pgd.51.2023.05.22.00.45.55; Mon, 22 May 2023 00:46:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=otLwxEON; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231675AbjEVHPH (ORCPT + 99 others); Mon, 22 May 2023 03:15:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232523AbjEVHOx (ORCPT ); Mon, 22 May 2023 03:14:53 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A882594; Mon, 22 May 2023 00:14:51 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3BF3961DC4; Mon, 22 May 2023 07:14:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EA4DBC4339E; Mon, 22 May 2023 07:14:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739690; bh=GI53KZcq3ZI3RxFd9aG0XytLXRdQ1EhHXwVomNHohgI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=otLwxEON9iJ+ZI6BCN9YAbKtZnBDHq6IdNPb/4XDqFwgee1+vXR5i5ac88qs0acoQ G67JEC/bGL1GStjgIDfG9HAmyx/PR1byEARlw3pScOH7bG8LkUxtPAtzLpTHfM+qWP Y26SWLmoKv2OwL7OrRwGQZELJUsfsCyHSTS6tX1nndOOdQCDfJ8DQWQo4kND9djWVG lWWWGEHVsvp8cRhPBspQpkgCys9yZyJRXJcA3Xqs3KzmNwViA/BXoZAH5kvrvQtWEz Y2Yf1PkKKKMebsw9TzPo+SEJrokn+xLLvF9LhcKe6zeq1EOXk8HhnsXVHxlM5MIWC+ 0jUyU+BU7vyjA== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 04/21] x86/efistub: Clear BSS in EFI handover protocol entrypoint Date: Mon, 22 May 2023 09:13:58 +0200 Message-Id: <20230522071415.501717-5-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2971; i=ardb@kernel.org; h=from:subject; bh=GI53KZcq3ZI3RxFd9aG0XytLXRdQ1EhHXwVomNHohgI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzCh+k2N2zVP21WnK71riE97Oi5L3ShPeM9esfPuSx i2fDRM7SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwET2GTIyXFu947S07Bq/pJS9 RRvOWewM0mj2ynzyvyB8ooh2R2fKX4b/LlJ2N3KZDJw8xXJT7Go7di1Kfdu/efIK3oNZKcf57H7 wAAA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579576765422644?= X-GMAIL-MSGID: =?utf-8?q?1766579576765422644?= The so-called EFI handover protocol is value-add from the distros that permits a loader to simply copy a PE kernel image into memory and call an alternative entrypoint that is described by an embedded boot_params structure. Most implementations of this protocol do not bother to check the PE header for minimum alignment, section placement, etc, and therefore also don't clear the image's BSS, or even allocate enough memory for it. Allocating more memory on the fly is rather difficult, but at least clear the BSS region explicitly when entering in this manner, so that the EFI stub code does not get confused by global variables that were not zero-initialized correctly. When booting in mixed mode, this BSS clearing must occur before any global state is created, so clear it in the 32-bit asm entry point. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/efi_mixed.S | 14 +++++++++++++- drivers/firmware/efi/libstub/x86-stub.c | 13 +++++++++++-- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S index 9308b595f6f0a5de..8a02a151806df14c 100644 --- a/arch/x86/boot/compressed/efi_mixed.S +++ b/arch/x86/boot/compressed/efi_mixed.S @@ -142,6 +142,18 @@ SYM_FUNC_END(__efi64_thunk) .code32 #ifdef CONFIG_EFI_HANDOVER_PROTOCOL SYM_FUNC_START(efi32_stub_entry) + call 1f +1: popl %ecx + + /* Clear BSS */ + xorl %eax, %eax + leal (_bss - 1b)(%ecx), %edi + leal (_ebss - 1b)(%ecx), %ecx + subl %edi, %ecx + shrl $2, %ecx + cld + rep stosl + add $0x4, %esp /* Discard return address */ popl %ecx popl %edx @@ -334,7 +346,7 @@ SYM_FUNC_END(efi32_pe_entry) .org efi32_stub_entry + 0x200 .code64 SYM_FUNC_START_NOALIGN(efi64_stub_entry) - jmp efi_stub_entry + jmp efi_handover_entry SYM_FUNC_END(efi64_stub_entry) #endif diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index d6a376e52cbe1399..d010448dffb12cb8 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -934,12 +934,21 @@ void __noreturn efi_stub_entry(efi_handle_t handle, } #ifdef CONFIG_EFI_HANDOVER_PROTOCOL +void efi_handover_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, + struct boot_params *boot_params) +{ + extern char _bss[], _ebss[]; + + memset(_bss, 0, _ebss - _bss); + efi_stub_entry(handle, sys_table_arg, boot_params); +} + #ifndef CONFIG_EFI_MIXED -extern __alias(efi_stub_entry) +extern __alias(efi_handover_entry) void efi32_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, struct boot_params *boot_params); -extern __alias(efi_stub_entry) +extern __alias(efi_handover_entry) void efi64_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, struct boot_params *boot_params); #endif From patchwork Mon May 22 07:13:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97119 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1267868vqo; Mon, 22 May 2023 00:32:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6Qk+2KFhEx+VO0uDch+DvsgX4aKPAz7cR0FdAzFssYIOLIiQXKEKapGt/D8pYS+Lb9edI1 X-Received: by 2002:a05:6a21:328b:b0:10b:8024:d253 with SMTP id yt11-20020a056a21328b00b0010b8024d253mr3104316pzb.26.1684740747119; Mon, 22 May 2023 00:32:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684740747; cv=none; d=google.com; s=arc-20160816; b=iUjUvzGfnqJO++PXpK0wERKo/nl4H44ZX7majKcc1TIIDEO2HiORObRLgDS9YzuMo+ nIVcru3tP8ockXWNkuyLOA40tE+WNn6PveKjH55XAtGumKVT4Uvej9vzlnEcdcVyzNSj 52nZg0EMevslpwbxwOq+HPH0jkRdCwq3BBG2hvBsFa0PB5MXeGzcQsCUi0e5jbuuBD75 UDVJPKM1a77iH0NuUopwVyWv7yvNQ73hxRfB/M5A26JBdhwfN7IBfeBuooxOjIP6oM1i Dnmb91RRp4pfA97qCQVNBqvZxLkNPM2khAPSQVl+MCiSJes5Wl1IyOe2UnlnObDumkNS RNUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=S3lEqGjsTGw1NWcFmbt4O+4A6ddMT7nl/cAxjp7Py8E=; b=WcXHTTAVZEExq7UkI0vVsQ8OD2MayItFsq+vMmwPTb7/s5qdLfT9jqbPA2NCVcLTD5 JV6te6FH6F4XPiGZTdGBRBTSycmb9Sn8Ei2MU9vf3p0Cr3zgwvf91AEaj1GdNpS9QHIS UX4GrHqw4lnuhrgtl+RzC1XULyE/W9qSavZRn0H0OOZHSMUS0YsqZepvOVWIJcNdsj6O NVgTiHGfB8nONqMu25SeE0qz2y7ZCGvx/TOlykfI3FrW4GkjQxdPGb1ZDVWD0+lOigTG MxdeuTAmceEm/rxYtvzECaLRJIlDXbrcPGkulqn3yB4DSWZyZUxt3Cdn8+iWzgnEbQFP 8i4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ahdZuLYw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 73-20020a63014c000000b0051b33c863ffsi1749389pgb.399.2023.05.22.00.32.13; Mon, 22 May 2023 00:32:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ahdZuLYw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232346AbjEVHPV (ORCPT + 99 others); Mon, 22 May 2023 03:15:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232537AbjEVHO4 (ORCPT ); Mon, 22 May 2023 03:14:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7113BA; Mon, 22 May 2023 00:14:55 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 61DBF61DDD; Mon, 22 May 2023 07:14:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C3BFC4339C; Mon, 22 May 2023 07:14:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739694; bh=ygI81p6SqewQoIn6tCgoZLUH2Qo5tPVgo3BNvwGSS3U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ahdZuLYw1TmYdmQL7lxGHEBiiBSANf79+lkAs61O4z58clTFkmUWH58Fsbdo33R3z f1fMye2kQbMk8gjq+efFzOBdtG5qIi9Ea8qvj08KORTxU9WKJePL9KJleO948dlGdd 1Q2JtizxxQVY0RkrEFWjyvwxepYUgaEaWVEH5w+BwdLHBpLNBkCSAGQ3YOS9BBBXND t1aep4slelOSfi3wf6U0TVJzig67BQfHew9FDQm0MMUJENIwTpNCZq9ILKsv1tu+7O bmRmAg52hOVvDqquTz9yxja6QwMK8OKV/WL0VdvLNQOEFNTWlhyGyXccFgxOoALZpG IlAcnhCBhUjBg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 05/21] x86/decompressor: Use proper sequence to take the address of the GOT Date: Mon, 22 May 2023 09:13:59 +0200 Message-Id: <20230522071415.501717-6-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1770; i=ardb@kernel.org; h=from:subject; bh=ygI81p6SqewQoIn6tCgoZLUH2Qo5tPVgo3BNvwGSS3U=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzHhfjGz12WInw7ydqfPDJ7Ukb5i9b7FGVn2Z7oyqH cdfVGp0lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIm0KTIyrF17+9OtoPo3ZXV+ 0yM+ap06MX1Rjqd/vtzOzMsersb9IYwMX2Pv6k44LDOnZ8bi5ZeDUs/EP/19p29RxDaZCRYi8/q LmAE= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766578713726066835?= X-GMAIL-MSGID: =?utf-8?q?1766578713726066835?= The 32-bit decompressor does not actually use a global offset table (GOT), but as is common for 32-bit position independent code, it uses the magic symbol _GLOBAL_OFFSET_TABLE_ as an anchor from which to derive the actual runtime addresses of other symbols, using special @GOTOFF symbol references that are resolved at link time, and populated with the distance between the address of the magic _GLOBAL_OFFSET_TABLE_ anchor and the address of the symbol in question. This means _GLOBAL_OFFSET_TABLE_ is the only symbol whose actual runtime address needs to be determined explicitly, which is one of the first things that happens in startup_32. However, it does so by taking the absolute address via the immediate field of an ADD instruction (plus a small offset), which seems to defeat the point. Fortunately, the assembler knows that _GLOBAL_OFFSET_TABLE_ is magic, and emits a special relative relocation instead, and so the resulting code works as expected. However, this is not obvious for someone reading the code, and the use of LEA with an explicit relative addend is more idiomatic so use that instead. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_32.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 8876ffe30e9a4819..3530465b5b85ccf3 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -58,7 +58,7 @@ SYM_FUNC_START(startup_32) leal (BP_scratch+4)(%esi), %esp call 1f 1: popl %edx - addl $_GLOBAL_OFFSET_TABLE_+(.-1b), %edx + leal (_GLOBAL_OFFSET_TABLE_ - 1b)(%edx), %edx /* Load new GDT */ leal gdt@GOTOFF(%edx), %eax From patchwork Mon May 22 07:14:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97109 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1262203vqo; Mon, 22 May 2023 00:17:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ54Lsx24O24g6zVJX3vCC8CFX9NlnkV2OC1f9S/l2jixDwP/CFGaojg/vcUgTJsz9aELqvO X-Received: by 2002:a05:6a00:2288:b0:643:aa8d:8cd7 with SMTP id f8-20020a056a00228800b00643aa8d8cd7mr15041345pfe.32.1684739875263; Mon, 22 May 2023 00:17:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684739875; cv=none; d=google.com; s=arc-20160816; b=xcani1YxqsCKLcsxhwrDF/44kbu4EA3DpXVHC5k42OOACXRLIYapTs22S8bIFm/Zvx VT/C78iDnGrdFnky5NHXwU4IoxvNUltgoLgyglvssQdCjsNJQ5dlwQuVGr69X3/f7ZNK He3kQyP+u1QVBPRjs9SI5yf3BQJgb12A+2Lw4WBQu+sInozvLteyUK4+cosQcXGxDGGN CEqmpOocyyNnOMXP9jKteQzPUunhWMX/sI0gNV8OSak0vP0fdZn3nd8fQ4Nt0fqlK/ZT YB4W+65ztx4frGYKlbKQHnTsk9Q9ssCb8+henEi7Ck+Lzv3ejDLJBsk3Q/v9KlkJMXyv 8V9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9LNa+G2wrQIjRvt2UCstrzgBbRTUvf6M5+JO2w9PXAw=; b=r+dmpDJu6UXravLIjJZepanfA2CA/u4g5xfCltPwz3CexKLlsma9/mAGV6SSVyehZ6 +CI9Vx2/qjoYZ3z0dLInS5bsqpRq9y8qrB0g0ZHagVdwf6tgJ8O14K/hgEIPpfnhu8XA PKmiO/75aqMplDhDr/ctZe0izMeY1U6NVhwlrnMV7UM1IjN+2J18h84hkEAVUj9uqtJ0 LgG3tySdRjaQkOVI9THKNKE5CdG1Py6g5MWdALbYuxe5Gr++OwNJ8ELINVB27jqwwtB9 BqcRKuy9pGFssbHAx+qy4dV8yDwvKhQRiPRqbF84y8GrSjtqFZSAabouuxUDSEFcS72A thmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=a7KdTq8P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e28-20020aa798dc000000b0064672de37cesi4416666pfm.192.2023.05.22.00.17.41; Mon, 22 May 2023 00:17:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=a7KdTq8P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232303AbjEVHPQ (ORCPT + 99 others); Mon, 22 May 2023 03:15:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232545AbjEVHPA (ORCPT ); Mon, 22 May 2023 03:15:00 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF9ECA1; Mon, 22 May 2023 00:14:59 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8671261DD4; Mon, 22 May 2023 07:14:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41F97C433D2; Mon, 22 May 2023 07:14:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739699; bh=XH7auhXZovyO0ZjhIGuK6rXf321+XRWDmJGikLNf51o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a7KdTq8PPZcLUMpaFnJwt9owjwLIw/qIfHVEF5sSQogAuB/WRsIxrAn774VuPIUJx N+FSlb/Akd4n2Us5eUEVyTfgY8jJChhAa7cRZWByVDu8c6ITJzdfi43CIiuM6/M1cs m00dGG46N2zKKp8HsTzugxFheiOff5FDccKpsUy2aqnG5Q1eoc5LSPnek94UTWhMmp 9VKR9uoZd59smBJfAndQiWoU5VfTupBjJ0IL6/buDPYmBOkayLh3NNaRtJr0/yj/ON FK1Z9YPDHdQjueJlsRk4sP9wHGA/L/SbfXFGUZ62SJoZsCvacM8D5SsQJbADTMeWeu cx6Xd2E5oTOAw== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 06/21] x86/decompressor: Store boot_params pointer in callee save register Date: Mon, 22 May 2023 09:14:00 +0200 Message-Id: <20230522071415.501717-7-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3619; i=ardb@kernel.org; h=from:subject; bh=XH7auhXZovyO0ZjhIGuK6rXf321+XRWDmJGikLNf51o=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzGRRpJHd1RnWG0UnzPsfEPSq6/POZO+LSS9tdY8IO KjNVzfuKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPZns3wzzbrXmbQ4X5Z9d5l m5nnXFj09m1EGWfxTRVXHwEJ9ax57owMPTt+Bh8LX35oLePmrOvs7veDBQ5ubGK1rRRLPtwsIbu FBwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766577799301205995?= X-GMAIL-MSGID: =?utf-8?q?1766577799301205995?= Instead of pushing and popping %RSI several times to preserve the struct boot_params pointer across the execution of the startup code, move it into a callee save register before the first call into C, and copy it back when needed. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 34 +++++++------------- 1 file changed, 11 insertions(+), 23 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 9f90661744741210..2d1b0ee94929f7ec 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -405,10 +405,14 @@ SYM_CODE_START(startup_64) lretq .Lon_kernel_cs: + /* + * RSI holds a pointer to a boot_params structure provided by the + * loader, and this needs to be preserved across C function calls. So + * move it into a callee saved register. + */ + movq %rsi, %r15 - pushq %rsi call load_stage1_idt - popq %rsi #ifdef CONFIG_AMD_MEM_ENCRYPT /* @@ -421,10 +425,8 @@ SYM_CODE_START(startup_64) * detection/setup to ensure that has been done in advance of any dependent * code. */ - pushq %rsi - movq %rsi, %rdi /* real mode address */ + movq %r15, %rdi /* pass struct boot_params pointer */ call sev_enable - popq %rsi #endif /* @@ -437,13 +439,9 @@ SYM_CODE_START(startup_64) * - Non zero RDX means trampoline needs to enable 5-level * paging. * - * RSI holds real mode data and needs to be preserved across - * this function call. */ - pushq %rsi - movq %rsi, %rdi /* real mode address */ + movq %r15, %rdi /* pass struct boot_params pointer */ call paging_prepare - popq %rsi /* Save the trampoline address in RCX */ movq %rax, %rcx @@ -468,14 +466,9 @@ trampoline_return: * * RDI is address of the page table to use instead of page table * in trampoline memory (if required). - * - * RSI holds real mode data and needs to be preserved across - * this function call. */ - pushq %rsi leaq rva(top_pgtable)(%rbx), %rdi call cleanup_trampoline - popq %rsi /* Zero EFLAGS */ pushq $0 @@ -485,7 +478,6 @@ trampoline_return: * Copy the compressed kernel to the end of our buffer * where decompression in place becomes safe. */ - pushq %rsi leaq (_bss-8)(%rip), %rsi leaq rva(_bss-8)(%rbx), %rdi movl $(_bss - startup_32), %ecx @@ -493,7 +485,6 @@ trampoline_return: std rep movsq cld - popq %rsi /* * The GDT may get overwritten either during the copy we just did or @@ -525,30 +516,27 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) shrq $3, %rcx rep stosq - pushq %rsi call load_stage2_idt /* Pass boot_params to initialize_identity_maps() */ - movq (%rsp), %rdi + movq %r15, %rdi /* pass struct boot_params pointer */ call initialize_identity_maps - popq %rsi /* * Do the extraction, and jump to the new kernel.. */ - pushq %rsi /* Save the real mode argument */ - movq %rsi, %rdi /* real mode address */ + movq %r15, %rdi /* pass struct boot_params pointer */ leaq boot_heap(%rip), %rsi /* malloc area for uncompression */ leaq input_data(%rip), %rdx /* input_data */ movl input_len(%rip), %ecx /* input_len */ movq %rbp, %r8 /* output target address */ movl output_len(%rip), %r9d /* decompressed length, end of relocs */ call extract_kernel /* returns kernel entry point in %rax */ - popq %rsi /* * Jump to the decompressed kernel. */ + movq %r15, %rsi jmp *%rax SYM_FUNC_END(.Lrelocated) From patchwork Mon May 22 07:14:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97124 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1269733vqo; Mon, 22 May 2023 00:37:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5T4zRBEVDYBb9v5IYT9n60QLGBNcrOgxCpKndu5bowclvp05LIvxp08Nw/WQUToCdBPRGp X-Received: by 2002:a17:903:185:b0:1af:b682:7a78 with SMTP id z5-20020a170903018500b001afb6827a78mr2473932plg.52.1684741047217; Mon, 22 May 2023 00:37:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741047; cv=none; d=google.com; s=arc-20160816; b=bm2kTu9JA4TB0XMXPkFzz1pykLkONNB+EdUp/+d3Cg1wwk8RKdojTmUogTtfUfQpAe gEbzuR5cqVAu969A37Mx6wdpHQ+UXneVDGd/IKN9qsMAB9TXUaOfBKSqjn9eSjY9oU1z RiytkhBXjuSIQsAWhIroO4Y2yn5n0Z6Cwx8DLkvctcoCbITaemM4EH9DkDaOK3LQpWdm LwAj6FPSGAoIiVN80bkS5rGGack2ySa8NC01rQJmKIiL2mXCTui7MKok/bxnCUGAWaCd B3mkltE2g3Zy7N6UaVH3+XJZLY4FAn3kNOkG7HWA8fLp8T5u/ZthPYrM0WZA1mruQdfn JEHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=sQZhXDdW/Pt1TLOgC10hAFzrGwzWSSI2c8Hy68wku88=; b=EfuLQuTPKmZYqLuihBUrKsdiKEAqXkPb81eFaTotKxf3NBJao8/+dUeProP4XgKgL/ 95+BQ6Ds1A/OF2U5QJQhHNNHe4eDQLv8G3nAl5SytMyb3u68J3MNhpygkyxOmd1EgilC BETPviogxgG8mC/irwehdL9Vwmk0lE77NK1jknW41A/JFb/0HNXFtlIKlcvozKCK5aqn 4Mj3wJfPEi81h0uNigR/8BDSARhghwZdfJBNHRjpGKJ0czi9K/bqbnlOtD8TrXSZsgdv pu3clMJum+PMq5WnLDj5qqUqz8MOnmyc6ck9rvH7jPnlpYH2wed0nNA7nH+WmtsM/Mmp lrTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=B8PFMI8j; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y2-20020a17090264c200b001a51c26f601si4055913pli.627.2023.05.22.00.37.13; Mon, 22 May 2023 00:37:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=B8PFMI8j; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231135AbjEVHPq (ORCPT + 99 others); Mon, 22 May 2023 03:15:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229987AbjEVHPF (ORCPT ); Mon, 22 May 2023 03:15:05 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 204D791; Mon, 22 May 2023 00:15:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AE44761DD3; Mon, 22 May 2023 07:15:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 670A6C4339B; Mon, 22 May 2023 07:14:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739703; bh=4Bh3UcI6mBb60v5tgLvoAeGffUld7BrhFCXiRmmP3sE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B8PFMI8j/D51LvwF865+aX+iv+KDHkvt8iC4EiQ0I5a5plkeEOGknXXGhlLSVCdBk VejQ45+yCQrtU4u/hGOoy1OuEhq0NmMSI0i8QdnnFfkAz8UNd9WpBLADjT12WaVSvd RRlX1XYUOOVEafN9pN1wTvSXiJOJp/7tfJUmHb9pIiGrOGSufq1ANncvuh8eubQiQP 08jLV5QPB+HNvBkF3CEIBrmDfEqhFjKun10kvVrQ94fWodKSsFzt0sdh8Ksp2p6uzl uOLwsHkHT44yvjrvzsjDZQ4J5TiXTfhbORKXoRieGKBGmJtBrf4tRGlejT/W1VCGap u1DFoz+N1jZiA== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 07/21] x86/decompressor: Call trampoline as a normal function Date: Mon, 22 May 2023 09:14:01 +0200 Message-Id: <20230522071415.501717-8-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2554; i=ardb@kernel.org; h=from:subject; bh=4Bh3UcI6mBb60v5tgLvoAeGffUld7BrhFCXiRmmP3sE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzPTyXcGvbvdk7s59zjTFLSjIQEnfS8e2IGX29+utW 1ncSkI6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwEQWPmX4X/zz6beN5SwCHx4L CT6Of3pzfUzP73c+guvUTitru/p8LWD4H64gNdtP63XgnDXiXh8fGXM4Lv4sUl68o3Hh5Tg7x91 xLAA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579027889199516?= X-GMAIL-MSGID: =?utf-8?q?1766579027889199516?= Move the long return to switch to 32-bit mode into the trampoline code so it can be called as an ordinary function. This will allow it to be called directly from C code in a subsequent patch. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 25 +++++++++----------- arch/x86/boot/compressed/pgtable.h | 2 +- 2 files changed, 12 insertions(+), 15 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 2d1b0ee94929f7ec..af45ddd8297a4a07 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -446,18 +446,9 @@ SYM_CODE_START(startup_64) /* Save the trampoline address in RCX */ movq %rax, %rcx - /* - * Load the address of trampoline_return() into RDI. - * It will be used by the trampoline to return to the main code. - */ - leaq trampoline_return(%rip), %rdi - - /* Switch to compatibility mode (CS.L = 0 CS.D = 1) via far return */ - pushq $__KERNEL32_CS leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax - pushq %rax - lretq -trampoline_return: + call *%rax + /* Restore the stack, the 32-bit trampoline uses its own stack */ leaq rva(boot_stack_end)(%rbx), %rsp @@ -540,16 +531,22 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) jmp *%rax SYM_FUNC_END(.Lrelocated) - .code32 /* * This is the 32-bit trampoline that will be copied over to low memory. * - * RDI contains the return address (might be above 4G). * ECX contains the base address of the trampoline memory. * Non zero RDX means trampoline needs to enable 5-level paging. */ SYM_CODE_START(trampoline_32bit_src) - /* Set up data and stack segments */ + popq %rdi + /* Switch to compatibility mode (CS.L = 0 CS.D = 1) via far return */ + pushq $__KERNEL32_CS + leaq 0f(%rip), %rax + pushq %rax + lretq + + .code32 +0: /* Set up data and stack segments */ movl $__KERNEL_DS, %eax movl %eax, %ds movl %eax, %ss diff --git a/arch/x86/boot/compressed/pgtable.h b/arch/x86/boot/compressed/pgtable.h index cc9b2529a08634b4..91dbb99203fbce2d 100644 --- a/arch/x86/boot/compressed/pgtable.h +++ b/arch/x86/boot/compressed/pgtable.h @@ -6,7 +6,7 @@ #define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0 #define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0x80 +#define TRAMPOLINE_32BIT_CODE_SIZE 0xA0 #define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZE From patchwork Mon May 22 07:14:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97110 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1262459vqo; Mon, 22 May 2023 00:18:39 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7lLpRgLOszILG9PGwF2LHT85louEa7mukm1Ruy1jjZVSNMmUvR8Da44UlzsJTecbYNo0CU X-Received: by 2002:a05:6a00:1a15:b0:643:b4d5:e3db with SMTP id g21-20020a056a001a1500b00643b4d5e3dbmr13607556pfv.4.1684739919213; Mon, 22 May 2023 00:18:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684739919; cv=none; d=google.com; s=arc-20160816; b=KMwnNJMppu2ErrRCy+tEer5LW+61xB+GVM7DcT117uLMQKVxiAy9hyqU1itd57tGgb pwX0rh6K8R6YbOZo5ospJFrtD3qqsCaU7dqFHWpYGLhDpiBw/DIVFjYQ7UGuK28QO5m2 F778KRhpaoJp80MzXTDkE2IoivMiyKbn6R2LlS5fINBH2HAghPDTzdS1p7AkM/R8d8Sp koKNNvTYCiOxMyDWiK/lGZucMZEKeSUnuvd4UWEkqQ6AALI/GL1Z/OFDm58mYV/W64rX 8K6VlAQFTJQM5vja1uneuOCzthjgohKx+5wU72jr2fhYohQr1gFfw1mP/roxc1vErN1N JBJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=owkHrfGk6rkCsLR4jaeO3WUYAvrwcw7T/GSFdPTyMwM=; b=ylr9FedMFO6khLEBlpyM84aaxehV8lei+6u8YcKXb7iGfEbAzBEp4FMBNoGvSHcEbg L2JNy1JIzw/yl8I9DOZLMyeJ83O/Qln7MBZcsmF37ifg32j10UU5l9OdgHQwOarYEgld 51TQmqQJJLxBbAcnmUBQS2Yu8gismfe/N6sfl1idmuZHT4sTyvmV9aGDMiuHVTtxOI6Q MKTqXqG078McyqonhDvUTGc9iQIRFl0fYV322cPDnhRNSTIJVWKK27mnrxvf983hEBkk PxBgL4G8sOGc+vx2kYGJBF8ebMM5B+LEoOp5eWpI6Y64381BIJCR4SKz9wr2qQGarCtk Ujjw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=q3m4hzTg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e28-20020aa798dc000000b0064672de37cesi4416666pfm.192.2023.05.22.00.18.25; Mon, 22 May 2023 00:18:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=q3m4hzTg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232266AbjEVHPv (ORCPT + 99 others); Mon, 22 May 2023 03:15:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232055AbjEVHPJ (ORCPT ); Mon, 22 May 2023 03:15:09 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D9D1C1; Mon, 22 May 2023 00:15:08 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D5FE961DD5; Mon, 22 May 2023 07:15:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DA42C4339E; Mon, 22 May 2023 07:15:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739707; bh=6LI2p3tMGpqgjf5Vh98U5/dlrPhBIFsp0XryDg0+x/4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q3m4hzTgh/bSksj7igaVybxKUaNE+hjOnVev/TCYk+HHZ507owq2NKfHGRoiNZK/l QfbcI9Ma3XpBQLOZHtNHyie3dGlmatDuZqAaMRDlF0wWDcyp6ow2m5eYSZMJEI2FBP yq/J57wA294JWZU6JyPyQnDgmsAj0rZvNF8n2+Kp0nvXreFuw2m1OxmPCXLMaWNqre kpeXldvteNljTO/ta8iKS1WeF5xUiFnDU7ga9Jn3JkeK6Xs1yb6zH+MUBa+dZiqoEz doP6jLyjVHoKSYYYRnJHYKKux5wrU+2cNWfesPqXEBuu5wjounLFCubCVTo1RzHrmt S71izQlFfOGxQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 08/21] x86/decompressor: Use standard calling convention for trampoline Date: Mon, 22 May 2023 09:14:02 +0200 Message-Id: <20230522071415.501717-9-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4143; i=ardb@kernel.org; h=from:subject; bh=6LI2p3tMGpqgjf5Vh98U5/dlrPhBIFsp0XryDg0+x/4=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzHxl6kWJ8zVOTq1sXadOmfbaxrX+bM1U2TXTp8ROI vgNn21HKQuDGAeDrJgii8Dsv+92np4oVes8SxZmDisTyBAGLk4BmEiPLyNDuw+Dff7Guc1PFHnn RRze9nLSf8nwF6Ifb13/ELn4cq4oPyPDiqVFEuUPjR8JiKZ07vJn71Z+12W017XkTZQr+75VlUc YAQ== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766577845168966788?= X-GMAIL-MSGID: =?utf-8?q?1766577845168966788?= Update the trampoline code so its arguments are passed via RDI and RSI, which matches the ordinary SysV calling convention for x86_64. This will allow this code to be called directly from C. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 30 +++++++++----------- arch/x86/boot/compressed/pgtable.h | 2 +- 2 files changed, 14 insertions(+), 18 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index af45ddd8297a4a07..a387cd80964e1a1e 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -443,9 +443,9 @@ SYM_CODE_START(startup_64) movq %r15, %rdi /* pass struct boot_params pointer */ call paging_prepare - /* Save the trampoline address in RCX */ - movq %rax, %rcx - + /* Pass the trampoline address and boolean flag as args #1 and #2 */ + movq %rax, %rdi + movq %rdx, %rsi leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax call *%rax @@ -534,11 +534,11 @@ SYM_FUNC_END(.Lrelocated) /* * This is the 32-bit trampoline that will be copied over to low memory. * - * ECX contains the base address of the trampoline memory. - * Non zero RDX means trampoline needs to enable 5-level paging. + * EDI contains the base address of the trampoline memory. + * Non-zero ESI means trampoline needs to enable 5-level paging. */ SYM_CODE_START(trampoline_32bit_src) - popq %rdi + popq %r8 /* Switch to compatibility mode (CS.L = 0 CS.D = 1) via far return */ pushq $__KERNEL32_CS leaq 0f(%rip), %rax @@ -552,7 +552,7 @@ SYM_CODE_START(trampoline_32bit_src) movl %eax, %ss /* Set up new stack */ - leal TRAMPOLINE_32BIT_STACK_END(%ecx), %esp + leal TRAMPOLINE_32BIT_STACK_END(%edi), %esp /* Disable paging */ movl %cr0, %eax @@ -560,7 +560,7 @@ SYM_CODE_START(trampoline_32bit_src) movl %eax, %cr0 /* Check what paging mode we want to be in after the trampoline */ - testl %edx, %edx + testl %esi, %esi jz 1f /* We want 5-level paging: don't touch CR3 if it already points to 5-level page tables */ @@ -575,21 +575,17 @@ SYM_CODE_START(trampoline_32bit_src) jz 3f 2: /* Point CR3 to the trampoline's new top level page table */ - leal TRAMPOLINE_32BIT_PGTABLE_OFFSET(%ecx), %eax + leal TRAMPOLINE_32BIT_PGTABLE_OFFSET(%edi), %eax movl %eax, %cr3 3: /* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */ - pushl %ecx - pushl %edx movl $MSR_EFER, %ecx rdmsr btsl $_EFER_LME, %eax /* Avoid writing EFER if no change was made (for TDX guest) */ jc 1f wrmsr -1: popl %edx - popl %ecx - +1: #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. @@ -606,14 +602,14 @@ SYM_CODE_START(trampoline_32bit_src) /* Enable PAE and LA57 (if required) paging modes */ orl $X86_CR4_PAE, %eax - testl %edx, %edx + testl %esi, %esi jz 1f orl $X86_CR4_LA57, %eax 1: movl %eax, %cr4 /* Calculate address of paging_enabled() once we are executing in the trampoline */ - leal .Lpaging_enabled - trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_OFFSET(%ecx), %eax + leal .Lpaging_enabled - trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_OFFSET(%edi), %eax /* Prepare the stack for far return to Long Mode */ pushl $__KERNEL_CS @@ -630,7 +626,7 @@ SYM_CODE_END(trampoline_32bit_src) .code64 SYM_FUNC_START_LOCAL_NOALIGN(.Lpaging_enabled) /* Return from the trampoline */ - jmp *%rdi + jmp *%r8 SYM_FUNC_END(.Lpaging_enabled) /* diff --git a/arch/x86/boot/compressed/pgtable.h b/arch/x86/boot/compressed/pgtable.h index 91dbb99203fbce2d..4e8cef135226bcbb 100644 --- a/arch/x86/boot/compressed/pgtable.h +++ b/arch/x86/boot/compressed/pgtable.h @@ -14,7 +14,7 @@ extern unsigned long *trampoline_32bit; -extern void trampoline_32bit_src(void *return_ptr); +extern void trampoline_32bit_src(void *trampoline, bool enable_5lvl); #endif /* __ASSEMBLER__ */ #endif /* BOOT_COMPRESSED_PAGETABLE_H */ From patchwork Mon May 22 07:14:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97138 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272804vqo; Mon, 22 May 2023 00:46:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6Lfgi9xxpLufFmcb5UZ8/9xp9q3hf2oGToRmRw7DIWfj2H7xxZYgQwEjUODrpYfrsBgFDO X-Received: by 2002:a17:90b:e8f:b0:24e:3b85:a8a with SMTP id fv15-20020a17090b0e8f00b0024e3b850a8amr10739574pjb.8.1684741562416; Mon, 22 May 2023 00:46:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741562; cv=none; d=google.com; s=arc-20160816; b=WWisiSXV+5ay6XKc2OiTplrnorlUvKVr1k1jyWWXkq/jV8kryLXjz/ATdJOgpc6pvR JE8ivtFwHL/Uu8r6NA171xzv76uxAvwzXUS/y/Or41pnfYO97ha1NIFtL6iZmWWYHa3a 8bMiAwfZez/IuKAVWIjsJZQ6YLF2WT1QMJ7N6CTXc2/uY+KfLlKmq4PDL+iGCuJ5FrqU b4Q8qqYKgoghTK6Y/NlkC7HsmrfKzsmLkHu0zswSWciOejIpuUu5kCRakMe8nKL0ULwz l05k2jY2KNT7OdmyjlqrstL7x6j1HoK2pPp5HgN+iG8TxU1C4tmPdoYYG5rI1TwQ7Jtr 2Xpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2nct0eXrdF4JNt6BmazV/eXEUxn2frpqXHOq85y9xRY=; b=bIE5DkepdhMv3ju4yjpDrnP/0Zt4gSaKi8JVxfxb0FoOmNDwZDlVjv+3nMWk+bauun 6n8z+Xr6Tueiam2wZ/eQXwDyqwPKgSFpw9osISAEvIATKVgLTyR+kH6Iu0TKgC5OImK7 16FRLiESdpp4yLphm1FBW1oqrJkxgtzkXqtHaxX11gFSLv2oddFw5OuJqaanXwM6FLKg wKKFY5L5tnf33qOSZ9+WhWDLc3PDcdr6FJFsz7JXVzKL3LfwYixqqTDOhfgNoimkIPd5 kyrdG8mYuU9XKho1XZrNpG9wHknBu0ueeco7HoiD790/8mr+W+1VQ2AsYc5HWGfOUW+G hEHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LcLHWjFN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t22-20020a17090ad51600b0024e33c63ed5si2477696pju.15.2023.05.22.00.45.50; Mon, 22 May 2023 00:46:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LcLHWjFN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231657AbjEVHPz (ORCPT + 99 others); Mon, 22 May 2023 03:15:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232318AbjEVHPR (ORCPT ); Mon, 22 May 2023 03:15:17 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66E19BA; Mon, 22 May 2023 00:15:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 04100611D8; Mon, 22 May 2023 07:15:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5FB1C4339C; Mon, 22 May 2023 07:15:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739711; bh=opd5H6IVQawxAyaERFSxkNMBcud1PLTZeWa/VKUckUQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LcLHWjFN1y4DO4wP08aTtHVSTOdt6iegTgkMZRlvdenRn54tcbRMqwcmm4s51XJRU xZTkpP6Zm1M6y2Z902fcBOSN2x0ubjIx//SFFBiZeo5Q+wzuTCm6bJpmWKWeBzao9h Jyg3v7Plta9HRrFr1tIn89Pr2CiYZs8/MD0hBsdGPom0aYdk9FEGKT077ychWS8QMD vWLFh0dhr5DxryBz3Oc/4DMQDswdPI99IrdreVCOLdpu6WjKJYdoxFyQgzejE8RlG4 7dyZ7FsJp3fCMSY2KpElpa1/6A7RON61LopYpBVED/ldN9TZVd/P6q5bb+kT2DDfLh mGS8mai9+jYEQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 09/21] x86/decompressor: Avoid the need for a stack in the 32-bit trampoline Date: Mon, 22 May 2023 09:14:03 +0200 Message-Id: <20230522071415.501717-10-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6184; i=ardb@kernel.org; h=from:subject; bh=opd5H6IVQawxAyaERFSxkNMBcud1PLTZeWa/VKUckUQ=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzOLwtagOtjZW63VHePMPTpXZwnphQ/3/P2tkN/wVz GmJ963rKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOJ72Jk+CRg5J8iU7T5gOip usWnl/ldi/z/YQn3xh3Prs1NbJW6c5fhfzjji+smJ5y3Nbsselyb47hLYZLzf489ky5wLQo0ZPV exQsA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579568528595190?= X-GMAIL-MSGID: =?utf-8?q?1766579568528595190?= The 32-bit trampoline no longer uses the stack for anything except performing a far return back to long mode. Currently, this stack is placed in the same page that carries the trampoline code, which means this page must be mapped writable and executable, and the stack is therefore executable as well. Replace the far return with a far jump, so that the return address can be pre-calculated and patched into the code before it is called. This removes the need for a stack entirely, and in a later patch, this will be taken advantage of by removing writable permissions from (and adding executable permissions to) this code page explicitly when booting via the EFI stub. Not touching the stack pointer also makes it more straight-forward to call the trampoline code as an ordinary 64-bit function from C code. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 45 ++++++++------------ arch/x86/boot/compressed/pgtable.h | 6 +-- arch/x86/boot/compressed/pgtable_64.c | 12 +++++- 3 files changed, 32 insertions(+), 31 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index a387cd80964e1a1e..741b4e8fefc915ea 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -449,9 +449,6 @@ SYM_CODE_START(startup_64) leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax call *%rax - /* Restore the stack, the 32-bit trampoline uses its own stack */ - leaq rva(boot_stack_end)(%rbx), %rsp - /* * cleanup_trampoline() would restore trampoline memory. * @@ -537,24 +534,22 @@ SYM_FUNC_END(.Lrelocated) * EDI contains the base address of the trampoline memory. * Non-zero ESI means trampoline needs to enable 5-level paging. */ + .section ".rodata", "a", @progbits SYM_CODE_START(trampoline_32bit_src) - popq %r8 /* Switch to compatibility mode (CS.L = 0 CS.D = 1) via far return */ pushq $__KERNEL32_CS leaq 0f(%rip), %rax pushq %rax lretq + /* + * The 32-bit code below will do a far jump back to long mode and end + * up here after reconfiguring the number of paging levels. + */ +.Lret: retq + .code32 -0: /* Set up data and stack segments */ - movl $__KERNEL_DS, %eax - movl %eax, %ds - movl %eax, %ss - - /* Set up new stack */ - leal TRAMPOLINE_32BIT_STACK_END(%edi), %esp - - /* Disable paging */ +0: /* Disable paging */ movl %cr0, %eax btrl $X86_CR0_PG_BIT, %eax movl %eax, %cr0 @@ -608,26 +603,22 @@ SYM_CODE_START(trampoline_32bit_src) 1: movl %eax, %cr4 - /* Calculate address of paging_enabled() once we are executing in the trampoline */ - leal .Lpaging_enabled - trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_OFFSET(%edi), %eax - - /* Prepare the stack for far return to Long Mode */ - pushl $__KERNEL_CS - pushl %eax - /* Enable paging again. */ movl %cr0, %eax btsl $X86_CR0_PG_BIT, %eax movl %eax, %cr0 - lret + /* + * Return to the 64-bit calling code using LJMP rather than LRET, to + * avoid the need for a 32-bit addressable stack. The destination + * address will be adjusted after the template code is copied into a + * 32-bit addressable buffer. + */ +.Ljmp: ljmpl $__KERNEL_CS, $(.Lret - trampoline_32bit_src) SYM_CODE_END(trampoline_32bit_src) - .code64 -SYM_FUNC_START_LOCAL_NOALIGN(.Lpaging_enabled) - /* Return from the trampoline */ - jmp *%r8 -SYM_FUNC_END(.Lpaging_enabled) +/* keep this right after trampoline_32bit_src() so we can infer its size */ +SYM_DATA(trampoline_ljmp_imm_offset, .word .Ljmp + 1 - trampoline_32bit_src) /* * The trampoline code has a size limit. @@ -636,7 +627,7 @@ SYM_FUNC_END(.Lpaging_enabled) */ .org trampoline_32bit_src + TRAMPOLINE_32BIT_CODE_SIZE - .code32 + .text SYM_FUNC_START_LOCAL_NOALIGN(.Lno_longmode) /* This isn't an x86-64 CPU, so hang intentionally, we cannot continue */ 1: diff --git a/arch/x86/boot/compressed/pgtable.h b/arch/x86/boot/compressed/pgtable.h index 4e8cef135226bcbb..131488f50af55d0a 100644 --- a/arch/x86/boot/compressed/pgtable.h +++ b/arch/x86/boot/compressed/pgtable.h @@ -6,9 +6,7 @@ #define TRAMPOLINE_32BIT_PGTABLE_OFFSET 0 #define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0xA0 - -#define TRAMPOLINE_32BIT_STACK_END TRAMPOLINE_32BIT_SIZE +#define TRAMPOLINE_32BIT_CODE_SIZE 0x80 #ifndef __ASSEMBLER__ @@ -16,5 +14,7 @@ extern unsigned long *trampoline_32bit; extern void trampoline_32bit_src(void *trampoline, bool enable_5lvl); +extern const u16 trampoline_ljmp_imm_offset; + #endif /* __ASSEMBLER__ */ #endif /* BOOT_COMPRESSED_PAGETABLE_H */ diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index 2ac12ff4111bf8c0..09fc18180929fab3 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -109,6 +109,7 @@ static unsigned long find_trampoline_placement(void) struct paging_config paging_prepare(void *rmode) { struct paging_config paging_config = {}; + void *tramp_code; /* Initialize boot_params. Required for cmdline_find_option_bool(). */ boot_params = rmode; @@ -143,9 +144,18 @@ struct paging_config paging_prepare(void *rmode) memset(trampoline_32bit, 0, TRAMPOLINE_32BIT_SIZE); /* Copy trampoline code in place */ - memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), + tramp_code = memcpy(trampoline_32bit + + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE); + /* + * Avoid the need for a stack in the 32-bit trampoline code, by using + * LJMP rather than LRET to return back to long mode. LJMP takes an + * immediate absolute address, so we have to adjust that based on the + * placement of the trampoline. + */ + *(u32 *)(tramp_code + trampoline_ljmp_imm_offset) += (unsigned long)tramp_code; + /* * The code below prepares page table in trampoline memory. * From patchwork Mon May 22 07:14:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97111 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1262848vqo; Mon, 22 May 2023 00:19:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4+rF4a0LXkf07md1WTwLPVjyWMbJHvDwhqJvMFoG9fWpBqc/YlfNSzxoVIOsK0R49BQepT X-Received: by 2002:a17:90b:1003:b0:255:5bde:e6cc with SMTP id gm3-20020a17090b100300b002555bdee6ccmr3263588pjb.17.1684739976380; Mon, 22 May 2023 00:19:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684739976; cv=none; d=google.com; s=arc-20160816; b=JKFzLPUZhSSCCbz1Wx4rx9XpWv9p/Mw2kM40i4g2I6g/kEK6/20HXEe9zSg+feccub zuTlxnHnKWbyJ2cc3XtocYAK/WB1+DpjIayQVjCTbfVQSRdO0S352z9jxkRz83sDwtpB TW6NOG2MP2Rw+o5ioXwchpUN2tJ92AMip6iMN89yLbP/rCywb7m1cvGV8ZV8V6ycV8op x90R9gn+7A7Ci0Jng9siM8mz5LLfHZYhnmcHvl7ekazjLuowJ+z9X1r9jLrldZFGpStt uxa3ZnepJrKFmj+6bDm8xoQCPZDQKSh9Y/J7bxjTs8J/PP+t8gxR0nQPmy345iBeduLR kulg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=oRBXgbs8R67IxSY7rXKZgl8HECs+aqz1XKXNDvM1rAU=; b=USDmXCsd6WEvnA3kgf7gFX1UM/1d10xy2zOtdXPd8qERDZDgxRSW7mt+2z6/Fdn1gX arRb1Z6S1qRCvFoxKbXUEUglhKFaPdKlvUtq5PxyWRdLCVQ69SFm/dVP7RhfbkxTpF4T rViwiyp6KqAABLvtSSv4bWZa5xOe6grOdLP0XWUDp6agX62Z469AfnlT6cZI26uoU2P6 7NrasCePaVW8LU+xvkJige6/rc8GTTnbTkk1I8uSHRzTyva9iXpCsMA8sfTqzOk3R02g j1Rubnxa/p3WrfqV3WnaKQDBnS90MN581YHMC7iLnsJ6jYiTxEJkV7WVl9SN5Do+HFUu Lpxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="hz/XLYpw"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x71-20020a63864a000000b0053b52fed717si3184011pgd.864.2023.05.22.00.19.20; Mon, 22 May 2023 00:19:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="hz/XLYpw"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232458AbjEVHQG (ORCPT + 99 others); Mon, 22 May 2023 03:16:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232097AbjEVHPd (ORCPT ); Mon, 22 May 2023 03:15:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A75D5CA; Mon, 22 May 2023 00:15:16 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2CBB061DC4; Mon, 22 May 2023 07:15:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DBAEFC433EF; Mon, 22 May 2023 07:15:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739715; bh=hFZ/SCviIEc1YZbiLBLlEDyweQ6PGSxYwFHksamIzHs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hz/XLYpwvrB7eaE67rIp6oVZcPncDEeoJ1HK+tnyQVcUCtXSHsbtsqPkd8b+q6RAT ZKfyjLbFV7cR2whGTuHdlUIvZT8cdvhBpe3lhbO6U9lffuWe2Tj3qFECI2hjnIyYpT uMkW92CnW/NWGG+IJ4qnp9ow11jZtgz7G6CdiN6OjrQ8dK9z8+LtMJQm8WuxcRJy5P 7QpuutM9ZljLjE83bqk3d/qSZPksgzgMO7Wf/FG5bfUEgPYezQ8xwYzWbRwLeEODq5 Xu4TCQTn8mY3nybucf17nyxw6gg3XkiWZpZElvymw03DzBd2SAmMdcjhcng0/0bvyv iWtsyc3P1mXfg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 10/21] x86/decompressor: Call trampoline directly from C code Date: Mon, 22 May 2023 09:14:04 +0200 Message-Id: <20230522071415.501717-11-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4966; i=ardb@kernel.org; h=from:subject; bh=hFZ/SCviIEc1YZbiLBLlEDyweQ6PGSxYwFHksamIzHs=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzPL8sq2/X/qHu2RslN1qcKxny6qQhYuW/epuYZguW MS46tj8jlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRoyGMDId17vRVz82v/sNW 3LB74ubOr+6SR34XTX9+UuHX+vULoh4x/OGZ2Gx+NysqlrmRZ573db4utlXfpc6IrTXQzc8zuWX xkgsA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766577905413286394?= X-GMAIL-MSGID: =?utf-8?q?1766577905413286394?= Instead of returning to the asm calling code to invoke the trampoline, call it straight from the C code that sets the scene. That way, the struct return type is no longer needed for returning two values, and the call can be made conditional more cleanly in a subsequent patch. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 20 +++----------- arch/x86/boot/compressed/pgtable_64.c | 28 ++++++++------------ 2 files changed, 15 insertions(+), 33 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 741b4e8fefc915ea..a60ec9283bd760e3 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -430,24 +430,12 @@ SYM_CODE_START(startup_64) #endif /* - * paging_prepare() sets up the trampoline and checks if we need to - * enable 5-level paging. - * - * paging_prepare() returns a two-quadword structure which lands - * into RDX:RAX: - * - Address of the trampoline is returned in RAX. - * - Non zero RDX means trampoline needs to enable 5-level - * paging. - * + * set_paging_levels() updates the number of paging levels using a + * trampoline in 32-bit addressable memory if the current number does + * not match the desired number. */ movq %r15, %rdi /* pass struct boot_params pointer */ - call paging_prepare - - /* Pass the trampoline address and boolean flag as args #1 and #2 */ - movq %rax, %rdi - movq %rdx, %rsi - leaq TRAMPOLINE_32BIT_CODE_OFFSET(%rax), %rax - call *%rax + call set_paging_levels /* * cleanup_trampoline() would restore trampoline memory. diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index 09fc18180929fab3..b62b6819dcdd01be 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -16,11 +16,6 @@ unsigned int __section(".data") pgdir_shift = 39; unsigned int __section(".data") ptrs_per_p4d = 1; #endif -struct paging_config { - unsigned long trampoline_start; - unsigned long l5_required; -}; - /* Buffer to preserve trampoline memory */ static char trampoline_save[TRAMPOLINE_32BIT_SIZE]; @@ -106,10 +101,10 @@ static unsigned long find_trampoline_placement(void) return bios_start - TRAMPOLINE_32BIT_SIZE; } -struct paging_config paging_prepare(void *rmode) +asmlinkage void set_paging_levels(void *rmode) { - struct paging_config paging_config = {}; - void *tramp_code; + void (*toggle_la57)(void *trampoline, bool enable_5lvl); + bool l5_required = false; /* Initialize boot_params. Required for cmdline_find_option_bool(). */ boot_params = rmode; @@ -130,12 +125,10 @@ struct paging_config paging_prepare(void *rmode) !cmdline_find_option_bool("no5lvl") && native_cpuid_eax(0) >= 7 && (native_cpuid_ecx(7) & (1 << (X86_FEATURE_LA57 & 31)))) { - paging_config.l5_required = 1; + l5_required = true; } - paging_config.trampoline_start = find_trampoline_placement(); - - trampoline_32bit = (unsigned long *)paging_config.trampoline_start; + trampoline_32bit = (unsigned long *)find_trampoline_placement(); /* Preserve trampoline memory */ memcpy(trampoline_save, trampoline_32bit, TRAMPOLINE_32BIT_SIZE); @@ -144,7 +137,7 @@ struct paging_config paging_prepare(void *rmode) memset(trampoline_32bit, 0, TRAMPOLINE_32BIT_SIZE); /* Copy trampoline code in place */ - tramp_code = memcpy(trampoline_32bit + + toggle_la57 = memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE); @@ -154,7 +147,8 @@ struct paging_config paging_prepare(void *rmode) * immediate absolute address, so we have to adjust that based on the * placement of the trampoline. */ - *(u32 *)(tramp_code + trampoline_ljmp_imm_offset) += (unsigned long)tramp_code; + *(u32 *)((u8 *)toggle_la57 + trampoline_ljmp_imm_offset) += + (unsigned long)toggle_la57; /* * The code below prepares page table in trampoline memory. @@ -170,10 +164,10 @@ struct paging_config paging_prepare(void *rmode) * We are not going to use the page table in trampoline memory if we * are already in the desired paging mode. */ - if (paging_config.l5_required == !!(native_read_cr4() & X86_CR4_LA57)) + if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) goto out; - if (paging_config.l5_required) { + if (l5_required) { /* * For 4- to 5-level paging transition, set up current CR3 as * the first and the only entry in a new top-level page table. @@ -196,7 +190,7 @@ struct paging_config paging_prepare(void *rmode) } out: - return paging_config; + toggle_la57(trampoline_32bit, l5_required); } void cleanup_trampoline(void *pgtable) From patchwork Mon May 22 07:14:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97113 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1263872vqo; Mon, 22 May 2023 00:22:05 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4AvlKGNmL0pPFQWYHG5rBIG4od8LQLwOgZ2O60nrh+ENgruEnMhqai3FCeSxHQrMSoRVNR X-Received: by 2002:a05:6a21:2d8c:b0:10a:e177:9e73 with SMTP id ty12-20020a056a212d8c00b0010ae1779e73mr5602930pzb.46.1684740125323; Mon, 22 May 2023 00:22:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684740125; cv=none; d=google.com; s=arc-20160816; b=vHO5M29QZm0yP27AebkEjkMDihszjuv4XHMfI1zxCay38T9QH3clYkgwLnajh81qlC cou6ESu9tIbX7LQODNsTcCQhxhOS2P4+YNl9g54GXyY5jEKgtefw4cc1oJqbRG81wMN7 46STFvaYyp2H135mlyxCdLScNds6yTFkRlcl9S7YK0qwz3Em7PYAntklKGlbjLuFWvbM lFi//pPyh2EbgFz8NJxtH9apa6Tf7pxfAUlXEKcRBXWZf6oZFK8NjEYqxbrWJNQrTjiA J17ZexRR5dAaRv/5OfY98OQ0WXrTQAaNLRMyFkpDx7xAWYV9K72NC5kcX2huvJ9vIial Vxtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=x6HDeOfTmKCWyOfu+z54dADCNk6vl1IOzYSyYB4KuDs=; b=I7AlzDAG2vV8uQtguQdJCOI+dYW2ir3+zT8FO53UMLfejZqct6GK4tsPhake6IX8AI bGU/NdzpW5CYcTJMrWtURF80VeYxYU8gzsE3TevY519CQrwltG25JxN07KlOOvOdJl5i DAHDloP7BHoPw7LhSH46tMMyCiiyJwLeBQ0mRIbpkIVQ2DTkQIv+AvRcZdBP6w4RoJVL EM1z47n/W8mztVg4L3h00019wBQlLDGrp5nURJ4Q8+QwtWnQLrv8Hteqg4NbaB1xpOUP F4LLnaW2UcC2b/g1TSAUJl3mblAAUMMXLu7nKipXckIjaUPefjSNgyyWDONSd36OYn74 Bsqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=FyjAA71P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x71-20020a63864a000000b0053b52fed717si3184011pgd.864.2023.05.22.00.21.49; Mon, 22 May 2023 00:22:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=FyjAA71P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232421AbjEVHQM (ORCPT + 99 others); Mon, 22 May 2023 03:16:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232397AbjEVHPr (ORCPT ); Mon, 22 May 2023 03:15:47 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CED7118F; Mon, 22 May 2023 00:15:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 532D261DD3; Mon, 22 May 2023 07:15:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0CA83C4339C; Mon, 22 May 2023 07:15:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739719; bh=u+mBV9LASMmynY9CQFGbcEYb3DED+j3/4fMymyiLQXA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FyjAA71PYrnpMLQHPHv32CH7/BWCS6DH91r8ny9yChy7sOq8/30xAg760z547M2O5 YthevGeJ5sdMO7RP0KpjBmHDCZm04SpHkKUXGGPnk1oh093YhuyEgjQbsYhBOFllUy hngucDLL8pwjGpkykQX8i/QhwBJRfES+9nDgsMOUkUjJRpT1tU/5p8VxYg/1KTb6DZ N/rpOcxxe8by8sDqBShDD0CHILaxXjWD9+EJ4pliMvzYNnCQbekaazrRfrBJ59Ut1T QfAGS2j0BtezeMB9Z74qvE1h6+fFWXtuqTyivAbAP7t6DnJzlG0gAYDK2sRSbMCqHR ufxYdIy6qrd2A== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 11/21] x86/decompressor: Only call the trampoline when changing paging levels Date: Mon, 22 May 2023 09:14:05 +0200 Message-Id: <20230522071415.501717-12-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3739; i=ardb@kernel.org; h=from:subject; bh=u+mBV9LASMmynY9CQFGbcEYb3DED+j3/4fMymyiLQXA=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzMrC9nxPor1CuD6POqfBqxxt7VtsE05xa3y19Hiz8 VL3Y9eOUhYGMQ4GWTFFFoHZf9/tPD1RqtZ5lizMHFYmkCEMXJwCMJENcxj+2cW3taQ3TdRs3W80 u1L4BD9nxvrHpRFMWyt/C83jrll2jpFhdVCjxOEvb1LvXTrkl1bDI7eq4PSsuY6lJSZ7+I/2FbI wAwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766578061932902533?= X-GMAIL-MSGID: =?utf-8?q?1766578061932902533?= Since the current and desired number of paging levels are known when the trampoline is being prepared, avoid calling the trampoline at all if it is clear that calling it is not going to result in a change to the number of paging levels. Given that the CPU is already running in long mode, the PAE and LA57 settings are necessarily consistent with the currently active page tables - the only difference is that CR4.MCE will always be preserved in this case, but it will be cleared by the real kernel startup code if CONFIG_X86_MCE is not enabled. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 21 +------------------- arch/x86/boot/compressed/pgtable_64.c | 18 +++++++---------- 2 files changed, 8 insertions(+), 31 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index a60ec9283bd760e3..403c96dae34d9c6d 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -387,10 +387,6 @@ SYM_CODE_START(startup_64) * For the trampoline, we need the top page table to reside in lower * memory as we don't have a way to load 64-bit values into CR3 in * 32-bit mode. - * - * We go though the trampoline even if we don't have to: if we're - * already in a desired paging mode. This way the trampoline code gets - * tested on every boot. */ /* Make sure we have GDT with 32-bit code segment */ @@ -542,25 +538,10 @@ SYM_CODE_START(trampoline_32bit_src) btrl $X86_CR0_PG_BIT, %eax movl %eax, %cr0 - /* Check what paging mode we want to be in after the trampoline */ - testl %esi, %esi - jz 1f - - /* We want 5-level paging: don't touch CR3 if it already points to 5-level page tables */ - movl %cr4, %eax - testl $X86_CR4_LA57, %eax - jnz 3f - jmp 2f -1: - /* We want 4-level paging: don't touch CR3 if it already points to 4-level page tables */ - movl %cr4, %eax - testl $X86_CR4_LA57, %eax - jz 3f -2: /* Point CR3 to the trampoline's new top level page table */ leal TRAMPOLINE_32BIT_PGTABLE_OFFSET(%edi), %eax movl %eax, %cr3 -3: + /* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */ movl $MSR_EFER, %ecx rdmsr diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index b62b6819dcdd01be..b92cf1d6e156d5f6 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -128,6 +128,13 @@ asmlinkage void set_paging_levels(void *rmode) l5_required = true; } + /* + * We are not going to use the trampoline if we + * are already in the desired paging mode. + */ + if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) + return; + trampoline_32bit = (unsigned long *)find_trampoline_placement(); /* Preserve trampoline memory */ @@ -155,18 +162,8 @@ asmlinkage void set_paging_levels(void *rmode) * * The new page table will be used by trampoline code for switching * from 4- to 5-level paging or vice versa. - * - * If switching is not required, the page table is unused: trampoline - * code wouldn't touch CR3. */ - /* - * We are not going to use the page table in trampoline memory if we - * are already in the desired paging mode. - */ - if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) - goto out; - if (l5_required) { /* * For 4- to 5-level paging transition, set up current CR3 as @@ -189,7 +186,6 @@ asmlinkage void set_paging_levels(void *rmode) (void *)src, PAGE_SIZE); } -out: toggle_la57(trampoline_32bit, l5_required); } From patchwork Mon May 22 07:14:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97131 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272629vqo; Mon, 22 May 2023 00:45:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7pQa3WSfhICMfg0TY2xa0KVq5XwI74NVhUv3xFqHjN8afKFn9UnwU9nxV0kdN0Ug0COKkY X-Received: by 2002:a17:902:e995:b0:1ae:536c:3d9c with SMTP id f21-20020a170902e99500b001ae536c3d9cmr11119078plb.37.1684741540204; Mon, 22 May 2023 00:45:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741540; cv=none; d=google.com; s=arc-20160816; b=f/z5S7VK2dZ4t4s4vhA9iviEgRNI/NkIuA5+e8YjKrYq9/2zyn/GGdOA6+noah9oFA yYb0eBxTEns5CrmgptkcaBM4gvyK8B+UNIhe3v/GyuiMabR6keASOTnSRidb+PHjiXFE suZJlW1EGS+J/OfVQBCKPWKr/yoqiXFL2ZzzzwefiarJEzBgOGX77xUe9aUUHvqpvwYc asb8fJd22NVpc9KBrZklAzotHuQx33u1if1zpSCL+JrXwcOHdlUtmrGbP5AFX+h2q3O5 vtDsNhIUHpPRW+r3usPhwxvAiNYDy4pF+k2ygJ5W7kKrClaFY0o8BGQjzsh/siS4HgCp fE4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2GvniZfDsJu6SuNQirP/r0oCYG6eEQ0It0hxOuT6TVk=; b=Ade4YWRETgTy1K91j/BXo2GsskZIvyU92doeolg4BDBMJWCpFrn8hANQOdvFCnXxu/ XJ4DX7M10qCqVFLvorPpOsr6cBt5yu7Zpfg2rJG9Z3lROqZeVbVmmfluD79dPv3FkUTs 4bhz/jyLdKJ0lWKfwmsAHu1Sf2kygPXzN/Jys/RcuQ+Dx0CRbRY3jHLxz6fCQl47G1Sg tD/2DmZ0XNeQWqWT7IC1ALvbBLgbnl/bzBfYn5i1bCpwZ/6j2qpdy57fgnNg5N5VUF6A nX/gRheLczhpGDew0Zjt9xIoYjNrFqjyA9VcixI2G4y6I2+OGseftGXk3LLo30yItWH4 R3fg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LYl06j+J; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i8-20020a170902c94800b0019e57f5a5aesi4441460pla.567.2023.05.22.00.45.27; Mon, 22 May 2023 00:45:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LYl06j+J; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232199AbjEVHQV (ORCPT + 99 others); Mon, 22 May 2023 03:16:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232504AbjEVHQD (ORCPT ); Mon, 22 May 2023 03:16:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FCAD1BB; Mon, 22 May 2023 00:15:25 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7ED8A61DE7; Mon, 22 May 2023 07:15:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3320DC433D2; Mon, 22 May 2023 07:15:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739723; bh=CnFv45/Vh35jv2ecaVpF4w7431pbCKxy7DMiPDdcENI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LYl06j+J8ieYZZca2k1PJDH91lGO+30SLJSBq06nguarkjbfMDpUecxsAnuYX0y+e j1phsGY8+nx4lIU611QIJ80L1lZFACYxOLOu7IXGc2t+9++aqhyOiyCjSyosdQlXPp jN9WBZRv4BcsgoHZgWDUZ0ijcx76sqNBlneVNv2yT05n15v4fhnHC7LzoUXH/anGg9 IuBSC8v/+PZ1uzkkKKqIawtbnM1Y4sDWqnsHDhgn86UhItg2W/bnWqu5i/GJpY7a4X uhfFYP62wf2i+zdTt7JrkrGy68+ZbyLwxdbZCGaYPlGJpxapQX1TjKNZmART5vzTqi 7Jf5leHkURz4A== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 12/21] x86/decompressor: Merge trampoline cleanup with switching code Date: Mon, 22 May 2023 09:14:06 +0200 Message-Id: <20230522071415.501717-13-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4166; i=ardb@kernel.org; h=from:subject; bh=CnFv45/Vh35jv2ecaVpF4w7431pbCKxy7DMiPDdcENI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzLpZ32dibazdja1Rn9b22qXov+fefvkk913jmvsPj pSsXHu3o5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzkoQPD/zwWY4mVUy4VsMvt jI2IfJU8PXCJlfixy8de2z44onBukRjDP6M66dpYdy6l77pmR7/Y/v/S9CB37XK3HfcSZ3ILMD/ kZQcA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579545363758305?= X-GMAIL-MSGID: =?utf-8?q?1766579545363758305?= Now that the trampoline setup code and the actual invocation of it are all done from the C routine, the trampoline cleanup can be merged into it as well, instead of returning to asm just to call another C function. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 13 +++------ arch/x86/boot/compressed/pgtable_64.c | 28 ++++++++------------ 2 files changed, 15 insertions(+), 26 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 403c96dae34d9c6d..b5bd6be035a7b7ec 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -429,19 +429,14 @@ SYM_CODE_START(startup_64) * set_paging_levels() updates the number of paging levels using a * trampoline in 32-bit addressable memory if the current number does * not match the desired number. + * + * RSI is the relocated address of the page table to use instead of + * page table in trampoline memory (if required). */ movq %r15, %rdi /* pass struct boot_params pointer */ + leaq rva(top_pgtable)(%rbx), %rsi call set_paging_levels - /* - * cleanup_trampoline() would restore trampoline memory. - * - * RDI is address of the page table to use instead of page table - * in trampoline memory (if required). - */ - leaq rva(top_pgtable)(%rbx), %rdi - call cleanup_trampoline - /* Zero EFLAGS */ pushq $0 popfq diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index b92cf1d6e156d5f6..eeddad8c8335655e 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -101,9 +101,10 @@ static unsigned long find_trampoline_placement(void) return bios_start - TRAMPOLINE_32BIT_SIZE; } -asmlinkage void set_paging_levels(void *rmode) +asmlinkage void set_paging_levels(void *rmode, void *pgtable) { void (*toggle_la57)(void *trampoline, bool enable_5lvl); + void *trampoline_pgtable; bool l5_required = false; /* Initialize boot_params. Required for cmdline_find_option_bool(). */ @@ -133,7 +134,7 @@ asmlinkage void set_paging_levels(void *rmode) * are already in the desired paging mode. */ if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) - return; + goto out; trampoline_32bit = (unsigned long *)find_trampoline_placement(); @@ -163,6 +164,8 @@ asmlinkage void set_paging_levels(void *rmode) * The new page table will be used by trampoline code for switching * from 4- to 5-level paging or vice versa. */ + trampoline_pgtable = trampoline_32bit + + TRAMPOLINE_32BIT_PGTABLE_OFFSET / sizeof(unsigned long); if (l5_required) { /* @@ -182,31 +185,21 @@ asmlinkage void set_paging_levels(void *rmode) * may be above 4G. */ src = *(unsigned long *)__native_read_cr3() & PAGE_MASK; - memcpy(trampoline_32bit + TRAMPOLINE_32BIT_PGTABLE_OFFSET / sizeof(unsigned long), - (void *)src, PAGE_SIZE); + memcpy(trampoline_pgtable, (void *)src, PAGE_SIZE); } toggle_la57(trampoline_32bit, l5_required); -} - -void cleanup_trampoline(void *pgtable) -{ - void *trampoline_pgtable; - - trampoline_pgtable = trampoline_32bit + TRAMPOLINE_32BIT_PGTABLE_OFFSET / sizeof(unsigned long); /* - * Move the top level page table out of trampoline memory, - * if it's there. + * Move the top level page table out of trampoline memory. */ - if ((void *)__native_read_cr3() == trampoline_pgtable) { - memcpy(pgtable, trampoline_pgtable, PAGE_SIZE); - native_write_cr3((unsigned long)pgtable); - } + memcpy(pgtable, trampoline_pgtable, PAGE_SIZE); + native_write_cr3((unsigned long)pgtable); /* Restore trampoline memory */ memcpy(trampoline_32bit, trampoline_save, TRAMPOLINE_32BIT_SIZE); +out: /* Initialize variables for 5-level paging */ #ifdef CONFIG_X86_5LEVEL if (__read_cr4() & X86_CR4_LA57) { @@ -215,4 +208,5 @@ void cleanup_trampoline(void *pgtable) ptrs_per_p4d = 512; } #endif + return; } From patchwork Mon May 22 07:14:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97128 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272474vqo; Mon, 22 May 2023 00:45:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5EXpdpA5dAGxluxLojg9XxHfS+4FnedBKhdqJNBcUyQ2sJK2PmPF5j7BZs/PT5coPMF1Jn X-Received: by 2002:a05:6a20:2453:b0:104:d028:ffe8 with SMTP id t19-20020a056a20245300b00104d028ffe8mr9653022pzc.45.1684741518005; Mon, 22 May 2023 00:45:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741517; cv=none; d=google.com; s=arc-20160816; b=oJMQ3uvj1SwGPEIxfwOnB0opCWACRdV0Rv1CydCMakXHz0jlWcWi8IVXKf6Z5R28IF at7XJqRejPgw/KW3JerbuzQN0nUQ00ywI5aW+XL8m40XuMilBbBupYWMT8oFXku4kXEl l+jt3ehuuaPrMD8tVeMhTzN7IgM1GiP8CaflHmq0u41KSumqfgMJKk5u61b/Z0NM7JCb krJwyfGPpugPkA1d0/C0Fs7l4QG31SAywJsUCMayzBcxK2l9zAX4KHb4HZKB/uLyXO5U hCCgJjJLt38EOxPw4dgkPqj69bt4OFQ87dcutG7hFiSm3QtiB1NWo/UBnma2P2TNm2R8 F0vQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Jn0587uIpTyGfApRe3P58rCG+/kGzNcaasrszTmnD/I=; b=kuH+TSdd26STozSVBdL58Y6zg+AQvRbR5eQjYNW/niw7QgUoRfy8VSnsLBBdib9+YX GzBjANdDRsLm2kSrBQNUNmo/yrJCNHEB+7DLlDyeRbDOPj+iP2h4w/w9FSdLtHHEqCd9 db3ODgq1osVz7qiUZFV3b6Hz/cENYsOk0EWJdQS6kktSlTuhmGNli3Bkmh3fht0GQzqF oQrvTjhjew5CaeByDFsfD5vcrLrPoIPxbo0NwxxbSRH8dS0Y6bLsPyPmP3Vgy7Hnd26P DnNVY5cYU8bvzGbCXgGO4HCkhPbShUcDJZoofXd8013czi45xdH/Jhscr8E2bzRj33mK jyIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="H7aiO/lW"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i11-20020a633c4b000000b0053486f3dcc2si4797562pgn.629.2023.05.22.00.45.06; Mon, 22 May 2023 00:45:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="H7aiO/lW"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232509AbjEVHQi (ORCPT + 99 others); Mon, 22 May 2023 03:16:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232532AbjEVHQG (ORCPT ); Mon, 22 May 2023 03:16:06 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D89EE59; Mon, 22 May 2023 00:15:29 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9DD5F61DC4; Mon, 22 May 2023 07:15:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 58C01C4339B; Mon, 22 May 2023 07:15:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739728; bh=4XvSZ9l1kBf4mLwN/wIv4K5HCtGQvEmq6xkR3el+340=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H7aiO/lWJufDh1NlapFTCIEbwmAVbkMfVBPsEKIAhC8X64Kaawi6tsFCgbQs8b5dz HLpu2oX3hRC5/L613vJ/95Qy9nRfyTNXomq2jTp4ZAh5z/a38pIoETEVdwMMxT29Tk 4TCdc3agdmeA9ni1I/ja960wXEVlAYi5F8XzxPaLI6Y3NQSccUGHNLV/eGQCygs7ba 0GtmBxgI1nF+ZSJKyAoDR/xVwNsTCiLCJfRtZvDuBjHp7QSJve50OiF3oMEvTafVsJ 4QxcORuFVfbNBPbhcKXcz5+xEZqqnSDv5oM+kWmLjCI/0R8d9pq/eq7RSmmM35aEZy QEt2k4NMLV0fQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 13/21] x86/efistub: Perform 4/5 level paging switch from the stub Date: Mon, 22 May 2023 09:14:07 +0200 Message-Id: <20230522071415.501717-14-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10043; i=ardb@kernel.org; h=from:subject; bh=4XvSZ9l1kBf4mLwN/wIv4K5HCtGQvEmq6xkR3el+340=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzLZoign7L+W9c+7v7ZMpqNi29qScDIu2B6NK2vYJD Rau1lIdpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCJzKxgZGgNuVug/9AroPKni I7drasVdAccv17ij0l8FNSQzzw/gZfgr5GFx23SVYse/Kz+MVJKM/i+avzci98uKtd9FNsx5uIy dHQA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579522080697381?= X-GMAIL-MSGID: =?utf-8?q?1766579522080697381?= In preparation for updating the EFI stub boot flow to avoid the bare metal decompressor code altogether, implement the support code for switching between 4 and 5 levels of paging before jumping to the kernel proper. This reuses the newly refactored trampoline that the bare metal decompressor uses, but relies on EFI APIs to allocate 32-bit addressable memory and remap it with the appropriate permissions. Given that the bare metal decompressor will no longer call into the trampoline if the number of paging levels is already set correctly, it is no longer needed to remove NX restrictions from the memory range where this trampoline may end up. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/Makefile | 1 + drivers/firmware/efi/libstub/efi-stub-helper.c | 2 + drivers/firmware/efi/libstub/efistub.h | 1 + drivers/firmware/efi/libstub/x86-5lvl.c | 94 ++++++++++++++++++++ drivers/firmware/efi/libstub/x86-stub.c | 45 ++++------ drivers/firmware/efi/libstub/x86-stub.h | 12 +++ 6 files changed, 128 insertions(+), 27 deletions(-) diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index 16d64a34d1e19465..ae8874401a9f1490 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -88,6 +88,7 @@ lib-$(CONFIG_EFI_GENERIC_STUB) += efi-stub.o string.o intrinsics.o systable.o \ lib-$(CONFIG_ARM) += arm32-stub.o lib-$(CONFIG_ARM64) += arm64.o arm64-stub.o smbios.o lib-$(CONFIG_X86) += x86-stub.o +lib-$(CONFIG_X86_64) += x86-5lvl.o lib-$(CONFIG_RISCV) += riscv.o riscv-stub.o lib-$(CONFIG_LOONGARCH) += loongarch.o loongarch-stub.o diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c b/drivers/firmware/efi/libstub/efi-stub-helper.c index 1e0203d74691ffcc..51779279fbff21b5 100644 --- a/drivers/firmware/efi/libstub/efi-stub-helper.c +++ b/drivers/firmware/efi/libstub/efi-stub-helper.c @@ -73,6 +73,8 @@ efi_status_t efi_parse_options(char const *cmdline) efi_loglevel = CONSOLE_LOGLEVEL_QUIET; } else if (!strcmp(param, "noinitrd")) { efi_noinitrd = true; + } else if (IS_ENABLED(CONFIG_X86_64) && !strcmp(param, "no5lvl")) { + efi_no5lvl = true; } else if (!strcmp(param, "efi") && val) { efi_nochunk = parse_option_str(val, "nochunk"); efi_novamap |= parse_option_str(val, "novamap"); diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h index 8659a01664b85d95..191698e8489d82e7 100644 --- a/drivers/firmware/efi/libstub/efistub.h +++ b/drivers/firmware/efi/libstub/efistub.h @@ -33,6 +33,7 @@ #define EFI_ALLOC_LIMIT ULONG_MAX #endif +extern bool efi_no5lvl; extern bool efi_nochunk; extern bool efi_nokaslr; extern int efi_loglevel; diff --git a/drivers/firmware/efi/libstub/x86-5lvl.c b/drivers/firmware/efi/libstub/x86-5lvl.c new file mode 100644 index 0000000000000000..f7284f6270abcc18 --- /dev/null +++ b/drivers/firmware/efi/libstub/x86-5lvl.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include + +#include +#include +#include + +#include "efistub.h" +#include "x86-stub.h" + +bool efi_no5lvl; + +static void (*la57_toggle)(void *trampoline, bool enable_5lvl); + +static const struct desc_struct gdt[] = { + [GDT_ENTRY_KERNEL32_CS] = GDT_ENTRY_INIT(0xc09b, 0, 0xfffff), + [GDT_ENTRY_KERNEL_CS] = GDT_ENTRY_INIT(0xa09b, 0, 0xfffff), +}; + +/* + * Enabling (or disabling) 5 level paging is tricky, because it can only be + * done from 32-bit mode with paging disabled. This means not only that the + * code itself must be running from 32-bit addressable physical memory, but + * also that the root page table must be 32-bit addressable, as we cannot + * program a 64-bit value into CR3 when running in 32-bit mode. + */ +efi_status_t efi_setup_5level_paging(void) +{ + u8 tmpl_size = (u8 *)&trampoline_ljmp_imm_offset - (u8 *)&trampoline_32bit_src; + efi_status_t status; + u8 *la57_code; + + if (!efi_is_64bit()) + return EFI_SUCCESS; + + /* check for 5 level paging support */ + if (native_cpuid_eax(0) < 7 || + !(native_cpuid_ecx(7) & (1 << (X86_FEATURE_LA57 & 31)))) + return EFI_SUCCESS; + + /* allocate some 32-bit addressable memory for code and a page table */ + status = efi_allocate_pages(2 * PAGE_SIZE, (unsigned long *)&la57_code, + U32_MAX); + if (status != EFI_SUCCESS) + return status; + + la57_toggle = memcpy(la57_code, trampoline_32bit_src, tmpl_size); + memset(la57_code + tmpl_size, 0x90, PAGE_SIZE - tmpl_size); + + /* + * To avoid having to allocate a 32-bit addressable stack, we use a + * ljmp to switch back to long mode. However, this takes an absolute + * address, so we have to poke it in at runtime. + */ + *(u32 *)&la57_code[trampoline_ljmp_imm_offset] += (unsigned long)la57_code; + + efi_adjust_memory_range_protection((unsigned long)la57_toggle, PAGE_SIZE); + + return EFI_SUCCESS; +} + +void efi_5level_switch(void) +{ + bool want_la57 = IS_ENABLED(CONFIG_X86_5LEVEL) && !efi_no5lvl; + bool have_la57 = native_read_cr4() & X86_CR4_LA57; + bool need_toggle = want_la57 ^ have_la57; + u64 *pgt = (void *)la57_toggle + PAGE_SIZE; + u64 *cr3 = (u64 *)__native_read_cr3(); + u64 *new_cr3; + + if (!la57_toggle || !need_toggle) + return; + + if (!have_la57) { + /* + * We are going to enable 5 level paging, so we need to + * allocate a root level page from the 32-bit addressable + * physical region, and plug the existing hierarchy into it. + */ + new_cr3 = memset(pgt, 0, PAGE_SIZE); + new_cr3[0] = (u64)cr3 | _PAGE_TABLE_NOENC; + } else { + // take the new root table pointer from the current entry #0 + new_cr3 = (u64 *)(cr3[0] & PAGE_MASK); + + // copy the new root level table if it is not 32-bit addressable + if ((u64)new_cr3 > U32_MAX) + new_cr3 = memcpy(pgt, new_cr3, PAGE_SIZE); + } + + native_load_gdt(&(struct desc_ptr){ sizeof(gdt) - 1, (u64)gdt }); + + la57_toggle(new_cr3, want_la57); +} diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index d010448dffb12cb8..229def7d9b028ceb 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -17,6 +17,7 @@ #include #include "efistub.h" +#include "x86-stub.h" /* Maximum physical address for 64-bit kernel with 4-level paging */ #define MAXMEM_X86_64_4LEVEL (1ull << 46) @@ -212,8 +213,8 @@ static void retrieve_apple_device_properties(struct boot_params *boot_params) } } -static void -adjust_memory_range_protection(unsigned long start, unsigned long size) +void efi_adjust_memory_range_protection(unsigned long start, + unsigned long size) { efi_status_t status; efi_gcd_memory_space_desc_t desc; @@ -267,35 +268,14 @@ adjust_memory_range_protection(unsigned long start, unsigned long size) } } -/* - * Trampoline takes 2 pages and can be loaded in first megabyte of memory - * with its end placed between 128k and 640k where BIOS might start. - * (see arch/x86/boot/compressed/pgtable_64.c) - * - * We cannot find exact trampoline placement since memory map - * can be modified by UEFI, and it can alter the computed address. - */ - -#define TRAMPOLINE_PLACEMENT_BASE ((128 - 8)*1024) -#define TRAMPOLINE_PLACEMENT_SIZE (640*1024 - (128 - 8)*1024) - void startup_32(struct boot_params *boot_params); static void setup_memory_protection(unsigned long image_base, unsigned long image_size) { - /* - * Allow execution of possible trampoline used - * for switching between 4- and 5-level page tables - * and relocated kernel image. - */ - - adjust_memory_range_protection(TRAMPOLINE_PLACEMENT_BASE, - TRAMPOLINE_PLACEMENT_SIZE); - #ifdef CONFIG_64BIT if (image_base != (unsigned long)startup_32) - adjust_memory_range_protection(image_base, image_size); + efi_adjust_memory_range_protection(image_base, image_size); #else /* * Clear protection flags on a whole range of possible @@ -305,8 +285,8 @@ setup_memory_protection(unsigned long image_base, unsigned long image_size) * need to remove possible protection on relocated image * itself disregarding further relocations. */ - adjust_memory_range_protection(LOAD_PHYSICAL_ADDR, - KERNEL_IMAGE_SIZE - LOAD_PHYSICAL_ADDR); + efi_adjust_memory_range_protection(LOAD_PHYSICAL_ADDR, + KERNEL_IMAGE_SIZE - LOAD_PHYSICAL_ADDR); #endif } @@ -804,6 +784,14 @@ void __noreturn efi_stub_entry(efi_handle_t handle, efi_dxe_table = NULL; } + if (IS_ENABLED(CONFIG_X86_64)) { + status = efi_setup_5level_paging(); + if (status != EFI_SUCCESS) { + efi_err("efi_setup_5level_paging() failed!\n"); + goto fail; + } + } + /* * If the kernel isn't already loaded at a suitable address, * relocate it. @@ -922,9 +910,12 @@ void __noreturn efi_stub_entry(efi_handle_t handle, goto fail; } - if (IS_ENABLED(CONFIG_X86_64)) + if (IS_ENABLED(CONFIG_X86_64)) { + efi_5level_switch(); + /* add offset of startup_64() */ bzimage_addr += 0x200; + } enter_kernel(bzimage_addr, boot_params); fail: diff --git a/drivers/firmware/efi/libstub/x86-stub.h b/drivers/firmware/efi/libstub/x86-stub.h new file mode 100644 index 0000000000000000..4190c881b4f27a41 --- /dev/null +++ b/drivers/firmware/efi/libstub/x86-stub.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include + +extern void trampoline_32bit_src(void *, bool); +extern const u16 trampoline_ljmp_imm_offset; + +void efi_adjust_memory_range_protection(unsigned long start, + unsigned long size); + +efi_status_t efi_setup_5level_paging(void); +void efi_5level_switch(void); From patchwork Mon May 22 07:14:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97114 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1264123vqo; Mon, 22 May 2023 00:22:48 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ59o5rRrkOSzfLpZZwCiE70kfODqWTCtlirDT8oFpt0k3V2Yy3J4X2YzKoHSNESh+AI83EP X-Received: by 2002:a05:6a21:6d86:b0:10a:e556:70ec with SMTP id wl6-20020a056a216d8600b0010ae55670ecmr6113336pzb.7.1684740168299; Mon, 22 May 2023 00:22:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684740168; cv=none; d=google.com; s=arc-20160816; b=izWUwWKWDlaIn5Q8wePPDR9R2+nirt/+0jbpEuWwwTgUw6UdwzjIl+VNv9ACG1aQHe inAme0fxtCZ1Xk1u9wIxbb8mJkEQEficHGk+7Qggb4JUIosQcp3PhZexBTrIfH+TKzNl Si12hDfeTsbZohaCZnZSw/9s11oUYjP+BJH7pevNj6sULBnq5lobjpb9xMHOjPmFOjvH eyizGNbq0/KqjPcXhtUyJbW7ZVOxkAbz3JNYHxeFjEOhNd0i17uJ+y/pafc7mPoB592z xbTbDWopmJD3vw/JWWbpvEJ6ZPdNKinvvUnwdqtnupkDC5zuuW4aPeAm0Y5irOf/GyyP yhqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=C3Bv8cUcHA/U96JPhUJqv2V3mQnBjuUWzbViEU2OyOo=; b=N/bYnS09n2ru5nF8FK0sJGJ+zy1r3kreARyN77Wjyzlq8kNJvlXg8j0M4RlOSpV16j vVSLe4E0ZeTcc4oe0JaT0nTPkcD+SrWd8hLuCoGEBxXoXV0MjOO8HKEcSe9/EWXNkoay o8WrGkW/+Co5zC8Zk6VLbWoT3X2lqbjnXwE+Km3ROL4evFoGYEsLqIrCH95Wd08u6oro xqvX+p/2EY5Z5QTQrQkDHUW8AQ2xwZ6bP/LqmWw10r3dbyosJJ4+L8+9UUypfAU4Rbg/ CyebwAEgJgYUBDBfftqKrZj+nO1ZxxRom2UYA4ovwsLP+5fC3P8BMaRFqmUMNjDWi2FW XzPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=M1Cqkfr0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x71-20020a63864a000000b0053b52fed717si3184011pgd.864.2023.05.22.00.22.32; Mon, 22 May 2023 00:22:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=M1Cqkfr0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232460AbjEVHQr (ORCPT + 99 others); Mon, 22 May 2023 03:16:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232483AbjEVHQI (ORCPT ); Mon, 22 May 2023 03:16:08 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DD8EE7E; Mon, 22 May 2023 00:15:33 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C96BB61DF3; Mon, 22 May 2023 07:15:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E058C433A1; Mon, 22 May 2023 07:15:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739732; bh=9rSHT6To50AeTePxnLLsddpAkNAPINJ4d8fPXZY5ZkI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M1Cqkfr0Z9+BTOytdvTGksJEHot4TmJ+SSmPbxA8uA0H0JIyxxXr7OrYxAThdfSBG iYflMl6nlU4kJa7YVVsFDDxSo3Ra8bxKCcoKC7q/k4BCvsnzX634ypUn0cHqaTHkVR dI8osqfR4xcpswOqzBhqtXf7Z3VV/+yL8Pq95Fd/dZAp/RtNNI/v7Q4/F8hNh140vO JhXdsG4ssDy37iY2/g3PC3rhfg/y3tgvz2LrhycqSPk3rkUIDJHCamrhbej2mKssnL /Kv+9i6uKQQRAvYrPNR4cLqIiJY3Ui1msoBE317GRBcyFoty923VZjR8uI0UIONO3f 3SIKPG1mpFOog== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 14/21] x86/efistub: Prefer EFI memory attributes protocol over DXE services Date: Mon, 22 May 2023 09:14:08 +0200 Message-Id: <20230522071415.501717-15-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3293; i=ardb@kernel.org; h=from:subject; bh=9rSHT6To50AeTePxnLLsddpAkNAPINJ4d8fPXZY5ZkI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzG71mWflt6IOMnrNKp69L/qBw85rtQ8dUnoi11ycK SDGfnJ9RykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiISh3DP7Up868GXGqfeiyx ft+nFzm7Gn/4NHqoKKilLbRZ58AT1sTwT2npF6eSaXWLZ/KqbN+UImqlvE30U35g2+EJXo4L5mQ 4cQAA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766578106671315929?= X-GMAIL-MSGID: =?utf-8?q?1766578106671315929?= Currently, the EFI stub relies on DXE services in some cases to clear non-execute restrictions from page allocations that need to be executable. This is dodgy, because DXE services are not specified by UEFI but by PI, and they are not intended for consumption by OS loaders. However, no alternative existed at the time. Now, there is a new UEFI protocol that should be used instead, so if it exists, prefer it over the DXE services calls. Signed-off-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/x86-stub.c | 29 ++++++++++++++------ 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 229def7d9b028ceb..fcdae5db0c63c7e5 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -26,6 +26,7 @@ const efi_system_table_t *efi_system_table; const efi_dxe_services_table_t *efi_dxe_table; u32 image_offset __section(".data"); static efi_loaded_image_t *image = NULL; +static efi_memory_attribute_protocol_t *memattr; static efi_status_t preserve_pci_rom_image(efi_pci_io_protocol_t *pci, struct pci_setup_rom **__rom) @@ -222,12 +223,18 @@ void efi_adjust_memory_range_protection(unsigned long start, unsigned long rounded_start, rounded_end; unsigned long unprotect_start, unprotect_size; - if (efi_dxe_table == NULL) - return; - rounded_start = rounddown(start, EFI_PAGE_SIZE); rounded_end = roundup(start + size, EFI_PAGE_SIZE); + if (memattr != NULL) { + efi_call_proto(memattr, clear_memory_attributes, rounded_start, + rounded_end - rounded_start, EFI_MEMORY_XP); + return; + } + + if (efi_dxe_table == NULL) + return; + /* * Don't modify memory region attributes, they are * already suitable, to lower the possibility to @@ -766,6 +773,7 @@ void __noreturn efi_stub_entry(efi_handle_t handle, efi_system_table_t *sys_table_arg, struct boot_params *boot_params) { + efi_guid_t guid = EFI_MEMORY_ATTRIBUTE_PROTOCOL_GUID; unsigned long bzimage_addr = (unsigned long)startup_32; unsigned long buffer_start, buffer_end; struct setup_header *hdr = &boot_params->hdr; @@ -777,13 +785,18 @@ void __noreturn efi_stub_entry(efi_handle_t handle, if (efi_system_table->hdr.signature != EFI_SYSTEM_TABLE_SIGNATURE) efi_exit(handle, EFI_INVALID_PARAMETER); - efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID); - if (efi_dxe_table && - efi_dxe_table->hdr.signature != EFI_DXE_SERVICES_TABLE_SIGNATURE) { - efi_warn("Ignoring DXE services table: invalid signature\n"); - efi_dxe_table = NULL; + if (IS_ENABLED(CONFIG_EFI_DXE_MEM_ATTRIBUTES)) { + efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID); + if (efi_dxe_table && + efi_dxe_table->hdr.signature != EFI_DXE_SERVICES_TABLE_SIGNATURE) { + efi_warn("Ignoring DXE services table: invalid signature\n"); + efi_dxe_table = NULL; + } } + /* grab the memory attributes protocol if it exists */ + efi_bs_call(locate_protocol, &guid, NULL, (void **)&memattr); + if (IS_ENABLED(CONFIG_X86_64)) { status = efi_setup_5level_paging(); if (status != EFI_SUCCESS) { From patchwork Mon May 22 07:14:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97126 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272007vqo; Mon, 22 May 2023 00:44:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5qyrQl3wwyr4vdp79dkE6M904uNI+xfC03FhcEOUFW4Ec5auRm0IR4gtwhVnjU68a4qv1E X-Received: by 2002:a17:902:f682:b0:1a6:db0a:8005 with SMTP id l2-20020a170902f68200b001a6db0a8005mr10775997plg.68.1684741441863; Mon, 22 May 2023 00:44:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741441; cv=none; d=google.com; s=arc-20160816; b=xEstC/mLogJinxPsQJF/9tKvq7cvgiKz8WzKT4DQlxPY+K3MhziuGHZT1cWBs5YqMA 45bFp+nqKbyaCbjdnSIMkeF9+imCma39p8k/y5fDvtFpMHTeW730OyNRAtRQGa27QRbq ZQpA1lewbw6pCfXrR72+D9vb6Nzg2WKDtF2LseBtnwcc0ReS1b1CGJ2niYfx86X1He9t H0u3e1EuoGHwNc8xk18WtnlMjfxwT/INW4NansDNeZ0aQ54Ru1dNSSFCab4052GJ+F9B OngOZ241Nk6C4h2N/SuWg0cwpGaD7PGVnnvDpaeQvfOe+ZsU5lslpYxdBl8Fjou8YNUG ZdHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=DB8oL3jXiN9e2dZTst6BBU9MW8a+CwDXQ55OFaZVfRU=; b=OxS6TB1X8GHMnOq/fLR5Rc9yGgN9A3ejYVvs+W3Y0QJD2wDDWRptgD67aHUO2kNJvt bmXDFH7R/hU4EZRBvW3SZ/QVf152WGjtGGxxA1HPoik4enydsL+DroWm2S6D3B+nH4zQ soTAAiG5UG424PerQhX+sGfWTl7aG4aYi+P6YXZJjw8vXtz5C916DT5X16JLmdorBbSk VEI84usTaa09HpK4tHuMevCHOSdVl3vDxZ6YxjgWdOMz26XoRxmynpE2yqXgajad7Odz ZBiPRYK0JEKJhxRDGLljG/LNZ/qKkekIZnj8HeDu4Dz2yeGs1RlojZhJmbZfXlJKcDtj HG0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="X/k25RQr"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 17-20020a170902ee5100b001a1a0db7f61si4161800plo.336.2023.05.22.00.43.32; Mon, 22 May 2023 00:44:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="X/k25RQr"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232541AbjEVHQ6 (ORCPT + 99 others); Mon, 22 May 2023 03:16:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232505AbjEVHQR (ORCPT ); Mon, 22 May 2023 03:16:17 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B68C10F4; Mon, 22 May 2023 00:15:37 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EBB5961DF4; Mon, 22 May 2023 07:15:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A567FC433EF; Mon, 22 May 2023 07:15:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739736; bh=TDhagb7/1BjVNI9fsBOdvguiTH1+pJKnlE7Km5AKGt0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X/k25RQrwho388uZuEtlF6MSH2EBdQnJmyiy5r1iU9TgEL5+t4CYqaj50iQtt75+u Qbqr1HwaRmOIAA7tpZ2RXqLdwR+pmCgRoTpQ6XB2SI4yYXYOKyKS3ruIbT3aCYv9Gm BadBzyHNU71Cut4XjqHwR/oPZVBfQU9FQAGzoR6jxLozp44/PzAqiTLdjLjyhIXzDL 2pvV5pxM0B0NfsXs9YF5SA7j7oe3OPbaAxx/DGmNIG9Y0mefuXpPuEtREMYvLqBMIB 2AWA8HVb9mCkDTj6/ng8cl4nwinPhOagqzaeX5vF7+z8TPN4iSsGLek5JCSV3LQCaZ KMKC1JT1BxKsA== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 15/21] decompress: Use 8 byte alignment Date: Mon, 22 May 2023 09:14:09 +0200 Message-Id: <20230522071415.501717-16-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=737; i=ardb@kernel.org; h=from:subject; bh=TDhagb7/1BjVNI9fsBOdvguiTH1+pJKnlE7Km5AKGt0=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzN4x+Byz1Ywd4vxs4ZLqct3rK24U3ZBefYk5ou4gx 1T+gDsdpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCJuFowMM8Uapp2MfjA5Yvqt yGfuB5VM3gd1rytn3aMTdf9rqf23n4wMH9Xfv3r8/8FEkdVGScfsb/9YxDCrYqv8iqazS94dmdj 5ghEA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579442090555125?= X-GMAIL-MSGID: =?utf-8?q?1766579442090555125?= The ZSTD decompressor requires malloc() allocations to be 8 byte aligned, so ensure that this the case. Signed-off-by: Ard Biesheuvel --- include/linux/decompress/mm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/decompress/mm.h b/include/linux/decompress/mm.h index 9192986b1a731323..ac862422df158bef 100644 --- a/include/linux/decompress/mm.h +++ b/include/linux/decompress/mm.h @@ -48,7 +48,7 @@ MALLOC_VISIBLE void *malloc(int size) if (!malloc_ptr) malloc_ptr = free_mem_ptr; - malloc_ptr = (malloc_ptr + 3) & ~3; /* Align */ + malloc_ptr = (malloc_ptr + 7) & ~7; /* Align */ p = (void *)malloc_ptr; malloc_ptr += size; From patchwork Mon May 22 07:14:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97130 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272601vqo; Mon, 22 May 2023 00:45:36 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6XQ5z9ukQDXnkIt6NLCWldcLiXndWHtAQtROYi67CyS/+C7KdfWmqpc94x7RfNFw1gFuSG X-Received: by 2002:a05:6a21:6811:b0:106:c9b7:c932 with SMTP id wr17-20020a056a21681100b00106c9b7c932mr4430836pzb.1.1684741536614; Mon, 22 May 2023 00:45:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741536; cv=none; d=google.com; s=arc-20160816; b=fhZoZeOZIFp2etdgSX3Xv9yA2D8n+SYhItidrep4YJdvhCoCJ/Fpclf5/kwsRsldj3 t43dN7mg6BsHAIHNl/BG7zGxiavj7CAhW9o2q/UN3KrLKWmaUITSeZnznYznMdOOIRjy WD1gQT4LqFkVt6LUZMNIHN+NxIApnvydULGDdsNZhKapbJCTCSsKfcqIeEdW9CR+1VKj NQrU1Of3EgRoWeXa5u7UDZ2dSX5agMl6gZegO8peUfpSaG2fy5dnTVaoRGw2jaJsEopK A4GAZBQAowM+y4udWCk2UhYe57vPw8sD8+QK1xjglSPmfwOa4PmUbMDxysNUxVZfFm4H /Nhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nEzGCrqtPMvKAlAsmeFYNAjbdU1NAif1Mw7xsg8ylaY=; b=o8rqGtmK0X8iuK3P2tCVfiW7Z+bCUB2fVeb3P7HB7jHmV3GeCN1C/i0t/AABChqMN5 E/MH//9hOuhCWKON9XA5MY7irGNAidvlO6cyuXyGu+DVjYKQGYGrh4Xihj9AqlIFBfvx 4FZF74iE1ZHSNEm60ChGk7gGTcFLcQZ0NicZGrAN9XCuW6oFnN/Ds4H5CmPLbysO59Tt XQaMu8TUDth9jOi2X51rEmG2dkzWqwbAGmsWhyR6pEDED4mZUMqLcWM24bGZqMGuGBi8 s3Inro30oejb1Y1KwP55LwJ9akKxI9MZHalVsV1Fl8vSk5WsRokzrYbV4f8a+DNRXPRv Ojcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=mirx97l6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g10-20020a636b0a000000b005348954e079si4427993pgc.95.2023.05.22.00.45.24; Mon, 22 May 2023 00:45:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=mirx97l6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232528AbjEVHRL (ORCPT + 99 others); Mon, 22 May 2023 03:17:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232476AbjEVHQj (ORCPT ); Mon, 22 May 2023 03:16:39 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF9E6170E; Mon, 22 May 2023 00:15:41 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1908961DF1; Mon, 22 May 2023 07:15:41 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CB936C4339E; Mon, 22 May 2023 07:15:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739740; bh=g3nSTumb/KYsp4GqRL0zJ1a7x2cI2HwxP02u+R5/mbI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mirx97l6GWeBA0OhV7TsvAblRauu2aWVvH38JpIG2zKIul+qU7f7MRIgo0GUnbemS qfLTjHv2YeUlFQHRaerfKF5pmzi1/odSwseeo2YmczUaGASU3y8null0bQphft+8xx RdBO6CCwWqblZD0t5wobdOgyDF888fx+hGQ5tnIWAp5PPktseCAsqL3xKFnPV3+57m d+jGNmqd001KMdQrQrfPAC4DnaqZFr5C6NEJ7xbB6LnPpyCICPdC5zIgppJ7eZgFco QcqnjtXwrECw8JDEUv/xXJYl+iJLuyktvL4R5QCSYl/EmnKO7pMPudypMuzhB6kWNT U5kBpXJjGNTNQ== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 16/21] x86/decompressor: Move global symbol references to C code Date: Mon, 22 May 2023 09:14:10 +0200 Message-Id: <20230522071415.501717-17-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5078; i=ardb@kernel.org; h=from:subject; bh=g3nSTumb/KYsp4GqRL0zJ1a7x2cI2HwxP02u+R5/mbI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzKFo2l9xTa26d75v9rwIzo5Ydy/OZve//yZNFamuE 06pdfJ0lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgInUSjAy7D6ZnzFzt9KtU3FW 0bFpX5a6CqtMvfh37t56xS9f0684NDD802tN7i2ZHW0muOyybCmDTICE6MUzu6Y/mSkjH1J/zE2 CAwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579541228554010?= X-GMAIL-MSGID: =?utf-8?q?1766579541228554010?= It is no longer necessary to be cautious when referring to global variables in the position independent decompressor code, now that it is built using PIE codegen and makes an assertion in the linker script that no GOT entries exist (which would require adjustment for the actual runtime load address of the decompressor binary). This means global variables can be referenced directly from C code, instead of having to pass their runtime addresses into C routines from asm code, which needs to happen at each call site. Do so for the code that will be called directly from the EFI stub after a subsequent patch, and avoid the need to duplicate this logic a third time. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_32.S | 8 -------- arch/x86/boot/compressed/head_64.S | 8 +------- arch/x86/boot/compressed/misc.c | 16 +++++++++------- 3 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 3530465b5b85ccf3..beee858058df4403 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -168,13 +168,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) */ /* push arguments for extract_kernel: */ - pushl output_len@GOTOFF(%ebx) /* decompressed length, end of relocs */ pushl %ebp /* output address */ - pushl input_len@GOTOFF(%ebx) /* input_len */ - leal input_data@GOTOFF(%ebx), %eax - pushl %eax /* input_data */ - leal boot_heap@GOTOFF(%ebx), %eax - pushl %eax /* heap area */ pushl %esi /* real mode pointer */ call extract_kernel /* returns kernel entry point in %eax */ addl $24, %esp @@ -202,8 +196,6 @@ SYM_DATA_END_LABEL(gdt, SYM_L_LOCAL, gdt_end) */ .bss .balign 4 -boot_heap: - .fill BOOT_HEAP_SIZE, 1, 0 boot_stack: .fill BOOT_STACK_SIZE, 1, 0 boot_stack_end: diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index b5bd6be035a7b7ec..3074d278c7e665d8 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -493,11 +493,7 @@ SYM_FUNC_START_LOCAL_NOALIGN(.Lrelocated) * Do the extraction, and jump to the new kernel.. */ movq %r15, %rdi /* pass struct boot_params pointer */ - leaq boot_heap(%rip), %rsi /* malloc area for uncompression */ - leaq input_data(%rip), %rdx /* input_data */ - movl input_len(%rip), %ecx /* input_len */ - movq %rbp, %r8 /* output target address */ - movl output_len(%rip), %r9d /* decompressed length, end of relocs */ + movq %rbp, %rsi /* output target address */ call extract_kernel /* returns kernel entry point in %rax */ /* @@ -636,8 +632,6 @@ SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end) */ .bss .balign 4 -SYM_DATA_LOCAL(boot_heap, .fill BOOT_HEAP_SIZE, 1, 0) - SYM_DATA_START_LOCAL(boot_stack) .fill BOOT_STACK_SIZE, 1, 0 .balign 16 diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 36535a3753f5d5fa..ad7a2297c9e186df 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -330,6 +330,11 @@ static size_t parse_elf(void *output) return ehdr.e_entry - LOAD_PHYSICAL_ADDR; } +static u8 boot_heap[BOOT_HEAP_SIZE] __aligned(4); + +extern unsigned char input_data[]; +extern unsigned int input_len, output_len; + /* * The compressed kernel image (ZO), has been moved so that its position * is against the end of the buffer used to hold the uncompressed kernel @@ -347,14 +352,11 @@ static size_t parse_elf(void *output) * |-------uncompressed kernel image---------| * */ -asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, - unsigned char *input_data, - unsigned long input_len, - unsigned char *output, - unsigned long output_len) +asmlinkage __visible void *extract_kernel(void *rmode, unsigned char *output) { const unsigned long kernel_total_size = VO__end - VO__text; unsigned long virt_addr = LOAD_PHYSICAL_ADDR; + memptr heap = (memptr)boot_heap; unsigned long needed_size; size_t entry_offset; @@ -412,7 +414,7 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, * entries. This ensures the full mapped area is usable RAM * and doesn't include any reserved areas. */ - needed_size = max(output_len, kernel_total_size); + needed_size = max((unsigned long)output_len, kernel_total_size); #ifdef CONFIG_X86_64 needed_size = ALIGN(needed_size, MIN_KERNEL_ALIGN); #endif @@ -443,7 +445,7 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, #ifdef CONFIG_X86_64 if (heap > 0x3fffffffffffUL) error("Destination address too large"); - if (virt_addr + max(output_len, kernel_total_size) > KERNEL_IMAGE_SIZE) + if (virt_addr + needed_size > KERNEL_IMAGE_SIZE) error("Destination virtual address is beyond the kernel mapping area"); #else if (heap > ((-__PAGE_OFFSET-(128<<20)-1) & 0x7fffffff)) From patchwork Mon May 22 07:14:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97133 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272651vqo; Mon, 22 May 2023 00:45:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ59bJfJN3Z5p1p8rPLvB6pnInuApsbW6p8QfMfXodL5qqx366jI/9DSVyTKF4mV3OtkKt2z X-Received: by 2002:a17:902:b411:b0:1aa:d866:33cf with SMTP id x17-20020a170902b41100b001aad86633cfmr8994890plr.56.1684741542224; Mon, 22 May 2023 00:45:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741542; cv=none; d=google.com; s=arc-20160816; b=WipIIEaBSSoIdWIvueKuHEZptEugru+QO2bWs6wh5XUcXkpvM17xHMUoPAu1Pc1Ygk WB6p20fHwlW2Hvg6Vq+qqrgiGFs2OGzLwKSIAkKW+a4/wdEudIu/FQ3+o796oTelMIO1 J4DqaPi5ZpJwWvSD30LOCY3Mqj9Ttt3JjPWFkpzuxzw0EV2DohhZ7hiBlhW1ei1MTuOW Ma1iadjPV77H4Uxcj/VeKnJ631nJU53hP3Rlo5dit9h+ic+snqf0KFAMuUEOjFMXcf4s 9QQhkmoFluWj5uwGkZHc2NCY8+VUPqI3O5P3R5gnHtc1HonF/KM8+neBwetjK34Rvs+Y LnIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Mn7RXE41n21od5stVTbucM8IPehZAKJb7S+xRWs313U=; b=D43WjcGOL9KxhFwmQP1Fz2Efyn68t+ClAfe9PyW3qY/vW46jKyf7w7cZsH6glk7RnV q8+t0wvRgHXoeeTXXNXIEkH36sFnZLCg67LE21Awff0TKCvD5tRZ9+yO0jtcUY3yJ87x hSlt8tSar2Q37IdllzmsVOwC9ttzzQ3Cp/HvdyarCsvKNW2FwnAGEQvHwqmzlyPVOiDz vNMaTNYvaj5mCjq3mXIPAqrdoQ6fdb53DvKG+yDSyH1JE/Ag2x1ljnrJ8TpV/SOxGrwC s32yi+LKw6uRWRb6N/G95xjowwPyz+YfBTnzLZTOfQ/xfz+ekt++J8TTQ1mUF4a3TYrd 7U6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=rr136Js0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 9-20020a170902c14900b001a1a9a1d336si4238083plj.206.2023.05.22.00.45.30; Mon, 22 May 2023 00:45:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=rr136Js0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232479AbjEVHRY (ORCPT + 99 others); Mon, 22 May 2023 03:17:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232500AbjEVHQy (ORCPT ); Mon, 22 May 2023 03:16:54 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84C62DB; Mon, 22 May 2023 00:15:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4293261DD9; Mon, 22 May 2023 07:15:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0AA3C433D2; Mon, 22 May 2023 07:15:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739744; bh=U1WWTB8Ed6jF5PagzQzhXVR5oyHC5bjwVmmRHfA2IP0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rr136Js0c7uMMk3uXQJZmoR+32dVkdLQdERBEOeclNrZle0SdfRIkLTAsRQdO77OR Ssee8IShaZWdYn3ltkiVok+AZkNQOFffZDPdO/WoMgLPJd5X1jmTygJgjNnIoSMDlw a938/yTL505LpkDrgCrhmTZvwidrQ27HKpIcXyWJ/mPtvQWfa8Tv/nfk9+ohhlwbCh Ox0Q2wYUfBL9gmwcNmAeeAbXfAW83O9H8eah3uMh9ywfT3FtxRL+dzCiJCfBXEaJrr flnvFdWOkh9mDJ5L5+m51UyipTj/owTVQQw/OURG9K5yog28dyhUW1cnLwDwh+S29T W8bd/ajAXFM7Q== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 17/21] x86/decompressor: Factor out kernel decompression and relocation Date: Mon, 22 May 2023 09:14:11 +0200 Message-Id: <20230522071415.501717-18-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2968; i=ardb@kernel.org; h=from:subject; bh=U1WWTB8Ed6jF5PagzQzhXVR5oyHC5bjwVmmRHfA2IP0=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzInp2qXX4X8a2I+/eyMtFH7p1BEGxr2LvH6UaC27O y/lfuHXjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRtOkM/+s0DRi736y1lWna uO3hvhy7XN5lpWtPqb3582tOtcwBoxcM/2v/zm2ZMdt7s6f9XN93Af9dVy39UvvofmyPo2eSzcL wFHYA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579547199032482?= X-GMAIL-MSGID: =?utf-8?q?1766579547199032482?= Factor out the decompressor sequence that invokes the decompressor, parses the ELF and applies the relocations so that it can be called directly from the EFI stub. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/misc.c | 28 ++++++++++++++++---- arch/x86/include/asm/boot.h | 8 ++++++ 2 files changed, 31 insertions(+), 5 deletions(-) diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index ad7a2297c9e186df..831485a66300319e 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -330,11 +330,33 @@ static size_t parse_elf(void *output) return ehdr.e_entry - LOAD_PHYSICAL_ADDR; } +const unsigned long kernel_total_size = VO__end - VO__text; + static u8 boot_heap[BOOT_HEAP_SIZE] __aligned(4); extern unsigned char input_data[]; extern unsigned int input_len, output_len; +unsigned long decompress_kernel(unsigned char *outbuf, unsigned long virt_addr, + void (*error)(char *x)) +{ + unsigned long entry; + + if (!free_mem_ptr) { + free_mem_ptr = (unsigned long)boot_heap; + free_mem_end_ptr = (unsigned long)boot_heap + sizeof(boot_heap); + } + + if (__decompress(input_data, input_len, NULL, NULL, outbuf, output_len, + NULL, error) < 0) + return ULONG_MAX; + + entry = parse_elf(outbuf); + handle_relocations(outbuf, output_len, virt_addr); + + return entry; +} + /* * The compressed kernel image (ZO), has been moved so that its position * is against the end of the buffer used to hold the uncompressed kernel @@ -354,7 +376,6 @@ extern unsigned int input_len, output_len; */ asmlinkage __visible void *extract_kernel(void *rmode, unsigned char *output) { - const unsigned long kernel_total_size = VO__end - VO__text; unsigned long virt_addr = LOAD_PHYSICAL_ADDR; memptr heap = (memptr)boot_heap; unsigned long needed_size; @@ -464,10 +485,7 @@ asmlinkage __visible void *extract_kernel(void *rmode, unsigned char *output) accept_memory(__pa(output), __pa(output) + needed_size); } - __decompress(input_data, input_len, NULL, NULL, output, output_len, - NULL, error); - entry_offset = parse_elf(output); - handle_relocations(output, output_len, virt_addr); + entry_offset = decompress_kernel(output, virt_addr, error); debug_putstr("done.\nBooting the kernel (entry_offset: 0x"); debug_puthex(entry_offset); diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index 9191280d9ea3160d..4ae14339cb8cc72d 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -62,4 +62,12 @@ # define BOOT_STACK_SIZE 0x1000 #endif +#ifndef __ASSEMBLY__ +extern unsigned int output_len; +extern const unsigned long kernel_total_size; + +unsigned long decompress_kernel(unsigned char *outbuf, unsigned long virt_addr, + void (*error)(char *x)); +#endif + #endif /* _ASM_X86_BOOT_H */ From patchwork Mon May 22 07:14:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97115 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1264943vqo; Mon, 22 May 2023 00:24:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4rWmmtII8ZO0V1ZtUMhM5sSadbbQ9OT8o31Pn/iqUHaL+8fhCd8Nt78NgKAFPX2udSWAfb X-Received: by 2002:a17:902:f7c6:b0:1ae:3036:b594 with SMTP id h6-20020a170902f7c600b001ae3036b594mr9206573plw.49.1684740295976; Mon, 22 May 2023 00:24:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684740295; cv=none; d=google.com; s=arc-20160816; b=rxNrl5eCijwzkF3yoqJP80NzK+pfNb9G/Yukm9QfoAcqazPSd75nOdsWwOZdn1zk6B LcUcsojlVVKajANMH/wdj77ymFbSA1wAfPxGoQvfk5E6LHHmljE5ZXbwkZu3NaNkDF6h nJd8W46M+BpLdYlLjrarcSr6g73dxsiLGkI5h+fexfHBI59PQOKL2Xgk51bmXpEIfM0j mGQ5K5SVbMXoQS4XrAUKonH7uUgiGPw1jGUWrvbo680ulGgN0fs8FtgHzEM7WdYvGDip LuGycFfjIrT2gv/pQk5S4iTwq8oFXkJIB9ZbLklRDBy1EPItiYtK9JrY/eaAq/mDU1+/ SNKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6rKAANIqTwIC8Ceh4KPhaYfQQa+5WnOGnakTa9Yj4l4=; b=VDC+ySDvBpjQcStKAQ7YHkGBti8++x6Mz3SFM1BmFDXzOrLS5eFvyzMTrgL2eo/jH1 3urpUeJWjvpvnp67yAdAam/KJQk+XGw3a/ocR/VIp0CYfF9LHaJ52TrmQ391hIFEJd5c UTIUSA8hm14Bes5V/ptMBhms5dtkLCYDhx55SHC8uI8HqZ+zIK0ZcdNsvQtJp73ZtS3w PNGK++j6c3B6Tkq5VAF2YUfEtOFOZczeN85WXYE+ODvrxlVs/RXIfjcRdOwFQ3K0rd+z UEtulaYOMM391tmXag4rWluRYygNl3E63paPMyBkAL8pZHJJi9BJM4qQJOwasZ/cuDA1 H7jg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=MeHO95+f; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q7-20020a170902a3c700b001ae533bccc7si4017986plb.322.2023.05.22.00.24.40; Mon, 22 May 2023 00:24:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=MeHO95+f; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232575AbjEVHR3 (ORCPT + 99 others); Mon, 22 May 2023 03:17:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232554AbjEVHQ7 (ORCPT ); Mon, 22 May 2023 03:16:59 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA313198E; Mon, 22 May 2023 00:15:52 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 66F3F61DFD; Mon, 22 May 2023 07:15:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 219F7C433A1; Mon, 22 May 2023 07:15:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739748; bh=XRnDiVEqXvkUypxOdLrzhfuiI1tjnMjnlxv76SViP7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MeHO95+fyW7ojbM1paHarGzzSqOBIfjxJ+yuheV73DAwvKeV08JYidpNzHaWZwuEg JsSeNEaKKmsXAw4HKwtOf5/AbOoeBxeRgkYcrkagdrmYxA8BFQyONaXX9ccphwv/s+ 6Ve8Wrvb5VjOKUh854UpOzm3GlqZavvhxINXZek4X+Aj6O2fKo05MXnihbwHMp/LDW 8EzhuHUU3XFVvk8yDQ6eMa12fYUToQbz9Xy3WDR+7Su3ITXzfPxAwR0DtgBKBA4gxP nJakjQDwZPHQ2RGHwTQvUGfTuf83W4djYeQkiSVqFtAeuDruz1hRuiwhE7+TkmIn47 MLycnjwgkG3Pg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 18/21] x86/head_64: Store boot_params pointer in callee-preserved register Date: Mon, 22 May 2023 09:14:12 +0200 Message-Id: <20230522071415.501717-19-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2440; i=ardb@kernel.org; h=from:subject; bh=XRnDiVEqXvkUypxOdLrzhfuiI1tjnMjnlxv76SViP7E=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzLlJ7ljnYiGh8BOsHcWd/55eUpsgaWUqviFc6vzTO wrbz1/oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABNpYmb4p5rbsdXfZndEyfJ/ kzQzpFoTKtpDvRQETDetu66UVX7lPMM/g/ezCpJcgz48fdXGI9W4aKn/5k1HdnQf6T4nrar5ZY4 FMwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766578240377067662?= X-GMAIL-MSGID: =?utf-8?q?1766578240377067662?= Instead of pushing/popping %RSI to/from the stack every time a function is called from startup_64(), store it in a callee preserved register and grab it from there when its value is actualled needed. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index a5df3e994f04f10f..95b12fdae10e1dc9 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -60,6 +60,7 @@ SYM_CODE_START_NOALIGN(startup_64) * compiled to run at we first fixup the physical addresses in our page * tables and then reload them. */ + mov %rsi, %r15 /* Preserve boot_params pointer */ /* Set up the stack for verify_cpu() */ leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp @@ -73,9 +74,7 @@ SYM_CODE_START_NOALIGN(startup_64) shrq $32, %rdx wrmsr - pushq %rsi call startup_64_setup_env - popq %rsi #ifdef CONFIG_AMD_MEM_ENCRYPT /* @@ -84,10 +83,8 @@ SYM_CODE_START_NOALIGN(startup_64) * which needs to be done before any CPUID instructions are executed in * subsequent code. */ - movq %rsi, %rdi - pushq %rsi + movq %r15, %rdi call sme_enable - popq %rsi #endif /* Now switch to __KERNEL_CS so IRET works reliably */ @@ -109,9 +106,7 @@ SYM_CODE_START_NOALIGN(startup_64) * programmed into CR3. */ leaq _text(%rip), %rdi - pushq %rsi call __startup_64 - popq %rsi /* Form the CR3 value being sure to include the CR3 modifier */ addq $(early_top_pgt - __START_KERNEL_map), %rax @@ -200,10 +195,8 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) * %rsi carries pointer to realmode data and is callee-clobbered. Save * and restore it. */ - pushq %rsi movq %rax, %rdi call sev_verify_cbit - popq %rsi /* * Switch to new page-table @@ -294,9 +287,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) wrmsr /* Setup and Load IDT */ - pushq %rsi call early_setup_idt - popq %rsi /* Check if nx is implemented */ movl $0x80000001, %eax @@ -334,7 +325,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* rsi is pointer to real mode structure with interesting info. pass it to C */ - movq %rsi, %rdi + movq %r15, %rdi .Ljump_to_C_code: /* From patchwork Mon May 22 07:14:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97118 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1265483vqo; Mon, 22 May 2023 00:26:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6UpXhY3zSnO+y3fNpGxq0mBQBz4fbVCbNkJkQ1b2r3znz+e6Z+mi1sS07Nr7z1FaQtxDoo X-Received: by 2002:a17:90a:2e12:b0:253:3ce4:b421 with SMTP id q18-20020a17090a2e1200b002533ce4b421mr9180801pjd.1.1684740388390; Mon, 22 May 2023 00:26:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684740388; cv=none; d=google.com; s=arc-20160816; b=u9c/XGbeVRtSceiFh94NmWkxaWAjKoz1KEaZMGHPoFaeMybjD12iulEbqv0Q/MiK6m tuuTivXBPBCVLD+TpE1x1aR78TzIUi5SjGYXMDHFQZ75yx75G6Na51doQZ1yHMUBUI+O QeyU8FfppnWmhqOR4qyM71/RBEI9QutXWvqlg/A4iDrWkJ7PYOA7F+8sCrTpl7wTWSZZ +MPL2OynSa+yL9zBgIhPHZudC/1oxNPvMCNXXSZZcIypKIFv3lblAO0rHODaRmNSil4q WSK0nN7ja+8cRrTwDu0O+LWCsKOFmqawQ+jQiJf5yHGa9JA0PDJcgttWNLkctCYP+f/f K8HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=U22TslAsOouX8iihIejwgc+OY0cfIU4IwzZif0GRiQ4=; b=OVZhqT1YlPU4/RSS9BfOiNUFX4QoasSMJM8RtkZr6Km7qjXn0cNMB3h592IlG4Mnau QQBuoW7VuGZiVdSDXGjr8IIU6oYP2Ep9QSS5AOdDSufwGUWeYt5zB791Z6B0vFR1O5Co zwf4EpXhykJdzD+dm9+8ZBGqn1IxcJC4k+MCnwYIpGSDyg9NRDUisTKgbell2I7qsV0X 2HkfMBesEO9S+qRevhOcDAbXx9xU6OWaWbd8xXHuWSj066Z8w93xRI/iOiSbABGxR7a4 mvz0skzlXqKhhdXNGPcVfUbXHk5TKRqdLNOiPywFfnfb7Ivaio6O9lp6HVDiLn71tHDN tMlQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=E2wFUzID; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g20-20020a17090ace9400b002528393e899si6293697pju.164.2023.05.22.00.26.13; Mon, 22 May 2023 00:26:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=E2wFUzID; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232582AbjEVHRc (ORCPT + 99 others); Mon, 22 May 2023 03:17:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232556AbjEVHQ7 (ORCPT ); Mon, 22 May 2023 03:16:59 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBFBA1992; Mon, 22 May 2023 00:15:53 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 96A0661DF3; Mon, 22 May 2023 07:15:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 468FBC433D2; Mon, 22 May 2023 07:15:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739753; bh=H6tv4XyD73jIrY44u3Vc1zagcnctlaNlh/vBGAcqcQA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=E2wFUzIDAOR6QyayDFrUO9ggUdmGt8ky4ywfcVIHBcYq8msVT7NL6QpGGGDVSh/RI 20HsTn93M0lS/Sd3lh0s2r2VTrLh6WeGNHXsMWsiGH7NTvS1vn5PyVGFE3C9/jJZdO fYqUB4ZbL2zsYDUnZqnLXW2LVjXn6cdb2M37BnJGPqZiLw4XQgXiXrZ8Jmijgn76x3 NDuukC4smqAgp+wmFMzDcPc3TCmyXgZ18a6A5y+p23h8+ZZCn9tU3dwia71EwobFg7 t82HPs86YGhxv7MBiGF4EE7xI+SEuB3vwjuy9skQ7VDfJt5ZktoEWiMT3ZZd0EHOrA Dgf/JRaW1/qLg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 19/21] efi/libstub: Add limit argument to efi_random_alloc() Date: Mon, 22 May 2023 09:14:13 +0200 Message-Id: <20230522071415.501717-20-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3943; i=ardb@kernel.org; h=from:subject; bh=H6tv4XyD73jIrY44u3Vc1zagcnctlaNlh/vBGAcqcQA=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzGXyAoneQ88jpqsqesR/2Hfc/2GR9mzPD9eXB70w3 6Mkrruvo5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzErY6RYSlPlnbe9/Kw6zyn 1D/uEP7OpRwSLrroaS6brFLgiY1xAowMH04Kzs8+xefMxCDU96iR7ZeaykI32cdSsc+u/JVR/GX OCwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766578337600744942?= X-GMAIL-MSGID: =?utf-8?q?1766578337600744942?= x86 will need to limit the kernel memory allocation to the lowest 512 MiB of memory, to match the behavior of the existing bare metal KASLR physical randomization logic. So in preparation for that, add a limit parameter to efi_random_alloc() and wire it up. Signed-off-by: Ard Biesheuvel --- drivers/firmware/efi/libstub/arm64-stub.c | 2 +- drivers/firmware/efi/libstub/efistub.h | 2 +- drivers/firmware/efi/libstub/randomalloc.c | 10 ++++++---- drivers/firmware/efi/libstub/zboot.c | 2 +- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c index 770b8ecb73984c61..8c40fc89f5f99209 100644 --- a/drivers/firmware/efi/libstub/arm64-stub.c +++ b/drivers/firmware/efi/libstub/arm64-stub.c @@ -106,7 +106,7 @@ efi_status_t handle_kernel_image(unsigned long *image_addr, */ status = efi_random_alloc(*reserve_size, min_kimg_align, reserve_addr, phys_seed, - EFI_LOADER_CODE); + EFI_LOADER_CODE, EFI_ALLOC_LIMIT); if (status != EFI_SUCCESS) efi_warn("efi_random_alloc() failed: 0x%lx\n", status); } else { diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h index 191698e8489d82e7..e90b8d1d5c7e8fd4 100644 --- a/drivers/firmware/efi/libstub/efistub.h +++ b/drivers/firmware/efi/libstub/efistub.h @@ -956,7 +956,7 @@ efi_status_t efi_get_random_bytes(unsigned long size, u8 *out); efi_status_t efi_random_alloc(unsigned long size, unsigned long align, unsigned long *addr, unsigned long random_seed, - int memory_type); + int memory_type, unsigned long alloc_limit); efi_status_t efi_random_get_seed(void); diff --git a/drivers/firmware/efi/libstub/randomalloc.c b/drivers/firmware/efi/libstub/randomalloc.c index 32c7a54923b4c127..674a064b8f7adc68 100644 --- a/drivers/firmware/efi/libstub/randomalloc.c +++ b/drivers/firmware/efi/libstub/randomalloc.c @@ -16,7 +16,8 @@ */ static unsigned long get_entry_num_slots(efi_memory_desc_t *md, unsigned long size, - unsigned long align_shift) + unsigned long align_shift, + u64 alloc_limit) { unsigned long align = 1UL << align_shift; u64 first_slot, last_slot, region_end; @@ -29,7 +30,7 @@ static unsigned long get_entry_num_slots(efi_memory_desc_t *md, return 0; region_end = min(md->phys_addr + md->num_pages * EFI_PAGE_SIZE - 1, - (u64)EFI_ALLOC_LIMIT); + alloc_limit); if (region_end < size) return 0; @@ -54,7 +55,8 @@ efi_status_t efi_random_alloc(unsigned long size, unsigned long align, unsigned long *addr, unsigned long random_seed, - int memory_type) + int memory_type, + unsigned long alloc_limit) { unsigned long total_slots = 0, target_slot; unsigned long total_mirrored_slots = 0; @@ -76,7 +78,7 @@ efi_status_t efi_random_alloc(unsigned long size, efi_memory_desc_t *md = (void *)map->map + map_offset; unsigned long slots; - slots = get_entry_num_slots(md, size, ilog2(align)); + slots = get_entry_num_slots(md, size, ilog2(align), alloc_limit); MD_NUM_SLOTS(md) = slots; total_slots += slots; if (md->attribute & EFI_MEMORY_MORE_RELIABLE) diff --git a/drivers/firmware/efi/libstub/zboot.c b/drivers/firmware/efi/libstub/zboot.c index e5d7fa1f1d8fd160..bdb17eac0cb401be 100644 --- a/drivers/firmware/efi/libstub/zboot.c +++ b/drivers/firmware/efi/libstub/zboot.c @@ -119,7 +119,7 @@ efi_zboot_entry(efi_handle_t handle, efi_system_table_t *systab) } status = efi_random_alloc(alloc_size, min_kimg_align, &image_base, - seed, EFI_LOADER_CODE); + seed, EFI_LOADER_CODE, EFI_ALLOC_LIMIT); if (status != EFI_SUCCESS) { efi_err("Failed to allocate memory\n"); goto free_cmdline; From patchwork Mon May 22 07:14:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97127 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1272167vqo; Mon, 22 May 2023 00:44:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5KQaF6Bu8/vBawry64bDvCkSoucN8Hx853zLgQ4DQXFeCGe7EY3z3POOBw3nJY3VsJ8k2s X-Received: by 2002:a05:6a00:16cc:b0:63b:8a91:e641 with SMTP id l12-20020a056a0016cc00b0063b8a91e641mr13720315pfc.11.1684741468055; Mon, 22 May 2023 00:44:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684741468; cv=none; d=google.com; s=arc-20160816; b=Q0nlm16MkkqE3z5Fp6OHJjIKfMeaWaUPAtBjeDoXrSNqwhGyexTJ7450/fgtlDKACO +oLdri7ODNh7pN+RBKphfsPrT+7q+8rpLTiLIfvrx1luBJBvhD0o84RWMv22iMh6qLp6 KWS0//6mBgKpVI8lBW76cB1/+aEn+0P1Jomu31D+2sI8a2oB0Qhg6QlZYaz+/fb741rd QdEIkN7HpGvF5dCrKfLs4IRC2i6AD4DCzRrFBTxWHOrct/pNrBfaU6nZCYEH3scYNPz3 c4zoptyIaR6oEN7rHAs7kfNOX6+OiRacWTKG/u7J3A+psTetEyQ6RmTi7+RI0EKhSZMH rwgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QFywEtNi6NYV4asW3p9U0VQVq5gDOqeDiImDhJeWp1o=; b=vF9ui7L2cvZn8ol0DDW1611dBdFRjZxTzw4ABUH6vsLhWvbyitmgYOKCObw9CCniNR ND6Pku5p8v9JPBXpQDbwHJwfwf0nDOyeHcUxBcT6k7Apz6ti0yv4Z2fbQqkAEPMBApQh dpBloF3ZhYvZFPEcX5Mo7RNu+lar3/C4yrryrDLlYQhpiZL+AVGKbAnOisCHiMrhv/Wj zDJB3/igEKSEcOTaonhwY2kepXm1AGPPx8ZmykNmuPygbaNyCc3gtOntbMpq0AZ5ntzH 9WKKyqsCdUeyh/YSDQAje9u//1CzhEUedtSFekwLVgf27QyVa2lof5juyJVSRE+I+xvn B9kw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lXnVT3BB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u72-20020a63794b000000b005130f15c5b1si4436071pgc.363.2023.05.22.00.44.15; Mon, 22 May 2023 00:44:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lXnVT3BB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232562AbjEVHRw (ORCPT + 99 others); Mon, 22 May 2023 03:17:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232331AbjEVHRN (ORCPT ); Mon, 22 May 2023 03:17:13 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAA35C1; Mon, 22 May 2023 00:16:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B99CC61DD4; Mon, 22 May 2023 07:15:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 732C5C4339E; Mon, 22 May 2023 07:15:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739757; bh=6oUMrt+Q90a10bD/8IHJeLA9pGO90K+akqZ6KAUmlwE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lXnVT3BBGvOGBrGEKonh+CX6QVme5/hVLSlw2yXeQXl2TLB09u+M3bh0UJGbN7e3F o8U8YhxkGjydM1bYGVigVs/MF0KAzwZLi5LYKcr1o5XY3UnGB2ww+7hjypc/bpEeqe izLcMA6n31A6gToJtRiUFZxrk7VJg7Odjtp+/9+fJg4kty9m2+PsdXmxw6vFWcLZOF rfl2lBhzbnM5V5Bo83XyeOR7f/BzdTfoI96wdGn21m6cXrIBFlVv1FQn8pL5cV1VOt RUKgSvqCDSyo3ss+WbaGlRvPs8PxDlIvcldFKFzdCDOWAcLG76Dktm2+sb9RtS0tvi Pydsneaa0xVqw== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 20/21] x86/efistub: Check SEV/SNP support while running in the firmware Date: Mon, 22 May 2023 09:14:14 +0200 Message-Id: <20230522071415.501717-21-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5088; i=ardb@kernel.org; h=from:subject; bh=6oUMrt+Q90a10bD/8IHJeLA9pGO90K+akqZ6KAUmlwE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzNXusDBrtcaNl5M7/t9ojDjNn1TBPrvD9+Knc1r97 rd8nz7oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOJaGf4nzbt4eUn1RdK08// X7SmYN975qIrAQVKX1i/8k/dHaCXuIHhf8TFRb+qSy8cfizF/+dYeVBLi8XlrmdXvFfZ/fsur5j 3lgkA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766579469251464648?= X-GMAIL-MSGID: =?utf-8?q?1766579469251464648?= Before refactoring the EFI stub boot flow to avoid the legacy bare metal decompressor, duplicate the SEV initialization and SNP feature check in the EFI stub before handing over to the kernel proper. This must be done after calling ExitBootServices(), to ensure that the SEV initialization does not corrupt any state that the firmware itself still relies on. This means that, unfortunately, the only recourse available when the SNP feature mask contains unsupported features is to terminate the virtual machine, which is what the bare metal decompressor does as well. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/sev.c | 19 ++++++++++++++----- arch/x86/include/asm/sev.h | 6 ++++++ drivers/firmware/efi/libstub/x86-stub.c | 17 +++++++++++++++++ 3 files changed, 37 insertions(+), 5 deletions(-) diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 014b89c890887b9a..d33d48359d09bfb5 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -315,23 +315,32 @@ static void enforce_vmpl0(void) */ #define SNP_FEATURES_PRESENT (0) +u64 snp_get_unsupported_features(void) +{ + if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED)) + return 0; + return sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT; +} + +void sev_es_terminate_snp_unsupported(void) +{ + sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED); +} + void snp_check_features(void) { u64 unsupported; - if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED)) - return; - /* * Terminate the boot if hypervisor has enabled any feature lacking * guest side implementation. Pass on the unsupported features mask through * EXIT_INFO_2 of the GHCB protocol so that those features can be reported * as part of the guest boot failure. */ - unsupported = sev_status & SNP_FEATURES_IMPL_REQ & ~SNP_FEATURES_PRESENT; + unsupported = snp_get_unsupported_features(); if (unsupported) { if (ghcb_version < 2 || (!boot_ghcb && !early_setup_ghcb())) - sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED); + sev_es_terminate_snp_unsupported(); sev_es_ghcb_terminate(boot_ghcb, SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED, unsupported); diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index 13dc2a9d23c1eb25..084a91aa5a6c708f 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -157,6 +157,7 @@ static __always_inline void sev_es_nmi_complete(void) __sev_es_nmi_complete(); } extern int __init sev_es_efi_map_ghcbs(pgd_t *pgd); +extern void sev_enable(struct boot_params *bp); static inline int rmpadjust(unsigned long vaddr, bool rmp_psize, unsigned long attrs) { @@ -202,12 +203,15 @@ void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio); +u64 snp_get_unsupported_features(void); +void sev_es_terminate_snp_unsupported(void); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } static inline int sev_es_setup_ap_jump_table(struct real_mode_header *rmh) { return 0; } static inline void sev_es_nmi_complete(void) { } static inline int sev_es_efi_map_ghcbs(pgd_t *pgd) { return 0; } +static inline void sev_enable(struct boot_params *bp) { } static inline int pvalidate(unsigned long vaddr, bool rmp_psize, bool validate) { return 0; } static inline int rmpadjust(unsigned long vaddr, bool rmp_psize, unsigned long attrs) { return 0; } static inline void setup_ghcb(void) { } @@ -225,6 +229,8 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in { return -ENOTTY; } +static inline u64 snp_get_unsupported_features(void) { return 0; } +static inline void sev_es_terminate_snp_unsupported(void) {} #endif #endif diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index fcdae5db0c63c7e5..02633199a8502b71 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -15,6 +15,7 @@ #include #include #include +#include #include "efistub.h" #include "x86-stub.h" @@ -923,6 +924,22 @@ void __noreturn efi_stub_entry(efi_handle_t handle, goto fail; } + /* + * Call the SEV init code while still running with the firmware's + * GDT/IDT, so #VC exceptions will be handled by EFI. + */ + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { + u64 unsupported; + + sev_enable(boot_params); + unsupported = snp_get_unsupported_features(); + if (unsupported) { + efi_err("Unsupported SEV-SNP features detected: 0x%llx\n", + unsupported); + sev_es_terminate_snp_unsupported(); + } + } + if (IS_ENABLED(CONFIG_X86_64)) { efi_5level_switch(); From patchwork Mon May 22 07:14:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 97116 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1264994vqo; Mon, 22 May 2023 00:25:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ45jyM+lnNeJAY2BeeI4ehgToS89n5Fe8PWop4xSWE4X30h6aufR6NvbBh1jbIWMLi7fO65 X-Received: by 2002:a05:6a00:1819:b0:646:e940:c2c4 with SMTP id y25-20020a056a00181900b00646e940c2c4mr13304950pfa.14.1684740302804; Mon, 22 May 2023 00:25:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684740302; cv=none; d=google.com; s=arc-20160816; b=eHsSasIICs8xxSNJ+C6u0/H0kTpJ8VHiLOZsDuSFjGNf6aLfV8sD8a6ajeJkC4fktS fydi6INS/gDxT4dyJ5TTV92BksZFvUKLDodN5BMj15oPWdm7/go0emdxY4hJ4BtagVo0 2aFEHYEqXZXWmzHJIbbC0KYi0gOCaaD9KJthtVgFmNIb7XYs8xcHm8j5ODzKCquYWzy3 KAdQcGjoz2QTxhfjctM7zlz6Yy1dN3WivnDmILxpPjADSwVEg/cQbZcFYROLPnRoibxm ARTXH/TVmu5OU807I+OZBy2Fh2YhHQjy5wq1oiT2BxtNmv5PKNgxoUrHe7C8e92NpS9F 2N0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=b6reh8uubLqTsvbJ2cQYzyC2ytZsv/100Al+2W2R3h4=; b=cPWFZaduqDfV68kP+84g7VS572qNxMUKGXR60DAr7kbax2m/dkjKueLxQtzJ1KiAuJ NataCH9WP3H7C9OYNpb/5ZoMImyCcHJuiGOsvPsHxx+Yl5pbmuA95I0eCei0KtzQI+Jh 6gjXSUO8AcpsBlxv7DyI0VI6ysI9tg9IfIBBIjy5zHbYKOS9pp2QssEkaStL7dktfSSU PN4zSD1RjlbKO9AIj+z2tMkhCxurW0qcfgDMNocgWURT0ru6f695cEPWo3OPMBr1o9Bn Em2bgUA5lip+BhusWY30OQpfmywR0F9dsQBsxPc5gWhQyyssDXZFYsUSLd2er7eG1U3h aPOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=GecyYXSG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f3-20020a625103000000b0064d4401edf1si4282002pfb.70.2023.05.22.00.24.48; Mon, 22 May 2023 00:25:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=GecyYXSG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232473AbjEVHR6 (ORCPT + 99 others); Mon, 22 May 2023 03:17:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232444AbjEVHRY (ORCPT ); Mon, 22 May 2023 03:17:24 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E7BE188; Mon, 22 May 2023 00:16:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E484D61DF9; Mon, 22 May 2023 07:16:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9AAD7C433EF; Mon, 22 May 2023 07:15:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684739761; bh=sEiDYrSPmg5nu7FmQd1VG/Fm5X7VeJdlmDiqajke56Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GecyYXSGZ78jK8QjPxxK0Y1d50Z6domMuX4wVdV1/MEP8TYfOC3yKXiy7f9M1HrHJ 9FO7X6zBGHIBRoyl9DI7GLmE9t3OduMh02T/479r6Prcs1jb7eLDY5n4JiX2g7pYiA RpBlfscCYZkh5gMuKvgDQcWV73xGQFQL69LwidMl835/pcn/NhIhk9SPteIlAuDYj6 at/KLa2fvsSrH/jDu27wTksNQzWrhRyZRjc66SdwPWd59EhmTTAPfXoSpshu6by2es fpLeOO+psdObhF+S5Eni98nVTjIGaDTKvbr/WRmIVbp+BUKw6N/zivyHNxAAzVJ00W GfB92Zm8sTo9Q== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds Subject: [PATCH v3 21/21] x86/efistub: Avoid legacy decompressor when doing EFI boot Date: Mon, 22 May 2023 09:14:15 +0200 Message-Id: <20230522071415.501717-22-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230522071415.501717-1-ardb@kernel.org> References: <20230522071415.501717-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17149; i=ardb@kernel.org; h=from:subject; bh=sEiDYrSPmg5nu7FmQd1VG/Fm5X7VeJdlmDiqajke56Q=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISVbzO2k3P/7C8xDj99auMlXuFSCUzlSaW78ZB5OhdcZ2 r/XfGjvKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABNRt2X4p1x3cJpJlVNqzfr2 2Xx691pzF1ucV9RUOx9ftPpPczSXCSPDLnWrL5X1cXanHzke0N4TdTWH9Ym2BO+E9TLXLzy2Fez iAQA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766578247743335559?= X-GMAIL-MSGID: =?utf-8?q?1766578247743335559?= The bare metal decompressor code was never really intended to run in a hosted environment such as the EFI boot services, and does a few things that are problematic in the context of EFI boot now that the logo requirements are getting tighter. In particular, the decompressor moves its own executable image around in memory, and relies on demand paging to populate the identity mappings, and these things are difficult to support in a context where memory is not permitted to be mapped writable and executable at the same time or, at the very least, is mapped non-executable by default, and needs special treatment for this restriction to be lifted. Since EFI already maps all of memory 1:1, it is unnecessary to create new page tables or handle page faults when decompressing the kernel. That means there is also no need to replace the special exception handlers for SEV. Generally, there is little need to do anything that the decompressor does beyond - initialize SEV encryption, if needed, - perform the 4/5 level paging switch, if needed, - decompress the kernel - relocate the kernel So do all of this from the EFI stub code, and avoid the bare metal decompressor altogether. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/Makefile | 5 + arch/x86/boot/compressed/efi_mixed.S | 55 ------- arch/x86/boot/compressed/head_32.S | 13 -- arch/x86/boot/compressed/head_64.S | 27 ---- arch/x86/include/asm/efi.h | 7 +- drivers/firmware/efi/libstub/x86-stub.c | 163 ++++++++------------ 6 files changed, 77 insertions(+), 193 deletions(-) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index b13a580210867ffb..535608fe72e11265 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -74,6 +74,11 @@ LDFLAGS_vmlinux += -z noexecstack ifeq ($(CONFIG_LD_IS_BFD),y) LDFLAGS_vmlinux += $(call ld-option,--no-warn-rwx-segments) endif +ifeq ($(CONFIG_EFI_STUB),y) +# ensure that the static EFI stub library will be pulled in, even if it is +# never referenced explicitly from the startup code +LDFLAGS_vmlinux += -u efi_pe_entry +endif LDFLAGS_vmlinux += -T hostprogs := mkpiggy diff --git a/arch/x86/boot/compressed/efi_mixed.S b/arch/x86/boot/compressed/efi_mixed.S index 8a02a151806df14c..f4e22ef774ab6b4a 100644 --- a/arch/x86/boot/compressed/efi_mixed.S +++ b/arch/x86/boot/compressed/efi_mixed.S @@ -269,10 +269,6 @@ SYM_FUNC_START_LOCAL(efi32_entry) jmp startup_32 SYM_FUNC_END(efi32_entry) -#define ST32_boottime 60 // offsetof(efi_system_table_32_t, boottime) -#define BS32_handle_protocol 88 // offsetof(efi_boot_services_32_t, handle_protocol) -#define LI32_image_base 32 // offsetof(efi_loaded_image_32_t, image_base) - /* * efi_status_t efi32_pe_entry(efi_handle_t image_handle, * efi_system_table_32_t *sys_table) @@ -280,8 +276,6 @@ SYM_FUNC_END(efi32_entry) SYM_FUNC_START(efi32_pe_entry) pushl %ebp movl %esp, %ebp - pushl %eax // dummy push to allocate loaded_image - pushl %ebx // save callee-save registers pushl %edi @@ -290,48 +284,8 @@ SYM_FUNC_START(efi32_pe_entry) movl $0x80000003, %eax // EFI_UNSUPPORTED jnz 2f - call 1f -1: pop %ebx - - /* Get the loaded image protocol pointer from the image handle */ - leal -4(%ebp), %eax - pushl %eax // &loaded_image - leal (loaded_image_proto - 1b)(%ebx), %eax - pushl %eax // pass the GUID address - pushl 8(%ebp) // pass the image handle - - /* - * Note the alignment of the stack frame. - * sys_table - * handle <-- 16-byte aligned on entry by ABI - * return address - * frame pointer - * loaded_image <-- local variable - * saved %ebx <-- 16-byte aligned here - * saved %edi - * &loaded_image - * &loaded_image_proto - * handle <-- 16-byte aligned for call to handle_protocol - */ - - movl 12(%ebp), %eax // sys_table - movl ST32_boottime(%eax), %eax // sys_table->boottime - call *BS32_handle_protocol(%eax) // sys_table->boottime->handle_protocol - addl $12, %esp // restore argument space - testl %eax, %eax - jnz 2f - movl 8(%ebp), %ecx // image_handle movl 12(%ebp), %edx // sys_table - movl -4(%ebp), %esi // loaded_image - movl LI32_image_base(%esi), %esi // loaded_image->image_base - leal (startup_32 - 1b)(%ebx), %ebp // runtime address of startup_32 - /* - * We need to set the image_offset variable here since startup_32() will - * use it before we get to the 64-bit efi_pe_entry() in C code. - */ - subl %esi, %ebp // calculate image_offset - movl %ebp, (image_offset - 1b)(%ebx) // save image_offset xorl %esi, %esi jmp efi32_entry // pass %ecx, %edx, %esi // no other registers remain live @@ -350,15 +304,6 @@ SYM_FUNC_START_NOALIGN(efi64_stub_entry) SYM_FUNC_END(efi64_stub_entry) #endif - .section ".rodata" - /* EFI loaded image protocol GUID */ - .balign 4 -SYM_DATA_START_LOCAL(loaded_image_proto) - .long 0x5b1b31a1 - .word 0x9562, 0x11d2 - .byte 0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b -SYM_DATA_END(loaded_image_proto) - .data .balign 8 SYM_DATA_START_LOCAL(efi32_boot_gdt) diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index beee858058df4403..cd9587fcd5084f22 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -84,19 +84,6 @@ SYM_FUNC_START(startup_32) #ifdef CONFIG_RELOCATABLE leal startup_32@GOTOFF(%edx), %ebx - -#ifdef CONFIG_EFI_STUB -/* - * If we were loaded via the EFI LoadImage service, startup_32() will be at an - * offset to the start of the space allocated for the image. efi_pe_entry() will - * set up image_offset to tell us where the image actually starts, so that we - * can use the full available buffer. - * image_offset = startup_32 - image_base - * Otherwise image_offset will be zero and has no effect on the calculations. - */ - subl image_offset@GOTOFF(%edx), %ebx -#endif - movl BP_kernel_alignment(%esi), %eax decl %eax addl %eax, %ebx diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 3074d278c7e665d8..aee518e71cdb7e75 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -146,19 +146,6 @@ SYM_FUNC_START(startup_32) #ifdef CONFIG_RELOCATABLE movl %ebp, %ebx - -#ifdef CONFIG_EFI_STUB -/* - * If we were loaded via the EFI LoadImage service, startup_32 will be at an - * offset to the start of the space allocated for the image. efi_pe_entry will - * set up image_offset to tell us where the image actually starts, so that we - * can use the full available buffer. - * image_offset = startup_32 - image_base - * Otherwise image_offset will be zero and has no effect on the calculations. - */ - subl rva(image_offset)(%ebp), %ebx -#endif - movl BP_kernel_alignment(%esi), %eax decl %eax addl %eax, %ebx @@ -335,20 +322,6 @@ SYM_CODE_START(startup_64) /* Start with the delta to where the kernel will run at. */ #ifdef CONFIG_RELOCATABLE leaq startup_32(%rip) /* - $startup_32 */, %rbp - -#ifdef CONFIG_EFI_STUB -/* - * If we were loaded via the EFI LoadImage service, startup_32 will be at an - * offset to the start of the space allocated for the image. efi_pe_entry will - * set up image_offset to tell us where the image actually starts, so that we - * can use the full available buffer. - * image_offset = startup_32 - image_base - * Otherwise image_offset will be zero and has no effect on the calculations. - */ - movl image_offset(%rip), %eax - subq %rax, %rbp -#endif - movl BP_kernel_alignment(%rsi), %eax decl %eax addq %rax, %rbp diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h index 8b4be7cecdb8eb73..b0994ae3bc23f84d 100644 --- a/arch/x86/include/asm/efi.h +++ b/arch/x86/include/asm/efi.h @@ -90,6 +90,8 @@ static inline void efi_fpu_end(void) } #ifdef CONFIG_X86_32 +#define EFI_X86_KERNEL_ALLOC_LIMIT (SZ_512M - 1) + #define arch_efi_call_virt_setup() \ ({ \ efi_fpu_begin(); \ @@ -103,8 +105,7 @@ static inline void efi_fpu_end(void) }) #else /* !CONFIG_X86_32 */ - -#define EFI_LOADER_SIGNATURE "EL64" +#define EFI_X86_KERNEL_ALLOC_LIMIT EFI_ALLOC_LIMIT extern asmlinkage u64 __efi_call(void *fp, ...); @@ -218,6 +219,8 @@ efi_status_t efi_set_virtual_address_map(unsigned long memory_map_size, #ifdef CONFIG_EFI_MIXED +#define EFI_ALLOC_LIMIT (efi_is_64bit() ? ULONG_MAX : U32_MAX) + #define ARCH_HAS_EFISTUB_WRAPPERS static inline bool efi_is_64bit(void) diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 02633199a8502b71..9c280d9e6e9ef58f 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -15,17 +15,14 @@ #include #include #include +#include #include #include "efistub.h" #include "x86-stub.h" -/* Maximum physical address for 64-bit kernel with 4-level paging */ -#define MAXMEM_X86_64_4LEVEL (1ull << 46) - const efi_system_table_t *efi_system_table; const efi_dxe_services_table_t *efi_dxe_table; -u32 image_offset __section(".data"); static efi_loaded_image_t *image = NULL; static efi_memory_attribute_protocol_t *memattr; @@ -276,33 +273,9 @@ void efi_adjust_memory_range_protection(unsigned long start, } } -void startup_32(struct boot_params *boot_params); - -static void -setup_memory_protection(unsigned long image_base, unsigned long image_size) -{ -#ifdef CONFIG_64BIT - if (image_base != (unsigned long)startup_32) - efi_adjust_memory_range_protection(image_base, image_size); -#else - /* - * Clear protection flags on a whole range of possible - * addresses used for KASLR. We don't need to do that - * on x86_64, since KASLR/extraction is performed after - * dedicated identity page tables are built and we only - * need to remove possible protection on relocated image - * itself disregarding further relocations. - */ - efi_adjust_memory_range_protection(LOAD_PHYSICAL_ADDR, - KERNEL_IMAGE_SIZE - LOAD_PHYSICAL_ADDR); -#endif -} - static const efi_char16_t apple[] = L"Apple"; -static void setup_quirks(struct boot_params *boot_params, - unsigned long image_base, - unsigned long image_size) +static void setup_quirks(struct boot_params *boot_params) { efi_char16_t *fw_vendor = (efi_char16_t *)(unsigned long) efi_table_attr(efi_system_table, fw_vendor); @@ -311,9 +284,6 @@ static void setup_quirks(struct boot_params *boot_params, if (IS_ENABLED(CONFIG_APPLE_PROPERTIES)) retrieve_apple_device_properties(boot_params); } - - if (IS_ENABLED(CONFIG_EFI_DXE_MEM_ATTRIBUTES)) - setup_memory_protection(image_base, image_size); } /* @@ -466,7 +436,6 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, } image_base = efi_table_attr(image, image_base); - image_offset = (void *)startup_32 - image_base; status = efi_allocate_pages(sizeof(struct boot_params), (unsigned long *)&boot_params, ULONG_MAX); @@ -757,6 +726,61 @@ static efi_status_t exit_boot(struct boot_params *boot_params, void *handle) return EFI_SUCCESS; } +static void efi_get_seed(void *seed, int size) +{ + efi_get_random_bytes(size, seed); + + /* + * This only updates seed[0] when running on 32-bit, but in that case, + * we don't use seed[1] anyway, as there is no virtual KASLR on 32-bit. + */ + *(unsigned long *)seed ^= kaslr_get_random_long("EFI"); +} + +static void error(char *str) +{ + efi_warn("Decompression failed: %s\n", str); +} + +static efi_status_t efi_decompress_kernel(unsigned long *kernel_entry) +{ + unsigned long virt_addr = LOAD_PHYSICAL_ADDR; + unsigned long addr, alloc_size, entry; + efi_status_t status; + u32 seed[2] = {}; + + /* determine the required size of the allocation */ + alloc_size = ALIGN(max((unsigned long)output_len, kernel_total_size), + MIN_KERNEL_ALIGN); + + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && !efi_nokaslr) { + u64 range = KERNEL_IMAGE_SIZE - LOAD_PHYSICAL_ADDR - kernel_total_size; + + efi_get_seed(seed, sizeof(seed)); + + virt_addr += (range * seed[1]) >> 32; + virt_addr &= ~(CONFIG_PHYSICAL_ALIGN - 1); + } + + status = efi_random_alloc(alloc_size, CONFIG_PHYSICAL_ALIGN, &addr, + seed[0], EFI_LOADER_CODE, + EFI_X86_KERNEL_ALLOC_LIMIT); + if (status != EFI_SUCCESS) + return status; + + entry = decompress_kernel((void *)addr, virt_addr, error); + if (entry == ULONG_MAX) { + efi_free(alloc_size, addr); + return EFI_LOAD_ERROR; + } + + *kernel_entry = addr + entry; + + efi_adjust_memory_range_protection(addr, kernel_total_size); + + return EFI_SUCCESS; +} + static void __noreturn enter_kernel(unsigned long kernel_addr, struct boot_params *boot_params) { @@ -775,10 +799,9 @@ void __noreturn efi_stub_entry(efi_handle_t handle, struct boot_params *boot_params) { efi_guid_t guid = EFI_MEMORY_ATTRIBUTE_PROTOCOL_GUID; - unsigned long bzimage_addr = (unsigned long)startup_32; - unsigned long buffer_start, buffer_end; struct setup_header *hdr = &boot_params->hdr; const struct linux_efi_initrd *initrd = NULL; + unsigned long kernel_entry; efi_status_t status; efi_system_table = sys_table_arg; @@ -806,60 +829,6 @@ void __noreturn efi_stub_entry(efi_handle_t handle, } } - /* - * If the kernel isn't already loaded at a suitable address, - * relocate it. - * - * It must be loaded above LOAD_PHYSICAL_ADDR. - * - * The maximum address for 64-bit is 1 << 46 for 4-level paging. This - * is defined as the macro MAXMEM, but unfortunately that is not a - * compile-time constant if 5-level paging is configured, so we instead - * define our own macro for use here. - * - * For 32-bit, the maximum address is complicated to figure out, for - * now use KERNEL_IMAGE_SIZE, which will be 512MiB, the same as what - * KASLR uses. - * - * Also relocate it if image_offset is zero, i.e. the kernel wasn't - * loaded by LoadImage, but rather by a bootloader that called the - * handover entry. The reason we must always relocate in this case is - * to handle the case of systemd-boot booting a unified kernel image, - * which is a PE executable that contains the bzImage and an initrd as - * COFF sections. The initrd section is placed after the bzImage - * without ensuring that there are at least init_size bytes available - * for the bzImage, and thus the compressed kernel's startup code may - * overwrite the initrd unless it is moved out of the way. - */ - - buffer_start = ALIGN(bzimage_addr - image_offset, - hdr->kernel_alignment); - buffer_end = buffer_start + hdr->init_size; - - if ((buffer_start < LOAD_PHYSICAL_ADDR) || - (IS_ENABLED(CONFIG_X86_32) && buffer_end > KERNEL_IMAGE_SIZE) || - (IS_ENABLED(CONFIG_X86_64) && buffer_end > MAXMEM_X86_64_4LEVEL) || - (image_offset == 0)) { - extern char _bss[]; - - status = efi_relocate_kernel(&bzimage_addr, - (unsigned long)_bss - bzimage_addr, - hdr->init_size, - hdr->pref_address, - hdr->kernel_alignment, - LOAD_PHYSICAL_ADDR); - if (status != EFI_SUCCESS) { - efi_err("efi_relocate_kernel() failed!\n"); - goto fail; - } - /* - * Now that we've copied the kernel elsewhere, we no longer - * have a set up block before startup_32(), so reset image_offset - * to zero in case it was set earlier. - */ - image_offset = 0; - } - #ifdef CONFIG_CMDLINE_BOOL status = efi_parse_options(CONFIG_CMDLINE); if (status != EFI_SUCCESS) { @@ -877,6 +846,12 @@ void __noreturn efi_stub_entry(efi_handle_t handle, } } + status = efi_decompress_kernel(&kernel_entry); + if (status != EFI_SUCCESS) { + efi_err("Failed to decompress kernel\n"); + goto fail; + } + /* * At this point, an initrd may already have been loaded by the * bootloader and passed via bootparams. We permit an initrd loaded @@ -916,7 +891,7 @@ void __noreturn efi_stub_entry(efi_handle_t handle, setup_efi_pci(boot_params); - setup_quirks(boot_params, bzimage_addr, buffer_end - buffer_start); + setup_quirks(boot_params); status = exit_boot(boot_params, handle); if (status != EFI_SUCCESS) { @@ -940,14 +915,10 @@ void __noreturn efi_stub_entry(efi_handle_t handle, } } - if (IS_ENABLED(CONFIG_X86_64)) { + if (IS_ENABLED(CONFIG_X86_64)) efi_5level_switch(); - /* add offset of startup_64() */ - bzimage_addr += 0x200; - } - - enter_kernel(bzimage_addr, boot_params); + enter_kernel(kernel_entry, boot_params); fail: efi_err("efi_stub_entry() failed!\n");