From patchwork Wed Jun 7 07:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 104297 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp75099vqr; Wed, 7 Jun 2023 00:36:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7X3Y266iBrAaZlrovNwiKqp9hSPY9977DPXEjEgvbp8fKzt4OayV1Z9qXJnr6GZbLgw/7K X-Received: by 2002:a92:d9cf:0:b0:32b:1536:f3e9 with SMTP id n15-20020a92d9cf000000b0032b1536f3e9mr6099519ilq.18.1686123391559; Wed, 07 Jun 2023 00:36:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686123391; cv=none; d=google.com; s=arc-20160816; b=EtindfOgxUWDJ7ckziJnMTAjRLwKpR5yLPgTbe/exI1obCZDLDsr7ZnIeoFf5K/qBC HiiAI883nGtbSMQjGC1FCJUqaS5xI8ON5OHT38kIV6vjOUxzHWRbRYWXYe8DjH5xf5vt FNgiyo5Hlr03+z8H03/RZwtTGXcRdd7HkgOC22f2DlC02HjmoqJOf7neDirapf/s3yyU aHbFKZ90J3k7Ln4oegj47N+3hR3Y0ekj0ynqNwoJKGWEdeahBWxV2Rl2AmcHFKulcRPK QqAW7KCW0ZDk4ZvYMIZ9yij36sv9D8g2U1w7mmUmJvsjmFBXPuRZWVodSAVLrwCESCcB IzSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uMDWIDF77iuBD0RJXEdt/pezidON+Mzi6E+7aSfz3v0=; b=KA/IgY9a5iwRuK+xV0Bhoj1Ip5YUzeOlyCaz88p9G1POODVBuJRE7A5bpaN7v4TLzi vFI1NyWO1azE1G+npjJoYdlB/+ZklkVWe+f/pWCy2foGlFZlk3TqJk/0qeusx7Zy5fnm JNdR3pO0MRxngQuGXzQ14NCyGjZ//LVJej6bGJOCp0IzndewEdM3nFv6zaCVSlkzlUxp LeYM4Tu68Ltdpx+SRU09H8yYdiikRsgiRLH13Uo/FChFlG18RluiSQS9trD+5efp4hEC 4ps7JJaKFc+9W7jiBgkg4lL6azEKQxsxNsraRa0oF5zCzJD5sgFef+D2BDaNaa/q6aq/ k5BA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sf3muz+U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t3-20020a639543000000b0053f955eda7csi3585288pgn.559.2023.06.07.00.36.17; Wed, 07 Jun 2023 00:36:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sf3muz+U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238886AbjFGH1p (ORCPT + 99 others); Wed, 7 Jun 2023 03:27:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239234AbjFGH00 (ORCPT ); Wed, 7 Jun 2023 03:26:26 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA9EF2D48; Wed, 7 Jun 2023 00:25:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9A81A61587; Wed, 7 Jun 2023 07:24:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39C82C4339E; Wed, 7 Jun 2023 07:24:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686122683; bh=PsQ8al6sNOe3j1ZfxUsRkdXdICx8owngaMHdAfuDB5Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sf3muz+U21ReBQT3a26E0Xc0gHR5+q0bZNlfAFZDNmrqxF45a2W+qlmcOKFt8YLx6 cIKC7efyPs4A+isbwJUTaaTwRZc40X3tSIkTR+7/w4ctUC1a6+b8x8EuELhEt4R7Aa X3wE3PIilL8K6RfvgtaupRHNRoQVCTIvZqPeGU33ZndhKo9UTGsUJgmOXxP0CStqTA 9vTbMOAEwdfS/nHYfQ5J7Zhf705WCAfJoi8SqB8eZRSmpJC/aJ6mPOQxkldq8hbF5n ftbVm+VTfJA1KNmcpRDn+Oo4BNkwpFjx3t2HROVveiTb+o+GNL/zGTQCRKwGD8RYLz Qvkpwqs7wt8Dg== From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Evgeniy Baskov , Borislav Petkov , Andy Lutomirski , Dave Hansen , Ingo Molnar , Peter Zijlstra , Thomas Gleixner , Alexey Khoroshilov , Peter Jones , Gerd Hoffmann , Dave Young , Mario Limonciello , Kees Cook , Tom Lendacky , "Kirill A . Shutemov" , Linus Torvalds , Joerg Roedel Subject: [PATCH v5 11/20] x86/decompressor: Only call the trampoline when changing paging levels Date: Wed, 7 Jun 2023 09:23:33 +0200 Message-Id: <20230607072342.4054036-12-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230607072342.4054036-1-ardb@kernel.org> References: <20230607072342.4054036-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3740; i=ardb@kernel.org; h=from:subject; bh=PsQ8al6sNOe3j1ZfxUsRkdXdICx8owngaMHdAfuDB5Q=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIaXBoEim8mh/RsPHj1ZOoj/7a5ce0ry+ZF/YMYsna48+5 C1LsLftKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABN58ZSRYYc6b9+P+09Zbucd 4rq9UN96o5hmV/Klzyom9mXsoYvFJjMy/D0SkDThzY/6mS4mXqsnPlNc9u9dvKLC/VN//v578P/ DNxYA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768028521697359214?= X-GMAIL-MSGID: =?utf-8?q?1768028521697359214?= Since the current and desired number of paging levels are known when the trampoline is being prepared, avoid calling the trampoline at all if it is clear that calling it is not going to result in a change to the number of paging levels. Given that the CPU is already running in long mode, the PAE and LA57 settings are necessarily consistent with the currently active page tables - the only difference is that CR4.MCE will always be preserved in this case, but it will be cleared by the real kernel startup code if CONFIG_X86_MCE is not enabled. Acked-by: Kirill A. Shutemov Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 21 +------------------- arch/x86/boot/compressed/pgtable_64.c | 18 +++++++---------- 2 files changed, 8 insertions(+), 31 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 3d4da7e5270c8d4d..577173be8ec805cd 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -387,10 +387,6 @@ SYM_CODE_START(startup_64) * For the trampoline, we need the top page table to reside in lower * memory as we don't have a way to load 64-bit values into CR3 in * 32-bit mode. - * - * We go though the trampoline even if we don't have to: if we're - * already in a desired paging mode. This way the trampoline code gets - * tested on every boot. */ /* Make sure we have GDT with 32-bit code segment */ @@ -542,25 +538,10 @@ SYM_CODE_START(trampoline_32bit_src) btrl $X86_CR0_PG_BIT, %eax movl %eax, %cr0 - /* Check what paging mode we want to be in after the trampoline */ - testl %esi, %esi - jz 1f - - /* We want 5-level paging: don't touch CR3 if it already points to 5-level page tables */ - movl %cr4, %eax - testl $X86_CR4_LA57, %eax - jnz 3f - jmp 2f -1: - /* We want 4-level paging: don't touch CR3 if it already points to 4-level page tables */ - movl %cr4, %eax - testl $X86_CR4_LA57, %eax - jz 3f -2: /* Point CR3 to the trampoline's new top level page table */ leal TRAMPOLINE_32BIT_PGTABLE_OFFSET(%edi), %eax movl %eax, %cr3 -3: + /* Set EFER.LME=1 as a precaution in case hypervsior pulls the rug */ movl $MSR_EFER, %ecx rdmsr diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index 1d28ad95ea839531..5b15d823e7010650 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -128,6 +128,13 @@ asmlinkage void set_paging_levels(void *rmode) l5_required = true; } + /* + * The trampoline will not be used if the paging mode is already set to + * the desired one. + */ + if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) + return; + trampoline_32bit = (unsigned long *)find_trampoline_placement(); /* Preserve trampoline memory */ @@ -155,18 +162,8 @@ asmlinkage void set_paging_levels(void *rmode) * * The new page table will be used by trampoline code for switching * from 4- to 5-level paging or vice versa. - * - * If switching is not required, the page table is unused: trampoline - * code wouldn't touch CR3. */ - /* - * We are not going to use the page table in trampoline memory if we - * are already in the desired paging mode. - */ - if (l5_required == !!(native_read_cr4() & X86_CR4_LA57)) - goto out; - if (l5_required) { /* * For 4- to 5-level paging transition, set up current CR3 as @@ -189,7 +186,6 @@ asmlinkage void set_paging_levels(void *rmode) (void *)src, PAGE_SIZE); } -out: toggle_la57(trampoline_32bit, l5_required); }