From patchwork Tue Feb 13 12:41:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 200415 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp507217dyb; Tue, 13 Feb 2024 04:43:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IEi7UT2/0rJ9ualPP02homoZLuXa2m1JKqEpsH+hcIFVAZMyfmSX5cZImk9vn+2X51P8OvE X-Received: by 2002:a17:906:e257:b0:a3c:9206:608f with SMTP id gq23-20020a170906e25700b00a3c9206608fmr4798254ejb.0.1707828223256; Tue, 13 Feb 2024 04:43:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707828223; cv=pass; d=google.com; s=arc-20160816; b=rFU4vk0OgyNFaJc34mFsux9CmNazRrMdl/IdibXSzb8vGodVK1BITauZz7hpqDDmKD LmvjCJ4im2ChfhnXUUfCd1Og1RQ2bfhSNnkO2Iou+bDR1d3vQ0pJ5Bg4089FD8gULVtQ N9fwZ7xSda5AWm8OqtVDtDDcRcORm5aYXTst+ri19SG5oi/m5IDNCTU94XeSA0k03hb7 s/AcR8xUP5Ozac+7ZMUJNtvRCET5ksR8vQGo+HZ1x23HvoRJizb/v8R4RItW8f6H4NTl Panjh8TkiXOnF9WTposCcy9OW7921B38f2vl6r7pQ3mpl8w6SXNiBf4Ify9GNVG5jOM9 cfDA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=SCp/zoIzhRhhb9JUvWveq8Y/lRzJCxLp1biyCfwnfq0=; fh=e+xdvUvoO6swtC40e28a1eDsy5sMVkWerbuU+KL0HAQ=; b=KuKxQli87doGJ88EvfGjxtrUR8HjE/nckgL1vMyXFtiZ4y1bYP1hmigmc9bn5PvYjo HH+fGZuGI7TmrSKCop2XlfI0yf/5Jtpqb7Q3Wve35ZAWRq62N5l8LEosOx+lnJDxMR3p AGno0nnJAth5hUDx94fmusM9nFta1yFkMFvldT4JqJaIPSxksgXwh4RcpIh/nfMESlKL 1hDZ3Ml+O3aCACvdAOueXsmjEUp/Idn3Y6fo5PpxU6w9jagJEccFh3gLGoF/KI7S9E8g bjzVmfKigI5kZOmgHOIHK/h9EcJmRhiHVt+9WkRxOovfpoEeuFWVfEPAib3M84vWh1iS CQeA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=vqgCL7dM; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-63524-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63524-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com X-Forwarded-Encrypted: i=2; AJvYcCXUUhhw+QD0Kyz7s6XF6NwsfzUJfrlOm1huvJ0WxcbNAR5+aPQkvsUKjqsuo1TT4qIBAkHvO4I14E+LiLrofscKq96g1w== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id h24-20020a17090619d800b00a3d0411ebc8si522084ejd.63.2024.02.13.04.43.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 04:43:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-63524-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=vqgCL7dM; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-63524-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-63524-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id AC30F1F259FE for ; Tue, 13 Feb 2024 12:43:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8FB5F52F94; Tue, 13 Feb 2024 12:42:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vqgCL7dM" Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE68651C42 for ; Tue, 13 Feb 2024 12:42:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707828134; cv=none; b=D6Wqp02rya+GeyxUYSt6bXSxCf74dHrXEgbgPcEcTo3RdFIiuUz0cP7up/6Qn0uwxaNt+RVr1Ji4T5Wp1bT12IxBSo9KWwBPZ96W4eKOra2UJIwIb8MMLJyPGyVmefxzw2Mz6GX3VXUxZpAVvVHY1CgRZGZDv15k8imnPAuW4y8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707828134; c=relaxed/simple; bh=y9rxsCe7t3+BoRvciY7zzDK+lsJfA46YxMM4xo0hlg0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JRiuChH6kBZjNdpXX05p04y+1mDtPxmpwGuozihydOlA9hUw69Y3x7hqYBcpPwwyz8Nq+uKQoGbIC5tIaKufVlRgRN8o9KU4s/uYlTKvecj3Y7ixAjclVmXFO9yyhropNowQkm75qaTd7129AxTSZRgvPfxo2vTOXY727T4wX+c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vqgCL7dM; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6077f931442so14216357b3.3 for ; Tue, 13 Feb 2024 04:42:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707828131; x=1708432931; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SCp/zoIzhRhhb9JUvWveq8Y/lRzJCxLp1biyCfwnfq0=; b=vqgCL7dMY/OFJl8YlLz32pwDD/XuxbQr5hLcfE4Gq0b+kPvmqrflXFRbpGLnIszomv BsnEKi82TtH8Sjs9c0g06lplf5Cxmryt9EQIRlvy9sc8WkgxbxPfH8+03WdFZkiFdbIU SS6H2buQAdnY82a8LZs8nBuVFtyM5Ii3gdBYb7DQoAyI1ZVKNOSXC0+8XbQ1bUqlhpjw fdxTva6JI6b8IapQF3mrfkuVPcHE+cmpvTThBGYLHfdAvazfo+ac5LyBLirMx51fY60i H3v7L9OFgccwAFEQ1fo3emfMczMcfCF+u4MsZXdVGKIzqAm3i6+Fxm7b4XilbJUi6eEt 54FA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707828131; x=1708432931; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SCp/zoIzhRhhb9JUvWveq8Y/lRzJCxLp1biyCfwnfq0=; b=f8U8y4Vs4vBexR/JrVUfL/X/qw6wRRpo5gyVF1s/Phb/nzuh9L9gMvqjYDxDdLIPap WLfS0L3QeJlU0DPGcyWLxw6w6NFr5SjNzwk4ZYCnXwP5rzUs5wemDrB3dkhDsFDU6fzx Lmbibw1ZDsZoS9y2SZ2XEW/RO/hjFY83LbJggoylyvC2aHYLjZQ2DsK/ePWIcdWpYfht mPmf6JaKDnS2NTu6bZmMhxmLz9o+aun3vZl/G5r3Ip3uL23haZaaxg6G+td7zR+lqiK/ hVAXgntUMBqTXloUal8gsRnqnnUQAP9t5BWRIumbNl2vRb+mxXvbm0VP8qGo9cxbczar QdAg== X-Gm-Message-State: AOJu0Ywrwq5QulubFrCn6h2IBR+tdO9eaMGdQmymGZqAon/aBv2DgBfm HRCzdmYSbkxf54LS4p87J7woW0Q2vjLwJqEzddMCLAUglSNztFyNrQI0Pee8PVpqK1o8uNTs/M1 KPc4kIXxbs1dsYCcAlRxhZJ1Wv0cpX2vRwcbGqwR4sLDUq5w9H12ELK/WQ+xqnnJi/rwmtJwvTU SFpRuoYRj4cil4GqZQEhiZfyRGwS+ljg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a81:6d05:0:b0:607:9963:2dd4 with SMTP id i5-20020a816d05000000b0060799632dd4mr90739ywc.9.1707828130803; Tue, 13 Feb 2024 04:42:10 -0800 (PST) Date: Tue, 13 Feb 2024 13:41:48 +0100 In-Reply-To: <20240213124143.1484862-13-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240213124143.1484862-13-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3321; i=ardb@kernel.org; h=from:subject; bh=z98YWbAEHVe7Ntbv2WAOVJQ9O8vTY0tfpKYTE0xX9/E=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIfV0ck841z3bDM4lraYBmROE2A0KGp8WRh3hbUm0T2C9Z bj2lENHKQuDGAeDrJgii8Dsv+92np4oVes8SxZmDisTyBAGLk4BmMj+XYwM8/bZqcV2fOwPq966 6h2LxWvenZlbOGWbAk95ne/M2CKXxshwU41PiU1nTfbReNcy8diKz7P1jp4JEnhypeRl+8m1ga9 ZAA== X-Mailer: git-send-email 2.43.0.687.g38aa6559b0-goog Message-ID: <20240213124143.1484862-17-ardb+git@google.com> Subject: [PATCH v4 04/11] x86/startup_64: Defer assignment of 5-level paging global variables From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Kees Cook , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790787687168931377 X-GMAIL-MSGID: 1790787687168931377 From: Ard Biesheuvel Assigning the 5-level paging related global variables from the earliest C code using explicit references that use the 1:1 translation of memory is unnecessary, as the startup code itself does not rely on them to create the initial page tables, and this is all it should be doing. So defer these assignments to the primary C entry code that executes via the ordinary kernel virtual mapping. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 44 +++++++------------- 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 4b08e321d168..4bcbd4ae2dc6 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -23,6 +23,7 @@ #include #include +#include #include #include #include @@ -77,24 +78,11 @@ static struct desc_struct startup_gdt[GDT_ENTRIES] __initdata = { [GDT_ENTRY_KERNEL_DS] = GDT_ENTRY_INIT(DESC_DATA64, 0, 0xfffff), }; -#ifdef CONFIG_X86_5LEVEL -static void __head *fixup_pointer(void *ptr, unsigned long physaddr) -{ - return ptr - (void *)_text + (void *)physaddr; -} - -static unsigned long __head *fixup_long(void *ptr, unsigned long physaddr) +static inline bool check_la57_support(void) { - return fixup_pointer(ptr, physaddr); -} - -static unsigned int __head *fixup_int(void *ptr, unsigned long physaddr) -{ - return fixup_pointer(ptr, physaddr); -} + if (!IS_ENABLED(CONFIG_X86_5LEVEL)) + return false; -static bool __head check_la57_support(unsigned long physaddr) -{ /* * 5-level paging is detected and enabled at kernel decompression * stage. Only check if it has been enabled there. @@ -102,21 +90,8 @@ static bool __head check_la57_support(unsigned long physaddr) if (!(native_read_cr4() & X86_CR4_LA57)) return false; - *fixup_int(&__pgtable_l5_enabled, physaddr) = 1; - *fixup_int(&pgdir_shift, physaddr) = 48; - *fixup_int(&ptrs_per_p4d, physaddr) = 512; - *fixup_long(&page_offset_base, physaddr) = __PAGE_OFFSET_BASE_L5; - *fixup_long(&vmalloc_base, physaddr) = __VMALLOC_BASE_L5; - *fixup_long(&vmemmap_base, physaddr) = __VMEMMAP_BASE_L5; - return true; } -#else -static bool __head check_la57_support(unsigned long physaddr) -{ - return false; -} -#endif static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) { @@ -180,7 +155,7 @@ unsigned long __head __startup_64(unsigned long physaddr, bool la57; int i; - la57 = check_la57_support(physaddr); + la57 = check_la57_support(); /* Is the address too large? */ if (physaddr >> MAX_PHYSMEM_BITS) @@ -463,6 +438,15 @@ asmlinkage __visible void __init __noreturn x86_64_start_kernel(char * real_mode (__START_KERNEL & PGDIR_MASK))); BUILD_BUG_ON(__fix_to_virt(__end_of_fixed_addresses) <= MODULES_END); + if (check_la57_support()) { + __pgtable_l5_enabled = 1; + pgdir_shift = 48; + ptrs_per_p4d = 512; + page_offset_base = __PAGE_OFFSET_BASE_L5; + vmalloc_base = __VMALLOC_BASE_L5; + vmemmap_base = __VMEMMAP_BASE_L5; + } + cr4_init_shadow(); /* Kill off the identity-map trampoline */