From patchwork Thu Jan 25 11:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192020 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573550dyi; Thu, 25 Jan 2024 03:34:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IFxFoCkxYdjySS+rD2e4pDd7RJH0dXOWbUhJNHKHGJe3G16091ozzE/TORn0UTF33ov+eBH X-Received: by 2002:a05:622a:138f:b0:42a:4c59:43fe with SMTP id o15-20020a05622a138f00b0042a4c5943femr953722qtk.102.1706182494405; Thu, 25 Jan 2024 03:34:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182494; cv=pass; d=google.com; s=arc-20160816; b=tkpvQj2ysCL83utjGm4J7ksDwXeKEhJhsCqXC6/eZFF+q9+EqnxlFSWMAiMgIqihLe CY27C/omC3t8R5jPG5qbRZkTgr0La6Pwi7MrfiL/hik0ZgiBzCDDZajgI+43O5BIvHom GNUQclR4AKk1yOA0ROca2FJJ5baxI88jEoVlColnxPlJPFz6yNsYQoZIdiqcxxybBKEx IX5b5EKNZLW87dj5OwaFUlTUxzjDAdqjmcV86pEMe2uZxO0HBbf//uZB71U3VYPlTjy6 fukHFxLLSBec33nVXjxs7JG443Ce4dz9itDIUILnKdeOyRPC6Z9yu4PAJbItI+bgnlEx QgMg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=1Kz7C+ixPzKX7RlC+dwS9psnBzxi2S77hDKHtKWjxqM=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=O2p5KwMxtLYNJfJP4CW9Ftn83ECaekjuCucZ3vVfsQ0mRQGbGf3+WSJBbRMo0clEzY 4F5zvxzUMclSYomNhW+4zgxEFfODJ6uWRzGvU/l8nCKchcX1Nrrmobtv0tykuwgO5gsh R9vrRa5n5/gVBO3yWXKvZnHcxkQ3qavquFc/7F5gPhkY6fPJ2CCK+X8H7Y0/H1ug7Q9Q 9LcCzyvVewE+Pavu3KOLdOc35vREvYUxRNgEZQopJ0Pw8845WMJQ6hDDV289k/pyTUtm 1FahJ/bGR7Sg3rGxhhkcFKcz599mNatxo0HmfuCh/0xFNMHT8d/gDCZbe7/wmnyd3vqw Z9LQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=LecD1d6T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id j16-20020ac85f90000000b0042a3d3520cbsi9553886qta.720.2024.01.25.03.34.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:34:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=LecD1d6T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 27A001C22C2E for ; Thu, 25 Jan 2024 11:34:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5AC704502D; Thu, 25 Jan 2024 11:33:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LecD1d6T" Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06DA94F216 for ; Thu, 25 Jan 2024 11:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182376; cv=none; b=cZMWWfPKf8/5iKywdwYxG5Ugn8BOdL6KpnZC8+v6TnotN1zMtJ5646dxrlnTtk6dYyJQhq/oN0Kgq3QbBlP4BsuIcfnwkukbVozigpNG5LsWzZxqLqn94Mp71Gxx9717+/39SnJ+wWjww4nxDPHN8b4OYbv9zkAcFqahVfAdHUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182376; c=relaxed/simple; bh=1jRwFT8KP5Sc2cp3hWq0TIt5gMnbctAz1NiKEF7cRfM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TJ8xML/dmS9p2YKH1q5xhNdMlQFVk30WRulPGDOv1KxL26LwPAKytLRJHzlQ4JJB4t0qjy4VHPumXWbxxF4gc+gY+ncnfp3rkX3w8Dzj4bLGk3iUkt45Kk44TGaFfg308vdwPnAusPqnIrJSTN8c7LS1fBThKElzd/IJSqKDE4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LecD1d6T; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-40e4478a3afso2716585e9.1 for ; Thu, 25 Jan 2024 03:32:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182371; x=1706787171; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1Kz7C+ixPzKX7RlC+dwS9psnBzxi2S77hDKHtKWjxqM=; b=LecD1d6TQ2ViDFF7DSxGHbMZealbvT2rcLpPji4ylD0OQy4ZeiE2cg6wNoWi7XDc9l 1KFrOMUEabKBp+wqZ5h4Iln5wJ6IdgyF5fW0M9mept1x/1OiXqqVWV3IjNJGRFwe0dn/ C2qA79F/We6WEZmWzZgOaKBNubIos2r/Jj8R4EcAOBQ7joBQ2firhx6NEA+9iuVwXewN 6ChsOXjBkbTUIJRd3G1j6AwgTz+67PWctKRwlKB8Q8n303kcE1jCGxFL+PTW53M5s+cK nC/l0zvPNr+OtCQlwD4jNss2t84CuegaiMfhc+gOtzF+7wHaxGpR5jVmLQSogpbO7Eqo ZwQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182371; x=1706787171; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1Kz7C+ixPzKX7RlC+dwS9psnBzxi2S77hDKHtKWjxqM=; b=N2+/wSVl4AyJVOI6jmPdjUrpMU5IroACBfOJCRnKCsY9rCCugVsdezTJ+5IeGhb1ir ZyKXeH/54PVXKmfBgA9EBXXKJMcMEcf1zurOWrtRnnAkfFgCxl6FFBFdnxdi9OgYaOGG kCFNi70D17IXP1QeQ+39y8H9W7DlTvNL8i+S3M+q135LdYZE0w4NCOLPOX8f4XPX3I0c 1vqqYdHO5u2I4lBl3zJYsnt1xmCxu3oicAJS0IV3PF5pW9GRgFbu/x6TjEzV8lsS4xa6 nLDoAo8dNtDk3S5fTzrVsS8+FhB3OmNIXGYfVoikU13Tu6Azw14964n3YBTTeK8t5jF0 KZuQ== X-Gm-Message-State: AOJu0YxMug7Ob09e2Jdz24LSvm4GRpqYQqF5e43ogW2I9M6iJcxpbcbq s4z3MCLsiPQu7G5GgIgBAFykYy/AH59DkAyLAc4jLSMMG6LP2UkxVegYRhMiUpoxoCalfotMzIl nNfhDCho/w4kkWiH75GG2SQCMKpcvl3/Z96sIOXJrxX7hk/FHFFpIzaY1GvhqYELXYyjiqU1rDe 123CHUxfxLAKd3nT457NbtqMdK5Jf/Wg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:3489:b0:40e:d20f:6e43 with SMTP id a9-20020a05600c348900b0040ed20f6e43mr21720wmq.1.1706182371411; Thu, 25 Jan 2024 03:32:51 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:25 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=11891; i=ardb@kernel.org; h=from:subject; bh=PzzWp9ixYoJ19nSdSRUo8K5t8tOJCydzPxdlM74OpZs=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT68PDJS839fA9XjdFTNBbZ5HLrO9tm63D9yocXXlZ+ M4Kk1vLOkpZGMQ4GGTFFFkEZv99t/P0RKla51myMHNYmUCGMHBxCsBEAr8x/OHevmDJd4ttvtP9 ReZN4p7dasPwKryKa1LuLw7diQkc584wMiyflRHO0nMyY6WDvk2S8u76iwt25X999OpCQQ77Nuv UfRwA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-25-ardb+git@google.com> Subject: [PATCH v2 06/17] x86/head64: Replace pointer fixups with PIE codegen From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062015419024945 X-GMAIL-MSGID: 1789062015419024945 From: Ard Biesheuvel Some of the C code in head64.c may be called from a different virtual address than it was linked at. Currently, we deal with this by using ordinary, position dependent codegen, and fixing up all symbol references on the fly. This is fragile and tricky to maintain. It is also unnecessary: we can use position independent codegen (with hidden visibility) to ensure that all compiler generated symbol references are RIP-relative, removing the need for fixups entirely. It does mean we need explicit references to kernel virtual addresses to be generated by hand, so generate those using a movabs instruction in inline asm in the handful places where we actually need this. Signed-off-by: Ard Biesheuvel --- arch/x86/Makefile | 5 ++ arch/x86/boot/compressed/Makefile | 2 +- arch/x86/include/asm/setup.h | 4 +- arch/x86/kernel/Makefile | 4 + arch/x86/kernel/head64.c | 88 +++++++------------- arch/x86/kernel/head_64.S | 5 +- 6 files changed, 45 insertions(+), 63 deletions(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 1a068de12a56..3c3c07cccd47 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -168,6 +168,11 @@ else KBUILD_CFLAGS += -mcmodel=kernel KBUILD_RUSTFLAGS += -Cno-redzone=y KBUILD_RUSTFLAGS += -Ccode-model=kernel + + PIE_CFLAGS := -fpie -mcmodel=small -fno-stack-protector \ + -include $(srctree)/include/linux/hidden.h + + export PIE_CFLAGS endif # diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index f19c038409aa..bccee07eae60 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -84,7 +84,7 @@ LDFLAGS_vmlinux += -T hostprogs := mkpiggy HOST_EXTRACFLAGS += -I$(srctree)/tools/include -sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABCDGRSTVW] \(_text\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' quiet_cmd_voffset = VOFFSET $@ cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@ diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h index 5c83729c8e71..b004f1b9a052 100644 --- a/arch/x86/include/asm/setup.h +++ b/arch/x86/include/asm/setup.h @@ -47,8 +47,8 @@ extern unsigned long saved_video_mode; extern void reserve_standard_io_resources(void); extern void i386_reserve_resources(void); -extern unsigned long __startup_64(unsigned long physaddr, struct boot_params *bp); -extern void startup_64_setup_env(unsigned long physbase); +extern unsigned long __startup_64(struct boot_params *bp); +extern void startup_64_setup_env(void); extern void early_setup_idt(void); extern void __init do_early_exception(struct pt_regs *regs, int trapnr); diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 0000325ab98f..65194ca79b5c 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -21,6 +21,10 @@ CFLAGS_REMOVE_sev.o = -pg CFLAGS_REMOVE_rethook.o = -pg endif +# head64.c contains C code that may execute from a different virtual address +# than it was linked at, so we always build it using PIE codegen +CFLAGS_head64.o += $(PIE_CFLAGS) + KASAN_SANITIZE_head$(BITS).o := n KASAN_SANITIZE_dumpstack.o := n KASAN_SANITIZE_dumpstack_$(BITS).o := n diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index d636bb02213f..a4a380494703 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -74,15 +74,10 @@ static struct desc_ptr startup_gdt_descr __initdata = { .address = 0, }; -static void __head *fixup_pointer(void *ptr, unsigned long physaddr) -{ - return ptr - (void *)_text + (void *)physaddr; -} - -static unsigned long __head *fixup_long(void *ptr, unsigned long physaddr) -{ - return fixup_pointer(ptr, physaddr); -} +#define __va_symbol(sym) ({ \ + unsigned long __v; \ + asm("movq $" __stringify(sym) ", %0":"=r"(__v)); \ + __v; }) static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) { @@ -99,8 +94,8 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv * attribute. */ if (sme_get_me_mask()) { - vaddr = (unsigned long)__start_bss_decrypted; - vaddr_end = (unsigned long)__end_bss_decrypted; + vaddr = __va_symbol(__start_bss_decrypted); + vaddr_end = __va_symbol(__end_bss_decrypted); for (; vaddr < vaddr_end; vaddr += PMD_SIZE) { /* @@ -127,25 +122,17 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv return sme_get_me_mask(); } -/* Code in __startup_64() can be relocated during execution, but the compiler - * doesn't have to generate PC-relative relocations when accessing globals from - * that function. Clang actually does not generate them, which leads to - * boot-time crashes. To work around this problem, every global pointer must - * be adjusted using fixup_pointer(). - */ -unsigned long __head __startup_64(unsigned long physaddr, - struct boot_params *bp) +unsigned long __head __startup_64(struct boot_params *bp) { + unsigned long physaddr = (unsigned long)_text; unsigned long load_delta, *p; unsigned long pgtable_flags; pgdval_t *pgd; p4dval_t *p4d; pudval_t *pud; pmdval_t *pmd, pmd_entry; - pteval_t *mask_ptr; bool la57; int i; - unsigned int *next_pgt_ptr; la57 = pgtable_l5_enabled(); @@ -157,7 +144,7 @@ unsigned long __head __startup_64(unsigned long physaddr, * Compute the delta between the address I am compiled to run at * and the address I am actually running at. */ - load_delta = physaddr - (unsigned long)(_text - __START_KERNEL_map); + load_delta = physaddr - (__va_symbol(_text) - __START_KERNEL_map); /* Is the address not 2M aligned? */ if (load_delta & ~PMD_MASK) @@ -168,26 +155,24 @@ unsigned long __head __startup_64(unsigned long physaddr, /* Fixup the physical addresses in the page table */ - pgd = fixup_pointer(early_top_pgt, physaddr); + pgd = (pgdval_t *)early_top_pgt; p = pgd + pgd_index(__START_KERNEL_map); if (la57) *p = (unsigned long)level4_kernel_pgt; else *p = (unsigned long)level3_kernel_pgt; - *p += _PAGE_TABLE_NOENC - __START_KERNEL_map + load_delta; + *p += _PAGE_TABLE_NOENC + sme_get_me_mask(); if (la57) { - p4d = fixup_pointer(level4_kernel_pgt, physaddr); + p4d = (p4dval_t *)level4_kernel_pgt; p4d[511] += load_delta; } - pud = fixup_pointer(level3_kernel_pgt, physaddr); - pud[510] += load_delta; - pud[511] += load_delta; + level3_kernel_pgt[510].pud += load_delta; + level3_kernel_pgt[511].pud += load_delta; - pmd = fixup_pointer(level2_fixmap_pgt, physaddr); for (i = FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) - pmd[i] += load_delta; + level2_fixmap_pgt[i].pmd += load_delta; /* * Set up the identity mapping for the switchover. These @@ -196,15 +181,13 @@ unsigned long __head __startup_64(unsigned long physaddr, * it avoids problems around wraparound. */ - next_pgt_ptr = fixup_pointer(&next_early_pgt, physaddr); - pud = fixup_pointer(early_dynamic_pgts[(*next_pgt_ptr)++], physaddr); - pmd = fixup_pointer(early_dynamic_pgts[(*next_pgt_ptr)++], physaddr); + pud = (pudval_t *)early_dynamic_pgts[next_early_pgt++]; + pmd = (pmdval_t *)early_dynamic_pgts[next_early_pgt++]; pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask(); if (la57) { - p4d = fixup_pointer(early_dynamic_pgts[(*next_pgt_ptr)++], - physaddr); + p4d = (p4dval_t *)early_dynamic_pgts[next_early_pgt++]; i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; pgd[i + 0] = (pgdval_t)p4d + pgtable_flags; @@ -225,8 +208,7 @@ unsigned long __head __startup_64(unsigned long physaddr, pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; /* Filter out unsupported __PAGE_KERNEL_* bits: */ - mask_ptr = fixup_pointer(&__supported_pte_mask, physaddr); - pmd_entry &= *mask_ptr; + pmd_entry &= __supported_pte_mask; pmd_entry += sme_get_me_mask(); pmd_entry += physaddr; @@ -252,14 +234,14 @@ unsigned long __head __startup_64(unsigned long physaddr, * error, causing the BIOS to halt the system. */ - pmd = fixup_pointer(level2_kernel_pgt, physaddr); + pmd = (pmdval_t *)level2_kernel_pgt; /* invalidate pages before the kernel image */ - for (i = 0; i < pmd_index((unsigned long)_text); i++) + for (i = 0; i < pmd_index(__va_symbol(_text)); i++) pmd[i] &= ~_PAGE_PRESENT; /* fixup pages that are part of the kernel image */ - for (; i <= pmd_index((unsigned long)_end); i++) + for (; i <= pmd_index(__va_symbol(_end)); i++) if (pmd[i] & _PAGE_PRESENT) pmd[i] += load_delta; @@ -271,7 +253,7 @@ unsigned long __head __startup_64(unsigned long physaddr, * Fixup phys_base - remove the memory encryption mask to obtain * the true physical address. */ - *fixup_long(&phys_base, physaddr) += load_delta - sme_get_me_mask(); + phys_base += load_delta - sme_get_me_mask(); return sme_postprocess_startup(bp, pmd); } @@ -553,22 +535,16 @@ static void set_bringup_idt_handler(gate_desc *idt, int n, void *handler) } /* This runs while still in the direct mapping */ -static void __head startup_64_load_idt(unsigned long physbase) +static void __head startup_64_load_idt(void) { - struct desc_ptr *desc = fixup_pointer(&bringup_idt_descr, physbase); - gate_desc *idt = fixup_pointer(bringup_idt_table, physbase); - - - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - void *handler; + gate_desc *idt = bringup_idt_table; + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) /* VMM Communication Exception */ - handler = fixup_pointer(vc_no_ghcb, physbase); - set_bringup_idt_handler(idt, X86_TRAP_VC, handler); - } + set_bringup_idt_handler(idt, X86_TRAP_VC, vc_no_ghcb); - desc->address = (unsigned long)idt; - native_load_idt(desc); + bringup_idt_descr.address = (unsigned long)idt; + native_load_idt(&bringup_idt_descr); } /* This is used when running on kernel addresses */ @@ -587,10 +563,10 @@ void early_setup_idt(void) /* * Setup boot CPU state needed before kernel switches to virtual addresses. */ -void __head startup_64_setup_env(unsigned long physbase) +void __head startup_64_setup_env(void) { /* Load GDT */ - startup_gdt_descr.address = (unsigned long)fixup_pointer(startup_gdt, physbase); + startup_gdt_descr.address = (unsigned long)startup_gdt; native_load_gdt(&startup_gdt_descr); /* New GDT is live - reload data segment registers */ @@ -598,5 +574,5 @@ void __head startup_64_setup_env(unsigned long physbase) "movl %%eax, %%ss\n" "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); - startup_64_load_idt(physbase); + startup_64_load_idt(); } diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 399241dcdbb5..b8704ac1a4da 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -67,8 +67,6 @@ SYM_CODE_START_NOALIGN(startup_64) /* Set up the stack for verify_cpu() */ leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp - leaq _text(%rip), %rdi - /* Setup GSBASE to allow stack canary access for C code */ movl $MSR_GS_BASE, %ecx leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx @@ -107,8 +105,7 @@ SYM_CODE_START_NOALIGN(startup_64) * is active) to be added to the initial pgdir entry that will be * programmed into CR3. */ - leaq _text(%rip), %rdi - movq %r15, %rsi + movq %r15, %rdi call __startup_64 /* Form the CR3 value being sure to include the CR3 modifier */