From patchwork Thu Jan 25 11:28:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192022 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573835dyi; Thu, 25 Jan 2024 03:35:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IE4AUfNShZ4vg4OloELd0adDy8mJTVaeQ+e6TIYtVPK6RjmtAhInceHqWpfkCygXD7W+yrI X-Received: by 2002:a2e:82c3:0:b0:2cd:cda6:55df with SMTP id n3-20020a2e82c3000000b002cdcda655dfmr625385ljh.62.1706182532761; Thu, 25 Jan 2024 03:35:32 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182532; cv=pass; d=google.com; s=arc-20160816; b=SKqcFhtEwLsW5l3m1Hfba+y6uqLPFnCjl6vNPiJdAuETnD0cMR3l3Jwkv/exBI1qRJ RcosDt+1PG55srh6yqj50GaamcSTOfE4WFWt7443gAMB/DYHSSWyALAbHs87ByNWWJ2R orssm5lO5C8pTHT3TSJOcIz48hBny8XFG+SFqA2XqrSZvq2DQTx1dcuGS6vKSCmKeFTV inNY2RJ27taFmCu40fmdZI0PHaO/NagHXMJwwbOveU/Ye08au5SK+RseapbZ1iSQ1mzZ eXrV0lpLuWMoArRREw6xMaexZeKEDH7FRueAONZMp+yQFljXkFD5VEJD1inlB3qDlv1F 9PIA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=RdRmN5FNC9Qw8qYorfxYqmWorkk70z2Gc350vuxR288=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=XKCVpb07/YyQozxcR6Pu2A954tv7BKbYRz911xjsMV8f7JS4FoDhv3hBpwt1GeQX7O gKZVQTPfhzjiBJxnLvrNpzLhdF8bu0jNWLQ/lRuqOLNbNkdK2fRsQaTfVhav9+0ilVLl wzOKsOVc2jj8A3fbkVII13bCB/lJ9dpJk2224ibX5R7nKeh7XwTbp61ivj45MK3jW8pp Z9wg2p5KPK/U4yFG3g4WL9EvB8NUaZYBuO7+I46PotlTr6SDKce0BhxI1zui+GmvAZzR SNAkhbsDzWWd2DoeJsjj0yJUjTSfMpgi3VDCxUl4TZqQUQu3QWfFlUhYJg9cH33RDOv9 Ipiw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=bCgvnz8T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id d11-20020a170906344b00b00a318faf3da4si267736ejb.241.2024.01.25.03.35.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:35:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=bCgvnz8T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 373641F241D9 for ; Thu, 25 Jan 2024 11:35:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6A9BE53819; Thu, 25 Jan 2024 11:33:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bCgvnz8T" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC8A3482DC for ; Thu, 25 Jan 2024 11:33:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182383; cv=none; b=RuibYks1Y+mp2VsMqsYlK6bCTt4P4GU241A9LadH4bil6XZpOKLTaM+6qno6UgqPvQyfDPYTQukocFmqOGDwey2bMUE/6p239RyxjdGfVK1VigkEqWx1hFk+Iko5QCDwI0b3BV5o6ijCgSicLqBNC2+oHGMnGWnBBIaYNwr6YA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182383; c=relaxed/simple; bh=nW9SmCdlADHOE6Omr4PFRwdaZThbp8qbwCml5fp8EWk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=X5Btt5WGZxFmswVVrvxE61A8hZ4/ESSmBN9B24QORpGd9KmAl6c5/o6TuAeNp1pEf8LT0+xlIxUL3HQ9lKzm3d+rSNbqAj8ltNvCfFNWG8h/7S7pA7zRA6Uu2dd3q2e4Qubfyi+LUzLIHqhUwazRdnb4aSF89mevIxXZgIo5EXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bCgvnz8T; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc2470bc0bdso8868666276.1 for ; Thu, 25 Jan 2024 03:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182380; x=1706787180; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=RdRmN5FNC9Qw8qYorfxYqmWorkk70z2Gc350vuxR288=; b=bCgvnz8Txbo20T4ZGLux/SN/pZi6d96ER9IuSuaxP3Gk0zphk0+N2Ppkp4Cc/TeLC7 pnHrNQTLf2n+UXXUbEnrE+p2zgbErCIQ4ruCbW5pSzMdwiVjGWbnpkrpdawgEFFCCWdA X3DcfNwr8Zn3SP67M8B/uG8R2FdS+FuV5lQ/Xw9LQ9RDlRoUgGrtdaRW58cFBZgl0xNC Qpwnt+4DX/IKygmLj03jF0eqVmugP6QB3B4Gqv74lhxkKukq0Hb4p1+rc1E1DAykFpmu Xj+BAARNwWMt0jNwvQG/DI2Ro22zMPYy0NeH8Ey18fbDk7ApNkqENThkKMfAzYvVYosj mKtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182380; x=1706787180; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RdRmN5FNC9Qw8qYorfxYqmWorkk70z2Gc350vuxR288=; b=xJzWsSh4RmOzO6gvcHGpLFBNWQE9bZe5nj7575pJdIUAowg0MilGrYcqAHvakpiyR3 lBKQlUsUe2ywvzKWkOErNeIDeIUBMsUji/0r8bnw02N5xtU55y1uWo/nusyVaFmT9m0d YM7IUEwWPP0qGB5gM1eNi/+/xo0GZ8UTeA6S8iB/OJFwF8e7bGUNYTbyhoSw/J1THI0X zjQCXawHY7bNZhuBkxhSjCchrt2iQrALyc1KQO5Wkn2vieoPv7GpkWvKNRT8xR0VIT7P myOopkmmlKFVq4EZ50GPzUEvRinhbGiggw7tj5tDcxhM6S/vjOyHEUikMC1tXZ9UF0Zn 56uw== X-Gm-Message-State: AOJu0YxWLX/IA+FC291LPQ3qqexT5ikk2pYKl7pmFzaaKoqoqzlK0OeU NBYN2hYxnEkyPLFjhjXspLT4PxOe/E2UAJDZWseHYkVgP0IFe60Itrz0N+gZwmP3WJfCdJfKpnN 81rUfzBqsor2aOHK+p8UiPNWp+9qqjHiMakKgZ5PMjll8Vf97wIrxdghk5/jxdgSgYuch5cdC1g gHi8G3FZjdeSDtTDgdgS5ojTI7WLwS9A== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:2407:b0:dc2:65e2:58f3 with SMTP id dr7-20020a056902240700b00dc265e258f3mr85436ybb.7.1706182380556; Thu, 25 Jan 2024 03:33:00 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:29 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3942; i=ardb@kernel.org; h=from:subject; bh=NDQz8Ub0YeUmHEZHj1/2gw+nBRrbtOLR0AuDt5JEbB0=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6+tOhnUbZt/4pv7feq263bHT+yJ7HN1ebknYPeV/T kbiHuunHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAifKYMf+Ue7yjove9+NW3v qrOrE8WS/Cq02Bf8ZNqfx9ay/nlsqxDDHy67p271b9cX1GyKdPLS1Im3nZDa0XjGtoK/97ZL+Rx VXgA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-29-ardb+git@google.com> Subject: [PATCH v2 10/17] x86/head64: Move early startup code into __pitext From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062055734616393 X-GMAIL-MSGID: 1789062055734616393 From: Ard Biesheuvel The boot CPU runs some early startup C code using a 1:1 mapping of memory, which deviates from the normal kernel virtual mapping that is used for calculating statically initialized pointer variables. This makes it necessary to strictly limit which C code will actually be called from that early boot path. Implement this by moving the early startup code into __pitext. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 9 ++++---- arch/x86/kernel/head_64.S | 24 ++++++++++++-------- 2 files changed, 20 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 993d888a3172..079e1adc6121 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -70,7 +70,8 @@ static struct desc_struct startup_gdt[GDT_ENTRIES] __initconst = { asm("movq $" __stringify(sym) ", %0":"=r"(__v)); \ __v; }) -static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) +static unsigned long __pitext sme_postprocess_startup(struct boot_params *bp, + pmdval_t *pmd) { unsigned long vaddr, vaddr_end; int i; @@ -113,7 +114,7 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv return sme_get_me_mask(); } -unsigned long __head __startup_64(struct boot_params *bp) +unsigned long __pitext __startup_64(struct boot_params *bp) { unsigned long physaddr = (unsigned long)_text; unsigned long load_delta, *p; @@ -508,7 +509,7 @@ void __init __noreturn x86_64_start_reservations(char *real_mode_data) */ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data; -static void early_load_idt(void (*handler)(void)) +static void __pitext early_load_idt(void (*handler)(void)) { gate_desc *idt = bringup_idt_table; struct desc_ptr bringup_idt_descr; @@ -539,7 +540,7 @@ void early_setup_idt(void) /* * Setup boot CPU state needed before kernel switches to virtual addresses. */ -void __head startup_64_setup_env(void) +void __pitext startup_64_setup_env(void) { struct desc_ptr startup_gdt_descr; diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index b8704ac1a4da..5defefcc7f50 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -42,6 +42,15 @@ L3_START_KERNEL = pud_index(__START_KERNEL_map) __HEAD .code64 SYM_CODE_START_NOALIGN(startup_64) + UNWIND_HINT_END_OF_STACK + jmp primary_startup_64 +SYM_CODE_END(startup_64) + + __PITEXT +#include "verify_cpu.S" +#include "sev_verify_cbit.S" + +SYM_CODE_START_LOCAL(primary_startup_64) UNWIND_HINT_END_OF_STACK /* * At this point the CPU runs in 64bit mode CS.L = 1 CS.D = 0, @@ -131,10 +140,12 @@ SYM_CODE_START_NOALIGN(startup_64) movq %rax, %cr3 /* Branch to the common startup code at its kernel virtual address */ - movq $common_startup_64, %rax ANNOTATE_RETPOLINE_SAFE - jmp *%rax -SYM_CODE_END(startup_64) + jmp *.Lcommon_startup_64(%rip) +SYM_CODE_END(primary_startup_64) + + __INITRODATA +SYM_DATA_LOCAL(.Lcommon_startup_64, .quad common_startup_64) .text SYM_CODE_START(secondary_startup_64) @@ -410,9 +421,6 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) int3 SYM_CODE_END(secondary_startup_64) -#include "verify_cpu.S" -#include "sev_verify_cbit.S" - #if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT) /* * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for @@ -539,10 +547,8 @@ SYM_CODE_END(early_idt_handler_common) * paravirtualized INTERRUPT_RETURN and pv-ops don't work that early. * * XXX it does, fix this. - * - * This handler will end up in the .init.text section and not be - * available to boot secondary CPUs. */ + __PITEXT SYM_CODE_START_NOALIGN(vc_no_ghcb) UNWIND_HINT_IRET_REGS offset=8 ENDBR