From patchwork Thu Jan 25 11:28:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192014 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1572771dyi; Thu, 25 Jan 2024 03:33:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IH+ON93OSohGQ1u3eqQBgA0dNTrH+JRGCCMPOjBd54yMZuh0+15BqFvVdo/MuGfV57sPflI X-Received: by 2002:a05:6808:2984:b0:3bd:4c75:e0ab with SMTP id ex4-20020a056808298400b003bd4c75e0abmr617544oib.48.1706182401812; Thu, 25 Jan 2024 03:33:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182401; cv=pass; d=google.com; s=arc-20160816; b=NlYW60R5rcFnvxh0bAmt9eUpWVOqbIqdBrBrgLuGIZ/U2slZxdAtpkDvjNVSxNJYFx gNNAp/QFnA3qZTtDsq9E5W3FGFJ1UbKP866CJ3iBBK8GJu4lF5KTYWj22PejvAspBo/0 sbdWQ4ume4wfnmBXJmZV/Bklywsh0O8yu6KhM6BtOqrcJfggaVioNlwbyS+1qMkgMGZ1 dPpRZRfCF3zyv/BEew0iT3ZW+2wICoV7WsfiEVpDonMTAVj12T6wloBwbdarI5B+96V0 vGgM+4cpmgzvke5NbIIsbA5JspBtrjoF/agGbPWwsOOGoqTjNSEm7SiFtOCnDJTuxBkg sjbQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=VlzuORvF37cHhQgjoovr2a86HYjlNB8YTzXdR4Eq8S8=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=MbQnTmzj9knrhaxwmOendE9Kaze5JSAfoNPRs38PoCApKLJh+K90RpZ3A0yRemBPr2 3FyBrGrIRJ6MvXItBUbZIMjdHIGpqkxfJIFVc7dU0pD1mZuysF+oXN/SDaQxiHAmCNP2 KCBRBGMFRLl3PfvWJM1/aU0PANabcm2BzLbWSFcByyFrjW1yj6CJiUiEram4Vfl0aWkC KNtExMF4OSpVYvNshm3qwaVB0FugVcaa+bsU4xCD68Zc6BQjTXbyNisucHhX28UNmyIS Ax70H7PWrM11MxzPI06GsuUztYBUQOA4lpCysf8NKuIS5jEPAcC4tHfRkEZy2bGxMwsZ ujZA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b="y9Cdx3//"; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38505-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38505-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id o11-20020a05622a044b00b004283ab4f730si12259136qtx.798.2024.01.25.03.33.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:33:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38505-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b="y9Cdx3//"; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38505-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38505-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 864A21C229C0 for ; Thu, 25 Jan 2024 11:33:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A6A5F4123F; Thu, 25 Jan 2024 11:32:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="y9Cdx3//" Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10FBF2CCAD for ; Thu, 25 Jan 2024 11:32:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182363; cv=none; b=FoWSy+FDdTIODBB26JnF7ARLHpRFiNAm3/2lkIx35cdN8q/iuOJzaVGo9aYtEjut2chrunQG3O+ZlHRitlj39wOH+qUzbBitmK4/LlfqOcErObjv3HSMwcHo6k0BVOFJJUDPjuXzQeW3elRSK8kBBGbcBZ/FjVcqye9566FfEzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182363; c=relaxed/simple; bh=ksYtquI/zNJoo09z1kn9EIIXegn4h7RVzbkZ+iPv7gQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TDa2wLBHY9ulS70450XvmV3x0TS5+CRU3YKsK/9VMxRqQJfKhu1eb5/fsFTDdAyfBx9m2gFVEszIyGmD+UukPkV0HzoDhiTck7hVIUwPt+6c0TnvDgCxChsC3v7XQEMo5xu1gXayn5rhzcV62vb0moES0kprKwB/hLADmERpU20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=y9Cdx3//; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-40eaf5c52d3so34985995e9.1 for ; Thu, 25 Jan 2024 03:32:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182360; x=1706787160; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VlzuORvF37cHhQgjoovr2a86HYjlNB8YTzXdR4Eq8S8=; b=y9Cdx3//r/vJpa/ctay760QmH747x6gRelL35ji6SSGpB7W1WT+qbtofUZFip8ilLN Qsj+Wob4Eh++Z+EFqd2glypUpg8sQioa3IcHyxrU3TFcfK4xv2cICDQRTiVpK7JpCA2W Nw5lC+luKKdAXrDKhL6z5b/81pbQk8p9QV2tNJs2PY2rEbeEF+mv7tBUN/W7coa3LiC7 i4WPUl6RRCcNk6cN5QPn2WWufat0n1YCKnNNI4AV185nnOfJ8d1Z6PiX0zfgpQHxhYHG SECQyLJFUBT7I1j4+PLzdPKzROI+KZFd5ite/NN1pRgHVpCUrkEXr5qBpXmj0NISsiKA b3Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182360; x=1706787160; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VlzuORvF37cHhQgjoovr2a86HYjlNB8YTzXdR4Eq8S8=; b=EcepAu1izAUfqiXOEWvwD8HwxUcrLQLJuGTv/lPS16pMU2C1NLVHUyAPezlxeqBp/h AEg/ZOtoMEF1naLUFkYwGa3DzUMrDZLlL6Y7SIxKotbGzGperTjpxzMqbUT92v8QzO+D K/Qvd+R6T7j1C0pkYWdNE9uu9PIpaH9BXIYAN/mPWLqN6PI6qLbabHRjTjTKaEqPtdcY BLV6KO3qhmGN/OmvrwlZZGjUU7UOjREDfKqJ0Uuf9a459XkGLegu4yT8y3EfE8ceuGtg 3yYcIAgxLbzo1zfV8CnPus6xjBjJQdm9YO5imoxkRGJQfxVQu++S4fOdGNe7JK9HzV3V cplQ== X-Gm-Message-State: AOJu0Yw+gorskywOORNKpR9C39vdG7c/kO5Id40qf0HuPq7seIMTgulY QIIgp8uM/fq8xqLnV/kIEXqK3oFoZCXwR2UEB4THh+CJaP9EdxsY/Zbb0jdmIIC68ebpokhnl5d RKATE+tB7YY12+9ItFTKk8ut0Xa6Ym0mZdA9lUkYv2pYB35jtGNXwwi81MVbKf0pYvkYYZUok75 N0glMrZFV/NnE6csR+e/dZdoHacTR3dw== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:c0f:b0:40e:d1ab:3421 with SMTP id fm15-20020a05600c0c0f00b0040ed1ab3421mr5142wmb.5.1706182359900; Thu, 25 Jan 2024 03:32:39 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:20 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2268; i=ardb@kernel.org; h=from:subject; bh=TSUaLCMOyIRIkxI+e58H4dzOT8JQVh2xiOSo20k09Yg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT69WZTg+YSttzMxXl1N4f2yxxuu1fZ8zluTsOCJY1N PRMiVzfUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACaSGcTIsGH3y0MrT1/cVLj5 bzrbMoGWoqPnJwQ3XZov0mO0bfWKo6aMDP+ecz9d8DR0D9NBuZinnFO5PjzMFnn5V+p00feprMs 9g/gA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-20-ardb+git@google.com> Subject: [PATCH v2 01/17] x86/startup_64: Drop long return to initial_code pointer From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789061918336083978 X-GMAIL-MSGID: 1789061918336083978 From: Ard Biesheuvel Since commit 866b556efa12 ("x86/head/64: Install startup GDT"), the primary startup sequence sets the code segment register (CS) to __KERNEL_CS before calling into the startup code shared between primary and secondary boot. This means a simple indirect call is sufficient here. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 35 ++------------------ 1 file changed, 3 insertions(+), 32 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index d4918d03efb4..4017a49d7b76 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -428,39 +428,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) movq %r15, %rdi .Ljump_to_C_code: - /* - * Jump to run C code and to be on a real kernel address. - * Since we are running on identity-mapped space we have to jump - * to the full 64bit address, this is only possible as indirect - * jump. In addition we need to ensure %cs is set so we make this - * a far return. - * - * Note: do not change to far jump indirect with 64bit offset. - * - * AMD does not support far jump indirect with 64bit offset. - * AMD64 Architecture Programmer's Manual, Volume 3: states only - * JMP FAR mem16:16 FF /5 Far jump indirect, - * with the target specified by a far pointer in memory. - * JMP FAR mem16:32 FF /5 Far jump indirect, - * with the target specified by a far pointer in memory. - * - * Intel64 does support 64bit offset. - * Software Developer Manual Vol 2: states: - * FF /5 JMP m16:16 Jump far, absolute indirect, - * address given in m16:16 - * FF /5 JMP m16:32 Jump far, absolute indirect, - * address given in m16:32. - * REX.W + FF /5 JMP m16:64 Jump far, absolute indirect, - * address given in m16:64. - */ - pushq $.Lafter_lret # put return address on stack for unwinder xorl %ebp, %ebp # clear frame pointer - movq initial_code(%rip), %rax - pushq $__KERNEL_CS # set correct cs - pushq %rax # target address in negative space - lretq -.Lafter_lret: - ANNOTATE_NOENDBR + ANNOTATE_RETPOLINE_SAFE + callq *initial_code(%rip) + int3 SYM_CODE_END(secondary_startup_64) #include "verify_cpu.S" From patchwork Thu Jan 25 11:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192015 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1572919dyi; Thu, 25 Jan 2024 03:33:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IGN4dQ8+l0FJLJ0qOzvOOe/KhgVu8eKU2Nk3wngm81QJ3sh4SyoYQoYCeub11ND4wqciiTF X-Received: by 2002:a2e:a40c:0:b0:2ce:fa57:4dd0 with SMTP id p12-20020a2ea40c000000b002cefa574dd0mr476471ljn.11.1706182418084; Thu, 25 Jan 2024 03:33:38 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182418; cv=pass; d=google.com; s=arc-20160816; b=K9j+6VcLhvdWFDAoKVy8FA5lpcJlMpwyCVbWgV7Fd4Xedgna/UCzFfghy2aDLMjIB4 LhoXBIXQBJbfKXCquvPmvjQSj+LvcTK+u6v+EbwDSXg0JDGiPFFhgTXnb6zVCSVyjsqF F5dvRQku3hoDK6Vah/o0sOKvN1F9vE02tJLDbUCcK0iFikvTH/GfghLzfWSj65DXLoBp DVyc1DEulR1w9yA6O9oGoqv7HRVV2hFCVLNViOEFIBAmgDantVizQPVEsIgNDh5bTHz9 Munkc0pCdN4ddfHebf8WPeVjCNNgQl3n9tYQjZy8QW0cE1wJ7BlymEUOUpCXJ0bmpZa2 H7xw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=iHVLq9Wa6m8PRjV68aukFNmIoFWxPV4eVUMjwp/oWNk=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=nbzg1Yr+E+6AsZi+1H+2AS8u9u40NnDLPj6hVZUEdNMyWXjSJL1S95vIvovTOZkIDl gqbWGtucmFiXG9YUHArbIt12Ordw/p0sBX1PRGVlkMBYMMIDT3eO7+N3vxUVj34yZER4 3osX2ltZb1FpD4qnPsbpofzs2BP1NBzJ99t7oKUb8GtTPMuuj0QpFT7Hu4EjNOUaTaua SZrAHCh8FC2LOo/ADO5N60h/bq5ujfkdt74qD7SzoQc/5tqw6IljjbpONoUMKzusC+Xa qqI2+Hpnz/duh+i/aSDq+UzVCokwLsZOcK0Zcu548HmbFZaJSyG7JyaYxX7QVw3Nw4qx 1jLQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=le7noE1q; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38506-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38506-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id h9-20020a056402280900b0055c5ca0f656si3661527ede.503.2024.01.25.03.33.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:33:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38506-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=le7noE1q; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38506-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38506-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 83A5C1F23D62 for ; Thu, 25 Jan 2024 11:33:37 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B37DD4594C; Thu, 25 Jan 2024 11:32:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="le7noE1q" Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 152D833CC9 for ; Thu, 25 Jan 2024 11:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182365; cv=none; b=idXcCyHPX2eCNwbSIp1uP17KCif0FqBa9C5bsiXZBdua4P0ookbyb1ibPsYoWFUNMLYcD63JxhiEcqrk1zdaBCulPQURDh4xFrGYdisOY/sVhpZkjuraOrSsNfxKM+Q9EzGERACUouJpDm4JWqkCSQp531gdIW0HcB7dzyqWfxU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182365; c=relaxed/simple; bh=NfXCrvSrRHu5/o/a38jIuQQEOM5CDJx3GD3fZgnkl0s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gmQZqiSXxKi01UiCHOsti2wYkpuH7wmPkEYaOGFOyz9YyA6u92asm08MPWw1upeDz7IF/JOGRR3TX5+m63VAOFbhN6wBIzmmOo40V+6bkwBjfEsUQZKH+zLGHZ8WyRcuOOugmhqpsQTC9UK0BNoYKUjqC7O16fPbtlLK6lzlte0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=le7noE1q; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3392c5e6dcdso249030f8f.0 for ; Thu, 25 Jan 2024 03:32:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182362; x=1706787162; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=iHVLq9Wa6m8PRjV68aukFNmIoFWxPV4eVUMjwp/oWNk=; b=le7noE1qf7mBa1wSBSLwLBSR76Pa9kx7N7YBkZGuTxppl23kHAp6gMLMWmAOvoeKEL DEVQTKZt21rJCv5Sp4O8o02+tEGPAawBRpOaLN2GIXyArDf4H0e5HuIIuUgDy9kSOnyx oTg81+DcEv3iwigMp7lBwpKw1gO7koP+yxrCC5205dutFJRoYqK41P3zR563vQdpaRAC emzZM9lCrfvEgFQd+hFPEJu4Fl+fDLmrNLRR3ap6t19ILXLptkxVRE13cMCv6YhQMlm3 YcFj5tzSdUjk7frSunXiFfx7VXm2G+xalp+5nkwllU95ZvJ8VA/n7APcnkJADVOQE6Zb SmPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182362; x=1706787162; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iHVLq9Wa6m8PRjV68aukFNmIoFWxPV4eVUMjwp/oWNk=; b=p76YVR31MrdAZWDZL2cygahOq877wu7eA/+WBdWa3ZZbJQRqV6B09MsF/Bhf2Z5p9X FQppvIvMC9HB2kksOuYRHIru9qOzYFZb43d7VLQ0FNK9yS96VdgEM7DkALp6/3AQaYGt lWHh3PSOL/1pmJyl3luGGrYBjD+BgNTFRTBFk6+PWZGtMOnK/Ud/QpgSCnh1SB+rZlpQ mI3Ie38f0a+NFFusK104LxFFyQBgBPPgDKPtPutFxXMFCECrryMMHOFOZgk4eq/X6Y1Y Qd0wUdraYROjPbf0mzxr1dUtFk85jeQ+CN8pR21y1cwMOeGh76liQCTlNqD/iKw6zk3x NS8w== X-Gm-Message-State: AOJu0YwsK7084vSiZ0KtiGXK2c6eHUyY8wDT5YIOVyj6CTfxMhiCmw6r djad5Va6UA+qGZ9hLSYXKd7BZNZpnWU2e5ISHAVXGi7zk7kf/+lzafm9KzLGLiZWOvVa0cjKGxM CUe4Ula3Jjw9FWEXrbL9EnGZ9jNsut11yZRsBklGlaCVBIxtxY+tKZx2l2iB540bdXxBb7e4PQE bCIYLGH5aInsl+O7etZCTLt7JH0biy8Q== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:3489:b0:40e:d20f:6e43 with SMTP id a9-20020a05600c348900b0040ed20f6e43mr21711wmq.1.1706182362257; Thu, 25 Jan 2024 03:32:42 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:21 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2687; i=ardb@kernel.org; h=from:subject; bh=XmOltJN4+bmiY4MWHeNLYuJPJ6TOu7DKSx2rFcdng6s=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT63We1716198s/rI9xig2a4ogw49NN/7UfS092zFnr vG19M3vO0pZGMQ4GGTFFFkEZv99t/P0RKla51myMHNYmUCGMHBxCsBE5M0YGfpWObw/1+Pu4DHj O3NQwTOG9GOlO/nK+iwWHXXX36D7O43hn7nK/YsyjduO3U2/XPBoM3+7+M7T4tcKePreLt7y9RX 3D0YA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-21-ardb+git@google.com> Subject: [PATCH v2 02/17] x86/startup_64: Simplify calculation of initial page table address From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789061935272076112 X-GMAIL-MSGID: 1789061935272076112 From: Ard Biesheuvel Determining the address of the initial page table to program into CR3 involves: - taking the physical address - adding the SME encryption mask On the primary entry path, the code is mapped using a 1:1 virtual to physical translation, so the physical address can be taken directly using a RIP-relative LEA instruction. On the secondary entry path, the address can be obtained by taking the offset from the virtual kernel base (__START_kernel_map) and adding the physical kernel base. This is all very straight-forward, but the current code makes a mess of this. Clean this up. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 25 ++++++-------------- 1 file changed, 7 insertions(+), 18 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 4017a49d7b76..6d24c2014759 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -113,13 +113,11 @@ SYM_CODE_START_NOALIGN(startup_64) call __startup_64 /* Form the CR3 value being sure to include the CR3 modifier */ - addq $(early_top_pgt - __START_KERNEL_map), %rax + leaq early_top_pgt(%rip), %rcx + addq %rcx, %rax #ifdef CONFIG_AMD_MEM_ENCRYPT mov %rax, %rdi - mov %rax, %r14 - - addq phys_base(%rip), %rdi /* * For SEV guests: Verify that the C-bit is correct. A malicious @@ -128,12 +126,6 @@ SYM_CODE_START_NOALIGN(startup_64) * the next RET instruction. */ call sev_verify_cbit - - /* - * Restore CR3 value without the phys_base which will be added - * below, before writing %cr3. - */ - mov %r14, %rax #endif jmp 1f @@ -173,18 +165,18 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* Clear %R15 which holds the boot_params pointer on the boot CPU */ xorq %r15, %r15 + /* Derive the runtime physical address of init_top_pgt[] */ + movq phys_base(%rip), %rax + addq $(init_top_pgt - __START_KERNEL_map), %rax + /* * Retrieve the modifier (SME encryption mask if SME is active) to be * added to the initial pgdir entry that will be programmed into CR3. */ #ifdef CONFIG_AMD_MEM_ENCRYPT - movq sme_me_mask, %rax -#else - xorq %rax, %rax + addq sme_me_mask(%rip), %rax #endif - /* Form the CR3 value being sure to include the CR3 modifier */ - addq $(init_top_pgt - __START_KERNEL_map), %rax 1: #ifdef CONFIG_X86_MCE @@ -211,9 +203,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) #endif movq %rcx, %cr4 - /* Setup early boot stage 4-/5-level pagetables. */ - addq phys_base(%rip), %rax - /* * Switch to new page-table * From patchwork Thu Jan 25 11:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192017 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573029dyi; Thu, 25 Jan 2024 03:33:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IHHXAE/neAuQaiXGAC2TCBFAzMfkwWx9Fzcu5VRkFGb0YevLHWbaU2F030OmUxdn9d8qIPg X-Received: by 2002:a0d:cb46:0:b0:5ff:5bac:a28a with SMTP id n67-20020a0dcb46000000b005ff5baca28amr437391ywd.76.1706182431599; Thu, 25 Jan 2024 03:33:51 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182431; cv=pass; d=google.com; s=arc-20160816; b=WiILwUoPu+0PXDhj0qeNkhtM5AmQfesaa85Qhr+1FMy1/ZMu0NLlpqjqG03xscT3nZ KFQJmv+e84kTvjzuk05b8vArpJB3Yw1or8AY7U34f+C02orQq5WmhgPdvhxdY/Tkj2MR hC0WqObTqLKNyi9AoyADCv9BclWY8QREh5ZrVjT0BkT4IE3zTMyfgWsag3MMjHW7SIvu 18836FxGY9/Ip96NpkfARFDYj71C5KJQCDtbsfiMOQKsdE6gxWG2f0+gW/qiZWwDwFrq RbzMD9vVyctQyJuOspBs9wJv5j9CJol0oDPJUqX+Td3CQqd90jg9UOrIn05qnira35sz 88oA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=S0g+jX4epnZI/ommXCdUCoMP4YiUzpooZ3mcs3Px4Jg=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=fmtIeq/0rjoKF0Ok7pHgIaMhD3C1SmsO3cPFfGDBDAJmAxWk1JrVpfblAW04PYl5GK 6AFyUrF2SLsMniVQoUMMfxeYzwRuF0cU15LgMeqHyZe9O9C1ZII4dW0K1L/NUB50T9Ny ZfyabYsMnarQ3Mw1iJp0N/eleO8wDQVPj417DBV2UXV+qg7emCK6IDNzqb7M+GJuWGn6 HunxrPzW2kpt5bxn+jAJkKOGKYFBDrQtanSMSe27y86jA9UAkJq7Z/W3CxcoXO+UYt2s xUfUv/AaaMs9uU97++Iw+Mse5wSAryZEg9tBtEYPzxG+QFSoVVhnhByt2t+/jGMQ8krA JvBw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=qugzgbHX; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38507-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38507-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id v21-20020a05620a0f1500b0078321fc9bd9si13411405qkl.367.2024.01.25.03.33.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:33:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38507-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=qugzgbHX; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38507-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38507-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 556511C224C9 for ; Thu, 25 Jan 2024 11:33:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 09D51481C5; Thu, 25 Jan 2024 11:32:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qugzgbHX" Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 699873EA62 for ; Thu, 25 Jan 2024 11:32:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182368; cv=none; b=DaxyUN3UDWED8g4kVnB9qaTFdN/u5PClvEQF6xwkJFUNFFgUSms/za3TzJfYK32t9CtAZo60s9xOUrjgofU9b6E+RWJaRgpLHGngLH1LRBNBRxjVW5iacfkQ3Gjkf0oDTtH2M6YQB21wkoRRL9Pll06kqNs4PIsiV83Cnvs1IIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182368; c=relaxed/simple; bh=mnmwDuuQkke36VfancYNvAaxHSgDhivKxa5loDOkS2c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GQSscS9T9GeKdLF4GkIwuYUtv89VeGsD0umI+1QrpXz2Vs0z6W8V1RSHyYWfZsdFcMiGNk1DDHI6ih5exZzc5G3rYCsYNd7ES9jtjq77mKMtU+UKVmG1skDSpwbBdOkCYf3BIeKK9IeiFGf5wUl0tBwSEqUSnvpExmEOAFRoWwo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qugzgbHX; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-40e40126031so62680105e9.0 for ; Thu, 25 Jan 2024 03:32:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182365; x=1706787165; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=S0g+jX4epnZI/ommXCdUCoMP4YiUzpooZ3mcs3Px4Jg=; b=qugzgbHX1rMaF5lqWyXeqvGOlJXLp1tvETs98a1sGz0+oWptkSv/uhAUwAkJ70R+xs +wRQ2+IISVWKNcwe+RYAJVREIMBsRwHv7ycoDZ/iKSqoI9PFZSe8QTIXkVWyRMk4G0H5 4OCJF9yBDRpdnyKU90r9K+noV2lZTvj+2OLo0hK+88XGQqP1K2iiz6aQX0Hkv3uqhUJX bddoNEy7K4RSVVoJEOJ2WK1pn213blkl8mb5zRFQyOeZhAXZlWqD0RSeGbiExbZD1uy7 S+7Rr9FSSNBNAa+izcxeWvVIAygMUqPvcsS95RUKSsZqlRcv3MaVFvPOV2rGl021j+zR ZHoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182365; x=1706787165; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=S0g+jX4epnZI/ommXCdUCoMP4YiUzpooZ3mcs3Px4Jg=; b=L/+ZKuoLhMW/DNgaGd6rPeCTGRTO1TjWzDjVTIZIejEMGOZaeAr6NwJJtQ93qMO1uy FKVutJuk+NDfZYIFlxZYgC26Av4+KnrePbmYD/MnIV0PX2qB7nyz7+Ln99LuAWza6407 z0okncv+Ib9MqB01x9iQXsA7B8uPXFEwYXsjBpLAV6NzEsYdUnReawieEN8rnbar4cSr gImQ4pIJIB7pzXFKspORS81Dq12qRsChWf/ha5fuP/31aPEgIzhb5Rn1pfKuxGC0Tx05 40RVS9knSHSV+upKi8V6nBlLY2yx9ivsuD/q7qNQ05MqBOBgONGvPEncckgx7ce0dzv+ c3Fw== X-Gm-Message-State: AOJu0Yxw/KBxojwOr5JPTkzvZaPTAIe3IB7zQvhQJbNFYLyLYse21ta4 ibmRz4iJar3vqPb43ZGN65+2uQHM1qCDNcbk7k2BT/9lxgpTvbyX74HTV4Ir76gz/ddOMSNhske dULhOdWQDoV66XKuxW/jMY7tRde4z7rKjLkizmMB/zT96XBdjmTWQxZx5WUDBW9DRYdQypxSVQt KVPeEKC6CEm85PQa8CLCyP+zBci94z+Q== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:b8e:b0:40e:cee1:a3f1 with SMTP id fl14-20020a05600c0b8e00b0040ecee1a3f1mr9662wmb.1.1706182364321; Thu, 25 Jan 2024 03:32:44 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:22 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2467; i=ardb@kernel.org; h=from:subject; bh=PKWmule5OrE2bPivbVWTqIoJwY0uJ2wdrmc5K/+7smk=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT663+0nAxozus5wUbzy1OXrD+cV2V7v20XxsYTc5+k f13z29xRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI9L2MDL372TmuvJBbWXus /djCrQU5T59Uznngy6q5dE+XJs/UljWMDHfzts/v7rPvUt7NVRC76GbrZakMA56gShW1vI1qEt2 P2AA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-22-ardb+git@google.com> Subject: [PATCH v2 03/17] x86/startup_64: Simplify CR4 handling in startup code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789061949288457851 X-GMAIL-MSGID: 1789061949288457851 From: Ard Biesheuvel When executing in long mode, the CR4.PAE and CR4.LA57 control bits cannot be updated, and so they can simply be preserved rather than reason about whether or not they need to be set. CR4.PSE has no effect in long mode so it can be omitted. CR4.PGE is used to flush the TLBs, by clearing it if it was set, and subsequently re-enabling it. So there is no need to set it just to disable and re-enable it later. CR4.MCE must be preserved unless the kernel was built without CONFIG_X86_MCE, in which case it must be cleared. Reimplement the above logic in a more straight-forward way, by defining a mask of CR4 bits to preserve, and applying that to CR4 at the point where it needs to be updated anyway. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 27 ++++++++------------ 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 6d24c2014759..2d361e0ac74e 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -179,6 +179,12 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) 1: + /* + * Define a mask of CR4 bits to preserve. PAE and LA57 cannot be + * modified while paging remains enabled. PGE will be toggled below if + * it is already set. + */ + orl $(X86_CR4_PAE | X86_CR4_PGE | X86_CR4_LA57), %edx #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. @@ -187,22 +193,9 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) * configured will crash the system regardless of the CR4.MCE value set * here. */ - movq %cr4, %rcx - andl $X86_CR4_MCE, %ecx -#else - movl $0, %ecx + orl $X86_CR4_MCE, %edx #endif - /* Enable PAE mode, PSE, PGE and LA57 */ - orl $(X86_CR4_PAE | X86_CR4_PSE | X86_CR4_PGE), %ecx -#ifdef CONFIG_X86_5LEVEL - testb $1, __pgtable_l5_enabled(%rip) - jz 1f - orl $X86_CR4_LA57, %ecx -1: -#endif - movq %rcx, %cr4 - /* * Switch to new page-table * @@ -218,10 +211,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) * entries from the identity mapping are flushed. */ movq %cr4, %rcx - movq %rcx, %rax - xorq $X86_CR4_PGE, %rcx + andl %edx, %ecx +0: btcl $X86_CR4_PGE_BIT, %ecx movq %rcx, %cr4 - movq %rax, %cr4 + jc 0b /* Ensure I am executing from virtual addresses */ movq $1f, %rax From patchwork Thu Jan 25 11:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192018 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573224dyi; Thu, 25 Jan 2024 03:34:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IES/9eZo11ZhleYsnnUWU1je+oPHPodrG+kdB+FKvu9vA8Bew+mevKLug5uedsNctyYnhNM X-Received: by 2002:a05:622a:1b91:b0:42a:32e5:85df with SMTP id bp17-20020a05622a1b9100b0042a32e585dfmr1169321qtb.113.1706182454281; Thu, 25 Jan 2024 03:34:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182454; cv=pass; d=google.com; s=arc-20160816; b=NgupvKEZ+yM/P6jOW2JGFxIIpAYm9VQ8Bo0Mmit5ymhB/nO4sBuP9SMSDFoQL2XlCa 4L8WorUmGtxG1zXJVn9kTmaTGOGH/K+ki5LBIisYyhnnJKvrpsLrfgfgb6B9xMmTmWmX KA6nievKjHFyme1u+9A2yL2i02ZeGSrbPVaNbhTVb8i9L7YlCT6JdC6fomEXyK54dMZJ U7S9QEttq6DEKdUQ47wm2TyH/XPublGWBOKp0fJfAt0O28Wf6p5gTY5+JO6Ia3mxgcmN yKfqOvCb/HNsZOHzyKIoXP56L3CAhOWu35e+ypBD73p3yD9HFs72Oph7jmr0bcBO8B/h qapw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=Qnt2FLKZgiwEBYPehLp29kLTMnOKVwtNBBB/C6C+Er8=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=dWvpvrm2h84vK6DoQ7KGGR28YFBoTI3xEsIBpqZuNVg1tq9TTUHAT1fxMbX71ydZZ5 Q1C5sHPbzHb/61Wi3QQPEmycJASKnYraX0sL47c8EuEWhIW+AB98Qr6Kb1CY++lOG8NB dj5XUo6a5Gcipjdnu7QTN5/2oohRVNZFYypsIJaoV0z0joAPXklhAPyjn9PRiNgS/175 9Bp4/Q6fmj/guE+0eWPgxsqHfrYKWJ5rSmtPAHBUtZiTFOlzLo3wQB6hzgczg5LHRPWa x5/3KX/BtdrTU2Uu64SQOujBoIA/qIWx28I4ciyGynw71PEZB19JkwggoJTKwz/QvCBe coeA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=kwMLQVLV; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38508-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38508-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id o2-20020a05622a008200b0042a3e1d1274si1499392qtw.243.2024.01.25.03.34.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:34:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38508-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=kwMLQVLV; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38508-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38508-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 068311C22F5E for ; Thu, 25 Jan 2024 11:34:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 729774F613; Thu, 25 Jan 2024 11:32:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kwMLQVLV" Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DE3345037 for ; Thu, 25 Jan 2024 11:32:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182370; cv=none; b=S+xlnHMkeXhtWmZryubK+xQKcoKHADkGB8jI05atOO8mYbPMhBQhioX/mXysvirEpeIqTQXqAdh7+Hw+uzxCF40vTFFzplMGWGc+66U8jmjdoh6Si5LQeizGkMb8orLitC3eFuSwVYosMvLzAwZP80umP93ezMMQ4ItknCiAZb8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182370; c=relaxed/simple; bh=EKses+iAL1UalTo7h086P7oqMox/oN2hrVZ5jU93KrU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TaLwwcm2j9Mp99lGO3vDCzQfxooAA90qFXpDnzK/HjJEcPmm/uz7thH8Qz89h/+irkDxOkoO+8Kt6/UF/Fizq/l7mhOrYFjdNnTOMW/qawJALoRnjSFoEPks0JHf36ANdykP8YIWgIFg+y3wFVY8zEUL4k0eYp2ijfc30I3tOmE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kwMLQVLV; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6029c8e8758so21158217b3.1 for ; Thu, 25 Jan 2024 03:32:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182367; x=1706787167; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Qnt2FLKZgiwEBYPehLp29kLTMnOKVwtNBBB/C6C+Er8=; b=kwMLQVLVI3r2POP1CxwjAUn10vQY1N88eWSfsrEvzljcBrE/nnGm+9VKB44FLYm/5A P2LqNHftlCsvCREn9Cot3Qyy4a1Zbw6V+cqsnWzuNo0ltbOnjq6q/w0EFWAgm40+UwbU L0OxGebMCnteWhg6b3Hxgk3gOp6P4/yjdeoPnX2OjeY7PTwkp+lzPHkTLJaLw6kI9nzB NL7mWawFOqZPR3vtV496a30uorAC4A1nsI1t07OxD7IiQxxCIz0lX4C05KjC8z7ABlUi YN5XP8I6FzuWyNv31Nq/O4i+tZ8Mug5OixBtPup7GLbiU0wWMscW6F+Y5M3JaTHMYj5T 3hvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182367; x=1706787167; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Qnt2FLKZgiwEBYPehLp29kLTMnOKVwtNBBB/C6C+Er8=; b=Or+aqk8aKRz3jzN4h7pWyiOFLfz7nLSOZvx/9He3UadLc5H/1lJDXdWh9AGq14zZyT osotdJBdnlkOXI8XCbTnue/NguVZLn8lgPiCwKaCCww/tn6a5cNtedJB0ovcSbCI2qgd wtRrN14BTjrnjP1rpKg4vUHW/1NCI4E1Y7x1GV+o3fucivng1SCB0u7UmwcbyioS+279 m1IYP4PimKRoxtlCwMh7XKzfU2aSDRFx8mW4aSiH4ccQLjWvCyrLRVnMH0OxdLQ8zsq+ UotSpsffwF3CuT+b6yrNEcx6x9TX6gpDGhZ4fZPzZoL2022Q5GDJ0T3/rZv4jJ+UJ3gL A/pg== X-Gm-Message-State: AOJu0YykAMUnpGOK9NuU1BCqDk4YGLuqGTOFmDAFerUCkBSIQzkrzVAW S9W4oWMCA5OkMu7jJpIvVuFvW1afgvxuo8nJi4TzKLzQuTxvn/xW7XiNT1YXCMyhY9xA3Bs979x HWljJwfyTK0QY92txL7V74ov8Zec+M5fj8npSQ4zR4E4qtGF/mZGqo+S+/CtKNJJgNiP+QOPrFS +i02SlnBk6A2MW1lx5gB99zDwvdkJytQ== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5b:ece:0:b0:dc6:207d:7797 with SMTP id a14-20020a5b0ece000000b00dc6207d7797mr60231ybs.3.1706182367074; Thu, 25 Jan 2024 03:32:47 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:23 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=7926; i=ardb@kernel.org; h=from:subject; bh=3eah7XU3z29gWglaF/LixaexnChg/lgSOeC5moLmsUE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6133PIb1SfHPmS+aX/j2c+EPb78zV5keLtcujM/lu OX9d71oRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjItz5Ghmlf//zyvLg17Yjh lNeSDW815f558k0o/XrmxLWJlZfv8rAzMlyqefc15QXLc5t1eSdreD3LO3446/yZJX9m2a582QO RK/kA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-23-ardb+git@google.com> Subject: [PATCH v2 04/17] x86/startup_64: Drop global variables to keep track of LA57 state From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789061972872567547 X-GMAIL-MSGID: 1789061972872567547 From: Ard Biesheuvel On x86_64, the core kernel is entered in long mode, which implies that paging is enabled. This means that the CR4.LA57 control bit is guaranteed to be in sync with the number of paging levels used by the kernel, and there is no need to store this in a variable. There is also no need to use variables for storing the calculations of pgdir_shift and ptrs_per_p4d, as they are easily determined on the fly. Other assignments of global variables related to the number of paging levels can be deferred to the primary C entrypoint that actually runs from the kernel virtual mapping. This removes the need for writing to __ro_after_init from the code that executes extremely early via the 1:1 mapping. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/pgtable_64.c | 2 - arch/x86/include/asm/pgtable_64_types.h | 15 +++--- arch/x86/kernel/cpu/common.c | 2 - arch/x86/kernel/head64.c | 52 ++++---------------- arch/x86/mm/kasan_init_64.c | 3 -- arch/x86/mm/mem_encrypt_identity.c | 9 ---- 6 files changed, 15 insertions(+), 68 deletions(-) diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compressed/pgtable_64.c index 51f957b24ba7..0586cc216aa6 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -128,8 +128,6 @@ asmlinkage void configure_5level_paging(struct boot_params *bp, void *pgtable) /* Initialize variables for 5-level paging */ __pgtable_l5_enabled = 1; - pgdir_shift = 48; - ptrs_per_p4d = 512; } /* diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 38b54b992f32..ecc010fbb377 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -22,28 +22,25 @@ typedef struct { pteval_t pte; } pte_t; typedef struct { pmdval_t pmd; } pmd_t; #ifdef CONFIG_X86_5LEVEL +#ifdef USE_EARLY_PGTABLE_L5 extern unsigned int __pgtable_l5_enabled; -#ifdef USE_EARLY_PGTABLE_L5 /* - * cpu_feature_enabled() is not available in early boot code. - * Use variable instead. + * CR4.LA57 may not be set to its final value yet in the early boot code. + * Use a variable instead. */ static inline bool pgtable_l5_enabled(void) { return __pgtable_l5_enabled; } #else -#define pgtable_l5_enabled() cpu_feature_enabled(X86_FEATURE_LA57) +#define pgtable_l5_enabled() !!(native_read_cr4() & X86_CR4_LA57) #endif /* USE_EARLY_PGTABLE_L5 */ #else #define pgtable_l5_enabled() 0 #endif /* CONFIG_X86_5LEVEL */ -extern unsigned int pgdir_shift; -extern unsigned int ptrs_per_p4d; - #endif /* !__ASSEMBLY__ */ #define SHARED_KERNEL_PMD 0 @@ -53,7 +50,7 @@ extern unsigned int ptrs_per_p4d; /* * PGDIR_SHIFT determines what a top-level page table entry can map */ -#define PGDIR_SHIFT pgdir_shift +#define PGDIR_SHIFT (pgtable_l5_enabled() ? 48 : 39) #define PTRS_PER_PGD 512 /* @@ -61,7 +58,7 @@ extern unsigned int ptrs_per_p4d; */ #define P4D_SHIFT 39 #define MAX_PTRS_PER_P4D 512 -#define PTRS_PER_P4D ptrs_per_p4d +#define PTRS_PER_P4D (pgtable_l5_enabled() ? 512 : 1) #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE - 1)) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 0b97bcde70c6..20ac11a2c06b 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1,6 +1,4 @@ // SPDX-License-Identifier: GPL-2.0-only -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 #include #include diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index dc0956067944..d636bb02213f 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -7,9 +7,6 @@ #define DISABLE_BRANCH_PROFILING -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include @@ -50,14 +47,6 @@ extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; static unsigned int __initdata next_early_pgt; pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX); -#ifdef CONFIG_X86_5LEVEL -unsigned int __pgtable_l5_enabled __ro_after_init; -unsigned int pgdir_shift __ro_after_init = 39; -EXPORT_SYMBOL(pgdir_shift); -unsigned int ptrs_per_p4d __ro_after_init = 1; -EXPORT_SYMBOL(ptrs_per_p4d); -#endif - #ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT unsigned long page_offset_base __ro_after_init = __PAGE_OFFSET_BASE_L4; EXPORT_SYMBOL(page_offset_base); @@ -95,37 +84,6 @@ static unsigned long __head *fixup_long(void *ptr, unsigned long physaddr) return fixup_pointer(ptr, physaddr); } -#ifdef CONFIG_X86_5LEVEL -static unsigned int __head *fixup_int(void *ptr, unsigned long physaddr) -{ - return fixup_pointer(ptr, physaddr); -} - -static bool __head check_la57_support(unsigned long physaddr) -{ - /* - * 5-level paging is detected and enabled at kernel decompression - * stage. Only check if it has been enabled there. - */ - if (!(native_read_cr4() & X86_CR4_LA57)) - return false; - - *fixup_int(&__pgtable_l5_enabled, physaddr) = 1; - *fixup_int(&pgdir_shift, physaddr) = 48; - *fixup_int(&ptrs_per_p4d, physaddr) = 512; - *fixup_long(&page_offset_base, physaddr) = __PAGE_OFFSET_BASE_L5; - *fixup_long(&vmalloc_base, physaddr) = __VMALLOC_BASE_L5; - *fixup_long(&vmemmap_base, physaddr) = __VMEMMAP_BASE_L5; - - return true; -} -#else -static bool __head check_la57_support(unsigned long physaddr) -{ - return false; -} -#endif - static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) { unsigned long vaddr, vaddr_end; @@ -189,7 +147,7 @@ unsigned long __head __startup_64(unsigned long physaddr, int i; unsigned int *next_pgt_ptr; - la57 = check_la57_support(physaddr); + la57 = pgtable_l5_enabled(); /* Is the address too large? */ if (physaddr >> MAX_PHYSMEM_BITS) @@ -486,6 +444,14 @@ asmlinkage __visible void __init __noreturn x86_64_start_kernel(char * real_mode (__START_KERNEL & PGDIR_MASK))); BUILD_BUG_ON(__fix_to_virt(__end_of_fixed_addresses) <= MODULES_END); +#ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT + if (pgtable_l5_enabled()) { + page_offset_base = __PAGE_OFFSET_BASE_L5; + vmalloc_base = __VMALLOC_BASE_L5; + vmemmap_base = __VMEMMAP_BASE_L5; + } +#endif + cr4_init_shadow(); /* Kill off the identity-map trampoline */ diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 0302491d799d..85ae1ef840cc 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -2,9 +2,6 @@ #define DISABLE_BRANCH_PROFILING #define pr_fmt(fmt) "kasan: " fmt -/* cpu_feature_enabled() cannot be used this early */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index d73aeb16417f..67d4530548ce 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -27,15 +27,6 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS -/* - * This code runs before CPU feature bits are set. By default, the - * pgtable_l5_enabled() function uses bit X86_FEATURE_LA57 to determine if - * 5-level paging is active, so that won't work here. USE_EARLY_PGTABLE_L5 - * is provided to handle this situation and, instead, use a variable that - * has been set by the early boot code. - */ -#define USE_EARLY_PGTABLE_L5 - #include #include #include From patchwork Thu Jan 25 11:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192021 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573740dyi; Thu, 25 Jan 2024 03:35:18 -0800 (PST) X-Google-Smtp-Source: AGHT+IE74N1WRkVyTnmUpcha04sXxRbNkZPksXrV7Q/X3lMo0plDJ2sxmiYrDU81Cdsf9ZLbHgB2 X-Received: by 2002:a05:6871:2b1c:b0:214:c782:8acb with SMTP id dr28-20020a0568712b1c00b00214c7828acbmr751907oac.27.1706182518267; Thu, 25 Jan 2024 03:35:18 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182518; cv=pass; d=google.com; s=arc-20160816; b=lae09wxskBql8/jfotyAPRNnJzrZCOrf43ZPBzexFGTF9wUd6dMIC6deaGsafrNkgF ICtbZZtdAksOx5UbDirnIUF+RjVFRdPPWpU2mBliWHSgKBq3VK7GCkX1fNCN1QsgQd83 DN5A2Z5uQbUBZZYPY1It5butEo57OKDOSgyyjyrsQBAhLNPIC21imyFhY1/DI2F3vzvb XFWZqeI7H+UXyiuPBoovoDv649MjCtLkiGlZy2XnFZffNhR9cJqdWOQhq461JpPAhnGB uZwRyVTM5zkfphEUl4cppH+S/CxJEBBcwt2vugyBENeZW45uFxGU7YOuCAb36AP9w3z4 6vMw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=rJoL2/hFc4gkGtzEyAanpsFO97W15h/UiL7Bxz9040w=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=wyECZ64jcWzxxDzELYYVl47BJu5PRwSbMViWrca6FWf5sNNNXZCRdKF0LNgK5/OKpY 8klHhFihdkOqUm3BMdInI0PtOok0vf5/IexiLx4+1eZZAnVYegvYKUWJCAbL+wH9VmXA VGPvDfJ8FumqzmfC7gcEaXWo/NlQLxY1dtgNdLwn8HafbRcuEp7pBaio5DPhMcy4CQ5p IemMobQcXPZJC8sXmfjErkT/tV+lidD95/jJiknx9caLCL7XSObTmLyAH085nBwU6p5D Br12lS5wOvsx4lKvyq7r1d65hao/HI+n1Ae/h+FTHLETR1v3DVjZVK94NJ9jljWh4Gp2 fMQg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=mnH1Vm06; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38509-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38509-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id bs123-20020a632881000000b005d44d025258si3443347pgb.629.2024.01.25.03.35.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:35:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38509-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=mnH1Vm06; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38509-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38509-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 5750E285888 for ; Thu, 25 Jan 2024 11:34:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 16A644F885; Thu, 25 Jan 2024 11:32:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mnH1Vm06" Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F6A04655C for ; Thu, 25 Jan 2024 11:32:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182371; cv=none; b=R7n1GaZXshBzVZwOdSd9siIsBpyKH6WI7MmyPdahYE25av7DnwFeY61PlB/JNRjyGIvLQUVzKSmTaw1y+Vaj7uVS8oc7g5LXRIab8dnHPLjbQmSsjb0sI+3kjfcZjByumyHDbZOYl7I917KJAOrJLuCj5nsFJFp9P+y7+rbUUBg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182371; c=relaxed/simple; bh=g6w4YKbdWdUPESXVeFJEr3y36v0ES/R9hznFmwq1kAk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=k3O300VXClCh1cWvWKa+9zjE/PdUs4lQWz2ZIAxd1y61fV1SU0GJLKCy4VK/wRgA3pkk/OsOYDeoFioTFRWkPETBe9wAub2qJDvZu4l7pOggX9pYQUXJTbEsljHbnXVsWq+CTd+sFIjATKyMBahQbfARYkZx7kl9P73xKw2qwxU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mnH1Vm06; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5ff84361ac3so7796337b3.1 for ; Thu, 25 Jan 2024 03:32:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182369; x=1706787169; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rJoL2/hFc4gkGtzEyAanpsFO97W15h/UiL7Bxz9040w=; b=mnH1Vm06WU4xdC8JIDlrHVpu+RgHWAlQ3lbx6dgdi3Zsh01r7+gLgT0q2XLlfncgl/ Rbn9GGt7zgBMwyijWQ+1t60q/ULLQHFkz97jbxWD2+QpZxVInRQSD8pPsDFd/Q7ENqut /w3FmxdTn2Mqxf8NjMVckX5qyuhUTauirJY9lo5/cC3AKxNWrqmKmfKXN+UqOxjFRn5K ha2DQwzA8yPdI6h6BnddYSmJWeAepqVI2UAazsG9qeBFWo/DbZWXS3qDEPISWqLJ9JUV 4UhjM40ihUMaNwKbCXyN6QwKNZeWN13py64Y84n4spENfgR4k9SpjqMuSMd4yrsQsg9I dE1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182369; x=1706787169; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rJoL2/hFc4gkGtzEyAanpsFO97W15h/UiL7Bxz9040w=; b=OroyN/e6Qu9SEXOYHvna9Kxypm0eWgUBfXg/Kq9IhZiB5j0pDa/b7GYW8zl4btrh0v IYITajAnbyLSoFMSX+nUaFBib04Zy6kplY8A3MmPN8xJrjoL0C6vp3y+0eGJxRC8pp42 tLkkO938rJxCwWxOzv2+XyApY7/HV91NlwhtSEeIMc76p/X3YSRiVZKOPh4U+gZ5pRtc AiI4bXdDJAKqX6YZFfK+1Wx3/yTfO7lhH95IRlc5tnt7OtGPxhxKq2hJB1Z++l5FCw0G e78DHEHAdXwTbE2jrbLpKMNl04PU6lYgBgi6IOJljwipvR6qT+AOmep3+K1iTP9G7sxR bmEA== X-Gm-Message-State: AOJu0YzUO5mo/MW1J88zOSFwFnmwaWIAsl+H8Vkoxrj7igahjsMB0iCS Y+ymV6I5NrW9sSiXodVd8K+TiDX6C3YIYs6qjg90SOsNxh9T57kQgIKuIaIo5uWhs82H5iHGxpn rhX3I/3b5C0O6BpHlDstP5n8RRQQjdj7bEhbSiO08CSxojoz/1oZmH0kwjp0PQJVmQTjG09gm8g dIIo++dZMl3+pvD7modLcqjWRu2vEzKA== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:2509:b0:dc2:51f6:9168 with SMTP id dt9-20020a056902250900b00dc251f69168mr447093ybb.2.1706182369248; Thu, 25 Jan 2024 03:32:49 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:24 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3319; i=ardb@kernel.org; h=from:subject; bh=t2swCS3/CT0QPpEAks7skAwmTJcH02OXVCoaa0u+RH0=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6z2rFV3aYh/+hMbW2pT79zN9nPJ7uduxr3f6Xjc+2 PFLP721o5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEyksYORYWFhVXFRQ+RcS7Hf vydLr3wt9ekEW7PGnoatB5+F3An6UMvw32GH9BnTl5Fz+AP4/srfvS7w5cfdXe5Tl62+rRU9Qes OPz8A X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-24-ardb+git@google.com> Subject: [PATCH v2 05/17] x86/startup_64: Simplify virtual switch on primary boot From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062039748723630 X-GMAIL-MSGID: 1789062039748723630 From: Ard Biesheuvel The secondary startup code is used on the primary boot path as well, but in this case, the initial part runs from a 1:1 mapping, until an explicit cross-jump is made to the kernel virtual mapping of the same code. On the secondary boot path, this jump is pointless as the code already executes from the mapping targeted by the jump. So combine this cross-jump with the jump from startup_64() into the common boot path. This simplifies the execution flow, and clearly separates code that runs from a 1:1 mapping from code that runs from the kernel virtual mapping. Note that this requires a page table switch, so hoist the CR3 assignment into startup_64() as well. Given that the secondary startup code does not require a special placement inside the executable, move it to the .text section. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 41 +++++++++----------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 2d361e0ac74e..399241dcdbb5 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -39,7 +39,6 @@ L4_START_KERNEL = l4_index(__START_KERNEL_map) L3_START_KERNEL = pud_index(__START_KERNEL_map) - .text __HEAD .code64 SYM_CODE_START_NOALIGN(startup_64) @@ -128,9 +127,19 @@ SYM_CODE_START_NOALIGN(startup_64) call sev_verify_cbit #endif - jmp 1f + /* + * Switch to early_top_pgt which still has the identity mappings + * present. + */ + movq %rax, %cr3 + + /* Branch to the common startup code at its kernel virtual address */ + movq $common_startup_64, %rax + ANNOTATE_RETPOLINE_SAFE + jmp *%rax SYM_CODE_END(startup_64) + .text SYM_CODE_START(secondary_startup_64) UNWIND_HINT_END_OF_STACK ANNOTATE_NOENDBR @@ -176,8 +185,15 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) #ifdef CONFIG_AMD_MEM_ENCRYPT addq sme_me_mask(%rip), %rax #endif + /* + * Switch to the init_top_pgt here, away from the trampoline_pgd and + * unmap the identity mapped ranges. + */ + movq %rax, %cr3 -1: +SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) + UNWIND_HINT_END_OF_STACK + ANNOTATE_NOENDBR // above /* * Define a mask of CR4 bits to preserve. PAE and LA57 cannot be @@ -195,17 +211,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) */ orl $X86_CR4_MCE, %edx #endif - - /* - * Switch to new page-table - * - * For the boot CPU this switches to early_top_pgt which still has the - * identity mappings present. The secondary CPUs will switch to the - * init_top_pgt here, away from the trampoline_pgd and unmap the - * identity mapped ranges. - */ - movq %rax, %cr3 - /* * Do a global TLB flush after the CR3 switch to make sure the TLB * entries from the identity mapping are flushed. @@ -216,14 +221,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) movq %rcx, %cr4 jc 0b - /* Ensure I am executing from virtual addresses */ - movq $1f, %rax - ANNOTATE_RETPOLINE_SAFE - jmp *%rax -1: - UNWIND_HINT_END_OF_STACK - ANNOTATE_NOENDBR // above - #ifdef CONFIG_SMP /* * For parallel boot, the APIC ID is read from the APIC, and then From patchwork Thu Jan 25 11:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192020 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573550dyi; Thu, 25 Jan 2024 03:34:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IFxFoCkxYdjySS+rD2e4pDd7RJH0dXOWbUhJNHKHGJe3G16091ozzE/TORn0UTF33ov+eBH X-Received: by 2002:a05:622a:138f:b0:42a:4c59:43fe with SMTP id o15-20020a05622a138f00b0042a4c5943femr953722qtk.102.1706182494405; Thu, 25 Jan 2024 03:34:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182494; cv=pass; d=google.com; s=arc-20160816; b=tkpvQj2ysCL83utjGm4J7ksDwXeKEhJhsCqXC6/eZFF+q9+EqnxlFSWMAiMgIqihLe CY27C/omC3t8R5jPG5qbRZkTgr0La6Pwi7MrfiL/hik0ZgiBzCDDZajgI+43O5BIvHom GNUQclR4AKk1yOA0ROca2FJJ5baxI88jEoVlColnxPlJPFz6yNsYQoZIdiqcxxybBKEx IX5b5EKNZLW87dj5OwaFUlTUxzjDAdqjmcV86pEMe2uZxO0HBbf//uZB71U3VYPlTjy6 fukHFxLLSBec33nVXjxs7JG443Ce4dz9itDIUILnKdeOyRPC6Z9yu4PAJbItI+bgnlEx QgMg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=1Kz7C+ixPzKX7RlC+dwS9psnBzxi2S77hDKHtKWjxqM=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=O2p5KwMxtLYNJfJP4CW9Ftn83ECaekjuCucZ3vVfsQ0mRQGbGf3+WSJBbRMo0clEzY 4F5zvxzUMclSYomNhW+4zgxEFfODJ6uWRzGvU/l8nCKchcX1Nrrmobtv0tykuwgO5gsh R9vrRa5n5/gVBO3yWXKvZnHcxkQ3qavquFc/7F5gPhkY6fPJ2CCK+X8H7Y0/H1ug7Q9Q 9LcCzyvVewE+Pavu3KOLdOc35vREvYUxRNgEZQopJ0Pw8845WMJQ6hDDV289k/pyTUtm 1FahJ/bGR7Sg3rGxhhkcFKcz599mNatxo0HmfuCh/0xFNMHT8d/gDCZbe7/wmnyd3vqw Z9LQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=LecD1d6T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id j16-20020ac85f90000000b0042a3d3520cbsi9553886qta.720.2024.01.25.03.34.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:34:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=LecD1d6T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38510-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 27A001C22C2E for ; Thu, 25 Jan 2024 11:34:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5AC704502D; Thu, 25 Jan 2024 11:33:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LecD1d6T" Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06DA94F216 for ; Thu, 25 Jan 2024 11:32:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182376; cv=none; b=cZMWWfPKf8/5iKywdwYxG5Ugn8BOdL6KpnZC8+v6TnotN1zMtJ5646dxrlnTtk6dYyJQhq/oN0Kgq3QbBlP4BsuIcfnwkukbVozigpNG5LsWzZxqLqn94Mp71Gxx9717+/39SnJ+wWjww4nxDPHN8b4OYbv9zkAcFqahVfAdHUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182376; c=relaxed/simple; bh=1jRwFT8KP5Sc2cp3hWq0TIt5gMnbctAz1NiKEF7cRfM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TJ8xML/dmS9p2YKH1q5xhNdMlQFVk30WRulPGDOv1KxL26LwPAKytLRJHzlQ4JJB4t0qjy4VHPumXWbxxF4gc+gY+ncnfp3rkX3w8Dzj4bLGk3iUkt45Kk44TGaFfg308vdwPnAusPqnIrJSTN8c7LS1fBThKElzd/IJSqKDE4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LecD1d6T; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-40e4478a3afso2716585e9.1 for ; Thu, 25 Jan 2024 03:32:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182371; x=1706787171; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1Kz7C+ixPzKX7RlC+dwS9psnBzxi2S77hDKHtKWjxqM=; b=LecD1d6TQ2ViDFF7DSxGHbMZealbvT2rcLpPji4ylD0OQy4ZeiE2cg6wNoWi7XDc9l 1KFrOMUEabKBp+wqZ5h4Iln5wJ6IdgyF5fW0M9mept1x/1OiXqqVWV3IjNJGRFwe0dn/ C2qA79F/We6WEZmWzZgOaKBNubIos2r/Jj8R4EcAOBQ7joBQ2firhx6NEA+9iuVwXewN 6ChsOXjBkbTUIJRd3G1j6AwgTz+67PWctKRwlKB8Q8n303kcE1jCGxFL+PTW53M5s+cK nC/l0zvPNr+OtCQlwD4jNss2t84CuegaiMfhc+gOtzF+7wHaxGpR5jVmLQSogpbO7Eqo ZwQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182371; x=1706787171; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1Kz7C+ixPzKX7RlC+dwS9psnBzxi2S77hDKHtKWjxqM=; b=N2+/wSVl4AyJVOI6jmPdjUrpMU5IroACBfOJCRnKCsY9rCCugVsdezTJ+5IeGhb1ir ZyKXeH/54PVXKmfBgA9EBXXKJMcMEcf1zurOWrtRnnAkfFgCxl6FFBFdnxdi9OgYaOGG kCFNi70D17IXP1QeQ+39y8H9W7DlTvNL8i+S3M+q135LdYZE0w4NCOLPOX8f4XPX3I0c 1vqqYdHO5u2I4lBl3zJYsnt1xmCxu3oicAJS0IV3PF5pW9GRgFbu/x6TjEzV8lsS4xa6 nLDoAo8dNtDk3S5fTzrVsS8+FhB3OmNIXGYfVoikU13Tu6Azw14964n3YBTTeK8t5jF0 KZuQ== X-Gm-Message-State: AOJu0YxMug7Ob09e2Jdz24LSvm4GRpqYQqF5e43ogW2I9M6iJcxpbcbq s4z3MCLsiPQu7G5GgIgBAFykYy/AH59DkAyLAc4jLSMMG6LP2UkxVegYRhMiUpoxoCalfotMzIl nNfhDCho/w4kkWiH75GG2SQCMKpcvl3/Z96sIOXJrxX7hk/FHFFpIzaY1GvhqYELXYyjiqU1rDe 123CHUxfxLAKd3nT457NbtqMdK5Jf/Wg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:3489:b0:40e:d20f:6e43 with SMTP id a9-20020a05600c348900b0040ed20f6e43mr21720wmq.1.1706182371411; Thu, 25 Jan 2024 03:32:51 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:25 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=11891; i=ardb@kernel.org; h=from:subject; bh=PzzWp9ixYoJ19nSdSRUo8K5t8tOJCydzPxdlM74OpZs=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT68PDJS839fA9XjdFTNBbZ5HLrO9tm63D9yocXXlZ+ M4Kk1vLOkpZGMQ4GGTFFFkEZv99t/P0RKla51myMHNYmUCGMHBxCsBEAr8x/OHevmDJd4ttvtP9 ReZN4p7dasPwKryKa1LuLw7diQkc584wMiyflRHO0nMyY6WDvk2S8u76iwt25X999OpCQQ77Nuv UfRwA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-25-ardb+git@google.com> Subject: [PATCH v2 06/17] x86/head64: Replace pointer fixups with PIE codegen From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062015419024945 X-GMAIL-MSGID: 1789062015419024945 From: Ard Biesheuvel Some of the C code in head64.c may be called from a different virtual address than it was linked at. Currently, we deal with this by using ordinary, position dependent codegen, and fixing up all symbol references on the fly. This is fragile and tricky to maintain. It is also unnecessary: we can use position independent codegen (with hidden visibility) to ensure that all compiler generated symbol references are RIP-relative, removing the need for fixups entirely. It does mean we need explicit references to kernel virtual addresses to be generated by hand, so generate those using a movabs instruction in inline asm in the handful places where we actually need this. Signed-off-by: Ard Biesheuvel --- arch/x86/Makefile | 5 ++ arch/x86/boot/compressed/Makefile | 2 +- arch/x86/include/asm/setup.h | 4 +- arch/x86/kernel/Makefile | 4 + arch/x86/kernel/head64.c | 88 +++++++------------- arch/x86/kernel/head_64.S | 5 +- 6 files changed, 45 insertions(+), 63 deletions(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 1a068de12a56..3c3c07cccd47 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -168,6 +168,11 @@ else KBUILD_CFLAGS += -mcmodel=kernel KBUILD_RUSTFLAGS += -Cno-redzone=y KBUILD_RUSTFLAGS += -Ccode-model=kernel + + PIE_CFLAGS := -fpie -mcmodel=small -fno-stack-protector \ + -include $(srctree)/include/linux/hidden.h + + export PIE_CFLAGS endif # diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index f19c038409aa..bccee07eae60 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -84,7 +84,7 @@ LDFLAGS_vmlinux += -T hostprogs := mkpiggy HOST_EXTRACFLAGS += -I$(srctree)/tools/include -sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABCDGRSTVW] \(_text\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' quiet_cmd_voffset = VOFFSET $@ cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@ diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h index 5c83729c8e71..b004f1b9a052 100644 --- a/arch/x86/include/asm/setup.h +++ b/arch/x86/include/asm/setup.h @@ -47,8 +47,8 @@ extern unsigned long saved_video_mode; extern void reserve_standard_io_resources(void); extern void i386_reserve_resources(void); -extern unsigned long __startup_64(unsigned long physaddr, struct boot_params *bp); -extern void startup_64_setup_env(unsigned long physbase); +extern unsigned long __startup_64(struct boot_params *bp); +extern void startup_64_setup_env(void); extern void early_setup_idt(void); extern void __init do_early_exception(struct pt_regs *regs, int trapnr); diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 0000325ab98f..65194ca79b5c 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -21,6 +21,10 @@ CFLAGS_REMOVE_sev.o = -pg CFLAGS_REMOVE_rethook.o = -pg endif +# head64.c contains C code that may execute from a different virtual address +# than it was linked at, so we always build it using PIE codegen +CFLAGS_head64.o += $(PIE_CFLAGS) + KASAN_SANITIZE_head$(BITS).o := n KASAN_SANITIZE_dumpstack.o := n KASAN_SANITIZE_dumpstack_$(BITS).o := n diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index d636bb02213f..a4a380494703 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -74,15 +74,10 @@ static struct desc_ptr startup_gdt_descr __initdata = { .address = 0, }; -static void __head *fixup_pointer(void *ptr, unsigned long physaddr) -{ - return ptr - (void *)_text + (void *)physaddr; -} - -static unsigned long __head *fixup_long(void *ptr, unsigned long physaddr) -{ - return fixup_pointer(ptr, physaddr); -} +#define __va_symbol(sym) ({ \ + unsigned long __v; \ + asm("movq $" __stringify(sym) ", %0":"=r"(__v)); \ + __v; }) static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) { @@ -99,8 +94,8 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv * attribute. */ if (sme_get_me_mask()) { - vaddr = (unsigned long)__start_bss_decrypted; - vaddr_end = (unsigned long)__end_bss_decrypted; + vaddr = __va_symbol(__start_bss_decrypted); + vaddr_end = __va_symbol(__end_bss_decrypted); for (; vaddr < vaddr_end; vaddr += PMD_SIZE) { /* @@ -127,25 +122,17 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv return sme_get_me_mask(); } -/* Code in __startup_64() can be relocated during execution, but the compiler - * doesn't have to generate PC-relative relocations when accessing globals from - * that function. Clang actually does not generate them, which leads to - * boot-time crashes. To work around this problem, every global pointer must - * be adjusted using fixup_pointer(). - */ -unsigned long __head __startup_64(unsigned long physaddr, - struct boot_params *bp) +unsigned long __head __startup_64(struct boot_params *bp) { + unsigned long physaddr = (unsigned long)_text; unsigned long load_delta, *p; unsigned long pgtable_flags; pgdval_t *pgd; p4dval_t *p4d; pudval_t *pud; pmdval_t *pmd, pmd_entry; - pteval_t *mask_ptr; bool la57; int i; - unsigned int *next_pgt_ptr; la57 = pgtable_l5_enabled(); @@ -157,7 +144,7 @@ unsigned long __head __startup_64(unsigned long physaddr, * Compute the delta between the address I am compiled to run at * and the address I am actually running at. */ - load_delta = physaddr - (unsigned long)(_text - __START_KERNEL_map); + load_delta = physaddr - (__va_symbol(_text) - __START_KERNEL_map); /* Is the address not 2M aligned? */ if (load_delta & ~PMD_MASK) @@ -168,26 +155,24 @@ unsigned long __head __startup_64(unsigned long physaddr, /* Fixup the physical addresses in the page table */ - pgd = fixup_pointer(early_top_pgt, physaddr); + pgd = (pgdval_t *)early_top_pgt; p = pgd + pgd_index(__START_KERNEL_map); if (la57) *p = (unsigned long)level4_kernel_pgt; else *p = (unsigned long)level3_kernel_pgt; - *p += _PAGE_TABLE_NOENC - __START_KERNEL_map + load_delta; + *p += _PAGE_TABLE_NOENC + sme_get_me_mask(); if (la57) { - p4d = fixup_pointer(level4_kernel_pgt, physaddr); + p4d = (p4dval_t *)level4_kernel_pgt; p4d[511] += load_delta; } - pud = fixup_pointer(level3_kernel_pgt, physaddr); - pud[510] += load_delta; - pud[511] += load_delta; + level3_kernel_pgt[510].pud += load_delta; + level3_kernel_pgt[511].pud += load_delta; - pmd = fixup_pointer(level2_fixmap_pgt, physaddr); for (i = FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) - pmd[i] += load_delta; + level2_fixmap_pgt[i].pmd += load_delta; /* * Set up the identity mapping for the switchover. These @@ -196,15 +181,13 @@ unsigned long __head __startup_64(unsigned long physaddr, * it avoids problems around wraparound. */ - next_pgt_ptr = fixup_pointer(&next_early_pgt, physaddr); - pud = fixup_pointer(early_dynamic_pgts[(*next_pgt_ptr)++], physaddr); - pmd = fixup_pointer(early_dynamic_pgts[(*next_pgt_ptr)++], physaddr); + pud = (pudval_t *)early_dynamic_pgts[next_early_pgt++]; + pmd = (pmdval_t *)early_dynamic_pgts[next_early_pgt++]; pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask(); if (la57) { - p4d = fixup_pointer(early_dynamic_pgts[(*next_pgt_ptr)++], - physaddr); + p4d = (p4dval_t *)early_dynamic_pgts[next_early_pgt++]; i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; pgd[i + 0] = (pgdval_t)p4d + pgtable_flags; @@ -225,8 +208,7 @@ unsigned long __head __startup_64(unsigned long physaddr, pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; /* Filter out unsupported __PAGE_KERNEL_* bits: */ - mask_ptr = fixup_pointer(&__supported_pte_mask, physaddr); - pmd_entry &= *mask_ptr; + pmd_entry &= __supported_pte_mask; pmd_entry += sme_get_me_mask(); pmd_entry += physaddr; @@ -252,14 +234,14 @@ unsigned long __head __startup_64(unsigned long physaddr, * error, causing the BIOS to halt the system. */ - pmd = fixup_pointer(level2_kernel_pgt, physaddr); + pmd = (pmdval_t *)level2_kernel_pgt; /* invalidate pages before the kernel image */ - for (i = 0; i < pmd_index((unsigned long)_text); i++) + for (i = 0; i < pmd_index(__va_symbol(_text)); i++) pmd[i] &= ~_PAGE_PRESENT; /* fixup pages that are part of the kernel image */ - for (; i <= pmd_index((unsigned long)_end); i++) + for (; i <= pmd_index(__va_symbol(_end)); i++) if (pmd[i] & _PAGE_PRESENT) pmd[i] += load_delta; @@ -271,7 +253,7 @@ unsigned long __head __startup_64(unsigned long physaddr, * Fixup phys_base - remove the memory encryption mask to obtain * the true physical address. */ - *fixup_long(&phys_base, physaddr) += load_delta - sme_get_me_mask(); + phys_base += load_delta - sme_get_me_mask(); return sme_postprocess_startup(bp, pmd); } @@ -553,22 +535,16 @@ static void set_bringup_idt_handler(gate_desc *idt, int n, void *handler) } /* This runs while still in the direct mapping */ -static void __head startup_64_load_idt(unsigned long physbase) +static void __head startup_64_load_idt(void) { - struct desc_ptr *desc = fixup_pointer(&bringup_idt_descr, physbase); - gate_desc *idt = fixup_pointer(bringup_idt_table, physbase); - - - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - void *handler; + gate_desc *idt = bringup_idt_table; + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) /* VMM Communication Exception */ - handler = fixup_pointer(vc_no_ghcb, physbase); - set_bringup_idt_handler(idt, X86_TRAP_VC, handler); - } + set_bringup_idt_handler(idt, X86_TRAP_VC, vc_no_ghcb); - desc->address = (unsigned long)idt; - native_load_idt(desc); + bringup_idt_descr.address = (unsigned long)idt; + native_load_idt(&bringup_idt_descr); } /* This is used when running on kernel addresses */ @@ -587,10 +563,10 @@ void early_setup_idt(void) /* * Setup boot CPU state needed before kernel switches to virtual addresses. */ -void __head startup_64_setup_env(unsigned long physbase) +void __head startup_64_setup_env(void) { /* Load GDT */ - startup_gdt_descr.address = (unsigned long)fixup_pointer(startup_gdt, physbase); + startup_gdt_descr.address = (unsigned long)startup_gdt; native_load_gdt(&startup_gdt_descr); /* New GDT is live - reload data segment registers */ @@ -598,5 +574,5 @@ void __head startup_64_setup_env(unsigned long physbase) "movl %%eax, %%ss\n" "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); - startup_64_load_idt(physbase); + startup_64_load_idt(); } diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 399241dcdbb5..b8704ac1a4da 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -67,8 +67,6 @@ SYM_CODE_START_NOALIGN(startup_64) /* Set up the stack for verify_cpu() */ leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp - leaq _text(%rip), %rdi - /* Setup GSBASE to allow stack canary access for C code */ movl $MSR_GS_BASE, %ecx leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx @@ -107,8 +105,7 @@ SYM_CODE_START_NOALIGN(startup_64) * is active) to be added to the initial pgdir entry that will be * programmed into CR3. */ - leaq _text(%rip), %rdi - movq %r15, %rsi + movq %r15, %rdi call __startup_64 /* Form the CR3 value being sure to include the CR3 modifier */ From patchwork Thu Jan 25 11:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192019 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573554dyi; Thu, 25 Jan 2024 03:34:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IGa7EjucKIgqZf4gCCqoaXwgOS8zmFwXKBdjobx0NliSmyZygM1Oe/jAWHox8afE0P7NcLw X-Received: by 2002:ac8:5c50:0:b0:42a:3275:a42b with SMTP id j16-20020ac85c50000000b0042a3275a42bmr914908qtj.44.1706182494963; Thu, 25 Jan 2024 03:34:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182494; cv=pass; d=google.com; s=arc-20160816; b=dthPWMu+xxHrKQ/DdGDxpuNgbI3ZQA/0CfBWrrLLtayTaRwKQsGSLsJ0yXEAqgftOn C995z//xWS8ayaKs3NV9kFGQLXW4XHULMY6kydLRufLx+8/Pn0qDa2ddmEFZAPJFZ229 18mMWK3mrU+AkZscYZKpTqfzJeYkrJvKoPuc+y0oTubcKdLu8wk0zxTIY5wpGCxNgaWp ICZWVAzyDUkVgV3+wv+7KA/pZIE/fGfY9uQdPJhSXrXJBEOcuCGImSVtdxo1+C1W+Mar F1g86PTn5Ea+1N2/Zww37/OSYJlL0jBJIN4f3U9Y1c6XdqWaXgyJNe1jCTxc91vnYw+W P7og== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=K393o/iOXcO+KXDFEg2lpPBpqdPHLp4izeuXizwM1H4=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=k/E9HaTVUuaQ1y5O/AsKRCAWII2kTC6d7jAohEQ7stDgQBfPv7HCpDfquz4O1N+Z1H xlXOUblDvuE5QL3jck3Mb4uXw9WnBDwOqw7N4VrHdcqsA5mW9BVxsp9Gs0HJqWMMAAIJ RNMBfxOJkPdW2KFCqZmEVJ3JRuhyaUbCR7lAko3cdNVkkB75Knl3qv0n1eLoyRKSjjaT my6JxEXitCBgXWFxmHdtYpwKRRVofFwdivr731M6CJutjO7yzRgxWce0imfurZEtTWJv zTryvCPOGvZI/9JGQwxn2rkay0dT9qFUBwjROEvdb/8uuYOBMh/XSPY0BMI7A3iPArPN 3CyA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=CceKwkEg; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38511-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38511-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id b7-20020a05622a020700b0042a3cdaec69si9392362qtx.112.2024.01.25.03.34.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:34:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38511-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=CceKwkEg; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38511-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38511-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B16B21C22398 for ; Thu, 25 Jan 2024 11:34:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 743BB45942; Thu, 25 Jan 2024 11:33:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CceKwkEg" Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EFA44F893 for ; Thu, 25 Jan 2024 11:32:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182377; cv=none; b=Lj/pMZNLmIfqYiw5jl5X446yCfbKG72qxtAow2EW76CBhz2/5cUbo+yQ9J0j/hqXuecqJlEt2MmPPMqZ9k3VjsIiwMFpJLSRhP18Z/Jcr5jMsWWuMjtmAnYHAQyX/P7NVRiK42kJ4Sa4ONo2WgPLXnsPuG7dA36yMi+jQl7pjlw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182377; c=relaxed/simple; bh=9H4JrfvvUnACeesgQKj2mOvmlC0TpA0V7SRr0Kxm8nk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Axuc6rgRYAP4R/BeFR+8xJmglNpfPKQlmrWZnm74IJfOfDzdykmpv1SKlq3q97eJgp27C0q8+5VPCis5tU4/Js193Y9sZ5QMyNKQ7iwhyvsm5y7nSzqGsqJdrG03PupABZo5dvFLxFvOr1cDgCtTIX3C9JxBLsOf/8CuxnZz/Q0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CceKwkEg; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-337d6024098so3870818f8f.1 for ; Thu, 25 Jan 2024 03:32:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182373; x=1706787173; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=K393o/iOXcO+KXDFEg2lpPBpqdPHLp4izeuXizwM1H4=; b=CceKwkEgY9fn7y0kKsYr8aBtywEMDbQKtOijG0xLcp3P8WXeImSYjPrcey7i293v6L hahVHOT/ANyOTE3PbaICYSOCWX6DRJz0auSW6bcQx5IF8beVzB+211A7PGXKw3IngN8y JQlJutT1SXBI1ChE2B7xzvnetcAFszCrh2coTIdps3q9c/g+WO2FTlTr0cbWtQWXS4Bf gT9MIK3PFs3zF6SfeEA3IS9EvdfUcbsfnCebaoRE6hx1kcT/vD4oKUO9DhyRXWsAE/QN vKodNcEzS6+QlFZteIRJ4Q0W/f7QE8BEAHSr3Cd31R7j+OIbauao951Rxy2zmuywvT8Y tjLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182373; x=1706787173; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=K393o/iOXcO+KXDFEg2lpPBpqdPHLp4izeuXizwM1H4=; b=RRArlG0w+EA7hK9pDtlOqc69j5EDmiyEco+D8Js+cfb+JuDf4LDXV+9it375vtaynV yGUNG4TeF+8/tUYR93VqJZiSKZkGe4HtilrpdpXErdwI3Kk0cSOQ7xNLTqqqyf/udu+w 6JaZgJ1tKWrxzAEBsBdA7Z89JWG9LRdBJLjDNVz3mPHuOPlnVOiKR9zh4t1S0dnfjHxN uBi75QdbEJ5vhtoj2tAiWKWtOu/SCsfxigucvhThtauhDRKHuuRDilSOPO707nBzJiM5 3X3bpgQl7ZnCS7F5d6r/MTjyIJVcGNEXAv/kAHF5vaLtsn3G9+rLhCYwL0Ze+iq4wGE/ ZEBA== X-Gm-Message-State: AOJu0Yy8MlsowkxJ4FKniAnylpy6/OOZdpajW00z9SVxayrxpPhWhAu/ HxKejJZyCbLJ4DZG1g3REJb+ZEhB/oYo1NoDTzknMhYk+XBz1Vvxso39+SnH4A4hi/ftHrnGXSq xfDcVfzFyg+kmNWsVB/nSSp/dUc8yXKn7f8/Rk31bsle5VcmFD/Zh4q57qomvXAJbDAG8mYJPD7 yPFGCLp3PaN2+fKmEz2sEpEmm6yGU+4w== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:3f94:b0:40e:9cd9:46f5 with SMTP id fs20-20020a05600c3f9400b0040e9cd946f5mr12071wmb.8.1706182373512; Thu, 25 Jan 2024 03:32:53 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:26 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4380; i=ardb@kernel.org; h=from:subject; bh=A4U+rfXqigBIzPefwYEqRJTK3Dd5A0XyBzuBQozAnUo=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT62ORs9oPc/Qadt0tbndi1tzVIbl1cf+vJo6HTaHnq uVPxLl3lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIk8DWP4w1F/53/Dulq+Y3y8 Mcf/KH6/sfNN4ZzVx312dp/ta/U0usvwh2/z7Sg997ZlyTavl/NuZ97mbifpJewt9a5j5ZSK+8L xXAA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-26-ardb+git@google.com> Subject: [PATCH v2 07/17] x86/head64: Simplify GDT/IDT initialization code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062015899317418 X-GMAIL-MSGID: 1789062015899317418 From: Ard Biesheuvel There used to be two separate code paths for programming the IDT early: one that was called via the 1:1 mapping, and one via the kernel virtual mapping, where the former used explicit pointer fixups to obtain 1:1 mapped addresses. That distinction is now gone so the GDT/IDT init code can be unified and simplified accordingly. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 59 +++++++------------- 1 file changed, 19 insertions(+), 40 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index a4a380494703..993d888a3172 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -59,21 +59,12 @@ EXPORT_SYMBOL(vmemmap_base); /* * GDT used on the boot CPU before switching to virtual addresses. */ -static struct desc_struct startup_gdt[GDT_ENTRIES] __initdata = { +static struct desc_struct startup_gdt[GDT_ENTRIES] __initconst = { [GDT_ENTRY_KERNEL32_CS] = GDT_ENTRY_INIT(DESC_CODE32, 0, 0xfffff), [GDT_ENTRY_KERNEL_CS] = GDT_ENTRY_INIT(DESC_CODE64, 0, 0xfffff), [GDT_ENTRY_KERNEL_DS] = GDT_ENTRY_INIT(DESC_DATA64, 0, 0xfffff), }; -/* - * Address needs to be set at runtime because it references the startup_gdt - * while the kernel still uses a direct mapping. - */ -static struct desc_ptr startup_gdt_descr __initdata = { - .size = sizeof(startup_gdt)-1, - .address = 0, -}; - #define __va_symbol(sym) ({ \ unsigned long __v; \ asm("movq $" __stringify(sym) ", %0":"=r"(__v)); \ @@ -363,7 +354,7 @@ void __init do_early_exception(struct pt_regs *regs, int trapnr) early_fixup_exception(regs, trapnr); } -/* Don't add a printk in there. printk relies on the PDA which is not initialized +/* Don't add a printk in there. printk relies on the PDA which is not initialized yet. */ void __init clear_bss(void) { @@ -517,47 +508,32 @@ void __init __noreturn x86_64_start_reservations(char *real_mode_data) */ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data; -static struct desc_ptr bringup_idt_descr = { - .size = (NUM_EXCEPTION_VECTORS * sizeof(gate_desc)) - 1, - .address = 0, /* Set at runtime */ -}; - -static void set_bringup_idt_handler(gate_desc *idt, int n, void *handler) -{ -#ifdef CONFIG_AMD_MEM_ENCRYPT - struct idt_data data; - gate_desc desc; - - init_idt_data(&data, n, handler); - idt_init_desc(&desc, &data); - native_write_idt_entry(idt, n, &desc); -#endif -} - -/* This runs while still in the direct mapping */ -static void __head startup_64_load_idt(void) +static void early_load_idt(void (*handler)(void)) { gate_desc *idt = bringup_idt_table; + struct desc_ptr bringup_idt_descr; + + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { + struct idt_data data; + gate_desc desc; - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) /* VMM Communication Exception */ - set_bringup_idt_handler(idt, X86_TRAP_VC, vc_no_ghcb); + init_idt_data(&data, X86_TRAP_VC, handler); + idt_init_desc(&desc, &data); + native_write_idt_entry(idt, X86_TRAP_VC, &desc); + } bringup_idt_descr.address = (unsigned long)idt; + bringup_idt_descr.size = sizeof(bringup_idt_table); native_load_idt(&bringup_idt_descr); } -/* This is used when running on kernel addresses */ void early_setup_idt(void) { - /* VMM Communication Exception */ - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) setup_ghcb(); - set_bringup_idt_handler(bringup_idt_table, X86_TRAP_VC, vc_boot_ghcb); - } - bringup_idt_descr.address = (unsigned long)bringup_idt_table; - native_load_idt(&bringup_idt_descr); + early_load_idt(vc_boot_ghcb); } /* @@ -565,8 +541,11 @@ void early_setup_idt(void) */ void __head startup_64_setup_env(void) { + struct desc_ptr startup_gdt_descr; + /* Load GDT */ startup_gdt_descr.address = (unsigned long)startup_gdt; + startup_gdt_descr.size = sizeof(startup_gdt) - 1; native_load_gdt(&startup_gdt_descr); /* New GDT is live - reload data segment registers */ @@ -574,5 +553,5 @@ void __head startup_64_setup_env(void) "movl %%eax, %%ss\n" "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); - startup_64_load_idt(); + early_load_idt(vc_no_ghcb); } From patchwork Thu Jan 25 11:28:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192023 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1574120dyi; Thu, 25 Jan 2024 03:36:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IHl+IB9iX/WITKSlqJN3y+HPUhJrju6FJ+U8uqpwNzGSJDO/T2yMSu4hXGbtvQ9rcg0NYn5 X-Received: by 2002:a17:903:298d:b0:1d8:20ea:4284 with SMTP id lm13-20020a170903298d00b001d820ea4284mr1312740plb.29.1706182570414; Thu, 25 Jan 2024 03:36:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182570; cv=pass; d=google.com; s=arc-20160816; b=TmKKI+4i2uJnsGcflTLNuFivT4fQCOU0eAqN+JClPevbXqPj1RdMzBNqHAEqvY6Vig IJD+WEZz/MarAxf2+W1CMqKkprkisDz38MrhHORa4V+kHsIszHk0QssoxL8vokz97rGr /nn0tCisBy05iHuwh/PPmv/JwpRrbVyk60mNJvfmzOaIgPN3mYJsPdvrZGX9E0cp+IeR c2RmtOh9SkwnhHNnNkwcbLLugKgXPSQ2RD0dPCPmuO26hDtUCg3w5hGF6oXaQPuGaswJ GNfmhOVPINKH7maoWLXcGFiMXsJVCxfi2ByD9QakeFLJCn7h6hocKIdbaMamNWgVBLhD nD+A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=usiDcusxejCbluC36Vpk0on80rVNZ7Itzv8e5WwmSQI=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=CU+OF6c4kTSnUwiTVU7VU/Osi47sR5JU3Dc4+bwf44+NkRojPPQNjD4rsCrZfxoOpF TvpRusgXqAZg3W25j+BNFtJ/hNT48OwSwb0GiRR8zFuTX9Yv1JpujtQGhgnGtLSqNRzV oD4hjF29f883L3DVxnyVNv6ED+Z8xv+jT81khHr93MZBnUknmyna3le9dmr4Z9kjRdLJ BGc3+2q3/+r9Mb2ABAdaSFXIG6py5iNL/fhsmeEEcJcrKW9zhf2WVyD2PtFQ/j6zFaE0 p/z9bwbUWZbvNFTP7gk6h8C+ip1HTLJ5IQrvaPHGBu0trsCHqoUNCwwh10jMgQTtPN0/ qmSA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=AfPFhgi+; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38513-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38513-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id b1-20020a170903228100b001d767df9dd9si5432536plh.333.2024.01.25.03.36.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:36:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38513-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=AfPFhgi+; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38513-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38513-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id B96AB2868CF for ; Thu, 25 Jan 2024 11:35:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7799E5100D; Thu, 25 Jan 2024 11:33:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AfPFhgi+" Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 244AB2EAEA for ; Thu, 25 Jan 2024 11:32:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182381; cv=none; b=hqxnIAQRAyGImatAUte9FczgNgZkjC7As8zl1tk+nbFf+5rPH3pNwMyIsOWSgNVd+/z4ewzNWMkZ5xy9MmKkQaEfQD0eWZ8eyOhiXDI3kQVeW06mzbaQqQHnhGAvi5mQBsAoFulm6co/wpyGaXFeIUpipisn927Ci5ZhteKYWYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182381; c=relaxed/simple; bh=BHUup4Ao15H5a0x4s+HXfGoWgWU/14gkkZKIK0ZXAdk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=nUsx6l+sjizG0mDqEoNPZHxlAPrimOxm6tM/XbXjM8DkacAfIJpDkieiFy0wcoOvBQe7bc7M90QW6EUadz7Hw3U3TcTOw6KAcC29DlsPYPkkIl6lKkUp6hCRcLQIhIXsOO4OO4pJakYCDQ9EMASHWc1oNAS3HBICxmT72kAl8jg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=AfPFhgi+; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-40e86ddcee5so49352285e9.2 for ; Thu, 25 Jan 2024 03:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182378; x=1706787178; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=usiDcusxejCbluC36Vpk0on80rVNZ7Itzv8e5WwmSQI=; b=AfPFhgi+OTBDvCG51iWK+mk9bdc3YtI0Wgk5Bt3uix0iQ1mGkhwwm4hyj3NwP7HTYl BUvwQJ3wYMcmEm5nc+eej38a1AFydvIiL5sN3Yd5gx4uIU0VmLCNDZuqxE6WyO7nEuMp BN0UkTiXViYzXfX+qOeIjMuugNnwWhjWnzvuZGcAtg7TPQ9xAt4o5lmwSM/U21dNoLty MTOAktO15RNlvph1D+xzZoLa2eTdd7wuu3SppPmbh8RbVOFlXBdecizDXPSxUCWZ8UJ5 RnPxnfoEBjl7gs8b6DZgYNwOitbxbAxw0Qe5kFEO41p4pu/UVZM8okzrdoC0gRfheUss 2vsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182378; x=1706787178; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=usiDcusxejCbluC36Vpk0on80rVNZ7Itzv8e5WwmSQI=; b=v0VPjNryIAPthgZs8vgrOAdQX+etIOTDGqSMvVBHxyQa0b8+KNvnFjdjOOGuTkiodw k3+V1oGeFgIR81trXFcbaAC2DcW6Ndtw5A/0ub4ajPC63nJvza//Ek/AEnSnql+0VScP 0u8sOt+0QoMs9EV9dddRuTKeYAjDPKjUXAPGndt9Z9uIDKmFWtolrjx3zTgSnDgX4Cvb D1RfLkvR+uAifvW7LDCh+fELUGH5eq2zTJBr1g20YjVBXqRWpnIyZOIB0aoUlpRUw2/i npjLHYGZkq2oDTPlWONMI0R20npXvxxEUUMSoS2/s6YsdkeQ3GmrWu7MOunMks7tqVI2 uH8Q== X-Gm-Message-State: AOJu0YwM8y6OToRrI2EMy9JGKuA6eizy3HvR7Xf2KOEOmKZzajlFEVxP /x17mkpPcYPqeHN67OLdNvaedVrXg9cCyKvuqj4CGcvPu3qjZtgEueK6a4+45+QMUZBvPCt8Wvi UJNvTBcoZ9N3qjHyXdJ/EuLGnNoszYYtyW8NlkmZzYYP+cQ5S2NoUqNM67rJVKJmOuMcRuMmiO6 JdOvRAfQIqU12TGJP+1OKdXnG613Fqmg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:1e1c:b0:40e:d2b5:f9c8 with SMTP id ay28-20020a05600c1e1c00b0040ed2b5f9c8mr10496wmb.2.1706182377907; Thu, 25 Jan 2024 03:32:57 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:28 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1335; i=ardb@kernel.org; h=from:subject; bh=+Hn8XwOONyFACfVl0+TV+ToDQrCQwdquUdW+H4YgWKM=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT68tdC0+Wx7q93Op2UuuemL2I66Gngl4O3H+7Ly35/ Kpcsku8o5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAExE+z4jwyFGw42c5z81zN/r MM1w9uVn5tMD5VhYn2RUh1t9nJx6ag4jw4GL548zXrGP75w8r+pPiuyaa+HcfdeX9M3nENW5KuM UxwMA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-28-ardb+git@google.com> Subject: [PATCH v2 09/17] x86: Move return_thunk to __pitext section From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062094545271499 X-GMAIL-MSGID: 1789062094545271499 From: Ard Biesheuvel The x86 return thunk will function correctly even when it is called via a different virtual mapping than the one it was linked at, so it can safely be moved to .pi.text. This allows other code in that section to call it. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/vmlinux.lds.S | 2 +- arch/x86/lib/retpoline.S | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index a349dbfc6d5a..77262e804250 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -134,7 +134,7 @@ SECTIONS SOFTIRQENTRY_TEXT #ifdef CONFIG_RETPOLINE *(.text..__x86.indirect_thunk) - *(.text..__x86.return_thunk) + *(.pi.text..__x86.return_thunk) #endif STATIC_CALL_TEXT diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S index 7b2589877d06..003b35445bbb 100644 --- a/arch/x86/lib/retpoline.S +++ b/arch/x86/lib/retpoline.S @@ -136,7 +136,7 @@ SYM_CODE_END(__x86_indirect_jump_thunk_array) * relocations for same-section JMPs and that breaks the returns * detection logic in apply_returns() and in objtool. */ - .section .text..__x86.return_thunk + .section .pi.text..__x86.return_thunk, "ax" #ifdef CONFIG_CPU_SRSO From patchwork Thu Jan 25 11:28:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192022 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1573835dyi; Thu, 25 Jan 2024 03:35:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IE4AUfNShZ4vg4OloELd0adDy8mJTVaeQ+e6TIYtVPK6RjmtAhInceHqWpfkCygXD7W+yrI X-Received: by 2002:a2e:82c3:0:b0:2cd:cda6:55df with SMTP id n3-20020a2e82c3000000b002cdcda655dfmr625385ljh.62.1706182532761; Thu, 25 Jan 2024 03:35:32 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182532; cv=pass; d=google.com; s=arc-20160816; b=SKqcFhtEwLsW5l3m1Hfba+y6uqLPFnCjl6vNPiJdAuETnD0cMR3l3Jwkv/exBI1qRJ RcosDt+1PG55srh6yqj50GaamcSTOfE4WFWt7443gAMB/DYHSSWyALAbHs87ByNWWJ2R orssm5lO5C8pTHT3TSJOcIz48hBny8XFG+SFqA2XqrSZvq2DQTx1dcuGS6vKSCmKeFTV inNY2RJ27taFmCu40fmdZI0PHaO/NagHXMJwwbOveU/Ye08au5SK+RseapbZ1iSQ1mzZ eXrV0lpLuWMoArRREw6xMaexZeKEDH7FRueAONZMp+yQFljXkFD5VEJD1inlB3qDlv1F 9PIA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=RdRmN5FNC9Qw8qYorfxYqmWorkk70z2Gc350vuxR288=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=XKCVpb07/YyQozxcR6Pu2A954tv7BKbYRz911xjsMV8f7JS4FoDhv3hBpwt1GeQX7O gKZVQTPfhzjiBJxnLvrNpzLhdF8bu0jNWLQ/lRuqOLNbNkdK2fRsQaTfVhav9+0ilVLl wzOKsOVc2jj8A3fbkVII13bCB/lJ9dpJk2224ibX5R7nKeh7XwTbp61ivj45MK3jW8pp Z9wg2p5KPK/U4yFG3g4WL9EvB8NUaZYBuO7+I46PotlTr6SDKce0BhxI1zui+GmvAZzR SNAkhbsDzWWd2DoeJsjj0yJUjTSfMpgi3VDCxUl4TZqQUQu3QWfFlUhYJg9cH33RDOv9 Ipiw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=bCgvnz8T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id d11-20020a170906344b00b00a318faf3da4si267736ejb.241.2024.01.25.03.35.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:35:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=bCgvnz8T; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38514-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 373641F241D9 for ; Thu, 25 Jan 2024 11:35:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6A9BE53819; Thu, 25 Jan 2024 11:33:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bCgvnz8T" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC8A3482DC for ; Thu, 25 Jan 2024 11:33:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182383; cv=none; b=RuibYks1Y+mp2VsMqsYlK6bCTt4P4GU241A9LadH4bil6XZpOKLTaM+6qno6UgqPvQyfDPYTQukocFmqOGDwey2bMUE/6p239RyxjdGfVK1VigkEqWx1hFk+Iko5QCDwI0b3BV5o6ijCgSicLqBNC2+oHGMnGWnBBIaYNwr6YA4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182383; c=relaxed/simple; bh=nW9SmCdlADHOE6Omr4PFRwdaZThbp8qbwCml5fp8EWk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=X5Btt5WGZxFmswVVrvxE61A8hZ4/ESSmBN9B24QORpGd9KmAl6c5/o6TuAeNp1pEf8LT0+xlIxUL3HQ9lKzm3d+rSNbqAj8ltNvCfFNWG8h/7S7pA7zRA6Uu2dd3q2e4Qubfyi+LUzLIHqhUwazRdnb4aSF89mevIxXZgIo5EXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bCgvnz8T; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc2470bc0bdso8868666276.1 for ; Thu, 25 Jan 2024 03:33:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182380; x=1706787180; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=RdRmN5FNC9Qw8qYorfxYqmWorkk70z2Gc350vuxR288=; b=bCgvnz8Txbo20T4ZGLux/SN/pZi6d96ER9IuSuaxP3Gk0zphk0+N2Ppkp4Cc/TeLC7 pnHrNQTLf2n+UXXUbEnrE+p2zgbErCIQ4ruCbW5pSzMdwiVjGWbnpkrpdawgEFFCCWdA X3DcfNwr8Zn3SP67M8B/uG8R2FdS+FuV5lQ/Xw9LQ9RDlRoUgGrtdaRW58cFBZgl0xNC Qpwnt+4DX/IKygmLj03jF0eqVmugP6QB3B4Gqv74lhxkKukq0Hb4p1+rc1E1DAykFpmu Xj+BAARNwWMt0jNwvQG/DI2Ro22zMPYy0NeH8Ey18fbDk7ApNkqENThkKMfAzYvVYosj mKtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182380; x=1706787180; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RdRmN5FNC9Qw8qYorfxYqmWorkk70z2Gc350vuxR288=; b=xJzWsSh4RmOzO6gvcHGpLFBNWQE9bZe5nj7575pJdIUAowg0MilGrYcqAHvakpiyR3 lBKQlUsUe2ywvzKWkOErNeIDeIUBMsUji/0r8bnw02N5xtU55y1uWo/nusyVaFmT9m0d YM7IUEwWPP0qGB5gM1eNi/+/xo0GZ8UTeA6S8iB/OJFwF8e7bGUNYTbyhoSw/J1THI0X zjQCXawHY7bNZhuBkxhSjCchrt2iQrALyc1KQO5Wkn2vieoPv7GpkWvKNRT8xR0VIT7P myOopkmmlKFVq4EZ50GPzUEvRinhbGiggw7tj5tDcxhM6S/vjOyHEUikMC1tXZ9UF0Zn 56uw== X-Gm-Message-State: AOJu0YxWLX/IA+FC291LPQ3qqexT5ikk2pYKl7pmFzaaKoqoqzlK0OeU NBYN2hYxnEkyPLFjhjXspLT4PxOe/E2UAJDZWseHYkVgP0IFe60Itrz0N+gZwmP3WJfCdJfKpnN 81rUfzBqsor2aOHK+p8UiPNWp+9qqjHiMakKgZ5PMjll8Vf97wIrxdghk5/jxdgSgYuch5cdC1g gHi8G3FZjdeSDtTDgdgS5ojTI7WLwS9A== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:2407:b0:dc2:65e2:58f3 with SMTP id dr7-20020a056902240700b00dc265e258f3mr85436ybb.7.1706182380556; Thu, 25 Jan 2024 03:33:00 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:29 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3942; i=ardb@kernel.org; h=from:subject; bh=NDQz8Ub0YeUmHEZHj1/2gw+nBRrbtOLR0AuDt5JEbB0=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6+tOhnUbZt/4pv7feq263bHT+yJ7HN1ebknYPeV/T kbiHuunHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAifKYMf+Ue7yjove9+NW3v qrOrE8WS/Cq02Bf8ZNqfx9ay/nlsqxDDHy67p271b9cX1GyKdPLS1Im3nZDa0XjGtoK/97ZL+Rx VXgA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-29-ardb+git@google.com> Subject: [PATCH v2 10/17] x86/head64: Move early startup code into __pitext From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062055734616393 X-GMAIL-MSGID: 1789062055734616393 From: Ard Biesheuvel The boot CPU runs some early startup C code using a 1:1 mapping of memory, which deviates from the normal kernel virtual mapping that is used for calculating statically initialized pointer variables. This makes it necessary to strictly limit which C code will actually be called from that early boot path. Implement this by moving the early startup code into __pitext. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 9 ++++---- arch/x86/kernel/head_64.S | 24 ++++++++++++-------- 2 files changed, 20 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 993d888a3172..079e1adc6121 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -70,7 +70,8 @@ static struct desc_struct startup_gdt[GDT_ENTRIES] __initconst = { asm("movq $" __stringify(sym) ", %0":"=r"(__v)); \ __v; }) -static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdval_t *pmd) +static unsigned long __pitext sme_postprocess_startup(struct boot_params *bp, + pmdval_t *pmd) { unsigned long vaddr, vaddr_end; int i; @@ -113,7 +114,7 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp, pmdv return sme_get_me_mask(); } -unsigned long __head __startup_64(struct boot_params *bp) +unsigned long __pitext __startup_64(struct boot_params *bp) { unsigned long physaddr = (unsigned long)_text; unsigned long load_delta, *p; @@ -508,7 +509,7 @@ void __init __noreturn x86_64_start_reservations(char *real_mode_data) */ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data; -static void early_load_idt(void (*handler)(void)) +static void __pitext early_load_idt(void (*handler)(void)) { gate_desc *idt = bringup_idt_table; struct desc_ptr bringup_idt_descr; @@ -539,7 +540,7 @@ void early_setup_idt(void) /* * Setup boot CPU state needed before kernel switches to virtual addresses. */ -void __head startup_64_setup_env(void) +void __pitext startup_64_setup_env(void) { struct desc_ptr startup_gdt_descr; diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index b8704ac1a4da..5defefcc7f50 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -42,6 +42,15 @@ L3_START_KERNEL = pud_index(__START_KERNEL_map) __HEAD .code64 SYM_CODE_START_NOALIGN(startup_64) + UNWIND_HINT_END_OF_STACK + jmp primary_startup_64 +SYM_CODE_END(startup_64) + + __PITEXT +#include "verify_cpu.S" +#include "sev_verify_cbit.S" + +SYM_CODE_START_LOCAL(primary_startup_64) UNWIND_HINT_END_OF_STACK /* * At this point the CPU runs in 64bit mode CS.L = 1 CS.D = 0, @@ -131,10 +140,12 @@ SYM_CODE_START_NOALIGN(startup_64) movq %rax, %cr3 /* Branch to the common startup code at its kernel virtual address */ - movq $common_startup_64, %rax ANNOTATE_RETPOLINE_SAFE - jmp *%rax -SYM_CODE_END(startup_64) + jmp *.Lcommon_startup_64(%rip) +SYM_CODE_END(primary_startup_64) + + __INITRODATA +SYM_DATA_LOCAL(.Lcommon_startup_64, .quad common_startup_64) .text SYM_CODE_START(secondary_startup_64) @@ -410,9 +421,6 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) int3 SYM_CODE_END(secondary_startup_64) -#include "verify_cpu.S" -#include "sev_verify_cbit.S" - #if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT) /* * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for @@ -539,10 +547,8 @@ SYM_CODE_END(early_idt_handler_common) * paravirtualized INTERRUPT_RETURN and pv-ops don't work that early. * * XXX it does, fix this. - * - * This handler will end up in the .init.text section and not be - * available to boot secondary CPUs. */ + __PITEXT SYM_CODE_START_NOALIGN(vc_no_ghcb) UNWIND_HINT_IRET_REGS offset=8 ENDBR From patchwork Thu Jan 25 11:28:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192025 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1574248dyi; Thu, 25 Jan 2024 03:36:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdwkZyiPT54E92kSSJxBS5n96fR+uD8LGji1kqddlGAmdMbukmMapanxUMZr3+iVATCVk/ X-Received: by 2002:a05:6e02:1a48:b0:361:bc55:ab83 with SMTP id u8-20020a056e021a4800b00361bc55ab83mr1044006ilv.16.1706182589392; Thu, 25 Jan 2024 03:36:29 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182589; cv=pass; d=google.com; s=arc-20160816; b=axtbzINkGoLIzUbUMbk/iy7uMU1Jj2smm6lTeGItXVjYTfT1CMxC6hRwkEOFzsK3+8 cnLj1EgjQAfcuFsBGisjBPvWYbzRUejalPKy6YB0zmViatuQgU7xKDwzwDK0F/Na8yYv rrWEjf4OxIwTsG+aXJmA9Tn5p8RE7ifrhUFqOneeoUTLsYoXb5bVwOSm9dDUGpftDj9+ fjE1bwztEpRIykKTvzHj+fXqMpdIMi/beAF9Kj5GPXWXQLGrwMNEEop+6lexCOYIAaQ1 qgbKjInYQuk7ghFLpC38hs5vTg/I0vjvu/zmHeCVYDM4AEoxqHKzuZlGOP+EUMJt6Tke taCg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=fLLl1bcVbRYtnIzLBeH2xJGti8beoepv/z/uSZlJPHo=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=tnXCoRchjExohgAoDVeMqYEBRo+wEXgkpnlfkDPDydZMKJe2EilI9XE8Vc/7FES8VJ S7F+rEwsDSyxDJXZeIedMoLSMXBv0nJ/IwIm1vMV99JolTHA97veYX0KSOsqkra2HH6A WcHv8eLlxsoCqQidlkJ9mzH/g6k0efS38hwq9vqCt8T9QRFOweBJZk2CZvDLsHrRsLgq 6PwgjQus5hYkGt4ikPuKzZ6k30rZWnH43vytaglBeY5WdQzF9as9e3mOLo2G32cQvJ0V J3+AgUUL7TT+MVm334wU82hpH4NT6Cu5xWucGfzsd7tyjmPBYwfpdd8v0kj8eHuDkpcj YZvA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=XQytFWvY; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38515-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38515-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id bw14-20020a056a02048e00b005cde387778esi7715312pgb.368.2024.01.25.03.36.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:36:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38515-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=XQytFWvY; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38515-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38515-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 01BDE28A282 for ; Thu, 25 Jan 2024 11:35:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D909055C0E; Thu, 25 Jan 2024 11:33:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XQytFWvY" Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A0FC51019 for ; Thu, 25 Jan 2024 11:33:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182386; cv=none; b=TTS7OO5nVmwjDeTQuBqQ6KITuZ4jK2ScRzosxruiV6JnFia+QlstNuqb0akkb3nexwb8oRwGpSsM/5XNdA5v+5B8TI/3iONXZddZxSHFcuQGMXfT+n+I1dOaLuJz+lyrvSDcH7b1IDEI37IumBxe1/07U6THjJ7OqrcwQlua88A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182386; c=relaxed/simple; bh=ywQcTtrCDMHJcI+QkXWX2nFyJwoVE8xehI2i+Z3nGsk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m4cLuWE8vnjuNKUo7UOYMX2+P18kQdMi3PHF2NXE8eVJRdQaOF5uqqfDLJEvtLLy8TQfbeChxFBL6RVBqmuM96eS2fLC76CFxmsuYCvYoJEGW7nsZpn8gH2SBngrl/CJ+H8h11RL0okHJNKVsYrAJaBC52AHJZZIknRSXcvGLTA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XQytFWvY; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-40e5f548313so56765455e9.1 for ; Thu, 25 Jan 2024 03:33:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182383; x=1706787183; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=fLLl1bcVbRYtnIzLBeH2xJGti8beoepv/z/uSZlJPHo=; b=XQytFWvYz0bgtEjZlafmrVEl4JEGRW1LWuubtwVLI9igMlHJWWkC1YTeP1FRIfRIAx lucFeZl+VkbkXqEHqmtIfC2zdtOJabRsOMcySxVIGN+SWIgdiMmSzR1DV+BRVBaHM6aL OyKMYAceo2D2x3l7aaGlWtnDOebsXAvRPQaPrGs1Wo2QYQXt5fbBwSjlC92u+ufupZrE MT9Fcz1Svw4NkYbd22PQwyARrrASBQorKidt+F5+IpxYIA1yz3OJRAuHuhJMBU76Z3St s6YLKYzMys6U5ZrHCK7wLb/IgTwzfheETOCp3HI58mbn+MewguaWN5vOkZN9fZ7rKWM6 quYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182383; x=1706787183; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fLLl1bcVbRYtnIzLBeH2xJGti8beoepv/z/uSZlJPHo=; b=tCW/sSfR2lTWyd5dtdB2PtkinaRxrtzm/kxN6pb402HoHkBHvXI9L6HQ9nnD6ZtwyK tavneyUPXqTGYBi6n6IM7KxqnTwEXlPJMbR1Ik4kgmfdOWCZ1bKTF/1EMWbLvlR1AWgB dygyoxkNyEP1QsEh4IW2ZrvIE1/qrRiTMsdvKfRv1FRxTGACmukcaft1wfuUd6YLZmSo M3XdPgUvb0opFNrHIAbLrEPHeKkZJZOVKwlO/4XmDXDfs1PYC4EWv97ssRPOSytKoTDF zUjTBuccHmg8SVHOmn2p7rkPadOwDkT81RiNQLZqrQjMA2i+9LXtrF3YAqNYVm/uR8xE 2rVQ== X-Gm-Message-State: AOJu0YymLKYUcRO7N7mYUycIRBhApTRXrAyKTYcDQd5zMeR+nF/whtk6 hC6tNoCU0ayvdXpfu8AT57wAmBRDOEk7iBmVGsrRj8cBSefj1toa1MeTu1HGWqR55biOuBvdNd4 jBdsc1PTj4kcKsSHKQJhL3fxGqsX4KFtCv1R/dXDhnmz+ZVdX3cuLJTvgZ2id0ibSr9J1zBC/cl nTZLiTIe3BprugU0g0Y/l6Tzo+uL9RPA== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:4e0c:b0:40e:c720:f327 with SMTP id b12-20020a05600c4e0c00b0040ec720f327mr14276wmq.4.1706182383051; Thu, 25 Jan 2024 03:33:03 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:30 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=979; i=ardb@kernel.org; h=from:subject; bh=DDaOUA3J+CFMHhm3e2XXAeDqtyhjkdr8A2J8hJRp8QM=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6wfmH+v9haLvWOZmMFrJXOT6Yz094E3RseZ9K47vm /XtUvaHjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRpg5Ghm0sH1ScTbRyzT9y LT5+VOxYRanmRG27Jp4J++av9Zuc7sLIsLeAS7395J5HEZYJvUVbV/Nxt13WjZNdJpJ7wl/vXc1 bLgA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-30-ardb+git@google.com> Subject: [PATCH v2 11/17] modpost: Warn about calls from __pitext into other text sections From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062114845909294 X-GMAIL-MSGID: 1789062114845909294 From: Ard Biesheuvel Ensure that code that is marked as being able to safely run from a 1:1 mapping does not call into other code which might lack that property. Signed-off-by: Ard Biesheuvel --- scripts/mod/modpost.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 962d00df47ab..33b56d6b4e7b 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -825,6 +825,7 @@ enum mismatch { ANY_INIT_TO_ANY_EXIT, ANY_EXIT_TO_ANY_INIT, EXTABLE_TO_NON_TEXT, + PI_TEXT_TO_NON_PI_TEXT, }; /** @@ -887,6 +888,11 @@ static const struct sectioncheck sectioncheck[] = { .bad_tosec = { ".altinstr_replacement", NULL }, .good_tosec = {ALL_TEXT_SECTIONS , NULL}, .mismatch = EXTABLE_TO_NON_TEXT, +}, +{ + .fromsec = { ALL_PI_TEXT_SECTIONS, NULL }, + .bad_tosec = { ALL_NON_PI_TEXT_SECTIONS, NULL }, + .mismatch = PI_TEXT_TO_NON_PI_TEXT, } }; From patchwork Thu Jan 25 11:28:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192027 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1574440dyi; Thu, 25 Jan 2024 03:36:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IFUJM9Aw022NZtS8BwqlehRB8kaliu7Hfjhx+ecXGu0hXKTE1IgCRyeTd/EDzCgi8A/UvLp X-Received: by 2002:a17:907:76c7:b0:a27:a977:9131 with SMTP id kf7-20020a17090776c700b00a27a9779131mr434093ejc.93.1706182609939; Thu, 25 Jan 2024 03:36:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182609; cv=pass; d=google.com; s=arc-20160816; b=KoiFE3Y1FwwsTbkvzYW3tq2U3FGlm62nTG9Itts4twd0UefdcN8iD0IzrKbqJRv+sC 3PBTCTL11gLKFZgNPpYH4PprB5SwriW1k95W1mjjS3VgkjlXPYIBNe6YfHL1ArLMKbwq J8ISbiTK43Oyv4U8dhYtQmj/grNPO/olLvxWQkCwcwv0oHe225lTTMB4c8FPN4Q94qOZ IrLo53SMr6IyX9IkOQe2SIkCHQUBOUhOwilZ//aPoA3rWcGSPwm7f3ymBKboLOW/7wd8 2GMRc+6M7ps78sQou1z0fe/0mfaMonFYGpPvQZ7JC0bOsVF0DgYKOrOd6Esq/vBjqRHz al3g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=6DdM1fA2/BUC7ScxyUb41JZesEtpKP4YlIE0jOqBF4A=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=KJvKkV0qaKthdxuPQgVATkNsY1k2FA34YaPupoIvFs3C0C0E+0DOwXcYgiZuo1E0Ix w8PB4cH+vwxhwlTtpejJu/hILBh1ny9Bign02VttSlVvQWJSVQu5TiUXVsEvv7UOqV6D Bnb152FeYMYJDkxOoKV1TUaDo86zrnJUZilXYAupW+o5ZmH1F3jhaksWSajJ7FOI8GqP gg39isTcwwNyd+tt9jidXlpbXiRGUb4sQKm0EVYxkxTP3mohSNucwYokBshWKSNaiVeh g3pFQo9aj009J/pLD6ykmUGbSpigcmW8igASxpYTeUr6Ak/fobcTbjW9L+3VWXNiwsMk Kuyw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=EUVR1Obo; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38516-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38516-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id t22-20020a17090616d600b00a3163680dcasi707483ejd.726.2024.01.25.03.36.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:36:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38516-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=EUVR1Obo; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38516-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38516-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 649831F22A53 for ; Thu, 25 Jan 2024 11:36:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E09045786F; Thu, 25 Jan 2024 11:33:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EUVR1Obo" Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DF1B54BE3 for ; Thu, 25 Jan 2024 11:33:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182390; cv=none; b=reLkhAoNVEZXpiOR0/VcdHHKjC9x3jTEXebpXVw6GwaLO2sHv6vOx3KFCufcFlvGEBbrBB1pV/FXMEPcHjDQcLUaFN/tzCiF+Hmg8SJ646VFpYb2Fmwj8+Iu4VCnQ8tMBiZ82sxuLiQpU/BlE/+CMLnMI/SvHfDEbml93hs7qWY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182390; c=relaxed/simple; bh=2tkXbI4kkP7Bg5PktzWdMGd8q8tWspLGt7tFV60fDKM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=S8YtOBVWoC/jIEAHW8UTU8HxkH3qTf1DPA2vGbCJDQdR4fA1EgOfm4xxEW1Ql3wlunBavQ+7S7Uw5iPI95i8ZrqmXlrI920gwuhPGYv3YF/5dAsIWGOahZF71VQgs/RQwODaKfSTdlm6DnI2CvI6ZnAModdC0ekQFRmT9h1spIQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EUVR1Obo; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-40e61491b81so2748295e9.0 for ; Thu, 25 Jan 2024 03:33:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182385; x=1706787185; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6DdM1fA2/BUC7ScxyUb41JZesEtpKP4YlIE0jOqBF4A=; b=EUVR1Obo8J5S4JUDK9Vu9muWNHvllPSi0sc9wX03vWhO4w3P2v3AJwxPr7E+J6mnby g8Ptd3X7l7POxEFaU7IAni4pJDqEEej63FRH0bXqUIwkx59lpKl2tykLvvIn9k4uSrsZ s7x+y/FSY9gMc1FeH3iJw/CUHTeBWbSXUoSYyeT3jhWB8qN8XLw9QqyrV7Nq/xNOUhe/ zuPhHi+6lfw6Q4D7l/CNDqH/50WeYu5+e72NuaLiU/9IQHkbpRmb/7r/DNAWU3Dia48w lXxNvIMGJfolyNivyguTFvdo8EAez9mnE19BYzisMpeo12AxT4PmETzIrepyPsbt3qfl ttrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182385; x=1706787185; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6DdM1fA2/BUC7ScxyUb41JZesEtpKP4YlIE0jOqBF4A=; b=wFwtu6f2znhDTRIN9QHd5gbz6Bu2/icE7YxNR1LStn9BN0Bwtz8SjUzAE33KyNz45b Xhl8TsjgOfGqxCIsn+YwcE7LIJM4o+uhug6KhFDCUSpkycCe6Adw3jvZcHH5TnfEKkIx JVPSSqhupk3o63WPKJra5WuVAqc66jTYX+oqlUjN1jMb9lMN4/RbA5/64ZCq/z+RuoLh gMJgHZQqoFs/r+bNd/GH4gOvqioTnzTGAE4lK4qnSZHF1NNjHE7d5Ec6zqFQDg+uhxQx ScjUA8mF/VBJw1mcBEkB0oRzu1MDfOYjKlVFhSmA91KEkVNygRgGVMjQKPa4q/3UbS/h PUhg== X-Gm-Message-State: AOJu0YywP3Z9ltZwW06GOa58ew3WHehYaRPk7HISy1nkIScP87BPWui+ j6G2sEvP7FRKm57Q6OX2pea/EflC+pORh5UO5IErF9vpfssO44DDXuxmrNEa3cxZ4SELxTwsocq boHKyJ44ho+LTqgz4In8kE2ZXr2V7OzuGmJY8SzxvcT7M+/KuXlVQ6ZjtMp5PdPv9/EfK0/v/Ir i94WteqLXQQu9L4PhAAilL82pbGQpfGw== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:3c8f:b0:40e:541d:dd64 with SMTP id bg15-20020a05600c3c8f00b0040e541ddd64mr40221wmb.4.1706182385129; Thu, 25 Jan 2024 03:33:05 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:31 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1516; i=ardb@kernel.org; h=from:subject; bh=XIt7BesiIe5Ouf8eLT5Jk6gT3ML8ZNdNuPHWU+248+Y=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT65eLE7uWmk2dq6Kz8dWG4PnqX5YX5X3TUYr0tLFyE m78IB/TUcrCIMbBICumyCIw+++7nacnStU6z5KFmcPKBDKEgYtTACZiEczI8LglN+rBwh+LaxmN ne73hRUuFn7LcdbD44WH1g6pXXLRfowMa/fpH3z+e/YjJ4esjoxCpfbczZOmiv4Of7fe3k3Zd6Y dDwA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-31-ardb+git@google.com> Subject: [PATCH v2 12/17] x86/coco: Make cc_set_mask() static inline From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062136600153413 X-GMAIL-MSGID: 1789062136600153413 From: Ard Biesheuvel Setting the cc_mask global variable may be done early in the boot while running fromm a 1:1 translation. This code is built with -fPIC in order to support this. Make cc_set_mask() static inline so it can execute safely in this context as well. Signed-off-by: Ard Biesheuvel --- arch/x86/coco/core.c | 7 +------ arch/x86/include/asm/coco.h | 8 +++++++- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c index eeec9986570e..d07be9d05cd0 100644 --- a/arch/x86/coco/core.c +++ b/arch/x86/coco/core.c @@ -14,7 +14,7 @@ #include enum cc_vendor cc_vendor __ro_after_init = CC_VENDOR_NONE; -static u64 cc_mask __ro_after_init; +u64 cc_mask __ro_after_init; static bool noinstr intel_cc_platform_has(enum cc_attr attr) { @@ -148,8 +148,3 @@ u64 cc_mkdec(u64 val) } } EXPORT_SYMBOL_GPL(cc_mkdec); - -__init void cc_set_mask(u64 mask) -{ - cc_mask = mask; -} diff --git a/arch/x86/include/asm/coco.h b/arch/x86/include/asm/coco.h index 6ae2d16a7613..ecc29d6136ad 100644 --- a/arch/x86/include/asm/coco.h +++ b/arch/x86/include/asm/coco.h @@ -13,7 +13,13 @@ enum cc_vendor { extern enum cc_vendor cc_vendor; #ifdef CONFIG_ARCH_HAS_CC_PLATFORM -void cc_set_mask(u64 mask); +static inline void cc_set_mask(u64 mask) +{ + extern u64 cc_mask; + + cc_mask = mask; +} + u64 cc_mkenc(u64 val); u64 cc_mkdec(u64 val); #else From patchwork Thu Jan 25 11:28:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192026 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1574403dyi; Thu, 25 Jan 2024 03:36:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IHNwzUSbuYSX6kfXitWcrH06ZA8dO7L26UkOgCzHNfm7JLpnHSZeCngcfIt+v1y7eWjzv/S X-Received: by 2002:a67:c99a:0:b0:469:a26c:cd40 with SMTP id y26-20020a67c99a000000b00469a26ccd40mr523491vsk.71.1706182605596; Thu, 25 Jan 2024 03:36:45 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182605; cv=pass; d=google.com; s=arc-20160816; b=hjQZdrjvWfp1K2EdoSIsI4HgNA0OGHQxdq4i/kXvvu/zXNORSAYdOZzFNjH2WwraqV m69MMUPrdtGWrY82HnI1pve14HSgGmQrN/7AHxVJODjvmSwCO1lGq09yIQbBw7QrIfzE cWtNnZ+9Y2k19dsGZ6XD6wh4eibec54fKMLSe7KWDNBAd6GlGsZ/ZpOEnwtOTBRk/RT3 o1mzl5sb1L7Fgm3Cu70pQ9kn+KingRaRkJufOtuEpwaB+uuIZwCRjt/AOtN7vEaw5rZA 7qP3fks5Xk3DQDR6UEFlvoX0tH2GqtUAbluNmGC2kX3pnpOc+o+j4SFSSnGYcxnnnC3S KzGw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=OB2IW1nCo9uLqanub4h0z041TKeK3DX8sIz1w6b+cd0=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=yezXqLd9wPZIJqXfRT1K5q/rQdg5GA3/JL69XxInjmk/MoyHi96UB/6IAqVeYRbFIs BqRYwUdPa5u6mYuI6ao9JFv8mzfUqWS2qmKCh4qUDAndEqmKYR2EAcX72NbLATWbxvN2 J2ZJ7kAAgajunkRM7/kDnUq8MEAWF2P09yL7QD/w7GwpDX5eaGdLRf/idYxFFKl6NeHn AOj2xzjRoyjhfHNrqa3cc8HPbAly7c2L4/t0HoWgX5kr61xpcIdj2tKD2uR2qn4djoX5 eNqXOGogaNmu2H2a0dPJQsAnyncLnON/VOL8tYYkQ2U6M6N79t0GfMiIUjO3Nl9cB4cj ai6w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=o9rxPBZL; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38517-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38517-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id k5-20020a0cabc5000000b00686a51ade2bsi5379916qvb.610.2024.01.25.03.36.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:36:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38517-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=o9rxPBZL; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38517-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38517-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 5206F1C2236F for ; Thu, 25 Jan 2024 11:36:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C90DE5A10B; Thu, 25 Jan 2024 11:33:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o9rxPBZL" Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3277C55797 for ; Thu, 25 Jan 2024 11:33:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182391; cv=none; b=MJYyKLFpD1V/wcx3cxUZfrK9kNC/Mh8aER68B2NN6ypdafQzKnmsACjuQVXflrahyfh1j3zu9admJl0CuUdS5uXn1vwNEPFGXE+f2Z07CcI4c9MJYJe0oeIJekDsVCh1fmm5jMl6sgwJBiQhf8RbEwppD3h49BDz/XHqYfoHz2E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182391; c=relaxed/simple; bh=i13AZ6vlGzK6PZ7ioBLkahjbrj9IBafL4dafK+/RwuI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PyChhWSwbldBvzrrwHIx2WksJ+b/A3A3Fxx0MB2cU6bc28hRTFXDLBA7/grILU+A+4BaGE39uQNS+Bv9kT+saWUKAGo8JtiMkMkysYFrtTG2Uiq+QU7liKXAKVIz/QcVsLG+wOY06BTwyGEvWFNdT3mCetZOd4t4Cg/fmChZwew= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o9rxPBZL; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-337d70f889cso4211888f8f.2 for ; Thu, 25 Jan 2024 03:33:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182387; x=1706787187; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=OB2IW1nCo9uLqanub4h0z041TKeK3DX8sIz1w6b+cd0=; b=o9rxPBZLK/GcgakL4l7qfM7kv0QYZDsjVFDDFfL5sip6AyIm5Npeu9VlVt46GjC04M CVS9Lz1E+2odSypQTQgFFu+++iYrXoyRHz4V4C+BCjWOqezxDXITkNcGY8DU5pL5p1XF XmNZiPKencNynBJy+RBW2XOhKBLYTs/4ypbtKZOQwPFYnjUWhNfWfS97Oej47MGA5WQA JaIZmoNl3y2XC10y9AuapSJLZWAoXARdZIggNuSa/Iazd9FmpQ4WnAfMfdUQcyW171Zm NbJrkU1OpYV7TT5MZcaiaj5drM3JB+0lsZ93AvQ/Z90Q4adiNnICbKLHIj8Rj3p7LWHb op/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182387; x=1706787187; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OB2IW1nCo9uLqanub4h0z041TKeK3DX8sIz1w6b+cd0=; b=Oj7JZSco45l6SlMdBnSvCCvl8FnVqMcTJltmcpvwe4VzF6iTyxBTo8ipWEPBtWXgD0 uJ40aM+ciIaxgcoJ9P0PEGkEZmsbfoDjweXss1k2/TAmv8HqfUhylTiuOFUFyWwSANrN e1lZx/etI+tOGnDDnE7Ky6vSIBwANt07bBCkpAfK1e0Kzsce45Lp2Yv6fa5e7BOtg3CI eLDSxPPRvqkJRR2jYWaU1fxK8FMgrYWk6L0+GRf12vTXD98KYgtubmzeRSoogGqlnqZq EH07dgiIpq78y9P1ruYQjRQxlE9Qb4Zo/aOxdK9H8QUOTnAYlx1aAjF6CCrK2mEurqKR FHpg== X-Gm-Message-State: AOJu0YzqmWSI7BMP61TU6Y1wF9lzFfdv3IG90LZ5mCJq/AaMfYnxSa0l GHZ76bVsnQRX9UvFsxJELKEnZSr0V/VXV2/cKSnsc3zNlzk2P6PiUcTsYdogSzjybPieIdnrxiB BST8jetPmRnt7PPVlPWlQIrRwAv7jPUN0UgVe90LVyRPhbdHgKhqbh+tdTl5AbhEqK2QaVM4hae X17o3eQe0tIjpvxhRf9QpiGddNhOgcNA== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6000:1d89:b0:337:689e:6358 with SMTP id bk9-20020a0560001d8900b00337689e6358mr2607wrb.10.1706182387308; Thu, 25 Jan 2024 03:33:07 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:32 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=19087; i=ardb@kernel.org; h=from:subject; bh=hTH7IeEeDnitmVEmG1phWeF915Vntt3b+X6pDwpSBQU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6/c5uZWGey9vDn6gtdyuOt1pd/TaM2wFS37tufHpG 1tEcuiajlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRS1UM/yz5T8x+cOn/veJf 97dfM/5UmnyD49qJ4996FynbztWVy0tn+B91MzGMyzX11u8pUxs11/0SMLYQNBEJarpQbLt1nuY RL1YA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-32-ardb+git@google.com> Subject: [PATCH v2 13/17] x86/sev: Make all code reachable from 1:1 mapping __pitext From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062131796122251 X-GMAIL-MSGID: 1789062131796122251 From: Ard Biesheuvel We cannot safely call any code when still executing from the 1:1 mapping at early boot. The SEV init code in particular does a fair amount of work this early, and calls into ordinary APIs, which is not safe. So annotate all SEV code used early as __pitext and along with it, some of the shared code that it relies on. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/sev.c | 3 ++ arch/x86/include/asm/mem_encrypt.h | 8 +-- arch/x86/include/asm/pgtable.h | 6 +-- arch/x86/include/asm/sev.h | 6 +-- arch/x86/kernel/sev-shared.c | 26 +++++----- arch/x86/kernel/sev.c | 14 +++--- arch/x86/lib/cmdline.c | 6 +-- arch/x86/lib/memcpy_64.S | 3 +- arch/x86/lib/memset_64.S | 3 +- arch/x86/mm/mem_encrypt_boot.S | 3 +- arch/x86/mm/mem_encrypt_identity.c | 52 ++++++++++++++------ arch/x86/mm/pti.c | 2 +- 12 files changed, 81 insertions(+), 51 deletions(-) diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 454acd7a2daf..22b9de2724f7 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -116,6 +116,9 @@ static bool fault_in_kernel_space(unsigned long address) #undef __init #define __init +#undef __pitext +#define __pitext + #define __BOOT_COMPRESSED /* Basic instruction decoding support needed */ diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index 359ada486fa9..48469e22a75e 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -46,8 +46,8 @@ void __init sme_unmap_bootdata(char *real_mode_data); void __init sme_early_init(void); -void __init sme_encrypt_kernel(struct boot_params *bp); -void __init sme_enable(struct boot_params *bp); +void sme_encrypt_kernel(struct boot_params *bp); +void sme_enable(struct boot_params *bp); int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size); int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size); @@ -75,8 +75,8 @@ static inline void __init sme_unmap_bootdata(char *real_mode_data) { } static inline void __init sme_early_init(void) { } -static inline void __init sme_encrypt_kernel(struct boot_params *bp) { } -static inline void __init sme_enable(struct boot_params *bp) { } +static inline void sme_encrypt_kernel(struct boot_params *bp) { } +static inline void sme_enable(struct boot_params *bp) { } static inline void sev_es_init_vc_handling(void) { } diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 9d077bca6a10..8f45255a8e32 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1412,7 +1412,7 @@ extern pmd_t pmdp_invalidate_ad(struct vm_area_struct *vma, * Returns true for parts of the PGD that map userspace and * false for the parts that map the kernel. */ -static inline bool pgdp_maps_userspace(void *__ptr) +static __always_inline bool pgdp_maps_userspace(void *__ptr) { unsigned long ptr = (unsigned long)__ptr; @@ -1435,7 +1435,7 @@ static inline int pgd_large(pgd_t pgd) { return 0; } * This generates better code than the inline assembly in * __set_bit(). */ -static inline void *ptr_set_bit(void *ptr, int bit) +static __always_inline void *ptr_set_bit(void *ptr, int bit) { unsigned long __ptr = (unsigned long)ptr; @@ -1450,7 +1450,7 @@ static inline void *ptr_clear_bit(void *ptr, int bit) return (void *)__ptr; } -static inline pgd_t *kernel_to_user_pgdp(pgd_t *pgdp) +static __always_inline pgd_t *kernel_to_user_pgdp(pgd_t *pgdp) { return ptr_set_bit(pgdp, PTI_PGTABLE_SWITCH_BIT); } diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index 5b4a1ce3d368..e3b55bd15ce1 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -201,14 +201,14 @@ struct snp_guest_request_ioctl; void setup_ghcb(void); void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, unsigned long npages); -void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, - unsigned long npages); +void early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, + unsigned long npages); void __init snp_prep_memory(unsigned long paddr, unsigned int sz, enum psc_op op); void snp_set_memory_shared(unsigned long vaddr, unsigned long npages); void snp_set_memory_private(unsigned long vaddr, unsigned long npages); void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); -void __init __noreturn snp_abort(void); +void __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio); void snp_accept_memory(phys_addr_t start, phys_addr_t end); u64 snp_get_unsupported_features(u64 status); diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c index 1d24ec679915..b432cac19d13 100644 --- a/arch/x86/kernel/sev-shared.c +++ b/arch/x86/kernel/sev-shared.c @@ -89,7 +89,8 @@ static bool __init sev_es_check_cpu_features(void) return true; } -static void __noreturn sev_es_terminate(unsigned int set, unsigned int reason) +static void __always_inline __noreturn sev_es_terminate(unsigned int set, + unsigned int reason) { u64 val = GHCB_MSR_TERM_REQ; @@ -222,10 +223,9 @@ static enum es_result verify_exception_info(struct ghcb *ghcb, struct es_em_ctxt return ES_VMM_ERROR; } -static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb, - struct es_em_ctxt *ctxt, - u64 exit_code, u64 exit_info_1, - u64 exit_info_2) +static enum es_result __pitext +sev_es_ghcb_hv_call(struct ghcb *ghcb, struct es_em_ctxt *ctxt, + u64 exit_code, u64 exit_info_1, u64 exit_info_2) { /* Fill in protocol and format specifiers */ ghcb->protocol_version = ghcb_version; @@ -241,7 +241,7 @@ static enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb, return verify_exception_info(ghcb, ctxt); } -static int __sev_cpuid_hv(u32 fn, int reg_idx, u32 *reg) +static int __pitext __sev_cpuid_hv(u32 fn, int reg_idx, u32 *reg) { u64 val; @@ -256,7 +256,7 @@ static int __sev_cpuid_hv(u32 fn, int reg_idx, u32 *reg) return 0; } -static int __sev_cpuid_hv_msr(struct cpuid_leaf *leaf) +static int __pitext __sev_cpuid_hv_msr(struct cpuid_leaf *leaf) { int ret; @@ -391,7 +391,7 @@ static u32 snp_cpuid_calc_xsave_size(u64 xfeatures_en, bool compacted) return xsave_size; } -static bool +static bool __pitext snp_cpuid_get_validated_func(struct cpuid_leaf *leaf) { const struct snp_cpuid_table *cpuid_table = snp_cpuid_get_table(); @@ -427,7 +427,8 @@ snp_cpuid_get_validated_func(struct cpuid_leaf *leaf) return false; } -static void snp_cpuid_hv(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf) +static void __pitext snp_cpuid_hv(struct ghcb *ghcb, struct es_em_ctxt *ctxt, + struct cpuid_leaf *leaf) { if (sev_cpuid_hv(ghcb, ctxt, leaf)) sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_CPUID_HV); @@ -528,7 +529,8 @@ static int snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt, * Returns -EOPNOTSUPP if feature not enabled. Any other non-zero return value * should be treated as fatal by caller. */ -static int snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf) +static int __pitext snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, + struct cpuid_leaf *leaf) { const struct snp_cpuid_table *cpuid_table = snp_cpuid_get_table(); @@ -570,7 +572,7 @@ static int snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_le * page yet, so it only supports the MSR based communication with the * hypervisor and only the CPUID exit-code. */ -void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) +void __pitext do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) { unsigned int subfn = lower_bits(regs->cx, 32); unsigned int fn = lower_bits(regs->ax, 32); @@ -1043,7 +1045,7 @@ static struct cc_blob_sev_info *find_cc_blob_setup_data(struct boot_params *bp) * mapping needs to be updated in sync with all the changes to virtual memory * layout and related mapping facilities throughout the boot process. */ -static void __init setup_cpuid_table(const struct cc_blob_sev_info *cc_info) +static void __pitext setup_cpuid_table(const struct cc_blob_sev_info *cc_info) { const struct snp_cpuid_table *cpuid_table_fw, *cpuid_table; int i; diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index c67285824e82..e5793505b307 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -682,8 +682,8 @@ static u64 __init get_jump_table_addr(void) return ret; } -static void early_set_pages_state(unsigned long vaddr, unsigned long paddr, - unsigned long npages, enum psc_op op) +static void __pitext early_set_pages_state(unsigned long vaddr, unsigned long paddr, + unsigned long npages, enum psc_op op) { unsigned long paddr_end; u64 val; @@ -758,8 +758,8 @@ void __init early_snp_set_memory_private(unsigned long vaddr, unsigned long padd early_set_pages_state(vaddr, paddr, npages, SNP_PAGE_STATE_PRIVATE); } -void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, - unsigned long npages) +void __pitext early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, + unsigned long npages) { /* * This can be invoked in early boot while running identity mapped, so @@ -2059,7 +2059,7 @@ bool __init handle_vc_boot_ghcb(struct pt_regs *regs) * * Scan for the blob in that order. */ -static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) +static __pitext struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) { struct cc_blob_sev_info *cc_info; @@ -2085,7 +2085,7 @@ static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) return cc_info; } -bool __init snp_init(struct boot_params *bp) +bool __pitext snp_init(struct boot_params *bp) { struct cc_blob_sev_info *cc_info; @@ -2107,7 +2107,7 @@ bool __init snp_init(struct boot_params *bp) return true; } -void __init __noreturn snp_abort(void) +void __pitext __noreturn snp_abort(void) { sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SNP_UNSUPPORTED); } diff --git a/arch/x86/lib/cmdline.c b/arch/x86/lib/cmdline.c index 80570eb3c89b..9f040b2882ae 100644 --- a/arch/x86/lib/cmdline.c +++ b/arch/x86/lib/cmdline.c @@ -119,7 +119,7 @@ __cmdline_find_option_bool(const char *cmdline, int max_cmdline_size, * Returns the length of the argument (regardless of if it was * truncated to fit in the buffer), or -1 on not found. */ -static int +static int __pitext __cmdline_find_option(const char *cmdline, int max_cmdline_size, const char *option, char *buffer, int bufsize) { @@ -203,12 +203,12 @@ __cmdline_find_option(const char *cmdline, int max_cmdline_size, return len; } -int cmdline_find_option_bool(const char *cmdline, const char *option) +int __pitext cmdline_find_option_bool(const char *cmdline, const char *option) { return __cmdline_find_option_bool(cmdline, COMMAND_LINE_SIZE, option); } -int cmdline_find_option(const char *cmdline, const char *option, char *buffer, +int __pitext cmdline_find_option(const char *cmdline, const char *option, char *buffer, int bufsize) { return __cmdline_find_option(cmdline, COMMAND_LINE_SIZE, option, diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S index 0ae2e1712e2e..48b0908d2c3e 100644 --- a/arch/x86/lib/memcpy_64.S +++ b/arch/x86/lib/memcpy_64.S @@ -4,11 +4,12 @@ #include #include #include +#include #include #include #include -.section .noinstr.text, "ax" + __PITEXT /* * memcpy - Copy a memory block. diff --git a/arch/x86/lib/memset_64.S b/arch/x86/lib/memset_64.S index 0199d56cb479..455424dcadc0 100644 --- a/arch/x86/lib/memset_64.S +++ b/arch/x86/lib/memset_64.S @@ -2,11 +2,12 @@ /* Copyright 2002 Andi Kleen, SuSE Labs */ #include +#include #include #include #include -.section .noinstr.text, "ax" + __PITEXT /* * ISO C memset - set a memory block to a byte value. This function uses fast diff --git a/arch/x86/mm/mem_encrypt_boot.S b/arch/x86/mm/mem_encrypt_boot.S index e25288ee33c2..f951f4f86e5c 100644 --- a/arch/x86/mm/mem_encrypt_boot.S +++ b/arch/x86/mm/mem_encrypt_boot.S @@ -7,6 +7,7 @@ * Author: Tom Lendacky */ +#include #include #include #include @@ -14,7 +15,7 @@ #include #include - .text + __PITEXT .code64 SYM_FUNC_START(sme_encrypt_execute) diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 67d4530548ce..20b23da4a26d 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -90,7 +90,7 @@ static char sme_cmdline_arg[] __initdata = "mem_encrypt"; static char sme_cmdline_on[] __initdata = "on"; static char sme_cmdline_off[] __initdata = "off"; -static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd) +static void __pitext sme_clear_pgd(struct sme_populate_pgd_data *ppd) { unsigned long pgd_start, pgd_end, pgd_size; pgd_t *pgd_p; @@ -105,7 +105,7 @@ static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd) memset(pgd_p, 0, pgd_size); } -static pud_t __init *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) +static pud_t __pitext *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) { pgd_t *pgd; p4d_t *p4d; @@ -142,7 +142,7 @@ static pud_t __init *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) return pud; } -static void __init sme_populate_pgd_large(struct sme_populate_pgd_data *ppd) +static void __pitext sme_populate_pgd_large(struct sme_populate_pgd_data *ppd) { pud_t *pud; pmd_t *pmd; @@ -158,7 +158,7 @@ static void __init sme_populate_pgd_large(struct sme_populate_pgd_data *ppd) set_pmd(pmd, __pmd(ppd->paddr | ppd->pmd_flags)); } -static void __init sme_populate_pgd(struct sme_populate_pgd_data *ppd) +static void __pitext sme_populate_pgd(struct sme_populate_pgd_data *ppd) { pud_t *pud; pmd_t *pmd; @@ -184,7 +184,7 @@ static void __init sme_populate_pgd(struct sme_populate_pgd_data *ppd) set_pte(pte, __pte(ppd->paddr | ppd->pte_flags)); } -static void __init __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) +static void __pitext __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) { while (ppd->vaddr < ppd->vaddr_end) { sme_populate_pgd_large(ppd); @@ -194,7 +194,7 @@ static void __init __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) } } -static void __init __sme_map_range_pte(struct sme_populate_pgd_data *ppd) +static void __pitext __sme_map_range_pte(struct sme_populate_pgd_data *ppd) { while (ppd->vaddr < ppd->vaddr_end) { sme_populate_pgd(ppd); @@ -204,7 +204,7 @@ static void __init __sme_map_range_pte(struct sme_populate_pgd_data *ppd) } } -static void __init __sme_map_range(struct sme_populate_pgd_data *ppd, +static void __pitext __sme_map_range(struct sme_populate_pgd_data *ppd, pmdval_t pmd_flags, pteval_t pte_flags) { unsigned long vaddr_end; @@ -228,22 +228,22 @@ static void __init __sme_map_range(struct sme_populate_pgd_data *ppd, __sme_map_range_pte(ppd); } -static void __init sme_map_range_encrypted(struct sme_populate_pgd_data *ppd) +static void __pitext sme_map_range_encrypted(struct sme_populate_pgd_data *ppd) { __sme_map_range(ppd, PMD_FLAGS_ENC, PTE_FLAGS_ENC); } -static void __init sme_map_range_decrypted(struct sme_populate_pgd_data *ppd) +static void __pitext sme_map_range_decrypted(struct sme_populate_pgd_data *ppd) { __sme_map_range(ppd, PMD_FLAGS_DEC, PTE_FLAGS_DEC); } -static void __init sme_map_range_decrypted_wp(struct sme_populate_pgd_data *ppd) +static void __pitext sme_map_range_decrypted_wp(struct sme_populate_pgd_data *ppd) { __sme_map_range(ppd, PMD_FLAGS_DEC_WP, PTE_FLAGS_DEC_WP); } -static unsigned long __init sme_pgtable_calc(unsigned long len) +static unsigned long __pitext sme_pgtable_calc(unsigned long len) { unsigned long entries = 0, tables = 0; @@ -280,7 +280,7 @@ static unsigned long __init sme_pgtable_calc(unsigned long len) return entries + tables; } -void __init sme_encrypt_kernel(struct boot_params *bp) +void __pitext sme_encrypt_kernel(struct boot_params *bp) { unsigned long workarea_start, workarea_end, workarea_len; unsigned long execute_start, execute_end, execute_len; @@ -493,7 +493,29 @@ void __init sme_encrypt_kernel(struct boot_params *bp) native_write_cr3(__native_read_cr3()); } -void __init sme_enable(struct boot_params *bp) +/** + * strncmp - Compare two length-limited strings + * @cs: One string + * @ct: Another string + * @count: The maximum number of bytes to compare + */ +static int __pitext __strncmp(const char *cs, const char *ct, size_t count) +{ + unsigned char c1, c2; + + while (count) { + c1 = *cs++; + c2 = *ct++; + if (c1 != c2) + return c1 < c2 ? -1 : 1; + if (!c1) + break; + count--; + } + return 0; +} + +void __pitext sme_enable(struct boot_params *bp) { const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off; unsigned int eax, ebx, ecx, edx; @@ -594,9 +616,9 @@ void __init sme_enable(struct boot_params *bp) if (cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer)) < 0) return; - if (!strncmp(buffer, cmdline_on, sizeof(buffer))) + if (!__strncmp(buffer, cmdline_on, sizeof(buffer))) sme_me_mask = me_mask; - else if (!strncmp(buffer, cmdline_off, sizeof(buffer))) + else if (!__strncmp(buffer, cmdline_off, sizeof(buffer))) sme_me_mask = 0; else sme_me_mask = active_by_default ? me_mask : 0; diff --git a/arch/x86/mm/pti.c b/arch/x86/mm/pti.c index 669ba1c345b3..8fd1b84ab40c 100644 --- a/arch/x86/mm/pti.c +++ b/arch/x86/mm/pti.c @@ -121,7 +121,7 @@ static int __init pti_parse_cmdline_nopti(char *arg) } early_param("nopti", pti_parse_cmdline_nopti); -pgd_t __pti_set_user_pgtbl(pgd_t *pgdp, pgd_t pgd) +pgd_t __pitext __pti_set_user_pgtbl(pgd_t *pgdp, pgd_t pgd) { /* * Changes to the high (kernel) portion of the kernelmode page From patchwork Thu Jan 25 11:28:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192024 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1574205dyi; Thu, 25 Jan 2024 03:36:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IGYSxVswOr/PG4CuOmd+ztoTeMhxQF7+1uXsE21Mf3oXMusOVecoR9hr2YPvvTfy+P415vl X-Received: by 2002:a05:620a:8d9:b0:783:c539:16a0 with SMTP id z25-20020a05620a08d900b00783c53916a0mr302078qkz.148.1706182585999; Thu, 25 Jan 2024 03:36:25 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182585; cv=pass; d=google.com; s=arc-20160816; b=ZaIe+qpOL5nlOE9QiceJ2Vnci6yLbIwBzmjdn5aYchMiptCBp726LAlpcYijm2Vvxk 1pZCTOQuYsJQw7cLL/iHdYe68KKnYxvWT2VIQ6JCaQfGPGr7GWtFnKWggL++Prn6vWf2 9w0fKBZcZlwV9B3+aOJvVWILT3Yf2Jkbs6bfVoBBYLCNy2GxjOdwdkNEewivtL+VAcNu jmYov330JUmmtluOTeszN8iqjMZRdA7MGu02Z/l/CkNOjej41BnP6wUsIK9X+OIAzEbC nesuHvYLU3ZSB5RtgpkQAwGw7EgbqJHfk4lBsQwxZLwXR5TJQl6zGw9wMvzBbAMSm9+U yxyg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=Vu5UpmPKB2YLCZqvfGM/x/cMDohVrJdP/wVfbDfAQyQ=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=nSjbh/JXilN4TMDMa+PqaRtYsztQlNUsURJKeKovQJVnWKNJ2azl3Mvcpg973Ge4VU v8aB9HsEzkv7YJa9SlZC/fPkROG2568DlaRoq4uPo+mmoEt2JFXAgsPVfvfSxuAQd7wO bBVs6heOi5pMq/XaoIbsb/wzRCFY5wCrt6wXJU7jRVyOAtBbmCvDAhnPBUNV/6ziYPvG 5vzMmYDAI7ctsjMkVYrf1pel5dCJdVRJBV1oiLHISFpAGcuYKf6KXhIOgwZh2K/y9nIo zJ6zI1DAQm75ab6NEO6guva+32jBxaCOYwoFcm/30hukbrFKo9tsn267xX90sWL3zi5j MnXQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=av+QYO+S; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38518-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38518-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id k16-20020ae9f110000000b00783a2469a57si7874069qkg.340.2024.01.25.03.36.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:36:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38518-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=av+QYO+S; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38518-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38518-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7BA921C235C8 for ; Thu, 25 Jan 2024 11:36:24 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EF2F658AA7; Thu, 25 Jan 2024 11:33:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="av+QYO+S" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AC8D55C3E for ; Thu, 25 Jan 2024 11:33:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182392; cv=none; b=J6TS4Kd1IylkbwlGpDM3U2T3r1TgAZImo5D1DJb16IDd3BmJAObBsOpEyliaZggzUYrW/nuWiVJ9CojHe9YBeWcrBFZlMcxey9guWANdX1kA2fHvLOMetKme/YFfc3Cy3ZgxOSw7aw6P7cXLTXEOkEeyyeXlRXph+LQmuzwAJF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182392; c=relaxed/simple; bh=0byiLPnDWEQfwFYPOageH65AoBGROCIbfIjdgEIOffg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qYKfoj+o2vS4KjcNAUuqZA+/qn1CvE3hcEPVMK+B/Vt5HWNIzR7pb66yaHplextg65YCpvycSpTG088PdG9mdA/4jM84NpCvBFiMJqoSPpEL6OUYXasPahCm1PD/ls++jGl1mftf4QxMnuYocCoj450bhxTOBdsXwSnrfRNko/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=av+QYO+S; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc37b0ac33cso4472739276.2 for ; Thu, 25 Jan 2024 03:33:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182389; x=1706787189; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Vu5UpmPKB2YLCZqvfGM/x/cMDohVrJdP/wVfbDfAQyQ=; b=av+QYO+S1Ypn4bbQm60XaJny8/I4M0doE43MnICJrrdmJvb0Mb1FrBppM6mJCTCyrs pdJpCQjDs2UqTdbuvnI4hdL3n5+qE5heWujv5qBDWibzy8RaEp4UJk+VCUluGFdahg6W 0CtKZnVW+YkK9Gcjj+wdRrdSdU8h1E3gR5pj54z9PHa37kpywZBd58xgfZ3QZ1VGT4xA XEwekOrgB2LYWcO5Y4rJY4ZCtNifrH4JTHO/EOZj8qU6xD8Ain5g6b16pbPilz2GIHUq 1StI/8LQNN3uwqjHZMwQFm7UcvvHFbgGYBXZNVQPaso2UlcPSwJTsHCfQB1voIDL/eeY YvlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182389; x=1706787189; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Vu5UpmPKB2YLCZqvfGM/x/cMDohVrJdP/wVfbDfAQyQ=; b=T4I8P75EculdxZjF4gGpDLsor80RT8s7YnmnOB8pBfmgr6dUmvty/MlbDhE9Mg8iAv Lgn4cNk2/W9Q8NwuDeCY44yr8YFKwDkV/Oy/uJFXTPtU532XXfDga5t/XzBLXpwcxrPy G64Zq1Ayv+uQRoRTIKirJuUKXe92mWJViUadhXWRsLkylgD/My3TaT7Nfa9wl69EMZMy 6E/H0I6mIrJptFoRN4GjJtqQSZ3R9kaPTt1SZ/izMrE2Caom4ggYNMD4JHFP/nKuZNT/ GhSPCp7E/PphRfhCa/eKgwBR+8/EIMkntJYJtigDRPlJm/8Rl1k7I4JNyERfVqnDRiXV ARqQ== X-Gm-Message-State: AOJu0YxMPwTKZTz68lHJeAvPoHm1vQLJKobRAEc0jxnALmPAVi8zTKbf Kw3rKWyuTu9IXpyQ3/3pD0OjBqJQNzuqIln8BS3Bt+xRrXpQSOTFKoqsa8Jrl6+B3kPY1Mk+fH3 Dz/SXQ6zPCx5xECBm8YyDbzrhLlURhm42kdl94xgqL64yNFB5dRScr0fyFra995QOYzthao765S u0p+72saz4Gu+wMGFUnBvpmrjdGENZsg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:1ac5:b0:dbe:642c:2124 with SMTP id db5-20020a0569021ac500b00dbe642c2124mr453115ybb.0.1706182389569; Thu, 25 Jan 2024 03:33:09 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:33 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1898; i=ardb@kernel.org; h=from:subject; bh=Hd3xYUTi1QV5SlyH4OmfppihXDWH6zHA/zraGYWG1jw=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT6y/2x/MP/uz3CVv7baNn60+dbbIHuA/9mfjj3f1fR 1/P6RIq7yhlYRDjYJAVU2QRmP333c7TE6VqnWfJwsxhZQIZwsDFKQATmaTAyPD2+pa2Dxbsd/fN jrsvvvyopfMmqYBp+f8FDP72z02YFf2V4X+B2iyJ35HcJiKreLw4Tjw9tO5KjeqymsqthxJdpnt 0O7MAAA== X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-33-ardb+git@google.com> Subject: [PATCH v2 14/17] x86/sev: Avoid WARN() in early code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062111365732165 X-GMAIL-MSGID: 1789062111365732165 From: Ard Biesheuvel Drop uses of WARN() from code that is reachable from the early primary boot path which executes via the initial 1:1 mapping before the kernel page tables are populated. This is unsafe and mostly pointless, given that printk() does not actually work yet at this point. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/sev.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index e5793505b307..8eb6454eadd6 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -698,7 +698,7 @@ static void __pitext early_set_pages_state(unsigned long vaddr, unsigned long pa if (op == SNP_PAGE_STATE_SHARED) { /* Page validation must be rescinded before changing to shared */ ret = pvalidate(vaddr, RMP_PG_SIZE_4K, false); - if (WARN(ret, "Failed to validate address 0x%lx ret %d", paddr, ret)) + if (ret) goto e_term; } @@ -711,21 +711,16 @@ static void __pitext early_set_pages_state(unsigned long vaddr, unsigned long pa val = sev_es_rd_ghcb_msr(); - if (WARN(GHCB_RESP_CODE(val) != GHCB_MSR_PSC_RESP, - "Wrong PSC response code: 0x%x\n", - (unsigned int)GHCB_RESP_CODE(val))) + if (GHCB_RESP_CODE(val) != GHCB_MSR_PSC_RESP) goto e_term; - if (WARN(GHCB_MSR_PSC_RESP_VAL(val), - "Failed to change page state to '%s' paddr 0x%lx error 0x%llx\n", - op == SNP_PAGE_STATE_PRIVATE ? "private" : "shared", - paddr, GHCB_MSR_PSC_RESP_VAL(val))) + if (GHCB_MSR_PSC_RESP_VAL(val)) goto e_term; if (op == SNP_PAGE_STATE_PRIVATE) { /* Page validation must be performed after changing to private */ ret = pvalidate(vaddr, RMP_PG_SIZE_4K, true); - if (WARN(ret, "Failed to validate address 0x%lx ret %d", paddr, ret)) + if (ret) goto e_term; } From patchwork Thu Jan 25 11:28:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192028 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1574562dyi; Thu, 25 Jan 2024 03:37:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IECf5KiZWW95zMtg3CmDv75hIwaWNvHBDFc8UqsLXnnAVYKMXEYpsUcpnyufrAis/xcMS2C X-Received: by 2002:a05:6512:1092:b0:510:cfa:994f with SMTP id j18-20020a056512109200b005100cfa994fmr568254lfg.100.1706182623259; Thu, 25 Jan 2024 03:37:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706182623; cv=pass; d=google.com; s=arc-20160816; b=OcTBfJXh68VNh2IlGQOgbky50repqSqOQZ85qlkrkMMEpTOaA/Eu8lpE37ezCaetYq y39qKuxDmaqy8K0wVv9Jap6cRO3WOc607hcssmIm0wVj+RhWB+KmMriNAPezefvKmDkJ k6HgKtWgne3NGTHqopb3O0K1JDaUyDbDSI5LcHySdU6f/0v686qfqqKBJ5eMrJQTF86x l9go8XVqwFIjiCNIO08A4EFWo4nCZiwb2yr0E+MxQvprRhHH/41aEu/KWFRc3AF3n5V1 A9QA6TljrWR3QZgq3gM1CzHPpmNucGuIhs1talcX35RMlnDmPhXldCYuV99sYyxqVfLH 1HKw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=7LDdj00C7C86np8himWDzcjTQoz4YXOOGxPlqG/+mK0=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=g9LWBYgrH2Yhp0tbEmacEhhngPsIC/CnuvEL+8MeNkUk6n6Sra4M2dV3CgWdO2x3SN rXI+Kz3WHVrIYkw7Drd7VrVJGs5FnE1JCbjjJYbHaWOtJfDFJAQQItqyayEvLDPgRkjA weMbhaXmKTQEkPKea0PkdD1JRha6F+2GVdzHnfe8dtJw4lWMmy5gBHG4ow0LZfFkbxTf yrGf/7OCFu0PZcWfbMJEH1Cf6cU8sixKkEhcpKPKAiMMVbYAIws1WPS1tkf7iJiePV1y qFsljG5alNZ40TVhOJvpKEirAR9yFlQN04fMuz1ITA2L7t11tqfRFLR6qL4OZkiboiDf fgCA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=taaxhFRv; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38519-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38519-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id l8-20020a50d6c8000000b0055c1c5ae6d2si4882202edj.685.2024.01.25.03.37.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:37:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38519-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=taaxhFRv; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38519-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38519-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 0BD181F265EB for ; Thu, 25 Jan 2024 11:36:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4987D59B54; Thu, 25 Jan 2024 11:33:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="taaxhFRv" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C59F557331 for ; Thu, 25 Jan 2024 11:33:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182394; cv=none; b=krlnG0qlmkIuE/b/Iy6bK6i9JveCXcn7RtC+XJyRp5vBwWScTmmHe0iS+/tpY2r9pyDxQmIJZF/9XvGy8nIUMOxobm0OulaitoHbgYp9R4y+iqAMmhSteDGZQ5aZOM0/ORGi+4Tbh377a8FZ6o0HaPZhLIVP0HUH8YYiVbcNdik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182394; c=relaxed/simple; bh=53cr8/bRwWd0m61pZCiciaZiTFFAHiTaLmG9qZPd8js=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tdSpCu95tmq2nZFiFfXICyigkK4zFQbfLHrUqqXnucw1f4NdcL99ifjVsJQQVJgbop8n8vEOgSCt/ueCotywld1Ch6YKhelygmaM5rxGh37FLLZtKNXV8AaGecM1iiDp4S3poDlSwOgkU8WP/c2Rhll5zIb9PrrTK5CTG+8KqvQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=taaxhFRv; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-d9a541b720aso9713596276.0 for ; Thu, 25 Jan 2024 03:33:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182392; x=1706787192; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7LDdj00C7C86np8himWDzcjTQoz4YXOOGxPlqG/+mK0=; b=taaxhFRvBJ7x27l0OSpr3vw8YcmBhyQ4c2nMIVUArmpw+BcvPfxbwyrfACHztLpiTk knCWGvpQr2vejXCH8yeUeWNrLRxBNfuRWLE0bdc07HrlzEQihHZu1oBwoAcf4F+c8SAI 9HJ3QelivOwHj0HG8vIqJSxsZEcqq531Eo6CZixbSsPOCJtOSd0fYwI/4isy6Gp/jzHp W5PccyVU8g+03rE5zXUChfWQb0IxL1w3Pu2XziIEViFfESgzYW8TTXNDuFcBkQ5xM64L 1p6H4uN7aYsC0okITA86UF+N9/r93rJBrs1wuHgsa2ec2aDhd9rSMJZtlDVXaVD9Yr0y nOrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182392; x=1706787192; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7LDdj00C7C86np8himWDzcjTQoz4YXOOGxPlqG/+mK0=; b=MTgFn0BDlOWaEjax88cRWu8GkIIBZ6KjxBU73ImtjHyrIpFlJxTMio/ISnDeaVyVtl 6R2JldYJJFKiIYbLE8s4hjLCFKiEYApibhdhnvND3rULlVSCGG6web5Hn6XTwJXaIZKm DGs9cAZqJz/GvRVPpPxs0q2WZvLD+F2yXbolMpMGViT4e9Rf35xcV/DkvS68ooxXcykh 116pWgy7/PRHbuwiA2pMfyxNxW2uQrQDrxHMArBZdL+J5laqEmlUPJAYfr9oOZAXzh+j W4aSVk/Y/ACWMc0qNuaIPFwrLYPHTwTyTsvkw/G6ZcU3PJ/pdxoWlMtPhwi384kfiLF0 hjGA== X-Gm-Message-State: AOJu0YwVFGtu1sU5SqX8kQDG73FWiUf+nRnwF2SWnkjDXQpY7d4cgQpD YIC6I9j9yKtRzfk0JrM7wg0U/tTakGYasXeIHCHpMMMdJmllHhuQsNI2Q1juA+mryQF/7xxRJ6f I2GK6ILrPUYESWv4p66wOKDDL8AydPlPYEWJMHAT7+79R4ggqAPULhXvRVpdRs37ChmF/KgfXDT GhFabys+MfHg7XF9Bm+TU+nWAM2OV/fg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:108f:b0:dc2:23d8:722d with SMTP id v15-20020a056902108f00b00dc223d8722dmr436585ybu.13.1706182391890; Thu, 25 Jan 2024 03:33:11 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:34 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2380; i=ardb@kernel.org; h=from:subject; bh=NfyMVMLTwE2gaCOtLA5OAbEVC8C/2zEsck2HEztektc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWT678l+dzpjnF+wZfLxO/X9M9rvaB34Fq1rvE771Va/ 2WDd+/oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABOZG8TIcPHCL6dXrmYPXq0u +/nAkX3twjvhvyf4n/khfu+DqbPAJz2Gv5KiT+q1eYp+JNx95BpofMp0+aemWZPF3+8rX3BPpu7 OSnYA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-34-ardb+git@google.com> Subject: [PATCH v2 15/17] x86/sev: Use PIC codegen for early SEV startup code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062150325049744 X-GMAIL-MSGID: 1789062150325049744 From: Ard Biesheuvel Use PIC codegen for the compilation units containing code that may be called very early during the boot, at which point the CPU still runs from the 1:1 mapping of memory. This is necessary to prevent the compiler from emitting absolute symbol references to addresses that are not mapped yet. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/Makefile | 1 + arch/x86/kernel/vmlinux.lds.S | 1 + arch/x86/lib/Makefile | 2 +- arch/x86/mm/Makefile | 3 ++- 4 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 65194ca79b5c..65677b25d803 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -24,6 +24,7 @@ endif # head64.c contains C code that may execute from a different virtual address # than it was linked at, so we always build it using PIE codegen CFLAGS_head64.o += $(PIE_CFLAGS) +CFLAGS_sev.o += $(PIE_CFLAGS) KASAN_SANITIZE_head$(BITS).o := n KASAN_SANITIZE_dumpstack.o := n diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 77262e804250..bbdccb6362a9 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -182,6 +182,7 @@ SECTIONS DATA_DATA CONSTRUCTORS + *(.data.rel .data.rel.*) /* rarely changed data like cpu maps */ READ_MOSTLY_DATA(INTERNODE_CACHE_BYTES) diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile index ea3a28e7b613..87c79bb8d386 100644 --- a/arch/x86/lib/Makefile +++ b/arch/x86/lib/Makefile @@ -24,7 +24,7 @@ ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_cmdline.o = -pg endif -CFLAGS_cmdline.o := -fno-stack-protector -fno-jump-tables +CFLAGS_cmdline.o := $(PIE_CFLAGS) endif inat_tables_script = $(srctree)/arch/x86/tools/gen-insn-attr-x86.awk diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index c80febc44cd2..b412009ae588 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -31,7 +31,8 @@ obj-y += pat/ # Make sure __phys_addr has no stackprotector CFLAGS_physaddr.o := -fno-stack-protector -CFLAGS_mem_encrypt_identity.o := -fno-stack-protector +CFLAGS_mem_encrypt_identity.o := $(PIE_CFLAGS) +CFLAGS_pti.o := $(PIE_CFLAGS) CFLAGS_fault.o := -I $(srctree)/$(src)/../include/asm/trace From patchwork Thu Jan 25 11:28:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192040 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1580315dyi; Thu, 25 Jan 2024 03:50:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IH5SSviNt3fTEuRGuWn03xrB74xKeUQBTMbaymTqAAj3DQfC9vZjKBqIgwAhFKMx07emk1y X-Received: by 2002:a17:902:d891:b0:1d4:b50d:dba9 with SMTP id b17-20020a170902d89100b001d4b50ddba9mr886224plz.71.1706183424162; Thu, 25 Jan 2024 03:50:24 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706183424; cv=pass; d=google.com; s=arc-20160816; b=EuM5TWmZhaqr44obOlkZ0dhbqMjod1oKpBPIhA5gA+79cyiIlJOmikPmQce9P/S3ZM N3JDctTqpnzm/lsQf/QelVxr6u3lryEQEq4jFQumcynphi4yrBls1FnRl/jGYx46dOB/ 4jcpOTbd+QHfVNLVFSvqW/maT4tD9q6kBkGz0PcVzR239QqHfWRI4pe8Pkmky6ycjgM3 SmTUmZycijxKh2fa1ZpZDCXp3vBCsQNq++QfzQ5FRL+ZCR6pr8M8a8rNNl1/Fn6ujVzb lZlx9JKj0xrT/Oq0akm1kxmIRwv3ifRggnehgLi1hS0qukPoou8bZ6R3J7SfWRX2HWWn BkpA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=hBUWnFeDK9kwJnQ8v39o1YEuf+vNzkyqOxxuFopn5io=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=osBrI23Tbyup8VVThDUky5GyXXfLDNsbZDbZ0EULKJRf1xqQgmldwl/+oPItbNQ77b ug4tVNfqghqkGTZ5uYba1S/GvCSYATLvN+wiyulqc5OzYWwNmStZh3EPsoBWhmgHQ66m dBwIAiF9DcfqFkqgXiMW7VmsGX+R9DOVT12070cFNGXKXLMe2QvtCldon5vp7LDKJOu8 lAWfB/0Q+5rI43L9fOTInGbdicORdFrEuiBEdmJcI/rCTXfztZPui1eCvtU52o6iwsWa ftD+5rXspdeYyhXN6qk5la1r1RiYyp4JTvzaQskjAKTfMxVBbXb1RlW5xwEOyqK0b4Z5 8kOw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=DfgQR618; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38520-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38520-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id d13-20020a170903230d00b001d7166e8ef9si12119277plh.226.2024.01.25.03.50.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:50:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38520-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=DfgQR618; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38520-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38520-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 224BB287A87 for ; Thu, 25 Jan 2024 11:37:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E80315A783; Thu, 25 Jan 2024 11:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DfgQR618" Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F414C59156 for ; Thu, 25 Jan 2024 11:33:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182397; cv=none; b=gS6RiDKUz4kZLcGsqKTuJcRX5ENiItf+bsvuOlHw9N4aOnRdv92XvHibIkTl4H6g1g8xKVpUXOw2OqS2vHoceb7IADn324XSUY+dKyGOE0DgmVpjqo5BRXJVEcrcUMG1imE9epJZHkbUwtVBloC8urhIjbw+vD8j0l/NM1lBuAE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182397; c=relaxed/simple; bh=P7SsUUZGFAoKRtQbrSSh80tauvm72OXVWNmZkHlq190=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Lg+rBeUFc59EwFDrez6qOwf5xulcJz6ipilQBf/PWvd1fuPrgji14KJdqu+vG0Lrj4GFFjmG5KQVpdq2rtOkgdFGCT9vdhoswnRiiBuZ9PPp8B9faM0MINdhiaXGKRa25WUy//fQXHD9sK6r8vKzKiH2lQWcSeL+G9ttDkfiCz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DfgQR618; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-40ed389a4dbso1313125e9.2 for ; Thu, 25 Jan 2024 03:33:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182394; x=1706787194; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hBUWnFeDK9kwJnQ8v39o1YEuf+vNzkyqOxxuFopn5io=; b=DfgQR618KVP9F9t3DuWRIAiCRRXQtg1odKkiO4gDT2Hl4pmtnL61E9g23dXvk/J7GD Fvttl5fiRd3KxTQfFUfxj9cg1SJiHjtAbs7KeQze4TuSh8qgPA6F0PE0C6iebjLC1X7t 59tl16FCWi3q3+cx4ac/TsRvYibARXRoID3HYC7Mw1999UKU+ChySSOkGyYr6AfCvOPe HZvxUT8j1g74HITP9jzfyMFt5YgNIGRZIMV1Pwj4Q9Ql08+1Jj/v0KcyrOZxPA5b9D8K EMBkr3nXE5XFMwI2esO8UcJGagZFId07+nOsaIXSPfhIJKsaIxjEDLjRdy3YpSQxPprJ FZYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182394; x=1706787194; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hBUWnFeDK9kwJnQ8v39o1YEuf+vNzkyqOxxuFopn5io=; b=j+2OjXxFGkLiqxmXfMBSP79BPGEK78yf/6m7FaSgwo5yTrff/ZmBMDbxKyrFRwJhvC 4VfCxpTKmH9Ssz8QWnyCCw2FqLps17+cqZ5YUti+yCiIczv9qeIGO0g5THO4vZ0qjH4I xH09PJlFKf4XHP19yc4y+E0xm0KCGc9sXRdxq3fRLxsmkNLax3y0xW4IoJMi7MUzqviO TLmCx3ZpS2FDjeJZnd1WSTjbvgAIFP1l+i7CCg3zKJcTO8NcxyUKF+xTOh0aNF+Kgdye zXjsJ+TMkomtokAvsXoxiuNZuIglKKIqzAV571Qm6apeO6dq9O72WUmyzQeE0KG3aIg9 d20Q== X-Gm-Message-State: AOJu0YylXfiUxdiCkn5wIxzevtBwc9Zq2N0h4bMSZ7Ds+Ov3BBb1BfFz CPeijEKhKcuDAZ1HIJ7BWqwzLDzAo/RrO4FfUCShnTlql1zvFP97c8sykVs28LZOA90ZAxi0b2T swQWvqJHFiMShTfVg4ewdmjSs10b9w7c7TwoQQz0agdttX4+xVEebG3GDfSNNf0H2HGbbvqt8Zr dqdqilBo20QGBQ597w82xjw+rJzLpjMA== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:600c:1e03:b0:40e:d31f:4cf8 with SMTP id ay3-20020a05600c1e0300b0040ed31f4cf8mr16460wmb.3.1706182394116; Thu, 25 Jan 2024 03:33:14 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:35 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3710; i=ardb@kernel.org; h=from:subject; bh=JT1olNiwpLiAazvRYxL/s0mN0D+4v2kwCKTsMENT5/8=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWTG5Om96fFG3IbTStkj79Symr2m+K3+H2fZ6N1zjRVN wPZNXs7SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwEQ2VjP893miFvbly6s19kvZ X+/6Hxq4oTuqS/Xx3b/1m3/WqujxPGf4p6Slc1LTL3iz04Wers3bufROzWqd1m7CrXkmqbRDa1c LMwA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-35-ardb+git@google.com> Subject: [PATCH v2 16/17] x86/sev: Drop inline asm LEA instructions for RIP-relative references From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062990014925475 X-GMAIL-MSGID: 1789062990014925475 From: Ard Biesheuvel The SEV code that may run early is now built with -fPIC and so there is no longer a need for explicit RIP-relative references in inline asm, given that is what the compiler will emit as well. Signed-off-by: Ard Biesheuvel --- arch/x86/mm/mem_encrypt_identity.c | 37 +++----------------- 1 file changed, 5 insertions(+), 32 deletions(-) diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 20b23da4a26d..2d857e3a560a 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -86,10 +86,6 @@ struct sme_populate_pgd_data { */ static char sme_workarea[2 * PMD_SIZE] __section(".init.scratch"); -static char sme_cmdline_arg[] __initdata = "mem_encrypt"; -static char sme_cmdline_on[] __initdata = "on"; -static char sme_cmdline_off[] __initdata = "off"; - static void __pitext sme_clear_pgd(struct sme_populate_pgd_data *ppd) { unsigned long pgd_start, pgd_end, pgd_size; @@ -333,14 +329,6 @@ void __pitext sme_encrypt_kernel(struct boot_params *bp) } #endif - /* - * We're running identity mapped, so we must obtain the address to the - * SME encryption workarea using rip-relative addressing. - */ - asm ("lea sme_workarea(%%rip), %0" - : "=r" (workarea_start) - : "p" (sme_workarea)); - /* * Calculate required number of workarea bytes needed: * executable encryption area size: @@ -350,7 +338,7 @@ void __pitext sme_encrypt_kernel(struct boot_params *bp) * pagetable structures for the encryption of the kernel * pagetable structures for workarea (in case not currently mapped) */ - execute_start = workarea_start; + execute_start = workarea_start = (unsigned long)sme_workarea; execute_end = execute_start + (PAGE_SIZE * 2) + PMD_SIZE; execute_len = execute_end - execute_start; @@ -517,9 +505,9 @@ static int __pitext __strncmp(const char *cs, const char *ct, size_t count) void __pitext sme_enable(struct boot_params *bp) { - const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off; unsigned int eax, ebx, ecx, edx; unsigned long feature_mask; + const char *cmdline_ptr; bool active_by_default; unsigned long me_mask; char buffer[16]; @@ -590,21 +578,6 @@ void __pitext sme_enable(struct boot_params *bp) goto out; } - /* - * Fixups have not been applied to phys_base yet and we're running - * identity mapped, so we must obtain the address to the SME command - * line argument data using rip-relative addressing. - */ - asm ("lea sme_cmdline_arg(%%rip), %0" - : "=r" (cmdline_arg) - : "p" (sme_cmdline_arg)); - asm ("lea sme_cmdline_on(%%rip), %0" - : "=r" (cmdline_on) - : "p" (sme_cmdline_on)); - asm ("lea sme_cmdline_off(%%rip), %0" - : "=r" (cmdline_off) - : "p" (sme_cmdline_off)); - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT)) active_by_default = true; else @@ -613,12 +586,12 @@ void __pitext sme_enable(struct boot_params *bp) cmdline_ptr = (const char *)((u64)bp->hdr.cmd_line_ptr | ((u64)bp->ext_cmd_line_ptr << 32)); - if (cmdline_find_option(cmdline_ptr, cmdline_arg, buffer, sizeof(buffer)) < 0) + if (cmdline_find_option(cmdline_ptr, "mem_encrypt", buffer, sizeof(buffer)) < 0) return; - if (!__strncmp(buffer, cmdline_on, sizeof(buffer))) + if (!__strncmp(buffer, "on", sizeof(buffer))) sme_me_mask = me_mask; - else if (!__strncmp(buffer, cmdline_off, sizeof(buffer))) + else if (!__strncmp(buffer, "off", sizeof(buffer))) sme_me_mask = 0; else sme_me_mask = active_by_default ? me_mask : 0; From patchwork Thu Jan 25 11:28:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 192039 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1579092dyi; Thu, 25 Jan 2024 03:47:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IFGE45Wlpdge6qhf2poCFN6Gi0NrzkfYoJvFCQ90XSFCdds8uvYFBK8b0r0N5O1lx0kELYX X-Received: by 2002:a92:d9c7:0:b0:361:aead:d3fc with SMTP id n7-20020a92d9c7000000b00361aeadd3fcmr877996ilq.109.1706183242900; Thu, 25 Jan 2024 03:47:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706183242; cv=pass; d=google.com; s=arc-20160816; b=O7Ho7JVqjPwJi7LcZZJ0jMtWz4npE3zJNfmnBe3DzE2RY9v8WghUF6C0VWEYdmx+mb myaa5wCYmCFwMZfG+f9Z+OtI888+oyyxOkj2M9MSQgAsQML2Wo3w50hF40sEYEeZgOyN O9Y/wsEmdFaMbLH/xULJkXyMRTojG0e7ngW+BuKtzbG7GhaKHsEPWOwTrYFGMmy9AOaS Y8UKQ2/fR4MU9jg/4ggceih9SBY/IYjYhFzkAJNt2cwAdgCGX2dMMApaekyCvqX71rIV QQO5/2MFHhJy4w3s4/DtN/4bN+jQ6ib77p3oYmSbGHluqN4RYkTkrdsBjMSyfTID+pSy 8DZw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=M0LWk2oJk+i7OdHacQS8szXmAws8+WmD6+X8i7Byook=; fh=Mjd69IxHltS/Jba8SYEPy4aDMrScq2KBDxG2XjETtWk=; b=w7Z7IB45qNuK9xnZGsollHV4uhxqK12lheywCP8yzklwMnVIAMcPoFqDbbDy9Sih4P pk0Ge52duIiiHB+cbRGZV3bVpgtYDehu9dMt4JnjnRbO9HZvdaJmpT2WeqV1QeG7ex8U jFH+sn7pY3zVQKem1xOaUS2IZ5+zo8nb9Q+7zm+bCSH+MUPIiJuQ88qD8zfs2f8j/K74 Ud6MBBlhCwXm+biykM0Au6sLBV1Dhyxc1V/0vsXd51awVCPMuDrkUNb1bRYSqbSte/at V0FBj3O3x+VWdeUvukrLCKyVGT2NnMbsXRq67Ft/rP7MQQMkwK2gf/vrN6R1W0BgkYTm 4sRg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=1Hth3OCF; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38521-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38521-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id bx2-20020a056a02050200b005ce46d33cdcsi13794904pgb.343.2024.01.25.03.47.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jan 2024 03:47:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-38521-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=1Hth3OCF; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-38521-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-38521-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3E61C2894DA for ; Thu, 25 Jan 2024 11:37:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DDE625B210; Thu, 25 Jan 2024 11:33:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1Hth3OCF" Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 943B45A78F for ; Thu, 25 Jan 2024 11:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182400; cv=none; b=kQMTBFZsefw7/HuQyn0W+w2Om1a7TeAsroWwYBWjOzQbCD/Ayq7/GJiLY8+YqqUQENcdAep4rJ5awwinc3pAI5tCB9vSJAA/Pieo96wd5Xue/iQ04qLDfSfwibm1mMVrNqjNpAyQxJNbsGsmyl1gtSRmJb4iaVn/oV2s/alpwmE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706182400; c=relaxed/simple; bh=OwW6wBuNpjoZwXNRvFnESPahBXLxFOjLZ++ayooCsxU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YtxZuSFWpwb6d/wWws6rNqd8h6LHGmDLSXMtq5ol1MrYBHM82BpQTJoboLNgiz0cLs7jbPbYHRHGsQ9+Esr66ri9EKWS7ZwhObbpZXdOhLyhEXI0quAGFSa6x7hyfGM7FypHdcs03tguWVWYnszM+fDYNTYsZZ1iFJ5E46wft5c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1Hth3OCF; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-337cfc83240so4415242f8f.3 for ; Thu, 25 Jan 2024 03:33:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706182397; x=1706787197; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=M0LWk2oJk+i7OdHacQS8szXmAws8+WmD6+X8i7Byook=; b=1Hth3OCF/4CFibvfC8gzhxHcuSVsOXu60uqUUY0oLKkk6pv9o8ElXY9L55+hXUqerX KI2YJgtufUVANACpD82Vrlpb2sBqdzq5E9nEkNSPb25Y8apUgrbN3mKsBd28WMgBNHeB TXVwmWITB+P/AHPqzmEDr3fJ3eCQczSz2kNyUnsCt1pH2pfbMuuoukd2B/YEvZr2z279 D3kNRoo6PPWNEKhLAKLucFVvGxlpu1xu0P5rB1Z5i01WBVudbDRLJ6roiGvcL1eWk90I nDy1WDyU5LL/LLTDJvSKDSIOSjBTdBWCY4oPH4FyYlKE+O8dLuFRKpZcSoBGIAulR1VK CglA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706182397; x=1706787197; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=M0LWk2oJk+i7OdHacQS8szXmAws8+WmD6+X8i7Byook=; b=xMQ4UP850NbUXj2pVZFxD8H5FcIm6/FZKqccfZGwCAzOGyHXCraWJ9WP9VYmM8ab2T ptUT3vMP4QXf7TMrd3GdwqlY0lIw1kBJem0SE0IsRiN6bxQdfpOdASwttQWjzxONNfA4 HYm5a7NxXtjTf3AWlFWUtybCBK+6WbBwsr6R9gGCUJge8MBZdR7kPkn+9ga9LW73UyIr SllHjRLTG7Jg06MVrPv/6wzeWpxHSdAYJFm3Yd0oex7WB8TSrp/wGySBbxpB8R3HAa43 FxCPcKfIhEJTq9Kzqwn6dvKoTW2+tA42KaAcNGqntxmKkfh3khog4X8HBk1Nu8w78hpP HDdw== X-Gm-Message-State: AOJu0Yx1XXwwu3oaog0BrzhaWF5pq3f2XcovqcoDJwTuaZUBRWqLhdps fftQ2RBXhk4+iXKffmBL7/IE063yvfNZpWVHbhPCuFyywRQH1BPyvuDR4s6SDw3REYW0fHX8IgJ dZiIhZBu0pfspwxIrW4sA/+hZUE/j8NMY4xa2Z7eAw70oR2F3ojoBEcObWAPbiwWBbA3XPhRbxO cefNkBjPEw6jrqcrPR4vDadDnQNMG1Fg== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a5d:4a04:0:b0:33a:d182:fda1 with SMTP id m4-20020a5d4a04000000b0033ad182fda1mr704wrq.11.1706182396410; Thu, 25 Jan 2024 03:33:16 -0800 (PST) Date: Thu, 25 Jan 2024 12:28:36 +0100 In-Reply-To: <20240125112818.2016733-19-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240125112818.2016733-19-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1282; i=ardb@kernel.org; h=from:subject; bh=hctyhPpJV0WOf9z7vrzpK1P7aPRf6ODhgzfVMyHZJzc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXWTG8sTO09G1eh3s38HbSvasYtlpd3913LP1YUZb+y3l Gtsejy1o4SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzkgx7D94wEFmeVtglT7X2n ae2pfHhf8JzyxIQtHd+dJiq5nqrMYmTYsNiqaMdW0aOzHgsVla44+tL+ZWwd59c1V192eaTtZ4v iBAA= X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240125112818.2016733-36-ardb+git@google.com> Subject: [PATCH v2 17/17] x86/startup_64: Don't bother setting up GS before the kernel is mapped From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789062800120926363 X-GMAIL-MSGID: 1789062800120926363 From: Ard Biesheuvel The code that executes from the early 1:1 mapping of the kernel should set up the kernel page tables and nothing else. C code that is linked into this code path is severely restricted in what it can do, and is therefore required to remain uninstrumented. It also built with -fPIC and without stack protector support. This makes it unnecessary to enable per-CPU variable access this early, and for the boot CPU, the initialization that occurs in the common CPU startup path is sufficient. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 7 ------- 1 file changed, 7 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 5defefcc7f50..2cce53b2cd70 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -76,13 +76,6 @@ SYM_CODE_START_LOCAL(primary_startup_64) /* Set up the stack for verify_cpu() */ leaq (__end_init_task - PTREGS_SIZE)(%rip), %rsp - /* Setup GSBASE to allow stack canary access for C code */ - movl $MSR_GS_BASE, %ecx - leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx - movl %edx, %eax - shrq $32, %rdx - wrmsr - call startup_64_setup_env /* Now switch to __KERNEL_CS so IRET works reliably */