From patchwork Mon Feb 26 14:29:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 206681 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2113976dyb; Mon, 26 Feb 2024 06:32:07 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWN+Lb2ipX6fuZNev4K9rgikt0Unj2+YjH24CjGZIkMkwyQ8HjxfAEkVGZkZ2CCyQtrmsVgwxRChl7LyqrHoZwZXsWrpA== X-Google-Smtp-Source: AGHT+IGXnT+f0baZosuUsEnXaiIVqKvNVnlR5gqkCahjdWGptQXgtEfKrIEhciRtw4H1yedjeSUV X-Received: by 2002:a05:6a21:3102:b0:1a1:863:e783 with SMTP id yz2-20020a056a21310200b001a10863e783mr763780pzb.17.1708957927347; Mon, 26 Feb 2024 06:32:07 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708957927; cv=pass; d=google.com; s=arc-20160816; b=z+ZCb57FJuyQYVHnrdfrCl/EFXggfY/ihWCZnTkL6gb5pwv0Cia+u5/Ag8ZTqj9FsH gMXsRd2qevsy/wsX0luzU3Zupo7b6eWzs4IzYphIvF3oWaFt3EaMf3Y8ZsIShnV4S1uR B2x6FzVq6PgS+c0j2hOCmWWSscTh4ANLR6xO3oj7TfqP6EN1D6bN+z9GQRAmSzO7Yv5m oenz9b3FbDV5klW2DVaOyYoXfvt6Jeq5W1/bDwSMrCqOrNVy6sRPW6DRMbRa59U187XL K9DZejh9n3bLY3OJtaVdJ6QDXTDXusH0GZqLQRDnbCNLF2gurb9lIc7slbSpOZYQWn6X Y3NA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=CpT/v58LZa+6DN+5ZZ1kc+YtmkGbFtn0oji87VtEKmc=; fh=GwInpN0wyaUEy1HIz2WqUneHJB46QWkS/TNrezKinns=; b=s4WLHGVHFOppLs5Eh2ji6twxkJcNrkII2UR/9WEiluvckxyCqweasPfbzt1xPBhqgJ Fl5JOiNNqGrxDusTJVy8mpbnpNCLJ8rdnXmUSU1zU0uXMj+/D9nNxEaHIsChx2ZHfDb4 ZQnwYlHU4oVkl3gj4vImdnnA68i1173usH1PROyP2eBS/v4umBwskAy/cxSoVclctW9Y j/7QD3hOT5Gv6jfV3lVni4G6MPl3i5tgmgd+KdtBF1gJjhLAh5j0/qi0wMpAtKyc38jb jotORmtRiJlz2tHHUPjg/j3GX5q4FfVdfU4YYqV0Yyq15iKIln9d/CpYLeEMvMlNZCui DP9g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=dbLixok0; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-81622-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81622-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id f11-20020a63e30b000000b005ce025116bbsi3727579pgh.858.2024.02.26.06.32.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:32:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81622-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=dbLixok0; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-81622-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81622-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 8F45B28AEDC for ; Mon, 26 Feb 2024 14:30:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0AA3F605C8; Mon, 26 Feb 2024 14:30:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dbLixok0" Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E844A12AAC4 for ; Mon, 26 Feb 2024 14:30:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708957804; cv=none; b=qEJzLkwJTEjNok8i25o63GygvXE99hPNUt/3dJOEZf4iHm8XrwZ13hZF4R72gjGzhNlypFWQreP9Aj+6VjTTYTJNB7aDhg+A48EP97KeSk4dUAFrsKMdeAbp7W9yVlJldzYbJZP9OxJzH6YhgUjxuf0S+qzJUnIqB/Gmtot1F50= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708957804; c=relaxed/simple; bh=wkfgi3tF51NfZa7UfUiPY8DHKdeYcx95mPJdjnoKkNE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jPug9fe0L+Q3pSYbYVPXzXHExV8Fq+h7HUx6jWmQljKs/4PSZUE/cVBQZlioIgm40V6pJT0WNk8xW9/RfV7PR2X9lrJK9QosaHYFd5jG0H13Ghxp9e2FegPzNIvJvakypf5IS8teX1cexi2E1QTh3ElAMdZVTlDLx46XJoHEjSs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dbLixok0; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-608d6e7314bso28507027b3.3 for ; Mon, 26 Feb 2024 06:30:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708957802; x=1709562602; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CpT/v58LZa+6DN+5ZZ1kc+YtmkGbFtn0oji87VtEKmc=; b=dbLixok0jF3K3eFbyGlBl51OgWZHSZN4xu+kyb9DRD8dcCkAEXPgqnMkUKVcBw5XSE k9SS+SRZNR/qfjgUpdJ1xxuh59QXFA9J/MXlNUgjK+CtozRn69iGHwnncCfqSayZqw4r PgDCaUNULDumN4h8Ato3TjBtmr6Lp4cV61iwlPkfk5Zu2lZXgPE0iA4YBt0pfDEI2r7A UM7bfLKq3oWrLvJdhcYBUDkUoLhw1Ny8MN30eFoDN7RAvH9qIQQBTlTLQP1/zcsITIZ5 PSQq+li96fapPwJNieRvCTxwuUa+ZDKkYAx7/KYfEFK31K3/kne+T/uaXUzUsg4bO8rx VclQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708957802; x=1709562602; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CpT/v58LZa+6DN+5ZZ1kc+YtmkGbFtn0oji87VtEKmc=; b=hM4ZDp8oStyTK6YJpdEeUYzAndoAE2cow1YlGZVTZ9ojfgwTRHNimTCV5OmW//gchE RXn9IfR9e7hPZs5hQXwJOFNvn6O2VP36rmL70a/xlY97SYgooaeMQ1hHQd4Vc6L1+y9u tOqU+wkoQhkRttiHSNTglxNhnxBvzOPDcji65zRSKzExddT4lC+GvkUfx0mW26Xk6CL7 LMvxdKpULESmAwFgaJ8fTrS7T/F+5CCg+0ikIU//+rD6huCmjCOORa60tWowgNQsxb85 4c/8///xwYV3XdkippGVckaCY9mPazT0QVIfnsRL4+ypHXtgsL6iyACVBAsnUnnOhSem 39mQ== X-Gm-Message-State: AOJu0YxZdGuSOVO69haOxkuoO19gVt2w2xhF17MsNBMbpJ5DJBq/z7OX vRfV6dvwwZAs0ABpDYL9obg8FBHFijWYPozG33MNUN/9JNSqY1nAgNjFIgTYnQHPH6yGiJDQUjU ceYB5wNoKEoMoJwt6HWbjmUtgrTedovHdC1/F/GrsfUpusNZsy8Tmh7sBUT4H425ZL2OmsiJoQC VPP/vqGrFPG3NgemorY7hg6y7bJ2NUgQ== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:100b:b0:dcc:8be2:7cb0 with SMTP id w11-20020a056902100b00b00dcc8be27cb0mr244198ybt.0.1708957801956; Mon, 26 Feb 2024 06:30:01 -0800 (PST) Date: Mon, 26 Feb 2024 15:29:54 +0100 In-Reply-To: <20240226142952.64769-12-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240226142952.64769-12-ardb+git@google.com> X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <20240226142952.64769-13-ardb+git@google.com> Subject: [PATCH v6 01/10] x86/startup_64: Simplify CR4 handling in startup code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Brian Gerst X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791972267490260412 X-GMAIL-MSGID: 1791972267490260412 From: Ard Biesheuvel When paging is enabled, the CR4.PAE and CR4.LA57 control bits cannot be changed, and so they can simply be preserved rather than reason about whether or not they need to be set. CR4.MCE should be preserved unless the kernel was built without CONFIG_X86_MCE, in which case it must be cleared. CR4.PSE should be set explicitly, regardless of whether or not it was set before. CR4.PGE is set explicitly, and then cleared and set again after programming CR3 in order to flush TLB entries based on global translations. This makes the first assignment redundant, and can therefore be omitted. So clear PGE by omitting it from the preserve mask, and set it again explicitly after switching to the new page tables. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 30 ++++++++------------ 1 file changed, 12 insertions(+), 18 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index d295bf68bf94..1b054585bfd1 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -185,6 +185,11 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) addq $(init_top_pgt - __START_KERNEL_map), %rax 1: + /* + * Create a mask of CR4 bits to preserve. Omit PGE in order to clean + * global 1:1 translations from the TLBs. + */ + movl $(X86_CR4_PAE | X86_CR4_LA57), %edx #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. @@ -193,20 +198,13 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) * configured will crash the system regardless of the CR4.MCE value set * here. */ - movq %cr4, %rcx - andl $X86_CR4_MCE, %ecx -#else - movl $0, %ecx + orl $X86_CR4_MCE, %edx #endif + movq %cr4, %rcx + andl %edx, %ecx - /* Enable PAE mode, PSE, PGE and LA57 */ - orl $(X86_CR4_PAE | X86_CR4_PSE | X86_CR4_PGE), %ecx -#ifdef CONFIG_X86_5LEVEL - testb $1, __pgtable_l5_enabled(%rip) - jz 1f - orl $X86_CR4_LA57, %ecx -1: -#endif + /* Even if ignored in long mode, set PSE uniformly on all logical CPUs. */ + btsl $X86_CR4_PSE_BIT, %ecx movq %rcx, %cr4 /* Setup early boot stage 4-/5-level pagetables. */ @@ -223,14 +221,10 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) movq %rax, %cr3 /* - * Do a global TLB flush after the CR3 switch to make sure the TLB - * entries from the identity mapping are flushed. + * Set CR4.PGE to re-enable global translations. */ - movq %cr4, %rcx - movq %rcx, %rax - xorq $X86_CR4_PGE, %rcx + btsl $X86_CR4_PGE_BIT, %ecx movq %rcx, %cr4 - movq %rax, %cr4 /* Ensure I am executing from virtual addresses */ movq $1f, %rax