From patchwork Tue Feb 6 22:36:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adam Dunlap X-Patchwork-Id: 197668 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp1863395dyb; Tue, 6 Feb 2024 14:37:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IHDz4SzouOvBoHXSb3y9oTKi5cE2BRntQ5CCDGRGaUZVEzNq5HT9sY22a+R4I9k349tewUS X-Received: by 2002:a05:6870:1588:b0:214:82c6:ee14 with SMTP id j8-20020a056870158800b0021482c6ee14mr4499151oab.58.1707259043179; Tue, 06 Feb 2024 14:37:23 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707259043; cv=pass; d=google.com; s=arc-20160816; b=DZjzw/KDF/1r+10VuZ+2mncduW/m6P98w3IOGEjred63xIfi2vt8b5QSFMK6O6Q/FU XlIhySyjIbrIpZmzJuienjoZ9MW3BEdwrkO/AcZeOOqQhAaIWQtcKx74HNLHAcJJ+dT6 a7E+YY/x4APdb1K2oHnhDsNuUJIbw9HIxXddKygT8o4StkitaXB67Md8qh6nxyz/CX4u oW5sLn3k4kVoI4MvIL52YX8qbyQ7QgZPUvUWveXa/fTVwctwqusLu/CVZ9y9k8/gIhYT Swldb9ePsD1uK8e3Bwx2JYT6+PpnQsDBOz14f9Vyo280zJhkgFUwdw1RbfA2l564iPeS usdw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:date:dkim-signature; bh=S9KktXhbpCP9EKhC0dGFOyn7FIX3yfw9fFagtMfvzvU=; fh=zqgi8rZTGlfskJKSmAzDDhYo7wNKs/KiC1xDgtr2oPY=; b=YnpUtlsVhuZW3hTKfcQd4kNal5NsiO0YoGGw2Upg58831McXu3phtfkvxY/dAFb85y aKCnkDakmgeIZcLI98USMocCmkBubaOQ81aZv5g/SQZSutI+OfcDfQ5lTNmBC0HaOf4V Zz7UdB1oVaJGZAZdu0iHc4Vjac+qWvDsETuQamhcgiEyEnMT4foUctlD4JS+xk7aRlJt GR7YEG+N04RhbU8wRD8M2oBZSzcUnVyx1dMxDlsUCEkTct4GPgoCWOhdG/k97hGYsQrE /ur53dF1PDDGaPkmncNXQWH1EuEYFFe+IWq/vbfEsBhT6QaQA3kOqxg0X1nGxC1Jls/X PSPw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=G5LxyzAO; arc=pass (i=1 spf=pass spfdomain=flex--acdunlap.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-55696-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-55696-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com X-Forwarded-Encrypted: i=1; AJvYcCVaajZ6SH+gHIYHxtTv6wMpgKxXlFC5iV9QnI6HSEVXfHt2HcmvWb3yvh6pvGU5EXzY6f/xw6iLHcZyU0QZx7I1XnMxQA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id q32-20020a631f60000000b005d96d182c41si2409449pgm.490.2024.02.06.14.37.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 14:37:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-55696-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=G5LxyzAO; arc=pass (i=1 spf=pass spfdomain=flex--acdunlap.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-55696-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-55696-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E4F1C288D58 for ; Tue, 6 Feb 2024 22:37:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5AAF51CD3F; Tue, 6 Feb 2024 22:36:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="G5LxyzAO" Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B13F51CD23 for ; Tue, 6 Feb 2024 22:36:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707259010; cv=none; b=EhbPB0TRkX/rWN34Nx1vHZQRClAYNvVtVJ9TBtP6CPZIfiHzfR/hWmYGUFGMJ0PN5bJRIaS9rt+fisnNuMCUjmJw3SmH5XgBkb9KlbjhNtG47Ohx66GUWLbu9PxHzwOPGDDpGzWLkJYO+tB5HsFqgI7dr2eIAF+sLrmDx/sQiDg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707259010; c=relaxed/simple; bh=N6KTTGVlioPgR9j+o8pygtYiTQpajG3elp7P2BRhHJ4=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=hVpOrXczGgyEXAfNTcYjviOe1gWrroMzPn44yEHYm5CoWXZ4z9uxNjynuvSelSMPqoQQsq6+o3TxXjP0OKiAyAbyOYJIAz2vW1mmzdcfhqMH2bQbG5kmtnBPguQ2/aPqhhF65E4wys67gtEbuOpzgPGEpPrpTBlynCFJUEAuBgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--acdunlap.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=G5LxyzAO; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--acdunlap.bounces.google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-604833eb9c3so230047b3.1 for ; Tue, 06 Feb 2024 14:36:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707259007; x=1707863807; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=S9KktXhbpCP9EKhC0dGFOyn7FIX3yfw9fFagtMfvzvU=; b=G5LxyzAO+ayBNRBm0069VkDnn7tWoSqNRp4l92H/aVhLoI/DkPKbdM69tW/Fu2QPI2 rhfsXF/soqYDh07e9TWooiKYsyOcyfHNB251QWlH4v67Pyz69bYy8uiN5haXKbC64uI2 lx77BlPY/e89QCLrArwxUd5NW515iTIMttNE+PgwHq3tealykgze1QNCGAVwmE6rZZ1A u3IlBwSzSii0AZyfU3iTE6bl+jLiDLAiOHoKov/85sqeBXqKaZZUQRMrBB9ReHilwygL TtisyvfXDx0Rcb5Sm7kEzgA5EsuTUc3IgVm/zpix9FgEas4u1g1Aoh4bDxlahOFLRB9V KGuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707259007; x=1707863807; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=S9KktXhbpCP9EKhC0dGFOyn7FIX3yfw9fFagtMfvzvU=; b=GNHk3DZsFFBMYS9idGF9RwE8WAeSDhbPpFTUzWhwMUu5yMbhUrtgwjYDPLjFSWhhem iLmcTpLaSKPmVc9fD397aybQ6V5PinEMEiziyJ1OlAz5RdNnUicl8Khmlbqo6r8UyF/1 YZnRv9VNRtbU3F+2ZyxdSH37CoaCaAzlnuqGgtdt3HB7z/psR12BcQ71GAS+dkECIitF wU6nWaKJ9qN9fM0K2xoUdy7YAOADEe9i2r8PUMzeaGHl11gxXtCVeCOO19HvRORkGKX7 i5cMBcnC54ZKVC4BbK9OLsv9oSgKNV+lYCOxyU1UEXaG5i0XgzOHnsrrQ6WQACo/szMY seaw== X-Gm-Message-State: AOJu0Yzm3gk8AG7edMUDU35bYy8629XbE9qP4zI/j6WFz5mZiY3p3T4b q84PSJsFpon+dmq0DM4/ybkqF+P7QBVfuXw+wGwzmko/driQ6UC0sRVwqKmE9GYXfg+pBMXPD93 INHD+fEBxbA== X-Received: from anticipation.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:4517]) (user=acdunlap job=sendgmr) by 2002:a05:690c:fd0:b0:604:1eea:a39e with SMTP id dg16-20020a05690c0fd000b006041eeaa39emr435630ywb.3.1707259007638; Tue, 06 Feb 2024 14:36:47 -0800 (PST) Date: Tue, 6 Feb 2024 14:36:20 -0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: <20240206223620.1833276-1-acdunlap@google.com> Subject: [PATCH v3] x86/asm: Force native_apic_mem_read to use mov From: Adam Dunlap To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt , "Peter Zijlstra (Intel)" , Arjan van de Ven , Wei Liu , linux-kernel@vger.kernel.org, llvm@lists.linux.dev, Jacob Xu , Alper Gun , Kevin Loughlin , Peter Gonda , Ard Biesheuvel Cc: Adam Dunlap X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790190858429088676 X-GMAIL-MSGID: 1790190858429088676 When done from a virtual machine, instructions that touch APIC memory must be emulated. By convention, MMIO access are typically performed via io.h helpers such as 'readl()' or 'writeq()' to simplify instruction emulation/decoding (ex: in KVM hosts and SEV guests) [0]. Currently, native_apic_mem_read does not follow this convention, allowing the compiler to emit instructions other than the mov generated by readl(). In particular, when compiled with clang and run as a SEV-ES or SEV-SNP guest, the compiler would emit a testl instruction which is not supported by the SEV-ES emulator, causing a boot failure in that environment. It is likely the same problem would happen in a TDX guest as that uses the same instruction emulator as SEV-ES. To make sure all emulators can emulate APIC memory reads via mov, use the readl function in native_apic_mem_read. It is expected that any emulator would support mov in any addressing mode it is the most generic and is what is ususally emitted currently. The testl instruction is emitted when native_apic_mem_read is inlined into __xapic_wait_icr_idle. The emulator comes from insn_decode_mmio in arch/x86/lib/insn-eval.c. It would not be worth it to extend insn_decode_mmio to support more instructions since, in theory, the compiler could choose to output nearly any instruction for such reads which would bloat the emulator beyond reason. An alterative to this approach would be to use inline assembly instead of the readl helper, as that is what native_apic_mem_write does. I consider using readl to be cleaner since it is documented to be a simple wrapper and inline assembly is less readable. native_apic_mem_write cannot be trivially updated to use writel since it appears to use custom asm to workaround for a processor-specific bug. [0] https://lore.kernel.org/all/20220405232939.73860-12-kirill.shutemov@linux.intel.com/ Signed-off-by: Adam Dunlap Tested-by: Kevin Loughlin --- Patch changelog: V1 -> V2: Replaced asm with readl function which does the same thing V2 -> V3: Updated commit message to show more motivation and justification Link to v2 discussion: https://lore.kernel.org/all/20220908170456.3177635-1-acdunlap@google.com/ arch/x86/include/asm/apic.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 9d159b771dc8..dddd3fc195ef 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -13,6 +13,7 @@ #include #include #include +#include #define ARCH_APICTIMER_STOPS_ON_C3 1 @@ -96,7 +97,7 @@ static inline void native_apic_mem_write(u32 reg, u32 v) static inline u32 native_apic_mem_read(u32 reg) { - return *((volatile u32 *)(APIC_BASE + reg)); + return readl((void __iomem *)(APIC_BASE + reg)); } static inline void native_apic_mem_eoi(void)