From patchwork Tue May 2 12:09:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 89334 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp568056vqo; Tue, 2 May 2023 05:24:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7v4xhpEVm9rLGiyISmdcGH+isG6nY3kwLM6VR8GQ6HNh3ncNtN+hglOlhcw6i5HAwUwaqN X-Received: by 2002:a05:6a20:c892:b0:ef:ef3d:6166 with SMTP id hb18-20020a056a20c89200b000efef3d6166mr17394329pzb.32.1683030297140; Tue, 02 May 2023 05:24:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683030297; cv=none; d=google.com; s=arc-20160816; b=Krzan4T2SY+BVPWl7w2Vyd2arYpYI8TeIxTZzmwI8vkQkl/6e/lPoDL2LoZUG0/g0n tEq+k6ObId08w7i3dYuQd3MZ/GghIGaMiI3PL/WKAcoMp6+HR3CVUsNYbQJ5sWBIijip Nw5XL7up4qwxHaQ7rYzVk/V7lqf+4OFLsWkQis2Nu0EBCXa55Vv7bevs0TauE13s59yM rAqg463UpF1NlTI2lnN7lSUrcv8Ey/qffx6yMqASR0CDeFi8vzzDhRYLjPT74LNx7GxS aosf9dzzSn9+qhQUQkank0Bb2RU0GzL0+5US23yfVbY5DtE5flPKPoIsGwvfP5fV4YUy vosA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RMQn8Yiv0jCwQnecK3T9cC+8gstSs8Pnq7WDZyHBVNw=; b=jNZ0UEehYYPHqPktBeYdzi1d9zFbcnM/v2nAEHQYXHTbIBuREg/ye4VAWXW0jNLfSz 91P/PbPdzVyTFUEN+mOsMNxdqBV6UmGwGDOsjVsSSjdILwpt/G2O8AOII32OlCm0/mzG 5Ypb7Xqx1M9okgtWYyU1idQ0u9EJqY2vqzV4CwiHaOrG0uzfGb/T4lv72P5G4nBm2X2m ZC6Le8lN4hMKbq4VTJfbh1rBcjoUIoZdpoXWuvam29hMlB6HZKpDds2DXPEoeMclXJtM VS4gAwIhiPra8ZHmF+rhqGmtAtCdOFEjITQ8URNUtY/aVFhaDiwSs1rzrbZBsNQAlT30 l9TA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=UthapKSl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y63-20020a638a42000000b0052c44914d73si1390592pgd.34.2023.05.02.05.24.42; Tue, 02 May 2023 05:24:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=UthapKSl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234070AbjEBMKg (ORCPT + 99 others); Tue, 2 May 2023 08:10:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234038AbjEBMKc (ORCPT ); Tue, 2 May 2023 08:10:32 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 483D25FCE for ; Tue, 2 May 2023 05:10:12 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DFC711F8D6; Tue, 2 May 2023 12:10:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1683029410; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RMQn8Yiv0jCwQnecK3T9cC+8gstSs8Pnq7WDZyHBVNw=; b=UthapKSln5sVxc4P6W4BSa2qf1rvyEffS6HG2itfjchsaqorftKG30uqY4qHygD7foz2Sd qDX60C23glOf63Dt33mUz/OjzAGUAEkfM3DRKtIV4jLuqobQl4wBPaZynsRN7Is2PLb0tE yKXtXL2TX9qODXdqlLZNvDgVEnbQqZM= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 92321139C3; Tue, 2 May 2023 12:10:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id GmhQIqL9UGTMLwAAMHmgww (envelope-from ); Tue, 02 May 2023 12:10:10 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: mikelley@microsoft.com, Juergen Gross , Boris Ostrovsky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , xen-devel@lists.xenproject.org Subject: [PATCH v6 05/16] x86/xen: set MTRR state when running as Xen PV initial domain Date: Tue, 2 May 2023 14:09:20 +0200 Message-Id: <20230502120931.20719-6-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230502120931.20719-1-jgross@suse.com> References: <20230502120931.20719-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764785177193448892?= X-GMAIL-MSGID: =?utf-8?q?1764785177193448892?= When running as Xen PV initial domain (aka dom0), MTRRs are disabled by the hypervisor, but the system should nevertheless use correct cache memory types. This has always kind of worked, as disabled MTRRs resulted in disabled PAT, too, so that the kernel avoided code paths resulting in inconsistencies. This bypassed all of the sanity checks the kernel is doing with enabled MTRRs in order to avoid memory mappings with conflicting memory types. This has been changed recently, leading to PAT being accepted to be enabled, while MTRRs stayed disabled. The result is that mtrr_type_lookup() no longer is accepting all memory type requests, but started to return WB even if UC- was requested. This led to driver failures during initialization of some devices. In reality MTRRs are still in effect, but they are under complete control of the Xen hypervisor. It is possible, however, to retrieve the MTRR settings from the hypervisor. In order to fix those problems, overwrite the MTRR state via mtrr_overwrite_state() with the MTRR data from the hypervisor, if the system is running as a Xen dom0. Fixes: 72cbc8f04fe2 ("x86/PAT: Have pat_enabled() properly reflect state when running on Xen") Signed-off-by: Juergen Gross Reviewed-by: Boris Ostrovsky --- V2: - new patch V3: - move the call of mtrr_overwrite_state() to xen_pv_init_platform() V4: - only call mtrr_overwrite_state() if any MTRR got from Xen (Boris Ostrovsky) --- arch/x86/xen/enlighten_pv.c | 52 +++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 093b78c8bbec..8732b85d5650 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -68,6 +68,7 @@ #include #include #include +#include #include #include #include @@ -119,6 +120,54 @@ static int __init parse_xen_msr_safe(char *str) } early_param("xen_msr_safe", parse_xen_msr_safe); +/* Get MTRR settings from Xen and put them into mtrr_state. */ +static void __init xen_set_mtrr_data(void) +{ +#ifdef CONFIG_MTRR + struct xen_platform_op op = { + .cmd = XENPF_read_memtype, + .interface_version = XENPF_INTERFACE_VERSION, + }; + unsigned int reg; + unsigned long mask; + uint32_t eax, width; + static struct mtrr_var_range var[MTRR_MAX_VAR_RANGES] __initdata; + + /* Get physical address width (only 64-bit cpus supported). */ + width = 36; + eax = cpuid_eax(0x80000000); + if ((eax >> 16) == 0x8000 && eax >= 0x80000008) { + eax = cpuid_eax(0x80000008); + width = eax & 0xff; + } + + for (reg = 0; reg < MTRR_MAX_VAR_RANGES; reg++) { + op.u.read_memtype.reg = reg; + if (HYPERVISOR_platform_op(&op)) + break; + + /* + * Only called in dom0, which has all RAM PFNs mapped at + * RAM MFNs, and all PCI space etc. is identity mapped. + * This means we can treat MFN == PFN regarding MTRR settings. + */ + var[reg].base_lo = op.u.read_memtype.type; + var[reg].base_lo |= op.u.read_memtype.mfn << PAGE_SHIFT; + var[reg].base_hi = op.u.read_memtype.mfn >> (32 - PAGE_SHIFT); + mask = ~((op.u.read_memtype.nr_mfns << PAGE_SHIFT) - 1); + mask &= (1UL << width) - 1; + if (mask) + mask |= MTRR_PHYSMASK_V; + var[reg].mask_lo = mask; + var[reg].mask_hi = mask >> 32; + } + + /* Only overwrite MTRR state if any MTRR could be got from Xen. */ + if (reg) + mtrr_overwrite_state(var, reg, MTRR_TYPE_UNCACHABLE); +#endif +} + static void __init xen_pv_init_platform(void) { /* PV guests can't operate virtio devices without grants. */ @@ -135,6 +184,9 @@ static void __init xen_pv_init_platform(void) /* pvclock is in shared info area */ xen_init_time_ops(); + + if (xen_initial_domain()) + xen_set_mtrr_data(); } static void __init xen_pv_guest_late_init(void)