From patchwork Mon Mar 6 16:34:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64956 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1953230wrd; Mon, 6 Mar 2023 09:00:21 -0800 (PST) X-Google-Smtp-Source: AK7set8dUdGBF3y6pQm8LR35teVf+ZcrGzBK/kFEi0rNU4lWapxDMssNbifSCeBhtK9bSvTtVRYB X-Received: by 2002:a17:903:22d2:b0:19c:d32a:befc with SMTP id y18-20020a17090322d200b0019cd32abefcmr12795706plg.15.1678122021009; Mon, 06 Mar 2023 09:00:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678122020; cv=none; d=google.com; s=arc-20160816; b=EA2xkxZFJHbD6GUs/j+OSXx8P5P+deYlOGIGLmpsF4G5O3fNA7H87n2Coo2bC5a4UM 3bEzGyIpgTDJiNkU31sQ/JLi/XX+lkRoT/lW7mXNinO16BCX+IFe7220VkLbMjXkCYrS wg5F/BXY43B7NwUg/ATFbA6U9TA4z8OaEXeoGhqRQrgW7bK2MzBzg0ZXYO1M5d54gz9C MJ/BEnr2j4ilXPI7FwF4deHtToezVa10zFatuEvuMLIUIRVSidxphx7ZIOFanj6OVEKa TKrSpYkeTzyx2Nc5Qd7awe1mZBmZ+6AUcd3OGmJ8Cdo15v7/mgPYHRtywKw1BHFV34NR RLow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wdZqXElp1B5EQPu6xSmwn38A5NK5TNGdQEx7TILYZLE=; b=gEyTKbvmQaMyEm1thRLGoXIc2r7ZODAcgItuHIqjmBUxyMm6zhfizb2qovagOmHb6x xJrI25DG9eXz64sz4Z55AJuS8xT8qstY75JQUHKwJWyWDKHraLnrBqaySdMvopsy4cpY 4jRKxuh9o6yyDmGWaDC4fpPncqICHSpW2xJNLNnwoU/KvOKxXDvgUz04ZyZiwZ2VvTJr FjV7zGo1iLzil41byBxocPKFlD+JQ3hqGVaUxT7FkxCsJ9YCzX9kFX7dDvtEyKdUpJXZ dq8DEB4ELmCTWZ/CMMj54kmFjEb/5ZW8Fi/mX4kqd4YdL6OCeKWfIdLExbHLWCNmkz8u nh8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=UHmhOeQR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kk16-20020a170903071000b0019ca3bea326si1681004plb.269.2023.03.06.09.00.07; Mon, 06 Mar 2023 09:00:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=UHmhOeQR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230241AbjCFQgo (ORCPT + 99 others); Mon, 6 Mar 2023 11:36:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48268 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229930AbjCFQfx (ORCPT ); Mon, 6 Mar 2023 11:35:53 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C1D739B8B for ; Mon, 6 Mar 2023 08:35:19 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 694DC21C6C; Mon, 6 Mar 2023 16:34:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120473; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wdZqXElp1B5EQPu6xSmwn38A5NK5TNGdQEx7TILYZLE=; b=UHmhOeQRK6Sy++tBvGVXthjYv7WqNFLqCX4CMLPwCve0KaCx8Wz1XB09BQlkb9YMNU7Jp3 th5A0ZTqwyvMf5kwHvBzhFpHuAuyDSbOKSJGdH9sNHPJaHKxKsslYoI+nxbCB+ZY/pAXKa /nnJrpGGPl4SVVwagBnyKD+PPbRyofk= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2C1D013A66; Mon, 6 Mar 2023 16:34:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id LV5sCRkWBmQbUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:34:33 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 01/12] x86/mtrr: split off physical address size calculation Date: Mon, 6 Mar 2023 17:34:14 +0100 Message-Id: <20230306163425.8324-2-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638476539101148?= X-GMAIL-MSGID: =?utf-8?q?1759638476539101148?= Move the calculation of the physical address size in mtrr_bp_init() into a helper function. This will be needed later. Do only the pure code movement without optimizing it. Signed-off-by: Juergen Gross --- V2: - new patch V3: - only move code, split off optimizations (Boris Petkov) --- arch/x86/kernel/cpu/mtrr/mtrr.c | 57 ++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtrr.c index 783f3210d582..8310bdb111d0 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -620,22 +620,14 @@ static struct syscore_ops mtrr_syscore_ops = { int __initdata changed_by_mtrr_cleanup; #define SIZE_OR_MASK_BITS(n) (~((1ULL << ((n) - PAGE_SHIFT)) - 1)) -/** - * mtrr_bp_init - initialize mtrrs on the boot CPU - * - * This needs to be called early; before any of the other CPUs are - * initialized (i.e. before smp_init()). - * - */ -void __init mtrr_bp_init(void) + +static unsigned int __init mtrr_calc_physbits(bool generic) { - const char *why = "(not available)"; - u32 phys_addr; + unsigned int phys_addr; phys_addr = 32; - if (boot_cpu_has(X86_FEATURE_MTRR)) { - mtrr_if = &generic_mtrr_ops; + if (generic) { size_or_mask = SIZE_OR_MASK_BITS(36); size_and_mask = 0x00f00000; phys_addr = 36; @@ -667,29 +659,44 @@ void __init mtrr_bp_init(void) size_and_mask = 0; phys_addr = 32; } + } else { + size_or_mask = SIZE_OR_MASK_BITS(32); + size_and_mask = 0; + } + + return phys_addr; +} + +/** + * mtrr_bp_init - initialize mtrrs on the boot CPU + * + * This needs to be called early; before any of the other CPUs are + * initialized (i.e. before smp_init()). + * + */ +void __init mtrr_bp_init(void) +{ + const char *why = "(not available)"; + unsigned int phys_addr; + + phys_addr = mtrr_calc_physbits(boot_cpu_has(X86_FEATURE_MTRR)); + + if (boot_cpu_has(X86_FEATURE_MTRR)) { + mtrr_if = &generic_mtrr_ops; } else { switch (boot_cpu_data.x86_vendor) { case X86_VENDOR_AMD: - if (cpu_feature_enabled(X86_FEATURE_K6_MTRR)) { - /* Pre-Athlon (K6) AMD CPU MTRRs */ + /* Pre-Athlon (K6) AMD CPU MTRRs */ + if (cpu_feature_enabled(X86_FEATURE_K6_MTRR)) mtrr_if = &amd_mtrr_ops; - size_or_mask = SIZE_OR_MASK_BITS(32); - size_and_mask = 0; - } break; case X86_VENDOR_CENTAUR: - if (cpu_feature_enabled(X86_FEATURE_CENTAUR_MCR)) { + if (cpu_feature_enabled(X86_FEATURE_CENTAUR_MCR)) mtrr_if = ¢aur_mtrr_ops; - size_or_mask = SIZE_OR_MASK_BITS(32); - size_and_mask = 0; - } break; case X86_VENDOR_CYRIX: - if (cpu_feature_enabled(X86_FEATURE_CYRIX_ARR)) { + if (cpu_feature_enabled(X86_FEATURE_CYRIX_ARR)) mtrr_if = &cyrix_mtrr_ops; - size_or_mask = SIZE_OR_MASK_BITS(32); - size_and_mask = 0; - } break; default: break; From patchwork Mon Mar 6 16:34:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64932 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1942817wrd; Mon, 6 Mar 2023 08:38:49 -0800 (PST) X-Google-Smtp-Source: AK7set/eU/5ktDM5Y0kv/GK0fJI71xqkvXwBTj99FlZUl/BOqNUmyIi4GgpZn889S1C1lm6I8cLd X-Received: by 2002:a05:6a20:3b85:b0:cc:fced:f715 with SMTP id b5-20020a056a203b8500b000ccfcedf715mr9866493pzh.22.1678120729057; Mon, 06 Mar 2023 08:38:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678120729; cv=none; d=google.com; s=arc-20160816; b=k/p6brdg6R8HS00z5Xa9DFfyK6mhpNWBVglH5p3349BnQjpA0ETnYfxl7GOHwtEE9c 55PR8OgtJ2i0lCzmVz44aSALDMkWjiJxj3kbiSZNj4/rpl95O14Nv2Co9jQHEWi8y4Ys JRxQhK9xmR7qVnxeutMlbwz2Dwa4eQqloFY6a8oUVbeCGWfM/1WyfkBcEh16RqpSs8l3 0f+CRnJLSLLp4bsCmcBt+70pGvMCoZSF6IEyIlyMdDjQNlXBN+CEgk6vMbm5HRMeJXRs 6pBuHNVTnY0oT3fe6pZoAjCfVoHqYfJyYqjSMVZC994saueA77dl5oODvJ5WxPnwRZZQ Frug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0JsZ/VnHkJzmuDE7gMbHKVhyFu+vf+6Nmu9WRKJMi3s=; b=x8JbELEthuRFYZ4bH9MqPFcwzsYxUZYU4QK928F7JIfrRU4S/ZX66PxYmTlGG0c0He IvfLmMVHg9LSYwXbWBEAQMeQtfaF/fgqBYLgaHwk6zARN1Wb3okeJSdyoFLo0oOnav00 20xb1LtJM+R51j//wNByOjRFqJL68a+9Tnkwcl2NmWqBO6eSXbONzoIP8YoAnZELbQko 0DNRU/8FEfG3zHnVQpr1/lG7myvjLq+kDJAAjQR97/4/+NJitGmg1y4HLrItH3wKrcGB aN92FOv6TdI4/F+C/zDwVNN6xKe6k3ZNVNjiQpQmIZWtJRrSHOY/aIdcWebNW1EdNH1j /MPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=FDrOH3Oo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 17-20020a630011000000b004fc274006d9si9631668pga.368.2023.03.06.08.38.35; Mon, 06 Mar 2023 08:38:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=FDrOH3Oo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230334AbjCFQgw (ORCPT + 99 others); Mon, 6 Mar 2023 11:36:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230007AbjCFQf4 (ORCPT ); Mon, 6 Mar 2023 11:35:56 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 632093B202 for ; Mon, 6 Mar 2023 08:35:21 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1778F1FDE4; Mon, 6 Mar 2023 16:34:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120479; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0JsZ/VnHkJzmuDE7gMbHKVhyFu+vf+6Nmu9WRKJMi3s=; b=FDrOH3OoQ6bIZecckFCAR+JTM5Ks/ekJZv6L41zO46FDqLHp32JNrBzlEuoz5EiibkZ3jQ NB8VXAbRlIRVaz/Go6GkYh2eMc5emfg3/a0/PnMKxdFR86myCbVYjKLbnNsU/gCMM/Zqbd xUW8WXTqZWk3lSUYNqL7DGsBJZakcl8= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id CF25613A66; Mon, 6 Mar 2023 16:34:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id TJhGMR4WBmQpUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:34:38 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 02/12] x86/mtrr: optimize mtrr_calc_physbits() Date: Mon, 6 Mar 2023 17:34:15 +0100 Message-Id: <20230306163425.8324-3-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759637121538244347?= X-GMAIL-MSGID: =?utf-8?q?1759637121538244347?= Optimize mtrr_calc_physbits() for better readability. Drop a stale comment, as reality has made it obsolete. Signed-off-by: Juergen Gross Signed-off-by: Juergen Gross Signed-off-by: Borislav Petkov (AMD) --- V3: - new patch, split off from previous patch (Boris Petkov) --- arch/x86/kernel/cpu/mtrr/mtrr.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtrr.c index 8310bdb111d0..7596ebeab929 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -619,8 +619,6 @@ static struct syscore_ops mtrr_syscore_ops = { int __initdata changed_by_mtrr_cleanup; -#define SIZE_OR_MASK_BITS(n) (~((1ULL << ((n) - PAGE_SHIFT)) - 1)) - static unsigned int __init mtrr_calc_physbits(bool generic) { unsigned int phys_addr; @@ -628,15 +626,8 @@ static unsigned int __init mtrr_calc_physbits(bool generic) phys_addr = 32; if (generic) { - size_or_mask = SIZE_OR_MASK_BITS(36); - size_and_mask = 0x00f00000; phys_addr = 36; - /* - * This is an AMD specific MSR, but we assume(hope?) that - * Intel will implement it too when they extend the address - * bus of the Xeon. - */ if (cpuid_eax(0x80000000) >= 0x80000008) { phys_addr = cpuid_eax(0x80000008) & 0xff; /* CPUID workaround for Intel 0F33/0F34 CPU */ @@ -647,23 +638,19 @@ static unsigned int __init mtrr_calc_physbits(bool generic) boot_cpu_data.x86_stepping == 0x4)) phys_addr = 36; - size_or_mask = SIZE_OR_MASK_BITS(phys_addr); - size_and_mask = ~size_or_mask & 0xfffff00000ULL; } else if (boot_cpu_data.x86_vendor == X86_VENDOR_CENTAUR && boot_cpu_data.x86 == 6) { /* * VIA C* family have Intel style MTRRs, * but don't support PAE */ - size_or_mask = SIZE_OR_MASK_BITS(32); - size_and_mask = 0; phys_addr = 32; } - } else { - size_or_mask = SIZE_OR_MASK_BITS(32); - size_and_mask = 0; } + size_or_mask = ~((1ULL << (phys_addr - PAGE_SHIFT)) - 1); + size_and_mask = ~size_or_mask & 0xfffff00000ULL; + return phys_addr; } From patchwork Mon Mar 6 16:34:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64934 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1942883wrd; Mon, 6 Mar 2023 08:38:55 -0800 (PST) X-Google-Smtp-Source: AK7set+C5BpArLIaFOyz86XP5U4MTZQVRyokFneD3ddARGHoCeQI4JCkzHrQ4TMytGjD/wovJBkI X-Received: by 2002:a17:903:11cf:b0:19e:6b50:e220 with SMTP id q15-20020a17090311cf00b0019e6b50e220mr175780plh.53.1678120735126; Mon, 06 Mar 2023 08:38:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678120735; cv=none; d=google.com; s=arc-20160816; b=NicY89vJYAYIE3SFjSkQhrFXt2WkWpNWjXd0kUJTwU0IQ+OfSvWId7Yz4OygfQDHUT sXu2TzGByrayi0ReCyhdKoTGEcns9z0Zc9I4Qz+uDqH+DetlMpLbvzTMIktZRUwJsT3G majkVmH1dttGi/6w9Q0vC2s4SQOuCptVhLVJG3Vu7uMfFeuxUUALjwG+lHQB7A1TBIGF EGMJOl05lMnF1D+TP8L0QkIX0FpbJEKt+EHkusZoHiPuikPq7h+m1E7LAtu/sf2AGzoB xbRUaS4tYtQhyPZRIgjmNYKKHJCYMrMLt/q4TszmFASmEE+s4VrQilSIrNV2GOa5jcNN LbXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mQIwzGwKV8auzcLGhE1h+Ttf3dgEINxZJAfs4KKmG8o=; b=MriJW3yOu9Bov5Nt4w+VOCGD/nH4ZQWxqrpqht9MEtHRJE+92f7AJWCwc+c1HFpymP UPWwZbXd0Gksg9/a/B4ESN4nNRbcmPyfsmWI0tZx5A1/u0Oda22YeHKVuejSj/QE3wJF oY3zluHOdkM6CtM+9rMnIqlRqUq55wTyhSgdE+whQxgElkp3YFsjzlcLlvzLCFcxJm+y vN9qTvLDpBa6JiZbIYkPTpOQkRuwLh88PW68E+NLzJ03eFZyRyzfIGJAMrdcFjo6rs25 19pLix6DTQPx3LAZGwBGp4Rl2vtFMiWLCteh9r6MWhDX5Zvbmxl0yrv30dR9/STdtsCk yLKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=lstrsomy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bj5-20020a170902850500b0019ca7a76e47si9286769plb.141.2023.03.06.08.38.40; Mon, 06 Mar 2023 08:38:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=lstrsomy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230376AbjCFQgz (ORCPT + 99 others); Mon, 6 Mar 2023 11:36:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230211AbjCFQgG (ORCPT ); Mon, 6 Mar 2023 11:36:06 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB30242BD9 for ; Mon, 6 Mar 2023 08:35:29 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AEEB01FDE5; Mon, 6 Mar 2023 16:34:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120484; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mQIwzGwKV8auzcLGhE1h+Ttf3dgEINxZJAfs4KKmG8o=; b=lstrsomyfTeRwpQM78KPNKAwcec5755jNucEwcWtQmzFrgh5sgfsT8yRNpGYrDsAGSe2T9 5aXuR0fQ2EH8f0gHCn1QIhiogy9S3plHmqbnuOKmlU+H3L+8NZaYaUyQybDtJ9cZgCo/fs /ZNxnmdBFtMH8CgiohSEGtRvniUszWk= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 72F0D13A66; Mon, 6 Mar 2023 16:34:44 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id brHMGiQWBmQzUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:34:44 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 03/12] x86/mtrr: support setting MTRR state for software defined MTRRs Date: Mon, 6 Mar 2023 17:34:16 +0100 Message-Id: <20230306163425.8324-4-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759637127796418401?= X-GMAIL-MSGID: =?utf-8?q?1759637127796418401?= When running virtualized, MTRR access can be reduced (e.g. in Xen PV guests or when running as a SEV-SNP guest under Hyper-V). Typically the hypervisor will reset the MTRR feature in CPUID data, resulting in no MTRR memory type information being available for the kernel. This has turned out to result in problems: - Hyper-V SEV-SNP guests using uncached mappings where they shouldn't - Xen PV dom0 mapping memory as WB which should be UC- instead Solve those problems by supporting to set a static MTRR state, overwriting the empty state used today. In case such a state has been set, don't call get_mtrr_state() in mtrr_bp_init(). The set state will only be used by mtrr_type_lookup(), as in all other cases mtrr_enabled() is being checked, which will return false. Accept the overwrite call only for selected cases when running as a guest. Disable X86_FEATURE_MTRR in order to avoid any MTRR modifications by just refusing them. Signed-off-by: Juergen Gross --- V2: - new patch V3: - omit fixed MTRRs, as those are currently not needed - disable X86_FEATURE_MTRR instead of testing it - provide a stub for !CONFIG_MTRR (Michael Kelley) - use cpu_feature_enabled() (Boris Petkov) - add tests for mtrr_overwrite_state() being allowed (Boris Petkov) V4: - add test for hv_is_isolation_supported() (Michael Kelley) --- arch/x86/include/asm/mtrr.h | 8 ++++++ arch/x86/kernel/cpu/mtrr/generic.c | 46 +++++++++++++++++++++++++++++- arch/x86/kernel/cpu/mtrr/mtrr.c | 9 ++++++ arch/x86/kernel/setup.c | 2 ++ 4 files changed, 64 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index f0eeaf6e5f5f..f1cb81330a64 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -31,6 +31,8 @@ */ # ifdef CONFIG_MTRR void mtrr_bp_init(void); +void mtrr_overwrite_state(struct mtrr_var_range *var, unsigned int num_var, + mtrr_type def_type); extern u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform); extern void mtrr_save_fixed_ranges(void *); extern void mtrr_save_state(void); @@ -48,6 +50,12 @@ void mtrr_disable(void); void mtrr_enable(void); void mtrr_generic_set_state(void); # else +static inline void mtrr_overwrite_state(struct mtrr_var_range *var, + unsigned int num_var, + mtrr_type def_type) +{ +} + static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform) { /* diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index ee09d359e08f..49b4cc923312 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -8,10 +8,12 @@ #include #include #include - +#include #include #include #include +#include +#include #include #include #include @@ -240,6 +242,48 @@ static u8 mtrr_type_lookup_variable(u64 start, u64 end, u64 *partial_end, return mtrr_state.def_type; } +/** + * mtrr_overwrite_state - set static MTRR state + * + * Used to set MTRR state via different means (e.g. with data obtained from + * a hypervisor). + * Is allowed only for special cases when running virtualized. Must be called + * from the x86_init.hyper.init_platform() hook. X86_FEATURE_MTRR must be off. + */ +void mtrr_overwrite_state(struct mtrr_var_range *var, unsigned int num_var, + mtrr_type def_type) +{ + unsigned int i; + + if (WARN_ON(mtrr_state_set || + hypervisor_is_type(X86_HYPER_NATIVE) || + !cpu_feature_enabled(X86_FEATURE_HYPERVISOR) || + (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP) && + !hv_is_isolation_supported() && + !cpu_feature_enabled(X86_FEATURE_XENPV) && + !cpu_feature_enabled(X86_FEATURE_TDX_GUEST)))) + return; + + /* Disable MTRR in order to disable MTRR modifications. */ + setup_clear_cpu_cap(X86_FEATURE_MTRR); + + if (var) { + if (num_var > MTRR_MAX_VAR_RANGES) { + pr_warn("Trying to overwrite MTRR state with %u variable entries\n", + num_var); + num_var = MTRR_MAX_VAR_RANGES; + } + for (i = 0; i < num_var; i++) + mtrr_state.var_ranges[i] = var[i]; + num_var_ranges = num_var; + } + + mtrr_state.def_type = def_type; + mtrr_state.enabled |= MTRR_STATE_MTRR_ENABLED; + + mtrr_state_set = 1; +} + /** * mtrr_type_lookup - look up memory type in MTRR * diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtrr.c index 7596ebeab929..5fe62ee0361b 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -666,6 +666,15 @@ void __init mtrr_bp_init(void) const char *why = "(not available)"; unsigned int phys_addr; + if (mtrr_state.enabled) { + /* Software overwrite of MTRR state, only for generic case. */ + mtrr_calc_physbits(true); + init_table(); + pr_info("MTRRs set to read-only\n"); + + return; + } + phys_addr = mtrr_calc_physbits(boot_cpu_has(X86_FEATURE_MTRR)); if (boot_cpu_has(X86_FEATURE_MTRR)) { diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 16babff771bd..0cccfeb67c3a 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1037,6 +1037,8 @@ void __init setup_arch(char **cmdline_p) /* * VMware detection requires dmi to be available, so this * needs to be done after dmi_setup(), for the boot CPU. + * For some guest types (Xen PV, SEV-SNP, TDX) it is required to be + * called before cache_bp_init() for setting up MTRR state. */ init_hypervisor_platform(); From patchwork Mon Mar 6 16:34:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64963 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1954723wrd; Mon, 6 Mar 2023 09:02:18 -0800 (PST) X-Google-Smtp-Source: AK7set8EQ9RIjnOixkct4nbPFIQyUV4Qs0ZNUHF2ZPVZmnzfHUa60sNEUlggl8J4maktAQioRsV/ X-Received: by 2002:aa7:c98a:0:b0:4ac:d2b4:ec2c with SMTP id c10-20020aa7c98a000000b004acd2b4ec2cmr8932138edt.29.1678122137978; Mon, 06 Mar 2023 09:02:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678122137; cv=none; d=google.com; s=arc-20160816; b=yslt9uXrtTUfLXVsNIXEik5lecRFeS/veApOYCr9ZlTTmnDhK+RAI+2kpj/s5PDO3T L3L78mhDSwbCXGxkT2jWE8ss3QXLPeNUlfUO6gq9mS9ds9I3FC+QcvvRPlDmO1yZeNFc 9mIYm0HHi4/NnRtvuHvyrJQAxk2XvxiE7+dZE43HxSqA9BTFVx5Zk0dVyVjaeV+jN/jF GcwdqkBxf2ky3jaoML/ONr7qj/tMxjscE9wlVXYmtYBUdgKScoOey0WeZ6mWp99jHdrm RJjKL6zFmvlx+eqlrFMY5vzTp9NQbiQtFNn1yEhZJPBfxVPOs7DhN21qEnwcrFOmQGnO BCoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EEyy0YGNrjiScFEJSCN9XqA2TAW4Ejl4yGBfINYDlUs=; b=Pi5poXcDx1PlqvoqpInW9Cv0UTpZYcvqJ2VWoD8iLzzEvDTSn0VC9DNfP37IWJlB/x L9C3e3Rlmg5Yp2nHj4n2oXFRhKngwAGzUKl4WyAXjoTiR4GiOdlnmRWTGRzLel206wZy IoXSmbIQ0oeTMkKeprYaGKXoo+WVYWQo72M7UZxhMw/ptUQ0k+yMpBQR7Al2wdNUyDeC +JHEwVOdwy/VOWVv8Yu9zuBFLnpLRaP+m2Sbgz54tQfcYRW0/TWiN63wImP4lSZH//M4 RdO2LBI7BDbjFz8eEaAhKbQlGiLPDBvYP3UF9ld58grTypwe0rIvOV8cawkX4l7KBk+R kq+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=hykKurK9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g6-20020aa7c586000000b004ab250bceecsi7238890edq.654.2023.03.06.09.01.53; Mon, 06 Mar 2023 09:02:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=hykKurK9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229843AbjCFQg7 (ORCPT + 99 others); Mon, 6 Mar 2023 11:36:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230287AbjCFQgL (ORCPT ); Mon, 6 Mar 2023 11:36:11 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53B2F4390C; Mon, 6 Mar 2023 08:35:37 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 7789921E5D; Mon, 6 Mar 2023 16:34:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120490; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EEyy0YGNrjiScFEJSCN9XqA2TAW4Ejl4yGBfINYDlUs=; b=hykKurK9/WXS6qISoyLDcDHoUPTa6DT2LVEsW+HX1QsXJ0G5yeT0pc2YbmUapmb9VennA3 yx5uPl6pHjSN+PR7U5sxC+wj8Z/kbztWb3NkzdcLlXeUJXYTivVOxlBOrfZl+IdV536VzX e3PDiwTRa+CSvX3UOk/PJo5Y3opyfRA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2756B13A66; Mon, 6 Mar 2023 16:34:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yD1zCCoWBmREUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:34:50 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-hyperv@vger.kernel.org Cc: Juergen Gross , "K. Y. Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 04/12] x86/hyperv: set MTRR state when running as SEV-SNP Hyper-V guest Date: Mon, 6 Mar 2023 17:34:17 +0100 Message-Id: <20230306163425.8324-5-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638598626976256?= X-GMAIL-MSGID: =?utf-8?q?1759638598626976256?= In order to avoid mappings using the UC- cache attribute, set the MTRR state to use WB caching as the default. This is needed in order to cope with the fact that PAT is enabled, while MTRRs are not supported by the hypervisor. Fixes: 90b926e68f50 ("x86/pat: Fix pat_x_mtrr_type() for MTRR disabled case") Signed-off-by: Juergen Gross --- V2: - new patch --- arch/x86/kernel/cpu/mshyperv.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f36dc2f796c5..0a6cc3cf8919 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -34,6 +34,7 @@ #include #include #include +#include /* Is Linux running as the root partition? */ bool hv_root_partition; @@ -408,6 +409,9 @@ static void __init ms_hyperv_init_platform(void) #ifdef CONFIG_SWIOTLB swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary; #endif + + /* Set WB as the default cache mode. */ + mtrr_overwrite_state(NULL, 0, MTRR_TYPE_WRBACK); } /* Isolation VMs are unenlightened SEV-based VMs, thus this check: */ if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { From patchwork Mon Mar 6 16:34:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64970 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1955513wrd; Mon, 6 Mar 2023 09:03:20 -0800 (PST) X-Google-Smtp-Source: AK7set8DHGnFlLshalNgfSESaSPGEoVaGYgKfwoTUUJlPjWZ6AD/lAI9D77QZDgJNBHbfTG53eUK X-Received: by 2002:aa7:dd06:0:b0:4aa:a4f1:3edb with SMTP id i6-20020aa7dd06000000b004aaa4f13edbmr10142958edv.29.1678122199868; Mon, 06 Mar 2023 09:03:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678122199; cv=none; d=google.com; s=arc-20160816; b=DUDGkQvPZpcDQMYvhZLLLfB9nDpC3cdGMvF4Vu8xbQkddh2g2u5kcklGkaTQjjkjzH Dsnysdx4KX0L9yP0qyUhf2ZqoNoQ0f5JpKvb3yWH3/8kmnKw5IT2wlyt8kDr2f2ovsCa yD8GY54QlwATPnIarz3eiiVEyk2DUV7GVgMC4w6Exnzv6z/UAfv8iw90ZC1o1GNjMJu7 53iw6zZLE3QSKuE7Bcy2ZQtshNMbOASUCdvni3M3LUAd1+vv4Sm8brv8sqs6pP/YlrDq aIW17sQigbei1NteUOoHVmG1vP+LawSFh56yDMcjAqMz/YMyZFMA/clATC0D8MrPgUWn Zhkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QeDAQjSRqeP0iwOfsSlKaCLfsyacyLvFFynhKMg6itw=; b=eNWmJVoPZmOQx0T/G4bbuUTF9wu+bdJw9YhY90AyTXfjCZF+Cpa6t0lH8eCR9BVsik yT+6yYqVk3RP8cevivGwU2ek9CM3/7cHVbF35aK6I5DzcRbZ1YEHwv6mhMyeaigC+1wR nfZcFB63QxYYUIFzjVcGsWGx/xdWl0HQ6BgMUiC4Jc+/QTNolqATsCQ+05HZWh2ZD0wV YiPM9ZKoiOHFwD+nGdbJ3DXcb7/BHX0ULpzBEr+lPpjdYilPfHCtlhVD+o4lGTJ6xNM4 S3weQ3AK1Mjt5sIfq8+S8jAVRgHvLvMiNb02y10lUfRPbI1INltBNUo/QKJg15YaFqyR NtHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=L0ORxZJR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id xa5-20020a170906fd8500b008e1cbdcd3ccsi1013933ejb.27.2023.03.06.09.02.55; Mon, 06 Mar 2023 09:03:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=L0ORxZJR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230120AbjCFQgh (ORCPT + 99 others); Mon, 6 Mar 2023 11:36:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230126AbjCFQfu (ORCPT ); Mon, 6 Mar 2023 11:35:50 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F8E13D92B for ; Mon, 6 Mar 2023 08:35:18 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2139B21E60; Mon, 6 Mar 2023 16:34:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120496; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QeDAQjSRqeP0iwOfsSlKaCLfsyacyLvFFynhKMg6itw=; b=L0ORxZJRMkPybl9qj+RAorA80nbejKg5XvHMdSpdYEzrYn4KgSUMFPaxfEgYjAJI2bDKF6 mzHQ5m87Ic6c/BgUzhbf/mMcZy2OPaG9/RLe5a2OW4fb8jiZZ/45mtEDztLmim1NkEWYVx QUxyHCFID5ix33QJnlXAQ1ezxKn5AN8= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D4D4813A66; Mon, 6 Mar 2023 16:34:55 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id PkrGMi8WBmRWUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:34:55 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Boris Ostrovsky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , xen-devel@lists.xenproject.org Subject: [PATCH v4 05/12] x86/xen: set MTRR state when running as Xen PV initial domain Date: Mon, 6 Mar 2023 17:34:18 +0100 Message-Id: <20230306163425.8324-6-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638663558623816?= X-GMAIL-MSGID: =?utf-8?q?1759638663558623816?= When running as Xen PV initial domain (aka dom0), MTRRs are disabled by the hypervisor, but the system should nevertheless use correct cache memory types. This has always kind of worked, as disabled MTRRs resulted in disabled PAT, too, so that the kernel avoided code paths resulting in inconsistencies. This bypassed all of the sanity checks the kernel is doing with enabled MTRRs in order to avoid memory mappings with conflicting memory types. This has been changed recently, leading to PAT being accepted to be enabled, while MTRRs stayed disabled. The result is that mtrr_type_lookup() no longer is accepting all memory type requests, but started to return WB even if UC- was requested. This led to driver failures during initialization of some devices. In reality MTRRs are still in effect, but they are under complete control of the Xen hypervisor. It is possible, however, to retrieve the MTRR settings from the hypervisor. In order to fix those problems, overwrite the MTRR state via mtrr_overwrite_state() with the MTRR data from the hypervisor, if the system is running as a Xen dom0. Fixes: 72cbc8f04fe2 ("x86/PAT: Have pat_enabled() properly reflect state when running on Xen") Signed-off-by: Juergen Gross Reviewed-by: Boris Ostrovsky --- V2: - new patch V3: - move the call of mtrr_overwrite_state() to xen_pv_init_platform() V4: - only call mtrr_overwrite_state() if any MTRR got from Xen (Boris Ostrovsky) --- arch/x86/xen/enlighten_pv.c | 52 +++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index bb59cc6ddb2d..12e6b6845870 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -68,6 +68,7 @@ #include #include #include +#include #include #include #include @@ -119,6 +120,54 @@ static int __init parse_xen_msr_safe(char *str) } early_param("xen_msr_safe", parse_xen_msr_safe); +/* Get MTRR settings from Xen and put them into mtrr_state. */ +static void __init xen_set_mtrr_data(void) +{ +#ifdef CONFIG_MTRR + struct xen_platform_op op = { + .cmd = XENPF_read_memtype, + .interface_version = XENPF_INTERFACE_VERSION, + }; + unsigned int reg; + unsigned long mask; + uint32_t eax, width; + static struct mtrr_var_range var[MTRR_MAX_VAR_RANGES] __initdata; + + /* Get physical address width (only 64-bit cpus supported). */ + width = 36; + eax = cpuid_eax(0x80000000); + if ((eax >> 16) == 0x8000 && eax >= 0x80000008) { + eax = cpuid_eax(0x80000008); + width = eax & 0xff; + } + + for (reg = 0; reg < MTRR_MAX_VAR_RANGES; reg++) { + op.u.read_memtype.reg = reg; + if (HYPERVISOR_platform_op(&op)) + break; + + /* + * Only called in dom0, which has all RAM PFNs mapped at + * RAM MFNs, and all PCI space etc. is identity mapped. + * This means we can treat MFN == PFN regarding MTTR settings. + */ + var[reg].base_lo = op.u.read_memtype.type; + var[reg].base_lo |= op.u.read_memtype.mfn << PAGE_SHIFT; + var[reg].base_hi = op.u.read_memtype.mfn >> (32 - PAGE_SHIFT); + mask = ~((op.u.read_memtype.nr_mfns << PAGE_SHIFT) - 1); + mask &= (1UL << width) - 1; + if (mask) + mask |= 1 << 11; + var[reg].mask_lo = mask; + var[reg].mask_hi = mask >> 32; + } + + /* Only overwrite MTRR state if any MTRR could be got from Xen. */ + if (reg) + mtrr_overwrite_state(var, reg, MTRR_TYPE_UNCACHABLE); +#endif +} + static void __init xen_pv_init_platform(void) { /* PV guests can't operate virtio devices without grants. */ @@ -135,6 +184,9 @@ static void __init xen_pv_init_platform(void) /* pvclock is in shared info area */ xen_init_time_ops(); + + if (xen_initial_domain()) + xen_set_mtrr_data(); } static void __init xen_pv_guest_late_init(void) From patchwork Mon Mar 6 16:34:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64937 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1943239wrd; Mon, 6 Mar 2023 08:39:36 -0800 (PST) X-Google-Smtp-Source: AK7set9qzPCSZhliisBXQi+adYX33j6SRQ0GmgUzPwruDHAh6L3s9zPeaXD0TtpwCJW3pJcLGseS X-Received: by 2002:a05:6a20:728b:b0:b8:5881:5c3b with SMTP id o11-20020a056a20728b00b000b858815c3bmr12192959pzk.58.1678120776618; Mon, 06 Mar 2023 08:39:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678120776; cv=none; d=google.com; s=arc-20160816; b=Ld3/6R6q4HYG18QanT+sO80o19h9GfwUZH/P9quA0/SBtfLLU9eoBk+7/mnzcJcEOT GqsWZObs9JOAg6aPoN0Adc0NZwCxUPpl2Vh7DlaxxUXh1xmj40Mxt4043SyrcMe9hpw3 iK14+qL2zdnwHhXijt/NqMk4x7Wo3Q5J/AKHIrJsaeVyonfCqLn3X7Wlbq7iRSxZ9cML gEEb1upkuXmQpx1Sfa1fXoaBlTgb5/I0KaXaU6fZnskICLwx/0ytEKQuEJVb6q2gvh2z 2JWIWgNzLkhPHz5TTmu9RbYOcLbC0vHlF2prnkBg57X2IGoYngGWMlPYj3D2X2h/tWfL 3LWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PG3Fi763G1leTyqQb5znTW1NqQX2oxdmW3GvkBcAAnI=; b=g2tC+hBPfVyh6nM/GjOeXPRaRMldadN5Lc24x6SDc9h/vvzpTpqWBYiajvnO2Pl12x kVcMODI6W5k6iaf++31tKh/Hc4cJgDCwvFsCQz/Y+nhxf2pSJKN9aKYfWi4/T6KHS+wx kle8CSEbP2dl7HXJr5pL2NSTCLpmIc7AUPXcM82Qdbf9hTupbYohmF3DbgIT13q8Ehk5 KoTDhQtTbJQzdQajxZJVzSDzgoD5X2ChVKFw4XjzLqmx3gQTq45hh5W/sG4NTD/xp9wp 8YxU5PnQGAa2OAC3NvM/G0JCWqbMDMVjpgYREIbTRRkr5AJxUJBki6WT2MGtLRmTqJEe ZcgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=HtKFnScb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h28-20020a056a00001c00b0059d1764718dsi9530017pfk.133.2023.03.06.08.39.23; Mon, 06 Mar 2023 08:39:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=HtKFnScb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230211AbjCFQhQ (ORCPT + 99 others); Mon, 6 Mar 2023 11:37:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230332AbjCFQgS (ORCPT ); Mon, 6 Mar 2023 11:36:18 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 537D4457D2 for ; Mon, 6 Mar 2023 08:35:44 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C766B21E5C; Mon, 6 Mar 2023 16:35:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120501; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PG3Fi763G1leTyqQb5znTW1NqQX2oxdmW3GvkBcAAnI=; b=HtKFnScbp1Txw0awnbbHJA1Vjgfo+RmS1dNRnBf/EAMOridZIFV9vJxDmHZ3C/WjgyacaN JFGJaACgXqxCpYVFs+p6ONCW6F0kxtb2fqtqiSWUAOBRJumG3Un6otqcUDrMODZUzAOQ5v xp5VC9YBvf8yEKzOc+lz0IH1Lq0RFDk= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8ED3A13A66; Mon, 6 Mar 2023 16:35:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id J+usITUWBmRoUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:01 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 06/12] x86/mtrr: replace vendor tests in MTRR code Date: Mon, 6 Mar 2023 17:34:19 +0100 Message-Id: <20230306163425.8324-7-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759637171173068012?= X-GMAIL-MSGID: =?utf-8?q?1759637171173068012?= Modern CPUs all share the same MTRR interface implemented via generic_mtrr_ops. At several places in MTRR code this generic interface is deduced via is_cpu(INTEL) tests, which is only working due to X86_VENDOR_INTEL being 0 (the is_cpu() macro is testing mtrr_if->vendor, which isn't explicitly set in generic_mtrr_ops). Fix that by replacing the is_cpu(INTEL) tests with testing for mtrr_if to be &generic_mtrr_ops. The only other place where the .vendor member of struct mtrr_ops is being used is in set_num_var_ranges(), where depending on the vendor the number of MTRR registers is determined. This can easily be changed by replacing .vendor with the static number of MTRR registers. It should be noted that the test "is_cpu(HYGON)" wasn't ever returning true, as there is no struct mtrr_ops with that vendor information. Signed-off-by: Juergen Gross --- V3: - new patch --- arch/x86/kernel/cpu/mtrr/amd.c | 2 +- arch/x86/kernel/cpu/mtrr/centaur.c | 2 +- arch/x86/kernel/cpu/mtrr/cleanup.c | 4 ++-- arch/x86/kernel/cpu/mtrr/cyrix.c | 2 +- arch/x86/kernel/cpu/mtrr/generic.c | 2 +- arch/x86/kernel/cpu/mtrr/mtrr.c | 8 +++----- arch/x86/kernel/cpu/mtrr/mtrr.h | 4 +--- 7 files changed, 10 insertions(+), 14 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/amd.c b/arch/x86/kernel/cpu/mtrr/amd.c index eff6ac62c0ff..ef3e8e42b782 100644 --- a/arch/x86/kernel/cpu/mtrr/amd.c +++ b/arch/x86/kernel/cpu/mtrr/amd.c @@ -110,7 +110,7 @@ amd_validate_add_page(unsigned long base, unsigned long size, unsigned int type) } const struct mtrr_ops amd_mtrr_ops = { - .vendor = X86_VENDOR_AMD, + .var_regs = 2, .set = amd_set_mtrr, .get = amd_get_mtrr, .get_free_region = generic_get_free_region, diff --git a/arch/x86/kernel/cpu/mtrr/centaur.c b/arch/x86/kernel/cpu/mtrr/centaur.c index b8a74eddde83..4466ddeb0125 100644 --- a/arch/x86/kernel/cpu/mtrr/centaur.c +++ b/arch/x86/kernel/cpu/mtrr/centaur.c @@ -112,7 +112,7 @@ centaur_validate_add_page(unsigned long base, unsigned long size, unsigned int t } const struct mtrr_ops centaur_mtrr_ops = { - .vendor = X86_VENDOR_CENTAUR, + .var_regs = 8, .set = centaur_set_mcr, .get = centaur_get_mcr, .get_free_region = centaur_get_free_region, diff --git a/arch/x86/kernel/cpu/mtrr/cleanup.c b/arch/x86/kernel/cpu/mtrr/cleanup.c index b5f43049fa5f..1c2c0c252fa5 100644 --- a/arch/x86/kernel/cpu/mtrr/cleanup.c +++ b/arch/x86/kernel/cpu/mtrr/cleanup.c @@ -689,7 +689,7 @@ int __init mtrr_cleanup(unsigned address_bits) int index_good; int i; - if (!is_cpu(INTEL) || enable_mtrr_cleanup < 1) + if (mtrr_if != &generic_mtrr_ops || enable_mtrr_cleanup < 1) return 0; rdmsr(MSR_MTRRdefType, def, dummy); @@ -886,7 +886,7 @@ int __init mtrr_trim_uncached_memory(unsigned long end_pfn) * Make sure we only trim uncachable memory on machines that * support the Intel MTRR architecture: */ - if (!is_cpu(INTEL) || disable_mtrr_trim) + if (mtrr_if != &generic_mtrr_ops || disable_mtrr_trim) return 0; rdmsr(MSR_MTRRdefType, def, dummy); diff --git a/arch/x86/kernel/cpu/mtrr/cyrix.c b/arch/x86/kernel/cpu/mtrr/cyrix.c index 173b9e01e623..238dad57d4d6 100644 --- a/arch/x86/kernel/cpu/mtrr/cyrix.c +++ b/arch/x86/kernel/cpu/mtrr/cyrix.c @@ -235,7 +235,7 @@ static void cyrix_set_arr(unsigned int reg, unsigned long base, } const struct mtrr_ops cyrix_mtrr_ops = { - .vendor = X86_VENDOR_CYRIX, + .var_regs = 8, .set = cyrix_set_arr, .get = cyrix_get_arr, .get_free_region = cyrix_get_free_region, diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index 49b4cc923312..e25a44c2c950 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -827,7 +827,7 @@ int generic_validate_add_page(unsigned long base, unsigned long size, * For Intel PPro stepping <= 7 * must be 4 MiB aligned and not touch 0x70000000 -> 0x7003FFFF */ - if (is_cpu(INTEL) && boot_cpu_data.x86 == 6 && + if (mtrr_if == &generic_mtrr_ops && boot_cpu_data.x86 == 6 && boot_cpu_data.x86_model == 1 && boot_cpu_data.x86_stepping <= 7) { if (base & ((1 << (22 - PAGE_SHIFT)) - 1)) { diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtrr.c index 5fe62ee0361b..0c83990501f5 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -108,14 +108,12 @@ static int have_wrcomb(void) /* This function returns the number of variable MTRRs */ static void __init set_num_var_ranges(bool use_generic) { - unsigned long config = 0, dummy; + unsigned long config, dummy; if (use_generic) rdmsr(MSR_MTRRcap, config, dummy); - else if (is_cpu(AMD) || is_cpu(HYGON)) - config = 2; - else if (is_cpu(CYRIX) || is_cpu(CENTAUR)) - config = 8; + else + config = mtrr_if->var_regs; num_var_ranges = config & 0xff; } diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h index 02eb5871492d..a3c362d3d5bf 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.h +++ b/arch/x86/kernel/cpu/mtrr/mtrr.h @@ -13,7 +13,7 @@ extern unsigned int mtrr_usage_table[MTRR_MAX_VAR_RANGES]; struct mtrr_ops { - u32 vendor; + u32 var_regs; void (*set)(unsigned int reg, unsigned long base, unsigned long size, mtrr_type type); void (*get)(unsigned int reg, unsigned long *base, @@ -54,8 +54,6 @@ bool get_mtrr_state(void); extern u64 size_or_mask, size_and_mask; extern const struct mtrr_ops *mtrr_if; -#define is_cpu(vnd) (mtrr_if && mtrr_if->vendor == X86_VENDOR_##vnd) - extern unsigned int num_var_ranges; extern u64 mtrr_tom2; extern struct mtrr_state_type mtrr_state; From patchwork Mon Mar 6 16:34:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64947 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1949828wrd; Mon, 6 Mar 2023 08:53:24 -0800 (PST) X-Google-Smtp-Source: AK7set9sjBKgSTO6SSl6WVW3UUA0YlaJzcGl0vCdyUW7DGLrcvb5xJTtJxRbWscBh3E5sBOPF8Cy X-Received: by 2002:a17:903:22c5:b0:19d:1686:989 with SMTP id y5-20020a17090322c500b0019d16860989mr14583070plg.59.1678121604042; Mon, 06 Mar 2023 08:53:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678121604; cv=none; d=google.com; s=arc-20160816; b=TdRzbjdNdLvzDIkWTGFORrLd744ZQ8+Zp8dHxJ4ZqW9AagTA+KYimCr6pQwr2xFDLL pr5ZIcgw4hq8nnT+0ooG44FsjzrEvEff2pUiYSEGWdFIFaWHHu2NgUI3iM/tAh0wZ5V2 LIiw93MRMzLTh6MWk/2OKaeBHbnSMldnmw0FudYIjw4O4RJx/Bs/UKRJhpjAOHCOWFp8 I18MPJvO1x0C+LtafoI0wa8go9Z0amm1YRsvkJoCoBOfzh8GNzbGd2LVdzG+eeI7hV1i 5HgYXjIxO0lwAlCmgD3MMKoNoC8r1FRsCnw3/qJMoGo3sLqDvA/I1HXOf7pwBeaHZvkp l4vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YeH0hO+srtzbwE8WX1MqylWAbFnZLLMVUNqDdf20xnc=; b=zsYzvGtdHTLZd66rxW4e4VpNPN+BaxiI9nyAncsVpF3u1dl25kO1l1rkeBxEHl3Jta YTjyonjptiq+UrcofxJV4PCesLN9qZpwaQn50r7W/zU6xAAFFKgeTyQFLnjOK1/ooMmH yVAJMZb/LBh5jx0mNCCnsWXG2dDlQOg7l8oCBQJIO5t/9WTfsZZI3TZOObKHxgkJvTQq bytG80HmK2QdhVnC2FpM+a1tv3CVHJtxm7VVq4B9h+8rrrOr9ckVDdzXxWmw6dddVZ+8 R1aXBiiUlntryAvI4Nf3EwOVK2jwmPoxEBgmnjF+4TTfXAe4rTiyWnz+EnlWjRNzhD6A PcEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=uMStH4Od; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z18-20020a1709028f9200b0019caa0b9701si9241302plo.170.2023.03.06.08.53.11; Mon, 06 Mar 2023 08:53:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=uMStH4Od; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230154AbjCFQgs (ORCPT + 99 others); Mon, 6 Mar 2023 11:36:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48054 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229973AbjCFQfy (ORCPT ); Mon, 6 Mar 2023 11:35:54 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FB203E607 for ; Mon, 6 Mar 2023 08:35:20 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6E91F21E61; Mon, 6 Mar 2023 16:35:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120507; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YeH0hO+srtzbwE8WX1MqylWAbFnZLLMVUNqDdf20xnc=; b=uMStH4OdzkDbYwZMmUuDbr+2yIPidh6L/M/CHXQI8mLkK3ohNycm7fb71xF7NZgzu06/lr XdeX6fnkZ5mQoaAOAVedcDzlLkngNvTY2X9XFSu1x/Qq7aeaAt7dqHqeYo/5+QfdQuXUvG 8Z5tU5HOGhxcPB3RKtCItgph/nWOzGA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 334DB13A66; Mon, 6 Mar 2023 16:35:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id +SczCzsWBmR3UwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:07 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 07/12] x86/mtrr: allocate mtrr_value array dynamically Date: Mon, 6 Mar 2023 17:34:20 +0100 Message-Id: <20230306163425.8324-8-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638038656601353?= X-GMAIL-MSGID: =?utf-8?q?1759638038656601353?= The mtrr_value[] array is a static variable, which is used only in a few configurations. Consuming 6kB is ridiculous for this case, especially as the array doesn't need to be that large and it can easily be allocated dynamically. Signed-off-by: Juergen Gross --- arch/x86/kernel/cpu/mtrr/mtrr.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtrr.c index 0c83990501f5..50cd2287b6e1 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -581,7 +581,7 @@ struct mtrr_value { unsigned long lsize; }; -static struct mtrr_value mtrr_value[MTRR_MAX_VAR_RANGES]; +static struct mtrr_value *mtrr_value; static int mtrr_save(void) { @@ -750,6 +750,7 @@ static int __init mtrr_init_finialize(void) * TBD: is there any system with such CPU which supports * suspend/resume? If no, we should remove the code. */ + mtrr_value = kcalloc(num_var_ranges, sizeof(*mtrr_value), GFP_KERNEL); register_syscore_ops(&mtrr_syscore_ops); return 0; From patchwork Mon Mar 6 16:34:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64971 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1955648wrd; Mon, 6 Mar 2023 09:03:30 -0800 (PST) X-Google-Smtp-Source: AK7set/gnN1OapL21PZ1SsgI829JZVNk8Ud3EBRWKJW3X+5rZfUBcpPmhQSVXZj6wIXpQpm38mih X-Received: by 2002:a17:907:80ca:b0:889:58bd:86f1 with SMTP id io10-20020a17090780ca00b0088958bd86f1mr11877799ejc.14.1678122210464; Mon, 06 Mar 2023 09:03:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678122210; cv=none; d=google.com; s=arc-20160816; b=rAzPNt3seFI9FmMEnVd0FnKSFlHHusD6qsWu30DMDUffxDcnDPxJ4FeMzj4u42YTzJ UB1RwayHFwRv1CSGXChUyUaYw6F97AfifvTPJn0gaLH/z3ivEnAlm3+GdTefZTYE9iPI j2KggdGdMLMno1KDCHRI5sgGvMOmg3Akn9SBx3s3JGD4LmrvcUEOjZiFeF9CB1+wSRGU PVZeLtaKwPAjn+YZpBG3TyY1aCpDBVkeOje3dkxXEDhtjVE6p+1d3vluM10WHl5lJpeT nvJf4FbWSYLlriM7+duz4w0nhOgNXeKeNR6w+KnJeOJBNdRS67UbShxNFfPtYH3Z8xOg B1Ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+DUOEsA+UNBr/vO361m2bPY9yhOH/7VFzmsusEs0LhE=; b=q0zX7LAuezfs9xr5JPuxElGjLB1xrD+Q+s887ovEAgA+ZAkUqD+bxEXbtJ2Mz1jCHX Etst8ps0trJy0EksF90JzyurMR5C2fVvUxl3X28PhnxahuFY/Dubm6sG9iv1gPcWae6N K/9TTaWplxTkMbZeDp1xp+/HNDAFrRucvjDwjQVQs8GZWlTLMbzlCUJbHwI7mNEArYaA 3PkC5XguGtde8+NJQl0HEWjUEzfydvoKKe4hhkEDp5Jh4/UqQdLTHlwite4YPFgQxydn eb9Hg7uYKsvXR2BxR+rk3O4y+xnZsjVNZLH9G9UoXbaBNH94qt8ZHGJYn/abWyMdCHMh Pw9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=Pnly3Qe5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w20-20020a50fa94000000b004ad0413e904si11139797edr.514.2023.03.06.09.03.06; Mon, 06 Mar 2023 09:03:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=Pnly3Qe5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229825AbjCFQjV (ORCPT + 99 others); Mon, 6 Mar 2023 11:39:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230332AbjCFQik (ORCPT ); Mon, 6 Mar 2023 11:38:40 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14F3839B84 for ; Mon, 6 Mar 2023 08:37:02 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1E29B21E64; Mon, 6 Mar 2023 16:35:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120513; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+DUOEsA+UNBr/vO361m2bPY9yhOH/7VFzmsusEs0LhE=; b=Pnly3Qe54P15pQsm2Gv3mdlsTA+CmSfI+mxgNBvgPwAwFC9OyyDO0YvJw1Bra3RgzqdXUA /eSZnKUVSwejcla2/oigzvVqA++wpOjmGXu/x/TmRkNEXjdSrnqDeiBg9c+2GaTbHDmmb5 9+SwcLb+0Cj2CbdamNUnmZ+UyrxTBjU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D4FC713A66; Mon, 6 Mar 2023 16:35:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id RcSdMkAWBmSLUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:12 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 08/12] x86/mtrr: add get_effective_type() service function Date: Mon, 6 Mar 2023 17:34:21 +0100 Message-Id: <20230306163425.8324-9-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638675202668125?= X-GMAIL-MSGID: =?utf-8?q?1759638675202668125?= Add a service function for obtaining the effective cache mode of overlapping MTRR registers. Make use of that function in check_type_overlap(). Signed-off-by: Juergen Gross --- V3: - new patch --- arch/x86/kernel/cpu/mtrr/generic.c | 39 +++++++++++++++--------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index e25a44c2c950..d271e0c73775 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -78,31 +78,30 @@ static u64 get_mtrr_size(u64 mask) return size; } +static u8 get_effective_type(u8 type1, u8 type2) +{ + if (type1 == MTRR_TYPE_UNCACHABLE || type2 == MTRR_TYPE_UNCACHABLE) + return MTRR_TYPE_UNCACHABLE; + + if ((type1 == MTRR_TYPE_WRBACK && type2 == MTRR_TYPE_WRTHROUGH) || + (type1 == MTRR_TYPE_WRTHROUGH && type2 == MTRR_TYPE_WRBACK)) + return MTRR_TYPE_WRTHROUGH; + + if (type1 != type2) + return MTRR_TYPE_UNCACHABLE; + + return type1; +} + /* * Check and return the effective type for MTRR-MTRR type overlap. - * Returns 1 if the effective type is UNCACHEABLE, else returns 0 + * Returns true if the effective type is UNCACHEABLE, else returns false */ -static int check_type_overlap(u8 *prev, u8 *curr) +static bool check_type_overlap(u8 *prev, u8 *curr) { - if (*prev == MTRR_TYPE_UNCACHABLE || *curr == MTRR_TYPE_UNCACHABLE) { - *prev = MTRR_TYPE_UNCACHABLE; - *curr = MTRR_TYPE_UNCACHABLE; - return 1; - } - - if ((*prev == MTRR_TYPE_WRBACK && *curr == MTRR_TYPE_WRTHROUGH) || - (*prev == MTRR_TYPE_WRTHROUGH && *curr == MTRR_TYPE_WRBACK)) { - *prev = MTRR_TYPE_WRTHROUGH; - *curr = MTRR_TYPE_WRTHROUGH; - } + *prev = *curr = get_effective_type(*curr, *prev); - if (*prev != *curr) { - *prev = MTRR_TYPE_UNCACHABLE; - *curr = MTRR_TYPE_UNCACHABLE; - return 1; - } - - return 0; + return *prev == MTRR_TYPE_UNCACHABLE; } /** From patchwork Mon Mar 6 16:34:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64974 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1955998wrd; Mon, 6 Mar 2023 09:03:59 -0800 (PST) X-Google-Smtp-Source: AK7set+UfUwcMVbfFOh+oE+MzxA2vnQLYhrEGaBTH/ftl0TAtOUptDqgI3Yv5IHP7hBTkmEq9pOw X-Received: by 2002:a17:907:c25:b0:8b2:fb3d:95e6 with SMTP id ga37-20020a1709070c2500b008b2fb3d95e6mr14114043ejc.74.1678122239384; Mon, 06 Mar 2023 09:03:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678122239; cv=none; d=google.com; s=arc-20160816; b=aFW+0ApwYHFyYnREEUjhUxIuY7XR/Gho80v+5BFGBEY3ZZKk0Sdw9dpDlbATQTDO0i dCbLyxsK1DLrRD4ym5tE2djAs6aMIjWkFtKNo+d+c2Pxx+fBbDA0vyVihRnezJmTEVXi wijvygrJyw1kHgtNg2chucLIx0l9DhrZzMZ4eKSdiI9KZTHXTCiH3/l/aWCO1bPtNrYk Jd/RhfkUjCUAFTTz3X9rVspeBY/6cJRNq9iEnA1/BDeGmOrTI+Qo78CcPnkZ2phRtvPw vl5DOrdsbRn55p2W6kFsvi/2yD1RclSI/j0wG8d5u/wk0RyP7lqwxZy8o0dSa8npc5uq Rp8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bw0w1eBhUpbFZKSgzj5kXYLWi6kQdU238jtWFg3CObg=; b=nhN4VT1IxNzmeEWtl34PwjbuH+BWKprjge3uoGUQJNIO7v7L5Vrm2lvjmSbGOY0BGt l4TUddR7/O3eSWEDMo33TnS9ptkrItf96S452NS3w1+0GzTEmoLZHlSpYmexq7MKCPS5 aPRVsetSKjSUiNZq1TPrgfxZMgMiua9BLgdj1891WqWCf9QamZbrLkHien1Gn6ktWe5s N9vJyhkOAg3LhEoSbqoHgQawfz6fiF8j3wbtaRg26BM2XOGOrZQZBgEFSOVoZ5KASWN7 Z4v2WyLDkKLkZdmG5LUb/nRAJ+gqC2qOE/tBBDogLjcWH3NNgAhS2MS1bQPUfD4ElEIN X6Mw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=XibzcWpX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id wg1-20020a17090705c100b008b17fe95459si6954511ejb.47.2023.03.06.09.03.35; Mon, 06 Mar 2023 09:03:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=XibzcWpX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230077AbjCFQm6 (ORCPT + 99 others); Mon, 6 Mar 2023 11:42:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230128AbjCFQmp (ORCPT ); Mon, 6 Mar 2023 11:42:45 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 935314DBF4 for ; Mon, 6 Mar 2023 08:42:12 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id C16561FDE7; Mon, 6 Mar 2023 16:35:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120518; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bw0w1eBhUpbFZKSgzj5kXYLWi6kQdU238jtWFg3CObg=; b=XibzcWpX1B89uabdMHwYGnLtuKsnFTD+BhbQurP8ulwE8n7GGjO/Sc6U2roCSHkKND2ds1 bTLKdLV2aKW5sUq0OIPBz9M2T+KGb8bswiL0H582D3SH0CbtI7WqESWwTXiazXv1vc+BKH pSWd+jNL+WJ141aIDGtJGiQuK9/veUU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8152A13A66; Mon, 6 Mar 2023 16:35:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id wGYsHkYWBmSZUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:18 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 09/12] x86/mtrr: construct a memory map with cache modes Date: Mon, 6 Mar 2023 17:34:22 +0100 Message-Id: <20230306163425.8324-10-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638705181541964?= X-GMAIL-MSGID: =?utf-8?q?1759638705181541964?= After MTRR initialization construct a memory map with cache modes from MTRR values. This will speed up lookups via mtrr_lookup_type() especially in case of overlapping MTRRs. This will be needed when switching the semantics of the "uniform" parameter of mtrr_lookup_type() from "only covered by one MTRR" to "memory range has a uniform cache mode", which is the data the callers really want to know. Today this information is not easily available, in case MTRRs are not well sorted regarding base address. The map will be built in __initdata. When memory management is up, the map will be moved to dynamically allocated memory, in order to avoid the need of an overly large array. The size of this array is calculated using the number of variable MTRR registers and the needed size for fixed entries. Only add the map creation and expansion for now. The lookup will be added later. When writing new MTRR entries in the running system rebuild the map inside the call from mtrr_rendezvous_handler() in order to avoid nasty race conditions with concurrent lookups. Signed-off-by: Juergen Gross --- V3: - new patch --- arch/x86/kernel/cpu/mtrr/generic.c | 254 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/mtrr/mtrr.c | 6 +- arch/x86/kernel/cpu/mtrr/mtrr.h | 3 + 3 files changed, 262 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index d271e0c73775..13bc637a50e9 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -33,6 +33,37 @@ static struct fixed_range_block fixed_range_blocks[] = { {} }; +struct cache_map { + u64 start; + u64 end; + u8 type; + bool fixed; +}; + +/* + * CACHE_MAP_MAX is the maximum number of memory ranges in cache_map, where + * no 2 adjacent ranges have the same cache mode (those would be merged). + * The number is based on the worst case: + * - no two adjacent fixed MTRRs share the same cache mode + * - one variable MTRR is spanning a huge area with mode WB + * - 255 variable MTRRs with mode UC all overlap with the WB MTRR, creating 2 + * additional ranges each (result like "ababababa...aba" with a = WB, b = UC), + * accounting for MTRR_MAX_VAR_RANGES * 2 - 1 range entries + * - a TOM2 area (even with overlapping an UC MTRR can't add 2 range entries + * to the possible maximum, as it always starts at 4GB, thus it can't be in + * the middle of that MTRR, unless that MTRR starts at 0, which would remove + * the initial "a" from the "abababa" pattern above) + * The map won't contain ranges with no matching MTRR (those fall back to the + * default cache mode). + */ +#define CACHE_MAP_MAX (MTRR_NUM_FIXED_RANGES + MTRR_MAX_VAR_RANGES * 2) + +static struct cache_map init_cache_map[CACHE_MAP_MAX] __initdata; +static struct cache_map *cache_map __refdata = init_cache_map; +static unsigned int cache_map_size = CACHE_MAP_MAX; +static unsigned int cache_map_n; +static unsigned int cache_map_fixed; + static unsigned long smp_changes_mask; static int mtrr_state_set; u64 mtrr_tom2; @@ -78,6 +109,20 @@ static u64 get_mtrr_size(u64 mask) return size; } +static u8 get_var_mtrr_state(unsigned int reg, u64 *start, u64 *size) +{ + struct mtrr_var_range *mtrr = mtrr_state.var_ranges + reg; + + if (!(mtrr->mask_lo & (1 << 11))) + return MTRR_TYPE_INVALID; + + *start = (((u64)mtrr->base_hi) << 32) + (mtrr->base_lo & PAGE_MASK); + *size = get_mtrr_size((((u64)mtrr->mask_hi) << 32) + + (mtrr->mask_lo & PAGE_MASK)); + + return mtrr->base_lo & 0xff; +} + static u8 get_effective_type(u8 type1, u8 type2) { if (type1 == MTRR_TYPE_UNCACHABLE || type2 == MTRR_TYPE_UNCACHABLE) @@ -241,6 +286,211 @@ static u8 mtrr_type_lookup_variable(u64 start, u64 end, u64 *partial_end, return mtrr_state.def_type; } +static void rm_map_entry_at(int idx) +{ + int i; + + for (i = idx; i < cache_map_n - 1; i++) + cache_map[i] = cache_map[i + 1]; + + cache_map_n--; +} + +/* + * Add an entry into cache_map at a specific index. + * Merges adjacent entries if appropriate. + * Return the number of merges for correcting the scan index. + */ +static int add_map_entry_at(u64 start, u64 end, u8 type, int idx) +{ + bool merge_prev, merge_next; + int i; + + if (start >= end) + return 0; + + merge_prev = (idx > 0 && !cache_map[idx - 1].fixed && + start == cache_map[idx - 1].end && + type == cache_map[idx - 1].type); + merge_next = (idx < cache_map_n && !cache_map[idx].fixed && + end == cache_map[idx].start && + type == cache_map[idx].type); + + if (merge_prev && merge_next) { + cache_map[idx - 1].end = cache_map[idx].end; + rm_map_entry_at(idx); + return 2; + } + if (merge_prev) { + cache_map[idx - 1].end = end; + return 1; + } + if (merge_next) { + cache_map[idx].start = start; + return 1; + } + + /* Sanity check: the array should NEVER be too small! */ + if (cache_map_n == cache_map_size) { + WARN(1, "MTRR cache mode memory map exhausted!\n"); + cache_map_n = cache_map_fixed; + return 0; + } + + for (i = cache_map_n; i > idx; i--) + cache_map[i] = cache_map[i - 1]; + + cache_map[idx].start = start; + cache_map[idx].end = end; + cache_map[idx].type = type; + cache_map[idx].fixed = false; + cache_map_n++; + + return 0; +} + +/* Clear a part of an entry. Return 1 if start of entry is still valid. */ +static int clr_map_range_at(u64 start, u64 end, int idx) +{ + int ret = start != cache_map[idx].start; + u64 tmp; + + if (start == cache_map[idx].start && end == cache_map[idx].end) { + rm_map_entry_at(idx); + } else if (start == cache_map[idx].start) { + cache_map[idx].start = end; + } else if (end == cache_map[idx].end) { + cache_map[idx].end = start; + } else { + tmp = cache_map[idx].end; + cache_map[idx].end = start; + add_map_entry_at(end, tmp, cache_map[idx].type, idx + 1); + } + + return ret; +} + +static void add_map_entry(u64 start, u64 end, u8 type) +{ + int i; + u8 new_type, old_type; + u64 tmp; + + for (i = 0; i < cache_map_n && start < end; i++) { + if (start >= cache_map[i].end) + continue; + + if (start < cache_map[i].start) { + /* Region start has no overlap. */ + tmp = min(end, cache_map[i].start); + i -= add_map_entry_at(start, tmp, type, i); + start = tmp; + continue; + } + + new_type = get_effective_type(type, cache_map[i].type); + old_type = cache_map[i].type; + + if (cache_map[i].fixed || new_type == old_type) { + /* Cut off start of new entry. */ + start = cache_map[i].end; + continue; + } + + tmp = min(end, cache_map[i].end); + i += clr_map_range_at(start, tmp, i); + i -= add_map_entry_at(start, tmp, new_type, i); + start = tmp; + } + + add_map_entry_at(start, end, type, i); +} + +/* Add variable MTRRs to cache map. */ +static void map_add_var(void) +{ + unsigned int i; + u64 start, size; + u8 type; + + /* Add AMD magic MTRR. */ + if (mtrr_tom2) { + add_map_entry(1ULL << 32, mtrr_tom2 - 1, MTRR_TYPE_WRBACK); + cache_map[cache_map_n - 1].fixed = true; + } + + for (i = 0; i < num_var_ranges; i++) { + type = get_var_mtrr_state(i, &start, &size); + if (type != MTRR_TYPE_INVALID) + add_map_entry(start, start + size, type); + } +} + +/* Rebuild map by replacing variable entries. */ +static void rebuild_map(void) +{ + cache_map_n = cache_map_fixed; + + map_add_var(); +} + +/* Build the cache_map containing the cache modes per memory range. */ +void mtrr_build_map(void) +{ + unsigned int i; + u64 start, end, size; + u8 type; + + if (!mtrr_state.enabled) + return; + + /* Add fixed MTRRs, optimize for adjacent entries with same type. */ + if (mtrr_state.enabled & MTRR_STATE_MTRR_FIXED_ENABLED) { + start = 0; + end = size = 0x10000; + type = mtrr_state.fixed_ranges[0]; + + for (i = 1; i < MTRR_NUM_FIXED_RANGES; i++) { + if (i == 8 || i == 24) + size >>= 2; + + if (mtrr_state.fixed_ranges[i] != type) { + add_map_entry(start, end, type); + start = end; + type = mtrr_state.fixed_ranges[i]; + } + end += size; + } + add_map_entry(start, end, type); + } + + /* Mark fixed and magic MTRR as fixed, they take precedence. */ + for (i = 0; i < cache_map_n; i++) + cache_map[i].fixed = true; + cache_map_fixed = cache_map_n; + + map_add_var(); +} + +/* Copy the cache_map from __initdata memory to dynamically allocated one. */ +void __init mtrr_copy_map(void) +{ + unsigned int new_size = cache_map_fixed + 2 * num_var_ranges; + + if (!mtrr_state.enabled || !new_size) { + cache_map = NULL; + return; + } + + mutex_lock(&mtrr_mutex); + + cache_map = kcalloc(new_size, sizeof(*cache_map), GFP_KERNEL); + memmove(cache_map, init_cache_map, cache_map_n * sizeof(*cache_map)); + cache_map_size = new_size; + + mutex_unlock(&mtrr_mutex); +} + /** * mtrr_overwrite_state - set static MTRR state * @@ -815,6 +1065,10 @@ static void generic_set_mtrr(unsigned int reg, unsigned long base, cache_enable(); local_irq_restore(flags); + + /* On the first cpu rebuild the cache mode memory map. */ + if (smp_processor_id() == cpumask_first(cpu_online_mask)) + rebuild_map(); } int generic_validate_add_page(unsigned long base, unsigned long size, diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.c b/arch/x86/kernel/cpu/mtrr/mtrr.c index 50cd2287b6e1..1dbb9fdfd87b 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.c +++ b/arch/x86/kernel/cpu/mtrr/mtrr.c @@ -65,7 +65,7 @@ static bool mtrr_enabled(void) } unsigned int mtrr_usage_table[MTRR_MAX_VAR_RANGES]; -static DEFINE_MUTEX(mtrr_mutex); +DEFINE_MUTEX(mtrr_mutex); u64 size_or_mask, size_and_mask; @@ -668,6 +668,7 @@ void __init mtrr_bp_init(void) /* Software overwrite of MTRR state, only for generic case. */ mtrr_calc_physbits(true); init_table(); + mtrr_build_map(); pr_info("MTRRs set to read-only\n"); return; @@ -705,6 +706,7 @@ void __init mtrr_bp_init(void) if (get_mtrr_state()) { memory_caching_control |= CACHE_MTRR; changed_by_mtrr_cleanup = mtrr_cleanup(phys_addr); + mtrr_build_map(); } else { mtrr_if = NULL; why = "by BIOS"; @@ -733,6 +735,8 @@ void mtrr_save_state(void) static int __init mtrr_init_finialize(void) { + mtrr_copy_map(); + if (!mtrr_enabled()) return 0; diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h index a3c362d3d5bf..6246a1d8650b 100644 --- a/arch/x86/kernel/cpu/mtrr/mtrr.h +++ b/arch/x86/kernel/cpu/mtrr/mtrr.h @@ -53,6 +53,7 @@ bool get_mtrr_state(void); extern u64 size_or_mask, size_and_mask; extern const struct mtrr_ops *mtrr_if; +extern struct mutex mtrr_mutex; extern unsigned int num_var_ranges; extern u64 mtrr_tom2; @@ -61,6 +62,8 @@ extern struct mtrr_state_type mtrr_state; void mtrr_state_warn(void); const char *mtrr_attrib_to_str(int x); void mtrr_wrmsr(unsigned, unsigned, unsigned); +void mtrr_build_map(void); +void mtrr_copy_map(void); /* CPU specific mtrr_ops vectors. */ extern const struct mtrr_ops amd_mtrr_ops; From patchwork Mon Mar 6 16:34:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64945 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1948810wrd; Mon, 6 Mar 2023 08:51:24 -0800 (PST) X-Google-Smtp-Source: AK7set+bVqEJQ8vWt/Rfu494DjoYUphTEh+d5CiJ2Y5TF1DCqcNgzF93cAdsKOBNZMRRhoaOiWT1 X-Received: by 2002:a17:907:8a1e:b0:8b1:264d:6187 with SMTP id sc30-20020a1709078a1e00b008b1264d6187mr14343767ejc.46.1678121484152; Mon, 06 Mar 2023 08:51:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678121484; cv=none; d=google.com; s=arc-20160816; b=WEb/ONzE0hA8k5fWxKzeZo4GPrz1txmNkVL863FCeghyuYtMUIFrKx4cJtp3J1seh1 5orbrngsMy323uhABoteTZAdCMiceL69YoaJf7tL32r1x2CvDhIy4xevszl+N/GSm7k+ +C7edIXf2OiSiJlsHquLgz51XBHOeVKHdqeJ3vdfYq9kqtPGqbZRy7T3Otrj1VoSsiOq 1LEGp/fkPzouYRYj9vltEavYLA21JgIo0yCrF60JPGcXI8FUccl1e6ORs9riC0jx3LbL yMjKOZc5JsVJZkt8CfVbimduRyiRVqvx95lTxlCfoubA+MbSGdnUcaAbUdHkTxA9Hjvc vFIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0lXReJ7uisxA2FLV7DzWQucN9HApMUmMzLqAa9e4gIU=; b=I6nudasTR7jkb1uLysqlKUo1QDw1LPWxQW13wYplXp907H1c6SukLpaXmTwkMDP7L8 jdn7mpjzXphIZu2w788f0KVE/5kIxpBZfRpPVep7OZ2kC6E0rNNUJgnLwYa14u8XVFCQ DcMoQSRnc7YBVJhzy2h5epIyqnN5syu8pvRyzMfJrGw/q6BvBJllk+j2VviMRxQ/UA1E aMvX7DPzmdGgRx8zx+gZxwTMsuYIWAW9o9SpjGz03Ey8o7psZ7gVDIGnRr2vVam+28OJ ljlN4JoAxNoIK2WytxIbKDtqF/t7ozKKsbWtl7uYq7Dy0dQXkjiGpghDuA70brZdwhR+ J75g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b="EWH/eNec"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e12-20020aa7d7cc000000b004acc7f8a77dsi10874361eds.639.2023.03.06.08.51.00; Mon, 06 Mar 2023 08:51:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b="EWH/eNec"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229676AbjCFQho (ORCPT + 99 others); Mon, 6 Mar 2023 11:37:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229640AbjCFQg4 (ORCPT ); Mon, 6 Mar 2023 11:36:56 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C3D23E613 for ; Mon, 6 Mar 2023 08:36:29 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 79F571FDEB; Mon, 6 Mar 2023 16:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120524; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0lXReJ7uisxA2FLV7DzWQucN9HApMUmMzLqAa9e4gIU=; b=EWH/eNeckPlceqcNII3hOpGH+qwAwWgiIv9wnK1K0Du0nHKsCKtNYvjxX71Hcg6EwN8TV4 78YrwWHkKsmtbKrQY38HsHCYMTjcjSe24nOz4+BKXL62k+STqYk7sS5sgGuDNJwRT/Ynvl K+QR17gqwIBW9uEgh7Cfz4zALoMk8TE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3DA5E13A66; Mon, 6 Mar 2023 16:35:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id TV68DUwWBmSqUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:24 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH v4 10/12] x86/mtrr: use new cache_map in mtrr_type_lookup() Date: Mon, 6 Mar 2023 17:34:23 +0100 Message-Id: <20230306163425.8324-11-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759637913327837794?= X-GMAIL-MSGID: =?utf-8?q?1759637913327837794?= Instead of crawling through the MTRR register state, use the new cache_map for looking up the cache type(s) of a memory region. This allows now to set the uniform parameter according to the uniformity of the cache mode of the region, instead of setting it only if the complete region is mapped by a single MTRR. This now includes even the region covered by the fixed MTRR registers. Make sure uniform is always set. Signed-off-by: Juergen Gross --- V3: - new patch V3.1: - fix type_merge() (Michael Kelley) V4: - fix type_merge() again (Michael Kelley) --- arch/x86/kernel/cpu/mtrr/generic.c | 227 ++++------------------------- 1 file changed, 32 insertions(+), 195 deletions(-) diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index 13bc637a50e9..de6d74d6e3a0 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -138,154 +138,6 @@ static u8 get_effective_type(u8 type1, u8 type2) return type1; } -/* - * Check and return the effective type for MTRR-MTRR type overlap. - * Returns true if the effective type is UNCACHEABLE, else returns false - */ -static bool check_type_overlap(u8 *prev, u8 *curr) -{ - *prev = *curr = get_effective_type(*curr, *prev); - - return *prev == MTRR_TYPE_UNCACHABLE; -} - -/** - * mtrr_type_lookup_fixed - look up memory type in MTRR fixed entries - * - * Return the MTRR fixed memory type of 'start'. - * - * MTRR fixed entries are divided into the following ways: - * 0x00000 - 0x7FFFF : This range is divided into eight 64KB sub-ranges - * 0x80000 - 0xBFFFF : This range is divided into sixteen 16KB sub-ranges - * 0xC0000 - 0xFFFFF : This range is divided into sixty-four 4KB sub-ranges - * - * Return Values: - * MTRR_TYPE_(type) - Matched memory type - * MTRR_TYPE_INVALID - Unmatched - */ -static u8 mtrr_type_lookup_fixed(u64 start, u64 end) -{ - int idx; - - if (start >= 0x100000) - return MTRR_TYPE_INVALID; - - /* 0x0 - 0x7FFFF */ - if (start < 0x80000) { - idx = 0; - idx += (start >> 16); - return mtrr_state.fixed_ranges[idx]; - /* 0x80000 - 0xBFFFF */ - } else if (start < 0xC0000) { - idx = 1 * 8; - idx += ((start - 0x80000) >> 14); - return mtrr_state.fixed_ranges[idx]; - } - - /* 0xC0000 - 0xFFFFF */ - idx = 3 * 8; - idx += ((start - 0xC0000) >> 12); - return mtrr_state.fixed_ranges[idx]; -} - -/** - * mtrr_type_lookup_variable - look up memory type in MTRR variable entries - * - * Return Value: - * MTRR_TYPE_(type) - Matched memory type or default memory type (unmatched) - * - * Output Arguments: - * repeat - Set to 1 when [start:end] spanned across MTRR range and type - * returned corresponds only to [start:*partial_end]. Caller has - * to lookup again for [*partial_end:end]. - * - * uniform - Set to 1 when an MTRR covers the region uniformly, i.e. the - * region is fully covered by a single MTRR entry or the default - * type. - */ -static u8 mtrr_type_lookup_variable(u64 start, u64 end, u64 *partial_end, - int *repeat, u8 *uniform) -{ - int i; - u64 base, mask; - u8 prev_match, curr_match; - - *repeat = 0; - *uniform = 1; - - prev_match = MTRR_TYPE_INVALID; - for (i = 0; i < num_var_ranges; ++i) { - unsigned short start_state, end_state, inclusive; - - if (!(mtrr_state.var_ranges[i].mask_lo & (1 << 11))) - continue; - - base = (((u64)mtrr_state.var_ranges[i].base_hi) << 32) + - (mtrr_state.var_ranges[i].base_lo & PAGE_MASK); - mask = (((u64)mtrr_state.var_ranges[i].mask_hi) << 32) + - (mtrr_state.var_ranges[i].mask_lo & PAGE_MASK); - - start_state = ((start & mask) == (base & mask)); - end_state = ((end & mask) == (base & mask)); - inclusive = ((start < base) && (end > base)); - - if ((start_state != end_state) || inclusive) { - /* - * We have start:end spanning across an MTRR. - * We split the region into either - * - * - start_state:1 - * (start:mtrr_end)(mtrr_end:end) - * - end_state:1 - * (start:mtrr_start)(mtrr_start:end) - * - inclusive:1 - * (start:mtrr_start)(mtrr_start:mtrr_end)(mtrr_end:end) - * - * depending on kind of overlap. - * - * Return the type of the first region and a pointer - * to the start of next region so that caller will be - * advised to lookup again after having adjusted start - * and end. - * - * Note: This way we handle overlaps with multiple - * entries and the default type properly. - */ - if (start_state) - *partial_end = base + get_mtrr_size(mask); - else - *partial_end = base; - - if (unlikely(*partial_end <= start)) { - WARN_ON(1); - *partial_end = start + PAGE_SIZE; - } - - end = *partial_end - 1; /* end is inclusive */ - *repeat = 1; - *uniform = 0; - } - - if ((start & mask) != (base & mask)) - continue; - - curr_match = mtrr_state.var_ranges[i].base_lo & 0xff; - if (prev_match == MTRR_TYPE_INVALID) { - prev_match = curr_match; - continue; - } - - *uniform = 0; - if (check_type_overlap(&prev_match, &curr_match)) - return curr_match; - } - - if (prev_match != MTRR_TYPE_INVALID) - return prev_match; - - return mtrr_state.def_type; -} - static void rm_map_entry_at(int idx) { int i; @@ -533,6 +385,20 @@ void mtrr_overwrite_state(struct mtrr_var_range *var, unsigned int num_var, mtrr_state_set = 1; } +static u8 type_merge(u8 type, u8 new_type, u8 *uniform) +{ + u8 effective_type; + + if (type == MTRR_TYPE_INVALID) + return new_type; + + effective_type = get_effective_type(type, new_type); + if (type != effective_type) + *uniform = 0; + + return effective_type; +} + /** * mtrr_type_lookup - look up memory type in MTRR * @@ -541,66 +407,37 @@ void mtrr_overwrite_state(struct mtrr_var_range *var, unsigned int num_var, * MTRR_TYPE_INVALID - MTRR is disabled * * Output Argument: - * uniform - Set to 1 when an MTRR covers the region uniformly, i.e. the - * region is fully covered by a single MTRR entry or the default - * type. + * uniform - Set to 1 when the returned MTRR type is valid for the whole + * region, set to 0 else. */ u8 mtrr_type_lookup(u64 start, u64 end, u8 *uniform) { - u8 type, prev_type, is_uniform = 1, dummy; - int repeat; - u64 partial_end; - - /* Make end inclusive instead of exclusive */ - end--; + u8 type = MTRR_TYPE_INVALID; + unsigned int i; - if (!mtrr_state_set) + if (!mtrr_state_set) { + *uniform = 0; /* Uniformity is unknown. */ return MTRR_TYPE_INVALID; + } + + *uniform = 1; if (!(mtrr_state.enabled & MTRR_STATE_MTRR_ENABLED)) return MTRR_TYPE_INVALID; - /* - * Look up the fixed ranges first, which take priority over - * the variable ranges. - */ - if ((start < 0x100000) && - (mtrr_state.have_fixed) && - (mtrr_state.enabled & MTRR_STATE_MTRR_FIXED_ENABLED)) { - is_uniform = 0; - type = mtrr_type_lookup_fixed(start, end); - goto out; - } - - /* - * Look up the variable ranges. Look of multiple ranges matching - * this address and pick type as per MTRR precedence. - */ - type = mtrr_type_lookup_variable(start, end, &partial_end, - &repeat, &is_uniform); + for (i = 0; i < cache_map_n && start < end; i++) { + if (start >= cache_map[i].end) + continue; + if (start < cache_map[i].start) + type = type_merge(type, mtrr_state.def_type, uniform); + type = type_merge(type, cache_map[i].type, uniform); - /* - * Common path is with repeat = 0. - * However, we can have cases where [start:end] spans across some - * MTRR ranges and/or the default type. Do repeated lookups for - * that case here. - */ - while (repeat) { - prev_type = type; - start = partial_end; - is_uniform = 0; - type = mtrr_type_lookup_variable(start, end, &partial_end, - &repeat, &dummy); - - if (check_type_overlap(&prev_type, &type)) - goto out; + start = cache_map[i].end; } - if (mtrr_tom2 && (start >= (1ULL<<32)) && (end < mtrr_tom2)) - type = MTRR_TYPE_WRBACK; + if (start < end) + type = type_merge(type, mtrr_state.def_type, uniform); -out: - *uniform = is_uniform; return type; } From patchwork Mon Mar 6 16:34:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64938 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1943298wrd; Mon, 6 Mar 2023 08:39:44 -0800 (PST) X-Google-Smtp-Source: AK7set94HliLkK3eoxYrbBKPGohW4u7K6pj3LqryJalG2z4kTpA0y8/nbnz8KwTyBLhSH7W6Gqek X-Received: by 2002:a17:90b:1b48:b0:237:401c:9bd5 with SMTP id nv8-20020a17090b1b4800b00237401c9bd5mr11488372pjb.46.1678120783766; Mon, 06 Mar 2023 08:39:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678120783; cv=none; d=google.com; s=arc-20160816; b=DQ7UOwNfcwrKNXIyg1WLqr20EobuLp/RXHytiRkWqDVjGvUh7ZYHdBPDBj7wRQ6i2h i0TWe3dWiGQutFyI3FdFz8AuVrkyeBqhdZsPb2P/P2bZ7UrqyyZdVec5q8lWnkiiWHi5 FIhk814sIgRDbpakHU/0ssvYWxHWS4QrUk0d1zGbBQ+HtnfTPNzVgR99cDsmKXREK1Mm cphKmPtCWK6vDsxuurOvQjjKnzSx71pcSIdIQu73Xwl677zldItZhWKQKAbPyG9IOIz4 vVNb8Jd6GhfuxtiElRZwHb4YJIhh4kZ71LkRrNrVQdJMHSZhzWLuGS+HIsRjw7pOGwCv 8EkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5wTDwZv06EmUbfQ1jDWIuEcre1E3183cNpPdoj5eT8U=; b=UVG9PJiKZZQzsIchfYOA1a1XQb+k/8Q2hX52wSa4Lfzwr6vdk+V4bxDfy4zWEnR0JZ 5rQBTlnpLxnRP53nkiXbLkXmMIIZH9WnoI0QqweeWeVA8BLyeKCTWe1iMmqdN4T6QebO NoGVNd2YY8xQF1KbMpf+2ilKKTCCRID8O1/RmkmnRXSl54Zj5drVC4tK+AOFVXCWP91q rc2J9Aic5+TqB9durSWYf5uZNfcQ23O/7wsx6L7nhjrdI494Mhi5yyo3DvQtqUvrm8rO Ou9X/YusYGTj3+PTpScflXagCMBM070qGB61KWoV3k7hyOsT/ygr2sEpHpY9T0ihQmgv FpBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=VAmf1J7B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s6-20020a634506000000b004fbd4ae5651si9839265pga.170.2023.03.06.08.39.29; Mon, 06 Mar 2023 08:39:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=VAmf1J7B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230261AbjCFQhT (ORCPT + 99 others); Mon, 6 Mar 2023 11:37:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229794AbjCFQg3 (ORCPT ); Mon, 6 Mar 2023 11:36:29 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6AC03C17 for ; Mon, 6 Mar 2023 08:36:00 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 301321FDEC; Mon, 6 Mar 2023 16:35:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120530; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5wTDwZv06EmUbfQ1jDWIuEcre1E3183cNpPdoj5eT8U=; b=VAmf1J7BrJMNASYAzGdYE3xlWvYmcISrJbv3oJgYGvqbgLPfCDHFE+ZVdv2q5+kTN2HgDF 0tdvJ4bpn3Hg618ZiJDgubq5lopaHA6vh8KSQzTIeuJLqXv+/lGpOHU0E/n556tuAR8jvV RWy/qWkiQJXZnQ/1Qqu4sc4KH5Byhsk= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DE67613A66; Mon, 6 Mar 2023 16:35:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id Vaq7NFEWBmSvUwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:29 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Linus Torvalds Subject: [PATCH v4 11/12] x86/mtrr: don't let mtrr_type_lookup() return MTRR_TYPE_INVALID Date: Mon, 6 Mar 2023 17:34:24 +0100 Message-Id: <20230306163425.8324-12-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759637178980683915?= X-GMAIL-MSGID: =?utf-8?q?1759637178980683915?= mtrr_type_lookup() should always return a valid memory type. In case there is no information available, it should return the default UC. This will remove the last case where mtrr_type_lookup() can return MTRR_TYPE_INVALID, so adjust the comment in include/uapi/asm/mtrr.h. Note that removing the MTRR_TYPE_INVALID #define from that header could break user code, so it has to stay. At the same time the mtrr_type_lookup() stub for the !CONFIG_MTRR case should set uniform to 1, as if the memory range would be covered by no MTRR at all. Suggested-by: Linus Torvalds Signed-off-by: Juergen Gross --- V2: - always set uniform - set uniform to 1 in case of disabled MTRRs (Linus Torvalds) V3: - adjust include/uapi/asm/mtrr.h comment --- arch/x86/include/asm/mtrr.h | 7 +++++-- arch/x86/include/uapi/asm/mtrr.h | 6 +++--- arch/x86/kernel/cpu/mtrr/generic.c | 4 ++-- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h index f1cb81330a64..0d7ea8a54d81 100644 --- a/arch/x86/include/asm/mtrr.h +++ b/arch/x86/include/asm/mtrr.h @@ -59,9 +59,12 @@ static inline void mtrr_overwrite_state(struct mtrr_var_range *var, static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform) { /* - * Return no-MTRRs: + * Return the default MTRR type, without any known other types in + * that range. */ - return MTRR_TYPE_INVALID; + *uniform = 1; + + return MTRR_TYPE_UNCACHABLE; } #define mtrr_save_fixed_ranges(arg) do {} while (0) #define mtrr_save_state() do {} while (0) diff --git a/arch/x86/include/uapi/asm/mtrr.h b/arch/x86/include/uapi/asm/mtrr.h index 376563f2bac1..4aa05c2ffa78 100644 --- a/arch/x86/include/uapi/asm/mtrr.h +++ b/arch/x86/include/uapi/asm/mtrr.h @@ -115,9 +115,9 @@ struct mtrr_state_type { #define MTRR_NUM_TYPES 7 /* - * Invalid MTRR memory type. mtrr_type_lookup() returns this value when - * MTRRs are disabled. Note, this value is allocated from the reserved - * values (0x7-0xff) of the MTRR memory types. + * Invalid MTRR memory type. No longer used outside of MTRR code. + * Note, this value is allocated from the reserved values (0x7-0xff) of + * the MTRR memory types. */ #define MTRR_TYPE_INVALID 0xff diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c index de6d74d6e3a0..3d48e9d06bfb 100644 --- a/arch/x86/kernel/cpu/mtrr/generic.c +++ b/arch/x86/kernel/cpu/mtrr/generic.c @@ -417,13 +417,13 @@ u8 mtrr_type_lookup(u64 start, u64 end, u8 *uniform) if (!mtrr_state_set) { *uniform = 0; /* Uniformity is unknown. */ - return MTRR_TYPE_INVALID; + return MTRR_TYPE_UNCACHABLE; } *uniform = 1; if (!(mtrr_state.enabled & MTRR_STATE_MTRR_ENABLED)) - return MTRR_TYPE_INVALID; + return MTRR_TYPE_UNCACHABLE; for (i = 0; i < cache_map_n && start < end; i++) { if (start >= cache_map[i].end) From patchwork Mon Mar 6 16:34:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juergen Gross X-Patchwork-Id: 64954 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1951924wrd; Mon, 6 Mar 2023 08:57:58 -0800 (PST) X-Google-Smtp-Source: AK7set+MQJQiiCg+UVa2K8GluJUJ4lEoinpgRW1k3vN8nDcEnlpZ1/Pmhmr1p0cvjvugzH683Gch X-Received: by 2002:a17:90b:4d85:b0:235:b712:4180 with SMTP id oj5-20020a17090b4d8500b00235b7124180mr10590333pjb.24.1678121877779; Mon, 06 Mar 2023 08:57:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678121877; cv=none; d=google.com; s=arc-20160816; b=wnnyvY7hlN3QOj9IcBMPYfmVdbB94ZRDVqXpEkWjJzopy4SisviB36cWs73QZRBs2U DWTAASG6vRxuvOzepRWC6zvyENCCJxSIymTggw6fhfRwvRnIXc6it6xxFlZkL+oAQ9xj 7bbfNbbnNtvGsJ3uvGf7hPDNBBgM8VmM44LF36216xBpi4FJKLnOVE27LN0Bw+/DGLL5 xUORUR8bvsR+Ex6XJCUy6W+GfaiIlUmZF4Pow4X07PlDxo2kg3ByYv407o7uzryn3TmB xCTohJdyJjM5lQW0pKfuYjaqGOcFBQkolMpFGScLLqxf48Ie12rZ9uFft98AgciEpoXG u06g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/LnG+vviZFEmSplserK7lU8lrYy0Xhh7tv0XqJyR/Mo=; b=FdkgaPvATgPdgF/aHEwuMZWlFlCPQa7jHSnCA7M5GMRtIa1pP2wG3EWS7j0mTem0GJ r2MUlf/8Kpct0UvzAOpWdiGmrErOGUgSROVOZrvxEhim7mSMIgEJ8QjDt1gUOK/YvRF0 qHcNQEG91u54miAaMQI0jbs12FP72eJ6G+ReW6lL81nE59VdWr5jwrOokMX4lSuIXjho jOgJE/xMGP41Dg7XCqMOI8YQjjTG5pYhwNlxqf0OXQUbPldgF5InyKKnK+WFt5HF2Sy+ TdBXguylHbEm04pysOMUhjsgAqYEfMBSSTRAh2W0xrzZXPXQr9ORe+9RmuEhZFL1u6+s nE7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=Pm4dbuFy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o15-20020a17090a678f00b00237155f2303si9487416pjj.136.2023.03.06.08.57.44; Mon, 06 Mar 2023 08:57:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.com header.s=susede1 header.b=Pm4dbuFy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=suse.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229832AbjCFQh2 (ORCPT + 99 others); Mon, 6 Mar 2023 11:37:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230143AbjCFQgi (ORCPT ); Mon, 6 Mar 2023 11:36:38 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 900D338B63 for ; Mon, 6 Mar 2023 08:36:12 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E23B821E68; Mon, 6 Mar 2023 16:35:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1678120535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/LnG+vviZFEmSplserK7lU8lrYy0Xhh7tv0XqJyR/Mo=; b=Pm4dbuFymyP28hC2S3dLLk5s9QEZCS4aQqKi80Fq1IBNB73YMwhWSuGo+/wrxAErADN/cN Kd+hgwoDljLDUWPLLH/Jng0pv8Fb9RyuangVMAuXAj+66tjANMk/zn/Fp/AjDR8u37KI0r +wVBlz4hM0ZY7RoAeJWXjixSGWZkKOo= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9544F13A66; Mon, 6 Mar 2023 16:35:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 7o0UI1cWBmS/UwAAMHmgww (envelope-from ); Mon, 06 Mar 2023 16:35:35 +0000 From: Juergen Gross To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Juergen Gross , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Linus Torvalds Subject: [PATCH v4 12/12] x86/mm: only check uniform after calling mtrr_type_lookup() Date: Mon, 6 Mar 2023 17:34:25 +0100 Message-Id: <20230306163425.8324-13-jgross@suse.com> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230306163425.8324-1-jgross@suse.com> References: <20230306163425.8324-1-jgross@suse.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638326108756217?= X-GMAIL-MSGID: =?utf-8?q?1759638326108756217?= Today pud_set_huge() and pmd_set_huge() test for the MTRR type to be WB or INVALID after calling mtrr_type_lookup(). Those tests can be dropped, as the only reason to not use a large mapping would be uniform being 0. Any MTRR type can be accepted as long as it applies to the whole memory range covered by the mapping, as the alternative would only be to map the same region with smaller pages instead, using the same PAT type as for the large mapping. Suggested-by: Linus Torvalds Signed-off-by: Juergen Gross --- V3: - adapt comment for pud_set_huge() --- arch/x86/mm/pgtable.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index e4f499eb0f29..15a8009a4480 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -702,14 +702,8 @@ void p4d_clear_huge(p4d_t *p4d) * pud_set_huge - setup kernel PUD mapping * * MTRRs can override PAT memory types with 4KiB granularity. Therefore, this - * function sets up a huge page only if any of the following conditions are met: - * - * - MTRRs are disabled, or - * - * - MTRRs are enabled and the range is completely covered by a single MTRR, or - * - * - MTRRs are enabled and the corresponding MTRR memory type is WB, which - * has no effect on the requested PAT memory type. + * function sets up a huge page only if the complete range has the same MTRR + * caching mode. * * Callers should try to decrease page size (1GB -> 2MB -> 4K) if the bigger * page mapping attempt fails. @@ -718,11 +712,10 @@ void p4d_clear_huge(p4d_t *p4d) */ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) { - u8 mtrr, uniform; + u8 uniform; - mtrr = mtrr_type_lookup(addr, addr + PUD_SIZE, &uniform); - if ((mtrr != MTRR_TYPE_INVALID) && (!uniform) && - (mtrr != MTRR_TYPE_WRBACK)) + mtrr_type_lookup(addr, addr + PUD_SIZE, &uniform); + if (!uniform) return 0; /* Bail out if we are we on a populated non-leaf entry: */ @@ -745,11 +738,10 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) */ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot) { - u8 mtrr, uniform; + u8 uniform; - mtrr = mtrr_type_lookup(addr, addr + PMD_SIZE, &uniform); - if ((mtrr != MTRR_TYPE_INVALID) && (!uniform) && - (mtrr != MTRR_TYPE_WRBACK)) { + mtrr_type_lookup(addr, addr + PMD_SIZE, &uniform); + if (!uniform) { pr_warn_once("%s: Cannot satisfy [mem %#010llx-%#010llx] with a huge-page mapping due to MTRR override.\n", __func__, addr, addr + PMD_SIZE); return 0;