From patchwork Tue Oct 3 16:07:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 147993 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2a8e:b0:403:3b70:6f57 with SMTP id in14csp2191291vqb; Tue, 3 Oct 2023 09:08:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHTJdwrs0JG+/was5O45sXoNpZ2Ut/QLbqCHFm6dKR5PyeK5bFfQgrxFGQSIVj1RWkmoIHl X-Received: by 2002:a05:6e02:1112:b0:351:80c:bc29 with SMTP id u18-20020a056e02111200b00351080cbc29mr14264009ilk.17.1696349328029; Tue, 03 Oct 2023 09:08:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696349328; cv=none; d=google.com; s=arc-20160816; b=MSZLMhBzpNNvN86olQ6fA9lxo/DbZf1hVMNrIxlBoHHPPAcPAbRsgeZRsDaZ5jxBKe bYUptOFeO1ebHg1G8oZFewZPL74ezpvND2XyD8lGkQHCv2kfvhDOZO2yXvVRzF0c7Kxu X1H+UGky7rB1j8yIZw7ejiWNGHTPYfD1jcp5hx2F52081ouwggbcBD39d558JyW4axDX 7/Lu9hjli3srujNxcL3oz/fexAz0gaO+EegxDnzpk0TD64EnOBMWYquZVMbnWAfNAMHG TntQgX0pnulvThc2Lr54HyhSqXlojsS7z7fqtMmQoQDgaFkHRQXMSxEMG/sKJFL623Ss Vxgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XR6nFYZorjWzzZtT+fqtv2IlTuH4V6qtkFxZVONPpB8=; fh=EIH9XAmicvPIUSP7TBeBhZ/WaoqG49JQ3xV1i3Gl7Co=; b=EQmdfjp9vXsUpIwr+mYfvg3mVp97jAvElcqjlim0wilaati+QOCSvw7H+CGMmXOFyk z1ND8CVKROeqx70gWAZGJMGghct9AEAUUXTITl9xhxRcu3fOC9sDAAWBiJRhOA2X7iSA qV7e+wJWi9YxqIPKf2t9Bt2nG/UnMiz62d/vxc3S0p7P+kf7Q/MhdYgLmGCUsonL+Krf ueUqQf3Ph+cplnOk0EZByZMFHfPgEh/LOFEAOZeqiM4T8a3HT0z0/rlG/We03iWn1qTf T1RvukZhbdxkxF4n+qE+lK0Ho4W+Gv8FSRe+Pot0sw5pLdsqH0w7CU2axWSji6C3+AiG dewg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=XeqgBi5F; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id b26-20020a6567da000000b005859a90c58csi1675201pgs.772.2023.10.03.09.08.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Oct 2023 09:08:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=XeqgBi5F; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id E4D8281A43D3; Tue, 3 Oct 2023 09:08:46 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240363AbjJCQIn (ORCPT + 17 others); Tue, 3 Oct 2023 12:08:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240302AbjJCQI3 (ORCPT ); Tue, 3 Oct 2023 12:08:29 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1626AD; Tue, 3 Oct 2023 09:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696349298; x=1727885298; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L/07qK3RivaXgr8+/bckLeXt73AIA+F9oksv2sUBzm8=; b=XeqgBi5FFLFHYHJ8TkPlDzDuhoWf+U1dIdJ3CF1piA5lm2EPcYLishW/ pYBRNi9Tfucmn0o1TTO6m0oWH7jRHgv3V0K8ZGV/FzzpxhojsZ8Blg3OR 0mNXTZS9tvldykQW865StLBMCQX1O4MPV9C/3iYWe03+cZOQ/++9tYkPx zXMC/PeF/X6LyORVxdR31HxGisEziFp+GY35qdF4VT01T+VUuzs91oJn6 +t/fvIeK+u/QDF8RcbrIojqJU0h9hLUa36P4TcVnPcupa1AlOWwAY+aQS gLZfA6uyZXu777VwdT5BM0PKhACYgn2t700YFCdioWxvr7Ly7khkLbPva A==; X-IronPort-AV: E=McAfee;i="6600,9927,10852"; a="447083318" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="447083318" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 09:08:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10852"; a="998098114" X-IronPort-AV: E=Sophos;i="6.03,197,1694761200"; d="scan'208";a="998098114" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Oct 2023 09:08:11 -0700 From: Tony Luck To: Fenghua Yu , Reinette Chatre , Peter Newman , Jonathan Corbet , Shuah Khan , x86@kernel.org Cc: Shaopeng Tan , James Morse , Jamie Iles , Babu Moger , Randy Dunlap , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v7 7/8] x86/resctrl: Sub NUMA Cluster detection and enable Date: Tue, 3 Oct 2023 09:07:58 -0700 Message-ID: <20231003160800.8601-8-tony.luck@intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231003160800.8601-1-tony.luck@intel.com> References: <20230928191350.205703-1-tony.luck@intel.com> <20231003160800.8601-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 03 Oct 2023 09:08:46 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778058162467640738 X-GMAIL-MSGID: 1778751193009272629 There isn't a simple h/w bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances. When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen. Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes all the required changes in Linux to operate in this reconfigured mode. Signed-off-by: Tony Luck Reviewed-by: Peter Newman --- Changes since last version: Moved kfree(node_caches); earlier, to the earliest point where it is no longer needed. Added Granite Rapids to list of CPU models that support SNC mode. arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 92 ++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 1d111350197f..393d1b047617 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1100,6 +1100,7 @@ #define MSR_IA32_QM_CTR 0xc8e #define MSR_IA32_PQR_ASSOC 0xc8f #define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0 #define MSR_IA32_L2_CBM_BASE 0xd10 #define MSR_IA32_MBA_THRTL_BASE 0xd50 diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index cd189b7ca6ea..dff294529f27 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -16,11 +16,14 @@ #define pr_fmt(fmt) "resctrl: " fmt +#include #include #include #include #include +#include +#include #include #include #include "internal.h" @@ -751,11 +754,42 @@ static void clear_closid_rmid(int cpu) wrmsr(MSR_IA32_PQR_ASSOC, 0, 0); } +/* + * The power-on reset value of MSR_RMID_SNC_CONFIG is 0x1 + * which indicates that RMIDs are configured in legacy mode. + * This mode is incompatible with Linux resctrl semantics + * as RMIDs are partitioned between SNC nodes, which requires + * a user to know which RMID is allocated to a task. + * Clearing bit 0 reconfigures the RMID counters for use + * in Sub NUMA Cluster mode. This mode is better for Linux. + * The RMID space is divided between all SNC nodes with the + * RMIDs renumbered to start from zero in each node when + * couning operations from tasks. Code to read the counters + * must adjust RMID counnter numbers based on SNC node. See + * __rmid_read() for code that does this. + */ +static void snc_remap_rmids(int cpu) +{ + u64 val; + + /* Only need to enable once per package. */ + if (cpumask_first(topology_core_cpumask(cpu)) != cpu) + return; + + rdmsrl(MSR_RMID_SNC_CONFIG, val); + val &= ~BIT_ULL(0); + wrmsrl(MSR_RMID_SNC_CONFIG, val); +} + static int resctrl_online_cpu(unsigned int cpu) { struct rdt_resource *r; mutex_lock(&rdtgroup_mutex); + + if (snc_nodes_per_l3_cache > 1) + snc_remap_rmids(cpu); + for_each_capable_rdt_resource(r) domain_add_cpu(cpu, r); /* The cpu is set in default rdtgroup after online. */ @@ -1010,11 +1044,69 @@ static __init bool get_rdt_resources(void) return (rdt_mon_capable || rdt_alloc_capable); } +/* CPU models that support MSR_RMID_SNC_CONFIG */ +static const struct x86_cpu_id snc_cpu_ids[] __initconst = { + X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_X, 0), + X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, 0), + X86_MATCH_INTEL_FAM6_MODEL(EMERALDRAPIDS_X, 0), + X86_MATCH_INTEL_FAM6_MODEL(GRANITERAPIDS_X, 0), + {} +}; + +/* + * There isn't a simple h/w bit that indicates whether a CPU is running + * in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the + * ratio of NUMA nodes to L3 cache instances. + * It is not possible to accurately detemine SNC state if the system is + * booted with a maxcpus=N parameter. That distorts the ratio of SNC nodes + * to L3 caches. It will be OK if system is booted with hyperthreading + * disabled (since this doesn't affect the ratio). + */ +static __init int snc_get_config(void) +{ + unsigned long *node_caches; + int mem_only_nodes = 0; + int cpu, node, ret; + int num_l3_caches; + + if (!x86_match_cpu(snc_cpu_ids)) + return 1; + + node_caches = bitmap_zalloc(nr_node_ids, GFP_KERNEL); + if (!node_caches) + return 1; + + cpus_read_lock(); + for_each_node(node) { + cpu = cpumask_first(cpumask_of_node(node)); + if (cpu < nr_cpu_ids) + set_bit(get_cpu_cacheinfo_id(cpu, 3), node_caches); + else + mem_only_nodes++; + } + cpus_read_unlock(); + + num_l3_caches = bitmap_weight(node_caches, nr_node_ids); + kfree(node_caches); + + if (!num_l3_caches) + return 1; + + ret = (nr_node_ids - mem_only_nodes) / num_l3_caches; + + if (ret > 1) + rdt_resources_all[RDT_RESOURCE_L3].r_resctrl.mon_scope = RESCTRL_NODE; + + return ret; +} + static __init void rdt_init_res_defs_intel(void) { struct rdt_hw_resource *hw_res; struct rdt_resource *r; + snc_nodes_per_l3_cache = snc_get_config(); + for_each_rdt_resource(r) { hw_res = resctrl_to_arch_res(r);