From patchwork Fri Oct 20 21:30:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 156335 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp1335143vqb; Fri, 20 Oct 2023 14:33:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHjocrV8Jp7/mCE6IGrgWcAPNSqfO9+MuzHSuVLXqmGUXoELhjIUshliEPqaCgz5XAC6CA5 X-Received: by 2002:a05:6a21:790a:b0:16b:f586:3c4 with SMTP id bg10-20020a056a21790a00b0016bf58603c4mr3177017pzc.58.1697837592123; Fri, 20 Oct 2023 14:33:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697837592; cv=none; d=google.com; s=arc-20160816; b=eVw0NMvngFrMQlFGLMEuVdk3+Kx9H1dqExDDzyYFcQNEtTYj92iHLzI74FCoX4+8v0 vI3ddFPzRLYlxxfZHv2ON6h1JDzx4KnEV9IWP7aF7ZQJEQ//mj0J3zWQorl/k/OcqnZR a1quw5kBUE+Y82khNE2u7OGkPkqJJQVpQ0wUHA1swEChO5+Q3XWyaK0c8CCaRF8+rrDH jS8/Qj0JkERlr5olVmnllE3vif0op8atQy90tWcaReFpM+Ng2s2PTUqufyHF/aaPK4k8 iEgXvF1h94k0+WKWiU5m82xBcNzf73yJrGxak3vWmBFRZAAulvqbi/Zm6zbqFMqRZfX5 hk2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=txy2+0Swt4im+GxUfCTrDkpFjJtBeVME+BWg2nyHV4E=; fh=EIH9XAmicvPIUSP7TBeBhZ/WaoqG49JQ3xV1i3Gl7Co=; b=wjfjqh+DKmkvho8HCNlXNp3N0PkIavi+RORXdFmLlaRav/6nfBNLEVIO3e4A+QBBhw YbZqmJn+PqljNQ9B8X6oz6wuSRTD6t40J/1sp3kP/Q7tREkm0mrs9tE0Ascxz6XDauul PyVwrD/wQp6btOFsQrKlz59el57IGQ9XXxp/g/X5zfAgb9S+khEJQEdyQ8N7ztiA79uL iokhkcA9mRwB7J2+BtwzYg+eSPOzraGmc6tNPVbWPT2mVg39RdTp9PULiFIXDQS5qL3O d5Oxs0wrUGZoNVW+RLcp+An61d9UClTSBEj4OrqB7EzGsQmDQn7ORm49LsgYrfAFjNKs xdkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CaS1gqko; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id d14-20020a056a00198e00b006933e8fec67si2761061pfl.227.2023.10.20.14.32.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 14:33:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CaS1gqko; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 5D0808038D9B; Fri, 20 Oct 2023 14:32:42 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233175AbjJTVcJ (ORCPT + 26 others); Fri, 20 Oct 2023 17:32:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233125AbjJTVbw (ORCPT ); Fri, 20 Oct 2023 17:31:52 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA5C410CB; Fri, 20 Oct 2023 14:31:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697837507; x=1729373507; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kTenDP0fIm+1slyIvN0UOJERAefoI2FWwB34+/nBKl8=; b=CaS1gqkoj2MDaoQAq91p+fZAd1l5/d6+cq8HkkjAburyKZAnqq2qeO6v epL+2iH5fn2If+2aVjeERtv5FEbCBiPMPApv9IHjgW48NwnAwLtDfT7lH IiCHMmq/zeB/E1uKJWl9B8IMllcjRvNsnehif6okYZvcY4rvRbmYtVZJi Ez984bD3hThvyt+3nTU5QGSaMEFpkcovqjgz/k0MfIgtznIkVOZWMWZ41 A/LTatn9hf7t/uPmx5E51DH4p7/srC+X8CdqKOjiSDER9582bMDPGwAqS +KUaA130yaUy4owsYdTYcrIDwyvIOR1C4WYA0LZfzAebiO0jS/8w9rBvr Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10869"; a="371647189" X-IronPort-AV: E=Sophos;i="6.03,239,1694761200"; d="scan'208";a="371647189" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2023 14:31:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10869"; a="881184540" X-IronPort-AV: E=Sophos;i="6.03,239,1694761200"; d="scan'208";a="881184540" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2023 14:31:12 -0700 From: Tony Luck To: Fenghua Yu , Reinette Chatre , Peter Newman , Jonathan Corbet , Shuah Khan , x86@kernel.org Cc: Shaopeng Tan , James Morse , Jamie Iles , Babu Moger , Randy Dunlap , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v9 7/8] x86/resctrl: Sub NUMA Cluster detection and enable Date: Fri, 20 Oct 2023 14:30:59 -0700 Message-ID: <20231020213100.123598-8-tony.luck@intel.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231020213100.123598-1-tony.luck@intel.com> References: <20231003213043.13565-1-tony.luck@intel.com> <20231020213100.123598-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Fri, 20 Oct 2023 14:32:42 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778058162467640738 X-GMAIL-MSGID: 1780311751018068115 There isn't a simple h/w bit that indicates whether a CPU is running in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the ratio of NUMA nodes to L3 cache instances. When SNC mode is detected, reconfigure the RMID counters by updating the MSR_RMID_SNC_CONFIG MSR on each socket as CPUs are seen. Clearing bit zero of the MSR divides the RMIDs and renumbers the ones on the second SNC node to start from zero. An earlier commit includes all the required changes in Linux to operate in this reconfigured mode. Reviewed-by: Peter Newman Signed-off-by: Tony Luck --- Changes since v6: Moved kfree(node_caches); earlier, to the earliest point where it is no longer needed. Added Granite Rapids to list of CPU models that support SNC mode. Added Peter's review tag arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 92 ++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index e3fa9cecd599..4285a5ee81fe 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1109,6 +1109,7 @@ #define MSR_IA32_QM_CTR 0xc8e #define MSR_IA32_PQR_ASSOC 0xc8f #define MSR_IA32_L3_CBM_BASE 0xc90 +#define MSR_RMID_SNC_CONFIG 0xca0 #define MSR_IA32_L2_CBM_BASE 0xd10 #define MSR_IA32_MBA_THRTL_BASE 0xd50 diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 0e418dd14070..ac187eb0440f 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -16,11 +16,14 @@ #define pr_fmt(fmt) "resctrl: " fmt +#include #include #include #include #include +#include +#include #include #include #include "internal.h" @@ -755,11 +758,42 @@ static void clear_closid_rmid(int cpu) wrmsr(MSR_IA32_PQR_ASSOC, 0, 0); } +/* + * The power-on reset value of MSR_RMID_SNC_CONFIG is 0x1 + * which indicates that RMIDs are configured in legacy mode. + * This mode is incompatible with Linux resctrl semantics + * as RMIDs are partitioned between SNC nodes, which requires + * a user to know which RMID is allocated to a task. + * Clearing bit 0 reconfigures the RMID counters for use + * in Sub NUMA Cluster mode. This mode is better for Linux. + * The RMID space is divided between all SNC nodes with the + * RMIDs renumbered to start from zero in each node when + * couning operations from tasks. Code to read the counters + * must adjust RMID counnter numbers based on SNC node. See + * __rmid_read() for code that does this. + */ +static void snc_remap_rmids(int cpu) +{ + u64 val; + + /* Only need to enable once per package. */ + if (cpumask_first(topology_core_cpumask(cpu)) != cpu) + return; + + rdmsrl(MSR_RMID_SNC_CONFIG, val); + val &= ~BIT_ULL(0); + wrmsrl(MSR_RMID_SNC_CONFIG, val); +} + static int resctrl_online_cpu(unsigned int cpu) { struct rdt_resource *r; mutex_lock(&rdtgroup_mutex); + + if (snc_nodes_per_l3_cache > 1) + snc_remap_rmids(cpu); + for_each_capable_rdt_resource(r) domain_add_cpu(cpu, r); /* The cpu is set in default rdtgroup after online. */ @@ -1014,11 +1048,69 @@ static __init bool get_rdt_resources(void) return (rdt_mon_capable || rdt_alloc_capable); } +/* CPU models that support MSR_RMID_SNC_CONFIG */ +static const struct x86_cpu_id snc_cpu_ids[] __initconst = { + X86_MATCH_INTEL_FAM6_MODEL(ICELAKE_X, 0), + X86_MATCH_INTEL_FAM6_MODEL(SAPPHIRERAPIDS_X, 0), + X86_MATCH_INTEL_FAM6_MODEL(EMERALDRAPIDS_X, 0), + X86_MATCH_INTEL_FAM6_MODEL(GRANITERAPIDS_X, 0), + {} +}; + +/* + * There isn't a simple h/w bit that indicates whether a CPU is running + * in Sub NUMA Cluster (SNC) mode. Infer the state by comparing the + * ratio of NUMA nodes to L3 cache instances. + * It is not possible to accurately determine SNC state if the system is + * booted with a maxcpus=N parameter. That distorts the ratio of SNC nodes + * to L3 caches. It will be OK if system is booted with hyperthreading + * disabled (since this doesn't affect the ratio). + */ +static __init int snc_get_config(void) +{ + unsigned long *node_caches; + int mem_only_nodes = 0; + int cpu, node, ret; + int num_l3_caches; + + if (!x86_match_cpu(snc_cpu_ids)) + return 1; + + node_caches = bitmap_zalloc(nr_node_ids, GFP_KERNEL); + if (!node_caches) + return 1; + + cpus_read_lock(); + for_each_node(node) { + cpu = cpumask_first(cpumask_of_node(node)); + if (cpu < nr_cpu_ids) + set_bit(get_cpu_cacheinfo_id(cpu, 3), node_caches); + else + mem_only_nodes++; + } + cpus_read_unlock(); + + num_l3_caches = bitmap_weight(node_caches, nr_node_ids); + kfree(node_caches); + + if (!num_l3_caches) + return 1; + + ret = (nr_node_ids - mem_only_nodes) / num_l3_caches; + + if (ret > 1) + rdt_resources_all[RDT_RESOURCE_L3].r_resctrl.mon_scope = RESCTRL_NODE; + + return ret; +} + static __init void rdt_init_res_defs_intel(void) { struct rdt_hw_resource *hw_res; struct rdt_resource *r; + snc_nodes_per_l3_cache = snc_get_config(); + for_each_rdt_resource(r) { hw_res = resctrl_to_arch_res(r);