Message ID | 20240130222034.37181-2-tony.luck@intel.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-45384-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1530426dyb; Tue, 30 Jan 2024 14:21:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IG/+VtoLJIfei4tyubQ3Qct+t9YK/q5ocsmKLwXKEiqMj1aXBrIjKx93C8krQYnytONo1UQ X-Received: by 2002:a05:6871:600c:b0:218:4c81:4f55 with SMTP id qx12-20020a056871600c00b002184c814f55mr1072oab.44.1706653288697; Tue, 30 Jan 2024 14:21:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706653288; cv=pass; d=google.com; s=arc-20160816; b=BlxxSRAjZpApapf3y1A7Si/mO+qk3kiiFEYVm7+ODCg5zRGBHkSD4AEYpHeVPHUAqG kxmLBw3dybwJVkpD+lWL2GWOMWZFRaf7oUgsTiV+RMRoa2GI066ejUXH841o+jcUapCU mJoUJNA90cCkJOgMLJJpR0oYnWNDlBQDN/v3MgeI46TXg06k/1N2+NTSxRRmAElp28a+ FJcJ6jPNX0D0VJAa55KxSyVk35eEI30bL9q4/f70RjGENm1zEOsUWntvPBNQ+A1HikJx 9xGvq2PRE5xTKYsxf5jAvVpMiHevi8ILnEyilvJEigumBHiJjZBa369Wm7by0D6CDtZ9 qgLA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=oDnUBp/mhIpXkEiffmtpFippFpQ8RJgZsyWzhLfLTlg=; fh=kDu4WSgSKQjfqS0+uR7gDD47kd+EMQAW47hbuuOMwvI=; b=LzskO15nm1FzFU33/FheLA5zaOaXlHoJzz6N6dXm7YCqGFgQQBDCv879FNp+yWrW/9 B1hOSgVqYdrpNSk36B2BEP/bbwUTasH06Hb9U8ndQnHkcE4wS4c6hNoT02rv+rDZrnBb rQpMniiBE/2CK+f+UURzaz3raKnOt91cPTfteFY5LqF0P/OOoJCvXBwKs//QvYEkZ5a+ ZCb9Yo6cKY0DSGzSThtNi7NE4l7G7lEVWfAWuMX6qX5/doJdjxRZIcwC2Rrzp20u/Ery LmtJiGuhkAPX8Gud01dm/CDqd6h587H0L2tIXHsHKXfPno3OCnyi++YcqiFtFSHxbppS VPHg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ejlFTTY7; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel+bounces-45384-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-45384-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id l187-20020a633ec4000000b005cd811e2ffesi8227021pga.18.2024.01.30.14.21.28 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jan 2024 14:21:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-45384-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ejlFTTY7; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel+bounces-45384-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-45384-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 4B32328782B for <ouuuleilei@gmail.com>; Tue, 30 Jan 2024 22:21:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id ACC847993B; Tue, 30 Jan 2024 22:20:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ejlFTTY7" Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E2B77D410; Tue, 30 Jan 2024 22:20:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.55.52.115 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706653246; cv=none; b=QtCeldqiBxdCRsEK29Id1ZOTDfG7RdBzlfZRGQCElqZRgsCh7VfbKURW+dVfpgu4IQvJmU89nHnCUGObxWUBebnMQIvsmGT9UhMwKSNkOinSCci0NmD3SBTqJGxkcl8qe4UtBXMpXC6QOWlhTkJXVRzrvduj03PbwdX0YcFzY24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706653246; c=relaxed/simple; bh=At0RKTLoBHv44HG3Uy9zivC4vUPHbbq8Oy87lxfsgWQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NvE7Xph6PYaRX0N+K/FY9Rq1rZRYMW1OJE7jrAy9Mqy0CgnYopP3dxr/lJ8IHVbU2XS7D2DrKLS3G6Ky1fdR/ivVi7+keSgvyESPLOp4Ic0qfQupyF1Z0ONJaB64kES/AZtUrmIE+mMNbMu+aqyK8+UVgCISmk+ISFAAtve+SxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ejlFTTY7; arc=none smtp.client-ip=192.55.52.115 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706653245; x=1738189245; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=At0RKTLoBHv44HG3Uy9zivC4vUPHbbq8Oy87lxfsgWQ=; b=ejlFTTY7YqCsg8cB+Cqd3xY4NnYlnfsLlT6LdrI7U6tvFIlg8HoJ4Vu/ Ayz3lZeTXD7pCaUL9mDIw8ehB41yytUgFjG5NcKUMi3+hmt7MH4s1ENoj bWk8ly4/vtX4dn04n8sqOH9dmqlAmayhqr7UpJ3rvIHuI3Wbuqzx4fok8 ca0d+Wq31N0AlIfsy1wzkyywB4HWfVCpT8g0bdgmFwPqm6NXQCBXoVN4y xwpvN2midCK4a5OJQRHH4+yCf7yX8aGQ3tCARM1vPvgnltLGoAUDQ5Nvs 5lZJzw0Ho7lvWON3ZY16yecVtv+4IhyF4iKc+pV5QB9EWd5VwGWPybf6U Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="403041708" X-IronPort-AV: E=Sophos;i="6.05,230,1701158400"; d="scan'208";a="403041708" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2024 14:20:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="1119412838" X-IronPort-AV: E=Sophos;i="6.05,230,1701158400"; d="scan'208";a="1119412838" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2024 14:20:42 -0800 From: Tony Luck <tony.luck@intel.com> To: Fenghua Yu <fenghua.yu@intel.com>, Reinette Chatre <reinette.chatre@intel.com>, Peter Newman <peternewman@google.com>, Jonathan Corbet <corbet@lwn.net>, Shuah Khan <skhan@linuxfoundation.org>, x86@kernel.org Cc: Shaopeng Tan <tan.shaopeng@fujitsu.com>, James Morse <james.morse@arm.com>, Jamie Iles <quic_jiles@quicinc.com>, Babu Moger <babu.moger@amd.com>, Randy Dunlap <rdunlap@infradead.org>, Drew Fustini <dfustini@baylibre.com>, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, patches@lists.linux.dev, Tony Luck <tony.luck@intel.com> Subject: [PATCH v15-RFC 1/8] x86/resctrl: Split the RDT_RESOURCE_L3 resource Date: Tue, 30 Jan 2024 14:20:27 -0800 Message-ID: <20240130222034.37181-2-tony.luck@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130222034.37181-1-tony.luck@intel.com> References: <20240126223837.21835-1-tony.luck@intel.com> <20240130222034.37181-1-tony.luck@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789555679027848209 X-GMAIL-MSGID: 1789555679027848209 |
Series |
Add support for Sub-NUMA cluster (SNC) systems
|
|
Commit Message
Luck, Tony
Jan. 30, 2024, 10:20 p.m. UTC
The RDT_RESOURCE_L3 is unique in that it is used for both monitoring
an control functions. This made sense while both uses had the same
scope. But systems with Sub-NUMA clustering enabled do not follow this
pattern.
Create a new resource: RDT_RESOURCE_L3_MON ready to take over the
monitoring functions.
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
arch/x86/kernel/cpu/resctrl/internal.h | 1 +
arch/x86/kernel/cpu/resctrl/core.c | 10 ++++++++++
2 files changed, 11 insertions(+)
Comments
Hi Tony, On 1/30/24 16:20, Tony Luck wrote: > The RDT_RESOURCE_L3 is unique in that it is used for both monitoring > an control functions. This made sense while both uses had the same an/and > scope. But systems with Sub-NUMA clustering enabled do not follow this > pattern. > > Create a new resource: RDT_RESOURCE_L3_MON ready to take over the > monitoring functions. > > Signed-off-by: Tony Luck <tony.luck@intel.com> > --- > arch/x86/kernel/cpu/resctrl/internal.h | 1 + > arch/x86/kernel/cpu/resctrl/core.c | 10 ++++++++++ > 2 files changed, 11 insertions(+) > > diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h > index 52e7e7deee10..c6051bc70e96 100644 > --- a/arch/x86/kernel/cpu/resctrl/internal.h > +++ b/arch/x86/kernel/cpu/resctrl/internal.h > @@ -429,6 +429,7 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); > extern struct dentry *debugfs_resctrl; > > enum resctrl_res_level { > + RDT_RESOURCE_L3_MON, > RDT_RESOURCE_L3, How about? RDT_RESOURCE_L3, RDT_RESOURCE_L3_MON, > RDT_RESOURCE_L2, > RDT_RESOURCE_MBA, > diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c > index aa9810a64258..c50f55d7790e 100644 > --- a/arch/x86/kernel/cpu/resctrl/core.c > +++ b/arch/x86/kernel/cpu/resctrl/core.c > @@ -60,6 +60,16 @@ mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, > #define domain_init(id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.domains) > > struct rdt_hw_resource rdt_resources_all[] = { > + [RDT_RESOURCE_L3_MON] = > + { > + .r_resctrl = { > + .rid = RDT_RESOURCE_L3_MON, > + .name = "L3", L3_MON ? > + .cache_level = 3, > + .domains = domain_init(RDT_RESOURCE_L3_MON), > + .fflags = RFTYPE_RES_CACHE, > + }, > + }, > [RDT_RESOURCE_L3] = > { > .r_resctrl = {
On Fri, Feb 09, 2024 at 09:28:16AM -0600, Moger, Babu wrote: > > enum resctrl_res_level { > > + RDT_RESOURCE_L3_MON, > > RDT_RESOURCE_L3, > > How about? > RDT_RESOURCE_L3, > RDT_RESOURCE_L3_MON, Does the order matter? I put the L3_MON one first because historically cache occupancy was the first resctrl tool. But if you have a better argument for the order, then I can change it. > > RDT_RESOURCE_L2, > > RDT_RESOURCE_MBA, > > diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c > > index aa9810a64258..c50f55d7790e 100644 > > --- a/arch/x86/kernel/cpu/resctrl/core.c > > +++ b/arch/x86/kernel/cpu/resctrl/core.c > > @@ -60,6 +60,16 @@ mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, > > #define domain_init(id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.domains) > > > > struct rdt_hw_resource rdt_resources_all[] = { > > + [RDT_RESOURCE_L3_MON] = > > + { > > + .r_resctrl = { > > + .rid = RDT_RESOURCE_L3_MON, > > + .name = "L3", > > L3_MON ? That was my first choice too. But I found: $ ls /sys/fs/resctrl/info L3 L3_MON_MON last_cmd_status MB This would be easy to fix ... just change this code to not append an extra "_MON" to the directory name: for_each_mon_capable_rdt_resource(r) { fflags = r->fflags | RFTYPE_MON_INFO; sprintf(name, "%s_MON", r->name); ret = rdtgroup_mkdir_info_resdir(r, name, fflags); if (ret) goto out_destroy; } But I also saw this: $ ls /sys/fs/resctrl/mon_data/ mon_L3_MON_00 mon_L3_MON_01 which didn't seem to have an easy fix. So I took the easy route and left the ".name" field as "L3_MON". -Tony
On 2/9/24 12:44, Tony Luck wrote: > On Fri, Feb 09, 2024 at 09:28:16AM -0600, Moger, Babu wrote: >>> enum resctrl_res_level { >>> + RDT_RESOURCE_L3_MON, >>> RDT_RESOURCE_L3, >> >> How about? >> RDT_RESOURCE_L3, >> RDT_RESOURCE_L3_MON, > > Does the order matter? I put the L3_MON one first because historically > cache occupancy was the first resctrl tool. But if you have a better > argument for the order, then I can change it. That is fine. No need to change. > >>> RDT_RESOURCE_L2, >>> RDT_RESOURCE_MBA, >>> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c >>> index aa9810a64258..c50f55d7790e 100644 >>> --- a/arch/x86/kernel/cpu/resctrl/core.c >>> +++ b/arch/x86/kernel/cpu/resctrl/core.c >>> @@ -60,6 +60,16 @@ mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, >>> #define domain_init(id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.domains) >>> >>> struct rdt_hw_resource rdt_resources_all[] = { >>> + [RDT_RESOURCE_L3_MON] = >>> + { >>> + .r_resctrl = { >>> + .rid = RDT_RESOURCE_L3_MON, >>> + .name = "L3", >> >> L3_MON ? > > That was my first choice too. But I found: > > $ ls /sys/fs/resctrl/info > L3 L3_MON_MON last_cmd_status MB > > This would be easy to fix ... just change this code to not append > an extra "_MON" to the directory name: > > for_each_mon_capable_rdt_resource(r) { > fflags = r->fflags | RFTYPE_MON_INFO; > sprintf(name, "%s_MON", r->name); > ret = rdtgroup_mkdir_info_resdir(r, name, fflags); > if (ret) > goto out_destroy; > } > > But I also saw this: > $ ls /sys/fs/resctrl/mon_data/ > mon_L3_MON_00 mon_L3_MON_01 > > which didn't seem to have an easy fix. So I took the easy route and left > the ".name" field as "L3_MON". > Ok. Sounds good.
diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 52e7e7deee10..c6051bc70e96 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -429,6 +429,7 @@ DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); extern struct dentry *debugfs_resctrl; enum resctrl_res_level { + RDT_RESOURCE_L3_MON, RDT_RESOURCE_L3, RDT_RESOURCE_L2, RDT_RESOURCE_MBA, diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index aa9810a64258..c50f55d7790e 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -60,6 +60,16 @@ mba_wrmsr_amd(struct rdt_domain *d, struct msr_param *m, #define domain_init(id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.domains) struct rdt_hw_resource rdt_resources_all[] = { + [RDT_RESOURCE_L3_MON] = + { + .r_resctrl = { + .rid = RDT_RESOURCE_L3_MON, + .name = "L3", + .cache_level = 3, + .domains = domain_init(RDT_RESOURCE_L3_MON), + .fflags = RFTYPE_RES_CACHE, + }, + }, [RDT_RESOURCE_L3] = { .r_resctrl = {