From patchwork Fri Oct 21 13:11:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6696 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690562wrr; Fri, 21 Oct 2022 06:13:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM62lukZNlMpfG3YNtFD/FjAPHmWjxTwi7Y3BCrSTAJBBTmKQkTyLJrT/AHAP1M1i2noQFw1 X-Received: by 2002:a17:90b:4b47:b0:20a:cfcb:8561 with SMTP id mi7-20020a17090b4b4700b0020acfcb8561mr21478588pjb.55.1666358000003; Fri, 21 Oct 2022 06:13:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666357999; cv=none; d=google.com; s=arc-20160816; b=IBysJVEYPFRkuN/ToJXsvDv2knesY4VKhXeR7iB9NbzmVuQTizOy4unMReIk8F3Mu+ WFzAcEAb6/QXIrKmu+YqbQkPU3GQ/kuK+bA647Wl2UC7MLlrgThMpz2VqCWdvtodQ4bt UFZ/zacLyT4OchOpW5UTXNCb3lu+i9b9k9/7Qqf8f85Pcp9HsIdYlybAuDtbOdbYv4Gt 3bnqTuFfhVBeX4nJP+U+OIIm8j8QknkM70nZKqNofPciLfQqlXttQMytnKeasSjK1A0L UbkgPUn9NVAmAf8ILhfgnpBk51TVuhoDYYnKxySvUHx+GC6sCQNW9UPtwRfl0C+9QfSM HDlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Ahm884K2hVyeyywfmDw/Ul9GywjSY+SlTfzorpSoi1Q=; b=rIop012v/cUfZ6l9sYW5bJGS/GWVuwP7T1+wDhBigaKoL5Y+uHfYNHymCa+yKetT9b 7r6ZwT2293VM1bjnsQ1ZB/Ws2ZQI6LwKgGNJvSytLtdoCUQZnAf5VWMjVE3ZV4sOhpnV 9Mt7SEgD7KA/EqXVWEGiZMkhuyPH+AQ211OeKwo1aFTRVlZI6LZQXoRp6X0C6CVAen/j 2lOc2tlXFLABba+o4gAuezeG5LEvNk3tlgsuChyP2jWU7c6geGr90lkzSSAg1McoJp9h 66e5JQwCwzZpuD8SB20huMVhIXfCxBpn38BIQzJ+o6EmJ99RSePuWeA464Os6oV6LEfP zO1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r2-20020a63ec42000000b00454bec0cb86si24954550pgj.182.2022.10.21.06.13.06; Fri, 21 Oct 2022 06:13:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230229AbiJUNMl (ORCPT + 99 others); Fri, 21 Oct 2022 09:12:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230163AbiJUNMg (ORCPT ); Fri, 21 Oct 2022 09:12:36 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 11B2D360A1 for ; Fri, 21 Oct 2022 06:12:29 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AD8BE1477; Fri, 21 Oct 2022 06:12:34 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 54F7F3F792; Fri, 21 Oct 2022 06:12:25 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 01/18] x86/resctrl: Track the closid with the rmid Date: Fri, 21 Oct 2022 13:11:47 +0000 Message-Id: <20221021131204.5581-2-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303006068748916?= X-GMAIL-MSGID: =?utf-8?q?1747303006068748916?= x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID. MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID. i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups. To allow rescrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86. Signed-off-by: James Morse Reviewed-by: Xin Hao Reviewed-by: Xin Hao --- Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing... --- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 54 +++++++++++++---------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++--- include/linux/resctrl.h | 11 ++++- 5 files changed, 49 insertions(+), 34 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 5f7128686cfd..4b243ba88882 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -519,7 +519,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); void closid_free(int closid); int alloc_rmid(void); -void free_rmid(u32 rmid); +void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); void mon_event_count(void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index efe0c30d3a12..f1f66c9942a5 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -25,6 +25,7 @@ #include "internal.h" struct rmid_entry { + u32 closid; u32 rmid; int busy; struct list_head list; @@ -136,7 +137,7 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val) return val; } -static inline struct rmid_entry *__rmid_entry(u32 rmid) +static inline struct rmid_entry *__rmid_entry(u32 closid, u32 rmid) { struct rmid_entry *entry; @@ -166,7 +167,8 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom, } void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid) + u32 closid, u32 rmid, + enum resctrl_event_id eventid) { struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); struct arch_mbm_state *am; @@ -185,7 +187,8 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) } int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid, u64 *val) + u32 closid, u32 rmid, enum resctrl_event_id eventid, + u64 *val) { struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); @@ -251,9 +254,9 @@ void __check_limbo(struct rdt_domain *d, bool force_free) if (nrmid >= r->num_rmid) break; - entry = __rmid_entry(nrmid); + entry = __rmid_entry(~0, nrmid); // temporary - if (resctrl_arch_rmid_read(r, d, entry->rmid, + if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val)) { rmid_dirty = true; } else { @@ -308,7 +311,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) cpu = get_cpu(); list_for_each_entry(d, &r->domains, list) { if (cpumask_test_cpu(cpu, &d->cpu_mask)) { - err = resctrl_arch_rmid_read(r, d, entry->rmid, + err = resctrl_arch_rmid_read(r, d, entry->closid, + entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val); if (err || val <= resctrl_rmid_realloc_threshold) @@ -332,7 +336,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) list_add_tail(&entry->list, &rmid_free_lru); } -void free_rmid(u32 rmid) +void free_rmid(u32 closid, u32 rmid) { struct rmid_entry *entry; @@ -341,7 +345,7 @@ void free_rmid(u32 rmid) lockdep_assert_held(&rdtgroup_mutex); - entry = __rmid_entry(rmid); + entry = __rmid_entry(closid, rmid); if (is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); @@ -349,15 +353,16 @@ void free_rmid(u32 rmid) list_add_tail(&entry->list, &rmid_free_lru); } -static int __mon_event_count(u32 rmid, struct rmid_read *rr) +static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) { struct mbm_state *m; u64 tval = 0; if (rr->first) - resctrl_arch_reset_rmid(rr->r, rr->d, rmid, rr->evtid); + resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid); - rr->err = resctrl_arch_rmid_read(rr->r, rr->d, rmid, rr->evtid, &tval); + rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid, + &tval); if (rr->err) return rr->err; @@ -400,7 +405,7 @@ static int __mon_event_count(u32 rmid, struct rmid_read *rr) * __mon_event_count() is compared with the chunks value from the previous * invocation. This must be called once per second to maintain values in MBps. */ -static void mbm_bw_count(u32 rmid, struct rmid_read *rr) +static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) { struct mbm_state *m = &rr->d->mbm_local[rmid]; u64 cur_bw, bytes, cur_bytes; @@ -430,7 +435,7 @@ void mon_event_count(void *info) rdtgrp = rr->rgrp; - ret = __mon_event_count(rdtgrp->mon.rmid, rr); + ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr); /* * For Ctrl groups read data from child monitor groups and @@ -441,7 +446,8 @@ void mon_event_count(void *info) if (rdtgrp->type == RDTCTRL_GROUP) { list_for_each_entry(entry, head, mon.crdtgrp_list) { - if (__mon_event_count(entry->mon.rmid, rr) == 0) + if (__mon_event_count(rdtgrp->closid, entry->mon.rmid, + rr) == 0) ret = 0; } } @@ -571,7 +577,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) +static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, + u32 closid, u32 rmid) { struct rmid_read rr; @@ -586,12 +593,12 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) if (is_mbm_total_enabled()) { rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID; rr.val = 0; - __mon_event_count(rmid, &rr); + __mon_event_count(closid, rmid, &rr); } if (is_mbm_local_enabled()) { rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID; rr.val = 0; - __mon_event_count(rmid, &rr); + __mon_event_count(closid, rmid, &rr); /* * Call the MBA software controller only for the @@ -599,7 +606,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) * the software controller explicitly. */ if (is_mba_sc(NULL)) - mbm_bw_count(rmid, &rr); + mbm_bw_count(closid, rmid, &rr); } } @@ -656,11 +663,11 @@ void mbm_handle_overflow(struct work_struct *work) d = container_of(work, struct rdt_domain, mbm_over.work); list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) { - mbm_update(r, d, prgrp->mon.rmid); + mbm_update(r, d, prgrp->closid, prgrp->mon.rmid); head = &prgrp->mon.crdtgrp_list; list_for_each_entry(crgrp, head, mon.crdtgrp_list) - mbm_update(r, d, crgrp->mon.rmid); + mbm_update(r, d, crgrp->closid, crgrp->mon.rmid); if (is_mba_sc(NULL)) update_mba_bw(prgrp, d); @@ -703,10 +710,11 @@ static int dom_data_init(struct rdt_resource *r) } /* - * RMID 0 is special and is always allocated. It's used for all - * tasks that are not monitored. + * RMID 0 is special and is always allocated. It's used for the + * default_rdtgroup control group, which will be setup later. See + * rdtgroup_setup_root(). */ - entry = __rmid_entry(0); + entry = __rmid_entry(0, 0); list_del(&entry->list); return 0; diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index d961ae3ed96e..4d3706f71ee3 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -738,7 +738,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp) * anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); ret = 0; goto out; @@ -773,7 +773,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret = rdtgroup_locksetup_user_restore(rdtgrp); if (ret) { - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); return ret; } diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index e5a48f05e787..f3b739c52e42 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2422,7 +2422,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list; list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) { - free_rmid(sentry->mon.rmid); + free_rmid(sentry->closid, sentry->mon.rmid); list_del(&sentry->mon.crdtgrp_list); if (atomic_read(&sentry->waitcount) != 0) @@ -2462,7 +2462,7 @@ static void rmdir_all_sub(void) cpumask_or(&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); kernfs_remove(rdtgrp->kn); list_del(&rdtgrp->rdtgroup_list); @@ -2955,7 +2955,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, return 0; out_idfree: - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); out_destroy: kernfs_put(rdtgrp->kn); kernfs_remove(rdtgrp->kn); @@ -2969,7 +2969,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp) { kernfs_remove(rgrp->kn); - free_rmid(rgrp->mon.rmid); + free_rmid(rgrp->closid, rgrp->mon.rmid); rdtgroup_remove(rgrp); } @@ -3118,7 +3118,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask) update_closid_rmid(tmpmask, NULL); rdtgrp->flags = RDT_DELETED; - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); /* * Remove the rdtgrp from the parent ctrl_mon group's list @@ -3164,8 +3164,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask) cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask); update_closid_rmid(tmpmask, NULL); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); closid_free(rdtgrp->closid); - free_rmid(rdtgrp->mon.rmid); rdtgroup_ctrl_remove(rdtgrp); diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 0cf5b20c6ddf..641aea580a1f 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -225,6 +225,8 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * for this resource and domain. * @r: resource that the counter should be read from. * @d: domain that the counter should be read from. + * @closid: closid that matches the rmid. The counter may + * match traffic of both closid and rmid, or rmid only. * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. @@ -235,20 +237,25 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * 0 on success, or -EIO, -EINVAL etc on error. */ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid, u64 *val); + u32 closid, u32 rmid, enum resctrl_event_id eventid, + u64 *val); + /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. * @r: The domain's resource. * @d: The rmid's domain. + * @closid: The closid that matches the rmid. Counters may match both + * closid and rmid, or rmid only. * @rmid: The rmid whose counter values should be reset. * @eventid: The eventid whose counter values should be reset. * * This can be called from any CPU. */ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid); + u32 closid, u32 rmid, + enum resctrl_event_id eventid); extern unsigned int resctrl_rmid_realloc_threshold; extern unsigned int resctrl_rmid_realloc_limit; From patchwork Fri Oct 21 13:11:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6695 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690536wrr; Fri, 21 Oct 2022 06:13:28 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5fyFIJqAAK1j1NI5khniSJ0TwyVKtyF0l2QpxyJ600v1MYkoCp+lvorTzMb7empuLVa1CY X-Received: by 2002:a05:6a00:218b:b0:569:a7c5:4cf7 with SMTP id h11-20020a056a00218b00b00569a7c54cf7mr5369097pfi.27.1666358008380; Fri, 21 Oct 2022 06:13:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358008; cv=none; d=google.com; s=arc-20160816; b=gLXrkySPnGbaklBUU/hsisKF2uHEsgLIsqXp7PWX1+hjyMJD9gL8oan0/tCUh8mmeK 4e8ERlw1X5mDDBQooa3TTOP3rYiiKDnKyIVPp4we7e1eIEY0eaWuD6UC45CoLCZVa4Ju 5HEhgFtaRmPECtVE/laOy8OanNO4v2Xkcdac4p+t5ZPU/k9vniMKkPcgPPRXXzKNi2iD chmEAkf5PHJJz02iBo2YqTycrdXTyqTMw9Q3ZiZc3IDWcyZ7fVp9iyO6Pvg3aE6oKcaa 7fEPf6GYbgKspB2nU35SjxR7eD8xZfADdyCrDMhe4GKfXnW+0phomBHvvyVKqhp+h29d P+4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=ZO6XDPpaHg90pPrG4ROdUP6qUSclNZhjiR1E0MFBG3o=; b=SeT6IvKgxGJ+zS0t0LsYqbo5Yh0N+LUbAhDEUfuZeEBIuDwBHXLN93HwoU06LCaT7F KYn+Q5F6DUnsldIEp5BR7Rv76s+KB444I9MmGl2b2GvfakMQbCJtQZnNy9E9sD97FlGV GGgnOkuoyl1jsTJXRs9oWKdSqc0WO3ns9FIuDQ797D09MeORWMJgOMq9MkUf9Jj2RO4B YRbBl1pgZDwLPIqJH3bi3t5k6sGtk9Ktu2gN79QqQzo+Fd5H1zbgISNOI7p2rRl9Lcax tDaZVPKrhejSO4WwhftdhxxrimllGcw63zCzXEoUIts0VErxDEC0Yp9uZEJMeD+89mfj S3iA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 186-20020a6301c3000000b0043c474c8942si25663744pgb.673.2022.10.21.06.13.14; Fri, 21 Oct 2022 06:13:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230331AbiJUNMy (ORCPT + 99 others); Fri, 21 Oct 2022 09:12:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230254AbiJUNMi (ORCPT ); Fri, 21 Oct 2022 09:12:38 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2D56526DB1A for ; Fri, 21 Oct 2022 06:12:34 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A13C11042; Fri, 21 Oct 2022 06:12:39 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CD25E3F792; Fri, 21 Oct 2022 06:12:28 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 02/18] x86/resctrl: Access per-rmid structures by index Date: Fri, 21 Oct 2022 13:11:48 +0000 Message-Id: <20221021131204.5581-3-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303015404614667?= X-GMAIL-MSGID: =?utf-8?q?1747303015404614667?= Because of the differences between Intel RDT/AMD QoS and Arm's MPAM monitors, RMID values on arm64 are not unique unless the CLOSID is also included. Bitmaps like rmid_busy_llc need to be sized by the number of unique entries for this resource. Add helpers to encode/decode the CLOSID and RMID to an index. The domain's busy_rmid_llc and the rmid_ptrs[] array are then sized by index. On x86, this is always just the RMID. This gives resctrl a unique value it can use to store monitor values, and allows MPAM to decode the closid when reading the hardware counters. Signed-off-by: James Morse --- arch/x86/include/asm/resctrl.h | 17 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 + arch/x86/kernel/cpu/resctrl/monitor.c | 75 +++++++++++++++++--------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 7 +-- 4 files changed, 72 insertions(+), 29 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index d24b04ebf950..523eabfa3193 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -96,6 +96,23 @@ static inline void resctrl_sched_in(void) __resctrl_sched_in(); } +static inline u32 resctrl_arch_system_num_rmid_idx(void) +{ + /* RMID are independent numbers for x86. num_rmid_idx==num_rmid */ + return boot_cpu_data.x86_cache_max_rmid + 1; +} + +static inline void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid) +{ + *rmid = idx; + *closid = ~0; +} + +static inline u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) +{ + return rmid; +} + void resctrl_cpu_detect(struct cpuinfo_x86 *c); #else diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 4b243ba88882..cb94c3e3fe36 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -8,6 +8,8 @@ #include #include +#include + #define MSR_IA32_L3_QOS_CFG 0xc81 #define MSR_IA32_L2_QOS_CFG 0xc82 #define MSR_IA32_L3_CBM_BASE 0xc90 diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index f1f66c9942a5..c95d259476d4 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -137,11 +137,24 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val) return val; } -static inline struct rmid_entry *__rmid_entry(u32 closid, u32 rmid) +/* + * x86 and arm64 differ in their handling of monitoring. + * x86's RMID are an independent number, there is one RMID '1'. + * arm64's PMG extend the PARTID/CLOSID space, there is one RMID '1' for each + * CLOSID. The RMID is no longer unique. + * To account for this, resctrl uses an index. On x86 this is just the RMID, + * on arm64 it encodes the CLOSID and RMID. This gives a unique number. + * + * The domain's rmid_busy_llc and rmid_ptrs are sized by index. The arch code + * must accept an attempt to read every index. + */ +static inline struct rmid_entry *__rmid_entry(u32 idx) { struct rmid_entry *entry; + u32 closid, rmid; - entry = &rmid_ptrs[rmid]; + entry = &rmid_ptrs[idx]; + resctrl_arch_rmid_idx_decode(idx, &closid, &rmid); WARN_ON(entry->rmid != rmid); return entry; @@ -238,8 +251,9 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, void __check_limbo(struct rdt_domain *d, bool force_free) { struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry; - u32 crmid = 1, nrmid; + u32 idx, cur_idx = 1; bool rmid_dirty; u64 val = 0; @@ -250,12 +264,11 @@ void __check_limbo(struct rdt_domain *d, bool force_free) * RMID and move it to the free list when the counter reaches 0. */ for (;;) { - nrmid = find_next_bit(d->rmid_busy_llc, r->num_rmid, crmid); - if (nrmid >= r->num_rmid) + idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx); + if (idx >= idx_limit) break; - entry = __rmid_entry(~0, nrmid); // temporary - + entry = __rmid_entry(idx); if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val)) { rmid_dirty = true; @@ -264,19 +277,21 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } if (force_free || !rmid_dirty) { - clear_bit(entry->rmid, d->rmid_busy_llc); + clear_bit(idx, d->rmid_busy_llc); if (!--entry->busy) { rmid_limbo_count--; list_add_tail(&entry->list, &rmid_free_lru); } } - crmid = nrmid + 1; + cur_idx = idx + 1; } } bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d) { - return find_first_bit(d->rmid_busy_llc, r->num_rmid) != r->num_rmid; + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); + + return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit; } /* @@ -306,6 +321,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) struct rdt_domain *d; int cpu, err; u64 val = 0; + u32 idx; + + idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); entry->busy = 0; cpu = get_cpu(); @@ -325,7 +343,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) */ if (!has_busy_rmid(r, d)) cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL); - set_bit(entry->rmid, d->rmid_busy_llc); + set_bit(idx, d->rmid_busy_llc); entry->busy++; } put_cpu(); @@ -338,14 +356,16 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) void free_rmid(u32 closid, u32 rmid) { + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); struct rmid_entry *entry; - if (!rmid) - return; - lockdep_assert_held(&rdtgroup_mutex); - entry = __rmid_entry(closid, rmid); + /* do not allow the default rmid to be free'd */ + if (!idx) + return; + + entry = __rmid_entry(idx); if (is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); @@ -355,6 +375,7 @@ void free_rmid(u32 closid, u32 rmid) static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) { + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); struct mbm_state *m; u64 tval = 0; @@ -371,10 +392,10 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) rr->val += tval; return 0; case QOS_L3_MBM_TOTAL_EVENT_ID: - m = &rr->d->mbm_total[rmid]; + m = &rr->d->mbm_total[idx]; break; case QOS_L3_MBM_LOCAL_EVENT_ID: - m = &rr->d->mbm_local[rmid]; + m = &rr->d->mbm_local[idx]; break; default: /* @@ -407,7 +428,8 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) */ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) { - struct mbm_state *m = &rr->d->mbm_local[rmid]; + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); + struct mbm_state *m = &rr->d->mbm_local[idx]; u64 cur_bw, bytes, cur_bytes; cur_bytes = rr->val; @@ -497,7 +519,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) { u32 closid, rmid, cur_msr_val, new_msr_val; struct mbm_state *pmbm_data, *cmbm_data; - u32 cur_bw, delta_bw, user_bw; + u32 cur_bw, delta_bw, user_bw, idx; struct rdt_resource *r_mba; struct rdt_domain *dom_mba; struct list_head *head; @@ -510,7 +532,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) closid = rgrp->closid; rmid = rgrp->mon.rmid; - pmbm_data = &dom_mbm->mbm_local[rmid]; + idx = resctrl_arch_rmid_idx_encode(closid, rmid); + pmbm_data = &dom_mbm->mbm_local[idx]; dom_mba = get_domain_from_cpu(smp_processor_id(), r_mba); if (!dom_mba) { @@ -693,19 +716,19 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) static int dom_data_init(struct rdt_resource *r) { + u32 nr_idx = resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry = NULL; - int i, nr_rmids; + int i; - nr_rmids = r->num_rmid; - rmid_ptrs = kcalloc(nr_rmids, sizeof(struct rmid_entry), GFP_KERNEL); + rmid_ptrs = kcalloc(nr_idx, sizeof(struct rmid_entry), GFP_KERNEL); if (!rmid_ptrs) return -ENOMEM; - for (i = 0; i < nr_rmids; i++) { + for (i = 0; i < nr_idx; i++) { entry = &rmid_ptrs[i]; INIT_LIST_HEAD(&entry->list); - entry->rmid = i; + resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid); list_add_tail(&entry->list, &rmid_free_lru); } @@ -714,7 +737,7 @@ static int dom_data_init(struct rdt_resource *r) * default_rdtgroup control group, which will be setup later. See * rdtgroup_setup_root(). */ - entry = __rmid_entry(0, 0); + entry = __rmid_entry(resctrl_arch_rmid_idx_encode(0, 0)); list_del(&entry->list); return 0; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index f3b739c52e42..9ce4746778f4 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3320,16 +3320,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) { + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); size_t tsize; if (is_llc_occupancy_enabled()) { - d->rmid_busy_llc = bitmap_zalloc(r->num_rmid, GFP_KERNEL); + d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL); if (!d->rmid_busy_llc) return -ENOMEM; } if (is_mbm_total_enabled()) { tsize = sizeof(*d->mbm_total); - d->mbm_total = kcalloc(r->num_rmid, tsize, GFP_KERNEL); + d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_total) { bitmap_free(d->rmid_busy_llc); return -ENOMEM; @@ -3337,7 +3338,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) } if (is_mbm_local_enabled()) { tsize = sizeof(*d->mbm_local); - d->mbm_local = kcalloc(r->num_rmid, tsize, GFP_KERNEL); + d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_local) { bitmap_free(d->rmid_busy_llc); kfree(d->mbm_total); From patchwork Fri Oct 21 13:11:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6697 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690614wrr; Fri, 21 Oct 2022 06:13:35 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4dqCwBHWKItDJYGZPTYXme5qF1kK55Wa+c3ObAh+mRvWzSt7sLCNuDmC+m8o6nFFNsqqRm X-Received: by 2002:a17:903:1250:b0:185:40c6:3c2c with SMTP id u16-20020a170903125000b0018540c63c2cmr19497759plh.64.1666358015053; Fri, 21 Oct 2022 06:13:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358015; cv=none; d=google.com; s=arc-20160816; b=n/SPj/QfGun4aaKK3LNVlAXclBxVJ5rc5i5nk9/QYuoL7uGw8JdNClW6AOl6FNBcTN Xw2+D5ls4eIff4tiZJqHQy5Q6OhnBFTd3KCqUsDLXECUoJwgkgMlOWprP2Mz7Bjx1CaT r0J7Z1m9cwv6L/ERCIoTMQjoZj0LhLjBeOatLpmMyRo46oIDCD9Wn4Rmi2n1qI72Lh51 zhPya7gCkgIUecRRQTHIFCMULtfmn9j2Qgt837oaIONMa/fqasX5rEfrM4AHVrO21rDK lQHp9Ja6V1oSLz2+utNybR+nh+hq+4kEcK1aD2jOdJgjJj85t16OaNjGlVLIJey7niE9 yAnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=EILKmT0Eh383MOLQwvVXC0UXgAgyas8vUmN9sOpBfiQ=; b=ZCRGkVND6IlxWdHXQ9g8Qj3rj3IgbRutkKWgJC9wEpJAk/D08n2wwxyu00UpY+GMpm jWAfEcGan2LiA/ajg/j7CaVtYFLqwNJYNpKU6/b419ZRiXW1BArWwMX1LWeVC7xWQXbY W3yBGFr0kpvxV5GJAd5pKhEl3OnZ8oGaGxoJJ9pRxpizW2n5PTIYzSAwB8qB1TXQzdHQ wFOpk7/c0eUGqW3phb4CpGg1b0G1jKaRMFEctYjRx70/qOu04LhMK+/hwswLA3dLT0BQ /f4ItAenH5zOliINOAL33eAcZIWgjSyJ0+kK2iN8Bc4ja7kDuttofL/O1h5M5NSKEpri 4i4A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d10-20020a056a00198a00b005633c373dcbsi25810619pfl.147.2022.10.21.06.13.21; Fri, 21 Oct 2022 06:13:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230232AbiJUNND (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230300AbiJUNMn (ORCPT ); Fri, 21 Oct 2022 09:12:43 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 38FB3263F14 for ; Fri, 21 Oct 2022 06:12:39 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6958D1480; Fri, 21 Oct 2022 06:12:44 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CCA373F792; Fri, 21 Oct 2022 06:12:33 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 03/18] x86/resctrl: Create helper for RMID allocation and mondata dir creation Date: Fri, 21 Oct 2022 13:11:49 +0000 Message-Id: <20221021131204.5581-4-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303022329122929?= X-GMAIL-MSGID: =?utf-8?q?1747303022329122929?= RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID. MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has. The RMID allocation needs to move to be after the CLOSID has been allocated. To make a subsequent change that does this easier to read, move the RMID allocation and mondata dir creation to a helper. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 42 +++++++++++++++++--------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 9ce4746778f4..841294ad6263 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2868,6 +2868,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) return 0; } +static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) +{ + int ret; + + if (!rdt_mon_capable) + return 0; + + ret = alloc_rmid(); + if (ret < 0) { + rdt_last_cmd_puts("Out of RMIDs\n"); + return ret; + } + rdtgrp->mon.rmid = ret; + + ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); + if (ret) { + rdt_last_cmd_puts("kernfs subdir error\n"); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + return ret; + } + + return 0; +} + static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, const char *name, umode_t mode, enum rdt_group_type rtype, struct rdtgroup **r) @@ -2933,20 +2957,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - if (rdt_mon_capable) { - ret = alloc_rmid(); - if (ret < 0) { - rdt_last_cmd_puts("Out of RMIDs\n"); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); + if (ret) + goto out_destroy; - ret = mkdir_mondata_all(kn, rdtgrp, &rdtgrp->mon.mon_data_kn); - if (ret) { - rdt_last_cmd_puts("kernfs subdir error\n"); - goto out_idfree; - } - } kernfs_activate(kn); /* @@ -2954,8 +2968,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, */ return 0; -out_idfree: - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); out_destroy: kernfs_put(rdtgrp->kn); kernfs_remove(rdtgrp->kn); From patchwork Fri Oct 21 13:11:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6698 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690668wrr; Fri, 21 Oct 2022 06:13:40 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7hu87RacOWi6F9ft0icvvgY1iiCBPRwHNRsSCWRyJmDsYqKTufMlUFDphPSQkHYb7xYywW X-Received: by 2002:aa7:9e0f:0:b0:562:7faf:e281 with SMTP id y15-20020aa79e0f000000b005627fafe281mr19324734pfq.58.1666358019900; Fri, 21 Oct 2022 06:13:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358019; cv=none; d=google.com; s=arc-20160816; b=YJX1K9zGtr9394UCQzaJF0HRQQFOiy7PmVVuilEXBWgoM4uNyP+qfpHY0SkwBcXSSF kAJJYoS0SwBgrVn0HvkkD626dvrDU2zmm11yb+5K5b8wCHVSTnq+Gr1QKLsGrfKxtsCb osCmlDJHhrPqzVh6eT15KuETWD12Kwwkl0ET+zYyS00mzWYn2suW48QnNIBMBIESAePo rMKtQT68/ld3sF4zO4mPX8E0ex+vOAYM0vwldU2uR81aU1BQizZyNTg3kI1FRKXbADE0 MkeZfawvSteRTb6D0+b1rO1Rv2HTZtVtP3HPO/yFx08BgwjjW1j5jEEpNvmrtbAf0xnV e70A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=bJyVlh/YYYvgqjKz85TevJrh6cWtEPyGfDaJXdqpVCA=; b=QkUyj/cSjfonPispPOGSMJXngSn3P1/8uZKQjrEf0TED+M1mqWXRNpMbih0g8j6vQn PpGb1UBWE8cyMlQUzZYZxVXTyhK2tsDZiImARvI3Q53hb1s4zsAoN66HMao1UCEGoIUG p0/d+28ChMspV49SMTrvy8zHVXXLRm71YCW2ZWEyFVBgl/fR7/Ye4/35a/F131CgX9Td RJvTod7nm7CiLQiWgMr3bMBb+PzGZkyU6GmMdJIVUI0eEVTNVICOKqmaw0BCxIS+VH9v zYdZIjT9yryFLfPMQAYPcrC58fih76ZQi452CTKjaHT3yBbRYvHI4IrWEQHhJoCsOPuB aXbQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p13-20020a056a000b4d00b0054ed3047ee2si25859929pfo.22.2022.10.21.06.13.26; Fri, 21 Oct 2022 06:13:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230108AbiJUNNF (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230314AbiJUNMv (ORCPT ); Fri, 21 Oct 2022 09:12:51 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4134826DB06 for ; Fri, 21 Oct 2022 06:12:41 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4B6E41477; Fri, 21 Oct 2022 06:12:47 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 967513F792; Fri, 21 Oct 2022 06:12:38 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 04/18] x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare() Date: Fri, 21 Oct 2022 13:11:50 +0000 Message-Id: <20221021131204.5581-5-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303027037632176?= X-GMAIL-MSGID: =?utf-8?q?1747303027037632176?= RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID. MPAM's equivalent of RMID are not an independent number, so can't be allocated until the closid is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has. The RMID allocation needs to move to be after the CLOSID has been allocated. Move the RMID allocation out of mkdir_rdt_prepare() to occur in its caller, after the mkdir_rdt_prepare() call. This allows the RMID allocator to know the CLOSID. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 29 +++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 841294ad6263..c67083a8a5f5 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2892,6 +2892,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) return 0; } +static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp) +{ + if (rdt_mon_capable) + free_rmid(rgrp->closid, rgrp->mon.rmid); +} + static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, const char *name, umode_t mode, enum rdt_group_type rtype, struct rdtgroup **r) @@ -2957,10 +2963,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); - if (ret) - goto out_destroy; - kernfs_activate(kn); /* @@ -2981,7 +2983,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp) { kernfs_remove(rgrp->kn); - free_rmid(rgrp->closid, rgrp->mon.rmid); rdtgroup_remove(rgrp); } @@ -3003,12 +3004,19 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, prgrp = rdtgrp->mon.parent; rdtgrp->closid = prgrp->closid; + ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); + if (ret) { + mkdir_rdt_prepare_clean(rdtgrp); + goto out_unlock; + } + /* * Add the rdtgrp to the list of rdtgrps the parent * ctrl_mon group has to track. */ list_add_tail(&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list); +out_unlock: rdtgroup_kn_unlock(parent_kn); return ret; } @@ -3039,10 +3047,15 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret = 0; rdtgrp->closid = closid; - ret = rdtgroup_init_alloc(rdtgrp); - if (ret < 0) + + ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); + if (ret) goto out_id_free; + ret = rdtgroup_init_alloc(rdtgrp); + if (ret < 0) + goto out_rmid_free; + list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups); if (rdt_mon_capable) { @@ -3061,6 +3074,8 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, out_del_list: list_del(&rdtgrp->rdtgroup_list); +out_rmid_free: + mkdir_rdt_prepare_rmid_free(rdtgrp); out_id_free: closid_free(closid); out_common_fail: From patchwork Fri Oct 21 13:11:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6699 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690736wrr; Fri, 21 Oct 2022 06:13:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6YOj8uHX2UvuAj/fGDR4JI/1yGefwwoj5qxvsLvBoITvUvEWyteo33wMaNT18QYhbt7Wy0 X-Received: by 2002:a17:90b:380b:b0:20d:7364:796f with SMTP id mq11-20020a17090b380b00b0020d7364796fmr21536516pjb.13.1666358026634; Fri, 21 Oct 2022 06:13:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358026; cv=none; d=google.com; s=arc-20160816; b=i2pHAiTqqMPHm1+p3EaKkQ58+Nl5ULmB5eXpTcjgKSJYhFfQCyeAwx6Rf+5Kp/0qbW gLZHvaXPwjqAEg324OQxKezmFhNqJXLACQ19JkGbE9g9Zad93ey72Y1OfGfPz8G1eWoL QS71W4HcWL6YGEr56DkSQv3P7x4tTw+TE/q9cgBa6cv8YfsW1upDBngZZWMgjdNEeZ9f dsfp1+A28fs1I5B2TSVUDEBM/2w38oweESkQCB3LYqw4APUphCcU44j7tBouX56hnweM SBSVT5Zxceb4eZe32y/awHTrUv8k36ZxoD4hhgzzVj+d+yD427S461k5zyKz+OSLaTxV he9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=lXjNcN8PgBr6PAB+9wphYnmmmXpIaw2HGk6HPKZVvHg=; b=A5dKll7TMqfP9CvDNmhaFj4BUY2tvRKyTgw5OE8T9W29GSMwoUW4YESE9m8AokhamV lvfGVqQNt+05Irrc6bxqgSmkBkAIicW2fqH+a9b3J8WwOizjdGvnzKNUW1iGSgcFBni1 Piwtf20AEj2tu72D9CBXmj2CzlMvRROmrfrLKJrMc2ekK133H3gF8AuzQU0hBW8QW32b jQL92qnjxqCfYr4mJ5eC7L01yHGBn814zcTqxdszAPkWOH3CPtcpgZQlPUCuRi+dhnQg bmuuDMjKvpR/d37ufROj9tcD3BE0zMUgw0ckPvFXR2WBwSE9cbFnPeY+WGA05w0mg4Mj vpEg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d2-20020a170902cec200b00172696f35e1si28228597plg.482.2022.10.21.06.13.33; Fri, 21 Oct 2022 06:13:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230340AbiJUNNI (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230317AbiJUNMv (ORCPT ); Fri, 21 Oct 2022 09:12:51 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 064AA27083F for ; Fri, 21 Oct 2022 06:12:44 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E400F1042; Fri, 21 Oct 2022 06:12:49 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 46B273F792; Fri, 21 Oct 2022 06:12:41 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 05/18] x86/resctrl: Allow RMID allocation to be scoped by CLOSID Date: Fri, 21 Oct 2022 13:11:51 +0000 Message-Id: <20221021131204.5581-6-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303033834810385?= X-GMAIL-MSGID: =?utf-8?q?1747303033834810385?= MPAMs RMID values are not unique unless the CLOSID is considered as well. alloc_rmid() expects the RMID to be an independent number. Pass the CLOSID in to alloc_rmid(). Use this to compare indexes when allocating. If the CLOSID is not relevant to the index, this ends up comparing the free RMID with itself, and the first free entry will be used. With MPAM the CLOSID is included in the index, so this becomes a walk of the free RMID entries, until one that matches the supplied CLOSID is found. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 44 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +- 4 files changed, 38 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index cb94c3e3fe36..c8c46fe088be 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -520,7 +520,7 @@ void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp); struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); void closid_free(int closid); -int alloc_rmid(void); +int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); void mon_event_count(void *info); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index c95d259476d4..59da256a77fe 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -294,25 +294,51 @@ bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d) return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit; } +static struct rmid_entry *resctrl_find_free_rmid(u32 closid) +{ + struct rmid_entry *itr; + u32 itr_idx, cmp_idx; + + if (list_empty(&rmid_free_lru)) + return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC); + + list_for_each_entry(itr, &rmid_free_lru, list) { + /* + * get the index of this free RMID, and the index it would need + * to be if it were used with this CLOSID. + * If the CLOSID is irrelevant on this architecture, these will + * always be the same. Otherwise they will only match if this + * RMID can be used with this CLOSID. + */ + itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid); + cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid); + + if (itr_idx == cmp_idx) + return itr; + } + + return ERR_PTR(-ENOSPC); +} + /* - * As of now the RMIDs allocation is global. + * As of now the RMIDs allocation is the same in each domain. * However we keep track of which packages the RMIDs * are used to optimize the limbo list management. + * The closid is ignored on x86. */ -int alloc_rmid(void) +int alloc_rmid(u32 closid) { struct rmid_entry *entry; lockdep_assert_held(&rdtgroup_mutex); - if (list_empty(&rmid_free_lru)) - return rmid_limbo_count ? -EBUSY : -ENOSPC; + entry = resctrl_find_free_rmid(closid); + if (!IS_ERR(entry)) { + list_del(&entry->list); + return entry->rmid; + } - entry = list_first_entry(&rmid_free_lru, - struct rmid_entry, list); - list_del(&entry->list); - - return entry->rmid; + return PTR_ERR(entry); } static void add_rmid_to_limbo(struct rmid_entry *entry) diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 4d3706f71ee3..acebbc35331f 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -763,7 +763,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) int ret; if (rdt_mon_capable) { - ret = alloc_rmid(); + ret = alloc_rmid(rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); return ret; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index c67083a8a5f5..ac88610a6946 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2875,7 +2875,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) if (!rdt_mon_capable) return 0; - ret = alloc_rmid(); + ret = alloc_rmid(rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); return ret; From patchwork Fri Oct 21 13:11:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6700 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690804wrr; Fri, 21 Oct 2022 06:13:53 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5N6Z+IxNUoxKZ8OS+952WvetipFOlymkClZLPTL1bYRzMlEwUq8bgeQM31E21A/BgnemNU X-Received: by 2002:a17:90b:4c42:b0:20d:7820:2e4e with SMTP id np2-20020a17090b4c4200b0020d78202e4emr57946618pjb.40.1666358032675; Fri, 21 Oct 2022 06:13:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358032; cv=none; d=google.com; s=arc-20160816; b=GyIi4Z3vo4OiFc0UVFVc8nji8mIIcM+Tu2MKiYRIuPnQyBhWvxNRG9k2oYIG6HQgxS 3cxTL3igCg9AJpkttfZAexT+MuumrqDZTjhPJgu+iAQJe+my3LemjwCIKljmSrgdVutN iJKIbCw68sJXpQj0vbQcU4/M7DM9FT+dm+7T9GVPsqNG13jwCYfNY3mlBEzSD78qJxGM eOjJ8QdcTqgGTuTWN/evzqsy7zupWNPAOkKSsmLsGAkOUbrQ7jThIMMj9hXiP1hpMIsI 6eLuoOvLCw4kIB20s/px7GJsrKalfE/DKfQLOdDwT6QgnSH55otz7YHntnx3J27FPcqH iutQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=4wkIYtHR8lkmbHJqFHsi4XLYvdtZ3D4UZqtzR7yPwvU=; b=WVCgQV6E3BnqSnOXVwCwcXHA1nypYEFjE+CUdC5uUAIw6D3iXnc7QdqtfzanQGn3Oi mNs/voNLH3yx+8FcouFSDUHDPs2t0VClZ0CVlA6S4ZMA/oyAv9qw61W2VfVq+lKiB84g pZyaW1YBNMA6BHM8/UjDZbtjoZFT12Oo+RNKmvmgxRmH7MVesJAqYByT4fY5rtDiT4FQ hSNSw2xKmSNDl0F0ZCdCiBEHapJ3lAdRRF8T9zeeid1LH8o85pWo3cun7SRLgfmcLyMF ce8bwnUMlv4mTKHJNyx0SHKqkYiGfuGXl34WNsZ13sy4wxvxaqwOt1K3v2PPeoMNnqdU XMgw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id rm9-20020a17090b3ec900b00200642481bdsi7170241pjb.11.2022.10.21.06.13.38; Fri, 21 Oct 2022 06:13:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230348AbiJUNNN (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230347AbiJUNM4 (ORCPT ); Fri, 21 Oct 2022 09:12:56 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 19B8B273556 for ; Fri, 21 Oct 2022 06:12:47 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8943E1BF3; Fri, 21 Oct 2022 06:12:52 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DF69E3F792; Fri, 21 Oct 2022 06:12:43 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 06/18] x86/resctrl: Allow the allocator to check if a CLOSID can allocate clean RMID Date: Fri, 21 Oct 2022 13:11:52 +0000 Message-Id: <20221021131204.5581-7-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303040506065758?= X-GMAIL-MSGID: =?utf-8?q?1747303040506065758?= MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups. This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo. Add a helper to allow the CLOSID allocator to check if a CLOSID has dirty RMID values. This behaviour is enabled by a kconfig option selected by the architecture, which avoids a pointless search for x86. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 31 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 +++++++++------ 3 files changed, 43 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index c8c46fe088be..faec12025a58 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -519,6 +519,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp); void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp); struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); +bool resctrl_closid_is_dirty(u32 closid); void closid_free(int closid); int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 59da256a77fe..99854ef4dee4 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -320,6 +320,37 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 closid) return ERR_PTR(-ENOSPC); } +/** + * resctrl_closid_is_dirty - Determine if clean RMID can be allocate for this + * CLOSID. + * @closid: The CLOSID that is being queried. + * + * MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocate CLOSID + * may not be able to allocate clean RMID. To avoid this the allocator will + * only return clean CLOSID. + */ +bool resctrl_closid_is_dirty(u32 closid) +{ + struct rmid_entry *entry; + int i; + + lockdep_assert_held(&rdtgroup_mutex); + + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) + return false; + + for (i = 0; i < resctrl_arch_system_num_rmid_idx(); i++) { + entry = &rmid_ptrs[i]; + if (entry->closid != closid) + continue; + + if (entry->busy) + return true; + } + + return false; +} + /* * As of now the RMIDs allocation is the same in each domain. * However we keep track of which packages the RMIDs diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index ac88610a6946..59f33adcf6f8 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -93,7 +93,7 @@ void rdt_last_cmd_printf(const char *fmt, ...) * - Our choices on how to configure each resource become progressively more * limited as the number of resources grows. */ -static int closid_free_map; +static unsigned long closid_free_map; static int closid_free_map_len; int closids_supported(void) @@ -119,14 +119,18 @@ static void closid_init(void) static int closid_alloc(void) { - u32 closid = ffs(closid_free_map); + u32 closid; - if (closid == 0) - return -ENOSPC; - closid--; - closid_free_map &= ~(1 << closid); + for_each_set_bit(closid, &closid_free_map, closid_free_map_len) { + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) && + resctrl_closid_is_dirty(closid)) + continue; - return closid; + clear_bit(closid, &closid_free_map); + return closid; + } + + return -ENOSPC; } void closid_free(int closid) From patchwork Fri Oct 21 13:11:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6701 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690840wrr; Fri, 21 Oct 2022 06:13:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4Su0XQqu0IOwgOSWSuti6lJex8hMev4/vzWGMp+zf0bpFPi1C1faOaxXOfTFw1mb58MZ7U X-Received: by 2002:a63:454d:0:b0:43c:e834:ec0 with SMTP id u13-20020a63454d000000b0043ce8340ec0mr16715024pgk.270.1666358034952; Fri, 21 Oct 2022 06:13:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358034; cv=none; d=google.com; s=arc-20160816; b=MXZr5gBCZflU4LEL8zdVr0qmSzQFnLESkyDOCJLCUrnIOylaDY7Y7hCO1B1RfJcVwK btqgE7fZIs0hcmz322aCek4BR5EyLMXcTF2qyc6e3KIcSe+tbxjcb9mF7xScij94g8ec MV8qckd0axmCO0s3lbD1KFkTX5eUfjUEkZtujgZScdsxBYcPEz4T0pHzF38LAY7sSXHj C5briPHzhBgq139zV+kSMGhQHQmevoMrjI1522JX4UtrKT4ab56gohmbdfJDiNyDRVpn Ch++GtP/I/ZcfuPmlvC235IT+4qpKcZfWqW0rHCgZOPCwGB8yM9sttm6Px0gTqYGgzDU PpOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=FvUGLdaWV3FZ6Nt872R2dJh8oAFcBSrMievVikrQNjE=; b=kQdxLiP95P5iTUqb2flWiYJxOCCOyvRAlIn486uAprg7Z0F6dC65zvMOvh7WFcMrAN yaMFM0DOwRfTgaVjwZ0i57TK8imn1KVwyhpbCvW2xN8NFYvboL5RmIGT7GoAeXDO1YRs ae335TfgCsnpdz46BPywi9YHLtVVNcv002SDQNjHuwz+oZh1YdGYBYWt+K9aPh5JgqpI XbEddFOW+jBbWDm/D8CKmqOmIvgXf54feog1SM34HMXKpDwSGQc8Ou2bt7z9lGXuRyQR RU3xjTrfpLUuecbx88EApDGpQZjTYFkfQeH1xdIMLfgBWn3RgOa9aN+rpbwXi1OBKMws +rAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y28-20020a63495c000000b0045a73a1790asi25788211pgk.613.2022.10.21.06.13.41; Fri, 21 Oct 2022 06:13:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230391AbiJUNNV (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230308AbiJUNNF (ORCPT ); Fri, 21 Oct 2022 09:13:05 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A99FD273565 for ; Fri, 21 Oct 2022 06:12:49 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4938F1480; Fri, 21 Oct 2022 06:12:55 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8596B3F792; Fri, 21 Oct 2022 06:12:46 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com, Valentin Schneider Subject: [PATCH 07/18] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers Date: Fri, 21 Oct 2022 13:11:53 +0000 Message-Id: <20221021131204.5581-8-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303042422263190?= X-GMAIL-MSGID: =?utf-8?q?1747303042422263190?= When switching tasks, the CLOSID and RMID that the new task should use are stored in struct task_struct. For x86 the CLOSID known by resctrl, the value in task_struct, and the value written to the CPU register are all the same thing. MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct task_struct implies the arch code knows whether resctrl is using CDP. Move the matching and setting of the struct task_struct properties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time. __rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance. To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together. Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID. CC: Valentin Schneider Signed-off-by: James Morse Reviewed-by: Valentin Schneider --- arch/x86/include/asm/resctrl.h | 18 ++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 57 +++++++++++++++----------- 2 files changed, 51 insertions(+), 24 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 523eabfa3193..edf4368ea25e 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -90,6 +90,24 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val) return val * scale; } +static inline void resctrl_arch_set_closid_rmid(struct task_struct *tsk, + u32 closid, u32 rmid) +{ + WRITE_ONCE(tsk->closid, closid); + WRITE_ONCE(tsk->rmid, rmid); +} + +static inline bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid) +{ + return READ_ONCE(tsk->closid) == closid; +} + +static inline bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 ignored, + u32 rmid) +{ + return READ_ONCE(tsk->rmid) == rmid; +} + static inline void resctrl_sched_in(void) { if (static_branch_likely(&rdt_enable_key)) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 59f33adcf6f8..0dd1b3391200 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -84,7 +84,7 @@ void rdt_last_cmd_printf(const char *fmt, ...) * * Using a global CLOSID across all resources has some advantages and * some drawbacks: - * + We can simply set "current->closid" to assign a task to a resource + * + We can simply set current's closid to assign a task to a resource * group. * + Context switch code can avoid extra memory references deciding which * CLOSID to load into the PQR_ASSOC MSR @@ -550,14 +550,26 @@ static void update_task_closid_rmid(struct task_struct *t) _update_task_closid_rmid(t); } +static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp) +{ + u32 closid, rmid = rdtgrp->mon.rmid; + + if (rdtgrp->type == RDTCTRL_GROUP) + closid = rdtgrp->closid; + else if (rdtgrp->type == RDTMON_GROUP) + closid = rdtgrp->mon.parent->closid; + else + return false; + + return resctrl_arch_match_closid(tsk, closid) && + resctrl_arch_match_rmid(tsk, closid, rmid); +} + static int __rdtgroup_move_task(struct task_struct *tsk, struct rdtgroup *rdtgrp) { /* If the task is already in rdtgrp, no need to move the task. */ - if ((rdtgrp->type == RDTCTRL_GROUP && tsk->closid == rdtgrp->closid && - tsk->rmid == rdtgrp->mon.rmid) || - (rdtgrp->type == RDTMON_GROUP && tsk->rmid == rdtgrp->mon.rmid && - tsk->closid == rdtgrp->mon.parent->closid)) + if (task_in_rdtgroup(tsk, rdtgrp)) return 0; /* @@ -568,19 +580,14 @@ static int __rdtgroup_move_task(struct task_struct *tsk, * For monitor groups, can move the tasks only from * their parent CTRL group. */ - - if (rdtgrp->type == RDTCTRL_GROUP) { - WRITE_ONCE(tsk->closid, rdtgrp->closid); - WRITE_ONCE(tsk->rmid, rdtgrp->mon.rmid); - } else if (rdtgrp->type == RDTMON_GROUP) { - if (rdtgrp->mon.parent->closid == tsk->closid) { - WRITE_ONCE(tsk->rmid, rdtgrp->mon.rmid); - } else { - rdt_last_cmd_puts("Can't move task to different control group\n"); - return -EINVAL; - } + if (rdtgrp->type == RDTMON_GROUP && + !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) { + rdt_last_cmd_puts("Can't move task to different control group\n"); + return -EINVAL; } + resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid, rdtgrp->mon.rmid); + /* * Ensure the task's closid and rmid are written before determining if * the task is current that will decide if it will be interrupted. @@ -600,14 +607,15 @@ static int __rdtgroup_move_task(struct task_struct *tsk, static bool is_closid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_alloc_capable && - (r->type == RDTCTRL_GROUP) && (t->closid == r->closid)); + return (rdt_alloc_capable && (r->type == RDTCTRL_GROUP) && + resctrl_arch_match_closid(t, r->closid)); } static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_mon_capable && - (r->type == RDTMON_GROUP) && (t->rmid == r->mon.rmid)); + return (rdt_mon_capable && (r->type == RDTMON_GROUP) && + resctrl_arch_match_rmid(t, r->mon.parent->closid, + r->mon.rmid)); } /** @@ -803,7 +811,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, rdtg->mode != RDT_MODE_EXCLUSIVE) continue; - if (rdtg->closid != tsk->closid) + if (!resctrl_arch_match_closid(tsk, rdtg->closid)) continue; seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "", @@ -811,7 +819,8 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, seq_puts(s, "mon:"); list_for_each_entry(crg, &rdtg->mon.crdtgrp_list, mon.crdtgrp_list) { - if (tsk->rmid != crg->mon.rmid) + if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid, + crg->mon.rmid)) continue; seq_printf(s, "%s", crg->kn->name); break; @@ -2402,8 +2411,8 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to, for_each_process_thread(p, t) { if (!from || is_closid_match(t, from) || is_rmid_match(t, from)) { - WRITE_ONCE(t->closid, to->closid); - WRITE_ONCE(t->rmid, to->mon.rmid); + resctrl_arch_set_closid_rmid(t, to->closid, + to->mon.rmid); /* * If the task is on a CPU, set the CPU in the mask. From patchwork Fri Oct 21 13:11:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6702 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp690881wrr; Fri, 21 Oct 2022 06:14:00 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7e8+XB/ZGgYAL9IrGnsazJ7y45JjxW6ii3NEcqXM4hEknCkk2/NCtcQIRta8eorKOalsEP X-Received: by 2002:a17:902:d4d1:b0:185:52c4:f4c6 with SMTP id o17-20020a170902d4d100b0018552c4f4c6mr18993261plg.154.1666358039755; Fri, 21 Oct 2022 06:13:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358039; cv=none; d=google.com; s=arc-20160816; b=zKOSnmqVGfj5BqICVXs+VrrBxsxBFz+/3sHO2j1+9xmryYnUKymnRooU3b6CAofzkc TB5BgVX2da4q6/afh1jeIrMY/v5Qh5xOXRA6K47D7hvLZH/USOO1UJLlbS4js+x69hqy v0lI6lITv5hE5qUSdw8VVLitPZD54Y9imBICTXwues8raqLQt0kZWgC+0NkqPP9c+YhK hq0TC+jJIp38i07jNDVJ/4D+J7Ge5ld6QMuKwiJBYrqkdUZYpRKeEcf/TccJK+6lmZWg gtLlJIQwWB16TBOh60DJ6dw3001O1FYYnQAZlfbeVTsYeH6Ot5VpFXF/mwsDjzDEiS5/ h2PA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=jubZ6/BMxwor3XW+SLtfJj5M2z6MRQDaJ2SOiwWNeJU=; b=pcu3OWpGAAAEBlfW3BVS4qs0fqrI0LkcJOOf1a+1FJSAyK5KTJ8crjbehbIrtufyIe paby6JyUK+5qjrCk+yyR/C3BpI02QV1D2TsUG8OE1StFYwYGpqv34uTXSVo1LwnhcN+i tClWd5hyrEMUV9+eVXQ5gpAK/rtVdpXsBob2k8nrRRllO5kCA+JmFsM6brESUVzsF1IL Gut2X1vg/fdQCKuKOuKj2ZjhgfSfUiwZQNws1ZiIBOVxPyzHOLBLHVWuWsg74XrPYa38 fRMWWndGI6zQW+fy77Zsfc4pNkQBWAJwdxWfFEGx7JDnWv+QbQjSrmBS1SPHZtcC6ttT 1B2A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l190-20020a6388c7000000b0046b2ea057e6si21571223pgd.608.2022.10.21.06.13.46; Fri, 21 Oct 2022 06:13:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230315AbiJUNNZ (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230224AbiJUNNL (ORCPT ); Fri, 21 Oct 2022 09:13:11 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9DA2D230A87 for ; Fri, 21 Oct 2022 06:12:52 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E3DB61477; Fri, 21 Oct 2022 06:12:57 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4556E3F792; Fri, 21 Oct 2022 06:12:49 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 08/18] x86/resctrl: Queue mon_event_read() instead of sending an IPI Date: Fri, 21 Oct 2022 13:11:54 +0000 Message-Id: <20221021131204.5581-9-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303047707862161?= X-GMAIL-MSGID: =?utf-8?q?1747303047707862161?= x86 is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed. Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs. These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced. mon_event_read() already invokes mon_event_count() via IPI, which means this isn't possible. Change mon_event_read() to schedule mon_event_count() on a remote CPU and wait, instead of sending an IPI. This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code). This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in monitor.c. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 7 +++++-- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++++-- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index 1dafbdc5ac31..842266d1d148 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -533,8 +533,11 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first) { + /* When picking a cpu from cpu_mask, ensure it can't race with cpuhp */ + lockdep_assert_held(&rdtgroup_mutex); + /* - * setup the parameters to send to the IPI to read the data. + * setup the parameters to pass to mon_event_count() to read the data. */ rr->rgrp = rdtgrp; rr->evtid = evtid; @@ -543,7 +546,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val = 0; rr->first = first; - smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1); + smp_call_on_cpu(cpumask_any(&d->cpu_mask), mon_event_count, rr, false); } int rdtgroup_mondata_show(struct seq_file *m, void *arg) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index faec12025a58..4d2f4524fe87 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -524,7 +524,7 @@ void closid_free(int closid); int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); -void mon_event_count(void *info); +int mon_event_count(void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 99854ef4dee4..2ca76cddb170 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -502,10 +502,10 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) } /* - * This is called via IPI to read the CQM/MBM counters + * This is scheduled by mon_event_read() to read the CQM/MBM counters * on a domain. */ -void mon_event_count(void *info) +int mon_event_count(void *info) { struct rdtgroup *rdtgrp, *entry; struct rmid_read *rr = info; @@ -538,6 +538,8 @@ void mon_event_count(void *info) */ if (ret == 0) rr->err = 0; + + return 0; } /* From patchwork Fri Oct 21 13:11:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6703 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691126wrr; Fri, 21 Oct 2022 06:14:22 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5sKhyEpTeSwm0KvlVcZkgV2lOn6eJ10xs9wsWcqRaeir23vIH2JcMQcFOT0Ts/Yr9tpGzN X-Received: by 2002:a17:90b:1e4d:b0:212:c6f4:2cff with SMTP id pi13-20020a17090b1e4d00b00212c6f42cffmr6306523pjb.240.1666358052269; Fri, 21 Oct 2022 06:14:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358052; cv=none; d=google.com; s=arc-20160816; b=ZTx32X/w/PwXcpEMvYuGy/PKHCPIU0Pa4OXWsEv5FnngZ8Lc8OY5W/xnXZs8mCkQO1 q6Piq1B8+bD9VWAUlXYLFBwmNI/1wmPG+fZ7mCRMLAELXzfPQm3wb9sj4PozmO3zMNeB CiDVPJUvgZY0PRmIQT0QIZ37DAJB4CBwmorfDuujatDCW0apdHEFVuQHQkC2KK6OTAbx LliD4s/I9KbCJ6ZFlP5osTkJl/kGhRnSMYEnP+ZA+oM0syaw24qD1VGkLfM3GLezF3Z+ Xt4JawxSyyeMi5OGjne0SzAYrctCtbhxkAAepDlTBrtGDt9+cMnCP6em9caRszG/Ekik HWXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=rD7mbJLAoQgQV8ybi2T1zGZ2c2nzotg4Zoxslt0m0nw=; b=Qjp7hOkrOqQgFdBrgKkbEnz1+/asUg911KX7fAZ4pjTJsz7DRBwFT4quLetUxt8OI1 p5JAtUNUS+a20D4sBEs/FsK+IaEL9suP0lOpGVKmv9s0kXXea/Oql9ibuINTIiKpJ//h IA0OrVgFNaFhI4PLNYgqkqCFWYQhwRFDjWfAsb0H4TLHWdj3zFycPJjm1b8ecUZHc3/c k1xiB3ZHeaPadpEAgRYvsDwyl8UMnkT2zKb/I6Z2dgdIB8oOw4sc8oYA+0duYqSyKSFz y4TLVnU/7Ocd5g0lguggLsE/c1DrxJ3KE7nPm9khFKLO8tswKRghTeBJcwSndzaijS+j X6yw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 4-20020a630c44000000b0043980f3cf24si26309679pgm.296.2022.10.21.06.13.59; Fri, 21 Oct 2022 06:14:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230415AbiJUNNa (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230343AbiJUNNJ (ORCPT ); Fri, 21 Oct 2022 09:13:09 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4849027357A for ; Fri, 21 Oct 2022 06:12:55 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8A4331042; Fri, 21 Oct 2022 06:13:00 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DFC8F3F792; Fri, 21 Oct 2022 06:12:51 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 09/18] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep Date: Fri, 21 Oct 2022 13:11:55 +0000 Message-Id: <20221021131204.5581-10-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303060808762680?= X-GMAIL-MSGID: =?utf-8?q?1747303060808762680?= MPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's resctrl_arch_rmid_read(). But add_rmid_to_limbo() calls resctrl_arch_rmid_read() from a non-preemptible context. add_rmid_to_limbo() is opportunistically reading the L3 occupancy counter on this domain to avoid adding the RMID to limbo if this domain's value has drifted below resctrl_rmid_realloc_threshold since the limbo handler last ran. Determining 'this domain' involves disabling preeption to prevent the thread being migrated to CPUs in a different domain between the check and resctrl_arch_rmid_read() call. The check is skipped for all remote domains. Instead, call resctrl_arch_rmid_read() for each domain, and get it to read the arch specific counter via IPI if its called on a CPU outside the target domain. By covering remote domains, this change stops the limbo handler from being started unnecessarily. This also allows resctrl_arch_rmid_read() to sleep. Signed-off-by: James Morse --- The alternative is to remove the counter read from this path altogether, and assume user-space would never try to re-allocate the last RMID before the limbo handler runs next. --- arch/x86/kernel/cpu/resctrl/monitor.c | 58 ++++++++++++++++++--------- 1 file changed, 38 insertions(+), 20 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 2ca76cddb170..e4ddc2477631 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -199,17 +199,19 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) return chunks >> shift; } -int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, - u32 closid, u32 rmid, enum resctrl_event_id eventid, - u64 *val) +struct __rmid_read_arg { - struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); - struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); - struct arch_mbm_state *am; - u64 msr_val, chunks; + u32 rmid; + enum resctrl_event_id eventid; - if (!cpumask_test_cpu(smp_processor_id(), &d->cpu_mask)) - return -EINVAL; + u64 msr_val; +}; + +static void __rmid_read(void *arg) +{ + enum resctrl_event_id eventid = ((struct __rmid_read_arg *)arg)->eventid; + u32 rmid = ((struct __rmid_read_arg *)arg)->rmid; + u64 msr_val; /* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured @@ -222,6 +224,28 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, wrmsr(MSR_IA32_QM_EVTSEL, eventid, rmid); rdmsrl(MSR_IA32_QM_CTR, msr_val); + ((struct __rmid_read_arg *)arg)->msr_val = msr_val; +} + +int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, + u32 closid, u32 rmid, enum resctrl_event_id eventid, + u64 *val) +{ + struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); + struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); + struct __rmid_read_arg arg; + struct arch_mbm_state *am; + u64 msr_val, chunks; + int err; + + arg.rmid = rmid; + arg.eventid = eventid; + + err = smp_call_function_any(&d->cpu_mask, __rmid_read, &arg, true); + if (err) + return err; + + msr_val = arg.msr_val; if (msr_val & RMID_VAL_ERROR) return -EIO; if (msr_val & RMID_VAL_UNAVAIL) @@ -376,23 +400,18 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) { struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; struct rdt_domain *d; - int cpu, err; u64 val = 0; u32 idx; + int err; idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); entry->busy = 0; - cpu = get_cpu(); list_for_each_entry(d, &r->domains, list) { - if (cpumask_test_cpu(cpu, &d->cpu_mask)) { - err = resctrl_arch_rmid_read(r, d, entry->closid, - entry->rmid, - QOS_L3_OCCUP_EVENT_ID, - &val); - if (err || val <= resctrl_rmid_realloc_threshold) - continue; - } + err = resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, + QOS_L3_OCCUP_EVENT_ID, &val); + if (err || val <= resctrl_rmid_realloc_threshold) + continue; /* * For the first limbo RMID in the domain, @@ -403,7 +422,6 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) set_bit(idx, d->rmid_busy_llc); entry->busy++; } - put_cpu(); if (entry->busy) rmid_limbo_count++; From patchwork Fri Oct 21 13:11:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6705 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691504wrr; Fri, 21 Oct 2022 06:15:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM664YZD7ilW9VKiHoPKE1Bak+duVt4dTDInAKo1EwuLtPdMMRCMDqjziagmrdI54QKnlaio X-Received: by 2002:a17:902:d4c5:b0:185:507b:5ef3 with SMTP id o5-20020a170902d4c500b00185507b5ef3mr19649172plg.108.1666358105420; Fri, 21 Oct 2022 06:15:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358105; cv=none; d=google.com; s=arc-20160816; b=OpcgwQKnkddnlpzvQkobnkh76QNZyA9mqb530hTQOQWr4ynlUW7CaOYG0ccEfr1lSe xGZeJ2KiUEQLP+OUxo0qSGazXQJUuGKsibnKiV6yi1lBLYxNB6yzCQGsllyOeBOFO8f7 WUuyYBnH5IoOItd62HmkCVOZoyjNfwVnkqdu3YFGKcQ8Cx31E6Eo/KNrHHe7v0CNoZkr yvIilJY2H890yNpAfed15wRVuJX9ec2h2jd1DkNEYWNcPaP2sMSw2cRMLbnC2UNtQXQr SfrpLBKwbobegbZSAwGUeDxwMZyfUHpGy9CsE3z7nIo0a/qGYmdEsWIXZRLup7beamvO 7LPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=2yHKkBinmFV1xq7kzN28qJNJx07qaStLsv5NVYojseA=; b=05ha9Eak9QljrVryerR6aWJOBuUtIi/7Xs3KzhNwI5JS9pXalvDMdcDZDtmN7aWBwa DTwbk89B6sjhAkKuDCBiqd4hzy0Hjk66s1BTEy4MU4MeB5C3eBmk3/9/501pVoSy8Nn8 VQTLpwIMf/OEZhvoqi+b5sR4CQdbT1hlofbL8Hos8GAXgDswUuBR/yG2SPUeK78uik+5 /OZsPeidgTXIJUBNR1gYFrQttphGcXZRLkPF94YPcsDAQeBMsLUHQAR/rBcm6svt1X6a PvXVn3yN/em5s/Ja0ZaS4SKURqkQ+G9gBDmaX5ZBXpAs0EnRpg0JU62+iF0S8iZxt7iI LUfg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z3-20020a17090ad78300b0020cedba54fcsi5853899pju.55.2022.10.21.06.14.49; Fri, 21 Oct 2022 06:15:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230378AbiJUNNk (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230362AbiJUNNO (ORCPT ); Fri, 21 Oct 2022 09:13:14 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A900F1645D0 for ; Fri, 21 Oct 2022 06:12:57 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 315101D34; Fri, 21 Oct 2022 06:13:03 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 853C83F792; Fri, 21 Oct 2022 06:12:54 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 10/18] x86/resctrl: Allow arch to allocate memory needed in resctrl_arch_rmid_read() Date: Fri, 21 Oct 2022 13:11:56 +0000 Message-Id: <20221021131204.5581-11-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303116881479078?= X-GMAIL-MSGID: =?utf-8?q?1747303116881479078?= Depending on the number of monitors available, Arm's MPAM may need to allocate a monitor prior to reading the counter value. Allocating a contended resource may involve sleeping. All callers of resctrl_arch_rmid_read() read the counter on more than one domain. If the monitor is allocated globally, there is no need to allocate and free it for each call to resctrl_arch_rmid_read(). Add arch hooks for this allocation, which need calling before resctrl_arch_rmid_read(). The allocated monitor is passed to resctrl_arch_rmid_read(), then freed again afterwards. The helper can be called on any CPU, and can sleep. Signed-off-by: James Morse --- arch/x86/include/asm/resctrl.h | 11 +++++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++++++++++++++++++++--- include/linux/resctrl.h | 4 +-- 4 files changed, 50 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index edf4368ea25e..4ce1376525e3 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -131,6 +131,17 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) return rmid; } +/* x86 can always read an rmid, nothing needs allocating */ +struct rdt_resource; +static inline int resctrl_arch_mon_ctx_alloc(struct rdt_resource *r, int evtid) +{ + might_sleep(); + return 0; +}; + +static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r, int evtid, + int ctx) { }; + void resctrl_cpu_detect(struct cpuinfo_x86 *c); #else diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 4d2f4524fe87..8e64f1306591 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -98,6 +98,7 @@ struct rmid_read { bool first; int err; u64 val; + int arch_mon_ctx; }; extern bool rdt_alloc_capable; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index e4ddc2477631..670cd5b7d733 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -15,6 +15,7 @@ * Software Developer Manual June 2016, volume 3, section 17.17. */ +#include #include #include #include @@ -229,7 +230,7 @@ static void __rmid_read(void *arg) int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, - u64 *val) + u64 *val, int ignored) { struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); @@ -278,9 +279,14 @@ void __check_limbo(struct rdt_domain *d, bool force_free) u32 idx_limit = resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry; u32 idx, cur_idx = 1; + int arch_mon_ctx; bool rmid_dirty; u64 val = 0; + arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID); + if (arch_mon_ctx < 0) + return; + /* * Skip RMID 0 and start from RMID 1 and check all the RMIDs that * are marked as busy for occupancy < threshold. If the occupancy @@ -294,7 +300,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) entry = __rmid_entry(idx); if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, - QOS_L3_OCCUP_EVENT_ID, &val)) { + QOS_L3_OCCUP_EVENT_ID, &val, + arch_mon_ctx)) { rmid_dirty = true; } else { rmid_dirty = (val >= resctrl_rmid_realloc_threshold); @@ -309,6 +316,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } cur_idx = idx + 1; } + + resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx); } bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d) @@ -400,16 +409,22 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) { struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; struct rdt_domain *d; + int arch_mon_ctx; u64 val = 0; u32 idx; int err; idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); + arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID); + if (arch_mon_ctx < 0) + return; + entry->busy = 0; list_for_each_entry(d, &r->domains, list) { err = resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, - QOS_L3_OCCUP_EVENT_ID, &val); + QOS_L3_OCCUP_EVENT_ID, &val, + arch_mon_ctx); if (err || val <= resctrl_rmid_realloc_threshold) continue; @@ -422,6 +437,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) set_bit(idx, d->rmid_busy_llc); entry->busy++; } + resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx); if (entry->busy) rmid_limbo_count++; @@ -458,7 +474,7 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid); rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid, - &tval); + &tval, rr->arch_mon_ctx); if (rr->err) return rr->err; @@ -531,6 +547,9 @@ int mon_event_count(void *info) int ret; rdtgrp = rr->rgrp; + rr->arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr->r, rr->evtid); + if (rr->arch_mon_ctx < 0) + return rr->arch_mon_ctx; ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr); @@ -557,6 +576,8 @@ int mon_event_count(void *info) if (ret == 0) rr->err = 0; + resctrl_arch_mon_ctx_free(rr->r, rr->evtid, rr->arch_mon_ctx); + return 0; } @@ -693,11 +714,21 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, if (is_mbm_total_enabled()) { rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID; rr.val = 0; + rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid); + if (rr.arch_mon_ctx < 0) + return; + __mon_event_count(closid, rmid, &rr); + + resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx); } if (is_mbm_local_enabled()) { rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID; rr.val = 0; + rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid); + if (rr.arch_mon_ctx < 0) + return; + __mon_event_count(closid, rmid, &rr); /* @@ -707,6 +738,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, */ if (is_mba_sc(NULL)) mbm_bw_count(closid, rmid, &rr); + resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx); } } diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 641aea580a1f..1f345d24e236 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -230,6 +230,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. + * @arch_mon_ctx: An allocated context from resctrl_arch_mon_ctx_alloc(). * * Call from process context on a CPU that belongs to domain @d. * @@ -238,8 +239,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); */ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, - u64 *val); - + u64 *val, int arch_mon_ctx); /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid From patchwork Fri Oct 21 13:11:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6704 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691471wrr; Fri, 21 Oct 2022 06:15:03 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6pdbu+sTDRprHhttDNTwPf34ZSycThEvD0d9KtfopRpyzEWY/6L3xLa0I28oKpZ9ZLUcfn X-Received: by 2002:a63:d613:0:b0:45a:654:cf16 with SMTP id q19-20020a63d613000000b0045a0654cf16mr16286166pgg.611.1666358102686; Fri, 21 Oct 2022 06:15:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358102; cv=none; d=google.com; s=arc-20160816; b=ZSbHGaKTl4trqPLJ4+IySRSerlim8iiKKJjByIr985+siIDGJkU0R8/hVSR4q2xZRA xgB3HkCudJvXAfLTPd10Kf0a/FAyNY6cyXSrxgULIkDKVqH24SRot7NCtQytmZHOwo5B 54LBDQiQhspYMqOEckRO96vNI0SblWrbM1b6SUBqPQnmgHGi0eoy2q5ktovouDNaI6zA Q2CpEEKxZC1+RpEpP3Gd3eD7g2Bro+eAWrH8oSN5dVEn4ZCutDuJqrsb88ZNtcxrWSIA OOlI3bNmorOIhiI01BmxgH+CAvpUx2jcWb3nowJgPXNR29q/+9YoWOdpwEkn0lutvgII A07w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=qWDQVGvzCAuxLZFsZCK91Y4YkxNnEutRYScrVvbaSuk=; b=uUz8E6Mnm5XPtDVtawddLnZxVzAGbbmhTciFBmj5c0qGv2wOfeJLbsr/7XeX8mRXSB RBQ3Xizzr5VowvuMpez+BMSsDeANUbgqqXLH/UYnOvxM8ypQ7ZH0oSf3qcetlml9Yp8r cZtF6s9iNZRJsZlD9QurUDn68Sd9g495h/AkZzApnAMT8ZI8BRvEHcmQSRha9Rkix+kK p4AkwLe9RoFkQM4OxDFBDLvJE54f85qd9j6EbacqhDs/zBS49h43a3Z7hcQrl0Mk3z87 YQXlXlbiQ4cleAq8/mZB2KtBQpfUhqF6rffAU1Ye1gPQ6xOCQ2TpjFFZOFM6uG5xqVSy YbZw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u4-20020a170902e80400b0017a7506e0dcsi27828267plg.231.2022.10.21.06.14.47; Fri, 21 Oct 2022 06:15:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230439AbiJUNNg (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230357AbiJUNNN (ORCPT ); Fri, 21 Oct 2022 09:13:13 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4056B26D911 for ; Fri, 21 Oct 2022 06:13:00 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CB4561D70; Fri, 21 Oct 2022 06:13:05 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2C8BE3F792; Fri, 21 Oct 2022 06:12:57 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 11/18] x86/resctrl: Make resctrl_mounted checks explicit Date: Fri, 21 Oct 2022 13:11:57 +0000 Message-Id: <20221021131204.5581-12-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303113843136385?= X-GMAIL-MSGID: =?utf-8?q?1747303113843136385?= The rdt_enable_key is switched when resctrl is mounted, and used to prevent a second mount of the filesystem. It also enables the architecture's context switch code. This requires another architecture to have the same set of static-keys, as resctrl depends on them too. Make the resctrl_mounted checks explicit: resctrl can keep track of whether it has been mounted once. This doesn't need to be combined with whether the arch code is context switching the CLOSID. Tests against the rdt_mon_enable_key become a test that resctrl is mounted and that monitoring is enabled. This will allow the static-key changing to be moved behind resctrl_arch_ calls. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 5 +++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 17 +++++++++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 8e64f1306591..a1f5bfdc62cc 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -105,6 +105,7 @@ extern bool rdt_alloc_capable; extern bool rdt_mon_capable; extern unsigned int rdt_mon_features; extern struct list_head resctrl_schema_all; +extern bool resctrl_mounted; enum rdt_group_type { RDTCTRL_GROUP = 0, diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 670cd5b7d733..043fcb32435f 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -788,7 +788,7 @@ void mbm_handle_overflow(struct work_struct *work) mutex_lock(&rdtgroup_mutex); - if (!static_branch_likely(&rdt_mon_enable_key)) + if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -816,8 +816,9 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - if (!static_branch_likely(&rdt_mon_enable_key)) + if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) return; + cpu = cpumask_any(&dom->cpu_mask); dom->mbm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->mbm_over, delay); diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 0dd1b3391200..89ce5ac7de35 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -42,6 +42,9 @@ LIST_HEAD(rdt_all_groups); /* list of entries for the schemata file */ LIST_HEAD(resctrl_schema_all); +/* the filesystem can only be mounted once */ +bool resctrl_mounted; + /* Kernel fs node for "info" directory under root */ static struct kernfs_node *kn_info; @@ -795,7 +798,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, mutex_lock(&rdtgroup_mutex); /* Return empty if resctrl has not been mounted. */ - if (!static_branch_unlikely(&rdt_enable_key)) { + if (!resctrl_mounted) { seq_puts(s, "res:\nmon:\n"); goto unlock; } @@ -2197,7 +2200,7 @@ static int rdt_get_tree(struct fs_context *fc) /* * resctrl file system can only be mounted once. */ - if (static_branch_unlikely(&rdt_enable_key)) { + if (resctrl_mounted) { ret = -EBUSY; goto out; } @@ -2245,8 +2248,10 @@ static int rdt_get_tree(struct fs_context *fc) if (rdt_mon_capable) static_branch_enable_cpuslocked(&rdt_mon_enable_key); - if (rdt_alloc_capable || rdt_mon_capable) + if (rdt_alloc_capable || rdt_mon_capable) { static_branch_enable_cpuslocked(&rdt_enable_key); + resctrl_mounted = true; + } if (is_mbm_enabled()) { r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -2513,6 +2518,7 @@ static void rdt_kill_sb(struct super_block *sb) static_branch_disable_cpuslocked(&rdt_alloc_enable_key); static_branch_disable_cpuslocked(&rdt_mon_enable_key); static_branch_disable_cpuslocked(&rdt_enable_key); + resctrl_mounted = false; kernfs_kill_sb(sb); mutex_unlock(&rdtgroup_mutex); cpus_read_unlock(); @@ -3337,7 +3343,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) * If resctrl is mounted, remove all the * per domain monitor data directories. */ - if (static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) rmdir_mondata_subdir_allrdtgrp(r, d->id); if (is_mbm_enabled()) @@ -3414,8 +3420,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) if (is_llc_occupancy_enabled()) INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo); - /* If resctrl is mounted, add per domain monitor data directories. */ - if (static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) mkdir_mondata_subdir_allrdtgrp(r, d); return 0; From patchwork Fri Oct 21 13:11:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6706 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691528wrr; Fri, 21 Oct 2022 06:15:07 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6F4TFSqgluypo7Y+YPIjjhyCG4jJs76k2n028ZToczIl1QwdtBSZ2S3meb1q81TAp87iJz X-Received: by 2002:a05:6a02:11a:b0:441:9db5:e8cc with SMTP id bg26-20020a056a02011a00b004419db5e8ccmr15866815pgb.345.1666358107102; Fri, 21 Oct 2022 06:15:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358107; cv=none; d=google.com; s=arc-20160816; b=qBokuPqLK9HE0rHl2Z2Df2QiFEBLaz4wmn6wanCHwLO4MmmbCI2G/uyLheObc0MZh6 ODUKU23yMnlW6qdPc5V1uxUOjOgS9xLA/iK3pxPGKUy6zKvXbRaNY2qWlAHQxbK8LAC2 1zzKvU/kdjBiEBr7OW7oAYB+6DqxsPKdnnzRKv0LDNi0BgwLHvUfuWv73tJDC4I7n4U6 iVym52YjghwhzHFnthJLTMPAv9sDbmCD40k8PzKBIt8u/IWJ6n4rto8lFvznYCaHv85r 2EThDHh/aAhAL3jxJgKPiftqaz/Me0NI9hQebdkJSD6b/11dksxevSdqpKUysct/kT2C IEDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=+Cl7KX9CNsVpI2i1NIf8Prx1fjCaohCzf3RQkbTP5/I=; b=z/22r2u2yDxLrxET+Lk6QJfUp4BZwZbvpXoNwSD28h1NSvIC9LR84Amh/m9R1hMF2k oMImBQouA/cDDyFIsIiom0j0ahyoV6TAFNU0Nno5wQMlAupuV1t8fQaSSSoyiD6YAsZA vfAAozrou+Ct/GVPUJKjyBsqfCV9vq+Q7KKCapqLwAj+AOy8xmXCuju/UlNoCQCokkss BjiUrRDrnTiGK7aw/5iFqF1VpJHIM7OhmAo9HZBobLdkP+D3ENubNTHiQheKnIq33/pL 54P0hf3UWoWbN6tZFQHOGUiNwjkZQJHNiV5a3EUeQNl3vni2W1ABUSxLOzTX8gQPMMpw 11Sw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n5-20020a632705000000b00460ea4393c9si23438242pgn.448.2022.10.21.06.14.52; Fri, 21 Oct 2022 06:15:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbiJUNNo (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230382AbiJUNNQ (ORCPT ); Fri, 21 Oct 2022 09:13:16 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id DAFEF24CCAD for ; Fri, 21 Oct 2022 06:13:03 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7007C1DB5; Fri, 21 Oct 2022 06:13:08 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C5A3B3F792; Fri, 21 Oct 2022 06:12:59 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 12/18] x86/resctrl: Move alloc/mon static keys into helpers Date: Fri, 21 Oct 2022 13:11:58 +0000 Message-Id: <20221021131204.5581-13-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303118531597291?= X-GMAIL-MSGID: =?utf-8?q?1747303118531597291?= resctrl enables three static keys depending on the features it has enabled. Another architecture's context switch code may look different, any static keys that control it should be buried behind helpers. Move the alloc/mon logic into arch-specific helpers as a preparatory step for making the rdt_enable_key's status something the arch code decides. This means other architectures don't have to mirror the static keys. Signed-off-by: James Morse --- arch/x86/include/asm/resctrl.h | 20 ++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++---- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 4ce1376525e3..ae3a87100626 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -37,6 +37,26 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); +static inline void resctrl_arch_enable_alloc(void) +{ + static_branch_enable_cpuslocked(&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_disable_alloc(void) +{ + static_branch_disable_cpuslocked(&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_enable_mon(void) +{ + static_branch_enable_cpuslocked(&rdt_mon_enable_key); +} + +static inline void resctrl_arch_disable_mon(void) +{ + static_branch_disable_cpuslocked(&rdt_mon_enable_key); +} + /* * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR * diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index a1f5bfdc62cc..f7ea5432cbdb 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -57,9 +57,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc) return container_of(kfc, struct rdt_fs_context, kfc); } -DECLARE_STATIC_KEY_FALSE(rdt_enable_key); -DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); - /** * struct mon_evt - Entry in the event list of a resource * @evtid: event id @@ -415,8 +412,6 @@ extern struct mutex rdtgroup_mutex; extern struct rdt_hw_resource rdt_resources_all[]; extern struct rdtgroup rdtgroup_default; -DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); - extern struct dentry *debugfs_resctrl; enum resctrl_res_level { diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 89ce5ac7de35..bd2489bae7cf 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2244,9 +2244,9 @@ static int rdt_get_tree(struct fs_context *fc) goto out_psl; if (rdt_alloc_capable) - static_branch_enable_cpuslocked(&rdt_alloc_enable_key); + resctrl_arch_enable_alloc(); if (rdt_mon_capable) - static_branch_enable_cpuslocked(&rdt_mon_enable_key); + resctrl_arch_enable_mon(); if (rdt_alloc_capable || rdt_mon_capable) { static_branch_enable_cpuslocked(&rdt_enable_key); @@ -2515,8 +2515,8 @@ static void rdt_kill_sb(struct super_block *sb) rdt_pseudo_lock_release(); rdtgroup_default.mode = RDT_MODE_SHAREABLE; schemata_list_destroy(); - static_branch_disable_cpuslocked(&rdt_alloc_enable_key); - static_branch_disable_cpuslocked(&rdt_mon_enable_key); + resctrl_arch_disable_alloc(); + resctrl_arch_disable_mon(); static_branch_disable_cpuslocked(&rdt_enable_key); resctrl_mounted = false; kernfs_kill_sb(sb); From patchwork Fri Oct 21 13:11:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6707 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691658wrr; Fri, 21 Oct 2022 06:15:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4mtNO7dOpHbhV10k0HVl3Pl3TaDX1jRAvj1I1rOH6N8Z01rOzvP0CHMd8Y/OaGZuRTlvgq X-Received: by 2002:a05:6a00:9a7:b0:563:a9e0:fe50 with SMTP id u39-20020a056a0009a700b00563a9e0fe50mr18845003pfg.45.1666358110497; Fri, 21 Oct 2022 06:15:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358110; cv=none; d=google.com; s=arc-20160816; b=IH7YkHpxH0bXQ5nQsWr10sYxKSpRSV0hW1OpaXHPZMONsrmgIHtsK0w9EXQcEA5Grj dZWm3Se57cGwuyaYxjhfpw/hYs8qa2H6wNhgLyPXFQIGPFUpRIptx8pJ5LId7e78Ir/q OSj6kq4YiuE3RBRR2p0enEUjzE4b0s8I6H4ciE/PhzGhOrp3U2cWT/DCbHYc514+RLHT CqqcVqJ8Kuzd6cRm+cvJ+YQI8lh6q8evCxZQewANtfMQuPuovK2IRqrMSy9D1TtDhPdV rw//H9HosYdxZwJqxo8Sq3LJo0ZE4QYfXqqBU08uB9/Wfwtz4pxHl+KpEfPzapqIQ4z9 QMCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=kWqINNdaMdOLRCjJmjINi23LtO5gbnxhZ8cCgV69a4w=; b=vzYXbIDoTSQJ1Pk7Jj46mf3BIrlMiklelOxf68QED66KmWdaoIMNJRdp3qY1AT7SgC 3sMdGHAusL/U1aiL3Qhvc5IMbnyGQhgBlm0joiGANsSR9MAlJdNLB8zLqd7FSpqwn4J5 Z+/ZUmDpp8h28Iso0RRVjiDKfZ+x3un59uRc3P6RneOEr+0yio1StjPb4xTx4lbyN8ZX qgul3Mh01Uh3QPgCgh1TK9RmwNURUqD2uymkbBq5EWwgfecoUtM4S7HFzOzsSd3JhIeD QnVvFQZFO0rZVoxDQ5shcOmNBPRwlu100TuU8/1rA4CrSnJx0ngzvS1yfqan8eLt7uqw OGJg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bx25-20020a056a00429900b0052d9e5f07d9si22848960pfb.210.2022.10.21.06.14.57; Fri, 21 Oct 2022 06:15:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230308AbiJUNN5 (ORCPT + 99 others); Fri, 21 Oct 2022 09:13:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230401AbiJUNNX (ORCPT ); Fri, 21 Oct 2022 09:13:23 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7A10926D91F for ; Fri, 21 Oct 2022 06:13:05 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 183F51E2F; Fri, 21 Oct 2022 06:13:11 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6C5023F792; Fri, 21 Oct 2022 06:13:02 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 13/18] x86/resctrl: Make rdt_enable_key the arch's decision to switch Date: Fri, 21 Oct 2022 13:11:59 +0000 Message-Id: <20221021131204.5581-14-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303122292602668?= X-GMAIL-MSGID: =?utf-8?q?1747303122292602668?= rdt_enable_key is switched when resctrl is mounted. It was also previously used to prevent a second mount of the filesystem. Any other architecture that wants to support resctrl has to provide identical static keys. Now that we have helpers for enablign and disabling the alloc/mon keys, resctrl doesn't need to switch this extra key, it can be done by the arch code. Use the static-key increment and decrement helpers, and change resctrl to ensure the calls are balanced. Signed-off-by: James Morse --- arch/x86/include/asm/resctrl.h | 4 ++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 11 +++++------ 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index ae3a87100626..00cffc43eb49 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -40,21 +40,25 @@ DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); static inline void resctrl_arch_enable_alloc(void) { static_branch_enable_cpuslocked(&rdt_alloc_enable_key); + static_branch_inc_cpuslocked(&rdt_enable_key); } static inline void resctrl_arch_disable_alloc(void) { static_branch_disable_cpuslocked(&rdt_alloc_enable_key); + static_branch_dec_cpuslocked(&rdt_enable_key); } static inline void resctrl_arch_enable_mon(void) { static_branch_enable_cpuslocked(&rdt_mon_enable_key); + static_branch_inc_cpuslocked(&rdt_enable_key); } static inline void resctrl_arch_disable_mon(void) { static_branch_disable_cpuslocked(&rdt_mon_enable_key); + static_branch_dec_cpuslocked(&rdt_enable_key); } /* diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index bd2489bae7cf..f076e516f381 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2248,10 +2248,8 @@ static int rdt_get_tree(struct fs_context *fc) if (rdt_mon_capable) resctrl_arch_enable_mon(); - if (rdt_alloc_capable || rdt_mon_capable) { - static_branch_enable_cpuslocked(&rdt_enable_key); + if (rdt_alloc_capable || rdt_mon_capable) resctrl_mounted = true; - } if (is_mbm_enabled()) { r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -2515,9 +2513,10 @@ static void rdt_kill_sb(struct super_block *sb) rdt_pseudo_lock_release(); rdtgroup_default.mode = RDT_MODE_SHAREABLE; schemata_list_destroy(); - resctrl_arch_disable_alloc(); - resctrl_arch_disable_mon(); - static_branch_disable_cpuslocked(&rdt_enable_key); + if (rdt_alloc_capable) + resctrl_arch_disable_alloc(); + if (rdt_mon_capable) + resctrl_arch_disable_mon(); resctrl_mounted = false; kernfs_kill_sb(sb); mutex_unlock(&rdtgroup_mutex); From patchwork Fri Oct 21 13:12:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6710 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691894wrr; Fri, 21 Oct 2022 06:15:44 -0700 (PDT) X-Google-Smtp-Source: AMsMyM56gXxNplatmSHa3HCCRhMxDrQtlEzJlu0d1NGwdfV4N8rMxBG0VTL1hi6AYkMC0X6ZUzEU X-Received: by 2002:a17:90b:3882:b0:20d:cf30:36ca with SMTP id mu2-20020a17090b388200b0020dcf3036camr35439844pjb.99.1666358144014; Fri, 21 Oct 2022 06:15:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358144; cv=none; d=google.com; s=arc-20160816; b=u+Nq/7wSfbwCacfG5hls+KR07z3AtQdRAMlVuyPoT/b7IHcbCKzIViOpTipfKT1jdY dKOn2f0aZAlwvJKABPpV2fQRMpbPSbx0roSxqNOU639iJWAyRRCK84vUDMMsj/cB0Wji 0UzxcOmefgblortKusHdneSDHcSGkib64GQFs9ZJO6KmhU5fdChWDtWqv7gYLRc9POOh vxi87TT+1v1mP7ZO8Oxp+Qnk2j4/IVpvUvOyG8Ti0RXb4AisoHgdWl+kokORsG95lcyy ovg5vITdazi5sK2Ae7M76s8ypdkYbylPRDoiGx4AnO50otGTvIFkCi+q1Igme4mIXSMz LDHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=R87KW5ujT3Q8CC/sVefT8GgGabyc/AEcl1KY1RQUtow=; b=iSRAhAvNDwlVjZKJMAl+B/LdzFSqvn4o8NJtgI8O3Lnrj8d8Nx+DvgX2BqGOs3Rgq1 ULzqroNZ2SnlVwN0HEjULrP7urHAarF5JTa+Wss8m+n3GuQaW8K0CDIamOFMaX9lpdrD Zx5z9fD1pU7iSaUqFJKQWGVQ4Z/qBFZw8x3gU4veVRPRuTtF8mNudkCSvwCT/wNRsg7q vpe9HrUKtT8wyb4Lihi+WC0Q/h/IKToz0jAzimRAYO7rY04VW6IIGv24EEdaz54gyKZo iQgZP0jpt5DiR6/6xTAQndK8vVNBEyFIWMH07fNlzdRKopOxC4YcWmvM9c6Bzn2frJPF BwnQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mi12-20020a17090b4b4c00b0020befc1a93asi7551290pjb.100.2022.10.21.06.15.30; Fri, 21 Oct 2022 06:15:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230405AbiJUNOu (ORCPT + 99 others); Fri, 21 Oct 2022 09:14:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230413AbiJUNN2 (ORCPT ); Fri, 21 Oct 2022 09:13:28 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id DFC0826D936 for ; Fri, 21 Oct 2022 06:13:12 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B38241E32; Fri, 21 Oct 2022 06:13:13 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 15A653F792; Fri, 21 Oct 2022 06:13:04 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 14/18] x86/resctrl: Add helpers for system wide mon/alloc capable Date: Fri, 21 Oct 2022 13:12:00 +0000 Message-Id: <20221021131204.5581-15-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303157725643335?= X-GMAIL-MSGID: =?utf-8?q?1747303157725643335?= resctrl reads rdt_alloc_capable or rdt_mon_capable to determine whether any of the resources support the corresponding features. resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing. This forces another architecture to have the same static-keys. As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key. After this change, only the architecture code manages and uses the static-keys to ensure __resctrl_sched_in() does not need runtime checks. This avoids multiple architectures having to define the same static-keys. Signed-off-by: James Morse --- arch/x86/include/asm/resctrl.h | 13 +++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 32 +++++++++++------------ 5 files changed, 34 insertions(+), 23 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 00cffc43eb49..c404d4611f83 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -33,10 +33,18 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); +extern bool rdt_alloc_capable; +extern bool rdt_mon_capable; + DECLARE_STATIC_KEY_FALSE(rdt_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); +static inline bool resctrl_arch_alloc_capable(void) +{ + return rdt_alloc_capable; +} + static inline void resctrl_arch_enable_alloc(void) { static_branch_enable_cpuslocked(&rdt_alloc_enable_key); @@ -49,6 +57,11 @@ static inline void resctrl_arch_disable_alloc(void) static_branch_dec_cpuslocked(&rdt_enable_key); } +static inline bool resctrl_arch_mon_capable(void) +{ + return rdt_mon_capable; +} + static inline void resctrl_arch_enable_mon(void) { static_branch_enable_cpuslocked(&rdt_mon_enable_key); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index f7ea5432cbdb..adbbfaabf70b 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -98,8 +98,6 @@ struct rmid_read { int arch_mon_ctx; }; -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable; extern unsigned int rdt_mon_features; extern struct list_head resctrl_schema_all; extern bool resctrl_mounted; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 043fcb32435f..a9af7c56a04a 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -788,7 +788,7 @@ void mbm_handle_overflow(struct work_struct *work) mutex_lock(&rdtgroup_mutex); - if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) + if (!resctrl_mounted || !resctrl_arch_mon_capable()) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -816,7 +816,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) + if (!resctrl_mounted || !resctrl_arch_mon_capable()) return; cpu = cpumask_any(&dom->cpu_mask); diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index acebbc35331f..48f196007e9c 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -567,7 +567,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp) if (ret) goto err_cpus; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = rdtgroup_kn_mode_restrict(rdtgrp, "mon_groups"); if (ret) goto err_cpus_list; @@ -614,7 +614,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp) if (ret) goto err_cpus; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = rdtgroup_kn_mode_restore(rdtgrp, "mon_groups", 0777); if (ret) goto err_cpus_list; @@ -762,7 +762,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) { int ret; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = alloc_rmid(rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index f076e516f381..81ed458be70b 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -610,13 +610,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk, static bool is_closid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_alloc_capable && (r->type == RDTCTRL_GROUP) && + return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) && resctrl_arch_match_closid(t, r->closid)); } static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_mon_capable && (r->type == RDTMON_GROUP) && + return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) && resctrl_arch_match_rmid(t, r->mon.parent->closid, r->mon.rmid)); } @@ -2221,7 +2221,7 @@ static int rdt_get_tree(struct fs_context *fc) if (ret < 0) goto out_schemata_free; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = mongroup_create_dir(rdtgroup_default.kn, &rdtgroup_default, "mon_groups", &kn_mongrp); @@ -2243,12 +2243,12 @@ static int rdt_get_tree(struct fs_context *fc) if (ret < 0) goto out_psl; - if (rdt_alloc_capable) + if (resctrl_arch_alloc_capable()) resctrl_arch_enable_alloc(); - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) resctrl_arch_enable_mon(); - if (rdt_alloc_capable || rdt_mon_capable) + if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable()) resctrl_mounted = true; if (is_mbm_enabled()) { @@ -2262,10 +2262,10 @@ static int rdt_get_tree(struct fs_context *fc) out_psl: rdt_pseudo_lock_release(); out_mondata: - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) kernfs_remove(kn_mondata); out_mongrp: - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) kernfs_remove(kn_mongrp); out_info: kernfs_remove(kn_info); @@ -2513,9 +2513,9 @@ static void rdt_kill_sb(struct super_block *sb) rdt_pseudo_lock_release(); rdtgroup_default.mode = RDT_MODE_SHAREABLE; schemata_list_destroy(); - if (rdt_alloc_capable) + if (resctrl_arch_alloc_capable()) resctrl_arch_disable_alloc(); - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) resctrl_arch_disable_mon(); resctrl_mounted = false; kernfs_kill_sb(sb); @@ -2890,7 +2890,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) { int ret; - if (!rdt_mon_capable) + if (!resctrl_arch_mon_capable()) return 0; ret = alloc_rmid(rdtgrp->closid); @@ -3076,7 +3076,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups); - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { /* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor. @@ -3131,14 +3131,14 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name, * allocation is supported, add a control and monitoring * subdirectory */ - if (rdt_alloc_capable && parent_kn == rdtgroup_default.kn) + if (resctrl_arch_alloc_capable() && parent_kn == rdtgroup_default.kn) return rdtgroup_mkdir_ctrl_mon(parent_kn, name, mode); /* * If RDT monitoring is supported and the parent directory is a valid * "mon_groups" directory, add a monitoring subdirectory. */ - if (rdt_mon_capable && is_mon_groups(parent_kn, name)) + if (resctrl_arch_mon_capable() && is_mon_groups(parent_kn, name)) return rdtgroup_mkdir_mon(parent_kn, name, mode); return -EPERM; @@ -3342,7 +3342,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) * If resctrl is mounted, remove all the * per domain monitor data directories. */ - if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && resctrl_arch_mon_capable()) rmdir_mondata_subdir_allrdtgrp(r, d->id); if (is_mbm_enabled()) @@ -3419,7 +3419,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) if (is_llc_occupancy_enabled()) INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo); - if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && resctrl_arch_mon_capable()) mkdir_mondata_subdir_allrdtgrp(r, d); return 0; From patchwork Fri Oct 21 13:12:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6708 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp691674wrr; Fri, 21 Oct 2022 06:15:23 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5oByA3Dio7wDlRNZOd3cMiaMTmnkAHZpN+CwA/IJ29nxtGbdC1VvsQcN90HMCNz3EgPJMB X-Received: by 2002:a17:90a:540e:b0:210:1e26:9422 with SMTP id z14-20020a17090a540e00b002101e269422mr13901845pjh.100.1666358123380; Fri, 21 Oct 2022 06:15:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358123; cv=none; d=google.com; s=arc-20160816; b=LkNDI9a+U/Yt25GgZNH7RoOIW9hMSRVRRJbW2yI1jVYZWBYq/LjHJ8Dt6XX03oTBPt 4Tq/9wOsoUM3d+o+0aLpzvPnncyIewPv0AiIfUwVVhT/d0jAQ417cZQkEniJ11hu2yaK YJ4qrqUv6Pkd8uW5mZKAEW2BLjmZk99Fr80TaiENbNsiExWkcSwF+5VOYPR3xbewn7IC dm+Crtp57ZawkrI+Hnf+nfD55hLWT9ZV4XLi/o2nsmNOHbdGpAbr7nU5wjaRAEy0w5Q0 yb6li45SZmWSoA6UQagMWBLba9o8CZLh+QkdGeAOjI2LnRP7497PHZCN6yxsnQBBMuTG a0xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=JB2mF0OIehw/wfG+fLWD7H5RHg15+9Hyve7cogGjmZM=; b=uxnKjnmxpRb+T/8bajsii7I2LBLRWI0QZcn4lUdZ1KCI3FHW4kunA9UbINyK/8AE+T KlsaxjR1CtJRJzG7NbiuaigMI1rJHN8iEVut0EYGu1LzWSi6cvnktJVimM+CGmdZGbY6 7N1R6ZbyWct2bfDQQ4PtDRuAiS8aUnuJsEftWe+6xNtTXiL+AXgMNccmL77dyCvy1Qeu Lqr9u0l33RjqL84jsi8k0pbuWlzBJKl33zzICJ5W8xjkeUnQLz3TE9vqLF1da3KEp0G0 8dcHMGokL1MCl3zgssSaOEoJNQmSM2N5wxX7YHPGp8OfqHc2GnMlefU9yTsGzaWhQLwL NvGw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o6-20020a056a001bc600b005644806915csi24645559pfw.42.2022.10.21.06.15.09; Fri, 21 Oct 2022 06:15:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230354AbiJUNOr (ORCPT + 99 others); Fri, 21 Oct 2022 09:14:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230361AbiJUNN1 (ORCPT ); Fri, 21 Oct 2022 09:13:27 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 86F9226D92A for ; Fri, 21 Oct 2022 06:13:10 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B7F31042; Fri, 21 Oct 2022 06:13:16 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B14323F792; Fri, 21 Oct 2022 06:13:07 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 15/18] x86/resctrl: Add cpu online callback for resctrl work Date: Fri, 21 Oct 2022 13:12:01 +0000 Message-Id: <20221021131204.5581-16-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303135393943494?= X-GMAIL-MSGID: =?utf-8?q?1747303135393943494?= The resctrl architecture specific code may need to create a domain when a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register. The resctrl filesystem code needs to update the rdtgroup_default cpu mask when cpus are brought online. Currently this is all done in one function, resctrl_online_cpu(). This will need to be split into architecture and filesystem parts before resctrl can be moved to /fs/. Pull the rdtgroup_default update work out as a filesystem specific cpu_online helper. resctrl_online_cpu() is the obvious name for this, which means the version in core.c needs renaming. resctrl_online_cpu() is called by the arch code once it has done the work to add the new cpu to any domains. In future patches, resctrl_online_cpu() will take the rdtgroup_mutex itself. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/core.c | 11 ++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 10 ++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index de62b0b87ced..511ced743a79 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -584,19 +584,20 @@ static void clear_closid_rmid(int cpu) wrmsr(IA32_PQR_ASSOC, 0, 0); } -static int resctrl_online_cpu(unsigned int cpu) +static int resctrl_arch_online_cpu(unsigned int cpu) { struct rdt_resource *r; + int err; mutex_lock(&rdtgroup_mutex); for_each_capable_rdt_resource(r) domain_add_cpu(cpu, r); - /* The cpu is set in default rdtgroup after online. */ - cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); clear_closid_rmid(cpu); + + err = resctrl_online_cpu(cpu); mutex_unlock(&rdtgroup_mutex); - return 0; + return err; } static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) @@ -923,7 +924,7 @@ static int __init resctrl_late_init(void) state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/resctrl/cat:online:", - resctrl_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, resctrl_offline_cpu); if (state < 0) return state; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 81ed458be70b..ef444a5690e9 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3425,6 +3425,16 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) return 0; } +int resctrl_online_cpu(unsigned int cpu) +{ + lockdep_assert_held(&rdtgroup_mutex); + + /* The cpu is set in default rdtgroup after online. */ + cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); + + return 0; +} + /* * rdtgroup_init - rdtgroup initialization * diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 1f345d24e236..b140ee6de00e 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -219,6 +219,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, u32 closid, enum resctrl_conf_type type); int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d); void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); +int resctrl_online_cpu(unsigned int cpu); /** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid From patchwork Fri Oct 21 13:12:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6711 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp695476wrr; Fri, 21 Oct 2022 06:22:54 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7bLWkEzngZSr2U7iEcMjcVSlklMdxYHX/2VjT1Q6ff6ZyAGravfiSRTVlOe0c18yWw2OzR X-Received: by 2002:a05:6402:371b:b0:460:ff7d:f511 with SMTP id ek27-20020a056402371b00b00460ff7df511mr6844078edb.148.1666358574287; Fri, 21 Oct 2022 06:22:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666358574; cv=none; d=google.com; s=arc-20160816; b=RHoZixgeb6qWrHWAGVd9twhCuU3QXU684EHQoC3YUbmjS+jOyCCtPN0C1IG01EjzN6 690q8U2epd0ketkILgfpKB0ds0s2PSV0dijgmiWa/j/HqOiJ7W53zn67/niD3FtAtGla p7AqYSTWCUkPD42cVphAM8TI6dsX58jmrAEDu5JJ0lnJFImFHRHNBWYLdytSUmCH5dGU KAShJjK4U+ygc1Poo4kmZUc9JLT8u6uePQ+1ooPuIdSOoriWKQZNcnx9xF244FP+/cxd L64zYZWkmvVUQqJd7CxgL9zDBdfVtfLRmgDr9hDUKOaEeGhenCIpnDt5Aj1QA1CpMGK7 lUPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=HHZYiQ0eAr9YV0VFcGzB23ho/xbGwdydMcPIDAl8Rrs=; b=ZLEVZEe1O2M/qk5zuOYX2wnTvetni4HbDQ+iITxMkfGWuD6E5/8F4xZ80n53n6J7hf kNGXXotmDrIWkXXV+CGJLFaOFMZc9N4B8ZZejNSs5PuOsawmCxgFg/qSV801Zjl0jwe/ YBf0vUoF6we8McnAjW8rNgxSi9fQiV5SrinzP/bBN/DMaY+IQ00ZyUEJCfcyZ6iWetBC cOD7VO20C7gORK+jhYKOTV1L6TcVNal5eVzEfaAXzMuggoYA+3NnwQAEn3lTv/hc0jCc Zin2I1qDRmuAEpD9q099h3n16KpbJKwZDLzyPW11mDkNo1o+jxRQchVewSlQTkc3KknS yqTw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p26-20020a170906a01a00b0078db89b2566si17756403ejy.699.2022.10.21.06.22.24; Fri, 21 Oct 2022 06:22:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230440AbiJUNPH (ORCPT + 99 others); Fri, 21 Oct 2022 09:15:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230442AbiJUNNl (ORCPT ); Fri, 21 Oct 2022 09:13:41 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CA077230A87 for ; Fri, 21 Oct 2022 06:13:23 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 05E231477; Fri, 21 Oct 2022 06:13:19 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5C5953F792; Fri, 21 Oct 2022 06:13:10 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 16/18] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu Date: Fri, 21 Oct 2022 13:12:02 +0000 Message-Id: <20221021131204.5581-17-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747303608349229842?= X-GMAIL-MSGID: =?utf-8?q?1747303608349229842?= When a cpu is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU. Once the offline callbacks have been split, cqm_setup_limbo_handler() will be called while the CPU that is going offline is still present in the cpu_mask. Pass the CPU to exclude to cqm_setup_limbo_handler() and mbm_setup_overflow_handler(). These functions can use cpumask_any_but() when selecting the CPU. -1 is used to indicate no CPUs need excluding. Signed-off-by: James Morse --- Both cpumask_any() and cpumask_any_but() return a value >= nr_cpu_ids on error. schedule_delayed_work_on() doesn't appear to check. Add the error handling to be robust. It doesn't look like its possible to hit this. --- arch/x86/kernel/cpu/resctrl/core.c | 6 ++-- arch/x86/kernel/cpu/resctrl/internal.h | 6 ++-- arch/x86/kernel/cpu/resctrl/monitor.c | 39 +++++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 4 +-- 4 files changed, 42 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 511ced743a79..e25d7a581b0d 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -563,12 +563,14 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) if (r == &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl) { if (is_mbm_enabled() && cpu == d->mbm_work_cpu) { cancel_delayed_work(&d->mbm_over); - mbm_setup_overflow_handler(d, 0); + /* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ + mbm_setup_overflow_handler(d, 0, -1); } if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && has_busy_rmid(r, d)) { cancel_delayed_work(&d->cqm_limbo); - cqm_setup_limbo_handler(d, 0); + /* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ + cqm_setup_limbo_handler(d, 0, -1); } } } diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index adbbfaabf70b..96535e359382 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -525,11 +525,13 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first); void mbm_setup_overflow_handler(struct rdt_domain *dom, - unsigned long delay_ms); + unsigned long delay_ms, + int exclude_cpu); void mbm_handle_overflow(struct work_struct *work); void __init intel_rdt_mbm_apply_quirk(void); bool is_mba_sc(struct rdt_resource *r); -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms); +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms, + int exclude_cpu); void cqm_handle_limbo(struct work_struct *work); bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d); void __check_limbo(struct rdt_domain *d, bool force_free); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index a9af7c56a04a..42a7fd6d56c9 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -433,7 +433,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) * setup up the limbo worker. */ if (!has_busy_rmid(r, d)) - cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL); + cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL, -1); set_bit(idx, d->rmid_busy_llc); entry->busy++; } @@ -766,15 +766,27 @@ void cqm_handle_limbo(struct work_struct *work) mutex_unlock(&rdtgroup_mutex); } -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms) +/** + * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this + * domain. + * @delay_ms: How far in the future the handler should run. + * @exclude_cpu: Which CPU the handler should not run on, -1 to pick any CPU. + */ +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms, + int exclude_cpu) { unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - cpu = cpumask_any(&dom->cpu_mask); + if (exclude_cpu == -1) + cpu = cpumask_any(&dom->cpu_mask); + else + cpu = cpumask_any_but(&dom->cpu_mask, exclude_cpu); + dom->cqm_work_cpu = cpu; - schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); + if (cpu < nr_cpu_ids) + schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); } void mbm_handle_overflow(struct work_struct *work) @@ -811,7 +823,14 @@ void mbm_handle_overflow(struct work_struct *work) mutex_unlock(&rdtgroup_mutex); } -void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) +/** + * mbm_setup_overflow_handler() - Schedule the overflow handler to run for this + * domain. + * @delay_ms: How far in the future the handler should run. + * @exclude_cpu: Which CPU the handler should not run on, -1 to pick any CPU. + */ +void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms, + int exclude_cpu) { unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; @@ -819,9 +838,15 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) if (!resctrl_mounted || !resctrl_arch_mon_capable()) return; - cpu = cpumask_any(&dom->cpu_mask); + if (exclude_cpu == -1) + cpu = cpumask_any(&dom->cpu_mask); + else + cpu = cpumask_any_but(&dom->cpu_mask, exclude_cpu); + dom->mbm_work_cpu = cpu; - schedule_delayed_work_on(cpu, &dom->mbm_over, delay); + + if (cpu < nr_cpu_ids) + schedule_delayed_work_on(cpu, &dom->mbm_over, delay); } static int dom_data_init(struct rdt_resource *r) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index ef444a5690e9..3d68a9cb1942 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2254,7 +2254,7 @@ static int rdt_get_tree(struct fs_context *fc) if (is_mbm_enabled()) { r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; list_for_each_entry(dom, &r->domains, list) - mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL); + mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL, -1); } goto out; @@ -3413,7 +3413,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) if (is_mbm_enabled()) { INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow); - mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL); + mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL, -1); } if (is_llc_occupancy_enabled()) From patchwork Fri Oct 21 13:12:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6717 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp699952wrr; Fri, 21 Oct 2022 06:32:24 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6iqO0LBwf9GoRGrSkf2G7MyPmuPaZdYbMBaA8HYiR7sdpbZo3xDeYZRqdw+1N+CWIwf18K X-Received: by 2002:a17:903:3249:b0:181:150c:fcc7 with SMTP id ji9-20020a170903324900b00181150cfcc7mr19937517plb.119.1666359144005; Fri, 21 Oct 2022 06:32:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666359144; cv=none; d=google.com; s=arc-20160816; b=SasxIu3lnJQ0nOQbtzaWtk3ZzKTpEz/HpiY52vlRdw1DBX+ukrtcof5MLo5Zvl7Zfg 3sjjMu9eSBTMgcmIb0i2vakRqioZk4cibkM86M3BrrBdtSAcQRffBqEEkHG6cpOjlb6P nokhL7hNAPawz++t6EE7DHKOSa/9MOmY6LhdAxWJoVPhyRvHB1OAdxU04XH0Edq8nzIc P6Vi+XJ17uXurVFfT88LEclYuSjmbih0uZU5kyEpZJt35+aWwNxiDXqkq2TXn7h0Qgjd yNeaovfB1eHdbsJzJVb6iaX0MTUf+EK1Y3qVJIIFIJ1X8m5Uc0MnPwLzjCUuNUbkfO9y nSQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Nzcce1+Ythej4jTR2tFbMxD2v8cJ9ApoYAFyD5AUKjs=; b=Hd9UUd5bmEjCvTsORREeSY1V5j5dKLzMAzb4EQLvfMldOku38pdXk5YW1nIl4ggl2C q1Mv65iq+tQnBNiAL0LHfkMIHrBILNLfuxq12FCop6LQq9aXPML1bR0C+peeh6zRV0xW M9vyeZdumETq13CMfr2OXUo6MqIHcJgrIcPNoKIa6+vuo1r1pGEfsmPvTBHbX7wckqjR hgWHoRKclj+4Tv/9Q5vFedXdhl0zBeIe/LN387GgeEAD8Y0+izvvo5uif3kpipWho8R6 KKhqi3IxiXCvgGhRB/gBbbNJvMUosRdUfJ+LlvbGhGKrvjtYTmFFxwmwzBjw9+6HiwAQ TEGg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nw9-20020a17090b254900b001f31da6b2d2si6217998pjb.154.2022.10.21.06.32.10; Fri, 21 Oct 2022 06:32:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230269AbiJUNPB (ORCPT + 99 others); Fri, 21 Oct 2022 09:15:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230429AbiJUNNe (ORCPT ); Fri, 21 Oct 2022 09:13:34 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E11DE26DB18 for ; Fri, 21 Oct 2022 06:13:15 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9F38F1D70; Fri, 21 Oct 2022 06:13:21 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 010313F792; Fri, 21 Oct 2022 06:13:12 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 17/18] x86/resctrl: Add cpu offline callback for resctrl work Date: Fri, 21 Oct 2022 13:12:03 +0000 Message-Id: <20221021131204.5581-18-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747304206177797632?= X-GMAIL-MSGID: =?utf-8?q?1747304206177797632?= The resctrl architecture specific code may need to free a domain when a CPU goes offline, it also needs to reset the CPUs PQR_ASSOC register. The resctrl filesystem code needs to move the overflow and limbo work to run on a different CPU, and clear this CPU from the cpu_mask of control and monitor groups. Currently this is all done in core.c and called from resctrl_offline_cpu(), making the split between architecture and filesystem code unclear. Move the filesystem work into a filesystem helper called resctrl_offline_cpu(), and rename the one in core.c resctrl_arch_offline_cpu(). The rdtgroup_mutex is unlocked and locked again in the call in preparation for changing the locking rules for the architecture code. resctrl_offline_cpu() is called before any of the resource/domains are updated, and makes use of the exclude_cpu feature we previously added. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/core.c | 39 ++++---------------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 39 ++++++++++++++++++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 45 insertions(+), 34 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index e25d7a581b0d..00b098b56e42 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -559,20 +559,6 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) return; } - - if (r == &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl) { - if (is_mbm_enabled() && cpu == d->mbm_work_cpu) { - cancel_delayed_work(&d->mbm_over); - /* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ - mbm_setup_overflow_handler(d, 0, -1); - } - if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && - has_busy_rmid(r, d)) { - cancel_delayed_work(&d->cqm_limbo); - /* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ - cqm_setup_limbo_handler(d, 0, -1); - } - } } static void clear_closid_rmid(int cpu) @@ -602,31 +588,15 @@ static int resctrl_arch_online_cpu(unsigned int cpu) return err; } -static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) +static int resctrl_arch_offline_cpu(unsigned int cpu) { - struct rdtgroup *cr; - - list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { - if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask)) { - break; - } - } -} - -static int resctrl_offline_cpu(unsigned int cpu) -{ - struct rdtgroup *rdtgrp; struct rdt_resource *r; mutex_lock(&rdtgroup_mutex); + resctrl_offline_cpu(cpu); + for_each_capable_rdt_resource(r) domain_remove_cpu(cpu, r); - list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) { - if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) { - clear_childcpus(rdtgrp, cpu); - break; - } - } clear_closid_rmid(cpu); mutex_unlock(&rdtgroup_mutex); @@ -926,7 +896,8 @@ static int __init resctrl_late_init(void) state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/resctrl/cat:online:", - resctrl_arch_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, + resctrl_arch_offline_cpu); if (state < 0) return state; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 3d68a9cb1942..f8ea04d95c42 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3435,6 +3435,45 @@ int resctrl_online_cpu(unsigned int cpu) return 0; } +static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) +{ + struct rdtgroup *cr; + + list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { + if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask)) + break; + } +} + +void resctrl_offline_cpu(unsigned int cpu) +{ + struct rdt_domain *d; + struct rdtgroup *rdtgrp; + struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + + lockdep_assert_held(&rdtgroup_mutex); + + list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) { + if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) { + clear_childcpus(rdtgrp, cpu); + break; + } + } + + d = get_domain_from_cpu(cpu, l3); + if (d) { + if (is_mbm_enabled() && cpu == d->mbm_work_cpu) { + cancel_delayed_work(&d->mbm_over); + mbm_setup_overflow_handler(d, 0, cpu); + } + if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && + has_busy_rmid(l3, d)) { + cancel_delayed_work(&d->cqm_limbo); + cqm_setup_limbo_handler(d, 0, cpu); + } + } +} + /* * rdtgroup_init - rdtgroup initialization * diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index b140ee6de00e..530fb21cf9a5 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -220,6 +220,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d); void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); int resctrl_online_cpu(unsigned int cpu); +void resctrl_offline_cpu(unsigned int cpu); /** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid From patchwork Fri Oct 21 13:12:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 6715 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp699985wrr; Fri, 21 Oct 2022 06:32:27 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6YEcb//EVgpg5E1tUlFy2IYmIONcdp/ov5PMtE4lQ1DGV5LSe25SWz+4VKjqwle7d90zPP X-Received: by 2002:a17:902:b101:b0:186:7067:3ea3 with SMTP id q1-20020a170902b10100b0018670673ea3mr6074087plr.59.1666359147023; Fri, 21 Oct 2022 06:32:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666359147; cv=none; d=google.com; s=arc-20160816; b=hAnnCnrLmbTy5ugMBNHuyEgaEEyg8qgGtTURkFMnHMlcgfJ8Ss8lxS7xEy9182aMY5 TxdgAdRxtl47LyJW9w/ymVmyG/8ybjLuxR6j15R4Dk7Pi/S0ngO6gB6BhgyRmtj6ooBJ vx9zxZUFj7qG6tXqslpJyaH5UMFhNdFoyLnhp+XTkiHpJUMlBX6RGZazF3rwTiaavPsN NpNdqYZUql8SAkgmZHFkLUZz/KFVdQLvkimQ+CZIqbF7VK/D1j1wcxGZypNZjUA4yPYL EMsHAn9kSWIVXb16LfS14Bu+2I/sq5ItocroX8dMvgqcEsaxPACFUKBG3+8rM5s7nVQu 3t8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=MyHF4529HX/5dgKLCv5hpzr1NaureL6SnfVoalU2q3w=; b=fX561ogl/q89w4zOhIUHVHYMjgLrVI4sQpKVvETbTfPnR82piefycl1P6BbBzzwF21 kfsvCIHeEsp/pussp0X9f2bPv5EJErWZmYep0WDNEkD2g55EUsq9bY4fl5ur0B9E7xfl PP2q5Cu/V1CNyMGyXszlDzhw5nfBtgNnSrs0iXeE4E9snOOd8MuFAU3ivlJEih9MNU7b GyNaITMUrik1+CcZFYNjNJCyor7HL6O56cgVwQR7GfunWdWBfqWcd/VKkwdoCrp3rBJ1 PhFnRickj6oe4apPnQVq8UCD0fXd8lCmdyGeTlHVhrLmGmpVGOdDBx5MdKPAI4aPuBN5 ORIg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l3-20020a655603000000b00434dd5dc135si26466196pgs.855.2022.10.21.06.32.11; Fri, 21 Oct 2022 06:32:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230424AbiJUNO4 (ORCPT + 99 others); Fri, 21 Oct 2022 09:14:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230434AbiJUNNf (ORCPT ); Fri, 21 Oct 2022 09:13:35 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 625F824CC22 for ; Fri, 21 Oct 2022 06:13:18 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 449001BF3; Fri, 21 Oct 2022 06:13:24 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9AE0E3F792; Fri, 21 Oct 2022 06:13:15 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, peternewman@google.com Subject: [PATCH 18/18] x86/resctrl: Separate arch and fs resctrl locks Date: Fri, 21 Oct 2022 13:12:04 +0000 Message-Id: <20221021131204.5581-19-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20221021131204.5581-1-james.morse@arm.com> References: <20221021131204.5581-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747304209383058180?= X-GMAIL-MSGID: =?utf-8?q?1747304209383058180?= resctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list. Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against cpu hotplug. The cpu hotplug code already has a mechanism to do this using cpus_read_lock(). MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory. Because resctrl_{on,off}line_cpu() take the rdtgroup_mutex as part of a cpuhp callback, cpus_read_lock() must always be taken first. rdtgroup_schemata_write() already does this. All but one of the filesystem code's domain list walkers are currently protected by the rdtgroup_mutex taken in rdtgroup_kn_lock_live(). The exception is rdt_bit_usage_show() which takes the lock directly. Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers. rdt_bit_usage_show() can walk the domain list under rcu_read_lock(). The other filesystem list walkers need to be able to sleep. Add cpus_read_lock() to rdtgroup_kn_lock_live() so that the cpuhp callbacks can't be invoked when file system operations are occurring. Add lockdep_assert_cpus_held() in the cases where the rdtgroup_kn_lock_live() call isn't obvious. Resctrl's domain online/offline calls now need to take the rdtgroup_mutex themselves. Signed-off-by: James Morse --- arch/x86/kernel/cpu/resctrl/core.c | 33 ++++++++------ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 14 ++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 3 ++ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 3 ++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 54 ++++++++++++++++++++--- include/linux/resctrl.h | 2 +- 6 files changed, 84 insertions(+), 25 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 00b098b56e42..c4e4ce5bf87f 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -25,8 +25,14 @@ #include #include "internal.h" -/* Mutex to protect rdtgroup access. */ -DEFINE_MUTEX(rdtgroup_mutex); +/* + * rdt_domain structures are kfree()d when their last cpu goes offline, + * and allocated when the first cpu in a new domain comes online. + * The rdt_resource's domain list is updated when this happens. The domain + * list is protected by RCU, but callers can also take the cpus_read_lock() + * to prevent modification if they need to sleep. All writers take this mutex: + */ +static DEFINE_MUTEX(domain_list_lock); /* * The cached resctrl_pqr_state is strictly per CPU and can never be @@ -489,6 +495,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) struct rdt_domain *d; int err; + lockdep_assert_held(&domain_list_lock); + d = rdt_find_domain(r, id, &add_pos); if (IS_ERR(d)) { pr_warn("Couldn't find cache id for CPU %d\n", cpu); @@ -522,11 +530,12 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) return; } - list_add_tail(&d->list, add_pos); + list_add_tail_rcu(&d->list, add_pos); err = resctrl_online_domain(r, d); if (err) { - list_del(&d->list); + list_del_rcu(&d->list); + synchronize_rcu(); domain_free(hw_dom); } } @@ -547,7 +556,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) cpumask_clear_cpu(cpu, &d->cpu_mask); if (cpumask_empty(&d->cpu_mask)) { resctrl_offline_domain(r, d); - list_del(&d->list); + list_del_rcu(&d->list); + synchronize_rcu(); /* * rdt_domain "d" is going to be freed below, so clear @@ -575,30 +585,27 @@ static void clear_closid_rmid(int cpu) static int resctrl_arch_online_cpu(unsigned int cpu) { struct rdt_resource *r; - int err; - mutex_lock(&rdtgroup_mutex); + mutex_lock(&domain_list_lock); for_each_capable_rdt_resource(r) domain_add_cpu(cpu, r); clear_closid_rmid(cpu); + mutex_unlock(&domain_list_lock); - err = resctrl_online_cpu(cpu); - mutex_unlock(&rdtgroup_mutex); - - return err; + return resctrl_online_cpu(cpu); } static int resctrl_arch_offline_cpu(unsigned int cpu) { struct rdt_resource *r; - mutex_lock(&rdtgroup_mutex); resctrl_offline_cpu(cpu); + mutex_lock(&domain_list_lock); for_each_capable_rdt_resource(r) domain_remove_cpu(cpu, r); clear_closid_rmid(cpu); - mutex_unlock(&rdtgroup_mutex); + mutex_unlock(&domain_list_lock); return 0; } diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index 842266d1d148..dc700a09d661 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -209,6 +209,9 @@ static int parse_line(char *line, struct resctrl_schema *s, struct rdt_domain *d; unsigned long dom_id; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP && r->rid == RDT_RESOURCE_MBA) { rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n"); @@ -314,6 +317,9 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) int cpu; u32 idx; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) return -ENOMEM; @@ -384,11 +390,9 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, return -EINVAL; buf[nbytes - 1] = '\0'; - cpus_read_lock(); rdtgrp = rdtgroup_kn_lock_live(of->kn); if (!rdtgrp) { rdtgroup_kn_unlock(of->kn); - cpus_read_unlock(); return -ENOENT; } rdt_last_cmd_clear(); @@ -452,7 +456,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, out: rdtgroup_kn_unlock(of->kn); - cpus_read_unlock(); return ret ?: nbytes; } @@ -472,6 +475,9 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo bool sep = false; u32 ctrl_val; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + seq_printf(s, "%*s:", max_name_width, schema->name); list_for_each_entry(dom, &r->domains, list) { if (sep) @@ -534,7 +540,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, int evtid, int first) { /* When picking a cpu from cpu_mask, ensure it can't race with cpuhp */ - lockdep_assert_held(&rdtgroup_mutex); + lockdep_assert_cpus_held(); /* * setup the parameters to pass to mon_event_count() to read the data. diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 42a7fd6d56c9..777085df07cf 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -414,6 +414,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) u32 idx; int err; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID); diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 48f196007e9c..690dc6c9f3d4 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -830,6 +830,9 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d) struct rdt_domain *d_i; bool ret = false; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (!zalloc_cpumask_var(&cpu_with_psl, GFP_KERNEL)) return true; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index f8ea04d95c42..5febb6dfaea0 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -35,6 +35,10 @@ DEFINE_STATIC_KEY_FALSE(rdt_enable_key); DEFINE_STATIC_KEY_FALSE(rdt_mon_enable_key); DEFINE_STATIC_KEY_FALSE(rdt_alloc_enable_key); + +/* Mutex to protect rdtgroup access. */ +DEFINE_MUTEX(rdtgroup_mutex); + static struct kernfs_root *rdt_root; struct rdtgroup rdtgroup_default; LIST_HEAD(rdt_all_groups); @@ -930,7 +934,8 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, mutex_lock(&rdtgroup_mutex); hw_shareable = r->cache.shareable_bits; - list_for_each_entry(dom, &r->domains, list) { + rcu_read_lock(); + list_for_each_entry_rcu(dom, &r->domains, list) { if (sep) seq_putc(seq, ';'); sw_shareable = 0; @@ -986,8 +991,10 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, } sep = true; } + rcu_read_unlock(); seq_putc(seq, '\n'); mutex_unlock(&rdtgroup_mutex); + return 0; } @@ -1227,6 +1234,9 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp) struct rdt_domain *d; u32 ctrl; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + list_for_each_entry(s, &resctrl_schema_all, list) { r = s->res; if (r->rid == RDT_RESOURCE_MBA) @@ -1860,6 +1870,9 @@ static int set_cache_qos_cfg(int level, bool enable) struct rdt_domain *d; int cpu; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (level == RDT_RESOURCE_L3) update = l3_qos_cfg_update; else if (level == RDT_RESOURCE_L2) @@ -2052,6 +2065,7 @@ struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn) atomic_inc(&rdtgrp->waitcount); kernfs_break_active_protection(kn); + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); /* Was this group deleted while we waited? */ @@ -2069,6 +2083,7 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn) return; mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); if (atomic_dec_and_test(&rdtgrp->waitcount) && (rdtgrp->flags & RDT_DELETED)) { @@ -2365,6 +2380,9 @@ static int reset_all_ctrls(struct rdt_resource *r) struct rdt_domain *d; int i, cpu; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) return -ENOMEM; @@ -2645,6 +2663,9 @@ static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn, struct rdt_domain *dom; int ret; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + list_for_each_entry(dom, &r->domains, list) { ret = mkdir_mondata_subdir(parent_kn, dom, r, prgrp); if (ret) @@ -3328,7 +3349,8 @@ static void domain_destroy_mon_state(struct rdt_domain *d) kfree(d->mbm_local); } -void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) +static void _resctrl_offline_domain(struct rdt_resource *r, + struct rdt_domain *d) { lockdep_assert_held(&rdtgroup_mutex); @@ -3363,6 +3385,13 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) domain_destroy_mon_state(d); } +void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) +{ + mutex_lock(&rdtgroup_mutex); + _resctrl_offline_domain(r, d); + mutex_unlock(&rdtgroup_mutex); +} + static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) { u32 idx_limit = resctrl_arch_system_num_rmid_idx(); @@ -3394,7 +3423,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) return 0; } -int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) +static int _resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) { int err; @@ -3425,12 +3454,23 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) return 0; } +int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) +{ + int err; + + mutex_lock(&rdtgroup_mutex); + err = _resctrl_online_domain(r, d); + mutex_unlock(&rdtgroup_mutex); + + return err; +} + int resctrl_online_cpu(unsigned int cpu) { - lockdep_assert_held(&rdtgroup_mutex); - + mutex_lock(&rdtgroup_mutex); /* The cpu is set in default rdtgroup after online. */ cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); + mutex_unlock(&rdtgroup_mutex); return 0; } @@ -3451,8 +3491,7 @@ void resctrl_offline_cpu(unsigned int cpu) struct rdtgroup *rdtgrp; struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; - lockdep_assert_held(&rdtgroup_mutex); - + mutex_lock(&rdtgroup_mutex); list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) { if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) { clear_childcpus(rdtgrp, cpu); @@ -3472,6 +3511,7 @@ void resctrl_offline_cpu(unsigned int cpu) cqm_setup_limbo_handler(d, 0, cpu); } } + mutex_unlock(&rdtgroup_mutex); } /* diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 530fb21cf9a5..c5af581790dc 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -153,7 +153,7 @@ struct resctrl_schema; * @cache_level: Which cache level defines scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. - * @domains: All domains for this resource + * @domains: RCU list of all domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.