From patchwork Fri Jul 28 16:42:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 127767 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp579034vqg; Fri, 28 Jul 2023 10:14:18 -0700 (PDT) X-Google-Smtp-Source: APBJJlFQAQhKl3ndBsyNd3h1DyGYlanhvCyC6i2TB1IeOkC60GeIwGT8x0RTittaV2PeubNU30m/ X-Received: by 2002:a05:6a20:430b:b0:13a:52ce:13cc with SMTP id h11-20020a056a20430b00b0013a52ce13ccmr2532416pzk.51.1690564458148; Fri, 28 Jul 2023 10:14:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690564458; cv=none; d=google.com; s=arc-20160816; b=BlWTCcyY3M/lN9Vc6/WVZNy0ZatwdVtZ4G1BxAwMSjxc7oAj2mxSb8nys9YTtrvOTr qatipHoaIsh325FxuP5LLeP8vATx+HUVywBS66bCVyHoPLPJLCxeOw0daLzPCK06AHOu +Kau8BcwlotfPSdZLG4Q1y5c8BSSVMVGtAhjr7JY6ItHUVKv+wwaogqHpRBS1ExwemjP bl2bA/E0zJX6O1aKui+WwEFrdbaCvGirIrYMJkgobdlMIbzq719g2/EczyW/6Pgz/RWD jmYGixvT0gUX3oiSo6E++xnAsf/DsDdOtSk2z0FKtQAXH3yoPwr/F67GAem/QBpUpc4v nmBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=pqwzG9EHEMQEBSaIklNJDqRTcsL9ENqnBXqFMdidqTU=; fh=n9L+e/Mo/wOalMjVbltm4zKnQoPh0Qbxoe+x3eznw8A=; b=FXtgRjNkxR//vXYAy0q8TVUiVo7/UTZTRqcNsKgsJYnbl3TZ2LFXkKmJLQMe7POaoq zow/rjU/i6H1sQXc7mrvBxzzDEWvID83GiCLf6J5sb6txEzcWsxGKi/0qP9tMuS3cSNE mpPZ1JTzP++UZo6eyNCpvEJDfQGSTKNZ9XmbpCwLcJhVEhaE3tTPeAH2saQDgqSMROZ3 4pX/0Qj/UkqDVC4GwIvD5J4KUpumwbBSppPxk5bs4e+FtPkhVyfMHpzr91JJgyl4r59I s2iN15Izgsg4Ob2mGPce83qWUO5fXNYReEK+z0DxC/S80EGsIuVu5ZJhxZcTK35uKTrX 124Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q202-20020a632ad3000000b0055785a37147si24395pgq.590.2023.07.28.10.14.05; Fri, 28 Jul 2023 10:14:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233779AbjG1Qp5 (ORCPT + 99 others); Fri, 28 Jul 2023 12:45:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233332AbjG1Qpa (ORCPT ); Fri, 28 Jul 2023 12:45:30 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id D9DE74ED0 for ; Fri, 28 Jul 2023 09:43:56 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9DEDDD75; Fri, 28 Jul 2023 09:44:39 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D79493F67D; Fri, 28 Jul 2023 09:43:53 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com Subject: [PATCH v5 13/24] x86/resctrl: Queue mon_event_read() instead of sending an IPI Date: Fri, 28 Jul 2023 16:42:43 +0000 Message-Id: <20230728164254.27562-14-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230728164254.27562-1-james.morse@arm.com> References: <20230728164254.27562-1-james.morse@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772685317501151202 X-GMAIL-MSGID: 1772685317501151202 Intel is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed. MPAM's CSU monitors are used to back the 'llc_occupancy' monitor file. The CSU counter is allowed to return 'not ready' for a small number of micro-seconds after programming. To allow one CSU hardware monitor to be used for multiple control or monitor groups, the CPU accessing the monitor needs to be able to block when configuring and reading the counter. Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs. These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced. mon_event_read() already invokes mon_event_count() via IPI, which means this isn't possible. On systems using nohz-full, some CPUs need to be interrupted to run kernel work as they otherwise stay in user-space running realtime workloads. Interrupting these CPUs should be avoided, and scheduling work on them may never complete. Change mon_event_read() to pick a housekeeping CPU, (one that is not using nohz_full) and schedule mon_event_count() and wait. If all the CPUs in a domain are using nohz-full, then an IPI is used as the fallback. This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code). This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in monitor.c. When the IPI fallback is used on machines where MPAM needs to make an access on multiple CPUs, the counter read will always fail. Tested-by: Shaopeng Tan Reviewed-by: Peter Newman Tested-by: Peter Newman Signed-off-by: James Morse --- Changes since v2: * Use cpumask_any_housekeeping() and fallback to an IPI if needed. Changes since v3: * Actually include the IPI fallback code. Changes since v4: * Tinkered with existing capitalisation. --- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 28 +++++++++++++++++++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- 2 files changed, 27 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index b44c487727d4..bd263b9a0abd 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "internal.h" /* @@ -520,12 +521,24 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, return ret; } +static int smp_mon_event_count(void *arg) +{ + mon_event_count(arg); + + return 0; +} + void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first) { + int cpu; + + /* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */ + lockdep_assert_held(&rdtgroup_mutex); + /* - * setup the parameters to send to the IPI to read the data. + * Setup the parameters to pass to mon_event_count() to read the data. */ rr->rgrp = rdtgrp; rr->evtid = evtid; @@ -534,7 +547,18 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val = 0; rr->first = first; - smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1); + cpu = cpumask_any_housekeeping(&d->cpu_mask); + + /* + * cpumask_any_housekeeping() prefers housekeeping CPUs, but + * are all the CPUs nohz_full? If yes, pick a CPU to IPI. + * MPAM's resctrl_arch_rmid_read() is unable to read the + * counters on some platforms if its called in irq context. + */ + if (tick_nohz_full_cpu(cpu)) + smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1); + else + smp_call_on_cpu(cpu, smp_mon_event_count, rr, false); } int rdtgroup_mondata_show(struct seq_file *m, void *arg) diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 62350bbd23e0..32569354c4f1 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -597,7 +597,7 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) } /* - * This is called via IPI to read the CQM/MBM counters + * This is scheduled by mon_event_read() to read the CQM/MBM counters * on a domain. */ void mon_event_count(void *info)