From patchwork Thu Sep 14 17:21:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 139723 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp505891vqi; Thu, 14 Sep 2023 10:23:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEwkZ0SqS0RqflmldvE/J1DKVBLZWKuV5fZc2+RxjwNI7GnUh/cN73IMLrfRvtVY8kqZ0z0 X-Received: by 2002:a05:6358:e486:b0:142:f1d5:ef89 with SMTP id by6-20020a056358e48600b00142f1d5ef89mr2896843rwb.5.1694712231076; Thu, 14 Sep 2023 10:23:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694712231; cv=none; d=google.com; s=arc-20160816; b=obSGvrl4UcSfh0qGm9TaPPhPwMt3VxfqZCUUWdBnZ84BWoKie6PzOnlZnl0xgf7NwE 7xDmcElybr2OaF4F+l4c27sI7/il3KD0h4t3uESWIrdrB/8rB4oqIMY4RvpK3LO6aRqW EWi5Kc1AS3XKVLwtKSDzPXagF41TsZhDjAojSfoNg+cd19PGk3Mg/9/cKI/xhbNL2jl0 JTwU1mEcK1/mi+cRRA64/Fi1I7hn90sfYE5sUPDknA0bFmofkB48k1ujSHcdM2dPvAs0 qE+glRGkz1115nbWPMnbrj/iWPND2M7v25HnWnqXNyPyJeVBxGMOuMWeWZZc5T/Pq5wK mGPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=5nmwMDtCic/CyX0b0MOtbO1oliwaPUuVjB3Q4PkccaA=; fh=6Nl2MjDBXofbhMXyzAhf7L7loTtEBflFxRNey74qSFc=; b=eWuX/rCNVcvThPKT+kXbnSCzg8Pf9pdczxepJQL3WdybyJ201DlZLBXCwBmOOclRtY 6zs+h3nIuflnnBT1LEH0uhczKeOjPlsYP5evss9/xpUeQw9fk6HWdPWvBofLdx5F2uMV ByeiFtRdApKf0FWnhGzRsCnsG6pEW+WHhgWG67BHHXnTzi8vamPV4NcmIEUe+VNsKIjL cq7HRefp2oAsC3a1FOwx7SVGf2NfBlQzd43yQEn2KJSENaqEhr5wTs6Ub88eWUR/STZ5 osjm3O0BTWQF0saKM0YNWJIDLIOxl7v8OfoRdwh4CJhpoUZsY7tU3n3WOZdZfCzVCZV+ ZNig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id v202-20020a6361d3000000b00565e6e7a80asi1809786pgb.349.2023.09.14.10.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 10:23:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 2207E837D4BA; Thu, 14 Sep 2023 10:23:23 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240416AbjINRXL (ORCPT + 33 others); Thu, 14 Sep 2023 13:23:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239846AbjINRWr (ORCPT ); Thu, 14 Sep 2023 13:22:47 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 120052724 for ; Thu, 14 Sep 2023 10:22:33 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0518413D5; Thu, 14 Sep 2023 10:23:10 -0700 (PDT) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 01BBB3F5A1; Thu, 14 Sep 2023 10:22:29 -0700 (PDT) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com Subject: [PATCH v6 12/24] x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow Date: Thu, 14 Sep 2023 17:21:26 +0000 Message-Id: <20230914172138.11977-13-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230914172138.11977-1-james.morse@arm.com> References: <20230914172138.11977-1-james.morse@arm.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Thu, 14 Sep 2023 10:23:23 -0700 (PDT) X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777034572263965270 X-GMAIL-MSGID: 1777034572263965270 The limbo and overflow code picks a CPU to use from the domain's list of online CPUs. Work is then scheduled on these CPUs to maintain the limbo list and any counters that may overflow. cpumask_any() may pick a CPU that is marked nohz_full, which will either penalise the work that CPU was dedicated to, or delay the processing of limbo list or counters that may overflow. Perhaps indefinitely. Delaying the overflow handling will skew the bandwidth values calculated by mba_sc, which expects to be called once a second. Add cpumask_any_housekeeping() as a replacement for cpumask_any() that prefers housekeeping CPUs. This helper will still return a nohz_full CPU if that is the only option. The CPU to use is re-evaluated each time the limbo/overflow work runs. This ensures the work will move off a nohz_full CPU once a housekeeping CPU is available. Reviewed-by: Shaopeng Tan Tested-by: Shaopeng Tan Tested-By: Peter Newman Signed-off-by: James Morse --- Changes since v3: * typos fixed Changes since v4: * Made temporary variables unsigned Changes since v5: * Restructured cpumask_any_housekeeping() to avoid later churn. --- arch/x86/kernel/cpu/resctrl/internal.h | 24 ++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 17 ++++++++++++----- 2 files changed, 36 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index f06d3d3e0808..37bb3de37a4a 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #define L3_QOS_CDP_ENABLE 0x01ULL @@ -55,6 +56,29 @@ /* Max event bits supported */ #define MAX_EVT_CONFIG_BITS GENMASK(6, 0) +/** + * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that + * aren't marked nohz_full + * @mask: The mask to pick a CPU from. + * + * Returns a CPU in @mask. If there are housekeeping CPUs that don't use + * nohz_full, these are preferred. + */ +static inline unsigned int cpumask_any_housekeeping(const struct cpumask *mask) +{ + unsigned int cpu, hk_cpu; + + cpu = cpumask_any(mask); + if (!tick_nohz_full_cpu(cpu)) + return cpu; + + hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask); + if (hk_cpu < nr_cpu_ids) + cpu = hk_cpu; + + return cpu; +} + struct rdt_fs_context { struct kernfs_fs_context kfc; bool enable_cdpl2; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 0bbed8c62d42..993837e46db1 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -782,9 +782,9 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, void cqm_handle_limbo(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL); - int cpu = smp_processor_id(); struct rdt_resource *r; struct rdt_domain *d; + int cpu; mutex_lock(&rdtgroup_mutex); @@ -793,8 +793,10 @@ void cqm_handle_limbo(struct work_struct *work) __check_limbo(d, false); - if (has_busy_rmid(d)) + if (has_busy_rmid(d)) { + cpu = cpumask_any_housekeeping(&d->cpu_mask); schedule_delayed_work_on(cpu, &d->cqm_limbo, delay); + } mutex_unlock(&rdtgroup_mutex); } @@ -804,7 +806,7 @@ void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms) unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - cpu = cpumask_any(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->cqm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); @@ -814,10 +816,10 @@ void mbm_handle_overflow(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL); struct rdtgroup *prgrp, *crgrp; - int cpu = smp_processor_id(); struct list_head *head; struct rdt_resource *r; struct rdt_domain *d; + int cpu; mutex_lock(&rdtgroup_mutex); @@ -838,6 +840,11 @@ void mbm_handle_overflow(struct work_struct *work) update_mba_bw(prgrp, d); } + /* + * Re-check for housekeeping CPUs. This allows the overflow handler to + * move off a nohz_full CPU quickly. + */ + cpu = cpumask_any_housekeeping(&d->cpu_mask); schedule_delayed_work_on(cpu, &d->mbm_over, delay); out_unlock: @@ -851,7 +858,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) if (!static_branch_likely(&rdt_mon_enable_key)) return; - cpu = cpumask_any(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->mbm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->mbm_over, delay); }