From patchwork Tue Feb 13 18:44:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200556 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp739519dyb; Tue, 13 Feb 2024 10:48:16 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUkPPS5435Gnl8cSi3eM6p9VoFckDcM2OihxbFwcSsLyarW33xMBbUr7LzROQNonqukikyLrFunMBW8Ha+GMApO7+MTcg== X-Google-Smtp-Source: AGHT+IFaRvKmkVxJlqXNDwKZnByG6fE6bEyTp711TaHHPrYW1WVd9//KZQeF5BJkPOAjwXohSEx5 X-Received: by 2002:a17:906:b21a:b0:a3c:8495:6b7e with SMTP id p26-20020a170906b21a00b00a3c84956b7emr183331ejz.1.1707850096704; Tue, 13 Feb 2024 10:48:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850096; cv=pass; d=google.com; s=arc-20160816; b=YjIXjETgNzZs5aWcQuijFcmAGqyRmBNQLu5BDAADLRjJYlN+focHQVnG/h5y4C0je1 wZfrjTsJrHnXwIt7WKaFe1o1M5WWom9/a7sAsurPfqmdRJBQuJuGAk39qS+T/ZHsZmE9 rlY8w44Aot3uML4K/6002xLX7POCtdZa6lRTAhU1LmCXV3v4jbQvzBRVjT5ZxGshUF/p L5CY0ni8/RCx8kB6upaWPEXhFRDIjWhPA1/X1OGRA3DI4WAOSg4xuVjTHqW5dMJt8Je+ MF/E42fTkS3rvvTjr1IluK2DRvIwFRQb8QUjr6fc99HEuxD5Xk1NRYYs2QPyzPeBZGGt fvew== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=bhQxGZVAEGUDnTH73bBgu+E4LP8c3vmP/LlRL93+uCE=; fh=wuucEwf2KxK+uHK6Y7ypIjudfr9Txw56+Yyo5iRnnys=; b=nYBkToRWkYV7eeqnzhaDcc2W19wZMx/PjM54hiWwl7EsDl6WPuwwblzIpveUJ//t94 Ym9jAc/EDXMAQaJNQWYOPvA+0gEi84Cj96mZVq9GrgMeFt894kTkuXcPAeblHJcfmPKK 65CwlI6QkN/MbYTK/DcQJaLpSyYFDdYeEtI5RaNgtm6aD1BlV7DdHnKfZonk7wY4KE+P AKB101fgftkZMQZhAL87DNdNTzVvprohjVTDhV9aDRgyd/EQGtgUjX+/CDOTP86KgWo/ vvEl5w+TuIEfM9p0hv9CQC9M9sLJPXZ8NC7PMmO056cOFPDGkZsiyLb1Be8p+bg3kdOl 7v7A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXokg0FhB2GMeKUzBwqq1DSMuYkTKMd7YP51zuKdvqKoi00XNJFYY77rYSsuWIrD/8Yw+AMo6UKQ+WZNaj9SIEf1wWXOg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id v18-20020a170906339200b00a3cf5d9315asi1032461eja.482.2024.02.13.10.48.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:48:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 02DC31F24866 for ; Tue, 13 Feb 2024 18:48:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 364BB612DB; Tue, 13 Feb 2024 18:45:46 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A3BA060EF3 for ; Tue, 13 Feb 2024 18:45:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849944; cv=none; b=lre71dxAPShje3zi66tZTF6BKn4nnsGFzxK9BsZFMzB0ec5aiQPVQrDX8lxoMGFL0qobsE/y5EpCdG7INDcy1vh+i5Idynfj9OO1mfikIXCSJjh3Qq/0mU45AY5Cwg9faRtMTEzWH1a7InEAObYz5dOFlQw6engffUJQURLEoQk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849944; c=relaxed/simple; bh=tCk5Rp6ufC7ekXhYD/3kjr90yolxWKM1koNMSXSV0dY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IRUgEIplDqbvtEK9mnIt05n9Ucgc3oa64ga/CGxoOlMgTKGSeQCNy9aZJGIhfGj2PBfqXt2zoraLxUJgpgOuMm9y/8LM31xyVyEMnVRJmNtlId+uOvY6TPU1/M13bv6S9VU8uyX2do+mxUseCN4pILmhpi1AtLBZZ4B7cZxfI1U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6B9081516; Tue, 13 Feb 2024 10:46:23 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0C89B3F766; Tue, 13 Feb 2024 10:45:38 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 12/24] x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow Date: Tue, 13 Feb 2024 18:44:26 +0000 Message-Id: <20240213184438.16675-13-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810623023326686 X-GMAIL-MSGID: 1790810623023326686 The limbo and overflow code picks a CPU to use from the domain's list of online CPUs. Work is then scheduled on these CPUs to maintain the limbo list and any counters that may overflow. cpumask_any() may pick a CPU that is marked nohz_full, which will either penalise the work that CPU was dedicated to, or delay the processing of limbo list or counters that may overflow. Perhaps indefinitely. Delaying the overflow handling will skew the bandwidth values calculated by mba_sc, which expects to be called once a second. Add cpumask_any_housekeeping() as a replacement for cpumask_any() that prefers housekeeping CPUs. This helper will still return a nohz_full CPU if that is the only option. The CPU to use is re-evaluated each time the limbo/overflow work runs. This ensures the work will move off a nohz_full CPU once a housekeeping CPU is available. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v3: * typos fixed Changes since v4: * Made temporary variables unsigned Changes since v5: * Restructured cpumask_any_housekeeping() to avoid later churn. Changes since v6: * Update mbm_work_cpu/cqm_work_cpu when rescheduling. --- arch/x86/kernel/cpu/resctrl/internal.h | 24 ++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 20 +++++++++++++------- 2 files changed, 37 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index b7b9d9230bef..81f5de916db8 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -7,6 +7,7 @@ #include #include #include +#include #include @@ -55,6 +56,29 @@ /* Max event bits supported */ #define MAX_EVT_CONFIG_BITS GENMASK(6, 0) +/** + * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that + * aren't marked nohz_full + * @mask: The mask to pick a CPU from. + * + * Returns a CPU in @mask. If there are housekeeping CPUs that don't use + * nohz_full, these are preferred. + */ +static inline unsigned int cpumask_any_housekeeping(const struct cpumask *mask) +{ + unsigned int cpu, hk_cpu; + + cpu = cpumask_any(mask); + if (!tick_nohz_full_cpu(cpu)) + return cpu; + + hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask); + if (hk_cpu < nr_cpu_ids) + cpu = hk_cpu; + + return cpu; +} + struct rdt_fs_context { struct kernfs_fs_context kfc; bool enable_cdpl2; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 101f1b112d17..38f85e53ca93 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -761,7 +761,6 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, void cqm_handle_limbo(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL); - int cpu = smp_processor_id(); struct rdt_domain *d; mutex_lock(&rdtgroup_mutex); @@ -770,8 +769,11 @@ void cqm_handle_limbo(struct work_struct *work) __check_limbo(d, false); - if (has_busy_rmid(d)) - schedule_delayed_work_on(cpu, &d->cqm_limbo, delay); + if (has_busy_rmid(d)) { + d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask); + schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo, + delay); + } mutex_unlock(&rdtgroup_mutex); } @@ -781,7 +783,7 @@ void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms) unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - cpu = cpumask_any(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->cqm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); @@ -791,7 +793,6 @@ void mbm_handle_overflow(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL); struct rdtgroup *prgrp, *crgrp; - int cpu = smp_processor_id(); struct list_head *head; struct rdt_resource *r; struct rdt_domain *d; @@ -815,7 +816,12 @@ void mbm_handle_overflow(struct work_struct *work) update_mba_bw(prgrp, d); } - schedule_delayed_work_on(cpu, &d->mbm_over, delay); + /* + * Re-check for housekeeping CPUs. This allows the overflow handler to + * move off a nohz_full CPU quickly. + */ + d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask); + schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay); out_unlock: mutex_unlock(&rdtgroup_mutex); @@ -828,7 +834,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) if (!static_branch_likely(&rdt_mon_enable_key)) return; - cpu = cpumask_any(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->mbm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->mbm_over, delay); }