From patchwork Tue Feb 13 18:44:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200570 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp750614dyb; Tue, 13 Feb 2024 11:06:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXNlIfOZdJdkSb92bZdLgn9Pf/V5oH1qx8GKpbYit9l0+lRwiJ/0FErqIOykjsonxeg77O2e91yEHSGBskbATjC+D2cDA== X-Google-Smtp-Source: AGHT+IEf1PkeQK/WWs6nVYajHCl+V6/eCL3J79k3dZWwrHUEXs2PGRhhAD90msCaDTNmRgA4BGCJ X-Received: by 2002:a17:90a:ca85:b0:298:a446:f929 with SMTP id y5-20020a17090aca8500b00298a446f929mr3920525pjt.11.1707851206392; Tue, 13 Feb 2024 11:06:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851206; cv=pass; d=google.com; s=arc-20160816; b=TkTJOY2MCl7P7FNjQZeluD1M+WGT71DX589SyhmTqRAF73rOI6ocEAv5g8ZCisMwIr iR1SY5SAjt3RWbplRpm+BLbgiGtZQrpSN3RW2ndQLltJ36wV2nuNZhkSCQWTBs6U6b5t MAPVKOYWnxlzSisi1RLON++Eg7FByhLDh8+1OYes5Re1SIDfrUN8LuhInm9u+RKzXhPl yNxu3EGUFe2bK7YsEealKf/AFvJI0NfjmskuPBYVAz2tmzStdg8qs+dYfp391h3LR8Gv sB64MyjoXL9Df60luWRMqos6pkB+WlSG1MBRbHmfBG29Dx/vhHPNYm/yfhVU2zQtbi3a jC1w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=FYL0k8ZSSa6/z6RfNHiTNqBJJVqEE+rEFz9X2dYGM2g=; fh=10QB5fcqKXTOqGF87TZL/6iDK/nGLe8DOYFsQBOjjnY=; b=M+BSYujzJW0O02J2h4E9LEK/MovlEf2jOzdiljVhlpb7z0jcOPyi0qUn28SzNMqAUc BHeUJqU/ye2jwxcSt8rQqBxueG3wJ4bKN+2s3WE+FMHoQMWhWFz5APvsxV2HqY81d7Xk ZN7eZc4VNNUfY1K4W5SzUgqHj6j1AsVO/MOKHk41tMYEgKx2BuJ5ruS9UXhiqJ2SBoHq 1zWAJHpSAh9cFAQJXnZw0uowwlsROSlDuyr1kAfLde63Aoz1eLaljM9UfRqjQBw8byDL Q/CIx8dcw4fqxht+wrP2bNzoSVrp0VVSZsMrgTNmlhNJxxeLrsXVTk1Y9toNINgxfYQt K0ew==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64092-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64092-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCVpg+jyALOc9kvqCgvI8YG/hrPeMZspKIpjJrdxwzz6i6ZSlKdZKIg5MIdVHJlh1OXa4gUPTRACbrN1lOHPAppzaUet9Q== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id t19-20020a639553000000b005d92a3f1196si2405298pgn.592.2024.02.13.11.06.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:06:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64092-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64092-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64092-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 8BA62B239F3 for ; Tue, 13 Feb 2024 18:45:36 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0239C57885; Tue, 13 Feb 2024 18:45:09 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8B8A71F618 for ; Tue, 13 Feb 2024 18:45:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849907; cv=none; b=rfsQBsAwNMu7ct4x4dFy/Fux/3NH8L0zWyGEmOGD7Jlt9/UCJrCfsWavHkXbofGM1dN/VZ77iJyLduFzW8eyFMgxetl3DgbNwEGyd36OAsV6rJvxdkxZByzqDIw0lYQl4/+quQLkPF3uCOeO9Y7s2oDposddtis8V6vfrCFkyL0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849907; c=relaxed/simple; bh=N9LgQW81xVaWBAUrrbwt8I6fqkUZ41bThuldzq3VtBk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=F8vQELYdFmqkFfRm+xTKgec1C1qPqgZYFwD/WnCLC4pmvXQ+/6yL4+OfK0JbLdHYBlmHeWbgTiK+YaOsZBxk9Fuw/KzGtziKn/KSJGURrYNfNbIXwQeeSnkYIu1qXbLUWTUJKOcqgjMOqIokZ405HoipPOE98T5Yt3GoZAI3aiU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B2F6DA7; Tue, 13 Feb 2024 10:45:46 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id ECE6F3F766; Tue, 13 Feb 2024 10:45:01 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Frederic Weisbecker Subject: [PATCH v9 01/24] tick/nohz: Move tick_nohz_full_mask declaration outside the #ifdef Date: Tue, 13 Feb 2024 18:44:15 +0000 Message-Id: <20240213184438.16675-2-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811786704839901 X-GMAIL-MSGID: 1790811786704839901 tick_nohz_full_mask lists the CPUs that are nohz_full. This is only needed when CONFIG_NO_HZ_FULL is defined. tick_nohz_full_cpu() allows a specific CPU to be tested against the mask, and evaluates to false when CONFIG_NO_HZ_FULL is not defined. The resctrl code needs to pick a CPU to run some work on, a new helper prefers housekeeping CPUs by examining the tick_nohz_full_mask. Hiding the declaration behind #ifdef CONFIG_NO_HZ_FULL forces all the users to be behind an ifdef too. Move the tick_nohz_full_mask declaration, this lets callers drop the ifdef, and guard access to tick_nohz_full_mask with IS_ENABLED() or something like tick_nohz_full_cpu(). The definition does not need to be moved as any callers should be removed at compile time unless CONFIG_NO_HZ_FULL is defined. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Thomas Gleixner Acked-by: Reinette Chatre # for resctrl dependency CC: Frederic Weisbecker --- include/linux/tick.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index 716d17f31c45..0fb903838dfb 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -164,9 +164,16 @@ static inline u64 get_cpu_idle_time_us(int cpu, u64 *unused) { return -1; } static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; } #endif /* !CONFIG_NO_HZ_COMMON */ +/* + * Mask of CPUs that are nohz_full. + * + * Users should be guarded by CONFIG_NO_HZ_FULL or a tick_nohz_full_cpu() + * check. + */ +extern cpumask_var_t tick_nohz_full_mask; + #ifdef CONFIG_NO_HZ_FULL extern bool tick_nohz_full_running; -extern cpumask_var_t tick_nohz_full_mask; static inline bool tick_nohz_full_enabled(void) { From patchwork Tue Feb 13 18:44:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200552 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp738311dyb; Tue, 13 Feb 2024 10:45:50 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWJPMq0ukvsd6ivKsPC6rQJwlxIprV7xu3hs4shbEBeIDtwh6d08YeO++tK9wxZ2j/cqNZHl9kW/nfbdwCHXjrWRhJA0A== X-Google-Smtp-Source: AGHT+IEUKHYtWFUsFglTljDuEyJ81k5biy2nD2X/7mjg0nCaJmE64Xn2dmIDDsvdfug+AMZpUOjF X-Received: by 2002:a05:6402:134f:b0:561:fd3f:d75a with SMTP id y15-20020a056402134f00b00561fd3fd75amr374807edw.28.1707849950388; Tue, 13 Feb 2024 10:45:50 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707849950; cv=pass; d=google.com; s=arc-20160816; b=FK3UzCkKpLU+s9n4lsRBf7YomgXW7KZoJV5t1Mu1+On+UEhyomh3MKrC+j5VF6gejb nBMirps4icTyG7hwGilFXPlrhKzRL5/1MyVrv7ZJz9cXNcUemYXRcHlTmH8z5I0TTATB oK0dTghwC+3jv1emSRYqQ15bZztofd7ufwgtDyMvn938WDlRCp37/u6h+fSbur6P9q8/ LcNw0W/GIAqtqq/yH334Pj6z06t9yJnLLQe6H/KtsW/Q6//BqXloA4MLXYT0fgdCAruN 5F/g/8yH7WEkOT+8gXHWftxmOyi60PgA3Qmyox8La/a1Ek/hyr7sD6eik5Pa+r/E7c47 Q7wQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=Ubd+KdrHeNuK4PbZyo8WR0IZ9Sea2C/GizjNaf/Dq1Y=; fh=NCE4Kspnr11lETLj+y+ZrRYg+Ch2b440ymcFSKRzp2s=; b=n7R34WQw5Fcatm/zDHzMDsQBWoQxP4XB/1b+Bngq6v0Li0i0oPRKbvr7/DGjGN2rqS KzXZf/w0I8rs3er0n02nQjZG32WkW+F32pn/SqSPQq+CioVGzSGEG3b3PIe4WdQtCkfF /aVCiH4AivDR+afgl0U5mFKhgBPgvaJTcsc1KSNJ6dmwNzbdBTGs4TOqbiwAB33TvWjy eA63zfvssI25wS4C6L2zNH1ITGx3zlxlnvqsmpbEWIQYP7oWSxxJ5PscesY11oeC+F7f P+6I229mNQu2fgUlLTSGf1RAb48rxDNm5x4aKAThFb68oKWm+BftZxdqGqr1GNsY1XlE nFiA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64093-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64093-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCVJll34QSB8kcrPHIZuaYjo8VHXWTFoXG1MF+Nc9uUS3E1EtmJ/6yDzF1ohfgKYht7Ugvr6zFaePp2Eo7on9yLOIaZIFg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id i1-20020a056402054100b0055ff1384e15si3988904edx.225.2024.02.13.10.45.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:45:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64093-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64093-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64093-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 06FFA1F24D3B for ; Tue, 13 Feb 2024 18:45:50 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3B8366087C; Tue, 13 Feb 2024 18:45:12 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 095355812E for ; Tue, 13 Feb 2024 18:45:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849910; cv=none; b=rs9R3C9D1RHwGouYsOY88kK9KtE1EvM2ug6CXE8MG/Aje2jXssesgNiwtBryyfT4FTAqh64/IyiDdUfcSQKPfaNrhPXwXQn95ukqhz5NXjdyAQODajg5Y3wMLXwct4Ce0boZ6TQeODL6olLI6pguqpYtZ7SWpkyqIrKvk/C8UUo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849910; c=relaxed/simple; bh=cnu/fw90MUyfw/nm4T5u6mCM9ebJAuqXfvQQKR65YCQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E/Jv+atH02zO82X6ucNsWuV+ERpv37M8Pb3CxqRPTPRyNBfIX5e0uiy+Uik1T4orz5sLossukihLe36rd3mTVId14k5ul+35M9qYCr4ZylrjNhD1vCa2n7FDknkZjN9rwmtTFCdywp0BFTG/uYRGstW2WY6TWd8kHnQmEvRBYcs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B8358FEC; Tue, 13 Feb 2024 10:45:49 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 532FC3F766; Tue, 13 Feb 2024 10:45:05 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 02/24] x86/resctrl: kfree() rmid_ptrs from resctrl_exit() Date: Tue, 13 Feb 2024 18:44:16 +0000 Message-Id: <20240213184438.16675-3-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810469652452446 X-GMAIL-MSGID: 1790810469652452446 rmid_ptrs[] is allocated from dom_data_init() but never free()d. While the exit text ends up in the linker script's DISCARD section, the direction of travel is for resctrl to be/have loadable modules. Add resctrl_put_mon_l3_config() to cleanup any memory allocated by rdt_get_mon_l3_config(). There is no reason to backport this to a stable kernel. Signed-off-by: James Morse Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Babu Moger Reviewed-by: Reinette Chatre --- Changes since v5: * This patch is new Changes since v6: * Removed struct rdt_resource argument, added __exit markers to match the only caller. * Adedd a whole stack of functions to maintain symmetry. Changes since v7: * Moved the eventual kfree() call to be after rdtgroup_exit() Changes since v8: * Removed an unused parameter from some unused code. --- arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 15 +++++++++++++++ 3 files changed, 22 insertions(+) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index aa9810a64258..9641c42d0f85 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -990,8 +990,14 @@ late_initcall(resctrl_late_init); static void __exit resctrl_exit(void) { + struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + cpuhp_remove_state(rdt_online); + rdtgroup_exit(); + + if (r->mon_capable) + rdt_put_mon_l3_config(); } __exitcall(resctrl_exit); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 52e7e7deee10..61c763604fc9 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -544,6 +544,7 @@ void closid_free(int closid); int alloc_rmid(void); void free_rmid(u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); +void __exit rdt_put_mon_l3_config(void); bool __init rdt_cpu_has(int flag); void mon_event_count(void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 3a6c069614eb..3a73db0579d8 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -719,6 +719,16 @@ static int dom_data_init(struct rdt_resource *r) return 0; } +static void __exit dom_data_exit(void) +{ + mutex_lock(&rdtgroup_mutex); + + kfree(rmid_ptrs); + rmid_ptrs = NULL; + + mutex_unlock(&rdtgroup_mutex); +} + static struct mon_evt llc_occupancy_event = { .name = "llc_occupancy", .evtid = QOS_L3_OCCUP_EVENT_ID, @@ -814,6 +824,11 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r) return 0; } +void __exit rdt_put_mon_l3_config(void) +{ + dom_data_exit(); +} + void __init intel_rdt_mbm_apply_quirk(void) { int cf_index; From patchwork Tue Feb 13 18:44:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200553 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp738415dyb; Tue, 13 Feb 2024 10:46:03 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW3hd4qgDfldiqKp54WETjxpDbqxLRztMz7AgzxlLIplNPlvaQ4jDnLBlwzQk7sFWh8o4CUg7BOlLwuZfrtGLPGB/E0wA== X-Google-Smtp-Source: AGHT+IEI9gUVvsA3on7CNS2QFwXuvztImwiEVPAx2cqqU0vEBmZE6XesuCZ+2qJ+8PtxpMWGGZ7u X-Received: by 2002:a05:6214:2268:b0:68c:a6ab:cedf with SMTP id gs8-20020a056214226800b0068ca6abcedfmr438962qvb.28.1707849963111; Tue, 13 Feb 2024 10:46:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707849963; cv=pass; d=google.com; s=arc-20160816; b=QPZKq0e7I6uCmp/qfgfLyyzHklNVAnKxMUNyzMjvKcTAQ4Lvvet8XDSDOLqN399gXT euY08t6weeGsmac+eJJeT0Ag7fcg2JRxUqcEPm+V1M45uMc9ledQA5bPTzipxjA8lCbV i2jildFNs2ZUUEe+K5Nq8Jq2jwTQdf9MVOTAyq4yR/XKbZ8zu2pUY/WIWi6A84BOm/61 /yi1Tz+HB+rfbYkU1KhDqFa2Fy0SPeOP214/PbVlZkxdVtJoQLgHf2ZnXjguyEiKN8+B VlISmyb5S5hveFv3wWjfeple6Bhmbq/v5viyQl5/BrF76ema26n1jIoNOCS11yMPCoSG m74A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=i/PWf0O7w77zQqYguicZljE1MqS+U4xfA2ohOvgrMKc=; fh=bjQffL4QSkRacjXWEz+X548T+NsKL6NWYfekoIwVOpM=; b=Z6QnVjgEIpCTL474IlbByGK1khhHhLcgxPpV81nF3sGnM1/1tL/JP7MAsF0uOzKGnl Yt+FCRnspHW+ZgdnVn5llIF8S+sA9lZGXepgHBW6SIidz2jkrdmQRX7hClBM/jCNnQx4 Wkl69yCIFKMkIv+VRmIu+Idt//hOFfH5ApeVWbExrq97y9Qt1bDgvhwqwZ6dGoyPBC+q sh3lTWnyEUYmc4Igka8DZg3mx87acXhk0//ctKl3nCV5ic2R7LMXr62eWWG7w8OnESyy 3s1ZnFIhgfd3ZCS/53egWidfSHwLOrb923M+AMSRPY2iT+psSYbJUN1cXN//MiO1JyqV 3FtQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64094-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64094-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWfSVAjdycaXQpbTsbSyxjRjPe/64/ZO9rFUbM4NM7tFz3NeS3Ar05cNDXKo0bRgHQtIYW5f2v5AK2TbPVsLrxwXz+zWg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id p10-20020a05621421ea00b0068cb4c6ecdbsi3448262qvj.317.2024.02.13.10.46.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:46:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64094-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64094-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64094-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id E20A21C20864 for ; Tue, 13 Feb 2024 18:46:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6C7C26089D; Tue, 13 Feb 2024 18:45:15 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6E93C60881 for ; Tue, 13 Feb 2024 18:45:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849914; cv=none; b=i+kyYcLScuE5xclZK28AYW0lmYje+C3JBL5JxGBMQzB7cZt+efSsJmOmhj5G0mht7m5MBFT9YMWjbGXaeBdDFdesCa42OGhrfJGQSNSdavdhbZbZYRja1uERBDnCiiKFQO7YH6uXRqEwoX5siruBIC3qzGEJMWmnch6ADRaVZTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849914; c=relaxed/simple; bh=e9JWYiCmeFsWinCmEWYZOTDNfITJiLE8EatsPyrLEfU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=e21cLK1Uef0QuLrhhti6QXmCdrlXaco+v6R5BI0p2zZw7Q15L1p/B+rEh5psyCpQ7nDX95HshEp08xA8GI3wsb5FMpNIGhhVYIcXfPQ9oxg6bpQuyo25GSwN0RXSHPKb+7EmyeioJudr9OM0dQSk1UEB0gkLAFqhwo/TNc7Y0b4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2FE2F1042; Tue, 13 Feb 2024 10:45:53 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AB1CB3F766; Tue, 13 Feb 2024 10:45:08 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger , =?utf-8?q?Ilpo_J=C3=A4rvinen?= Subject: [PATCH v9 03/24] x86/resctrl: Create helper for RMID allocation and mondata dir creation Date: Tue, 13 Feb 2024 18:44:17 +0000 Message-Id: <20240213184438.16675-4-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810482892951971 X-GMAIL-MSGID: 1790810482892951971 When monitoring is supported, each monitor and control group is allocated an RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID. MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has. The RMID allocation needs to move to be after the CLOSID has been allocated. Move the RMID allocation and mondata dir creation to a helper. Signed-off-by: James Morse Tested-by: Peter Newman Tested-by: Shaopeng Tan Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Ilpo Järvinen Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger Reviewed-by: David Hildenbrand --- Changes since v4: * Fixed typo in commit message, moved some words around. --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 42 +++++++++++++++++--------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index aa24343f1d23..4ea5a871be49 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3288,6 +3288,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) return ret; } +static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) +{ + int ret; + + if (!rdt_mon_capable) + return 0; + + ret = alloc_rmid(); + if (ret < 0) { + rdt_last_cmd_puts("Out of RMIDs\n"); + return ret; + } + rdtgrp->mon.rmid = ret; + + ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); + if (ret) { + rdt_last_cmd_puts("kernfs subdir error\n"); + free_rmid(rdtgrp->mon.rmid); + return ret; + } + + return 0; +} + static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, const char *name, umode_t mode, enum rdt_group_type rtype, struct rdtgroup **r) @@ -3360,20 +3384,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - if (rdt_mon_capable) { - ret = alloc_rmid(); - if (ret < 0) { - rdt_last_cmd_puts("Out of RMIDs\n"); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); + if (ret) + goto out_destroy; - ret = mkdir_mondata_all(kn, rdtgrp, &rdtgrp->mon.mon_data_kn); - if (ret) { - rdt_last_cmd_puts("kernfs subdir error\n"); - goto out_idfree; - } - } kernfs_activate(kn); /* @@ -3381,8 +3395,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, */ return 0; -out_idfree: - free_rmid(rdtgrp->mon.rmid); out_destroy: kernfs_put(rdtgrp->kn); kernfs_remove(rdtgrp->kn); From patchwork Tue Feb 13 18:44:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200567 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp742886dyb; Tue, 13 Feb 2024 10:55:16 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVYKLoFhY/Ug1Qp4XBq1KVYit4QbBj6bMuc8v9e5ADDW/cH6ra8wZvmLznT3dtsVi1q9Jac8G2ZimC7Og57KtQnK3JBnw== X-Google-Smtp-Source: AGHT+IGGjCwLen/3NxTbZyStjtv3vxa2EPhmYrju1xtQ2iho2spV9VAWNvZZT+t/aghWe4ESGW7W X-Received: by 2002:a17:90a:dc12:b0:298:c3b8:5e15 with SMTP id i18-20020a17090adc1200b00298c3b85e15mr404467pjv.14.1707850516078; Tue, 13 Feb 2024 10:55:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850516; cv=pass; d=google.com; s=arc-20160816; b=SQS6j4w4LYTrSbPSGxu00/stNEVyCINe+mp+w/vTnmKPupTeZHq8GScHCw0Id5Lux3 RAIYvQssg0QS4oVw8+S4ZiJfjnbJTbYlQ3uw4VglQvb2Gjaw1cqU9tDIxQjZqT6/oI72 bUKD9/ZNM6V4gd05Oq2llKwuQEaF6JXdb9JjctZKdlul47nt1c/oRIh9jrH3oPeYiK01 FSANi0ZxwfjX+ASFtoRqzANUBwnP/Y0CEKQNvRxgj20SE6HhMvaMMkLH48j6lVcI5JNV IEIt62HjzWJAjHs5QHHy2UCduHHB9d/53vDXZu8eQqE+7OPiWRAgfdZIeatxeEXpw23B QIVw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=hmLpwX2wiOP/xgTST0rAoarfL9MYtsAs81Y9M0YoZuM=; fh=59i1OFjG0+TtAS1HTCAtk0aaAOuQs3Xjv4pSc2gIHho=; b=ppCImOw5zw/mmvRXZmIrSToxbKvUitcDSV5dhNGiZ8SRTTQ5zc0FpQBF9KdcvRWWAO zQ3e3ySq3RMWTOr7CHgnDIxkLVl5KIGR2k9a03rhz6CxVn6O9LzvNO7HVvae/5uNfHGq JZsNoru0fAy/064szKq2c6iF0UI3mpwQsllu14BLliRPMQm19OTWdDAObu7AsSFeSWPF +In7cfhzFlVJPHuaEooiyBSWfg7bH7Dbh4UqkseYL8Y1BuJ3iBXFoc/FqITGW6CNxs0W R6CCUo0iOf7zgt9fKF+sYsOu1Bc738sx9xaLNDw4JGO8MJ2JUTZGww5QkMbe7DBDZC1w K4Zg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64095-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64095-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWlfvSozdemX4/oTl+I8203FD9U+SnxGf+Haec2eTcdSsCHLBc/G8Hnzj6pnVvkqG8l8veBBkOsDOKoSDgXOCfgR7kXPw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id v23-20020a17090abb9700b002969bca1b53si2394811pjr.96.2024.02.13.10.55.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:55:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64095-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64095-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64095-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 93F9728DB2C for ; Tue, 13 Feb 2024 18:46:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5A32A60BA2; Tue, 13 Feb 2024 18:45:19 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E294A60B81 for ; Tue, 13 Feb 2024 18:45:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849917; cv=none; b=YIMburLYXqd+GRb3xJd306lo3ifFqlK2zc811WCjXdyf/stAJm58MbgqSmASQXKyKkL9hBWYYkDYfF1ZeWLNS1iIVKA4nzGT9h9flLiAKwAYHrTMoNQRP06iKErMDiYUEIVp/1VRNN5Tr4r2BstvXLDQKUwwvOdbkLNOHgm32KY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849917; c=relaxed/simple; bh=YcQFSWDQA1m6PdSQ0Ol/gs7JJFkLp0C6qtBwalerQcc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ekEWN2In9NXxcQbKi9l7LxmZsYEbaBsbobU2XojtCWCArseZG3vSjZy8K+xoXsMwXy9KTlJvgeRW/Mov/p7nHIAH8aF0XYAjCdRx8OZU6JMUab4vKjQayi80Id2Wh/7ADVAkzKYxDL/Ohau+btMUYda6XgKJs9jY7qnU5o3t/HA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8740413D5; Tue, 13 Feb 2024 10:45:56 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2C8623F766; Tue, 13 Feb 2024 10:45:12 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 04/24] x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare() Date: Tue, 13 Feb 2024 18:44:18 +0000 Message-Id: <20240213184438.16675-5-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811062777822663 X-GMAIL-MSGID: 1790811062777822663 RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID. MPAM's equivalent of RMID is not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has. The RMID allocation needs to move to be after the CLOSID has been allocated. Move the RMID allocation out of mkdir_rdt_prepare() to occur in its caller, after the mkdir_rdt_prepare() call. This allows the RMID allocator to know the CLOSID. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v2: * Moved kernfs_activate() later to preserve atomicity of files being visible Changes since v5: * Renamed out_id_free as out_closid_free. --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 35 +++++++++++++++++++------- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 4ea5a871be49..f455a10b74ab 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3312,6 +3312,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) return 0; } +static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp) +{ + if (rdt_mon_capable) + free_rmid(rgrp->mon.rmid); +} + static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, const char *name, umode_t mode, enum rdt_group_type rtype, struct rdtgroup **r) @@ -3384,12 +3390,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); - if (ret) - goto out_destroy; - - kernfs_activate(kn); - /* * The caller unlocks the parent_kn upon success. */ @@ -3408,7 +3408,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp) { kernfs_remove(rgrp->kn); - free_rmid(rgrp->mon.rmid); rdtgroup_remove(rgrp); } @@ -3430,12 +3429,21 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, prgrp = rdtgrp->mon.parent; rdtgrp->closid = prgrp->closid; + ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); + if (ret) { + mkdir_rdt_prepare_clean(rdtgrp); + goto out_unlock; + } + + kernfs_activate(rdtgrp->kn); + /* * Add the rdtgrp to the list of rdtgrps the parent * ctrl_mon group has to track. */ list_add_tail(&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list); +out_unlock: rdtgroup_kn_unlock(parent_kn); return ret; } @@ -3466,9 +3474,16 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret = 0; rdtgrp->closid = closid; + + ret = mkdir_rdt_prepare_rmid_alloc(rdtgrp); + if (ret) + goto out_closid_free; + + kernfs_activate(rdtgrp->kn); + ret = rdtgroup_init_alloc(rdtgrp); if (ret < 0) - goto out_id_free; + goto out_rmid_free; list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups); @@ -3488,7 +3503,9 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, out_del_list: list_del(&rdtgrp->rdtgroup_list); -out_id_free: +out_rmid_free: + mkdir_rdt_prepare_rmid_free(rdtgrp); +out_closid_free: closid_free(closid); out_common_fail: mkdir_rdt_prepare_clean(rdtgrp); From patchwork Tue Feb 13 18:44:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200571 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp751219dyb; Tue, 13 Feb 2024 11:07:42 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUS5MzvUD0Wu7gEUA+mk5liuI16hFrjhEYIScJzDobU00Nz1Zbb02UdQefppMv5vaOqgay2FyBI7KPeEJGyxzKpnKMN+Q== X-Google-Smtp-Source: AGHT+IElaQjWBo9Kge60aAZiaAVVsS/baVrBUChLjPJZhjNV+24RYvTjEoPUHm51M+5UKIGYy5oQ X-Received: by 2002:a17:903:298d:b0:1d9:b789:b1b9 with SMTP id lm13-20020a170903298d00b001d9b789b1b9mr681798plb.40.1707851262194; Tue, 13 Feb 2024 11:07:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851262; cv=pass; d=google.com; s=arc-20160816; b=oGNWeDUrAO1xBdw1O3IZI9Ud1M7puoebL5mK6XN7aeI7OQZrhdKkC6IALnXjtlv4yy ujIYyQmDDCYhPx9bJB5WqolgzCTaTzMYQC25LgN/RgIn1A/XXvHz26+VTwaZrpdJ/3E+ YFQ4HrYPFyH56XKiEv5iguX9mXhH+NoPNy8QfluqloYyvH3sV6kGeUCP8a+tyhN6v+lx 6a8HUlcErk9yXtQrfcQj7ZprT+kr/ec8byNbCMkjOuG3A44VxFZkBRmxB0ODMcprZ6w2 l7R6NFePjO9gL2AeA9letd3o39Gli+egDyg7nOVfTdLzxbj5VL+PlSbgJVh/d7RCISHC Wm8g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=O0bf0BwylMAj6rgipP4vdA5g7rGD87UOcoHufVEr260=; fh=LrX7/fEezrsj0wC8qr3bkAOSb/QbiuQlERayqeqwkpM=; b=glruYpZxZKduLDTDfFvydwk4zjj/Y0l57mgclyNhjtIzeIOFYw2bZFiLMaPNX8Y3PO 7tQy9wzO9fGPMaY6ipOF4MHeEbAwX2eMacZWmTqdYnYNE23Mf1xHrNBy9lPmJBKohq4G 44QAMOPZRxT9MQuvpyG+/29rRkhHcSPMeNaYQVmFG76uVOi6RI5K0n4E/hwj/kiYYFK6 ldKXuAWWSbzRq4t2zJ6VGGdSnaRYw0Whn5IIMqR3CTtO1XzVHVotdq0tYyw8ozdOb8/z O3bw5w2CxiRfMp/jurTFsbPIB5Sh+nmMDiGAsaLymNRxN7ayJu9u9GXgMUktJ/EpWihr azeg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64096-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64096-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCUw/vvacGKoyCe8LOSghCsFZiuoNDBVEiUuqsJ31eEkDX7Z0nx9taYAkBTSdQQM2+P9e0BomBUlzFMlTJ9sMJ0tE3EVjg== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id y3-20020a17090322c300b001d76a322667si2558259plg.72.2024.02.13.11.07.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:07:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64096-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64096-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64096-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 8E74EB26773 for ; Tue, 13 Feb 2024 18:46:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1AA6F1E49E; Tue, 13 Feb 2024 18:45:25 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 17DA660B9C for ; Tue, 13 Feb 2024 18:45:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849921; cv=none; b=hGL6DUTUfQfbSAKMcDKrI+8ymxF0YsyivutDXlQ5Dwod2FWiL14MtIaqEchzW1hBlXhuJ4LKarDTFH+3PG7SFfrWLnIg9FhTAJFg4E8Qe1h2dReAEfOcqK8CQWIP4tF3N1hhrkZ31oQhQGYJk6TDx7tIUQK8ZyFoFWPy9hyKgAQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849921; c=relaxed/simple; bh=+cI0zo69QHig7B+XUzI4+SP7IUCyYfDOcVRw0taJjdU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VUXpBRa0sjSmQ2hevmZYNLTRu+a1JRI7XsvN13m21ZcFR9xVNQDGh7xGtTZs6l7scq5cwQun1CkL9WyIazuWuoIHBhkrtg3F2fIIe3gUG+cm+1Txh0Qe2u9gGSHd9NhwdQxVFYcHJ9IEISouZuF3zvlefrlDLRQjkCOkXsAapD4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ED9AB143D; Tue, 13 Feb 2024 10:45:59 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 76F643F766; Tue, 13 Feb 2024 10:45:15 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 05/24] x86/resctrl: Track the closid with the rmid Date: Tue, 13 Feb 2024 18:44:19 +0000 Message-Id: <20240213184438.16675-6-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811845258021817 X-GMAIL-MSGID: 1790811845258021817 x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID. MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID. i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups. To allow resctrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Xin Hao Reviewed-by: Reinette Chatre --- Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing... Changes since v1: * Added comment in struct rmid_entry Changes since v2: * Moved X86_RESCTRL_BAD_CLOSID from a subsequent patch Chances since v3: * Renamed X86_RESCTRL_BAD_CLOSID to EMPTY * Clarified a few comments and kernel-doc Changes since v5: * Use entry->closid from the iterator, instead of the parent control group. * Move the reserved defines into this patch to reduce the churn. * Added some kernel doc. * Renamed some arch closid parameters as 'unused'. Changes since v6: * Changes to comments. Changes since v7: * Changes to comments. --- arch/x86/include/asm/resctrl.h | 7 +++ arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 73 +++++++++++++++-------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++-- include/linux/resctrl.h | 16 ++++- 6 files changed, 77 insertions(+), 37 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 255a78d9d906..cc6e1bce7b1a 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -7,6 +7,13 @@ #include #include +/* + * This value can never be a valid CLOSID, and is used when mapping a + * (closid, rmid) pair to an index and back. On x86 only the RMID is + * needed. The index is a software defined value. + */ +#define X86_RESCTRL_EMPTY_CLOSID ((u32)~0) + /** * struct resctrl_pqr_state - State cache for the PQR MSR * @cur_rmid: The cached Resource Monitoring ID diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 61c763604fc9..ae0e3338abc4 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -542,7 +542,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); void closid_free(int closid); int alloc_rmid(void); -void free_rmid(u32 rmid); +void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); void __exit rdt_put_mon_l3_config(void); bool __init rdt_cpu_has(int flag); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 3a73db0579d8..3dad4134d2c9 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -24,7 +24,20 @@ #include "internal.h" +/** + * struct rmid_entry - dirty tracking for all RMID. + * @closid: The CLOSID for this entry. + * @rmid: The RMID for this entry. + * @busy: The number of domains with cached data using this RMID. + * @list: Member of the rmid_free_lru list when busy == 0. + * + * Depending on the architecture the correct monitor is accessed using + * both @closid and @rmid, or @rmid only. + * + * Take the rdtgroup_mutex when accessing. + */ struct rmid_entry { + u32 closid; u32 rmid; int busy; struct list_head list; @@ -136,7 +149,7 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val) return val; } -static inline struct rmid_entry *__rmid_entry(u32 rmid) +static inline struct rmid_entry *__rmid_entry(u32 closid, u32 rmid) { struct rmid_entry *entry; @@ -190,7 +203,8 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom, } void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid) + u32 unused, u32 rmid, + enum resctrl_event_id eventid) { struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); struct arch_mbm_state *am; @@ -230,7 +244,8 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) } int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid, u64 *val) + u32 unused, u32 rmid, enum resctrl_event_id eventid, + u64 *val) { struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); @@ -285,9 +300,9 @@ void __check_limbo(struct rdt_domain *d, bool force_free) if (nrmid >= r->num_rmid) break; - entry = __rmid_entry(nrmid); + entry = __rmid_entry(X86_RESCTRL_EMPTY_CLOSID, nrmid);// temporary - if (resctrl_arch_rmid_read(r, d, entry->rmid, + if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val)) { rmid_dirty = true; } else { @@ -342,7 +357,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) cpu = get_cpu(); list_for_each_entry(d, &r->domains, list) { if (cpumask_test_cpu(cpu, &d->cpu_mask)) { - err = resctrl_arch_rmid_read(r, d, entry->rmid, + err = resctrl_arch_rmid_read(r, d, entry->closid, + entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val); if (err || val <= resctrl_rmid_realloc_threshold) @@ -366,7 +382,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) list_add_tail(&entry->list, &rmid_free_lru); } -void free_rmid(u32 rmid) +void free_rmid(u32 closid, u32 rmid) { struct rmid_entry *entry; @@ -375,7 +391,7 @@ void free_rmid(u32 rmid) lockdep_assert_held(&rdtgroup_mutex); - entry = __rmid_entry(rmid); + entry = __rmid_entry(closid, rmid); if (is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); @@ -383,8 +399,8 @@ void free_rmid(u32 rmid) list_add_tail(&entry->list, &rmid_free_lru); } -static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 rmid, - enum resctrl_event_id evtid) +static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid, + u32 rmid, enum resctrl_event_id evtid) { switch (evtid) { case QOS_L3_MBM_TOTAL_EVENT_ID: @@ -396,20 +412,21 @@ static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 rmid, } } -static int __mon_event_count(u32 rmid, struct rmid_read *rr) +static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) { struct mbm_state *m; u64 tval = 0; if (rr->first) { - resctrl_arch_reset_rmid(rr->r, rr->d, rmid, rr->evtid); - m = get_mbm_state(rr->d, rmid, rr->evtid); + resctrl_arch_reset_rmid(rr->r, rr->d, closid, rmid, rr->evtid); + m = get_mbm_state(rr->d, closid, rmid, rr->evtid); if (m) memset(m, 0, sizeof(struct mbm_state)); return 0; } - rr->err = resctrl_arch_rmid_read(rr->r, rr->d, rmid, rr->evtid, &tval); + rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid, + &tval); if (rr->err) return rr->err; @@ -421,6 +438,7 @@ static int __mon_event_count(u32 rmid, struct rmid_read *rr) /* * mbm_bw_count() - Update bw count from values previously read by * __mon_event_count(). + * @closid: The closid used to identify the cached mbm_state. * @rmid: The rmid used to identify the cached mbm_state. * @rr: The struct rmid_read populated by __mon_event_count(). * @@ -429,7 +447,7 @@ static int __mon_event_count(u32 rmid, struct rmid_read *rr) * __mon_event_count() is compared with the chunks value from the previous * invocation. This must be called once per second to maintain values in MBps. */ -static void mbm_bw_count(u32 rmid, struct rmid_read *rr) +static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) { struct mbm_state *m = &rr->d->mbm_local[rmid]; u64 cur_bw, bytes, cur_bytes; @@ -456,7 +474,7 @@ void mon_event_count(void *info) rdtgrp = rr->rgrp; - ret = __mon_event_count(rdtgrp->mon.rmid, rr); + ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr); /* * For Ctrl groups read data from child monitor groups and @@ -467,7 +485,8 @@ void mon_event_count(void *info) if (rdtgrp->type == RDTCTRL_GROUP) { list_for_each_entry(entry, head, mon.crdtgrp_list) { - if (__mon_event_count(entry->mon.rmid, rr) == 0) + if (__mon_event_count(entry->closid, entry->mon.rmid, + rr) == 0) ret = 0; } } @@ -578,7 +597,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) resctrl_arch_update_one(r_mba, dom_mba, closid, CDP_NONE, new_msr_val); } -static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) +static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, + u32 closid, u32 rmid) { struct rmid_read rr; @@ -593,12 +613,12 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) if (is_mbm_total_enabled()) { rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID; rr.val = 0; - __mon_event_count(rmid, &rr); + __mon_event_count(closid, rmid, &rr); } if (is_mbm_local_enabled()) { rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID; rr.val = 0; - __mon_event_count(rmid, &rr); + __mon_event_count(closid, rmid, &rr); /* * Call the MBA software controller only for the @@ -606,7 +626,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) * the software controller explicitly. */ if (is_mba_sc(NULL)) - mbm_bw_count(rmid, &rr); + mbm_bw_count(closid, rmid, &rr); } } @@ -663,11 +683,11 @@ void mbm_handle_overflow(struct work_struct *work) d = container_of(work, struct rdt_domain, mbm_over.work); list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) { - mbm_update(r, d, prgrp->mon.rmid); + mbm_update(r, d, prgrp->closid, prgrp->mon.rmid); head = &prgrp->mon.crdtgrp_list; list_for_each_entry(crgrp, head, mon.crdtgrp_list) - mbm_update(r, d, crgrp->mon.rmid); + mbm_update(r, d, crgrp->closid, crgrp->mon.rmid); if (is_mba_sc(NULL)) update_mba_bw(prgrp, d); @@ -710,10 +730,11 @@ static int dom_data_init(struct rdt_resource *r) } /* - * RMID 0 is special and is always allocated. It's used for all - * tasks that are not monitored. + * RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and + * are always allocated. These are used for the rdtgroup_default + * control group, which will be setup later in rdtgroup_init(). */ - entry = __rmid_entry(0); + entry = __rmid_entry(RESCTRL_RESERVED_CLOSID, RESCTRL_RESERVED_RMID); list_del(&entry->list); return 0; diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 8f559eeae08e..65bee6f11015 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -752,7 +752,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp) * anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); ret = 0; goto out; @@ -787,7 +787,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret = rdtgroup_locksetup_user_restore(rdtgrp); if (ret) { - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); return ret; } diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index f455a10b74ab..ad7da7254f4d 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2837,7 +2837,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list; list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) { - free_rmid(sentry->mon.rmid); + free_rmid(sentry->closid, sentry->mon.rmid); list_del(&sentry->mon.crdtgrp_list); if (atomic_read(&sentry->waitcount) != 0) @@ -2877,7 +2877,7 @@ static void rmdir_all_sub(void) cpumask_or(&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); kernfs_remove(rdtgrp->kn); list_del(&rdtgrp->rdtgroup_list); @@ -3305,7 +3305,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); if (ret) { rdt_last_cmd_puts("kernfs subdir error\n"); - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); return ret; } @@ -3315,7 +3315,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp) { if (rdt_mon_capable) - free_rmid(rgrp->mon.rmid); + free_rmid(rgrp->closid, rgrp->mon.rmid); } static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, @@ -3574,7 +3574,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask) update_closid_rmid(tmpmask, NULL); rdtgrp->flags = RDT_DELETED; - free_rmid(rdtgrp->mon.rmid); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); /* * Remove the rdtgrp from the parent ctrl_mon group's list @@ -3620,8 +3620,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask) cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask); update_closid_rmid(tmpmask, NULL); + free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); closid_free(rdtgrp->closid); - free_rmid(rdtgrp->mon.rmid); rdtgroup_ctrl_remove(rdtgrp); diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 66942d7fba7f..bd4ec22b5a96 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -6,6 +6,10 @@ #include #include +/* CLOSID, RMID value used by the default control group */ +#define RESCTRL_RESERVED_CLOSID 0 +#define RESCTRL_RESERVED_RMID 0 + #ifdef CONFIG_PROC_CPU_RESCTRL int proc_resctrl_show(struct seq_file *m, @@ -225,6 +229,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * for this resource and domain. * @r: resource that the counter should be read from. * @d: domain that the counter should be read from. + * @closid: closid that matches the rmid. Depending on the architecture, the + * counter may match traffic of both @closid and @rmid, or @rmid + * only. * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. @@ -235,20 +242,25 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * 0 on success, or -EIO, -EINVAL etc on error. */ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid, u64 *val); + u32 closid, u32 rmid, enum resctrl_event_id eventid, + u64 *val); + /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. * @r: The domain's resource. * @d: The rmid's domain. + * @closid: closid that matches the rmid. Depending on the architecture, the + * counter may match traffic of both @closid and @rmid, or @rmid only. * @rmid: The rmid whose counter values should be reset. * @eventid: The eventid whose counter values should be reset. * * This can be called from any CPU. */ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid); + u32 closid, u32 rmid, + enum resctrl_event_id eventid); /** * resctrl_arch_reset_rmid_all() - Reset all private state associated with From patchwork Tue Feb 13 18:44:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200554 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp738813dyb; Tue, 13 Feb 2024 10:46:52 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXKPH6W6FbgHMrqtmt5EhHhmdJrJyTQl4qUAhaMfjsRADvTm0dvv5xn7vaKu0fESlm4TdhS0OiPELUr5yFNRlV5uONBpA== X-Google-Smtp-Source: AGHT+IGoXIx1Kp7KVhNeE0iQ1HPuO6wpTuyrlDuYjNO22ghiQTjJ+aRGBPncEqtrC8CB6m68YIfl X-Received: by 2002:a05:6402:515:b0:560:f198:2487 with SMTP id m21-20020a056402051500b00560f1982487mr465111edv.25.1707850011887; Tue, 13 Feb 2024 10:46:51 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850011; cv=pass; d=google.com; s=arc-20160816; b=C49FlWX5jn4D1cepteN44P1ZaB8sXLgpo1mNiocoUgcdDYn6sadPL33w8/fdwsvw4j Kb/B9lldLk3Hy/4cvAVgKJCIPMZN58qOUE1QD+qQjxRbFwwVAdks7Grmq+cyl2OAJsOY KyvrlCCqjvGWJgjzLlrftanSXvfKHy0YG0sK91hS+rJFwKwHz0Ki1ZJEVk7azBSjFKD4 HexK6d4g05OyRAmr8Wudj9amlI0p+iWgFa8S5124DVvt0BilfYnZ3PorE4dgDUE6Led0 u1v+LXmqYhEViR7XgyRJWb1nKmAlVQyiso98HOhKCDlAQvEtMVgwo73oVkK66Xo1bac5 7Eag== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=HBMjpKIEmpOsC3Yu9R6ikDfXct12/eMdQ0w+cVVIXoo=; fh=Cv2WTpk9xuPNw2zV5SuUrZejhNUpC/4Ik7celjYg2UA=; b=e3A/z7ZrUEd+0NN8LJEsw3Epo5VLYDpFdmgHn86BcwpmKvTyYqdVip6IlBBPoQmsv3 qFzd1CiCHUnwvc5ZVHUQEiXMsENnMJQDrvNV+qU/wjdspFgnl84g0mLb4l+346FGh7H4 2iooC1PTZoF2R7pk6ldnN4a38snXfZ9waIYElNUADhJuj69u+RzZIHCaTSfP9Y+Cn1xz vOGUWmXWGVsTOnAObE4NpzCTd6UnTVA0QAXZPoers0Zo4SchNMPbHillhcD1xsDhowie LrVVwvocAzZOjAUKgMcCaeknDGQfbsJj+TiuJLJDpy38Qm5FPAtDyisWNs2FfXs39uLw Pfbw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64097-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64097-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCVyZY6lozXjPUu0yeIuwqpzYdA3hgScKjwvHfcMOtrdqvNRtUv5g+bw4gQVitUzqKD6ySBFc2v/4ALJ8trr7yLiS1Vtdw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id u13-20020a50eacd000000b00561a837743csi2417245edp.12.2024.02.13.10.46.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:46:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64097-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64097-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64097-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 4CCB41F225DD for ; Tue, 13 Feb 2024 18:46:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 54CEB60BAD; Tue, 13 Feb 2024 18:45:28 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9C0B61CA94 for ; Tue, 13 Feb 2024 18:45:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849924; cv=none; b=T/UoxT9vVIFQiOU9gnjXImp4mwqBBpYePteq5hYc2zA7xPMN0hbaPQVv0hiNoF0H1A0EUcXZ1IlcxAHh+iWd9v1UNFj7Rzjqxf9XWUkrkjgHssm+ZowVx4703qdpcw5yfJNYXbjjzSz6HItFuDUeKElrQc1Ntba/RsqZDwJ8VCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849924; c=relaxed/simple; bh=L6HQEEeKgPgOcHw/oUivZxQriLrDcyicOQ8it5NvtYQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UrDYqBfJSBdu4bUVntn/ImNGCSjfDEsNgbb+CEH0baTktBB6Cfr5vRUDfDyxannAPjwDOuSEFo9WKaa2Zr+d/Pk3ywHL2SCKEKlVg/Cccfm5bwvRy4iCfURN5Uz15DdChENIrwxf2NP2Umt8VmStm3+bU5OH+ahR/EM04nQ1rE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 667781474; Tue, 13 Feb 2024 10:46:03 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E6D303F766; Tue, 13 Feb 2024 10:45:18 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 06/24] x86/resctrl: Access per-rmid structures by index Date: Tue, 13 Feb 2024 18:44:20 +0000 Message-Id: <20240213184438.16675-7-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810534287349151 X-GMAIL-MSGID: 1790810534287349151 x86 systems identify traffic using the CLOSID and RMID. The CLOSID is used to lookup the control policy, the RMID is used for monitoring. For x86 these are independent numbers. Arm's MPAM has equivalent features PARTID and PMG, where the PARTID is used to lookup the control policy. The PMG in contrast is a small number of bits that are used to subdivide PARTID when monitoring. The cache-occupancy monitors require the PARTID to be specified when monitoring. This means MPAM's PMG field is not unique. There are multiple PMG-0, one per allocated CLOSID/PARTID. If PMG is treated as equivalent to RMID, it cannot be allocated as an independent number. Bitmaps like rmid_busy_llc need to be sized by the number of unique entries for this resource. Treat the combined CLOSID and RMID as an index, and provide architecture helpers to pack and unpack an index. This makes the MPAM values unique. The domain's rmid_busy_llc and rmid_ptrs[] are then sized by index, as are domain mbm_local[] and mbm_total[]. x86 can ignore the CLOSID field when packing and unpacking an index, and report as many indexes as RMID. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Babu Moger Reviewed-by: Reinette Chatre --- Changes since v1: * Added X86_BAD_CLOSID macro to make it clear what this value means * Added second WARN_ON() for closid checking, and made both _ONCE() Changes since v2: * Added RESCTRL_RESERVED_CLOSID * Removed a newline * Repharsed some comments * Renamed a variable 'ignore'd * Moved X86_RESCTRL_BAD_CLOSID to a previous patch Changes since v3: * Changed a variable name * Fixed various typos Changes since v4: * Removed resource parameter from has_busy_rmid() * Rewrote commit message Changes since v5: * Used RESCTRL_RESERVED_RMID in clear_closid_rmid(). * Added comment against free_rmid()s index comparison tricks. Changes since v6: * Added a newline between some #includes. * Juggled some plurals in a comment. Changes since v7: * Added more uses of RESCTRL_RESERVED_foo * Removed use of resource in cqm_handle_limbo() that gcc >12 spots as unused. --- arch/x86/include/asm/resctrl.h | 17 +++++ arch/x86/kernel/cpu/resctrl/core.c | 13 ++-- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/monitor.c | 98 +++++++++++++++++--------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 13 ++-- 5 files changed, 100 insertions(+), 45 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index cc6e1bce7b1a..db4c84dde2d5 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -101,6 +101,23 @@ static inline void resctrl_sched_in(struct task_struct *tsk) __resctrl_sched_in(tsk); } +static inline u32 resctrl_arch_system_num_rmid_idx(void) +{ + /* RMID are independent numbers for x86. num_rmid_idx == num_rmid */ + return boot_cpu_data.x86_cache_max_rmid + 1; +} + +static inline void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid) +{ + *rmid = idx; + *closid = X86_RESCTRL_EMPTY_CLOSID; +} + +static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid) +{ + return rmid; +} + void resctrl_cpu_detect(struct cpuinfo_x86 *c); #else diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 9641c42d0f85..d1dc80a21ea9 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -587,7 +587,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) mbm_setup_overflow_handler(d, 0); } if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && - has_busy_rmid(r, d)) { + has_busy_rmid(d)) { cancel_delayed_work(&d->cqm_limbo); cqm_setup_limbo_handler(d, 0); } @@ -598,11 +598,12 @@ static void clear_closid_rmid(int cpu) { struct resctrl_pqr_state *state = this_cpu_ptr(&pqr_state); - state->default_closid = 0; - state->default_rmid = 0; - state->cur_closid = 0; - state->cur_rmid = 0; - wrmsr(MSR_IA32_PQR_ASSOC, 0, 0); + state->default_closid = RESCTRL_RESERVED_CLOSID; + state->default_rmid = RESCTRL_RESERVED_RMID; + state->cur_closid = RESCTRL_RESERVED_CLOSID; + state->cur_rmid = RESCTRL_RESERVED_RMID; + wrmsr(MSR_IA32_PQR_ASSOC, RESCTRL_RESERVED_RMID, + RESCTRL_RESERVED_CLOSID); } static int resctrl_online_cpu(unsigned int cpu) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index ae0e3338abc4..cbba782acd0c 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -8,6 +8,8 @@ #include #include +#include + #define L3_QOS_CDP_ENABLE 0x01ULL #define L2_QOS_CDP_ENABLE 0x01ULL @@ -558,7 +560,7 @@ void __init intel_rdt_mbm_apply_quirk(void); bool is_mba_sc(struct rdt_resource *r); void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms); void cqm_handle_limbo(struct work_struct *work); -bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d); +bool has_busy_rmid(struct rdt_domain *d); void __check_limbo(struct rdt_domain *d, bool force_free); void rdt_domain_reconfigure_cdp(struct rdt_resource *r); void __init thread_throttle_mode_init(void); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 3dad4134d2c9..bc5ceef143ab 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -149,12 +149,29 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val) return val; } -static inline struct rmid_entry *__rmid_entry(u32 closid, u32 rmid) +/* + * x86 and arm64 differ in their handling of monitoring. + * x86's RMID are independent numbers, there is only one source of traffic + * with an RMID value of '1'. + * arm64's PMG extends the PARTID/CLOSID space, there are multiple sources of + * traffic with a PMG value of '1', one for each CLOSID, meaning the RMID + * value is no longer unique. + * To account for this, resctrl uses an index. On x86 this is just the RMID, + * on arm64 it encodes the CLOSID and RMID. This gives a unique number. + * + * The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The arch code + * must accept an attempt to read every index. + */ +static inline struct rmid_entry *__rmid_entry(u32 idx) { struct rmid_entry *entry; + u32 closid, rmid; - entry = &rmid_ptrs[rmid]; - WARN_ON(entry->rmid != rmid); + entry = &rmid_ptrs[idx]; + resctrl_arch_rmid_idx_decode(idx, &closid, &rmid); + + WARN_ON_ONCE(entry->closid != closid); + WARN_ON_ONCE(entry->rmid != rmid); return entry; } @@ -284,8 +301,9 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, void __check_limbo(struct rdt_domain *d, bool force_free) { struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry; - u32 crmid = 1, nrmid; + u32 idx, cur_idx = 1; bool rmid_dirty; u64 val = 0; @@ -296,12 +314,11 @@ void __check_limbo(struct rdt_domain *d, bool force_free) * RMID and move it to the free list when the counter reaches 0. */ for (;;) { - nrmid = find_next_bit(d->rmid_busy_llc, r->num_rmid, crmid); - if (nrmid >= r->num_rmid) + idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx); + if (idx >= idx_limit) break; - entry = __rmid_entry(X86_RESCTRL_EMPTY_CLOSID, nrmid);// temporary - + entry = __rmid_entry(idx); if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, QOS_L3_OCCUP_EVENT_ID, &val)) { rmid_dirty = true; @@ -310,19 +327,21 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } if (force_free || !rmid_dirty) { - clear_bit(entry->rmid, d->rmid_busy_llc); + clear_bit(idx, d->rmid_busy_llc); if (!--entry->busy) { rmid_limbo_count--; list_add_tail(&entry->list, &rmid_free_lru); } } - crmid = nrmid + 1; + cur_idx = idx + 1; } } -bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d) +bool has_busy_rmid(struct rdt_domain *d) { - return find_first_bit(d->rmid_busy_llc, r->num_rmid) != r->num_rmid; + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); + + return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit; } /* @@ -352,6 +371,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) struct rdt_domain *d; int cpu, err; u64 val = 0; + u32 idx; + + idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); entry->busy = 0; cpu = get_cpu(); @@ -369,9 +391,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) * For the first limbo RMID in the domain, * setup up the limbo worker. */ - if (!has_busy_rmid(r, d)) + if (!has_busy_rmid(d)) cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL); - set_bit(entry->rmid, d->rmid_busy_llc); + set_bit(idx, d->rmid_busy_llc); entry->busy++; } put_cpu(); @@ -384,14 +406,21 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) void free_rmid(u32 closid, u32 rmid) { + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); struct rmid_entry *entry; - if (!rmid) - return; - lockdep_assert_held(&rdtgroup_mutex); - entry = __rmid_entry(closid, rmid); + /* + * Do not allow the default rmid to be free'd. Comparing by index + * allows architectures that ignore the closid parameter to avoid an + * unnecessary check. + */ + if (idx == resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID, + RESCTRL_RESERVED_RMID)) + return; + + entry = __rmid_entry(idx); if (is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); @@ -402,11 +431,13 @@ void free_rmid(u32 closid, u32 rmid) static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id evtid) { + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); + switch (evtid) { case QOS_L3_MBM_TOTAL_EVENT_ID: - return &d->mbm_total[rmid]; + return &d->mbm_total[idx]; case QOS_L3_MBM_LOCAL_EVENT_ID: - return &d->mbm_local[rmid]; + return &d->mbm_local[idx]; default: return NULL; } @@ -449,7 +480,8 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) */ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) { - struct mbm_state *m = &rr->d->mbm_local[rmid]; + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); + struct mbm_state *m = &rr->d->mbm_local[idx]; u64 cur_bw, bytes, cur_bytes; cur_bytes = rr->val; @@ -538,9 +570,9 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) struct mbm_state *pmbm_data, *cmbm_data; struct rdt_resource *r_mba; struct rdt_domain *dom_mba; + u32 cur_bw, user_bw, idx; struct list_head *head; struct rdtgroup *entry; - u32 cur_bw, user_bw; if (!is_mbm_local_enabled()) return; @@ -549,7 +581,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) closid = rgrp->closid; rmid = rgrp->mon.rmid; - pmbm_data = &dom_mbm->mbm_local[rmid]; + idx = resctrl_arch_rmid_idx_encode(closid, rmid); + pmbm_data = &dom_mbm->mbm_local[idx]; dom_mba = get_domain_from_cpu(smp_processor_id(), r_mba); if (!dom_mba) { @@ -638,17 +671,15 @@ void cqm_handle_limbo(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL); int cpu = smp_processor_id(); - struct rdt_resource *r; struct rdt_domain *d; mutex_lock(&rdtgroup_mutex); - r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; d = container_of(work, struct rdt_domain, cqm_limbo.work); __check_limbo(d, false); - if (has_busy_rmid(r, d)) + if (has_busy_rmid(d)) schedule_delayed_work_on(cpu, &d->cqm_limbo, delay); mutex_unlock(&rdtgroup_mutex); @@ -713,19 +744,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) static int dom_data_init(struct rdt_resource *r) { + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry = NULL; - int i, nr_rmids; + u32 idx; + int i; - nr_rmids = r->num_rmid; - rmid_ptrs = kcalloc(nr_rmids, sizeof(struct rmid_entry), GFP_KERNEL); + rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL); if (!rmid_ptrs) return -ENOMEM; - for (i = 0; i < nr_rmids; i++) { + for (i = 0; i < idx_limit; i++) { entry = &rmid_ptrs[i]; INIT_LIST_HEAD(&entry->list); - entry->rmid = i; + resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid); list_add_tail(&entry->list, &rmid_free_lru); } @@ -734,7 +766,9 @@ static int dom_data_init(struct rdt_resource *r) * are always allocated. These are used for the rdtgroup_default * control group, which will be setup later in rdtgroup_init(). */ - entry = __rmid_entry(RESCTRL_RESERVED_CLOSID, RESCTRL_RESERVED_RMID); + idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID, + RESCTRL_RESERVED_RMID); + entry = __rmid_entry(idx); list_del(&entry->list); return 0; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index ad7da7254f4d..a7dbc0e7e559 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3853,8 +3853,8 @@ static void __init rdtgroup_setup_default(void) { mutex_lock(&rdtgroup_mutex); - rdtgroup_default.closid = 0; - rdtgroup_default.mon.rmid = 0; + rdtgroup_default.closid = RESCTRL_RESERVED_CLOSID; + rdtgroup_default.mon.rmid = RESCTRL_RESERVED_RMID; rdtgroup_default.type = RDTCTRL_GROUP; INIT_LIST_HEAD(&rdtgroup_default.mon.crdtgrp_list); @@ -3889,7 +3889,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) if (is_mbm_enabled()) cancel_delayed_work(&d->mbm_over); - if (is_llc_occupancy_enabled() && has_busy_rmid(r, d)) { + if (is_llc_occupancy_enabled() && has_busy_rmid(d)) { /* * When a package is going down, forcefully * decrement rmid->ebusy. There is no way to know @@ -3907,16 +3907,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) { + u32 idx_limit = resctrl_arch_system_num_rmid_idx(); size_t tsize; if (is_llc_occupancy_enabled()) { - d->rmid_busy_llc = bitmap_zalloc(r->num_rmid, GFP_KERNEL); + d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL); if (!d->rmid_busy_llc) return -ENOMEM; } if (is_mbm_total_enabled()) { tsize = sizeof(*d->mbm_total); - d->mbm_total = kcalloc(r->num_rmid, tsize, GFP_KERNEL); + d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_total) { bitmap_free(d->rmid_busy_llc); return -ENOMEM; @@ -3924,7 +3925,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) } if (is_mbm_local_enabled()) { tsize = sizeof(*d->mbm_local); - d->mbm_local = kcalloc(r->num_rmid, tsize, GFP_KERNEL); + d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL); if (!d->mbm_local) { bitmap_free(d->rmid_busy_llc); kfree(d->mbm_total); From patchwork Tue Feb 13 18:44:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200569 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp743627dyb; Tue, 13 Feb 2024 10:56:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWWAWWB5x2uUp/z6gw0Ia0VX65kNVnuGObxFIazcH9pOWZrtIK86pKj5zVEg+B9Lb8XY6//MS1oO0wtIrbJuwJmtK6ULg== X-Google-Smtp-Source: AGHT+IHAoNiiJKb9odHTXSn0JYSAeRganbMvKq8JJyWoinGGpgGYfqXQUCy/eIAK1Dy4ePOKWur9 X-Received: by 2002:a17:90a:34c8:b0:298:ae6f:313e with SMTP id m8-20020a17090a34c800b00298ae6f313emr363376pjf.29.1707850606503; Tue, 13 Feb 2024 10:56:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850606; cv=pass; d=google.com; s=arc-20160816; b=nGAMQqtMkaDzcGyBmkxB1n4UpwMQta/56Rd1CkC3a8QZ0uOG6VzzjfXWESgpVYrgM/ EvPsOpjoVRvGL8RRmOJwaT/xPLFn2B4qqIcrN7naTySw2p3YnHdrWpUJe/ZifQS23q96 oJDDacG2YVW25hx31Ge9RByJySPghdeVxsMIT0QXge8V1hlw1WKoLeoaEQpwJ7ccA1p+ Xyx2GW+dXwsjtAfiekyn6P6sfhz60D24hICQXMFmeKTlAw8sP301a/aBMqYp409g3FgF Q/iJzizVVQBUsoQkh+5cuEZBz0W3v+ud5eVPKMjyZkwgGu8CByL1JlnuZfdEOnoaDe0R 0/VQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=yKCbwn03evTkb1pYfysjE7wPYkPceMkGFBq5p9Eo8IQ=; fh=1AaxPsA6uh8uoO61UTqjBmgr2+czbwMr+PSIFfcH9rM=; b=STVj/UlCAiuJjAra+0F02nnNWNms5jrFIrIS3XKs8aSyJbB+YXFZ6sBd4LdEYnb6r8 EF8bY8Px8wXw0PbSCvwovPzGTS48KvENHlzBua1uykuboH9ky9EHd/6iAcNlDu8/hksu wCnRph3UkPlyL1C6ReI6HcSq6wsRWh44/HuW+tvr6PZ1pdJwwo5UhT+0rNro2WHl63tU MFWetn8yCufydAIJEQF3jcVPRqiRkhVOCEf791kFmZHJDMyYvmEJnzkjceLvMlpmJsOP pomzs4H0KfcqSPFviceNUm9w5OV5P0Zw4LUYMmmE/0gk5hnhVDPPLd3w2IIAh2B2EIlJ uV5g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64098-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64098-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCU4/VRBFyT6ptAgcvHTvut/SUtEfIj5HDkOJaZnKugrYTDGvoo5Hnr/PX9W4eXs44JSoZ0vk6VCW/RgYen3bNh6AnPYUg== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id bo10-20020a17090b090a00b00296fd518c20si2428782pjb.80.2024.02.13.10.56.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:56:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64098-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64098-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64098-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id C6C4128E3FD for ; Tue, 13 Feb 2024 18:47:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C938160DC8; Tue, 13 Feb 2024 18:45:29 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 150681F5FA for ; Tue, 13 Feb 2024 18:45:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849927; cv=none; b=nuoYLLFLaMp1XgURrTY/L+ggPExOp/8qopx8QuvWKKFFeOWmceQKKwDwZc/NMgXRuVt1KuMobnjFF4Ga2TeZBLL4JsCyXj4cdUz9L/UUKOPX8LcTCFk72/xcVVz3/z5J6VyvLDW7950XCImSRVo+xDwe2OD1kBpsrHPX5SH2K0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849927; c=relaxed/simple; bh=6ebG69cbFczaqBbQqA7uAG5WhTVnWJljhAVqTzlCvT4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=M3B0ciqvFjs//ge32fkB5CS6wS/TbZlguDiOrME+IEhPHU9R0Wnbk2Tgn8ilWHRr3wwJ+SUAbfKooc3SOys+IC+HMFCqf2n4p3OGrJ3AiKcf3b0d8GcGXvLjj5R11tBAc41KWAeG9VzbdN8Biescj3Z7JHv7IGoGGa3Ip7EVnq4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B23AE1476; Tue, 13 Feb 2024 10:46:06 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5AA3A3F766; Tue, 13 Feb 2024 10:45:22 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 07/24] x86/resctrl: Allow RMID allocation to be scoped by CLOSID Date: Tue, 13 Feb 2024 18:44:21 +0000 Message-Id: <20240213184438.16675-8-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811157174057031 X-GMAIL-MSGID: 1790811157174057031 MPAMs RMID values are not unique unless the CLOSID is considered as well. alloc_rmid() expects the RMID to be an independent number. Pass the CLOSID in to alloc_rmid(). Use this to compare indexes when allocating. If the CLOSID is not relevant to the index, this ends up comparing the free RMID with itself, and the first free entry will be used. With MPAM the CLOSID is included in the index, so this becomes a walk of the free RMID entries, until one that matches the supplied CLOSID is found. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v2; * Rephrased comment in resctrl_find_free_rmid() to describe this in terms of list_entry_first() * Rephrased comment above alloc_rmid() Changes since v3: * Flipped conditions in alloc_rmid() Changes since v4: * Typo in comment Changes since v5: * Reworded two comments. Changes since v6: * Fixed a typo and some whitespace. --- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 51 +++++++++++++++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +- 4 files changed, 41 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index cbba782acd0c..872ba1a34103 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -543,7 +543,7 @@ void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp); struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); void closid_free(int closid); -int alloc_rmid(void); +int alloc_rmid(u32 closid); void free_rmid(u32 closid, u32 rmid); int rdt_get_mon_l3_config(struct rdt_resource *r); void __exit rdt_put_mon_l3_config(void); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index bc5ceef143ab..c49f2e89ef29 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -344,24 +344,49 @@ bool has_busy_rmid(struct rdt_domain *d) return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit; } -/* - * As of now the RMIDs allocation is global. - * However we keep track of which packages the RMIDs - * are used to optimize the limbo list management. - */ -int alloc_rmid(void) +static struct rmid_entry *resctrl_find_free_rmid(u32 closid) { - struct rmid_entry *entry; - - lockdep_assert_held(&rdtgroup_mutex); + struct rmid_entry *itr; + u32 itr_idx, cmp_idx; if (list_empty(&rmid_free_lru)) - return rmid_limbo_count ? -EBUSY : -ENOSPC; + return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC); + + list_for_each_entry(itr, &rmid_free_lru, list) { + /* + * Get the index of this free RMID, and the index it would need + * to be if it were used with this CLOSID. + * If the CLOSID is irrelevant on this architecture, the two + * index values are always the same on every entry and thus the + * very first entry will be returned. + */ + itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid); + cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid); + + if (itr_idx == cmp_idx) + return itr; + } + + return ERR_PTR(-ENOSPC); +} + +/* + * For MPAM the RMID value is not unique, and has to be considered with + * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which + * allows all domains to be managed by a single free list. + * Each domain also has a rmid_busy_llc to reduce the work of the limbo handler. + */ +int alloc_rmid(u32 closid) +{ + struct rmid_entry *entry; + + lockdep_assert_held(&rdtgroup_mutex); + + entry = resctrl_find_free_rmid(closid); + if (IS_ERR(entry)) + return PTR_ERR(entry); - entry = list_first_entry(&rmid_free_lru, - struct rmid_entry, list); list_del(&entry->list); - return entry->rmid; } diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 65bee6f11015..d8f44113ed1f 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -777,7 +777,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) int ret; if (rdt_mon_capable) { - ret = alloc_rmid(); + ret = alloc_rmid(rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); return ret; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index a7dbc0e7e559..dcffd1c4a476 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -3295,7 +3295,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) if (!rdt_mon_capable) return 0; - ret = alloc_rmid(); + ret = alloc_rmid(rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); return ret; From patchwork Tue Feb 13 18:44:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200568 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp743088dyb; Tue, 13 Feb 2024 10:55:41 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWORfNh2StetbZXGfK5CsId7WSgMVZ9wEb2uNp2D9MZy/4tp/EHvYWRQoAEBnn0ooBwZoiVVUg9xgPr7NQ/JPbby6f0vQ== X-Google-Smtp-Source: AGHT+IH4HzclASkBD7IjrX6AnUHW2PRE2ET+x/ez8DB+zZKdYkrowjkV+K5RJ35zkkqoA12+1Vv1 X-Received: by 2002:a05:6a21:9206:b0:19c:9419:1f37 with SMTP id tl6-20020a056a21920600b0019c94191f37mr584589pzb.13.1707850541258; Tue, 13 Feb 2024 10:55:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850541; cv=pass; d=google.com; s=arc-20160816; b=ZfNqWD+EjQPvcv5soWw8a8Q91aKqyCxVRhhISMn16eA3WQJbH4kffSMc7WshGFsmQh 7qNKX+D/TANAhFD3p0qFYGhR9zUBOJaygj5gQUKjOwZ5LChEEt42JVqJSfken4JZWV7D ufOPtRzzYUVUmGvWpTozpJktmjTuKyxM0dx2ZwAeXG96529tZccCiQH1Uiq7hbi0T31C tlghPBQcg129Ep+0ztVUWGB8QhxOjnj4Mc4F9g8/n5yxqghsSpPaABd8GPF/lOp77FWF 3IB4PB6u9xTxh2h/MV1NLxOmXqaWeUZ2FBY19HB44fnMBy1aGbO08oRl2Bk6ZH+CM0+b GI/w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=b9PHOAX1gF4vsYMdgCQKu/jQ2P8XIjL6hdCH9UxfwMM=; fh=IeLxMHZPFEjvEyd0d3Vwy8nHQ17LewkoTLBUafCWwlY=; b=nVK+7wPNjGiYjSIg+BEOVy9AMiJzbXP8tRC4WqZDZFLqqXNIn2ZPK76+gREhxX//TZ ZZJfd3gPunXr2pszGf8Hjo7i5+jGAasf///NasjI+2kOHqF2yTlDnq7k7VMdljD64/HL F4U+A8uJzUQ5r+u+TVZCw8YBW/EwK0x3PdTBbZYNG2wIsxBlNd14cNsJQ5bO4SYaxjF8 ug2cPejmmi1EVQsA2dHd1ExRgLnfKZ4NbUm2yHDfImu7UsBAVP8gjqql/PD1ZyWlISIA a5iuzKYY3aObRqgc2meu1spq6NGCbKCO4r8SL/h08bWLNvaEpiKJjhAE27NCWxbu8BOq lRgA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64099-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64099-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCUJKC+YtFeY+Nm0YHZ7ecEepkpffIET61hCobssSioZm7IVxre8yba6HpIc6WMvJjhaGyfUky7j0iz0RUFvmTCSFJ0nJQ== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id q13-20020a65684d000000b005dc76f56a18si2349525pgt.559.2024.02.13.10.55.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:55:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64099-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64099-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64099-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E9D6228DFC4 for ; Tue, 13 Feb 2024 18:47:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C2F4660DC7; Tue, 13 Feb 2024 18:45:32 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2AAED60BBB for ; Tue, 13 Feb 2024 18:45:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849930; cv=none; b=sUjLOxSXB3wC0YK7axWwv6ccejLqraEa5/H8H56w4dcx5YBehU7tb2uuy+haB4zu/4BnCN+c28rKzuLP2/wToUd7o7Ol67aVHUEQoIk6DNRDxauvAnvoRPddkx0I7lSxE+V6HoojWUclZBWoGsidPSe+8soIxy+paMVo9L8MlnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849930; c=relaxed/simple; bh=4Ked7jnGlDXK2s0lINFNKoZlGzapZ/Q5SvYuHDZAEHk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CwX1Du3i6YQ91Xtw7aDhqKlTC20mjcxcFMR7CVZ91CXjazkUOBg/8VGKcN09b0QhfHaGm2hgFPN8zSipD53dQVI7u9rVS3jUHdLihHxhWPHjwx9Cl9/UHYCwRKO/tfi4BuBBF14W2XQfXrV9PDlcQbY2fPWpzNrMvUrnwdgQsUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 01C1C1477; Tue, 13 Feb 2024 10:46:10 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9D6383F766; Tue, 13 Feb 2024 10:45:25 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 08/24] x86/resctrl: Track the number of dirty RMID a CLOSID has Date: Tue, 13 Feb 2024 18:44:22 +0000 Message-Id: <20240213184438.16675-9-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811089089135347 X-GMAIL-MSGID: 1790811089089135347 MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups. This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo. Keep track of the number of RMID held in limbo each CLOSID has. This will allow a future helper to find the 'cleanest' CLOSID when allocating. The array is only needed when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined. This will never be the case on x86. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v4: * Moved closid_num_dirty_rmid[] update under entry->busy check * Take the mutex in dom_data_init() as the caller doesn't. Changes since v5: * Added braces after an else. * Made closid_num_dirty_rmid an unsigned int. * Moved mutex_lock() in dom_data_init() to cover the whole function. Changes since v6: * Made closid_num_dirty_rmid[] and associated tmp variables u32. Changes since v7: * Clobber kfree()d variable with NULL. * Guard the use of closid_num_dirty_rmid with IS_ENABLED() so it can be optimised out on x86. Changes since v8: * Added a comment about over allocation is the arch doesn't pre-sanitise values. MPAM does, and is the only user of this code. * Made comments over closid_num_dirty_rmid() not a kdoc comment. --- arch/x86/kernel/cpu/resctrl/monitor.c | 75 +++++++++++++++++++++++---- 1 file changed, 65 insertions(+), 10 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index c49f2e89ef29..13b0c8d14f3d 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -50,6 +50,13 @@ struct rmid_entry { */ static LIST_HEAD(rmid_free_lru); +/* + * @closid_num_dirty_rmid The number of dirty RMID each CLOSID has. + * Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined. + * Indexed by CLOSID. Protected by rdtgroup_mutex. + */ +static u32 *closid_num_dirty_rmid; + /* * @rmid_limbo_count - count of currently unused but (potentially) * dirty RMIDs. @@ -292,6 +299,17 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, return 0; } +static void limbo_release_entry(struct rmid_entry *entry) +{ + lockdep_assert_held(&rdtgroup_mutex); + + rmid_limbo_count--; + list_add_tail(&entry->list, &rmid_free_lru); + + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) + closid_num_dirty_rmid[entry->closid]--; +} + /* * Check the RMIDs that are marked as busy for this domain. If the * reported LLC occupancy is below the threshold clear the busy bit and @@ -328,10 +346,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) if (force_free || !rmid_dirty) { clear_bit(idx, d->rmid_busy_llc); - if (!--entry->busy) { - rmid_limbo_count--; - list_add_tail(&entry->list, &rmid_free_lru); - } + if (!--entry->busy) + limbo_release_entry(entry); } cur_idx = idx + 1; } @@ -398,6 +414,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) u64 val = 0; u32 idx; + lockdep_assert_held(&rdtgroup_mutex); + idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); entry->busy = 0; @@ -423,10 +441,13 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) } put_cpu(); - if (entry->busy) + if (entry->busy) { rmid_limbo_count++; - else + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) + closid_num_dirty_rmid[entry->closid]++; + } else { list_add_tail(&entry->list, &rmid_free_lru); + } } void free_rmid(u32 closid, u32 rmid) @@ -770,13 +791,39 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) static int dom_data_init(struct rdt_resource *r) { u32 idx_limit = resctrl_arch_system_num_rmid_idx(); + u32 num_closid = resctrl_arch_get_num_closid(r); struct rmid_entry *entry = NULL; + int err = 0, i; u32 idx; - int i; + + mutex_lock(&rdtgroup_mutex); + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + u32 *tmp; + + /* + * If the architecture hasn't provided a sanitised value here, + * this may result in larger arrays than necessary. Resctrl will + * use a smaller system wide value based on the resources in + * use. + */ + tmp = kcalloc(num_closid, sizeof(*tmp), GFP_KERNEL); + if (!tmp) { + err = -ENOMEM; + goto out_unlock; + } + + closid_num_dirty_rmid = tmp; + } rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL); - if (!rmid_ptrs) - return -ENOMEM; + if (!rmid_ptrs) { + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + kfree(closid_num_dirty_rmid); + closid_num_dirty_rmid = NULL; + } + err = -ENOMEM; + goto out_unlock; + } for (i = 0; i < idx_limit; i++) { entry = &rmid_ptrs[i]; @@ -796,13 +843,21 @@ static int dom_data_init(struct rdt_resource *r) entry = __rmid_entry(idx); list_del(&entry->list); - return 0; +out_unlock: + mutex_unlock(&rdtgroup_mutex); + + return err; } static void __exit dom_data_exit(void) { mutex_lock(&rdtgroup_mutex); + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + kfree(closid_num_dirty_rmid); + closid_num_dirty_rmid = NULL; + } + kfree(rmid_ptrs); rmid_ptrs = NULL; From patchwork Tue Feb 13 18:44:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200555 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp739156dyb; Tue, 13 Feb 2024 10:47:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWpJ0x2NcFQok5Osck8QErRYcK6N+hR6D02c5QtvCiRgWqKi2kVP8OEBBNTmeM8CwJTHuyp9+C201//eAGgoQ0uP9e5vw== X-Google-Smtp-Source: AGHT+IEJI78ufR9GZw7HCdwcalVWP2+RZrZAQeDh42ESgg+Jtf5rJWRekrrzAoyLLt0EX2acoBF8 X-Received: by 2002:a05:6808:603:b0:3bf:f758:8096 with SMTP id y3-20020a056808060300b003bff7588096mr249840oih.32.1707850050499; Tue, 13 Feb 2024 10:47:30 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850050; cv=pass; d=google.com; s=arc-20160816; b=w1RAXzvjTnq1Zn6uWxsSbjywl0CrP3vrKBv0aMtTpC10vLHlwhKXo+uvq4zpqyIYRa 2aolaTm2gNVjmVSAUrTDfw0DLqrptEF548ZYH9iJ6+IajCycVyxO1HiqeNFHhyE/NAZN gQSGI+rBevpIqIXRHu72W3Z1Ik1Wt0vJg7fT1c3Wa2hFccDC04mJOXqxohR8U0U86yFM PEhn9DIiMj6CPK8ctG++HIfr2+F7y4srTpTGFO/YPREM0VBkukzI+qzMaJHvDaxAKgRc r6TxYT6iZTanT/9R01XpMlXpbrvQGdBJIxk6HkzkfE8gGK0wZgM24Jrtj3ShemSAjbXI huPA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=uOLb68ruCI0vMDHIPjk7bP5Iv2bM/c+aWXUyBejWPHI=; fh=/wOQK4fZBvqJ8oHRmgoyS4NgoVvh7P9Khhlkp5sv4JI=; b=MrN3hGv+31xC094yEv23PvsnfuhTu+pFYQlKCIkWy/ipc/X9q9BRYR2K7fWs/5nG1i 1oxKxQNP/imogh1G5mZIPtp7cVLq5bkxvoD/e7sy5IHCap91MsaUqlYbdyK3oM12SJxY M5MiNE33cUkx4KnBkMKOIMMaJhK+eLWWIkv/6aUNsXQbITjNvMNR7gXVGSQ0KvwAyGms byNVoCj7W5guxsrzjAI7XTapp09g3YpKuydXWPCsSAazZRUhU9Ao5tJOKZ5r8bsZaP+d INgMCANw6M5J8bQ8dJ0bxRGnFvDjpNXV9QAPM173G5UMfloxEmj3JQZLOZRcPaEEKQu7 iCoA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64100-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64100-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWP4O33GR976uh26ajoZDIWsxos7Q6mLxTFm387NsC0B83C4Gnxkx+4O8eY7THsByWm0CDKt5CT+0Jxv2V/79QugpgOgw== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id i14-20020ad44bae000000b0068c9c9584ffsi1466608qvw.500.2024.02.13.10.47.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:47:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64100-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64100-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64100-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4A5EB1C2329D for ; Tue, 13 Feb 2024 18:47:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BB74A60DEE; Tue, 13 Feb 2024 18:45:35 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A420460DC3 for ; Tue, 13 Feb 2024 18:45:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849934; cv=none; b=IOhEl7UF1hS5RYY+BZQyNeRhlCRINnonwg1Y4/kJ1hH1o8BdFPDn1mrzEp51oSPYxBCITJtiL6BXgxwNyXy8shI+Rv5+ocV/T5dQbQbitacaqzRTPTn2jQs4wtsjf+QQXoAFUlh1KgaGHKplM0DGPJCSrK8KsD4LWWNpfz4Hj5k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849934; c=relaxed/simple; bh=SohyraE/+H1Ez4/07fPwKDGgs8I/0aYXelB5GAm+lJQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HTW1zPvPEcZDJKACjUBgMVmDnDKzLSRRAEb50nUgxc/ycnn6nSTMnukNMqkCeKbFw49/9GhIFrsQ/E/sCr0Epj+9F1IUKVLHmJR6SxAjXvWhNxvL0nHVhdqesgGwt1vgTEzbN7wHXbp6I2jD+iEQkSL85A+9tKPnSWfzSGKfzfw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5E3111480; Tue, 13 Feb 2024 10:46:13 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id ED7E73F766; Tue, 13 Feb 2024 10:45:28 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 09/24] x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding Date: Tue, 13 Feb 2024 18:44:23 +0000 Message-Id: <20240213184438.16675-10-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810574110404273 X-GMAIL-MSGID: 1790810574110404273 The resctrl CLOSID allocator uses a single 32bit word to track which CLOSID are free. The setting and clearing of bits is open coded. Convert the existing open coded bit manipulations of closid_free_map to use __set_bit() and friends. These don't need to be atomic as this list is protected by the mutex. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger Reviewed-by: David Hildenbrand --- Changes since v6: * Use the __ inatomic helpers and add lockdep_assert_held() annotations to document how this is safe. * Fixed a resctrl_closid_is_free()/closid_allocated() rename in the commit message. * Use RESCTRL_RESERVED_CLOSID to improve readability. Changes since v7: * Removed paragraph explaining why this should be done now due to badword 'subsequent'. * Changed a comment to refer to RESCTRL_RESERVED_CLOSID. --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index dcffd1c4a476..bc6e0f83c847 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -111,7 +111,7 @@ void rdt_staged_configs_clear(void) * - Our choices on how to configure each resource become progressively more * limited as the number of resources grows. */ -static int closid_free_map; +static unsigned long closid_free_map; static int closid_free_map_len; int closids_supported(void) @@ -130,8 +130,8 @@ static void closid_init(void) closid_free_map = BIT_MASK(rdt_min_closid) - 1; - /* CLOSID 0 is always reserved for the default group */ - closid_free_map &= ~1; + /* RESCTRL_RESERVED_CLOSID is always reserved for the default group */ + __clear_bit(RESCTRL_RESERVED_CLOSID, &closid_free_map); closid_free_map_len = rdt_min_closid; } @@ -139,17 +139,21 @@ static int closid_alloc(void) { u32 closid = ffs(closid_free_map); + lockdep_assert_held(&rdtgroup_mutex); + if (closid == 0) return -ENOSPC; closid--; - closid_free_map &= ~(1 << closid); + __clear_bit(closid, &closid_free_map); return closid; } void closid_free(int closid) { - closid_free_map |= 1 << closid; + lockdep_assert_held(&rdtgroup_mutex); + + __set_bit(closid, &closid_free_map); } /** @@ -161,7 +165,9 @@ void closid_free(int closid) */ static bool closid_allocated(unsigned int closid) { - return (closid_free_map & (1 << closid)) == 0; + lockdep_assert_held(&rdtgroup_mutex); + + return !test_bit(closid, &closid_free_map); } /** From patchwork Tue Feb 13 18:44:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200572 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp751934dyb; Tue, 13 Feb 2024 11:08:58 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU1m37ZUrcNFVxsqTAHSHI1KELZCHaonCRAU+Kle3x0Oah8XG/tm8DcTtxN6Xl1kGnT+WrvwNoU0NbH6TEFlmiC5q0FRA== X-Google-Smtp-Source: AGHT+IFLCAMQvc2QiDu9P6whLq5GsLM3hp68jDRmWViWT0VoHtXduQvnVmpB7GhWZpaZt/ILVWgu X-Received: by 2002:a05:6a21:9207:b0:19e:9a65:75eb with SMTP id tl7-20020a056a21920700b0019e9a6575ebmr688507pzb.40.1707851337884; Tue, 13 Feb 2024 11:08:57 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851337; cv=pass; d=google.com; s=arc-20160816; b=NV1WDKmINmymwLbx6V4SK8vl04UuatxWI1GXH8kcHzplmG2xEDpeaTfff3bCLUSHY6 tk0ZeNiCC/tkFuX9Yr5zOs24wqwqzuS3RPh8t+TjGtONtV2OU/p6Oq7jtIhuFCLvsjZB /jjtY66oF6eZood7Xs/AN++xHZVQuVsuIPjR3B/LtKVSrBMx5DXBjZW3G5oPR1NzpQSo Lg/iGPFLBQ2TrWd6qoc4DmloVHeAjZYhR0LioSLH8+b8KJRfMhe8GLtA3yfPE2w3BEpj kn0FUfcS1npYuOzlnQeeK712HJr2sjKCYlA3pc5J1m+Ph8JlHiMB96P2ITv0bCcc8vlL w70A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=zR9kH6lRo7Yq+kADqTku3v4kD42UAF0nDQHUdwThyNA=; fh=QhdVNWv++wRsnmoUk9HiNBcZkt0k9rQbSanFxuCh7tw=; b=GOd2AGwaAXZ2gzoHtwLe1lppqeiqaGyGLc6vApiVM3b4jzS1QlWNfwLRDIoNCHxPRk mGty4PsdyCRnR7YGhANouC7aRDroF5Tzk7RlMm1Hq34G8Ugt0UndbofgSF/mdRyS76x5 4cD3DnPHN7ndQxuCJlmm0vOnJ1HEFT9sZOSa1XvqTVN/f3HZYBVp2qSgYn3zHuOisW+7 NfgVLq3WkRKtodOP9u9Sd3FCtSp/lNDqwq1LWOgJ/tsOvxy06f3x56DoYRASRpwwNe4d OEgJ4vu4+ukrfcuaQfNX2n2O1CEoAz/9t7turaM+1SAgPbcsONkDXSn4BYQSkNQN+Y2Z 2OKg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64101-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64101-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXNmrySKLP/eRpCWEhZXJtXFqSkidHSxa7j25PQp6d3PZzfQtjDrasmtkuBRldkHb6D96+TZXPv6+3ccr6pQTIe8loOLw== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id q3-20020a656a83000000b005cd966560b7si2486026pgu.626.2024.02.13.11.08.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:08:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64101-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64101-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64101-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 17046B274C4 for ; Tue, 13 Feb 2024 18:47:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6D20460ED0; Tue, 13 Feb 2024 18:45:40 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EA85360DF1 for ; Tue, 13 Feb 2024 18:45:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849938; cv=none; b=Ul4JBOvLScMPCbyXCU8ZY5MuQcDtkG1CcIOt3VIrqR0NpHAwfsr4mSNZGPYuACi5nO2A6MDA7ekmVCTFy42ymiRcamuaRygum5BUsS6myGnU+bvKVvk9q1hyO5sOqNRmcHsCfsFZpFGtNuAV6V6S3VfV90ipioArRB2W7BTukQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849938; c=relaxed/simple; bh=yTn+5rNsttXSA8wv9R6eV/BdH3FviFAT3quk383HYUk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=peokL6DiyP0FtTuaMoHPuehamH8jIAVUFV0Lvu8r8kwQWlE4PXo7+MrbvwhBROik/DypoqCTJ4Q2EcaBMiTsAS5+3YWeFEVGKIcFMnMoBnh5T0O5lJvOl36aIvVhezqhLCDBKOQtTbQZwJoikbMMuVKmqZpKTWqpCeBGW7DnFqc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C0CB314BF; Tue, 13 Feb 2024 10:46:16 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 64A323F766; Tue, 13 Feb 2024 10:45:32 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 10/24] x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid Date: Tue, 13 Feb 2024 18:44:24 +0000 Message-Id: <20240213184438.16675-11-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811924338529113 X-GMAIL-MSGID: 1790811924338529113 MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups. This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo. Instead of allocating the first free CLOSID, on architectures where CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is enabled, search closid_num_dirty_rmid[] to find the cleanest CLOSID. The CLOSID found is returned to closid_alloc() for the free list to be updated. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v4: * Dropped stale section from comment Changes since v5: * Renamed some variables. Changes since v7: * Made comments over closid_num_dirty_rmid() not a kdoc comment. --- arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/monitor.c | 45 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 ++++++++--- 3 files changed, 61 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 872ba1a34103..b7b9d9230bef 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -566,5 +566,7 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r); void __init thread_throttle_mode_init(void); void __init mbm_config_rftype_init(const char *config); void rdt_staged_configs_clear(void); +bool closid_allocated(unsigned int closid); +int resctrl_find_cleanest_closid(void); #endif /* _ASM_X86_RESCTRL_INTERNAL_H */ diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 13b0c8d14f3d..101f1b112d17 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -386,6 +386,51 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 closid) return ERR_PTR(-ENOSPC); } +/** + * resctrl_find_cleanest_closid() - Find a CLOSID where all the associated + * RMID are clean, or the CLOSID that has + * the most clean RMID. + * + * MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocated CLOSID + * may not be able to allocate clean RMID. To avoid this the allocator will + * choose the CLOSID with the most clean RMID. + * + * When the CLOSID and RMID are independent numbers, the first free CLOSID will + * be returned. + */ +int resctrl_find_cleanest_closid(void) +{ + u32 cleanest_closid = ~0; + int i = 0; + + lockdep_assert_held(&rdtgroup_mutex); + + if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) + return -EIO; + + for (i = 0; i < closids_supported(); i++) { + int num_dirty; + + if (closid_allocated(i)) + continue; + + num_dirty = closid_num_dirty_rmid[i]; + if (num_dirty == 0) + return i; + + if (cleanest_closid == ~0) + cleanest_closid = i; + + if (num_dirty < closid_num_dirty_rmid[cleanest_closid]) + cleanest_closid = i; + } + + if (cleanest_closid == ~0) + return -ENOSPC; + + return cleanest_closid; +} + /* * For MPAM the RMID value is not unique, and has to be considered with * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index bc6e0f83c847..8fc46204a6cc 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -137,13 +137,22 @@ static void closid_init(void) static int closid_alloc(void) { - u32 closid = ffs(closid_free_map); + int cleanest_closid; + u32 closid; lockdep_assert_held(&rdtgroup_mutex); - if (closid == 0) - return -ENOSPC; - closid--; + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) { + cleanest_closid = resctrl_find_cleanest_closid(); + if (cleanest_closid < 0) + return cleanest_closid; + closid = cleanest_closid; + } else { + closid = ffs(closid_free_map); + if (closid == 0) + return -ENOSPC; + closid--; + } __clear_bit(closid, &closid_free_map); return closid; @@ -163,7 +172,7 @@ void closid_free(int closid) * Return: true if @closid is currently associated with a resource group, * false if @closid is free */ -static bool closid_allocated(unsigned int closid) +bool closid_allocated(unsigned int closid) { lockdep_assert_held(&rdtgroup_mutex); From patchwork Tue Feb 13 18:44:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200573 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp752037dyb; Tue, 13 Feb 2024 11:09:11 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWhXrvZz3B7eI/e8PLGIF9XGAwCC2Da2GJG9W7Y8l9na5a48jw0faaf4jwbPrH+udEqoBcmeUAYJ9LXifvwrzrph/j2lw== X-Google-Smtp-Source: AGHT+IHjtbNcPrD413zbTFi2iaJFzAGaILkrO4FZUOhQGxhZrPpgb4bFOqCgt959FKBoPIZnaV2O X-Received: by 2002:a17:90a:c986:b0:296:2358:6e28 with SMTP id w6-20020a17090ac98600b0029623586e28mr4941820pjt.14.1707851350972; Tue, 13 Feb 2024 11:09:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851350; cv=pass; d=google.com; s=arc-20160816; b=SYx0QXCLsHBmGQRh0lCYbUhH+3iAeE0t0WDEBYAQbFRQiMDaUKJGLXFWtozGKXBRBJ ygR3xNJTBK0Y0oer9bWqLp4y2dB2Y/jL3plIzSDAhD8POcHjt7nH5+PaRCbeFSvrzhhz pZd4PxnIAmYs+ge1eQXISFeq3+oGB48VoLj8BsgsBTi0m/70uu7SsxylrimoX89m0jP+ 9+QbiospniC+zFvJ0Z0bnQ+iVlFp2OhAIa79cuB9CRDRktmtEX6E2E5quw9/qKjMMJK9 Of9pLqKNXb4ztPgrQ2nCwMG45fFD3hy7G+CIlGhu8P5TQegNqXc9KqVKhfXtzcyoPPv+ PSKA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=H0ewcUl90AYjSAONBYc2Zj+coQCcDC3/UITKLPfs5ww=; fh=9fcsUVSkpnL1qz2T0tD0+iYYUhJosalix5l2lgqyLKA=; b=jwAN/x2jjh1iQoOOg6IQwfP0T2JdqlcSr29EA7dfwUUAAr9q/HWooPFI8W/heQyqNu qa/y82hSejMTnJhVr37e0hk9KWnnL38SCo4K4lNpduC3dkl3o/Dwx34nKKDNqTYtv+Jm +pk0vdscj3i/w13LvMeWEbnSuHLkk4/7Us1u9N5U5+nS4oY7IiZ4A6CFcE6ikk7vDASQ 2aAmYWvq3ECitM53x8lae9xCRXCnHnwMWL+PifHBXVClTO4ND//aYH8JfpUSKET0KNpB Ae9K4uQc642MdCpJtwpc+7L6zRd6mezT7mC7YoyGDf6Z7rO0JPZKXQAYqG5MPzLy8EEX tBzg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64102-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64102-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCUj8Jf7aBNnycb989TKVewcJbC4xIhWsjKhk+m85ypt701lDO+wglvTc8EGb3+r9+Tkoy7VU+9IsU3O6T7ae2TDGllnEA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id f28-20020a63381c000000b005d8b5aace83si2322262pga.289.2024.02.13.11.09.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:09:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64102-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64102-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64102-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id C5D04B275F3 for ; Tue, 13 Feb 2024 18:48:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 76AE360ED2; Tue, 13 Feb 2024 18:45:43 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6397760DFD for ; Tue, 13 Feb 2024 18:45:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849941; cv=none; b=gFw1tncHu2+LVInaO1ZoL4Pzu/tsYoombmw3QTJLexHTFgjEw3sqXP4lHahfsviSuYFH62GjLW1fms8jmMqizNo2D4Iu3Pqx581UEzZphfdgecVFQH5LKKGXT7XS78CNxSzedEoXWwBVPG6tyaAatSSpzN1EkGW+59NXUN962lw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849941; c=relaxed/simple; bh=QrE8SkUDD8C7YoMMIPm2bwyYJzX8gB1Lw0s5UB0P0to=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ouxp7WOS2BZoEs0+EE6Khq+EM3OxFG88/8XSZvIi4sC+NK59f2txR+BEgSk4RJ57TWyJCOQzbSsfIZPs/P8TBIdKmtbOyLHuesRXvAYIt+mzPTuvgAyfJu6BvuP0bipnoYDntFgwxAlIuAzcYP3yJnN2Lrpz+zojIX+gpQvy9mc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 228751515; Tue, 13 Feb 2024 10:46:20 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BD3A33F766; Tue, 13 Feb 2024 10:45:35 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 11/24] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers Date: Tue, 13 Feb 2024 18:44:25 +0000 Message-Id: <20240213184438.16675-12-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811938195050860 X-GMAIL-MSGID: 1790811938195050860 When switching tasks, the CLOSID and RMID that the new task should use are stored in struct task_struct. For x86 the CLOSID known by resctrl, the value in task_struct, and the value written to the CPU register are all the same thing. MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct task_struct implies the arch code knows whether resctrl is using CDP. Move the matching and setting of the struct task_struct properties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time. __rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance. To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together. Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v2: * __rdtgroup_move_task() changed to set CLOSID from different CLOSID place depending on group type --- arch/x86/include/asm/resctrl.h | 18 ++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++++++++++---------- 2 files changed, 56 insertions(+), 24 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index db4c84dde2d5..1d274dbabc44 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -95,6 +95,24 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val) return val * scale; } +static inline void resctrl_arch_set_closid_rmid(struct task_struct *tsk, + u32 closid, u32 rmid) +{ + WRITE_ONCE(tsk->closid, closid); + WRITE_ONCE(tsk->rmid, rmid); +} + +static inline bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid) +{ + return READ_ONCE(tsk->closid) == closid; +} + +static inline bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 ignored, + u32 rmid) +{ + return READ_ONCE(tsk->rmid) == rmid; +} + static inline void resctrl_sched_in(struct task_struct *tsk) { if (static_branch_likely(&rdt_enable_key)) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 8fc46204a6cc..e42cbdf8f6a3 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -102,7 +102,7 @@ void rdt_staged_configs_clear(void) * * Using a global CLOSID across all resources has some advantages and * some drawbacks: - * + We can simply set "current->closid" to assign a task to a resource + * + We can simply set current's closid to assign a task to a resource * group. * + Context switch code can avoid extra memory references deciding which * CLOSID to load into the PQR_ASSOC MSR @@ -574,14 +574,26 @@ static void update_task_closid_rmid(struct task_struct *t) _update_task_closid_rmid(t); } +static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp) +{ + u32 closid, rmid = rdtgrp->mon.rmid; + + if (rdtgrp->type == RDTCTRL_GROUP) + closid = rdtgrp->closid; + else if (rdtgrp->type == RDTMON_GROUP) + closid = rdtgrp->mon.parent->closid; + else + return false; + + return resctrl_arch_match_closid(tsk, closid) && + resctrl_arch_match_rmid(tsk, closid, rmid); +} + static int __rdtgroup_move_task(struct task_struct *tsk, struct rdtgroup *rdtgrp) { /* If the task is already in rdtgrp, no need to move the task. */ - if ((rdtgrp->type == RDTCTRL_GROUP && tsk->closid == rdtgrp->closid && - tsk->rmid == rdtgrp->mon.rmid) || - (rdtgrp->type == RDTMON_GROUP && tsk->rmid == rdtgrp->mon.rmid && - tsk->closid == rdtgrp->mon.parent->closid)) + if (task_in_rdtgroup(tsk, rdtgrp)) return 0; /* @@ -592,19 +604,19 @@ static int __rdtgroup_move_task(struct task_struct *tsk, * For monitor groups, can move the tasks only from * their parent CTRL group. */ - - if (rdtgrp->type == RDTCTRL_GROUP) { - WRITE_ONCE(tsk->closid, rdtgrp->closid); - WRITE_ONCE(tsk->rmid, rdtgrp->mon.rmid); - } else if (rdtgrp->type == RDTMON_GROUP) { - if (rdtgrp->mon.parent->closid == tsk->closid) { - WRITE_ONCE(tsk->rmid, rdtgrp->mon.rmid); - } else { - rdt_last_cmd_puts("Can't move task to different control group\n"); - return -EINVAL; - } + if (rdtgrp->type == RDTMON_GROUP && + !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) { + rdt_last_cmd_puts("Can't move task to different control group\n"); + return -EINVAL; } + if (rdtgrp->type == RDTMON_GROUP) + resctrl_arch_set_closid_rmid(tsk, rdtgrp->mon.parent->closid, + rdtgrp->mon.rmid); + else + resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid, + rdtgrp->mon.rmid); + /* * Ensure the task's closid and rmid are written before determining if * the task is current that will decide if it will be interrupted. @@ -626,14 +638,15 @@ static int __rdtgroup_move_task(struct task_struct *tsk, static bool is_closid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_alloc_capable && - (r->type == RDTCTRL_GROUP) && (t->closid == r->closid)); + return (rdt_alloc_capable && (r->type == RDTCTRL_GROUP) && + resctrl_arch_match_closid(t, r->closid)); } static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_mon_capable && - (r->type == RDTMON_GROUP) && (t->rmid == r->mon.rmid)); + return (rdt_mon_capable && (r->type == RDTMON_GROUP) && + resctrl_arch_match_rmid(t, r->mon.parent->closid, + r->mon.rmid)); } /** @@ -884,7 +897,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, rdtg->mode != RDT_MODE_EXCLUSIVE) continue; - if (rdtg->closid != tsk->closid) + if (!resctrl_arch_match_closid(tsk, rdtg->closid)) continue; seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "", @@ -892,7 +905,8 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, seq_puts(s, "mon:"); list_for_each_entry(crg, &rdtg->mon.crdtgrp_list, mon.crdtgrp_list) { - if (tsk->rmid != crg->mon.rmid) + if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid, + crg->mon.rmid)) continue; seq_printf(s, "%s", crg->kn->name); break; @@ -2820,8 +2834,8 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to, for_each_process_thread(p, t) { if (!from || is_closid_match(t, from) || is_rmid_match(t, from)) { - WRITE_ONCE(t->closid, to->closid); - WRITE_ONCE(t->rmid, to->mon.rmid); + resctrl_arch_set_closid_rmid(t, to->closid, + to->mon.rmid); /* * Order the closid/rmid stores above before the loads From patchwork Tue Feb 13 18:44:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200556 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp739519dyb; Tue, 13 Feb 2024 10:48:16 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUkPPS5435Gnl8cSi3eM6p9VoFckDcM2OihxbFwcSsLyarW33xMBbUr7LzROQNonqukikyLrFunMBW8Ha+GMApO7+MTcg== X-Google-Smtp-Source: AGHT+IFaRvKmkVxJlqXNDwKZnByG6fE6bEyTp711TaHHPrYW1WVd9//KZQeF5BJkPOAjwXohSEx5 X-Received: by 2002:a17:906:b21a:b0:a3c:8495:6b7e with SMTP id p26-20020a170906b21a00b00a3c84956b7emr183331ejz.1.1707850096704; Tue, 13 Feb 2024 10:48:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850096; cv=pass; d=google.com; s=arc-20160816; b=YjIXjETgNzZs5aWcQuijFcmAGqyRmBNQLu5BDAADLRjJYlN+focHQVnG/h5y4C0je1 wZfrjTsJrHnXwIt7WKaFe1o1M5WWom9/a7sAsurPfqmdRJBQuJuGAk39qS+T/ZHsZmE9 rlY8w44Aot3uML4K/6002xLX7POCtdZa6lRTAhU1LmCXV3v4jbQvzBRVjT5ZxGshUF/p L5CY0ni8/RCx8kB6upaWPEXhFRDIjWhPA1/X1OGRA3DI4WAOSg4xuVjTHqW5dMJt8Je+ MF/E42fTkS3rvvTjr1IluK2DRvIwFRQb8QUjr6fc99HEuxD5Xk1NRYYs2QPyzPeBZGGt fvew== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=bhQxGZVAEGUDnTH73bBgu+E4LP8c3vmP/LlRL93+uCE=; fh=wuucEwf2KxK+uHK6Y7ypIjudfr9Txw56+Yyo5iRnnys=; b=nYBkToRWkYV7eeqnzhaDcc2W19wZMx/PjM54hiWwl7EsDl6WPuwwblzIpveUJ//t94 Ym9jAc/EDXMAQaJNQWYOPvA+0gEi84Cj96mZVq9GrgMeFt894kTkuXcPAeblHJcfmPKK 65CwlI6QkN/MbYTK/DcQJaLpSyYFDdYeEtI5RaNgtm6aD1BlV7DdHnKfZonk7wY4KE+P AKB101fgftkZMQZhAL87DNdNTzVvprohjVTDhV9aDRgyd/EQGtgUjX+/CDOTP86KgWo/ vvEl5w+TuIEfM9p0hv9CQC9M9sLJPXZ8NC7PMmO056cOFPDGkZsiyLb1Be8p+bg3kdOl 7v7A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXokg0FhB2GMeKUzBwqq1DSMuYkTKMd7YP51zuKdvqKoi00XNJFYY77rYSsuWIrD/8Yw+AMo6UKQ+WZNaj9SIEf1wWXOg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id v18-20020a170906339200b00a3cf5d9315asi1032461eja.482.2024.02.13.10.48.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:48:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64103-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 02DC31F24866 for ; Tue, 13 Feb 2024 18:48:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 364BB612DB; Tue, 13 Feb 2024 18:45:46 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A3BA060EF3 for ; Tue, 13 Feb 2024 18:45:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849944; cv=none; b=lre71dxAPShje3zi66tZTF6BKn4nnsGFzxK9BsZFMzB0ec5aiQPVQrDX8lxoMGFL0qobsE/y5EpCdG7INDcy1vh+i5Idynfj9OO1mfikIXCSJjh3Qq/0mU45AY5Cwg9faRtMTEzWH1a7InEAObYz5dOFlQw6engffUJQURLEoQk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849944; c=relaxed/simple; bh=tCk5Rp6ufC7ekXhYD/3kjr90yolxWKM1koNMSXSV0dY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IRUgEIplDqbvtEK9mnIt05n9Ucgc3oa64ga/CGxoOlMgTKGSeQCNy9aZJGIhfGj2PBfqXt2zoraLxUJgpgOuMm9y/8LM31xyVyEMnVRJmNtlId+uOvY6TPU1/M13bv6S9VU8uyX2do+mxUseCN4pILmhpi1AtLBZZ4B7cZxfI1U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6B9081516; Tue, 13 Feb 2024 10:46:23 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0C89B3F766; Tue, 13 Feb 2024 10:45:38 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 12/24] x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow Date: Tue, 13 Feb 2024 18:44:26 +0000 Message-Id: <20240213184438.16675-13-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810623023326686 X-GMAIL-MSGID: 1790810623023326686 The limbo and overflow code picks a CPU to use from the domain's list of online CPUs. Work is then scheduled on these CPUs to maintain the limbo list and any counters that may overflow. cpumask_any() may pick a CPU that is marked nohz_full, which will either penalise the work that CPU was dedicated to, or delay the processing of limbo list or counters that may overflow. Perhaps indefinitely. Delaying the overflow handling will skew the bandwidth values calculated by mba_sc, which expects to be called once a second. Add cpumask_any_housekeeping() as a replacement for cpumask_any() that prefers housekeeping CPUs. This helper will still return a nohz_full CPU if that is the only option. The CPU to use is re-evaluated each time the limbo/overflow work runs. This ensures the work will move off a nohz_full CPU once a housekeeping CPU is available. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v3: * typos fixed Changes since v4: * Made temporary variables unsigned Changes since v5: * Restructured cpumask_any_housekeeping() to avoid later churn. Changes since v6: * Update mbm_work_cpu/cqm_work_cpu when rescheduling. --- arch/x86/kernel/cpu/resctrl/internal.h | 24 ++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 20 +++++++++++++------- 2 files changed, 37 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index b7b9d9230bef..81f5de916db8 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -7,6 +7,7 @@ #include #include #include +#include #include @@ -55,6 +56,29 @@ /* Max event bits supported */ #define MAX_EVT_CONFIG_BITS GENMASK(6, 0) +/** + * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that + * aren't marked nohz_full + * @mask: The mask to pick a CPU from. + * + * Returns a CPU in @mask. If there are housekeeping CPUs that don't use + * nohz_full, these are preferred. + */ +static inline unsigned int cpumask_any_housekeeping(const struct cpumask *mask) +{ + unsigned int cpu, hk_cpu; + + cpu = cpumask_any(mask); + if (!tick_nohz_full_cpu(cpu)) + return cpu; + + hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask); + if (hk_cpu < nr_cpu_ids) + cpu = hk_cpu; + + return cpu; +} + struct rdt_fs_context { struct kernfs_fs_context kfc; bool enable_cdpl2; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 101f1b112d17..38f85e53ca93 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -761,7 +761,6 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, void cqm_handle_limbo(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL); - int cpu = smp_processor_id(); struct rdt_domain *d; mutex_lock(&rdtgroup_mutex); @@ -770,8 +769,11 @@ void cqm_handle_limbo(struct work_struct *work) __check_limbo(d, false); - if (has_busy_rmid(d)) - schedule_delayed_work_on(cpu, &d->cqm_limbo, delay); + if (has_busy_rmid(d)) { + d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask); + schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo, + delay); + } mutex_unlock(&rdtgroup_mutex); } @@ -781,7 +783,7 @@ void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms) unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - cpu = cpumask_any(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->cqm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); @@ -791,7 +793,6 @@ void mbm_handle_overflow(struct work_struct *work) { unsigned long delay = msecs_to_jiffies(MBM_OVERFLOW_INTERVAL); struct rdtgroup *prgrp, *crgrp; - int cpu = smp_processor_id(); struct list_head *head; struct rdt_resource *r; struct rdt_domain *d; @@ -815,7 +816,12 @@ void mbm_handle_overflow(struct work_struct *work) update_mba_bw(prgrp, d); } - schedule_delayed_work_on(cpu, &d->mbm_over, delay); + /* + * Re-check for housekeeping CPUs. This allows the overflow handler to + * move off a nohz_full CPU quickly. + */ + d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask); + schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay); out_unlock: mutex_unlock(&rdtgroup_mutex); @@ -828,7 +834,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) if (!static_branch_likely(&rdt_mon_enable_key)) return; - cpu = cpumask_any(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->mbm_work_cpu = cpu; schedule_delayed_work_on(cpu, &dom->mbm_over, delay); } From patchwork Tue Feb 13 18:44:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200557 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp739630dyb; Tue, 13 Feb 2024 10:48:31 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUB4EbVOX5i3gmAt3GU2jvgG1uCalmFgDx2zWtgN86N7dhkE5vSOKgFjAG7KU+HEol1YVRfqoOULn9GN1S60OA4H4k9ig== X-Google-Smtp-Source: AGHT+IEqAvqkUHInl0GFc/kIbBNlZZdMsJyFRsPbmJmX49I58BCTz1T0C0cXnkHhxRBEzdefU8BB X-Received: by 2002:a17:906:b852:b0:a3c:c97e:7d63 with SMTP id ga18-20020a170906b85200b00a3cc97e7d63mr116402ejb.48.1707850111167; Tue, 13 Feb 2024 10:48:31 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850111; cv=pass; d=google.com; s=arc-20160816; b=qrFC7ygQJsMccfhj7ZNR5+2h1XWVGUz3mXXYwv3I3MVlOf2B035gGuPOFWzU007Zk9 3NpCaBfDR+ngWgjo/qJwqNqJtgJagL1xxGulblYiRFbTPf4FwJ4r8mBeHeKwR7UuyLt/ HBAiZ/IhSVryEltoVcJAuMW+YPCY5w1q5M6j4eN+J4OgsvMdx3/h6k+JAOZaSja1YhTY Qt+Kgv1VXUTxnEQXYBIGyQe9gkOMwFrclyT8FHJhnUUhPWdjcSOrdCrK/vQWfRzCCOhN O3vX34G3vBtjeZCOZfPXK/8uZyuzbXYO6bMNQ/B73ACrM9j3q6N3fBcEFdHWafOr/CS5 5GSA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=CK5bHYyML+5ggglC8i1zhWHzgKvxzECq517FxCys7WU=; fh=WfSQBSaCeP2KwkGTqRiYyP/3ood+v3b1D5w8ZCIHPGM=; b=HzeUdsNa/U1xeM484gIcx3duzyhSu9+h8ADBYkSZTKt8Vf6Y/yHjZ4pkhrzXZba+S/ ogFhmwU6EoBHlgjHQ4nh467VECr3VsInJs3Fow6DH7pkR2FT05G46/5wYYO1GPfbFUhV PJUsh2Ig5j4DPhAyN7vmabVBEmwKCKpXgdy/aeQxNA83Sm/X+1+tyDpFwmOxPeCX7Hwn JX6EbE8j/aEezMLbUr8vF1lh6l/nll8qbHiuEZAZPEc6csOeIbIHQKAlema8S7Kq6PjY yt1oiOdAfwbfempaRaaEM8aTMbE7HwYnD2hFDihWP28zCIcheOrO4CRd9Js/tOb3N0rd eqaA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64104-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64104-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCVRgZglpCtjb3LZODRBCqGlOyc9JF4E2CmiQcaHGm3pCab31/k7OS5cxJXJlwbhGaBKvxwI09iU3mILJb/eBXWe92n0sQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id v18-20020a170906339200b00a3cf5d9315asi1032461eja.482.2024.02.13.10.48.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:48:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64104-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64104-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64104-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 9D1A71F2200B for ; Tue, 13 Feb 2024 18:48:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2460A612C6; Tue, 13 Feb 2024 18:45:49 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C08C6612D6 for ; Tue, 13 Feb 2024 18:45:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849947; cv=none; b=aSZzROPtIigeAp6weMkI1GIsEJ2NqvdbxRZkO+fuJKtuTVxIVzXRrpssRSferYekCF2gRtncXb1zmRLzfALxW43wisBTi3Wpc0o791ez7EqhOCawpfGFUiuoFCeIJuRE0O4aZkHMN+lkzQFxbMqsndDvDU5vYmZhSGoZ/IuV4ks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849947; c=relaxed/simple; bh=KFwlStvMlPXKUvkhpDhAxvJVdMWOp52J5AhonkDwfAg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PS0NFqwBEcxuktQ7obfYm5LopptCSaOZ/YCMrA1B9t4bVUyDp9sHqWgB5/5jJlErAM4dBSokuDtwTnggCDxIVtWuZwszcd2JUdIx+ChTmDOV5eoIV05l0j8ei24qy0NZgn6oM7qh0C3rwrAEV54nMMi3yKSi2No8lgPX1nFGTaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BF3191570; Tue, 13 Feb 2024 10:46:26 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 677783F766; Tue, 13 Feb 2024 10:45:42 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 13/24] x86/resctrl: Queue mon_event_read() instead of sending an IPI Date: Tue, 13 Feb 2024 18:44:27 +0000 Message-Id: <20240213184438.16675-14-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810638028029377 X-GMAIL-MSGID: 1790810638028029377 Intel is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed. MPAM's CSU monitors are used to back the 'llc_occupancy' monitor file. The CSU counter is allowed to return 'not ready' for a small number of micro-seconds after programming. To allow one CSU hardware monitor to be used for multiple control or monitor groups, the CPU accessing the monitor needs to be able to block when configuring and reading the counter. Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs. These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced. mon_event_read() already invokes mon_event_count() via IPI, which means this isn't possible. On systems using nohz-full, some CPUs need to be interrupted to run kernel work as they otherwise stay in user-space running realtime workloads. Interrupting these CPUs should be avoided, and scheduling work on them may never complete. Change mon_event_read() to pick a housekeeping CPU, (one that is not using nohz_full) and schedule mon_event_count() and wait. If all the CPUs in a domain are using nohz-full, then an IPI is used as the fallback. This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code). This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in monitor.c. When the IPI fallback is used on machines where MPAM needs to make an access on multiple CPUs, the counter read will always fail. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Peter Newman Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v2: * Use cpumask_any_housekeeping() and fallback to an IPI if needed. Changes since v3: * Actually include the IPI fallback code. Changes since v4: * Tinkered with existing capitalisation. Changes since v5: * Added a newline. Changes since v6: * Moved lockdep annotations to a later patch. --- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 26 +++++++++++++++++++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index beccb0e87ba7..d07f99245851 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -19,6 +19,8 @@ #include #include #include +#include + #include "internal.h" /* @@ -522,12 +524,21 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, return ret; } +static int smp_mon_event_count(void *arg) +{ + mon_event_count(arg); + + return 0; +} + void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first) { + int cpu; + /* - * setup the parameters to send to the IPI to read the data. + * Setup the parameters to pass to mon_event_count() to read the data. */ rr->rgrp = rdtgrp; rr->evtid = evtid; @@ -536,7 +547,18 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val = 0; rr->first = first; - smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1); + cpu = cpumask_any_housekeeping(&d->cpu_mask); + + /* + * cpumask_any_housekeeping() prefers housekeeping CPUs, but + * are all the CPUs nohz_full? If yes, pick a CPU to IPI. + * MPAM's resctrl_arch_rmid_read() is unable to read the + * counters on some platforms if its called in irq context. + */ + if (tick_nohz_full_cpu(cpu)) + smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1); + else + smp_call_on_cpu(cpu, smp_mon_event_count, rr, false); } int rdtgroup_mondata_show(struct seq_file *m, void *arg) diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 38f85e53ca93..fd060ef86f38 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -585,7 +585,7 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) } /* - * This is called via IPI to read the CQM/MBM counters + * This is scheduled by mon_event_read() to read the CQM/MBM counters * on a domain. */ void mon_event_count(void *info) From patchwork Tue Feb 13 18:44:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200574 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp752494dyb; Tue, 13 Feb 2024 11:09:56 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWhjkHiM3fOnZEetGldZUQQrG7Z0obB0i4b2PhH4rOxDGc8nS5blw7o5nT3tr2AqYbkculwirlW6n1U97sm11CWGOh62Q== X-Google-Smtp-Source: AGHT+IHkcpPN5qz5rlqTNe3JOQNiHwf6/LEwPSrsr4Lh/ODOfKVPtx6kIcBm9YKA+HbVyCQhAp+0 X-Received: by 2002:aa7:86c1:0:b0:6e1:782:ce26 with SMTP id h1-20020aa786c1000000b006e10782ce26mr173223pfo.19.1707851396615; Tue, 13 Feb 2024 11:09:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851396; cv=pass; d=google.com; s=arc-20160816; b=tlYZCHCZq2bDA7VAitLoQa1zLvl49tbbXNyATWLgUq7pD13CLwS94cOXG3GaMj5WFp 6K6ZMbF6ZlYmCkhYIMmqP5wyqswK+hmsBN9nzQlFBukOTVFXFZcI0h62uL+vIpvQLdBe LLs3obIxFy5EyfWFSgfIpUVe8nHRLzORgXO48z5GPBH4tumEqaIxX7MV29HxGGmc7DTt ejDZ8BDdRBNYFc+zU405K8jxZaCcIq01JuxZsJlNArDUgfQguCWrDE+AmbqLZ+kcSwtt aMt1OWoWMMKWtMQGXd035HoaHxbJAmflTXblZxUSN3aeF+e4HQRiSFld1pi+EgkRZnUR AIWw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=X+o34MG7Ip0Rl/SXSUzsSR0bS8mdCqY7DX+o4M/C1Yk=; fh=KNQ0IyKiVinzRHYEVjLVZh2k08LBXkzQWQvf7DBDRgY=; b=L3vHF+gLZBvN/HB/Ij9cvkG+tNr/sKrZRwec/3mkNDwdXdtMdhkzvknzaCWqzKulRh fqWMG0d8ewxl9yirJaLI7TlrMwjY+PrXWuBGRbW6+bKHIurrfO80wMqkrC/2VMJpv63y eT7TqRKcBG+XhKZQeFuLoJDyCNX0sAB6hv6zRbOk75EK/xxNVAlC+kbDNZC+psdurjhd J0MN7zURVyGQ2laGbA5WlpbzugqcSNkM3RR8516EoXHO4KqXxuWrUIHyFEggkuqPI9r9 j+/xxvGO9IF9+bjzmxfttgVL4tmC2skJTBrc8JhivjpucCNpmJsDXPreGbNUJC8YxI8H TljQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64105-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64105-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXdOovdXAHCPkzF1fslxWoT9Kbv6Cs/gvOJJNChxV94QpyhY7hkI9QHEY131oSji0MP2Gzw1mqG7PDNvH4HB0vrtYDx1g== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id a8-20020a056a000c8800b006e0f47e6491si2171096pfv.0.2024.02.13.11.09.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:09:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64105-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64105-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64105-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 6262EB2806C for ; Tue, 13 Feb 2024 18:48:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 624D86167A; Tue, 13 Feb 2024 18:45:53 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4B43C612FB for ; Tue, 13 Feb 2024 18:45:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849951; cv=none; b=qw0DfcrOKjCV5gI/HEwjf9jpx8gD9E79x33nDYpWCCmO/37n95XbfYYumobcXOjq1DPjlqqUeW8uR0cRHjdCASwmVu5hhbmycsjVVFfQ3pdJspoGoFPQg5LoT7IAyK3RacnU1Cf7N5vDIuaBraYN2S4o7fEYO6Eejv4zOQMlxg8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849951; c=relaxed/simple; bh=ZaRjZOZBOoo8Deyb17/rBdMZ+TyPLScylxjo8oaeW24=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qeLMmUANxhusH7KED10Pqocjl3nL1MWzhhPEOiRIc8IXbTQqHvf0yhYZbtEMXrI3vmsYbogD1sOE0sGLsKPYHz3vnhfVGG45x4CcQf0y0qXRCQQ/kanGKsHg1nw7Xxa7AiI5TlTm9vcDI0M9Gq7IgKr7NbRrxhmPBBcahINowT8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 114431576; Tue, 13 Feb 2024 10:46:30 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AB1B43F766; Tue, 13 Feb 2024 10:45:45 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 14/24] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep Date: Tue, 13 Feb 2024 18:44:28 +0000 Message-Id: <20240213184438.16675-15-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790811985596652910 X-GMAIL-MSGID: 1790811985596652910 MPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's resctrl_arch_rmid_read(). resctrl_arch_rmid_read() may be called via IPI meaning it is unable to sleep. In this case resctrl_arch_rmid_read() should return an error if it needs to sleep. This will only affect MPAM platforms where the cache occupancy counter isn't available immediately, nohz_full is in use, and there are no housekeeping CPUs in the necessary domain. There are three callers of resctrl_arch_rmid_read(): __mon_event_count() and __check_limbo() are both called from a non-migrateable context. mon_event_read() invokes __mon_event_count() using smp_call_on_cpu(), which adds work to the target CPUs workqueue. rdtgroup_mutex() is held, meaning this cannot race with the resctrl cpuhp callback. __check_limbo() is invoked via schedule_delayed_work_on() also adds work to a per-cpu workqueue. The remaining call is add_rmid_to_limbo() which is called in response to a user-space syscall that frees an RMID. This opportunistically reads the LLC occupancy counter on the current domain to see if the RMID is over the dirty threshold. This has to disable preemption to avoid reading the wrong domain's value. Disabling pre-emption here prevents resctrl_arch_rmid_read() from sleeping. add_rmid_to_limbo() walks each domain, but only reads the counter on one domain. If the system has more than one domain, the RMID will always be added to the limbo list. If the RMIDs usage was not over the threshold, it will be removed from the list when __check_limbo() runs. Make this the default behaviour. Free RMIDs are always added to the limbo list for each domain. The user visible effect of this is that a clean RMID is not available for re-allocation immediately after 'rmdir()' completes, this behaviour was never portable as it never happened on a machine with multiple domains. Removing this path allows resctrl_arch_rmid_read() to sleep if its called with interrupts unmasked. Document this is the expected behaviour, and add a might_sleep() annotation to catch changes that won't work on arm64. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- The previous version allowed resctrl_arch_rmid_read() to be called on the wrong CPUs, but now that this needs to take nohz_full and housekeeping into account, its too complex. Changes since v3: * Removed error handling for smp_call_function_any(), this can't race with the cpuhp callbacks as both hold rdtgroup_mutex. * Switched to the alternative of removing the counter read, this simplifies things dramatically. Changes since v4: * Messed with capitalisation. * Removed some dead code now that entry->busy will never be zero in add_rmid_to_limbo(). * Rephrased the comment above resctrl_arch_rmid_read_context_check(). Changes since v5: * Really rephrased the comment above resctrl_arch_rmid_read_context_check(). --- arch/x86/kernel/cpu/resctrl/monitor.c | 25 +++++-------------------- include/linux/resctrl.h | 23 ++++++++++++++++++++++- 2 files changed, 27 insertions(+), 21 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index fd060ef86f38..e8aeff6673ea 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -277,6 +277,8 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u64 msr_val, chunks; int ret; + resctrl_arch_rmid_read_context_check(); + if (!cpumask_test_cpu(smp_processor_id(), &d->cpu_mask)) return -EINVAL; @@ -455,8 +457,6 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) { struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; struct rdt_domain *d; - int cpu, err; - u64 val = 0; u32 idx; lockdep_assert_held(&rdtgroup_mutex); @@ -464,17 +464,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); entry->busy = 0; - cpu = get_cpu(); list_for_each_entry(d, &r->domains, list) { - if (cpumask_test_cpu(cpu, &d->cpu_mask)) { - err = resctrl_arch_rmid_read(r, d, entry->closid, - entry->rmid, - QOS_L3_OCCUP_EVENT_ID, - &val); - if (err || val <= resctrl_rmid_realloc_threshold) - continue; - } - /* * For the first limbo RMID in the domain, * setup up the limbo worker. @@ -484,15 +474,10 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) set_bit(idx, d->rmid_busy_llc); entry->busy++; } - put_cpu(); - if (entry->busy) { - rmid_limbo_count++; - if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) - closid_num_dirty_rmid[entry->closid]++; - } else { - list_add_tail(&entry->list, &rmid_free_lru); - } + rmid_limbo_count++; + if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) + closid_num_dirty_rmid[entry->closid]++; } void free_rmid(u32 closid, u32 rmid) diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index bd4ec22b5a96..8649fc84aac2 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -236,7 +236,12 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. * - * Call from process context on a CPU that belongs to domain @d. + * Some architectures need to sleep when first programming some of the counters. + * (specifically: arm64's MPAM cache occupancy counters can return 'not ready' + * for a short period of time). Call from a non-migrateable process context on + * a CPU that belongs to domain @d. e.g. use smp_call_on_cpu() or + * schedule_work_on(). This function can be called with interrupts masked, + * e.g. using smp_call_function_any(), but may consistently return an error. * * Return: * 0 on success, or -EIO, -EINVAL etc on error. @@ -245,6 +250,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, u64 *val); +/** + * resctrl_arch_rmid_read_context_check() - warn about invalid contexts + * + * When built with CONFIG_DEBUG_ATOMIC_SLEEP generate a warning when + * resctrl_arch_rmid_read() is called with preemption disabled. + * + * The contract with resctrl_arch_rmid_read() is that if interrupts + * are unmasked, it can sleep. This allows NOHZ_FULL systems to use an + * IPI, (and fail if the call needed to sleep), while most of the time + * the work is scheduled, allowing the call to sleep. + */ +static inline void resctrl_arch_rmid_read_context_check(void) +{ + if (!irqs_disabled()) + might_sleep(); +} /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid From patchwork Tue Feb 13 18:44:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200575 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp752631dyb; Tue, 13 Feb 2024 11:10:12 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUH/Kva/La7EmfKnc0K6wshE/LzZR8EV+9x7QKTEhe1F1G3h5BOYOVkHMto6L5xTZK8XdDX90Uns9gZ+1/K4oSf3BP/qQ== X-Google-Smtp-Source: AGHT+IFCpOIE31EIfpCBbBlKQIdOhCij+htAZdzIQ//2+xNEr7Wb5E6L0QOo0MA+61lb/9P9IXkC X-Received: by 2002:a05:6a20:9597:b0:19c:9eea:e731 with SMTP id iu23-20020a056a20959700b0019c9eeae731mr627610pzb.40.1707851412351; Tue, 13 Feb 2024 11:10:12 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851412; cv=pass; d=google.com; s=arc-20160816; b=XGa/qQFE68SbgQ1Vp0BNxMKn9e5P+b8ng8/2qs1gu5v6v9jpxlWzsd960QAhKWczFM 5w61cyUX/wtIMR6diqdRmaiGoHk9fXguPY/EQy6RVQSlphYLrPMmYPEuvgb2mK/zgQpf xjNDTxCmM5GWYikTs/VrsnXk7raBqQKFcOOME28Oq8fY35k2mSceVVvFoplyZ1mvYA7w P5weZypTffa5gAg1HLaTD/7L6TTUzBMjmnJJWzu4IxTFLCsmfgxurVaMDYWR+uL9Me6J wugYY56768qSKnBqoZdhusZl1ej1eTMg9mpLi5hv+8wkM4OJefDg3XAVHnnAWtvVs/g2 h0rA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=zdQONivDQiwDYeI16fxlx3omaWSRkjyjTxQvDXJGd/A=; fh=q1IpxysXUzuLl65yAvUVGhEHIThnCIaqrsrEGP4VNi8=; b=vRGFX1AJojMSNerQaqXm6iCc+iR2rZst1nCZvNXG75xSb+sU4S0XbiA9dCEnHiSzZJ 2xpmuvDiIHDe1/pHvGNgNoMW+OmAWI9J+JMHb9wcLw4iTNdhd8PtWeQQkzj3MJ69fOEP Co/Go4m3iRX8yaJDGR655mOIaLg/QfwL3l66g6ddomDcB1D+C0e83v0bUGJnM183+DFA lFpZeZrIo6wDFF3QkoDP7znZKZi26MHXEPTB1SvD5ZeorgDF8hhbP2ym5lOfjeVZrfx4 60+HTBn4M3w9rAPHyhSLIb9txRQwtWOY+/lF5WF79uAQY0jKlbKS0/SAqYi/zRuIfshX W0Gg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64106-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64106-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXS0dFSRso5tU2RZ/p6nf/8WqwfIiF71VmiALY0MtYi/bPKZYk/jSJ4B2fk0Aan9/sZQxIyBzEpNXnvdzNvSPaFGvOO9A== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id d2-20020a631d42000000b005dc81a6b2absi1804924pgm.792.2024.02.13.11.10.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:10:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64106-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64106-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64106-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 1869DB28317 for ; Tue, 13 Feb 2024 18:49:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D3C486169C; Tue, 13 Feb 2024 18:45:56 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 93A9861664 for ; Tue, 13 Feb 2024 18:45:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849954; cv=none; b=fhbhVlJ+0KGLHaAjlIzLOtbe7b7MsbX6ztAOCG/RXeLm1+YERkltYrjCFxGc5i9HXz6CCYMtldzmtLjViHvvOAgOMYbUoXqfUR2dEDFjxxYngg3lSZK58QwzL/t8D6qnAtzJg1i/qNGI3yChwx0Ed3Y3V/sKjpdXUJe1e3d4+00= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849954; c=relaxed/simple; bh=0PsV4UkfxjfIIYMAqXVZWfwiBlgMM8uu8FaU8sF2C0Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tt++1pxFnt1icYd7tZSjHqcP8zGCt2ZVMFeMW6Qf5y8tT4UiOoMmSyNHCSuJQTZlbjEjnrEJFM28+VMRo/vTLooZOU6VA0EWW5n9AloziZM/cMVONcWoaZa9FS2RhPCXaHH4+qJ02XY0RSZahZfgo61Gqsto7WkBiOjMjohq3PI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B1431595; Tue, 13 Feb 2024 10:46:33 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 01BCE3F766; Tue, 13 Feb 2024 10:45:48 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 15/24] x86/resctrl: Allow arch to allocate memory needed in resctrl_arch_rmid_read() Date: Tue, 13 Feb 2024 18:44:29 +0000 Message-Id: <20240213184438.16675-16-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790812002502513451 X-GMAIL-MSGID: 1790812002502513451 Depending on the number of monitors available, Arm's MPAM may need to allocate a monitor prior to reading the counter value. Allocating a contended resource may involve sleeping. __check_limbo() and mon_event_count() each make multiple calls to resctrl_arch_rmid_read(), to avoid extra work on contended systems, the allocation should be valid for multiple invocations of resctrl_arch_rmid_read(). The memory or hardware allocated is not specific to a domain. Add arch hooks for this allocation, which need calling before resctrl_arch_rmid_read(). The allocated monitor is passed to resctrl_arch_rmid_read(), then freed again afterwards. The helper can be called on any CPU, and can sleep. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v3: * Expanded comment. * Removed stray header include. * Reworded commit message. * Made ctx a void * instead of an int. Changes since v4: * Used IS_ERR() in more places. Changes since v5: * Pass the error back from mon_event_read() as -EINVAL/Unavailable. * Add some ratelimited warnings when failing to allocate a mon context Changes since v6: * Added a newline. Changes since v7: * Replaced explanation of why the memory allocation should be global as the old explanation made no sense now that the patches have changed order. --- arch/x86/include/asm/resctrl.h | 11 +++++++ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 7 +++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 35 +++++++++++++++++++++-- include/linux/resctrl.h | 5 +++- 5 files changed, 55 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 1d274dbabc44..29c4cc343787 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -136,6 +136,17 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid) return rmid; } +/* x86 can always read an rmid, nothing needs allocating */ +struct rdt_resource; +static inline void *resctrl_arch_mon_ctx_alloc(struct rdt_resource *r, int evtid) +{ + might_sleep(); + return NULL; +}; + +static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r, int evtid, + void *ctx) { }; + void resctrl_cpu_detect(struct cpuinfo_x86 *c); #else diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index d07f99245851..a033e8e32108 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -546,6 +546,11 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->d = d; rr->val = 0; rr->first = first; + rr->arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, evtid); + if (IS_ERR(rr->arch_mon_ctx)) { + rr->err = -EINVAL; + return; + } cpu = cpumask_any_housekeeping(&d->cpu_mask); @@ -559,6 +564,8 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, smp_call_function_any(&d->cpu_mask, mon_event_count, rr, 1); else smp_call_on_cpu(cpu, smp_mon_event_count, rr, false); + + resctrl_arch_mon_ctx_free(r, evtid, rr->arch_mon_ctx); } int rdtgroup_mondata_show(struct seq_file *m, void *arg) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 81f5de916db8..e089d1a1a055 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -137,6 +137,7 @@ struct rmid_read { bool first; int err; u64 val; + void *arch_mon_ctx; }; extern bool rdt_alloc_capable; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index e8aeff6673ea..9b503e6ac490 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -269,7 +269,7 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 unused, u32 rmid, enum resctrl_event_id eventid, - u64 *val) + u64 *val, void *ignored) { struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r); struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); @@ -324,9 +324,17 @@ void __check_limbo(struct rdt_domain *d, bool force_free) u32 idx_limit = resctrl_arch_system_num_rmid_idx(); struct rmid_entry *entry; u32 idx, cur_idx = 1; + void *arch_mon_ctx; bool rmid_dirty; u64 val = 0; + arch_mon_ctx = resctrl_arch_mon_ctx_alloc(r, QOS_L3_OCCUP_EVENT_ID); + if (IS_ERR(arch_mon_ctx)) { + pr_warn_ratelimited("Failed to allocate monitor context: %ld", + PTR_ERR(arch_mon_ctx)); + return; + } + /* * Skip RMID 0 and start from RMID 1 and check all the RMIDs that * are marked as busy for occupancy < threshold. If the occupancy @@ -340,7 +348,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) entry = __rmid_entry(idx); if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid, - QOS_L3_OCCUP_EVENT_ID, &val)) { + QOS_L3_OCCUP_EVENT_ID, &val, + arch_mon_ctx)) { rmid_dirty = true; } else { rmid_dirty = (val >= resctrl_rmid_realloc_threshold); @@ -353,6 +362,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } cur_idx = idx + 1; } + + resctrl_arch_mon_ctx_free(r, QOS_L3_OCCUP_EVENT_ID, arch_mon_ctx); } bool has_busy_rmid(struct rdt_domain *d) @@ -533,7 +544,7 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) } rr->err = resctrl_arch_rmid_read(rr->r, rr->d, closid, rmid, rr->evtid, - &tval); + &tval, rr->arch_mon_ctx); if (rr->err) return rr->err; @@ -722,11 +733,27 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, if (is_mbm_total_enabled()) { rr.evtid = QOS_L3_MBM_TOTAL_EVENT_ID; rr.val = 0; + rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid); + if (IS_ERR(rr.arch_mon_ctx)) { + pr_warn_ratelimited("Failed to allocate monitor context: %ld", + PTR_ERR(rr.arch_mon_ctx)); + return; + } + __mon_event_count(closid, rmid, &rr); + + resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx); } if (is_mbm_local_enabled()) { rr.evtid = QOS_L3_MBM_LOCAL_EVENT_ID; rr.val = 0; + rr.arch_mon_ctx = resctrl_arch_mon_ctx_alloc(rr.r, rr.evtid); + if (IS_ERR(rr.arch_mon_ctx)) { + pr_warn_ratelimited("Failed to allocate monitor context: %ld", + PTR_ERR(rr.arch_mon_ctx)); + return; + } + __mon_event_count(closid, rmid, &rr); /* @@ -736,6 +763,8 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, */ if (is_mba_sc(NULL)) mbm_bw_count(closid, rmid, &rr); + + resctrl_arch_mon_ctx_free(rr.r, rr.evtid, rr.arch_mon_ctx); } } diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 8649fc84aac2..bf460c912bf5 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -235,6 +235,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. + * @arch_mon_ctx: An architecture specific value from + * resctrl_arch_mon_ctx_alloc(), for MPAM this identifies + * the hardware monitor allocated for this read request. * * Some architectures need to sleep when first programming some of the counters. * (specifically: arm64's MPAM cache occupancy counters can return 'not ready' @@ -248,7 +251,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); */ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, - u64 *val); + u64 *val, void *arch_mon_ctx); /** * resctrl_arch_rmid_read_context_check() - warn about invalid contexts From patchwork Tue Feb 13 18:44:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200558 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740018dyb; Tue, 13 Feb 2024 10:49:16 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXbvwyky/mxT+M9TYSNqjovlo2gHjpOywrDGdyeUesd+R0sde+AQH+GZCbj9Mi1Z+ukL3rOaZQzzUjqSXPAZKdd3suhBw== X-Google-Smtp-Source: AGHT+IEmcPzpEkArJ3i9CJjQCMrpwlnKd4gwsJ20RpqoX5n8HHp8FbFgNc6cEQA+GU7jJN02DDcm X-Received: by 2002:a17:906:c342:b0:a3c:458d:a1a8 with SMTP id ci2-20020a170906c34200b00a3c458da1a8mr3115836ejb.12.1707850156247; Tue, 13 Feb 2024 10:49:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850156; cv=pass; d=google.com; s=arc-20160816; b=UfhDzp3oEVGIq/KHl7R5D9I46NDP20fDDsmHGl0cTv1R4H7+scTJRyjqxj77rZvohd +nhBsiyFyjOLjaa0gybm9Q0Y0LIyMibM9jBVfGxMx8S9A4I8wIb/p3hjaL2E31zTH5xq BPs1wXcvsy2fFkIbTQjnFYe0Z5HURV7S5uXSmR122JnXEjH0IfCHey9Ds/3bcqAC9Arl MHfP6eWzqsN8hkE8h18GyVFxTLuqtyotkOhpJjCWhP23suMhBj/cYAxduTod12GSpqxX ZMby9UQDEr4mwahkBAQPx12Fj9bKz8ZZ2Btn6qO2lBe8mhB6BFEdOPGmwma0ZUcv0h2M DArA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=aSRLvq58JaYagwB2EEEXkWyu1KJ52lodhrIKrgBv7M4=; fh=mEz4a413jnV6pYvXGhXxsLHa8xFVCuTeLi7nfnZvu58=; b=MgZAtGJZOykMXdzE85BPnfYLBTIxHaksmzRwbIZcyl1hpMs4hjsVU4N25SRKK7/1T/ oBrGhtKOBC6WMzqNnuv//uNEknyPYQskzASujOWZ1XAmNgO3EmddTOGM8FLQQw7wesAx R5TAA7OnUqMLTuCgMLrsHTXAjFYlF7vOMgYUUh8jZjONDi03ZArXG6z/7wA2ztW75aNn e0Kxjmv6RlHMTgqYyuQgZU7jdsuW2KDXT1dawe/3T1PyBGV4/xX/Ouqi9FKrrceWBNOQ /opeyLC4jkl5K3qrrLYSPjNCICLSu+YQGf+0qk8Jl2Ra/3i9/Wr3ybGLGkbVEov0A9lk NStg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64107-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64107-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXR5CX/39QhSMMYMo0zQK5d/Ie/TYLzStfQQopn16pydlWbHSGOGWhiSPPGBcgW02/LUnaNLin9nDFl3qAJ7DBdsuneqg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id v18-20020a170906339200b00a3cf5d9315asi1032461eja.482.2024.02.13.10.49.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:49:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64107-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64107-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64107-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id B0C7E1F24E8E for ; Tue, 13 Feb 2024 18:49:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9E67262162; Tue, 13 Feb 2024 18:45:59 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EA7FD61692 for ; Tue, 13 Feb 2024 18:45:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849957; cv=none; b=GEuOCL0JMwMkKif5VHW47F+hF0L/LK7EFlq/lHBDHavfW4KHAejxRPoFNaMj7exI0QLcR2OXhdoB5Fl8aIpQG47+AeJJTRKbdWEbVXftp9NaSd7eXl7+WvFAKFYjC/RSgHUNmvPV5oTmp73xANOnPqxCjsL/4MFF8ft2uJWCu8o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849957; c=relaxed/simple; bh=s5BgjTafmzNShrIDz3D0IMsmPwUKqJnbt2szR6I5DCo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=g+oGh4eZkCwPSqnAz1NZXwXWQDVBAO65eoRwRazGW23mjETOA2ezUDrCkyF1shmP3dktyQUxN6IMilTucQzPyykDilSbEwKeomsMbP46OIHI7AHBFXKRux2ifyVvDtW7aOLq/qPRBRN5qO2VvTk4C94tS+xFOR9S6eqHY1cquKA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A05BA1596; Tue, 13 Feb 2024 10:46:36 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 45C193F766; Tue, 13 Feb 2024 10:45:52 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 16/24] x86/resctrl: Make resctrl_mounted checks explicit Date: Tue, 13 Feb 2024 18:44:30 +0000 Message-Id: <20240213184438.16675-17-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810685228018642 X-GMAIL-MSGID: 1790810685228018642 The rdt_enable_key is switched when resctrl is mounted, and used to prevent a second mount of the filesystem. It also enables the architecture's context switch code. This requires another architecture to have the same set of static-keys, as resctrl depends on them too. The existing users of these static-keys are implicitly also checking if the filesystem is mounted. Make the resctrl_mounted checks explicit: resctrl can keep track of whether it has been mounted once. This doesn't need to be combined with whether the arch code is context switching the CLOSID. rdt_mon_enable_key is never used just to test that resctrl is mounted, but does also have this implication. Add a resctrl_mounted to all uses of rdt_mon_enable_key. This will allow the static-key changing to be moved behind resctrl_arch_ calls. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v3: * Removed a newline. * Rephrased commit message Changes since v4: * Rephrased comment. Changes since v7: * Removed a sentence with the word 'subsequent' in it. --- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 12 ++++++++++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 23 +++++++++++++++++------ 3 files changed, 28 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index e089d1a1a055..9bfda6963794 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -144,6 +144,7 @@ extern bool rdt_alloc_capable; extern bool rdt_mon_capable; extern unsigned int rdt_mon_features; extern struct list_head resctrl_schema_all; +extern bool resctrl_mounted; enum rdt_group_type { RDTCTRL_GROUP = 0, diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 9b503e6ac490..d5d8a58d96f2 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -813,7 +813,11 @@ void mbm_handle_overflow(struct work_struct *work) mutex_lock(&rdtgroup_mutex); - if (!static_branch_likely(&rdt_mon_enable_key)) + /* + * If the filesystem has been unmounted this work no longer needs to + * run. + */ + if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -846,7 +850,11 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - if (!static_branch_likely(&rdt_mon_enable_key)) + /* + * When a domain comes online there is no guarantee the filesystem is + * mounted. If not, there is no need to catch counter overflow. + */ + if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) return; cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->mbm_work_cpu = cpu; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index e42cbdf8f6a3..857fbbc3c839 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -42,6 +42,9 @@ LIST_HEAD(rdt_all_groups); /* list of entries for the schemata file */ LIST_HEAD(resctrl_schema_all); +/* The filesystem can only be mounted once. */ +bool resctrl_mounted; + /* Kernel fs node for "info" directory under root */ static struct kernfs_node *kn_info; @@ -881,7 +884,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, mutex_lock(&rdtgroup_mutex); /* Return empty if resctrl has not been mounted. */ - if (!static_branch_unlikely(&rdt_enable_key)) { + if (!resctrl_mounted) { seq_puts(s, "res:\nmon:\n"); goto unlock; } @@ -2608,7 +2611,7 @@ static int rdt_get_tree(struct fs_context *fc) /* * resctrl file system can only be mounted once. */ - if (static_branch_unlikely(&rdt_enable_key)) { + if (resctrl_mounted) { ret = -EBUSY; goto out; } @@ -2669,8 +2672,10 @@ static int rdt_get_tree(struct fs_context *fc) if (rdt_mon_capable) static_branch_enable_cpuslocked(&rdt_mon_enable_key); - if (rdt_alloc_capable || rdt_mon_capable) + if (rdt_alloc_capable || rdt_mon_capable) { static_branch_enable_cpuslocked(&rdt_enable_key); + resctrl_mounted = true; + } if (is_mbm_enabled()) { r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -2944,6 +2949,7 @@ static void rdt_kill_sb(struct super_block *sb) static_branch_disable_cpuslocked(&rdt_alloc_enable_key); static_branch_disable_cpuslocked(&rdt_mon_enable_key); static_branch_disable_cpuslocked(&rdt_enable_key); + resctrl_mounted = false; kernfs_kill_sb(sb); mutex_unlock(&rdtgroup_mutex); cpus_read_unlock(); @@ -3913,7 +3919,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) * If resctrl is mounted, remove all the * per domain monitor data directories. */ - if (static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) rmdir_mondata_subdir_allrdtgrp(r, d->id); if (is_mbm_enabled()) @@ -3990,8 +3996,13 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) if (is_llc_occupancy_enabled()) INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo); - /* If resctrl is mounted, add per domain monitor data directories. */ - if (static_branch_unlikely(&rdt_mon_enable_key)) + /* + * If the filesystem is not mounted then only the default resource group + * exists. Creation of its directories is deferred until mount time + * by rdt_get_tree() calling mkdir_mondata_all(). + * If resctrl is mounted, add per domain monitor data directories. + */ + if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) mkdir_mondata_subdir_allrdtgrp(r, d); return 0; From patchwork Tue Feb 13 18:44:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200559 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740135dyb; Tue, 13 Feb 2024 10:49:31 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW8JlOVucmG+XH2aJcH7+YesO6GwTlH7xk0pFORI0C94lGp8M3v2TLZOvTecfGswRDFg3zEMYKZa9T7U9XG2/zc0Dq3vw== X-Google-Smtp-Source: AGHT+IGjiMkfv/1cFNgNtNO1vMSw5VYXkmFsyH7+V35yfQM+ur7XWsQGnmvC3LCAp+hpLjeqt2JX X-Received: by 2002:a17:906:68c5:b0:a3d:2762:71f7 with SMTP id y5-20020a17090668c500b00a3d276271f7mr129478ejr.66.1707850171026; Tue, 13 Feb 2024 10:49:31 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850171; cv=pass; d=google.com; s=arc-20160816; b=BymBV39EQ+YDQUOEanJdEsiQcXRQYx/77PyKxNhfONGQSooGdsZj5KkZsGP5z5wDFf pwrptKqfJJ0C1QqGxDuTE6X3Hmvjm3oUPtG9oThPFfKZUOObEl5plaNNfM5vdtO44Hqz hqEAtQZGH7Me+arK0bcvFN273Oc05VP6JIdRZFAs2LZ7737cIi4WV3zLmSDKXqR2+y3e 4LidMr1CvezKfHSAVAUOXGAfOwYDP5nOqcP/skhM+b76Oa4Lk6Jiuk16WvACYw+jw3Ov f1JaWfSKHt5oZ0cWz7Hb3g47Ygxs+Fi0OZCskemy4wfjXqr4jHgaLchCxnX8xm7bl2CS GQ/A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=0xSLpzHEYb4dnRSqP8O6AV1o/h86rsyj77f2d4QllGo=; fh=wG4cjF+ek8oNyKmwd2VRUpXWDhBfri9YzY3svl5gssg=; b=tvEFU8402wyYWBQi1kHSEQISgBM1zb0wq3SjlNs3qfBpatwaw6H0GwQyoIAbKBrtnT /1wuohtRRAOtaMvvvzWrolYzhPSdARxBO9iph3H2XdIM15X954PM/1A39uGNQFltT8/2 7u2NnGXRphHUhQYpvjkybd8ViAQq5zHIVFhVJm8DUhu5hqGwfw1bGoaIdxGFTGyWTuyh hy0wINBJAUlurA6q/rNrS13s4xx5pM0za8RnEZC0c1/izM3lvMEuoWY001hnGDa70W48 t7UT4m8Tc5KqPSDlu4ktvcr6YUr5jcwyaa4t9lkzuBDgW31Qx2eMWYpihlrgtYiFPeJk C3GA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64108-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64108-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWMW0Ru4dK2wuPyloPVH9+WV3Lv7kRuNtIW4pv1srQscXHc0La+PgDKEecXqFqVz+1G8E1U5KeIcdsVCCRJaSTD2HBmlA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id s17-20020a170906455100b00a3d19189d73si466601ejq.810.2024.02.13.10.49.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:49:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64108-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64108-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64108-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 98B0B1F24D81 for ; Tue, 13 Feb 2024 18:49:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 12EBF605C6; Tue, 13 Feb 2024 18:46:03 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1BFF562157 for ; Tue, 13 Feb 2024 18:45:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849961; cv=none; b=i+MGT9QwshAXBC3H5eTVDaG4fDM0r6k4uzuJybiAs51gtDVV3ud8b9Z5gCsNGHsL4e/GM2vofHpPMavPb/eWG9za7aQNg7WPnkROPplGYZDXgaF2pPtOXjXSnSS/LHJG3MKtSwfRlabiHp9eLPa3PTcvRl7vmeWzna7k5NeIwGo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849961; c=relaxed/simple; bh=UTm+/uA3DbKSZsz9TADTmie/tQ3Sskf0znguZ3b19zQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=enBk/c3ynC8r9yEIN4DK+xAHJGZWpbZilAqo90CI3QxYUYSvHWZmUZtuFnukDSFk2JbL2P6jcmnVi4pIhsZCLxrRGvxaL/k4PzBn3/hXnRM7tx4ZzHd5hLR06jGai9bpeqchbPB7sPiWO/A57y6GOwTS1KOFTQYAoKxE58e3aLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 080201576; Tue, 13 Feb 2024 10:46:40 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A21E93F766; Tue, 13 Feb 2024 10:45:55 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 17/24] x86/resctrl: Move alloc/mon static keys into helpers Date: Tue, 13 Feb 2024 18:44:31 +0000 Message-Id: <20240213184438.16675-18-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810700828864938 X-GMAIL-MSGID: 1790810700828864938 resctrl enables three static keys depending on the features it has enabled. Another architecture's context switch code may look different, any static keys that control it should be buried behind helpers. Move the alloc/mon logic into arch-specific helpers as a preparatory step for making the rdt_enable_key's status something the arch code decides. This means other architectures don't have to mirror the static keys. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- arch/x86/include/asm/resctrl.h | 20 ++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++---- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 29c4cc343787..3c9137b6ad4f 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -42,6 +42,26 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); +static inline void resctrl_arch_enable_alloc(void) +{ + static_branch_enable_cpuslocked(&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_disable_alloc(void) +{ + static_branch_disable_cpuslocked(&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_enable_mon(void) +{ + static_branch_enable_cpuslocked(&rdt_mon_enable_key); +} + +static inline void resctrl_arch_disable_mon(void) +{ + static_branch_disable_cpuslocked(&rdt_mon_enable_key); +} + /* * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR * diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 9bfda6963794..78580855139d 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -94,9 +94,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc) return container_of(kfc, struct rdt_fs_context, kfc); } -DECLARE_STATIC_KEY_FALSE(rdt_enable_key); -DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); - /** * struct mon_evt - Entry in the event list of a resource * @evtid: event id @@ -452,8 +449,6 @@ extern struct mutex rdtgroup_mutex; extern struct rdt_hw_resource rdt_resources_all[]; extern struct rdtgroup rdtgroup_default; -DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); - extern struct dentry *debugfs_resctrl; enum resctrl_res_level { diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 857fbbc3c839..231207f09e04 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2668,9 +2668,9 @@ static int rdt_get_tree(struct fs_context *fc) goto out_psl; if (rdt_alloc_capable) - static_branch_enable_cpuslocked(&rdt_alloc_enable_key); + resctrl_arch_enable_alloc(); if (rdt_mon_capable) - static_branch_enable_cpuslocked(&rdt_mon_enable_key); + resctrl_arch_enable_mon(); if (rdt_alloc_capable || rdt_mon_capable) { static_branch_enable_cpuslocked(&rdt_enable_key); @@ -2946,8 +2946,8 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode = RDT_MODE_SHAREABLE; schemata_list_destroy(); rdtgroup_destroy_root(); - static_branch_disable_cpuslocked(&rdt_alloc_enable_key); - static_branch_disable_cpuslocked(&rdt_mon_enable_key); + resctrl_arch_disable_alloc(); + resctrl_arch_disable_mon(); static_branch_disable_cpuslocked(&rdt_enable_key); resctrl_mounted = false; kernfs_kill_sb(sb); From patchwork Tue Feb 13 18:44:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200576 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp754948dyb; Tue, 13 Feb 2024 11:14:28 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX3w1GbqPYOK+lm997u133v0eTqlrUbHLHFe+PW43j86p0iWXHIxzTCassIqDIbJfmAKWd/UclvUn+3gZKhO2n9/uEz6Q== X-Google-Smtp-Source: AGHT+IEHXd3QgiMHEYS83idbEWEZzuKnuUys7HLOmTnrzwaakLPV/gDXQydKboFkBmjCChbfFjVu X-Received: by 2002:a17:902:82ca:b0:1d9:63e2:680a with SMTP id u10-20020a17090282ca00b001d963e2680amr408620plz.24.1707851668480; Tue, 13 Feb 2024 11:14:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851668; cv=pass; d=google.com; s=arc-20160816; b=WreUhlgMkAIGN4xSmS5uOcr/YGt637d3oTExupS86uV3cYxYCerou3rSHP0HhoEIVy NABJC6oxTFMp7NYOGhlMgSJDYUudkPSnZDkeMhZmHbFx/u63nKBaUnq2516LwANr9+Du A/k/dP1AwUrMj6REa/tfFzW7k8ClrLjn//YHfeBlO8UdDxWyEJmGW9HqdTxCwIVFP0Ft xQVrI48VrIYdLFVSxXUAorTIuxqRkewLhzapT0DXrgaxDwQrl067bAIdcW7iZSb3O6l3 vXXaJaglTyBhxEUACXTnsRuVC26Tp9YfJiPyXYGugLGrRUEUOqwC0Gy7aH4xvOBat57K josA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=dyg1PQBXjzOurYsqs4MsUkQbyNLjRgUYb1TxLj6nCEU=; fh=TIs4PTuzSMCW1Sfje5KP6cWhkWVWXYmbfsx/8bwFJXw=; b=MIEJrqNuyge86EhldSAVuwd4p25IpS1yBazpUlbuPKdWHwtSMBMbcShZDmI1KtS4a5 D1Vl3dfPEQRVZZLNk9deVMPspP9vmXo+P63s+lhlCgJQzmegfYArgh5NmRadVewWuTWp yZxh3AmiLFWD5QYpB2IcDZoaNgCa0G/idzUHqNNutSNp/vg6Ysdm7X2r8qgy6XSNgeuo xLbXOuzrrO5hG4rm+43+CcqNV12vxUL3ZJRkwWrhzY7vofBeHuscTJtlqS5z1SqZDrbK Q3X3QD1WUb2mCsELsir2LFizMcMVb4U8ore0MG6RJv2LkdP07yvEkQPwnfqc+pC/+Aeq Oqeg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64109-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64109-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCVNFTKVk+CwXXAVuIjCIf1TnknWcEaB9Pr4bqDaWaHdQJOtyGN7VN2qKl1ipP5wStnfHH85FCuV0Yoig8gq4PGi2bOIjw== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id qj6-20020a17090b28c600b00297315d3f7bsi2529282pjb.26.2024.02.13.11.14.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:14:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64109-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64109-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64109-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 8F04FB26579 for ; Tue, 13 Feb 2024 18:49:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 989C36088D; Tue, 13 Feb 2024 18:46:05 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 94C955FB88 for ; Tue, 13 Feb 2024 18:46:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849964; cv=none; b=upvH4ghxjxXfH2VP4hUYdl9Pbw50ZGGuUGB6Px140XTw5LY5QC0AdNaCZNOZHLWwDRPMU+5OfNUToEQHoYqiQ1BGmzq2/GMrMAar30zqToMRWvdwVjuBG684UsnYloUcsHI653K2jnU/RqxaIrRQphd6YSduFYMRrM4tRD9+lHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849964; c=relaxed/simple; bh=GBgK+HKsOLOZBaUQG/gKw1CrHgLc7JCW+Za9mnYk4R8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=c3jZQS4xYpHp2JKT4mO5XPhsSGHWW57R1EF6vHQ5TdHCs0nvVSUALnsFdvEABr8orfgm5Upr51IUOzUwDJgsq2/bVJZQn1ubVDTlVtyoHwSmqEKPXoZl8CzsXpdn/y1VFUWgRAjKpuojrFL/9kLzIzG2HCnrjkCxqzd0jLBpt40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4C52B15A1; Tue, 13 Feb 2024 10:46:43 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E8A6D3F766; Tue, 13 Feb 2024 10:45:58 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 18/24] x86/resctrl: Make rdt_enable_key the arch's decision to switch Date: Tue, 13 Feb 2024 18:44:32 +0000 Message-Id: <20240213184438.16675-19-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790812270950718437 X-GMAIL-MSGID: 1790812270950718437 rdt_enable_key is switched when resctrl is mounted. It was also previously used to prevent a second mount of the filesystem. Any other architecture that wants to support resctrl has to provide identical static keys. Now that there are helpers for enabling and disabling the alloc/mon keys, resctrl doesn't need to switch this extra key, it can be done by the arch code. Use the static-key increment and decrement helpers, and change resctrl to ensure the calls are balanced. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- arch/x86/include/asm/resctrl.h | 4 ++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 11 +++++------ 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index 3c9137b6ad4f..b74aa34dc9e8 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -45,21 +45,25 @@ DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); static inline void resctrl_arch_enable_alloc(void) { static_branch_enable_cpuslocked(&rdt_alloc_enable_key); + static_branch_inc_cpuslocked(&rdt_enable_key); } static inline void resctrl_arch_disable_alloc(void) { static_branch_disable_cpuslocked(&rdt_alloc_enable_key); + static_branch_dec_cpuslocked(&rdt_enable_key); } static inline void resctrl_arch_enable_mon(void) { static_branch_enable_cpuslocked(&rdt_mon_enable_key); + static_branch_inc_cpuslocked(&rdt_enable_key); } static inline void resctrl_arch_disable_mon(void) { static_branch_disable_cpuslocked(&rdt_mon_enable_key); + static_branch_dec_cpuslocked(&rdt_enable_key); } /* diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 231207f09e04..7e57ac9d81f7 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2672,10 +2672,8 @@ static int rdt_get_tree(struct fs_context *fc) if (rdt_mon_capable) resctrl_arch_enable_mon(); - if (rdt_alloc_capable || rdt_mon_capable) { - static_branch_enable_cpuslocked(&rdt_enable_key); + if (rdt_alloc_capable || rdt_mon_capable) resctrl_mounted = true; - } if (is_mbm_enabled()) { r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -2946,9 +2944,10 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode = RDT_MODE_SHAREABLE; schemata_list_destroy(); rdtgroup_destroy_root(); - resctrl_arch_disable_alloc(); - resctrl_arch_disable_mon(); - static_branch_disable_cpuslocked(&rdt_enable_key); + if (rdt_alloc_capable) + resctrl_arch_disable_alloc(); + if (rdt_mon_capable) + resctrl_arch_disable_mon(); resctrl_mounted = false; kernfs_kill_sb(sb); mutex_unlock(&rdtgroup_mutex); From patchwork Tue Feb 13 18:44:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200577 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp755182dyb; Tue, 13 Feb 2024 11:14:54 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVhWWkxr0jPewupW/+aObUF71c3+/dZliP1n7g3xNnmwgiWRsmAPRMhZ29YztPO9hlclw6w3FoazAYnsOGBKK4sVO7zKA== X-Google-Smtp-Source: AGHT+IFFJ+Jx7Fn01cKUAqQlKcbJzcmqSsZIjvnbwgSBPfmLnKH297ubwJpkfO+75to9nj4snvFQ X-Received: by 2002:a05:6358:2c8e:b0:179:1f8b:445a with SMTP id l14-20020a0563582c8e00b001791f8b445amr239844rwm.22.1707851693840; Tue, 13 Feb 2024 11:14:53 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707851693; cv=pass; d=google.com; s=arc-20160816; b=PVOg7VQuHOFJYdIZDyVdUfXn5c+gTQLOAT5v8Ke/sIyOsrX36icZ3y5GHd3lIA68d8 R4jVx3dhtevTpSMhHISFelECN8OLuLAngL5VI1pTSeGTaacW3Oh9IDEGdYxhVmUxQfCx LsjZs2lSF+T4cULyCjgP4CrlW7V9PJgNZGIZU6YSf0nkoUjus6yeagbt/wAfYjhgfNcC YiqJ55aMwOA6mWleGF8q/D5C4Pu3kc0dBatLQOx+mYiyBpk2AbhB7E370i/PGonH3v1w KT1EMbwPH1crFjbMpYmmQkcFyTIz9z6pYEkthbGxvvDdID/NzHue7Ou2l+gvSgaAbwZV DHcg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=iPtQ9Q4hFdpafvMfmWtCfKzooXUbncU5WsfaS9PfDQI=; fh=KVkeWiS8viO4V9z1gZqU0zTTjCFtdLuJT7PARKScAAE=; b=SMavo8kvRUt1lftqzIJ950i1STJbHRaD3O+sXZcWGElVZ8i3L491yu8JgknEI483Yn HFrscLCkfUlbv4J63/UB2SnszZ3tqesWjY10uXJHgYG6VOEspiqKiIPSZspvG0m92kXG dBX0T5UTGhiFPVktfnQuL4iDtiGutchTnu6rlFMPBZuDQkpbK9Ig+NifVXaepmpNNlf5 tDeBKBzbjlynMpTWbXgTKQ/QpN5fD5RHJJMyIS3ZpbcLCETZWK4ajtKj3uU78pQM0bWX hor9BrZUUryP89CNoxsl5mLrxVEP3jg9Dv99TkiceStdSVwp6UC4T4nVuXR083fgEV0N aN3A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64110-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64110-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXrAbHcAxz0gA51R2i9H4s2Anf0gRPzZ/pYDEPKlO16mTTMV4NaUMp5nkJ6+UH8hZgg3nfMuSkFzFShJF3Pi5plPizH2A== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id bx41-20020a056a02052900b005dab535faa8si2580736pgb.140.2024.02.13.11.14.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 11:14:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64110-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64110-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64110-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 02EDCB28AB3 for ; Tue, 13 Feb 2024 18:50:03 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9A5CD61680; Tue, 13 Feb 2024 18:46:10 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 02D0F60893 for ; Tue, 13 Feb 2024 18:46:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849967; cv=none; b=WIAtz2Owcz8R3IIvpwvaV3mPNoDGbEH3u/nJWesVpgUVT5Y/J9h2sQEKa809eUUSb3wQ9S9cRpuQ50NMeqr37CcwfqQgLPAc4dtRUqKSIKnGYjiHJRc0TR1VSsXZPh6IxjP9d6fs5/apbiRdA02psEuvfzdodlttdqIznThPtZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849967; c=relaxed/simple; bh=c1TtCoEKYnIksbt49IoR7GOnaTPAQOZuCem4lYk23X4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Trd0/pB98BejqHUemFGoLHCHHsoj3JAmE3fXid+Vm2E7PsqBbTDYEnxYcpP8VQ9eNIoJKuf45LUt6R7FXSkAxPU3BSLE1juUf1lquHKJ1wOw62iuBlfNPUZ+IFIa5r6hzX0J7YNfCsVEGu6OCg9c63oglJmDn//X0QWciO1LvPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B04711595; Tue, 13 Feb 2024 10:46:46 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 45D503F766; Tue, 13 Feb 2024 10:46:02 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 19/24] x86/resctrl: Add helpers for system wide mon/alloc capable Date: Tue, 13 Feb 2024 18:44:33 +0000 Message-Id: <20240213184438.16675-20-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790812297727269948 X-GMAIL-MSGID: 1790812297727269948 resctrl reads rdt_alloc_capable or rdt_mon_capable to determine whether any of the resources support the corresponding features. resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing. This forces another architecture to have the same static-keys. As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key. After this change, only the architecture code manages and uses the static-keys to ensure __resctrl_sched_in() does not need runtime checks. This avoids multiple architectures having to define the same static-keys. Cases where the static-key implicitly tested if the resctrl filesystem was mounted all have an explicit check added by a previous patch. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v1: * Added missing conversion in mkdir_rdt_prepare_rmid_free() Changes since v3: * Expanded the commit message. Change since v7: * Added a few missing resctrl_arch_mon_capable() that crept in during a rebase. --- arch/x86/include/asm/resctrl.h | 13 ++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 38 +++++++++++------------ 5 files changed, 37 insertions(+), 26 deletions(-) diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h index b74aa34dc9e8..12dbd2588ca7 100644 --- a/arch/x86/include/asm/resctrl.h +++ b/arch/x86/include/asm/resctrl.h @@ -38,10 +38,18 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); +extern bool rdt_alloc_capable; +extern bool rdt_mon_capable; + DECLARE_STATIC_KEY_FALSE(rdt_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key); DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key); +static inline bool resctrl_arch_alloc_capable(void) +{ + return rdt_alloc_capable; +} + static inline void resctrl_arch_enable_alloc(void) { static_branch_enable_cpuslocked(&rdt_alloc_enable_key); @@ -54,6 +62,11 @@ static inline void resctrl_arch_disable_alloc(void) static_branch_dec_cpuslocked(&rdt_enable_key); } +static inline bool resctrl_arch_mon_capable(void) +{ + return rdt_mon_capable; +} + static inline void resctrl_arch_enable_mon(void) { static_branch_enable_cpuslocked(&rdt_mon_enable_key); diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 78580855139d..3ee855c37447 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -137,8 +137,6 @@ struct rmid_read { void *arch_mon_ctx; }; -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable; extern unsigned int rdt_mon_features; extern struct list_head resctrl_schema_all; extern bool resctrl_mounted; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index d5d8a58d96f2..92d7ba674003 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -817,7 +817,7 @@ void mbm_handle_overflow(struct work_struct *work) * If the filesystem has been unmounted this work no longer needs to * run. */ - if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) + if (!resctrl_mounted || !resctrl_arch_mon_capable()) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; @@ -854,7 +854,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) * When a domain comes online there is no guarantee the filesystem is * mounted. If not, there is no need to catch counter overflow. */ - if (!resctrl_mounted || !static_branch_likely(&rdt_mon_enable_key)) + if (!resctrl_mounted || !resctrl_arch_mon_capable()) return; cpu = cpumask_any_housekeeping(&dom->cpu_mask); dom->mbm_work_cpu = cpu; diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index d8f44113ed1f..8056bed033cc 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -581,7 +581,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp) if (ret) goto err_cpus; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = rdtgroup_kn_mode_restrict(rdtgrp, "mon_groups"); if (ret) goto err_cpus_list; @@ -628,7 +628,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp) if (ret) goto err_cpus; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = rdtgroup_kn_mode_restore(rdtgrp, "mon_groups", 0777); if (ret) goto err_cpus_list; @@ -776,7 +776,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) { int ret; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = alloc_rmid(rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 7e57ac9d81f7..ed5fc677a99d 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -641,13 +641,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk, static bool is_closid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_alloc_capable && (r->type == RDTCTRL_GROUP) && + return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) && resctrl_arch_match_closid(t, r->closid)); } static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_mon_capable && (r->type == RDTMON_GROUP) && + return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) && resctrl_arch_match_rmid(t, r->mon.parent->closid, r->mon.rmid)); } @@ -2632,7 +2632,7 @@ static int rdt_get_tree(struct fs_context *fc) closid_init(); - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) flags |= RFTYPE_MON; ret = rdtgroup_add_files(rdtgroup_default.kn, flags); @@ -2645,7 +2645,7 @@ static int rdt_get_tree(struct fs_context *fc) if (ret < 0) goto out_schemata_free; - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret = mongroup_create_dir(rdtgroup_default.kn, &rdtgroup_default, "mon_groups", &kn_mongrp); @@ -2667,12 +2667,12 @@ static int rdt_get_tree(struct fs_context *fc) if (ret < 0) goto out_psl; - if (rdt_alloc_capable) + if (resctrl_arch_alloc_capable()) resctrl_arch_enable_alloc(); - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) resctrl_arch_enable_mon(); - if (rdt_alloc_capable || rdt_mon_capable) + if (resctrl_arch_alloc_capable() || resctrl_arch_mon_capable()) resctrl_mounted = true; if (is_mbm_enabled()) { @@ -2686,10 +2686,10 @@ static int rdt_get_tree(struct fs_context *fc) out_psl: rdt_pseudo_lock_release(); out_mondata: - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) kernfs_remove(kn_mondata); out_mongrp: - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) kernfs_remove(kn_mongrp); out_info: kernfs_remove(kn_info); @@ -2944,9 +2944,9 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode = RDT_MODE_SHAREABLE; schemata_list_destroy(); rdtgroup_destroy_root(); - if (rdt_alloc_capable) + if (resctrl_arch_alloc_capable()) resctrl_arch_disable_alloc(); - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) resctrl_arch_disable_mon(); resctrl_mounted = false; kernfs_kill_sb(sb); @@ -3326,7 +3326,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) { int ret; - if (!rdt_mon_capable) + if (!resctrl_arch_mon_capable()) return 0; ret = alloc_rmid(rdtgrp->closid); @@ -3348,7 +3348,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp) { - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) free_rmid(rgrp->closid, rgrp->mon.rmid); } @@ -3412,7 +3412,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, if (rtype == RDTCTRL_GROUP) { files = RFTYPE_BASE | RFTYPE_CTRL; - if (rdt_mon_capable) + if (resctrl_arch_mon_capable()) files |= RFTYPE_MON; } else { files = RFTYPE_BASE | RFTYPE_MON; @@ -3521,7 +3521,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, list_add(&rdtgrp->rdtgroup_list, &rdt_all_groups); - if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { /* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor. @@ -3576,14 +3576,14 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name, * allocation is supported, add a control and monitoring * subdirectory */ - if (rdt_alloc_capable && parent_kn == rdtgroup_default.kn) + if (resctrl_arch_alloc_capable() && parent_kn == rdtgroup_default.kn) return rdtgroup_mkdir_ctrl_mon(parent_kn, name, mode); /* * If RDT monitoring is supported and the parent directory is a valid * "mon_groups" directory, add a monitoring subdirectory. */ - if (rdt_mon_capable && is_mon_groups(parent_kn, name)) + if (resctrl_arch_mon_capable() && is_mon_groups(parent_kn, name)) return rdtgroup_mkdir_mon(parent_kn, name, mode); return -EPERM; @@ -3918,7 +3918,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) * If resctrl is mounted, remove all the * per domain monitor data directories. */ - if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && resctrl_arch_mon_capable()) rmdir_mondata_subdir_allrdtgrp(r, d->id); if (is_mbm_enabled()) @@ -4001,7 +4001,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) * by rdt_get_tree() calling mkdir_mondata_all(). * If resctrl is mounted, add per domain monitor data directories. */ - if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && resctrl_arch_mon_capable()) mkdir_mondata_subdir_allrdtgrp(r, d); return 0; From patchwork Tue Feb 13 18:44:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200560 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740468dyb; Tue, 13 Feb 2024 10:50:15 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWNT48Zxl7NQjtEcGJqbCq1XtPjZd/90h5GIbpxVu7cwbNPVh0IueSAffQxz7GAmc85qLDA980fkwkO0dmnPyxj88cF4Q== X-Google-Smtp-Source: AGHT+IHuNGeg3q9nJuzLo4ZhTlxi6QshaDa7ZQMXkIPclXezuysAAHeKnWW8OJx8jbbKXhDEZpOo X-Received: by 2002:a17:903:2284:b0:1d9:6063:89ce with SMTP id b4-20020a170903228400b001d9606389cemr4380372plh.31.1707850215241; Tue, 13 Feb 2024 10:50:15 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850215; cv=pass; d=google.com; s=arc-20160816; b=YFB/ZasCMvE2qjPrsh0kuu7LaujX8VtqW5G4VekvTFkzfQbq60pEWi/NHouEO3SaV3 Htdcmh8tOwrir0CXJQHfzJSvcfrpQeUbFe1eBep6tEMF9fEDKYcTeX9X2gnzc6N4i02F t4MxVE300GfWVv1UAKN5mK/Xiekxjc5dkGFKnSMLZuX4bSfqf1Y2RehhxRrr6DDh7/Uf aQp2dyTONj7dgHPgB+pJxJEbWLvx4W/a+0KrP2QFc++aPKRV4mRkfsJR4txcJuMM78Zo h7aU5cyNOUjlJ2fSsMfZhOGi3JNukxdtfRlqg+Oz7OiqgyfOZd3lWGcJBjHqjPMGVEFX TIOw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=F90YUNjz9EenpVEYCt+7SQWg4jTdf2x1VJplA11wTRw=; fh=dax1g7XTFfJLj00JREuJ7zWiJmQ7UDIdZSL00KcC16s=; b=Cv7PTbUuq54aUyEjYQzTYcMj03gg/jUi9+Jacwk1zK4/3omEY7LpvvBa8jnHfaogsD /InYDoNTE0Bm3pbszLYfBZc6h9DZLcKbjQqgm0OlyS6KX/lRDYFjAozBZ7s92L5NDZtO qsmtPX6/TKuTCuqlr3hs4SSTECmaNZYharCygOpYI6DgIoxSITWQdhaRBbD/mwNKgMBY LfjKNzdu2N/RUlVL3utVfzRY+R7rsERBlHuE/Wu2KfPHVTPJI7q8N27H4GOxphyeyTLT TiLtao5VoBrUcUlVFP7zDR5j76KF/8lQjEf7o2r/xVRLQOHNmt2JnXT8K++af0qIFnxi pxbA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64111-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64111-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCURIErx8unQrpvaWm+uQ3rgQUko8BTP4gPwAWHSXyHwM9i5HI1FNo+9LOrmN/9CFhjoRlsnCg8BvnukZr3oHzcbMEQiiw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id w26-20020a63af1a000000b005dc488101casi2313156pge.341.2024.02.13.10.50.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:50:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64111-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64111-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64111-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id D8AC7290624 for ; Tue, 13 Feb 2024 18:50:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 69CD0626A4; Tue, 13 Feb 2024 18:46:12 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 557E360B8B for ; Tue, 13 Feb 2024 18:46:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849970; cv=none; b=SvsU3LtYZHGs6xmMZmKEyq9zplT1JPClYKDe5K1YsaQNjm3kZyh4e7hOZWzD7/yRxLAFeeyK20enlfJs5x67ri4NpyTAFB3XXEOJgpdHEn3NvZpNR145VZizsslozwxK04ujut1qE6rM37Zag9B2HXj76EHgI5Ohhiy78FsKMUw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849970; c=relaxed/simple; bh=LMKCBuYvFmeGar0ZaGdEj75Qzv9aXDJQEL9P02vZS08=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AxvXy0EQh70BNGYz9SCaLsJwUhW5JOAX6JlzMBK1aQibkUVmzyh9zIc2Y8E/kbnbBxfRPkjo54zBTFqsyiUg22LKmZbhxQr+IZz1GkZPZAtKI4wvWTKms09ewGhkJTjTGQB6Yi5bZjuqVYDZ4rYjwCWVp1xm2lmBTbTjQev9WP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 18B781576; Tue, 13 Feb 2024 10:46:50 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B41CC3F766; Tue, 13 Feb 2024 10:46:05 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 20/24] x86/resctrl: Add CPU online callback for resctrl work Date: Tue, 13 Feb 2024 18:44:34 +0000 Message-Id: <20240213184438.16675-21-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810747007900117 X-GMAIL-MSGID: 1790810747007900117 The resctrl architecture specific code may need to create a domain when a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register. The resctrl filesystem code needs to update the rdtgroup_default CPU mask when CPUs are brought online. Currently this is all done in one function, resctrl_online_cpu(). This will need to be split into architecture and filesystem parts before resctrl can be moved to /fs/. Pull the rdtgroup_default update work out as a filesystem specific cpu_online helper. resctrl_online_cpu() is the obvious name for this, which means the version in core.c needs renaming. resctrl_online_cpu() is called by the arch code once it has done the work to add the new CPU to any domains. In future patches, resctrl_online_cpu() will take the rdtgroup_mutex itself. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v3: * Renamed err to ret Changes since v4: * Changes in capitalisation. Changes since v5: * More changes in capitalisation. * Made resctrl_online_cpu() return void. --- arch/x86/kernel/cpu/resctrl/core.c | 8 ++++---- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++++++ include/linux/resctrl.h | 1 + 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index d1dc80a21ea9..4627d447bc3d 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -606,16 +606,16 @@ static void clear_closid_rmid(int cpu) RESCTRL_RESERVED_CLOSID); } -static int resctrl_online_cpu(unsigned int cpu) +static int resctrl_arch_online_cpu(unsigned int cpu) { struct rdt_resource *r; mutex_lock(&rdtgroup_mutex); for_each_capable_rdt_resource(r) domain_add_cpu(cpu, r); - /* The cpu is set in default rdtgroup after online. */ - cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); clear_closid_rmid(cpu); + + resctrl_online_cpu(cpu); mutex_unlock(&rdtgroup_mutex); return 0; @@ -967,7 +967,7 @@ static int __init resctrl_late_init(void) state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/resctrl/cat:online:", - resctrl_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, resctrl_offline_cpu); if (state < 0) return state; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index ed5fc677a99d..38d3b19a3aca 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4007,6 +4007,14 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) return 0; } +void resctrl_online_cpu(unsigned int cpu) +{ + lockdep_assert_held(&rdtgroup_mutex); + + /* The CPU is set in default rdtgroup after online. */ + cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); +} + /* * rdtgroup_init - rdtgroup initialization * diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index bf460c912bf5..4c4bad3c34e4 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -223,6 +223,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, u32 closid, enum resctrl_conf_type type); int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d); void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); +void resctrl_online_cpu(unsigned int cpu); /** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid From patchwork Tue Feb 13 18:44:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200562 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740816dyb; Tue, 13 Feb 2024 10:51:00 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUw/lrvwLsiZTG9GFgDKu6tIbU0OLFiWR1J7qgD/0yBczCVBTYHqRb8dcrSWBuomvv15aBeLqU26vdonww7BIfTyRCFww== X-Google-Smtp-Source: AGHT+IHnrEK4ICdqejFa1P/DYFgSEh3vCA769JEL05fxv4FDMMWlrg/dl1PRw7H6SMAGQZ26dfk6 X-Received: by 2002:a05:6214:c49:b0:68c:96dd:7383 with SMTP id r9-20020a0562140c4900b0068c96dd7383mr279365qvj.38.1707850259853; Tue, 13 Feb 2024 10:50:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850259; cv=pass; d=google.com; s=arc-20160816; b=DVQVHFNEhR29exIM6YWSLRXx7g4Uz+HHlZWh00NHVydOIDiyKNt6SNDC/4fSAG53sv wjWqkNzzbjXmzotHWJKiPWvjkypt09fcnhMBpzd45cMMCw++UqYJ7eti37jJh4QryWsB Sipp0GzPo2dIeopqBuYOoKvKkcH+mVlavf1q6ts2i5HnT4bQ593u00TkvLMVLPQ+d1I7 0J4FoNNu/aloKGgTmmcYtQ6r/vyN8HgxZ2L9gxComQEEvnbhuhP3dkRiv0cjVuzUGyVz jvMPs3Fs0+RpA4/21mHG5eOApw/es5GEWV/y7Rj/AIRIanlOvke7ctMYz0qM4cuS6OQD BAcA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=DxNaBv0cDnqaWD80+uIXOzZoFNTx5oUxwESSpKVQwWc=; fh=dZjzQueZAIW9U0B48Hu1+8VpLj9A1ap5z0qzbSubL9k=; b=gjp3s/B1GaUnYsAQZhzuPsS+Tz18cnPem9nzrbnkNYljxAKTWn4ITx2ckEHL1lDMFV 6qDWgZhhX4NuAcPpnooAJImn6AWaLKd6k8jFKpp2YLS3W8w5sy7RTK68k7Pwrfqw5GE9 3ARtDThoJMPxwBwVLKa/RCFQlqTuNSzAEonB/rJKeYroZrCsxY+v01w6IGTOvhIt8FBQ gF6xxGqIO+OwwJPHGC4/NUOvdV7dWlPzxjMiXPY5CWxZ5LvqiiaeEQx8BV4pbALMDXkW CX1vwLjTZGheQBfWWQkU4t49k8Q+hxW6pkzwInsDkxn2cuCfb5Zh+jk5Qeru+INYKqy1 G7nw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64112-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64112-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWdlzxHOloIR1tFORFjZX1TQ+0jSP/kG8IBEiLytaiQeqerhDvM/6lZJHoRXuVoHHwqvIsWHlLuJ69cUm3nj0PKtvkV/w== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id o11-20020a0562140e4b00b0068cb0ec2748si3281115qvc.581.2024.02.13.10.50.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:50:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64112-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64112-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64112-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 1042A1C264F0 for ; Tue, 13 Feb 2024 18:50:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 44176626DF; Tue, 13 Feb 2024 18:46:17 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A0AE1626A9 for ; Tue, 13 Feb 2024 18:46:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849974; cv=none; b=D9a7NZYqKU+uCbzx35KFhxh9M8OCQbZhNpSYJIknPgRb5nZ/hhZd+60F8Qc4EF9vYUWuZbzgjbmhiRp/jCSZMMDNhv4KqlQgz+/+l/OhhHYecWYvFMIvuB2IOItc/KP8rbx+vXG849K0WEJ4YhBCqtPjHAc6CNrT2uUjKtm3RQU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849974; c=relaxed/simple; bh=ZuU+KSA4ulzLmwkUHi/QZtXEigOgOSRtK0hCV6hjL3Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=d12AItPXRe/rkt68E78jR2LSWmGT/JfZocOBx+r9od+VJfqL/QW7wJPxPBF2o9Vvt7lpmuUkaSOA6kIQ+NGdaXkdGkftAYuYAFRSlH84/sMV5dbjwe4LliMmqsYpKICIJxKbIt4VEgBaXB/CN2C/37Uva3il37XVLmoEZWRHmdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6B7D01596; Tue, 13 Feb 2024 10:46:53 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 120403F766; Tue, 13 Feb 2024 10:46:08 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu Date: Tue, 13 Feb 2024 18:44:35 +0000 Message-Id: <20240213184438.16675-22-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810793878933196 X-GMAIL-MSGID: 1790810793878933196 When a CPU is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU. Once the offline callbacks have been split, cqm_setup_limbo_handler() will be called while the CPU that is going offline is still present in the cpu_mask. Pass the CPU to exclude to cqm_setup_limbo_handler() and mbm_setup_overflow_handler(). These functions can use a variant of cpumask_any_but() when selecting the CPU. -1 is used to indicate no CPUs need excluding. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Babu Moger Reviewed-by: Reinette Chatre --- Changes since v2: * Rephrased a comment to avoid a two letter bad-word. (we) * Avoid assigning mbm_work_cpu if the domain is going to be free()d * Added cpumask_any_housekeeping_but(), I dislike the name Changes since v3: * Marked an explanatory comment as temporary as the subsequent patch is no longer adjacent. Changes since v4: * Check against RESCTRL_PICK_ANY_CPU instead of -1. * Leave cqm_work_cpu as nr_cpu_ids when no CPU is available. * Made cpumask_any_housekeeping_but() more readable. Changes since v5: * Changes in captialisation, and a typo. * Merged cpumask helpers. Changes since v6: * Added the missing dom parameter to some kernel doc. * Re-added use of cpumask_any_but(), * Expanded comment above cpumask_any_housekeeping(), * Added some more comments for good measure. * Added explicit IS_ENABLED() check as gcc-12 doesn't seem to work this out. Changes since v7: * Removed the paragraph with the badword 'subsequent'. * Reworded a comment with the badword 'we'. * Fixed a bounds checking bug in cpumask_any_housekeeping(). --- arch/x86/kernel/cpu/resctrl/core.c | 8 +++-- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 33 ++++++++++++++---- arch/x86/kernel/cpu/resctrl/monitor.c | 42 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 6 ++-- include/linux/resctrl.h | 2 ++ 6 files changed, 72 insertions(+), 21 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 4627d447bc3d..55322ba629da 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -584,12 +584,16 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) if (r == &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl) { if (is_mbm_enabled() && cpu == d->mbm_work_cpu) { cancel_delayed_work(&d->mbm_over); - mbm_setup_overflow_handler(d, 0); + /* + * temporary: exclude_cpu=-1 as this CPU has already + * been removed by cpumask_clear_cpu()d + */ + mbm_setup_overflow_handler(d, 0, RESCTRL_PICK_ANY_CPU); } if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && has_busy_rmid(d)) { cancel_delayed_work(&d->cqm_limbo); - cqm_setup_limbo_handler(d, 0); + cqm_setup_limbo_handler(d, 0, RESCTRL_PICK_ANY_CPU); } } } diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index a033e8e32108..64db51455df3 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -552,7 +552,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, return; } - cpu = cpumask_any_housekeeping(&d->cpu_mask); + cpu = cpumask_any_housekeeping(&d->cpu_mask, RESCTRL_PICK_ANY_CPU); /* * cpumask_any_housekeeping() prefers housekeeping CPUs, but diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index 3ee855c37447..c99f26ebe7a6 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -60,19 +60,36 @@ * cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. + * @exclude_cpu:The CPU to avoid picking. * - * Returns a CPU in @mask. If there are housekeeping CPUs that don't use - * nohz_full, these are preferred. + * Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping + * CPUs that don't use nohz_full, these are preferred. Pass + * RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. + * + * When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping(const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping(const struct cpumask *mask, int exclude_cpu) { unsigned int cpu, hk_cpu; - cpu = cpumask_any(mask); - if (!tick_nohz_full_cpu(cpu)) + if (exclude_cpu == RESCTRL_PICK_ANY_CPU) + cpu = cpumask_any(mask); + else + cpu = cpumask_any_but(mask, exclude_cpu); + + if (!IS_ENABLED(CONFIG_NO_HZ_FULL)) + return cpu; + + /* If the CPU picked isn't marked nohz_full nothing more needs doing. */ + if (cpu < nr_cpu_ids && !tick_nohz_full_cpu(cpu)) return cpu; + /* Try to find a CPU that isn't nohz_full to use in preference */ hk_cpu = cpumask_nth_andnot(0, mask, tick_nohz_full_mask); + if (hk_cpu == exclude_cpu) + hk_cpu = cpumask_nth_andnot(1, mask, tick_nohz_full_mask); + if (hk_cpu < nr_cpu_ids) cpu = hk_cpu; @@ -573,11 +590,13 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first); void mbm_setup_overflow_handler(struct rdt_domain *dom, - unsigned long delay_ms); + unsigned long delay_ms, + int exclude_cpu); void mbm_handle_overflow(struct work_struct *work); void __init intel_rdt_mbm_apply_quirk(void); bool is_mba_sc(struct rdt_resource *r); -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms); +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms, + int exclude_cpu); void cqm_handle_limbo(struct work_struct *work); bool has_busy_rmid(struct rdt_domain *d); void __check_limbo(struct rdt_domain *d, bool force_free); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 92d7ba674003..67edd4c440f0 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -481,7 +481,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) * setup up the limbo worker. */ if (!has_busy_rmid(d)) - cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL); + cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL, + RESCTRL_PICK_ANY_CPU); set_bit(idx, d->rmid_busy_llc); entry->busy++; } @@ -784,7 +785,8 @@ void cqm_handle_limbo(struct work_struct *work) __check_limbo(d, false); if (has_busy_rmid(d)) { - d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask); + d->cqm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask, + RESCTRL_PICK_ANY_CPU); schedule_delayed_work_on(d->cqm_work_cpu, &d->cqm_limbo, delay); } @@ -792,15 +794,25 @@ void cqm_handle_limbo(struct work_struct *work) mutex_unlock(&rdtgroup_mutex); } -void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms) +/** + * cqm_setup_limbo_handler() - Schedule the limbo handler to run for this + * domain. + * @dom: The domain the limbo handler should run for. + * @delay_ms: How far in the future the handler should run. + * @exclude_cpu: Which CPU the handler should not run on, + * RESCTRL_PICK_ANY_CPU to pick any CPU. + */ +void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms, + int exclude_cpu) { unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; - cpu = cpumask_any_housekeeping(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu); dom->cqm_work_cpu = cpu; - schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); + if (cpu < nr_cpu_ids) + schedule_delayed_work_on(cpu, &dom->cqm_limbo, delay); } void mbm_handle_overflow(struct work_struct *work) @@ -838,14 +850,24 @@ void mbm_handle_overflow(struct work_struct *work) * Re-check for housekeeping CPUs. This allows the overflow handler to * move off a nohz_full CPU quickly. */ - d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask); + d->mbm_work_cpu = cpumask_any_housekeeping(&d->cpu_mask, + RESCTRL_PICK_ANY_CPU); schedule_delayed_work_on(d->mbm_work_cpu, &d->mbm_over, delay); out_unlock: mutex_unlock(&rdtgroup_mutex); } -void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) +/** + * mbm_setup_overflow_handler() - Schedule the overflow handler to run for this + * domain. + * @dom: The domain the overflow handler should run for. + * @delay_ms: How far in the future the handler should run. + * @exclude_cpu: Which CPU the handler should not run on, + * RESCTRL_PICK_ANY_CPU to pick any CPU. + */ +void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms, + int exclude_cpu) { unsigned long delay = msecs_to_jiffies(delay_ms); int cpu; @@ -856,9 +878,11 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) */ if (!resctrl_mounted || !resctrl_arch_mon_capable()) return; - cpu = cpumask_any_housekeeping(&dom->cpu_mask); + cpu = cpumask_any_housekeeping(&dom->cpu_mask, exclude_cpu); dom->mbm_work_cpu = cpu; - schedule_delayed_work_on(cpu, &dom->mbm_over, delay); + + if (cpu < nr_cpu_ids) + schedule_delayed_work_on(cpu, &dom->mbm_over, delay); } static int dom_data_init(struct rdt_resource *r) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 38d3b19a3aca..f5688c79d94f 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -2678,7 +2678,8 @@ static int rdt_get_tree(struct fs_context *fc) if (is_mbm_enabled()) { r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; list_for_each_entry(dom, &r->domains, list) - mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL); + mbm_setup_overflow_handler(dom, MBM_OVERFLOW_INTERVAL, + RESCTRL_PICK_ANY_CPU); } goto out; @@ -3989,7 +3990,8 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) if (is_mbm_enabled()) { INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow); - mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL); + mbm_setup_overflow_handler(d, MBM_OVERFLOW_INTERVAL, + RESCTRL_PICK_ANY_CPU); } if (is_llc_occupancy_enabled()) diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 4c4bad3c34e4..ccbbbe5d18d3 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -10,6 +10,8 @@ #define RESCTRL_RESERVED_CLOSID 0 #define RESCTRL_RESERVED_RMID 0 +#define RESCTRL_PICK_ANY_CPU -1 + #ifdef CONFIG_PROC_CPU_RESCTRL int proc_resctrl_show(struct seq_file *m, From patchwork Tue Feb 13 18:44:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200561 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740696dyb; Tue, 13 Feb 2024 10:50:42 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUrS1rEhflSk6j4CEDv3k7jFRYCOI8LARg+FeUIn+yfs2aV2wup0U6jnAY2ISDdYMIKujYHy63BDcJEWF+Ko/9I/pReNg== X-Google-Smtp-Source: AGHT+IEfAJ8W+IYPzNSoKVxVsF3B/fwm3dOuyCv2hgvG9Q9NEGDUWIqtwdkBnvy2slVOA8QWa9yv X-Received: by 2002:a05:6830:1b63:b0:6df:b685:84b9 with SMTP id d3-20020a0568301b6300b006dfb68584b9mr505878ote.32.1707850242479; Tue, 13 Feb 2024 10:50:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850242; cv=pass; d=google.com; s=arc-20160816; b=ODZ1nzeXn9buRHk3tIc9dP3+GoNfkgfZXltzInwctRaASQhXZWgIJmmX6jc1jQ6Tfe GgE3gjdHVH7dM01WiPVUE05MaGhGmMagxfZYGbHXKOykI1VU6ayK/gHEjzs1CDaEZ/Nl dJ91EqsrGLbdWLDmiiHXYT9iP+W96E1nkqUO3dR1mZ8dQziBy2ElJuXILs9TXXWGNMGz 7qWqLHmmKzFKKkNlAM+i152ysSII7skw5YZ5Np/9NCePlYx/wGPqG4QxBWX0RpYTIGBG AJQO+kMnAbO1khsw2yhvaLkLWdu8DIr1muvIs7jTW/4nI6UBhoLr0izIOQ/GL7pmVMyD cZQA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=Bjp/kM0pwgfnsnvfEvpgsjKtodmxi49LEm2hNwZluXY=; fh=TNr15Cc4ZCIZJVPrLskO4e2yEOS9MSgT4tyALxMaFzI=; b=F8lTuhOs3MXMnVnk9MhYTQKaw6DX8P7fmqXTy7Z32c1RicnIL+sUuVh+QjC4cvqH8s jzzM5nwyJRf6UQnSqQo6ct5KREYI8OODbACMy+tcXv1rB40WFuvHlIISW7vHGDCyra/w u0MfQrrFm573Lnf189BGICZMBe+CFf2+DsWJ3kXwOl3vqv15KPRHYMzXPNYKnHVV+H3c hl4+ODy95nwzXG5QdlJp0XV63h4CI7ThN4C24vLmhYfH54K6KG1e7EuVDPdy9+Qi8Uak jLKZoVgsxNS26W/MYmlxHJ7HZAzV6Knjqc0BTb2MidOwTCzZnE+YkL2XBOK1X5WHbDQK usYQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64113-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64113-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWqsaZrabRv8a6BdD5KgURWW937mUfMJ1Dh+xnObnoscF5HX2Dw4i1EnyPG9OO7DFayC2eqAoXAmnzly2wEXl0ju9Y6tA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id l66-20020a633e45000000b005dc19ac7173si2350815pga.21.2024.02.13.10.50.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:50:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64113-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64113-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64113-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3D9F0290591 for ; Tue, 13 Feb 2024 18:50:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 36DF360894; Tue, 13 Feb 2024 18:46:19 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 03D4D626C9 for ; Tue, 13 Feb 2024 18:46:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849977; cv=none; b=fxQMFBnAMDc2bxRS+FqCp+9ut53dZE7V51A1QyeAYojV1XyrB68WD2MwGldPyXFWjmByHDMhNtqdjyRwvLSMdtxHJ/HHxSiOJYX8uIGH+8PTz9zGorPkgpz5clZiOjpY0PVltGMEEhbVguynmehaoOJZ3GMwEZxkmTBTGvQ1ukc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849977; c=relaxed/simple; bh=R72bTUh1j2/iRl0BP2jxva1B3BK/NOYi0JFQYorCmiI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JNVztWokUMwwK6GJlGKEfHUw2ORFs26t+AVBoitEikULS7Z48TYfQ+QBFL53T5TM84x/PeP9k7EoRQwZKwD6r06BgFw27SOSssBY/CaORDcfv5yJyV8NuOeEYB8ol0b+o4iTfqpcMcCec595Y9Hl7ONGaPVxpCD9pkyvIz3J56k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C3B7E1595; Tue, 13 Feb 2024 10:46:56 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6587A3F766; Tue, 13 Feb 2024 10:46:12 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 22/24] x86/resctrl: Add CPU offline callback for resctrl work Date: Tue, 13 Feb 2024 18:44:36 +0000 Message-Id: <20240213184438.16675-23-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810775887414856 X-GMAIL-MSGID: 1790810775887414856 The resctrl architecture specific code may need to free a domain when a CPU goes offline, it also needs to reset the CPUs PQR_ASSOC register. Amongst other things, the resctrl filesystem code needs to clear this CPU from the cpu_mask of any control and monitor groups. Currently this is all done in core.c and called from resctrl_offline_cpu(), making the split between architecture and filesystem code unclear. Move the filesystem work to remove the CPU from the control and monitor groups into a filesystem helper called resctrl_offline_cpu(), and rename the one in core.c resctrl_arch_offline_cpu(). Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- arch/x86/kernel/cpu/resctrl/core.c | 25 +++++-------------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 24 ++++++++++++++++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 30 insertions(+), 20 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 55322ba629da..4aedefa22f61 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -625,31 +625,15 @@ static int resctrl_arch_online_cpu(unsigned int cpu) return 0; } -static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) +static int resctrl_arch_offline_cpu(unsigned int cpu) { - struct rdtgroup *cr; - - list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { - if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask)) { - break; - } - } -} - -static int resctrl_offline_cpu(unsigned int cpu) -{ - struct rdtgroup *rdtgrp; struct rdt_resource *r; mutex_lock(&rdtgroup_mutex); + resctrl_offline_cpu(cpu); + for_each_capable_rdt_resource(r) domain_remove_cpu(cpu, r); - list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) { - if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) { - clear_childcpus(rdtgrp, cpu); - break; - } - } clear_closid_rmid(cpu); mutex_unlock(&rdtgroup_mutex); @@ -971,7 +955,8 @@ static int __init resctrl_late_init(void) state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/resctrl/cat:online:", - resctrl_arch_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, + resctrl_arch_offline_cpu); if (state < 0) return state; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index f5688c79d94f..5bd3d8fb3f67 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4017,6 +4017,30 @@ void resctrl_online_cpu(unsigned int cpu) cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); } +static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) +{ + struct rdtgroup *cr; + + list_for_each_entry(cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { + if (cpumask_test_and_clear_cpu(cpu, &cr->cpu_mask)) + break; + } +} + +void resctrl_offline_cpu(unsigned int cpu) +{ + struct rdtgroup *rdtgrp; + + lockdep_assert_held(&rdtgroup_mutex); + + list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) { + if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) { + clear_childcpus(rdtgrp, cpu); + break; + } + } +} + /* * rdtgroup_init - rdtgroup initialization * diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index ccbbbe5d18d3..270ff1d5c051 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -226,6 +226,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d); void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); void resctrl_online_cpu(unsigned int cpu); +void resctrl_offline_cpu(unsigned int cpu); /** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid From patchwork Tue Feb 13 18:44:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200564 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740919dyb; Tue, 13 Feb 2024 10:51:14 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCURzK4gIkeHP69vMjOhd33n86lHSZ/fTGehzP4Nt4JZciDpOrxEhkroD2dc/etLDDGDhPCVPsK2CunIl+lSOb6BBQqEWQ== X-Google-Smtp-Source: AGHT+IGF0lniPjJgjb72EfXex6xDTucskC18h36wAc/Z/nY12oSsF9lPICZQBe2C+4lhOkQmY/Kl X-Received: by 2002:ac8:5909:0:b0:42c:6923:6e52 with SMTP id 9-20020ac85909000000b0042c69236e52mr338465qty.17.1707850273812; Tue, 13 Feb 2024 10:51:13 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850273; cv=pass; d=google.com; s=arc-20160816; b=xTduTe2p/RFp3dQW+8WgV4sMAeryaUSOHBl9p65/xxeIiQojuCZrpRCNS1ipoGx7UM zqJS9DYZbj0tFaDTbapTq/tVpoQXb6ZLuPDGi4+ZBaap3T8eqcUHR28xHKfhLYkpo3z6 4KD56Qi3npqpzvIqUpVI5vz5eahtxU3X/e4SDnZYLbhxIq+8VW4dZV1+2rzdR42SYHMt ZVP38TRWrrm1xuA809KCU8ICrugTVsEf79k91svLiIJeNmSQMLXa3OF2ui3igg7ZpbB3 uARZMuYmHeAWb9AHbm3HvQ+y/RbbQg9kwG3+okLv9HrILWjKBdR0FnQyuZTdyJqQHYNf feWQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=FoON0ULOmmeByLgj/eRokC6XIVi03jW4OrcL3lQWo8A=; fh=loFYBUI7/zRwwbKkRN9zrQ4QXQQXCSF85bNP2+Rk8Bc=; b=kXcOjYob8YEDCPIydC6AdSQSx2BKMKOybv2cEDjiFBR8G4ehShFetmV0uCok0a9BO5 pqP8eiqqg+8Bma6/TP7wJCCDMo/hSsA6GOMrCfW0YgGqLV98mLAmNuNI4I/noSA/FGrt 8NLlY0Rs7l29mwCaLJEgQayypJmX5OdTCuBKEFUQXA3i04HBTtt86ub0AtSz7Gw9PS56 tlhvD143wKRHBDBmgoka4TTvCkBFEtAbNBQ3sk8l46t5+uN6BGeiHzdIxPvjNIC1uVdY ZdQpbCDRjCYvfv+gi19qwl/0p/Z9H8rZGN7eM/YV93e9EGyDo80JFOgnFOEBYvqu5Wbm EFKQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64114-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64114-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCXIcjx/g8SfpTZEo581er8+FrNUl3+GafAad2omFdX6ISH4M2O9XtFeIF9e+N0o3U61PCIj5L0tSPIUWvHWitDodQrbeQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id b7-20020ac85bc7000000b0042c2f47ede5si3766621qtb.467.2024.02.13.10.51.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:51:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64114-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64114-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64114-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 5102C1C2474C for ; Tue, 13 Feb 2024 18:50:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 530A2111B2; Tue, 13 Feb 2024 18:46:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 25F9360893 for ; Tue, 13 Feb 2024 18:46:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849980; cv=none; b=dA7g8jmASAluzF2MweLBCDzw+fpH5jNfYxU+naFWBz6LHAvp8/88L3oHDHFxpdfm9n0oDgg3Hy4PrFD8PJOQqEscNDvgUUw8zuM4rmX82r3qdj1GA2I6rgOgFsxn64MBhgu7md7OK3jH4eLm/ghfH4Jihza0lgLbRAG1OopAHgk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849980; c=relaxed/simple; bh=hHAjGEeRkRs4daqFFngb0LSIvNeLboUSwjB1bPyY14U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QFM0AJNeVcjFtxdHPV1IxBd5PqhO3BieqnNXvfW8Z4mALn+OrpyZSxtn4Eqc7qnXkO4X/6pv/wJPwPc+tmGTNL0fduCVZ8gzFz2ZsFDqY9nIrdnGOuXLG1RqojqdiRAEXdTQ2prBa8vhZKZe8F3xUqkJAjN/FY+UmHO3zWBl1Tg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1CB8B1576; Tue, 13 Feb 2024 10:47:00 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B71123F766; Tue, 13 Feb 2024 10:46:15 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 23/24] x86/resctrl: Move domain helper migration into resctrl_offline_cpu() Date: Tue, 13 Feb 2024 18:44:37 +0000 Message-Id: <20240213184438.16675-24-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810808702939932 X-GMAIL-MSGID: 1790810808702939932 When a CPU is taken offline the resctrl filesystem code needs to check if it was the CPU nominated to perform the periodic overflow and limbo work. If so, another CPU needs to be chosen to do this work. This is currently done in core.c, mixed in with the code that removes the CPU from the domain's mask, and potentially free()s the domain. Move the migration of the overflow and limbo helpers into the filesystem code, into resctrl_offline_cpu(). As resctrl_offline_cpu() runs before the architecture code has removed the CPU from the domain mask, the callers need to be told which CPU is being removed, to avoid picking it as the new CPU. This uses the exclude_cpu feature previously added. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v5: * Changed fir tree order of variables. * Added mon-capable check for cpu offline. --- arch/x86/kernel/cpu/resctrl/core.c | 16 ---------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 ++++++++++++++++++ 2 files changed, 18 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 4aedefa22f61..b03a6c658ae5 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -580,22 +580,6 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) return; } - - if (r == &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl) { - if (is_mbm_enabled() && cpu == d->mbm_work_cpu) { - cancel_delayed_work(&d->mbm_over); - /* - * temporary: exclude_cpu=-1 as this CPU has already - * been removed by cpumask_clear_cpu()d - */ - mbm_setup_overflow_handler(d, 0, RESCTRL_PICK_ANY_CPU); - } - if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && - has_busy_rmid(d)) { - cancel_delayed_work(&d->cqm_limbo); - cqm_setup_limbo_handler(d, 0, RESCTRL_PICK_ANY_CPU); - } - } } static void clear_closid_rmid(int cpu) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 5bd3d8fb3f67..777e9f680332 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -4029,7 +4029,9 @@ static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) void resctrl_offline_cpu(unsigned int cpu) { + struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; struct rdtgroup *rdtgrp; + struct rdt_domain *d; lockdep_assert_held(&rdtgroup_mutex); @@ -4039,6 +4041,22 @@ void resctrl_offline_cpu(unsigned int cpu) break; } } + + if (!l3->mon_capable) + return; + + d = get_domain_from_cpu(cpu, l3); + if (d) { + if (is_mbm_enabled() && cpu == d->mbm_work_cpu) { + cancel_delayed_work(&d->mbm_over); + mbm_setup_overflow_handler(d, 0, cpu); + } + if (is_llc_occupancy_enabled() && cpu == d->cqm_work_cpu && + has_busy_rmid(d)) { + cancel_delayed_work(&d->cqm_limbo); + cqm_setup_limbo_handler(d, 0, cpu); + } + } } /* From patchwork Tue Feb 13 18:44:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 200563 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp740882dyb; Tue, 13 Feb 2024 10:51:09 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCV+HiqbMgFSkegNvmUu9QpOVsDNpbABpWC0eRYAFF0MVn75VLYdrPUSuzEsevctIJGcT4zonatEBV3eQh8NOVmcQCOwRQ== X-Google-Smtp-Source: AGHT+IENE7o54DHfV98xwzv8WlcrOs2lwRJuk/G6NuzZHzGWyEOQ3o8vKdjmm39oYnKlXzoW2V2b X-Received: by 2002:a17:902:7b8e:b0:1da:3278:e6b with SMTP id w14-20020a1709027b8e00b001da32780e6bmr329444pll.66.1707850269095; Tue, 13 Feb 2024 10:51:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707850269; cv=pass; d=google.com; s=arc-20160816; b=H3mZmmlBxa2GzWre6ZKEqLvSbcOwAzlIkBzvLi+aeqcbFAk6J0F0N/kdv8zHmQfsi7 L5BHmUnqmlD5IbQmF7mqq8RL46d0xaAA1t7fombt6cEmWcn2mliHZEB8GM98K8g4MtDB WMsC2xCsY4eR7dyQ0+w0EGbLlRQVPXf9bF0wbG5uqm56ZYkjJhwKD9bFzav6NPsFDPPn VOlsyxIXpsB8NyrGSe5Vl3DzvSYl0hWKv/faMjTloIpuqu8drCpPRI0r0B0IMIDMRJr7 l8PSU+MejatGPsN4BQfZLgSkNjXHX+eP9UbbLp1coAaYTaFMRXjek5RjO/HOddvNyiRd 5edw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=kkK5KrmQpChoZjSiMOMFZ3Z2pE4LTMsIj2v2n/pgaR4=; fh=2joRu/J9GaRgY6PGc8GPuqz4rH2aertUskPPZ9gGlvo=; b=l4Be05jzSuf33PXY8wzkoKzOA9eMI3aQAoZ+1e5y6Nnq840w1oyv5oy0hQ3tL0LH/u /nCoMyofQfwJRxrkgBIdjrEbCeO38NpPqhoAXrg8rVWLIXPcbfgZzv0cqnQR43q2UnMu VKAA8rchcHiBRBR+VAf60ZeGc2emeOjd9WoNKT5TtONYIKo1cf4LtuBg8lrGuCSpxi2D QNCd1BBXdtYpl9iUo7GyTPjPAXeim6RYXEeDN2upLYo9mYI6F5K3fCLAw5D3xlcBWMUs GPU+QCLlPBxxUSqczg7RtA/5aNW6XUODjQpchel5tygc1NcMRdMUmQpkf4efeBS6WC9b LLXw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64115-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64115-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com X-Forwarded-Encrypted: i=2; AJvYcCWr+lECxy10waBvl/3+JABs0Dnr5N7iCnAQd2soBH1xNNXNj17FiEUz3HY4+yAeSKJ8L+q5kxj6o8h7TS3Y437GirsA1Q== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id k12-20020a170902d58c00b001d8f4b57bf7si2390366plh.107.2024.02.13.10.51.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 10:51:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64115-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-64115-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64115-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 9F83B290646 for ; Tue, 13 Feb 2024 18:51:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A48E662810; Tue, 13 Feb 2024 18:46:28 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 709B2626C9 for ; Tue, 13 Feb 2024 18:46:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849984; cv=none; b=j4F8zAxohRnbBagq+u7/mDFE2Hrq8Fh6ffnUQ/KgvTMXWdvFTjRMPH0OFAo9rxnkV4Y056Hqm00hWFpvVHIlVMIqcSt76RhRC/YbeHYIt+mYehi47z7II2NzFbFkKcSpLxewUPB9QK14f6GdxMxlRBaXr/VIWKaKLbOD7ZnpyZE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707849984; c=relaxed/simple; bh=qTpSJectV554qx1dawyxGSS7lymy21OoN0CWoKuzujk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IlvzK7k8oyJv1h9v8fUZM5Wwc2FkqYYmRUYjWSHmiIiUGxPfnh+qQloxeBnfwxEglvZ0yzOeUZgej/LGvxUXruvlt7AyzEFK7v+5GCY7tDlRIrsvW61yTR8i9ec/pv5dqHP8mH4MYk4SV0JnBx78dAY5Gs11FrpGJVh6GhBngsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7268C1596; Tue, 13 Feb 2024 10:47:03 -0800 (PST) Received: from merodach.members.linode.com (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 139B33F766; Tue, 13 Feb 2024 10:46:18 -0800 (PST) From: James Morse To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , James Morse , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, baolin.wang@linux.alibaba.com, Jamie Iles , Xin Hao , peternewman@google.com, dfustini@baylibre.com, amitsinght@marvell.com, David Hildenbrand , Babu Moger Subject: [PATCH v9 24/24] x86/resctrl: Separate arch and fs resctrl locks Date: Tue, 13 Feb 2024 18:44:38 +0000 Message-Id: <20240213184438.16675-25-james.morse@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240213184438.16675-1-james.morse@arm.com> References: <20240213184438.16675-1-james.morse@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790810803590326938 X-GMAIL-MSGID: 1790810803590326938 resctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list. Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against CPU hotplug. The CPU hotplug code already has a mechanism to do this using cpus_read_lock(). MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory. Because resctrl_{on,off}line_cpu() take the rdtgroup_mutex as part of a cpuhp callback, cpus_read_lock() must always be taken first. rdtgroup_schemata_write() already does this. Most of the filesystem code's domain list walkers are currently protected by the rdtgroup_mutex taken in rdtgroup_kn_lock_live(). The exceptions are rdt_bit_usage_show() and the mon_config helpers which take the lock directly. Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers. rdt_bit_usage_show() could walk the domain list using RCU, but to keep all the filesystem operations the same, this is changed to call cpus_read_lock(). The mon_config helpers send multiple IPIs, take the cpus_read_lock() in these cases. The other filesystem list walkers need to be able to sleep. Add cpus_read_lock() to rdtgroup_kn_lock_live() so that the cpuhp callbacks can't be invoked when file system operations are occurring. Add lockdep_assert_cpus_held() in the cases where the rdtgroup_kn_lock_live() call isn't obvious. Resctrl's domain online/offline calls now need to take the rdtgroup_mutex themselves. Signed-off-by: James Morse Tested-by: Shaopeng Tan Tested-by: Peter Newman Tested-by: Babu Moger Tested-by: Carl Worth # arm64 Reviewed-by: Shaopeng Tan Reviewed-by: Reinette Chatre Reviewed-by: Babu Moger --- Changes since v2: * Reworded a comment, * Added a lockdep assertion * Moved clear_closid_rmid() outside the locked region of cpu online/offline Changes since v3: * Added a header include Changes since v5: * Made rdt_bit_usage_show() take the cpus_read_lock() instead of using RCU. Changes since v6: * Added lockdep_is_cpus_held() to get_domain_from_cpu(). * Added cpus_read_lock() around overflow and limbo handlers. --- arch/x86/kernel/cpu/resctrl/core.c | 44 +++++++++++---- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 15 ++++- arch/x86/kernel/cpu/resctrl/monitor.c | 8 +++ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 3 + arch/x86/kernel/cpu/resctrl/rdtgroup.c | 68 ++++++++++++++++++----- include/linux/resctrl.h | 2 +- 6 files changed, 112 insertions(+), 28 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index b03a6c658ae5..9f1aa555a8ea 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -16,6 +16,7 @@ #define pr_fmt(fmt) "resctrl: " fmt +#include #include #include #include @@ -25,8 +26,15 @@ #include #include "internal.h" -/* Mutex to protect rdtgroup access. */ -DEFINE_MUTEX(rdtgroup_mutex); +/* + * rdt_domain structures are kfree()d when their last CPU goes offline, + * and allocated when the first CPU in a new domain comes online. + * The rdt_resource's domain list is updated when this happens. Readers of + * the domain list must either take cpus_read_lock(), or rely on an RCU + * read-side critical section, to avoid observing concurrent modification. + * All writers take this mutex: + */ +static DEFINE_MUTEX(domain_list_lock); /* * The cached resctrl_pqr_state is strictly per CPU and can never be @@ -354,6 +362,15 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) { struct rdt_domain *d; + /* + * Walking r->domains, ensure it can't race with cpuhp. + * Because this is called via IPI by rdt_ctrl_update(), assertions + * about locks this thread holds will lead to false positives. Check + * someone is holding the CPUs lock. + */ + if (IS_ENABLED(CONFIG_LOCKDEP)) + lockdep_is_cpus_held(); + list_for_each_entry(d, &r->domains, list) { /* Find the domain that contains this CPU */ if (cpumask_test_cpu(cpu, &d->cpu_mask)) @@ -510,6 +527,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) struct rdt_domain *d; int err; + lockdep_assert_held(&domain_list_lock); + d = rdt_find_domain(r, id, &add_pos); if (IS_ERR(d)) { pr_warn("Couldn't find cache id for CPU %d\n", cpu); @@ -543,11 +562,12 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r) return; } - list_add_tail(&d->list, add_pos); + list_add_tail_rcu(&d->list, add_pos); err = resctrl_online_domain(r, d); if (err) { - list_del(&d->list); + list_del_rcu(&d->list); + synchronize_rcu(); domain_free(hw_dom); } } @@ -558,6 +578,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) struct rdt_hw_domain *hw_dom; struct rdt_domain *d; + lockdep_assert_held(&domain_list_lock); + d = rdt_find_domain(r, id, NULL); if (IS_ERR_OR_NULL(d)) { pr_warn("Couldn't find cache id for CPU %d\n", cpu); @@ -568,7 +590,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) cpumask_clear_cpu(cpu, &d->cpu_mask); if (cpumask_empty(&d->cpu_mask)) { resctrl_offline_domain(r, d); - list_del(&d->list); + list_del_rcu(&d->list); + synchronize_rcu(); /* * rdt_domain "d" is going to be freed below, so clear @@ -598,13 +621,13 @@ static int resctrl_arch_online_cpu(unsigned int cpu) { struct rdt_resource *r; - mutex_lock(&rdtgroup_mutex); + mutex_lock(&domain_list_lock); for_each_capable_rdt_resource(r) domain_add_cpu(cpu, r); - clear_closid_rmid(cpu); + mutex_unlock(&domain_list_lock); + clear_closid_rmid(cpu); resctrl_online_cpu(cpu); - mutex_unlock(&rdtgroup_mutex); return 0; } @@ -613,13 +636,14 @@ static int resctrl_arch_offline_cpu(unsigned int cpu) { struct rdt_resource *r; - mutex_lock(&rdtgroup_mutex); resctrl_offline_cpu(cpu); + mutex_lock(&domain_list_lock); for_each_capable_rdt_resource(r) domain_remove_cpu(cpu, r); + mutex_unlock(&domain_list_lock); + clear_closid_rmid(cpu); - mutex_unlock(&rdtgroup_mutex); return 0; } diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c index 64db51455df3..dc59643498bf 100644 --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c @@ -212,6 +212,9 @@ static int parse_line(char *line, struct resctrl_schema *s, struct rdt_domain *d; unsigned long dom_id; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP && (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA)) { rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n"); @@ -316,6 +319,9 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid) struct rdt_domain *d; u32 idx; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) return -ENOMEM; @@ -381,11 +387,9 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, return -EINVAL; buf[nbytes - 1] = '\0'; - cpus_read_lock(); rdtgrp = rdtgroup_kn_lock_live(of->kn); if (!rdtgrp) { rdtgroup_kn_unlock(of->kn); - cpus_read_unlock(); return -ENOENT; } rdt_last_cmd_clear(); @@ -447,7 +451,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, out: rdt_staged_configs_clear(); rdtgroup_kn_unlock(of->kn); - cpus_read_unlock(); return ret ?: nbytes; } @@ -467,6 +470,9 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clo bool sep = false; u32 ctrl_val; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + seq_printf(s, "%*s:", max_name_width, schema->name); list_for_each_entry(dom, &r->domains, list) { if (sep) @@ -537,6 +543,9 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, { int cpu; + /* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + /* * Setup the parameters to pass to mon_event_count() to read the data. */ diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index 67edd4c440f0..c34a35ec0f03 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -15,6 +15,7 @@ * Software Developer Manual June 2016, volume 3, section 17.17. */ +#include #include #include #include @@ -472,6 +473,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) lockdep_assert_held(&rdtgroup_mutex); + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid); entry->busy = 0; @@ -778,6 +782,7 @@ void cqm_handle_limbo(struct work_struct *work) unsigned long delay = msecs_to_jiffies(CQM_LIMBOCHECK_INTERVAL); struct rdt_domain *d; + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); d = container_of(work, struct rdt_domain, cqm_limbo.work); @@ -792,6 +797,7 @@ void cqm_handle_limbo(struct work_struct *work) } mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); } /** @@ -823,6 +829,7 @@ void mbm_handle_overflow(struct work_struct *work) struct rdt_resource *r; struct rdt_domain *d; + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); /* @@ -856,6 +863,7 @@ void mbm_handle_overflow(struct work_struct *work) out_unlock: mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); } /** diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index 8056bed033cc..884b88e25141 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -844,6 +844,9 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d) struct rdt_domain *d_i; bool ret = false; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (!zalloc_cpumask_var(&cpu_with_psl, GFP_KERNEL)) return true; diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 777e9f680332..011e17efb1a6 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -35,6 +35,10 @@ DEFINE_STATIC_KEY_FALSE(rdt_enable_key); DEFINE_STATIC_KEY_FALSE(rdt_mon_enable_key); DEFINE_STATIC_KEY_FALSE(rdt_alloc_enable_key); + +/* Mutex to protect rdtgroup access. */ +DEFINE_MUTEX(rdtgroup_mutex); + static struct kernfs_root *rdt_root; struct rdtgroup rdtgroup_default; LIST_HEAD(rdt_all_groups); @@ -1014,6 +1018,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, bool sep = false; u32 ctrl_val; + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); hw_shareable = r->cache.shareable_bits; list_for_each_entry(dom, &r->domains, list) { @@ -1074,6 +1079,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, } seq_putc(seq, '\n'); mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); return 0; } @@ -1329,6 +1335,9 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp) struct rdt_domain *d; u32 ctrl; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + list_for_each_entry(s, &resctrl_schema_all, list) { r = s->res; if (r->rid == RDT_RESOURCE_MBA || r->rid == RDT_RESOURCE_SMBA) @@ -1593,6 +1602,7 @@ static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid struct rdt_domain *dom; bool sep = false; + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); list_for_each_entry(dom, &r->domains, list) { @@ -1609,6 +1619,7 @@ static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtid seq_puts(s, "\n"); mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); return 0; } @@ -1690,6 +1701,9 @@ static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid) unsigned long dom_id, val; struct rdt_domain *d; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + next: if (!tok || tok[0] == '\0') return 0; @@ -1736,6 +1750,7 @@ static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of, if (nbytes == 0 || buf[nbytes - 1] != '\n') return -EINVAL; + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); rdt_last_cmd_clear(); @@ -1745,6 +1760,7 @@ static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of, ret = mon_config_write(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID); mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); return ret ?: nbytes; } @@ -1760,6 +1776,7 @@ static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of, if (nbytes == 0 || buf[nbytes - 1] != '\n') return -EINVAL; + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); rdt_last_cmd_clear(); @@ -1769,6 +1786,7 @@ static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of, ret = mon_config_write(r, buf, QOS_L3_MBM_LOCAL_EVENT_ID); mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); return ret ?: nbytes; } @@ -2245,6 +2263,9 @@ static int set_cache_qos_cfg(int level, bool enable) struct rdt_domain *d; int cpu; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (level == RDT_RESOURCE_L3) update = l3_qos_cfg_update; else if (level == RDT_RESOURCE_L2) @@ -2444,6 +2465,7 @@ struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn) rdtgroup_kn_get(rdtgrp, kn); + cpus_read_lock(); mutex_lock(&rdtgroup_mutex); /* Was this group deleted while we waited? */ @@ -2461,6 +2483,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn) return; mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); + rdtgroup_kn_put(rdtgrp, kn); } @@ -2793,6 +2817,9 @@ static int reset_all_ctrls(struct rdt_resource *r) struct rdt_domain *d; int i; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + if (!zalloc_cpumask_var(&cpu_mask, GFP_KERNEL)) return -ENOMEM; @@ -3077,6 +3104,9 @@ static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn, struct rdt_domain *dom; int ret; + /* Walking r->domains, ensure it can't race with cpuhp */ + lockdep_assert_cpus_held(); + list_for_each_entry(dom, &r->domains, list) { ret = mkdir_mondata_subdir(parent_kn, dom, r, prgrp); if (ret) @@ -3907,13 +3937,13 @@ static void domain_destroy_mon_state(struct rdt_domain *d) void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) { - lockdep_assert_held(&rdtgroup_mutex); + mutex_lock(&rdtgroup_mutex); if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) mba_sc_domain_destroy(r, d); if (!r->mon_capable) - return; + goto out_unlock; /* * If resctrl is mounted, remove all the @@ -3938,6 +3968,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) } domain_destroy_mon_state(d); + +out_unlock: + mutex_unlock(&rdtgroup_mutex); } static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) @@ -3973,20 +4006,22 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) { - int err; + int err = 0; - lockdep_assert_held(&rdtgroup_mutex); + mutex_lock(&rdtgroup_mutex); - if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) + if (supports_mba_mbps() && r->rid == RDT_RESOURCE_MBA) { /* RDT_RESOURCE_MBA is never mon_capable */ - return mba_sc_domain_allocate(r, d); + err = mba_sc_domain_allocate(r, d); + goto out_unlock; + } if (!r->mon_capable) - return 0; + goto out_unlock; err = domain_setup_mon_state(r, d); if (err) - return err; + goto out_unlock; if (is_mbm_enabled()) { INIT_DELAYED_WORK(&d->mbm_over, mbm_handle_overflow); @@ -4006,15 +4041,18 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) if (resctrl_mounted && resctrl_arch_mon_capable()) mkdir_mondata_subdir_allrdtgrp(r, d); - return 0; +out_unlock: + mutex_unlock(&rdtgroup_mutex); + + return err; } void resctrl_online_cpu(unsigned int cpu) { - lockdep_assert_held(&rdtgroup_mutex); - + mutex_lock(&rdtgroup_mutex); /* The CPU is set in default rdtgroup after online. */ cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask); + mutex_unlock(&rdtgroup_mutex); } static void clear_childcpus(struct rdtgroup *r, unsigned int cpu) @@ -4033,8 +4071,7 @@ void resctrl_offline_cpu(unsigned int cpu) struct rdtgroup *rdtgrp; struct rdt_domain *d; - lockdep_assert_held(&rdtgroup_mutex); - + mutex_lock(&rdtgroup_mutex); list_for_each_entry(rdtgrp, &rdt_all_groups, rdtgroup_list) { if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) { clear_childcpus(rdtgrp, cpu); @@ -4043,7 +4080,7 @@ void resctrl_offline_cpu(unsigned int cpu) } if (!l3->mon_capable) - return; + goto out_unlock; d = get_domain_from_cpu(cpu, l3); if (d) { @@ -4057,6 +4094,9 @@ void resctrl_offline_cpu(unsigned int cpu) cqm_setup_limbo_handler(d, 0, cpu); } } + +out_unlock: + mutex_unlock(&rdtgroup_mutex); } /* diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index 270ff1d5c051..a365f67131ec 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -159,7 +159,7 @@ struct resctrl_schema; * @cache_level: Which cache level defines scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. - * @domains: All domains for this resource + * @domains: RCU list of all domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.