From patchwork Tue Jan 23 09:20:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haifeng Xu X-Patchwork-Id: 190802 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp217691dyi; Tue, 23 Jan 2024 01:32:04 -0800 (PST) X-Google-Smtp-Source: AGHT+IEi+42WSoIj0gIJFnay/s6vkmtcge//aQmI0/6mSy0r3ckIKhmsAwdmtN7ukQZ4UEtyKJWo X-Received: by 2002:a17:906:2694:b0:a2e:6f5f:2662 with SMTP id t20-20020a170906269400b00a2e6f5f2662mr2206479ejc.65.1706002324337; Tue, 23 Jan 2024 01:32:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706002324; cv=pass; d=google.com; s=arc-20160816; b=rLFR2IrZQqFR6/GTu6eKRJwA9zfqR/qApSKcYAyMs8WpMOCUNiOfqTbCYD5kXUYuQ4 9SeexIndLucDWrik8pq0xwWo7Was3+9enIyIvVR/EqsmWgd03FfN6urLbBu15mDe6hWk ZsN1SP9q/JQK+5CiB2TeFeu5V6hT4BkEjk7vdnUOVfVPVvvH3I+nEZ0Ro9/y6whHfb3I rDLfyoxwgOXmo3rgnYwqbyVfsSam4iLVmOfMl45oiIMHLY90WXvccHzW7XutE0gJ/+0C aZWvyPHugv7nJnnFQCkYxdfkEjx8gCLgSTNE3OoJe5CdarXgoGeCEhEw4EfiFu38QEXi 6r8g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=CJkjXjTnrwSkzf7VwgN4WhxQt30ievtg/kefs5USDPY=; fh=evP20v0Z2agBF6766Fg4f0yypEL+LBR7IJvWx8Uqu9c=; b=Qlzd9oI4GMGjdqxNKLVVB5XlmAA15C7TI/CF1oZvidCiqDDdObFuy9PGemhZ+XadVb afCz+zqRxD3j4BpctfhC/gggt5wP51iBgiiCy96rlPnesSReV1mID4LNJs09p3mS4mcH l0fc93yuUg+2y1SL0LqFuVUpYdjSND+PTM0skQEGCco9m2g54Y4CCmfS4Zm/qzxK9B6C AzqmDSS3FocvlPCHRSev5Fhl1JmOHnHPx+YzttwYnqV5Qlq5kFqMTvyIIbIRWe9vA6M5 0QbCP25g4EI2xgh1w3MQCn7CXZR8/wWZmFu63vOmhlqp+1HuOA6p0UIk+XlsoZX/dBGV xtDw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@shopee.com header.s=shopee.com header.b="XS/Uh0iJ"; arc=pass (i=1 spf=pass spfdomain=shopee.com dkim=pass dkdomain=shopee.com dmarc=pass fromdomain=shopee.com); spf=pass (google.com: domain of linux-kernel+bounces-34988-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-34988-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=shopee.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id b3-20020a170906d10300b00a3052aa8f8fsi1850957ejz.48.2024.01.23.01.32.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 01:32:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-34988-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@shopee.com header.s=shopee.com header.b="XS/Uh0iJ"; arc=pass (i=1 spf=pass spfdomain=shopee.com dkim=pass dkdomain=shopee.com dmarc=pass fromdomain=shopee.com); spf=pass (google.com: domain of linux-kernel+bounces-34988-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-34988-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=shopee.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 0C7E21F298AF for ; Tue, 23 Jan 2024 09:22:26 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6B5CF5A0F1; Tue, 23 Jan 2024 09:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shopee.com header.i=@shopee.com header.b="XS/Uh0iJ" Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E88505822D for ; Tue, 23 Jan 2024 09:21:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706001689; cv=none; b=qL7xfFavtSW/BBf2Bx6CTl9dP8Mp+Y82rcKaw7o7vKvU84aVE5VU+o5tvm3oQ4ENsSRNbLRYCDd69pPwUbTBbkgeQxCnL8NvGD+xw0rCuVneszmVb0+bjo2uRDmm44q8ZqIZeBuG3jk19j25gMnwG+a0Ds215erQNfaEmnk0h4E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706001689; c=relaxed/simple; bh=7C+tIDAexxZtGUr88BRAtpwjZhSDpQTHpgA/Tvvz8pE=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=uImfW+mF+ZRDRvcf0FE/lH2++JVRhv7YE+SlblCVNiJn9+CIml5ldsAOB+H3/Q0Juyo3+r0F9DZbXOTvHUwiDul5BALRnxM7k4xowa7a4E+gPBxXRr1ZahA7EaU7OhXi9j/vdZaS6AyBSsyICysQBLxTm75YSRmAS0gRfbVZmJU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=shopee.com; spf=pass smtp.mailfrom=shopee.com; dkim=pass (2048-bit key) header.d=shopee.com header.i=@shopee.com header.b=XS/Uh0iJ; arc=none smtp.client-ip=209.85.210.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=shopee.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shopee.com Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-6de424cef01so1978236a34.2 for ; Tue, 23 Jan 2024 01:21:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shopee.com; s=shopee.com; t=1706001687; x=1706606487; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=CJkjXjTnrwSkzf7VwgN4WhxQt30ievtg/kefs5USDPY=; b=XS/Uh0iJorVptS03/u+UFJN+KSIIFSp9eCUFBGIA37rnVHIqG4AWowTdmCfjc+9wi/ t2dIoDL09B3yHW5ynzErHlDlVEw4Wy05CYoGCqUz3DYhfj9M5Jnyx25FOylgwEROim6m 9EK8iJewUKCpd3LqLEJOgdCeQwD1WQFSdBRmpjlie8jr+ehTzS/a6698KN9jbYhxPb8V SISEJFKhm4XmSKCQrU8iWGiN2mhUSRor9bgtPyFYX3wTIcKVEDfx9zXcfVxEVuh9WP7n O8n67zzLZIJCytnUew/QJm8Oq/SNLn6M0wcAJT4LMoyDvDjs1Jh1VlxylswImmie39gA gkMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706001687; x=1706606487; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=CJkjXjTnrwSkzf7VwgN4WhxQt30ievtg/kefs5USDPY=; b=GlMfRSVjsthWASX7p8CPpr/1rOzzMrchmlJbyTUK0GF162Fqd/gIwyNa1TLIn7DHH3 1nLAYz/lUYLacnhS3/DmeXNWb3lmeINkcwOFQlYaBax2bYm+F4v0RzDQr5JiF/Itf9ff wFDjximO/pPPbfTrrAZno/HM5U5kxYM2pT4uXltOnLqgwhGmpN5PSOvIURsANEBlOpG+ DFxDD0jdoiCDfmHow1jLRIZ2PrCxGNIEaZF81sFVPAb9N2HtjbwrOBwraveXm5S+BvRv u3RisZEA0EVpc5oxxCmYWtRUYVH8tH285QlnEw3mTZwyqdygrZEQlYDvwRTiZDJMqNnf hZhQ== X-Gm-Message-State: AOJu0YwVBF0vaxTQ1hpSxzE1APqlJdk2N8RQLCzs8hhhdcutOqRgAXmc lSa9CHcrqukrOiNvYfMtMdsXxEDaaUd9W5szp0NG3++heJB75sCUWLADAs0f8H8= X-Received: by 2002:a05:6358:60cb:b0:176:2f2:1c5a with SMTP id i11-20020a05635860cb00b0017602f21c5amr4868638rwi.30.1706001686799; Tue, 23 Jan 2024 01:21:26 -0800 (PST) Received: from ubuntu-haifeng.default.svc.cluster.local ([101.127.248.173]) by smtp.gmail.com with ESMTPSA id i15-20020aa787cf000000b006dbddea71e2sm3650707pfo.79.2024.01.23.01.21.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 01:21:26 -0800 (PST) From: Haifeng Xu To: reinette.chatre@intel.com Cc: fenghua.yu@intel.com, babu.moger@amd.com, peternewman@google.com, x86@kernel.org, linux-kernel@vger.kernel.org, Haifeng Xu Subject: [PATCH 2/3] x86/resctrl: Display the number of available RMIDs Date: Tue, 23 Jan 2024 09:20:23 +0000 Message-Id: <20240123092024.1271882-2-haifeng.xu@shopee.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788873093229871604 X-GMAIL-MSGID: 1788873093229871604 Now we can know the total number of RMIDs from: /sys/fs/resctrl/info/L3_MON/num_rmids But for users, they don't know the number of available RMIDs. Especially when llc_occupancy is enabled, though the monitor group is removed, its rmid may not be freed. Users may confused that though the number of control groups and monitor groups is less than num_rmids, they still fail to create a new monitor group, so add a RFTYPE_TOP_INFO file 'free_rmids' that tells users how many free rmids are left. Signed-off-by: Haifeng Xu --- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 16 ++++++++++++++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 ++++++++++++++++ 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h index a4f1aa15f0a2..4b0392bd3a65 100644 --- a/arch/x86/kernel/cpu/resctrl/internal.h +++ b/arch/x86/kernel/cpu/resctrl/internal.h @@ -117,6 +117,7 @@ struct rmid_read { extern bool rdt_alloc_capable; extern bool rdt_mon_capable; extern unsigned int rdt_mon_features; +extern unsigned int free_rmids; extern struct list_head resctrl_schema_all; enum rdt_group_type { diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c index f136ac046851..edf8b6e1017c 100644 --- a/arch/x86/kernel/cpu/resctrl/monitor.c +++ b/arch/x86/kernel/cpu/resctrl/monitor.c @@ -51,6 +51,11 @@ static unsigned int rmid_limbo_count; */ static struct rmid_entry *rmid_ptrs; +/* + * This is the number of available RMIDs. + */ +unsigned int free_rmids; + /* * Global boolean for rdt_monitor which is true if any * resource monitoring is enabled. @@ -299,6 +304,7 @@ void __check_limbo(struct rdt_domain *d, bool force_free) if (!--entry->busy) { rmid_limbo_count--; list_add_tail(&entry->list, &rmid_free_lru); + free_rmids++; } } crmid = nrmid + 1; @@ -327,6 +333,7 @@ int alloc_rmid(void) entry = list_first_entry(&rmid_free_lru, struct rmid_entry, list); list_del(&entry->list); + free_rmids--; return entry->rmid; } @@ -362,8 +369,10 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) if (entry->busy) rmid_limbo_count++; - else + else { list_add_tail(&entry->list, &rmid_free_lru); + free_rmids++; + } } void free_rmid(u32 rmid) @@ -379,8 +388,10 @@ void free_rmid(u32 rmid) if (is_llc_occupancy_enabled()) add_rmid_to_limbo(entry); - else + else { list_add_tail(&entry->list, &rmid_free_lru); + free_rmids++; + } } static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 rmid, @@ -737,6 +748,7 @@ static int dom_data_init(struct rdt_resource *r) */ entry = __rmid_entry(0); list_del(&entry->list); + free_rmids = nr_rmids - 1; return 0; } diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 577d870ac45f..1eac0ca97b81 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -925,6 +925,15 @@ static int rdt_free_closids_show(struct kernfs_open_file *of, return 0; } +static int rdt_free_rmids_show(struct kernfs_open_file *of, + struct seq_file *seq, void *v) +{ + mutex_lock(&rdtgroup_mutex); + seq_printf(seq, "%d\n", free_rmids); + mutex_unlock(&rdtgroup_mutex); + return 0; +} + static int rdt_num_closids_show(struct kernfs_open_file *of, struct seq_file *seq, void *v) { @@ -1775,6 +1784,13 @@ static struct rftype res_common_files[] = { .seq_show = rdt_free_closids_show, .fflags = RFTYPE_TOP_INFO, }, + { + .name = "free_rmids", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdt_free_rmids_show, + .fflags = RFTYPE_TOP_INFO, + }, { .name = "num_closids", .mode = 0444, From patchwork Tue Jan 23 09:20:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haifeng Xu X-Patchwork-Id: 190800 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp214629dyi; Tue, 23 Jan 2024 01:23:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IF3g925NwxJ3af6L15jfnrmRTmwJA+ySae4h+n9PYLL1D6wId22y0qK6ri9LT4tFU3mFWi+ X-Received: by 2002:a05:6358:9386:b0:176:36c5:fdce with SMTP id h6-20020a056358938600b0017636c5fdcemr3812097rwb.63.1706001834962; Tue, 23 Jan 2024 01:23:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706001834; cv=pass; d=google.com; s=arc-20160816; b=WoFs4ZI26NQaz4znmlNrcXGXti5igrAkZ/dzn3/xJ/LEc91IBhBvEa+FP34foXd+nF Es7Hlp3JMkCgUWyc531ZmlFSYI9N9U63lM1TCFs6gMD+O1u1NGNqoz5ZnH2tMxk2g2qL V34riBUOL4UVm/nW/UxP/IjzVfmO1DO3u6OFeGPRAU/lYma6W3Y1RJVGTn7scOTNAA4+ mCkfj2R7WOZn55l+v2obA5NNgNBzsjEpnaiso6km0ShW80X97wfeGR6d+t8OKNpsGeJx gZSoyjyD4EqODm0y4qm1VD0a5uS17H3Ux06grj9iZzwb/kh8JnVuYq7rkIfpWhp9qvyu tTfQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=F8mS8d8vBcR03ufqO/xF4nBQTf7/wmjAomjX8LcNRSg=; fh=evP20v0Z2agBF6766Fg4f0yypEL+LBR7IJvWx8Uqu9c=; b=R37/2j7mG/AKgq1aHjD9bg7P9MJZY/jEbIOP++GRRx+NJyzMNdt4b+P9yYzpmAnaLs 5RVibx3kZ47R2Za7qU3P09W7jU7tYJVloK+aImrDMFR8ecKAUGE+0XeJQ6biVNNBQRJE AcgZBML9lqP220cH+iH4R5r5rZQIumUUvmNo/clux6sLp5Hjj4TQMq0BUnUx0FQwhofp 3hr5RXSZYR3wJjrBdo4+jmhkNO/hn1XKLudABLE/oUoRlG1ZBUgmyxXgZt/Rf+s7c30u EmlzSo4n/rfvJwZCcUbxUJrnvb9X48UufrDdS5TJL/ZzLce8DAmj7GMHTedCx63RQ/7I ooUg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@shopee.com header.s=shopee.com header.b=cBRGXvP2; arc=pass (i=1 spf=pass spfdomain=shopee.com dkim=pass dkdomain=shopee.com dmarc=pass fromdomain=shopee.com); spf=pass (google.com: domain of linux-kernel+bounces-34989-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-34989-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=shopee.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id le12-20020a056a004fcc00b006d9906df4ccsi11839065pfb.162.2024.01.23.01.23.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 01:23:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-34989-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@shopee.com header.s=shopee.com header.b=cBRGXvP2; arc=pass (i=1 spf=pass spfdomain=shopee.com dkim=pass dkdomain=shopee.com dmarc=pass fromdomain=shopee.com); spf=pass (google.com: domain of linux-kernel+bounces-34989-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-34989-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=shopee.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id B9E7B28C279 for ; Tue, 23 Jan 2024 09:22:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B8C585A0FF; Tue, 23 Jan 2024 09:21:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shopee.com header.i=@shopee.com header.b="cBRGXvP2" Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E5045A0E6 for ; Tue, 23 Jan 2024 09:21:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706001692; cv=none; b=MRiu8OHY4AjyYc4gajZfvTahpSlqWsMFPxfv1IUTvd7OcXoWVvlBZLpVlKfIw5yItqX6NTJrSLkTAw9h2xgqHsL8UuzTSu3GL4WtXiyMw0WTCFpteTzxlX3WDJtzJkUHFKMxWIUlKHhSgwAcEovt5AFzNVO5DhD0T29qG3R2Qnw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706001692; c=relaxed/simple; bh=kdjTVDnt3T5rMo3cn2OSR3tL1n1pa7btbCx55IWmczM=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=U9IUWW8JY5kCb42r2BfwpZYvlkdthl+qBtovmy6RMSkIehCAc4PjDDOatsoqYNXGF4l65jawS8Pe7W5OnkGRHxGqknmTmaXEIHf3ncNgbeeWoqQ0e9tR5eycX/q2oKsTt9K4lF8td5+EzwXUPg404AHMHDK+8ZXG5SR9yaLCrXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=shopee.com; spf=pass smtp.mailfrom=shopee.com; dkim=pass (2048-bit key) header.d=shopee.com header.i=@shopee.com header.b=cBRGXvP2; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=shopee.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shopee.com Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6dd6c9cb6a8so254273b3a.1 for ; Tue, 23 Jan 2024 01:21:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shopee.com; s=shopee.com; t=1706001690; x=1706606490; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=F8mS8d8vBcR03ufqO/xF4nBQTf7/wmjAomjX8LcNRSg=; b=cBRGXvP2X0oPEjeR6Psmi0aaMHXROkwDez6uRSjNTkxdN3P7pR4KrmsJuThkflFXw4 sdOwUp31IAKrMO82MxiYeqFCcV2AOGSh7xm/XM7jMc75d2B67IW+Asr5zwykuZh7mwtj EsB8En1Tkh+tJfHpyHIyltItXD0EDrcVnDDGVM8q62/nGddISw0ZvD8bYaTlvSl1IbN3 6YwX8RDWLN9oax6nKzrraqjFMOFtgIcv/Kl/cHFuyleFdC3NTQiWMCJf3p0ANwsFFGPM ENCG8O4a1HGPPJvkKG5JYPXpaleVqlivOL6L2T9k/3TczseX7MyEt6hV/ugGPgD3WW7A R4kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706001690; x=1706606490; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=F8mS8d8vBcR03ufqO/xF4nBQTf7/wmjAomjX8LcNRSg=; b=pRAr1omnYxLmxrrv1EKT6lbk0louYV3xbALnZgWE/s3gfRXVUKuCh1JVvtnxxR6Ti2 8P1Ik3C/3jGAS+MJL4cuT52tAP0RijM9MIX4S2X6s6k4bGz2lotxHmpElRtMhH94cjdi GFCMLN9z4NY/3C0/+ENHH5nvbOcZbEEWa+xzX/6MUNHZhA1Scp+rzKP8WCHRWl/GJQDG b1heg85Op+kk+bb8mzA8LwkO6bN36iseJlj7zyf+o5RfVXjrxMPAseo19LQEibm1xkmK I/CSxFXrr71Tupo51kaUhOa4IIuJ+ZzDzCt8OYYd1I6eIpShFhE6v3jwcppv82kzmmJu eTIQ== X-Gm-Message-State: AOJu0Ywg0japA6CqQ01YLEdbrEmzpeYRk86l5/dBbVzBcs44dLoxGVOc gbJt+jT7/OOAony6N6qMxmUUZ/9Kpt98kRaRwOWfxGbsu0Y6xuLyos6FHPXX1Tg= X-Received: by 2002:a05:6a00:2d95:b0:6d9:ac71:3b52 with SMTP id fb21-20020a056a002d9500b006d9ac713b52mr3796848pfb.54.1706001690073; Tue, 23 Jan 2024 01:21:30 -0800 (PST) Received: from ubuntu-haifeng.default.svc.cluster.local ([101.127.248.173]) by smtp.gmail.com with ESMTPSA id i15-20020aa787cf000000b006dbddea71e2sm3650707pfo.79.2024.01.23.01.21.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 01:21:29 -0800 (PST) From: Haifeng Xu To: reinette.chatre@intel.com Cc: fenghua.yu@intel.com, babu.moger@amd.com, peternewman@google.com, x86@kernel.org, linux-kernel@vger.kernel.org, Haifeng Xu Subject: [PATCH 3/3] x86/resctrl: Display cache occupancy of busy RMIDs Date: Tue, 23 Jan 2024 09:20:24 +0000 Message-Id: <20240123092024.1271882-3-haifeng.xu@shopee.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788872580002805489 X-GMAIL-MSGID: 1788872580002805489 If llc_occupany is enabled, the RMID may not be freed immediately unless its llc_occupany is less than the resctrl_rmid_realloc_threshold. In our production environment, those unused RMIDs get stuck in the limbo list forever because their llc_occupancy are larger than the threshold. After turning it up , we can successfully free unused RMIDs and create new monitor groups. In order to accquire the llc_occupancy of RMIDs in each rdt domain, we use perf tool to track and filter the log manually. It's not efficient enough. Therefore, we can add a RFTYPE_TOP_INFO file 'busy_rmids_info' that tells users the llc_occupancy of busy RMIDs. It can also help to guide users how much the resctrl_rmid_realloc_threshold should be. Signed-off-by: Haifeng Xu --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 70 ++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 1eac0ca97b81..88dadb87f4e1 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -32,6 +32,12 @@ #include #include "internal.h" +struct busy_rmids_info { + struct rdt_resource *r; + struct rdt_domain *d; + struct seq_file *seq; +}; + DEFINE_STATIC_KEY_FALSE(rdt_enable_key); DEFINE_STATIC_KEY_FALSE(rdt_mon_enable_key); DEFINE_STATIC_KEY_FALSE(rdt_alloc_enable_key); @@ -934,6 +940,63 @@ static int rdt_free_rmids_show(struct kernfs_open_file *of, return 0; } +void rdt_domain_busy_rmids_show(void *info) +{ + + struct rdt_resource *r; + struct rdt_domain *d; + struct seq_file *seq; + struct busy_rmids_info *rmids_info = info; + u32 crmid = 1, nrmid; + u64 val; + int ret; + + r = rmids_info->r; + d = rmids_info->d; + seq = rmids_info->seq; + + seq_printf(seq, "domain-%d busy rmids.\n", d->id); + + for (;;) { + nrmid = find_next_bit(d->rmid_busy_llc, r->num_rmid, crmid); + if (nrmid >= r->num_rmid) + break; + + ret = resctrl_arch_rmid_read(r, d, nrmid, QOS_L3_OCCUP_EVENT_ID, &val); + switch (ret) { + case -EIO: + seq_printf(seq, "I/O Error\n"); + return; + case -EINVAL: + seq_printf(seq, "Invalid Argument\n"); + return; + default: + seq_printf(seq, "rmid:%d llc_occupancy:%llu\n", nrmid, val); + } + crmid = nrmid + 1; + } +} + +static int rdt_busy_rmids_info_show(struct kernfs_open_file *of, + struct seq_file *seq, void *v) +{ + + struct rdt_domain *d; + struct rdt_resource *r; + struct busy_rmids_info info; + + mutex_lock(&rdtgroup_mutex); + r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl; + info.r = r; + info.seq = seq; + list_for_each_entry(d, &r->domains, list) { + info.d = d; + smp_call_function_any(&d->cpu_mask, rdt_domain_busy_rmids_show, &info, 1); + } + mutex_unlock(&rdtgroup_mutex); + return 0; +} + static int rdt_num_closids_show(struct kernfs_open_file *of, struct seq_file *seq, void *v) { @@ -1791,6 +1854,13 @@ static struct rftype res_common_files[] = { .seq_show = rdt_free_rmids_show, .fflags = RFTYPE_TOP_INFO, }, + { + .name = "busy_rmids_info", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdt_busy_rmids_info_show, + .fflags = RFTYPE_TOP_INFO, + }, { .name = "num_closids", .mode = 0444,