From patchwork Fri Nov 4 20:01:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Moger, Babu" X-Patchwork-Id: 15773 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp609946wru; Fri, 4 Nov 2022 13:02:56 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5CAHSe6Wp69Jye38Zjttro9+qgQv+wfqNT6uwD5NSyrq0EBbdxbcLQDV+ZpH5jl0wqaojT X-Received: by 2002:a63:69c2:0:b0:46a:eeb1:e784 with SMTP id e185-20020a6369c2000000b0046aeeb1e784mr31476605pgc.589.1667592175823; Fri, 04 Nov 2022 13:02:55 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1667592175; cv=pass; d=google.com; s=arc-20160816; b=Hmyurnny39obW/W+hgVcAhC4kUSXjZdOaNVg8xxgVNAiC/m2zJ6Vtw+8Bl0hkFI4yi 48z6w9K4TLp0UK3RL7nhLZ4pAJMe97PtNZlQqMJRlz5LgpU2DCChqHxJRIV/YWZIJ7kg OWtMkOC6xpMIcKq/aR9fJD72SQyvPsK6nh2pQ4llzb/X2on370AQXgeqDIuHjWXB3Jxa g2qcQiosSqjPvFZWxaTN3FYdhRjRFBASnWAw27BVLJgfXxmgsRpXTC0XUqrhdaF4uyJR v7a1gGyuMKS+uh5/1rrLYZmQKkscoplnEVn/KHnUTz4srF5FGk5ViyPppfFTJ8wfqFRD 7Dow== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:cc:to:from :subject:dkim-signature; bh=EhPsXuc7Ix2Dzi+ZlbtEA0mEhLgjscPZHWQv6Fb9h2g=; b=oQXeCBC6Z1Plm3pG2TpXIRAXhNF06RihfNK6Z7QFujHbQp3rovrOetXCUvV/tJMxx/ RS4nuEj3SlY6ymR4adrj4VwR0rJ3SMXhv52pgqeTsNq6gKjNq/bbGNPMPxq0w0DyHFpr ieYyKiEXRwJspEkgSz+jGyVEfaFftV2WpIRhvUznzIqLLQgPJiPpNd01B1uBl7zTTi1J RcIkMkUeXSGewEUKVkMOprOqgqk7P8yh8CNrm28wH9s8K63w924hPc8uAUjx4WX5UNI8 of6pktAQjcEYXpbMHO30lVos2IE/NBP/utZCXtYLY+hZAPdwXxDZOn5BeguMZcjWa7gq 6FrA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b="H2wC0/cB"; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m11-20020a17090a858b00b002135e1f5a7bsi4080822pjn.155.2022.11.04.13.02.34; Fri, 04 Nov 2022 13:02:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b="H2wC0/cB"; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229961AbiKDUB6 (ORCPT + 99 others); Fri, 4 Nov 2022 16:01:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229933AbiKDUBV (ORCPT ); Fri, 4 Nov 2022 16:01:21 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2050.outbound.protection.outlook.com [40.107.100.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EE2C45A37; Fri, 4 Nov 2022 13:01:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T17cwrFzi0/pI0U4J3jC62qPpKJE/EaxOyEITknh1IJ7OGu9x44gUx28MS8eofnOTny3dEkROI43AzfQeR04qfOFOkFNH03toAlVUeMX1OiGHWuDCgKgDYfnoyRl2v5esJraBz9lGftw+tib2T9zXUZhg8T2j8admMMZM9KST4sZHEy18rAcSEd1u/PvW0xP3xEQ+y1PjbZh/7oe4GNOiX7XG7R7h2xqCu+hYgjEJd1kFz78dxcBBWRuHLO1hd+bftTFyY8qT91v3m5HqPzo9zNwK3e/3w1lM6hhU76sB/mY4CfWdfuMXPy9uuYpRvU4CfR3ouZ18ZglLK5j3UkYDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EhPsXuc7Ix2Dzi+ZlbtEA0mEhLgjscPZHWQv6Fb9h2g=; b=ngynHBVA4AzA/Zo7QlN9TfHPCitGyDF7xyG4Grt4eE53ix+bKb7KTDLgFN7Dqnci03Ryn1eSrhk4VR+CUXTnBkvMpj1CyUIU/46Q/FOuzwFXHLOoGfZo15ISdmCdcBWUVgR9599mctlnOMe+ia/tIj9BgmL8xYP+GlOGSQTLZE0Qyiws1uv+J5H75EhPuoP6eeERBB144dIs1DochFJDcnPsDKEktEJVE5PG77mU8VUYWENiwKkb9haHEpCWJRraZ3GFboc673C35NQ8l6eqUBjIPAUmYmjJ/6feiiyN4SZBRRVCMgc7L4bWcx8H0Ko3k3S0WtDFnhi8rOUJAdygbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=quicinc.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EhPsXuc7Ix2Dzi+ZlbtEA0mEhLgjscPZHWQv6Fb9h2g=; b=H2wC0/cBIahPQjilhd+6Qy+JhPqxmHEmqnL9z9ENOHK2LxHoZeR7PlwiYXIdcXuv+DCfId69ax+291lkONczQX0mWq3albrLb7Frrt/Li7hmubh1huw/pihsU4b4/yLuvONNc0x6IKvrl8E+pHP1YUWUtbeMx0IpHDbYkGHA5k0= Received: from DS7PR05CA0035.namprd05.prod.outlook.com (2603:10b6:8:2f::18) by DM6PR12MB5007.namprd12.prod.outlook.com (2603:10b6:5:20d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22; Fri, 4 Nov 2022 20:01:12 +0000 Received: from DM6NAM11FT100.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2f:cafe::60) by DS7PR05CA0035.outlook.office365.com (2603:10b6:8:2f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.8 via Frontend Transport; Fri, 4 Nov 2022 20:01:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT100.mail.protection.outlook.com (10.13.172.247) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5791.20 via Frontend Transport; Fri, 4 Nov 2022 20:01:12 +0000 Received: from [127.0.1.1] (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 4 Nov 2022 15:01:09 -0500 Subject: [PATCH v8 10/13] x86/resctrl: Add sysfs interface to write mbm_total_bytes_config From: Babu Moger To: , , , , CC: , , , , , , , , , , , , , , , , , , , , , , , , Date: Fri, 4 Nov 2022 15:01:09 -0500 Message-ID: <166759206900.3281208.11975514088019160962.stgit@bmoger-ubuntu> In-Reply-To: <166759188265.3281208.11769277079826754455.stgit@bmoger-ubuntu> References: <166759188265.3281208.11769277079826754455.stgit@bmoger-ubuntu> User-Agent: StGit/1.1.dev103+g5369f4c MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT100:EE_|DM6PR12MB5007:EE_ X-MS-Office365-Filtering-Correlation-Id: d1ee638a-3275-480d-26df-08dabe9f52ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hG9n9cAnSqeTnoVfSyZ4ZqXfNvZ7C3UCGLGEyeZUaJtDIYSGb8c0jEc/KmDh392cfyVhuIqf86ufCG+kTfVimswHTczsj+GbZGUerEmv46633lb67/0Q6lF90quRw8u2jYvWa59y8g3rrSqQv3q9+sty302zTkSQCsP2LNmVxtV1THl7HGOvhJQ0pQxuWPlFo47Mcx1SzEfi432BuGuTLMH4QkRTg0+7+zqtwkggCckVHqZYEQZOL9bwTy/mX58mGHic5UIIjEg6EQRjSxuQjuHODeWfp//JNdq8DD6jpsAr/+ypiRCa4/QnxKD7Q1FPqSuxYM4oiYdE65D5nYevtmAkGt1cGRcYVTGOXVzmpEWbtY4aHWeivyk8zvI1dz+mnoUzAjrPfAbbvP85QakQbjPyQlw+4u5Cwd1aNqcELG+Z7UP6D5sm8lrH8UUFiEwd9FSJrM8fKNr+U7QVN9r+hhOzaQLsA3+DdY8bNHIK6KBK4E5iCd6Ga+KY5IqvNT2UoG20QwSnbLinufEGNvik1o13v5NO8FY+Iv68nDOwbrFd1fkYZXcNiGvYL9O4r2xgEqbhSt2OWPCnIWBjkqWKk1ppok9HKoEa+K325Wu3rJh7ZUpM8yDOwZzWLtbFOwp6yDFoYYAKOoJjDmNt0xKf7mwQlXNcUHmTwvoU48L2s6C9LCsI+tcC8GIQiUKJ/urM4YunhiOOHKFyHYKTu3P4+C4sqGJfCGXhN1CpHz4TmlWDUKSVv3DtL3PCvJARVmA93+kqgLve+fuEOvkh4g2ivV7xlBpDyDqO1uaeSBMx8j87gF9cWP5gDlrD5DHpO2vysXHrBYm5F5SUVmbC22XtoA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(7916004)(4636009)(376002)(396003)(346002)(136003)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(83380400001)(70586007)(8676002)(4326008)(70206006)(356005)(47076005)(86362001)(426003)(40480700001)(316002)(103116003)(33716001)(16576012)(40460700003)(54906003)(2906002)(81166007)(82310400005)(36860700001)(82740400003)(110136005)(8936002)(41300700001)(478600001)(9686003)(26005)(7416002)(5660300002)(16526019)(186003)(44832011)(336012)(71626007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Nov 2022 20:01:12.3778 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d1ee638a-3275-480d-26df-08dabe9f52ca X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT100.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB5007 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748597133675062377?= X-GMAIL-MSGID: =?utf-8?q?1748597133675062377?= The current event configuration for mbm_total_bytes can be changed by the user by writing to the file /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config. The event configuration settings are domain specific and will affect all the CPUs in the domain. Following are the types of events supported: ==== =========================================================== Bits Description ==== =========================================================== 6 Dirty Victims from the QOS domain to all types of memory 5 Reads to slow memory in the non-local NUMA domain 4 Reads to slow memory in the local NUMA domain 3 Non-temporal writes to non-local NUMA domain 2 Non-temporal writes to local NUMA domain 1 Reads to memory in the non-local NUMA domain 0 Reads to memory in the local NUMA domain ==== =========================================================== For example: To change the mbm_total_bytes to count only reads on domain 0, the bits 0, 1, 4 and 5 needs to be set, which is 110011b (in hex 0x33). Run the command. $echo 0=0x33 > /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config To change the mbm_total_bytes to count all the slow memory reads on domain 1, the bits 4 and 5 needs to be set which is 110000b (in hex 0x30). Run the command. $echo 1=0x30 > /sys/fs/resctrl/info/L3_MON/mbm_total_bytes_config Signed-off-by: Babu Moger --- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 130 ++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c index 18f9588a41cf..0cdccb69386e 100644 --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c @@ -1505,6 +1505,133 @@ static int mbm_local_bytes_config_show(struct kernfs_open_file *of, return 0; } +static void mon_event_config_write(void *info) +{ + struct mon_config_info *mon_info = info; + u32 index; + + index = mon_event_config_index_get(mon_info->evtid); + if (index >= MAX_CONFIG_EVENTS) { + pr_warn_once("Invalid event id %d\n", mon_info->evtid); + return; + } + wrmsr(MSR_IA32_EVT_CFG_BASE + index, mon_info->mon_config, 0); +} + +static int mbm_config_write(struct rdt_resource *r, struct rdt_domain *d, + u32 evtid, u32 val) +{ + struct mon_config_info mon_info = {0}; + int ret = 0; + + rdt_last_cmd_clear(); + + /* mon_config cannot be more than the supported set of events */ + if (val > MAX_EVT_CONFIG_BITS) { + rdt_last_cmd_puts("Invalid event configuration\n"); + return -EINVAL; + } + + /* + * Read the current config value first. If both are same then + * we don't need to write it again. + */ + mon_info.evtid = evtid; + mondata_config_read(d, &mon_info); + if (mon_info.mon_config == val) + goto write_exit; + + mon_info.mon_config = val; + + /* + * Update MSR_IA32_EVT_CFG_BASE MSRs on all the CPUs in the + * domain. The MSRs offset from MSR MSR_IA32_EVT_CFG_BASE + * are scoped at the domain level. Writing any of these MSRs + * on one CPU is supposed to be observed by all CPUs in the + * domain. However, the hardware team recommends to update + * these MSRs on all the CPUs in the domain. + */ + on_each_cpu_mask(&d->cpu_mask, mon_event_config_write, &mon_info, 1); + + /* + * When an Event Configuration is changed, the bandwidth counters + * for all RMIDs and Events will be cleared by the hardware. The + * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for + * every RMID on the next read to any event for every RMID. + * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62) + * cleared while it is tracked by the hardware. Clear the + * mbm_local and mbm_total counts for all the RMIDs. + */ + memset(d->mbm_local, 0, sizeof(struct mbm_state) * r->num_rmid); + memset(d->mbm_total, 0, sizeof(struct mbm_state) * r->num_rmid); + +write_exit: + return ret; +} + +static int mon_config_parse(struct rdt_resource *r, char *tok, u32 evtid) +{ + char *dom_str = NULL, *id_str; + unsigned long dom_id, val; + struct rdt_domain *d; + int ret = 0; + +next: + if (!tok || tok[0] == '\0') + return 0; + + /* Start processing the strings for each domain */ + dom_str = strim(strsep(&tok, ";")); + id_str = strsep(&dom_str, "="); + + if (!dom_str || kstrtoul(id_str, 10, &dom_id)) { + rdt_last_cmd_puts("Missing '=' or non-numeric domain id\n"); + return -EINVAL; + } + + if (!dom_str || kstrtoul(dom_str, 16, &val)) { + rdt_last_cmd_puts("Missing '=' or non-numeric event configuration value\n"); + return -EINVAL; + } + + list_for_each_entry(d, &r->domains, list) { + if (d->id == dom_id) { + ret = mbm_config_write(r, d, evtid, val); + if (ret) + return -EINVAL; + goto next; + } + } + + return -EINVAL; +} + +static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, + loff_t off) +{ + struct rdt_resource *r = of->kn->parent->priv; + int ret; + + /* Valid input requires a trailing newline */ + if (nbytes == 0 || buf[nbytes - 1] != '\n') + return -EINVAL; + + cpus_read_lock(); + mutex_lock(&rdtgroup_mutex); + + rdt_last_cmd_clear(); + + buf[nbytes - 1] = '\0'; + + ret = mon_config_parse(r, buf, QOS_L3_MBM_TOTAL_EVENT_ID); + + mutex_unlock(&rdtgroup_mutex); + cpus_read_unlock(); + + return ret ?: nbytes; +} + /* rdtgroup information files for one cache resource. */ static struct rftype res_common_files[] = { { @@ -1605,9 +1732,10 @@ static struct rftype res_common_files[] = { }, { .name = "mbm_total_bytes_config", - .mode = 0444, + .mode = 0644, .kf_ops = &rdtgroup_kf_single_ops, .seq_show = mbm_total_bytes_config_show, + .write = mbm_total_bytes_config_write, }, { .name = "mbm_local_bytes_config",