From patchwork Thu Oct 5 05:23:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandipan Das X-Patchwork-Id: 148907 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp425785vqb; Thu, 5 Oct 2023 09:41:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEb8a2gC3st90StE/Rr4OyCPnbkuZaGE5BI/ARvivE3isEoF6rT3dW/gR1H+L++vNaSk0RV X-Received: by 2002:a17:902:e746:b0:1c7:3224:9137 with SMTP id p6-20020a170902e74600b001c732249137mr6136823plf.6.1696524109881; Thu, 05 Oct 2023 09:41:49 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1696524109; cv=pass; d=google.com; s=arc-20160816; b=VgsjyF2O8envREr3qRY99wlZdsq38bfgn96PFORUMQ2R/VsWUa27Tv9GEijmT8Ifo+ +foTrEKSrsLZ0E3116RgHkXLcy0x9l72u29/6cP9fgOmYLd9RSdEVRQEWVlyw50MK6jX 48goi32GYlRP2vsagyH/nkgCOE3Zd7mhhuAzQucBtJr7AvXGV25vZFuafuvdLasTDJaJ 3rA0uHZK9h5XXpV00sTB/Odt8ECF6LddXALKoSZ10dSR8jWxbo4/cjXmbwzSsL/HqdMK A4iP/PkRVl2cCGEuqO/LWPoGSQJvVEtJ3zxf/9cQV8JMyQtQfrEcAnozgEIkqGmLtzCm xPmQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0BXSBwuJ4YsRWA1Xrxys8uIlw2CPbb6D1MSI86imjQw=; fh=DHqst7/L7qcDaGO45kwMD1/18OMA+IdBCna8yDZwwes=; b=XtISbDlgT6NO2DQ2kqMDcLMZNxc+96QN4+qBb3mda4wVLeNz16H1DkQNNsGrBwBcMD 92zKXvsdhHDzVRwwRiskuKlnh0cgTzMPYnc+nC6ZzCMwPerRi7DmFnsmo03/SBTpmZkC KakbxM40mjSTzGuV5t/YPsD4a2BwjbdXJSzaXuHHhje2QpjBpPE6A7dXHkPlKNdoK5xI CsIawn4WifPIO/0NH9zJGJzaD3MmdiXE2POiNgIlezoWx9zJxsJTABqii19+uHVcl61U rbEiJ5ehtIKV3vp8iJnMhG0M0ov/XOErYQam8pGOjsqwHhrGqje5sO6i+RCa8fGBNwp2 ZAJQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b="o/+DaQYZ"; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id e14-20020a170902cf4e00b001b7d2b55d8asi1696446plg.626.2023.10.05.09.41.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 09:41:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b="o/+DaQYZ"; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 0D8CC80C0DE0; Thu, 5 Oct 2023 09:41:20 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233761AbjJEQks (ORCPT + 18 others); Thu, 5 Oct 2023 12:40:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234813AbjJEQjV (ORCPT ); Thu, 5 Oct 2023 12:39:21 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2061f.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::61f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EF564ED7; Wed, 4 Oct 2023 22:24:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nRKWCSDBFzPZB1v4TvggdPSlp0osBhYz6APVeG+57IdL8ohOTmy2pIB5xq7Vd/eEhXrr7+BWpI78A2cLPQ1lWjEfFQCHH1el/IdsisOH5pO3/trEW0H5XZxtXhUF0TdqcGzAspVdKNxsAD7dEc+fcZFWLPI1ozCbqBZSxqBMTFKNl+YjGFiWn9a7Q7ucxYUpNaKEHx2NFmQrTEI0NeSJtaNAtGUr0w07YG82JwEljaIEAWVNVolEUXklamcHSetiroOLGqkR49odndOId+a6sTn+d9r60zcwr58GiwDCsV6zDnRSTy4Qhe3f+nFTKlaSEqfSMtqj9CJhY6VOx6DZAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0BXSBwuJ4YsRWA1Xrxys8uIlw2CPbb6D1MSI86imjQw=; b=Ei+PuaqHxy5CitsWzMOktmrzWOq0+/Czxf/4sSNw92c4rWKYgKOphQ0wVNf5ZxEGIgBCC5HvwCnvqpTHfAhI54PC7dUHVk+wJU//cEotGGOAjJayAdjY4WoG72p2ABvqEQZA6SJ+Zn4taa1wzrMWnzda6OslP2s2CtoyB2Yr9W2vjM14GFGbGbXZ3PNI9O18Xl5Jk/U8BYbSXVUgd9XVhVeIUk59k8fm9C6uMHxC3Efqmk0/QKz8sc1TPG7Or9LZMRNDR8VibP8x8JPT63nEE17+vae+S+M6wrsd+MuY2ZQ6Qkc+6/qulr4HmfS8IEk6+Idl0qIgI8lQwqEZ7prBZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0BXSBwuJ4YsRWA1Xrxys8uIlw2CPbb6D1MSI86imjQw=; b=o/+DaQYZv2/NHkRSV/OK5DU5Vc6BIU2Rofd7ydjgQimzgX913acrHuP09PVSyBSAm/JCLEiS/9X6mkWSwP1Ek5fg+9IWGDS/NXP8AzSdK2ZeUOZAoiBwrFLF5gh4xUvwlqoGkuvWBPXBAdO+7u8WFFTwHtHHwvVGMxN0wpKdfLg= Received: from BLAPR03CA0177.namprd03.prod.outlook.com (2603:10b6:208:32f::30) by MN2PR12MB4334.namprd12.prod.outlook.com (2603:10b6:208:1d1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.34; Thu, 5 Oct 2023 05:24:17 +0000 Received: from BL6PEPF0001AB4D.namprd04.prod.outlook.com (2603:10b6:208:32f:cafe::23) by BLAPR03CA0177.outlook.office365.com (2603:10b6:208:32f::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.25 via Frontend Transport; Thu, 5 Oct 2023 05:24:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4D.mail.protection.outlook.com (10.167.242.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.14 via Frontend Transport; Thu, 5 Oct 2023 05:24:17 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 00:24:10 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH v2 1/6] perf/x86/amd/uncore: Refactor uncore management Date: Thu, 5 Oct 2023 10:53:11 +0530 Message-ID: <24b38c49a5dae65d8c96e5d75a2b96ae97aaa651.1696425185.git.sandipan.das@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4D:EE_|MN2PR12MB4334:EE_ X-MS-Office365-Filtering-Correlation-Id: ffe175f6-113d-4cb2-2d20-08dbc5635263 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4i0KpmIjsy5sRL87qalO3Z/2DF0/2lpVAYXs3ixlSHnD5gMovD6x15/5FAhVfvzZ6jS6lpO+wq2tqF/KEj2LlNqJaAmYyjilL4QRdrn9I2mQ56xwUktJxtta3wJVgk9rQDl3JAgHQpyu0DKIniYwCch+wYcR1I/GVIIKKRUjvbRPgNnnxSru2AB1qvQhGADs4LNVKNXteSd5E3RlevP3uIQsbftr2Pmsh0/+PkA22vD4QlzfcucYBZVbAAE5oY0unLRxhQQU9G3OKymr4FTeZtDphCBVVMJsXIf7nfUl3gZVnlfnvifpdhy8s8cbtn/vDv6iqgNb2otiPo1/NoYN1/OhLIkr0ZF1fXslQcwt79fWxP+TVEnU3RsnePXEc/UPIwxBL9jUNlf0H1j9LToeDsxlmw4HiNF2yXzrFPs1dSnN3+1w8oMbiD+RvVSsyzd3Oeg4R1lHiiAe4Hro/iX8x8s+MbUUcxO3/QhzRCb9tLpjAu45zudclwRc6v7jdcqY88PhgUZngJ6EuBzxfxaHj6NUwXhtQInqCX/5WMwquhVP5wX/w3OqWExiXv3AB5hk6yE2LpcbsFe0AIu3fVbVtulvcmETp1stpM7RPuE/R1VH0bUqrs3EXzrjbUFb268ZwJXMGsUfNevGRMHcSYI/AmzSiYDXR4XDkDVs+8akOeFEkX23KcT3nkM/ybCs7nkqgKJ7ij1CGzKNbl9G3dSmD/AVLiUe5MF6MiiPK9Aku6bR3cEC5QoaKYF4FvqWN+6q+yO+qrzLkMpo70adPXjFYA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(136003)(376002)(230922051799003)(82310400011)(1800799009)(186009)(64100799003)(451199024)(36840700001)(46966006)(40470700004)(36756003)(82740400003)(7696005)(81166007)(110136005)(86362001)(478600001)(356005)(54906003)(316002)(70586007)(41300700001)(2616005)(26005)(16526019)(70206006)(8936002)(4326008)(8676002)(336012)(5660300002)(426003)(83380400001)(44832011)(30864003)(40460700003)(47076005)(36860700001)(2906002)(40480700001)(7416002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 05:24:17.7253 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ffe175f6-113d-4cb2-2d20-08dbc5635263 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4D.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4334 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_PASS,SPF_NONE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 05 Oct 2023 09:41:20 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778934465491505823 X-GMAIL-MSGID: 1778934465491505823 Since struct amd_uncore is used to manage per-cpu contexts, rename it to amd_uncore_ctx in order to better reflect its purpose. Add a new struct amd_uncore_pmu to encapsulate all attributes which are shared by per-cpu contexts for a corresponding PMU. These include the number of counters, active mask, MSR and RDPMC base addresses, etc. Since the struct pmu is now embedded, the corresponding amd_uncore_pmu for a given event can be found by simply using container_of(). Finally, move all PMU-specific code to separate functions. While the original event management functions continue to provide the base functionality, all PMU-specific quirks and customizations are applied in separate functions. The motivation is to simplify the management of uncore PMUs. Signed-off-by: Sandipan Das --- arch/x86/events/amd/uncore.c | 737 ++++++++++++++++++----------------- 1 file changed, 379 insertions(+), 358 deletions(-) diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c index 83f15fe411b3..ffcecda13d65 100644 --- a/arch/x86/events/amd/uncore.c +++ b/arch/x86/events/amd/uncore.c @@ -27,56 +27,41 @@ #define COUNTER_SHIFT 16 +#define NUM_UNCORES_MAX 2 /* DF (or NB) and L3 (or L2) */ +#define UNCORE_NAME_LEN 16 + #undef pr_fmt #define pr_fmt(fmt) "amd_uncore: " fmt static int pmu_version; -static int num_counters_llc; -static int num_counters_nb; -static bool l3_mask; static HLIST_HEAD(uncore_unused_list); -struct amd_uncore { +struct amd_uncore_ctx { int id; int refcnt; int cpu; - int num_counters; - int rdpmc_base; - u32 msr_base; - cpumask_t *active_mask; - struct pmu *pmu; struct perf_event **events; struct hlist_node node; }; -static struct amd_uncore * __percpu *amd_uncore_nb; -static struct amd_uncore * __percpu *amd_uncore_llc; - -static struct pmu amd_nb_pmu; -static struct pmu amd_llc_pmu; - -static cpumask_t amd_nb_active_mask; -static cpumask_t amd_llc_active_mask; - -static bool is_nb_event(struct perf_event *event) -{ - return event->pmu->type == amd_nb_pmu.type; -} +struct amd_uncore_pmu { + char name[UNCORE_NAME_LEN]; + int num_counters; + int rdpmc_base; + u32 msr_base; + cpumask_t active_mask; + struct pmu pmu; + struct amd_uncore_ctx * __percpu *ctx; + int (*id)(unsigned int cpu); +}; -static bool is_llc_event(struct perf_event *event) -{ - return event->pmu->type == amd_llc_pmu.type; -} +static struct amd_uncore_pmu pmus[NUM_UNCORES_MAX]; +static int num_pmus __read_mostly; -static struct amd_uncore *event_to_amd_uncore(struct perf_event *event) +static struct amd_uncore_pmu *event_to_amd_uncore_pmu(struct perf_event *event) { - if (is_nb_event(event) && amd_uncore_nb) - return *per_cpu_ptr(amd_uncore_nb, event->cpu); - else if (is_llc_event(event) && amd_uncore_llc) - return *per_cpu_ptr(amd_uncore_llc, event->cpu); - - return NULL; + return container_of(event->pmu, struct amd_uncore_pmu, pmu); } static void amd_uncore_read(struct perf_event *event) @@ -118,7 +103,7 @@ static void amd_uncore_stop(struct perf_event *event, int flags) hwc->state |= PERF_HES_STOPPED; if ((flags & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) { - amd_uncore_read(event); + event->pmu->read(event); hwc->state |= PERF_HES_UPTODATE; } } @@ -126,15 +111,16 @@ static void amd_uncore_stop(struct perf_event *event, int flags) static int amd_uncore_add(struct perf_event *event, int flags) { int i; - struct amd_uncore *uncore = event_to_amd_uncore(event); + struct amd_uncore_pmu *pmu = event_to_amd_uncore_pmu(event); + struct amd_uncore_ctx *ctx = *per_cpu_ptr(pmu->ctx, event->cpu); struct hw_perf_event *hwc = &event->hw; /* are we already assigned? */ - if (hwc->idx != -1 && uncore->events[hwc->idx] == event) + if (hwc->idx != -1 && ctx->events[hwc->idx] == event) goto out; - for (i = 0; i < uncore->num_counters; i++) { - if (uncore->events[i] == event) { + for (i = 0; i < pmu->num_counters; i++) { + if (ctx->events[i] == event) { hwc->idx = i; goto out; } @@ -142,8 +128,8 @@ static int amd_uncore_add(struct perf_event *event, int flags) /* if not, take the first available counter */ hwc->idx = -1; - for (i = 0; i < uncore->num_counters; i++) { - if (cmpxchg(&uncore->events[i], NULL, event) == NULL) { + for (i = 0; i < pmu->num_counters; i++) { + if (cmpxchg(&ctx->events[i], NULL, event) == NULL) { hwc->idx = i; break; } @@ -153,23 +139,13 @@ static int amd_uncore_add(struct perf_event *event, int flags) if (hwc->idx == -1) return -EBUSY; - hwc->config_base = uncore->msr_base + (2 * hwc->idx); - hwc->event_base = uncore->msr_base + 1 + (2 * hwc->idx); - hwc->event_base_rdpmc = uncore->rdpmc_base + hwc->idx; + hwc->config_base = pmu->msr_base + (2 * hwc->idx); + hwc->event_base = pmu->msr_base + 1 + (2 * hwc->idx); + hwc->event_base_rdpmc = pmu->rdpmc_base + hwc->idx; hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED; - /* - * The first four DF counters are accessible via RDPMC index 6 to 9 - * followed by the L3 counters from index 10 to 15. For processors - * with more than four DF counters, the DF RDPMC assignments become - * discontiguous as the additional counters are accessible starting - * from index 16. - */ - if (is_nb_event(event) && hwc->idx >= NUM_COUNTERS_NB) - hwc->event_base_rdpmc += NUM_COUNTERS_L3; - if (flags & PERF_EF_START) - amd_uncore_start(event, PERF_EF_RELOAD); + event->pmu->start(event, PERF_EF_RELOAD); return 0; } @@ -177,55 +153,36 @@ static int amd_uncore_add(struct perf_event *event, int flags) static void amd_uncore_del(struct perf_event *event, int flags) { int i; - struct amd_uncore *uncore = event_to_amd_uncore(event); + struct amd_uncore_pmu *pmu = event_to_amd_uncore_pmu(event); + struct amd_uncore_ctx *ctx = *per_cpu_ptr(pmu->ctx, event->cpu); struct hw_perf_event *hwc = &event->hw; - amd_uncore_stop(event, PERF_EF_UPDATE); + event->pmu->stop(event, PERF_EF_UPDATE); - for (i = 0; i < uncore->num_counters; i++) { - if (cmpxchg(&uncore->events[i], event, NULL) == event) + for (i = 0; i < pmu->num_counters; i++) { + if (cmpxchg(&ctx->events[i], event, NULL) == event) break; } hwc->idx = -1; } -/* - * Return a full thread and slice mask unless user - * has provided them - */ -static u64 l3_thread_slice_mask(u64 config) -{ - if (boot_cpu_data.x86 <= 0x18) - return ((config & AMD64_L3_SLICE_MASK) ? : AMD64_L3_SLICE_MASK) | - ((config & AMD64_L3_THREAD_MASK) ? : AMD64_L3_THREAD_MASK); - - /* - * If the user doesn't specify a threadmask, they're not trying to - * count core 0, so we enable all cores & threads. - * We'll also assume that they want to count slice 0 if they specify - * a threadmask and leave sliceid and enallslices unpopulated. - */ - if (!(config & AMD64_L3_F19H_THREAD_MASK)) - return AMD64_L3_F19H_THREAD_MASK | AMD64_L3_EN_ALL_SLICES | - AMD64_L3_EN_ALL_CORES; - - return config & (AMD64_L3_F19H_THREAD_MASK | AMD64_L3_SLICEID_MASK | - AMD64_L3_EN_ALL_CORES | AMD64_L3_EN_ALL_SLICES | - AMD64_L3_COREID_MASK); -} - static int amd_uncore_event_init(struct perf_event *event) { - struct amd_uncore *uncore; + struct amd_uncore_pmu *pmu; + struct amd_uncore_ctx *ctx; struct hw_perf_event *hwc = &event->hw; - u64 event_mask = AMD64_RAW_EVENT_MASK_NB; if (event->attr.type != event->pmu->type) return -ENOENT; - if (pmu_version >= 2 && is_nb_event(event)) - event_mask = AMD64_PERFMON_V2_RAW_EVENT_MASK_NB; + if (event->cpu < 0) + return -EINVAL; + + pmu = event_to_amd_uncore_pmu(event); + ctx = *per_cpu_ptr(pmu->ctx, event->cpu); + if (!ctx) + return -ENODEV; /* * NB and Last level cache counters (MSRs) are shared across all cores @@ -235,28 +192,14 @@ static int amd_uncore_event_init(struct perf_event *event) * out. So we do not support sampling and per-thread events via * CAP_NO_INTERRUPT, and we do not enable counter overflow interrupts: */ - hwc->config = event->attr.config & event_mask; + hwc->config = event->attr.config; hwc->idx = -1; - if (event->cpu < 0) - return -EINVAL; - - /* - * SliceMask and ThreadMask need to be set for certain L3 events. - * For other events, the two fields do not affect the count. - */ - if (l3_mask && is_llc_event(event)) - hwc->config |= l3_thread_slice_mask(event->attr.config); - - uncore = event_to_amd_uncore(event); - if (!uncore) - return -ENODEV; - /* * since request can come in to any of the shared cores, we will remap * to a single common cpu. */ - event->cpu = uncore->cpu; + event->cpu = ctx->cpu; return 0; } @@ -278,17 +221,10 @@ static ssize_t amd_uncore_attr_show_cpumask(struct device *dev, struct device_attribute *attr, char *buf) { - cpumask_t *active_mask; - struct pmu *pmu = dev_get_drvdata(dev); - - if (pmu->type == amd_nb_pmu.type) - active_mask = &amd_nb_active_mask; - else if (pmu->type == amd_llc_pmu.type) - active_mask = &amd_llc_active_mask; - else - return 0; + struct pmu *ptr = dev_get_drvdata(dev); + struct amd_uncore_pmu *pmu = container_of(ptr, struct amd_uncore_pmu, pmu); - return cpumap_print_to_pagebuf(true, buf, active_mask); + return cpumap_print_to_pagebuf(true, buf, &pmu->active_mask); } static DEVICE_ATTR(cpumask, S_IRUGO, amd_uncore_attr_show_cpumask, NULL); @@ -396,113 +332,57 @@ static const struct attribute_group *amd_uncore_l3_attr_update[] = { NULL, }; -static struct pmu amd_nb_pmu = { - .task_ctx_nr = perf_invalid_context, - .attr_groups = amd_uncore_df_attr_groups, - .name = "amd_nb", - .event_init = amd_uncore_event_init, - .add = amd_uncore_add, - .del = amd_uncore_del, - .start = amd_uncore_start, - .stop = amd_uncore_stop, - .read = amd_uncore_read, - .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, - .module = THIS_MODULE, -}; - -static struct pmu amd_llc_pmu = { - .task_ctx_nr = perf_invalid_context, - .attr_groups = amd_uncore_l3_attr_groups, - .attr_update = amd_uncore_l3_attr_update, - .name = "amd_l2", - .event_init = amd_uncore_event_init, - .add = amd_uncore_add, - .del = amd_uncore_del, - .start = amd_uncore_start, - .stop = amd_uncore_stop, - .read = amd_uncore_read, - .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, - .module = THIS_MODULE, -}; - -static struct amd_uncore *amd_uncore_alloc(unsigned int cpu) -{ - return kzalloc_node(sizeof(struct amd_uncore), GFP_KERNEL, - cpu_to_node(cpu)); -} - -static inline struct perf_event ** -amd_uncore_events_alloc(unsigned int num, unsigned int cpu) -{ - return kzalloc_node(sizeof(struct perf_event *) * num, GFP_KERNEL, - cpu_to_node(cpu)); -} - static int amd_uncore_cpu_up_prepare(unsigned int cpu) { - struct amd_uncore *uncore_nb = NULL, *uncore_llc = NULL; - - if (amd_uncore_nb) { - *per_cpu_ptr(amd_uncore_nb, cpu) = NULL; - uncore_nb = amd_uncore_alloc(cpu); - if (!uncore_nb) - goto fail; - uncore_nb->cpu = cpu; - uncore_nb->num_counters = num_counters_nb; - uncore_nb->rdpmc_base = RDPMC_BASE_NB; - uncore_nb->msr_base = MSR_F15H_NB_PERF_CTL; - uncore_nb->active_mask = &amd_nb_active_mask; - uncore_nb->pmu = &amd_nb_pmu; - uncore_nb->events = amd_uncore_events_alloc(num_counters_nb, cpu); - if (!uncore_nb->events) + struct amd_uncore_pmu *pmu; + struct amd_uncore_ctx *ctx; + int node = cpu_to_node(cpu), i; + + for (i = 0; i < num_pmus; i++) { + pmu = &pmus[i]; + *per_cpu_ptr(pmu->ctx, cpu) = NULL; + ctx = kzalloc_node(sizeof(struct amd_uncore_ctx), GFP_KERNEL, + node); + if (!ctx) goto fail; - uncore_nb->id = -1; - *per_cpu_ptr(amd_uncore_nb, cpu) = uncore_nb; - } - if (amd_uncore_llc) { - *per_cpu_ptr(amd_uncore_llc, cpu) = NULL; - uncore_llc = amd_uncore_alloc(cpu); - if (!uncore_llc) - goto fail; - uncore_llc->cpu = cpu; - uncore_llc->num_counters = num_counters_llc; - uncore_llc->rdpmc_base = RDPMC_BASE_LLC; - uncore_llc->msr_base = MSR_F16H_L2I_PERF_CTL; - uncore_llc->active_mask = &amd_llc_active_mask; - uncore_llc->pmu = &amd_llc_pmu; - uncore_llc->events = amd_uncore_events_alloc(num_counters_llc, cpu); - if (!uncore_llc->events) + ctx->cpu = cpu; + ctx->events = kzalloc_node(sizeof(struct perf_event *) * + pmu->num_counters, GFP_KERNEL, + node); + if (!ctx->events) goto fail; - uncore_llc->id = -1; - *per_cpu_ptr(amd_uncore_llc, cpu) = uncore_llc; + + ctx->id = -1; + *per_cpu_ptr(pmu->ctx, cpu) = ctx; } return 0; fail: - if (uncore_nb) { - kfree(uncore_nb->events); - kfree(uncore_nb); - } + /* Rollback */ + for (; i >= 0; i--) { + pmu = &pmus[i]; + ctx = *per_cpu_ptr(pmu->ctx, cpu); + if (!ctx) + continue; - if (uncore_llc) { - kfree(uncore_llc->events); - kfree(uncore_llc); + kfree(ctx->events); + kfree(ctx); } return -ENOMEM; } -static struct amd_uncore * -amd_uncore_find_online_sibling(struct amd_uncore *this, - struct amd_uncore * __percpu *uncores) +static struct amd_uncore_ctx * +amd_uncore_find_online_sibling(struct amd_uncore_ctx *this, + struct amd_uncore_pmu *pmu) { unsigned int cpu; - struct amd_uncore *that; + struct amd_uncore_ctx *that; for_each_online_cpu(cpu) { - that = *per_cpu_ptr(uncores, cpu); + that = *per_cpu_ptr(pmu->ctx, cpu); if (!that) continue; @@ -523,24 +403,16 @@ amd_uncore_find_online_sibling(struct amd_uncore *this, static int amd_uncore_cpu_starting(unsigned int cpu) { - unsigned int eax, ebx, ecx, edx; - struct amd_uncore *uncore; - - if (amd_uncore_nb) { - uncore = *per_cpu_ptr(amd_uncore_nb, cpu); - cpuid(0x8000001e, &eax, &ebx, &ecx, &edx); - uncore->id = ecx & 0xff; - - uncore = amd_uncore_find_online_sibling(uncore, amd_uncore_nb); - *per_cpu_ptr(amd_uncore_nb, cpu) = uncore; - } - - if (amd_uncore_llc) { - uncore = *per_cpu_ptr(amd_uncore_llc, cpu); - uncore->id = get_llc_id(cpu); + struct amd_uncore_pmu *pmu; + struct amd_uncore_ctx *ctx; + int i; - uncore = amd_uncore_find_online_sibling(uncore, amd_uncore_llc); - *per_cpu_ptr(amd_uncore_llc, cpu) = uncore; + for (i = 0; i < num_pmus; i++) { + pmu = &pmus[i]; + ctx = *per_cpu_ptr(pmu->ctx, cpu); + ctx->id = pmu->id(cpu); + ctx = amd_uncore_find_online_sibling(ctx, pmu); + *per_cpu_ptr(pmu->ctx, cpu) = ctx; } return 0; @@ -548,195 +420,359 @@ static int amd_uncore_cpu_starting(unsigned int cpu) static void uncore_clean_online(void) { - struct amd_uncore *uncore; + struct amd_uncore_ctx *ctx; struct hlist_node *n; - hlist_for_each_entry_safe(uncore, n, &uncore_unused_list, node) { - hlist_del(&uncore->node); - kfree(uncore->events); - kfree(uncore); + hlist_for_each_entry_safe(ctx, n, &uncore_unused_list, node) { + hlist_del(&ctx->node); + kfree(ctx->events); + kfree(ctx); } } -static void uncore_online(unsigned int cpu, - struct amd_uncore * __percpu *uncores) +static int amd_uncore_cpu_online(unsigned int cpu) { - struct amd_uncore *uncore = *per_cpu_ptr(uncores, cpu); + struct amd_uncore_pmu *pmu; + struct amd_uncore_ctx *ctx; + int i; uncore_clean_online(); - if (cpu == uncore->cpu) - cpumask_set_cpu(cpu, uncore->active_mask); + for (i = 0; i < num_pmus; i++) { + pmu = &pmus[i]; + ctx = *per_cpu_ptr(pmu->ctx, cpu); + if (cpu == ctx->cpu) + cpumask_set_cpu(cpu, &pmu->active_mask); + } + + return 0; } -static int amd_uncore_cpu_online(unsigned int cpu) +static int amd_uncore_cpu_down_prepare(unsigned int cpu) { - if (amd_uncore_nb) - uncore_online(cpu, amd_uncore_nb); + struct amd_uncore_ctx *this, *that; + struct amd_uncore_pmu *pmu; + int i, j; + + for (i = 0; i < num_pmus; i++) { + pmu = &pmus[i]; + this = *per_cpu_ptr(pmu->ctx, cpu); + + /* this cpu is going down, migrate to a shared sibling if possible */ + for_each_online_cpu(j) { + that = *per_cpu_ptr(pmu->ctx, j); + + if (cpu == j) + continue; + + if (this == that) { + perf_pmu_migrate_context(&pmu->pmu, cpu, j); + cpumask_clear_cpu(cpu, &pmu->active_mask); + cpumask_set_cpu(j, &pmu->active_mask); + that->cpu = j; + break; + } + } + } - if (amd_uncore_llc) - uncore_online(cpu, amd_uncore_llc); + return 0; +} + +static int amd_uncore_cpu_dead(unsigned int cpu) +{ + struct amd_uncore_ctx *ctx; + struct amd_uncore_pmu *pmu; + int i; + + for (i = 0; i < num_pmus; i++) { + pmu = &pmus[i]; + ctx = *per_cpu_ptr(pmu->ctx, cpu); + if (cpu == ctx->cpu) + cpumask_clear_cpu(cpu, &pmu->active_mask); + + if (!--ctx->refcnt) { + kfree(ctx->events); + kfree(ctx); + } + + *per_cpu_ptr(pmu->ctx, cpu) = NULL; + } return 0; } -static void uncore_down_prepare(unsigned int cpu, - struct amd_uncore * __percpu *uncores) +static int amd_uncore_df_id(unsigned int cpu) { - unsigned int i; - struct amd_uncore *this = *per_cpu_ptr(uncores, cpu); + unsigned int eax, ebx, ecx, edx; - if (this->cpu != cpu) - return; + cpuid(0x8000001e, &eax, &ebx, &ecx, &edx); - /* this cpu is going down, migrate to a shared sibling if possible */ - for_each_online_cpu(i) { - struct amd_uncore *that = *per_cpu_ptr(uncores, i); + return ecx & 0xff; +} - if (cpu == i) - continue; +static int amd_uncore_df_event_init(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + int ret = amd_uncore_event_init(event); - if (this == that) { - perf_pmu_migrate_context(this->pmu, cpu, i); - cpumask_clear_cpu(cpu, that->active_mask); - cpumask_set_cpu(i, that->active_mask); - that->cpu = i; - break; - } - } + if (ret || pmu_version < 2) + return ret; + + hwc->config = event->attr.config & + (pmu_version >= 2 ? AMD64_PERFMON_V2_RAW_EVENT_MASK_NB : + AMD64_RAW_EVENT_MASK_NB); + + return 0; } -static int amd_uncore_cpu_down_prepare(unsigned int cpu) +static int amd_uncore_df_add(struct perf_event *event, int flags) { - if (amd_uncore_nb) - uncore_down_prepare(cpu, amd_uncore_nb); + int ret = amd_uncore_add(event, flags & ~PERF_EF_START); + struct hw_perf_event *hwc = &event->hw; + + if (ret) + return ret; + + /* + * The first four DF counters are accessible via RDPMC index 6 to 9 + * followed by the L3 counters from index 10 to 15. For processors + * with more than four DF counters, the DF RDPMC assignments become + * discontiguous as the additional counters are accessible starting + * from index 16. + */ + if (hwc->idx >= NUM_COUNTERS_NB) + hwc->event_base_rdpmc += NUM_COUNTERS_L3; - if (amd_uncore_llc) - uncore_down_prepare(cpu, amd_uncore_llc); + /* Delayed start after rdpmc base update */ + if (flags & PERF_EF_START) + amd_uncore_start(event, PERF_EF_RELOAD); return 0; } -static void uncore_dead(unsigned int cpu, struct amd_uncore * __percpu *uncores) +static int amd_uncore_df_init(void) { - struct amd_uncore *uncore = *per_cpu_ptr(uncores, cpu); + struct attribute **df_attr = amd_uncore_df_format_attr; + struct amd_uncore_pmu *pmu = &pmus[num_pmus]; + union cpuid_0x80000022_ebx ebx; + int ret; - if (cpu == uncore->cpu) - cpumask_clear_cpu(cpu, uncore->active_mask); + if (!boot_cpu_has(X86_FEATURE_PERFCTR_NB)) + return 0; - if (!--uncore->refcnt) { - kfree(uncore->events); - kfree(uncore); + /* + * For Family 17h and above, the Northbridge counters are repurposed + * as Data Fabric counters. The PMUs are exported based on family as + * either NB or DF. + */ + strscpy(pmu->name, boot_cpu_data.x86 >= 0x17 ? "amd_df" : "amd_nb", + sizeof(pmu->name)); + + pmu->num_counters = NUM_COUNTERS_NB; + pmu->msr_base = MSR_F15H_NB_PERF_CTL; + pmu->rdpmc_base = RDPMC_BASE_NB; + pmu->id = amd_uncore_df_id; + + if (pmu_version >= 2) { + *df_attr++ = &format_attr_event14v2.attr; + *df_attr++ = &format_attr_umask12.attr; + ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES); + pmu->num_counters = ebx.split.num_df_pmc; + } else if (boot_cpu_data.x86 >= 0x17) { + *df_attr = &format_attr_event14.attr; + } + + pmu->ctx = alloc_percpu(struct amd_uncore_ctx *); + if (!pmu->ctx) + return -ENOMEM; + + pmu->pmu = (struct pmu) { + .task_ctx_nr = perf_invalid_context, + .attr_groups = amd_uncore_df_attr_groups, + .name = pmu->name, + .event_init = amd_uncore_df_event_init, + .add = amd_uncore_df_add, + .del = amd_uncore_del, + .start = amd_uncore_start, + .stop = amd_uncore_stop, + .read = amd_uncore_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, + .module = THIS_MODULE, + }; + + ret = perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1); + if (ret) { + free_percpu(pmu->ctx); + pmu->ctx = NULL; + return ret; } - *per_cpu_ptr(uncores, cpu) = NULL; + pr_info("%d %s %s counters detected\n", pmu->num_counters, + boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON" : "", + pmu->pmu.name); + + num_pmus++; + + return 0; } -static int amd_uncore_cpu_dead(unsigned int cpu) +static int amd_uncore_l3_id(unsigned int cpu) { - if (amd_uncore_nb) - uncore_dead(cpu, amd_uncore_nb); + return get_llc_id(cpu); +} + +static int amd_uncore_l3_event_init(struct perf_event *event) +{ + int ret = amd_uncore_event_init(event); + struct hw_perf_event *hwc = &event->hw; + u64 config = event->attr.config; + u64 mask; + + hwc->config = config & AMD64_RAW_EVENT_MASK_NB; + + /* + * SliceMask and ThreadMask need to be set for certain L3 events. + * For other events, the two fields do not affect the count. + */ + if (ret || boot_cpu_data.x86 < 0x17) + return ret; + + mask = config & (AMD64_L3_F19H_THREAD_MASK | AMD64_L3_SLICEID_MASK | + AMD64_L3_EN_ALL_CORES | AMD64_L3_EN_ALL_SLICES | + AMD64_L3_COREID_MASK); + + if (boot_cpu_data.x86 <= 0x18) + mask = ((config & AMD64_L3_SLICE_MASK) ? : AMD64_L3_SLICE_MASK) | + ((config & AMD64_L3_THREAD_MASK) ? : AMD64_L3_THREAD_MASK); + + /* + * If the user doesn't specify a ThreadMask, they're not trying to + * count core 0, so we enable all cores & threads. + * We'll also assume that they want to count slice 0 if they specify + * a ThreadMask and leave SliceId and EnAllSlices unpopulated. + */ + else if (!(config & AMD64_L3_F19H_THREAD_MASK)) + mask = AMD64_L3_F19H_THREAD_MASK | AMD64_L3_EN_ALL_SLICES | + AMD64_L3_EN_ALL_CORES; - if (amd_uncore_llc) - uncore_dead(cpu, amd_uncore_llc); + hwc->config |= mask; return 0; } -static int __init amd_uncore_init(void) +static int amd_uncore_l3_init(void) { - struct attribute **df_attr = amd_uncore_df_format_attr; struct attribute **l3_attr = amd_uncore_l3_format_attr; - union cpuid_0x80000022_ebx ebx; - int ret = -ENODEV; + struct amd_uncore_pmu *pmu = &pmus[num_pmus]; + int ret; - if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD && - boot_cpu_data.x86_vendor != X86_VENDOR_HYGON) - return -ENODEV; + if (!boot_cpu_has(X86_FEATURE_PERFCTR_LLC)) + return 0; - if (!boot_cpu_has(X86_FEATURE_TOPOEXT)) - return -ENODEV; + /* + * For Family 17h and above, L3 cache counters are available instead + * of L2 cache counters. The PMUs are exported based on family as + * either L2 or L3. + */ + strscpy(pmu->name, boot_cpu_data.x86 >= 0x17 ? "amd_l3" : "amd_l2", + sizeof(pmu->name)); - if (boot_cpu_has(X86_FEATURE_PERFMON_V2)) - pmu_version = 2; + pmu->num_counters = NUM_COUNTERS_L2; + pmu->msr_base = MSR_F16H_L2I_PERF_CTL; + pmu->rdpmc_base = RDPMC_BASE_LLC; + pmu->id = amd_uncore_l3_id; - num_counters_nb = NUM_COUNTERS_NB; - num_counters_llc = NUM_COUNTERS_L2; if (boot_cpu_data.x86 >= 0x17) { - /* - * For F17h and above, the Northbridge counters are - * repurposed as Data Fabric counters. Also, L3 - * counters are supported too. The PMUs are exported - * based on family as either L2 or L3 and NB or DF. - */ - num_counters_llc = NUM_COUNTERS_L3; - amd_nb_pmu.name = "amd_df"; - amd_llc_pmu.name = "amd_l3"; - l3_mask = true; + *l3_attr++ = &format_attr_event8.attr; + *l3_attr++ = &format_attr_umask8.attr; + *l3_attr++ = boot_cpu_data.x86 >= 0x19 ? + &format_attr_threadmask2.attr : + &format_attr_threadmask8.attr; + pmu->num_counters = NUM_COUNTERS_L3; } - if (boot_cpu_has(X86_FEATURE_PERFCTR_NB)) { - if (pmu_version >= 2) { - *df_attr++ = &format_attr_event14v2.attr; - *df_attr++ = &format_attr_umask12.attr; - } else if (boot_cpu_data.x86 >= 0x17) { - *df_attr = &format_attr_event14.attr; - } + pmu->ctx = alloc_percpu(struct amd_uncore_ctx *); + if (!pmu->ctx) + return -ENOMEM; + + pmu->pmu = (struct pmu) { + .task_ctx_nr = perf_invalid_context, + .attr_groups = amd_uncore_l3_attr_groups, + .attr_update = amd_uncore_l3_attr_update, + .name = pmu->name, + .event_init = amd_uncore_l3_event_init, + .add = amd_uncore_add, + .del = amd_uncore_del, + .start = amd_uncore_start, + .stop = amd_uncore_stop, + .read = amd_uncore_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, + .module = THIS_MODULE, + }; + + ret = perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1); + if (ret) { + free_percpu(pmu->ctx); + pmu->ctx = NULL; + return ret; + } - amd_uncore_nb = alloc_percpu(struct amd_uncore *); - if (!amd_uncore_nb) { - ret = -ENOMEM; - goto fail_nb; - } - ret = perf_pmu_register(&amd_nb_pmu, amd_nb_pmu.name, -1); - if (ret) - goto fail_nb; + pr_info("%d %s %s counters detected\n", pmu->num_counters, + boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON" : "", + pmu->pmu.name); - if (pmu_version >= 2) { - ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES); - num_counters_nb = ebx.split.num_df_pmc; - } + num_pmus++; - pr_info("%d %s %s counters detected\n", num_counters_nb, - boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON" : "", - amd_nb_pmu.name); + return 0; +} - ret = 0; - } +static void uncore_free(void) +{ + struct amd_uncore_pmu *pmu; + int i; - if (boot_cpu_has(X86_FEATURE_PERFCTR_LLC)) { - if (boot_cpu_data.x86 >= 0x19) { - *l3_attr++ = &format_attr_event8.attr; - *l3_attr++ = &format_attr_umask8.attr; - *l3_attr++ = &format_attr_threadmask2.attr; - } else if (boot_cpu_data.x86 >= 0x17) { - *l3_attr++ = &format_attr_event8.attr; - *l3_attr++ = &format_attr_umask8.attr; - *l3_attr++ = &format_attr_threadmask8.attr; - } + for (i = 0; i < num_pmus; i++) { + pmu = &pmus[i]; + if (!pmu->ctx) + continue; - amd_uncore_llc = alloc_percpu(struct amd_uncore *); - if (!amd_uncore_llc) { - ret = -ENOMEM; - goto fail_llc; - } - ret = perf_pmu_register(&amd_llc_pmu, amd_llc_pmu.name, -1); - if (ret) - goto fail_llc; - - pr_info("%d %s %s counters detected\n", num_counters_llc, - boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON" : "", - amd_llc_pmu.name); - ret = 0; + perf_pmu_unregister(&pmu->pmu); + free_percpu(pmu->ctx); + pmu->ctx = NULL; } + num_pmus = 0; +} + +static int __init amd_uncore_init(void) +{ + int ret; + + if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD && + boot_cpu_data.x86_vendor != X86_VENDOR_HYGON) + return -ENODEV; + + if (!boot_cpu_has(X86_FEATURE_TOPOEXT)) + return -ENODEV; + + if (boot_cpu_has(X86_FEATURE_PERFMON_V2)) + pmu_version = 2; + + ret = amd_uncore_df_init(); + if (ret) + goto fail; + + ret = amd_uncore_l3_init(); + if (ret) + goto fail; + /* * Install callbacks. Core will call them for each online cpu. */ if (cpuhp_setup_state(CPUHP_PERF_X86_AMD_UNCORE_PREP, "perf/x86/amd/uncore:prepare", amd_uncore_cpu_up_prepare, amd_uncore_cpu_dead)) - goto fail_llc; + goto fail; if (cpuhp_setup_state(CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING, "perf/x86/amd/uncore:starting", @@ -753,12 +789,8 @@ static int __init amd_uncore_init(void) cpuhp_remove_state(CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING); fail_prep: cpuhp_remove_state(CPUHP_PERF_X86_AMD_UNCORE_PREP); -fail_llc: - if (boot_cpu_has(X86_FEATURE_PERFCTR_NB)) - perf_pmu_unregister(&amd_nb_pmu); - free_percpu(amd_uncore_llc); -fail_nb: - free_percpu(amd_uncore_nb); +fail: + uncore_free(); return ret; } @@ -768,18 +800,7 @@ static void __exit amd_uncore_exit(void) cpuhp_remove_state(CPUHP_AP_PERF_X86_AMD_UNCORE_ONLINE); cpuhp_remove_state(CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING); cpuhp_remove_state(CPUHP_PERF_X86_AMD_UNCORE_PREP); - - if (boot_cpu_has(X86_FEATURE_PERFCTR_LLC)) { - perf_pmu_unregister(&amd_llc_pmu); - free_percpu(amd_uncore_llc); - amd_uncore_llc = NULL; - } - - if (boot_cpu_has(X86_FEATURE_PERFCTR_NB)) { - perf_pmu_unregister(&amd_nb_pmu); - free_percpu(amd_uncore_nb); - amd_uncore_nb = NULL; - } + uncore_free(); } module_init(amd_uncore_init); From patchwork Thu Oct 5 05:23:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandipan Das X-Patchwork-Id: 148732 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp332260vqb; Thu, 5 Oct 2023 07:21:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IECdAjaxmTBorXABw4eKQ+J+yVYC00vOEGtVTZz6Bwz0aeXRgBLEIJ1E+fOVlhXd0e6pgLS X-Received: by 2002:a05:6a20:7485:b0:15e:e0fd:98e7 with SMTP id p5-20020a056a20748500b0015ee0fd98e7mr5942089pzd.20.1696515659920; Thu, 05 Oct 2023 07:20:59 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1696515659; cv=pass; d=google.com; s=arc-20160816; b=tQYzbZcZj8RYJPuRe6qXNos5LL0+Ty2ES4hjfTwzagaHwWU/G/Fd87DbCBOF14Lc4t ol8JkiXakiz/pk9L768WNEGfVTz2aJvAAVdwXsTVuQr0WT3DUwOnOoQXN/wX8+oRujGj 3IRwTcvZfTbPgutW72FPLzs9PKJhartAkN+7FXfHhmDc+Bmw1ZnOIjh8kHOBqSYcs0Fv EipVRzL9QGtVL8J6cR0MAUUroHOBs50sUZIiwN5z1Thf27o9R9yA1HLWzkzqV9f9jSFj UzvjGhGITvB46DlC+sx6MmvbCNj7WK+mEI8qzlfc33oq/DGQcvxLLp5zYYHHKfLh1+oF kDEA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=od0csuepmdl4p8pUBMe+CC9x6u2ItCRn2R588yC5dLQ=; fh=DHqst7/L7qcDaGO45kwMD1/18OMA+IdBCna8yDZwwes=; b=mnw7lgS3XTID2vrdrGQ7OCz+u+H8aufWWsEAZENQxmZ7Vm4njYAw3k7XKOJJssZB51 KOMppOw8JYhtLjw0Iwg7CINlnbNYn1F+yv0tzO+XQ4vUPJL/FBWsG0y3nneh5Qq994XZ l91J91XSBozJl0mzmDHbfzv5wLuZcPYGeUvtpNe35nygYQ8swLx4cXD8lNu4uSu3KMJJ gVyp14adikHE54rhEPG5yl6fj2SxP4Fo/ozcYyPTYqwrVFlA5To8yuTcPh7n2P0IZAAe OJLs5cSZ59uhm7lPxOT5xxBIvVa4XjnJX6rZo+yltOTMWYidzjmiPWgZ9BYLq4ZqSlbU 6d+g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=4NAZzrpu; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id y18-20020a63b512000000b005897813624fsi687645pge.476.2023.10.05.07.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 07:20:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=4NAZzrpu; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 6C7AD82685DE; Thu, 5 Oct 2023 07:20:04 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233272AbjJEOTC (ORCPT + 19 others); Thu, 5 Oct 2023 10:19:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233278AbjJEOQg (ORCPT ); Thu, 5 Oct 2023 10:16:36 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2060e.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::60e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C549E4EE0; Wed, 4 Oct 2023 22:24:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ISwroJJf8A34zhxa4jSUmKBUHNO65cJUYXEf93atWOKKJX3FijsG3UY5Sgi0FnXOGTVf7axVUTNkbvzDo858GIGirFocXM/rxfuDcGYMx6wnPcBbqvibMmaWtzFcpBkzM5QZ5hNi+GDORoJDPsxwAd3QKbRZqzsAzippRfQu468TJn0M6Cloj5aacbhIqVPD7ldFkwkYyszVvFEx20XlKPTp/MhkJ4keaqL65iJkHa7CUHrg8Gjl+474bRGr9t4lNgL+TawFaRQLYAFWZk5yH3m1Ozqfvsk4V1u35EJWG6hFXA5e+efe8LyKmUYdYx67vXcE13Qv9tqgYgL8jqW1Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=od0csuepmdl4p8pUBMe+CC9x6u2ItCRn2R588yC5dLQ=; b=KKffqHHqnToHxljz0HFSh7XSjjVfewYKE9L3ZAwPGV6AVXVFU4fjNiXKAbkrGQvCNPP+kVPWqqYtredBLR6rrOlPGOItCSTCibuCAXBhL0ezdFwnKtZfDBtMs4YaZusa5dKdEqewVy29zKrfK0WO1QE/8Tuk+ugffBHeyyHdyqY7fNvkn0H8jZ+RSgJSZ6+L1EfIaubI6SdSAVYabTiA4fNv0wY9pCt3ZJE4zlzIeByP0purhFLUVRToQ12F5zLSJWXqllTMuvf+hDEjYcXC7DZRSBqdyPlREIM3FeBJWbG1MG7yhu5ZHpq/dS3Gtkc+lZgRfjojJ+og0Oy2zhEJcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=od0csuepmdl4p8pUBMe+CC9x6u2ItCRn2R588yC5dLQ=; b=4NAZzrpuyxYuuZNapO8SFQUlf0VzRxkPqAqurI5U/iNGL4PP7aHQjDQkEqV8m/M4yfGFhuARZyH9TWl2BYIAsEQhl99Myr2AFpOJKsMXqE7Mcf2rrEDKjaWjE03tJkDZfTr5mn7kol/Tj0nLBIk83TPYfNwi0VwRGk47B0oFv5k= Received: from MN2PR18CA0015.namprd18.prod.outlook.com (2603:10b6:208:23c::20) by MW3PR12MB4441.namprd12.prod.outlook.com (2603:10b6:303:59::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33; Thu, 5 Oct 2023 05:24:38 +0000 Received: from BL6PEPF0001AB4B.namprd04.prod.outlook.com (2603:10b6:208:23c:cafe::a9) by MN2PR18CA0015.outlook.office365.com (2603:10b6:208:23c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.25 via Frontend Transport; Thu, 5 Oct 2023 05:24:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4B.mail.protection.outlook.com (10.167.242.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.14 via Frontend Transport; Thu, 5 Oct 2023 05:24:37 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 00:24:30 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH v2 2/6] perf/x86/amd/uncore: Move discovery and registration Date: Thu, 5 Oct 2023 10:53:12 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4B:EE_|MW3PR12MB4441:EE_ X-MS-Office365-Filtering-Correlation-Id: 40f15927-2a74-4fc8-bd5b-08dbc5635e69 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uQwfvXRVVPFYcT34JRFf4ZoSdHVdvIb5nHTFG25rZFcFgvBmQlqiDEodhbKGVAAplvyZrqNCIhYRHjknug3bsVPKfmRSP79r52X5Q3k3zNLnIehgNeOj976t3cOmwsQGuJkDofEM9EbAKPxOiUzWXRrCJPBy/IwNLOteQD6mJxruYIabG9QGED56/VhQ3EtaDp1uEptzbBrbGIv9cI1GHAkKPCubVCVDr6zDD4kjXvFBJ4uZVRSdJYC/Va9QEnYrsJoFY0OU5lhdDWQURvqjFv4wk3P1xizEKPCF8+VH47iiHQLgpjS4TIUJC/8RMqkkwxsUdYJ5x1lPEXFW/SieNa4gM2KuhP3gncAg3w8YBirURgoEbljq7FiN2xNp27US7kVw8sSX0tKhGvbrd3VE4eqDGt782eeCgSbuJ/6j+iPcoByYB16FT3xjiUqS0v7tcLT2Du/ujEFA2rt4EzewiKQQzi1aDRa8/09v0zl6sfX4sHLZUVKS8BfYXCWGWJi8ZHZN7A6Vm8Mjp/ofOENiB7U/dqeUhbx+c0PLvHwwZVmdLYYck5PFdwdH7aSPxuDcdr+5yQp7zoIVf9PLWXXPO1snPnwG69QcEjiMPVncpACoINzNcWVaActG/AAp8RpksFtILXf+fgIBjDHScckW4897Emdy3nSoIovtb/ZRhaQC3VqTjcrBiZmjA3r4AIez/kWfA/FIjkmW+7A4cqN3hGQ+Envo5XmVowQpNzEzZ/OfmamaHIfbBzHbsF/6ZFNK67z6QfUTgiTf6vOVqP5nXQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(346002)(136003)(39860400002)(396003)(230922051799003)(64100799003)(186009)(451199024)(1800799009)(82310400011)(40470700004)(46966006)(36840700001)(7696005)(2616005)(40460700003)(81166007)(356005)(86362001)(36860700001)(82740400003)(40480700001)(36756003)(47076005)(426003)(336012)(2906002)(30864003)(44832011)(6666004)(7416002)(478600001)(83380400001)(26005)(4326008)(8936002)(8676002)(16526019)(316002)(41300700001)(5660300002)(70586007)(70206006)(110136005)(54906003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 05:24:37.8698 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 40f15927-2a74-4fc8-bd5b-08dbc5635e69 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4441 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Thu, 05 Oct 2023 07:20:04 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778925604994140785 X-GMAIL-MSGID: 1778925604994140785 Uncore PMUs have traditionally been registered in the module init path. This is fine for the existing DF and L3 PMUs since the CPUID information does not vary across CPUs but not for the memory controller (UMC) PMUs since information like active memory channels can vary for each socket depending on how the DIMMs have been physically populated. To overcome this, the discovery of PMU information using CPUID is moved to the startup of UNCORE_STARTING. This cannot be done in the startup of UNCORE_PREP since the hotplug callback does not run on the CPU that is being brought online. Previously, the startup of UNCORE_PREP was used for allocating uncore contexts following which, the startup of UNCORE_STARTING was used to find and reuse an existing sibling context, if possible. Any unused contexts were added to a list for reclaimation later during the startup of UNCORE_ONLINE. Since all required CPUID info is now available only after the startup of UNCORE_STARTING has completed, context allocation has been moved to the startup of UNCORE_ONLINE. Before allocating contexts, the first CPU that comes online has to take up the additional responsibility of registering the PMUs. This is a one-time process though. Since sibling discovery now happens prior to deciding whether a new context is required, there is no longer a need to track and free up unused contexts. The teardown of UNCORE_ONLINE and UNCORE_PREP functionally remain the same. Overall, the flow of control described above is achieved using the following handlers for managing uncore PMUs. It is mandatory to define them for each type of uncore PMU. * scan() runs during startup of UNCORE_STARTING and collects PMU info using CPUID. * init() runs during startup of UNCORE_ONLINE, registers PMUs and sets up uncore contexts. * move() runs during teardown of UNCORE_ONLINE and migrates uncore contexts to a shared sibling, if possible. * free() runs during teardown of UNCORE_PREP and frees up uncore contexts. Signed-off-by: Sandipan Das --- arch/x86/events/amd/uncore.c | 494 +++++++++++++++++++++-------------- 1 file changed, 305 insertions(+), 189 deletions(-) diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c index ffcecda13d65..ff1d09cc07ad 100644 --- a/arch/x86/events/amd/uncore.c +++ b/arch/x86/events/amd/uncore.c @@ -26,8 +26,6 @@ #define RDPMC_BASE_LLC 10 #define COUNTER_SHIFT 16 - -#define NUM_UNCORES_MAX 2 /* DF (or NB) and L3 (or L2) */ #define UNCORE_NAME_LEN 16 #undef pr_fmt @@ -35,10 +33,7 @@ static int pmu_version; -static HLIST_HEAD(uncore_unused_list); - struct amd_uncore_ctx { - int id; int refcnt; int cpu; struct perf_event **events; @@ -53,11 +48,36 @@ struct amd_uncore_pmu { cpumask_t active_mask; struct pmu pmu; struct amd_uncore_ctx * __percpu *ctx; - int (*id)(unsigned int cpu); }; -static struct amd_uncore_pmu pmus[NUM_UNCORES_MAX]; -static int num_pmus __read_mostly; +enum { + UNCORE_TYPE_DF, + UNCORE_TYPE_L3, + + UNCORE_TYPE_MAX +}; + +union amd_uncore_info { + struct { + u64 aux_data:32; /* auxiliary data */ + u64 num_pmcs:8; /* number of counters */ + u64 cid:8; /* context id */ + } split; + u64 full; +}; + +struct amd_uncore { + union amd_uncore_info * __percpu info; + struct amd_uncore_pmu *pmus; + unsigned int num_pmus; + bool init_done; + void (*scan)(struct amd_uncore *uncore, unsigned int cpu); + int (*init)(struct amd_uncore *uncore, unsigned int cpu); + void (*move)(struct amd_uncore *uncore, unsigned int cpu); + void (*free)(struct amd_uncore *uncore, unsigned int cpu); +}; + +static struct amd_uncore uncores[UNCORE_TYPE_MAX]; static struct amd_uncore_pmu *event_to_amd_uncore_pmu(struct perf_event *event) { @@ -332,182 +352,192 @@ static const struct attribute_group *amd_uncore_l3_attr_update[] = { NULL, }; -static int amd_uncore_cpu_up_prepare(unsigned int cpu) +static __always_inline +int amd_uncore_ctx_cid(struct amd_uncore *uncore, unsigned int cpu) { - struct amd_uncore_pmu *pmu; - struct amd_uncore_ctx *ctx; - int node = cpu_to_node(cpu), i; - - for (i = 0; i < num_pmus; i++) { - pmu = &pmus[i]; - *per_cpu_ptr(pmu->ctx, cpu) = NULL; - ctx = kzalloc_node(sizeof(struct amd_uncore_ctx), GFP_KERNEL, - node); - if (!ctx) - goto fail; - - ctx->cpu = cpu; - ctx->events = kzalloc_node(sizeof(struct perf_event *) * - pmu->num_counters, GFP_KERNEL, - node); - if (!ctx->events) - goto fail; - - ctx->id = -1; - *per_cpu_ptr(pmu->ctx, cpu) = ctx; - } - - return 0; - -fail: - /* Rollback */ - for (; i >= 0; i--) { - pmu = &pmus[i]; - ctx = *per_cpu_ptr(pmu->ctx, cpu); - if (!ctx) - continue; - - kfree(ctx->events); - kfree(ctx); - } + union amd_uncore_info *info = per_cpu_ptr(uncore->info, cpu); + return info->split.cid; +} - return -ENOMEM; +static __always_inline +int amd_uncore_ctx_num_pmcs(struct amd_uncore *uncore, unsigned int cpu) +{ + union amd_uncore_info *info = per_cpu_ptr(uncore->info, cpu); + return info->split.num_pmcs; } -static struct amd_uncore_ctx * -amd_uncore_find_online_sibling(struct amd_uncore_ctx *this, - struct amd_uncore_pmu *pmu) +static void amd_uncore_ctx_free(struct amd_uncore *uncore, unsigned int cpu) { - unsigned int cpu; - struct amd_uncore_ctx *that; + struct amd_uncore_pmu *pmu; + struct amd_uncore_ctx *ctx; + int i; - for_each_online_cpu(cpu) { - that = *per_cpu_ptr(pmu->ctx, cpu); + if (!uncore->init_done) + return; - if (!that) + for (i = 0; i < uncore->num_pmus; i++) { + pmu = &uncore->pmus[i]; + ctx = *per_cpu_ptr(pmu->ctx, cpu); + if (!ctx) continue; - if (this == that) - continue; + if (cpu == ctx->cpu) + cpumask_clear_cpu(cpu, &pmu->active_mask); - if (this->id == that->id) { - hlist_add_head(&this->node, &uncore_unused_list); - this = that; - break; + if (!--ctx->refcnt) { + kfree(ctx->events); + kfree(ctx); } - } - this->refcnt++; - return this; + *per_cpu_ptr(pmu->ctx, cpu) = NULL; + } } -static int amd_uncore_cpu_starting(unsigned int cpu) +static int amd_uncore_ctx_init(struct amd_uncore *uncore, unsigned int cpu) { + struct amd_uncore_ctx *curr, *prev; struct amd_uncore_pmu *pmu; - struct amd_uncore_ctx *ctx; - int i; + int node, cid, i, j; - for (i = 0; i < num_pmus; i++) { - pmu = &pmus[i]; - ctx = *per_cpu_ptr(pmu->ctx, cpu); - ctx->id = pmu->id(cpu); - ctx = amd_uncore_find_online_sibling(ctx, pmu); - *per_cpu_ptr(pmu->ctx, cpu) = ctx; - } + if (!uncore->init_done || !uncore->num_pmus) + return 0; - return 0; -} + cid = amd_uncore_ctx_cid(uncore, cpu); -static void uncore_clean_online(void) -{ - struct amd_uncore_ctx *ctx; - struct hlist_node *n; + for (i = 0; i < uncore->num_pmus; i++) { + pmu = &uncore->pmus[i]; + *per_cpu_ptr(pmu->ctx, cpu) = NULL; + curr = NULL; - hlist_for_each_entry_safe(ctx, n, &uncore_unused_list, node) { - hlist_del(&ctx->node); - kfree(ctx->events); - kfree(ctx); - } -} + /* Find a sibling context */ + for_each_online_cpu(j) { + if (cpu == j) + continue; -static int amd_uncore_cpu_online(unsigned int cpu) -{ - struct amd_uncore_pmu *pmu; - struct amd_uncore_ctx *ctx; - int i; + prev = *per_cpu_ptr(pmu->ctx, j); + if (!prev) + continue; - uncore_clean_online(); + if (cid == amd_uncore_ctx_cid(uncore, j)) { + curr = prev; + break; + } + } + + /* Allocate context if sibling does not exist */ + if (!curr) { + node = cpu_to_node(cpu); + curr = kzalloc_node(sizeof(*curr), GFP_KERNEL, node); + if (!curr) + goto fail; + + curr->cpu = cpu; + curr->events = kzalloc_node(sizeof(*curr->events) * + pmu->num_counters, + GFP_KERNEL, node); + if (!curr->events) { + kfree(curr); + goto fail; + } - for (i = 0; i < num_pmus; i++) { - pmu = &pmus[i]; - ctx = *per_cpu_ptr(pmu->ctx, cpu); - if (cpu == ctx->cpu) cpumask_set_cpu(cpu, &pmu->active_mask); + } + + curr->refcnt++; + *per_cpu_ptr(pmu->ctx, cpu) = curr; } return 0; + +fail: + amd_uncore_ctx_free(uncore, cpu); + + return -ENOMEM; } -static int amd_uncore_cpu_down_prepare(unsigned int cpu) +static void amd_uncore_ctx_move(struct amd_uncore *uncore, unsigned int cpu) { - struct amd_uncore_ctx *this, *that; + struct amd_uncore_ctx *curr, *next; struct amd_uncore_pmu *pmu; int i, j; - for (i = 0; i < num_pmus; i++) { - pmu = &pmus[i]; - this = *per_cpu_ptr(pmu->ctx, cpu); + if (!uncore->init_done) + return; - /* this cpu is going down, migrate to a shared sibling if possible */ - for_each_online_cpu(j) { - that = *per_cpu_ptr(pmu->ctx, j); + for (i = 0; i < uncore->num_pmus; i++) { + pmu = &uncore->pmus[i]; + curr = *per_cpu_ptr(pmu->ctx, cpu); + if (!curr) + continue; - if (cpu == j) + /* Migrate to a shared sibling if possible */ + for_each_online_cpu(j) { + next = *per_cpu_ptr(pmu->ctx, j); + if (!next || cpu == j) continue; - if (this == that) { + if (curr == next) { perf_pmu_migrate_context(&pmu->pmu, cpu, j); cpumask_clear_cpu(cpu, &pmu->active_mask); cpumask_set_cpu(j, &pmu->active_mask); - that->cpu = j; + next->cpu = j; break; } } } +} + +static int amd_uncore_cpu_starting(unsigned int cpu) +{ + struct amd_uncore *uncore; + int i; + + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; + uncore->scan(uncore, cpu); + } return 0; } -static int amd_uncore_cpu_dead(unsigned int cpu) +static int amd_uncore_cpu_online(unsigned int cpu) { - struct amd_uncore_ctx *ctx; - struct amd_uncore_pmu *pmu; + struct amd_uncore *uncore; int i; - for (i = 0; i < num_pmus; i++) { - pmu = &pmus[i]; - ctx = *per_cpu_ptr(pmu->ctx, cpu); - if (cpu == ctx->cpu) - cpumask_clear_cpu(cpu, &pmu->active_mask); + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; + if (uncore->init(uncore, cpu)) + break; + } - if (!--ctx->refcnt) { - kfree(ctx->events); - kfree(ctx); - } + return 0; +} - *per_cpu_ptr(pmu->ctx, cpu) = NULL; +static int amd_uncore_cpu_down_prepare(unsigned int cpu) +{ + struct amd_uncore *uncore; + int i; + + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; + uncore->move(uncore, cpu); } return 0; } -static int amd_uncore_df_id(unsigned int cpu) +static int amd_uncore_cpu_dead(unsigned int cpu) { - unsigned int eax, ebx, ecx, edx; + struct amd_uncore *uncore; + int i; - cpuid(0x8000001e, &eax, &ebx, &ecx, &edx); + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; + uncore->free(uncore, cpu); + } - return ecx & 0xff; + return 0; } static int amd_uncore_df_event_init(struct perf_event *event) @@ -550,41 +580,66 @@ static int amd_uncore_df_add(struct perf_event *event, int flags) return 0; } -static int amd_uncore_df_init(void) +static +void amd_uncore_df_ctx_scan(struct amd_uncore *uncore, unsigned int cpu) { - struct attribute **df_attr = amd_uncore_df_format_attr; - struct amd_uncore_pmu *pmu = &pmus[num_pmus]; union cpuid_0x80000022_ebx ebx; - int ret; + union amd_uncore_info info; if (!boot_cpu_has(X86_FEATURE_PERFCTR_NB)) - return 0; + return; + + info.split.aux_data = 0; + info.split.num_pmcs = NUM_COUNTERS_NB; + info.split.cid = topology_die_id(cpu); + + if (pmu_version >= 2) { + ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES); + info.split.num_pmcs = ebx.split.num_df_pmc; + } + + *per_cpu_ptr(uncore->info, cpu) = info; +} + +static +int amd_uncore_df_ctx_init(struct amd_uncore *uncore, unsigned int cpu) +{ + struct attribute **df_attr = amd_uncore_df_format_attr; + struct amd_uncore_pmu *pmu; + + /* Run just once */ + if (uncore->init_done) + return amd_uncore_ctx_init(uncore, cpu); + + /* No grouping, single instance for a system */ + uncore->pmus = kzalloc(sizeof(*uncore->pmus), GFP_KERNEL); + if (!uncore->pmus) { + uncore->num_pmus = 0; + goto done; + } /* * For Family 17h and above, the Northbridge counters are repurposed * as Data Fabric counters. The PMUs are exported based on family as * either NB or DF. */ + pmu = &uncore->pmus[0]; strscpy(pmu->name, boot_cpu_data.x86 >= 0x17 ? "amd_df" : "amd_nb", sizeof(pmu->name)); - - pmu->num_counters = NUM_COUNTERS_NB; + pmu->num_counters = amd_uncore_ctx_num_pmcs(uncore, cpu); pmu->msr_base = MSR_F15H_NB_PERF_CTL; pmu->rdpmc_base = RDPMC_BASE_NB; - pmu->id = amd_uncore_df_id; if (pmu_version >= 2) { *df_attr++ = &format_attr_event14v2.attr; *df_attr++ = &format_attr_umask12.attr; - ebx.full = cpuid_ebx(EXT_PERFMON_DEBUG_FEATURES); - pmu->num_counters = ebx.split.num_df_pmc; } else if (boot_cpu_data.x86 >= 0x17) { *df_attr = &format_attr_event14.attr; } pmu->ctx = alloc_percpu(struct amd_uncore_ctx *); if (!pmu->ctx) - return -ENOMEM; + goto done; pmu->pmu = (struct pmu) { .task_ctx_nr = perf_invalid_context, @@ -600,25 +655,22 @@ static int amd_uncore_df_init(void) .module = THIS_MODULE, }; - ret = perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1); - if (ret) { + if (perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1)) { free_percpu(pmu->ctx); pmu->ctx = NULL; - return ret; + goto done; } - pr_info("%d %s %s counters detected\n", pmu->num_counters, - boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON" : "", + pr_info("%d %s%s counters detected\n", pmu->num_counters, + boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON " : "", pmu->pmu.name); - num_pmus++; + uncore->num_pmus = 1; - return 0; -} +done: + uncore->init_done = true; -static int amd_uncore_l3_id(unsigned int cpu) -{ - return get_llc_id(cpu); + return amd_uncore_ctx_init(uncore, cpu); } static int amd_uncore_l3_event_init(struct perf_event *event) @@ -660,27 +712,52 @@ static int amd_uncore_l3_event_init(struct perf_event *event) return 0; } -static int amd_uncore_l3_init(void) +static +void amd_uncore_l3_ctx_scan(struct amd_uncore *uncore, unsigned int cpu) { - struct attribute **l3_attr = amd_uncore_l3_format_attr; - struct amd_uncore_pmu *pmu = &pmus[num_pmus]; - int ret; + union amd_uncore_info info; if (!boot_cpu_has(X86_FEATURE_PERFCTR_LLC)) - return 0; + return; + + info.split.aux_data = 0; + info.split.num_pmcs = NUM_COUNTERS_L2; + info.split.cid = get_llc_id(cpu); + + if (boot_cpu_data.x86 >= 0x17) + info.split.num_pmcs = NUM_COUNTERS_L3; + + *per_cpu_ptr(uncore->info, cpu) = info; +} + +static +int amd_uncore_l3_ctx_init(struct amd_uncore *uncore, unsigned int cpu) +{ + struct attribute **l3_attr = amd_uncore_l3_format_attr; + struct amd_uncore_pmu *pmu; + + /* Run just once */ + if (uncore->init_done) + return amd_uncore_ctx_init(uncore, cpu); + + /* No grouping, single instance for a system */ + uncore->pmus = kzalloc(sizeof(*uncore->pmus), GFP_KERNEL); + if (!uncore->pmus) { + uncore->num_pmus = 0; + goto done; + } /* * For Family 17h and above, L3 cache counters are available instead * of L2 cache counters. The PMUs are exported based on family as * either L2 or L3. */ + pmu = &uncore->pmus[0]; strscpy(pmu->name, boot_cpu_data.x86 >= 0x17 ? "amd_l3" : "amd_l2", sizeof(pmu->name)); - - pmu->num_counters = NUM_COUNTERS_L2; + pmu->num_counters = amd_uncore_ctx_num_pmcs(uncore, cpu); pmu->msr_base = MSR_F16H_L2I_PERF_CTL; pmu->rdpmc_base = RDPMC_BASE_LLC; - pmu->id = amd_uncore_l3_id; if (boot_cpu_data.x86 >= 0x17) { *l3_attr++ = &format_attr_event8.attr; @@ -688,12 +765,11 @@ static int amd_uncore_l3_init(void) *l3_attr++ = boot_cpu_data.x86 >= 0x19 ? &format_attr_threadmask2.attr : &format_attr_threadmask8.attr; - pmu->num_counters = NUM_COUNTERS_L3; } pmu->ctx = alloc_percpu(struct amd_uncore_ctx *); if (!pmu->ctx) - return -ENOMEM; + goto done; pmu->pmu = (struct pmu) { .task_ctx_nr = perf_invalid_context, @@ -710,43 +786,45 @@ static int amd_uncore_l3_init(void) .module = THIS_MODULE, }; - ret = perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1); - if (ret) { + if (perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1)) { free_percpu(pmu->ctx); pmu->ctx = NULL; - return ret; + goto done; } - pr_info("%d %s %s counters detected\n", pmu->num_counters, - boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON" : "", + pr_info("%d %s%s counters detected\n", pmu->num_counters, + boot_cpu_data.x86_vendor == X86_VENDOR_HYGON ? "HYGON " : "", pmu->pmu.name); - num_pmus++; - - return 0; -} - -static void uncore_free(void) -{ - struct amd_uncore_pmu *pmu; - int i; + uncore->num_pmus = 1; - for (i = 0; i < num_pmus; i++) { - pmu = &pmus[i]; - if (!pmu->ctx) - continue; - - perf_pmu_unregister(&pmu->pmu); - free_percpu(pmu->ctx); - pmu->ctx = NULL; - } +done: + uncore->init_done = true; - num_pmus = 0; + return amd_uncore_ctx_init(uncore, cpu); } +static struct amd_uncore uncores[UNCORE_TYPE_MAX] = { + /* UNCORE_TYPE_DF */ + { + .scan = amd_uncore_df_ctx_scan, + .init = amd_uncore_df_ctx_init, + .move = amd_uncore_ctx_move, + .free = amd_uncore_ctx_free, + }, + /* UNCORE_TYPE_L3 */ + { + .scan = amd_uncore_l3_ctx_scan, + .init = amd_uncore_l3_ctx_init, + .move = amd_uncore_ctx_move, + .free = amd_uncore_ctx_free, + }, +}; + static int __init amd_uncore_init(void) { - int ret; + struct amd_uncore *uncore; + int ret, i; if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD && boot_cpu_data.x86_vendor != X86_VENDOR_HYGON) @@ -758,20 +836,27 @@ static int __init amd_uncore_init(void) if (boot_cpu_has(X86_FEATURE_PERFMON_V2)) pmu_version = 2; - ret = amd_uncore_df_init(); - if (ret) - goto fail; + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; - ret = amd_uncore_l3_init(); - if (ret) - goto fail; + BUG_ON(!uncore->scan); + BUG_ON(!uncore->init); + BUG_ON(!uncore->move); + BUG_ON(!uncore->free); + + uncore->info = alloc_percpu(union amd_uncore_info); + if (!uncore->info) { + ret = -ENOMEM; + goto fail; + } + }; /* * Install callbacks. Core will call them for each online cpu. */ if (cpuhp_setup_state(CPUHP_PERF_X86_AMD_UNCORE_PREP, "perf/x86/amd/uncore:prepare", - amd_uncore_cpu_up_prepare, amd_uncore_cpu_dead)) + NULL, amd_uncore_cpu_dead)) goto fail; if (cpuhp_setup_state(CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING, @@ -790,17 +875,48 @@ static int __init amd_uncore_init(void) fail_prep: cpuhp_remove_state(CPUHP_PERF_X86_AMD_UNCORE_PREP); fail: - uncore_free(); + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; + if (uncore->info) { + free_percpu(uncore->info); + uncore->info = NULL; + } + } return ret; } static void __exit amd_uncore_exit(void) { + struct amd_uncore *uncore; + struct amd_uncore_pmu *pmu; + int i, j; + cpuhp_remove_state(CPUHP_AP_PERF_X86_AMD_UNCORE_ONLINE); cpuhp_remove_state(CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING); cpuhp_remove_state(CPUHP_PERF_X86_AMD_UNCORE_PREP); - uncore_free(); + + for (i = 0; i < UNCORE_TYPE_MAX; i++) { + uncore = &uncores[i]; + if (!uncore->info) + continue; + + free_percpu(uncore->info); + uncore->info = NULL; + + for (j = 0; j < uncore->num_pmus; j++) { + pmu = &uncore->pmus[j]; + if (!pmu->ctx) + continue; + + perf_pmu_unregister(&pmu->pmu); + free_percpu(pmu->ctx); + pmu->ctx = NULL; + } + + kfree(uncore->pmus); + uncore->pmus = NULL; + } } module_init(amd_uncore_init); From patchwork Thu Oct 5 05:23:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandipan Das X-Patchwork-Id: 148733 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp332890vqb; Thu, 5 Oct 2023 07:21:58 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFb37SA6cgMsghpD4FCFy2Vr7ZJ2H13cz9hLafQe3jZbWIGhxS2X3tHGOxh/H285Nc/YUPx X-Received: by 2002:a17:902:680b:b0:1c3:ed30:ce04 with SMTP id h11-20020a170902680b00b001c3ed30ce04mr4600796plk.25.1696515717899; Thu, 05 Oct 2023 07:21:57 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1696515717; cv=pass; d=google.com; s=arc-20160816; b=JREdrYEtefCVFJwd27wk9QqWIgMSMKz5PacEGYvI8jlaZa/r6q8JCqgKaaMJ9fHhC7 bgzxpCfYNrOpIXAwI5+a273P8CBVmC0Ywnmk1s4n/Ikj40pnMcDoKroNw638KSfQFO/a HwCFseZuZfUBqPB1Y9Zw0NHgGmezTgbag+m5Po1dnKxx6PWhcQ4iX3R1rNP9niMsphVb 1O/uCWlFpTyTIIB4RVmETsaqqMCg+3wADy3ujcPUNb41qgMz3bveQc9yKnp4t7lknYgq ei3X3/W7YNwW5hML2Zy0STZkjD9Le0A+qoNgyL/g83a7iAx0Nqsr9yiFitD6SSzEVQhE zjPg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=iMmsOE81tcPEf1TtjF3ppaSEoCx22VbO2B6jtxXluH0=; fh=DHqst7/L7qcDaGO45kwMD1/18OMA+IdBCna8yDZwwes=; b=yvr7ywzzBqqo58LczbRqAkaU2UnYf/a9xaD+GfqWJW82TknhoiAR7EB7E90yPlZl5k rD4sTHALZk95rFL1lxgfeYiAscsgIo6L2Nggi4L3pZ+4obsMg3rxFZul5vsnHGBL+Koj KRpd99zIk8Jgt4CczGgpB0ebNIoytddxiOmPs+fxSQ3wAZzknAOj9QTlVWVglM0s/wQw IkEI5dQY3udwkNTMpRK/Zy/gqId95KiPtHPYa47iR2mv0jLuz8PXci6hl1Ca7z7ZCenX AFibrJV/4LdN3x8JpoM0GuX2Z2gJKMRfXs92Jwa1RW91P+kAiZcMODXcmf/fyx8zdJDM r6ag== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Ng0H7XoR; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id q15-20020a170902eb8f00b001c7269bc9f5si1620062plg.152.2023.10.05.07.21.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 07:21:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Ng0H7XoR; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id AF801833F8D3; Thu, 5 Oct 2023 07:21:56 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234426AbjJEOV1 (ORCPT + 19 others); Thu, 5 Oct 2023 10:21:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234132AbjJEOTa (ORCPT ); Thu, 5 Oct 2023 10:19:30 -0400 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on20616.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eb2::616]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DFE604EE8; Wed, 4 Oct 2023 22:25:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hwUkETit4DiIn0ywnoatas1t2leECTCqul9RbuDwZpcOq50/pu7KMUF9pPrpuJ8HWpnOXemSUiwfNkRhNLbp5utDrvHMXi3czwTICbBE1CYSYNsdD6In9dE96wqmnA1xskf8OApwAw/Sfua5ABGipHQNKCl6U+OvyuUCVdfPCyE26U+OyAN/jzqtDFhx8Hedy63vMEZim9ZqvU1UN1paw3Hv00emr8lIVoC3f7qaXUUyNnOJcsAPV3sJTZOw2efyv+vCcZgI6g93hweAJAIaoy1DBrvGL3pqNqn4uXrbNYTfvEb7VsH1bcEMt98rVksLzexLuI8BMLKrbG522AdmQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iMmsOE81tcPEf1TtjF3ppaSEoCx22VbO2B6jtxXluH0=; b=SHELh+otjLc8OwQ7sPP9+++xxEraFJ1MV6IvqWSBrfz7iQ7pF/r1Kwh1QFntf/GDYLTL9rYxxPOD7dPLSX+CQAqYxoM74qg2a0EpZzuEZGNGRcW5xwgNfwgBPKvDXSWje9s9c7DaVy4NmUOr9XTib7F5/IPlh+fBVQMeYGBhocKR6L6tMHfQKGORuGM90GFmh/PXrZGFmr2wu7x6EEvwrh2FIzr4XfrXKj3Qr9oD2dkV1QSzidO+TQ2czMl6z3fTShfaMC1zlvRLjVD7M4iIKOLFtrRsKZ08EEapFKQnKwDgH6fK8qzurbuD91dkx9J/roEbxhcM+zw5AMCBCoJcdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iMmsOE81tcPEf1TtjF3ppaSEoCx22VbO2B6jtxXluH0=; b=Ng0H7XoRZJqmjJtQmRP1Y61TyXRIZViakPPnBW/zDrxzIvDtNvwrSAe/FhAF6IUAPYmyJwiXEUJFeCW+j1yFU2bONqHKzPOA+/+doSViqq5P063ub72mQI6AcuCH70600QeGWbl2d8kabH7eTmPqrEEQIacMKoOr9NOhd4vb0gw= Received: from BLAPR03CA0164.namprd03.prod.outlook.com (2603:10b6:208:32f::20) by MW3PR12MB4476.namprd12.prod.outlook.com (2603:10b6:303:2d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33; Thu, 5 Oct 2023 05:25:29 +0000 Received: from BL6PEPF0001AB4D.namprd04.prod.outlook.com (2603:10b6:208:32f:cafe::e6) by BLAPR03CA0164.outlook.office365.com (2603:10b6:208:32f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.25 via Frontend Transport; Thu, 5 Oct 2023 05:25:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4D.mail.protection.outlook.com (10.167.242.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.14 via Frontend Transport; Thu, 5 Oct 2023 05:25:29 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 00:25:22 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH v2 3/6] perf/x86/amd/uncore: Use rdmsr if rdpmc is unavailable Date: Thu, 5 Oct 2023 10:53:13 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4D:EE_|MW3PR12MB4476:EE_ X-MS-Office365-Filtering-Correlation-Id: 8c9f51d5-3c52-48d2-a7af-08dbc5637cf0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XWDc+s6gLUKOA8dNPPGBMtSyogT0JJieYaTg33uFDBTUEZULvsmrSZr9kyHmc840UxEvc9BkKzpX/ExMTEO1WGJYru5A6I/W7/hrDl6CxvzeKeR28nuLzkKS85PAJxa2rIlppbX8dYhBJgJEyv0OWrrfx7NRN2x+l8moNYh7nSvQY52/H4aS5BgmnVOOnvlJPqz8frX2VEhIVOA9HzsrUffVeGgrF7AIZB+TpxiGSNUc04Wa337YMi9kSoaMxrKgB9qtYpeb5nsqoPvBhAZhRNwFDBa8csXtySDVdLH2jWbB2P8PyKdtfRDo+01D3ehI0VqsD+zWkGiA/OmoUU0xQUG4MlTfHG8ovimspgae5/U92moewpbzWju/LLTfuV/o1YHeRJ/EoWf2qhmNMFbzzJ+tlixIutfzubVyhoYraxGjL+8hbjZ6hW+eWpnHYyQnXdGU/CzpW875bswu6WuJQ4taqH3x6aMZPh1UpjLmZTltHORdLgNQUHQt0z8MKJHVvbi7Dm+g9ur1v86blm1bRx1dRst7cdcYOI7qPysJCXB9r5A+CrfMtcL0MtoIDrgAAmKSwbloLjiUvTNJS5o2iI3V9AruwGQ5SLMN/Rwp5MvtFI4xlxSqstUASyb0KAngASvriHdJb8/thZn46K2CRuqYauzuKqmk18D8fvnr2ar7UeDvm8zdgESulXRnwEL6Npj07Q3ky8OYezFs2KMMxwJbBRvk7px3MA5oM2f/HXNKxN5m8dtlp1cG1OjyY5HIGAggs4E8mWrZGAYuqqle2w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(376002)(396003)(39860400002)(230922051799003)(1800799009)(64100799003)(451199024)(186009)(82310400011)(36840700001)(40470700004)(46966006)(40460700003)(40480700001)(7696005)(478600001)(6666004)(16526019)(82740400003)(36860700001)(356005)(86362001)(81166007)(110136005)(2906002)(4326008)(2616005)(47076005)(426003)(336012)(26005)(83380400001)(7416002)(36756003)(70206006)(70586007)(44832011)(8936002)(41300700001)(5660300002)(54906003)(8676002)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 05:25:29.1325 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8c9f51d5-3c52-48d2-a7af-08dbc5637cf0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4D.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4476 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_PASS,SPF_NONE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 05 Oct 2023 07:21:56 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778925665640404371 X-GMAIL-MSGID: 1778925665640404371 Not all uncore PMUs may support the use of the RDPMC instruction for reading counters. In such cases, read the count from the corresponding PERF_CTR register using the RDMSR instruction. Signed-off-by: Sandipan Das --- arch/x86/events/amd/uncore.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c index ff1d09cc07ad..2fe623923034 100644 --- a/arch/x86/events/amd/uncore.c +++ b/arch/x86/events/amd/uncore.c @@ -96,7 +96,16 @@ static void amd_uncore_read(struct perf_event *event) */ prev = local64_read(&hwc->prev_count); - rdpmcl(hwc->event_base_rdpmc, new); + + /* + * Some uncore PMUs do not have RDPMC assignments. In such cases, + * read counts directly from the corresponding PERF_CTR. + */ + if (hwc->event_base_rdpmc < 0) + rdmsrl(hwc->event_base, new); + else + rdpmcl(hwc->event_base_rdpmc, new); + local64_set(&hwc->prev_count, new); delta = (new << COUNTER_SHIFT) - (prev << COUNTER_SHIFT); delta >>= COUNTER_SHIFT; @@ -164,6 +173,9 @@ static int amd_uncore_add(struct perf_event *event, int flags) hwc->event_base_rdpmc = pmu->rdpmc_base + hwc->idx; hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED; + if (pmu->rdpmc_base < 0) + hwc->event_base_rdpmc = -1; + if (flags & PERF_EF_START) event->pmu->start(event, PERF_EF_RELOAD); From patchwork Thu Oct 5 05:23:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandipan Das X-Patchwork-Id: 148726 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp330650vqb; Thu, 5 Oct 2023 07:18:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEQdqcUlbj27SQV98aFTCCB1Hkx+KUGdHqSUkiDWeTajkXKJ44QEd7e0Xrj4Mrv7OH2lQwt X-Received: by 2002:a05:6a21:1a5:b0:15d:8366:65b8 with SMTP id le37-20020a056a2101a500b0015d836665b8mr2081417pzb.13.1696515516018; Thu, 05 Oct 2023 07:18:36 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1696515516; cv=pass; d=google.com; s=arc-20160816; b=jQsDhSkh64TxNSFLV7AXU+Rz88jZkokQu7ZqurkBBXWkZcr+F8+NtK0ZH4AvTz5MhX rIFnTgfWQvHKxPYchGj4wg/39YeZjpKs8uqX2kxOCvp8ax2ozK6iyYYeOEB1k2nx8Nyl wAMWoNOAVl0eaNzVh9loaKPpME7EZXqkiHivqAz/lYHZsfkaf1pCZGISW48qZKGpodTT TokT+pBpEz9gMzVTtsS6Z9wshRzrFKci+n6FHr6QlJ4+OPCqq2sdZXOU7csyRgy0a1u+ F+sJgD/ZVx13Eu5idGwNX9+gnKk2NNAmTyXMRL4R+4AHgYfVxvvkDWD3j7Vi65P3EmEP w8Vg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LVlPFoNlbno4/HbumKBlL+FDU30UKhtlZ5NY6J/E0i0=; fh=DHqst7/L7qcDaGO45kwMD1/18OMA+IdBCna8yDZwwes=; b=N63KMJivlXOUri05Us+h0Ucd6sAI4SehP0VzTCTvGaDgGLLpqanMaxKqJnccLj0+K8 Z3IUK5+DI+5c9PhqVVJwxDlzTtdlEd6H0SiK0L/DZ0QgZdxWopXjsrbdvM3WGNtWTBN2 smK7GO0hxLUrgHT/E2bZIgidpCTbPM1wlVBTQZKq+bMpie+UsK4In1iUFF7jh1LzLN8P KAsdl2+TRscLi+OAhh76Fj/JBmYm+1s+lSVS+ENUXPR+TvWXoqAfWqrmhIf/Ati5ccjG XWzoYS8awNSQuv+6Af/bn3X96CAmZAJjKyux07XvQE988Q7h6FmjmcluU4V7l6bRLTtd 1vjA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=HWWrk35I; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id m16-20020a63fd50000000b00589336ba9f8si1532226pgj.830.2023.10.05.07.18.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 07:18:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=HWWrk35I; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 50B6E82C516F; Thu, 5 Oct 2023 07:18:35 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232814AbjJEOR5 (ORCPT + 19 others); Thu, 5 Oct 2023 10:17:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232827AbjJEOQB (ORCPT ); Thu, 5 Oct 2023 10:16:01 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on20626.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::626]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CEB1D4EF1; Wed, 4 Oct 2023 22:26:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ErQpUDBvtGGQUWvuQvxpNv1bXzPqHXZidUQ5Y4ruHAnuEc06hzptgyxHfD3C5iY2PuAL4zYT75C6HGfJi7uSP6fYOQuUmOZun45oDKIO0GMtDAmf+2NpyM0/TW42DaE9ofrQd5YE4z80v/q+Le4SdPCJRd+T20mE08bueoGmkwfpewbpucQvYi8RCeXSgJNrg5r4Guha1nf+mnBujJd8hiMLlktx5K5Vhb+B0yqXyEcjv5imtA1omQ/9IUX1hbT0Xl7nlwq4yHQJKRBeKObRzKCYlY6F68xL1fnLZbbmz1myDivXcv3c6yaNKWvrVc3h1/rjFG/M89ltTFjpJlkguQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LVlPFoNlbno4/HbumKBlL+FDU30UKhtlZ5NY6J/E0i0=; b=Fkl7mYkWalYmdZFctX04gHfPYtctWtpoIGBX4UvFXX3/RBkXxcw4guIqeRetoyTVF8LPT/ufnCVHPwFEY00x80rKhOJkgCU46H/9uU9i6WGKDomThZktjUh5IcAYDwi/t10f3vQmUhR+PaqvkFKIVFFxlYECWTls1ay+nEVysa+nOoO/v+vHD5seJh7qBszu12l9ycZSJ6pxVJBFpqMi4yfbvIB0rFXTeB2TG9dZcyCt38EECCrSp+bXq1KoPWlIQXv7hhwJjccjcMjZWxWaqcTa2sdbD5UtIsHpowulIg21avRoWoedfuUvElCwuMLO+QNrwOov8/i+MEslsFNTFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LVlPFoNlbno4/HbumKBlL+FDU30UKhtlZ5NY6J/E0i0=; b=HWWrk35IoZNvlj3DH3mRiaqg3y6LCMVTDEuwQUrdO+JTsmSfbKEhGCIVIspRdr/oPdbrAEHh3modsS3smL23GExxp//3KM7cbya9pJ1CwFbJRKoLz5HS87s+Rll4YuAahlVoGKA+dMq2dnBuH9k3W2i/VR0vohS5qQwROLdli6s= Received: from BL0PR01CA0013.prod.exchangelabs.com (2603:10b6:208:71::26) by BN9PR12MB5353.namprd12.prod.outlook.com (2603:10b6:408:102::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33; Thu, 5 Oct 2023 05:25:48 +0000 Received: from BL6PEPF0001AB4E.namprd04.prod.outlook.com (2603:10b6:208:71:cafe::7d) by BL0PR01CA0013.outlook.office365.com (2603:10b6:208:71::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33 via Frontend Transport; Thu, 5 Oct 2023 05:25:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4E.mail.protection.outlook.com (10.167.242.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.14 via Frontend Transport; Thu, 5 Oct 2023 05:25:48 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 00:25:42 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH v2 4/6] perf/x86/amd/uncore: Add group exclusivity Date: Thu, 5 Oct 2023 10:53:14 +0530 Message-ID: <937d6d71010a48ea4e069f4904b3116a5f99ecdf.1696425185.git.sandipan.das@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4E:EE_|BN9PR12MB5353:EE_ X-MS-Office365-Filtering-Correlation-Id: 7bba36d5-453f-4493-3690-08dbc563887c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /u3zoD/R6E24IiRHuF+ZkKnJcE5hoGCbmI6QxTQIm5lPRRkRQ4rY0oxCNvElr2hsQyCKkl6HyvaPs+JFbIO5YMG0+UshIOsfYA1DZ/KYr9/WLNJalvv8FVLzyq/or+E6vM0rGKvjLwKUmtZ7R95nOnByQJf5oJJb9duNehKZuNDebOh8IHqsdaHTathRMXai6fsRGqiuZon7zTnBj7lIifkcTrdhdAHVohw/scinOraUIYQudVPN++mqzScxE3uJthadr3bw7khOz0QDrs4s0sFqQKXXcoiLjqawYTOZRJKEYCIBANdc3Wnpy57wkGvIoHHLa2jm10BrTdgmoXe2FWa7Eh0TWAFz+0xAEG8KE166A+ccMp+xOaHRnXFcWqF+6vI76ZPtJ0Wh7iFvKU4KuhOTVAT/es8oYrr2vLMfNm869EOj2sG7xedhZoaTGpN9K8cw/x7V28snoDxqsVXwKQKWJvTo1GnLMMNoLpDck+p9iVOGTBs2S8+oamNFgaa3int8ixJIttKvQGC8SdOHbnOMpxrCPz2srbiOYtH477fLq2AZaqwLD+QNdvKLtk/YzQG76N35w9wOlTj1YoOPcUIS2ng9Qal0+1F7njfF7Zevh6MOlDi7Wv9VcIjLVGVJD8MprCyYjhiLbf1TG7tMxb8ETSs9bWbVnow7UTH2uyIBJSOJPilbmUeoGgS1QWYKGy4ytdLlVGfoGMFJQYkzugTWVa5OtLhqaHNmYFojAKuqFAXme9k2XNzYROp2MGcTTZeQkSfVSG6YfmCO32CbJA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(346002)(136003)(39860400002)(396003)(230922051799003)(451199024)(1800799009)(82310400011)(186009)(64100799003)(46966006)(40470700004)(36840700001)(110136005)(2616005)(70206006)(7696005)(41300700001)(316002)(54906003)(70586007)(44832011)(8676002)(4326008)(5660300002)(16526019)(26005)(8936002)(6666004)(478600001)(336012)(426003)(40460700003)(7416002)(82740400003)(2906002)(83380400001)(47076005)(36756003)(356005)(36860700001)(40480700001)(81166007)(86362001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 05:25:48.5020 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7bba36d5-453f-4493-3690-08dbc563887c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4E.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5353 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, SPF_HELO_PASS,SPF_NONE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 05 Oct 2023 07:18:35 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778925453985988510 X-GMAIL-MSGID: 1778925453985988510 In some cases, it may be necessary to restrict opening PMU events to a subset of CPUs. E.g. Unified Memory Controller (UMC) PMUs are specific to each active memory channel and the MSR address space for the PERF_CTL and PERF_CTR registers is reused on each socket. Thus, opening events for a specific UMC PMU should be restricted to CPUs belonging to the same socket as that of the UMC. The "cpumask" of the PMU should also reflect this accordingly. Uncore PMUs which require this can use the new group attribute in struct amd_uncore_pmu to set a valid group ID during the scan() phase. Later, during init(), an uncore context for a CPU will be unavailable if the group ID does not match. Signed-off-by: Sandipan Das --- arch/x86/events/amd/uncore.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c index 2fe623923034..318982951dd2 100644 --- a/arch/x86/events/amd/uncore.c +++ b/arch/x86/events/amd/uncore.c @@ -27,6 +27,7 @@ #define COUNTER_SHIFT 16 #define UNCORE_NAME_LEN 16 +#define UNCORE_GROUP_MAX 256 #undef pr_fmt #define pr_fmt(fmt) "amd_uncore: " fmt @@ -45,6 +46,7 @@ struct amd_uncore_pmu { int num_counters; int rdpmc_base; u32 msr_base; + int group; cpumask_t active_mask; struct pmu pmu; struct amd_uncore_ctx * __percpu *ctx; @@ -61,6 +63,7 @@ union amd_uncore_info { struct { u64 aux_data:32; /* auxiliary data */ u64 num_pmcs:8; /* number of counters */ + u64 gid:8; /* group id */ u64 cid:8; /* context id */ } split; u64 full; @@ -371,6 +374,13 @@ int amd_uncore_ctx_cid(struct amd_uncore *uncore, unsigned int cpu) return info->split.cid; } +static __always_inline +int amd_uncore_ctx_gid(struct amd_uncore *uncore, unsigned int cpu) +{ + union amd_uncore_info *info = per_cpu_ptr(uncore->info, cpu); + return info->split.gid; +} + static __always_inline int amd_uncore_ctx_num_pmcs(struct amd_uncore *uncore, unsigned int cpu) { @@ -409,18 +419,23 @@ static int amd_uncore_ctx_init(struct amd_uncore *uncore, unsigned int cpu) { struct amd_uncore_ctx *curr, *prev; struct amd_uncore_pmu *pmu; - int node, cid, i, j; + int node, cid, gid, i, j; if (!uncore->init_done || !uncore->num_pmus) return 0; cid = amd_uncore_ctx_cid(uncore, cpu); + gid = amd_uncore_ctx_gid(uncore, cpu); for (i = 0; i < uncore->num_pmus; i++) { pmu = &uncore->pmus[i]; *per_cpu_ptr(pmu->ctx, cpu) = NULL; curr = NULL; + /* Check for group exclusivity */ + if (gid != pmu->group) + continue; + /* Find a sibling context */ for_each_online_cpu(j) { if (cpu == j) @@ -603,6 +618,7 @@ void amd_uncore_df_ctx_scan(struct amd_uncore *uncore, unsigned int cpu) info.split.aux_data = 0; info.split.num_pmcs = NUM_COUNTERS_NB; + info.split.gid = 0; info.split.cid = topology_die_id(cpu); if (pmu_version >= 2) { @@ -641,6 +657,7 @@ int amd_uncore_df_ctx_init(struct amd_uncore *uncore, unsigned int cpu) pmu->num_counters = amd_uncore_ctx_num_pmcs(uncore, cpu); pmu->msr_base = MSR_F15H_NB_PERF_CTL; pmu->rdpmc_base = RDPMC_BASE_NB; + pmu->group = amd_uncore_ctx_gid(uncore, cpu); if (pmu_version >= 2) { *df_attr++ = &format_attr_event14v2.attr; @@ -734,6 +751,7 @@ void amd_uncore_l3_ctx_scan(struct amd_uncore *uncore, unsigned int cpu) info.split.aux_data = 0; info.split.num_pmcs = NUM_COUNTERS_L2; + info.split.gid = 0; info.split.cid = get_llc_id(cpu); if (boot_cpu_data.x86 >= 0x17) @@ -770,6 +788,7 @@ int amd_uncore_l3_ctx_init(struct amd_uncore *uncore, unsigned int cpu) pmu->num_counters = amd_uncore_ctx_num_pmcs(uncore, cpu); pmu->msr_base = MSR_F16H_L2I_PERF_CTL; pmu->rdpmc_base = RDPMC_BASE_LLC; + pmu->group = amd_uncore_ctx_gid(uncore, cpu); if (boot_cpu_data.x86 >= 0x17) { *l3_attr++ = &format_attr_event8.attr; From patchwork Thu Oct 5 05:23:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandipan Das X-Patchwork-Id: 148740 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp335858vqb; Thu, 5 Oct 2023 07:26:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGNuAgelL3NcI0l0BewNyx+YM8Z0gv4lv84fypJBPKgU2JlOLhTT9Zs/GMudLlO4jomgdkH X-Received: by 2002:a05:6a20:244f:b0:152:be08:b013 with SMTP id t15-20020a056a20244f00b00152be08b013mr6353675pzc.42.1696516011271; Thu, 05 Oct 2023 07:26:51 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1696516011; cv=pass; d=google.com; s=arc-20160816; b=dCHHydoK4U/n/iWeKL01dvGUrHlpKeDuQPuITSbLAp4yimd+eSklHfZu70DF4uQy0a B+6p3mBeLWeQoa05bdaEbXhoneeoQhEgcnJVn6lWUQYzovlr4tMT5zFSRDuz9R41nUqF uURBFjPrmeNPAsyjismNNQWiDQQR/uXGQnWk4hJUW0Wk37YyydKyLHZ6tdhMLa3MW4YH vsJ5/X5WWNOU4vrSfkdEMKfEEGXY0dA5us+VHyTr5PdIOBiV5zzg/YMc1/9cV9w2nPRp 5j6W+q+eqV2Ybqr12iqX2ZzPEXBs1rAJlM6SH0ci+PfdQqnUsAXpqmvlfCJCd1NL35FU FCBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uUdMjEm7deVyT2kl20ILuy91w+wua2S02CT89YQQHXI=; fh=DHqst7/L7qcDaGO45kwMD1/18OMA+IdBCna8yDZwwes=; b=iLyqxsbeXav9afyNB36seXartmXR/GniAde0/s7kq66yxuvGzcQH5PUw1Oe8tBLFLn 4CET09fJu11Qb7toew4hBQ5GY+DwGh4rskimoP0r1PMMI8vq2XJ/mSwO2v2J3QmiRXoB J4FYWDNs3LAlQwbJi15NP85CB5LVD5aLv5a8/H2SenIA1BiA9rlcI690MKWGOdr6oyY4 oRwgiEmkVAOD9CZKx2McfiX0LhbEZwP8VTNCpGMT1UdzVQ1Y1XgCU0pJ/gcCsDd6/X4D UVgZ+gnxYQvda2RyCxiadj8TK3igYSf4ig8ABPvLSq+UWmSguiHlvwcLyCJiriF/KThk N36g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=s5xEkKKU; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id bv11-20020a056a00414b00b0068fcb779a3esi1459884pfb.278.2023.10.05.07.26.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 07:26:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=s5xEkKKU; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 638BA81C19D3; Thu, 5 Oct 2023 07:26:15 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233225AbjJEOWW (ORCPT + 19 others); Thu, 5 Oct 2023 10:22:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232371AbjJEOUU (ORCPT ); Thu, 5 Oct 2023 10:20:20 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2061f.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::61f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76DCD4EF7; Wed, 4 Oct 2023 22:26:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Zij66N1wJS5G/NfiODgrrYDvRl/uRPaaSWSG0TON7GcSxyCxJEYa21f1UhBV0hekYfkjekQVjprpD2pyykgvv+JxkMfq54bqgoI9R/uCeeBPuPjAJdsIXwZKSWC8aoYUjwJiUGHf7uAi51bP3iKzq07+9+7dZxettU4V1YDL82HZUmNU5xlr761I8eJUwz7vfoIJM2fG9pgubNpSSsmDDVtem46l0wgqUbFaSRDIOR3+h6ACxkRvbR2oL26vIoy/l8LCocViIAUHIGwyfyghIgK3vGVznF44GbzpRiJ4MBKtBGocId6c6o7DvYPLtav59hce42CGlm/MdLZeNZhxhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uUdMjEm7deVyT2kl20ILuy91w+wua2S02CT89YQQHXI=; b=ZBbNy8i6XtC6mw8TzGXE8ErOmVM5I5nM4qdJZd5dmumG1xXXZgQAIOiYvjFwacWXausktx3efDx7SbQVIXRt22ccIbmmUdV7QDEiitCkb2ISQIp9Pa2WdkG9R7vJYtvSFYx74xjWDunFvlSDiQ6rd5gAqUZOSgGbdyDYSkXfaLZU0NJxx+w96/Ey3eq3d3+jHVAn3FG6CMCpRw1Bh0vqyuVFxbnJ69KqsA+P+l/NtyrVUI2d0AOqbNMVYHWWAdp3CBgKiPnjDIxPInNLWdpuUzUhVpaytcdBtDTjYd3HapyZnbwf3j+2UT++jZ66OzQ2XZnTg2i+tSxVwha/go07tg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uUdMjEm7deVyT2kl20ILuy91w+wua2S02CT89YQQHXI=; b=s5xEkKKUf6KHL+Q+iMH/omiN9EE/hEBGhNX++eMiPZhR+xmJTPKJK2pFBQQ09UQ0ySncsbfvDtQMRiPJlAjxuX1kae8IapulU1MqSDMh8ZvRJO5aQ6iLCQMbFYPM7LECD2w5Db4KUVZlTqVBWS/UjaH/26QoCNaNhhcyLOlLSgE= Received: from DM5PR07CA0111.namprd07.prod.outlook.com (2603:10b6:4:ae::40) by DM4PR12MB8450.namprd12.prod.outlook.com (2603:10b6:8:188::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.33; Thu, 5 Oct 2023 05:26:35 +0000 Received: from DS1PEPF0001708E.namprd03.prod.outlook.com (2603:10b6:4:ae:cafe::74) by DM5PR07CA0111.outlook.office365.com (2603:10b6:4:ae::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.26 via Frontend Transport; Thu, 5 Oct 2023 05:26:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DS1PEPF0001708E.mail.protection.outlook.com (10.167.17.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.14 via Frontend Transport; Thu, 5 Oct 2023 05:26:33 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 00:26:27 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH v2 5/6] perf/x86/amd/uncore: Add memory controller support Date: Thu, 5 Oct 2023 10:53:15 +0530 Message-ID: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF0001708E:EE_|DM4PR12MB8450:EE_ X-MS-Office365-Filtering-Correlation-Id: f3d22065-a943-4f72-4427-08dbc563a37a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GCqYyR6j/Zf5gY+GZyk02G7u6iK2dd2h2lp16jGIpD1zHR5YRt0/716M2Kk+2rdFPvX0HiyaEhyfBx8GSEPrjblyUjsTACXoY77yoR8EhWT4+wzLShCkG1XAif4nLWumNsT9wT++1FiOITLAyVA09h6udfjklcwHoKkuGZXm69WX/2ReQNVClFKmjqsMllet/hLQkr9TmdS5SyMmgrN0OlGvKqtGhS+bDuFPxReKWNJZxVN5+PcImmaMk14UXF7okhWHGlw6HIOAmc7aBCxvGcIZ3Z2FT1O1AZlPN6RomWXt1k/zycgb76OzgLFp5fiT/t6v47Dnv2itAWYvYqqkkXGUyJRLv8Qh4LcyezjmICh96oLENBgqisaulPbOJgCbkMsYgtOtDS6ky14nXWvivRodjZpFGZuyWB0BeGI8cWtWGK4EVq4bkluAuVY/W7HLTqTaZJRWjy5VpNN2ODpc6mjN/MJoDhyaLiLLXmCqgqKMo1kymJqC1xfDntNyVm7JGI5hiOh795Wep+mixQeKGUsPsLMSH8YLW7q9CHqjVIRGALzSnqjwAqNoRxMRWTaL2+4qM+3A4ylcWPXmZF6WJNwjW3U5BLFVNkim7bdNCoVjs7QuIbvlYTsVwTyph8QKzgQjUrRntgcGe1kwIRLZDKWQ9nhRvqInLXw4ej6zXRbelRD5pUfiViaz2btJQS+1HYLuuQskxXOcaYBNY/brHz8uvbZsaHFWj2ShfdEZnZQjLwJuD95yfIZ1IOjgP1+TjVVMOUI4I2aIytqFlj2+Ug== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:ErrorRetry;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(39860400002)(136003)(346002)(230922051799003)(64100799003)(82310400011)(451199024)(1800799009)(186009)(36840700001)(46966006)(40470700004)(110136005)(54906003)(70206006)(40480700001)(70586007)(316002)(36860700001)(41300700001)(44832011)(40460700003)(426003)(336012)(7416002)(2906002)(36756003)(83380400001)(7696005)(6666004)(26005)(5660300002)(4326008)(8936002)(8676002)(16526019)(2616005)(81166007)(82740400003)(356005)(478600001)(47076005)(86362001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 05:26:33.6940 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f3d22065-a943-4f72-4427-08dbc563a37a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF0001708E.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8450 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 05 Oct 2023 07:26:15 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778925973046926830 X-GMAIL-MSGID: 1778925973046926830 Unified Memory Controller (UMC) events were introduced with Zen 4 as a part of the Performance Monitoring Version 2 (PerfMonV2) enhancements. An event is specified using the EventSelect bits and the RdWrMask bits can be used for additional filtering of read and write requests. As of now, a maximum of 12 channels of DDR5 are available on each socket and each channel is controlled by a dedicated UMC. Each UMC, in turn, has its own set of performance monitoring counters. Since the MSR address space for the UMC PERF_CTL and PERF_CTR registers are reused across sockets, uncore groups are created on the basis of socket IDs. Hence, group exclusivity is mandatory while opening events so that events for an UMC can only be opened on CPUs which are on the same socket as the corresponding memory channel. For each socket, the total number of available UMC counters and active memory channels are determined from CPUID leaf 0x80000022 EBX and ECX respectively. Usually, on Zen 4, each UMC has four counters. MSR assignments are determined on the basis of active UMCs. E.g. if UMCs 1, 4 and 9 are active for a given socket, then * UMC 1 gets MSRs 0xc0010800 to 0xc0010807 as PERF_CTLs and PERF_CTRs * UMC 4 gets MSRs 0xc0010808 to 0xc001080f as PERF_CTLs and PERF_CTRs * UMC 9 gets MSRs 0xc0010810 to 0xc0010817 as PERF_CTLs and PERF_CTRs If there are sockets without any online CPUs when the amd_uncore driver is loaded, UMCs for such sockets will not be discoverable since the mechanism relies on executing the CPUID instruction on an online CPU from the socket. Signed-off-by: Sandipan Das --- arch/x86/events/amd/uncore.c | 156 +++++++++++++++++++++++++++++- arch/x86/include/asm/msr-index.h | 4 + arch/x86/include/asm/perf_event.h | 9 ++ 3 files changed, 168 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/amd/uncore.c b/arch/x86/events/amd/uncore.c index 318982951dd2..9b444ce24108 100644 --- a/arch/x86/events/amd/uncore.c +++ b/arch/x86/events/amd/uncore.c @@ -55,6 +55,7 @@ struct amd_uncore_pmu { enum { UNCORE_TYPE_DF, UNCORE_TYPE_L3, + UNCORE_TYPE_UMC, UNCORE_TYPE_MAX }; @@ -286,7 +287,7 @@ static struct device_attribute format_attr_##_var = \ DEFINE_UNCORE_FORMAT_ATTR(event12, event, "config:0-7,32-35"); DEFINE_UNCORE_FORMAT_ATTR(event14, event, "config:0-7,32-35,59-60"); /* F17h+ DF */ DEFINE_UNCORE_FORMAT_ATTR(event14v2, event, "config:0-7,32-37"); /* PerfMonV2 DF */ -DEFINE_UNCORE_FORMAT_ATTR(event8, event, "config:0-7"); /* F17h+ L3 */ +DEFINE_UNCORE_FORMAT_ATTR(event8, event, "config:0-7"); /* F17h+ L3, PerfMonV2 UMC */ DEFINE_UNCORE_FORMAT_ATTR(umask8, umask, "config:8-15"); DEFINE_UNCORE_FORMAT_ATTR(umask12, umask, "config:8-15,24-27"); /* PerfMonV2 DF */ DEFINE_UNCORE_FORMAT_ATTR(coreid, coreid, "config:42-44"); /* F19h L3 */ @@ -296,6 +297,7 @@ DEFINE_UNCORE_FORMAT_ATTR(threadmask2, threadmask, "config:56-57"); /* F19h L DEFINE_UNCORE_FORMAT_ATTR(enallslices, enallslices, "config:46"); /* F19h L3 */ DEFINE_UNCORE_FORMAT_ATTR(enallcores, enallcores, "config:47"); /* F19h L3 */ DEFINE_UNCORE_FORMAT_ATTR(sliceid, sliceid, "config:48-50"); /* F19h L3 */ +DEFINE_UNCORE_FORMAT_ATTR(rdwrmask, rdwrmask, "config:8-9"); /* PerfMonV2 UMC */ /* Common DF and NB attributes */ static struct attribute *amd_uncore_df_format_attr[] = { @@ -312,6 +314,13 @@ static struct attribute *amd_uncore_l3_format_attr[] = { NULL, }; +/* Common UMC attributes */ +static struct attribute *amd_uncore_umc_format_attr[] = { + &format_attr_event8.attr, /* event */ + &format_attr_rdwrmask.attr, /* rdwrmask */ + NULL, +}; + /* F17h unique L3 attributes */ static struct attribute *amd_f17h_uncore_l3_format_attr[] = { &format_attr_slicemask.attr, /* slicemask */ @@ -349,6 +358,11 @@ static struct attribute_group amd_f19h_uncore_l3_format_group = { .is_visible = amd_f19h_uncore_is_visible, }; +static struct attribute_group amd_uncore_umc_format_group = { + .name = "format", + .attrs = amd_uncore_umc_format_attr, +}; + static const struct attribute_group *amd_uncore_df_attr_groups[] = { &amd_uncore_attr_group, &amd_uncore_df_format_group, @@ -367,6 +381,12 @@ static const struct attribute_group *amd_uncore_l3_attr_update[] = { NULL, }; +static const struct attribute_group *amd_uncore_umc_attr_groups[] = { + &amd_uncore_attr_group, + &amd_uncore_umc_format_group, + NULL, +}; + static __always_inline int amd_uncore_ctx_cid(struct amd_uncore *uncore, unsigned int cpu) { @@ -835,6 +855,133 @@ int amd_uncore_l3_ctx_init(struct amd_uncore *uncore, unsigned int cpu) return amd_uncore_ctx_init(uncore, cpu); } +static int amd_uncore_umc_event_init(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + int ret = amd_uncore_event_init(event); + + if (ret) + return ret; + + hwc->config = event->attr.config & AMD64_PERFMON_V2_RAW_EVENT_MASK_UMC; + + return 0; +} + +static void amd_uncore_umc_start(struct perf_event *event, int flags) +{ + struct hw_perf_event *hwc = &event->hw; + + if (flags & PERF_EF_RELOAD) + wrmsrl(hwc->event_base, (u64)local64_read(&hwc->prev_count)); + + hwc->state = 0; + wrmsrl(hwc->config_base, (hwc->config | AMD64_PERFMON_V2_ENABLE_UMC)); + perf_event_update_userpage(event); +} + +static +void amd_uncore_umc_ctx_scan(struct amd_uncore *uncore, unsigned int cpu) +{ + union cpuid_0x80000022_ebx ebx; + union amd_uncore_info info; + unsigned int eax, ecx, edx; + + if (pmu_version < 2) + return; + + cpuid(EXT_PERFMON_DEBUG_FEATURES, &eax, &ebx.full, &ecx, &edx); + info.split.aux_data = ecx; /* stash active mask */ + info.split.num_pmcs = ebx.split.num_umc_pmc; + info.split.gid = topology_die_id(cpu); + info.split.cid = topology_die_id(cpu); + *per_cpu_ptr(uncore->info, cpu) = info; +} + +static +int amd_uncore_umc_ctx_init(struct amd_uncore *uncore, unsigned int cpu) +{ + DECLARE_BITMAP(gmask, UNCORE_GROUP_MAX) = { 0 }; + u8 group_num_pmus[UNCORE_GROUP_MAX] = { 0 }; + u8 group_num_pmcs[UNCORE_GROUP_MAX] = { 0 }; + union amd_uncore_info info; + struct amd_uncore_pmu *pmu; + int index = 0, gid, i; + + if (pmu_version < 2) + return 0; + + /* Run just once */ + if (uncore->init_done) + return amd_uncore_ctx_init(uncore, cpu); + + /* Find unique groups */ + for_each_online_cpu(i) { + info = *per_cpu_ptr(uncore->info, i); + gid = info.split.gid; + if (test_bit(gid, gmask)) + continue; + + __set_bit(gid, gmask); + group_num_pmus[gid] = hweight32(info.split.aux_data); + group_num_pmcs[gid] = info.split.num_pmcs; + uncore->num_pmus += group_num_pmus[gid]; + } + + uncore->pmus = kzalloc(sizeof(*uncore->pmus) * uncore->num_pmus, + GFP_KERNEL); + if (!uncore->pmus) { + uncore->num_pmus = 0; + goto done; + } + + for_each_set_bit(gid, gmask, UNCORE_GROUP_MAX) { + for (i = 0; i < group_num_pmus[gid]; i++) { + pmu = &uncore->pmus[index]; + snprintf(pmu->name, sizeof(pmu->name), "amd_umc_%d", index); + pmu->num_counters = group_num_pmcs[gid] / group_num_pmus[gid]; + pmu->msr_base = MSR_F19H_UMC_PERF_CTL + i * pmu->num_counters * 2; + pmu->rdpmc_base = -1; + pmu->group = gid; + + pmu->ctx = alloc_percpu(struct amd_uncore_ctx *); + if (!pmu->ctx) + goto done; + + pmu->pmu = (struct pmu) { + .task_ctx_nr = perf_invalid_context, + .attr_groups = amd_uncore_umc_attr_groups, + .name = pmu->name, + .event_init = amd_uncore_umc_event_init, + .add = amd_uncore_add, + .del = amd_uncore_del, + .start = amd_uncore_umc_start, + .stop = amd_uncore_stop, + .read = amd_uncore_read, + .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, + .module = THIS_MODULE, + }; + + if (perf_pmu_register(&pmu->pmu, pmu->pmu.name, -1)) { + free_percpu(pmu->ctx); + pmu->ctx = NULL; + goto done; + } + + pr_info("%d %s counters detected\n", pmu->num_counters, + pmu->pmu.name); + + index++; + } + } + +done: + uncore->num_pmus = index; + uncore->init_done = true; + + return amd_uncore_ctx_init(uncore, cpu); +} + static struct amd_uncore uncores[UNCORE_TYPE_MAX] = { /* UNCORE_TYPE_DF */ { @@ -850,6 +997,13 @@ static struct amd_uncore uncores[UNCORE_TYPE_MAX] = { .move = amd_uncore_ctx_move, .free = amd_uncore_ctx_free, }, + /* UNCORE_TYPE_UMC */ + { + .scan = amd_uncore_umc_ctx_scan, + .init = amd_uncore_umc_ctx_init, + .move = amd_uncore_ctx_move, + .free = amd_uncore_ctx_free, + }, }; static int __init amd_uncore_init(void) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 1d111350197f..dc159acb350a 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -637,6 +637,10 @@ /* AMD Last Branch Record MSRs */ #define MSR_AMD64_LBR_SELECT 0xc000010e +/* Fam 19h MSRs */ +#define MSR_F19H_UMC_PERF_CTL 0xc0010800 +#define MSR_F19H_UMC_PERF_CTR 0xc0010801 + /* Fam 17h MSRs */ #define MSR_F17H_IRPERF 0xc00000e9 diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h index 85a9fd5a3ec3..2618ec7c3d1d 100644 --- a/arch/x86/include/asm/perf_event.h +++ b/arch/x86/include/asm/perf_event.h @@ -112,6 +112,13 @@ (AMD64_PERFMON_V2_EVENTSEL_EVENT_NB | \ AMD64_PERFMON_V2_EVENTSEL_UMASK_NB) +#define AMD64_PERFMON_V2_ENABLE_UMC BIT_ULL(31) +#define AMD64_PERFMON_V2_EVENTSEL_EVENT_UMC GENMASK_ULL(7, 0) +#define AMD64_PERFMON_V2_EVENTSEL_RDWRMASK_UMC GENMASK_ULL(9, 8) +#define AMD64_PERFMON_V2_RAW_EVENT_MASK_UMC \ + (AMD64_PERFMON_V2_EVENTSEL_EVENT_UMC | \ + AMD64_PERFMON_V2_EVENTSEL_RDWRMASK_UMC) + #define AMD64_NUM_COUNTERS 4 #define AMD64_NUM_COUNTERS_CORE 6 #define AMD64_NUM_COUNTERS_NB 4 @@ -232,6 +239,8 @@ union cpuid_0x80000022_ebx { unsigned int lbr_v2_stack_sz:6; /* Number of Data Fabric Counters */ unsigned int num_df_pmc:6; + /* Number of Unified Memory Controller Counters */ + unsigned int num_umc_pmc:6; } split; unsigned int full; }; From patchwork Thu Oct 5 05:23:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandipan Das X-Patchwork-Id: 148714 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2016:b0:403:3b70:6f57 with SMTP id fe22csp324259vqb; Thu, 5 Oct 2023 07:09:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFgg0ia6um/ehq7tevTc5WPWiSnU3ROtVytWDqMmhiMhC/11hOnqcKsGusfaIq01XB+6aGF X-Received: by 2002:a05:6a00:2e89:b0:693:3851:bd98 with SMTP id fd9-20020a056a002e8900b006933851bd98mr5651218pfb.2.1696514995030; Thu, 05 Oct 2023 07:09:55 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1696514995; cv=pass; d=google.com; s=arc-20160816; b=Mm/wnjsUbd+yDASI2UCLLR3tVY64Hlm2qrsS+en2qGc7jDtw3Fn5VbXzT6gDgdjoAI B77l91ldyMyYsgqpzwt9Gz3sBLY52/bVGlLZisryIhN4WF7gAOA1vlXaGS56Zug+zXEW 4p/0Oy2LZE2DWP2587ges4joWpObTvROaB71Pv8jCUKqXUPoZWV9BvlBoeToEBC1cshF hyiLpRW7clVfpSJ5D8RPCc/oee0d2TP+jsOahDZtYq77pdn4AlROuVAMVRXciqS0hOOG 6zhXy9ER3rKxhI52Af65uMeB0uUf3Q/K8v9nLP6M4idX/SHi+wNNOFZKGO6MYIPjjbFd 1l7Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pasd/eTIvhNEgyrJP1iQxHoAyNdcqhtMCtq+P5vd+XY=; fh=DHqst7/L7qcDaGO45kwMD1/18OMA+IdBCna8yDZwwes=; b=HhC69Xxja4V1AAuYe44TmkYFKKOKeNAnkd/TG5984yygw1ghGOqXS6q3YWAlshWTU0 IuJFmbUVAe1272vSrAv8DQr9O7o2g0etES3GkUlqzUUCwXMhMp+Yy1HWAlsblFn2p/Du HBp7H0JSDnLsLfqn26kdJKtHHgBQR5oa+ICD0HL7tKud8kFSeFLiv8tPl+TBSmGjX8jZ IAlJUIhbU4FVQ++FJq8Wlym2zqwZq1YhbdcqdUPSvYKFOv/UazOm+uCkyEgPflQduWgJ kL4rg6olclYoWgbi875nsGbTwj8Gq2OcJwk69lHAp9Y3MZxKXQmXqzT2imM96JUwthLn VeIA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Czc+eeJM; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id m189-20020a633fc6000000b00565660b78d3si1505003pga.591.2023.10.05.07.09.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 07:09:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@amd.com header.s=selector1 header.b=Czc+eeJM; arc=pass (i=1 spf=pass spfdomain=amd.com dmarc=pass fromdomain=amd.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amd.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id C88BE802A0FB; Thu, 5 Oct 2023 07:09:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229896AbjJEOH1 (ORCPT + 19 others); Thu, 5 Oct 2023 10:07:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232029AbjJEOFR (ORCPT ); Thu, 5 Oct 2023 10:05:17 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eae::601]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B8964EF8; Wed, 4 Oct 2023 22:27:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GtUEfG/BvQq8qcTEYqI9uCEhLiuhljSYW2o6fIux2fjxvIMpOWgq2emAL1tn+4q9Ujexm9hzMgBizJ+F1PS4gnMvGdQGw25hC3uJiiXzFbQs9hmL7EuqGv4uSwEb7uDrfNa3+CxAu1fDFP3dU7QNLrXn5qXjRFtrAT+I1eyxaOLUVAM0eIXwqk4yRWos/UePGjPNEEA2IYrsGJpjZ3sY1BnXGEElSt4U6/KVG0Nq8DmTIl5WCqwZYUL1G31PkOI5QW2rQL4V63ihKjD6iGoi6jiPezmWo8nwEPWFqW8KXywKG3ZfHnP3/qvFAm5ziRqORaIYwoHUqEdOSg9X7+aRlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pasd/eTIvhNEgyrJP1iQxHoAyNdcqhtMCtq+P5vd+XY=; b=ZK5+/SAzOAcnIXwqrhE5Gpc0v9kMt1h0shWz5Bb31C/pusfRVrl7nwyH95BUa9H4MBiGV1iKiPLPryafIqPyfYGfBvbobKe3ErAVLjlYoVFCg60vqW6S6n/CvYEnpVLmsvOU+Qz9P/2NZpgZoitOVQ7mOAXEsUj2TLGj42TMvsrgYC0+18tbv7gUDenJP1/jUJqSKKgsFzK8QOfoY9nt/natoJoToMfjl7ekf9Wi3TII2G9MTqQyqi4yZkqwwRdRH+5OvngmGIzweu/vvmFEc0tAJXoiKo+PImzrEtnL7d8xfdNIIyNVeEafa8wAfGqjXyMSwQSIkRAKm8L7MwGNZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pasd/eTIvhNEgyrJP1iQxHoAyNdcqhtMCtq+P5vd+XY=; b=Czc+eeJMh5HHk9pICG1vCPe+lzongqnJGQ8Ua2CcaNUHkkyaVBVcSGtVIikQR/aQ0h/FPJOW+9ltD3ODtNDlino9pdvNvEE+4iAdFeMVScy3yhV9H9lyB7x0/CuB7GiAJ5nka0Na7jzrO0sY9limafH5bhisosEw5f6dzAU32+E= Received: from DM6PR06CA0071.namprd06.prod.outlook.com (2603:10b6:5:54::48) by PH7PR12MB7843.namprd12.prod.outlook.com (2603:10b6:510:27e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.37; Thu, 5 Oct 2023 05:26:53 +0000 Received: from DS1PEPF00017090.namprd03.prod.outlook.com (2603:10b6:5:54:cafe::ee) by DM6PR06CA0071.outlook.office365.com (2603:10b6:5:54::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.26 via Frontend Transport; Thu, 5 Oct 2023 05:26:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DS1PEPF00017090.mail.protection.outlook.com (10.167.17.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.14 via Frontend Transport; Thu, 5 Oct 2023 05:26:53 +0000 Received: from sindhu.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 5 Oct 2023 00:26:46 -0500 From: Sandipan Das To: , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH v2 6/6] perf vendor events amd: Add Zen 4 memory controller events Date: Thu, 5 Oct 2023 10:53:16 +0530 Message-ID: <3a7245598265737ae4296fa6b0ab03a51708c502.1696425185.git.sandipan.das@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS1PEPF00017090:EE_|PH7PR12MB7843:EE_ X-MS-Office365-Filtering-Correlation-Id: cc401e67-02ee-4c97-91ac-08dbc563af00 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HCBhQ7AIZhjFLuErfBbZlE1aoyUuGwRUyU3LixzxCpjX/54fhdtwr8nEyJk3zK7g73NLdDJLRxs3w6MvhFZFPfdOn0jAFXdoxvySMDXqtLStqJIC/nxb7QviL2jIQKVOoU7PzJpJxjw2EBAiUFuYxeIUzYCXALgF+Vklp6jODzeIz4YI0fW+v96dV9SHfkjwUqTqmadoZoAAQNhLi0xtcrzKft/GzegtqOsU7syn2KE9Edp+l3CIziCupg+zL5SrApUSysaY3B5kX0MzT9E0cqMgW1zkvyoLkZA+yJFUfxMC9aAmiH7j8S/WSI7pdKWbn2K577ppJ8mud8xov0MfdbleGONjBIl3uiNxHII9bhB/Zw/q1onstAlnOWePfD68aizXEtJe9i9t/wOn/wRSxw34xpv2wjG4QAkfCRb+1F2q6cRifr17FtojsWNWWWwhSGrU/+RzEFQMjn7QuDAS09w+akuAa4UwC625i5z8atNXoJKlS92hGxZW01lL6p39W9ua+HySK+ZBla7mV6xVqJSCeXxaa1SWA8/4q6xCSnL//A/+d4JGvp7jXGPXpw1zvoOAiUYpZdxGmWwJCQz0mWZ0yZuoYSXz4rBHXpW3gsNLeRoZQKMXfRx7pVqX8Sakh+o4vMIlouCauIwIsm16l+xEJDijdPW7zrHJm3O9czI2mYcMP5nDPoNS74KX41KtVhnlHuJMPLoz1ejOQQ8c3xhX2tbTeb1B+dbtRhIJYUT76zOvrHe5zZVQlStNz0xE1g3DA8K2+5cuFWrSWPTuPA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(136003)(396003)(39860400002)(346002)(230922051799003)(82310400011)(1800799009)(451199024)(64100799003)(186009)(40470700004)(36840700001)(46966006)(7416002)(44832011)(5660300002)(8676002)(2906002)(4326008)(8936002)(41300700001)(70206006)(70586007)(2616005)(316002)(40460700003)(54906003)(110136005)(36860700001)(40480700001)(26005)(336012)(426003)(6666004)(36756003)(7696005)(16526019)(356005)(81166007)(82740400003)(83380400001)(86362001)(47076005)(478600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2023 05:26:53.0760 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cc401e67-02ee-4c97-91ac-08dbc563af00 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS1PEPF00017090.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7843 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 05 Oct 2023 07:09:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778924907261327415 X-GMAIL-MSGID: 1778924907261327415 Make the jevents parser aware of the Unified Memory Controller (UMC) PMU and add events taken from Section 8.2.1 "UMC Performance Monitor Events" of the Processor Programming Reference (PPR) for AMD Family 19h Model 11h processors. The events capture UMC command activity such as CAS, ACTIVATE, PRECHARGE etc. while the metrics derive data bus utilization and memory bandwidth out of these events. Signed-off-by: Sandipan Das Acked-by: Ian Rogers --- .../arch/x86/amdzen4/memory-controller.json | 101 ++++++++++++++++++ .../arch/x86/amdzen4/recommended.json | 84 +++++++++++++++ tools/perf/pmu-events/jevents.py | 2 + 3 files changed, 187 insertions(+) create mode 100644 tools/perf/pmu-events/arch/x86/amdzen4/memory-controller.json diff --git a/tools/perf/pmu-events/arch/x86/amdzen4/memory-controller.json b/tools/perf/pmu-events/arch/x86/amdzen4/memory-controller.json new file mode 100644 index 000000000000..55263e5e4f69 --- /dev/null +++ b/tools/perf/pmu-events/arch/x86/amdzen4/memory-controller.json @@ -0,0 +1,101 @@ +[ + { + "EventName": "umc_mem_clk", + "PublicDescription": "Number of memory clock cycles.", + "EventCode": "0x00", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_act_cmd.all", + "PublicDescription": "Number of ACTIVATE commands sent.", + "EventCode": "0x05", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_act_cmd.rd", + "PublicDescription": "Number of ACTIVATE commands sent for reads.", + "EventCode": "0x05", + "RdWrMask": "0x1", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_act_cmd.wr", + "PublicDescription": "Number of ACTIVATE commands sent for writes.", + "EventCode": "0x05", + "RdWrMask": "0x2", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_pchg_cmd.all", + "PublicDescription": "Number of PRECHARGE commands sent.", + "EventCode": "0x06", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_pchg_cmd.rd", + "PublicDescription": "Number of PRECHARGE commands sent for reads.", + "EventCode": "0x06", + "RdWrMask": "0x1", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_pchg_cmd.wr", + "PublicDescription": "Number of PRECHARGE commands sent for writes.", + "EventCode": "0x06", + "RdWrMask": "0x2", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_cas_cmd.all", + "PublicDescription": "Number of CAS commands sent.", + "EventCode": "0x0a", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_cas_cmd.rd", + "PublicDescription": "Number of CAS commands sent for reads.", + "EventCode": "0x0a", + "RdWrMask": "0x1", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_cas_cmd.wr", + "PublicDescription": "Number of CAS commands sent for writes.", + "EventCode": "0x0a", + "RdWrMask": "0x2", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_data_slot_clks.all", + "PublicDescription": "Number of clocks used by the data bus.", + "EventCode": "0x14", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_data_slot_clks.rd", + "PublicDescription": "Number of clocks used by the data bus for reads.", + "EventCode": "0x14", + "RdWrMask": "0x1", + "PerPkg": "1", + "Unit": "UMCPMC" + }, + { + "EventName": "umc_data_slot_clks.wr", + "PublicDescription": "Number of clocks used by the data bus for writes.", + "EventCode": "0x14", + "RdWrMask": "0x2", + "PerPkg": "1", + "Unit": "UMCPMC" + } +] diff --git a/tools/perf/pmu-events/arch/x86/amdzen4/recommended.json b/tools/perf/pmu-events/arch/x86/amdzen4/recommended.json index 5e6a793acf7b..96e06401c6cb 100644 --- a/tools/perf/pmu-events/arch/x86/amdzen4/recommended.json +++ b/tools/perf/pmu-events/arch/x86/amdzen4/recommended.json @@ -330,5 +330,89 @@ "MetricGroup": "data_fabric", "PerPkg": "1", "ScaleUnit": "6.103515625e-5MiB" + }, + { + "MetricName": "umc_data_bus_utilization", + "BriefDescription": "Memory controller data bus utilization.", + "MetricExpr": "d_ratio(umc_data_slot_clks.all / 2, umc_mem_clk)", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "100%" + }, + { + "MetricName": "umc_cas_cmd_rate", + "BriefDescription": "Memory controller CAS command rate.", + "MetricExpr": "d_ratio(umc_cas_cmd.all * 1000, umc_mem_clk)", + "MetricGroup": "memory_controller", + "PerPkg": "1" + }, + { + "MetricName": "umc_cas_cmd_read_ratio", + "BriefDescription": "Ratio of memory controller CAS commands for reads.", + "MetricExpr": "d_ratio(umc_cas_cmd.rd, umc_cas_cmd.all)", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "100%" + }, + { + "MetricName": "umc_cas_cmd_write_ratio", + "BriefDescription": "Ratio of memory controller CAS commands for writes.", + "MetricExpr": "d_ratio(umc_cas_cmd.wr, umc_cas_cmd.all)", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "100%" + }, + { + "MetricName": "umc_mem_read_bandwidth", + "BriefDescription": "Estimated memory read bandwidth.", + "MetricExpr": "(umc_cas_cmd.rd * 64) / 1e6 / duration_time", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "1MB/s" + }, + { + "MetricName": "umc_mem_write_bandwidth", + "BriefDescription": "Estimated memory write bandwidth.", + "MetricExpr": "(umc_cas_cmd.wr * 64) / 1e6 / duration_time", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "1MB/s" + }, + { + "MetricName": "umc_mem_bandwidth", + "BriefDescription": "Estimated combined memory bandwidth.", + "MetricExpr": "(umc_cas_cmd.all * 64) / 1e6 / duration_time", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "1MB/s" + }, + { + "MetricName": "umc_cas_cmd_read_ratio", + "BriefDescription": "Ratio of memory controller CAS commands for reads.", + "MetricExpr": "d_ratio(umc_cas_cmd.rd, umc_cas_cmd.all)", + "MetricGroup": "memory_controller", + "PerPkg": "1", + "ScaleUnit": "100%" + }, + { + "MetricName": "umc_cas_cmd_rate", + "BriefDescription": "Memory controller CAS command rate.", + "MetricExpr": "d_ratio(umc_cas_cmd.all * 1000, umc_mem_clk)", + "MetricGroup": "memory_controller", + "PerPkg": "1" + }, + { + "MetricName": "umc_activate_cmd_rate", + "BriefDescription": "Memory controller ACTIVATE command rate.", + "MetricExpr": "d_ratio(umc_act_cmd.all * 1000, umc_mem_clk)", + "MetricGroup": "memory_controller", + "PerPkg": "1" + }, + { + "MetricName": "umc_precharge_cmd_rate", + "BriefDescription": "Memory controller PRECHARGE command rate.", + "MetricExpr": "d_ratio(umc_pchg_cmd.all * 1000, umc_mem_clk)", + "MetricGroup": "memory_controller", + "PerPkg": "1" } ] diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py index 72ba4a9239c6..ff202e0b7b0a 100755 --- a/tools/perf/pmu-events/jevents.py +++ b/tools/perf/pmu-events/jevents.py @@ -286,6 +286,7 @@ class JsonEvent: 'imx8_ddr': 'imx8_ddr', 'L3PMC': 'amd_l3', 'DFPMC': 'amd_df', + 'UMCPMC': 'amd_umc', 'cpu_core': 'cpu_core', 'cpu_atom': 'cpu_atom', 'ali_drw': 'ali_drw', @@ -345,6 +346,7 @@ class JsonEvent: ('Invert', 'inv='), ('SampleAfterValue', 'period='), ('UMask', 'umask='), + ('RdWrMask', 'rdwrmask='), ] for key, value in event_fields: if key in jd and jd[key] != '0':