From patchwork Mon Nov 20 22:19:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 167410 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp252046vqb; Mon, 20 Nov 2023 14:20:04 -0800 (PST) X-Google-Smtp-Source: AGHT+IEAVRubJCBBBQPs1M4iZtTN/rqc8/UjhQ/YcVNryPJVHvsFd/YSr2KQNzhDNlnC/T64DTrP X-Received: by 2002:a05:6a20:1604:b0:185:a90d:363d with SMTP id l4-20020a056a20160400b00185a90d363dmr7579918pzj.2.1700518804027; Mon, 20 Nov 2023 14:20:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700518804; cv=none; d=google.com; s=arc-20160816; b=u/5c3tgdlbnwimvVwFh2LnvrQTvBnALD+OJtg8rh86x7KJP+CbVCcJm5x+5M+ihEII RACjUWO3dy6JXXIzwyojxbIkzJ6OYSrM3ZLGgQOcub6dRVNqr11pYh4t8i6Q0/E9fIFn hxdquAQPWUc6mE4NE53VzU9fkNLjjjLu0MA+5iN5pjLbLKJEEPd6jQ8r1Q4nsekQTNuI pl16XIhxGMBkLZ4j/WMRKfu0Iqc+jOpHbeNaELt6+72UJXDdd1+PAtm+/eR2jBJEDO3M 9cJZS+vM+oq8yF/ARIJOrxs4OIL1gnNUNytPYQLlkySzuVpCoeoEhtcG9duj/iuQhyWG 1KqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=6XC8XlavK49xNjfUz810sKQmeHvQWvE5qIjS3kpF1wE=; fh=b3TeBIs+CttSBZm6Hnbb3kJFExQaS46Vc9Vx9fOC4sY=; b=WnH7uy1smgezywFr4prn23jQwxBrTIgIAKHgdDtm6u4Kt7ZeBr5cJ/ibrZ2tRqY4Rn KufTzpQlXCeVxIlgXQxxze4WLFLbaPGixdEnjktGVU5LI1+bjLLV6lISiqyqHr/dZCb+ NxM6mZYVn6wILEwQ7bLd0AQBN+5IBF8xvbmevnKygrpCd28hHrKWwOJyaxT4kfJuLrT8 2bu7cFAeozg7i/rAp44K00CfBJkd9M1dSmM9Yy7K8f7kYqR7HVd8KEKPeSGUx8YMA3em JUBfvw7o68dGd1hHe4pxWYZdJEU/E8a7zJp9cmc+oT2iWjq7mivNwKuFcov9y3lWb4lb 9SWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Mm5lJ0bv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id bm22-20020a656e96000000b005bddb7ff530si9477833pgb.124.2023.11.20.14.20.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 14:20:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Mm5lJ0bv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 006C2802F30A; Mon, 20 Nov 2023 14:20:03 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232620AbjKTWTm (ORCPT + 27 others); Mon, 20 Nov 2023 17:19:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232689AbjKTWTi (ORCPT ); Mon, 20 Nov 2023 17:19:38 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C374ACA for ; Mon, 20 Nov 2023 14:19:34 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1cc4f777ab9so38675465ad.0 for ; Mon, 20 Nov 2023 14:19:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700518774; x=1701123574; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=6XC8XlavK49xNjfUz810sKQmeHvQWvE5qIjS3kpF1wE=; b=Mm5lJ0bvDxChRHIn951jLh/fx2lDrdn/3fJ1BotetQ0vSkNo2i/D8qsxVEWl0LfwED uEwoNnxwaRZFjmb3y+PrbGY3pXDobRwNUNbJNDgSQ+ZiXS/sQUQOeY2xvQEcFENDJhPy kbVi8/VJ8dz8vBQMTXqMIFpIobxtMOAruHJiKtJWWo2oJduKIpuI0iEtSg2J1IpfUz0D 0eUhnLa4Yhn33355fjyoJQV4XObXSH5gfd730YJzwA2kucmCSnpUl//WOsPZGcIfDXYh NGubz5RxoDCNSTTvg7mDaelhDnnoqaaMdNZt7GA2J1rJ6t9cN4zfWFMYneYRiaZtl/CF eGOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700518774; x=1701123574; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6XC8XlavK49xNjfUz810sKQmeHvQWvE5qIjS3kpF1wE=; b=r3B9OTVJlUwBWTWrpfT5bLn8fAErAy32nE+uE/llHOElz1ygiyuEcrSPUucvdQDxdB U6BvRUlvr/jwNIRi2YaAxKXphsgOAYnGVcA6f2jMRTVwi0Z1cJ5LjCmQMDn+UGZcqDpT WhqH6bbz2+cRRAfqRGDwNaFdTw/XjMKoQiR7VVrB0CuF8s2ltcvATvT8ijHRLXq3GAe5 Xw01uA0qACy/jbik/jzw2Kuw31GefJYnYZ8W9RXmmukPZyLyMGgOsb0vccJvvI1oARr0 zKS5Llg4gAB5qZp3iX9U1X6Zymfduj3/T+1vz1/HOOAG486qzeAZTJLco1ae9JxcH9SZ Qykg== X-Gm-Message-State: AOJu0Ywfsil389fUsKTAAHWWwP+qFd0mI/euV0fqo7CBe/e8BbgW++Dr RmTQ6KeEZuwqCXve63PCay0= X-Received: by 2002:a17:902:b486:b0:1ca:e78b:35dc with SMTP id y6-20020a170902b48600b001cae78b35dcmr6445266plr.27.1700518774127; Mon, 20 Nov 2023 14:19:34 -0800 (PST) Received: from bangji.hsd1.ca.comcast.net ([2601:647:6780:42e0:75c6:4212:ae99:93b6]) by smtp.gmail.com with ESMTPSA id d4-20020a170902c18400b001c9c47d6cb9sm3528246pld.99.2023.11.20.14.19.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 14:19:33 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , LKML , Ian Rogers , Kan Liang , Mingwei Zhang Subject: [PATCH 1/3] perf/core: Update perf_adjust_freq_unthr_context() Date: Mon, 20 Nov 2023 14:19:30 -0800 Message-ID: <20231120221932.213710-1-namhyung@kernel.org> X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 14:20:03 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783123205647483092 X-GMAIL-MSGID: 1783123205647483092 It was unnecessarily disabling and enabling PMUs for each event. It should be done at PMU level. Add pmu_ctx->nr_freq counter to check it at each PMU. As pmu context has separate active lists for pinned group and flexible group, factor out a new function to do the job. Another minor optimization is that it can skip PMUs w/ CAP_NO_INTERRUPT even if it needs to unthrottle sampling events. Signed-off-by: Namhyung Kim Reviewed-by: Ian Rogers Reviewed-by: Kan Liang Tested-by: Mingwei Zhang --- include/linux/perf_event.h | 1 + kernel/events/core.c | 68 +++++++++++++++++++++++--------------- 2 files changed, 43 insertions(+), 26 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 0367d748fae0..3eb17dc89f5e 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -879,6 +879,7 @@ struct perf_event_pmu_context { unsigned int nr_events; unsigned int nr_cgroups; + unsigned int nr_freq; atomic_t refcount; /* event <-> epc */ struct rcu_head rcu_head; diff --git a/kernel/events/core.c b/kernel/events/core.c index 3eb26c2c6e65..53e2ad73102d 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -2275,8 +2275,10 @@ event_sched_out(struct perf_event *event, struct perf_event_context *ctx) if (!is_software_event(event)) cpc->active_oncpu--; - if (event->attr.freq && event->attr.sample_freq) + if (event->attr.freq && event->attr.sample_freq) { ctx->nr_freq--; + epc->nr_freq--; + } if (event->attr.exclusive || !cpc->active_oncpu) cpc->exclusive = 0; @@ -2531,9 +2533,10 @@ event_sched_in(struct perf_event *event, struct perf_event_context *ctx) if (!is_software_event(event)) cpc->active_oncpu++; - if (event->attr.freq && event->attr.sample_freq) + if (event->attr.freq && event->attr.sample_freq) { ctx->nr_freq++; - + epc->nr_freq++; + } if (event->attr.exclusive) cpc->exclusive = 1; @@ -4096,30 +4099,14 @@ static void perf_adjust_period(struct perf_event *event, u64 nsec, u64 count, bo } } -/* - * combine freq adjustment with unthrottling to avoid two passes over the - * events. At the same time, make sure, having freq events does not change - * the rate of unthrottling as that would introduce bias. - */ -static void -perf_adjust_freq_unthr_context(struct perf_event_context *ctx, bool unthrottle) +static void perf_adjust_freq_unthr_events(struct list_head *event_list) { struct perf_event *event; struct hw_perf_event *hwc; u64 now, period = TICK_NSEC; s64 delta; - /* - * only need to iterate over all events iff: - * - context have events in frequency mode (needs freq adjust) - * - there are events to unthrottle on this cpu - */ - if (!(ctx->nr_freq || unthrottle)) - return; - - raw_spin_lock(&ctx->lock); - - list_for_each_entry_rcu(event, &ctx->event_list, event_entry) { + list_for_each_entry(event, event_list, active_list) { if (event->state != PERF_EVENT_STATE_ACTIVE) continue; @@ -4127,8 +4114,6 @@ perf_adjust_freq_unthr_context(struct perf_event_context *ctx, bool unthrottle) if (!event_filter_match(event)) continue; - perf_pmu_disable(event->pmu); - hwc = &event->hw; if (hwc->interrupts == MAX_INTERRUPTS) { @@ -4138,7 +4123,7 @@ perf_adjust_freq_unthr_context(struct perf_event_context *ctx, bool unthrottle) } if (!event->attr.freq || !event->attr.sample_freq) - goto next; + continue; /* * stop the event and update event->count @@ -4160,8 +4145,39 @@ perf_adjust_freq_unthr_context(struct perf_event_context *ctx, bool unthrottle) perf_adjust_period(event, period, delta, false); event->pmu->start(event, delta > 0 ? PERF_EF_RELOAD : 0); - next: - perf_pmu_enable(event->pmu); + } +} + +/* + * combine freq adjustment with unthrottling to avoid two passes over the + * events. At the same time, make sure, having freq events does not change + * the rate of unthrottling as that would introduce bias. + */ +static void +perf_adjust_freq_unthr_context(struct perf_event_context *ctx, bool unthrottle) +{ + struct perf_event_pmu_context *pmu_ctx; + + /* + * only need to iterate over all events iff: + * - context have events in frequency mode (needs freq adjust) + * - there are events to unthrottle on this cpu + */ + if (!(ctx->nr_freq || unthrottle)) + return; + + raw_spin_lock(&ctx->lock); + + list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) { + if (!(pmu_ctx->nr_freq || unthrottle)) + continue; + if (pmu_ctx->pmu->capabilities & PERF_PMU_CAP_NO_INTERRUPT) + continue; + + perf_pmu_disable(pmu_ctx->pmu); + perf_adjust_freq_unthr_events(&pmu_ctx->pinned_active); + perf_adjust_freq_unthr_events(&pmu_ctx->flexible_active); + perf_pmu_enable(pmu_ctx->pmu); } raw_spin_unlock(&ctx->lock); From patchwork Mon Nov 20 22:19:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 167411 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp252055vqb; Mon, 20 Nov 2023 14:20:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IE38Lymslnbp/pxGQNaA2B58Xmg/aZFVRkIx9EHR9Q23cUHhavNTOxPIEN9KPvrl8UoGjgW X-Received: by 2002:a05:6830:22f0:b0:6d6:490f:f027 with SMTP id t16-20020a05683022f000b006d6490ff027mr8984532otc.37.1700518805534; Mon, 20 Nov 2023 14:20:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700518805; cv=none; d=google.com; s=arc-20160816; b=ZLBcbTDbzso1HaZtboUX2a4SVbxpHrKvcQCfU8p0wmWcMxjFMOOeHt93JkI1GTtXRX 7ftIKZESaJREeH1lvX49OCW9utAMSoKqXdk2RFzk55Fl2RX+5cdeDcwTIF/Ju5EY2RQ1 H+yqYXVuCkXQSwTDl52ipnBSIz1n9UdBBolufn/OYBZLWHtnpyJc1Y6mgB7FjHNLCYm9 8K8KdnFn7n/pyqAvROo2jE1Sl7q63j7umh8HgrX85KGGGJtLAQLIYh9WdcQzdmH1X3Ug rFJuXMfvArPEshtXKpHhgDum9OnbnVb9q1t6H4ana9Z4AxP48m94e5a5FjXaPT60PQlU oydg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=2XOoFK7iiRtg8F9n1yVwN4BqrDcgETLBQgehwgQkqSk=; fh=b3TeBIs+CttSBZm6Hnbb3kJFExQaS46Vc9Vx9fOC4sY=; b=iAhAc0LnOdfeIxpIs2Fk1nrcSpYKzQZVy537WuHEwRz2g7vgpzW4SXaY+FLtUNmTg8 WuDTr3ZXksKaL89KpQpLQfpHr4cgZmVT/xlxdEhX5g0Y4uA8lFi6nphGliLwqHnStN62 9nsXSoSHB3algF4V9oYFj3NPNgX7DKVQKkjZF/bf76CqxndKiultDaq1WZoDmkyH8WPJ JIRwAEHqQ6iyZfRV/aSkzKNS5lbOHzrU7gDBh15hhVu9oVOCcjJ1Ln8SdOhigwf2MaPT ofp5xTs0IcSSGc7Lw5CpN5A/BWiyERZaqOO1koPOk+hDeXjohKM9w/Qiw6xo2X35VF0y aFIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=YktyFqrI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id c27-20020a630d1b000000b005c2060c72fbsi8315994pgl.166.2023.11.20.14.20.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 14:20:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=YktyFqrI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id E3794809B749; Mon, 20 Nov 2023 14:19:59 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232626AbjKTWTl (ORCPT + 27 others); Mon, 20 Nov 2023 17:19:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229798AbjKTWTj (ORCPT ); Mon, 20 Nov 2023 17:19:39 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3511ED for ; Mon, 20 Nov 2023 14:19:35 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1cf6af8588fso4598305ad.0 for ; Mon, 20 Nov 2023 14:19:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700518775; x=1701123575; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=2XOoFK7iiRtg8F9n1yVwN4BqrDcgETLBQgehwgQkqSk=; b=YktyFqrINYIrDa2sVdT9aw/3Nv6up+b0BlReQBNqLCzQaC2aB5bh4d4owSVBMc7bKu DhG9U0c3IkBP1fhHK6UMRZc/8uM1hr4i7fBe3rDJuHLmZi4c3DdWaP+ZkK/J8I6eOUTv dOj/6LINxR0oKsgggtdt6oGO9p2yPesBl3OkJlAI9inEdxAlkqd/+DXBhwfenZHYgEe8 sKX5I9faUGnQJcictpeHNWc7CiWFqlYsfRgzmESHLj7lQ+94pSCXz8LQQu89NVqfot80 qUSpvU4QxxGA9BCK/GNSuDaB1xBYNwEUAReZS7NaU5ljsAgQqoN33RrMe8NcJ3I3KrX8 bV0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700518775; x=1701123575; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=2XOoFK7iiRtg8F9n1yVwN4BqrDcgETLBQgehwgQkqSk=; b=SuW/Yypu2azxNLcDX8BOgoPKeXVar/xkO2cC/KnXnVNaWBglvNYRcMgYq/N3EqZJRf 2e1IALH7G3pXKRHg+9DW1+taE7D3pwdvnzsY0RWsGUMFUbl8KEKjKKKJzObN6nx6ZG7I 9AcbhFF6nwME2OMTyVOB15CLik/b14Up3HNjlcDMVz80ji9aCaT5zWBT2H26Ratv9U3+ oOZhm9sZVOWkEkYPNTk6dN6KxzwUTEj3Dt17VPtzzy7rdn+cRbC3oymd8W86LgYhilit DzvNIwf3n+q4dSykr9EsJ/uCTTuhtRuhWxvqyrKAVa6ee41I41/FPHRTR0++G8S21PHT JsuQ== X-Gm-Message-State: AOJu0Yw9MnpuEvZ8QYbxqj3Vj+E9Ye8P06zP7F0WzKoLMzO9s3szisKo VPQlNHrSUOazvmQLSfl3CRI= X-Received: by 2002:a17:902:7682:b0:1b8:954c:1f6 with SMTP id m2-20020a170902768200b001b8954c01f6mr10400426pll.36.1700518775351; Mon, 20 Nov 2023 14:19:35 -0800 (PST) Received: from bangji.hsd1.ca.comcast.net ([2601:647:6780:42e0:75c6:4212:ae99:93b6]) by smtp.gmail.com with ESMTPSA id d4-20020a170902c18400b001c9c47d6cb9sm3528246pld.99.2023.11.20.14.19.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 14:19:35 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , LKML , Ian Rogers , Kan Liang , Mingwei Zhang Subject: [PATCH 2/3] perf/core: Reduce PMU access to adjust sample freq Date: Mon, 20 Nov 2023 14:19:31 -0800 Message-ID: <20231120221932.213710-2-namhyung@kernel.org> X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog In-Reply-To: <20231120221932.213710-1-namhyung@kernel.org> References: <20231120221932.213710-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 20 Nov 2023 14:20:00 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783123206525588576 X-GMAIL-MSGID: 1783123206525588576 For throttled events, it first starts the event and then stop unnecessarily. As it's already stopped, it can directly adjust the frequency and then move on. Signed-off-by: Namhyung Kim Reviewed-by: Kan Liang --- kernel/events/core.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 53e2ad73102d..fd3449e4d081 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -4119,10 +4119,15 @@ static void perf_adjust_freq_unthr_events(struct list_head *event_list) if (hwc->interrupts == MAX_INTERRUPTS) { hwc->interrupts = 0; perf_log_throttle(event, 1); - event->pmu->start(event, 0); - } - if (!event->attr.freq || !event->attr.sample_freq) + if (!event->attr.freq || !event->attr.sample_freq) { + delta = 0; + goto next; + } + + if (event->hw.state & PERF_HES_STOPPED) + goto adjust; + } else if (!event->attr.freq || !event->attr.sample_freq) continue; /* @@ -4130,6 +4135,7 @@ static void perf_adjust_freq_unthr_events(struct list_head *event_list) */ event->pmu->stop(event, PERF_EF_UPDATE); +adjust: now = local64_read(&event->count); delta = now - hwc->freq_count_stamp; hwc->freq_count_stamp = now; @@ -4144,6 +4150,7 @@ static void perf_adjust_freq_unthr_events(struct list_head *event_list) if (delta > 0) perf_adjust_period(event, period, delta, false); +next: event->pmu->start(event, delta > 0 ? PERF_EF_RELOAD : 0); } } From patchwork Mon Nov 20 22:19:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 167412 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp252084vqb; Mon, 20 Nov 2023 14:20:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IHISBRmpYFKjMS/y3C1bUu02dFzr3CrpLaE4KywxGyDE0M7g+xCQokuPwnppEu2F0RsC5JW X-Received: by 2002:a05:6a20:3942:b0:189:c852:562e with SMTP id r2-20020a056a20394200b00189c852562emr9758336pzg.38.1700518808549; Mon, 20 Nov 2023 14:20:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700518808; cv=none; d=google.com; s=arc-20160816; b=eerglpaC4QTrsmh6mYTatlN+ufVvVeu1hkir2ZY5L0Q8Zbgreq+hi2NgmSJZsgiObk C6WrGsg+K+siwgV0ZAYB4bLIZ15XHLCLqOVZBXN+2YRhi7GaZKwet8QFJC/FsbekChow CWEXEQST2YoathVtQN7WqRXzYD8xbrmJKqrTgEVmUnsDtUNPzEuXkzJyPbrZWzkH7Qk5 i/OwjpH2aSaYuk84hhY6o2v53OmEAV/Tc50Kh4JmLz4Fzj9miUD63q1r53hZ+jkaaE/0 0JitWBmTferGZq7TIpUhSnLrH7CYNrrojK28aJYYWt9cDqor8gHcgf7RIQEAPj7uWXL+ R0NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=ZU36NcT9RPyJC6TQdfEVGdreh7ck6Zb0oyJfbLqLJ4s=; fh=b3TeBIs+CttSBZm6Hnbb3kJFExQaS46Vc9Vx9fOC4sY=; b=YDtP4XoEj4yWprYaL9n/hse3skhpUHWORTYajT3nDdH0aQcuvIZVH7VFrYATt/+PGD kHCs6oRw/W1B0QcfD/NDI6kmvKE3X4NI2K9N4PcC/UE+WOCJXBiYtqhlYxyYmmf3LXVX PwhLqiR6/b0SX95INAel3ZCikQMlwGaDGeM639Ft/Pn5UdUEnvF7GIf7clgObybLYCzu /2teLcPGGNF5VBcFhrSiVBVH3Os8AHvCDDPMZhX1MhZu26dECOipFHmd28yz7oJhy0JY hGvcR6EC+wzlVnSmOUfJyB8d7azxGnfRaSUjaXoMofTPSgwUvUpCSPh5xr4cMLSxPX0J pp1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=BHBZSCe9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id i17-20020a170902c95100b001c3da86939csi9244059pla.546.2023.11.20.14.20.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 14:20:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=BHBZSCe9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 62A358032EEF; Mon, 20 Nov 2023 14:20:06 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232677AbjKTWTp (ORCPT + 27 others); Mon, 20 Nov 2023 17:19:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232587AbjKTWTk (ORCPT ); Mon, 20 Nov 2023 17:19:40 -0500 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FEE391 for ; Mon, 20 Nov 2023 14:19:37 -0800 (PST) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1cc2575dfc7so34247355ad.1 for ; Mon, 20 Nov 2023 14:19:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700518776; x=1701123576; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=ZU36NcT9RPyJC6TQdfEVGdreh7ck6Zb0oyJfbLqLJ4s=; b=BHBZSCe9TsuJXRnWjynxNsorVEt2YpNDI5jBGvOdc0lS2BVO9kGRDmab+buvBF1XDF tu7hhr7ZR57P73qhf0EssUW7071iC1TrGWXMxb+lHzpNMTmhWE+USEYdog1DDnmPwp4Q 0vIg0Yx3r5cEKb5thX8sJKUejl3RdXbrFh3xyIeSHaCbSIsJUpsmIVmkVzoqERBCFbI1 dMuzWfVSVANEeHQJT/1YFOUWgXz08OkQuHh4w0MqSasRlrR26BbTI7010TgZh5i0YnB8 4HECP1qFKRiwv3VUpWERU4lX4yA1OgrQEvnawmrFjngupfDQAPLfkrRQV6hqQ89LY1ps Jkgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700518776; x=1701123576; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ZU36NcT9RPyJC6TQdfEVGdreh7ck6Zb0oyJfbLqLJ4s=; b=oRbh/2nLsIn5vL9DcpTuDtfwbE53T25yoO4k5WXlbXDhAGPV3zsWJOf3JCF1LwtZ/j ZfknrOOgbn6E6kqscGIjQrO0nBVSLrDIPPfAz2Xs7ex1MZbAj6eZd7SDyr7QOIVfNKH+ myYUjEZ9SuUJVAwOUIm9FVBe0xmw3VaUHdGuugvuqcKJiBaUcL9jPJgTl9hVEyx+THOr biBgcFX/wNzwtJ5N1H3HOd/7tBtrX8ASs6kMaKFWHr68EEDkT1fyJ0WhcFVaenHOBioc 0uW2OoDJMfEbBMlu77C9NrxIKJstGdvP3wC27PwhvDha0Sis9E7k6H8RGSoGfNIkpuco Y3WA== X-Gm-Message-State: AOJu0YzWNk03KCIBqzJQqTfFRJR8K82BDcVMyPFVjKXIlU0V6NKEfDJJ lzm/INv/xYYlt7z2H9FopeY= X-Received: by 2002:a17:902:a985:b0:1cc:58f1:8646 with SMTP id bh5-20020a170902a98500b001cc58f18646mr7364589plb.50.1700518776489; Mon, 20 Nov 2023 14:19:36 -0800 (PST) Received: from bangji.hsd1.ca.comcast.net ([2601:647:6780:42e0:75c6:4212:ae99:93b6]) by smtp.gmail.com with ESMTPSA id d4-20020a170902c18400b001c9c47d6cb9sm3528246pld.99.2023.11.20.14.19.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 14:19:36 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Peter Zijlstra , Ingo Molnar Cc: Mark Rutland , Alexander Shishkin , Arnaldo Carvalho de Melo , LKML , Ian Rogers , Kan Liang , Mingwei Zhang Subject: [PATCH 3/3] perf/x86: Add CAP_NO_INTERRUPT for uncore PMUs Date: Mon, 20 Nov 2023 14:19:32 -0800 Message-ID: <20231120221932.213710-3-namhyung@kernel.org> X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog In-Reply-To: <20231120221932.213710-1-namhyung@kernel.org> References: <20231120221932.213710-1-namhyung@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 20 Nov 2023 14:20:06 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783123210208432177 X-GMAIL-MSGID: 1783123210208432177 It doesn't support sampling in uncore PMU events. While it's technically possible to generate interrupts, let's treat it as if it has no interrupt in order to skip the freq adjust/unthrottling logic in the timer handler which is only meaningful to sampling events. Also remove the sampling event check because it'd be done in the general code in the perf_event_open syscall. Signed-off-by: Namhyung Kim --- arch/x86/events/intel/uncore.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/arch/x86/events/intel/uncore.c b/arch/x86/events/intel/uncore.c index 69043e02e8a7..f7e6228bd1b1 100644 --- a/arch/x86/events/intel/uncore.c +++ b/arch/x86/events/intel/uncore.c @@ -744,10 +744,6 @@ static int uncore_pmu_event_init(struct perf_event *event) if (pmu->func_id < 0) return -ENOENT; - /* Sampling not supported yet */ - if (hwc->sample_period) - return -EINVAL; - /* * Place all uncore events for a particular physical package * onto a single cpu @@ -919,7 +915,12 @@ static int uncore_pmu_register(struct intel_uncore_pmu *pmu) .stop = uncore_pmu_event_stop, .read = uncore_pmu_event_read, .module = THIS_MODULE, - .capabilities = PERF_PMU_CAP_NO_EXCLUDE, + /* + * It doesn't allow sampling for uncore events, let's + * treat the PMU has no interrupts to skip them in the + * perf_adjust_freq_unthr_context(). + */ + .capabilities = PERF_PMU_CAP_NO_EXCLUDE | PERF_PMU_CAP_NO_INTERRUPT, .attr_update = pmu->type->attr_update, }; } else {