From patchwork Thu Feb 2 05:04:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 4851 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp44749wrn; Wed, 1 Feb 2023 21:06:29 -0800 (PST) X-Google-Smtp-Source: AK7set/lG0utlS+Zh4Wr4qRrW4ijqZSl7HaDC/A8GuQECao+8VzVWmDjqrZ8+hGQeu+JaznSAj1S X-Received: by 2002:a17:906:7955:b0:886:221b:44e5 with SMTP id l21-20020a170906795500b00886221b44e5mr6612654ejo.62.1675314388921; Wed, 01 Feb 2023 21:06:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675314388; cv=none; d=google.com; s=arc-20160816; b=TleG24WpdmUM1uzLU8CSrLaIjYSlZoaDMOj9N0P2GhEgFqJA3O5Q+LrbCGCAehDb+M dlqEnDK/y94f7hEFS7kMkC3LK2newMszVSsfjfd3JUjbdd/NbL4eizf8tiYurmXP87JP fWv8PoDbozNDF0ROetTxXeZ+ge7DqNN7OfZtNKBBzaeYtObSH3T23SyeGBr9H4R1PUcf /BV9DPhADveKLVHhlAV9dN5STw93T1AwEpAX7Q8ScSohOvMAvnABnJNpfs1SCqi6cAtg q4LHUc+ExFfV5tTEY58/64LxcPn4E5SCgMpP5nlSAk2pA0II4F5waa6eYDv4ThP+oJTx VIzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=R0tc7/8E01DSEPlgGlBLAhea+NkOTJGGCeqxDO2l9is=; b=UbvBicLSL8lJMKNyrGnPRUyAvlapcyY8GB0ShPN/GU4tCx53+TSb7piTtP1a+IkRjv vKgPtdssIW0ZFy1tsz9ER37NoOsnQQjaPi9SE2vWvy9Pmt2JkyEDunRtPlN6vhQHSsvU cXgGgWkkXdEv2VLqsN5OVgOBIiarW9ST/8Wx09H2OVTBjoPphsWcQsOETJkYZTgctPsN /K5mkn6QYOjwBtBDoZ89ionfB6bWZ6xCXkspvL6otli6cs5zmUE5ziLrDagOIAZUey8j ptpUNh6aDIw4QtLj7qUnoUZFG5wqLTcFXhRPGe0DOs9AaPARmEypfk68Le690WujUFV9 0fKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=aYKDxdFz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ka6-20020a170907920600b0087bd4e90784si19538506ejb.850.2023.02.01.21.06.04; Wed, 01 Feb 2023 21:06:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=aYKDxdFz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230515AbjBBFFC (ORCPT + 99 others); Thu, 2 Feb 2023 00:05:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229651AbjBBFFA (ORCPT ); Thu, 2 Feb 2023 00:05:00 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10B6813D6F; Wed, 1 Feb 2023 21:04:59 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id b5so734530plz.5; Wed, 01 Feb 2023 21:04:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=R0tc7/8E01DSEPlgGlBLAhea+NkOTJGGCeqxDO2l9is=; b=aYKDxdFzWm0VsQWWGRFnUmPGy5i3KtoBAa5VGTHfg7T6ts0uICPU4COZjD0ANDl2q7 sT4hjskqYL+57eAT8sQilzV1p7sNWrseUvS4rgaTcCqsnDfLW0073AZHfBGq5Zsauc3D ZFDRC5ew/sj2xrzl+2qdPJpR05ZU5z5MoP08hOI68X/+n/mjWkSG8gHtwDz7W7SJ3r56 itPamYEmXl7Yp8jhDjiivuY/R/xrYvlEnSH4tTbisrLZ2c0B8+0V7ONFmWd3ulHKoDvo hTP1ILuSptmwCJOUZiArTwpvy+/QBKtmspZYaB6aWdzHy7dRceLx5TPXJcN6/gAnrHdN AxFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=R0tc7/8E01DSEPlgGlBLAhea+NkOTJGGCeqxDO2l9is=; b=AWorGOX7gnOVqRyHdzWSUyOIfgKRcjP4say+RmBXs5hh0G2Yq8RCQoa1ZEVW0Wbsk0 zEYdqs92zpIYkZ1BpQ+ri2glZEXkUDbcDE9wnMjhVrquX0blorqyvfjLGOl7mjfmEmNx nTdJmcFRapNojqQtoeCOAwHips5ZV5NWpr5JCJmcepjQZolo2KTskk8gUNckcmgyWmyQ O93Zc7Uh4CGZEzjoIFYhcgi0OXw+ujmOjCZ0Yj0VO97jDw7+yNxvK2RaxD5VIqkKracD RPXvtPIkDV3Fm5wbdgba6VETqdBAZ5NlI2mhFUDCGUe3Wu4uqBBW78PD0/vkT0NRIS15 Fv7A== X-Gm-Message-State: AO0yUKUrv4xYRjaqFRJLSRSDZMVI3Qs9Apra9o6EqkS3K70flvzKTNkk O1jInIyfT7zpEb4M7MC4Pko= X-Received: by 2002:a17:90b:1b09:b0:22c:8baa:c7e6 with SMTP id nu9-20020a17090b1b0900b0022c8baac7e6mr5166978pjb.20.1675314298419; Wed, 01 Feb 2023 21:04:58 -0800 (PST) Received: from balhae.hsd1.ca.comcast.net ([2601:647:6780:ff0:8899:7288:e54d:8b87]) by smtp.gmail.com with ESMTPSA id t3-20020a17090aba8300b0022bb3ee9b68sm2238374pjr.13.2023.02.01.21.04.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 21:04:57 -0800 (PST) Sender: Namhyung Kim From: Namhyung Kim To: Arnaldo Carvalho de Melo , Jiri Olsa Cc: Ingo Molnar , Peter Zijlstra , LKML , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org, Song Liu , Hao Luo , bpf@vger.kernel.org Subject: [PATCH 0/4] perf lock contention: Improve aggr x filter combination (v1) Date: Wed, 1 Feb 2023 21:04:51 -0800 Message-Id: <20230202050455.2187592-1-namhyung@kernel.org> X-Mailer: git-send-email 2.39.1.456.gfc5497dd1b-goog MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756694460285924823?= X-GMAIL-MSGID: =?utf-8?q?1756694460285924823?= Hello, The callstack filter can be useful to debug lock issues but it has a limitation that it only works with caller aggregation mode (which is the default setting). IOW it cannot filter by callstack when showing tasks or lock addresses/names. But sometimes users want to use the filter for other aggregation mode. Like "show me lock addresses/names from this caller only" or "show me tasks having these callers". When it's using tracepoint events from the data file, the situation is good since the tracepoints have all the necessary info. But when using BPF it needs to extend the key of lock stat BPF map to have more than one info like 'pid + stack_id' or 'lock_addr + stack_id'. As callstack filter works in userspace, it should save the both info. With this change we can now use the -S/--callstack-filter with the -t/--threads option or -l/--lock-addr option. It's also possible to use it with other filter options. The following example shows the top 5 tasks that have contention somewhere in the epoll handling. $ sudo perf lock con -abt -S epoll -E5 -- sleep 1 contended total wait max wait avg wait pid comm 2 58.64 us 32.38 us 29.32 us 1514752 Chrome_IOThread 3 29.31 us 12.65 us 9.77 us 3773 Xorg 1 17.45 us 17.45 us 17.45 us 1514906 Chrome_ChildIOT 1 15.41 us 15.41 us 15.41 us 1515382 Chrome_ChildIOT 1 12.52 us 12.52 us 12.52 us 293878 IPC I/O Parent You get get the code at 'perf/lock-filter-v1' branch in git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git Thanks, Namhyung Namhyung Kim (4): perf lock contention: Factor out lock_contention_get_name() perf lock contention: Use lock_stat_find{,new} perf lock contention: Support filters for different aggregation perf test: Add more test cases for perf lock contention tools/perf/builtin-lock.c | 79 ++++---- tools/perf/tests/shell/lock_contention.sh | 66 ++++++- tools/perf/util/bpf_lock_contention.c | 178 +++++++++++------- .../perf/util/bpf_skel/lock_contention.bpf.c | 15 +- tools/perf/util/bpf_skel/lock_data.h | 4 +- tools/perf/util/lock-contention.h | 5 + 6 files changed, 234 insertions(+), 113 deletions(-) base-commit: 7cfa9f5e440054db7c7e28e83a045d36993ff958