mbox series

[v2,0/3] perf lock contention: Improve aggr x filter combination

Message ID 20230203021324.143540-1-namhyung@kernel.org
Headers
Series perf lock contention: Improve aggr x filter combination |

Message

Namhyung Kim Feb. 3, 2023, 2:13 a.m. UTC 
  Hello,

The callstack filter can be useful to debug lock issues but it has a
limitation that it only works with caller aggregation mode (which is the
default setting).  IOW it cannot filter by callstack when showing tasks
or lock addresses/names.

But sometimes users want to use the filter for other aggregation mode.
Like "show me lock addresses/names from this caller only" or "show me
tasks having these callers".

When it's using tracepoint events from the data file, the situation is
good since the tracepoints have all the necessary info.  But when using
BPF it needs to extend the key of lock stat BPF map to have more than
one info like 'pid + stack_id' or 'lock_addr + stack_id'.  As callstack
filter works in userspace, it should save the both info.

With this change we can now use the -S/--callstack-filter with the
-t/--threads option or -l/--lock-addr option.  It's also possible to use
it with other filter options.

The following example shows the top 5 tasks that have contention
somewhere in the epoll handling.

  $ sudo perf lock con -abt -S epoll -E5 -- sleep 1
   contended   total wait     max wait     avg wait          pid   comm

           2     58.64 us     32.38 us     29.32 us      1514752   Chrome_IOThread
           3     29.31 us     12.65 us      9.77 us         3773   Xorg
           1     17.45 us     17.45 us     17.45 us      1514906   Chrome_ChildIOT
           1     15.41 us     15.41 us     15.41 us      1515382   Chrome_ChildIOT
           1     12.52 us     12.52 us     12.52 us       293878   IPC I/O Parent

You get get the code at 'perf/lock-filter-v1' branch in

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung

Namhyung Kim (3):
  perf lock contention: Factor out lock_contention_get_name()
  perf lock contention: Use lock_stat_find{,new}
  perf lock contention: Support filters for different aggregation

 tools/perf/builtin-lock.c                     |  79 ++++----
 tools/perf/util/Build                         |   5 +-
 tools/perf/util/bpf_lock_contention.c         | 180 +++++++++++-------
 .../perf/util/bpf_skel/lock_contention.bpf.c  |  15 +-
 tools/perf/util/bpf_skel/lock_data.h          |   4 +-
 tools/perf/util/lock-contention.h             |   5 +
 6 files changed, 178 insertions(+), 110 deletions(-)

Comments

Arnaldo Carvalho de Melo Feb. 3, 2023, 8:12 p.m. UTC | #1 
Em Thu, Feb 02, 2023 at 06:13:21PM -0800, Namhyung Kim escreveu:
> Hello,
> 
> The callstack filter can be useful to debug lock issues but it has a
> limitation that it only works with caller aggregation mode (which is the
> default setting).  IOW it cannot filter by callstack when showing tasks
> or lock addresses/names.

Thanks, applied.

- Arnaldo

 
> But sometimes users want to use the filter for other aggregation mode.
> Like "show me lock addresses/names from this caller only" or "show me
> tasks having these callers".
> 
> When it's using tracepoint events from the data file, the situation is
> good since the tracepoints have all the necessary info.  But when using
> BPF it needs to extend the key of lock stat BPF map to have more than
> one info like 'pid + stack_id' or 'lock_addr + stack_id'.  As callstack
> filter works in userspace, it should save the both info.
> 
> With this change we can now use the -S/--callstack-filter with the
> -t/--threads option or -l/--lock-addr option.  It's also possible to use
> it with other filter options.
> 
> The following example shows the top 5 tasks that have contention
> somewhere in the epoll handling.
> 
>   $ sudo perf lock con -abt -S epoll -E5 -- sleep 1
>    contended   total wait     max wait     avg wait          pid   comm
> 
>            2     58.64 us     32.38 us     29.32 us      1514752   Chrome_IOThread
>            3     29.31 us     12.65 us      9.77 us         3773   Xorg
>            1     17.45 us     17.45 us     17.45 us      1514906   Chrome_ChildIOT
>            1     15.41 us     15.41 us     15.41 us      1515382   Chrome_ChildIOT
>            1     12.52 us     12.52 us     12.52 us       293878   IPC I/O Parent
> 
> You get get the code at 'perf/lock-filter-v1' branch in
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
> 
> Thanks,
> Namhyung
> 
> Namhyung Kim (3):
>   perf lock contention: Factor out lock_contention_get_name()
>   perf lock contention: Use lock_stat_find{,new}
>   perf lock contention: Support filters for different aggregation
> 
>  tools/perf/builtin-lock.c                     |  79 ++++----
>  tools/perf/util/Build                         |   5 +-
>  tools/perf/util/bpf_lock_contention.c         | 180 +++++++++++-------
>  .../perf/util/bpf_skel/lock_contention.bpf.c  |  15 +-
>  tools/perf/util/bpf_skel/lock_data.h          |   4 +-
>  tools/perf/util/lock-contention.h             |   5 +
>  6 files changed, 178 insertions(+), 110 deletions(-)
> 
> -- 
> 2.39.1.519.gcb327c4b5f-goog
>