[v5,00/16] perf kvm: Support histograms and TUI mode

  This patch set is to enable histograms and (partially) TUI mode in perf
kvm tool; patch set v1 [1] gives brief introduction for the change.

We can use below commands for testing this series:

In a terminal, you could launch a virtual machine with qemu command; in
below case, I downloaded a Ubuntu (or Debian) iso file and used it as
the file system image:

  $ qemu-system-x86_64 -M pc -enable-kvm -cpu host -m 4096 -hda ubuntu-22.04-desktop-amd64.iso

Then in another terminal, I can use below command to capture KVM trace
data and report the result:

  # cd linux/tools/perf
  # ./perf kvm stat record
  # ./perf kvm stat report          => Output in TUI mode
  # ./perf kvm stat report --stdio  => Output in stdio mode

Changes from v4:
* Fixed building failure in patch 04; compiled successfully for every
  patch to ensure bisection.

Changes from v3:
* Updated patch 04 to change comparison function return type from int to
  int64_t to avoid overflow;
* Added a refactoring patch 05 "perf kvm: Use macro to replace variable
  'decode_str_len'";
* Refined to use hists__add_entry_ops() and hist_entry__cmp() to compare
  hist entry and add new entries, to support comparison between hist
  entry, added a new patch 09; (Namhyung Kim)
* Updated patch 10 for comparison event name; (Namhyung Kim)
* Refined patch 11 to move init_stats() and hists__inc_nr_samples()
  into kvm_he_zalloc(); (Namhyung Kim)
* Minor refactoring for sort_result() and kvm_hists__init() in patch 15.

Changes from v2:
* Found building failure with command 'make VF=1 DEBUG=1 NO_SLANG=1',
  fixed it in the patch 13. (James Clark)

Changes from v1:
* Updated the patch 13 'perf kvm: Add TUI mode for stat report' to avoid
  building failure if no support HAVE_SLANG_SUPPORT.

[1] https://lore.kernel.org/lkml/20230226042053.1492409-1-leo.yan@linaro.org/


Leo Yan (16):
  perf kvm: Refactor overall statistics
  perf kvm: Add pointer to 'perf_kvm_stat' in kvm event
  perf kvm: Move up metrics helpers
  perf kvm: Use subtraction for comparison metrics
  perf kvm: Use macro to replace variable 'decode_str_len'
  perf kvm: Introduce histograms data structures
  perf kvm: Pass argument 'sample' to kvm_alloc_init_event()
  perf kvm: Parse address location for samples
  perf hist: Add 'kvm_info' field in histograms entry
  perf kvm: Add dimensions for KVM event statistics
  perf kvm: Use histograms list to replace cached list
  perf kvm: Polish sorting key
  perf kvm: Support printing attributions for dimensions
  perf kvm: Add dimensions for percentages
  perf kvm: Add TUI mode for stat report
  perf kvm: Update documentation to reflect new changes

 tools/perf/Documentation/perf-kvm.txt   |   9 +-
 tools/perf/arch/arm64/util/kvm-stat.c   |   1 -
 tools/perf/arch/powerpc/util/kvm-stat.c |   3 +-
 tools/perf/arch/s390/util/kvm-stat.c    |   1 -
 tools/perf/arch/x86/util/kvm-stat.c     |   7 +-
 tools/perf/builtin-annotate.c           |   2 +-
 tools/perf/builtin-c2c.c                |   4 +-
 tools/perf/builtin-diff.c               |   4 +-
 tools/perf/builtin-kvm.c                | 859 +++++++++++++++++++-----
 tools/perf/tests/hists_link.c           |   4 +-
 tools/perf/util/hist.c                  |  19 +-
 tools/perf/util/hist.h                  |   3 +
 tools/perf/util/kvm-stat.h              |  33 +-
 tools/perf/util/sort.h                  |   1 +
 14 files changed, 745 insertions(+), 205 deletions(-)
  

Em Wed, Mar 15, 2023 at 10:50:56PM +0800, Leo Yan escreveu:
> This patch set is to enable histograms and (partially) TUI mode in perf
> kvm tool; patch set v1 [1] gives brief introduction for the change.
> 
> We can use below commands for testing this series:
> 
> In a terminal, you could launch a virtual machine with qemu command; in
> below case, I downloaded a Ubuntu (or Debian) iso file and used it as
> the file system image:
> 
>   $ qemu-system-x86_64 -M pc -enable-kvm -cpu host -m 4096 -hda ubuntu-22.04-desktop-amd64.iso
> 
> Then in another terminal, I can use below command to capture KVM trace
> data and report the result:
> 
>   # cd linux/tools/perf
>   # ./perf kvm stat record
>   # ./perf kvm stat report          => Output in TUI mode
>   # ./perf kvm stat report --stdio  => Output in stdio mode
> 
> Changes from v4:
> * Fixed building failure in patch 04; compiled successfully for every
>   patch to ensure bisection.

Trying to fix this:

⬢[acme@toolbox perf-tools-next]$ git log --oneline -1 ; time make -C tools/perf build-test
cbd137d776f05b70 (HEAD -> perf-tools-next) perf kvm: Update documentation to reflect new changes
make: Entering directory '/var/home/acme/git/perf-tools-next/tools/perf'
- tarpkg: ./tests/perf-targz-src-pkg .
                 make_static: cd . && make LDFLAGS=-static NO_PERF_READ_VDSO32=1 NO_PERF_READ_VDSOX32=1 NO_JVMTI=1 NO_LIBTRACEEVENT=1 NO_LIBELF=1 -j32  DESTDIR=/tmp/tmp.AJidGMSQnF
cd . && make LDFLAGS=-static NO_PERF_READ_VDSO32=1 NO_PERF_READ_VDSOX32=1 NO_JVMTI=1 NO_LIBTRACEEVENT=1 NO_LIBELF=1 -j32 DESTDIR=/tmp/tmp.AJidGMSQnF
  BUILD:   Doing 'make -j32' parallel build
  HOSTCC  fixdep.o
  HOSTLD  fixdep-in.o
  LINK    fixdep
Makefile.config:678: Disabling post unwind, no support found.
Makefile.config:746: No libcrypto.h found, disables jitted code injection, please install openssl-devel or libssl-dev
Makefile.config:758: slang not found, disables TUI support. Please install slang-devel, libslang-dev or libslang2-dev
Makefile.config:805: Missing perl devel files. Disabling perl scripting support, please install perl-ExtUtils-Embed/libperl-dev
Makefile.config:845: No 'Python.h' was found: disables Python support - please install python-devel/python-dev
Makefile.config:950: No liblzma found, disables xz kernel module decompression, please install xz-devel/liblzma-dev
Makefile.config:963: No libzstd found, disables trace compression, please install libzstd-dev[el] and/or set LIBZSTD_DIR
Makefile.config:974: No libcap found, disables capability support, please install libcap-devel/libcap-dev
Makefile.config:987: No numa.h found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev
Makefile.config:1046: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev
Makefile.config:1137: libpfm4 not found, disables libpfm4 support. Please install libpfm4-dev

Auto-detecting system features:
...                                   dwarf: [ OFF ]
...                      dwarf_getlocations: [ OFF ]
...                                   glibc: [ on  ]
...                                  libbfd: [ OFF ]
...                          libbfd-buildid: [ OFF ]
...                                  libcap: [ OFF ]
...                                  libelf: [ OFF ]
...                                 libnuma: [ OFF ]
...                  numa_num_possible_cpus: [ OFF ]
...                                 libperl: [ OFF ]
...                               libpython: [ OFF ]
...                               libcrypto: [ OFF ]
...                               libunwind: [ OFF ]
...                      libdw-dwarf-unwind: [ OFF ]
...                                    zlib: [ OFF ]
...                                    lzma: [ OFF ]
...                               get_cpuid: [ on  ]
...                                     bpf: [ on  ]
...                                  libaio: [ on  ]
...                                 libzstd: [ OFF ]

  GEN     common-cmds.h
  CC      dlfilters/dlfilter-test-api-v0.o
  CC      dlfilters/dlfilter-show-cycles.o
  LINK    dlfilters/dlfilter-show-cycles.so
  LINK    dlfilters/dlfilter-test-api-v0.so
  INSTALL /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/include/subcmd/exec-cmd.h
  INSTALL /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/include/subcmd/help.h
  INSTALL /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/include/subcmd/pager.h
  INSTALL /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/include/subcmd/parse-options.h
  INSTALL /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/include/subcmd/run-command.h
  CC      /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/exec-cmd.o
  CC      /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/help.o
  CC      /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/pager.o
  <SNIP>
  CC      /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/parse-options.o
  CC      /var/home/acme/git/perf-tools-next/tools/perf/libsubcmd/run-command.o
  CC      util/sample-raw.o
  CC      util/s390-sample-raw.o
  CC      util/amd-sample-raw.o
  CC      util/ordered-events.o
builtin-kvm.c:512:13: error: ‘print_result’ used but never defined [-Werror]
  512 | static void print_result(struct perf_kvm_stat *kvm);
      |             ^~~~~~~~~~~~
builtin-kvm.c:591:13: error: ‘kvm_display’ defined but not used [-Werror=unused-function]
  591 | static void kvm_display(struct perf_kvm_stat *kvm)
      |             ^~~~~~~~~~~
builtin-kvm.c:506:12: error: ‘kvm_hists__reinit’ defined but not used [-Werror=unused-function]
  506 | static int kvm_hists__reinit(const char *output, const char *sort)
      |            ^~~~~~~~~~~~~~~~~
builtin-kvm.c:498:12: error: ‘kvm_hists__init’ defined but not used [-Werror=unused-function]
  498 | static int kvm_hists__init(void)
      |            ^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[5]: *** [/var/home/acme/git/perf-tools-next/tools/build/Makefile.build:97: builtin-kvm.o] Error 1
make[5]: *** Waiting for unfinished jobs....
  CC      tests/perf-time-to-tsc.o
  CC      util/namespaces.o
  

On Wed, Mar 15, 2023 at 04:35:04PM -0300, Arnaldo Carvalho de Melo wrote:

[...]

> Trying to fix this:
> 
> ⬢[acme@toolbox perf-tools-next]$ git log --oneline -1 ; time make -C tools/perf build-test

I tried to run the build-test after I fetched the branch
acme/tmp.perf-tools-next and found errors; seems the errors are not
related with perf kvm but it's good to check a bit.

/usr/bin/ld: /usr/lib/llvm-15/lib/libclangSema.a(SemaRISCVVectorLookup.cpp.o): in function `clang::CreateRISCVIntrinsicManager(clang::Sema&)':
(.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x1b8): undefined reference to `clang::RISCV::RVVIntrinsic::computeBuiltinTypes(llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>, bool, bool, bool, unsigned int)'
/usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x1f9): undefined reference to `clang::RISCV::RVVIntrinsic::computeBuiltinTypes(llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>, bool, bool, bool, unsigned int)'
/usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x2cc): undefined reference to `clang::RISCV::RVVType::computeTypes(clang::RISCV::BasicType, int, unsigned int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
/usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x2ec): undefined reference to `clang::RISCV::RVVIntrinsic::getSuffixStr[abi:cxx11](clang::RISCV::BasicType, int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
/usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x307): undefined reference to `clang::RISCV::RVVIntrinsic::getSuffixStr[abi:cxx11](clang::RISCV::BasicType, int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
/usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x36c): undefined reference to `clang::RISCV::RVVType::computeTypes(clang::RISCV::BasicType, int, unsigned int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
collect2: error: ld returned 1 exit status

The complete building log is in the link:
https://termbin.com/jujt.

Later I think it would be a good habit to run build-test on my
patches locally :)

Thanks,
Leo
  

Em Thu, Mar 16, 2023 at 11:10:06AM +0800, Leo Yan escreveu:
> On Wed, Mar 15, 2023 at 04:35:04PM -0300, Arnaldo Carvalho de Melo wrote:
> 
> [...]
> 
> > Trying to fix this:
> > 
> > ⬢[acme@toolbox perf-tools-next]$ git log --oneline -1 ; time make -C tools/perf build-test
> 
> I tried to run the build-test after I fetched the branch
> acme/tmp.perf-tools-next and found errors; seems the errors are not
> related with perf kvm but it's good to check a bit.
> 
> /usr/bin/ld: /usr/lib/llvm-15/lib/libclangSema.a(SemaRISCVVectorLookup.cpp.o): in function `clang::CreateRISCVIntrinsicManager(clang::Sema&)':
> (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x1b8): undefined reference to `clang::RISCV::RVVIntrinsic::computeBuiltinTypes(llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>, bool, bool, bool, unsigned int)'
> /usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x1f9): undefined reference to `clang::RISCV::RVVIntrinsic::computeBuiltinTypes(llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>, bool, bool, bool, unsigned int)'
> /usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x2cc): undefined reference to `clang::RISCV::RVVType::computeTypes(clang::RISCV::BasicType, int, unsigned int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
> /usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x2ec): undefined reference to `clang::RISCV::RVVIntrinsic::getSuffixStr[abi:cxx11](clang::RISCV::BasicType, int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
> /usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x307): undefined reference to `clang::RISCV::RVVIntrinsic::getSuffixStr[abi:cxx11](clang::RISCV::BasicType, int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
> /usr/bin/ld: (.text._ZN5clang27CreateRISCVIntrinsicManagerERNS_4SemaE+0x36c): undefined reference to `clang::RISCV::RVVType::computeTypes(clang::RISCV::BasicType, int, unsigned int, llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>)'
> collect2: error: ld returned 1 exit status

Nevermind this one, it is related to linking with clang libs, something
we'll deprecated.
 
> The complete building log is in the link:
> https://termbin.com/jujt.
> 
> Later I think it would be a good habit to run build-test on my
> patches locally :)

great!
 
> Thanks,
> Leo