[2/6] tools/perf/json: Add cache metrics for s390 z16

  Add metrics for s390 z16
- Percentage sourced from Level 2 cache
- Percentage sourced from Level 3 on same chip cache
- Percentage sourced from Level 4 Local cache on same book
- Percentage sourced from Level 4 Remote cache on different book
- Percentage sourced from memory

For details about the formulas see this documentation:
https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf

Outpuf after:
 # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
 .... dd output deleted

 Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':

                 0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
           431,866      L1I_DIR_WRITES
             2,395      IDCW_OFF_DRAWER_IV
                 0      ICW_OFF_DRAWER
                 0      IDCW_OFF_DRAWER_DRAWER_HIT
             1,437      DCW_OFF_DRAWER
       425,960,793      L1D_DIR_WRITES

      12.165030699 seconds time elapsed

       0.001037000 seconds user
      12.162140000 seconds sys

 #

Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
---
 .../arch/s390/cf_z16/transaction.json         | 25 +++++++++++++++++++
 1 file changed, 25 insertions(+)
  

On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>
> Add metrics for s390 z16
> - Percentage sourced from Level 2 cache
> - Percentage sourced from Level 3 on same chip cache
> - Percentage sourced from Level 4 Local cache on same book
> - Percentage sourced from Level 4 Remote cache on different book
> - Percentage sourced from memory
>
> For details about the formulas see this documentation:
> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
>
> Outpuf after:
>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
>  .... dd output deleted
>
>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
>
>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
>            431,866      L1I_DIR_WRITES
>              2,395      IDCW_OFF_DRAWER_IV
>                  0      ICW_OFF_DRAWER
>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
>              1,437      DCW_OFF_DRAWER
>        425,960,793      L1D_DIR_WRITES
>
>       12.165030699 seconds time elapsed
>
>        0.001037000 seconds user
>       12.162140000 seconds sys
>
>  #
>
> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>

Acked-by: Ian Rogers <irogers@google.com>

Thanks,
Ian

> ---
>  .../arch/s390/cf_z16/transaction.json         | 25 +++++++++++++++++++
>  1 file changed, 25 insertions(+)
>
> diff --git a/tools/perf/pmu-events/arch/s390/cf_z16/transaction.json b/tools/perf/pmu-events/arch/s390/cf_z16/transaction.json
> index 86bf83b4504e..dde0735a7d22 100644
> --- a/tools/perf/pmu-events/arch/s390/cf_z16/transaction.json
> +++ b/tools/perf/pmu-events/arch/s390/cf_z16/transaction.json
> @@ -18,5 +18,30 @@
>      "BriefDescription": "Level One Miss per 100 Instructions",
>      "MetricName": "l1mp",
>      "MetricExpr": "((L1I_DIR_WRITES + L1D_DIR_WRITES) / INSTRUCTIONS) * 100"
> +  },
> +  {
> +    "BriefDescription": "Percentage sourced from Level 2 cache",
> +    "MetricName": "l2p",
> +    "MetricExpr": "((DCW_REQ + DCW_REQ_IV + ICW_REQ + ICW_REQ_IV) / (L1I_DIR_WRITES + L1D_DIR_WRITES)) * 100"
> +  },
> +  {
> +    "BriefDescription": "Percentage sourced from Level 3 on same chip cache",
> +    "MetricName": "l3p",
> +    "MetricExpr": "((DCW_REQ_CHIP_HIT + DCW_ON_CHIP + DCW_ON_CHIP_IV + DCW_ON_CHIP_CHIP_HIT + ICW_REQ_CHIP_HIT + ICW_ON_CHIP + ICW_ON_CHIP_IV + ICW_ON_CHIP_CHIP_HIT) / (L1I_DIR_WRITES + L1D_DIR_WRITES)) * 100"
> +  },
> +  {
> +    "BriefDescription": "Percentage sourced from Level 4 Local cache on same book",
> +    "MetricName": "l4lp",
> +    "MetricExpr": "((DCW_REQ_DRAWER_HIT + DCW_ON_CHIP_DRAWER_HIT + DCW_ON_MODULE + DCW_ON_DRAWER + IDCW_ON_MODULE_IV + IDCW_ON_MODULE_CHIP_HIT + IDCW_ON_MODULE_DRAWER_HIT + IDCW_ON_DRAWER_IV + IDCW_ON_DRAWER_CHIP_HIT + IDCW_ON_DRAWER_DRAWER_HIT + ICW_REQ_DRAWER_HIT + ICW_ON_CHIP_DRAWER_HIT + ICW_ON_MODULE + ICW_ON_DRAWER) / (L1I_DIR_WRITES + L1D_DIR_WRITES)) * 100"
> +  },
> +  {
> +    "BriefDescription": "Percentage sourced from Level 4 Remote cache on different book",
> +    "MetricName": "l4rp",
> +    "MetricExpr": "((DCW_OFF_DRAWER + IDCW_OFF_DRAWER_IV + IDCW_OFF_DRAWER_CHIP_HIT + IDCW_OFF_DRAWER_DRAWER_HIT + ICW_OFF_DRAWER) / (L1I_DIR_WRITES + L1D_DIR_WRITES)) * 100"
> +  },
> +  {
> +    "BriefDescription": "Percentage sourced from memory",
> +    "MetricName": "memp",
> +    "MetricExpr": "((DCW_ON_CHIP_MEMORY + DCW_ON_MODULE_MEMORY + DCW_ON_DRAWER_MEMORY + DCW_OFF_DRAWER_MEMORY + ICW_ON_CHIP_MEMORY + ICW_ON_MODULE_MEMORY + ICW_ON_DRAWER_MEMORY + ICW_OFF_DRAWER_MEMORY) / (L1I_DIR_WRITES + L1D_DIR_WRITES)) * 100"
>    }
>  ]
> --
> 2.39.1
>
  

Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
> >
> > Add metrics for s390 z16
> > - Percentage sourced from Level 2 cache
> > - Percentage sourced from Level 3 on same chip cache
> > - Percentage sourced from Level 4 Local cache on same book
> > - Percentage sourced from Level 4 Remote cache on different book
> > - Percentage sourced from memory
> >
> > For details about the formulas see this documentation:
> > https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
> >
> > Outpuf after:
> >  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
> >  .... dd output deleted
> >
> >  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
> >
> >                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
> >            431,866      L1I_DIR_WRITES
> >              2,395      IDCW_OFF_DRAWER_IV
> >                  0      ICW_OFF_DRAWER
> >                  0      IDCW_OFF_DRAWER_DRAWER_HIT
> >              1,437      DCW_OFF_DRAWER
> >        425,960,793      L1D_DIR_WRITES
> >
> >       12.165030699 seconds time elapsed
> >
> >        0.001037000 seconds user
> >       12.162140000 seconds sys
> >
> >  #
> >
> > Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
> > Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
> 
> Acked-by: Ian Rogers <irogers@google.com>

Thanks, applied the first two patches, please address the review
suggestions for patches 3-6 and resubmit only those.

The patches will be in the public perf-tools-next branch later today.

- Arnaldo
  

On 3/13/23 19:33, Arnaldo Carvalho de Melo wrote:
> Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
>> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>>>
>>> Add metrics for s390 z16
>>> - Percentage sourced from Level 2 cache
>>> - Percentage sourced from Level 3 on same chip cache
>>> - Percentage sourced from Level 4 Local cache on same book
>>> - Percentage sourced from Level 4 Remote cache on different book
>>> - Percentage sourced from memory
>>>
>>> For details about the formulas see this documentation:
>>> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
>>>
>>> Outpuf after:
>>>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
>>>  .... dd output deleted
>>>
>>>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
>>>
>>>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
>>>            431,866      L1I_DIR_WRITES
>>>              2,395      IDCW_OFF_DRAWER_IV
>>>                  0      ICW_OFF_DRAWER
>>>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
>>>              1,437      DCW_OFF_DRAWER
>>>        425,960,793      L1D_DIR_WRITES
>>>
>>>       12.165030699 seconds time elapsed
>>>
>>>        0.001037000 seconds user
>>>       12.162140000 seconds sys
>>>
>>>  #
>>>
>>> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
>>> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
>>
>> Acked-by: Ian Rogers <irogers@google.com>
> 
> Thanks, applied the first two patches, please address the review
> suggestions for patches 3-6 and resubmit only those.
> 
> The patches will be in the public perf-tools-next branch later today.
> 
> - Arnaldo
> 

I would really prefer the current implementation without using "ScaleUnit": "100%"
The reason is that these formulars are given to me from the s390 Performance team.
They want to use the exact same formulars on all platforms running on s390
which includes z/OS and z/VM. This way they are sure to get the same numbers.

Hope this background info helps.

Thanks a lot.
  

On Tue, Mar 14, 2023 at 1:20 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>
> On 3/13/23 19:33, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
> >> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
> >>>
> >>> Add metrics for s390 z16
> >>> - Percentage sourced from Level 2 cache
> >>> - Percentage sourced from Level 3 on same chip cache
> >>> - Percentage sourced from Level 4 Local cache on same book
> >>> - Percentage sourced from Level 4 Remote cache on different book
> >>> - Percentage sourced from memory
> >>>
> >>> For details about the formulas see this documentation:
> >>> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
> >>>
> >>> Outpuf after:
> >>>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
> >>>  .... dd output deleted
> >>>
> >>>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
> >>>
> >>>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
> >>>            431,866      L1I_DIR_WRITES
> >>>              2,395      IDCW_OFF_DRAWER_IV
> >>>                  0      ICW_OFF_DRAWER
> >>>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
> >>>              1,437      DCW_OFF_DRAWER
> >>>        425,960,793      L1D_DIR_WRITES
> >>>
> >>>       12.165030699 seconds time elapsed
> >>>
> >>>        0.001037000 seconds user
> >>>       12.162140000 seconds sys
> >>>
> >>>  #
> >>>
> >>> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
> >>> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
> >>
> >> Acked-by: Ian Rogers <irogers@google.com>
> >
> > Thanks, applied the first two patches, please address the review
> > suggestions for patches 3-6 and resubmit only those.
> >
> > The patches will be in the public perf-tools-next branch later today.
> >
> > - Arnaldo
> >
>
> I would really prefer the current implementation without using "ScaleUnit": "100%"
> The reason is that these formulars are given to me from the s390 Performance team.
> They want to use the exact same formulars on all platforms running on s390
> which includes z/OS and z/VM. This way they are sure to get the same numbers.
>
> Hope this background info helps.

For the series:
Acked-by: Ian Rogers <irogers@google.com>

Using ScaleUnit won't change the result. A ScaleUnit of "100%" means
scale the result up by multiplying by 100 and then apply the % after
the value. Another nit is having metrics that place their units in the
name, like _percent, is usually a sign the name can be better. Perhaps
we can follow up with some clean up.

Thanks,
Ian

> Thanks a lot.
> --
> Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
> --
> Vorsitzender des Aufsichtsrats: Gregor Pillen
> Geschäftsführung: David Faller
> Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
>
  

Em Tue, Mar 14, 2023 at 09:34:46AM -0700, Ian Rogers escreveu:
> On Tue, Mar 14, 2023 at 1:20 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
> >
> > On 3/13/23 19:33, Arnaldo Carvalho de Melo wrote:
> > > Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
> > >> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
> > >>>
> > >>> Add metrics for s390 z16
> > >>> - Percentage sourced from Level 2 cache
> > >>> - Percentage sourced from Level 3 on same chip cache
> > >>> - Percentage sourced from Level 4 Local cache on same book
> > >>> - Percentage sourced from Level 4 Remote cache on different book
> > >>> - Percentage sourced from memory
> > >>>
> > >>> For details about the formulas see this documentation:
> > >>> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
> > >>>
> > >>> Outpuf after:
> > >>>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
> > >>>  .... dd output deleted
> > >>>
> > >>>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
> > >>>
> > >>>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
> > >>>            431,866      L1I_DIR_WRITES
> > >>>              2,395      IDCW_OFF_DRAWER_IV
> > >>>                  0      ICW_OFF_DRAWER
> > >>>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
> > >>>              1,437      DCW_OFF_DRAWER
> > >>>        425,960,793      L1D_DIR_WRITES
> > >>>
> > >>>       12.165030699 seconds time elapsed
> > >>>
> > >>>        0.001037000 seconds user
> > >>>       12.162140000 seconds sys
> > >>>
> > >>>  #
> > >>>
> > >>> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
> > >>> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
> > >>
> > >> Acked-by: Ian Rogers <irogers@google.com>
> > >
> > > Thanks, applied the first two patches, please address the review
> > > suggestions for patches 3-6 and resubmit only those.
> > >
> > > The patches will be in the public perf-tools-next branch later today.
> > >
> > > - Arnaldo
> > >
> >
> > I would really prefer the current implementation without using "ScaleUnit": "100%"
> > The reason is that these formulars are given to me from the s390 Performance team.
> > They want to use the exact same formulars on all platforms running on s390
> > which includes z/OS and z/VM. This way they are sure to get the same numbers.
> >
> > Hope this background info helps.
> 
> For the series:
> Acked-by: Ian Rogers <irogers@google.com>

Thanks, applied.

- Arnaldo

 
> Using ScaleUnit won't change the result. A ScaleUnit of "100%" means
> scale the result up by multiplying by 100 and then apply the % after
> the value. Another nit is having metrics that place their units in the
> name, like _percent, is usually a sign the name can be better. Perhaps
> we can follow up with some clean up.
> 
> Thanks,
> Ian
> 
> > Thanks a lot.
> > --
> > Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
> > --
> > Vorsitzender des Aufsichtsrats: Gregor Pillen
> > Geschäftsführung: David Faller
> > Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
> >
  

On 3/14/23 17:34, Ian Rogers wrote:
> On Tue, Mar 14, 2023 at 1:20 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>>
>> On 3/13/23 19:33, Arnaldo Carvalho de Melo wrote:
>>> Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
>>>> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>>>>>
>>>>> Add metrics for s390 z16
>>>>> - Percentage sourced from Level 2 cache
>>>>> - Percentage sourced from Level 3 on same chip cache
>>>>> - Percentage sourced from Level 4 Local cache on same book
>>>>> - Percentage sourced from Level 4 Remote cache on different book
>>>>> - Percentage sourced from memory
>>>>>
>>>>> For details about the formulas see this documentation:
>>>>> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
>>>>>
>>>>> Outpuf after:
>>>>>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
>>>>>  .... dd output deleted
>>>>>
>>>>>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
>>>>>
>>>>>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
>>>>>            431,866      L1I_DIR_WRITES
>>>>>              2,395      IDCW_OFF_DRAWER_IV
>>>>>                  0      ICW_OFF_DRAWER
>>>>>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
>>>>>              1,437      DCW_OFF_DRAWER
>>>>>        425,960,793      L1D_DIR_WRITES
>>>>>
>>>>>       12.165030699 seconds time elapsed
>>>>>
>>>>>        0.001037000 seconds user
>>>>>       12.162140000 seconds sys
>>>>>
>>>>>  #
>>>>>
>>>>> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
>>>>> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
>>>>
>>>> Acked-by: Ian Rogers <irogers@google.com>
>>>
>>> Thanks, applied the first two patches, please address the review
>>> suggestions for patches 3-6 and resubmit only those.
>>>
>>> The patches will be in the public perf-tools-next branch later today.
>>>
>>> - Arnaldo
>>>
>>
>> I would really prefer the current implementation without using "ScaleUnit": "100%"
>> The reason is that these formulars are given to me from the s390 Performance team.
>> They want to use the exact same formulars on all platforms running on s390
>> which includes z/OS and z/VM. This way they are sure to get the same numbers.
>>
>> Hope this background info helps.
> 
> For the series:
> Acked-by: Ian Rogers <irogers@google.com>
> 
> Using ScaleUnit won't change the result. A ScaleUnit of "100%" means
> scale the result up by multiplying by 100 and then apply the % after
> the value. Another nit is having metrics that place their units in the
> name, like _percent, is usually a sign the name can be better. Perhaps
> we can follow up with some clean up.
> 
> Thanks,
> Ian

Thanks Ian,
I put the ScaleUnit thing on my todo list and will provide a clean up...
  

Em Tue, Mar 14, 2023 at 06:36:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Mar 14, 2023 at 09:34:46AM -0700, Ian Rogers escreveu:
> > On Tue, Mar 14, 2023 at 1:20 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
> > >
> > > On 3/13/23 19:33, Arnaldo Carvalho de Melo wrote:
> > > > Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
> > > >> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
> > > >>>
> > > >>> Add metrics for s390 z16
> > > >>> - Percentage sourced from Level 2 cache
> > > >>> - Percentage sourced from Level 3 on same chip cache
> > > >>> - Percentage sourced from Level 4 Local cache on same book
> > > >>> - Percentage sourced from Level 4 Remote cache on different book
> > > >>> - Percentage sourced from memory
> > > >>>
> > > >>> For details about the formulas see this documentation:
> > > >>> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
> > > >>>
> > > >>> Outpuf after:
> > > >>>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
> > > >>>  .... dd output deleted
> > > >>>
> > > >>>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
> > > >>>
> > > >>>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
> > > >>>            431,866      L1I_DIR_WRITES
> > > >>>              2,395      IDCW_OFF_DRAWER_IV
> > > >>>                  0      ICW_OFF_DRAWER
> > > >>>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
> > > >>>              1,437      DCW_OFF_DRAWER
> > > >>>        425,960,793      L1D_DIR_WRITES
> > > >>>
> > > >>>       12.165030699 seconds time elapsed
> > > >>>
> > > >>>        0.001037000 seconds user
> > > >>>       12.162140000 seconds sys
> > > >>>
> > > >>>  #
> > > >>>
> > > >>> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
> > > >>> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
> > > >>
> > > >> Acked-by: Ian Rogers <irogers@google.com>
> > > >
> > > > Thanks, applied the first two patches, please address the review
> > > > suggestions for patches 3-6 and resubmit only those.
> > > >
> > > > The patches will be in the public perf-tools-next branch later today.
> > > >
> > > > - Arnaldo
> > > >
> > >
> > > I would really prefer the current implementation without using "ScaleUnit": "100%"
> > > The reason is that these formulars are given to me from the s390 Performance team.
> > > They want to use the exact same formulars on all platforms running on s390
> > > which includes z/OS and z/VM. This way they are sure to get the same numbers.
> > >
> > > Hope this background info helps.
> > 
> > For the series:
> > Acked-by: Ian Rogers <irogers@google.com>
> 
> Thanks, applied.
> 
> - Arnaldo

While trying to cross build to s390 on:

ubuntu:18.04

using python3
 

   CC      /tmp/build/perf/tests/parse-events.o
Exception processing pmu-events/arch/s390/cf_z16/extended.json
Traceback (most recent call last):
  File "pmu-events/jevents.py", line 997, in <module>
    main()
  File "pmu-events/jevents.py", line 979, in main
    ftw(arch_path, [], preprocess_one_file)
  File "pmu-events/jevents.py", line 935, in ftw
    ftw(item.path, parents + [item.name], action)
  File "pmu-events/jevents.py", line 933, in ftw
    action(parents, item)
  File "pmu-events/jevents.py", line 514, in preprocess_one_file
    for event in read_json_events(item.path, topic):
  File "pmu-events/jevents.py", line 388, in read_json_events
    events = json.load(open(path), object_hook=JsonEvent)
  File "/usr/lib/python3.6/json/__init__.py", line 296, in load
    return loads(fp.read(),
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4271: ordinal not in range(128)

>  
> > Using ScaleUnit won't change the result. A ScaleUnit of "100%" means
> > scale the result up by multiplying by 100 and then apply the % after
> > the value. Another nit is having metrics that place their units in the
> > name, like _percent, is usually a sign the name can be better. Perhaps
> > we can follow up with some clean up.
> > 
> > Thanks,
> > Ian
> > 
> > > Thanks a lot.
> > > --
> > > Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
> > > --
> > > Vorsitzender des Aufsichtsrats: Gregor Pillen
> > > Geschäftsführung: David Faller
> > > Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
> > >
> 
> -- 
> 
> - Arnaldo
  

On 3/22/23 21:59, Arnaldo Carvalho de Melo wrote:
> Em Tue, Mar 14, 2023 at 06:36:24PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Tue, Mar 14, 2023 at 09:34:46AM -0700, Ian Rogers escreveu:
>>> On Tue, Mar 14, 2023 at 1:20 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>>>>
>>>> On 3/13/23 19:33, Arnaldo Carvalho de Melo wrote:
>>>>> Em Mon, Mar 13, 2023 at 08:22:44AM -0700, Ian Rogers escreveu:
>>>>>> On Mon, Mar 13, 2023 at 1:30 AM Thomas Richter <tmricht@linux.ibm.com> wrote:
>>>>>>>
>>>>>>> Add metrics for s390 z16
>>>>>>> - Percentage sourced from Level 2 cache
>>>>>>> - Percentage sourced from Level 3 on same chip cache
>>>>>>> - Percentage sourced from Level 4 Local cache on same book
>>>>>>> - Percentage sourced from Level 4 Remote cache on different book
>>>>>>> - Percentage sourced from memory
>>>>>>>
>>>>>>> For details about the formulas see this documentation:
>>>>>>> https://www.ibm.com/support/pages/system/files/inline-files/CPU%20MF%20Formulas%20including%20z16%20-%20May%202022_1.pdf
>>>>>>>
>>>>>>> Outpuf after:
>>>>>>>  # ./perf stat -M l4rp -- dd if=/dev/zero of=/dev/null bs=10M count=10K
>>>>>>>  .... dd output deleted
>>>>>>>
>>>>>>>  Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=10M count=10K':
>>>>>>>
>>>>>>>                  0      IDCW_OFF_DRAWER_CHIP_HIT         #     0.00 l4rp
>>>>>>>            431,866      L1I_DIR_WRITES
>>>>>>>              2,395      IDCW_OFF_DRAWER_IV
>>>>>>>                  0      ICW_OFF_DRAWER
>>>>>>>                  0      IDCW_OFF_DRAWER_DRAWER_HIT
>>>>>>>              1,437      DCW_OFF_DRAWER
>>>>>>>        425,960,793      L1D_DIR_WRITES
>>>>>>>
>>>>>>>       12.165030699 seconds time elapsed
>>>>>>>
>>>>>>>        0.001037000 seconds user
>>>>>>>       12.162140000 seconds sys
>>>>>>>
>>>>>>>  #
>>>>>>>
>>>>>>> Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
>>>>>>> Acked-By: Sumanth Korikkar <sumanthk@linux.ibm.com>
>>>>>>
>>>>>> Acked-by: Ian Rogers <irogers@google.com>
>>>>>
>>>>> Thanks, applied the first two patches, please address the review
>>>>> suggestions for patches 3-6 and resubmit only those.
>>>>>
>>>>> The patches will be in the public perf-tools-next branch later today.
>>>>>
>>>>> - Arnaldo
>>>>>
>>>>
>>>> I would really prefer the current implementation without using "ScaleUnit": "100%"
>>>> The reason is that these formulars are given to me from the s390 Performance team.
>>>> They want to use the exact same formulars on all platforms running on s390
>>>> which includes z/OS and z/VM. This way they are sure to get the same numbers.
>>>>
>>>> Hope this background info helps.
>>>
>>> For the series:
>>> Acked-by: Ian Rogers <irogers@google.com>
>>
>> Thanks, applied.
>>
>> - Arnaldo
> 
> While trying to cross build to s390 on:
> 
> ubuntu:18.04
> 
> using python3
>  
> 
>    CC      /tmp/build/perf/tests/parse-events.o
> Exception processing pmu-events/arch/s390/cf_z16/extended.json
> Traceback (most recent call last):
>   File "pmu-events/jevents.py", line 997, in <module>
>     main()
>   File "pmu-events/jevents.py", line 979, in main
>     ftw(arch_path, [], preprocess_one_file)
>   File "pmu-events/jevents.py", line 935, in ftw
>     ftw(item.path, parents + [item.name], action)
>   File "pmu-events/jevents.py", line 933, in ftw
>     action(parents, item)
>   File "pmu-events/jevents.py", line 514, in preprocess_one_file
>     for event in read_json_events(item.path, topic):
>   File "pmu-events/jevents.py", line 388, in read_json_events
>     events = json.load(open(path), object_hook=JsonEvent)
>   File "/usr/lib/python3.6/json/__init__.py", line 296, in load
>     return loads(fp.read(),
>   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
>     return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4271: ordinal not in range(128)
> 
>

Hmmm, this is very strange. After reading this mail I installed Ubuntu 18.04
on my s390 system. The build works fine, no errors at all.


# pmu-events/jevents.py s390 all pmu-events/arch pmu-events/pmu-events.c
# ll pmu-events/pmu-events.c
-rw-r--r-- 1 root root 317284 Mar 23 10:46 pmu-events/pmu-events.c
#

The file has the correct contents and the build works fine too.
# make 
....
Auto-detecting system features:
...                                   dwarf: [ on  ]
...                      dwarf_getlocations: [ on  ]
...                                   glibc: [ on  ]
...                                  libbfd: [ on  ]
...                          libbfd-buildid: [ on  ]
...                                  libcap: [ OFF ]
...                                  libelf: [ on  ]
...                                 libnuma: [ on  ]
...                  numa_num_possible_cpus: [ on  ]
...                                 libperl: [ on  ]
...                               libpython: [ on  ]
...                               libcrypto: [ on  ]
...                               libunwind: [ OFF ]
...                      libdw-dwarf-unwind: [ on  ]
...                                    zlib: [ on  ]
...                                    lzma: [ on  ]
...                               get_cpuid: [ OFF ]
...                                     bpf: [ on  ]
...                                  libaio: [ on  ]
...                                 libzstd: [ OFF ]

  INSTALL libsubcmd_headers
  INSTALL libsymbol_headers
  INSTALL libperf_headers
  INSTALL libapi_headers
  INSTALL libbpf_headers
  CC      pmu-events/pmu-events.o
  LD      pmu-events/pmu-events-in.o
  LINK    perf
# ./perf list | grep -A 20 basic:
basic:
  CPU_CYCLES
       [Cycle Count. Unit: cpum_cf]
  INSTRUCTIONS
       [Instruction Count. Unit: cpum_cf]
  L1D_DIR_WRITES
       [Level-1 D-Cache Directory Write Count. Unit: cpum_cf]
  L1D_PENALTY_CYCLES
       [Level-1 D-Cache Penalty Cycle Count. Unit: cpum_cf]
  L1I_DIR_WRITES
       [Level-1 I-Cache Directory Write Count. Unit: cpum_cf]
  L1I_PENALTY_CYCLES
       [Level-1 I-Cache Penalty Cycle Count. Unit: cpum_cf]
  PROBLEM_STATE_CPU_CYCLES
       [Problem-State Cycle Count. Unit: cpum_cf]
  PROBLEM_STATE_INSTRUCTIONS
       [Problem-State Instruction Count. Unit: cpum_cf]


So everythings works as usual.
  

On Thu, Mar 23, 2023 at 10:51:16AM +0100, Thomas Richter wrote:
> On 3/22/23 21:59, Arnaldo Carvalho de Melo wrote:
> > While trying to cross build to s390 on:
> > 
> > ubuntu:18.04
> > 
> > using python3
> >  
> > 
> >    CC      /tmp/build/perf/tests/parse-events.o
> > Exception processing pmu-events/arch/s390/cf_z16/extended.json
> > Traceback (most recent call last):
> >   File "pmu-events/jevents.py", line 997, in <module>
> >     main()
> >   File "pmu-events/jevents.py", line 979, in main
> >     ftw(arch_path, [], preprocess_one_file)
> >   File "pmu-events/jevents.py", line 935, in ftw
> >     ftw(item.path, parents + [item.name], action)
> >   File "pmu-events/jevents.py", line 933, in ftw
> >     action(parents, item)
> >   File "pmu-events/jevents.py", line 514, in preprocess_one_file
> >     for event in read_json_events(item.path, topic):
> >   File "pmu-events/jevents.py", line 388, in read_json_events
> >     events = json.load(open(path), object_hook=JsonEvent)
> >   File "/usr/lib/python3.6/json/__init__.py", line 296, in load
> >     return loads(fp.read(),
> >   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
> >     return codecs.ascii_decode(input, self.errors)[0]
> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4271: ordinal not in range(128)
> > 
> >
> 
> Hmmm, this is very strange. After reading this mail I installed Ubuntu 18.04
> on my s390 system. The build works fine, no errors at all.
> 
> 
> # pmu-events/jevents.py s390 all pmu-events/arch pmu-events/pmu-events.c
> # ll pmu-events/pmu-events.c
> -rw-r--r-- 1 root root 317284 Mar 23 10:46 pmu-events/pmu-events.c
> #
> 
> The file has the correct contents and the build works fine too.
> # make 

The file contains UTF-8 characters, which were already present before
your patch. Guess you need to provide an addon patch which converts to
plain ASCII.
  

Em Thu, Mar 23, 2023 at 11:06:15AM +0100, Heiko Carstens escreveu:
> On Thu, Mar 23, 2023 at 10:51:16AM +0100, Thomas Richter wrote:
> > On 3/22/23 21:59, Arnaldo Carvalho de Melo wrote:
> > > While trying to cross build to s390 on:
> > > 
> > > ubuntu:18.04
> > > 
> > > using python3
> > >  
> > > 
> > >    CC      /tmp/build/perf/tests/parse-events.o
> > > Exception processing pmu-events/arch/s390/cf_z16/extended.json
> > > Traceback (most recent call last):
> > >   File "pmu-events/jevents.py", line 997, in <module>
> > >     main()
> > >   File "pmu-events/jevents.py", line 979, in main
> > >     ftw(arch_path, [], preprocess_one_file)
> > >   File "pmu-events/jevents.py", line 935, in ftw
> > >     ftw(item.path, parents + [item.name], action)
> > >   File "pmu-events/jevents.py", line 933, in ftw
> > >     action(parents, item)
> > >   File "pmu-events/jevents.py", line 514, in preprocess_one_file
> > >     for event in read_json_events(item.path, topic):
> > >   File "pmu-events/jevents.py", line 388, in read_json_events
> > >     events = json.load(open(path), object_hook=JsonEvent)
> > >   File "/usr/lib/python3.6/json/__init__.py", line 296, in load
> > >     return loads(fp.read(),
> > >   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
> > >     return codecs.ascii_decode(input, self.errors)[0]
> > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4271: ordinal not in range(128)
> > > 
> > >
> > 
> > Hmmm, this is very strange. After reading this mail I installed Ubuntu 18.04
> > on my s390 system. The build works fine, no errors at all.
> > 
> > 
> > # pmu-events/jevents.py s390 all pmu-events/arch pmu-events/pmu-events.c
> > # ll pmu-events/pmu-events.c
> > -rw-r--r-- 1 root root 317284 Mar 23 10:46 pmu-events/pmu-events.c
> > #
> > 
> > The file has the correct contents and the build works fine too.
> > # make 
> 
> The file contains UTF-8 characters, which were already present before
> your patch. Guess you need to provide an addon patch which converts to
> plain ASCII.

Yeah, and in this s390 perf test build container in the past I didn't
have the needed python3-dev package needed to build jevents.py, so it
was being disabled and the problem was left unnoticed.

Now that it is a opt-out feature, I installed python3-dev, jevents.py
got built and then this failure surfaced.

- Arnaldo