[v3,5/6] perf vendor events arm64: Add PE utilization metrics for neoverse-n2

  Add PE utilization related metrics.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
---
 .../arch/arm64/arm/neoverse-n2/metrics.json        | 45 ++++++++++++++++++++++
 1 file changed, 45 insertions(+)
  

On Thu, Nov 24, 2022 at 9:15 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>
> Add PE utilization related metrics.
>
> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
> ---
>  .../arch/arm64/arm/neoverse-n2/metrics.json        | 45 ++++++++++++++++++++++
>  1 file changed, 45 insertions(+)
>
> diff --git a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
> index 23c7d62..7b54819 100644
> --- a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
> +++ b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
> @@ -189,5 +189,50 @@
>          "MetricGroup": "Branch",
>          "MetricName": "branch_miss_pred_rate",
>          "ScaleUnit": "100%"
> +    },
> +    {
> +        "MetricExpr": "instructions / CPU_CYCLES",
> +        "PublicDescription": "The average number of instructions executed for each cycle.",
> +        "BriefDescription": "Instructions per cycle",
> +        "MetricGroup": "PEutilization",
> +        "MetricName": "ipc"
> +    },

A related useful metric is percentage of peak, so if the peak IPC is 8
(usually a constant related to the number of functional units) then
you can just compute the ratio of IPC with this.

> +    {
> +        "MetricExpr": "INST_RETIRED / CPU_CYCLES",
> +        "PublicDescription": "Architecturally executed Instructions Per Cycle (IPC)",
> +        "BriefDescription": "Architecturally executed Instructions Per Cycle (IPC)",


The duplicated descriptions are unnecessary. Drop the public one for
consistency with what we do for Intel:
https://github.com/intel/perfmon/blob/main/scripts/create_perf_json.py#L299

> +        "MetricGroup": "PEutilization",
> +        "MetricName": "retired_ipc"
> +    },
> +    {
> +        "MetricExpr": "INST_SPEC / CPU_CYCLES",
> +        "PublicDescription": "Speculatively executed Instructions Per Cycle (IPC)",
> +        "BriefDescription": "Speculatively executed Instructions Per Cycle (IPC)",
> +        "MetricGroup": "PEutilization",
> +        "MetricName": "spec_ipc"
> +    },
> +    {
> +        "MetricExpr": "OP_RETIRED / OP_SPEC",
> +        "PublicDescription": "Fraction of operations retired",
> +        "BriefDescription": "Fraction of operations retired",

Would instructions be clearer than operations here?

> +        "MetricGroup": "PEutilization",
> +        "MetricName": "retired_rate",
> +        "ScaleUnit": "100%"
> +    },
> +    {
> +        "MetricExpr": "1 - OP_RETIRED / OP_SPEC",

Should OP_RETIRED be greater than OP_SPEC? In which case won't this
metric be negative?

> +        "PublicDescription": "Fraction of operations wasted",
> +        "BriefDescription": "Fraction of operations wasted",
> +        "MetricGroup": "PEutilization",
> +        "MetricName": "wasted_rate",
> +        "ScaleUnit": "100%"
> +    },
> +    {
> +        "MetricExpr": "OP_RETIRED / OP_SPEC * (1 - (STALL_SLOT - CPU_CYCLES) / (CPU_CYCLES * 5))",
> +        "PublicDescription": "Utilization of CPU",
> +        "BriefDescription": "Utilization of CPU",

Some more detail in the description would be useful.

> +        "MetricGroup": "PEutilization",
> +        "MetricName": "cpu_utilization",
> +        "ScaleUnit": "100%"
>      }
>  ]
> --
> 1.8.3.1
>
  

在 2022/12/1 上午2:58, Ian Rogers 写道:
> On Thu, Nov 24, 2022 at 9:15 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>
>> Add PE utilization related metrics.
>>
>> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
>> ---
>>  .../arch/arm64/arm/neoverse-n2/metrics.json        | 45 ++++++++++++++++++++++
>>  1 file changed, 45 insertions(+)
>>
>> diff --git a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
>> index 23c7d62..7b54819 100644
>> --- a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
>> +++ b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
>> @@ -189,5 +189,50 @@
>>          "MetricGroup": "Branch",
>>          "MetricName": "branch_miss_pred_rate",
>>          "ScaleUnit": "100%"
>> +    },
>> +    {
>> +        "MetricExpr": "instructions / CPU_CYCLES",
>> +        "PublicDescription": "The average number of instructions executed for each cycle.",
>> +        "BriefDescription": "Instructions per cycle",
>> +        "MetricGroup": "PEutilization",
>> +        "MetricName": "ipc"
>> +    },
> 
> A related useful metric is percentage of peak, so if the peak IPC is 8
> (usually a constant related to the number of functional units) then
> you can just compute the ratio of IPC with this.
> 

Glad to discuss these with you.
The peak ipc value of neoverse-n2 is 5. Maybe I should add an ipc_rate metric?

>> +    {
>> +        "MetricExpr": "INST_RETIRED / CPU_CYCLES",
>> +        "PublicDescription": "Architecturally executed Instructions Per Cycle (IPC)",
>> +        "BriefDescription": "Architecturally executed Instructions Per Cycle (IPC)",
> 
> 
> The duplicated descriptions are unnecessary. Drop the public one for
> consistency with what we do for Intel:
> https://github.com/intel/perfmon/blob/main/scripts/create_perf_json.py#L299
> 

Sounds good, will do.

>> +        "MetricGroup": "PEutilization",
>> +        "MetricName": "retired_ipc"
>> +    },
>> +    {
>> +        "MetricExpr": "INST_SPEC / CPU_CYCLES",
>> +        "PublicDescription": "Speculatively executed Instructions Per Cycle (IPC)",
>> +        "BriefDescription": "Speculatively executed Instructions Per Cycle (IPC)",
>> +        "MetricGroup": "PEutilization",
>> +        "MetricName": "spec_ipc"
>> +    },
>> +    {
>> +        "MetricExpr": "OP_RETIRED / OP_SPEC",
>> +        "PublicDescription": "Fraction of operations retired",
>> +        "BriefDescription": "Fraction of operations retired",
> 
> Would instructions be clearer than operations here?
> 

operation and instruction are different. OP_RETIRED counts any operation (not instruction)
that has been architecturally executed, For example, speculatively executed operations that
have been abandoned for a branch mispredict will not be counted. So I think operation might
be more accurate.

>> +        "MetricGroup": "PEutilization",
>> +        "MetricName": "retired_rate",
>> +        "ScaleUnit": "100%"
>> +    },
>> +    {
>> +        "MetricExpr": "1 - OP_RETIRED / OP_SPEC",
> 
> Should OP_RETIRED be greater than OP_SPEC? In which case won't this
> metric be negative?
> 

OP_RETIRED will not be greater than OP_SPEC. OP_SPEC counts any operation that has been
speculatively executed. OP_SPEC is a superset of the OP_RETIRED event. There is a
description about OP_SPEC and OP_RETIRED in this neoverse-n2 document.
Link: https://documentation-service.arm.com/static/62cfe21e31ea212bb6627393?token=

>> +        "PublicDescription": "Fraction of operations wasted",
>> +        "BriefDescription": "Fraction of operations wasted",
>> +        "MetricGroup": "PEutilization",
>> +        "MetricName": "wasted_rate",
>> +        "ScaleUnit": "100%"
>> +    },
>> +    {
>> +        "MetricExpr": "OP_RETIRED / OP_SPEC * (1 - (STALL_SLOT - CPU_CYCLES) / (CPU_CYCLES * 5))",
>> +        "PublicDescription": "Utilization of CPU",
>> +        "BriefDescription": "Utilization of CPU",
> 
> Some more detail in the description would be useful.
> 

Ok, I'll describe it in more detail. CPU_utilization reflects the truly effective ratio of operation
executed by the CPU, which means that misprediction and stall are not included. Note that stall_slot
minus cpu_cycles is a correction to the stall_slot error count.

>> +        "MetricGroup": "PEutilization",
>> +        "MetricName": "cpu_utilization",
>> +        "ScaleUnit": "100%"
>>      }
>>  ]
>> --
>> 1.8.3.1
>>
  

On Thu, Dec 1, 2022 at 3:08 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>
>
>
> 在 2022/12/1 上午2:58, Ian Rogers 写道:
> > On Thu, Nov 24, 2022 at 9:15 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
> >>
> >> Add PE utilization related metrics.
> >>
> >> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
> >> ---
> >>  .../arch/arm64/arm/neoverse-n2/metrics.json        | 45 ++++++++++++++++++++++
> >>  1 file changed, 45 insertions(+)
> >>
> >> diff --git a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
> >> index 23c7d62..7b54819 100644
> >> --- a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
> >> +++ b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
> >> @@ -189,5 +189,50 @@
> >>          "MetricGroup": "Branch",
> >>          "MetricName": "branch_miss_pred_rate",
> >>          "ScaleUnit": "100%"
> >> +    },
> >> +    {
> >> +        "MetricExpr": "instructions / CPU_CYCLES",
> >> +        "PublicDescription": "The average number of instructions executed for each cycle.",
> >> +        "BriefDescription": "Instructions per cycle",
> >> +        "MetricGroup": "PEutilization",
> >> +        "MetricName": "ipc"
> >> +    },
> >
> > A related useful metric is percentage of peak, so if the peak IPC is 8
> > (usually a constant related to the number of functional units) then
> > you can just compute the ratio of IPC with this.
> >
>
> Glad to discuss these with you.
> The peak ipc value of neoverse-n2 is 5. Maybe I should add an ipc_rate metric?
>
> >> +    {
> >> +        "MetricExpr": "INST_RETIRED / CPU_CYCLES",
> >> +        "PublicDescription": "Architecturally executed Instructions Per Cycle (IPC)",
> >> +        "BriefDescription": "Architecturally executed Instructions Per Cycle (IPC)",
> >
> >
> > The duplicated descriptions are unnecessary. Drop the public one for
> > consistency with what we do for Intel:
> > https://github.com/intel/perfmon/blob/main/scripts/create_perf_json.py#L299
> >
>
> Sounds good, will do.
>
> >> +        "MetricGroup": "PEutilization",
> >> +        "MetricName": "retired_ipc"
> >> +    },
> >> +    {
> >> +        "MetricExpr": "INST_SPEC / CPU_CYCLES",
> >> +        "PublicDescription": "Speculatively executed Instructions Per Cycle (IPC)",
> >> +        "BriefDescription": "Speculatively executed Instructions Per Cycle (IPC)",
> >> +        "MetricGroup": "PEutilization",
> >> +        "MetricName": "spec_ipc"
> >> +    },
> >> +    {
> >> +        "MetricExpr": "OP_RETIRED / OP_SPEC",
> >> +        "PublicDescription": "Fraction of operations retired",
> >> +        "BriefDescription": "Fraction of operations retired",
> >
> > Would instructions be clearer than operations here?
> >
>
> operation and instruction are different. OP_RETIRED counts any operation (not instruction)
> that has been architecturally executed, For example, speculatively executed operations that
> have been abandoned for a branch mispredict will not be counted. So I think operation might
> be more accurate.

Thanks, I see this note in the N2 PMU guide:

"""
For PMU event definitions, some events specifically count
instructions, while other events count micro-operations (which are
referred to as operations). Please be aware of the use of the word
"operations" or "instructions" in the event description.
"""

From your explanation I wasn't sure if operation was a superset of
instruction that included both retired and speculated ones, or whether
operation had another meaning. I don't see operation being used in the
micro-operation sense elsewhere in the ARM perf json, I think
micro-operation is more consistent and also clearer:
https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/pmu-events/arch/arm64/arm/cortex-a75/pipeline.json?h=perf/core#n27

Perhaps the description can be something like:
Of all the micro-operations issued, what percentage were retired. A
lower number indicates bad speculation.

An alternate way to add documentation is the perf wiki's glossary:
https://perf.wiki.kernel.org/index.php/Glossary

I added the Neoverse N2 PMU Guide to:
https://perf.wiki.kernel.org/index.php/Useful_Links#Manuals

Thanks,
Ian

> >> +        "MetricGroup": "PEutilization",
> >> +        "MetricName": "retired_rate",
> >> +        "ScaleUnit": "100%"
> >> +    },
> >> +    {
> >> +        "MetricExpr": "1 - OP_RETIRED / OP_SPEC",
> >
> > Should OP_RETIRED be greater than OP_SPEC? In which case won't this
> > metric be negative?
> >
>
> OP_RETIRED will not be greater than OP_SPEC. OP_SPEC counts any operation that has been
> speculatively executed. OP_SPEC is a superset of the OP_RETIRED event. There is a
> description about OP_SPEC and OP_RETIRED in this neoverse-n2 document.
> Link: https://documentation-service.arm.com/static/62cfe21e31ea212bb6627393?token=
>
> >> +        "PublicDescription": "Fraction of operations wasted",
> >> +        "BriefDescription": "Fraction of operations wasted",
> >> +        "MetricGroup": "PEutilization",
> >> +        "MetricName": "wasted_rate",
> >> +        "ScaleUnit": "100%"
> >> +    },
> >> +    {
> >> +        "MetricExpr": "OP_RETIRED / OP_SPEC * (1 - (STALL_SLOT - CPU_CYCLES) / (CPU_CYCLES * 5))",
> >> +        "PublicDescription": "Utilization of CPU",
> >> +        "BriefDescription": "Utilization of CPU",
> >
> > Some more detail in the description would be useful.
> >
>
> Ok, I'll describe it in more detail. CPU_utilization reflects the truly effective ratio of operation
> executed by the CPU, which means that misprediction and stall are not included. Note that stall_slot
> minus cpu_cycles is a correction to the stall_slot error count.
>
> >> +        "MetricGroup": "PEutilization",
> >> +        "MetricName": "cpu_utilization",
> >> +        "ScaleUnit": "100%"
> >>      }
> >>  ]
> >> --
> >> 1.8.3.1
> >>
  

在 2022/12/3 上午4:05, Ian Rogers 写道:
> On Thu, Dec 1, 2022 at 3:08 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>
>>
>>
>> 在 2022/12/1 上午2:58, Ian Rogers 写道:
>>> On Thu, Nov 24, 2022 at 9:15 AM Jing Zhang <renyu.zj@linux.alibaba.com> wrote:
>>>>
>>>> Add PE utilization related metrics.
>>>>
>>>> Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
>>>> ---
>>>>  .../arch/arm64/arm/neoverse-n2/metrics.json        | 45 ++++++++++++++++++++++
>>>>  1 file changed, 45 insertions(+)
>>>>
>>>> diff --git a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
>>>> index 23c7d62..7b54819 100644
>>>> --- a/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
>>>> +++ b/tools/perf/pmu-events/arch/arm64/arm/neoverse-n2/metrics.json
>>>> @@ -189,5 +189,50 @@
>>>>          "MetricGroup": "Branch",
>>>>          "MetricName": "branch_miss_pred_rate",
>>>>          "ScaleUnit": "100%"
>>>> +    },
>>>> +    {
>>>> +        "MetricExpr": "instructions / CPU_CYCLES",
>>>> +        "PublicDescription": "The average number of instructions executed for each cycle.",
>>>> +        "BriefDescription": "Instructions per cycle",
>>>> +        "MetricGroup": "PEutilization",
>>>> +        "MetricName": "ipc"
>>>> +    },
>>>
>>> A related useful metric is percentage of peak, so if the peak IPC is 8
>>> (usually a constant related to the number of functional units) then
>>> you can just compute the ratio of IPC with this.
>>>
>>
>> Glad to discuss these with you.
>> The peak ipc value of neoverse-n2 is 5. Maybe I should add an ipc_rate metric?
>>
>>>> +    {
>>>> +        "MetricExpr": "INST_RETIRED / CPU_CYCLES",
>>>> +        "PublicDescription": "Architecturally executed Instructions Per Cycle (IPC)",
>>>> +        "BriefDescription": "Architecturally executed Instructions Per Cycle (IPC)",
>>>
>>>
>>> The duplicated descriptions are unnecessary. Drop the public one for
>>> consistency with what we do for Intel:
>>> https://github.com/intel/perfmon/blob/main/scripts/create_perf_json.py#L299
>>>
>>
>> Sounds good, will do.
>>
>>>> +        "MetricGroup": "PEutilization",
>>>> +        "MetricName": "retired_ipc"
>>>> +    },
>>>> +    {
>>>> +        "MetricExpr": "INST_SPEC / CPU_CYCLES",
>>>> +        "PublicDescription": "Speculatively executed Instructions Per Cycle (IPC)",
>>>> +        "BriefDescription": "Speculatively executed Instructions Per Cycle (IPC)",
>>>> +        "MetricGroup": "PEutilization",
>>>> +        "MetricName": "spec_ipc"
>>>> +    },
>>>> +    {
>>>> +        "MetricExpr": "OP_RETIRED / OP_SPEC",
>>>> +        "PublicDescription": "Fraction of operations retired",
>>>> +        "BriefDescription": "Fraction of operations retired",
>>>
>>> Would instructions be clearer than operations here?
>>>
>>
>> operation and instruction are different. OP_RETIRED counts any operation (not instruction)
>> that has been architecturally executed, For example, speculatively executed operations that
>> have been abandoned for a branch mispredict will not be counted. So I think operation might
>> be more accurate.
> 
> Thanks, I see this note in the N2 PMU guide:
> 
> """
> For PMU event definitions, some events specifically count
> instructions, while other events count micro-operations (which are
> referred to as operations). Please be aware of the use of the word
> "operations" or "instructions" in the event description.
> """
> 
> From your explanation I wasn't sure if operation was a superset of
> instruction that included both retired and speculated ones, or whether
> operation had another meaning. I don't see operation being used in the
> micro-operation sense elsewhere in the ARM perf json, I think
> micro-operation is more consistent and also clearer:
> https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/pmu-events/arch/arm64/arm/cortex-a75/pipeline.json?h=perf/core#n27
> 
> Perhaps the description can be something like:
> Of all the micro-operations issued, what percentage were retired. A
> lower number indicates bad speculation.
> 
> An alternate way to add documentation is the perf wiki's glossary:
> https://perf.wiki.kernel.org/index.php/Glossary
> 
> I added the Neoverse N2 PMU Guide to:
> https://perf.wiki.kernel.org/index.php/Useful_Links#Manuals
> 

Thanks.

The operation here is micro-operation, perhaps it is more accurate to change it to micro-operation.

Description of op_retired and op_spec:
https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git/tree/tools/perf/pmu-events/arch/arm64/common-and-microarch.json?h=perf/core#n315

The event of op_retired counts Micro-operation architecturally executed. The counter counts each
operation counted by OP_SPEC that would be executed in a simple sequential execution of the program.

The event of op_spec counts Micro-operation speculatively executed. The counter counts the number
of operations executed by the processing element, including those that are executed speculatively
and would not be executed in a simple sequential execution of the program.

So "op_retired/op_spec" is indeed "of all the micro-operations issued, what percentage were retired".
But not "a lower number indicates bad speculation". I think "retired" here means "committed".

In the N2 PMU guide:
"""
If the branch is mispredicted, and the instructions are speculatively executed, they will not be
considered architecturally executed. The Arm® Architecture Reference Manual also refers to
architecturally executed instructions as “retired” or “committed”. Speculatively executed instructions
that are not architecturally executed will be abandoned; that is, their results will be discarded and
not counted as part of the program flow.
"""

> 
>>>> +        "MetricGroup": "PEutilization",
>>>> +        "MetricName": "retired_rate",
>>>> +        "ScaleUnit": "100%"
>>>> +    },
>>>> +    {
>>>> +        "MetricExpr": "1 - OP_RETIRED / OP_SPEC",
>>>
>>> Should OP_RETIRED be greater than OP_SPEC? In which case won't this
>>> metric be negative?
>>>
>>
>> OP_RETIRED will not be greater than OP_SPEC. OP_SPEC counts any operation that has been
>> speculatively executed. OP_SPEC is a superset of the OP_RETIRED event. There is a
>> description about OP_SPEC and OP_RETIRED in this neoverse-n2 document.
>> Link: https://documentation-service.arm.com/static/62cfe21e31ea212bb6627393?token=
>>
>>>> +        "PublicDescription": "Fraction of operations wasted",
>>>> +        "BriefDescription": "Fraction of operations wasted",
>>>> +        "MetricGroup": "PEutilization",
>>>> +        "MetricName": "wasted_rate",
>>>> +        "ScaleUnit": "100%"
>>>> +    },
>>>> +    {
>>>> +        "MetricExpr": "OP_RETIRED / OP_SPEC * (1 - (STALL_SLOT - CPU_CYCLES) / (CPU_CYCLES * 5))",
>>>> +        "PublicDescription": "Utilization of CPU",
>>>> +        "BriefDescription": "Utilization of CPU",
>>>
>>> Some more detail in the description would be useful.
>>>
>>
>> Ok, I'll describe it in more detail. CPU_utilization reflects the truly effective ratio of operation
>> executed by the CPU, which means that misprediction and stall are not included. Note that stall_slot
>> minus cpu_cycles is a correction to the stall_slot error count.
>>
>>>> +        "MetricGroup": "PEutilization",
>>>> +        "MetricName": "cpu_utilization",
>>>> +        "ScaleUnit": "100%"
>>>>      }
>>>>  ]
>>>> --
>>>> 1.8.3.1
>>>>