aarch64: disable LDP via tuning structure for -mcpu=ampere1

Message ID 20230413232157.1487389-1-philipp.tomsich@vrull.eu
State Accepted
Headers
Series aarch64: disable LDP via tuning structure for -mcpu=ampere1 |

Checks

Context Check Description
snail/gcc-patch-check success Github commit url

Commit Message

Philipp Tomsich April 13, 2023, 11:21 p.m. UTC
  AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
Given the chance that this causes instructions to slip into the next
decoding cycle and the additional overheads when handling
cacheline-crossing LDP instructions, we disable the generation of LDP
isntructions through the tuning structure from instruction combining
(such as in peephole2).

Given the code-density benefits in builtins and prologue/epilogue
expansion, we allow LDPs there.

This commit:
 * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
 * allows -moverride=tune=... to override this

Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Co-Authored-By: Di Zhao <di.zhao@amperecomputing.com>

gcc/ChangeLog:

	* config/aarch64/aarch64-tuning-flags.def (AARCH64_EXTRA_TUNING_OPTION):
	Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
	* config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
	Check for the above tuning option when processing loads.

---

 gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
 gcc/config/aarch64/aarch64.cc               | 8 +++++++-
 2 files changed, 10 insertions(+), 1 deletion(-)
  

Comments

Kyrylo Tkachov April 14, 2023, 9:20 a.m. UTC | #1
Hi Philipp,

> -----Original Message-----
> From: Philipp Tomsich <philipp.tomsich@vrull.eu>
> Sent: Friday, April 14, 2023 12:22 AM
> To: gcc-patches@gcc.gnu.org
> Cc: Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>; Philipp Tomsich
> <philipp.tomsich@vrull.eu>; Di Zhao <di.zhao@amperecomputing.com>
> Subject: [PATCH] aarch64: disable LDP via tuning structure for -
> mcpu=ampere1
> 
> AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
> Given the chance that this causes instructions to slip into the next
> decoding cycle and the additional overheads when handling
> cacheline-crossing LDP instructions, we disable the generation of LDP
> isntructions through the tuning structure from instruction combining
> (such as in peephole2).
> 
> Given the code-density benefits in builtins and prologue/epilogue
> expansion, we allow LDPs there.

LDPs are indeed quite an important part of the ISA for code density and there are, in principle, second-order benefits from using them, like keeping the instruction cache footprint low (which can be important for large workloads).
Did you gather some benchmarks showing a benefit of disabling them in this manner?

> 
> This commit:
>  * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
>  * allows -moverride=tune=... to override this
> 
> Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> Co-Authored-By: Di Zhao <di.zhao@amperecomputing.com>
> 
> gcc/ChangeLog:
> 
> 	* config/aarch64/aarch64-tuning-flags.def
> (AARCH64_EXTRA_TUNING_OPTION):
> 	Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
> 	* config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
> 	Check for the above tuning option when processing loads.
> 
> ---
> 
>  gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
>  gcc/config/aarch64/aarch64.cc               | 8 +++++++-
>  2 files changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> b/gcc/config/aarch64/aarch64-tuning-flags.def
> index 712895a5263..52112ba7c48 100644
> --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> @@ -44,6 +44,9 @@ AARCH64_EXTRA_TUNING_OPTION
> ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
>  /* Disallow load/store pair instructions on Q-registers.  */
>  AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs",
> NO_LDP_STP_QREGS)
> 
> +/* Disallow load-pair instructions to be formed in combine/peephole.  */
> +AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine",
> NO_LDP_COMBINE)
> +
>  AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs",
> RENAME_LOAD_REGS)
> 
>  AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants",
> CSE_SVE_VL_CONSTANTS)
> diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
> index f4ef22ce02f..8dc1a9ceb17 100644
> --- a/gcc/config/aarch64/aarch64.cc
> +++ b/gcc/config/aarch64/aarch64.cc
> @@ -1971,7 +1971,7 @@ static const struct tune_params ampere1a_tunings
> =
>    2,	/* min_div_recip_mul_df.  */
>    0,	/* max_case_values.  */
>    tune_params::AUTOPREFETCHER_WEAK,	/* autoprefetcher_model.  */
> -  (AARCH64_EXTRA_TUNE_NONE),		/* tune_flags.  */
> +  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),	/* tune_flags.  */
>    &ampere1_prefetch_tune
>  };
> 
> @@ -26053,6 +26053,12 @@ aarch64_operands_ok_for_ldpstp (rtx
> *operands, bool load,
>    enum reg_class rclass_1, rclass_2;
>    rtx mem_1, mem_2, reg_1, reg_2;
> 
> +  /* Allow the tuning structure to disable LDP instruction formation
> +     from combining instructions (e.g., in peephole2).  */
> +  if (load && (aarch64_tune_params.extra_tuning_flags
> +	       & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
> +    return false;

If we do decide to do this, I think this is not a complete approach. See the similar tuning flag AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS.
There's various other places in the backend that would need to be adjusted to avoid bringing loads together for the peephole2s to merge (the sched_fusion stuff).
Plus there's the cpymem expansions that would generate load pairs too...

We'd want some testcases added to check that LDPs are blocked too...

Thanks,
Kyrill

> +
>    if (load)
>      {
>        mem_1 = operands[1];
> --
> 2.34.1
  
Philipp Tomsich April 14, 2023, 9:31 a.m. UTC | #2
Kyrylo,

On Fri, 14 Apr 2023 at 11:21, Kyrylo Tkachov <Kyrylo.Tkachov@arm.com> wrote:
>
> Hi Philipp,
>
> > -----Original Message-----
> > From: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > Sent: Friday, April 14, 2023 12:22 AM
> > To: gcc-patches@gcc.gnu.org
> > Cc: Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>; Philipp Tomsich
> > <philipp.tomsich@vrull.eu>; Di Zhao <di.zhao@amperecomputing.com>
> > Subject: [PATCH] aarch64: disable LDP via tuning structure for -
> > mcpu=ampere1
> >
> > AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
> > Given the chance that this causes instructions to slip into the next
> > decoding cycle and the additional overheads when handling
> > cacheline-crossing LDP instructions, we disable the generation of LDP
> > isntructions through the tuning structure from instruction combining
> > (such as in peephole2).
> >
> > Given the code-density benefits in builtins and prologue/epilogue
> > expansion, we allow LDPs there.
>
> LDPs are indeed quite an important part of the ISA for code density and there are, in principle, second-order benefits from using them, like keeping the instruction cache footprint low (which can be important for large workloads).
> Did you gather some benchmarks showing a benefit of disabling them in this manner?


This has been benchmark-driven, but I need to follow up separately (as
I the final numbers are with the folks that have access to the
benchmark machines)..

>
> > This commit:
> >  * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
> >  * allows -moverride=tune=... to override this
> >
> > Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > Co-Authored-By: Di Zhao <di.zhao@amperecomputing.com>
> >
> > gcc/ChangeLog:
> >
> >       * config/aarch64/aarch64-tuning-flags.def
> > (AARCH64_EXTRA_TUNING_OPTION):
> >       Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
> >       * config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
> >       Check for the above tuning option when processing loads.
> >
> > ---
> >
> >  gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
> >  gcc/config/aarch64/aarch64.cc               | 8 +++++++-
> >  2 files changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> > b/gcc/config/aarch64/aarch64-tuning-flags.def
> > index 712895a5263..52112ba7c48 100644
> > --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> > +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> > @@ -44,6 +44,9 @@ AARCH64_EXTRA_TUNING_OPTION
> > ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
> >  /* Disallow load/store pair instructions on Q-registers.  */
> >  AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs",
> > NO_LDP_STP_QREGS)
> >
> > +/* Disallow load-pair instructions to be formed in combine/peephole.  */
> > +AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine",
> > NO_LDP_COMBINE)
> > +
> >  AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs",
> > RENAME_LOAD_REGS)
> >
> >  AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants",
> > CSE_SVE_VL_CONSTANTS)
> > diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
> > index f4ef22ce02f..8dc1a9ceb17 100644
> > --- a/gcc/config/aarch64/aarch64.cc
> > +++ b/gcc/config/aarch64/aarch64.cc
> > @@ -1971,7 +1971,7 @@ static const struct tune_params ampere1a_tunings
> > =
> >    2, /* min_div_recip_mul_df.  */
> >    0, /* max_case_values.  */
> >    tune_params::AUTOPREFETCHER_WEAK,  /* autoprefetcher_model.  */
> > -  (AARCH64_EXTRA_TUNE_NONE),         /* tune_flags.  */
> > +  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),       /* tune_flags.  */
> >    &ampere1_prefetch_tune
> >  };
> >
> > @@ -26053,6 +26053,12 @@ aarch64_operands_ok_for_ldpstp (rtx
> > *operands, bool load,
> >    enum reg_class rclass_1, rclass_2;
> >    rtx mem_1, mem_2, reg_1, reg_2;
> >
> > +  /* Allow the tuning structure to disable LDP instruction formation
> > +     from combining instructions (e.g., in peephole2).  */
> > +  if (load && (aarch64_tune_params.extra_tuning_flags
> > +            & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
> > +    return false;
>
> If we do decide to do this, I think this is not a complete approach. See the similar tuning flag AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS.
> There's various other places in the backend that would need to be adjusted to avoid bringing loads together for the peephole2s to merge (the sched_fusion stuff).
> Plus there's the cpymem expansions that would generate load pairs too...

I have add-on patches for these, but given that I don't have direct
access to the benchmarking machine and the benchmarks have been run
with this functionality only, I didn't submit them for the time being.
Do you see a path to get this in during the current cycle and defer
the add-on patches (happy to resubmit as a series) only?

> We'd want some testcases added to check that LDPs are blocked too...
>
> Thanks,
> Kyrill
>
> > +
> >    if (load)
> >      {
> >        mem_1 = operands[1];
> > --
> > 2.34.1
>
  
Philipp Tomsich April 14, 2023, 9:51 a.m. UTC | #3
For phase 1, we plan to replace this with a feature to allow
finer-grained control over when to use LDP or STP (i.e., control these
independently) with the following scopes and policies:
 - scopes are: { sched-fusion, mem, pro/epilogue, peephole }
 - policies are: { default (from tuning), always, never, aligned (to
2x element size) }
Happy to get this fuller solution already onto the list, if it helps
with forward-progress on the localised change.

The current patch tries to be minimally invasive (i.e., it doesn't touch STP).
It intentionally avoids modifying the sched-fusion logic (which
requires refactoring, as it doesn't differentiate between the load and
store cases), pro/epilogue creation and mem* function expansion.

Philipp.

On Fri, 14 Apr 2023 at 11:31, Philipp Tomsich <philipp.tomsich@vrull.eu> wrote:
>
> Kyrylo,
>
> On Fri, 14 Apr 2023 at 11:21, Kyrylo Tkachov <Kyrylo.Tkachov@arm.com> wrote:
> >
> > Hi Philipp,
> >
> > > -----Original Message-----
> > > From: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > > Sent: Friday, April 14, 2023 12:22 AM
> > > To: gcc-patches@gcc.gnu.org
> > > Cc: Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>; Philipp Tomsich
> > > <philipp.tomsich@vrull.eu>; Di Zhao <di.zhao@amperecomputing.com>
> > > Subject: [PATCH] aarch64: disable LDP via tuning structure for -
> > > mcpu=ampere1
> > >
> > > AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
> > > Given the chance that this causes instructions to slip into the next
> > > decoding cycle and the additional overheads when handling
> > > cacheline-crossing LDP instructions, we disable the generation of LDP
> > > isntructions through the tuning structure from instruction combining
> > > (such as in peephole2).
> > >
> > > Given the code-density benefits in builtins and prologue/epilogue
> > > expansion, we allow LDPs there.
> >
> > LDPs are indeed quite an important part of the ISA for code density and there are, in principle, second-order benefits from using them, like keeping the instruction cache footprint low (which can be important for large workloads).
> > Did you gather some benchmarks showing a benefit of disabling them in this manner?
>
>
> This has been benchmark-driven, but I need to follow up separately (as
> I the final numbers are with the folks that have access to the
> benchmark machines)..
>
> >
> > > This commit:
> > >  * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
> > >  * allows -moverride=tune=... to override this
> > >
> > > Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > > Co-Authored-By: Di Zhao <di.zhao@amperecomputing.com>
> > >
> > > gcc/ChangeLog:
> > >
> > >       * config/aarch64/aarch64-tuning-flags.def
> > > (AARCH64_EXTRA_TUNING_OPTION):
> > >       Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
> > >       * config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
> > >       Check for the above tuning option when processing loads.
> > >
> > > ---
> > >
> > >  gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
> > >  gcc/config/aarch64/aarch64.cc               | 8 +++++++-
> > >  2 files changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> > > b/gcc/config/aarch64/aarch64-tuning-flags.def
> > > index 712895a5263..52112ba7c48 100644
> > > --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> > > +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> > > @@ -44,6 +44,9 @@ AARCH64_EXTRA_TUNING_OPTION
> > > ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
> > >  /* Disallow load/store pair instructions on Q-registers.  */
> > >  AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs",
> > > NO_LDP_STP_QREGS)
> > >
> > > +/* Disallow load-pair instructions to be formed in combine/peephole.  */
> > > +AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine",
> > > NO_LDP_COMBINE)
> > > +
> > >  AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs",
> > > RENAME_LOAD_REGS)
> > >
> > >  AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants",
> > > CSE_SVE_VL_CONSTANTS)
> > > diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
> > > index f4ef22ce02f..8dc1a9ceb17 100644
> > > --- a/gcc/config/aarch64/aarch64.cc
> > > +++ b/gcc/config/aarch64/aarch64.cc
> > > @@ -1971,7 +1971,7 @@ static const struct tune_params ampere1a_tunings
> > > =
> > >    2, /* min_div_recip_mul_df.  */
> > >    0, /* max_case_values.  */
> > >    tune_params::AUTOPREFETCHER_WEAK,  /* autoprefetcher_model.  */
> > > -  (AARCH64_EXTRA_TUNE_NONE),         /* tune_flags.  */
> > > +  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),       /* tune_flags.  */
> > >    &ampere1_prefetch_tune
> > >  };
> > >
> > > @@ -26053,6 +26053,12 @@ aarch64_operands_ok_for_ldpstp (rtx
> > > *operands, bool load,
> > >    enum reg_class rclass_1, rclass_2;
> > >    rtx mem_1, mem_2, reg_1, reg_2;
> > >
> > > +  /* Allow the tuning structure to disable LDP instruction formation
> > > +     from combining instructions (e.g., in peephole2).  */
> > > +  if (load && (aarch64_tune_params.extra_tuning_flags
> > > +            & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
> > > +    return false;
> >
> > If we do decide to do this, I think this is not a complete approach. See the similar tuning flag AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS.
> > There's various other places in the backend that would need to be adjusted to avoid bringing loads together for the peephole2s to merge (the sched_fusion stuff).
> > Plus there's the cpymem expansions that would generate load pairs too...
>
> I have add-on patches for these, but given that I don't have direct
> access to the benchmarking machine and the benchmarks have been run
> with this functionality only, I didn't submit them for the time being.
> Do you see a path to get this in during the current cycle and defer
> the add-on patches (happy to resubmit as a series) only?
>
> > We'd want some testcases added to check that LDPs are blocked too...
> >
> > Thanks,
> > Kyrill
> >
> > > +
> > >    if (load)
> > >      {
> > >        mem_1 = operands[1];
> > > --
> > > 2.34.1
> >
  
Philipp Tomsich April 14, 2023, 10:25 a.m. UTC | #4
On Fri, 14 Apr 2023 at 11:31, Philipp Tomsich <philipp.tomsich@vrull.eu>
wrote:

> Kyrylo,
>
> On Fri, 14 Apr 2023 at 11:21, Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>
> wrote:
> >
> > Hi Philipp,
> >
> > > -----Original Message-----
> > > From: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > > Sent: Friday, April 14, 2023 12:22 AM
> > > To: gcc-patches@gcc.gnu.org
> > > Cc: Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>; Philipp Tomsich
> > > <philipp.tomsich@vrull.eu>; Di Zhao <di.zhao@amperecomputing.com>
> > > Subject: [PATCH] aarch64: disable LDP via tuning structure for -
> > > mcpu=ampere1
> > >
> > > AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
> > > Given the chance that this causes instructions to slip into the next
> > > decoding cycle and the additional overheads when handling
> > > cacheline-crossing LDP instructions, we disable the generation of LDP
> > > isntructions through the tuning structure from instruction combining
> > > (such as in peephole2).
> > >
> > > Given the code-density benefits in builtins and prologue/epilogue
> > > expansion, we allow LDPs there.
> >
> > LDPs are indeed quite an important part of the ISA for code density and
> there are, in principle, second-order benefits from using them, like
> keeping the instruction cache footprint low (which can be important for
> large workloads).
> > Did you gather some benchmarks showing a benefit of disabling them in
> this manner?
>
> This has been benchmark-driven, but I need to follow up separately (as
> I the final numbers are with the folks that have access to the
> benchmark machines)..
>

Here are the numbers for the submitted change for AmpereOne:
   503.bwaves_r.        -0.88%
   507.cactuBSSN_r    0.35%
   508.namd_r             3.09%
   510.parest_r           -2.99%
   511.povray_r            5.54%
   519.lbm_r               15.83%
   521.wrf_r                  0.56%
   526.blender_r           2.47%
   527.cam4_r              0.70%
   538.imagick_r           0.00%
   544.nab_r                -0.33%
   549.fotonik3d_r.      -0.42%
   554.roms_r               0.00%
   = total                       1.79%


> >
> > > This commit:
> > >  * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
> > >  * allows -moverride=tune=... to override this
> > >
> > > Signed-off-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > > Co-Authored-By: Di Zhao <di.zhao@amperecomputing.com>
> > >
> > > gcc/ChangeLog:
> > >
> > >       * config/aarch64/aarch64-tuning-flags.def
> > > (AARCH64_EXTRA_TUNING_OPTION):
> > >       Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
> > >       * config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
> > >       Check for the above tuning option when processing loads.
> > >
> > > ---
> > >
> > >  gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
> > >  gcc/config/aarch64/aarch64.cc               | 8 +++++++-
> > >  2 files changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> > > b/gcc/config/aarch64/aarch64-tuning-flags.def
> > > index 712895a5263..52112ba7c48 100644
> > > --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> > > +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> > > @@ -44,6 +44,9 @@ AARCH64_EXTRA_TUNING_OPTION
> > > ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
> > >  /* Disallow load/store pair instructions on Q-registers.  */
> > >  AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs",
> > > NO_LDP_STP_QREGS)
> > >
> > > +/* Disallow load-pair instructions to be formed in combine/peephole.
> */
> > > +AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine",
> > > NO_LDP_COMBINE)
> > > +
> > >  AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs",
> > > RENAME_LOAD_REGS)
> > >
> > >  AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants",
> > > CSE_SVE_VL_CONSTANTS)
> > > diff --git a/gcc/config/aarch64/aarch64.cc
> b/gcc/config/aarch64/aarch64.cc
> > > index f4ef22ce02f..8dc1a9ceb17 100644
> > > --- a/gcc/config/aarch64/aarch64.cc
> > > +++ b/gcc/config/aarch64/aarch64.cc
> > > @@ -1971,7 +1971,7 @@ static const struct tune_params ampere1a_tunings
> > > =
> > >    2, /* min_div_recip_mul_df.  */
> > >    0, /* max_case_values.  */
> > >    tune_params::AUTOPREFETCHER_WEAK,  /* autoprefetcher_model.  */
> > > -  (AARCH64_EXTRA_TUNE_NONE),         /* tune_flags.  */
> > > +  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),       /* tune_flags.  */
> > >    &ampere1_prefetch_tune
> > >  };
> > >
> > > @@ -26053,6 +26053,12 @@ aarch64_operands_ok_for_ldpstp (rtx
> > > *operands, bool load,
> > >    enum reg_class rclass_1, rclass_2;
> > >    rtx mem_1, mem_2, reg_1, reg_2;
> > >
> > > +  /* Allow the tuning structure to disable LDP instruction formation
> > > +     from combining instructions (e.g., in peephole2).  */
> > > +  if (load && (aarch64_tune_params.extra_tuning_flags
> > > +            & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
> > > +    return false;
> >
> > If we do decide to do this, I think this is not a complete approach. See
> the similar tuning flag AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS.
> > There's various other places in the backend that would need to be
> adjusted to avoid bringing loads together for the peephole2s to merge (the
> sched_fusion stuff).
> > Plus there's the cpymem expansions that would generate load pairs too...
>
> I have add-on patches for these, but given that I don't have direct
> access to the benchmarking machine and the benchmarks have been run
> with this functionality only, I didn't submit them for the time being.
> Do you see a path to get this in during the current cycle and defer
> the add-on patches (happy to resubmit as a series) only?
>
> > We'd want some testcases added to check that LDPs are blocked too...
> >
> > Thanks,
> > Kyrill
> >
> > > +
> > >    if (load)
> > >      {
> > >        mem_1 = operands[1];
> > > --
> > > 2.34.1
> >
>
  
Kyrylo Tkachov April 14, 2023, 11:02 a.m. UTC | #5
Hi Philipp,

From: Philipp Tomsich <philipp.tomsich@vrull.eu> 
Sent: Friday, April 14, 2023 11:26 AM
To: Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>
Cc: gcc-patches@gcc.gnu.org; Di Zhao <di.zhao@amperecomputing.com>
Subject: Re: [PATCH] aarch64: disable LDP via tuning structure for -mcpu=ampere1



On Fri, 14 Apr 2023 at 11:31, Philipp Tomsich <mailto:philipp.tomsich@vrull.eu> wrote:
Kyrylo,

On Fri, 14 Apr 2023 at 11:21, Kyrylo Tkachov <mailto:Kyrylo.Tkachov@arm.com> wrote:
>
> Hi Philipp,
>
> > -----Original Message-----
> > From: Philipp Tomsich <mailto:philipp.tomsich@vrull.eu>
> > Sent: Friday, April 14, 2023 12:22 AM
> > To: mailto:gcc-patches@gcc.gnu.org
> > Cc: Kyrylo Tkachov <mailto:Kyrylo.Tkachov@arm.com>; Philipp Tomsich
> > <mailto:philipp.tomsich@vrull.eu>; Di Zhao <mailto:di.zhao@amperecomputing.com>
> > Subject: [PATCH] aarch64: disable LDP via tuning structure for -
> > mcpu=ampere1
> >
> > AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
> > Given the chance that this causes instructions to slip into the next
> > decoding cycle and the additional overheads when handling
> > cacheline-crossing LDP instructions, we disable the generation of LDP
> > isntructions through the tuning structure from instruction combining
> > (such as in peephole2).
> >
> > Given the code-density benefits in builtins and prologue/epilogue
> > expansion, we allow LDPs there.
>
> LDPs are indeed quite an important part of the ISA for code density and there are, in principle, second-order benefits from using them, like keeping the instruction cache footprint low (which can be important for large workloads).
> Did you gather some benchmarks showing a benefit of disabling them in this manner?

>This has been benchmark-driven, but I need to follow up separately (as
>I the final numbers are with the folks that have access to the
>benchmark machines)..

>Here are the numbers for the submitted change for AmpereOne:
>   503.bwaves_r.        -0.88%
>   507.cactuBSSN_r    0.35%
>   508.namd_r             3.09%
>   510.parest_r           -2.99%
>   511.povray_r            5.54%
>   519.lbm_r               15.83%
>   521.wrf_r                  0.56%
>   526.blender_r           2.47%
>   527.cam4_r              0.70%
>   538.imagick_r           0.00%
>   544.nab_r                -0.33%
>  549.fotonik3d_r.      -0.42%
>   554.roms_r               0.00%
>   = total                       1.79%
 
Thanks for getting these, the gains are quite significant.

>
> > This commit:
> >  * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
> >  * allows -moverride=tune=... to override this
> >
> > Signed-off-by: Philipp Tomsich <mailto:philipp.tomsich@vrull.eu>
> > Co-Authored-By: Di Zhao <mailto:di.zhao@amperecomputing.com>
> >
> > gcc/ChangeLog:
> >
> >       * config/aarch64/aarch64-tuning-flags.def
> > (AARCH64_EXTRA_TUNING_OPTION):
> >       Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
> >       * config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
> >       Check for the above tuning option when processing loads.
> >
> > ---
> >
> >  gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
> >  gcc/config/aarch64/aarch64.cc               | 8 +++++++-
> >  2 files changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> > b/gcc/config/aarch64/aarch64-tuning-flags.def
> > index 712895a5263..52112ba7c48 100644
> > --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> > +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> > @@ -44,6 +44,9 @@ AARCH64_EXTRA_TUNING_OPTION
> > ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
> >  /* Disallow load/store pair instructions on Q-registers.  */
> >  AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs",
> > NO_LDP_STP_QREGS)
> >
> > +/* Disallow load-pair instructions to be formed in combine/peephole.  */
> > +AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine",
> > NO_LDP_COMBINE)
> > +
> >  AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs",
> > RENAME_LOAD_REGS)
> >
> >  AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants",
> > CSE_SVE_VL_CONSTANTS)
> > diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
> > index f4ef22ce02f..8dc1a9ceb17 100644
> > --- a/gcc/config/aarch64/aarch64.cc
> > +++ b/gcc/config/aarch64/aarch64.cc
> > @@ -1971,7 +1971,7 @@ static const struct tune_params ampere1a_tunings
> > =
> >    2, /* min_div_recip_mul_df.  */
> >    0, /* max_case_values.  */
> >    tune_params::AUTOPREFETCHER_WEAK,  /* autoprefetcher_model.  */
> > -  (AARCH64_EXTRA_TUNE_NONE),         /* tune_flags.  */
> > +  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),       /* tune_flags.  */
> >    &ampere1_prefetch_tune
> >  };
> >
> > @@ -26053,6 +26053,12 @@ aarch64_operands_ok_for_ldpstp (rtx
> > *operands, bool load,
> >    enum reg_class rclass_1, rclass_2;
> >    rtx mem_1, mem_2, reg_1, reg_2;
> >
> > +  /* Allow the tuning structure to disable LDP instruction formation
> > +     from combining instructions (e.g., in peephole2).  */
> > +  if (load && (aarch64_tune_params.extra_tuning_flags
> > +            & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
> > +    return false;
>
> If we do decide to do this, I think this is not a complete approach. See the similar tuning flag AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS.
> There's various other places in the backend that would need to be adjusted to avoid bringing loads together for the peephole2s to merge (the sched_fusion stuff).
> Plus there's the cpymem expansions that would generate load pairs too...

>I have add-on patches for these, but given that I don't have direct
>access to the benchmarking machine and the benchmarks have been run
>with this functionality only, I didn't submit them for the time being.
>Do you see a path to get this in during the current cycle and defer
>the add-on patches (happy to resubmit as a series) only?

Yeah, I agree that we'll want something with minimal risk at this stage.
Just to confirm, have you tried the pre-existing AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS for AmpereOne and it didn't give the benefit that this patch does?
I would think that AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS would be beneficial as well, though maybe blocking Q-reg STPs is undesirable?

If you are planning to implement this properly in stage 1 for GCC 14 then I'd be okay with this approach if you add a TODO marker in this hunk describing the work needed to do this more comprehensively.
It seems we'll want a more flexible hierarchy for controlling the emission LDPs, STPs per mode size similar to how we use cpu_approx_modes ☹
Talking offline to Richard, one thing you may want to try is restricting LDP formation to only aligned addresses, similar to what AARCH64_EXTRA_TUNE_SLOW_UNALIGNED_LDPW does.
But this patch is okay for trunk if testing shows no problems.

Looking forward to a stage 1 series 😊
Thanks,
Kyrill

> We'd want some testcases added to check that LDPs are blocked too...
>
> Thanks,
> Kyrill
>
> > +
> >    if (load)
> >      {
> >        mem_1 = operands[1];
> > --
> > 2.34.1
>
  
Philipp Tomsich April 14, 2023, 11:16 a.m. UTC | #6
On Fri, 14 Apr 2023 at 13:02, Kyrylo Tkachov <Kyrylo.Tkachov@arm.com> wrote:

> Hi Philipp,
>
> From: Philipp Tomsich <philipp.tomsich@vrull.eu>
> Sent: Friday, April 14, 2023 11:26 AM
> To: Kyrylo Tkachov <Kyrylo.Tkachov@arm.com>
> Cc: gcc-patches@gcc.gnu.org; Di Zhao <di.zhao@amperecomputing.com>
> Subject: Re: [PATCH] aarch64: disable LDP via tuning structure for
> -mcpu=ampere1
>
>
>
> On Fri, 14 Apr 2023 at 11:31, Philipp Tomsich <mailto:
> philipp.tomsich@vrull.eu> wrote:
> Kyrylo,
>
> On Fri, 14 Apr 2023 at 11:21, Kyrylo Tkachov <mailto:
> Kyrylo.Tkachov@arm.com> wrote:
> >
> > Hi Philipp,
> >
> > > -----Original Message-----
> > > From: Philipp Tomsich <mailto:philipp.tomsich@vrull.eu>
> > > Sent: Friday, April 14, 2023 12:22 AM
> > > To: mailto:gcc-patches@gcc.gnu.org
> > > Cc: Kyrylo Tkachov <mailto:Kyrylo.Tkachov@arm.com>; Philipp Tomsich
> > > <mailto:philipp.tomsich@vrull.eu>; Di Zhao <mailto:
> di.zhao@amperecomputing.com>
> > > Subject: [PATCH] aarch64: disable LDP via tuning structure for -
> > > mcpu=ampere1
> > >
> > > AmpereOne (-mcpu=ampere1) breaks LDP instructions into two uops.
> > > Given the chance that this causes instructions to slip into the next
> > > decoding cycle and the additional overheads when handling
> > > cacheline-crossing LDP instructions, we disable the generation of LDP
> > > isntructions through the tuning structure from instruction combining
> > > (such as in peephole2).
> > >
> > > Given the code-density benefits in builtins and prologue/epilogue
> > > expansion, we allow LDPs there.
> >
> > LDPs are indeed quite an important part of the ISA for code density and
> there are, in principle, second-order benefits from using them, like
> keeping the instruction cache footprint low (which can be important for
> large workloads).
> > Did you gather some benchmarks showing a benefit of disabling them in
> this manner?
>
> >This has been benchmark-driven, but I need to follow up separately (as
> >I the final numbers are with the folks that have access to the
> >benchmark machines)..
>
> >Here are the numbers for the submitted change for AmpereOne:
> >   503.bwaves_r.        -0.88%
> >   507.cactuBSSN_r    0.35%
> >   508.namd_r             3.09%
> >   510.parest_r           -2.99%
> >   511.povray_r            5.54%
> >   519.lbm_r               15.83%
> >   521.wrf_r                  0.56%
> >   526.blender_r           2.47%
> >   527.cam4_r              0.70%
> >   538.imagick_r           0.00%
> >   544.nab_r                -0.33%
> >  549.fotonik3d_r.      -0.42%
> >   554.roms_r               0.00%
> >   = total                       1.79%
>
> Thanks for getting these, the gains are quite significant.
>
> >
> > > This commit:
> > >  * adds a new tuning option AARCH64_EXTRA_TUNE_NO_LDP_COMBINE
> > >  * allows -moverride=tune=... to override this
> > >
> > > Signed-off-by: Philipp Tomsich <mailto:philipp.tomsich@vrull.eu>
> > > Co-Authored-By: Di Zhao <mailto:di.zhao@amperecomputing.com>
> > >
> > > gcc/ChangeLog:
> > >
> > >       * config/aarch64/aarch64-tuning-flags.def
> > > (AARCH64_EXTRA_TUNING_OPTION):
> > >       Add AARCH64_EXTRA_TUNE_NO_LDP_COMBINE.
> > >       * config/aarch64/aarch64.cc (aarch64_operands_ok_for_ldpstp):
> > >       Check for the above tuning option when processing loads.
> > >
> > > ---
> > >
> > >  gcc/config/aarch64/aarch64-tuning-flags.def | 3 +++
> > >  gcc/config/aarch64/aarch64.cc               | 8 +++++++-
> > >  2 files changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
> > > b/gcc/config/aarch64/aarch64-tuning-flags.def
> > > index 712895a5263..52112ba7c48 100644
> > > --- a/gcc/config/aarch64/aarch64-tuning-flags.def
> > > +++ b/gcc/config/aarch64/aarch64-tuning-flags.def
> > > @@ -44,6 +44,9 @@ AARCH64_EXTRA_TUNING_OPTION
> > > ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
> > >  /* Disallow load/store pair instructions on Q-registers.  */
> > >  AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs",
> > > NO_LDP_STP_QREGS)
> > >
> > > +/* Disallow load-pair instructions to be formed in combine/peephole.
> */
> > > +AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine",
> > > NO_LDP_COMBINE)
> > > +
> > >  AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs",
> > > RENAME_LOAD_REGS)
> > >
> > >  AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants",
> > > CSE_SVE_VL_CONSTANTS)
> > > diff --git a/gcc/config/aarch64/aarch64.cc
> b/gcc/config/aarch64/aarch64.cc
> > > index f4ef22ce02f..8dc1a9ceb17 100644
> > > --- a/gcc/config/aarch64/aarch64.cc
> > > +++ b/gcc/config/aarch64/aarch64.cc
> > > @@ -1971,7 +1971,7 @@ static const struct tune_params ampere1a_tunings
> > > =
> > >    2, /* min_div_recip_mul_df.  */
> > >    0, /* max_case_values.  */
> > >    tune_params::AUTOPREFETCHER_WEAK,  /* autoprefetcher_model.  */
> > > -  (AARCH64_EXTRA_TUNE_NONE),         /* tune_flags.  */
> > > +  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),       /* tune_flags.  */
> > >    &ampere1_prefetch_tune
> > >  };
> > >
> > > @@ -26053,6 +26053,12 @@ aarch64_operands_ok_for_ldpstp (rtx
> > > *operands, bool load,
> > >    enum reg_class rclass_1, rclass_2;
> > >    rtx mem_1, mem_2, reg_1, reg_2;
> > >
> > > +  /* Allow the tuning structure to disable LDP instruction formation
> > > +     from combining instructions (e.g., in peephole2).  */
> > > +  if (load && (aarch64_tune_params.extra_tuning_flags
> > > +            & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
> > > +    return false;
> >
> > If we do decide to do this, I think this is not a complete approach. See
> the similar tuning flag AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS.
> > There's various other places in the backend that would need to be
> adjusted to avoid bringing loads together for the peephole2s to merge (the
> sched_fusion stuff).
> > Plus there's the cpymem expansions that would generate load pairs too...
>
> >I have add-on patches for these, but given that I don't have direct
> >access to the benchmarking machine and the benchmarks have been run
> >with this functionality only, I didn't submit them for the time being.
> >Do you see a path to get this in during the current cycle and defer
> >the add-on patches (happy to resubmit as a series) only?
>
> Yeah, I agree that we'll want something with minimal risk at this stage.
> Just to confirm, have you tried the pre-existing
> AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS for AmpereOne and it didn't give the
> benefit that this patch does?
> I would think that AARCH64_EXTRA_TUNE_NO_LDP_STP_QREGS would be beneficial
> as well, though maybe blocking Q-reg STPs is undesirable?
>

The issue on lbm are LDPs on 64bit registers that are not aligned to to a
16byte boundary (i.e., only naturally aligned).  Disabling LDPs for 32bit
quantities provided some small additive gains.
So disabling Q-regs doesn't help and should even be beneficial in the
memcpy case.


> If you are planning to implement this properly in stage 1 for GCC 14 then
> I'd be okay with this approach if you add a TODO marker in this hunk
> describing the work needed to do this more comprehensively.
> It seems we'll want a more flexible hierarchy for controlling the emission
> LDPs, STPs per mode size similar to how we use cpu_approx_modes ☹
> Talking offline to Richard, one thing you may want to try is restricting
> LDP formation to only aligned addresses, similar to what
> AARCH64_EXTRA_TUNE_SLOW_UNALIGNED_LDPW does.

But this patch is okay for trunk if testing shows no problems.
>

I'll resubmit with the two following changes

   - the TODO marker
   - the testcase

asking for an OK.

Philipp.



> Looking forward to a stage 1 series 😊
> Thanks,
> Kyrill
>
> > We'd want some testcases added to check that LDPs are blocked too...
> >
> > Thanks,
> > Kyrill
> >
> > > +
> > >    if (load)
> > >      {
> > >        mem_1 = operands[1];
> > > --
> > > 2.34.1
> >
>
  

Patch

diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def b/gcc/config/aarch64/aarch64-tuning-flags.def
index 712895a5263..52112ba7c48 100644
--- a/gcc/config/aarch64/aarch64-tuning-flags.def
+++ b/gcc/config/aarch64/aarch64-tuning-flags.def
@@ -44,6 +44,9 @@  AARCH64_EXTRA_TUNING_OPTION ("cheap_shift_extend", CHEAP_SHIFT_EXTEND)
 /* Disallow load/store pair instructions on Q-registers.  */
 AARCH64_EXTRA_TUNING_OPTION ("no_ldp_stp_qregs", NO_LDP_STP_QREGS)
 
+/* Disallow load-pair instructions to be formed in combine/peephole.  */
+AARCH64_EXTRA_TUNING_OPTION ("no_ldp_combine", NO_LDP_COMBINE)
+
 AARCH64_EXTRA_TUNING_OPTION ("rename_load_regs", RENAME_LOAD_REGS)
 
 AARCH64_EXTRA_TUNING_OPTION ("cse_sve_vl_constants", CSE_SVE_VL_CONSTANTS)
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index f4ef22ce02f..8dc1a9ceb17 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -1971,7 +1971,7 @@  static const struct tune_params ampere1a_tunings =
   2,	/* min_div_recip_mul_df.  */
   0,	/* max_case_values.  */
   tune_params::AUTOPREFETCHER_WEAK,	/* autoprefetcher_model.  */
-  (AARCH64_EXTRA_TUNE_NONE),		/* tune_flags.  */
+  (AARCH64_EXTRA_TUNE_NO_LDP_COMBINE),	/* tune_flags.  */
   &ampere1_prefetch_tune
 };
 
@@ -26053,6 +26053,12 @@  aarch64_operands_ok_for_ldpstp (rtx *operands, bool load,
   enum reg_class rclass_1, rclass_2;
   rtx mem_1, mem_2, reg_1, reg_2;
 
+  /* Allow the tuning structure to disable LDP instruction formation
+     from combining instructions (e.g., in peephole2).  */
+  if (load && (aarch64_tune_params.extra_tuning_flags
+	       & AARCH64_EXTRA_TUNE_NO_LDP_COMBINE))
+    return false;
+
   if (load)
     {
       mem_1 = operands[1];