[committed,00/22] arm: testsuite: clean up some architecture-specific tests

  A lot of the arm-specific compiler tests require a specific CPU or
architecture to be specified.  This causes problems if the test suite
run is set up to test a specific architecture or CPU that differs from
the test's requirements.  An exmple I use commonly is

	set target_list { "arm-qemu{,-mthumb}" }

but it is possible to also test other architectures or CPUs this way, for
example,

	set target_list { "arm-qemu{,-mthumb,
	  -march=armv6t2+fp/-mfloat-abi=hard,
	  -march=armv8-a+simd/-mthumb/-mfloat-abi=hard,
	  -mcpu=cortex-m33/-mfloat-abi=softfp,
	  -mcpu=cortex-m55/-mfloat-abi=hard,
	  -mcpu=cortex-m23}" }

[line breaks inserted for readability]

tests 7 permutations of
 - base configuration
 - base configuration with -mthumb
 - armv6t2 with FP and a hard-float ABI
 - armv8-a with Neon and thumb and the hard-float ABI
 - cortex-m33 with the softfp ABI
 - cortex-m55 with the hard-float ABI
 - cortex-m23

Over time we have developed a series of checks that can be used to
ensure that we test what we want to test and don't test if the options
conflict, but these have been applied somewhat haphazzardly and as the
framework has been improved tests haven't been updated to make full
use of the tests.

This patch series deploys the framework dg- directives more widely
across the arm-specific tests to make testing more consistent.  On
that long list of permutations above this results in the following
changes:

16 tests move from FAIL to PASS.
21 new FAILS.  
562 new tests that PASS
74 tests that passed have been removed

The new FAILs are real issues on targets that only support
single-precision FP and should be investigated at some point, but
probably aren't urgent given the use cases for cores with this issue.

The tests that have been removed come from the fact that we now more
accurately test that option combinations won't cause problems; they
are related to the fact that if the testrun config specifies -mcpu,
but the test sets -march, then we can get an architecture conflict.
I have some ideas about how to address this, but that's for a later
test series.

committed to master branch.

R.

Richard Earnshaw (22):
  arm: testsuite: correctly detect armv6t2 hardware for acle execution
    tests
  arm: testsuite: correctly detect hard_float
  arm: testsuite: avoid hard-float ABI incompatibility with -march
  arm: testsuite: avoid problems with -mfpu=auto in pacbti-m-predef-11.c
  arm: testsuite: avoid problems with -mfpu=auto in attr-crypto.c
  arm: testsuite: avoid problems with -mfpu=auto in attr_thumb-static2.c
  arm: testsuite: tidy up pre-run check for g2.c
  arm: testsuite: improve compatibility of arm/lto/pr96939_1.c
  arm: testsuite: tidy up pr65647-2.c pre-checks.
  arm: testsuite: improve compatibility of arm/pr78353-*.c
  arm: testsuite: improve compatibility of pr88648-asm-syntax-unified.c
  arm: testsuite: improve compatibility of pragma_arch_attribute*.c
  arm: testsuite: improve compatibility of pragma_arch_switch_2.c
  arm: testsuite: modernize framework usage for arm/scd42-2.c
  arm: testsuite: improve compatibility of ftest-armv7m-thumb.c
  arm: testsuite: improve compatibility of gcc.target/arm/macro_defs*.c
  arm: testsuite: improve compatibility of
    gcc.target/arm/optional_thumb-*.c
  arm: testsuite: improve compatibility of gcc.target/arm/pr19599.c
  arm: testsuite: improve compatibility of gcc.target/arm/pr59575.c
  testsuite: arm: tighten up mode-specific ISA tests
  arm: testsuite: fix some more architecture tests
  arm: testsuite: improve compatibility of gcc.dg/debug/pr57351.c

 gcc/testsuite/gcc.dg/debug/pr57351.c          |  7 +-
 .../arm/acle/data-intrinsics-armv6.c          |  2 +-
 .../arm/acle/data-intrinsics-rbit.c           |  2 +-
 .../gcc.target/arm/acle/pacbti-m-predef-11.c  |  2 +-
 gcc/testsuite/gcc.target/arm/attr-crypto.c    |  2 +-
 .../gcc.target/arm/attr_thumb-static2.c       |  2 +-
 .../gcc.target/arm/ftest-armv7m-thumb.c       |  3 +-
 gcc/testsuite/gcc.target/arm/g2.c             | 10 +-
 gcc/testsuite/gcc.target/arm/lto/pr96939_1.c  |  2 +-
 gcc/testsuite/gcc.target/arm/macro_defs0.c    |  7 +-
 gcc/testsuite/gcc.target/arm/macro_defs1.c    |  6 +-
 gcc/testsuite/gcc.target/arm/macro_defs2.c    |  6 +-
 .../gcc.target/arm/optional_thumb-1.c         |  2 +-
 .../gcc.target/arm/optional_thumb-3.c         |  4 +-
 gcc/testsuite/gcc.target/arm/pr19599.c        |  2 +-
 gcc/testsuite/gcc.target/arm/pr59575.c        |  4 +-
 gcc/testsuite/gcc.target/arm/pr60650-2.c      |  4 +-
 gcc/testsuite/gcc.target/arm/pr60657.c        |  4 +-
 gcc/testsuite/gcc.target/arm/pr60663.c        |  4 +-
 gcc/testsuite/gcc.target/arm/pr65647-2.c      |  3 +-
 gcc/testsuite/gcc.target/arm/pr78353-1.c      |  3 +-
 gcc/testsuite/gcc.target/arm/pr78353-2.c      |  3 +-
 gcc/testsuite/gcc.target/arm/pr81863.c        |  4 +-
 .../arm/pr88648-asm-syntax-unified.c          |  2 +-
 gcc/testsuite/gcc.target/arm/pr97969.c        |  4 +-
 gcc/testsuite/gcc.target/arm/pr98931.c        |  5 +-
 .../gcc.target/arm/pragma_arch_attribute.c    |  6 +-
 .../gcc.target/arm/pragma_arch_attribute_2.c  |  2 +-
 .../gcc.target/arm/pragma_arch_attribute_3.c  |  2 +-
 .../gcc.target/arm/pragma_arch_switch_2.c     |  5 +-
 gcc/testsuite/gcc.target/arm/scd42-2.c        |  9 +-
 gcc/testsuite/gcc.target/arm/tail-long-call.c |  6 +-
 gcc/testsuite/lib/target-supports.exp         | 92 +++++++++++++------
 33 files changed, 130 insertions(+), 91 deletions(-)
  

Hi Richard,

On Mon, 13 Nov 2023 at 15:28, Richard Earnshaw <rearnsha@arm.com> wrote:
>
>
> A number of tests in the gcc testsuite, especially for arm-specific
> targets, add various flags to control the architecture.  These run
> into problems when the compiler is configured with -mfpu=auto if the
> new architecture lacks an architectural feature that implies we have
> floating-point instructions.
>
> The testsuite makes this worse as it falls foul of this requirement in
> the base architecture strings provided by target-supports.exp.
>
> To fix this we add "+fp", or something equivalent to this, to all the
> base architecture specifications.  The feature will be ignored if the
> float ABI is set to soft.
>
> gcc/testsuite:
>
>         * lib/target-supports.exp (check_effective_target_arm_arch_FUNC_ok):
>         Add base FPU specifications to all architectures that can support
>         one.
> ---
>  gcc/testsuite/lib/target-supports.exp | 50 +++++++++++++--------------
>  1 file changed, 25 insertions(+), 25 deletions(-)
>

Our CI has detected some regressions with this patch, in particular
when testing for cortex-m55:

with -mthumb/-march=armv8.1-m.main+mve.fp+fp.dp/-mtune=cortex-m55/-mfloat-abi=hard/-mfpu=auto
and GCC configured with --disable-multilib --with-mode=thumb
--with-arch=armv8.1-m.main+mve.fp+fp.dp --with-float=hard

you can see our logs here:
https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m55_hard_eabi-build/209/artifact/artifacts/00-sumfiles/

Thanks,

Christophe
  

On Mon, 20 Nov 2023 at 13:44, Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
>
>
> On 20/11/2023 10:23, Christophe Lyon wrote:
> > Hi Richard,
> >
> > On Mon, 13 Nov 2023 at 15:28, Richard Earnshaw <rearnsha@arm.com> wrote:
> >>
> >>
> >> A number of tests in the gcc testsuite, especially for arm-specific
> >> targets, add various flags to control the architecture.  These run
> >> into problems when the compiler is configured with -mfpu=auto if the
> >> new architecture lacks an architectural feature that implies we have
> >> floating-point instructions.
> >>
> >> The testsuite makes this worse as it falls foul of this requirement in
> >> the base architecture strings provided by target-supports.exp.
> >>
> >> To fix this we add "+fp", or something equivalent to this, to all the
> >> base architecture specifications.  The feature will be ignored if the
> >> float ABI is set to soft.
> >>
> >> gcc/testsuite:
> >>
> >>         * lib/target-supports.exp (check_effective_target_arm_arch_FUNC_ok):
> >>         Add base FPU specifications to all architectures that can support
> >>         one.
> >> ---
> >>  gcc/testsuite/lib/target-supports.exp | 50 +++++++++++++--------------
> >>  1 file changed, 25 insertions(+), 25 deletions(-)
> >>
> >
> > Our CI has detected some regressions with this patch, in particular
> > when testing for cortex-m55:
> >
> > with
> > -mthumb/-march=armv8.1-m.main+mve.fp+fp.dp/-mtune=cortex-m55/-mfloat-abi=hard/-mfpu=auto
> > and GCC configured with --disable-multilib --with-mode=thumb
> > --with-arch=armv8.1-m.main+mve.fp+fp.dp --with-float=hard
> >
> > you can see our logs here:
> > https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m55_hard_eabi-build/209/artifact/artifacts/00-sumfiles/ <https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m55_hard_eabi-build/209/artifact/artifacts/00-sumfiles/>
> >
> > Thanks,
> >
> > Christophe
>
> What exactly am I supposed to be looking at here?  I see no description
> of what those logs represent.  If they are supposed to be before and
> after, then why does the after only run a tiny fraction of the testsuite
> (Running gcc.git~master/gcc/testsuite/gcc.target/arm/arm.exp ...
> Running gcc.git~master/gcc/testsuite/gcc.target/arm/cmse/cmse.exp ...
> Running gcc.git~master/gcc/testsuite/gcc.target/arm/lto/lto.exp ...)
>
> The logs give no clue as to why the reminder of the testsuite wasn't run.
>
> Please don't make me guess.
>

Here is a summary with the list of regressions:
https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-thumb_m55_hard_eabi-build/209/artifact/artifacts/notify/regressions.sum/*view*/

I thought you'd be able to find your way in the logs above, the .0
files contain the logs of the initial full testsuite run, and .1 ones
contain the logs of the second run of the testsuite, restricted to the
.exp files where we detected regressions. So looking at gcc.log.1.xz
will give you details of the regressions shown in the link above.

Christophe

> R.