testsuite: Default make check-g++ vs. tests for newest C++ standard

Message ID Y1AXafpqS9xxvvTp@tucnak
State Unresolved
Headers
Series testsuite: Default make check-g++ vs. tests for newest C++ standard |

Checks

Context Check Description
snail/gcc-patch-check warning Git am fail log

Commit Message

Jakub Jelinek Oct. 19, 2022, 3:27 p.m. UTC
  Hi!

On Wed, Oct 19, 2022 at 09:10:48AM -0400, Jason Merrill wrote:
> > The screw-up on my side with libstdc++ testing (tested normally rather
> > than in C++23 mode) makes me wonder if we couldn't tweak the default
> > testing.
> > Dunno what libstdc++ testing normally does (just C++17?), make check-g++
> > tests by default { 98, 14, 17, 20 } (and I regularly use
> > GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
> > cover libstdc++ I guess).
> > When adding tests for upcoming C++ version, one always has a dilemma
> > whether to use explicit // { dg-options "-std=c++2b" }
> > or -std=gnu++2b and similar, then the test works in all modes, but it might
> > be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
> > test so that when 23 is tested by default and say 26 or 29 appears too,
> > we test it also in those modes, or just go with
> > // { dg-do whatever { target c++23 } }
> > which has the disadvantage that it is skipped when testing by default and
> > one only tests it if he asks for the newer version.
> > 
> > I wonder if we couldn't for the default testing (when one doesn't
> > specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
> > improve things a little bit by automatically treat those
> > // { dg-do whatever { target c++23 } }
> > tests as // { dg-options "-std=c++2b" }.
> 
> That would be great.

Here is a patch that implements it.
On make check-g++ RUNTESTFLAGS=dg.exp=cpp23/*.C
the vanilla vs. patched gcc difference is:
                === g++ Summary ===

# of expected passes            1743
# of expected failures          2
# of unsupported tests          224
vs.
                === g++ Summary ===

# of expected passes            1829
# of expected failures          2
# of unsupported tests          112

(g++.sum details attached).

2022-10-19  Jakub Jelinek  <jakub@redhat.com>

	* lib/g++-dg.exp (g++-dg-runtest): When using defaulted
	std_list, if test has { dg-do * { target c++23 } } directive,
	use { 23 } with which the test will run instead of { 98 14 17 20 }
	which would make it UNSUPPORTED in all cases.



	Jakub
--- testsuite/g++/g++.sum	2022-10-19 17:06:38.351280055 +0200
+++ testsuite/g++/g++.sum	2022-10-19 17:11:15.948537868 +0200
@@ -647,56 +647,39 @@ PASS: g++.dg/cpp23/auto-array4.C  -std=g
 PASS: g++.dg/cpp23/auto-array4.C  -std=gnu++20  (test for warnings, line 12)
 PASS: g++.dg/cpp23/auto-array4.C  -std=gnu++20  at line 13 (test for warnings, line 12)
 PASS: g++.dg/cpp23/auto-array4.C  -std=gnu++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C  -std=c++20
+PASS: g++.dg/cpp23/auto-fncast1.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast10.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast11.C  -std=c++23  (test for errors, line 19)
+PASS: g++.dg/cpp23/auto-fncast11.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast12.C  -std=c++23  (test for errors, line 7)
+PASS: g++.dg/cpp23/auto-fncast12.C  -std=c++23  (test for errors, line 8)
+PASS: g++.dg/cpp23/auto-fncast12.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast2.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 10)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 11)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 13)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 14)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 15)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 16)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 17)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 18)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 19)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23  (test for errors, line 20)
+PASS: g++.dg/cpp23/auto-fncast3.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast4.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast5.C  -std=c++23  (test for errors, line 14)
+PASS: g++.dg/cpp23/auto-fncast5.C  -std=c++23  (test for errors, line 15)
+PASS: g++.dg/cpp23/auto-fncast5.C  -std=c++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/auto-fncast6.C  -std=c++98
 UNSUPPORTED: g++.dg/cpp23/auto-fncast6.C  -std=c++14
 UNSUPPORTED: g++.dg/cpp23/auto-fncast6.C  -std=c++17
 PASS: g++.dg/cpp23/auto-fncast6.C  -std=c++20  (test for errors, line 10)
 PASS: g++.dg/cpp23/auto-fncast6.C  -std=c++20  (test for errors, line 11)
 PASS: g++.dg/cpp23/auto-fncast6.C  -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C  -std=c++20
+PASS: g++.dg/cpp23/auto-fncast7.C  -std=c++23  (test for errors, line 8)
+PASS: g++.dg/cpp23/auto-fncast7.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast8.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast9.C  -std=c++23 (test for excess errors)
 PASS: g++.dg/cpp23/charlit-encoding1.C  -std=c++98 (test for excess errors)
 PASS: g++.dg/cpp23/charlit-encoding1.C  -std=c++98 execution test
 PASS: g++.dg/cpp23/charlit-encoding1.C  -std=c++14 (test for excess errors)
@@ -705,10 +688,9 @@ PASS: g++.dg/cpp23/charlit-encoding1.C
 PASS: g++.dg/cpp23/charlit-encoding1.C  -std=c++17 execution test
 PASS: g++.dg/cpp23/charlit-encoding1.C  -std=c++20 (test for excess errors)
 PASS: g++.dg/cpp23/charlit-encoding1.C  -std=c++20 execution test
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C  -std=c++20
+PASS: g++.dg/cpp23/concepts-err1.C  -std=c++23  (test for errors, line 5)
+PASS: g++.dg/cpp23/concepts-err1.C  -std=c++23  (test for errors, line 6)
+PASS: g++.dg/cpp23/concepts-err1.C  -std=c++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/consteval-if1.C  -std=gnu++98
 UNSUPPORTED: g++.dg/cpp23/consteval-if1.C  -std=gnu++14
 UNSUPPORTED: g++.dg/cpp23/consteval-if1.C  -std=gnu++17
@@ -904,60 +886,54 @@ PASS: g++.dg/cpp23/constexpr-nonlit7.C
 PASS: g++.dg/cpp23/constexpr-nonlit7.C  -std=c++17 (test for excess errors)
 PASS: g++.dg/cpp23/constexpr-nonlit7.C  -std=c++20 (test for excess errors)
 PASS: g++.dg/cpp23/cplusplus.C   (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/decltype1.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/decltype1.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/decltype1.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/decltype1.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/decltype2.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/decltype2.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/decltype2.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/decltype2.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision1.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision1.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision1.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision1.C  -std=c++20
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for errors, line 14)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for warnings, line 16)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for errors, line 20)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for errors, line 41)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for warnings, line 48)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for errors, line 62)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for warnings, line 69)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23  (test for warnings, line 83)
+PASS: g++.dg/cpp23/decltype1.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/decltype2.C  -std=c++23  (test for warnings, line 35)
+PASS: g++.dg/cpp23/decltype2.C  -std=c++23  (test for warnings, line 47)
+PASS: g++.dg/cpp23/decltype2.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision1.C  -std=c++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/elision2.C  -std=c++98
 UNSUPPORTED: g++.dg/cpp23/elision2.C  -std=c++14
 UNSUPPORTED: g++.dg/cpp23/elision2.C  -std=c++17
 PASS: g++.dg/cpp23/elision2.C  -std=c++20  (test for errors, line 24)
 PASS: g++.dg/cpp23/elision2.C  -std=c++20  (test for errors, line 30)
 PASS: g++.dg/cpp23/elision2.C  -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/elision3.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision3.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision3.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision3.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision4.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision4.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision4.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision4.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision5.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision5.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision5.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision5.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision6.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision6.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision6.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision6.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision7.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision7.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision7.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision7.C  -std=c++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C  -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C  -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C  -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C  -std=gnu++20
+PASS: g++.dg/cpp23/elision3.C  -std=c++23  (test for errors, line 15)
+PASS: g++.dg/cpp23/elision3.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision4.C  -std=c++23  (test for errors, line 14)
+PASS: g++.dg/cpp23/elision4.C  -std=c++23  (test for errors, line 27)
+PASS: g++.dg/cpp23/elision4.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision5.C  -std=c++23  (test for errors, line 34)
+PASS: g++.dg/cpp23/elision5.C  -std=c++23  (test for errors, line 43)
+PASS: g++.dg/cpp23/elision5.C  -std=c++23  (test for warnings, line 52)
+PASS: g++.dg/cpp23/elision5.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision6.C  -std=c++23  (test for errors, line 16)
+PASS: g++.dg/cpp23/elision6.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision7.C  -std=c++23  (test for errors, line 44)
+PASS: g++.dg/cpp23/elision7.C  -std=c++23  (test for warnings, line 46)
+PASS: g++.dg/cpp23/elision7.C  -std=c++23  (test for errors, line 53)
+PASS: g++.dg/cpp23/elision7.C  -std=c++23  (test for errors, line 65)
+PASS: g++.dg/cpp23/elision7.C  -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/ext-floating1.C  -std=gnu++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/ext-floating10.C  -std=gnu++98
 UNSUPPORTED: g++.dg/cpp23/ext-floating10.C  -std=gnu++14
 UNSUPPORTED: g++.dg/cpp23/ext-floating10.C  -std=gnu++17
 UNSUPPORTED: g++.dg/cpp23/ext-floating10.C  -std=gnu++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C  -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C  -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C  -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C  -std=gnu++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C  -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C  -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C  -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C  -std=gnu++20
+PASS: g++.dg/cpp23/ext-floating11.C  -std=gnu++23 (test for excess errors)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23  (test for warnings, line 55)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23  (test for warnings, line 59)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23  (test for warnings, line 63)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23  (test for warnings, line 81)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23  (test for warnings, line 85)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23  (test for warnings, line 110)
+PASS: g++.dg/cpp23/ext-floating2.C  -std=gnu++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/ext-floating3.C  -std=gnu++98
 UNSUPPORTED: g++.dg/cpp23/ext-floating3.C  -std=gnu++14
 UNSUPPORTED: g++.dg/cpp23/ext-floating3.C  -std=gnu++17
@@ -970,10 +946,7 @@ UNSUPPORTED: g++.dg/cpp23/ext-floating5.
 UNSUPPORTED: g++.dg/cpp23/ext-floating5.C  -std=gnu++14
 UNSUPPORTED: g++.dg/cpp23/ext-floating5.C  -std=gnu++17
 UNSUPPORTED: g++.dg/cpp23/ext-floating5.C  -std=gnu++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C  -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C  -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C  -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C  -std=gnu++20
+PASS: g++.dg/cpp23/ext-floating6.C  -std=gnu++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/ext-floating7.C  -std=gnu++98
 UNSUPPORTED: g++.dg/cpp23/ext-floating7.C  -std=gnu++14
 UNSUPPORTED: g++.dg/cpp23/ext-floating7.C  -std=gnu++17
@@ -996,10 +969,13 @@ PASS: g++.dg/cpp23/init-stmt1.C  -std=c+
 PASS: g++.dg/cpp23/init-stmt1.C  -std=c++20  (test for errors, line 20)
 PASS: g++.dg/cpp23/init-stmt1.C  -std=c++20  (test for errors, line 25)
 PASS: g++.dg/cpp23/init-stmt1.C  -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C  -std=c++20
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23  (test for errors, line 12)
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23  (test for errors, line 14)
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23  (test for errors, line 16)
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23  (test for errors, line 18)
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23  (test for errors, line 20)
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23  (test for errors, line 22)
+PASS: g++.dg/cpp23/init-stmt2.C  -std=c++23 (test for excess errors)
 PASS: g++.dg/cpp23/label1.C  -std=c++98  (test for errors, line 14)
 PASS: g++.dg/cpp23/label1.C  -std=c++98  (test for errors, line 20)
 PASS: g++.dg/cpp23/label1.C  -std=c++98  (test for errors, line 28)
@@ -1043,10 +1019,14 @@ PASS: g++.dg/cpp23/label1.C  -std=c++20
 PASS: g++.dg/cpp23/label1.C  -std=c++20  (test for errors, line 84)
 PASS: g++.dg/cpp23/label1.C  -std=c++20  (test for errors, line 88)
 PASS: g++.dg/cpp23/label1.C  -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/label2.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/label2.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/label2.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/label2.C  -std=c++20
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 12)
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 22)
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 29)
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 30)
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 37)
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 44)
+PASS: g++.dg/cpp23/label2.C  -std=c++23  (test for errors, line 52)
+PASS: g++.dg/cpp23/label2.C  -std=c++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/lambda-attr1.C  -std=c++98
 PASS: g++.dg/cpp23/lambda-attr1.C  -std=c++14  (test for warnings, line 7)
 PASS: g++.dg/cpp23/lambda-attr1.C  -std=c++14  (test for warnings, line 10)
@@ -1130,10 +1110,7 @@ PASS: g++.dg/cpp23/lambda-specifiers1.C
 PASS: g++.dg/cpp23/lambda-specifiers1.C  -std=gnu++20  (test for warnings, line 16)
 PASS: g++.dg/cpp23/lambda-specifiers1.C  -std=gnu++20  (test for warnings, line 17)
 PASS: g++.dg/cpp23/lambda-specifiers1.C  -std=gnu++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C  -std=c++20
+PASS: g++.dg/cpp23/lambda-specifiers2.C  -std=c++23 (test for excess errors)
 PASS: g++.dg/cpp23/lookup2.C  -std=c++98 (test for excess errors)
 PASS: g++.dg/cpp23/lookup2.C  -std=c++14 (test for excess errors)
 PASS: g++.dg/cpp23/lookup2.C  -std=c++17 (test for excess errors)
@@ -1688,10 +1665,7 @@ PASS: g++.dg/cpp23/normalize7.C  -std=gn
 PASS: g++.dg/cpp23/normalize7.C  -std=gnu++20  (test for errors, line 6)
 PASS: g++.dg/cpp23/normalize7.C  -std=gnu++20  (test for errors, line 8)
 PASS: g++.dg/cpp23/normalize7.C  -std=gnu++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C  -std=c++98
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C  -std=c++14
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C  -std=c++17
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C  -std=c++20
+PASS: g++.dg/cpp23/size_t-literals.C  -std=c++23 (test for excess errors)
 UNSUPPORTED: g++.dg/cpp23/static-operator-call1.C  -std=gnu++98
 PASS: g++.dg/cpp23/static-operator-call1.C  -std=gnu++14  (test for warnings, line 8)
 PASS: g++.dg/cpp23/static-operator-call1.C  -std=gnu++14  (test for warnings, line 30)
@@ -1980,8 +1954,8 @@ PASS: g++.dg/cpp23/whitespace-splice1.C
 
 		=== g++ Summary ===
 
-# of expected passes		1743
+# of expected passes		1829
 # of expected failures		2
-# of unsupported tests		224
+# of unsupported tests		112
 /usr/src/gcc/obj/gcc/xg++  version 13.0.0 20221018 (experimental) (GCC)
  

Comments

Jason Merrill Oct. 19, 2022, 3:47 p.m. UTC | #1
On 10/19/22 11:27, Jakub Jelinek wrote:
> Hi!
> 
> On Wed, Oct 19, 2022 at 09:10:48AM -0400, Jason Merrill wrote:
>>> The screw-up on my side with libstdc++ testing (tested normally rather
>>> than in C++23 mode) makes me wonder if we couldn't tweak the default
>>> testing.
>>> Dunno what libstdc++ testing normally does (just C++17?), make check-g++
>>> tests by default { 98, 14, 17, 20 } (and I regularly use
>>> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
>>> cover libstdc++ I guess).
>>> When adding tests for upcoming C++ version, one always has a dilemma
>>> whether to use explicit // { dg-options "-std=c++2b" }
>>> or -std=gnu++2b and similar, then the test works in all modes, but it might
>>> be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
>>> test so that when 23 is tested by default and say 26 or 29 appears too,
>>> we test it also in those modes, or just go with
>>> // { dg-do whatever { target c++23 } }
>>> which has the disadvantage that it is skipped when testing by default and
>>> one only tests it if he asks for the newer version.
>>>
>>> I wonder if we couldn't for the default testing (when one doesn't
>>> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
>>> improve things a little bit by automatically treat those
>>> // { dg-do whatever { target c++23 } }
>>> tests as // { dg-options "-std=c++2b" }.
>>
>> That would be great.
> 
> Here is a patch that implements it.
> On make check-g++ RUNTESTFLAGS=dg.exp=cpp23/*.C
> the vanilla vs. patched gcc difference is:
>                  === g++ Summary ===
> 
> # of expected passes            1743
> # of expected failures          2
> # of unsupported tests          224
> vs.
>                  === g++ Summary ===
> 
> # of expected passes            1829
> # of expected failures          2
> # of unsupported tests          112
> 
> (g++.sum details attached).

OK, thanks.

> 2022-10-19  Jakub Jelinek  <jakub@redhat.com>
> 
> 	* lib/g++-dg.exp (g++-dg-runtest): When using defaulted
> 	std_list, if test has { dg-do * { target c++23 } } directive,
> 	use { 23 } with which the test will run instead of { 98 14 17 20 }
> 	which would make it UNSUPPORTED in all cases.
> 
> --- gcc/testsuite/lib/g++-dg.exp.jj	2022-03-30 09:11:53.306103969 +0200
> +++ gcc/testsuite/lib/g++-dg.exp	2022-10-19 17:10:51.574866364 +0200
> @@ -53,7 +53,16 @@ proc g++-dg-runtest { testcases flags de
>   	    if { [llength $gpp_std_list] > 0 } {
>   		set std_list $gpp_std_list
>   	    } else {
> -		set std_list { 98 14 17 20 }
> +		# If the test requires a newer C++ version than which
> +		# is tested by default, use that C++ version for that
> +		# single test.  This should be updated or commented
> +		# out whenever the default std_list is updated or newer
> +		# C++ effective target is added.
> +		if [search_for $test "{ dg-do * { target c++23 } }"] {
> +		    set std_list { 23 }
> +		} else {
> +		    set std_list { 98 14 17 20 }
> +		}
>   	    }
>   	    set option_list { }
>   	    foreach x $std_list {
> 
> 
> 	Jakub
  

Patch

--- gcc/testsuite/lib/g++-dg.exp.jj	2022-03-30 09:11:53.306103969 +0200
+++ gcc/testsuite/lib/g++-dg.exp	2022-10-19 17:10:51.574866364 +0200
@@ -53,7 +53,16 @@  proc g++-dg-runtest { testcases flags de
 	    if { [llength $gpp_std_list] > 0 } {
 		set std_list $gpp_std_list
 	    } else {
-		set std_list { 98 14 17 20 }
+		# If the test requires a newer C++ version than which
+		# is tested by default, use that C++ version for that
+		# single test.  This should be updated or commented
+		# out whenever the default std_list is updated or newer
+		# C++ effective target is added.
+		if [search_for $test "{ dg-do * { target c++23 } }"] {
+		    set std_list { 23 }
+		} else {
+		    set std_list { 98 14 17 20 }
+		}
 	    }
 	    set option_list { }
 	    foreach x $std_list {