[PING^5] nvptx: Allow '--with-arch' to override the default '-misa' (was: nvptx multilib setup)
Commit Message
Hi Tom!
Ping.
Grüße
Thomas
On 2022-07-27T17:48:58+0200, I wrote:
> Hi Tom!
>
> Ping.
>
>
> Grüße
> Thomas
>
>
> On 2022-07-20T14:46:03+0200, I wrote:
>> Hi Tom!
>>
>> Ping.
>>
>>
>> Grüße
>> Thomas
>>
>>
>> On 2022-07-13T10:42:44+0200, I wrote:
>>> Hi Tom!
>>>
>>> Ping.
>>>
>>>
>>> Grüße
>>> Thomas
>>>
>>>
>>> On 2022-07-05T16:59:23+0200, I wrote:
>>>> Hi Tom!
>>>>
>>>> Ping.
>>>>
>>>>
>>>> Grüße
>>>> Thomas
>>>>
>>>>
>>>> On 2022-06-15T23:18:10+0200, I wrote:
>>>>> Hi Tom!
>>>>>
>>>>> On 2022-05-13T16:20:14+0200, I wrote:
>>>>>> On 2022-02-04T13:09:29+0100, Tom de Vries via Gcc <gcc@gcc.gnu.org> wrote:
>>>>>>> On 2/4/22 08:21, Thomas Schwinge wrote:
>>>>>>>> On 2022-02-03T13:35:55+0000, "vries at gcc dot gnu.org via Gcc-bugs" <gcc-bugs@gcc.gnu.org> wrote:
>>>>>>>>> I've tested this using (recommended) driver 470.94 on boards:
>>>>>>
>>>>>>>>> while iterating over dimensions { -mptx=3.1 , -mptx=6.3 } x { GOMP_NVPTX_JIT=-O0, <default> }.
>>>>>>>>
>>>>>>>> Do you use separate (nvptx-none offload target only?) builds for
>>>>>>>> different '-mptx' variants (likewise: '-misa'), or have you hacked up the
>>>>>>>> multilib configuration?
>>>>>>>
>>>>>>> Neither, I'm using --target_board=unix/foffload= for that.
>>>>>>
>>>>>> ACK, I see. So these flags then only affect GCC/nvptx code generation
>>>>>> for the actual user code (here: GCC libgomp test cases), but for the
>>>>>> GCC/nvptx target libraries (such as: libc, libm, libgfortran, libgomp --
>>>>>> the latter especially relevant for OpenMP), it uses PTX code from one of
>>>>>> the two "pre-compiled" GCC/nvptx multilibs: default or '-mptx=3.1'.
>>>>>>
>>>>>> Meaning, one can't just use such a flag for "completely building code"
>>>>>> for a specific configuration. Random example,
>>>>>> '-foffload-options=nvptx-none=-march=sm_75': as GCC/nvptx target
>>>>>> libraries aren't being built for '-march=sm_75' multilib,
>>>>>> '-foffload-options=nvptx-none=-march=sm_75' uses the default multilib,
>>>>>> which isn't '-march=sm_75'.
>>>>>>
>>>>>>
>>>>>>> ('gcc/config/nvptx/t-nvptx:MULTILIB_OPTIONS'
>>>>>>>> etc., I suppose?) Should we add a few representative configurations to
>>>>>>>> be built by default? And/or, should we have a way to 'configure' per
>>>>>>>> user needs (I suppose: '--with-multilib-list=[...]', as supported for a
>>>>>>>> few other targets?)? (I see there's also a new
>>>>>>>> '--with-multilib-generator=[...]', haven't looked in detail.) No matter
>>>>>>>> which way: again, combinatorial explosion is a problem, of course...
>>>>>>>
>>>>>>> As far as I know, the gcc build doesn't finish when switching default to
>>>>>>> higher than sm_35, so there's little point to go to a multilib setup at
>>>>>>> this point. But once we fix that, we could reconsider, otherwise,
>>>>>>> things are likely to regress again.
>>>>>>
>>>>>> As far as I remember, several issues have been fixed. Still waiting for
>>>>>> Roger's "middle-end: Support ABIs that pass FP values as wider integers"
>>>>>> or something similar, but that PR104489 issue is being worked around by
>>>>>> "Limit HFmode support to mexperimental", if I got that right.
>>>>>>
>>>>>> Now I'm not suggesting we should now enable all or any random GCC/nvptx
>>>>>> multilibs, to get all these variants of GCC/nvptx target libraries built;
>>>>>> especially also given that GCC/nvptx code generation currently doesn't
>>>>>> make too much use of the new capabilities.
>>>>>>
>>>>>> However, we do have a specific request that a customer would like to be
>>>>>> able to change at GCC 'configure' time the GCC/nvptx default multilib
>>>>>> (including that being used for building corresponding GCC/nvptx target
>>>>>> libraries).
>>>>>>
>>>>>> Per 'gcc/doc/install.texi', I do see that some GCC targets allow for
>>>>>> GCC 'configure'-time '--with-multilib-list=[...]', or
>>>>>> '--with-multilib-generator=[...]', and I suppose we could be doing
>>>>>> something similar? But before starting implementing, I'd like your
>>>>>> input, as you'll be the one to approve in the end. And/or, maybe you've
>>>>>> already made up your own ideas about that?
>>>>>
>>>>> So, instead of "random GCC/nvptx multilib configuration" (last
>>>>> paragraph), I've come up with a way to implement our customer's request
>>>>> (second last paragraph): 'configure' GCC/nvptx '--with-arch=sm_70'.
>>>>>
>>>>> I think I've implemented this in a way so that "random GCC/nvptx multilib
>>>>> configuration" may eventually be implemented on top of that. For easy
>>>>> review/testing I've split my changes into three commits, see attached
>>>>> "nvptx: Make default '-misa=sm_30' explicit",
>>>>> "nvptx: Introduce dummy multilib option for default '-misa=sm_30'",
>>>>> "nvptx: Allow '--with-arch' to override the default '-misa'".
>>>>>
>>>>> To the best of my knowledge, the first two patches do not change any
>>>>> user-visible behavior (I generally 'diff'ed target libraries, and
>>>>> compared a good number of 'gcc -print-multi-directory [flags]'), and
>>>>> likewise with the third patch, given implicit (default) or explicit
>>>>> '--with-arch=sm_30', and that with '--with-arch=sm_70', for example, the
>>>>> '-misa=sm_70' multilib variants are used for implicit (default) or
>>>>> explicit '-misa=sm_70' or higher, and the '-misa=sm_30' multilib variants
>>>>> are used for explicit lower '-misa'.
>>>>>
>>>>> What do you think, OK to push to master branch?
>>>>>
>>>>>
>>>>> Grüße
>>>>> Thomas
-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955
Comments
On 8/6/22 21:20, Thomas Schwinge wrote:
> Hi Tom!
>
Hi Thomas,
thanks for doing this.
Series approved.
As I mentioned, I'm not completely happy with the multilib name, but I
don't think it makes sense to post-pone approval for this.
Thanks,
- Tom
> Ping.
>
>
> Grüße
> Thomas
>
>
> On 2022-07-27T17:48:58+0200, I wrote:
>> Hi Tom!
>>
>> Ping.
>>
>>
>> Grüße
>> Thomas
>>
>>
>> On 2022-07-20T14:46:03+0200, I wrote:
>>> Hi Tom!
>>>
>>> Ping.
>>>
>>>
>>> Grüße
>>> Thomas
>>>
>>>
>>> On 2022-07-13T10:42:44+0200, I wrote:
>>>> Hi Tom!
>>>>
>>>> Ping.
>>>>
>>>>
>>>> Grüße
>>>> Thomas
>>>>
>>>>
>>>> On 2022-07-05T16:59:23+0200, I wrote:
>>>>> Hi Tom!
>>>>>
>>>>> Ping.
>>>>>
>>>>>
>>>>> Grüße
>>>>> Thomas
>>>>>
>>>>>
>>>>> On 2022-06-15T23:18:10+0200, I wrote:
>>>>>> Hi Tom!
>>>>>>
>>>>>> On 2022-05-13T16:20:14+0200, I wrote:
>>>>>>> On 2022-02-04T13:09:29+0100, Tom de Vries via Gcc <gcc@gcc.gnu.org> wrote:
>>>>>>>> On 2/4/22 08:21, Thomas Schwinge wrote:
>>>>>>>>> On 2022-02-03T13:35:55+0000, "vries at gcc dot gnu.org via Gcc-bugs" <gcc-bugs@gcc.gnu.org> wrote:
>>>>>>>>>> I've tested this using (recommended) driver 470.94 on boards:
>>>>>>>
>>>>>>>>>> while iterating over dimensions { -mptx=3.1 , -mptx=6.3 } x { GOMP_NVPTX_JIT=-O0, <default> }.
>>>>>>>>>
>>>>>>>>> Do you use separate (nvptx-none offload target only?) builds for
>>>>>>>>> different '-mptx' variants (likewise: '-misa'), or have you hacked up the
>>>>>>>>> multilib configuration?
>>>>>>>>
>>>>>>>> Neither, I'm using --target_board=unix/foffload= for that.
>>>>>>>
>>>>>>> ACK, I see. So these flags then only affect GCC/nvptx code generation
>>>>>>> for the actual user code (here: GCC libgomp test cases), but for the
>>>>>>> GCC/nvptx target libraries (such as: libc, libm, libgfortran, libgomp --
>>>>>>> the latter especially relevant for OpenMP), it uses PTX code from one of
>>>>>>> the two "pre-compiled" GCC/nvptx multilibs: default or '-mptx=3.1'.
>>>>>>>
>>>>>>> Meaning, one can't just use such a flag for "completely building code"
>>>>>>> for a specific configuration. Random example,
>>>>>>> '-foffload-options=nvptx-none=-march=sm_75': as GCC/nvptx target
>>>>>>> libraries aren't being built for '-march=sm_75' multilib,
>>>>>>> '-foffload-options=nvptx-none=-march=sm_75' uses the default multilib,
>>>>>>> which isn't '-march=sm_75'.
>>>>>>>
>>>>>>>
>>>>>>>> ('gcc/config/nvptx/t-nvptx:MULTILIB_OPTIONS'
>>>>>>>>> etc., I suppose?) Should we add a few representative configurations to
>>>>>>>>> be built by default? And/or, should we have a way to 'configure' per
>>>>>>>>> user needs (I suppose: '--with-multilib-list=[...]', as supported for a
>>>>>>>>> few other targets?)? (I see there's also a new
>>>>>>>>> '--with-multilib-generator=[...]', haven't looked in detail.) No matter
>>>>>>>>> which way: again, combinatorial explosion is a problem, of course...
>>>>>>>>
>>>>>>>> As far as I know, the gcc build doesn't finish when switching default to
>>>>>>>> higher than sm_35, so there's little point to go to a multilib setup at
>>>>>>>> this point. But once we fix that, we could reconsider, otherwise,
>>>>>>>> things are likely to regress again.
>>>>>>>
>>>>>>> As far as I remember, several issues have been fixed. Still waiting for
>>>>>>> Roger's "middle-end: Support ABIs that pass FP values as wider integers"
>>>>>>> or something similar, but that PR104489 issue is being worked around by
>>>>>>> "Limit HFmode support to mexperimental", if I got that right.
>>>>>>>
>>>>>>> Now I'm not suggesting we should now enable all or any random GCC/nvptx
>>>>>>> multilibs, to get all these variants of GCC/nvptx target libraries built;
>>>>>>> especially also given that GCC/nvptx code generation currently doesn't
>>>>>>> make too much use of the new capabilities.
>>>>>>>
>>>>>>> However, we do have a specific request that a customer would like to be
>>>>>>> able to change at GCC 'configure' time the GCC/nvptx default multilib
>>>>>>> (including that being used for building corresponding GCC/nvptx target
>>>>>>> libraries).
>>>>>>>
>>>>>>> Per 'gcc/doc/install.texi', I do see that some GCC targets allow for
>>>>>>> GCC 'configure'-time '--with-multilib-list=[...]', or
>>>>>>> '--with-multilib-generator=[...]', and I suppose we could be doing
>>>>>>> something similar? But before starting implementing, I'd like your
>>>>>>> input, as you'll be the one to approve in the end. And/or, maybe you've
>>>>>>> already made up your own ideas about that?
>>>>>>
>>>>>> So, instead of "random GCC/nvptx multilib configuration" (last
>>>>>> paragraph), I've come up with a way to implement our customer's request
>>>>>> (second last paragraph): 'configure' GCC/nvptx '--with-arch=sm_70'.
>>>>>>
>>>>>> I think I've implemented this in a way so that "random GCC/nvptx multilib
>>>>>> configuration" may eventually be implemented on top of that. For easy
>>>>>> review/testing I've split my changes into three commits, see attached
>>>>>> "nvptx: Make default '-misa=sm_30' explicit",
>>>>>> "nvptx: Introduce dummy multilib option for default '-misa=sm_30'",
>>>>>> "nvptx: Allow '--with-arch' to override the default '-misa'".
>>>>>>
>>>>>> To the best of my knowledge, the first two patches do not change any
>>>>>> user-visible behavior (I generally 'diff'ed target libraries, and
>>>>>> compared a good number of 'gcc -print-multi-directory [flags]'), and
>>>>>> likewise with the third patch, given implicit (default) or explicit
>>>>>> '--with-arch=sm_30', and that with '--with-arch=sm_70', for example, the
>>>>>> '-misa=sm_70' multilib variants are used for implicit (default) or
>>>>>> explicit '-misa=sm_70' or higher, and the '-misa=sm_30' multilib variants
>>>>>> are used for explicit lower '-misa'.
>>>>>>
>>>>>> What do you think, OK to push to master branch?
>>>>>>
>>>>>>
>>>>>> Grüße
>>>>>> Thomas
>
>
> -----------------
> Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955
From 8343b7d0ed96cf5d45c12b36ea34a156229b8745 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Sat, 11 Jun 2022 19:37:10 +0200
Subject: [PATCH 3/3] nvptx: Allow '--with-arch' to override the default
'-misa'
---
gcc/config.gcc | 5 ++
gcc/config/nvptx/gen-multilib-matches.sh | 60 ++++++++++++++++++++++++
gcc/config/nvptx/t-nvptx | 21 +++++++--
gcc/doc/install.texi | 9 ++++
gcc/doc/invoke.texi | 4 +-
5 files changed, 93 insertions(+), 6 deletions(-)
create mode 100755 gcc/config/nvptx/gen-multilib-matches.sh
@@ -5442,10 +5442,15 @@ case "${target}" in
nvptx-*)
supported_defaults=arch
TM_MULTILIB_CONFIG=$with_arch
+ #TODO 'sm_[...]' list per 'nvptx-sm.def'.
case $with_arch in
sm_30 )
# OK; default.
;;
+ sm_35 | sm_53 | sm_70 | sm_75 | sm_80 )
+ # OK, but we'd like 'sm_30', too.
+ TM_MULTILIB_CONFIG="$TM_MULTILIB_CONFIG sm_30"
+ ;;
* )
echo "Unknown arch used in --with-arch=$with_arch" 1>&2
exit 1
new file mode 100755
@@ -0,0 +1,60 @@
+#!/bin/sh
+
+# Print nvptx 'MULTILIB_MATCHES'
+
+# Copyright (C) 2022 Free Software Foundation, Inc.
+#
+# This file is part of GCC.
+#
+# GCC is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC; see the file COPYING3. If not see
+# <http://www.gnu.org/licenses/>.
+
+set -e
+
+nvptx_sm_def="$1/nvptx-sm.def"
+multilib_options_isa_default=$2
+multilib_options_isa_list=$3
+
+sms=$(grep ^NVPTX_SM $nvptx_sm_def | sed 's/.*(//;s/,.*//')
+
+# Every variant in 'sms' has to either be remapped to the default variant
+# ('.', which is always built), or does get built as non-default variant
+# ('misa=sm_SM'; thus not remapped), or has to be remapped to the "next lower"
+# variant that does get built.
+
+# The "lowest" variant has to be built.
+sm_next_lower=INVALID
+
+for sm in $sms; do
+ if [ x"sm_$sm" = x"$multilib_options_isa_default" ]; then
+ sm_map=.
+ elif expr " $multilib_options_isa_list " : ".* sm_$sm " > /dev/null; then
+ sm_map=
+ else
+ sm_map=$sm_next_lower
+ fi
+
+ if [ x"$sm_map" = x ]; then
+ sm_next_lower=$sm
+ else
+ # Output format as required for 'MULTILIB_MATCHES'.
+ if [ x"$sm_map" = x. ]; then
+ echo ".=misa?sm_$sm"
+ else
+ echo "misa?sm_$sm_map=misa?sm_$sm"
+ fi
+
+ sm_next_lower=$sm_map
+ fi
+done
@@ -42,11 +42,22 @@ MULTILIB_OPTIONS += mgomp
multilib_options_isa_list := $(TM_MULTILIB_CONFIG)
multilib_options_isa_default := $(word 1,$(multilib_options_isa_list))
-# Add the default '-misa' as a multilib option:
-MULTILIB_OPTIONS += misa=$(multilib_options_isa_default)
-# ..., but don't handle it specially (remap to default):
-MULTILIB_MATCHES += .=misa?$(multilib_options_isa_default)
-# ..., and don't actually build it:
+multilib_options_misa_list := $(addprefix misa=,$(multilib_options_isa_list))
+# Add the requested '-misa' variants as a multilib option ('misa=VAR1/misa=VAR2/misa=VAR3' etc.):
+empty :=
+space := $(empty) $(empty)
+MULTILIB_OPTIONS += $(subst $(space),/,$(multilib_options_misa_list))
+# ..., and remap '-misa' variants as appropriate:
+multilib_matches := $(shell $(srcdir)/config/nvptx/gen-multilib-matches.sh $(srcdir)/config/nvptx $(multilib_options_isa_default) "$(multilib_options_isa_list)")
+MULTILIB_MATCHES += $(multilib_matches)
+# ..., and don't actually build what's the default '-misa':
MULTILIB_EXCEPTIONS += *misa=$(multilib_options_isa_default)*
MULTILIB_OPTIONS += mptx=3.1
+# Filter out invalid '-misa'/'-mptx=3.1' combinations; per 'nvptx-sm.def',
+# 'nvptx.opt:ptx_version', 'nvptx.cc:first_ptx_version_supporting_sm'
+# (that is, '-mptx=3.1' only for sm_30, sm_35 variants):
+MULTILIB_EXCEPTIONS += $(foreach misa,$(filter-out %=sm_30 %=sm_35,$(multilib_options_misa_list)),*$(misa)/mptx=3.1)
+# ..., and special care has to be taken if '-mptx=3.1' is invalid for the
+# default variant:
+MULTILIB_EXCEPTIONS += $(if $(filter-out sm_30 sm_35,$(multilib_options_isa_default)),mgomp/mptx=3.1 mptx=3.1)
@@ -4617,6 +4617,15 @@ the GCC sources.
Use the @option{--disable-sjlj-exceptions} and
@option{--enable-newlib-io-long-long} options when configuring.
+The @option{--with-arch} option may be specified to override the
+default value for the @option{-march} option, and to also build
+corresponding target libraries.
+The default is @option{--with-arch=sm_30}.
+
+For example, if @option{--with-arch=sm_70} is specified,
+@option{-march=sm_30} and @option{-march=sm_70} target libraries are
+built, and code generation defaults to @option{-march=sm_70}.
+
@html
<hr />
@end html
@@ -27549,7 +27549,9 @@ supported.
Generate code for the specified PTX ISA target architecture
(e.g.@: @samp{sm_35}). Valid architecture strings are @samp{sm_30},
@samp{sm_35}, @samp{sm_53}, @samp{sm_70}, @samp{sm_75} and
-@samp{sm_80}. The default target architecture is sm_30.
+@samp{sm_80}.
+The default depends on how the compiler has been configured, see
+@option{--with-arch}.
This option sets the value of the preprocessor macro
@code{__PTX_SM__}; for instance, for @samp{sm_35}, it has the value
--
2.25.1