[v5,0/5] P1689R5 support

  Hi,

This patch series adds initial support for ISO C++'s [P1689R5][], a
format for describing C++ module requirements and provisions based on
the source code. This is required because compiling C++ with modules is
not embarrassingly parallel and need to be ordered to ensure that
`import some_module;` can be satisfied in time by making sure that any
TU with `export import some_module;` is compiled first.

[P1689R5]: https://isocpp.org/files/papers/P1689R5.html

I've also added patches to include imported module CMI files and the
module mapper file as dependencies of the compilation. I briefly looked
into adding dependencies on response files as well, but that appeared to
need some code contortions to have a `class mkdeps` available before
parsing the command line or to keep the information around until one was
made.

I'd like feedback on the approach taken here with respect to the
user-visible flags. I'll also note that header units are not supported
at this time because the current `-E` behavior with respect to `import
<some_header>;` is to search for an appropriate `.gcm` file which is not
something such a "scan" can support. A new mode will likely need to be
created (e.g., replacing `-E` with `-fc++-module-scanning` or something)
where headers are looked up "normally" and processed only as much as
scanning requires.

FWIW, Clang as taken an alternate approach with its `clang-scan-deps`
tool rather than using the compiler directly.

Thanks,

--Ben

---
v4 -> v5:

- add dependency tracking for imported modules to `-MF`
- add dependency tracking for static module mapper files given to
  `-fmodule-mapper=`

v3 -> v4:

- add missing spaces between function names and arguments

v2 -> v3:

- changelog entries moved to commit messages
- documentation updated/added in the UTF-8 routine editing

v1 -> v2:

- removal of the `deps_write(extra)` parameter to option-checking where
  ndeeded
- default parameter of `cpp_finish(fdeps_stream = NULL)`
- unification of libcpp UTF-8 validity functions from v1
- test cases for flag parsing states (depflags-*) and p1689 output
  (p1689-*)

Ben Boeckel (5):
  libcpp: reject codepoints above 0x10FFFF
  libcpp: add a function to determine UTF-8 validity of a C string
  p1689r5: initial support
  c++modules: report imported CMI files as dependencies
  c++modules: report module mapper files as a dependency

 gcc/c-family/c-opts.cc                        |  40 +++-
 gcc/c-family/c.opt                            |  12 +
 gcc/cp/mapper-client.cc                       |   4 +
 gcc/cp/mapper-client.h                        |   1 +
 gcc/cp/module.cc                              |  23 +-
 gcc/doc/invoke.texi                           |  15 ++
 gcc/testsuite/g++.dg/modules/depflags-f-MD.C  |   2 +
 gcc/testsuite/g++.dg/modules/depflags-f.C     |   1 +
 gcc/testsuite/g++.dg/modules/depflags-fi.C    |   3 +
 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-fj.C    |   4 +
 .../g++.dg/modules/depflags-fjo-MD.C          |   4 +
 gcc/testsuite/g++.dg/modules/depflags-fjo.C   |   5 +
 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-fo.C    |   4 +
 gcc/testsuite/g++.dg/modules/depflags-j-MD.C  |   2 +
 gcc/testsuite/g++.dg/modules/depflags-j.C     |   3 +
 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C |   3 +
 gcc/testsuite/g++.dg/modules/depflags-jo.C    |   4 +
 gcc/testsuite/g++.dg/modules/depflags-o-MD.C  |   2 +
 gcc/testsuite/g++.dg/modules/depflags-o.C     |   3 +
 gcc/testsuite/g++.dg/modules/modules.exp      |   1 +
 gcc/testsuite/g++.dg/modules/p1689-1.C        |  18 ++
 gcc/testsuite/g++.dg/modules/p1689-1.exp.json |  27 +++
 gcc/testsuite/g++.dg/modules/p1689-2.C        |  16 ++
 gcc/testsuite/g++.dg/modules/p1689-2.exp.json |  16 ++
 gcc/testsuite/g++.dg/modules/p1689-3.C        |  14 ++
 gcc/testsuite/g++.dg/modules/p1689-3.exp.json |  16 ++
 gcc/testsuite/g++.dg/modules/p1689-4.C        |  14 ++
 gcc/testsuite/g++.dg/modules/p1689-4.exp.json |  14 ++
 gcc/testsuite/g++.dg/modules/p1689-5.C        |  14 ++
 gcc/testsuite/g++.dg/modules/p1689-5.exp.json |  14 ++
 gcc/testsuite/g++.dg/modules/test-p1689.py    | 222 ++++++++++++++++++
 gcc/testsuite/lib/modules.exp                 |  71 ++++++
 libcpp/charset.cc                             |  28 ++-
 libcpp/include/cpplib.h                       |  12 +-
 libcpp/include/mkdeps.h                       |  17 +-
 libcpp/init.cc                                |  13 +-
 libcpp/internal.h                             |   2 +
 libcpp/mkdeps.cc                              | 149 +++++++++++-
 40 files changed, 789 insertions(+), 30 deletions(-)
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fi.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o-MD.C
 create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.exp.json
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.exp.json
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.exp.json
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.exp.json
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.C
 create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.exp.json
 create mode 100644 gcc/testsuite/g++.dg/modules/test-p1689.py
 create mode 100644 gcc/testsuite/lib/modules.exp


base-commit: 9d4c00cdaccc3decd07740e817387ce844ef3ac9
  

On Wed, Jan 25, 2023 at 16:06:31 -0500, Ben Boeckel wrote:
> This patch series adds initial support for ISO C++'s [P1689R5][], a
> format for describing C++ module requirements and provisions based on
> the source code. This is required because compiling C++ with modules is
> not embarrassingly parallel and need to be ordered to ensure that
> `import some_module;` can be satisfied in time by making sure that any
> TU with `export import some_module;` is compiled first.
> 
> [P1689R5]: https://isocpp.org/files/papers/P1689R5.html
> 
> I've also added patches to include imported module CMI files and the
> module mapper file as dependencies of the compilation. I briefly looked
> into adding dependencies on response files as well, but that appeared to
> need some code contortions to have a `class mkdeps` available before
> parsing the command line or to keep the information around until one was
> made.
> 
> I'd like feedback on the approach taken here with respect to the
> user-visible flags. I'll also note that header units are not supported
> at this time because the current `-E` behavior with respect to `import
> <some_header>;` is to search for an appropriate `.gcm` file which is not
> something such a "scan" can support. A new mode will likely need to be
> created (e.g., replacing `-E` with `-fc++-module-scanning` or something)
> where headers are looked up "normally" and processed only as much as
> scanning requires.
> 
> FWIW, Clang as taken an alternate approach with its `clang-scan-deps`
> tool rather than using the compiler directly.

Ping? It'd be nice to have this supported in at least GCC 14 (since it
missed 13).

Thanks,

--Ben
  

Hi Ben,

Am 25.01.23 um 22:06 schrieb Ben Boeckel via Gcc-patches:
> Hi,
> 
> This patch series adds initial support for ISO C++'s [P1689R5][], a
> format for describing C++ module requirements and provisions based on
> the source code. This is required because compiling C++ with modules is
> not embarrassingly parallel and need to be ordered to ensure that
> `import some_module;` can be satisfied in time by making sure that any
> TU with `export import some_module;` is compiled first.
> 
> [P1689R5]: https://isocpp.org/files/papers/P1689R5.html

while that paper mentions Fortran, the patch in its present version
does not seem to implement anything related to Fortran and does not
touch the gfortran frontend.  Or am I missing anything?  Otherwise,
could you give an example how it would be used with Fortran?

Thus I'd say that it is OK from the gfortran side.

Thanks,
Harald

> I've also added patches to include imported module CMI files and the
> module mapper file as dependencies of the compilation. I briefly looked
> into adding dependencies on response files as well, but that appeared to
> need some code contortions to have a `class mkdeps` available before
> parsing the command line or to keep the information around until one was
> made.
> 
> I'd like feedback on the approach taken here with respect to the
> user-visible flags. I'll also note that header units are not supported
> at this time because the current `-E` behavior with respect to `import
> <some_header>;` is to search for an appropriate `.gcm` file which is not
> something such a "scan" can support. A new mode will likely need to be
> created (e.g., replacing `-E` with `-fc++-module-scanning` or something)
> where headers are looked up "normally" and processed only as much as
> scanning requires.
> 
> FWIW, Clang as taken an alternate approach with its `clang-scan-deps`
> tool rather than using the compiler directly.
> 
> Thanks,
> 
> --Ben
> 
> ---
> v4 -> v5:
> 
> - add dependency tracking for imported modules to `-MF`
> - add dependency tracking for static module mapper files given to
>    `-fmodule-mapper=`
> 
> v3 -> v4:
> 
> - add missing spaces between function names and arguments
> 
> v2 -> v3:
> 
> - changelog entries moved to commit messages
> - documentation updated/added in the UTF-8 routine editing
> 
> v1 -> v2:
> 
> - removal of the `deps_write(extra)` parameter to option-checking where
>    ndeeded
> - default parameter of `cpp_finish(fdeps_stream = NULL)`
> - unification of libcpp UTF-8 validity functions from v1
> - test cases for flag parsing states (depflags-*) and p1689 output
>    (p1689-*)
> 
> Ben Boeckel (5):
>    libcpp: reject codepoints above 0x10FFFF
>    libcpp: add a function to determine UTF-8 validity of a C string
>    p1689r5: initial support
>    c++modules: report imported CMI files as dependencies
>    c++modules: report module mapper files as a dependency
> 
>   gcc/c-family/c-opts.cc                        |  40 +++-
>   gcc/c-family/c.opt                            |  12 +
>   gcc/cp/mapper-client.cc                       |   4 +
>   gcc/cp/mapper-client.h                        |   1 +
>   gcc/cp/module.cc                              |  23 +-
>   gcc/doc/invoke.texi                           |  15 ++
>   gcc/testsuite/g++.dg/modules/depflags-f-MD.C  |   2 +
>   gcc/testsuite/g++.dg/modules/depflags-f.C     |   1 +
>   gcc/testsuite/g++.dg/modules/depflags-fi.C    |   3 +
>   gcc/testsuite/g++.dg/modules/depflags-fj-MD.C |   3 +
>   gcc/testsuite/g++.dg/modules/depflags-fj.C    |   4 +
>   .../g++.dg/modules/depflags-fjo-MD.C          |   4 +
>   gcc/testsuite/g++.dg/modules/depflags-fjo.C   |   5 +
>   gcc/testsuite/g++.dg/modules/depflags-fo-MD.C |   3 +
>   gcc/testsuite/g++.dg/modules/depflags-fo.C    |   4 +
>   gcc/testsuite/g++.dg/modules/depflags-j-MD.C  |   2 +
>   gcc/testsuite/g++.dg/modules/depflags-j.C     |   3 +
>   gcc/testsuite/g++.dg/modules/depflags-jo-MD.C |   3 +
>   gcc/testsuite/g++.dg/modules/depflags-jo.C    |   4 +
>   gcc/testsuite/g++.dg/modules/depflags-o-MD.C  |   2 +
>   gcc/testsuite/g++.dg/modules/depflags-o.C     |   3 +
>   gcc/testsuite/g++.dg/modules/modules.exp      |   1 +
>   gcc/testsuite/g++.dg/modules/p1689-1.C        |  18 ++
>   gcc/testsuite/g++.dg/modules/p1689-1.exp.json |  27 +++
>   gcc/testsuite/g++.dg/modules/p1689-2.C        |  16 ++
>   gcc/testsuite/g++.dg/modules/p1689-2.exp.json |  16 ++
>   gcc/testsuite/g++.dg/modules/p1689-3.C        |  14 ++
>   gcc/testsuite/g++.dg/modules/p1689-3.exp.json |  16 ++
>   gcc/testsuite/g++.dg/modules/p1689-4.C        |  14 ++
>   gcc/testsuite/g++.dg/modules/p1689-4.exp.json |  14 ++
>   gcc/testsuite/g++.dg/modules/p1689-5.C        |  14 ++
>   gcc/testsuite/g++.dg/modules/p1689-5.exp.json |  14 ++
>   gcc/testsuite/g++.dg/modules/test-p1689.py    | 222 ++++++++++++++++++
>   gcc/testsuite/lib/modules.exp                 |  71 ++++++
>   libcpp/charset.cc                             |  28 ++-
>   libcpp/include/cpplib.h                       |  12 +-
>   libcpp/include/mkdeps.h                       |  17 +-
>   libcpp/init.cc                                |  13 +-
>   libcpp/internal.h                             |   2 +
>   libcpp/mkdeps.cc                              | 149 +++++++++++-
>   40 files changed, 789 insertions(+), 30 deletions(-)
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fi.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o-MD.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.exp.json
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.exp.json
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.exp.json
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.exp.json
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.exp.json
>   create mode 100644 gcc/testsuite/g++.dg/modules/test-p1689.py
>   create mode 100644 gcc/testsuite/lib/modules.exp
> 
> 
> base-commit: 9d4c00cdaccc3decd07740e817387ce844ef3ac9
  

On Thu, Feb 02, 2023 at 21:24:12 +0100, Harald Anlauf wrote:
> Am 25.01.23 um 22:06 schrieb Ben Boeckel via Gcc-patches:
> > Hi,
> >
> > This patch series adds initial support for ISO C++'s [P1689R5][], a
> > format for describing C++ module requirements and provisions based on
> > the source code. This is required because compiling C++ with modules is
> > not embarrassingly parallel and need to be ordered to ensure that
> > `import some_module;` can be satisfied in time by making sure that any
> > TU with `export import some_module;` is compiled first.
> >
> > [P1689R5]: https://isocpp.org/files/papers/P1689R5.html
> 
> while that paper mentions Fortran, the patch in its present version
> does not seem to implement anything related to Fortran and does not
> touch the gfortran frontend.  Or am I missing anything?  Otherwise,
> could you give an example how it would be used with Fortran?

Correct. Still trying to put the walls back together after modules
KoolAid Man'd their way into the build graph structure :) . Being able
to drop our Fortran parser (well, we'd have to drop support for Fortran
compilers that exist today…so maybe in 2075 or something) and rely on
compilers to tell us the information would be amazing though :) .

FWIW, the initial revision of the patchset did touch the gfortran
frontend, but the new parameter is now defaulted and therefore the
callsite doesn't need an update anymore. I still thought it worthwhile
to keep the Fortran side aware of what is going on in the space.

The link to Fortran comes up because the build graph problem is
isomorphic (Fortran supports exporting multiple modules from a single
TU, but it's not relevant at the graph level; it's the zero -> any case
that is hard), CMake "solved" it already, and C++ is going to have a
*lot* more "I want to consume $other_project's modules using my favorite
compiler/flags" than seems to happen in Fortran. If you're interested,
this is the paper showing how we do it:

    https://mathstuf.fedorapeople.org/fortran-modules/fortran-modules.html

> Thus I'd say that it is OK from the gfortran side.

Eventually we'll like to get gfortran supporting this type of scanning,
but…as above.

Thanks,

--Ben
  

On Wed, Jan 25, 2023 at 1:07 PM Ben Boeckel via Fortran
<fortran@gcc.gnu.org> wrote:
>
> Hi,
>
> This patch series adds initial support for ISO C++'s [P1689R5][], a
> format for describing C++ module requirements and provisions based on
> the source code. This is required because compiling C++ with modules is
> not embarrassingly parallel and need to be ordered to ensure that
> `import some_module;` can be satisfied in time by making sure that any
> TU with `export import some_module;` is compiled first.


I like how folks are complaining that GCC outputs POSIX makefile
syntax from GCC's dependency files which are supposed to be in POSIX
Makefile syntax.
It seems like rather the build tools are people like to use are not
understanding POSIX makefile syntax any more rather.
Also I am not a fan of json, it is too verbose for no use. Maybe it is
time to go back to standardizing a new POSIX makefile syntax rather
than changing C++ here.

Thanks,
Andrew Pinski

>
> [P1689R5]: https://isocpp.org/files/papers/P1689R5.html
>
> I've also added patches to include imported module CMI files and the
> module mapper file as dependencies of the compilation. I briefly looked
> into adding dependencies on response files as well, but that appeared to
> need some code contortions to have a `class mkdeps` available before
> parsing the command line or to keep the information around until one was
> made.
>
> I'd like feedback on the approach taken here with respect to the
> user-visible flags. I'll also note that header units are not supported
> at this time because the current `-E` behavior with respect to `import
> <some_header>;` is to search for an appropriate `.gcm` file which is not
> something such a "scan" can support. A new mode will likely need to be
> created (e.g., replacing `-E` with `-fc++-module-scanning` or something)
> where headers are looked up "normally" and processed only as much as
> scanning requires.
>
> FWIW, Clang as taken an alternate approach with its `clang-scan-deps`
> tool rather than using the compiler directly.
>
> Thanks,
>
> --Ben
>
> ---
> v4 -> v5:
>
> - add dependency tracking for imported modules to `-MF`
> - add dependency tracking for static module mapper files given to
>   `-fmodule-mapper=`
>
> v3 -> v4:
>
> - add missing spaces between function names and arguments
>
> v2 -> v3:
>
> - changelog entries moved to commit messages
> - documentation updated/added in the UTF-8 routine editing
>
> v1 -> v2:
>
> - removal of the `deps_write(extra)` parameter to option-checking where
>   ndeeded
> - default parameter of `cpp_finish(fdeps_stream = NULL)`
> - unification of libcpp UTF-8 validity functions from v1
> - test cases for flag parsing states (depflags-*) and p1689 output
>   (p1689-*)
>
> Ben Boeckel (5):
>   libcpp: reject codepoints above 0x10FFFF
>   libcpp: add a function to determine UTF-8 validity of a C string
>   p1689r5: initial support
>   c++modules: report imported CMI files as dependencies
>   c++modules: report module mapper files as a dependency
>
>  gcc/c-family/c-opts.cc                        |  40 +++-
>  gcc/c-family/c.opt                            |  12 +
>  gcc/cp/mapper-client.cc                       |   4 +
>  gcc/cp/mapper-client.h                        |   1 +
>  gcc/cp/module.cc                              |  23 +-
>  gcc/doc/invoke.texi                           |  15 ++
>  gcc/testsuite/g++.dg/modules/depflags-f-MD.C  |   2 +
>  gcc/testsuite/g++.dg/modules/depflags-f.C     |   1 +
>  gcc/testsuite/g++.dg/modules/depflags-fi.C    |   3 +
>  gcc/testsuite/g++.dg/modules/depflags-fj-MD.C |   3 +
>  gcc/testsuite/g++.dg/modules/depflags-fj.C    |   4 +
>  .../g++.dg/modules/depflags-fjo-MD.C          |   4 +
>  gcc/testsuite/g++.dg/modules/depflags-fjo.C   |   5 +
>  gcc/testsuite/g++.dg/modules/depflags-fo-MD.C |   3 +
>  gcc/testsuite/g++.dg/modules/depflags-fo.C    |   4 +
>  gcc/testsuite/g++.dg/modules/depflags-j-MD.C  |   2 +
>  gcc/testsuite/g++.dg/modules/depflags-j.C     |   3 +
>  gcc/testsuite/g++.dg/modules/depflags-jo-MD.C |   3 +
>  gcc/testsuite/g++.dg/modules/depflags-jo.C    |   4 +
>  gcc/testsuite/g++.dg/modules/depflags-o-MD.C  |   2 +
>  gcc/testsuite/g++.dg/modules/depflags-o.C     |   3 +
>  gcc/testsuite/g++.dg/modules/modules.exp      |   1 +
>  gcc/testsuite/g++.dg/modules/p1689-1.C        |  18 ++
>  gcc/testsuite/g++.dg/modules/p1689-1.exp.json |  27 +++
>  gcc/testsuite/g++.dg/modules/p1689-2.C        |  16 ++
>  gcc/testsuite/g++.dg/modules/p1689-2.exp.json |  16 ++
>  gcc/testsuite/g++.dg/modules/p1689-3.C        |  14 ++
>  gcc/testsuite/g++.dg/modules/p1689-3.exp.json |  16 ++
>  gcc/testsuite/g++.dg/modules/p1689-4.C        |  14 ++
>  gcc/testsuite/g++.dg/modules/p1689-4.exp.json |  14 ++
>  gcc/testsuite/g++.dg/modules/p1689-5.C        |  14 ++
>  gcc/testsuite/g++.dg/modules/p1689-5.exp.json |  14 ++
>  gcc/testsuite/g++.dg/modules/test-p1689.py    | 222 ++++++++++++++++++
>  gcc/testsuite/lib/modules.exp                 |  71 ++++++
>  libcpp/charset.cc                             |  28 ++-
>  libcpp/include/cpplib.h                       |  12 +-
>  libcpp/include/mkdeps.h                       |  17 +-
>  libcpp/init.cc                                |  13 +-
>  libcpp/internal.h                             |   2 +
>  libcpp/mkdeps.cc                              | 149 +++++++++++-
>  40 files changed, 789 insertions(+), 30 deletions(-)
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-f.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fi.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fj.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fjo.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-fo.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-j.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-jo.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o-MD.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/depflags-o.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-1.exp.json
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-2.exp.json
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-3.exp.json
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-4.exp.json
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/p1689-5.exp.json
>  create mode 100644 gcc/testsuite/g++.dg/modules/test-p1689.py
>  create mode 100644 gcc/testsuite/lib/modules.exp
>
>
> base-commit: 9d4c00cdaccc3decd07740e817387ce844ef3ac9
> --
> 2.39.0
>
  

On Fri, 3 Feb 2023, 04:09 Andrew Pinski via Gcc, <gcc@gcc.gnu.org> wrote:

> On Wed, Jan 25, 2023 at 1:07 PM Ben Boeckel via Fortran
> <fortran@gcc.gnu.org> wrote:
> >
> > Hi,
> >
> > This patch series adds initial support for ISO C++'s [P1689R5][], a
> > format for describing C++ module requirements and provisions based on
> > the source code. This is required because compiling C++ with modules is
> > not embarrassingly parallel and need to be ordered to ensure that
> > `import some_module;` can be satisfied in time by making sure that any
> > TU with `export import some_module;` is compiled first.
>
>
> I like how folks are complaining that GCC outputs POSIX makefile
> syntax from GCC's dependency files which are supposed to be in POSIX
> Makefile syntax.
> It seems like rather the build tools are people like to use are not
> understanding POSIX makefile syntax any more rather.
> Also I am not a fan of json, it is too verbose for no use. Maybe it is
> time to go back to standardizing a new POSIX makefile syntax rather
> than changing C++ here.
>


That would take a decade or more. It's too late for POSIX 202x and the pace
that POSIX agrees on makefile features is incredibly slow.
  

On Fri, 3 Feb 2023 at 08:58, Jonathan Wakely wrote:
>
>
>
> On Fri, 3 Feb 2023, 04:09 Andrew Pinski via Gcc, <gcc@gcc.gnu.org> wrote:
>>
>> On Wed, Jan 25, 2023 at 1:07 PM Ben Boeckel via Fortran
>> <fortran@gcc.gnu.org> wrote:
>> >
>> > Hi,
>> >
>> > This patch series adds initial support for ISO C++'s [P1689R5][], a
>> > format for describing C++ module requirements and provisions based on
>> > the source code. This is required because compiling C++ with modules is
>> > not embarrassingly parallel and need to be ordered to ensure that
>> > `import some_module;` can be satisfied in time by making sure that any
>> > TU with `export import some_module;` is compiled first.
>>
>>
>> I like how folks are complaining that GCC outputs POSIX makefile
>> syntax from GCC's dependency files which are supposed to be in POSIX
>> Makefile syntax.
>> It seems like rather the build tools are people like to use are not
>> understanding POSIX makefile syntax any more rather.
>> Also I am not a fan of json, it is too verbose for no use. Maybe it is
>> time to go back to standardizing a new POSIX makefile syntax rather
>> than changing C++ here.
>
>
>
> That would take a decade or more. It's too late for POSIX 202x and the pace that POSIX agrees on makefile features is incredibly slow.

Also, name+=value is *not* POSIX make syntax today, that's an
extension. That's why the tools don't always support it.
So I don't think it's true that GCC's dependency files are in POSIX syntax.

POSIX 202x does add support for it, but it will take some time for it
to be supported everywhere.
  

On Fri, Feb 03, 2023 at 09:10:21 +0000, Jonathan Wakely wrote:
> On Fri, 3 Feb 2023 at 08:58, Jonathan Wakely wrote:
> > On Fri, 3 Feb 2023, 04:09 Andrew Pinski via Gcc, <gcc@gcc.gnu.org> wrote:
> >> On Wed, Jan 25, 2023 at 1:07 PM Ben Boeckel via Fortran
> >> <fortran@gcc.gnu.org> wrote:
> >> > This patch series adds initial support for ISO C++'s [P1689R5][], a
> >> > format for describing C++ module requirements and provisions based on
> >> > the source code. This is required because compiling C++ with modules is
> >> > not embarrassingly parallel and need to be ordered to ensure that
> >> > `import some_module;` can be satisfied in time by making sure that any
> >> > TU with `export import some_module;` is compiled first.
> >>
> >> I like how folks are complaining that GCC outputs POSIX makefile
> >> syntax from GCC's dependency files which are supposed to be in POSIX
> >> Makefile syntax.
> >> It seems like rather the build tools are people like to use are not
> >> understanding POSIX makefile syntax any more rather.
> >> Also I am not a fan of json, it is too verbose for no use. Maybe it is
> >> time to go back to standardizing a new POSIX makefile syntax rather
> >> than changing C++ here.

I'm not complaining that dependency files are in POSIX (or even
POSIX-to-be) syntax. The information requires a bit more structure than
some variable assignments and I don't expect anything trying to read
them to start trying to understand `VAR_$(DEREF)=` and the behaviors of
`:=` versus `=` assignment to get this reliably.

> > That would take a decade or more. It's too late for POSIX 202x and
> > the pace that POSIX agrees on makefile features is incredibly slow.
> 
> Also, name+=value is *not* POSIX make syntax today, that's an
> extension. That's why the tools don't always support it.
> So I don't think it's true that GCC's dependency files are in POSIX syntax.
> 
> POSIX 202x does add support for it, but it will take some time for it
> to be supported everywhere.

Additionally, while the *syntax* might be supported, encoding all of
P1689 in it would require additional work (e.g., key/value variable
assignments or something). Batch scanning would also be…interesting.
Also note that the imported modules' location cannot be known before
scanning in general, so all you get are "logical names" that you need a
collator to link up with other scan results anyways. Tools such as
`make` and `ninja` cannot know, in general, how to do this linking
between arbitrary targets (e.g., there may be a debug and release build
of the same module in the graph and knowing which to use requires
higher-level info about the entire build graph; modules may also be
considered "private" and not accessible everywhere and therefore should
also not be hooked up across different target boundaries).

While the `CXX_MODULES +=` approach can work for simple cases (a
pseudo-implicit build), it is quite insufficient for the general case.

--Ben