[1/3] Factor out jobserver_active_p.

  Patch can bootstrap on x86_64-linux-gnu and survives regression tests.

Ready to be installed?
Thanks,
Martin

gcc/ChangeLog:

	* gcc.cc (driver::detect_jobserver): Remove and move to
	jobserver.h.
	* lto-wrapper.cc (jobserver_active_p): Likewise.
	(run_gcc): Likewise.
	* jobserver.h: New file.
---
 gcc/gcc.cc         | 36 +++-----------------
 gcc/jobserver.h    | 85 ++++++++++++++++++++++++++++++++++++++++++++++
 gcc/lto-wrapper.cc | 43 +++++------------------
 3 files changed, 97 insertions(+), 67 deletions(-)
 create mode 100644 gcc/jobserver.h
  

On Tue, Aug 9, 2022 at 2:03 PM Martin Liška <mliska@suse.cz> wrote:
>
> Patch can bootstrap on x86_64-linux-gnu and survives regression tests.
>
> Ready to be installed?
> Thanks,
> Martin
>
> gcc/ChangeLog:
>
>         * gcc.cc (driver::detect_jobserver): Remove and move to
>         jobserver.h.
>         * lto-wrapper.cc (jobserver_active_p): Likewise.
>         (run_gcc): Likewise.
>         * jobserver.h: New file.
> ---
>  gcc/gcc.cc         | 36 +++-----------------
>  gcc/jobserver.h    | 85 ++++++++++++++++++++++++++++++++++++++++++++++
>  gcc/lto-wrapper.cc | 43 +++++------------------
>  3 files changed, 97 insertions(+), 67 deletions(-)
>  create mode 100644 gcc/jobserver.h
>
> diff --git a/gcc/gcc.cc b/gcc/gcc.cc
> index 5cbb38560b2..69fbd293eaa 100644
> --- a/gcc/gcc.cc
> +++ b/gcc/gcc.cc
> @@ -43,6 +43,7 @@ compilation is specified by a string called a "spec".  */
>  #include "opts.h"
>  #include "filenames.h"
>  #include "spellcheck.h"
> +#include "jobserver.h"
>
>
>
> @@ -9178,38 +9179,9 @@ driver::final_actions () const
>  void
>  driver::detect_jobserver () const
>  {
> -  /* Detect jobserver and drop it if it's not working.  */
> -  const char *makeflags = env.get ("MAKEFLAGS");
> -  if (makeflags != NULL)
> -    {
> -      const char *needle = "--jobserver-auth=";
> -      const char *n = strstr (makeflags, needle);
> -      if (n != NULL)
> -       {
> -         int rfd = -1;
> -         int wfd = -1;
> -
> -         bool jobserver
> -           = (sscanf (n + strlen (needle), "%d,%d", &rfd, &wfd) == 2
> -              && rfd > 0
> -              && wfd > 0
> -              && is_valid_fd (rfd)
> -              && is_valid_fd (wfd));
> -
> -         /* Drop the jobserver if it's not working now.  */
> -         if (!jobserver)
> -           {
> -             unsigned offset = n - makeflags;
> -             char *dup = xstrdup (makeflags);
> -             dup[offset] = '\0';
> -
> -             const char *space = strchr (makeflags + offset, ' ');
> -             if (space != NULL)
> -               strcpy (dup + offset, space);
> -             xputenv (concat ("MAKEFLAGS=", dup, NULL));
> -           }
> -       }
> -    }
> +  jobserver_info jinfo;
> +  if (!jinfo.is_active && !jinfo.skipped_makeflags.empty ())
> +    xputenv (jinfo.skipped_makeflags.c_str ());
>  }
>
>  /* Determine what the exit code of the driver should be.  */
> diff --git a/gcc/jobserver.h b/gcc/jobserver.h
> new file mode 100644
> index 00000000000..85453dd3c79
> --- /dev/null
> +++ b/gcc/jobserver.h
> @@ -0,0 +1,85 @@
> +/* GNU make's jobserver related functionality.
> +   Copyright (C) 2022 Free Software Foundation, Inc.
> +
> +This file is part of GCC.
> +
> +GCC is free software; you can redistribute it and/or modify it under
> +the terms of the GNU General Public License as published by the Free
> +Software Foundation; either version 3, or (at your option) any later
> +version.
> +
> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> +for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GCC; see the file COPYING3.  If not see
> +<http://www.gnu.org/licenses/>.
> +
> +See dbgcnt.def for usage information.  */
> +
> +#ifndef GCC_JOBSERVER_H
> +#define GCC_JOBSERVER_H
> +
> +#include <string>

C++ standard library includes have to go through system.h (#define
INCLUDE_STRING).

Does the API really have to use std::string?

> +
> +using namespace std;
> +
> +struct jobserver_info
> +{
> +  /* Default constructor.  */
> +  jobserver_info ();
> +
> +  /* Error message if there is a problem.  */
> +  string error_msg = "";
> +  /* Skipped MAKEFLAGS where --jobserver-auth is skipped.  */
> +  string skipped_makeflags = "";
> +  /* File descriptor for reading used for jobserver communication.  */
> +  int rfd = -1;
> +  /* File descriptor for writing used for jobserver communication.  */
> +  int wfd = -1;
> +  /* Return true if jobserver is active.  */
> +  bool is_active = false;
> +};
> +
> +jobserver_info::jobserver_info ()
> +{
> +  /* Detect jobserver and drop it if it's not working.  */
> +  string js_needle = "--jobserver-auth=";
> +
> +  const char *envval = getenv ("MAKEFLAGS");
> +  if (envval != NULL)
> +    {
> +      string makeflags = envval;
> +      size_t n = makeflags.rfind (js_needle);
> +      if (n != string::npos)
> +       {
> +         if (sscanf (makeflags.c_str () + n + js_needle.size (),
> +                     "%d,%d", &rfd, &wfd) == 2
> +             && rfd > 0
> +             && wfd > 0
> +             && is_valid_fd (rfd)
> +             && is_valid_fd (wfd))
> +           is_active = true;
> +         else
> +           {
> +             string dup = makeflags.substr (0, n);
> +             size_t pos = makeflags.find (' ', n);
> +             if (pos != string::npos)
> +               dup += makeflags.substr (pos);
> +             skipped_makeflags = "MAKEFLAGS=" + dup;
> +             error_msg
> +               = "cannot access %<" + js_needle + "%> file descriptors";
> +           }
> +       }
> +      error_msg = "%<" + js_needle + "%> is not present in %<MAKEFLAGS%>";
> +    }
> +  else
> +    error_msg = "%<MAKEFLAGS%> environment variable is unset";
> +
> +  if (!error_msg.empty ())
> +    error_msg = "jobserver is not available: " + error_msg;
> +}
> +
> +#endif /* GCC_JOBSERVER_H */
> diff --git a/gcc/lto-wrapper.cc b/gcc/lto-wrapper.cc
> index 795ab74555c..9279958d055 100644
> --- a/gcc/lto-wrapper.cc
> +++ b/gcc/lto-wrapper.cc
> @@ -49,6 +49,8 @@ along with GCC; see the file COPYING3.  If not see
>  #include "lto-section-names.h"
>  #include "collect-utils.h"
>  #include "opts-diagnostic.h"
> +#include "opt-suggestions.h"
> +#include "jobserver.h"
>
>  /* Environment variable, used for passing the names of offload targets from GCC
>     driver to lto-wrapper.  */
> @@ -1336,35 +1338,6 @@ init_num_threads (void)
>  #endif
>  }
>
> -/* Test and return reason why a jobserver cannot be detected.  */
> -
> -static const char *
> -jobserver_active_p (void)
> -{
> -  #define JS_PREFIX "jobserver is not available: "
> -  #define JS_NEEDLE "--jobserver-auth="
> -
> -  const char *makeflags = getenv ("MAKEFLAGS");
> -  if (makeflags == NULL)
> -    return JS_PREFIX "%<MAKEFLAGS%> environment variable is unset";
> -
> -  const char *n = strstr (makeflags, JS_NEEDLE);
> -  if (n == NULL)
> -    return JS_PREFIX "%<" JS_NEEDLE "%> is not present in %<MAKEFLAGS%>";
> -
> -  int rfd = -1;
> -  int wfd = -1;
> -
> -  if (sscanf (n + strlen (JS_NEEDLE), "%d,%d", &rfd, &wfd) == 2
> -      && rfd > 0
> -      && wfd > 0
> -      && is_valid_fd (rfd)
> -      && is_valid_fd (wfd))
> -    return NULL;
> -  else
> -    return JS_PREFIX "cannot access %<" JS_NEEDLE "%> file descriptors";
> -}
> -
>  /* Print link to -flto documentation with a hint message.  */
>
>  void
> @@ -1422,7 +1395,6 @@ run_gcc (unsigned argc, char *argv[])
>    bool jobserver_requested = false;
>    int auto_parallel = 0;
>    bool no_partition = false;
> -  const char *jobserver_error = NULL;
>    bool fdecoded_options_first = true;
>    vec<cl_decoded_option> fdecoded_options;
>    fdecoded_options.create (16);
> @@ -1653,14 +1625,14 @@ run_gcc (unsigned argc, char *argv[])
>      }
>    else
>      {
> -      jobserver_error = jobserver_active_p ();
> -      if (jobserver && jobserver_error != NULL)
> +      jobserver_info jinfo;
> +      if (jobserver && !jinfo.is_active)
>         {
>           /* Fall back to auto parallelism.  */
>           jobserver = 0;
>           auto_parallel = 1;
>         }
> -      else if (!jobserver && jobserver_error == NULL)
> +      else if (!jobserver && jinfo.is_active)
>         {
>           parallel = 1;
>           jobserver = 1;
> @@ -1971,9 +1943,10 @@ cont:
>
>        if (nr > 1)
>         {
> -         if (jobserver_requested && jobserver_error != NULL)
> +         jobserver_info jinfo;
> +         if (jobserver_requested && !jinfo.is_active)
>             {
> -             warning (0, jobserver_error);
> +             warning (0, jinfo.error_msg.c_str ());
>               print_lto_docs_link ();
>             }
>           else if (parallel == 0)
> --
> 2.37.1
>
>
  

On 8/10/22 08:56, Richard Biener wrote:
> C++ standard library includes have to go through system.h (#define
> INCLUDE_STRING).

Oh, yeah. That means I need to rely on the flat header files :/

> 
> Does the API really have to use std::string?

I would like to. My main motivation is std::string::rfind function that
has no C equivalent (would be rstrstr).

Martin
  

On Wed, Aug 10, 2022 at 9:17 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 8/10/22 08:56, Richard Biener wrote:
> > C++ standard library includes have to go through system.h (#define
> > INCLUDE_STRING).
>
> Oh, yeah. That means I need to rely on the flat header files :/
>
> >
> > Does the API really have to use std::string?
>
> I would like to. My main motivation is std::string::rfind function that
> has no C equivalent (would be rstrstr).

The old code happily uses strstr though, not worrying about
finding the last instance of --jobserver-auth?

Anyway, I'm not going to insist - I just noticed the actual
users use .c_str on the error message and adjusting the
environment for a not working jobserver is done
inconsistently.  Since I'm coming from C I was more
expecting sth like

 bool jobserver_active = probe_jobserver (true /* diagnose */);

rather than pulling in a class instance from an all-inline
implementation.  But hey ;)


>
> Martin
  

On 8/10/22 09:47, Richard Biener wrote:
> On Wed, Aug 10, 2022 at 9:17 AM Martin Liška <mliska@suse.cz> wrote:
>>
>> On 8/10/22 08:56, Richard Biener wrote:
>>> C++ standard library includes have to go through system.h (#define
>>> INCLUDE_STRING).
>>
>> Oh, yeah. That means I need to rely on the flat header files :/
>>
>>>
>>> Does the API really have to use std::string?
>>
>> I would like to. My main motivation is std::string::rfind function that
>> has no C equivalent (would be rstrstr).
> 
> The old code happily uses strstr though, not worrying about
> finding the last instance of --jobserver-auth?

Yes, sorry, I forgot to mention that, it's something I was notified by the GNU make
developer here: https://savannah.gnu.org/bugs/index.php?57242#comment13

> 
> Anyway, I'm not going to insist - I just noticed the actual
> users use .c_str on the error message and adjusting the
> environment for a not working jobserver is done
> inconsistently.  Since I'm coming from C I was more
> expecting sth like
> 
>  bool jobserver_active = probe_jobserver (true /* diagnose */);

Well, the main problem is that I need to "extra" a bunch of information
when parsing the env variable (and each consumer needs something else,
so that's why the jobserver_info members). It was very ugly having all
these return values being given as params (of pointer type).

Martin

> 
> rather than pulling in a class instance from an all-inline
> implementation.  But hey ;)
> 
> 
>>
>> Martin
  

On Wed, Aug 10, 2022 at 11:30 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 8/10/22 09:47, Richard Biener wrote:
> > On Wed, Aug 10, 2022 at 9:17 AM Martin Liška <mliska@suse.cz> wrote:
> >>
> >> On 8/10/22 08:56, Richard Biener wrote:
> >>> C++ standard library includes have to go through system.h (#define
> >>> INCLUDE_STRING).
> >>
> >> Oh, yeah. That means I need to rely on the flat header files :/
> >>
> >>>
> >>> Does the API really have to use std::string?
> >>
> >> I would like to. My main motivation is std::string::rfind function that
> >> has no C equivalent (would be rstrstr).
> >
> > The old code happily uses strstr though, not worrying about
> > finding the last instance of --jobserver-auth?
>
> Yes, sorry, I forgot to mention that, it's something I was notified by the GNU make
> developer here: https://savannah.gnu.org/bugs/index.php?57242#comment13
>
> >
> > Anyway, I'm not going to insist - I just noticed the actual
> > users use .c_str on the error message and adjusting the
> > environment for a not working jobserver is done
> > inconsistently.  Since I'm coming from C I was more
> > expecting sth like
> >
> >  bool jobserver_active = probe_jobserver (true /* diagnose */);
>
> Well, the main problem is that I need to "extra" a bunch of information
> when parsing the env variable (and each consumer needs something else,
> so that's why the jobserver_info members). It was very ugly having all
> these return values being given as params (of pointer type).

Yeah, fair enough.

> Martin
>
> >
> > rather than pulling in a class instance from an all-inline
> > implementation.  But hey ;)
> >
> >
> >>
> >> Martin
>
  

On 8/10/22 12:47, Richard Biener wrote:
> Yeah, fair enough.

I'm going to install the v3 where I renamed jobserver.h
and moved the ctor implementation to opts-common.cc.

Cheers,
Martin
  

On Wed, Aug 10, 2022 at 1:11 PM Martin Liška <mliska@suse.cz> wrote:
>
> On 8/10/22 12:47, Richard Biener wrote:
> > Yeah, fair enough.
>
> I'm going to install the v3 where I renamed jobserver.h
> and moved the ctor implementation to opts-common.cc.

For the record, the v3 series is OK

> Cheers,
> Martin