c++ modules: lazy loading from within template [PR99377]

  Here when lazily loading the binding for f at parse time from the
template g, processing_template_decl is set and thus the call to
note_vague_linkage_fn from module_state::read_cluster has no effect,
and we never push f onto deferred_fns and end up never emitting its
definition.

ISTM the behavior of the lazy loading machinery shouldn't be sensitive
to whether we're inside a template, and therefore we should probably be
clearing processing_template_decl somewhere e.g in lazy_load_binding.
This is sufficient to fix the testcase.

But it also seems the processing_template_decl test in
note_vague_linkage_fn, added by r8-7539-g977bc3ee11383e for PR84973, is
perhaps too strong: if the intent is to avoid deferring output for
uninstantiated templates, we should make sure that DECL in question is
an uninstantiated template by checking e.g. value_dependent_expression_p.
This too is sufficient to fix the testcase (since f isn't a template)
and survives bootstrap and regtest.

Does one or the other approach look like the correct fix for this PR?

	PR c++/99377

gcc/cp/ChangeLog:

	* decl2.cc (note_vague_linkage_fn): Relax processing_template_decl
	test to value_dependent_expression_p.
	* module.cc (lazy_load_binding): Clear processing_template_decl.

gcc/testsuite/ChangeLog:

	* g++.dg/modules/pr99377-2_a.C: New test.
	* g++.dg/modules/pr99377-2_b.C: New test.
---
 gcc/cp/decl2.cc                            | 2 +-
 gcc/cp/module.cc                           | 2 ++
 gcc/testsuite/g++.dg/modules/pr99377-2_a.C | 5 +++++
 gcc/testsuite/g++.dg/modules/pr99377-2_b.C | 6 ++++++
 4 files changed, 14 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_a.C
 create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_b.C
  

On Tue, 4 Oct 2022, Patrick Palka wrote:

> Here when lazily loading the binding for f at parse time from the
> template g, processing_template_decl is set and thus the call to
> note_vague_linkage_fn from module_state::read_cluster has no effect,
> and we never push f onto deferred_fns and end up never emitting its
> definition.
> 
> ISTM the behavior of the lazy loading machinery shouldn't be sensitive
> to whether we're inside a template, and therefore we should probably be
> clearing processing_template_decl somewhere e.g in lazy_load_binding.
> This is sufficient to fix the testcase.
> 
> But it also seems the processing_template_decl test in
> note_vague_linkage_fn, added by r8-7539-g977bc3ee11383e for PR84973, is
> perhaps too strong: if the intent is to avoid deferring output for
> uninstantiated templates, we should make sure that DECL in question is
> an uninstantiated template by checking e.g. value_dependent_expression_p.
> This too is sufficient to fix the testcase (since f isn't a template)
> and survives bootstrap and regtest.
> 
> Does one or the other approach look like the correct fix for this PR?
> 
> 	PR c++/99377
> 
> gcc/cp/ChangeLog:
> 
> 	* decl2.cc (note_vague_linkage_fn): Relax processing_template_decl
> 	test to value_dependent_expression_p.
> 	* module.cc (lazy_load_binding): Clear processing_template_decl.
> 
> gcc/testsuite/ChangeLog:
> 
> 	* g++.dg/modules/pr99377-2_a.C: New test.
> 	* g++.dg/modules/pr99377-2_b.C: New test.
> ---
>  gcc/cp/decl2.cc                            | 2 +-
>  gcc/cp/module.cc                           | 2 ++
>  gcc/testsuite/g++.dg/modules/pr99377-2_a.C | 5 +++++
>  gcc/testsuite/g++.dg/modules/pr99377-2_b.C | 6 ++++++
>  4 files changed, 14 insertions(+), 1 deletion(-)
>  create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_a.C
>  create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> 
> diff --git a/gcc/cp/decl2.cc b/gcc/cp/decl2.cc
> index 9f18466192f..5af4d17ee3b 100644
> --- a/gcc/cp/decl2.cc
> +++ b/gcc/cp/decl2.cc
> @@ -876,7 +876,7 @@ check_classfn (tree ctype, tree function, tree template_parms)
>  void
>  note_vague_linkage_fn (tree decl)
>  {
> -  if (processing_template_decl)
> +  if (value_dependent_expression_p (decl))
>      return;
>  
>    DECL_DEFER_OUTPUT (decl) = 1;
> diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
> index 500ac06563a..79cbb346ffa 100644
> --- a/gcc/cp/module.cc
> +++ b/gcc/cp/module.cc
> @@ -19074,6 +19074,8 @@ lazy_load_binding (unsigned mod, tree ns, tree id, binding_slot *mslot)
>  
>    timevar_start (TV_MODULE_IMPORT);
>  
> +  processing_template_decl_sentinel ptds;
> +
>    /* Stop GC happening, even in outermost loads (because our caller
>       could well be building up a lookup set).  */
>    function_depth++;
> diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_a.C b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
> new file mode 100644
> index 00000000000..26e2bccbbbe
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
> @@ -0,0 +1,5 @@
> +// PR c++/99377
> +// { dg-additional-options -fmodules-ts }
> +// { dg-module-cmi pr99377 }
> +export module pr99377;
> +export inline void f() { }
> diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_b.C b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> new file mode 100644
> index 00000000000..69571952c8a
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> @@ -0,0 +1,6 @@
> +// PR c++/99377
> +// { dg-additional-options -fmodules-ts }
> +// { dg-do link }
> +import pr99377;
> +template<class> void g() { f(); }
> +int main() { g<int>(); }

This test can be simplified to:

import pr99377;
template<class> void g() { f(); }
int main() { f(); }

which probably illustrates the bug better: we lazily load f at the
point of its first use, which in this case is in the (uninstantiated)
template g with processing_template_decl != 0.  Thus during lazy loading
the call to note_vague_linkage_fn has no effect and we never emit the
definition of f.
  

On 10/4/22 13:36, Patrick Palka wrote:
> Here when lazily loading the binding for f at parse time from the
> template g, processing_template_decl is set and thus the call to
> note_vague_linkage_fn from module_state::read_cluster has no effect,
> and we never push f onto deferred_fns and end up never emitting its
> definition.
> 
> ISTM the behavior of the lazy loading machinery shouldn't be sensitive
> to whether we're inside a template, and therefore we should probably be
> clearing processing_template_decl somewhere e.g in lazy_load_binding.
> This is sufficient to fix the testcase.

yeah, I remember hitting issues with this, but thought I'd got rid of the need 
to override processing_template_decl.  Do you also need to override it in 
lazy_load_pendings though? that's a lazy loader and my suspicion is it might be 
susceptible to the same issues.

> 
> But it also seems the processing_template_decl test in
> note_vague_linkage_fn, added by r8-7539-g977bc3ee11383e for PR84973, is
> perhaps too strong: if the intent is to avoid deferring output for
> uninstantiated templates, we should make sure that DECL in question is
> an uninstantiated template by checking e.g. value_dependent_expression_p.
> This too is sufficient to fix the testcase (since f isn't a template)
> and survives bootstrap and regtest.

I think this is an orthogonal issue -- can we remove it from this patch?


> 
> Does one or the other approach look like the correct fix for this PR?
> 
> 	PR c++/99377
> 
> gcc/cp/ChangeLog:
> 
> 	* decl2.cc (note_vague_linkage_fn): Relax processing_template_decl
> 	test to value_dependent_expression_p.
> 	* module.cc (lazy_load_binding): Clear processing_template_decl.
> 
> gcc/testsuite/ChangeLog:
> 
> 	* g++.dg/modules/pr99377-2_a.C: New test.
> 	* g++.dg/modules/pr99377-2_b.C: New test.
> ---
>   gcc/cp/decl2.cc                            | 2 +-
>   gcc/cp/module.cc                           | 2 ++
>   gcc/testsuite/g++.dg/modules/pr99377-2_a.C | 5 +++++
>   gcc/testsuite/g++.dg/modules/pr99377-2_b.C | 6 ++++++
>   4 files changed, 14 insertions(+), 1 deletion(-)
>   create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_a.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> 
> diff --git a/gcc/cp/decl2.cc b/gcc/cp/decl2.cc
> index 9f18466192f..5af4d17ee3b 100644
> --- a/gcc/cp/decl2.cc
> +++ b/gcc/cp/decl2.cc
> @@ -876,7 +876,7 @@ check_classfn (tree ctype, tree function, tree template_parms)
>   void
>   note_vague_linkage_fn (tree decl)
>   {
> -  if (processing_template_decl)
> +  if (value_dependent_expression_p (decl))
>       return;
>   
>     DECL_DEFER_OUTPUT (decl) = 1;
> diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
> index 500ac06563a..79cbb346ffa 100644
> --- a/gcc/cp/module.cc
> +++ b/gcc/cp/module.cc
> @@ -19074,6 +19074,8 @@ lazy_load_binding (unsigned mod, tree ns, tree id, binding_slot *mslot)
>   
>     timevar_start (TV_MODULE_IMPORT);
>   
> +  processing_template_decl_sentinel ptds;
> +
>     /* Stop GC happening, even in outermost loads (because our caller
>        could well be building up a lookup set).  */
>     function_depth++;
> diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_a.C b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
> new file mode 100644
> index 00000000000..26e2bccbbbe
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
> @@ -0,0 +1,5 @@
> +// PR c++/99377
> +// { dg-additional-options -fmodules-ts }
> +// { dg-module-cmi pr99377 }
> +export module pr99377;
> +export inline void f() { }
> diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_b.C b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> new file mode 100644
> index 00000000000..69571952c8a
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> @@ -0,0 +1,6 @@
> +// PR c++/99377
> +// { dg-additional-options -fmodules-ts }
> +// { dg-do link }
> +import pr99377;
> +template<class> void g() { f(); }
> +int main() { g<int>(); }
  

On Mon, 10 Oct 2022, Nathan Sidwell wrote:

> On 10/4/22 13:36, Patrick Palka wrote:
> > Here when lazily loading the binding for f at parse time from the
> > template g, processing_template_decl is set and thus the call to
> > note_vague_linkage_fn from module_state::read_cluster has no effect,
> > and we never push f onto deferred_fns and end up never emitting its
> > definition.
> > 
> > ISTM the behavior of the lazy loading machinery shouldn't be sensitive
> > to whether we're inside a template, and therefore we should probably be
> > clearing processing_template_decl somewhere e.g in lazy_load_binding.
> > This is sufficient to fix the testcase.
> 
> yeah, I remember hitting issues with this, but thought I'd got rid of the need
> to override processing_template_decl.  Do you also need to override it in
> lazy_load_pendings though? that's a lazy loader and my suspicion is it might
> be susceptible to the same issues.

Hmm yeah, looks like we need to override it in lazy_load_pendings too:
I ran the testsuite with gcc_assert (!processing_template_decl) added to
module_state::read_cluster and if we don't also override it in
lazy_load_binding then the assert triggers for pr99425-2_b.X.

> 
> > 
> > But it also seems the processing_template_decl test in
> > note_vague_linkage_fn, added by r8-7539-g977bc3ee11383e for PR84973, is
> > perhaps too strong: if the intent is to avoid deferring output for
> > uninstantiated templates, we should make sure that DECL in question is
> > an uninstantiated template by checking e.g. value_dependent_expression_p.
> > This too is sufficient to fix the testcase (since f isn't a template)
> > and survives bootstrap and regtest.
> 
> I think this is an orthogonal issue -- can we remove it from this patch?

Done.

-- >8 --

Subject: [PATCH] c++ modules: lazy loading from within template [PR99377]

Here when lazily loading the binding for f at parse time from the
template g, processing_template_decl is set and thus the call to
note_vague_linkage_fn from module_state::read_cluster has no effect,
and we never push f onto deferred_fns and end up never emitting its
definition.

ISTM the behavior of the lazy loading machinery shouldn't be sensitive
to whether we're inside a template, and therefore we should be clearing
processing_template_decl in the entrypoints lazy_load_binding and
lazy_load_pendings.

	PR c++/99377

gcc/cp/ChangeLog:

	* module.cc (lazy_load_binding): Clear processing_template_decl.
	(lazy_load_pendings): Likewise.

gcc/testsuite/ChangeLog:

	* g++.dg/modules/pr99377-2_a.C: New test.
	* g++.dg/modules/pr99377-2_b.C: New test.
---
 gcc/cp/module.cc                           | 8 ++++++++
 gcc/testsuite/g++.dg/modules/pr99377-2_a.C | 5 +++++
 gcc/testsuite/g++.dg/modules/pr99377-2_b.C | 6 ++++++
 3 files changed, 19 insertions(+)
 create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_a.C
 create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_b.C

diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
index 94fbee85225..7c48602136c 100644
--- a/gcc/cp/module.cc
+++ b/gcc/cp/module.cc
@@ -19081,6 +19081,10 @@ lazy_load_binding (unsigned mod, tree ns, tree id, binding_slot *mslot)
 
   timevar_start (TV_MODULE_IMPORT);
 
+  /* Make sure lazy loading from a template context behaves as if
+     from a non-template context.  */
+  processing_template_decl_sentinel ptds;
+
   /* Stop GC happening, even in outermost loads (because our caller
      could well be building up a lookup set).  */
   function_depth++;
@@ -19129,6 +19133,10 @@ lazy_load_binding (unsigned mod, tree ns, tree id, binding_slot *mslot)
 void
 lazy_load_pendings (tree decl)
 {
+  /* Make sure lazy loading from a template context behaves as if
+     from a non-template context.  */
+  processing_template_decl_sentinel ptds;
+
   tree key_decl;
   pending_key key;
   key.ns = find_pending_key (decl, &key_decl);
diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_a.C b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
new file mode 100644
index 00000000000..26e2bccbbbe
--- /dev/null
+++ b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
@@ -0,0 +1,5 @@
+// PR c++/99377
+// { dg-additional-options -fmodules-ts }
+// { dg-module-cmi pr99377 }
+export module pr99377;
+export inline void f() { }
diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_b.C b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
new file mode 100644
index 00000000000..69571952c8a
--- /dev/null
+++ b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
@@ -0,0 +1,6 @@
+// PR c++/99377
+// { dg-additional-options -fmodules-ts }
+// { dg-do link }
+import pr99377;
+template<class> void g() { f(); }
+int main() { g<int>(); }
  

On 10/11/22 10:58, Patrick Palka wrote:
> On Mon, 10 Oct 2022, Nathan Sidwell wrote:
> 
>> On 10/4/22 13:36, Patrick Palka wrote:
>>> Here when lazily loading the binding for f at parse time from the
>>> template g, processing_template_decl is set and thus the call to
>>> note_vague_linkage_fn from module_state::read_cluster has no effect,
>>> and we never push f onto deferred_fns and end up never emitting its
>>> definition.
>>>
>>> ISTM the behavior of the lazy loading machinery shouldn't be sensitive
>>> to whether we're inside a template, and therefore we should probably be
>>> clearing processing_template_decl somewhere e.g in lazy_load_binding.
>>> This is sufficient to fix the testcase.
>>
>> yeah, I remember hitting issues with this, but thought I'd got rid of the need
>> to override processing_template_decl.  Do you also need to override it in
>> lazy_load_pendings though? that's a lazy loader and my suspicion is it might
>> be susceptible to the same issues.
> 
> Hmm yeah, looks like we need to override it in lazy_load_pendings too:
> I ran the testsuite with gcc_assert (!processing_template_decl) added to
> module_state::read_cluster and if we don't also override it in
> lazy_load_binding then the assert triggers for pr99425-2_b.X.
> 
>>
>>>
>>> But it also seems the processing_template_decl test in
>>> note_vague_linkage_fn, added by r8-7539-g977bc3ee11383e for PR84973, is
>>> perhaps too strong: if the intent is to avoid deferring output for
>>> uninstantiated templates, we should make sure that DECL in question is
>>> an uninstantiated template by checking e.g. value_dependent_expression_p.
>>> This too is sufficient to fix the testcase (since f isn't a template)
>>> and survives bootstrap and regtest.
>>
>> I think this is an orthogonal issue -- can we remove it from this patch?
> 
> Done.
> 
> -- >8 --
> 
> Subject: [PATCH] c++ modules: lazy loading from within template [PR99377]
> 
> Here when lazily loading the binding for f at parse time from the
> template g, processing_template_decl is set and thus the call to
> note_vague_linkage_fn from module_state::read_cluster has no effect,
> and we never push f onto deferred_fns and end up never emitting its
> definition.
> 
> ISTM the behavior of the lazy loading machinery shouldn't be sensitive
> to whether we're inside a template, and therefore we should be clearing
> processing_template_decl in the entrypoints lazy_load_binding and
> lazy_load_pendings.
> 
> 	PR c++/99377
> 
> gcc/cp/ChangeLog:
> 
> 	* module.cc (lazy_load_binding): Clear processing_template_decl.
> 	(lazy_load_pendings): Likewise.

ok, thanks

> 
> gcc/testsuite/ChangeLog:
> 
> 	* g++.dg/modules/pr99377-2_a.C: New test.
> 	* g++.dg/modules/pr99377-2_b.C: New test.
> ---
>   gcc/cp/module.cc                           | 8 ++++++++
>   gcc/testsuite/g++.dg/modules/pr99377-2_a.C | 5 +++++
>   gcc/testsuite/g++.dg/modules/pr99377-2_b.C | 6 ++++++
>   3 files changed, 19 insertions(+)
>   create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_a.C
>   create mode 100644 gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> 
> diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc
> index 94fbee85225..7c48602136c 100644
> --- a/gcc/cp/module.cc
> +++ b/gcc/cp/module.cc
> @@ -19081,6 +19081,10 @@ lazy_load_binding (unsigned mod, tree ns, tree id, binding_slot *mslot)
>   
>     timevar_start (TV_MODULE_IMPORT);
>   
> +  /* Make sure lazy loading from a template context behaves as if
> +     from a non-template context.  */
> +  processing_template_decl_sentinel ptds;
> +
>     /* Stop GC happening, even in outermost loads (because our caller
>        could well be building up a lookup set).  */
>     function_depth++;
> @@ -19129,6 +19133,10 @@ lazy_load_binding (unsigned mod, tree ns, tree id, binding_slot *mslot)
>   void
>   lazy_load_pendings (tree decl)
>   {
> +  /* Make sure lazy loading from a template context behaves as if
> +     from a non-template context.  */
> +  processing_template_decl_sentinel ptds;
> +
>     tree key_decl;
>     pending_key key;
>     key.ns = find_pending_key (decl, &key_decl);
> diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_a.C b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
> new file mode 100644
> index 00000000000..26e2bccbbbe
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/modules/pr99377-2_a.C
> @@ -0,0 +1,5 @@
> +// PR c++/99377
> +// { dg-additional-options -fmodules-ts }
> +// { dg-module-cmi pr99377 }
> +export module pr99377;
> +export inline void f() { }
> diff --git a/gcc/testsuite/g++.dg/modules/pr99377-2_b.C b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> new file mode 100644
> index 00000000000..69571952c8a
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/modules/pr99377-2_b.C
> @@ -0,0 +1,6 @@
> +// PR c++/99377
> +// { dg-additional-options -fmodules-ts }
> +// { dg-do link }
> +import pr99377;
> +template<class> void g() { f(); }
> +int main() { g<int>(); }