Add attribute hot judgement for INLINE_HINT_known_hot hint.
Checks
Commit Message
Hi Honza,
This patch is to add attribute hot judgement for INLINE_HINT_known_hot hint.
We set up INLINE_HINT_known_hot hint only when we have profile feedback,
now add function attribute judgement for it, when both caller and callee
have __attribute__((hot)), we will also set up INLINE_HINT_known_hot hint
for it.
With this patch applied
Ratio Codesize
ADL Multi-copy: 538.imagic_r 16.7% 1.6%
SPR Multi-copy: 538.imagic_r 15% 1.7%
ICX Multi-copy: 538.imagic_r 15.2% 1.7%
CLX Multi-copy: 538.imagic_r 12.7% 1.7%
Znver3 Multi-copy: 538.imagic_r 10.6% 1.5%
Bootstrap and regtest pending on x86_64-unknown-linux-gnu.
OK for trunk?
Thanks,
Lili.
gcc/ChangeLog
* ipa-inline-analysis.cc (do_estimate_edge_time): Add function attribute
judgement for INLINE_HINT_known_hot hint.
---
gcc/ipa-inline-analysis.cc | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
Comments
> Hi Honza,
>
> This patch is to add attribute hot judgement for INLINE_HINT_known_hot hint.
>
> We set up INLINE_HINT_known_hot hint only when we have profile feedback,
> now add function attribute judgement for it, when both caller and callee
> have __attribute__((hot)), we will also set up INLINE_HINT_known_hot hint
> for it.
>
> With this patch applied
> Ratio Codesize
> ADL Multi-copy: 538.imagic_r 16.7% 1.6%
> SPR Multi-copy: 538.imagic_r 15% 1.7%
> ICX Multi-copy: 538.imagic_r 15.2% 1.7%
> CLX Multi-copy: 538.imagic_r 12.7% 1.7%
> Znver3 Multi-copy: 538.imagic_r 10.6% 1.5%
>
> Bootstrap and regtest pending on x86_64-unknown-linux-gnu.
> OK for trunk?
>
> Thanks,
> Lili.
>
> gcc/ChangeLog
>
> * ipa-inline-analysis.cc (do_estimate_edge_time): Add function attribute
> judgement for INLINE_HINT_known_hot hint.
Thank you. Can you please also add a testcase that tests for this.
So you modify imagemagick marking attribute hot on the specific inline?
I will try to also look again at your earlier patch - I had very busy
summer and unfortunately lost track on this one.
Honza
> ---
> gcc/ipa-inline-analysis.cc | 13 +++++++++----
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/gcc/ipa-inline-analysis.cc b/gcc/ipa-inline-analysis.cc
> index 1ca685d1b0e..7bd29c36590 100644
> --- a/gcc/ipa-inline-analysis.cc
> +++ b/gcc/ipa-inline-analysis.cc
> @@ -48,6 +48,7 @@ along with GCC; see the file COPYING3. If not see
> #include "ipa-utils.h"
> #include "cfgexpand.h"
> #include "gimplify.h"
> +#include "attribs.h"
>
> /* Cached node/edge growths. */
> fast_call_summary<edge_growth_cache_entry *, va_heap> *edge_growth_cache = NULL;
> @@ -249,15 +250,19 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time)
> hints = estimates.hints;
> }
>
> - /* When we have profile feedback, we can quite safely identify hot
> - edges and for those we disable size limits. Don't do that when
> - probability that caller will call the callee is low however, since it
> + /* When we have profile feedback or function attribute, we can quite safely
> + identify hot edges and for those we disable size limits. Don't do that
> + when probability that caller will call the callee is low however, since it
> may hurt optimization of the caller's hot path. */
> - if (edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
> + if ((edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
> && (edge->count.ipa () * 2
> > (edge->caller->inlined_to
> ? edge->caller->inlined_to->count.ipa ()
> : edge->caller->count.ipa ())))
> + || (lookup_attribute ("hot", DECL_ATTRIBUTES (edge->caller->decl))
> + != NULL
> + && lookup_attribute ("hot", DECL_ATTRIBUTES (edge->callee->decl))
> + != NULL))
> hints |= INLINE_HINT_known_hot;
>
> gcc_checking_assert (size >= 0);
> --
> 2.17.1
>
> Thank you. Can you please also add a testcase that tests for this.
> So you modify imagemagick marking attribute hot on the specific inline?
Thanks Honza. Added the testcase. I didn't modify source code of 538.imagic_r, the original source code has attribute like:
#define magick_hot_spot __attribute__((__hot__))
static Cache *SetPixelCacheNexusPixels( ... ) magick_hot_spot;
> I will try to also look again at your earlier patch - I had very busy summer and
> unfortunately lost track on this one.
>
NP, I guessed you were busy during that time, my earlier patch was partially duplicated with function "Elimination_by_inlining_prob",
except "parameter points to caller local memory" part, maybe we can find a suitable place to add local memory part to the IPA.
> Honza
gcc/ChangeLog
* ipa-inline-analysis.cc (do_estimate_edge_time): Add function attribute
judgement for INLINE_HINT_known_hot hint.
gcc/testsuite/ChangeLog:
* gcc.dg/ipa/inlinehint-6.c: New test.
---
gcc/ipa-inline-analysis.cc | 13 ++++---
gcc/testsuite/gcc.dg/ipa/inlinehint-6.c | 47 +++++++++++++++++++++++++
2 files changed, 56 insertions(+), 4 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
diff --git a/gcc/ipa-inline-analysis.cc b/gcc/ipa-inline-analysis.cc
index 1ca685d1b0e..7bd29c36590 100644
--- a/gcc/ipa-inline-analysis.cc
+++ b/gcc/ipa-inline-analysis.cc
@@ -48,6 +48,7 @@ along with GCC; see the file COPYING3. If not see
#include "ipa-utils.h"
#include "cfgexpand.h"
#include "gimplify.h"
+#include "attribs.h"
/* Cached node/edge growths. */
fast_call_summary<edge_growth_cache_entry *, va_heap> *edge_growth_cache = NULL;
@@ -249,15 +250,19 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time)
hints = estimates.hints;
}
- /* When we have profile feedback, we can quite safely identify hot
- edges and for those we disable size limits. Don't do that when
- probability that caller will call the callee is low however, since it
+ /* When we have profile feedback or function attribute, we can quite safely
+ identify hot edges and for those we disable size limits. Don't do that
+ when probability that caller will call the callee is low however, since it
may hurt optimization of the caller's hot path. */
- if (edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
+ if ((edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
&& (edge->count.ipa () * 2
> (edge->caller->inlined_to
? edge->caller->inlined_to->count.ipa ()
: edge->caller->count.ipa ())))
+ || (lookup_attribute ("hot", DECL_ATTRIBUTES (edge->caller->decl))
+ != NULL
+ && lookup_attribute ("hot", DECL_ATTRIBUTES (edge->callee->decl))
+ != NULL))
hints |= INLINE_HINT_known_hot;
gcc_checking_assert (size >= 0);
diff --git a/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c b/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
new file mode 100644
index 00000000000..1f3be641c6d
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
@@ -0,0 +1,47 @@
+/* { dg-options "-O3 -c -fdump-ipa-inline-details -fno-early-inlining -fno-ipa-cp" } */
+/* { dg-add-options bind_pic_locally } */
+
+#define size_t long long int
+
+struct A
+{
+ size_t f1, f2, f3, f4;
+};
+struct C
+{
+ struct A a;
+ size_t b;
+};
+struct C x;
+
+__attribute__((hot)) struct C callee (struct A *a, struct C *c)
+{
+ c->a=(*a);
+
+ if((c->b + 7) & 17)
+ {
+ c->a.f1 = c->a.f2 + c->a.f1;
+ c->a.f2 = c->a.f3 - c->a.f2;
+ c->a.f3 = c->a.f2 + c->a.f3;
+ c->a.f4 = c->a.f2 - c->a.f4;
+ c->b = c->a.f2;
+
+ }
+ return *c;
+}
+
+__attribute__((hot)) struct C caller (size_t d, size_t e, size_t f, size_t g, struct C *c)
+{
+ struct A a;
+ a.f1 = 1 + d;
+ a.f2 = e;
+ a.f3 = 12 + f;
+ a.f4 = 68 + g;
+ if (c->b > 0)
+ return callee (&a, c);
+ else
+ return *c;
+}
+
+/* { dg-final { scan-ipa-dump "known_hot" "inline" } } */
+
Hi Honza,
Gentle ping https://gcc.gnu.org/pipermail/gcc-patches/2022-September/601934.html
Thanks,
Lili.
> -----Original Message-----
> From: Gcc-patches <gcc-patches-bounces+lili.cui=intel.com@gcc.gnu.org> On
> Behalf Of Cui, Lili via Gcc-patches
> Sent: Wednesday, September 21, 2022 5:22 PM
> To: Jan Hubicka <hubicka@ucw.cz>
> Cc: Lu, Hongjiu <hongjiu.lu@intel.com>; Liu, Hongtao
> <hongtao.liu@intel.com>; gcc-patches@gcc.gnu.org
> Subject: RE: [PATCH] Add attribute hot judgement for
> INLINE_HINT_known_hot hint.
>
> > Thank you. Can you please also add a testcase that tests for this.
> > So you modify imagemagick marking attribute hot on the specific inline?
>
> Thanks Honza. Added the testcase. I didn't modify source code of
> 538.imagic_r, the original source code has attribute like:
>
> #define magick_hot_spot __attribute__((__hot__)) static Cache
> *SetPixelCacheNexusPixels( ... ) magick_hot_spot;
>
> > I will try to also look again at your earlier patch - I had very busy
> > summer and unfortunately lost track on this one.
> >
> NP, I guessed you were busy during that time, my earlier patch was partially
> duplicated with function "Elimination_by_inlining_prob", except "parameter
> points to caller local memory" part, maybe we can find a suitable place to
> add local memory part to the IPA.
>
> > Honza
>
> gcc/ChangeLog
>
> * ipa-inline-analysis.cc (do_estimate_edge_time): Add function attribute
> judgement for INLINE_HINT_known_hot hint.
>
> gcc/testsuite/ChangeLog:
>
> * gcc.dg/ipa/inlinehint-6.c: New test.
> ---
> gcc/ipa-inline-analysis.cc | 13 ++++---
> gcc/testsuite/gcc.dg/ipa/inlinehint-6.c | 47 +++++++++++++++++++++++++
> 2 files changed, 56 insertions(+), 4 deletions(-) create mode 100644
> gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
>
> diff --git a/gcc/ipa-inline-analysis.cc b/gcc/ipa-inline-analysis.cc index
> 1ca685d1b0e..7bd29c36590 100644
> --- a/gcc/ipa-inline-analysis.cc
> +++ b/gcc/ipa-inline-analysis.cc
> @@ -48,6 +48,7 @@ along with GCC; see the file COPYING3. If not see
> #include "ipa-utils.h"
> #include "cfgexpand.h"
> #include "gimplify.h"
> +#include "attribs.h"
>
> /* Cached node/edge growths. */
> fast_call_summary<edge_growth_cache_entry *, va_heap>
> *edge_growth_cache = NULL; @@ -249,15 +250,19 @@
> do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time)
> hints = estimates.hints;
> }
>
> - /* When we have profile feedback, we can quite safely identify hot
> - edges and for those we disable size limits. Don't do that when
> - probability that caller will call the callee is low however, since it
> + /* When we have profile feedback or function attribute, we can quite
> safely
> + identify hot edges and for those we disable size limits. Don't do that
> + when probability that caller will call the callee is low however,
> + since it
> may hurt optimization of the caller's hot path. */
> - if (edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
> + if ((edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
> && (edge->count.ipa () * 2
> > (edge->caller->inlined_to
> ? edge->caller->inlined_to->count.ipa ()
> : edge->caller->count.ipa ())))
> + || (lookup_attribute ("hot", DECL_ATTRIBUTES (edge->caller->decl))
> + != NULL
> + && lookup_attribute ("hot", DECL_ATTRIBUTES (edge->callee->decl))
> + != NULL))
> hints |= INLINE_HINT_known_hot;
>
> gcc_checking_assert (size >= 0);
> diff --git a/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> b/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> new file mode 100644
> index 00000000000..1f3be641c6d
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> @@ -0,0 +1,47 @@
> +/* { dg-options "-O3 -c -fdump-ipa-inline-details -fno-early-inlining
> +-fno-ipa-cp" } */
> +/* { dg-add-options bind_pic_locally } */
> +
> +#define size_t long long int
> +
> +struct A
> +{
> + size_t f1, f2, f3, f4;
> +};
> +struct C
> +{
> + struct A a;
> + size_t b;
> +};
> +struct C x;
> +
> +__attribute__((hot)) struct C callee (struct A *a, struct C *c) {
> + c->a=(*a);
> +
> + if((c->b + 7) & 17)
> + {
> + c->a.f1 = c->a.f2 + c->a.f1;
> + c->a.f2 = c->a.f3 - c->a.f2;
> + c->a.f3 = c->a.f2 + c->a.f3;
> + c->a.f4 = c->a.f2 - c->a.f4;
> + c->b = c->a.f2;
> +
> + }
> + return *c;
> +}
> +
> +__attribute__((hot)) struct C caller (size_t d, size_t e, size_t f,
> +size_t g, struct C *c) {
> + struct A a;
> + a.f1 = 1 + d;
> + a.f2 = e;
> + a.f3 = 12 + f;
> + a.f4 = 68 + g;
> + if (c->b > 0)
> + return callee (&a, c);
> + else
> + return *c;
> +}
> +
> +/* { dg-final { scan-ipa-dump "known_hot" "inline" } } */
> +
> --
> 2.17.1
>
> Thanks,
> Lili.
Hi Honza,
Gentle ping https://gcc.gnu.org/pipermail/gcc-patches/2022-September/601934.html
Thanks,
Lili.
> -----Original Message-----
> From: Cui, Lili <lili.cui@intel.com>
> Sent: Saturday, October 8, 2022 8:33 AM
> To: Cui, Lili <lili.cui@intel.com>; Jan Hubicka <hubicka@ucw.cz>
> Cc: Lu, Hongjiu <hongjiu.lu@intel.com>; Liu, Hongtao
> <hongtao.liu@intel.com>; gcc-patches@gcc.gnu.org
> Subject: Ping^1 [PATCH] Add attribute hot judgement for
> INLINE_HINT_known_hot hint.
>
> Hi Honza,
>
> Gentle ping https://gcc.gnu.org/pipermail/gcc-patches/2022-
> September/601934.html
>
> Thanks,
> Lili.
>
> > -----Original Message-----
> > From: Gcc-patches <gcc-patches-bounces+lili.cui=intel.com@gcc.gnu.org>
> > On Behalf Of Cui, Lili via Gcc-patches
> > Sent: Wednesday, September 21, 2022 5:22 PM
> > To: Jan Hubicka <hubicka@ucw.cz>
> > Cc: Lu, Hongjiu <hongjiu.lu@intel.com>; Liu, Hongtao
> > <hongtao.liu@intel.com>; gcc-patches@gcc.gnu.org
> > Subject: RE: [PATCH] Add attribute hot judgement for
> > INLINE_HINT_known_hot hint.
> >
> > > Thank you. Can you please also add a testcase that tests for this.
> > > So you modify imagemagick marking attribute hot on the specific inline?
> >
> > Thanks Honza. Added the testcase. I didn't modify source code of
> > 538.imagic_r, the original source code has attribute like:
> >
> > #define magick_hot_spot __attribute__((__hot__)) static Cache
> > *SetPixelCacheNexusPixels( ... ) magick_hot_spot;
> >
> > > I will try to also look again at your earlier patch - I had very
> > > busy summer and unfortunately lost track on this one.
> > >
> > NP, I guessed you were busy during that time, my earlier patch was
> > partially duplicated with function "Elimination_by_inlining_prob",
> > except "parameter points to caller local memory" part, maybe we can
> > find a suitable place to add local memory part to the IPA.
> >
> > > Honza
> >
> > gcc/ChangeLog
> >
> > * ipa-inline-analysis.cc (do_estimate_edge_time): Add function attribute
> > judgement for INLINE_HINT_known_hot hint.
> >
> > gcc/testsuite/ChangeLog:
> >
> > * gcc.dg/ipa/inlinehint-6.c: New test.
> > ---
> > gcc/ipa-inline-analysis.cc | 13 ++++---
> > gcc/testsuite/gcc.dg/ipa/inlinehint-6.c | 47
> > +++++++++++++++++++++++++
> > 2 files changed, 56 insertions(+), 4 deletions(-) create mode 100644
> > gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> >
> > diff --git a/gcc/ipa-inline-analysis.cc b/gcc/ipa-inline-analysis.cc
> > index
> > 1ca685d1b0e..7bd29c36590 100644
> > --- a/gcc/ipa-inline-analysis.cc
> > +++ b/gcc/ipa-inline-analysis.cc
> > @@ -48,6 +48,7 @@ along with GCC; see the file COPYING3. If not see
> > #include "ipa-utils.h"
> > #include "cfgexpand.h"
> > #include "gimplify.h"
> > +#include "attribs.h"
> >
> > /* Cached node/edge growths. */
> > fast_call_summary<edge_growth_cache_entry *, va_heap>
> > *edge_growth_cache = NULL; @@ -249,15 +250,19 @@
> do_estimate_edge_time
> > (struct cgraph_edge *edge, sreal *ret_nonspec_time)
> > hints = estimates.hints;
> > }
> >
> > - /* When we have profile feedback, we can quite safely identify hot
> > - edges and for those we disable size limits. Don't do that when
> > - probability that caller will call the callee is low however, since it
> > + /* When we have profile feedback or function attribute, we can
> > + quite
> > safely
> > + identify hot edges and for those we disable size limits. Don't do that
> > + when probability that caller will call the callee is low
> > + however, since it
> > may hurt optimization of the caller's hot path. */
> > - if (edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
> > + if ((edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
> > && (edge->count.ipa () * 2
> > > (edge->caller->inlined_to
> > ? edge->caller->inlined_to->count.ipa ()
> > : edge->caller->count.ipa ())))
> > + || (lookup_attribute ("hot", DECL_ATTRIBUTES (edge->caller->decl))
> > + != NULL
> > + && lookup_attribute ("hot", DECL_ATTRIBUTES (edge->callee->decl))
> > + != NULL))
> > hints |= INLINE_HINT_known_hot;
> >
> > gcc_checking_assert (size >= 0);
> > diff --git a/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> > b/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> > new file mode 100644
> > index 00000000000..1f3be641c6d
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.dg/ipa/inlinehint-6.c
> > @@ -0,0 +1,47 @@
> > +/* { dg-options "-O3 -c -fdump-ipa-inline-details -fno-early-inlining
> > +-fno-ipa-cp" } */
> > +/* { dg-add-options bind_pic_locally } */
> > +
> > +#define size_t long long int
> > +
> > +struct A
> > +{
> > + size_t f1, f2, f3, f4;
> > +};
> > +struct C
> > +{
> > + struct A a;
> > + size_t b;
> > +};
> > +struct C x;
> > +
> > +__attribute__((hot)) struct C callee (struct A *a, struct C *c) {
> > + c->a=(*a);
> > +
> > + if((c->b + 7) & 17)
> > + {
> > + c->a.f1 = c->a.f2 + c->a.f1;
> > + c->a.f2 = c->a.f3 - c->a.f2;
> > + c->a.f3 = c->a.f2 + c->a.f3;
> > + c->a.f4 = c->a.f2 - c->a.f4;
> > + c->b = c->a.f2;
> > +
> > + }
> > + return *c;
> > +}
> > +
> > +__attribute__((hot)) struct C caller (size_t d, size_t e, size_t f,
> > +size_t g, struct C *c) {
> > + struct A a;
> > + a.f1 = 1 + d;
> > + a.f2 = e;
> > + a.f3 = 12 + f;
> > + a.f4 = 68 + g;
> > + if (c->b > 0)
> > + return callee (&a, c);
> > + else
> > + return *c;
> > +}
> > +
> > +/* { dg-final { scan-ipa-dump "known_hot" "inline" } } */
> > +
> > --
> > 2.17.1
> >
> > Thanks,
> > Lili.
@@ -48,6 +48,7 @@ along with GCC; see the file COPYING3. If not see
#include "ipa-utils.h"
#include "cfgexpand.h"
#include "gimplify.h"
+#include "attribs.h"
/* Cached node/edge growths. */
fast_call_summary<edge_growth_cache_entry *, va_heap> *edge_growth_cache = NULL;
@@ -249,15 +250,19 @@ do_estimate_edge_time (struct cgraph_edge *edge, sreal *ret_nonspec_time)
hints = estimates.hints;
}
- /* When we have profile feedback, we can quite safely identify hot
- edges and for those we disable size limits. Don't do that when
- probability that caller will call the callee is low however, since it
+ /* When we have profile feedback or function attribute, we can quite safely
+ identify hot edges and for those we disable size limits. Don't do that
+ when probability that caller will call the callee is low however, since it
may hurt optimization of the caller's hot path. */
- if (edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
+ if ((edge->count.ipa ().initialized_p () && edge->maybe_hot_p ()
&& (edge->count.ipa () * 2
> (edge->caller->inlined_to
? edge->caller->inlined_to->count.ipa ()
: edge->caller->count.ipa ())))
+ || (lookup_attribute ("hot", DECL_ATTRIBUTES (edge->caller->decl))
+ != NULL
+ && lookup_attribute ("hot", DECL_ATTRIBUTES (edge->callee->decl))
+ != NULL))
hints |= INLINE_HINT_known_hot;
gcc_checking_assert (size >= 0);