diff mbox series

middle-end: correct loop bounds for early breaks and peeled vector loops

Message ID patch-18055-tamar@arm.com
State Unresolved
Headers
Series middle-end: correct loop bounds for early breaks and peeled vector loops |

Checks

Context Check Description
snail/gcc-patch-check warning Git am fail log

Commit Message

Tamar Christina Dec. 6, 2023, 4:03 a.m. UTC 
  Hi All,

While waiting for reviews I've continued to run more test.
In this case this was one found running 32-bit systems.

While we calculate the right latch count for the epilog,
the vectorizer overrides SCEV and so unrolling goes wrong.

This updates the bounds for the case where we've peeled a
vector iteration.

Testcase in early break testsuite adjusted to test for this
and I'll merge this commit in the main one.

Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.

Ok for master?

Thanks,
Tamar

gcc/ChangeLog:

	* tree-vect-loop-manip.cc (vect_do_peeling): Adjust bounds.

--- inline copy of patch -- 
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644




--
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
       if (bound_scalar.is_constant (&bound))
 	{
 	  gcc_assert (bound != 0);
+	  /* Adjust the upper bound by the extra peeled vector iteration if we
+	     are an epilogue of an peeled vect loop and not VLA.  For VLA the
+	     loop bounds are unknown.  */
+	  if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)
+	      && vf.is_constant ())
+	    bound += vf.to_constant ();
 	  /* -1 to convert loop iterations to latch iterations.  */
 	  record_niter_bound (epilog, bound - 1, false, true);
 	  scale_loop_profile (epilog, profile_probability::always (),

Comments

Richard Biener Dec. 6, 2023, 8:03 a.m. UTC | #1 
On Wed, 6 Dec 2023, Tamar Christina wrote:

> Hi All,
> 
> While waiting for reviews I've continued to run more test.
> In this case this was one found running 32-bit systems.
> 
> While we calculate the right latch count for the epilog,
> the vectorizer overrides SCEV and so unrolling goes wrong.
> 
> This updates the bounds for the case where we've peeled a
> vector iteration.
> 
> Testcase in early break testsuite adjusted to test for this
> and I'll merge this commit in the main one.
> 
> Bootstrapped Regtested on aarch64-none-linux-gnu and no issues.
> 
> Ok for master?

OK.

> Thanks,
> Tamar
> 
> gcc/ChangeLog:
> 
> 	* tree-vect-loop-manip.cc (vect_do_peeling): Adjust bounds.
> 
> --- inline copy of patch -- 
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index 4edde4443ecd98775972f39b3fe839255db12b04..7d48502e2e46240553509dfa6d75fcab7fea36d3 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -3457,6 +3457,12 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
>        if (bound_scalar.is_constant (&bound))
>  	{
>  	  gcc_assert (bound != 0);
> +	  /* Adjust the upper bound by the extra peeled vector iteration if we
> +	     are an epilogue of an peeled vect loop and not VLA.  For VLA the
> +	     loop bounds are unknown.  */
> +	  if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)
> +	      && vf.is_constant ())
> +	    bound += vf.to_constant ();
>  	  /* -1 to convert loop iterations to latch iterations.  */
>  	  record_niter_bound (epilog, bound - 1, false, true);
>  	  scale_loop_profile (epilog, profile_probability::always (),
> 
> 
> 
> 
>
diff mbox series

Patch

--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -3457,6 +3457,12 @@  vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
       if (bound_scalar.is_constant (&bound))
 	{
 	  gcc_assert (bound != 0);
+	  /* Adjust the upper bound by the extra peeled vector iteration if we
+	     are an epilogue of an peeled vect loop and not VLA.  For VLA the
+	     loop bounds are unknown.  */
+	  if (LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)
+	      && vf.is_constant ())
+	    bound += vf.to_constant ();
 	  /* -1 to convert loop iterations to latch iterations.  */
 	  record_niter_bound (epilog, bound - 1, false, true);
 	  scale_loop_profile (epilog, profile_probability::always (),