RISC-V: Disable AVL propagation of vrgather instruction
Checks
Commit Message
This patch fixes following FAILs in zvl1024b of both RV32/RV64:
FAIL: gcc.c-torture/execute/990128-1.c -O2 execution test
FAIL: gcc.c-torture/execute/990128-1.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/990128-1.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/990128-1.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.c-torture/execute/990128-1.c -O3 -g execution test
FAIL: gcc.dg/torture/pr58955-2.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
The root case can be simpliy described in this following small case:
https://godbolt.org/z/7GaxbEGzG
#include "riscv_vector.h"
typedef int64_t v1024b __attribute__ ((vector_size (128)));
void foo (void *out, void *in, int64_t a, int64_t b)
{
v1024b v = {a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a};
v1024b v2 = {b,b,b,b,b,b,b,b,b,b,b,b,b,b,b,b};
v1024b index = *(v1024b*)in;
v1024b v3 = __builtin_shuffle (v, v2, index);
__riscv_vse64_v_i64m1 (out, (vint64m1_t)v3, 10);
}
Incorrect ASM:
foo:
li a5,31
vsetivli zero,10,e64,m1,ta,mu
vmv.v.x v2,a5
vl1re64.v v1,0(a1)
vmv.v.x v4,a2
vand.vv v1,v1,v2
vmv.v.x v3,a3
vmsgeu.vi v0,v1,16
vrgather.vv v2,v4,v1 --> AVL = VLMAX according to codes.
vadd.vi v1,v1,-16
vrgather.vv v2,v3,v1,v0.t --> AVL = VLMAX according to codes.
vse64.v v2,0(a0) --> AVL = 10 according to codes.
ret
For vrgather dest, source, index instruction, when index may has the value > the following store AVL
that is index value > 10. In this situation, the codes above will end up with:
The source vector of vrgather has undefined value on index >= AVL (which is 10 in this case).
So disable AVL propagation for vrgather instruction.
PR target/112599
gcc/ChangeLog:
* config/riscv/riscv-avlprop.cc (alv_can_be_propagated_p): New function.
(vlmax_ta_p): Disable vrgather AVL propagation.
gcc/testsuite/ChangeLog:
* gcc.target/riscv/rvv/autovec/pr112599-1.c: New test.
---
gcc/config/riscv/riscv-avlprop.cc | 13 ++++++++++++-
.../gcc.target/riscv/rvv/autovec/pr112599-1.c | 17 +++++++++++++++++
2 files changed, 29 insertions(+), 1 deletion(-)
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/pr112599-1.c
Comments
Oh. You mean this patch also fixes FLTO failed case ?
juzhe.zhong@rivai.ai
From: Robin Dapp
Date: 2023-11-23 19:55
To: Juzhe-Zhong; gcc-patches
CC: rdapp.gcc; kito.cheng; kito.cheng; jeffreyalaw
Subject: Re: [PATCH] RISC-V: Disable AVL propagation of vrgather instruction
I was just about to post a similar-ish patch that fixes pr65518.c
but you were faster ;)
Therefore LGTM. You can add PR/target 112670.
Regards
Robin
Thanks Robin.
I have sent V2:
https://gcc.gnu.org/pipermail/gcc-patches/2023-November/637921.html
with adding PR/target 112670
Could you commit it for me ?
I am sorry that make you doing redundant work.
I didn't realize they are same issue :)
juzhe.zhong@rivai.ai
From: Robin Dapp
Date: 2023-11-23 19:58
To: juzhe.zhong@rivai.ai; gcc-patches
CC: rdapp.gcc; kito.cheng; Kito.cheng; jeffreyalaw
Subject: Re: [PATCH] RISC-V: Disable AVL propagation of vrgather instruction
> Oh. You mean this patch also fixes FLTO failed case ?
Yes, it's the same issue. There we have a fixed vl (known via LTO)
that is being propagated "into" gathers and we end up missing
gather elements.
Regards
Robin
@@ -104,10 +104,21 @@ avlprop_type_to_str (enum avlprop_type type)
}
}
+/* Return true if the AVL of the INSN can be propagated. */
+static bool
+alv_can_be_propagated_p (rtx_insn *rinsn)
+{
+ /* The index of "vrgather dest, source, index" may pick up the
+ element which has index >= AVL, so we can't strip the elements
+ that has index >= AVL of source register. */
+ return get_attr_type (rinsn) != TYPE_VGATHER;
+}
+
static bool
vlmax_ta_p (rtx_insn *rinsn)
{
- return vlmax_avl_type_p (rinsn) && tail_agnostic_p (rinsn);
+ return vlmax_avl_type_p (rinsn) && tail_agnostic_p (rinsn)
+ && alv_can_be_propagated_p (rinsn);
}
static machine_mode
new file mode 100644
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zfh_zvl1024b -mabi=lp64d -O3 --param=riscv-autovec-preference=fixed-vlmax" } */
+
+#include "riscv_vector.h"
+
+typedef int64_t v1024b __attribute__ ((vector_size (128)));
+
+void foo (void *out, void *in, int64_t a, int64_t b)
+{
+ v1024b v = {a,a,a,a,a,a,a,a,a,a,a,a,a,a,a,a};
+ v1024b v2 = {b,b,b,b,b,b,b,b,b,b,b,b,b,b,b,b};
+ v1024b index = *(v1024b*)in;
+ v1024b v3 = __builtin_shuffle (v, v2, index);
+ __riscv_vse64_v_i64m1 (out, (vint64m1_t)v3, 10);
+}
+
+/* { dg-final { scan-assembler {vsetivli\s+zero,\s*16} } } */