From patchwork Wed Oct 18 10:25:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 154795 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4690795vqb; Wed, 18 Oct 2023 03:26:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHRcSaRgvQUXWOaXu2TeVNA23vkvlJQbGXuDivpBFeSlU5+NHnTamyBLyDaNPm0niL+4JQy X-Received: by 2002:a05:620a:4706:b0:76c:8b47:f394 with SMTP id bs6-20020a05620a470600b0076c8b47f394mr5693978qkb.2.1697624771632; Wed, 18 Oct 2023 03:26:11 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1697624771; cv=pass; d=google.com; s=arc-20160816; b=Ci/FFC+kMwT9luWqD3feNSohiTZ8W8MmCl76nSBqS/g/3m6RYWiG55Mh9Blqhuwlfq kgf2qJcr/UUuTJHf0wsfmG2D6yjcNFy1j31WRUbhGIRMDiOPZpJyl9Jgb8u/mMK0ydgT LMFBW02fk1vQoSWaheQKHhTAV6aFBjSPw2tGl7B1tD1O20a6CWCIrjhxN0oFq2pu7u3i i3moGIanQfkK9GeITCjzxm+59EThnobUHbj7xzUYIMwQ041Ltnalny8b5qJvYTTEA3Ec OQRBu0cYHpjW84lsw7vavklhuav/sPa5HsbPJ6LTLvADqYYTNhOI7tImVUcVQxxkD/vq dGBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:arc-filter:dmarc-filter:delivered-to; bh=J3ujvIeHULkLAcXnaz0HSYv8wrPKIIpCSbeYzJ+D4ww=; fh=12MRPJmZ1mgDpHqWoogMKqnaGRGM2b7lcuJroqfjJiw=; b=jL08ViHTC7rLfD2cJZ1fGR6Eg0fzI1ayUrBzOBOhdM20D+3l9UBfxGFCZGk94Idwjs zZDlxncL9EjrHOy/oWW6qZt43+FXnnFoEs41O7pxWYgxxIO7iHu6WO1FKEESujEOUdot 5a0Xu/19q9Or3XZK7i52k+kKUBhV2f4hEhZJAbXJUumt1xhDorJ0j2Ku9uMC8Xl9qHv+ fF+mkPbp5JVJfSYQLCMoVGxQ+XkctDKzj5lVrOAmbSQ3jtDP7B/quCfeCtDfZ2dcVXEN JNk6HzXthsRBTPNdumyeQo61Ux5iuMT3ZUIauOrTi4KNwWao5bRwwgvEvI+Ix6iDTqib Dc3w== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id de10-20020a05620a370a00b007706b14735dsi2447987qkb.643.2023.10.18.03.26.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Oct 2023 03:26:11 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 59AB33858C54 for ; Wed, 18 Oct 2023 10:26:11 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgsg2.qq.com (smtpbgsg2.qq.com [54.254.200.128]) by sourceware.org (Postfix) with ESMTPS id 35FB83858C54 for ; Wed, 18 Oct 2023 10:25:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 35FB83858C54 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 35FB83858C54 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.254.200.128 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697624745; cv=none; b=bYvAWDeHZC+jQyKpcnxnKFl+3DUkwYjlMf4y1COSqZ0JXsgwX8b73/RVb/oeIdxG/4UHnJhJsccYcsikglPL2MJv3T0Q8Plaf+ZDtOqrLnCxoLToeVb7b/uYLhttdZEh/aQ1omrFmd/ymVIkw5tg8CGB1KEXsorlihoTC7SM/D8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697624745; c=relaxed/simple; bh=zZBjmixDMOx20PmAAeUa6RK5Xf4v+9vMrmz64ezHMPI=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=HvBoH7exxMu79CabX5jV93lHoVKWYEFcf/ZtovVY4Z0bbBY7G5QxZZ09twpramHS+4XygALtFn1qPRWWxkDaVG4MQnreO3i5LMtXq70KVKu9Mr4TTUSMD7TaVPMtR7h0ryoxHrYwLcAMM3IJPQY+j9E1gIi+3FpbiddRkblRVgk= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp90t1697624735tndpwdwx Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 18 Oct 2023 18:25:34 +0800 (CST) X-QQ-SSF: 01400000000000G0V000000A0000000 X-QQ-FEAT: eTp2eCgHrZkN7qyVPADZQsvvWRLY6NMoZ6J95gSaJXWS5EhkoCHB5vj5VLOlt XQJYyeONvj1ZjiPmuVdXBCr/eGrVFDFdgyGPBtFZxdFk+VSPMTcR7SLlbi/cUterNA8irma sSSluggedOTBgtbxodevbT7lE4Mv3Nxtyf1POD+bs8JTzgJWWvnpbJqbDqAzdsL7pxIxfol //qdQSITV8rExUevuxGWXYN/XH3Gwima1Ox2PtraenM0XKGygqxNdYEDxbFEyTEFw9eNiGn ehqcF63cPiS3f2txoT271udYOoEo/3RsJUYCsp4D2NPzC1K6DKQKASUw+Khp082nGSSUmaU xvufZm6rBvLsWb9AxYFSqQMR0z8ga+bwSFkWxp0b5XUxcmqo8BeFnnIvfMRSszaeksp3WWw mOBpbT/DUv6ZuC17+IymvQ== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 6224230576106643045 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH V2] RISC-V: Fix failed hoist in LICM of vmv.v.x instruction Date: Wed, 18 Oct 2023 18:25:33 +0800 Message-Id: <20231018102533.2643245-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SCC_5_SHORT_WORD_LINES, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780088359943953414 X-GMAIL-MSGID: 1780088592630024959 Confirm dynamic LMUL algorithm works well for choosing LMUL = 4 for the PR: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111848 But it generate horrible register spillings. The root cause is that we didn't hoist the vmv.v.x outside the loop which increase the SLP loop register pressure. So, change the COSNT_VECTOR move into vec_duplicate splitter that we can gain better optimizations: 1. better LICM. 2. More opportunities of transforming 'vv' into 'vx' in the future. Before this patch: f3: ble a4,zero,.L8 csrr t0,vlenb slli t1,t0,4 csrr a6,vlenb sub sp,sp,t1 csrr a5,vlenb slli a6,a6,3 slli a5,a5,2 add a6,a6,sp vsetvli a7,zero,e16,m8,ta,ma slli a4,a4,3 vid.v v8 addi t6,a5,-1 vand.vi v8,v8,-2 neg t5,a5 vs8r.v v8,0(sp) vadd.vi v8,v8,1 vs8r.v v8,0(a6) j .L4 .L12: vsetvli a7,zero,e16,m8,ta,ma .L4: csrr t0,vlenb slli t0,t0,3 vl8re16.v v16,0(sp) add t0,t0,sp vmv.v.x v8,t6 mv t1,a4 vand.vv v24,v16,v8 mv a6,a4 vl8re16.v v16,0(t0) vand.vv v8,v16,v8 bleu a4,a5,.L3 mv a6,a5 .L3: vsetvli zero,a6,e8,m4,ta,ma vle8.v v20,0(a2) vle8.v v16,0(a3) vsetvli a7,zero,e8,m4,ta,ma vrgatherei16.vv v4,v20,v24 vadd.vv v4,v16,v4 vsetvli zero,a6,e8,m4,ta,ma vse8.v v4,0(a0) vle8.v v20,0(a2) vsetvli a7,zero,e8,m4,ta,ma vrgatherei16.vv v4,v20,v8 vadd.vv v4,v4,v16 vsetvli zero,a6,e8,m4,ta,ma vse8.v v4,0(a1) add a4,a4,t5 add a0,a0,a5 add a3,a3,a5 add a1,a1,a5 add a2,a2,a5 bgtu t1,a5,.L12 csrr t0,vlenb slli t1,t0,4 add sp,sp,t1 jr ra .L8: ret After this patch: f3: ble a4,zero,.L6 csrr a6,vlenb csrr a5,vlenb slli a6,a6,2 slli a5,a5,2 addi a6,a6,-1 slli a4,a4,3 neg t5,a5 vsetvli t1,zero,e16,m8,ta,ma vmv.v.x v24,a6 vid.v v8 vand.vi v8,v8,-2 vadd.vi v16,v8,1 vand.vv v8,v8,v24 vand.vv v16,v16,v24 .L4: mv t1,a4 mv a6,a4 bleu a4,a5,.L3 mv a6,a5 .L3: vsetvli zero,a6,e8,m4,ta,ma vle8.v v28,0(a2) vle8.v v24,0(a3) vsetvli a7,zero,e8,m4,ta,ma vrgatherei16.vv v4,v28,v8 vadd.vv v4,v24,v4 vsetvli zero,a6,e8,m4,ta,ma vse8.v v4,0(a0) vle8.v v28,0(a2) vsetvli a7,zero,e8,m4,ta,ma vrgatherei16.vv v4,v28,v16 vadd.vv v4,v4,v24 vsetvli zero,a6,e8,m4,ta,ma vse8.v v4,0(a1) add a4,a4,t5 add a0,a0,a5 add a3,a3,a5 add a1,a1,a5 add a2,a2,a5 bgtu t1,a5,.L4 .L6: ret Note that this patch triggers multiple FAILs: FAIL: gcc.target/riscv/rvv/autovec/cond/cond_arith_run-3.c execution test FAIL: gcc.target/riscv/rvv/autovec/cond/cond_arith_run-3.c execution test FAIL: gcc.target/riscv/rvv/autovec/cond/cond_arith_run-4.c execution test FAIL: gcc.target/riscv/rvv/autovec/cond/cond_arith_run-4.c execution test FAIL: gcc.target/riscv/rvv/autovec/cond/cond_arith_run-8.c execution test FAIL: gcc.target/riscv/rvv/autovec/cond/cond_arith_run-8.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_load_run-1.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_load_run-1.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_load_run-2.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_load_run-2.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_store_run-1.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_store_run-1.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_store_run-2.c execution test FAIL: gcc.target/riscv/rvv/autovec/gather-scatter/strided_store_run-2.c execution test They failed are all because of bugs on VSETVL PASS: 10dd4: 0c707057 vsetvli zero,zero,e8,mf2,ta,ma 10dd8: 5e06b8d7 vmv.v.i v17,13 10ddc: 9ed030d7 vmv1r.v v1,v13 10de0: b21040d7 vncvt.x.x.w v1,v1 ----> raise illegal instruction since we don't have SEW = 8 -> SEW = 4 narrowing. 10de4: 5e0785d7 vmv.v.v v11,v15 Confirm the recent VSETVL refactor patch: https://gcc.gnu.org/pipermail/gcc-patches/2023-October/633231.html fixed all of them. So this patch should be committed after the VSETVL refactor patch. PR target/111848 gcc/ChangeLog: * config/riscv/riscv-selftests.cc (run_const_vector_selftests): Adapt selftest. * config/riscv/riscv-v.cc (expand_const_vector): Change it into vec_duplicate splitter. gcc/testsuite/ChangeLog: * gcc.dg/vect/costmodel/riscv/rvv/dynamic-lmul2-7.c: Adapt test. * gcc.dg/vect/costmodel/riscv/rvv/pr111848.c: New test. --- gcc/config/riscv/riscv-selftests.cc | 14 ++++---- gcc/config/riscv/riscv-v.cc | 27 ++++++++++++-- .../costmodel/riscv/rvv/dynamic-lmul2-7.c | 3 +- .../vect/costmodel/riscv/rvv/pr111848.c | 35 +++++++++++++++++++ 4 files changed, 68 insertions(+), 11 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr111848.c diff --git a/gcc/config/riscv/riscv-selftests.cc b/gcc/config/riscv/riscv-selftests.cc index cdc863ee4f7..0ac17fb70a1 100644 --- a/gcc/config/riscv/riscv-selftests.cc +++ b/gcc/config/riscv/riscv-selftests.cc @@ -267,15 +267,14 @@ run_const_vector_selftests (void) rtx dup = gen_const_vec_duplicate (mode, GEN_INT (val)); emit_move_insn (dest, dup); rtx_insn *insn = get_last_insn (); - rtx src = XEXP (SET_SRC (PATTERN (insn)), 1); + rtx src = SET_SRC (PATTERN (insn)); /* 1. Should be vmv.v.i for in rang of -16 ~ 15. 2. Should be vmv.v.x for exceed -16 ~ 15. */ if (IN_RANGE (val, -16, 15)) - ASSERT_TRUE (rtx_equal_p (src, dup)); - else ASSERT_TRUE ( - rtx_equal_p (src, - gen_rtx_VEC_DUPLICATE (mode, XEXP (src, 0)))); + rtx_equal_p (XEXP (SET_SRC (PATTERN (insn)), 1), dup)); + else + ASSERT_TRUE (GET_CODE (src) == VEC_DUPLICATE); end_sequence (); } } @@ -294,10 +293,9 @@ run_const_vector_selftests (void) rtx dup = gen_const_vec_duplicate (mode, ele); emit_move_insn (dest, dup); rtx_insn *insn = get_last_insn (); - rtx src = XEXP (SET_SRC (PATTERN (insn)), 1); + rtx src = SET_SRC (PATTERN (insn)); /* Should always be vfmv.v.f. */ - ASSERT_TRUE ( - rtx_equal_p (src, gen_rtx_VEC_DUPLICATE (mode, XEXP (src, 0)))); + ASSERT_TRUE (GET_CODE (src) == VEC_DUPLICATE); end_sequence (); } } diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index 895c11d13fc..6116f5df504 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -1001,8 +1001,31 @@ expand_const_vector (rtx target, rtx src) } else { - rtx ops[] = {tmp, elt}; - emit_vlmax_insn (code_for_pred_broadcast (mode), UNARY_OP, ops); + /* Emit vec_duplicate split pattern before RA so that + we could have a better optimization opportunity in LICM + which will hoist vmv.v.x outside the loop and in fwprop && combine + which will transform 'vv' into 'vx' instruction. + + The reason we don't emit vec_duplicate split pattern during + RA since the split stage after RA is a too late stage to generate + RVV instruction which need an additional register (We can't + allocate a new register after RA) for VL operand of vsetvl + instruction (vsetvl a5, zero). */ + if (lra_in_progress) + { + rtx ops[] = {tmp, elt}; + emit_vlmax_insn (code_for_pred_broadcast (mode), UNARY_OP, ops); + } + else + { + struct expand_operand ops[2]; + enum insn_code icode = optab_handler (vec_duplicate_optab, mode); + gcc_assert (icode != CODE_FOR_nothing); + create_output_operand (&ops[0], tmp, mode); + create_input_operand (&ops[1], elt, GET_MODE_INNER (mode)); + expand_insn (icode, 2, ops); + tmp = ops[0].value; + } } if (tmp != target) diff --git a/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/dynamic-lmul2-7.c b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/dynamic-lmul2-7.c index 3dfc6f16a25..2a735d8c6b6 100644 --- a/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/dynamic-lmul2-7.c +++ b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/dynamic-lmul2-7.c @@ -18,7 +18,8 @@ bar (int *x, int a, int b, int n) } /* { dg-final { scan-assembler {e32,m2} } } */ -/* { dg-final { scan-assembler-times {csrr} 1 } } */ +/* { dg-final { scan-assembler-not {jr} } } */ +/* { dg-final { scan-assembler-times {ret} 2 } } * /* { dg-final { scan-tree-dump-times "Maximum lmul = 8" 1 "vect" } } */ /* { dg-final { scan-tree-dump-times "Maximum lmul = 4" 1 "vect" } } */ /* { dg-final { scan-tree-dump-times "Maximum lmul = 2" 1 "vect" } } */ diff --git a/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr111848.c b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr111848.c new file mode 100644 index 00000000000..b203ca907fa --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/costmodel/riscv/rvv/pr111848.c @@ -0,0 +1,35 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -fdump-tree-vect-details" } */ + +void +f3 (uint8_t *restrict a, uint8_t *restrict b, + uint8_t *restrict c, uint8_t *restrict d, + int n) +{ + for (int i = 0; i < n; ++i) + { + a[i * 8] = c[i * 8] + d[i * 8]; + a[i * 8 + 1] = c[i * 8] + d[i * 8 + 1]; + a[i * 8 + 2] = c[i * 8 + 2] + d[i * 8 + 2]; + a[i * 8 + 3] = c[i * 8 + 2] + d[i * 8 + 3]; + a[i * 8 + 4] = c[i * 8 + 4] + d[i * 8 + 4]; + a[i * 8 + 5] = c[i * 8 + 4] + d[i * 8 + 5]; + a[i * 8 + 6] = c[i * 8 + 6] + d[i * 8 + 6]; + a[i * 8 + 7] = c[i * 8 + 6] + d[i * 8 + 7]; + b[i * 8] = c[i * 8 + 1] + d[i * 8]; + b[i * 8 + 1] = c[i * 8 + 1] + d[i * 8 + 1]; + b[i * 8 + 2] = c[i * 8 + 3] + d[i * 8 + 2]; + b[i * 8 + 3] = c[i * 8 + 3] + d[i * 8 + 3]; + b[i * 8 + 4] = c[i * 8 + 5] + d[i * 8 + 4]; + b[i * 8 + 5] = c[i * 8 + 5] + d[i * 8 + 5]; + b[i * 8 + 6] = c[i * 8 + 7] + d[i * 8 + 6]; + b[i * 8 + 7] = c[i * 8 + 7] + d[i * 8 + 7]; + } +} + +/* { dg-final { scan-assembler {e8,m4} } } */ +/* { dg-final { scan-assembler-not {jr} } } */ +/* { dg-final { scan-assembler-times {ret} 1 } } * +/* { dg-final { scan-tree-dump-times "Maximum lmul = 4" 1 "vect" } } */ +/* { dg-final { scan-tree-dump-not "Maximum lmul = 2" "vect" } } */ +/* { dg-final { scan-tree-dump-not "Maximum lmul = 1" "vect" } } */