From patchwork Thu Sep 21 08:20:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 142774 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp4689274vqi; Thu, 21 Sep 2023 01:21:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFnWfeFY/FLLyDlxK4hnWrvj9C7ghS2seAyebSwVbCpfHJBeGpcRHFXF2oXdeRBCu/64oek X-Received: by 2002:a17:906:739b:b0:9a1:da9a:f1c0 with SMTP id f27-20020a170906739b00b009a1da9af1c0mr4223610ejl.48.1695284478745; Thu, 21 Sep 2023 01:21:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695284478; cv=none; d=google.com; s=arc-20160816; b=JGq/fZdrKCMkQZI1t62E8igOUeOi15o9UHx8ZxWud45PBIHDesqpomPocDi66yw+QE 8etIJJak9TUdnsnWXhP7YVZNlJLi/tktve0pDT8IcP9pWg2itHf48KBUiKAGps8COZ6A fWmgqjkyYUoA2b4JEUphmWbr2qzjeCs9hvbcvTh2OhvsFrcjZh+sU+twCQ21s758K6Ko oTswU4tlv3HWMHNqiTd3rK+4s/dZgePOFSBhWjna+07V1MoSXJDk5SBKGvKz6u11WyhY tgUySz3qpqB9W/E2QDU3mM4KaC3Hfl9Wxd5bw9W0IuZbxb0kfzDMwR9KgeVGtArAXc+C 9WvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=cO8O7CtTHzJcEnsAEt6okNt2rU7Gqt53ONN7Xf+mpeA=; fh=12MRPJmZ1mgDpHqWoogMKqnaGRGM2b7lcuJroqfjJiw=; b=CjIboVlF+E+EyALOLeo22mAXIX7N216/TGdDADSXGHnClIktQDXb0uKvscWnmB0iPN hCa2jAr3i++Wy7VUoCQ/3dlyKsCZPQ41pnwQQCU0yRTEE5Z1+mXzxy35QC+Zp6t5njw0 yU/PYxAcAOQ8Xws80R+CxIpRe6KSyXtWshZSlXjPOvKB1X6SyjDoajhVyNUCHVccvHh8 s0QB6SuwwoydlOLet+roTnJaV3ZaknYepZn5chXSkFdvaMEtgio5Efd0mkxC8MTrXBAl vhV/BgTPdAy7qdyYqhyVWt1EBy3ucbJM2tzkzMYp0Ea3SVR8eREgS79vSS80OBx2J4ev kaMQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id h11-20020a170906854b00b00992d0de8760si726459ejy.911.2023.09.21.01.21.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Sep 2023 01:21:18 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 459913858017 for ; Thu, 21 Sep 2023 08:21:17 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgsg2.qq.com (smtpbgsg2.qq.com [54.254.200.128]) by sourceware.org (Postfix) with ESMTPS id 88A9A3858D3C for ; Thu, 21 Sep 2023 08:20:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 88A9A3858D3C Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp77t1695284420t7v1afuw Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Thu, 21 Sep 2023 16:20:18 +0800 (CST) X-QQ-SSF: 01400000000000G0V000000A0000000 X-QQ-FEAT: vrqOr+ppv0vZWJnp0bweQUgifVu5UnnSKCLix9gC6heGNbnzdDzZdntCV5gGc pXPq24+AyHe3X4dovZFk4/yEiX89zk735vjVypnnF2kgYaEUpkrri9Qc5S7UpVCMohGS8F+ Xjnf11AhglxFzgQm/kyVsnUDjgXfXylVcKAAb6VeMOqWC/rTkHnv2ApoPDO4bb1a23HQEwG NUdXmDR/e6OO7WP4gobJhueN45W2aMdPrKAZsZG4vfdiSmnjTUrDkKRmXSpUldYxcG6ZKD6 oGD+YBMqey5lfz1pMqlfE22Ppxe02vZYRs4rLMd6k2vQ09/LZtLwvWObypZa2esa7BToxcq vDN8H/QzQivc0Vzs/kT907zxHwJT1QPHqvQmbYr6fbllTZT+R/tCSC1l4AQ/tgEFUKoTX60 lR+lTcQOErQ= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 16415926014246469450 From: Juzhe-Zhong To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH] RISC-V: Enable undefined support for RVV auto-vectorization[PR110751] Date: Thu, 21 Sep 2023 16:20:17 +0800 Message-Id: <20230921082017.1456735-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-11.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777634617311488537 X-GMAIL-MSGID: 1777634617311488537 Now GCC middle-end can support undefined value which is traslated into (scratch:mode). This patch is to enable RISC-V backend undefine value in ELSE value of COND_LEN_xxx/COND_xxx. Consider this following case: __attribute__((noipa)) void vrem_int8_t (int8_t * __restrict dst, int8_t * __restrict a, int8_t * __restrict b, int n) { for (int i = 0; i < n; i++) dst[i] = a[i] % b[i]; } Before this patch: vrem_int8_t: ble a3,zero,.L5 vsetvli a5,zero,e8,m1,ta,ma vmv.v.i v4,0 ---> redundant. .L3: vsetvli a5,a3,e8,m1,tu,ma ---> should be TA. vmv1r.v v1,v4 ---> redudant. vle8.v v3,0(a1) vle8.v v2,0(a2) sub a3,a3,a5 vrem.vv v1,v3,v2 vse8.v v1,0(a0) add a1,a1,a5 add a2,a2,a5 add a0,a0,a5 bne a3,zero,.L3 .L5: ret After this patch: vrem_int8_t: ble a3,zero,.L5 .L3: vsetvli a5,a3,e8,m1,ta,ma vle8.v v1,0(a1) vle8.v v2,0(a2) sub a3,a3,a5 vrem.vv v1,v1,v2 vse8.v v1,0(a0) add a1,a1,a5 add a2,a2,a5 add a0,a0,a5 bne a3,zero,.L3 .L5: ret PR target/110751 gcc/ChangeLog: * config/riscv/autovec.md: Enable scratch rtx in ELSE operand. * config/riscv/predicates.md (autovec_else_operand): New predicate. * config/riscv/riscv-v.cc (get_else_operand): New function. (expand_cond_len_unop): Adapt ELSE value. (expand_cond_len_binop): Ditto. (expand_cond_len_ternop): Ditto. * config/riscv/riscv.cc (riscv_preferred_else_value): New function. (TARGET_PREFERRED_ELSE_VALUE): New targethook. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv-nofm.c: Adapt test. * gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv.c: Ditto. * gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv-nofm.c: Ditto. * gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv.c: Ditto. * gcc.target/riscv/rvv/autovec/binop/vrem-rv32gcv.c: Ditto. * gcc.target/riscv/rvv/autovec/binop/vrem-rv64gcv.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-1.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-10.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-11.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-12.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-2.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-3.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-4.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-5.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-6.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-7.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-8.c: Ditto. * gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-9.c: Ditto. --- gcc/config/riscv/autovec.md | 48 +++++++++---------- gcc/config/riscv/predicates.md | 4 ++ gcc/config/riscv/riscv-v.cc | 13 +++-- gcc/config/riscv/riscv.cc | 16 +++++++ .../rvv/autovec/binop/vdiv-rv32gcv-nofm.c | 6 +++ .../riscv/rvv/autovec/binop/vdiv-rv32gcv.c | 6 +++ .../rvv/autovec/binop/vdiv-rv64gcv-nofm.c | 6 +++ .../riscv/rvv/autovec/binop/vdiv-rv64gcv.c | 6 +++ .../riscv/rvv/autovec/binop/vrem-rv32gcv.c | 6 +++ .../riscv/rvv/autovec/binop/vrem-rv64gcv.c | 6 +++ .../riscv/rvv/autovec/ternop/ternop_nofm-1.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-10.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-11.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-12.c | 2 + .../riscv/rvv/autovec/ternop/ternop_nofm-2.c | 6 +-- .../riscv/rvv/autovec/ternop/ternop_nofm-3.c | 3 +- .../riscv/rvv/autovec/ternop/ternop_nofm-4.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-5.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-6.c | 1 + .../riscv/rvv/autovec/ternop/ternop_nofm-7.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-8.c | 4 +- .../riscv/rvv/autovec/ternop/ternop_nofm-9.c | 1 + 22 files changed, 105 insertions(+), 53 deletions(-) diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index 55c0a04df3b..f0f1abc4e82 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -1496,7 +1496,7 @@ (match_operand: 1 "vector_mask_operand") (any_int_unop:VI (match_operand:VI 2 "register_operand")) - (match_operand:VI 3 "register_operand")] + (match_operand:VI 3 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1512,7 +1512,7 @@ (match_operand: 1 "vector_mask_operand") (any_int_unop:VI (match_operand:VI 2 "register_operand")) - (match_operand:VI 3 "register_operand") + (match_operand:VI 3 "autovec_else_operand") (match_operand 4 "autovec_length_operand") (match_operand 5 "const_0_operand")] "TARGET_VECTOR" @@ -1534,7 +1534,7 @@ (match_operand: 1 "vector_mask_operand") (any_float_unop_nofrm:VF (match_operand:VF 2 "register_operand")) - (match_operand:VF 3 "register_operand")] + (match_operand:VF 3 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1550,7 +1550,7 @@ (match_operand: 1 "vector_mask_operand") (any_float_unop_nofrm:VF (match_operand:VF 2 "register_operand")) - (match_operand:VF 3 "register_operand") + (match_operand:VF 3 "autovec_else_operand") (match_operand 4 "autovec_length_operand") (match_operand 5 "const_0_operand")] "TARGET_VECTOR" @@ -1573,7 +1573,7 @@ (any_shift:VI (match_operand:VI 2 "register_operand") (match_operand:VI 3 "vector_shift_operand")) - (match_operand:VI 4 "register_operand")] + (match_operand:VI 4 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1590,7 +1590,7 @@ (any_shift:VI (match_operand:VI 2 "register_operand") (match_operand:VI 3 "vector_shift_operand")) - (match_operand:VI 4 "register_operand") + (match_operand:VI 4 "autovec_else_operand") (match_operand 5 "autovec_length_operand") (match_operand 6 "const_0_operand")] "TARGET_VECTOR" @@ -1614,7 +1614,7 @@ (any_int_binop_no_shift:VI (match_operand:VI 2 "") (match_operand:VI 3 "")) - (match_operand:VI 4 "register_operand")] + (match_operand:VI 4 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1631,7 +1631,7 @@ (any_int_binop_no_shift:VI (match_operand:VI 2 "") (match_operand:VI 3 "")) - (match_operand:VI 4 "register_operand") + (match_operand:VI 4 "autovec_else_operand") (match_operand 5 "autovec_length_operand") (match_operand 6 "const_0_operand")] "TARGET_VECTOR" @@ -1655,7 +1655,7 @@ (any_float_binop:VF (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand")) - (match_operand:VF 4 "register_operand")] + (match_operand:VF 4 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1672,7 +1672,7 @@ (any_float_binop:VF (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand")) - (match_operand:VF 4 "register_operand") + (match_operand:VF 4 "autovec_else_operand") (match_operand 5 "autovec_length_operand") (match_operand 6 "const_0_operand")] "TARGET_VECTOR" @@ -1694,7 +1694,7 @@ (any_float_binop_nofrm:VF (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand")) - (match_operand:VF 4 "register_operand")] + (match_operand:VF 4 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1711,7 +1711,7 @@ (any_float_binop_nofrm:VF (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand")) - (match_operand:VF 4 "register_operand") + (match_operand:VF 4 "autovec_else_operand") (match_operand 5 "autovec_length_operand") (match_operand 6 "const_0_operand")] "TARGET_VECTOR" @@ -1734,7 +1734,7 @@ (match_operand:VI 2 "register_operand") (match_operand:VI 3 "register_operand") (match_operand:VI 4 "register_operand") - (match_operand:VI 5 "register_operand")] + (match_operand:VI 5 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1751,7 +1751,7 @@ (match_operand:VI 2 "register_operand") (match_operand:VI 3 "register_operand") (match_operand:VI 4 "register_operand") - (match_operand:VI 5 "register_operand") + (match_operand:VI 5 "autovec_else_operand") (match_operand 6 "autovec_length_operand") (match_operand 7 "const_0_operand")] "TARGET_VECTOR" @@ -1767,7 +1767,7 @@ (match_operand:VI 2 "register_operand") (match_operand:VI 3 "register_operand") (match_operand:VI 4 "register_operand") - (match_operand:VI 5 "register_operand")] + (match_operand:VI 5 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1784,7 +1784,7 @@ (match_operand:VI 2 "register_operand") (match_operand:VI 3 "register_operand") (match_operand:VI 4 "register_operand") - (match_operand:VI 5 "register_operand") + (match_operand:VI 5 "autovec_else_operand") (match_operand 6 "autovec_length_operand") (match_operand 7 "const_0_operand")] "TARGET_VECTOR" @@ -1807,7 +1807,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand")] + (match_operand:VF 5 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1824,7 +1824,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand") + (match_operand:VF 5 "autovec_else_operand") (match_operand 6 "autovec_length_operand") (match_operand 7 "const_0_operand")] "TARGET_VECTOR" @@ -1840,7 +1840,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand")] + (match_operand:VF 5 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1857,7 +1857,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand") + (match_operand:VF 5 "autovec_else_operand") (match_operand 6 "autovec_length_operand") (match_operand 7 "const_0_operand")] "TARGET_VECTOR" @@ -1873,7 +1873,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand")] + (match_operand:VF 5 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1890,7 +1890,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand") + (match_operand:VF 5 "autovec_else_operand") (match_operand 6 "autovec_length_operand") (match_operand 7 "const_0_operand")] "TARGET_VECTOR" @@ -1906,7 +1906,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand")] + (match_operand:VF 5 "autovec_else_operand")] "TARGET_VECTOR" { /* Normalize into cond_len_* operations. */ @@ -1923,7 +1923,7 @@ (match_operand:VF 2 "register_operand") (match_operand:VF 3 "register_operand") (match_operand:VF 4 "register_operand") - (match_operand:VF 5 "register_operand") + (match_operand:VF 5 "autovec_else_operand") (match_operand 6 "autovec_length_operand") (match_operand 7 "const_0_operand")] "TARGET_VECTOR" diff --git a/gcc/config/riscv/predicates.md b/gcc/config/riscv/predicates.md index bf241890e7e..6b72a5f4e07 100644 --- a/gcc/config/riscv/predicates.md +++ b/gcc/config/riscv/predicates.md @@ -452,6 +452,10 @@ (ior (match_operand 0 "register_operand") (match_operand 0 "vector_undef_operand"))) +(define_predicate "autovec_else_operand" + (ior (match_operand 0 "register_operand") + (match_operand 0 "scratch_operand"))) + (define_predicate "vector_arith_operand" (ior (match_operand 0 "register_operand") (and (match_code "const_vector") diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index 64a71a128d4..366f0659817 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -3024,6 +3024,13 @@ expand_cond_len_op (unsigned icode, insn_flags op_type, rtx *ops, rtx len) emit_nonvlmax_insn (icode, insn_flags, ops, len); } +/* Return RVV_VUNDEF if the ELSE value is scratch rtx. */ +static rtx +get_else_operand (rtx op) +{ + return GET_CODE (op) == SCRATCH ? RVV_VUNDEF (GET_MODE (op)) : op; +} + /* Expand unary ops COND_LEN_*. */ void expand_cond_len_unop (unsigned icode, rtx *ops) @@ -3031,7 +3038,7 @@ expand_cond_len_unop (unsigned icode, rtx *ops) rtx dest = ops[0]; rtx mask = ops[1]; rtx src = ops[2]; - rtx merge = ops[3]; + rtx merge = get_else_operand (ops[3]); rtx len = ops[4]; rtx cond_ops[] = {dest, mask, merge, src}; @@ -3046,7 +3053,7 @@ expand_cond_len_binop (unsigned icode, rtx *ops) rtx mask = ops[1]; rtx src1 = ops[2]; rtx src2 = ops[3]; - rtx merge = ops[4]; + rtx merge = get_else_operand (ops[4]); rtx len = ops[5]; rtx cond_ops[] = {dest, mask, merge, src1, src2}; @@ -3218,7 +3225,7 @@ expand_cond_len_ternop (unsigned icode, rtx *ops) rtx src1 = ops[2]; rtx src2 = ops[3]; rtx src3 = ops[4]; - rtx merge = ops[5]; + rtx merge = get_else_operand (ops[5]); rtx len = ops[6]; rtx cond_ops[] = {dest, mask, src1, src2, src3, merge}; diff --git a/gcc/config/riscv/riscv.cc b/gcc/config/riscv/riscv.cc index 6158953098d..6e7a719e7a0 100644 --- a/gcc/config/riscv/riscv.cc +++ b/gcc/config/riscv/riscv.cc @@ -72,6 +72,7 @@ along with GCC; see the file COPYING3. If not see #include "gimple-expr.h" #include "tree-vectorizer.h" #include "gcse.h" +#include "tree-dfa.h" /* This file should be included last. */ #include "target-def.h" @@ -9585,6 +9586,18 @@ riscv_vectorize_create_costs (vec_info *vinfo, bool costing_for_scalar) return new vector_costs (vinfo, costing_for_scalar); } +/* Implement TARGET_PREFERRED_ELSE_VALUE. */ + +static tree +riscv_preferred_else_value (unsigned ifn, tree vectype, unsigned int nops, + tree *ops) +{ + if (riscv_v_ext_mode_p (TYPE_MODE (vectype))) + return get_or_create_ssa_default_def (cfun, create_tmp_var (vectype)); + + return default_preferred_else_value (ifn, vectype, nops, ops); +} + /* Initialize the GCC target structure. */ #undef TARGET_ASM_ALIGNED_HI_OP #define TARGET_ASM_ALIGNED_HI_OP "\t.half\t" @@ -9901,6 +9914,9 @@ riscv_vectorize_create_costs (vec_info *vinfo, bool costing_for_scalar) #undef TARGET_VECTORIZE_CREATE_COSTS #define TARGET_VECTORIZE_CREATE_COSTS riscv_vectorize_create_costs +#undef TARGET_PREFERRED_ELSE_VALUE +#define TARGET_PREFERRED_ELSE_VALUE riscv_preferred_else_value + struct gcc_target targetm = TARGET_INITIALIZER; #include "gt-riscv.h" diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv-nofm.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv-nofm.c index 5ce2e57e265..f7d77047ad1 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv-nofm.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv-nofm.c @@ -10,3 +10,9 @@ /* { dg-final { scan-tree-dump-times "\.COND_LEN_DIV" 16 "optimized" } } */ /* { dg-final { scan-tree-dump-times "\.COND_LEN_RDIV" 6 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv1r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv2r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv4r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv8r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.i} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv.c index 9b984dda452..bb421fa7134 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv32gcv.c @@ -12,3 +12,9 @@ /* { dg-final { scan-assembler-times {\tvfmul\.vv} 3 } } */ /* { dg-final { scan-tree-dump-times "\.COND_LEN_DIV" 16 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv1r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv2r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv4r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv8r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.i} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv-nofm.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv-nofm.c index 7b1aa28e45e..0dd4df6a5c5 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv-nofm.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv-nofm.c @@ -10,3 +10,9 @@ /* { dg-final { scan-tree-dump-times "\.COND_LEN_DIV" 16 "optimized" } } */ /* { dg-final { scan-tree-dump-times "\.COND_LEN_RDIV" 6 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv1r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv2r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv4r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv8r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.i} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv.c index ca4d23bb1ed..9764cc3f1fd 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vdiv-rv64gcv.c @@ -12,3 +12,9 @@ /* { dg-final { scan-assembler-times {\tvfmul\.vv} 3 } } */ /* { dg-final { scan-tree-dump-times "\.COND_LEN_DIV" 16 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv1r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv2r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv4r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv8r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.i} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv32gcv.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv32gcv.c index 6d0493b3194..7628f4a3d26 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv32gcv.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv32gcv.c @@ -5,3 +5,9 @@ /* { dg-final { scan-assembler-times {\tvrem\.vv} 8 } } */ /* { dg-final { scan-assembler-times {\tvremu\.vv} 8 } } */ /* { dg-final { scan-tree-dump-times "\.COND_LEN_MOD" 16 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv1r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv2r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv4r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv8r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.i} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv64gcv.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv64gcv.c index 24b2bc81a90..8af9a8b5745 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv64gcv.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/binop/vrem-rv64gcv.c @@ -6,3 +6,9 @@ /* { dg-final { scan-assembler-times {\tvrem\.vv} 8 } } */ /* { dg-final { scan-assembler-times {\tvremu\.vv} 8 } } */ /* { dg-final { scan-tree-dump-times "\.COND_LEN_MOD" 16 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv1r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv2r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv4r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv8r\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.v} } } */ +/* { dg-final { scan-assembler-not {\tvmv\.v\.i} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-1.c index 52861ee104d..c5fab3f1f38 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-1.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-1.c @@ -3,7 +3,5 @@ #include "ternop-1.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FMA" 3 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-10.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-10.c index 01a5e995253..a65c398cba3 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-10.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-10.c @@ -3,7 +3,5 @@ #include "ternop-10.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FNMS" 3 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-11.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-11.c index 19671424660..9725cfad7ca 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-11.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-11.c @@ -3,7 +3,5 @@ #include "ternop-11.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FNMS" 3 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-12.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-12.c index f2c2c735db9..97be71c4bd2 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-12.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-12.c @@ -4,3 +4,5 @@ #include "ternop-12.c" /* { dg-final { scan-tree-dump-times "COND_LEN_FNMS" 3 "optimized" } } */ +/* { dg-final { scan-assembler-times {\tvmv} 3 } } */ + diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-2.c index ba07596046b..965365da4bb 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-2.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-2.c @@ -4,8 +4,6 @@ #include "ternop-2.c" /* { dg-final { scan-assembler-times {\tvmacc\.vv} 8 } } */ -/* { dg-final { scan-assembler-times {\tvfmacc\.vv} 9 } } */ -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ +/* { dg-final { scan-assembler-times {\tvfma[c-d][c-d]\.vv} 9 } } */ /* { dg-final { scan-tree-dump-times "COND_LEN_FMA" 9 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-3.c index bc44644baf3..de6d40431f8 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-3.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-3.c @@ -4,5 +4,6 @@ #include "ternop-3.c" /* { dg-final { scan-assembler-times {\tvmacc\.vv} 8 } } */ -/* { dg-final { scan-assembler-times {\tvfmacc\.vv} 9 } } */ +/* { dg-final { scan-assembler-times {\tvfma[c-d][c-d]\.vv} 9 } } */ /* { dg-final { scan-tree-dump-times "COND_LEN_FMA" 9 "optimized" } } */ +/* { dg-final { scan-assembler-times {\tvmv} 11 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-4.c index 96dd8a0ddca..4d73a541b0b 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-4.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-4.c @@ -3,7 +3,5 @@ #include "ternop-4.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FNMA" 3 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-5.c index 2664efaeef5..6fa28a23f3f 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-5.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-5.c @@ -3,7 +3,5 @@ #include "ternop-5.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FNMA" 3 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-6.c index 9f3d1204fc5..33faf0582a7 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-6.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-6.c @@ -4,3 +4,4 @@ #include "ternop-6.c" /* { dg-final { scan-tree-dump-times "COND_LEN_FNMA" 3 "optimized" } } */ +/* { dg-final { scan-assembler-times {\tvmv} 11 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-7.c index 8c07f1941ec..44807993c33 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-7.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-7.c @@ -3,7 +3,5 @@ #include "ternop-7.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FMS" 3 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-8.c index e0cbc53227a..c89f5836bcb 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-8.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-8.c @@ -3,7 +3,5 @@ #include "ternop-8.c" -/* TODO: we don't have undefine IR for COND_LEN_* operations, - which will produce redundant move instructions here. - Will add assembler-not check of 'vmv' instructions in the future. */ /* { dg-final { scan-tree-dump-times "COND_LEN_FMS" 9 "optimized" } } */ +/* { dg-final { scan-assembler-not {\tvmv} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-9.c index f080d3a3cb2..2de649b1db8 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-9.c +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/ternop/ternop_nofm-9.c @@ -4,3 +4,4 @@ #include "ternop-9.c" /* { dg-final { scan-tree-dump-times "COND_LEN_FMS" 9 "optimized" } } */ +/* { dg-final { scan-assembler-times {\tvmv} 3 } } */