From patchwork Tue Jun 6 11:46:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 103738 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3330055vqr; Tue, 6 Jun 2023 04:48:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6sEVTV0BJAKeoRYopUchU6Rup/E00QKyuRMGd0Gvw32UK29QgGYgar0l+iiSqRRx7b7rjk X-Received: by 2002:a17:907:7f10:b0:974:4f34:b04c with SMTP id qf16-20020a1709077f1000b009744f34b04cmr2346729ejc.41.1686052106824; Tue, 06 Jun 2023 04:48:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686052106; cv=none; d=google.com; s=arc-20160816; b=fclK6Eu4hkkaiJXqslWRolILirNOZ1oCtTfddM2rU+qNRZK0H7O/dqOTEYArUq7PoJ L72dKhS6dFLBtqFTR25zLTUDl1OqAXv2b3awWpfmGaf6RumR7fOswlmfQ9mW8Mghzdj1 t6CO1BF4tBjJGPKUd2ajcikQJmzDuYLbx3pLp1Ztlhhe5NEBvGuXPlQFr5IOlGpZRd52 Xwte1nncnHsRvGJmQyHVNOHVQOVkZ48qhZQJpqi2MneVX6oo/yT74/7gtUZEgGnJgcFa 2QtA034fQRNkKEAkPLSdiYN1TnRszdnQlCfgTZ4ZD+hh7287lD3Sw7r4NY6YrtqtrtRn GHPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=cvL5moQaj1ucmgK4qwEEBr8lXidlbxFknwe3kMbCT1o=; b=Vj00mSVmDPvERA1loR/y05d94Snhix8kQyC+D1I5Nsp1t1FfPSMwnv1t/ozo5e1Hqw qSO7SjAbQykM+hAtKCXH/RZbmUNitFo5jZOeYDy7aFCxiL5KtOXvodwRjXD6/1wDBHIJ ZuZjG1nyd/7EpDeGax9EHTTurQcLhuCgr1jHOAOLyqIoSbqcdrCpu9oWY/ACig1rbmHa 7TMsDtuvySromNv86PdjZAOXMzJGdY75RmYXeQHRMD6aQbZrh06JNLExXnIf98i5xsVT /fEN5XwzAtXLSvTD/bjA5P9x0As0OZFV2N7Emq5xWE+6yuo8PRXH6p0NNlVRCSOVf2JH Mdhw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id ho6-20020a1709070e8600b00977c8ca1c6asi5282508ejc.729.2023.06.06.04.48.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 04:48:26 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B604A3846440 for ; Tue, 6 Jun 2023 11:47:09 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgau1.qq.com (smtpbgau1.qq.com [54.206.16.166]) by sourceware.org (Postfix) with ESMTPS id 34C6F38582BC for ; Tue, 6 Jun 2023 11:46:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 34C6F38582BC Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp75t1686051995t0cf4pc8 Received: from rios-cad5.localdomain ( [58.60.1.11]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 06 Jun 2023 19:46:34 +0800 (CST) X-QQ-SSF: 01400000000000F0S000000A0000000 X-QQ-FEAT: 3M0okmaRx3jnVbcp/vXF+d+npf48OMYVaT2R/tOQgQxRG0Uow63cJeP7zQO14 ShJ2WPptajwLRGbHb7cSZjm3//ji7QcDx88X/aI4gSi2uopr7eRj6tPRHukF8pqDPEZpvVP 3uXGdw4Mqa8Ooh0thgwGlAxyxkNQEm1cliSZxtwTTaGGUQRTB0lcCwXjmgfbOTJ8xkdBUXZ ozajlz1vYlxsM7qA1WWyPxtjMMdoBf2GdzNIFIkQphwp0+BgZajNtBIDrf2QARE/iHaQ73Q I43Cls30DOmGKbNca2PA4M9UmTRtMUznns7v4T5SCs5NSnGADc3mmCl4M5NOFmiqNMfHITS 1A8nEeVpagRG0wRFgj7L4fvPKPWCZKK7EDncXH2p1C4AZ42JQeQ4O3rkv7CwRi8TM7AnP/k L/uD17W5DBr8uiTZt2Etvg== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 6662725065064746320 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: kito.cheng@sifive.com, palmer@rivosinc.com, rdapp.gcc@gmail.com, jeffreyalaw@gmail.com, Juzhe-Zhong Subject: [PATCH V2] RISC-V: Add RVV vwmacc/vwmaccu/vwmaccsu combine lowering optmization Date: Tue, 6 Jun 2023 19:46:30 +0800 Message-Id: <20230606114632.1629751-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767946937790470274?= X-GMAIL-MSGID: =?utf-8?q?1767953773927497936?= From: Juzhe-Zhong Fix according to comments from Robin of V1 patch. This patch add combine optimization for following case: __attribute__ ((noipa)) void vwmaccsu (int16_t *__restrict dst, int8_t *__restrict a, uint8_t *__restrict b, int n) { for (int i = 0; i < n; i++) dst[i] += (int16_t) a[i] * (int16_t) b[i]; } Before this patch: ... vsext.vf2 vzext.vf2 vmadd.vv .. After this patch: ... vwmaccsu.vv ... gcc/ChangeLog: * config/riscv/autovec-opt.md (*_fma): New pattern. (*single_mult_plus): Ditto. (*double_mult_plus): Ditto. (*sign_zero_extend_fma): Ditto. (*zero_sign_extend_fma): Ditto. * config/riscv/riscv-protos.h (enum insn_type): New enum. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/widen/widen-8.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen-9.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen-complicate-5.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen-complicate-6.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen_run-8.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen_run-9.c: New test. --- gcc/config/riscv/autovec-opt.md | 162 ++++++++++++++++++ gcc/config/riscv/riscv-protos.h | 1 + .../riscv/rvv/autovec/widen/widen-8.c | 27 +++ .../riscv/rvv/autovec/widen/widen-9.c | 23 +++ .../rvv/autovec/widen/widen-complicate-5.c | 32 ++++ .../rvv/autovec/widen/widen-complicate-6.c | 30 ++++ .../riscv/rvv/autovec/widen/widen_run-8.c | 38 ++++ .../riscv/rvv/autovec/widen/widen_run-9.c | 35 ++++ 8 files changed, 348 insertions(+) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-8.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-9.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-5.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-6.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-8.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-9.c diff --git a/gcc/config/riscv/autovec-opt.md b/gcc/config/riscv/autovec-opt.md index f6052b50572..1c36b5f56be 100644 --- a/gcc/config/riscv/autovec-opt.md +++ b/gcc/config/riscv/autovec-opt.md @@ -170,3 +170,165 @@ } [(set_attr "type" "vmalu") (set_attr "mode" "")]) + +;; ========================================================================= +;; == Widening Ternary arithmetic +;; ========================================================================= + +;; ------------------------------------------------------------------------- +;; ---- [INT] VWMACC +;; ------------------------------------------------------------------------- +;; Includes: +;; - vwmacc.vv +;; - vwmaccu.vv +;; ------------------------------------------------------------------------- + +;; Combine ext + ext + fma ===> widen fma. +;; Most of circumstantces, LoopVectorizer will generate the following IR: +;; vect__8.64_40 = (vector([4,4]) int) vect__7.63_41; +;; vect__11.68_35 = (vector([4,4]) int) vect__10.67_36; +;; vect__13.70_33 = .FMA (vect__11.68_35, vect__8.64_40, vect__4.60_45); +(define_insn_and_split "*_fma" + [(set (match_operand:VWEXTI 0 "register_operand") + (plus:VWEXTI + (mult:VWEXTI + (any_extend:VWEXTI + (match_operand: 2 "register_operand")) + (any_extend:VWEXTI + (match_operand: 3 "register_operand"))) + (match_operand:VWEXTI 1 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] + { + rtx ops[] = {operands[0], operands[1], operands[2], operands[3]}; + riscv_vector::emit_vlmax_ternary_insn (code_for_pred_widen_mul_plus (, mode), + riscv_vector::RVV_WIDEN_TERNOP, ops); + DONE; + } + [(set_attr "type" "viwmuladd") + (set_attr "mode" "")]) + +;; This helps to match ext + fma to enhance the combine optimizations. +(define_insn_and_split "*single_mult_plus" + [(set (match_operand:VWEXTI 0 "register_operand") + (plus:VWEXTI + (mult:VWEXTI + (any_extend:VWEXTI + (match_operand: 2 "register_operand")) + (match_operand:VWEXTI 3 "register_operand")) + (match_operand:VWEXTI 1 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] + { + insn_code icode = code_for_pred_vf2 (, mode); + rtx tmp = gen_reg_rtx (mode); + rtx ext_ops[] = {tmp, operands[2]}; + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, ext_ops); + + rtx dst = expand_ternary_op (mode, fma_optab, tmp, operands[3], + operands[1], operands[0], 0); + emit_move_insn (operands[0], dst); + DONE; + } + [(set_attr "type" "viwmuladd") + (set_attr "mode" "")]) + +;; Combine ext + ext + mult + plus ===> widen fma. +;; We have some special cases generated by LoopVectorizer: +;; vect__8.18_46 = (vector([8,8]) signed short) vect__7.17_47; +;; vect__11.22_41 = (vector([8,8]) signed short) vect__10.21_42; +;; vect__12.23_40 = vect__11.22_41 * vect__8.18_46; +;; vect__14.25_38 = vect__13.24_39 + vect__5.14_51; +;; This situation doesn't generate FMA IR. +(define_insn_and_split "*double_mult_plus" + [(set (match_operand:VWEXTI 0 "register_operand") + (if_then_else:VWEXTI + (unspec: + [(match_operand: 1 "vector_mask_operand") + (match_operand 6 "vector_length_operand") + (match_operand 7 "const_int_operand") + (match_operand 8 "const_int_operand") + (match_operand 9 "const_int_operand") + (reg:SI VL_REGNUM) + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) + (plus:VWEXTI + (if_then_else:VWEXTI + (unspec: + [(match_dup 1) + (match_dup 6) + (match_dup 7) + (match_dup 8) + (match_dup 9) + (reg:SI VL_REGNUM) + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) + (mult:VWEXTI + (any_extend:VWEXTI + (match_operand: 4 "register_operand")) + (any_extend:VWEXTI + (match_operand: 5 "register_operand"))) + (match_operand:VWEXTI 2 "vector_undef_operand")) + (match_operand:VWEXTI 3 "register_operand")) + (match_dup 2)))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] + { + emit_insn (gen_pred_widen_mul_plus (, mode, operands[0], + operands[1], operands[3], operands[4], + operands[5], operands[6], operands[7], + operands[8], operands[9])); + DONE; + } + [(set_attr "type" "viwmuladd") + (set_attr "mode" "")]) + +;; Combine sign_extend + zero_extend + fma ===> widen fma (su). +(define_insn_and_split "*sign_zero_extend_fma" + [(set (match_operand:VWEXTI 0 "register_operand") + (plus:VWEXTI + (mult:VWEXTI + (sign_extend:VWEXTI + (match_operand: 2 "register_operand")) + (zero_extend:VWEXTI + (match_operand: 3 "register_operand"))) + (match_operand:VWEXTI 1 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] + { + riscv_vector::emit_vlmax_ternary_insn (code_for_pred_widen_mul_plussu (mode), + riscv_vector::RVV_WIDEN_TERNOP, operands); + DONE; + } + [(set_attr "type" "viwmuladd") + (set_attr "mode" "")]) + +;; This helps to match zero_extend + sign_extend + fma +;; to enhance the combine optimizations. +(define_insn_and_split "*zero_sign_extend_fma" + [(set (match_operand:VWEXTI 0 "register_operand") + (plus:VWEXTI + (mult:VWEXTI + (zero_extend:VWEXTI + (match_operand: 2 "register_operand")) + (sign_extend:VWEXTI + (match_operand: 3 "register_operand"))) + (match_operand:VWEXTI 1 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] + { + rtx ops[] = {operands[0], operands[1], operands[3], operands[2]}; + riscv_vector::emit_vlmax_ternary_insn (code_for_pred_widen_mul_plussu (mode), + riscv_vector::RVV_WIDEN_TERNOP, ops); + DONE; + } + [(set_attr "type" "viwmuladd") + (set_attr "mode" "")]) diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h index 27ecd16e496..b311b937f17 100644 --- a/gcc/config/riscv/riscv-protos.h +++ b/gcc/config/riscv/riscv-protos.h @@ -143,6 +143,7 @@ enum insn_type RVV_CMP_MU_OP = RVV_CMP_OP + 2, /* +2 means mask and maskoff operand. */ RVV_UNOP_MU = RVV_UNOP + 2, /* Likewise. */ RVV_TERNOP = 5, + RVV_WIDEN_TERNOP = 4, RVV_SCALAR_MOV_OP = 4, /* +1 for VUNDEF according to vector.md. */ }; enum vlmul_type diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-8.c new file mode 100644 index 00000000000..f3ca07c02e0 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-8.c @@ -0,0 +1,27 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define TEST_TYPE(TYPE1, TYPE2) \ + __attribute__ ((noipa)) void vwmacc_##TYPE1_##TYPE2 (TYPE1 *__restrict dst, \ + TYPE2 *__restrict a, \ + TYPE2 *__restrict b, \ + int n) \ + { \ + for (int i = 0; i < n; i++) \ + dst[i] += (TYPE1) a[i] * (TYPE1) b[i]; \ + } + +#define TEST_ALL() \ + TEST_TYPE (int16_t, int8_t) \ + TEST_TYPE (uint16_t, uint8_t) \ + TEST_TYPE (int32_t, int16_t) \ + TEST_TYPE (uint32_t, uint16_t) \ + TEST_TYPE (int64_t, int32_t) \ + TEST_TYPE (uint64_t, uint32_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvwmacc\.vv} 3 } } */ +/* { dg-final { scan-assembler-times {\tvwmaccu\.vv} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-9.c new file mode 100644 index 00000000000..969a1e8f80c --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-9.c @@ -0,0 +1,23 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define TEST_TYPE(TYPE1, TYPE2, TYPE3) \ + __attribute__ ((noipa)) void vwmacc_##TYPE1_##TYPE2 (TYPE1 *__restrict dst, \ + TYPE2 *__restrict a, \ + TYPE3 *__restrict b, \ + int n) \ + { \ + for (int i = 0; i < n; i++) \ + dst[i] += (TYPE1) a[i] * (TYPE1) b[i]; \ + } + +#define TEST_ALL() \ + TEST_TYPE (int16_t, int8_t, uint8_t) \ + TEST_TYPE (int32_t, int16_t, uint16_t) \ + TEST_TYPE (int64_t, int32_t, uint32_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvwmaccsu\.vv} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-5.c new file mode 100644 index 00000000000..187b6db21fd --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-5.c @@ -0,0 +1,32 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define TEST_TYPE(TYPE1, TYPE2) \ + __attribute__ ((noipa)) void vwadd_##TYPE1_##TYPE2 ( \ + TYPE1 *__restrict dst, TYPE1 *__restrict dst2, TYPE1 *__restrict dst3, \ + TYPE1 *__restrict dst4, TYPE2 *__restrict a, TYPE2 *__restrict b, \ + TYPE2 *__restrict a2, TYPE2 *__restrict b2, int n) \ + { \ + for (int i = 0; i < n; i++) \ + { \ + dst[i] += (TYPE1) a[i] * (TYPE1) b[i]; \ + dst2[i] += (TYPE1) a2[i] * (TYPE1) b[i]; \ + dst3[i] += (TYPE1) a2[i] * (TYPE1) a[i]; \ + dst4[i] += (TYPE1) a[i] * (TYPE1) b2[i]; \ + } \ + } + +#define TEST_ALL() \ + TEST_TYPE (int16_t, int8_t) \ + TEST_TYPE (uint16_t, uint8_t) \ + TEST_TYPE (int32_t, int16_t) \ + TEST_TYPE (uint32_t, uint16_t) \ + TEST_TYPE (int64_t, int32_t) \ + TEST_TYPE (uint64_t, uint32_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvwmacc\.vv} 12 } } */ +/* { dg-final { scan-assembler-times {\tvwmaccu\.vv} 12 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-6.c new file mode 100644 index 00000000000..fa56f21aa81 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-6.c @@ -0,0 +1,30 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define TEST_TYPE(TYPE1, TYPE2, TYPE3) \ + __attribute__ ((noipa)) void vwadd_##TYPE1_##TYPE2 ( \ + TYPE1 *__restrict dst, TYPE1 *__restrict dst2, TYPE1 *__restrict dst3, \ + TYPE1 *__restrict dst4, TYPE2 *__restrict a, TYPE3 *__restrict b, \ + TYPE3 *__restrict a2, TYPE2 *__restrict b2, int n) \ + { \ + for (int i = 0; i < n; i++) \ + { \ + dst[i] += (TYPE1) a[i] * (TYPE1) b[i]; \ + dst2[i] += (TYPE1) a2[i] * (TYPE1) b[i]; \ + dst3[i] += (TYPE1) a2[i] * (TYPE1) a[i]; \ + dst4[i] += (TYPE1) a[i] * (TYPE1) b2[i]; \ + } \ + } + +#define TEST_ALL() \ + TEST_TYPE (int16_t, int8_t, uint8_t) \ + TEST_TYPE (int32_t, int16_t, uint16_t) \ + TEST_TYPE (int64_t, int32_t, uint32_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvwmaccsu\.vv} 6 } } */ +/* { dg-final { scan-assembler-times {\tvwmacc\.vv} 3 } } */ +/* { dg-final { scan-assembler-times {\tvwmaccu\.vv} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-8.c new file mode 100644 index 00000000000..f4840d30dc2 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-8.c @@ -0,0 +1,38 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include +#include "widen-8.c" + +#define SZ 512 + +#define RUN(TYPE1, TYPE2, LIMIT) \ + TYPE2 a##TYPE2[SZ]; \ + TYPE2 b##TYPE2[SZ]; \ + TYPE1 dst##TYPE1[SZ]; \ + TYPE1 dst2##TYPE1[SZ]; \ + for (int i = 0; i < SZ; i++) \ + { \ + a##TYPE2[i] = LIMIT + i % 8723; \ + b##TYPE2[i] = LIMIT + i & 1964; \ + dst##TYPE1[i] = LIMIT + i & 628; \ + dst2##TYPE1[i] = LIMIT + i & 628; \ + } \ + vwmacc_##TYPE1_##TYPE2 (dst##TYPE1, a##TYPE2, b##TYPE2, SZ); \ + for (int i = 0; i < SZ; i++) \ + assert (dst##TYPE1[i] \ + == ((TYPE1) a##TYPE2[i] * (TYPE1) b##TYPE2[i]) + dst2##TYPE1[i]); + +#define RUN_ALL() \ + RUN (int16_t, int8_t, -128) \ + RUN (uint16_t, uint8_t, 255) \ + RUN (int32_t, int16_t, -32768) \ + RUN (uint32_t, uint16_t, 65535) \ + RUN (int64_t, int32_t, -2147483648) \ + RUN (uint64_t, uint32_t, 4294967295) + +int +main () +{ + RUN_ALL () +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-9.c new file mode 100644 index 00000000000..2caa09a2c5a --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-9.c @@ -0,0 +1,35 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include +#include "widen-9.c" + +#define SZ 512 + +#define RUN(TYPE1, TYPE2, TYPE3, LIMIT) \ + TYPE2 a##TYPE2[SZ]; \ + TYPE3 b##TYPE3[SZ]; \ + TYPE1 dst##TYPE1[SZ]; \ + TYPE1 dst2##TYPE1[SZ]; \ + for (int i = 0; i < SZ; i++) \ + { \ + a##TYPE2[i] = LIMIT + i % 8723; \ + b##TYPE3[i] = LIMIT + i & 1964; \ + dst##TYPE1[i] = LIMIT + i & 728; \ + dst2##TYPE1[i] = LIMIT + i & 728; \ + } \ + vwmacc_##TYPE1_##TYPE2 (dst##TYPE1, a##TYPE2, b##TYPE3, SZ); \ + for (int i = 0; i < SZ; i++) \ + assert (dst##TYPE1[i] \ + == ((TYPE1) a##TYPE2[i] * (TYPE1) b##TYPE3[i]) + dst2##TYPE1[i]); + +#define RUN_ALL() \ + RUN (int16_t, int8_t, uint8_t, -128) \ + RUN (int32_t, int16_t, uint16_t, -32768) \ + RUN (int64_t, int32_t, uint32_t, -2147483648) + +int +main () +{ + RUN_ALL () +}