From patchwork Thu Jun 1 07:31:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 101791 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp113165vqr; Thu, 1 Jun 2023 00:32:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6jekIbgLRw9RMUV71cQHSPQgoCAabqdVSO53mKQku80QUIrBhcxxHfMaKIUPum9o8qaCx8 X-Received: by 2002:a17:907:7216:b0:973:d80f:ea69 with SMTP id dr22-20020a170907721600b00973d80fea69mr8070038ejc.29.1685604749148; Thu, 01 Jun 2023 00:32:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685604749; cv=none; d=google.com; s=arc-20160816; b=hlhmAMDCZrYtQLPB0PNka2Q37vVFterEGMnAz8RlImj0b29psYEW/7jLtHvOsRIn4R saTaOmGEP+Tyz+Dnr6fmWDAJpaN5EskBUSaIAHCupmVMzAZv3p+nkEC9frcZSmuZ2x+K FPYq3o+oBMM16BepMgDSHSP1IoTpsYuUC9/JwLKSLnm5XiqqLpKZYT7RPlq/lrAlycS/ 2b0EBIODx5z2Z9cAVYKcfl3UgLzGEqi/JvvM0KZOoy0LgrMa6GCYUnluyqor8osqte5X PPE/Dj4b2xMF/097mM4MJBzMkji2v9KSjd/2U1BTMJ26xfa2mL9cI4sTweIum3zYCe0F u4ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=M/cZQ7cUYHN4MgzY0iJRVoNa55t5tNLxwDDvaKlIdJo=; b=jRd9w6sIFPAoImzitywwwmAqccr5aZQWXAkX5fqxuDuYf3VOhVS0p4tKUuvTPIWija 8jp2yeTp2UYwDuchhL0ENpjS04lvwVWGi32BEi3ZSBat17V74Tn4oS4XD7F0QapbzDxZ Zyxyo6NAio4ItANCrmTSbTTdQqOckbytN1YnR76rRGjdUeOteTY5M9i11/o+fyrwE5dy 4EiY7XiVKZ7mBlGeSDWvsxNnb0UCfwKbnioNSoyWxeDRc1SJFdh6qSvUxGIho/8+NtFT O4p7258RL/gg5IRY+f7ODdZ60MYj3X/DVQqepKpVwpKPKY6i6ybB2cBMGTwSyws3NLhJ B+EA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id ez4-20020a1709070bc400b00965ff5ddac8si4800074ejc.682.2023.06.01.00.32.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Jun 2023 00:32:29 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B53FE385771C for ; Thu, 1 Jun 2023 07:32:22 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg153.qq.com (smtpbg153.qq.com [13.245.218.24]) by sourceware.org (Postfix) with ESMTPS id 512FA3858CDB for ; Thu, 1 Jun 2023 07:31:50 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 512FA3858CDB Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp72t1685604702tf75p5wb Received: from server1.localdomain ( [58.60.1.22]) by bizesmtp.qq.com (ESMTP) with id ; Thu, 01 Jun 2023 15:31:40 +0800 (CST) X-QQ-SSF: 01400000000000F0R000000A0000000 X-QQ-FEAT: q+EIYT+FhZqX5HJc5PtbdmSfWN0BMDBN/QIUOG5eE/tBHKyhZniDAk6leNl0t 5XB8zJEEQhf0kPtOQfAOddvsCMSWZZsjiOgQAyjYaUou4RQhD6KC73dbqyH/PPLkFL8xNpM WL94/rO/5FhausExaCdAoYMSTlRLTEvAMhQwVl0xbvkjqME2Uxh2ta9R1p1ggTGBv02bjCV Yj3EvCBFoQiTSZouWe94pj6uBIBMq+h+eBhWv0bdMTyz5lN/B3BsTmDCXHLzK2Y21tDuWYy k8+ZyFRyZvZLrI5G9hShX0vBMbXbz7nK9QTYLEQtOKl6v1VTkcNs4IZrcPAPsDnbxnSi90w sTgCLmO+MsCsV8Oldg0WOdVx83Dkyejo1ZtIp+yDio89zrRZG7anDxoqQExEBj5QrDVL8KK lev6Guu7xsBdtE9QSllvWA== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 13437052013462881775 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, palmer@dabbelt.com, palmer@rivosinc.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH] RISC-V: Add pseudo vwmul.wv pattern to enhance vwmul.vv instruction optimizations Date: Thu, 1 Jun 2023 15:31:36 +0800 Message-Id: <20230601073136.193330-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767484685454703522?= X-GMAIL-MSGID: =?utf-8?q?1767484685454703522?= From: Juzhe-Zhong This patch is to enhance vwmul.vv combine optimizations. Consider this following code: void vwadd_int16_t_int8_t (int16_t *__restrict dst, int16_t *__restrict dst2, int16_t *__restrict dst3, int16_t *__restrict dst4, int8_t *__restrict a, int8_t *__restrict b, int8_t *__restrict a2, int8_t *__restrict b2, int n) { for (int i = 0; i < n; i++) { dst[i] = (int16_t) a[i] * (int16_t) b[i]; dst2[i] = (int16_t) a2[i] * (int16_t) b[i]; dst3[i] = (int16_t) a2[i] * (int16_t) a[i]; dst4[i] = (int16_t) a[i] * (int16_t) b2[i]; } } In such complicate case, the operand is not single used, used by multiple statements. GCC combine optimization will iterate the combination of the operands. First round -> combine one of the operand and change vsext + vmul into vwmul.wv Second round -> combine the other operand and change vwmul.wv into vwmul.vv Notice when I add a pseudo vwmul.wv pattern, it makes vwmulsu.vv testcase fail since GCC prefer such pattern order: (mul: (zero_extend) (sign_exted)) So change vwmulsu.vv instruction operands order. gcc/ChangeLog: * config/riscv/vector.md: Shift zero_extend and sign_extend order. * config/riscv/autovec-opt.md: New file. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/widen/widen-7.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen-complicate-3.c: New test. * gcc.target/riscv/rvv/autovec/widen/widen_run-7.c: New test. --- gcc/config/riscv/autovec-opt.md | 56 +++++++++++++++++++ gcc/config/riscv/vector.md | 9 +-- .../riscv/rvv/autovec/widen/widen-7.c | 27 +++++++++ .../rvv/autovec/widen/widen-complicate-3.c | 32 +++++++++++ .../riscv/rvv/autovec/widen/widen_run-7.c | 34 +++++++++++ 5 files changed, 154 insertions(+), 4 deletions(-) create mode 100644 gcc/config/riscv/autovec-opt.md create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-7.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-3.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-7.c diff --git a/gcc/config/riscv/autovec-opt.md b/gcc/config/riscv/autovec-opt.md new file mode 100644 index 00000000000..5b7dc9bef8c --- /dev/null +++ b/gcc/config/riscv/autovec-opt.md @@ -0,0 +1,56 @@ +;; Machine description for optimization of RVV auto-vectorization. +;; Copyright (C) 2023 Free Software Foundation, Inc. +;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies Ltd. + +;; This file is part of GCC. + +;; GCC is free software; you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation; either version 3, or (at your option) +;; any later version. + +;; GCC is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with GCC; see the file COPYING3. If not see +;; . + +;; We don't have vwmul.wv instruction like vwadd.wv in RVV. +;; This pattern is an intermediate RTL IR as a pseudo vwmul.wv to enhance +;; optimization of instructions combine. +(define_insn_and_split "@pred_single_widen_mul" + [(set (match_operand:VWEXTI 0 "register_operand" "=&vr,&vr") + (if_then_else:VWEXTI + (unspec: + [(match_operand: 1 "vector_mask_operand" "vmWc1,vmWc1") + (match_operand 5 "vector_length_operand" " rK, rK") + (match_operand 6 "const_int_operand" " i, i") + (match_operand 7 "const_int_operand" " i, i") + (match_operand 8 "const_int_operand" " i, i") + (reg:SI VL_REGNUM) + (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) + (mult:VWEXTI + (any_extend:VWEXTI + (match_operand: 4 "register_operand" " vr, vr")) + (match_operand:VWEXTI 3 "register_operand" " vr, vr")) + (match_operand:VWEXTI 2 "vector_merge_operand" " vu, 0")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + [(const_int 0)] + { + insn_code icode = code_for_pred_vf2 (, mode); + rtx tmp = gen_reg_rtx (mode); + rtx ops[] = {tmp, operands[4]}; + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, ops); + + emit_insn (gen_pred (MULT, mode, operands[0], operands[1], operands[2], + operands[3], tmp, operands[5], operands[6], + operands[7], operands[8])); + DONE; + } + [(set_attr "type" "viwmul") + (set_attr "mode" "")]) diff --git a/gcc/config/riscv/vector.md b/gcc/config/riscv/vector.md index c74dce89db6..bb333a9b410 100644 --- a/gcc/config/riscv/vector.md +++ b/gcc/config/riscv/vector.md @@ -24,7 +24,7 @@ ;; ;; - Intrinsics (https://github.com/riscv/rvv-intrinsic-doc) ;; - Auto-vectorization (autovec.md) -;; - Combine optimization (TBD) +;; - Optimization (autovec-opt.md) (include "vector-iterators.md") @@ -3207,10 +3207,10 @@ (reg:SI VL_REGNUM) (reg:SI VTYPE_REGNUM)] UNSPEC_VPREDICATE) (mult:VWEXTI - (sign_extend:VWEXTI - (match_operand: 3 "register_operand" " vr, vr")) (zero_extend:VWEXTI - (match_operand: 4 "register_operand" " vr, vr"))) + (match_operand: 4 "register_operand" " vr, vr")) + (sign_extend:VWEXTI + (match_operand: 3 "register_operand" " vr, vr"))) (match_operand:VWEXTI 2 "vector_merge_operand" " vu, 0")))] "TARGET_VECTOR" "vwmulsu.vv\t%0,%3,%4%p1" @@ -8422,3 +8422,4 @@ ) (include "autovec.md") +(include "autovec-opt.md") diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-7.c new file mode 100644 index 00000000000..cc43d9ba3fe --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-7.c @@ -0,0 +1,27 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define TEST_TYPE(TYPE1, TYPE2) \ + __attribute__ ((noipa)) void vwmul_##TYPE1_##TYPE2 (TYPE1 *__restrict dst, \ + TYPE2 *__restrict a, \ + TYPE1 *__restrict b, \ + int n) \ + { \ + for (int i = 0; i < n; i++) \ + dst[i] = ((TYPE1) a[i]) * b[i]; \ + } + +#define TEST_ALL() \ + TEST_TYPE (int16_t, int8_t) \ + TEST_TYPE (uint16_t, uint8_t) \ + TEST_TYPE (int32_t, int16_t) \ + TEST_TYPE (uint32_t, uint16_t) \ + TEST_TYPE (int64_t, int32_t) \ + TEST_TYPE (uint64_t, uint32_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvsext\.vf2} 3 } } */ +/* { dg-final { scan-assembler-times {\tvzext\.vf2} 3 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-3.c new file mode 100644 index 00000000000..e1fd79430c3 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen-complicate-3.c @@ -0,0 +1,32 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define TEST_TYPE(TYPE1, TYPE2) \ + __attribute__ ((noipa)) void vwadd_##TYPE1_##TYPE2 ( \ + TYPE1 *__restrict dst, TYPE1 *__restrict dst2, TYPE1 *__restrict dst3, \ + TYPE1 *__restrict dst4, TYPE2 *__restrict a, TYPE2 *__restrict b, \ + TYPE2 *__restrict a2, TYPE2 *__restrict b2, int n) \ + { \ + for (int i = 0; i < n; i++) \ + { \ + dst[i] = (TYPE1) a[i] * (TYPE1) b[i]; \ + dst2[i] = (TYPE1) a2[i] * (TYPE1) b[i]; \ + dst3[i] = (TYPE1) a2[i] * (TYPE1) a[i]; \ + dst4[i] = (TYPE1) a[i] * (TYPE1) b2[i]; \ + } \ + } + +#define TEST_ALL() \ + TEST_TYPE (int16_t, int8_t) \ + TEST_TYPE (uint16_t, uint8_t) \ + TEST_TYPE (int32_t, int16_t) \ + TEST_TYPE (uint32_t, uint16_t) \ + TEST_TYPE (int64_t, int32_t) \ + TEST_TYPE (uint64_t, uint32_t) + +TEST_ALL () + +/* { dg-final { scan-assembler-times {\tvwmul\.vv} 12 } } */ +/* { dg-final { scan-assembler-times {\tvwmulu\.vv} 12 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-7.c new file mode 100644 index 00000000000..4abddd5d718 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/widen/widen_run-7.c @@ -0,0 +1,34 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include +#include "widen-7.c" + +#define SZ 512 + +#define RUN(TYPE1, TYPE2, LIMIT) \ + TYPE2 a##TYPE2[SZ]; \ + TYPE1 b##TYPE1[SZ]; \ + TYPE1 dst##TYPE1[SZ]; \ + for (int i = 0; i < SZ; i++) \ + { \ + a##TYPE2[i] = LIMIT + i % LIMIT; \ + b##TYPE1[i] = LIMIT + i & LIMIT; \ + } \ + vwmul_##TYPE1_##TYPE2 (dst##TYPE1, a##TYPE2, b##TYPE1, SZ); \ + for (int i = 0; i < SZ; i++) \ + assert (dst##TYPE1[i] == (((TYPE1) a##TYPE2[i]) * b##TYPE1[i])); + +#define RUN_ALL() \ + RUN (int16_t, int8_t, -128) \ + RUN (uint16_t, uint8_t, 255) \ + RUN (int32_t, int16_t, -32768) \ + RUN (uint32_t, uint16_t, 65535) \ + RUN (int64_t, int32_t, -2147483648) \ + RUN (uint64_t, uint32_t, 4294967295) + +int +main () +{ + RUN_ALL () +}