From patchwork Tue Sep 12 16:25:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 138297 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp527692vqx; Tue, 12 Sep 2023 09:27:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEN0SoKT9wm+J10p3vJuwSWrFk91SJW0i3zMgtG1vbcbf+QoNDi9aPFBc4Kq8WllqZtZ1T+ X-Received: by 2002:a50:ec8a:0:b0:523:2274:91cb with SMTP id e10-20020a50ec8a000000b00523227491cbmr138723edr.12.1694536073007; Tue, 12 Sep 2023 09:27:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694536072; cv=none; d=google.com; s=arc-20160816; b=xs+ho4+vNVfoxFTVCeZ7ThLNhZCFu7O66srNE6YOc7Dc5OmUURzZ8JMixYevY1IsLk iRSgnVt897A3juKbDGHskrYvVX8eY2y2Eu/AjYSb4ZhqpM8W4XC8HhBiYvMOoHadoeF/ fZpKsUWpmNy/RAa5keMmcgVKd49FYdEeAvRCDiO/wd9K97MzS4MHXdS8Ctbr8KGWI7ki kk11ZTe6m4AGWgoBiKabnedONB3HSdk2rfW/En9jJKznU7ED0fk5Nu8Xobr2J3Kw03Ig HqeCUDRzHxdwpWpBLXDZk6l9odF56h5pLG/dkq8zRgzUNq4mPxYEwYbCrZPUSw7ppyUz kmvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=mz/QCqhS3e1a8sBHlavh8jTt2fHT/81jqtdKRhOTKLg=; fh=x/Q0OlwHuvCZ3FpkiZPiUSvevOYVxUAi4aNnf76mUPQ=; b=gT8gKUIUyq2mqA/wGLkhR5F91RdVuVOyc6VbcZ8Bjf16bWel4OtI3W3PzUD1U9WSOy Lwa4V5IPaakKkNvafexJpqoDEwqgYzPcG7SVPM61WDWxv+d/G5vNvcb4umitwOBHQ3zA t9awRWuJjMsgY+XG6JtS/GjLc7Jg7qFjCw1cwAeuEtvBjPi9/3Ud1LtY8UDqygKpte1a YYW/Xlr0XDVTEqq3Zy0FO9JferzqVkOK3gitfyY+saGgRY/BRIQy4RhbawOjNh5awoKU 0uvkAH5X8Z7ABlXidwBJPpDXh3PjxK5qVGrTn9UM8hIkypJcZxlrNw90CqlkW6Ewem+U pAIg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id c13-20020aa7c98d000000b0052c98ac3c73si9136285edt.193.2023.09.12.09.27.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 09:27:52 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D95483853D3A for ; Tue, 12 Sep 2023 16:26:35 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgau2.qq.com (smtpbgau2.qq.com [54.206.34.216]) by sourceware.org (Postfix) with ESMTPS id 66F7B3857029 for ; Tue, 12 Sep 2023 16:25:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 66F7B3857029 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp84t1694535942txp5qmoc Received: from rios-cad122.hadoop.rioslab.org ( [58.60.1.26]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 13 Sep 2023 00:25:41 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: WfQrrMZtGtGGI89fuPLbG8wrj65w3IScrJYcZtddxhlWudwXfEmNGzQ2vL0Ke QY1LZy5fxZ1Ppr98p8yCj1d3nl1qCJ1Nqq+ydmZj3DKyCd3sHgp57zzZQAbYMhcrgtRC+1G +cObMovrTwrnXJGyhF/kdi/TCKg+QUEWNdlHPuobGfwI7UQqQLdGHrdRJmaurGad2i5+RID zSqxDD1/j51frQCq4vf1mpTaXb3+TsntGnFeWd+JXpvtptrFyhC7zioGlCuhe5eIN2/b3Oi YcHDFVzbJYa6f4Jl+qpSdF1s7PV3zrebRUm5ImnYVJD4W73TJKh5gUyqpLtxogSfLcIG3uf F6eHwxyKsQrgnOYCx7rqC8Zi4eW21JWf4wpzHGJChbopEjjAl/U2BOVx60bw5wFGwPHWgUI X-QQ-GoodBg: 2 X-BIZMAIL-ID: 5474063613046441319 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, kito.cheng@gmail.com, rdapp.gcc@gmail.com, palmer@rivosinc.com, jeffreyalaw@gmail.com, lehua.ding@rivai.ai Subject: [PATCH] RISC-V: Support cond vmulh.vv and vmulu.vv Date: Wed, 13 Sep 2023 00:25:40 +0800 Message-Id: <20230912162540.2001990-1-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-9.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776849857075978290 X-GMAIL-MSGID: 1776849857075978290 This patch adds combine patterns to combine vmulh[u].vv + vcond_mask to mask vmulh[u].vv. For vmulsu.vv, it can not be produced in midend currently. We will send another patch to take this issue. gcc/ChangeLog: * config/riscv/autovec-opt.md (*cond_3_highpart): New combine pattern. * config/riscv/autovec.md (smul3_highpart): Mrege smul and umul. (3_highpart): Merged pattern. (umul3_highpart): Mrege smul and umul. * config/riscv/vector-iterators.md (umul): New iterators. (UNSPEC_VMULHU): New iterators. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/cond/cond_mulh-1.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_mulh-2.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-1.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-2.c: New test. --- gcc/config/riscv/autovec-opt.md | 23 ++++++++++++- gcc/config/riscv/autovec.md | 22 ++---------- gcc/config/riscv/vector-iterators.md | 4 +++ .../riscv/rvv/autovec/cond/cond_mulh-1.c | 29 ++++++++++++++++ .../riscv/rvv/autovec/cond/cond_mulh-2.c | 30 ++++++++++++++++ .../riscv/rvv/autovec/cond/cond_mulh_run-1.c | 32 +++++++++++++++++ .../riscv/rvv/autovec/cond/cond_mulh_run-2.c | 34 +++++++++++++++++++ 7 files changed, 154 insertions(+), 20 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-2.c diff --git a/gcc/config/riscv/autovec-opt.md b/gcc/config/riscv/autovec-opt.md index 0d2721f0b29..552be48bf73 100644 --- a/gcc/config/riscv/autovec-opt.md +++ b/gcc/config/riscv/autovec-opt.md @@ -970,6 +970,28 @@ } [(set_attr "type" "vnshift")]) +;; Combine vmulh.vv/vmulhu.vv + vcond_mask +(define_insn_and_split "*cond_3_highpart" + [(set (match_operand:VFULLI 0 "register_operand") + (if_then_else:VFULLI + (match_operand: 1 "register_operand") + (mulh:VFULLI + (match_operand:VFULLI 2 "register_operand") + (match_operand:VFULLI 3 "register_operand")) + (match_operand:VFULLI 4 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] +{ + insn_code icode = code_for_pred_mulh (, mode); + rtx ops[] = {operands[0], operands[1], operands[2], operands[3], operands[4], + gen_int_mode (GET_MODE_NUNITS (mode), Pmode)}; + riscv_vector::expand_cond_len_binop (icode, ops); + DONE; +} +[(set_attr "type" "vector")]) + ;; ============================================================================= ;; Combine extend + binop to widen_binop ;; ============================================================================= @@ -1172,7 +1194,6 @@ } [(set_attr "type" "vfwmul")]) - ;; ============================================================================= ;; Misc combine patterns ;; ============================================================================= diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index e9dd40af935..b4ac22bb97b 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -1569,9 +1569,9 @@ ;; - vmulhu.vv ;; ------------------------------------------------------------------------- -(define_insn_and_split "smul3_highpart" +(define_insn_and_split "3_highpart" [(set (match_operand:VFULLI 0 "register_operand") - (smul_highpart:VFULLI + (mulh:VFULLI (match_operand:VFULLI 1 "register_operand") (match_operand:VFULLI 2 "register_operand")))] "TARGET_VECTOR && can_create_pseudo_p ()" @@ -1579,23 +1579,7 @@ "&& 1" [(const_int 0)] { - insn_code icode = code_for_pred_mulh (UNSPEC_VMULHS, mode); - riscv_vector::emit_vlmax_insn (icode, riscv_vector::BINARY_OP, operands); - DONE; -} -[(set_attr "type" "vimul")]) - -(define_insn_and_split "umul3_highpart" - [(set (match_operand:VFULLI 0 "register_operand") - (umul_highpart:VFULLI - (match_operand:VFULLI 1 "register_operand") - (match_operand:VFULLI 2 "register_operand")))] - "TARGET_VECTOR && can_create_pseudo_p ()" - "#" - "&& 1" - [(const_int 0)] -{ - insn_code icode = code_for_pred_mulh (UNSPEC_VMULHU, mode); + insn_code icode = code_for_pred_mulh (, mode); riscv_vector::emit_vlmax_insn (icode, riscv_vector::BINARY_OP, operands); DONE; } diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md index 2f7f7cbe08c..e70a9bc5c74 100644 --- a/gcc/config/riscv/vector-iterators.md +++ b/gcc/config/riscv/vector-iterators.md @@ -2354,6 +2354,10 @@ (define_code_iterator sat_int_plus_binop [ss_plus us_plus]) (define_code_iterator sat_int_minus_binop [ss_minus us_minus]) +(define_code_iterator mulh [smul_highpart umul_highpart]) +(define_code_attr mulh_table [(smul_highpart "smul") (umul_highpart "umul")]) +(define_code_attr MULH_UNSPEC [(smul_highpart "UNSPEC_VMULHS") (umul_highpart "UNSPEC_VMULHU")]) + (define_code_iterator any_widen_binop [plus minus mult]) (define_code_iterator plus_minus [plus minus]) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-1.c new file mode 100644 index 00000000000..6ff2dc580a4 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-1.c @@ -0,0 +1,29 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv_zvfh -mabi=ilp32d --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define DEF_LOOP(TYPE, WTYPE) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##WTYPE (TYPE *__restrict a, TYPE *__restrict b, \ + TYPE *__restrict c, TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + c[i] = pred[i] \ + ? (TYPE) (((WTYPE) a[i] * (WTYPE) b[i]) >> sizeof (TYPE) * 8) \ + : c[i]; \ + } + +#define TEST_ALL(T) \ + T (int8_t, int16_t) \ + T (int16_t, int32_t) \ + T (int32_t, int64_t) \ + T (uint8_t, uint16_t) \ + T (uint16_t, uint32_t) \ + T (uint32_t, uint64_t) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler-times {\tvmulh\.vv\tv[0-9]+,v[0-9]+,v[0-9]+,v0.t} 3 } } */ +/* { dg-final { scan-assembler-times {\tvmulhu\.vv\tv[0-9]+,v[0-9]+,v[0-9]+,v0.t} 3 } } */ +/* { dg-final { scan-assembler-not {\tvmerge\.vvm\t} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-2.c new file mode 100644 index 00000000000..c4c2b50f203 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh-2.c @@ -0,0 +1,30 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv_zvfh -mabi=ilp32d --param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include + +#define DEF_LOOP(TYPE1, WTYPE1, TYPE2, WTYPE2, TYPE3) \ + void __attribute__ ((noipa)) \ + test_##TYPE1##_##TYPE2 (TYPE1 *__restrict a, TYPE2 *__restrict b, \ + TYPE3 *__restrict c, TYPE3 *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + c[i] \ + = pred[i] \ + ? (TYPE3) (((WTYPE1) a[i] * (WTYPE2) b[i]) >> sizeof (TYPE1) * 8) \ + : c[i]; \ + } + +#define TEST_ALL(T) \ + T (int8_t, int16_t, uint8_t, uint16_t, int8_t) \ + T (uint8_t, uint16_t, int8_t, int16_t, int8_t) \ + T (int16_t, int32_t, uint16_t, uint32_t, int16_t) \ + T (uint16_t, uint32_t, int16_t, int32_t, int16_t) \ + T (int32_t, int64_t, uint32_t, uint64_t, int32_t) \ + T (uint32_t, uint64_t, int32_t, int64_t, int32_t) + +TEST_ALL (DEF_LOOP) + +/* FIXME: need midend support: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111381 */ +/* { dg-final { scan-assembler-times {\tvmulhsu\.vv\tv[0-9]+,v[0-9]+,v[0-9]+,v0.t} 6 {xfail riscv*-*-*} } } */ +/* { dg-final { scan-assembler-not {\tvmerge\.vvm\t} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-1.c new file mode 100644 index 00000000000..8260472add7 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-1.c @@ -0,0 +1,32 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_mulh-1.c" + +#define N 99 + +#define TEST_LOOP(TYPE, WTYPE) \ + { \ + TYPE a[N], b[N], c[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = i * i; \ + b[i] = ((i + 2) % 3) * (i + 1); \ + pred[i] = i % 3; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##WTYPE (a, b, c, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (c[i] \ + != (pred[i] \ + ? (TYPE) (((WTYPE) a[i] * (WTYPE) b[i]) >> sizeof (TYPE) * 8) \ + : c[i])) \ + __builtin_abort (); \ + } + +int +main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-2.c new file mode 100644 index 00000000000..24612ff84f7 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_mulh_run-2.c @@ -0,0 +1,34 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable -fno-vect-cost-model" } */ + +#include "cond_mulh-2.c" + +#define N 99 + +#define TEST_LOOP(TYPE1, WTYPE1, TYPE2, WTYPE2, TYPE3) \ + { \ + TYPE1 a[N]; \ + TYPE2 b[N]; \ + TYPE3 c[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = i * i; \ + b[i] = ((i + 2) % 3) * (i + 1); \ + pred[i] = i % 3; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE1##_##TYPE2 (a, b, c, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (c[i] \ + != (pred[i] ? (TYPE3) (((WTYPE1) a[i] * (WTYPE2) b[i]) \ + >> sizeof (TYPE1) * 8) \ + : c[i])) \ + __builtin_abort (); \ + } + +int +main () +{ + TEST_ALL (TEST_LOOP) + return 0; +}