From patchwork Mon Sep 4 04:49:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 137438 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp890732vqo; Sun, 3 Sep 2023 21:49:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFYCB+9NufKdI4G9SlfZHToCeYn2stOBh0U3FsYZ7akKipmWI96TOWQLoSfUbDqLdxMthph X-Received: by 2002:aa7:dcd2:0:b0:522:6e3f:b65 with SMTP id w18-20020aa7dcd2000000b005226e3f0b65mr5672434edu.33.1693802988453; Sun, 03 Sep 2023 21:49:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693802988; cv=none; d=google.com; s=arc-20160816; b=V2wBH0aSHVN7wf1x+Oosz78AcIvM1SONC+7Ova+SdnXuXV9Gopae3h3cTyW270eDCW FpTEBlsBc+3Ah5I3PePNX7WY/coTKIK5IRIGk8EoprNkBQNhvs539wtRiZA7B4DY16h3 TjgMO/WehsfjlS1bb8Cte5oo2E+RGNC/wuG3y/+6BkV3DfNNIXQNv7P2JwHfLqVFb8qH u6TX7Zx2RVBqZieV3ZavTzYPvqqi0VlCg7HvjdRBHS0BOxT+OI31sB3+SwqXSyKdMf51 e+u8rJSEVz/zgF8DR0QuW2PfSXLoxroLl1b9o5D00I28bl6/fSE+cNbcPK0d8rep+klK 00RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:to :from:dmarc-filter:delivered-to; bh=7ymqcYtJo9vRigA63MF2kLitaFUg9nAA2Nf5AeY/L4w=; fh=flMKfU26Zf2bCSWDpkELM+XX9P5dENvjDCVykGrzV2k=; b=nAukTJI1HK98hpCWLQyxJ3YDZmmBlpW/fmYL4fLl3FFRbBe+0jwgZs2U3kgtC2kNm+ KF6fRmwy6ubIqceu3ll6g8etwR19IKoQwo6DqpE8PlMNnBuBdBDI6TaA3zd2EMg/zXsd aIjRV9yATtLUiVNNfed4W6xSn/qCrHo+3QehBrB1Vm9ZIqva51Lvo4D4syK1UqTBP9KE 7NyH5sRIwv/S+yibGtz2pU3JINkXeMkc0IpHaBKewnApQ0AX4MJaAGU99bKcmY3JN1QH 5Hk/8Q4KAqQ4SCCYCDjZQ2g6Oa/XHVdaOOvzUV6iqgNJ5bSM6xKkxa86ru5yfAKdx1dk 40Qg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id d25-20020a056402079900b0052566663542si5974613edy.76.2023.09.03.21.49.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Sep 2023 21:49:48 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D642B385770A for ; Mon, 4 Sep 2023 04:49:42 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgeu1.qq.com (smtpbgeu1.qq.com [52.59.177.22]) by sourceware.org (Postfix) with ESMTPS id 4DAC43858D33 for ; Mon, 4 Sep 2023 04:49:13 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4DAC43858D33 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp91t1693802947tp7oohd8 Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Mon, 04 Sep 2023 12:49:06 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: +rn5L4K3jcwP3KknfnyzjBxb5sx0h2VG/TGxlbRDxsZUw5N1GkZ9ESFWUKQBr XLFJbKDX2hOE7FZIm9RpJ+Q9C/xk7ntaplUaBlOkR46sVDXqFyFRk+KVEs5DQlW5i3FKq92 0j/y+mGeXT6+hIp3guL0iAMSrFJsfgVqVFeFpCNvJYuGKxovGlJOmtMlMv5PmoOQih6ufYs WvcIn02ekWFo38MpvB8//QDEy2IFF+252lbg1AloCbroOMAXfxmjHhgnM+YSiVJFLcw1HlN eSVMo5tTKu3g2vOR9MlV8FYIKq4NrjCyDgFsu+1rOlULDPQz8nI8qMQLkwX0SKZGfaIv5Xp 3cMDf1cNRJyNUgpFu0tWEW/2ROQV7x1egpT4J2SkJzjToc2EfFdUIoiBdXQgA4GGQb0UTAW X-QQ-GoodBg: 2 X-BIZMAIL-ID: 17084482950575883563 From: Lehua Ding To: gcc-patches@gcc.gnu.org Subject: [PATCH] RISC-V: Add conditional sqrt autovec pattern Date: Mon, 4 Sep 2023 12:49:06 +0800 Message-Id: <20230904044906.2546875-1-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS, TXREP, T_SPF_HELO_TEMPERROR autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kito.cheng@gmail.com, juzhe.zhong@rivai.ai Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776081162027837329 X-GMAIL-MSGID: 1776081162027837329 This patch adds a combined pattern for combining vfsqrt.v and vcond_mask. gcc/ChangeLog: * config/riscv/autovec-opt.md (*cond_): Add sqrt + vcond_mask combine pattern. * config/riscv/autovec.md (2): Change define_expand to define_insn_and_split. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/cond/cond_sqrt-1.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_sqrt-2.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-1.c: New test. * gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-2.c: New test. --- gcc/config/riscv/autovec-opt.md | 20 +++++++++++++ gcc/config/riscv/autovec.md | 7 +++-- .../riscv/rvv/autovec/cond/cond_sqrt-1.c | 24 +++++++++++++++ .../riscv/rvv/autovec/cond/cond_sqrt-2.c | 24 +++++++++++++++ .../riscv/rvv/autovec/cond/cond_sqrt_run-1.c | 29 +++++++++++++++++++ .../riscv/rvv/autovec/cond/cond_sqrt_run-2.c | 29 +++++++++++++++++++ 6 files changed, 131 insertions(+), 2 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-2.c diff --git a/gcc/config/riscv/autovec-opt.md b/gcc/config/riscv/autovec-opt.md index 1ca5ce97193..d9863c76654 100644 --- a/gcc/config/riscv/autovec-opt.md +++ b/gcc/config/riscv/autovec-opt.md @@ -730,6 +730,26 @@ DONE; }) +;; Combine vfsqrt.v and cond_mask +(define_insn_and_split "*cond_" + [(set (match_operand:VF 0 "register_operand") + (if_then_else:VF + (match_operand: 1 "register_operand") + (any_float_unop:VF + (match_operand:VF 2 "register_operand")) + (match_operand:VF 3 "register_operand")))] + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] +{ + insn_code icode = code_for_pred (, mode); + rtx ops[] = {operands[0], operands[1], operands[2], operands[3], + gen_int_mode (GET_MODE_NUNITS (mode), Pmode)}; + riscv_vector::expand_cond_len_unop (icode, ops); + DONE; +}) + ;; Combine vlmax neg and UNSPEC_VCOPYSIGN (define_insn_and_split "*copysign_neg" [(set (match_operand:VF 0 "register_operand") diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index 0f9d1fe2c8e..c220fda312e 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -994,11 +994,14 @@ ;; Includes: ;; - vfsqrt.v ;; ------------------------------------------------------------------------------- -(define_expand "2" +(define_insn_and_split "2" [(set (match_operand:VF 0 "register_operand") (any_float_unop:VF (match_operand:VF 1 "register_operand")))] - "TARGET_VECTOR" + "TARGET_VECTOR && can_create_pseudo_p ()" + "#" + "&& 1" + [(const_int 0)] { insn_code icode = code_for_pred (, mode); riscv_vector::emit_vlmax_insn (icode, riscv_vector::UNARY_OP_FRM_DYN, operands); diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-1.c new file mode 100644 index 00000000000..21219b43d9d --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-1.c @@ -0,0 +1,24 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model -ffast-math" } */ + +#include + +#define DEF_LOOP(TYPE, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##OP (TYPE *__restrict r, TYPE *__restrict a, \ + TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + r[i] = pred[i] ? OP (a[i]) : a[i]; \ + } + +#define TEST_ALL(T) \ + T (_Float16, __builtin_sqrtf16) \ + T (float, __builtin_sqrtf) \ + T (double, __builtin_sqrt) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler-times {\tvfsqrt\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ + +/* { dg-final { scan-assembler {\tvsetvli\t[a-z0-9]+,[a-z0-9]+,e[0-9]+,m[f0-9]+,t[au],mu} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-2.c new file mode 100644 index 00000000000..2fcdc339e70 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt-2.c @@ -0,0 +1,24 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv64gcv_zvfh -mabi=lp64d --param riscv-autovec-preference=scalable -fno-vect-cost-model -ffast-math" } */ + +#include + +#define DEF_LOOP(TYPE, OP) \ + void __attribute__ ((noipa)) \ + test_##TYPE##_##OP (TYPE *__restrict r, TYPE *__restrict a, \ + TYPE *__restrict b, TYPE *__restrict pred, int n) \ + { \ + for (int i = 0; i < n; ++i) \ + r[i] = pred[i] ? OP (a[i]) : b[i]; \ + } + +#define TEST_ALL(T) \ + T (_Float16, __builtin_sqrtf16) \ + T (float, __builtin_sqrtf) \ + T (double, __builtin_sqrt) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler-times {\tvfsqrt\.v\tv[0-9]+,v[0-9]+,v0\.t} 3 } } */ + +/* { dg-final { scan-assembler {\tvsetvli\t[a-z0-9]+,[a-z0-9]+,e[0-9]+,m[f0-9]+,t[au],mu} } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-1.c new file mode 100644 index 00000000000..c6f9ba85790 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-1.c @@ -0,0 +1,29 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model -ffast-math " } */ + +#include "cond_sqrt-1.c" +#include + +#define N 99 + +#define TEST_LOOP(TYPE, OP) \ + { \ + TYPE r[N], a[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : 2); \ + pred[i] = (i % 7 < 4); \ + asm volatile("" ::: "memory"); \ + } \ + test_##TYPE##_##OP (r, a, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : a[i])) \ + __builtin_abort (); \ + } + +int +main () +{ + TEST_ALL (TEST_LOOP) + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-2.c new file mode 100644 index 00000000000..5cfcfed568a --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cond/cond_sqrt_run-2.c @@ -0,0 +1,29 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param riscv-autovec-preference=scalable -fno-vect-cost-model -ffast-math" } */ + +#include "cond_sqrt-2.c" + +#define N 99 + +#define TEST_LOOP(TYPE, OP) \ + { \ + TYPE r[N], a[N], b[N], pred[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + a[i] = (i & 1 ? i : 3 * i) * (i % 3 == 0 ? 1 : 2); \ + b[i] = (i % 9) * (i % 7 + 1); \ + pred[i] = (i % 7 < 4); \ + asm volatile("" ::: "memory"); \ + } \ + test_##TYPE##_##OP (r, a, b, pred, N); \ + for (int i = 0; i < N; ++i) \ + if (r[i] != (pred[i] ? OP (a[i]) : b[i])) \ + __builtin_abort (); \ + } + +int +main () +{ + TEST_ALL (TEST_LOOP) + return 0; +}