From patchwork Thu Sep 21 15:18:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Pan2" X-Patchwork-Id: 142804 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp4925180vqi; Thu, 21 Sep 2023 08:19:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHUfrp+PZ2+dKP+5bASgtysnDJI3a4EthvRt4MH2xBcbFxV+Jh23i29WUsar6dqqf2V/lKm X-Received: by 2002:a2e:9257:0:b0:2bf:e596:36f9 with SMTP id v23-20020a2e9257000000b002bfe59636f9mr5078484ljg.13.1695309597505; Thu, 21 Sep 2023 08:19:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695309597; cv=none; d=google.com; s=arc-20160816; b=gommDuYszdmtQjnbo5lZ5KXWZG4GhFAvz6mn20ITtkFkKQpwXreLKPo0D+ikz2LEDp 6WlyrpRQ1n/goisw+C4uCc4IjP066xHg1n2Mo288+iBKSpNghT4EX/hFk7mOQRoR3JpV u5W8emABpUDcqfODdjreWhipvOD/xSTrEcvoS7Gs4KhWlBzKifqi19Cy1EiNl/Euktjv 8/VdcXzHhZcTnX/GpuTrDt9nKgSZFLOeUu7LrnYiI9LXw8mIIfcE1puu9RSWRTuBWb0n BvNjiaSGMOCXRoXPxz1wfQ+z357u1736ljtAth7GYc/59DCbzjea/gp8mirmuIBMitXZ 4FTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature:dmarc-filter:delivered-to; bh=XNpghvdODQ6tg9oymAPW4V2HN3+JK7mwnaw5MGnntBA=; fh=yqBQmCEeFYB2Wjmf8l8QkV/dOy5iKwSEx/iU/FYQjxU=; b=Yk0cGIwgAzku0U+suIbAOORBMgkcGLwAHVepWdS69R4bm206RADmXbsu/wxzieuOiE qtRo2c+QGmP/KtDVEP/s7OXLqP+g/qmHTWSRT835pV/KAXcIA3bJtXmgNQ6pvePoSkY6 293NfA7H7yd47t1NQ0rKZiOqivqjlTHpk3bpBybCTH0wox4RrBVi5i3HR5ApuUfI20va pDm2Vbm5+5ic6tXCo4k5gvoEddq1iUyPlf1L4rqUnwQxcI1XmsAyo4Z/zzzmqyocvJnO TSU2RM/KoAk9G5XYMl8IsTdHXfWhSAM4RucMWz2hl3Q6snETBELQJsiUpxmmVLHC/XfG BZRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QSZJPBLS; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id w27-20020a1709060a1b00b009ad7ec350e8si1342888ejf.665.2023.09.21.08.19.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Sep 2023 08:19:57 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QSZJPBLS; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B60753857359 for ; Thu, 21 Sep 2023 15:19:44 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.136]) by sourceware.org (Postfix) with ESMTPS id 0309A3858D39 for ; Thu, 21 Sep 2023 15:19:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0309A3858D39 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695309555; x=1726845555; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NJy8hR1ggkbCunO7nJj2vdMHZHKgCt5CIA1yF61z0R8=; b=QSZJPBLScGyAdiroMUI7Yv92St0BJCzuwAX5ZlxKB6mXgWjo3ynkdqgz 5/Z82S2AvfujvNTOmLDbClOnFpidD2HsGEJORZubTFidkBmoemDb3CFcw SAziNW7fq7arZBJN/mau0zbIlRTQ8yg0K6jKijxg2Up+TgCRfAH81D9L5 Z6vXsDQO4Xft70DdSNwghUI+WXXjMXLraysvdLlrOrn5MUSMQFr89PfqB 4tMc8JhF0tMFA8vCZ0V0w3QHuh7og7/VmPJHKEyjxLbfO5lPmXP/EJuxe oPKiYX+girUqX4LgSDpfudDAmYO+Z3vpglVRE6cDbqPQ7GW67fFoYPT86 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10840"; a="359932810" X-IronPort-AV: E=Sophos;i="6.03,165,1694761200"; d="scan'208";a="359932810" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2023 08:18:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10840"; a="747159647" X-IronPort-AV: E=Sophos;i="6.03,165,1694761200"; d="scan'208";a="747159647" Received: from shvmail02.sh.intel.com ([10.239.244.9]) by orsmga002.jf.intel.com with ESMTP; 21 Sep 2023 08:18:52 -0700 Received: from pli-ubuntu.sh.intel.com (pli-ubuntu.sh.intel.com [10.239.159.47]) by shvmail02.sh.intel.com (Postfix) with ESMTP id 7AA1B1005697; Thu, 21 Sep 2023 23:18:51 +0800 (CST) From: pan2.li@intel.com To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, pan2.li@intel.com, yanzhang.wang@intel.com, kito.cheng@gmail.com Subject: [PATCH v3] RISC-V: Support ceil and ceilf auto-vectorization Date: Thu, 21 Sep 2023 23:18:50 +0800 Message-Id: <20230921151850.1697755-1-pan2.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230920023059.1728132-1-pan2.li@intel.com> References: <20230920023059.1728132-1-pan2.li@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777522024262700176 X-GMAIL-MSGID: 1777660956510647161 From: Pan Li This patch would like to support auto-vectorization for both the ceil and ceilf of math.h. It depends on the -ffast-math option. When we would like to call ceil/ceilf like v2 = ceil (v1), we will convert it into below insn (reference the implementation of llvm). * vfcvt.x.f v3, v1, RUP * vfcvt.f.x v2, v3 However, the floating point value may not need the cvt as above if its mantissa is zero. For example single precision floating point below. +-----------+---------------+ | float | binary layout | +-----------+---------------+ | 8388607.5 | 0x4affffff | | 8388608.0 | 0x4b000000 | | 8388609.0 | 0x4b000001 | +-----------+---------------+ All single floating point great than 8388608.0 will have all zero mantisaa. We leverage vmflt and mask to filter them out in vector and only do the cvt on mask. Befor this patch: math-ceil-1.c:21:1: missed: couldn't vectorize loop ... .L3: flw fa0,0(s0) addi s0,s0,4 addi s1,s1,4 call ceilf fsw fa0,-4(s1) bne s0,s2,.L3 After this patch: ... fsrmi 3 .L4: vfabs.v v0,v1 vmv1r.v v2,v1 vmflt.vv v0,v0,v4 sub a3,a3,a4 vfcvt.x.f.v v3,v1,v0.t vfcvt.f.x.v v2,v3,v0.t vfsgnj.vv v2,v2,v1 bne .L4 .L14: fsrm a6 ret Please note VLS mode is also involved in this patch and covered by the test cases. gcc/ChangeLog: * config/riscv/autovec.md (ceil2): New pattern. * config/riscv/riscv-protos.h (enum insn_flags): New enum type. (enum insn_type): Ditto. (expand_vec_ceil): New function decl. * config/riscv/riscv-v.cc (gen_ceil_const_fp): New function impl. (expand_vec_float_cmp_mask): Ditto. (expand_vec_copysign): Ditto. (expand_vec_ceil): Ditto. * config/riscv/vector.md: Add VLS mode support. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/vls/def.h: New macro. * gcc.target/riscv/rvv/autovec/math-ceil-1.c: New test. * gcc.target/riscv/rvv/autovec/math-ceil-2.c: New test. * gcc.target/riscv/rvv/autovec/math-ceil-3.c: New test. * gcc.target/riscv/rvv/autovec/math-ceil-run-1.c: New test. * gcc.target/riscv/rvv/autovec/math-ceil-run-2.c: New test. * gcc.target/riscv/rvv/autovec/test-math.h: New test. * gcc.target/riscv/rvv/autovec/vls/math-ceil-1.c: New test. Signed-off-by: Pan Li Signed-off-by: Pan Li --- gcc/config/riscv/autovec.md | 16 +++ gcc/config/riscv/riscv-protos.h | 5 + gcc/config/riscv/riscv-v.cc | 116 ++++++++++++++++++ gcc/config/riscv/vector.md | 2 +- .../riscv/rvv/autovec/math-ceil-1.c | 26 ++++ .../riscv/rvv/autovec/math-ceil-2.c | 26 ++++ .../riscv/rvv/autovec/math-ceil-3.c | 28 +++++ .../riscv/rvv/autovec/math-ceil-run-1.c | 39 ++++++ .../riscv/rvv/autovec/math-ceil-run-2.c | 39 ++++++ .../gcc.target/riscv/rvv/autovec/test-math.h | 38 ++++++ .../gcc.target/riscv/rvv/autovec/vls/def.h | 8 ++ .../riscv/rvv/autovec/vls/math-ceil-1.c | 43 +++++++ 12 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-3.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-1.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/test-math.h create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/math-ceil-1.c diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index f0f1abc4e82..1b4bd82f9ec 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -2239,3 +2239,19 @@ (define_expand "avg3_ceil" riscv_vector::emit_vlmax_insn (icode, riscv_vector::BINARY_OP, ops3); DONE; }) + +;; ------------------------------------------------------------------------- +;; ---- [FP] Math.h. +;; ------------------------------------------------------------------------- +;; Includes: +;; - ceil/ceilf +;; ------------------------------------------------------------------------- +(define_expand "ceil2" + [(match_operand:V_VLSF 0 "register_operand") + (match_operand:V_VLSF 1 "register_operand")] + "TARGET_VECTOR && !flag_trapping_math && !flag_rounding_math" + { + riscv_vector::expand_vec_ceil (operands[0], operands[1], mode, mode); + DONE; + } +) diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h index 9ea0bcf15d3..07b4ffe3edf 100644 --- a/gcc/config/riscv/riscv-protos.h +++ b/gcc/config/riscv/riscv-protos.h @@ -250,6 +250,9 @@ enum insn_flags : unsigned int /* flags for the floating-point rounding mode. */ /* Means INSN has FRM operand and the value is FRM_DYN. */ FRM_DYN_P = 1 << 15, + + /* Means INSN has FRM operand and the value is FRM_RUP. */ + FRM_RUP_P = 1 << 16, }; enum insn_type : unsigned int @@ -290,6 +293,7 @@ enum insn_type : unsigned int UNARY_OP_TAMA = __MASK_OP_TAMA | UNARY_OP_P, UNARY_OP_TAMU = __MASK_OP_TAMU | UNARY_OP_P, UNARY_OP_FRM_DYN = UNARY_OP | FRM_DYN_P, + UNARY_OP_TAMU_FRM_RUP = UNARY_OP_TAMU | FRM_RUP_P, /* Binary operator. */ BINARY_OP = __NORMAL_OP | BINARY_OP_P, @@ -432,6 +436,7 @@ bool expand_vec_cmp_float (rtx, rtx_code, rtx, rtx, bool); void expand_cond_len_unop (unsigned, rtx *); void expand_cond_len_binop (unsigned, rtx *); void expand_reduction (unsigned, unsigned, rtx *, rtx); +void expand_vec_ceil (rtx, rtx, machine_mode, machine_mode); #endif bool sew64_scalar_helper (rtx *, rtx *, rtx, machine_mode, bool, void (*)(rtx *, rtx)); diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index 4b9a494f8eb..35143bc6421 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -323,6 +323,8 @@ public: /* Add rounding mode operand. */ if (m_insn_flags & FRM_DYN_P) add_rounding_mode_operand (FRM_DYN); + if (m_insn_flags & FRM_RUP_P) + add_rounding_mode_operand (FRM_RUP); gcc_assert (insn_data[(int) icode].n_operands == m_opno); expand (icode, any_mem_p); @@ -3439,4 +3441,118 @@ cmp_lmul_gt_one (machine_mode mode) return false; } +/* We don't have to convert the floating point to integer when the + mantissa is zero. Thus, ther will be a limitation for both the + single and double precision floating point. There will be no + mantissa if the floating point is greater than the limit. + + 1. Single floating point. + +-----------+---------------+ + | float | binary layout | + +-----------+---------------+ + | 8388607.5 | 0x4affffff | + +-----------+---------------+ + | 8388608.0 | 0x4b000000 | + +-----------+---------------+ + | 8388609.0 | 0x4b000001 | + +-----------+---------------+ + | ... | ... | + + All single floating point will be unchanged for ceil if it is + greater than and equal to 8388608. + + 2. Double floating point. + +--------------------+--------------------+ + | float | binary layout | + +--------------------+--------------------+ + | 4503599627370495.5 | 0X432fffffffffffff | + +--------------------+--------------------+ + | 4503599627370496.0 | 0X4330000000000000 | + +--------------------+--------------------+ + | 4503599627370497.0 | 0X4340000000000000 | + +--------------------+--------------------+ + | ... | ... | + + All double floating point will be unchanged for ceil if it is + greater than and equal to 4503599627370496. + */ +static rtx +gen_ceil_const_fp (machine_mode inner_mode) +{ + REAL_VALUE_TYPE real; + + if (inner_mode == E_SFmode) + real_from_integer (&real, inner_mode, 8388608, SIGNED); + else if (inner_mode == E_DFmode) + real_from_integer (&real, inner_mode, 4503599627370496, SIGNED); + else + gcc_unreachable (); + + return const_double_from_real_value (real, inner_mode); +} + +static rtx +expand_vec_float_cmp_mask (rtx fp_vector, rtx_code code, rtx fp_scalar, + machine_mode vec_fp_mode) +{ + /* Step-1: Get the abs float value for mask generation. */ + rtx tmp = gen_reg_rtx (vec_fp_mode); + rtx abs_ops[] = {tmp, fp_vector}; + insn_code icode = code_for_pred (ABS, vec_fp_mode); + emit_vlmax_insn (icode, UNARY_OP, abs_ops); + + /* Step-2: Prepare the scalar float compare register. */ + rtx fp_reg = gen_reg_rtx (GET_MODE_INNER (vec_fp_mode)); + emit_insn (gen_move_insn (fp_reg, fp_scalar)); + + /* Step-3: Prepare the vector float compare register. */ + rtx vec_dup = gen_reg_rtx (vec_fp_mode); + icode = code_for_pred_broadcast (vec_fp_mode); + rtx vfmv_ops[] = {vec_dup, fp_reg}; + emit_vlmax_insn (icode, UNARY_OP, vfmv_ops); + + /* Step-4: Generate the mask. */ + machine_mode mask_mode = get_mask_mode (vec_fp_mode); + rtx mask = gen_reg_rtx (mask_mode); + expand_vec_cmp (mask, code, tmp, vec_dup); + + return mask; +} + +static void +expand_vec_copysign (rtx op_dest, rtx op_src_0, rtx op_src_1, + machine_mode vec_mode) +{ + rtx sgnj_ops[] = {op_dest, op_src_0, op_src_1}; + insn_code icode = code_for_pred (UNSPEC_VCOPYSIGN, vec_mode); + + emit_vlmax_insn (icode, BINARY_OP, sgnj_ops); +} + +void +expand_vec_ceil (rtx op_0, rtx op_1, machine_mode vec_fp_mode, + machine_mode vec_int_mode) +{ + /* Step-1: Generate the mask on const fp. */ + rtx const_fp = gen_ceil_const_fp (GET_MODE_INNER (vec_fp_mode)); + rtx mask = expand_vec_float_cmp_mask (op_1, LT, const_fp, vec_fp_mode); + + /* Step-2: Convert to integer on mask, with rounding up (aka ceil). */ + rtx tmp = gen_reg_rtx (vec_int_mode); + rtx cvt_x_ops[] = {tmp, mask, tmp, op_1}; + insn_code icode = code_for_pred_fcvt_x_f (UNSPEC_VFCVT, vec_fp_mode); + emit_vlmax_insn (icode, UNARY_OP_TAMU_FRM_RUP, cvt_x_ops); + + /* Step-3: Convert to floating-point on mask for the final result. + To avoid unnecessary frm register access, we use RUP here and it will + never do the rounding up because the tmp rtx comes from the float + to int conversion. */ + rtx cvt_fp_ops[] = {op_0, mask, op_1, tmp}; + icode = code_for_pred (FLOAT, vec_fp_mode); + emit_vlmax_insn (icode, UNARY_OP_TAMU_FRM_RUP, cvt_fp_ops); + + /* Step-4: Retrieve the sign bit. */ + expand_vec_copysign (op_0, op_0, op_1, vec_fp_mode); +} + } // namespace riscv_vector diff --git a/gcc/config/riscv/vector.md b/gcc/config/riscv/vector.md index 36f0256c747..73f90dea36b 100644 --- a/gcc/config/riscv/vector.md +++ b/gcc/config/riscv/vector.md @@ -7438,7 +7438,7 @@ (define_insn "@pred_fcvt_x_f" (reg:SI VTYPE_REGNUM) (reg:SI FRM_REGNUM)] UNSPEC_VPREDICATE) (unspec: - [(match_operand:VF 3 "register_operand" " vr, vr, vr, vr")] VFCVTS) + [(match_operand:V_VLSF 3 "register_operand" " vr, vr, vr, vr")] VFCVTS) (match_operand: 2 "vector_merge_operand" " vu, 0, vu, 0")))] "TARGET_VECTOR" "vfcvt.x.f.v\t%0,%3%p1" diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-1.c new file mode 100644 index 00000000000..0908ef269bd --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-1.c @@ -0,0 +1,26 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fno-vect-cost-model -ffast-math -fno-schedule-insns -fno-schedule-insns2" } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +#include "test-math.h" + +/* +** test_float___builtin_ceilf: +** frrm\s+[atx][0-9]+ +** ... +** fsrmi\s+3 +** ... +** vsetvli\s+[atx][0-9]+,\s*zero,\s*e32,\s*m1,\s*ta,\s*mu +** vfabs\.v\s+v[0-9]+,\s*v[0-9]+ +** ... +** vmflt\.vv\s+v0,\s*v[0-9]+,\s*v[0-9]+ +** ... +** vfcvt\.x\.f\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t +** ... +** vfcvt\.f\.x\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t +** vfsgnj\.vv\s+v[0-9]+,v[0-9]+,v[0-9]+ +** ... +** fsrm\s+[atx][0-9]+ +** ... +*/ +TEST_CEIL(float, __builtin_ceilf) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-2.c new file mode 100644 index 00000000000..65d4807edef --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-2.c @@ -0,0 +1,26 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fno-vect-cost-model -ffast-math -fno-schedule-insns -fno-schedule-insns2" } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +#include "test-math.h" + +/* +** test_double___builtin_ceil: +** frrm\s+[atx][0-9]+ +** ... +** fsrmi\s+3 +** ... +** vsetvli\s+[atx][0-9]+,\s*zero,\s*e64,\s*m1,\s*ta,\s*mu +** vfabs\.v\s+v[0-9]+,\s*v[0-9]+ +** ... +** vmflt\.vv\s+v0,\s*v[0-9]+,\s*v[0-9]+ +** ... +** vfcvt\.x\.f\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t +** ... +** vfcvt\.f\.x\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t +** vfsgnj\.vv\s+v[0-9]+,v[0-9]+,v[0-9]+ +** ... +** fsrm\s+[atx][0-9]+ +** ... +*/ +TEST_CEIL(double, __builtin_ceil) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-3.c new file mode 100644 index 00000000000..416698a753e --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-3.c @@ -0,0 +1,28 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fno-vect-cost-model -ffast-math -fno-schedule-insns -fno-schedule-insns2" } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +#include "test-math.h" + +/* +** test_float___builtin_ceilf: +** frrm\s+[atx][0-9]+ +** ... +** fsrmi\s+3 +** ... +** vsetvli\s+[atx][0-9]+,\s*zero,\s*e32,\s*m1,\s*ta,\s*mu +** vfabs\.v\s+v[0-9]+,\s*v[0-9]+ +** ... +** vmflt\.vv\s+v0,\s*v[0-9]+,\s*v[0-9]+ +** ... +** vfcvt\.x\.f\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t +** ... +** vfcvt\.f\.x\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t +** vfsgnj\.vv\s+v[0-9]+,v[0-9]+,v[0-9]+ +** ... +** vmerge\.vvm\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+,\s*v0 +** ... +** fsrm\s+[atx][0-9]+ +** ... +*/ +TEST_COND_CEIL(float, __builtin_ceilf) diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-1.c new file mode 100644 index 00000000000..1bd5829eeca --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-1.c @@ -0,0 +1,39 @@ +/* { dg-do run { target { riscv_vector && riscv_zvfh_hw } } } */ +/* { dg-additional-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fno-vect-cost-model -ffast-math" } */ + +#include "test-math.h" + +#define ARRAY_SIZE 128 + +float in[ARRAY_SIZE]; +float out[ARRAY_SIZE]; +float ref[ARRAY_SIZE]; + +TEST_CEIL (float, __builtin_ceilf) +TEST_ASSERT (float) + +TEST_INIT (float, 1.2, 2.0, 1) +TEST_INIT (float, -1.2, -1.0, 2) +TEST_INIT (float, 3.0, 3.0, 3) +TEST_INIT (float, 8388607.5, 8388608.0, 4) +TEST_INIT (float, 8388609.0, 8388609.0, 5) +TEST_INIT (float, 0.0, 0.0, 6) +TEST_INIT (float, -0.0, -0.0, 7) +TEST_INIT (float, -8388607.5, -8388607.0, 8) +TEST_INIT (float, -8388608.0, -8388608.0, 9) + +int +main () +{ + RUN_TEST (float, 1, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 2, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 3, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 4, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 5, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 6, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 7, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 8, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + RUN_TEST (float, 9, __builtin_ceilf, in, out, ref, ARRAY_SIZE); + + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-2.c new file mode 100644 index 00000000000..d9386215efb --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/math-ceil-run-2.c @@ -0,0 +1,39 @@ +/* { dg-do run { target { riscv_vector && riscv_zvfh_hw } } } */ +/* { dg-additional-options "-march=rv64gcv -mabi=lp64d -O3 -ftree-vectorize -fno-vect-cost-model -ffast-math" } */ + +#include "test-math.h" + +#define ARRAY_SIZE 128 + +double in[ARRAY_SIZE]; +double out[ARRAY_SIZE]; +double ref[ARRAY_SIZE]; + +TEST_CEIL (double, __builtin_ceil) +TEST_ASSERT (double) + +TEST_INIT (double, 1.2, 2.0, 1) +TEST_INIT (double, -1.2, -1.0, 2) +TEST_INIT (double, 3.0, 3.0, 3) +TEST_INIT (double, 4503599627370495.5, 4503599627370496.0, 4) +TEST_INIT (double, 4503599627370497.0, 4503599627370497.0, 5) +TEST_INIT (double, 0.0, 0.0, 6) +TEST_INIT (double, -0.0, -0.0, 7) +TEST_INIT (double, -4503599627370495.5, -4503599627370495.0, 8) +TEST_INIT (double, -4503599627370496.0, -4503599627370496.0, 9) + +int +main () +{ + RUN_TEST (double, 1, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 2, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 3, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 4, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 5, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 6, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 7, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 8, __builtin_ceil, in, out, ref, ARRAY_SIZE); + RUN_TEST (double, 9, __builtin_ceil, in, out, ref, ARRAY_SIZE); + + return 0; +} diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/test-math.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/test-math.h new file mode 100644 index 00000000000..6e913da37f4 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/test-math.h @@ -0,0 +1,38 @@ +#define TEST_CEIL(TYPE, CALL) \ + void test_##TYPE##_##CALL (TYPE *out, TYPE *in, unsigned count) \ + { \ + for (unsigned i = 0; i < count; i++) \ + out[i] = CALL (in[i]); \ + } + +#define TEST_COND_CEIL(TYPE, CALL) \ + void test_##TYPE##_##CALL (TYPE *out, int *cond, TYPE *in, unsigned count) \ + { \ + for (unsigned i = 0; i < count; i++) \ + out[i] = cond[i] ? CALL (in[i]) : in[i]; \ + } + +#define TEST_INIT(TYPE, VAL_IN, VAL_REF, NUM) \ + void test_##TYPE##_init_##NUM (TYPE *in, TYPE *ref, unsigned size) \ + { \ + for (unsigned i = 0; i < size; i++) \ + { \ + in[i] = VAL_IN; \ + ref[i] = VAL_REF; \ + } \ + } + +#define TEST_ASSERT(TYPE) \ + void test_##TYPE##_assert (TYPE *out, TYPE *ref, unsigned size) \ + { \ + for (unsigned i = 0; i < size; i++) \ + { \ + if (out[i] != ref[i]) \ + __builtin_abort (); \ + } \ + } + +#define RUN_TEST(TYPE, NUM, CALL, IN, OUT, REF, SIZE) \ + test_##TYPE##_init_##NUM (IN, REF, SIZE); \ + test_##TYPE##_##CALL (OUT, IN, SIZE); \ + test_##TYPE##_assert (OUT, REF, SIZE); diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h index 74685f8d05e..ccc1d1d70ab 100644 --- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h @@ -518,3 +518,11 @@ typedef double v512df __attribute__ ((vector_size (4096))); for (int i = 0; i < NUM; i++) \ dst[i] = ((TYPE2) a[i] + b[i] + 1) >> 1; \ } + +#define DEF_CALL_V(PREFIX, NUM, TYPE, CALL) \ + void __attribute__ ((noinline, noclone)) \ + PREFIX##_##TYPE##NUM (TYPE *restrict a, TYPE *restrict b) \ + { \ + for (int i = 0; i < NUM; ++i) \ + a[i] = CALL (b[i]); \ + } diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/math-ceil-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/math-ceil-1.c new file mode 100644 index 00000000000..cd19a686c86 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/math-ceil-1.c @@ -0,0 +1,43 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -mabi=lp64d -O3 --param=riscv-autovec-lmul=m8 -ffast-math -fdump-tree-optimized" } */ + +#include "def.h" + +DEF_CALL_V (ceilf, 1, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 2, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 4, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 8, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 16, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 32, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 64, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 128, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 256, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 512, float, __builtin_ceilf) +DEF_CALL_V (ceilf, 1024, float, __builtin_ceilf) + +DEF_CALL_V (ceil, 1, double, __builtin_ceil) +DEF_CALL_V (ceil, 2, double, __builtin_ceil) +DEF_CALL_V (ceil, 4, double, __builtin_ceil) +DEF_CALL_V (ceil, 8, double, __builtin_ceil) +DEF_CALL_V (ceil, 16, double, __builtin_ceil) +DEF_CALL_V (ceil, 32, double, __builtin_ceil) +DEF_CALL_V (ceil, 64, double, __builtin_ceil) +DEF_CALL_V (ceil, 128, double, __builtin_ceil) +DEF_CALL_V (ceil, 256, double, __builtin_ceil) +DEF_CALL_V (ceil, 512, double, __builtin_ceil) + +/* { dg-final { scan-assembler-not {csrr} } } */ +/* { dg-final { scan-tree-dump-not "1,1" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "2,2" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "4,4" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "16,16" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "32,32" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "64,64" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "128,128" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "256,256" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "512,512" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "1024,1024" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "2048,2048" "optimized" } } */ +/* { dg-final { scan-tree-dump-not "4096,4096" "optimized" } } */ +/* { dg-final { scan-assembler-times {vfcvt\.x\.f\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t} 19 } } */ +/* { dg-final { scan-assembler-times {vfcvt\.f\.x\.v\s+v[0-9]+,\s*v[0-9]+,\s*v0\.t} 19 } } */