From patchwork Thu Oct 12 06:02:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: liuhongt X-Patchwork-Id: 151760 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp1012678vqb; Wed, 11 Oct 2023 23:05:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGydG6KuhL3koh0g5bi6c4pPRt8RyHJXLM6yyM+mFZEzr863QaLkSA/bfI5OrMcuQ3JhWJq X-Received: by 2002:aa7:cb93:0:b0:52a:943:9abd with SMTP id r19-20020aa7cb93000000b0052a09439abdmr20082144edt.30.1697090716454; Wed, 11 Oct 2023 23:05:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697090716; cv=none; d=google.com; s=arc-20160816; b=Ez+gViCekOcYiKj322d3VdjAYAu5dTiIOQid5FlUr2O+YNFmKVmyTu/i0XjWesAVon 3kjQaKFk0Ypw0MqfBvQPos/t2KcF7CQuIRApu6tRb1ZI1jWAMjHRhPrvrfo5zExvqsZV //J7lf32Z4b9UyRvn3xa7mU0WnOV932DYt8anQxq23jn4YGXLSylsbP739+nua1uE/w7 vhjZEP9sGLMiwN/1BHzml3XB7VBVBECoo9Q/RBlZqUh9If9y9mQV0Cn75vpFoeGPFA/D 4VdEddj5zvinaPoCdEMfCu+Xn/JUUM8PchrCmbAby+E70xChDtdwb/UhzqKnuP/KRarj hr7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:cc:to:from:dkim-signature :dmarc-filter:delivered-to; bh=jmnd6V4BA70XC2Fft0KzrlYrOxxEgVKKyw2qXlp0OCM=; fh=ChXOctppJn0KECDRINafwUY5xHRufGHaa0Ju9pddrcQ=; b=m8NLndpKIBwCxXd6ntVumq1VcLbuZVb4qYL13RAPBKrLofnenFCQik8La7j2Tkamgl KMsCCbRD+ITRzohUAyh3vELQzTK7WSbzNZy5/8DTMmIoGhhjJCvfOVyiOBPKyneyYCHT 9XCyLPHRgcDb/ULrnfbzszSvQV7V8S258WyrbXmPMFL8IWkt5HnW5XAR6YTeFR0Sb+2S fyrsgTMEuaoRvD/GZwrA+onNLholGYNTWHPq2hisLXF8G47nDeN4CqqpOqYCe1gwfjld Kek0PS5QDBP+v7KxCot1o0K8QYPCqgUdDv6OXWjK4HtPyS2lzF/vRDBOd++RR4s8HC9p YPfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=a2UadfEw; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id u23-20020a056402065700b0053621d9f96bsi7443526edx.592.2023.10.11.23.05.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Oct 2023 23:05:16 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=a2UadfEw; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0BB863856962 for ; Thu, 12 Oct 2023 06:04:46 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by sourceware.org (Postfix) with ESMTPS id 01BD53858C52 for ; Thu, 12 Oct 2023 06:04:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 01BD53858C52 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697090657; x=1728626657; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=WN3FN+nUEDPuuAzDpFkCWLN62TtwkQ+U4XKxCUH05cM=; b=a2UadfEwJIjFCKh1TLpsZc400xVU96QG7UY+z01sSSpN8QKELsPMZFls OwVYaOMwD0GueH3wZ2v4Hb5inMZXplITu/+KLwMGi4AUTRjHp1iV5NLWA Y62imIiAWtgfThO7mPTvwCp7iXCkQW6Gb32Xg/h59uu78SmpWqRB4Gfjx QIf7A8Q6WmIgtchff1rM/mXEWAoqHpJAlVoA2E5v5HtWLyHYr3jvfbEGY HxKCV+Jn3074m4uyf6gHYi8nKfp5lmjRU5QJRWxQPvIm0bqTXEYKkKjDP BhqEvMXlPHLd90TkzOPHRD4j4AWEVUUFE1CyjYu9+ZXm5aTQVTrxlSCGw A==; X-IronPort-AV: E=McAfee;i="6600,9927,10860"; a="471096530" X-IronPort-AV: E=Sophos;i="6.03,218,1694761200"; d="scan'208";a="471096530" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Oct 2023 23:04:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10860"; a="877984837" X-IronPort-AV: E=Sophos;i="6.03,218,1694761200"; d="scan'208";a="877984837" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by orsmga004.jf.intel.com with ESMTP; 11 Oct 2023 23:04:10 -0700 Received: from shliclel4217.sh.intel.com (shliclel4217.sh.intel.com [10.239.240.127]) by shvmail03.sh.intel.com (Postfix) with ESMTP id 883C510056F1; Thu, 12 Oct 2023 14:04:09 +0800 (CST) From: liuhongt To: gcc-patches@gcc.gnu.org Cc: crazylht@gmail.com, hjl.tools@gmail.com Subject: [PATCH 1/2] Enable vectorization for V2HF/V4HF rounding operations and sqrt. Date: Thu, 12 Oct 2023 14:02:08 +0800 Message-Id: <20231012060209.4130200-1-hongtao.liu@intel.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779528594940014133 X-GMAIL-MSGID: 1779528594940014133 For lrint/lround/lceil/lfoor is not vectorized due to vectorization restriction. When input element size is different from output element size, vectorization relies on the old TARGET_VECTORIZE_BUILTIN_VECTORIZED_FUNCTION intstead of the modern standand pattern name. The patch only supports standard pattern name, doesn't update ix86_builtin_vectorized_function. Bootstrapped and regtested on x86_64-pc-linux-gnu{-m32,}. Ready push to trunk. gcc/ChangeLog: * config/i386/i386-expand.cc (ix86_sse_copysign_to_positive): Handle HFmode. (ix86_expand_round_sse4): Ditto. * config/i386/i386.md (roundhf2): New expander. (lroundhf2): Ditto. (lrinthf2): Ditto. (lhf2): Ditto. * config/i386/mmx.md (sqrt2): Ditto. (btrunc2): Ditto. (nearbyint2): Ditto. (rint2): Ditto. (lrint2): Ditto. (floor2): Ditto. (lfloor2): Ditto. (ceil2): Ditto. (lceil2): Ditto. (round2): Ditto. (lround2): Ditto. * config/i386/sse.md (lrint2): Ditto. (lfloor2): Ditto. (lceil2): Ditto. (lround2): Ditto. (sse4_1_round): Extend to V8HF. (round2): Extend to V8HF/V16HF/V32HF. gcc/testsuite/ChangeLog: * gcc.target/i386/part-vect-roundhf.c: New test. * gcc.target/i386/part-vect-sqrtph-1.c: New test. --- gcc/config/i386/i386-expand.cc | 6 + gcc/config/i386/i386.md | 38 +++ gcc/config/i386/mmx.md | 191 ++++++++++++++- gcc/config/i386/sse.md | 60 ++++- .../gcc.target/i386/part-vect-roundhf.c | 217 ++++++++++++++++++ .../gcc.target/i386/part-vect-sqrtph-1.c | 20 ++ 6 files changed, 521 insertions(+), 11 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/part-vect-roundhf.c create mode 100644 gcc/testsuite/gcc.target/i386/part-vect-sqrtph-1.c diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc index 425f3531862..b81b5cc030c 100644 --- a/gcc/config/i386/i386-expand.cc +++ b/gcc/config/i386/i386-expand.cc @@ -18434,6 +18434,8 @@ ix86_sse_copysign_to_positive (rtx result, rtx abs_value, rtx sign, rtx mask) vmode = V4SFmode; else if (mode == DFmode) vmode = V2DFmode; + else if (mode == HFmode) + vmode = V8HFmode; else vmode = mode; @@ -18970,6 +18972,10 @@ ix86_expand_round_sse4 (rtx op0, rtx op1) switch (mode) { + case E_HFmode: + gen_copysign = gen_copysignhf3; + gen_round = gen_sse4_1_roundhf2; + break; case E_SFmode: gen_copysign = gen_copysignsf3; gen_round = gen_sse4_1_roundsf2; diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md index 65a0dd025c7..41173cb3452 100644 --- a/gcc/config/i386/i386.md +++ b/gcc/config/i386/i386.md @@ -21741,6 +21741,15 @@ (define_expand "nearbyint2" DONE; }) +(define_expand "roundhf2" + [(match_operand:HF 0 "register_operand") + (match_operand:HF 1 "register_operand")] + "TARGET_AVX512FP16 && !flag_trapping_math && !flag_rounding_math" +{ + ix86_expand_round_sse4 (operands[0], operands[1]); + DONE; +}) + (define_expand "round2" [(match_operand:X87MODEF 0 "register_operand") (match_operand:X87MODEF 1 "nonimmediate_operand")] @@ -21792,6 +21801,22 @@ (define_insn "lrintxf2" [(set_attr "type" "fpspc") (set_attr "mode" "")]) +(define_expand "lroundhf2" + [(set (match_operand:SWI248 0 "register_operand") + (unspec:SWI248 [(match_operand:HF 1 "nonimmediate_operand")] + UNSPEC_FIX_NOTRUNC))] + "TARGET_AVX512FP16 && !flag_trapping_math && !flag_rounding_math" +{ + ix86_expand_lround (operands[0], operands[1]); + DONE; +}) + +(define_expand "lrinthf2" + [(set (match_operand:SWI48 0 "register_operand") + (unspec:SWI48 [(match_operand:HF 1 "nonimmediate_operand")] + UNSPEC_FIX_NOTRUNC))] + "TARGET_AVX512FP16") + (define_expand "lrint2" [(set (match_operand:SWI48 0 "register_operand") (unspec:SWI48 [(match_operand:MODEF 1 "nonimmediate_operand")] @@ -22034,6 +22059,19 @@ (define_expand "lxf2" && (!TARGET_SSE_MATH || TARGET_MIX_SSE_I387) && flag_unsafe_math_optimizations") +(define_expand "lhf2" + [(set (match_operand:SWI48 0 "nonimmediate_operand") + (unspec:SWI48 [(match_operand:HF 1 "register_operand")] + FIST_ROUNDING))] + "TARGET_AVX512FP16" +{ + rtx tmp = gen_reg_rtx (HFmode); + emit_insn (gen_sse4_1_roundhf2 (tmp, operands[1], + GEN_INT (ROUND_ | ROUND_NO_EXC))); + emit_insn (gen_fix_trunchf2 (operands[0], tmp)); + DONE; +}) + (define_expand "l2" [(parallel [(set (match_operand:SWI48 0 "nonimmediate_operand") (unspec:SWI48 [(match_operand:MODEF 1 "register_operand")] diff --git a/gcc/config/i386/mmx.md b/gcc/config/i386/mmx.md index c84a37a8444..8375100d4bf 100644 --- a/gcc/config/i386/mmx.md +++ b/gcc/config/i386/mmx.md @@ -103,7 +103,8 @@ (define_mode_attr mmxintvecmode (V4HF "V4HF") (V2HF "V2HI")]) (define_mode_attr mmxintvecmodelower - [(V2SF "v2si") (V2SI "v2si") (V4HI "v4hi") (V8QI "v8qi")]) + [(V2SF "v2si") (V2SI "v2si") (V4HI "v4hi") (V8QI "v8qi") + (V4HF "v4hi") (V2HF "v2hi")]) ;; Mapping of vector modes to a vector mode of double size (define_mode_attr mmxdoublevecmode @@ -2053,6 +2054,21 @@ (define_expand "3" DONE; }) +(define_expand "sqrt2" + [(set (match_operand:VHF_32_64 0 "register_operand") + (sqrt:VHF_32_64 + (match_operand:VHF_32_64 1 "nonimmediate_operand")))] + "TARGET_AVX512FP16 && TARGET_AVX512VL && ix86_partial_vec_fp_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_sqrtv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + DONE; +}) + (define_expand "2" [(set (match_operand:VHF_32_64 0 "register_operand") (absneg:VHF_32_64 @@ -2088,6 +2104,179 @@ (define_insn_and_split "*mmx_nabs2" [(set (match_dup 0) (ior: (match_dup 1) (match_dup 2)))]) +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +;; +;; Parallel half-precision floating point rounding operations. +;; +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + +(define_expand "btrunc2" + [(match_operand:VHF_32_64 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_btruncv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "nearbyint2" + [(match_operand:VHF_32_64 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_nearbyintv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "rint2" + [(match_operand:VHF_32_64 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_rintv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "lrint2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_lrintv8hfv8hi2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "floor2" + [(match_operand:VHF_32_64 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_floorv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "lfloor2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_lfloorv8hfv8hi2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "ceil2" + [(match_operand:VHF_32_64 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_ceilv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "lceil2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_lceilv8hfv8hi2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "round2" + [(match_operand:VHF_32_64 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_roundv8hf2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + +(define_expand "lround2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_32_64 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && TARGET_AVX512VL + && ix86_partial_vec_fp_math + && !flag_trapping_math" +{ + rtx op1 = gen_reg_rtx (V8HFmode); + rtx op0 = gen_reg_rtx (V8HFmode); + + emit_insn (gen_mov__to_sse (op1, operands[1])); + emit_insn (gen_lroundv8hfv8hi2 (op0, op1)); + emit_move_insn (operands[0], lowpart_subreg (mode, op0, V8HFmode)); + + DONE; +}) + ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; ;; Parallel half-precision floating point logical operations diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md index 22e43eb3f92..4602edf2374 100644 --- a/gcc/config/i386/sse.md +++ b/gcc/config/i386/sse.md @@ -7092,6 +7092,13 @@ (define_expand "vec_unpacks_hi_" DONE; }) +(define_expand "lrint2" + [(set (match_operand: 0 "register_operand") + (unspec: + [(match_operand:VHF_AVX512VL 1 "register_operand")] + UNSPEC_FIX_NOTRUNC))] + "TARGET_AVX512FP16") + (define_insn "avx512fp16_vcvtph2_" [(set (match_operand:VI248_AVX512VL 0 "register_operand" "=v") (unspec:VI248_AVX512VL @@ -24183,13 +24190,13 @@ (define_expand "_round_vec_pack_sfix" }) (define_insn "sse4_1_round" - [(set (match_operand:VF_128 0 "register_operand" "=Yr,*x,x,v") - (vec_merge:VF_128 - (unspec:VF_128 - [(match_operand:VF_128 2 "nonimmediate_operand" "Yrjm,*xjm,xjm,vm") + [(set (match_operand:VFH_128 0 "register_operand" "=Yr,*x,x,v") + (vec_merge:VFH_128 + (unspec:VFH_128 + [(match_operand:VFH_128 2 "nonimmediate_operand" "Yrjm,*xjm,xjm,vm") (match_operand:SI 3 "const_0_to_15_operand")] UNSPEC_ROUND) - (match_operand:VF_128 1 "register_operand" "0,0,x,v") + (match_operand:VFH_128 1 "register_operand" "0,0,x,v") (const_int 1)))] "TARGET_SSE4_1" { @@ -24201,7 +24208,7 @@ (define_insn "sse4_1_round" case 2: return "vround\t{%3, %2, %1, %0|%0, %1, %2, %3}"; case 3: - if (x86_evex_reg_mentioned_p (operands, 3)) + if (x86_evex_reg_mentioned_p (operands, 3) || mode == V8HFmode) return "vrndscale\t{%3, %2, %1, %0|%0, %1, %2, %3}"; else return "vround\t{%3, %2, %1, %0|%0, %1, %2, %3}"; @@ -24264,6 +24271,17 @@ (define_expand "floor2" "TARGET_SSE4_1 && !flag_trapping_math" "operands[2] = GEN_INT (ROUND_FLOOR | ROUND_NO_EXC);") +(define_expand "lfloor2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_AVX512VL 1 "nonimmediate_operand")] + "TARGET_AVX512FP16 && !flag_trapping_math" +{ + rtx tmp = gen_reg_rtx (mode); + emit_insn (gen_floor2 (tmp, operands[1])); + emit_insn (gen_fix_trunc2 (operands[0], tmp)); + DONE; +}) + (define_expand "lfloor2" [(match_operand: 0 "register_operand") (match_operand:VF1_VF2_AVX512DQ 1 "register_operand")] @@ -24284,6 +24302,17 @@ (define_expand "ceil2" "TARGET_SSE4_1 && !flag_trapping_math" "operands[2] = GEN_INT (ROUND_CEIL | ROUND_NO_EXC);") +(define_expand "lceil2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_AVX512VL 1 "register_operand")] + "TARGET_AVX512FP16 && !flag_trapping_math" +{ + rtx tmp = gen_reg_rtx (mode); + emit_insn (gen_ceil2 (tmp, operands[1])); + emit_insn (gen_fix_trunc2 (operands[0], tmp)); + DONE; +}) + (define_expand "lceil2" [(match_operand: 0 "register_operand") (match_operand:VF1_VF2_AVX512DQ 1 "register_operand")] @@ -24306,11 +24335,11 @@ (define_expand "btrunc2" (define_expand "round2" [(set (match_dup 3) - (plus:VF - (match_operand:VF 1 "register_operand") + (plus:VFH + (match_operand:VFH 1 "register_operand") (match_dup 2))) - (set (match_operand:VF 0 "register_operand") - (unspec:VF + (set (match_operand:VFH 0 "register_operand") + (unspec:VFH [(match_dup 3) (match_dup 4)] UNSPEC_ROUND))] "TARGET_SSE4_1 && !flag_trapping_math" @@ -24338,6 +24367,17 @@ (define_expand "round2" operands[4] = GEN_INT (ROUND_TRUNC); }) +(define_expand "lround2" + [(match_operand: 0 "register_operand") + (match_operand:VHF_AVX512VL 1 "register_operand")] + "TARGET_AVX512FP16 && !flag_trapping_math" +{ + rtx tmp = gen_reg_rtx (mode); + emit_insn (gen_round2 (tmp, operands[1])); + emit_insn (gen_fix_trunc2 (operands[0], tmp)); + DONE; +}) + (define_expand "lround2" [(match_operand: 0 "register_operand") (match_operand:VF1_VF2_AVX512DQ 1 "register_operand")] diff --git a/gcc/testsuite/gcc.target/i386/part-vect-roundhf.c b/gcc/testsuite/gcc.target/i386/part-vect-roundhf.c new file mode 100644 index 00000000000..38235c157b2 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/part-vect-roundhf.c @@ -0,0 +1,217 @@ +/* { dg-do run { target avx512fp16 } } */ +/* { dg-options "-O1 -mavx512fp16 -mavx512vl -fdump-tree-slp-details -fdump-tree-optimized" } */ + +extern void abort (); + +static void do_test (void); + +#define DO_TEST do_test +#define AVX512FP16 +#include "avx512-check.h" + +#define N 16 +_Float16 b[N] = {-1.2f, 3.4f, -5.6f, 7.8f, + -9.0f, 1.0f, -2.0f, 3.0f, + -4.0f, -5.0f, 6.0f, 7.0f, + -8.0f, -9.0f, 10.0f, 11.0f}; +_Float16 r[N]; + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +round_32 (void) +{ + r[0] = __builtin_roundf16 (b[0]); + r[1] = __builtin_roundf16 (b[1]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +round_64 (void) +{ + r[0] = __builtin_roundf16 (b[0]); + r[1] = __builtin_roundf16 (b[1]); + r[2] = __builtin_roundf16 (b[2]); + r[3] = __builtin_roundf16 (b[3]); +} + +void +__attribute__((noipa,noinline,optimize("O2"))) +rint_32 (void) +{ + r[0] = __builtin_rintf16 (b[0]); + r[1] = __builtin_rintf16 (b[1]); +} + +void +__attribute__((noipa,noinline,optimize("O2"))) +rint_64 (void) +{ + r[0] = __builtin_rintf16 (b[0]); + r[1] = __builtin_rintf16 (b[1]); + r[2] = __builtin_rintf16 (b[2]); + r[3] = __builtin_rintf16 (b[3]); +} + +void +__attribute__((noipa,noinline,optimize("O2"))) +nearbyint_32 (void) +{ + r[0] = __builtin_nearbyintf16 (b[0]); + r[1] = __builtin_nearbyintf16 (b[1]); +} + +void +__attribute__((noipa,noinline,optimize("O2"))) +nearbyint_64 (void) +{ + r[0] = __builtin_nearbyintf16 (b[0]); + r[1] = __builtin_nearbyintf16 (b[1]); + r[2] = __builtin_nearbyintf16 (b[2]); + r[3] = __builtin_nearbyintf16 (b[3]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +trunc_32 (void) +{ + r[0] = __builtin_truncf16 (b[0]); + r[1] = __builtin_truncf16 (b[1]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +trunc_64 (void) +{ + r[0] = __builtin_truncf16 (b[0]); + r[1] = __builtin_truncf16 (b[1]); + r[2] = __builtin_truncf16 (b[2]); + r[3] = __builtin_truncf16 (b[3]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +floor_32 (void) +{ + r[0] = __builtin_floorf16 (b[0]); + r[1] = __builtin_floorf16 (b[1]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +floor_64 (void) +{ + r[0] = __builtin_floorf16 (b[0]); + r[1] = __builtin_floorf16 (b[1]); + r[2] = __builtin_floorf16 (b[2]); + r[3] = __builtin_floorf16 (b[3]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +ceil_32 (void) +{ + r[0] = __builtin_ceilf16 (b[0]); + r[1] = __builtin_ceilf16 (b[1]); +} + +void +__attribute__((noipa,noinline,optimize("Ofast"))) +ceil_64 (void) +{ + r[0] = __builtin_ceilf16 (b[0]); + r[1] = __builtin_ceilf16 (b[1]); + r[2] = __builtin_ceilf16 (b[2]); + r[3] = __builtin_ceilf16 (b[3]); +} + +_Float16 +__attribute__((noipa,noinline,optimize("Ofast"))) +dummy_roundf16 (_Float16 a) +{ + return __builtin_roundf16 (a); +} +static void +__attribute__ ((noinline, noclone)) +do_test (void) +{ + round_32 (); + /* check results: */ + for (int i = 0; i != 2; i++) + if (r[i] != dummy_roundf16 (b[i])) + abort (); + + round_64 (); + /* check results: */ + for (int i = 0; i != 4; i++) + if (r[i] != dummy_roundf16 (b[i])) + abort (); + + rint_32 (); + /* check results: */ + for (int i = 0; i != 2; i++) + if (r[i] != __builtin_rintf16 (b[i])) + abort (); + + rint_64 (); + /* check results: */ + for (int i = 0; i != 4; i++) + if (r[i] != __builtin_rintf16 (b[i])) + abort (); + + nearbyint_32 (); + /* check results: */ + for (int i = 0; i != 2; i++) + if (r[i] != __builtin_nearbyintf16 (b[i])) + abort (); + + nearbyint_64 (); + /* check results: */ + for (int i = 0; i != 4; i++) + if (r[i] != __builtin_nearbyintf16 (b[i])) + abort (); + + trunc_32 (); + /* check results: */ + for (int i = 0; i != 2; i++) + if (r[i] != __builtin_truncf16 (b[i])) + abort (); + + trunc_64 (); + /* check results: */ + for (int i = 0; i != 4; i++) + if (r[i] != __builtin_truncf16 (b[i])) + abort (); + + floor_32 (); + /* check results: */ + for (int i = 0; i != 2; i++) + if (r[i] != __builtin_floorf16 (b[i])) + abort (); + + floor_64 (); + /* check results: */ + for (int i = 0; i != 4; i++) + if (r[i] != __builtin_floorf16 (b[i])) + abort (); + + ceil_32 (); + /* check results: */ + for (int i = 0; i != 2; i++) + if (r[i] != __builtin_ceilf16 (b[i])) + abort (); + + ceil_64 (); + /* check results: */ + for (int i = 0; i != 4; i++) + if (r[i] != __builtin_ceilf16 (b[i])) + abort (); +} + +/* { dg-final { scan-tree-dump-times "vectorized using 8 byte vectors" 6 "slp2" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times "vectorized using 4 byte vectors" 6 "slp2" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times {(?n).CEIL \(vect} 2 "optimized" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times {(?n).FLOOR \(vect} 2 "optimized" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times {(?n).ROUND \(vect} 2 "optimized" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times {(?n).RINT \(vect} 2 "optimized" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times {(?n).NEARBYINT \(vect} 2 "optimized" { target { ! ia32 } } } } */ +/* { dg-final { scan-tree-dump-times {(?n).TRUNC \(vect} 2 "optimized" { target { ! ia32 } } } } */ diff --git a/gcc/testsuite/gcc.target/i386/part-vect-sqrtph-1.c b/gcc/testsuite/gcc.target/i386/part-vect-sqrtph-1.c new file mode 100644 index 00000000000..b7f9e7fb9b2 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/part-vect-sqrtph-1.c @@ -0,0 +1,20 @@ +/* { dg-do compile } */ +/* { dg-options "-mavx512fp16 -mavx512vl -Ofast" } */ +/* { dg-final { scan-assembler-times {(?n)vsqrtph[ \t].*%xmm[0-9]} 2 { target { ! ia32 } } } } */ +/* { dg-final { scan-assembler-times {(?n)vsqrtph[ \t].*%xmm[0-9]} 2 { target { ! ia32 } } } } */ + +void +foo16_sqrt (_Float16* a, _Float16* __restrict c) +{ + c[0] = __builtin_sqrtf16 (a[0]); + c[1] = __builtin_sqrtf16 (a[1]); +} + +void +foo32_sqrt(_Float16* a, _Float16* __restrict c) +{ + c[0] = __builtin_sqrtf16 (a[0]); + c[1] = __builtin_sqrtf16 (a[1]); + c[2] = __builtin_sqrtf16 (a[2]); + c[3] = __builtin_sqrtf16 (a[3]); +}