From patchwork Mon Nov 20 04:23:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Meissner X-Patchwork-Id: 166944 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1974496vqn; Sun, 19 Nov 2023 20:24:31 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdQmKzMnb5uGsUZJrUb476RVKbIyJyZbgCYXtUCea2cq63JGtaAgxtbvbB6wOOAsa0ahgH X-Received: by 2002:a05:622a:13c8:b0:421:c3a9:1e4f with SMTP id p8-20020a05622a13c800b00421c3a91e4fmr8577679qtk.7.1700454271233; Sun, 19 Nov 2023 20:24:31 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1700454271; cv=pass; d=google.com; s=arc-20160816; b=CUs8D1AJYL4ZZOlqUWeTTvLDoVjwkG3v+3ce3RAeL+2n9UyE+V5GrYaB/FxIxLsFY1 zvfj1u4i9KItYObzrEHCUjVpS+y3/EG9ESoVBjtrSitq01aL9dRZDR4Rux95ApUq1E2n cHdfmKLfmoDs/kiLJZ5TIU7XiV19lX0Enjq/1X8X2f7LXVisEkJHAlkNBJlyVJLbg3Uk OKmJPZRdIK200JrzWYNEdZNFg7XU8e7tuqn+unookZCyefQ7CzWRdXtFD2SlW3INQoE0 6Bz2yjpYHkaA5cGrbi72dHIrz9BKYSI2fub6jlLae5YXoHRaZEqjM32jCjiSTUFN+8Fr Cmzw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:in-reply-to:content-disposition :mime-version:references:mail-followup-to:message-id:subject:to:from :date:dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=/8EYxFycTBCeELh9p+HR2BJBK1Zu0ADnWH5rPx8AyKM=; fh=jH+DijE7mz3ySVsRmzRqEe/ioBeGu3vnvA+jm2JjCm8=; b=BJ4Uxff3YyxM1LGKJWavLYeIkDBHnMnE3zIWIb3dh8G4e0IdT7Yj4ZUAR7FeltAuBd NgUeLBa++U9EygyC6hF5MRAlqZWRuOYzgoos+lvI9uKUe+5zflSEZiAUtADlMqNFmm28 sF8Hzw5EitXvesu5WgIhuw8oC5O9spuQcIF2nuEN8r6ZVtQPrK1MPYfmNFth8F1+eLe4 wHMvp3MEGoASTNmXyjXUyHTYuSnYuF2qOVxJGet+Aln/pbd07V7ayZeyVTKElHpyE7mx fcs10aKhPhIQjHp/t6xhs+KnfUW4EYog2t4fqUCOZz60IhuTS5ZZUqun9yEze29VIe4B zANA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b="jp/w5Ico"; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id t27-20020a05622a181b00b0041812c64682si6402862qtc.112.2023.11.19.20.24.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 20:24:31 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b="jp/w5Ico"; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 051A03858421 for ; Mon, 20 Nov 2023 04:24:31 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id C6D2B3858C78 for ; Mon, 20 Nov 2023 04:24:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C6D2B3858C78 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linux.ibm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C6D2B3858C78 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700454246; cv=none; b=NF69YZyHimdFZY+l6WhkDmWZy9KUZu9epB6oX7yAqwWm1vhpCVqlHCb0tF3z+AWfi4iVCsCkQENiENgup8bxHAEs6eG0Fj0DTZvw3kjUOdk6c4KDpUmxAjjjqYfSoXFxyhPC5OO6r/aBA+F+uy+giLcttzRAQMu4liEEcEvl8OM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700454246; c=relaxed/simple; bh=F/tV1B8HrVZN1r/P9sBuSIXOAEHLz6vQKeaUxfG9JvQ=; h=DKIM-Signature:Date:From:To:Subject:Message-ID:MIME-Version; b=pfYKnUs+qz7ylH/HCXUFB/L1kDF9aSeJaIXFskCQzTb6vNp5Np/EUZH4HYcqIwkkfmXLb8u5SsfBazLBRFAtcevktrq8P6j5L7B+zNW19UpenO9+RACbDYEp9d5EXZJtvtIwezBcBkK4Se4XThS6VzTjbgOpuvOIwJrxD583Z6A= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from pps.filterd (m0353726.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3AK2jbfi018549; Mon, 20 Nov 2023 04:24:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : subject : message-id : references : mime-version : content-type : in-reply-to; s=pp1; bh=/8EYxFycTBCeELh9p+HR2BJBK1Zu0ADnWH5rPx8AyKM=; b=jp/w5IcojWSXSNpn4cWYhyN4/dzkVyInCysuYHcsGijwhik9I007g/JT9Hp5u6M/bxNJ 47aqEFeky7nYYWTZZrIP8ijq2Ofm/S7epCaMo9TrLvoguWw7csUp0aWL359NcNYFWeDw MFtI7hvDwtSDSsdw5cFjdn8QT+vpDZsItq8Ii53mLL5Di++IQ5iBXJ9SL3SMNeRe+e+s zXVldRxuZNCE3IchZAsup5wkW2L/GPbsdmYe1TAqphfy3pJPxU4p4ZG2TCwwcLQZFTI3 2+ptA0BMLzGIFBc6rxcsC60c4ZCZi1G9tayy76nDSaxEVZXhp74E1gLi8+d4VxSnI6Ib Yg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3uf0k3k177-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 Nov 2023 04:24:02 +0000 Received: from m0353726.ppops.net (m0353726.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3AK4Fb3H009953; Mon, 20 Nov 2023 04:24:01 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3uf0k3k16w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 Nov 2023 04:24:01 +0000 Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3AK1bhx0000491; Mon, 20 Nov 2023 04:24:00 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 3ufaa1paqb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 20 Nov 2023 04:24:00 +0000 Received: from smtpav01.wdc07v.mail.ibm.com (smtpav01.wdc07v.mail.ibm.com [10.39.53.228]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 3AK4O0Hs20644450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 20 Nov 2023 04:24:00 GMT Received: from smtpav01.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E30C158059; Mon, 20 Nov 2023 04:23:59 +0000 (GMT) Received: from smtpav01.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2BF3E5804B; Mon, 20 Nov 2023 04:23:59 +0000 (GMT) Received: from cowardly-lion.the-meissners.org (unknown [9.61.1.46]) by smtpav01.wdc07v.mail.ibm.com (Postfix) with ESMTPS; Mon, 20 Nov 2023 04:23:59 +0000 (GMT) Date: Sun, 19 Nov 2023 23:23:57 -0500 From: Michael Meissner To: Michael Meissner , gcc-patches@gcc.gnu.org, Segher Boessenkool , "Kewen.Lin" , David Edelsohn , Peter Bergner Subject: [PATCH 2/4] Vector pair floating point support for PowerPC Message-ID: Mail-Followup-To: Michael Meissner , gcc-patches@gcc.gnu.org, Segher Boessenkool , "Kewen.Lin" , David Edelsohn , Peter Bergner References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-TM-AS-GCONF: 00 X-Proofpoint-GUID: y1_MIaAR-h1y8XpwRJbQPzWzpQRk1NRB X-Proofpoint-ORIG-GUID: -9_K4bu6J29YCLcQxwaG1gwtDp8mVgrd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-11-20_01,2023-11-17_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 phishscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2311200029 X-Spam-Status: No, score=-10.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783055538062585705 X-GMAIL-MSGID: 1783055538062585705 The first patch in the vector pair series was previous posted. This patch needs that first patch. The first patch implemented the basic modes, and it allows for initialization of the modes. In addition, I added some optimizations for extracting and setting fields within the vector pair. This is the second patch in the vector pair series. It adds the basic support to do the normal floating point arithmetic operations like add, subtract, etc. I have also put in combine insns to enable combining the fma (fused multiply-add) instructions with negation to generate the 4 fma operations on the PowerPC. The third patch will implement the integer vector pair support. The fourth patch will provide new tests to the test suite. When I test a saxpy type loop (a[i] += (b[i] * c[i])), I generally see a 10% improvement over either auto-factorization, or just using the vector types. I have tested these patches on a little endian power10 system. With -vector-size-32 disabled by default, there are no regressions in the test suite. I have also built and run the tests on both little endian power 9 and big endian 9 power systems, and there are no regressions. Can I check these patches into the master branch? 2023-11-19 Michael Meisner gcc/ * config/rs6000/rs6000-protos.h (split_unary_vector_pair): New declaration. (split_binary_vector_pair): Likewise. (split_fma_vector_pair): Likewise. * config/rs6000/rs6000.cc (split_unary_vector_pair): New function. (split_binary_vector_pair): Likewise. (split_fma_vector_pair): Likewise. * config/rs6000/vector-pair.md (VPAIR_FP): New mode iterator. (VPAIR_FP_UNARY): New code iterator. (VPAIR_FP_BINARY): Likewise. (vpair_op): New code attribute. (2, VPAIR_FP and VPAIR_FP_UNARY iterators): New insns. (sqrtv8sf2): Likewise. (sqrtv4df2): Likewise. (nabs2): Likewise. (3, VPAIR_FP and VP_FP_BINARY iterators): Likewise. (divv8sf3): Likewise. (divv4df3): Likewise. (fma4): Likewise. (fms4): Likewise. (nfma4): Likewise. (nfms4): Likewise. (fma_fpcontract_4): Likewise. (fms_fpcontract_4): Likewise. (nfma_fpcontract_): Likewise. (nfms_fpcontract_): Likewise. --- gcc/config/rs6000/rs6000-protos.h | 5 + gcc/config/rs6000/rs6000.cc | 74 +++++++ gcc/config/rs6000/vector-pair.md | 310 ++++++++++++++++++++++++++++++ 3 files changed, 389 insertions(+) diff --git a/gcc/config/rs6000/rs6000-protos.h b/gcc/config/rs6000/rs6000-protos.h index e17d73cb4ca..dac48f199ab 100644 --- a/gcc/config/rs6000/rs6000-protos.h +++ b/gcc/config/rs6000/rs6000-protos.h @@ -141,6 +141,11 @@ extern void rs6000_emit_swsqrt (rtx, rtx, bool); extern void output_toc (FILE *, rtx, int, machine_mode); extern void rs6000_fatal_bad_address (rtx); extern rtx create_TOC_reference (rtx, rtx); +extern void split_unary_vector_pair (machine_mode, rtx [], rtx (*)(rtx, rtx)); +extern void split_binary_vector_pair (machine_mode, rtx [], + rtx (*)(rtx, rtx, rtx)); +extern void split_fma_vector_pair (machine_mode, rtx [], + rtx (*)(rtx, rtx, rtx, rtx)); extern void rs6000_split_multireg_move (rtx, rtx); extern void rs6000_emit_le_vsx_permute (rtx, rtx, machine_mode); extern void rs6000_emit_le_vsx_move (rtx, rtx, machine_mode); diff --git a/gcc/config/rs6000/rs6000.cc b/gcc/config/rs6000/rs6000.cc index c9bd8c35e63..aeac7c9fa42 100644 --- a/gcc/config/rs6000/rs6000.cc +++ b/gcc/config/rs6000/rs6000.cc @@ -27634,6 +27634,80 @@ rs6000_split_logical (rtx operands[3], return; } +/* Split a unary vector pair insn into two separate vector insns. */ + +void +split_unary_vector_pair (machine_mode mode, /* vector mode. */ + rtx operands[], /* dest, src. */ + rtx (*func)(rtx, rtx)) /* create insn. */ +{ + rtx op0 = operands[0]; + rtx op1 = operands[1]; + machine_mode orig_mode = GET_MODE (op0); + + rtx reg0_vector0 = simplify_gen_subreg (mode, op0, orig_mode, 0); + rtx reg1_vector0 = simplify_gen_subreg (mode, op1, orig_mode, 0); + rtx reg0_vector1 = simplify_gen_subreg (mode, op0, orig_mode, 16); + rtx reg1_vector1 = simplify_gen_subreg (mode, op1, orig_mode, 16); + + emit_insn (func (reg0_vector0, reg1_vector0)); + emit_insn (func (reg0_vector1, reg1_vector1)); + return; +} + +/* Split a binary vector pair insn into two separate vector insns. */ + +void +split_binary_vector_pair (machine_mode mode, /* vector mode. */ + rtx operands[], /* dest, src. */ + rtx (*func)(rtx, rtx, rtx)) /* create insn. */ +{ + rtx op0 = operands[0]; + rtx op1 = operands[1]; + rtx op2 = operands[2]; + machine_mode orig_mode = GET_MODE (op0); + + rtx reg0_vector0 = simplify_gen_subreg (mode, op0, orig_mode, 0); + rtx reg1_vector0 = simplify_gen_subreg (mode, op1, orig_mode, 0); + rtx reg2_vector0 = simplify_gen_subreg (mode, op2, orig_mode, 0); + rtx reg0_vector1 = simplify_gen_subreg (mode, op0, orig_mode, 16); + rtx reg1_vector1 = simplify_gen_subreg (mode, op1, orig_mode, 16); + rtx reg2_vector1 = simplify_gen_subreg (mode, op2, orig_mode, 16); + + emit_insn (func (reg0_vector0, reg1_vector0, reg2_vector0)); + emit_insn (func (reg0_vector1, reg1_vector1, reg2_vector1)); + return; +} + +/* Split a fused multiply-add vector pair insn into two separate vector + insns. */ + +void +split_fma_vector_pair (machine_mode mode, /* vector mode. */ + rtx operands[], /* dest, src. */ + rtx (*func)(rtx, rtx, rtx, rtx)) /* create insn. */ +{ + rtx op0 = operands[0]; + rtx op1 = operands[1]; + rtx op2 = operands[2]; + rtx op3 = operands[3]; + machine_mode orig_mode = GET_MODE (op0); + + rtx reg0_vector0 = simplify_gen_subreg (mode, op0, orig_mode, 0); + rtx reg1_vector0 = simplify_gen_subreg (mode, op1, orig_mode, 0); + rtx reg2_vector0 = simplify_gen_subreg (mode, op2, orig_mode, 0); + rtx reg3_vector0 = simplify_gen_subreg (mode, op3, orig_mode, 0); + + rtx reg0_vector1 = simplify_gen_subreg (mode, op0, orig_mode, 16); + rtx reg1_vector1 = simplify_gen_subreg (mode, op1, orig_mode, 16); + rtx reg2_vector1 = simplify_gen_subreg (mode, op2, orig_mode, 16); + rtx reg3_vector1 = simplify_gen_subreg (mode, op3, orig_mode, 16); + + emit_insn (func (reg0_vector0, reg1_vector0, reg2_vector0, reg3_vector0)); + emit_insn (func (reg0_vector1, reg1_vector1, reg2_vector1, reg3_vector1)); + return; +} + /* Emit instructions to move SRC to DST. Called by splitters for multi-register moves. It will emit at most one instruction for each register that is accessed; that is, it won't emit li/lis pairs diff --git a/gcc/config/rs6000/vector-pair.md b/gcc/config/rs6000/vector-pair.md index 068f562200a..8e2d7e5cc5b 100644 --- a/gcc/config/rs6000/vector-pair.md +++ b/gcc/config/rs6000/vector-pair.md @@ -31,9 +31,34 @@ ;; integer vector pairs for perumte operations (and eventually compare). (define_mode_iterator VPAIR [V32QI V16HI V8SI V4DI V8SF V4DF]) +;; Floating point vector pair ops +(define_mode_iterator VPAIR_FP [V8SF V4DF]) + +;; Iterator for floating point unary/binary operations. +(define_code_iterator VPAIR_FP_UNARY [abs neg]) +(define_code_iterator VPAIR_FP_BINARY [plus minus mult smin smax]) + ;; Iterator for vector pairs with double word elements (define_mode_iterator VPAIR_DWORD [V4DI V4DF]) +;; Give the insn name from the opertion +(define_code_attr vpair_op [(abs "abs") + (div "div") + (and "and") + (fma "fma") + (ior "ior") + (minus "sub") + (mult "mul") + (neg "neg") + (not "one_cmpl") + (plus "add") + (smin "smin") + (smax "smax") + (sqrt "sqrt") + (umin "umin") + (umax "umax") + (xor "xor")]) + ;; Map vector pair mode to vector mode in upper case after the vector pair is ;; split to two vectors. (define_mode_attr VPAIR_VECTOR [(V32QI "V16QI") @@ -317,3 +342,288 @@ (define_expand "vpair_splat_" emit_insn (gen_vpair_concat_ (op0, tmp, tmp)); DONE; }) + +;; Vector pair floating point arithmetic unary operations +(define_insn_and_split "2" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa") + (VPAIR_FP_UNARY:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_unary_vector_pair (mode, operands, + gen_2); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Sqrt needs different type attributes between V8SF and V4DF +(define_insn_and_split "sqrtv8sf2" + [(set (match_operand:V8SF 0 "vsx_register_operand" "=wa") + (sqrt:V8SF + (match_operand:V8SF 1 "vsx_register_operand" "wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_unary_vector_pair (V4SFmode, operands, gen_sqrtv4sf2); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfdiv")]) + +(define_insn_and_split "sqrtv4df2" + [(set (match_operand:V4DF 0 "vsx_register_operand" "=wa") + (sqrt:V4DF + (match_operand:V4DF 1 "vsx_register_operand" "wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_unary_vector_pair (V2DFmode, operands, gen_sqrtv2df2); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecdiv")]) + +;; Optimize negative absolute value (both floating point and integer) +(define_insn_and_split "nabs2" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa") + (neg:VPAIR_FP + (abs:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "wa"))))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_unary_vector_pair (mode, operands, + gen_vsx_nabs2); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Vector pair floating point arithmetic binary operations +(define_insn_and_split "3" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa") + (VPAIR_FP_BINARY:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_binary_vector_pair (mode, operands, + gen_3); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Divide needs different type attributes between V8SF and V4DF +(define_insn_and_split "divv8sf3" + [(set (match_operand:V8SF 0 "vsx_register_operand" "=wa") + (div:V8SF + (match_operand:V8SF 1 "vsx_register_operand" "wa") + (match_operand:V8SF 2 "vsx_register_operand" "wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_binary_vector_pair (V4SFmode, operands, gen_divv4sf3); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfdiv")]) + +(define_insn_and_split "divv4df3" + [(set (match_operand:V4DF 0 "vsx_register_operand" "=wa") + (div:V4DF + (match_operand:V4DF 1 "vsx_register_operand" "wa") + (match_operand:V4DF 2 "vsx_register_operand" "wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_binary_vector_pair (V2DFmode, operands, gen_divv2df3); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecdiv")]) + +;; Vector pair floating point fused multiply-add +(define_insn_and_split "fma4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (fma:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0") + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_fma_vector_pair (mode, operands, + gen_fma4); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Vector pair floating point fused multiply-subtract +(define_insn_and_split "fms4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (fma:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0") + (neg:VPAIR_FP + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa"))))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_fma_vector_pair (mode, operands, + gen_fms4); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Vector pair floating point negative fused multiply-add +(define_insn_and_split "nfma4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (neg:VPAIR_FP + (fma:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0") + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa"))))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_fma_vector_pair (mode, operands, + gen_nfma4); + DONE; +} + [(set_attr "length" "8")]) + +;; Vector pair floating point fused negative multiply-subtract +(define_insn_and_split "nfms4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (neg:VPAIR_FP + (fma:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0") + (neg:VPAIR_FP + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa")))))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32" + "#" + "&& reload_completed" + [(const_int 0)] +{ + split_fma_vector_pair (mode, operands, + gen_nfms4); + DONE; +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Optimize vector pair (a * b) + c into fma (a, b, c) +(define_insn_and_split "*fma_fpcontract_4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (plus:VPAIR_FP + (mult:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0")) + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32 + && flag_fp_contract_mode == FP_CONTRACT_FAST" + "#" + "&& 1" + [(set (match_dup 0) + (fma:VPAIR_FP (match_dup 1) + (match_dup 2) + (match_dup 3)))] +{ +} + [(set_attr "length" "8")]) + +;; Optimize vector pair (a * b) - c into fma (a, b, -c) +(define_insn_and_split "*fms_fpcontract_4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (minus:VPAIR_FP + (mult:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0")) + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa")))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32 + && flag_fp_contract_mode == FP_CONTRACT_FAST" + "#" + "&& 1" + [(set (match_dup 0) + (fma:VPAIR_FP (match_dup 1) + (match_dup 2) + (neg:VPAIR_FP + (match_dup 3))))] +{ +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) + +;; Optimize vector pair -((a * b) + c) into -fma (a, b, c) +(define_insn_and_split "*nfma_fpcontract_4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (neg:VPAIR_FP + (plus:VPAIR_FP + (mult:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0")) + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa"))))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32 + && flag_fp_contract_mode == FP_CONTRACT_FAST" + "#" + "&& 1" + [(set (match_dup 0) + (neg:VPAIR_FP + (fma:VPAIR_FP (match_dup 1) + (match_dup 2) + (match_dup 3))))] +{ +} + [(set_attr "length" "8")]) + +;; Optimize vector pair -((a * b) - c) into -fma (a, b, -c) +(define_insn_and_split "*nfms_fpcontract_4" + [(set (match_operand:VPAIR_FP 0 "vsx_register_operand" "=wa,wa") + (neg:VPAIR_FP + (minus:VPAIR_FP + (mult:VPAIR_FP + (match_operand:VPAIR_FP 1 "vsx_register_operand" "%wa,wa") + (match_operand:VPAIR_FP 2 "vsx_register_operand" "wa,0")) + (match_operand:VPAIR_FP 3 "vsx_register_operand" "0,wa"))))] + "TARGET_MMA && TARGET_VECTOR_SIZE_32 + && flag_fp_contract_mode == FP_CONTRACT_FAST" + "#" + "&& 1" + [(set (match_dup 0) + (neg:VPAIR_FP + (fma:VPAIR_FP (match_dup 1) + (match_dup 2) + (neg:VPAIR_FP + (match_dup 3)))))] +{ +} + [(set_attr "length" "8") + (set_attr "type" "vecfloat")]) +