From patchwork Fri Jan 5 23:42:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Meissner X-Patchwork-Id: 185569 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp6527688dyb; Fri, 5 Jan 2024 15:42:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IE9QbYEBcxm/gGf1i/VRGQXjWbjXh7skpxxX2YoqeXK4sp0kEVdaxcXq5uzt0SkrbiH6LoY X-Received: by 2002:ac8:7d02:0:b0:429:76fb:3f81 with SMTP id g2-20020ac87d02000000b0042976fb3f81mr263236qtb.14.1704498174882; Fri, 05 Jan 2024 15:42:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1704498174; cv=pass; d=google.com; s=arc-20160816; b=xq/XyyrhzBCg6Yb3AzYzW0ykC/wY/IffzxKJjiA0O0aqLW90aUaRUAgQcJp1AQPw7d PxcE2u+hITTbMBqrD5zjHLwozw/vjxtyuqQWz1+RBiI/fQHzFvyQGJ7acc6uvXy8/8+X ADQyp5xeWBLMhcEromQyGrID8Qy2/3+FlDTsExx3kv8gWKsaTNRFRqmmYLls5GfAIGF3 2qmJujLXXbVAJJXEVc88meLvJLsLNaChmieS/SeJWv12FBCJzSPAVDw8Fj++DVQcsR52 09sHsC2iHNy0mhRrfD+RjQxEsQnyMg7o776VGTsevpllX/RZOXf8QvM+SvC0SsQsiTDo gV9Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:in-reply-to:content-disposition :mime-version:references:mail-followup-to:message-id:subject:to:from :date:dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=ur2wo2urk04Snpcnb80HcN+ux1WnbbCCbog4l0mpdpo=; fh=jH+DijE7mz3ySVsRmzRqEe/ioBeGu3vnvA+jm2JjCm8=; b=bvhxjQ9oc9//ghedV30StVKslTRKsPOhWu7xwD6JQ6IEE7xiQ67ojKdL+Vju3QehQv vVC8em0BVbfmqoZkov3ew4ONpd5dbiA/f4yTAE8IInj2qJOlNBHeqU2YtBCEcMep8Ol6 01aPg87NEdRnrfQifpaBKGLAOV5DwE3kEZs897PlYUCZM55fqtyCjb6i5uhgPmQVaerI IvAgwniSD9cTnJICIKbCUDKWpgbsqh+5DxtbQcshVl1wqv0tUceGtBKnghD8fgzBdYto mBNNp8e967Jd9LYZw+TfHfr3DD27mGtNFYp5QdpfGOD1QMiFYpuIJS+093Zstzc3hnBd cttQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=Egz56SM7; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id f28-20020a05622a1a1c00b00427931c3f73si2820328qtb.643.2024.01.05.15.42.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 15:42:54 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=Egz56SM7; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8F503385DC08 for ; Fri, 5 Jan 2024 23:42:54 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id 0D48C3858419 for ; Fri, 5 Jan 2024 23:42:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0D48C3858419 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linux.ibm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0D48C3858419 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1704498133; cv=none; b=FIGBjUGAz8LtG41T01rDAI+vw2jNt9XuT0qc0MNsdHHWGUwsjTybZ7ZP7cMh7z/UBIW5cJwst8fLUTca/JgIS8lLIPaNNtNmZAlmB0inuhq7ESmHQPMzUnogG3NQ/qy7hT5vqiw1xvlZqAL8l/b9rTP7V4h56Gxmb3ozbhNqJ/s= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1704498133; c=relaxed/simple; bh=/BRovvFLkogB0/K8hjAL0e9cINm7maxSIkjb+6cKMgg=; h=DKIM-Signature:Date:From:To:Subject:Message-ID:MIME-Version; b=O3jWcgemHZTKFJvsj2njlSN7KbQTfqzXZHnbLqWJOEWoDag4oI7yMO07+hE/PXWXfIBxpy1Hn24KUemUwTI0gQg/e3BSGRrc1AS5tcUU8FkpgVTQeo07Vr20DypqCp1Tidf7n61YUC4ecoDuNXFqLQHTd9rmfnB7T8d7VkesW6Q= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from pps.filterd (m0353728.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 405MmAeA003364; Fri, 5 Jan 2024 23:42:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : subject : message-id : references : mime-version : content-type : in-reply-to; s=pp1; bh=ur2wo2urk04Snpcnb80HcN+ux1WnbbCCbog4l0mpdpo=; b=Egz56SM7GopQC9sFMAMus3xzGJsNCNiCKRUZINUZRW0Vcf7EcUiz0fNJeHTLdTSkRbvd 8rZCrifuS7TqWtB1N/jsN5ZJOLqozLom81pCwwI/tkZmOyvKliXkNGGkiPjZCuaep9oF rapQJDX9PQAJAUxOwqH9sqMDbGkq2F15eUvCoZoXpH2WbHj4CX1uSxskylncSvrtPOlX tXDlX43xx35TBoT/lefhaZqzHYKecT7dwewJe6CxMDbjvUnEtz1tcv4GnQ7w62YMy11R MwFuHBLOCW+R4Ur3f8Y1iCkzn8WYYsRzOH+amDt7flzwmbf5nwDuYnoW+2wUlNCkFQ5d Gg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3vervxbadq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 05 Jan 2024 23:42:07 +0000 Received: from m0353728.ppops.net (m0353728.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 405Ng7Pi028425; Fri, 5 Jan 2024 23:42:07 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3vervxbade-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 05 Jan 2024 23:42:07 +0000 Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 405L4dsl024559; Fri, 5 Jan 2024 23:42:06 GMT Received: from smtprelay07.wdc07v.mail.ibm.com ([172.16.1.74]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 3vb082t1s2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 05 Jan 2024 23:42:06 +0000 Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay07.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 405Ng4LY15401718 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 5 Jan 2024 23:42:05 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9D31F58056; Fri, 5 Jan 2024 23:42:04 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2211058052; Fri, 5 Jan 2024 23:42:04 +0000 (GMT) Received: from cowardly-lion.the-meissners.org (unknown [9.61.128.150]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTPS; Fri, 5 Jan 2024 23:42:04 +0000 (GMT) Date: Fri, 5 Jan 2024 18:42:02 -0500 From: Michael Meissner To: Michael Meissner , gcc-patches@gcc.gnu.org, Segher Boessenkool , "Kewen.Lin" , David Edelsohn , Peter Bergner Subject: Repost [PATCH 6/6] PowerPC: Add support for 1,024 bit DMR registers. Message-ID: Mail-Followup-To: Michael Meissner , gcc-patches@gcc.gnu.org, Segher Boessenkool , "Kewen.Lin" , David Edelsohn , Peter Bergner References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 7FWnuQpbCJ6b80ZQXR29FayQShVtpscf X-Proofpoint-GUID: aqbNBOH8lQbcw1X7CCVC4COv5CxDw7mb X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-05_08,2024-01-05_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 mlxscore=0 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401050181 X-Spam-Status: No, score=-11.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787295878024953412 X-GMAIL-MSGID: 1787295878024953412 This patch is a prelimianry patch to add the full 1,024 bit dense math register (DMRs) for -mcpu=future. The MMA 512-bit accumulators map onto the top of the DMR register. This patch only adds the new 1,024 bit register support. It does not add support for any instructions that need 1,024 bit registers instead of 512 bit registers. I used the new mode 'TDOmode' to be the opaque mode used for 1,204 bit registers. The 'wD' constraint added in previous patches is used for these registers. I added support to do load and store of DMRs via the VSX registers, since there are no load/store dense math instructions. I added the new keyword '__dmr' to create 1,024 bit types that can be loaded into DMRs. At present, I don't have aliases for __dmr512 and __dmr1024 that we've discussed internally. The patches have been tested on both little and big endian systems. Can I check it into the master branch? 2024-01-05 Michael Meissner gcc/ * config/rs6000/mma.md (UNSPEC_DM_INSERT512_UPPER): New unspec. (UNSPEC_DM_INSERT512_LOWER): Likewise. (UNSPEC_DM_EXTRACT512): Likewise. (UNSPEC_DMR_RELOAD_FROM_MEMORY): Likewise. (UNSPEC_DMR_RELOAD_TO_MEMORY): Likewise. (movtdo): New define_expand and define_insn_and_split to implement 1,024 bit DMR registers. (movtdo_insert512_upper): New insn. (movtdo_insert512_lower): Likewise. (movtdo_extract512): Likewise. (reload_dmr_from_memory): Likewise. (reload_dmr_to_memory): Likewise. * config/rs6000/rs6000-builtin.cc (rs6000_type_string): Add DMR support. (rs6000_init_builtins): Add support for __dmr keyword. * config/rs6000/rs6000-call.cc (rs6000_return_in_memory): Add support for TDOmode. (rs6000_function_arg): Likewise. * config/rs6000/rs6000-modes.def (TDOmode): New mode. * config/rs6000/rs6000.cc (rs6000_hard_regno_nregs_internal): Add support for TDOmode. (rs6000_hard_regno_mode_ok_uncached): Likewise. (rs6000_hard_regno_mode_ok): Likewise. (rs6000_modes_tieable_p): Likewise. (rs6000_debug_reg_global): Likewise. (rs6000_setup_reg_addr_masks): Likewise. (rs6000_init_hard_regno_mode_ok): Add support for TDOmode. Setup reload hooks for DMR mode. (reg_offset_addressing_ok_p): Add support for TDOmode. (rs6000_emit_move): Likewise. (rs6000_secondary_reload_simple_move): Likewise. (rs6000_secondary_reload_class): Likewise. (rs6000_mangle_type): Add mangling for __dmr type. (rs6000_dmr_register_move_cost): Add support for TDOmode. (rs6000_split_multireg_move): Likewise. (rs6000_invalid_conversion): Likewise. * config/rs6000/rs6000.h (VECTOR_ALIGNMENT_P): Add TDOmode. (enum rs6000_builtin_type_index): Add DMR type nodes. (dmr_type_node): Likewise. (ptr_dmr_type_node): Likewise. gcc/testsuite/ * gcc.target/powerpc/dm-1024bit.c: New test. --- gcc/config/rs6000/mma.md | 152 ++++++++++++++++++ gcc/config/rs6000/rs6000-builtin.cc | 13 ++ gcc/config/rs6000/rs6000-call.cc | 13 +- gcc/config/rs6000/rs6000-modes.def | 4 + gcc/config/rs6000/rs6000.cc | 135 ++++++++++++---- gcc/config/rs6000/rs6000.h | 7 +- gcc/testsuite/gcc.target/powerpc/dm-1024bit.c | 63 ++++++++ 7 files changed, 351 insertions(+), 36 deletions(-) create mode 100644 gcc/testsuite/gcc.target/powerpc/dm-1024bit.c diff --git a/gcc/config/rs6000/mma.md b/gcc/config/rs6000/mma.md index f06e6bbb184..37de9030903 100644 --- a/gcc/config/rs6000/mma.md +++ b/gcc/config/rs6000/mma.md @@ -92,6 +92,11 @@ (define_c_enum "unspec" UNSPEC_MMA_XXMFACC UNSPEC_MMA_XXMTACC UNSPEC_DM_ASSEMBLE_ACC + UNSPEC_DM_INSERT512_UPPER + UNSPEC_DM_INSERT512_LOWER + UNSPEC_DM_EXTRACT512 + UNSPEC_DMR_RELOAD_FROM_MEMORY + UNSPEC_DMR_RELOAD_TO_MEMORY ]) (define_c_enum "unspecv" @@ -879,3 +884,150 @@ (define_insn "mma_" [(set_attr "type" "mma") (set_attr "prefixed" "yes") (set_attr "isa" "dm,not_dm,not_dm")]) + + +;; TDOmode (i.e. __dmr). +(define_expand "movtdo" + [(set (match_operand:TDO 0 "nonimmediate_operand") + (match_operand:TDO 1 "input_operand"))] + "TARGET_DENSE_MATH" +{ + rs6000_emit_move (operands[0], operands[1], TDOmode); + DONE; +}) + +(define_insn_and_split "*movtdo" + [(set (match_operand:TDO 0 "nonimmediate_operand" "=wa,m,wa,wD,wD,wa") + (match_operand:TDO 1 "input_operand" "m,wa,wa,wa,wD,wD"))] + "TARGET_DENSE_MATH + && (gpc_reg_operand (operands[0], TDOmode) + || gpc_reg_operand (operands[1], TDOmode))" + "@ + # + # + # + # + dmmr %0,%1 + #" + "&& reload_completed + && (!dmr_operand (operands[0], TDOmode) || !dmr_operand (operands[1], TDOmode))" + [(const_int 0)] +{ + rtx op0 = operands[0]; + rtx op1 = operands[1]; + + if (REG_P (op0) && REG_P (op1)) + { + int regno0 = REGNO (op0); + int regno1 = REGNO (op1); + + if (DMR_REGNO_P (regno0) && VSX_REGNO_P (regno1)) + { + rtx op1_upper = gen_rtx_REG (XOmode, regno1); + rtx op1_lower = gen_rtx_REG (XOmode, regno1 + 4); + emit_insn (gen_movtdo_insert512_upper (op0, op1_upper)); + emit_insn (gen_movtdo_insert512_lower (op0, op0, op1_lower)); + DONE; + } + + else if (VSX_REGNO_P (regno0) && DMR_REGNO_P (regno1)) + { + rtx op0_upper = gen_rtx_REG (XOmode, regno0); + rtx op0_lower = gen_rtx_REG (XOmode, regno0 + 4); + emit_insn (gen_movtdo_extract512 (op0_upper, op1, const0_rtx)); + emit_insn (gen_movtdo_extract512 (op0_lower, op1, const1_rtx)); + DONE; + } + } + + rs6000_split_multireg_move (operands[0], operands[1]); + DONE; +} + [(set_attr "type" "vecload,vecstore,vecmove,vecmove,vecmove,vecmove") + (set_attr "length" "*,*,32,8,*,8") + (set_attr "max_prefixed_insns" "4,4,*,*,*,*")]) + +;; Move from VSX registers to DMR registers via two insert 512 bit +;; instructions. +(define_insn "movtdo_insert512_upper" + [(set (match_operand:TDO 0 "dmr_operand" "=wD") + (unspec:TDO [(match_operand:XO 1 "vsx_register_operand" "wa")] + UNSPEC_DM_INSERT512_UPPER))] + "TARGET_DENSE_MATH" + "dmxxinstdmr512 %0,%1,%Y1,0" + [(set_attr "type" "mma")]) + +(define_insn "movtdo_insert512_lower" + [(set (match_operand:TDO 0 "dmr_operand" "=wD") + (unspec:TDO [(match_operand:TDO 1 "dmr_operand" "0") + (match_operand:XO 2 "vsx_register_operand" "wa")] + UNSPEC_DM_INSERT512_LOWER))] + "TARGET_DENSE_MATH" + "dmxxinstdmr512 %0,%2,%Y2,1" + [(set_attr "type" "mma")]) + +;; Move from DMR registers to VSX registers via two extract 512 bit +;; instructions. +(define_insn "movtdo_extract512" + [(set (match_operand:XO 0 "vsx_register_operand" "=wa") + (unspec:XO [(match_operand:TDO 1 "dmr_operand" "wD") + (match_operand 2 "const_0_to_1_operand" "n")] + UNSPEC_DM_EXTRACT512))] + "TARGET_DENSE_MATH" + "dmxxextfdmr512 %0,%Y0,%1,%2" + [(set_attr "type" "mma")]) + +;; Reload DMR registers from memory +(define_insn_and_split "reload_dmr_from_memory" + [(set (match_operand:TDO 0 "dmr_operand" "=wD") + (unspec:TDO [(match_operand:TDO 1 "memory_operand" "m")] + UNSPEC_DMR_RELOAD_FROM_MEMORY)) + (clobber (match_operand:XO 2 "vsx_register_operand" "=wa"))] + "TARGET_DENSE_MATH" + "#" + "&& reload_completed" + [(const_int 0)] +{ + rtx dest = operands[0]; + rtx src = operands[1]; + rtx tmp = operands[2]; + rtx mem_upper = adjust_address (src, XOmode, BYTES_BIG_ENDIAN ? 0 : 32); + rtx mem_lower = adjust_address (src, XOmode, BYTES_BIG_ENDIAN ? 32 : 0); + + emit_move_insn (tmp, mem_upper); + emit_insn (gen_movtdo_insert512_upper (dest, tmp)); + + emit_move_insn (tmp, mem_lower); + emit_insn (gen_movtdo_insert512_lower (dest, dest, tmp)); + DONE; +} + [(set_attr "length" "16") + (set_attr "max_prefixed_insns" "2") + (set_attr "type" "vecload")]) + +;; Reload dense math registers to memory +(define_insn_and_split "reload_dmr_to_memory" + [(set (match_operand:TDO 0 "memory_operand" "=m") + (unspec:TDO [(match_operand:TDO 1 "dmr_operand" "wD")] + UNSPEC_DMR_RELOAD_TO_MEMORY)) + (clobber (match_operand:XO 2 "vsx_register_operand" "=wa"))] + "TARGET_DENSE_MATH" + "#" + "&& reload_completed" + [(const_int 0)] +{ + rtx dest = operands[0]; + rtx src = operands[1]; + rtx tmp = operands[2]; + rtx mem_upper = adjust_address (dest, XOmode, BYTES_BIG_ENDIAN ? 0 : 32); + rtx mem_lower = adjust_address (dest, XOmode, BYTES_BIG_ENDIAN ? 32 : 0); + + emit_insn (gen_movtdo_extract512 (tmp, src, const0_rtx)); + emit_move_insn (mem_upper, tmp); + + emit_insn (gen_movtdo_extract512 (tmp, src, const1_rtx)); + emit_move_insn (mem_lower, tmp); + DONE; +} + [(set_attr "length" "16") + (set_attr "max_prefixed_insns" "2")]) diff --git a/gcc/config/rs6000/rs6000-builtin.cc b/gcc/config/rs6000/rs6000-builtin.cc index 6698274031b..54868d2009c 100644 --- a/gcc/config/rs6000/rs6000-builtin.cc +++ b/gcc/config/rs6000/rs6000-builtin.cc @@ -495,6 +495,8 @@ const char *rs6000_type_string (tree type_node) return "__vector_pair"; else if (type_node == vector_quad_type_node) return "__vector_quad"; + else if (type_node == dmr_type_node) + return "__dmr"; return "unknown"; } @@ -781,6 +783,17 @@ rs6000_init_builtins (void) t = build_qualified_type (vector_quad_type_node, TYPE_QUAL_CONST); ptr_vector_quad_type_node = build_pointer_type (t); + dmr_type_node = make_node (OPAQUE_TYPE); + SET_TYPE_MODE (dmr_type_node, TDOmode); + TYPE_SIZE (dmr_type_node) = bitsize_int (GET_MODE_BITSIZE (TDOmode)); + TYPE_PRECISION (dmr_type_node) = GET_MODE_BITSIZE (TDOmode); + TYPE_SIZE_UNIT (dmr_type_node) = size_int (GET_MODE_SIZE (TDOmode)); + SET_TYPE_ALIGN (dmr_type_node, 512); + TYPE_USER_ALIGN (dmr_type_node) = 0; + lang_hooks.types.register_builtin_type (dmr_type_node, "__dmr"); + t = build_qualified_type (dmr_type_node, TYPE_QUAL_CONST); + ptr_dmr_type_node = build_pointer_type (t); + tdecl = add_builtin_type ("__bool char", bool_char_type_node); TYPE_NAME (bool_char_type_node) = tdecl; diff --git a/gcc/config/rs6000/rs6000-call.cc b/gcc/config/rs6000/rs6000-call.cc index 8c590903c86..6e2465204cf 100644 --- a/gcc/config/rs6000/rs6000-call.cc +++ b/gcc/config/rs6000/rs6000-call.cc @@ -437,7 +437,8 @@ rs6000_return_in_memory (const_tree type, const_tree fntype ATTRIBUTE_UNUSED) if (cfun && !cfun->machine->mma_return_type_error && TREE_TYPE (cfun->decl) == fntype - && (TYPE_MODE (type) == OOmode || TYPE_MODE (type) == XOmode)) + && (TYPE_MODE (type) == OOmode || TYPE_MODE (type) == XOmode + || TYPE_MODE (type) == TDOmode)) { /* Record we have now handled function CFUN, so the next time we are called, we do not re-report the same error. */ @@ -1641,6 +1642,16 @@ rs6000_function_arg (cumulative_args_t cum_v, const function_arg_info &arg) return NULL_RTX; } + if (mode == TDOmode) + { + if (TYPE_CANONICAL (type) != NULL_TREE) + type = TYPE_CANONICAL (type); + error ("invalid use of dense math operand of type %qs as a function " + "parameter", + IDENTIFIER_POINTER (DECL_NAME (TYPE_NAME (type)))); + return NULL_RTX; + } + /* Return a marker to indicate whether CR1 needs to set or clear the bit that V.4 uses to say fp args were passed in registers. Assume that we don't need the marker for software floating point, diff --git a/gcc/config/rs6000/rs6000-modes.def b/gcc/config/rs6000/rs6000-modes.def index 094b246c834..60ebb363196 100644 --- a/gcc/config/rs6000/rs6000-modes.def +++ b/gcc/config/rs6000/rs6000-modes.def @@ -86,3 +86,7 @@ PARTIAL_INT_MODE (TI, 128, PTI); /* Modes used by __vector_pair and __vector_quad. */ OPAQUE_MODE (OO, 32); OPAQUE_MODE (XO, 64); + +/* Modes used by __dmr. */ +OPAQUE_MODE (TDO, 128); + diff --git a/gcc/config/rs6000/rs6000.cc b/gcc/config/rs6000/rs6000.cc index 59517c8608d..aed4b72c4ea 100644 --- a/gcc/config/rs6000/rs6000.cc +++ b/gcc/config/rs6000/rs6000.cc @@ -1846,7 +1846,9 @@ rs6000_hard_regno_nregs_internal (int regno, machine_mode mode) 128-bit floating point that can go in vector registers, which has VSX memory addressing. */ if (FP_REGNO_P (regno)) - reg_size = (VECTOR_MEM_VSX_P (mode) || VECTOR_ALIGNMENT_P (mode) + reg_size = (VECTOR_MEM_VSX_P (mode) + || VECTOR_ALIGNMENT_P (mode) + || mode == TDOmode ? UNITS_PER_VSX_WORD : UNITS_PER_FP_WORD); @@ -1880,9 +1882,9 @@ rs6000_hard_regno_mode_ok_uncached (int regno, machine_mode mode) /* On ISA 3.1 (power10), MMA accumulator modes need FPR registers divisible by 4. - If dense math is enabled, allow all VSX registers plus the DMR registers. - We need to make sure we don't cross between the boundary of FPRs and - traditional Altiviec registers. */ + If dense math is enabled, allow all VSX registers plus the dense math + registers. We need to make sure we don't cross between the boundary of + FPRs and traditional Altiviec registers. */ if (mode == XOmode) { if (TARGET_MMA && !TARGET_DENSE_MATH) @@ -1904,7 +1906,27 @@ rs6000_hard_regno_mode_ok_uncached (int regno, machine_mode mode) return 0; } - /* No other types other than XOmode can go in DMRs. */ + /* Dense math register modes need DMR registers or VSX registers divisible by + 2. We need to make sure we don't cross between the boundary of FPRs and + traditional Altiviec registers. */ + if (mode == TDOmode) + { + if (!TARGET_DENSE_MATH) + return 0; + + if (DMR_REGNO_P (regno)) + return 1; + + if (FP_REGNO_P (regno)) + return ((regno & 1) == 0 && regno <= LAST_FPR_REGNO - 7); + + if (ALTIVEC_REGNO_P (regno)) + return ((regno & 1) == 0 && regno <= LAST_ALTIVEC_REGNO - 7); + + return 0; + } + + /* No other types other than XOmode or TDOmode can go in DMRs. */ if (DMR_REGNO_P (regno)) return 0; @@ -2012,9 +2034,11 @@ rs6000_hard_regno_mode_ok (unsigned int regno, machine_mode mode) GPR registers, and TImode can go in any GPR as well as VSX registers (PR 57744). - Similarly, don't allow OOmode (vector pair, restricted to even VSX - registers) or XOmode (vector quad, restricted to FPR registers divisible - by 4) to tie with other modes. + Similarly, don't allow OOmode (vector pair), XOmode (vector quad), or + TDOmode (dmr register) to pair with anything else. Vector pairs are + restricted to even/odd VSX registers. Without dense math, vector quads are + limited to FPR registers divisible by 4. With dense math, vector quads are + limited to even VSX registers or DMR registers. Altivec/VSX vector tests were moved ahead of scalar float mode, so that IEEE 128-bit floating point on VSX systems ties with other vectors. */ @@ -2023,7 +2047,8 @@ static bool rs6000_modes_tieable_p (machine_mode mode1, machine_mode mode2) { if (mode1 == PTImode || mode1 == OOmode || mode1 == XOmode - || mode2 == PTImode || mode2 == OOmode || mode2 == XOmode) + || mode1 == TDOmode || mode2 == PTImode || mode2 == OOmode + || mode2 == XOmode || mode2 == TDOmode) return mode1 == mode2; if (ALTIVEC_OR_VSX_VECTOR_MODE (mode1)) @@ -2314,6 +2339,7 @@ rs6000_debug_reg_global (void) V4DFmode, OOmode, XOmode, + TDOmode, CCmode, CCUNSmode, CCEQmode, @@ -2679,7 +2705,7 @@ rs6000_setup_reg_addr_masks (void) /* Special case DMR registers. */ if (rc == RELOAD_REG_DMR) { - if (TARGET_DENSE_MATH && m2 == XOmode) + if (TARGET_DENSE_MATH && (m2 == XOmode || m2 == TDOmode)) { addr_mask = RELOAD_REG_VALID; reg_addr[m].addr_mask[rc] = addr_mask; @@ -2786,12 +2812,14 @@ rs6000_setup_reg_addr_masks (void) /* Vector pairs can do both indexed and offset loads if the instructions are enabled, otherwise they can only do offset loads - since it will be broken into two vector moves. Vector quads can - only do offset loads. If the user restricted generation of either - of the LXVP or STXVP instructions, do not allow indexed mode so - that we can split the load/store. */ + since it will be broken into two vector moves. If the user + restricted generation of either of the LXVP or STXVP instructions, + do not allow indexed mode so that we can split the load/store. + + Vector quads and dense math 1,024 bit registers can only do offset + loads. */ else if ((addr_mask != 0) && TARGET_MMA - && (m2 == OOmode || m2 == XOmode)) + && (m2 == OOmode || m2 == XOmode || m2 == TDOmode)) { addr_mask |= RELOAD_REG_OFFSET; if (rc == RELOAD_REG_FPR || rc == RELOAD_REG_VMX) @@ -3021,6 +3049,14 @@ rs6000_init_hard_regno_mode_ok (bool global_init_p) rs6000_vector_align[XOmode] = 512; } + /* Add support for 1,024 bit DMR registers. */ + if (TARGET_DENSE_MATH) + { + rs6000_vector_unit[TDOmode] = VECTOR_NONE; + rs6000_vector_mem[TDOmode] = VECTOR_VSX; + rs6000_vector_align[TDOmode] = 512; + } + /* Register class constraints for the constraints that depend on compile switches. When the VSX code was added, different constraints were added based on the type (DFmode, V2DFmode, V4SFmode). For the vector types, all @@ -3234,6 +3270,12 @@ rs6000_init_hard_regno_mode_ok (bool global_init_p) } } + if (TARGET_DENSE_MATH) + { + reg_addr[TDOmode].reload_load = CODE_FOR_reload_dmr_from_memory; + reg_addr[TDOmode].reload_store = CODE_FOR_reload_dmr_to_memory; + } + /* Precalculate HARD_REGNO_NREGS. */ for (r = 0; HARD_REGISTER_NUM_P (r); ++r) for (m = 0; m < NUM_MACHINE_MODES; ++m) @@ -8800,12 +8842,15 @@ reg_offset_addressing_ok_p (machine_mode mode) return mode_supports_dq_form (mode); break; - /* The vector pair/quad types support offset addressing if the - underlying vectors support offset addressing. */ + /* The vector pair/quad types and the dense math types support offset + addressing if the underlying vectors support offset addressing. */ case E_OOmode: case E_XOmode: return TARGET_MMA; + case E_TDOmode: + return TARGET_DENSE_MATH; + case E_SDmode: /* If we can do direct load/stores of SDmode, restrict it to reg+reg addressing for the LFIWZX and STFIWX instructions. */ @@ -11354,6 +11399,12 @@ rs6000_emit_move (rtx dest, rtx source, machine_mode mode) (mode == OOmode) ? "__vector_pair" : "__vector_quad"); break; + case E_TDOmode: + if (CONST_INT_P (operands[1])) + error ("%qs is an opaque type, and you cannot set it to constants", + "__dmr"); + break; + case E_SImode: case E_DImode: /* Use default pattern for address of ELF small data */ @@ -12817,7 +12868,7 @@ rs6000_secondary_reload_simple_move (enum rs6000_reg_type to_type, /* We can transfer between VSX registers and DMR registers without needing extra registers. */ - if (TARGET_DENSE_MATH && mode == XOmode + if (TARGET_DENSE_MATH && (mode == XOmode || mode == TDOmode) && ((to_type == DMR_REG_TYPE && from_type == VSX_REG_TYPE) || (to_type == VSX_REG_TYPE && from_type == DMR_REG_TYPE))) return true; @@ -13618,6 +13669,9 @@ rs6000_preferred_reload_class (rtx x, enum reg_class rclass) if (mode == XOmode) return TARGET_DENSE_MATH ? VSX_REGS : FLOAT_REGS; + if (mode == TDOmode) + return VSX_REGS; + if (GET_MODE_CLASS (mode) == MODE_INT) return GENERAL_REGS; } @@ -13741,8 +13795,9 @@ rs6000_secondary_reload_class (enum reg_class rclass, machine_mode mode, else regno = -1; - /* DMR registers don't have loads or stores. We have to go through the VSX - registers to load XOmode (vector quad). */ + /* Dense math registers don't have loads or stores. We have to go through + the VSX registers to load XOmode (vector quad) and TDOmode (dmr 1024 + bit). */ if (TARGET_DENSE_MATH && rclass == DM_REGS) return VSX_REGS; @@ -20830,6 +20885,8 @@ rs6000_mangle_type (const_tree type) return "u13__vector_pair"; if (type == vector_quad_type_node) return "u13__vector_quad"; + if (type == dmr_type_node) + return "u5__dmr"; /* For all other types, use the default mangling. */ return NULL; @@ -22954,6 +23011,10 @@ rs6000_dmr_register_move_cost (machine_mode mode, reg_class_t rclass) if (mode == XOmode) return reg_move_base; + /* __dmr (i.e. TDOmode) is transferred in 2 instructions. */ + else if (mode == TDOmode) + return reg_move_base * 2; + else return reg_move_base * 2 * hard_regno_nregs (FIRST_DMR_REGNO, mode); } @@ -27651,9 +27712,10 @@ rs6000_split_multireg_move (rtx dst, rtx src) mode = GET_MODE (dst); nregs = hard_regno_nregs (reg, mode); - /* If we have a vector quad register for MMA, and this is a load or store, - see if we can use vector paired load/stores. */ - if (mode == XOmode && TARGET_MMA + /* If we have a vector quad register for MMA or DMR register for dense math, + and this is a load or store, see if we can use vector paired + load/stores. */ + if ((mode == XOmode || mode == TDOmode) && TARGET_MMA && (MEM_P (dst) || MEM_P (src))) { reg_mode = OOmode; @@ -27661,7 +27723,7 @@ rs6000_split_multireg_move (rtx dst, rtx src) } /* If we have a vector pair/quad mode, split it into two/four separate vectors. */ - else if (mode == OOmode || mode == XOmode) + else if (mode == OOmode || mode == XOmode || mode == TDOmode) reg_mode = V1TImode; else if (FP_REGNO_P (reg)) reg_mode = DECIMAL_FLOAT_MODE_P (mode) ? DDmode : @@ -27707,13 +27769,13 @@ rs6000_split_multireg_move (rtx dst, rtx src) return; } - /* The __vector_pair and __vector_quad modes are multi-register - modes, so if we have to load or store the registers, we have to be - careful to properly swap them if we're in little endian mode - below. This means the last register gets the first memory - location. We also need to be careful of using the right register - numbers if we are splitting XO to OO. */ - if (mode == OOmode || mode == XOmode) + /* The __vector_pair, __vector_quad, and __dmr modes are multi-register + modes, so if we have to load or store the registers, we have to be careful + to properly swap them if we're in little endian mode below. This means + the last register gets the first memory location. We also need to be + careful of using the right register numbers if we are splitting XO to + OO. */ + if (mode == OOmode || mode == XOmode || mode == TDOmode) { nregs = hard_regno_nregs (reg, mode); int reg_mode_nregs = hard_regno_nregs (reg, reg_mode); @@ -27850,7 +27912,7 @@ rs6000_split_multireg_move (rtx dst, rtx src) overlap. */ int i; /* XO/OO are opaque so cannot use subregs. */ - if (mode == OOmode || mode == XOmode ) + if (mode == OOmode || mode == XOmode || mode == TDOmode) { for (i = nregs - 1; i >= 0; i--) { @@ -28024,7 +28086,7 @@ rs6000_split_multireg_move (rtx dst, rtx src) continue; /* XO/OO are opaque so cannot use subregs. */ - if (mode == OOmode || mode == XOmode ) + if (mode == OOmode || mode == XOmode || mode == TDOmode) { rtx dst_i = gen_rtx_REG (reg_mode, REGNO (dst) + j); rtx src_i = gen_rtx_REG (reg_mode, REGNO (src) + j); @@ -29006,7 +29068,8 @@ rs6000_invalid_conversion (const_tree fromtype, const_tree totype) if (frommode != tomode) { - /* Do not allow conversions to/from XOmode and OOmode types. */ + /* Do not allow conversions to/from XOmode, OOmode, and TDOmode + types. */ if (frommode == XOmode) return N_("invalid conversion from type %<__vector_quad%>"); if (tomode == XOmode) @@ -29015,6 +29078,10 @@ rs6000_invalid_conversion (const_tree fromtype, const_tree totype) return N_("invalid conversion from type %<__vector_pair%>"); if (tomode == OOmode) return N_("invalid conversion to type %<__vector_pair%>"); + if (frommode == TDOmode) + return N_("invalid conversion from type %<__dmr%>"); + if (tomode == TDOmode) + return N_("invalid conversion to type %<__dmr%>"); } /* Conversion allowed. */ diff --git a/gcc/config/rs6000/rs6000.h b/gcc/config/rs6000/rs6000.h index 22efac4a80c..9711777b5cd 100644 --- a/gcc/config/rs6000/rs6000.h +++ b/gcc/config/rs6000/rs6000.h @@ -1004,7 +1004,8 @@ enum data_align { align_abi, align_opt, align_both }; /* Modes that are not vectors, but require vector alignment. Treat these like vectors in terms of loads and stores. */ #define VECTOR_ALIGNMENT_P(MODE) \ - (FLOAT128_VECTOR_P (MODE) || (MODE) == OOmode || (MODE) == XOmode) + (FLOAT128_VECTOR_P (MODE) || (MODE) == OOmode || (MODE) == XOmode \ + || (MODE) == TDOmode) #define ALTIVEC_VECTOR_MODE(MODE) \ ((MODE) == V16QImode \ @@ -2293,6 +2294,7 @@ enum rs6000_builtin_type_index RS6000_BTI_const_str, /* pointer to const char * */ RS6000_BTI_vector_pair, /* unsigned 256-bit types (vector pair). */ RS6000_BTI_vector_quad, /* unsigned 512-bit types (vector quad). */ + RS6000_BTI_dmr, /* unsigned 1,024-bit types (dmr). */ RS6000_BTI_const_ptr_void, /* const pointer to void */ RS6000_BTI_ptr_V16QI, RS6000_BTI_ptr_V1TI, @@ -2331,6 +2333,7 @@ enum rs6000_builtin_type_index RS6000_BTI_ptr_dfloat128, RS6000_BTI_ptr_vector_pair, RS6000_BTI_ptr_vector_quad, + RS6000_BTI_ptr_dmr, RS6000_BTI_ptr_long_long, RS6000_BTI_ptr_long_long_unsigned, RS6000_BTI_MAX @@ -2388,6 +2391,7 @@ enum rs6000_builtin_type_index #define const_str_type_node (rs6000_builtin_types[RS6000_BTI_const_str]) #define vector_pair_type_node (rs6000_builtin_types[RS6000_BTI_vector_pair]) #define vector_quad_type_node (rs6000_builtin_types[RS6000_BTI_vector_quad]) +#define dmr_type_node (rs6000_builtin_types[RS6000_BTI_dmr]) #define pcvoid_type_node (rs6000_builtin_types[RS6000_BTI_const_ptr_void]) #define ptr_V16QI_type_node (rs6000_builtin_types[RS6000_BTI_ptr_V16QI]) #define ptr_V1TI_type_node (rs6000_builtin_types[RS6000_BTI_ptr_V1TI]) @@ -2426,6 +2430,7 @@ enum rs6000_builtin_type_index #define ptr_dfloat128_type_node (rs6000_builtin_types[RS6000_BTI_ptr_dfloat128]) #define ptr_vector_pair_type_node (rs6000_builtin_types[RS6000_BTI_ptr_vector_pair]) #define ptr_vector_quad_type_node (rs6000_builtin_types[RS6000_BTI_ptr_vector_quad]) +#define ptr_dmr_type_node (rs6000_builtin_types[RS6000_BTI_ptr_dmr]) #define ptr_long_long_integer_type_node (rs6000_builtin_types[RS6000_BTI_ptr_long_long]) #define ptr_long_long_unsigned_type_node (rs6000_builtin_types[RS6000_BTI_ptr_long_long_unsigned]) diff --git a/gcc/testsuite/gcc.target/powerpc/dm-1024bit.c b/gcc/testsuite/gcc.target/powerpc/dm-1024bit.c new file mode 100644 index 00000000000..0a9884ddf63 --- /dev/null +++ b/gcc/testsuite/gcc.target/powerpc/dm-1024bit.c @@ -0,0 +1,63 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target powerpc_dense_math_ok } */ +/* { dg-options "-mdejagnu-cpu=future -O2" } */ + +/* Test basic load/store for __dmr type. */ + +#ifndef CONSTRAINT +#if defined(USE_D) +#define CONSTRAINT "d" + +#elif defined(USE_V) +#define CONSTRAINT "v" + +#elif defined(USE_WA) +#define CONSTRAINT "wa" + +#else +#define CONSTRAINT "wD" +#endif +#endif +const char constraint[] = CONSTRAINT; + +void foo_mem_asm (__dmr *p, __dmr *q) +{ + /* 2 LXVP instructions. */ + __dmr vq = *p; + + /* 2 DMXXINSTDMR512 instructions to transfer VSX to DMR. */ + __asm__ ("# foo (" CONSTRAINT ") %A0" : "+" CONSTRAINT (vq)); + /* 2 DMXXEXTFDMR512 instructions to transfer DMR to VSX. */ + + /* 2 STXVP instructions. */ + *q = vq; +} + +void foo_mem_asm2 (__dmr *p, __dmr *q) +{ + /* 2 LXVP instructions. */ + __dmr vq = *p; + __dmr vq2; + __dmr vq3; + + /* 2 DMXXINSTDMR512 instructions to transfer VSX to DMR. */ + __asm__ ("# foo1 (" CONSTRAINT ") %A0" : "+" CONSTRAINT (vq)); + /* 2 DMXXEXTFDMR512 instructions to transfer DMR to VSX. */ + + vq2 = vq; + __asm__ ("# foo2 (wa) %0" : "+wa" (vq2)); + + /* 2 STXVP instructions. */ + *q = vq2; +} + +void foo_mem (__dmr *p, __dmr *q) +{ + /* 2 LXVP, 2 STXVP instructions, no DMR transfer. */ + *q = *p; +} + +/* { dg-final { scan-assembler-times {\mdmxxextfdmr512\M} 4 } } */ +/* { dg-final { scan-assembler-times {\mdmxxinstdmr512\M} 4 } } */ +/* { dg-final { scan-assembler-times {\mlxvp\M} 12 } } */ +/* { dg-final { scan-assembler-times {\mstxvp\M} 12 } } */