From patchwork Wed Jun 28 13:48:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 113911 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8943689vqr; Wed, 28 Jun 2023 06:55:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ51NBYUeSWm+v8SzhangagW3DYYD+kqRw5DkBwV4kGrarcxvdnUcb/h4x118TvKMwKIDNyd X-Received: by 2002:aa7:de06:0:b0:51d:d41b:26d4 with SMTP id h6-20020aa7de06000000b0051dd41b26d4mr487381edv.39.1687960550174; Wed, 28 Jun 2023 06:55:50 -0700 (PDT) Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id n14-20020aa7d04e000000b005164220ba75si5254023edo.41.2023.06.28.06.55.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jun 2023 06:55:50 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=gWsiAbvM; arc=fail (signature failed); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 66C3C38319CD for ; Wed, 28 Jun 2023 13:51:50 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 66C3C38319CD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1687960310; bh=rA1tT9dzOqEFbBF1yolFgDs6NQL5g44v6fbA2ZuswCU=; h=Date:To:Cc:Subject:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=gWsiAbvMY+voII1ouqAf66f85PVmhowTK/GVEm/jBgSd6zyrFx8+iPkqZ5uvljKFp AB8PnOYf3BFvm0ldYGjtSjo4xPZJ+CXsrd7Y2Z054iHcGpkOblO3M2fGcXtwOUxALV uEDGQcA1IkcSTlDH415SVlj/G3nbXIZJNsCeZuJU= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on2073.outbound.protection.outlook.com [40.107.7.73]) by sourceware.org (Postfix) with ESMTPS id 6CD89385660A for ; Wed, 28 Jun 2023 13:48:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6CD89385660A Received: from AS9PR06CA0360.eurprd06.prod.outlook.com (2603:10a6:20b:466::16) by PAWPR08MB9124.eurprd08.prod.outlook.com (2603:10a6:102:330::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.26; Wed, 28 Jun 2023 13:48:20 +0000 Received: from AM7EUR03FT010.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:466:cafe::1e) by AS9PR06CA0360.outlook.office365.com (2603:10a6:20b:466::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.34 via Frontend Transport; Wed, 28 Jun 2023 13:48:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT010.mail.protection.outlook.com (100.127.141.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.18 via Frontend Transport; Wed, 28 Jun 2023 13:48:20 +0000 Received: ("Tessian outbound c08fa2e31830:v142"); Wed, 28 Jun 2023 13:48:20 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 068e4d210c05b97c X-CR-MTA-TID: 64aa7808 Received: from aa16e7ce21e0.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 585A6D15-EE0F-4B03-A6E3-E890F2D8294A.1; Wed, 28 Jun 2023 13:48:14 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id aa16e7ce21e0.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 28 Jun 2023 13:48:14 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JflZhMrLw2wYG3oZtUkVxze1y0LS92Usq7WsBR7svWOWiDtU8rPRd67FvVmn2bt1g7yPFllOOcLOIMxTY7JNZbu/h1wjriejL+XubxWFTSmM6z5y03kkW0z/p8f8c5iYLYzV0N7ILyrszR1QaHzQcjE6sVT7NWyCDgmHyTr4S6gj0iZcsej+5FGhKBXoG02MpZGWIuxe0zn93h43QEcK80lOJykLnqsXRdBVW/cdCEqa9Izz9mjHzZ7D3paW9nGgPX6uP+ZJgoWoUdOlIAyWJClcqyxZ8DLYtlxjwvDVwiK7uiHAIPJc7trSt1VsNX0l54DksccdVkad4E1ZVgo+zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rA1tT9dzOqEFbBF1yolFgDs6NQL5g44v6fbA2ZuswCU=; b=e6xUl1I4iYamUvUtXB6UUOsRx20lb5QlA8z+KYuGWf1z+a26zN/SYrbC+6EkdfOfy8/dIE5K2g81ht14/0K3KHl7ChK5oHp7siV8ELH2C+IwFWbpNBGQpU0m2YJGfA67xBdhkToJIzCv2NUfR9BDQOewpuVX69yimgcTdDGw4pd1SvdEtLUwyVq1f8y03Z09bADj//otqvXbJQLOleWgyLVSEuT3N/SNzeMfPk+x6rXQ+ov2Sko8+m0uA7nvpqM8JASGr2Gh1E2nsFFGvWc3hSOnsZkFDxfAauGQyhc+tlz7X5cgXE3+E+wWhiebyy8+0pD8H6b/RCa/H1Xe8fWEdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by AM7PR08MB5398.eurprd08.prod.outlook.com (2603:10a6:20b:103::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.26; Wed, 28 Jun 2023 13:48:12 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0%6]) with mapi id 15.20.6521.026; Wed, 28 Jun 2023 13:48:12 +0000 Date: Wed, 28 Jun 2023 14:48:10 +0100 To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, Richard.Earnshaw@arm.com, Marcus.Shawcroft@arm.com, Kyrylo.Tkachov@arm.com, richard.sandiford@arm.com Subject: [PATCH 15/19]AArch64: Add implementation for vector cbranch for Advanced SIMD Message-ID: Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO4P123CA0351.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18d::14) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|AM7PR08MB5398:EE_|AM7EUR03FT010:EE_|PAWPR08MB9124:EE_ X-MS-Office365-Filtering-Correlation-Id: c2b2a0df-e764-4ae6-7a79-08db77de55ac x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: ewpUcndxfaKB0wLlckpIGXiSnk7RuOG6uLZA09MglTrZKVz86/HNpzOVJeK2J/VXHcqoM+/fy98s/Bz5upJz+mRGIlaKm9h4sqau6WHEVFuu/sePzC+1zgcaKKGlH5EhM6mJHCT2A1jdze4Cv4yXeN+ht/Mbb6Ja6hhop+HuI46VsVosTzmPHiUqhctsuQiRGfXK9a/QVWMrhaenN1EiotE2olBpHJNON/Ercm84fXoJWZfPYGQ09RTfJ4MEWv0Ep/K8UJQ3tTuhRHjPVBZ9V2Z79oP549p4rIIk0kOWWlNseSVCFFsGahsfaUDy+v9fB5zEyok2OYcaJ7U35xIiR2hvIvfuy4+QK1u0a1srU4c8yG90+DYQhVvF5MX6x9q0N1tLE5xkdT6gIfGKA/HjwkwInRlg2iADbMWdEN/Zc9IqucTtTlE6fnYW//s/T58bxTCvvj6ln5hQFN/TYIQfAr2znGf6AmhPmh/G6dMiUIFG2Wvy+MNr2X+xBuMvLyEf4xSR3sb0N1MSJ3pNzqNQoIjWXynt2qEXjvcLu0KzGDaUqX8Ol0DZKyiHwAQGaFRWThtzYHrnIicN+jOMuzXxD4Si7arzd+XAeZU0GPf7pkE= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(136003)(376002)(346002)(39860400002)(366004)(396003)(451199021)(6486002)(4743002)(33964004)(26005)(2616005)(478600001)(186003)(2906002)(6506007)(44144004)(5660300002)(44832011)(36756003)(316002)(235185007)(8676002)(66946007)(41300700001)(38100700002)(4326008)(86362001)(6916009)(66476007)(66556008)(8936002)(6512007)(84970400001)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5398 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT010.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: dd828ea5-e9a6-486c-5efa-08db77de50a6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F5S9StlQiViI9rZtAXzEl/IJvIP/O2JKxOM0Fs4dc4qPQvubXKDFxlQmL3ukKkkXED0hRjuaSmKuailYZU6IYGDk7lrXtkDdjUD4t37nzMDS5s+ms+z0j4bPVoRI30WWGcaKSRGTw8L9WWUw0Wa+1N/TwjilJ2totGzVDbOXKAKKxMW5NttU2D0ZyVbfZgV+3mm1vzVYJ8EvuOJU5MghH1Prw6t00tkrv+WxTb/pE5FtcN1Ve7yVuoAFaABYo0txvsVU2CqOCZHx5zPXkJf/7oIaVftLTCrWZoixTRexZPVP2+FDhN3Bd1RjAKEA62Zjrooz7hbSptq5RjKnhwgcB+BU2xlg+q4es+h4XUWgbXHdFonUUKGeArLUI5/4eM1X/dODZQ3qGoaWK676nBULGJ/0QOwfTrR3RVGqt/4CP4g5FzODJ7IkgD86z6FakiQiEmiUTzDndvPu32J/SQ+qWBsnMSyGkwhWhrm2bCX7nrH/QUHFXH9K+frtvdmU3kk4I9Mzscpc87Lfby6AQKmD3X1ozoxzGh1SDNxu1YxxAAlGdt+YbCCzUCpjIEomtCPlhiySkGNWhf2EqWqeOiZQacBNmE9GOWeDEkYee5MGnkZ2E1m+FoFzCJq90snUsj4/44P/CC5nKSwdOhKlGDRBcohz44LKA3/9p4S9KTCrUyelzt8esQGtWytfqAE64IRqvZuV/Gvj2Cb5q/05RyeZGB176eWsaSIHNAAYpHZ9woYk09B4hy/waZ/XGCPHLLwiFpxMH0NDP3duc4Rnh8ePVQ== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(396003)(136003)(376002)(346002)(451199021)(40470700004)(46966006)(36840700001)(84970400001)(36756003)(82310400005)(36860700001)(6506007)(70206006)(81166007)(82740400003)(40460700003)(5660300002)(44832011)(356005)(86362001)(316002)(6916009)(4326008)(41300700001)(8936002)(40480700001)(235185007)(70586007)(8676002)(6486002)(47076005)(6512007)(33964004)(2906002)(44144004)(186003)(26005)(336012)(4743002)(478600001)(2616005)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2023 13:48:20.6087 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c2b2a0df-e764-4ae6-7a79-08db77de55ac X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT010.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR08MB9124 X-Spam-Status: No, score=-12.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_LOTSOFHASH, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769954921760512341?= X-GMAIL-MSGID: =?utf-8?q?1769954921760512341?= Hi All, This adds an implementation for conditional branch optab for AArch64. For e.g. void f1 () { for (int i = 0; i < N; i++) { b[i] += a[i]; if (a[i] > 0) break; } } For 128-bit vectors we generate: cmgt v1.4s, v1.4s, #0 umaxp v1.4s, v1.4s, v1.4s fmov x3, d1 cbnz x3, .L8 and of 64-bit vector we can omit the compression: cmgt v1.2s, v1.2s, #0 fmov x2, d1 cbz x2, .L13 Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * config/aarch64/aarch64-simd.md (cbranch4): New. gcc/testsuite/ChangeLog: * gcc.target/aarch64/vect-early-break-cbranch.c: New test. --- inline copy of patch -- diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md index 90118c6348e9614bef580d1dc94c0c1841dd5204..cd5ec35c3f53028f14828bd70a92924f62524c15 100644 --- diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md index 90118c6348e9614bef580d1dc94c0c1841dd5204..cd5ec35c3f53028f14828bd70a92924f62524c15 100644 --- a/gcc/config/aarch64/aarch64-simd.md +++ b/gcc/config/aarch64/aarch64-simd.md @@ -3830,6 +3830,46 @@ (define_expand "vcond_mask_" DONE; }) +;; Patterns comparing two vectors and conditionally jump + +(define_expand "cbranch4" + [(set (pc) + (if_then_else + (match_operator 0 "aarch64_equality_operator" + [(match_operand:VDQ_I 1 "register_operand") + (match_operand:VDQ_I 2 "aarch64_simd_reg_or_zero")]) + (label_ref (match_operand 3 "")) + (pc)))] + "TARGET_SIMD" +{ + auto code = GET_CODE (operands[0]); + rtx tmp = operands[1]; + + /* If comparing against a non-zero vector we have to do a comparison first + so we can have a != 0 comparison with the result. */ + if (operands[2] != CONST0_RTX (mode)) + emit_insn (gen_vec_cmp (tmp, operands[0], operands[1], + operands[2])); + + /* For 64-bit vectors we need no reductions. */ + if (known_eq (128, GET_MODE_BITSIZE (mode))) + { + /* Always reduce using a V4SI. */ + rtx reduc = gen_lowpart (V4SImode, tmp); + rtx res = gen_reg_rtx (V4SImode); + emit_insn (gen_aarch64_umaxpv4si (res, reduc, reduc)); + emit_move_insn (tmp, gen_lowpart (mode, res)); + } + + rtx val = gen_reg_rtx (DImode); + emit_move_insn (val, gen_lowpart (DImode, tmp)); + + rtx cc_reg = aarch64_gen_compare_reg (code, val, const0_rtx); + rtx cmp_rtx = gen_rtx_fmt_ee (code, DImode, cc_reg, const0_rtx); + emit_jump_insn (gen_condjump (cmp_rtx, cc_reg, operands[3])); + DONE; +}) + ;; Patterns comparing two vectors to produce a mask. (define_expand "vec_cmp" diff --git a/gcc/testsuite/gcc.target/aarch64/vect-early-break-cbranch.c b/gcc/testsuite/gcc.target/aarch64/vect-early-break-cbranch.c new file mode 100644 index 0000000000000000000000000000000000000000..c0363c3787270507d7902bb2ac0e39faef63a852 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/vect-early-break-cbranch.c @@ -0,0 +1,124 @@ +/* { dg-do compile } */ +/* { dg-options "-O3" } */ +/* { dg-final { check-function-bodies "**" "" "" { target lp64 } } } */ + +#pragma GCC target "+nosve" + +#define N 640 +int a[N] = {0}; +int b[N] = {0}; + + +/* +** f1: +** ... +** cmgt v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f1 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] > 0) + break; + } +} + +/* +** f2: +** ... +** cmge v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f2 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] >= 0) + break; + } +} + +/* +** f3: +** ... +** cmeq v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f3 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] == 0) + break; + } +} + +/* +** f4: +** ... +** cmtst v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f4 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] != 0) + break; + } +} + +/* +** f5: +** ... +** cmlt v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f5 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] < 0) + break; + } +} + +/* +** f6: +** ... +** cmle v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f6 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] <= 0) + break; + } +} --- a/gcc/config/aarch64/aarch64-simd.md +++ b/gcc/config/aarch64/aarch64-simd.md @@ -3830,6 +3830,46 @@ (define_expand "vcond_mask_" DONE; }) +;; Patterns comparing two vectors and conditionally jump + +(define_expand "cbranch4" + [(set (pc) + (if_then_else + (match_operator 0 "aarch64_equality_operator" + [(match_operand:VDQ_I 1 "register_operand") + (match_operand:VDQ_I 2 "aarch64_simd_reg_or_zero")]) + (label_ref (match_operand 3 "")) + (pc)))] + "TARGET_SIMD" +{ + auto code = GET_CODE (operands[0]); + rtx tmp = operands[1]; + + /* If comparing against a non-zero vector we have to do a comparison first + so we can have a != 0 comparison with the result. */ + if (operands[2] != CONST0_RTX (mode)) + emit_insn (gen_vec_cmp (tmp, operands[0], operands[1], + operands[2])); + + /* For 64-bit vectors we need no reductions. */ + if (known_eq (128, GET_MODE_BITSIZE (mode))) + { + /* Always reduce using a V4SI. */ + rtx reduc = gen_lowpart (V4SImode, tmp); + rtx res = gen_reg_rtx (V4SImode); + emit_insn (gen_aarch64_umaxpv4si (res, reduc, reduc)); + emit_move_insn (tmp, gen_lowpart (mode, res)); + } + + rtx val = gen_reg_rtx (DImode); + emit_move_insn (val, gen_lowpart (DImode, tmp)); + + rtx cc_reg = aarch64_gen_compare_reg (code, val, const0_rtx); + rtx cmp_rtx = gen_rtx_fmt_ee (code, DImode, cc_reg, const0_rtx); + emit_jump_insn (gen_condjump (cmp_rtx, cc_reg, operands[3])); + DONE; +}) + ;; Patterns comparing two vectors to produce a mask. (define_expand "vec_cmp" diff --git a/gcc/testsuite/gcc.target/aarch64/vect-early-break-cbranch.c b/gcc/testsuite/gcc.target/aarch64/vect-early-break-cbranch.c new file mode 100644 index 0000000000000000000000000000000000000000..c0363c3787270507d7902bb2ac0e39faef63a852 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/vect-early-break-cbranch.c @@ -0,0 +1,124 @@ +/* { dg-do compile } */ +/* { dg-options "-O3" } */ +/* { dg-final { check-function-bodies "**" "" "" { target lp64 } } } */ + +#pragma GCC target "+nosve" + +#define N 640 +int a[N] = {0}; +int b[N] = {0}; + + +/* +** f1: +** ... +** cmgt v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f1 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] > 0) + break; + } +} + +/* +** f2: +** ... +** cmge v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f2 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] >= 0) + break; + } +} + +/* +** f3: +** ... +** cmeq v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f3 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] == 0) + break; + } +} + +/* +** f4: +** ... +** cmtst v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f4 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] != 0) + break; + } +} + +/* +** f5: +** ... +** cmlt v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f5 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] < 0) + break; + } +} + +/* +** f6: +** ... +** cmle v[0-9]+.4s, v[0-9]+.4s, #0 +** umaxp v[0-9]+.4s, v[0-9]+.4s, v[0-9]+.4s +** fmov x[0-9]+, d[0-9]+ +** cbnz x[0-9]+, \.L[0-9]+ +** ... +*/ +void f6 () +{ + for (int i = 0; i < N; i++) + { + b[i] += a[i]; + if (a[i] <= 0) + break; + } +}