From patchwork Wed Jun 28 13:45:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 113898 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8939670vqr; Wed, 28 Jun 2023 06:48:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6Ty21egIGteYbQZL4N05Mj35PNDsphmIMYGDnimFECbiJhlb0hynUAHFss0zeZXDjEpS3c X-Received: by 2002:a2e:9203:0:b0:2b6:a682:9aa5 with SMTP id k3-20020a2e9203000000b002b6a6829aa5mr5543777ljg.38.1687960130539; Wed, 28 Jun 2023 06:48:50 -0700 (PDT) Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id y11-20020aa7c24b000000b0051d8027e02esi5035902edo.461.2023.06.28.06.48.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jun 2023 06:48:50 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b="TpuB/rU+"; arc=fail (signature failed); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 25D93385482E for ; Wed, 28 Jun 2023 13:47:02 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 25D93385482E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1687960022; bh=vnyl2ykqiTPU/rE/94Uw6LzwwEbBWqcwZD6fJoKmkqc=; h=Date:To:Cc:Subject:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=TpuB/rU+q1CIwMYk46I7TqGGZO0VKVej/q9LiJpx5QDR2wcgxSO+0hYJpkYjmANvo IJNhqYPSdknfuLYSMEol7orCnU6SXLgaYd7eTjyfkDtoJR88HIwjg4uf7M6OkNdGeC CToqO/CwfiIq1NX+GCAtwlsJWYwiWdzQqLw56iWA= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2061.outbound.protection.outlook.com [40.107.20.61]) by sourceware.org (Postfix) with ESMTPS id F14733856DDE for ; Wed, 28 Jun 2023 13:46:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F14733856DDE Received: from DBBPR09CA0037.eurprd09.prod.outlook.com (2603:10a6:10:d4::25) by VE1PR08MB5647.eurprd08.prod.outlook.com (2603:10a6:800:1b2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.26; Wed, 28 Jun 2023 13:46:04 +0000 Received: from DBAEUR03FT057.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:d4:cafe::ea) by DBBPR09CA0037.outlook.office365.com (2603:10a6:10:d4::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.19 via Frontend Transport; Wed, 28 Jun 2023 13:46:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT057.mail.protection.outlook.com (100.127.142.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.20 via Frontend Transport; Wed, 28 Jun 2023 13:46:04 +0000 Received: ("Tessian outbound 546d04a74417:v142"); Wed, 28 Jun 2023 13:46:04 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 052834bfdca2bf1f X-CR-MTA-TID: 64aa7808 Received: from ebb8f32f352e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E385BCCF-273F-4C15-8116-D86C9497F9E5.1; Wed, 28 Jun 2023 13:45:58 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ebb8f32f352e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 28 Jun 2023 13:45:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RfCDBbhZ87wDtTkXkXK/mM/WqUkehbWGBxxn2g/hOSBJa8vWU1LdX0qSK/wAiTAO33NGAYTEThPjKJT/wj9rIQL2Hn5myy9f3fFWjc0VTpS7k8NKtp1R23sqXaPjBQAWKKj9v0YhXan5gYkibv8SJg522u6AEPb1kByRWuD8giGIYfuNq2o5PYUJQrHAfdEI8NRgnT1eKHAvYvgyTbTp5+2ZsydZ0pFekrwxvAao7qyBXSdsc8TCqCBbp8dK2+IkvKlJ7zMrn2hXJobskzvWQ3xBPxVP6lQKcdwiBVNasAqRwNDShTjC8vteeqckOX242YP0K088ngZ8xrpDAQYX9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vnyl2ykqiTPU/rE/94Uw6LzwwEbBWqcwZD6fJoKmkqc=; b=XAC9WIh5kiltj2xmaRn9jeCZRvkg87Ed4jPxp+tjjxiBPIaEok4bRFllNVeMvKWLqqAC4tBcVl8PzGmBdHp3Um4nkaDfrNL+YZPMpjWszJub27my1AGfmT9AQblUGPCIvJ9b477pUx47EmVW5xW7TDTFnXB6UtJhCBOvxnZAde2RApPxcKOipBc7oBhATRM5bQ8UuKN0lX5BMhYEQ+jsycYLVV59nG92sHE4Jo2OElDKC9T1BdugqNr5L3a7XCNE6IzS1OxLdvkxjgrfLqYyk37YsPnn7qKubz8GDRAO96xiwmtqBRLw6KnMpgEzs4Ecokh5Ki1l6Y6uaKbqnbLgLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DB5PR08MB9970.eurprd08.prod.outlook.com (2603:10a6:10:489::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.23; Wed, 28 Jun 2023 13:45:56 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0%6]) with mapi id 15.20.6521.026; Wed, 28 Jun 2023 13:45:56 +0000 Date: Wed, 28 Jun 2023 14:45:51 +0100 To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, Richard.Earnshaw@arm.com, Marcus.Shawcroft@arm.com, Kyrylo.Tkachov@arm.com, richard.sandiford@arm.com Subject: [PATCH 9/19]AArch64 middle-end: refactor vectorizable_comparison to make the main body re-usable. Message-ID: Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: VI1PR0902CA0060.eurprd09.prod.outlook.com (2603:10a6:802:1::49) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|DB5PR08MB9970:EE_|DBAEUR03FT057:EE_|VE1PR08MB5647:EE_ X-MS-Office365-Filtering-Correlation-Id: 9c055c20-5782-46ad-f3ba-08db77de0465 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: r+LpjBy6bSqlUZ73X+Al/ikVYRF4fFlCxWqjkaHx5au7FMDxabXx6zQoXmPkHdiSGh7pctoS/OKxx7KjYBqSM0ZEQWhYJ+3+v0Xa5Z4DKTDHrx7Rhwonm1gDJY6gQNx7YxC0mOioTQa7RtcfB5unVlH5tAbk8lCj3OmeiUZ7mKnHvOMIUHegAXsCzcVg8luzqDfiCvmVoylTIaJH4y/rxPCpwj2r4yFyGignmIhrvhXO9y8SAf/Bvy81R3aX5MC2LxrCRtHDp5bVMfz0Ov9aWDFmO+VoKEdLpawI/IZrAcdT+jAINXlJxsEieRS32PXkZ8WdqUjm+Jc5rBp3Ij2s3l6ormaV9eO10GwSrx6vodSyfp6ZDtT9XAk+NAF1ZZTERnMVUCdtlzFp/nWd/h6nOCDMDLSuZMa4TGzLoBzQE8mfaeuG3R5E+Gxi6ISMKnicXxho9dPH2TSqs4r6iaMDCScQqqzSI66I25oOin0UrQpWJIDB3NSQRsLXt5uV+A5J9t2yy0rcmloVyNdqdWJgHfd5exlHy3B7xUZz/aVTBv8In7Sre3DwRY5k44aKtPPW9vG77qZPnpecxdbD3I27sbffs0INeoWpDu/9G/xswALY9d1gh4ZPtJZkEFSUtFQo X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(136003)(346002)(396003)(376002)(366004)(39860400002)(451199021)(66946007)(66476007)(6916009)(44832011)(316002)(66556008)(478600001)(4326008)(36756003)(8936002)(235185007)(8676002)(5660300002)(86362001)(2906002)(33964004)(41300700001)(44144004)(6486002)(186003)(4743002)(6506007)(6512007)(6666004)(38100700002)(26005)(2616005)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR08MB9970 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT057.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 09e8b3ef-72ff-4774-3f67-08db77ddff8e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lTx3nzH8anSeT4/F0ANP98x3moB1NpmS2usqA9zYiCga56zcAtQlgnuKgRR0fcDukWBIgpSF5llv0j5rdo8LR6+JpCmFTRSJ14p2sW1ZANDKbFvPnf4b+K/LH7Yya+nRr3bCpqkpLieKkyqrQhHBEKDUWDx3C9qAvseupDHqBuQxkxQ/RUzruOfQqiLk0XTnQ8DuLbDjO+J4lKfSX+OMsxjIms8jxrkvIF5elrdUFcB8nfMu/U1GfZYMp/q6bijJrLpwTZemKHfJytlFzhXYcaSfEGCwq0AzgsN3izaO4I+jTyhherZTn257MzIDnleI6gfetO16Zx+MJEEKBaDvjazOqnsI3DtQHbGVzLjdQY87SwENUjjVm2w9UOqIV02lNtCcU8fXwxzFfVrcuvnKwI7g28U+JXNiW96R6MOi4CB8gbgTACIdiSwd5u7/NHlFGO+tXbCAnCs9WpLEd25lq7BRt2CBBt2Bid/c+esWUKZjE5umjXlpSJqlp7TRKM2PoHNQW1JoM2cBCasanis/Et4ackzHOhgXmUwdLsgxss4asvCEQVkROGeOQrBpaFcvlAtZiaqMKbTDC+q3onSWl1J7IR0h6v8ATgEqCDxlXZiPKUdFTBBSqDXCF4HCE7hfRuDjfhYaJ73kjbvVs7NY02kJ5knEvgvBBFFszuLtNcok94+SLfFRhFNmpbOj2r75WcDjo45fJHY90wKGimd0YhLke8x6XU5N4Js9hboQihBGpwXi86srZGpl788skYcgrRwh/cRehVMh3u4oNh5neg== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(346002)(39860400002)(396003)(376002)(451199021)(40470700004)(46966006)(36840700001)(235185007)(82310400005)(186003)(2906002)(26005)(5660300002)(41300700001)(33964004)(82740400003)(8676002)(6666004)(6916009)(36756003)(316002)(2616005)(478600001)(6486002)(336012)(86362001)(4326008)(4743002)(81166007)(44144004)(47076005)(70586007)(8936002)(44832011)(356005)(40460700003)(40480700001)(70206006)(6506007)(6512007)(36860700001)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2023 13:46:04.3136 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9c055c20-5782-46ad-f3ba-08db77de0465 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT057.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5647 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769954481606761993?= X-GMAIL-MSGID: =?utf-8?q?1769954481606761993?= Hi All, Vectorization of a gcond starts off essentially the same as vectorizing a comparison witht he only difference being how the operands are extracted. This refactors vectorable_comparison such that we now have a generic function that can be used from vectorizable_early_break. The refactoring splits the gassign checks and actual validation/codegen off to a helper function. No change in functionality expected. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-stmts.cc (vectorizable_comparison): Refactor, splitting body to ... (vectorizable_comparison_1): ...This. --- inline copy of patch -- diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index ae24f3e66e63d9bd9763284a47fb2c911335c4c1..f3e33cd4ed125b9564ca81acd197693fc3457c31 100644 --- diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index ae24f3e66e63d9bd9763284a47fb2c911335c4c1..f3e33cd4ed125b9564ca81acd197693fc3457c31 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -11332,21 +11332,22 @@ vectorizable_condition (vec_info *vinfo, /* vectorizable_comparison. - Check if STMT_INFO is comparison expression that can be vectorized. +/* Helper of vectorizable_comparison. + + Check if STMT_INFO is comparison expression CODE that can be vectorized. If VEC_STMT is also passed, vectorize STMT_INFO: create a vectorized comparison, put it in VEC_STMT, and insert it at GSI. Return true if STMT_INFO is vectorizable in this way. */ static bool -vectorizable_comparison (vec_info *vinfo, - stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, - gimple **vec_stmt, - slp_tree slp_node, stmt_vector_for_cost *cost_vec) +vectorizable_comparison_1 (vec_info *vinfo, tree vectype, + stmt_vec_info stmt_info, tree_code code, + gimple_stmt_iterator *gsi, gimple **vec_stmt, + slp_tree slp_node, stmt_vector_for_cost *cost_vec) { tree lhs, rhs1, rhs2; tree vectype1 = NULL_TREE, vectype2 = NULL_TREE; - tree vectype = STMT_VINFO_VECTYPE (stmt_info); tree vec_rhs1 = NULL_TREE, vec_rhs2 = NULL_TREE; tree new_temp; loop_vec_info loop_vinfo = dyn_cast (vinfo); @@ -11354,7 +11355,7 @@ vectorizable_comparison (vec_info *vinfo, int ndts = 2; poly_uint64 nunits; int ncopies; - enum tree_code code, bitop1 = NOP_EXPR, bitop2 = NOP_EXPR; + enum tree_code bitop1 = NOP_EXPR, bitop2 = NOP_EXPR; int i; bb_vec_info bb_vinfo = dyn_cast (vinfo); vec vec_oprnds0 = vNULL; @@ -11377,14 +11378,6 @@ vectorizable_comparison (vec_info *vinfo, ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); - if (STMT_VINFO_DEF_TYPE (stmt_info) != vect_internal_def) - return false; - - gassign *stmt = dyn_cast (stmt_info->stmt); - if (!stmt) - return false; - - code = gimple_assign_rhs_code (stmt); if (TREE_CODE_CLASS (code) != tcc_comparison) return false; @@ -11499,7 +11492,6 @@ vectorizable_comparison (vec_info *vinfo, return false; } - STMT_VINFO_TYPE (stmt_info) = comparison_vec_info_type; vect_model_simple_cost (vinfo, stmt_info, ncopies * (1 + (bitop2 != NOP_EXPR)), dts, ndts, slp_node, cost_vec); @@ -11565,6 +11557,44 @@ vectorizable_comparison (vec_info *vinfo, return true; } +/* vectorizable_comparison. + + Check if STMT_INFO is comparison expression that can be vectorized. + If VEC_STMT is also passed, vectorize STMT_INFO: create a vectorized + comparison, put it in VEC_STMT, and insert it at GSI. + + Return true if STMT_INFO is vectorizable in this way. */ + +static bool +vectorizable_comparison (vec_info *vinfo, + stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, + gimple **vec_stmt, + slp_tree slp_node, stmt_vector_for_cost *cost_vec) +{ + bb_vec_info bb_vinfo = dyn_cast (vinfo); + + if (!STMT_VINFO_RELEVANT_P (stmt_info) && !bb_vinfo) + return false; + + if (STMT_VINFO_DEF_TYPE (stmt_info) != vect_internal_def) + return false; + + gassign *stmt = dyn_cast (stmt_info->stmt); + if (!stmt) + return false; + + enum tree_code code = gimple_assign_rhs_code (stmt); + tree vectype = STMT_VINFO_VECTYPE (stmt_info); + if (!vectorizable_comparison_1 (vinfo, vectype, stmt_info, code, gsi, + vec_stmt, slp_node, cost_vec)) + return false; + + if (!vec_stmt) + STMT_VINFO_TYPE (stmt_info) = comparison_vec_info_type; + + return true; +} + /* If SLP_NODE is nonnull, return true if vectorizable_live_operation can handle all live statements in the node. Otherwise return true if STMT_INFO is not live or if vectorizable_live_operation can handle it. --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -11332,21 +11332,22 @@ vectorizable_condition (vec_info *vinfo, /* vectorizable_comparison. - Check if STMT_INFO is comparison expression that can be vectorized. +/* Helper of vectorizable_comparison. + + Check if STMT_INFO is comparison expression CODE that can be vectorized. If VEC_STMT is also passed, vectorize STMT_INFO: create a vectorized comparison, put it in VEC_STMT, and insert it at GSI. Return true if STMT_INFO is vectorizable in this way. */ static bool -vectorizable_comparison (vec_info *vinfo, - stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, - gimple **vec_stmt, - slp_tree slp_node, stmt_vector_for_cost *cost_vec) +vectorizable_comparison_1 (vec_info *vinfo, tree vectype, + stmt_vec_info stmt_info, tree_code code, + gimple_stmt_iterator *gsi, gimple **vec_stmt, + slp_tree slp_node, stmt_vector_for_cost *cost_vec) { tree lhs, rhs1, rhs2; tree vectype1 = NULL_TREE, vectype2 = NULL_TREE; - tree vectype = STMT_VINFO_VECTYPE (stmt_info); tree vec_rhs1 = NULL_TREE, vec_rhs2 = NULL_TREE; tree new_temp; loop_vec_info loop_vinfo = dyn_cast (vinfo); @@ -11354,7 +11355,7 @@ vectorizable_comparison (vec_info *vinfo, int ndts = 2; poly_uint64 nunits; int ncopies; - enum tree_code code, bitop1 = NOP_EXPR, bitop2 = NOP_EXPR; + enum tree_code bitop1 = NOP_EXPR, bitop2 = NOP_EXPR; int i; bb_vec_info bb_vinfo = dyn_cast (vinfo); vec vec_oprnds0 = vNULL; @@ -11377,14 +11378,6 @@ vectorizable_comparison (vec_info *vinfo, ncopies = vect_get_num_copies (loop_vinfo, vectype); gcc_assert (ncopies >= 1); - if (STMT_VINFO_DEF_TYPE (stmt_info) != vect_internal_def) - return false; - - gassign *stmt = dyn_cast (stmt_info->stmt); - if (!stmt) - return false; - - code = gimple_assign_rhs_code (stmt); if (TREE_CODE_CLASS (code) != tcc_comparison) return false; @@ -11499,7 +11492,6 @@ vectorizable_comparison (vec_info *vinfo, return false; } - STMT_VINFO_TYPE (stmt_info) = comparison_vec_info_type; vect_model_simple_cost (vinfo, stmt_info, ncopies * (1 + (bitop2 != NOP_EXPR)), dts, ndts, slp_node, cost_vec); @@ -11565,6 +11557,44 @@ vectorizable_comparison (vec_info *vinfo, return true; } +/* vectorizable_comparison. + + Check if STMT_INFO is comparison expression that can be vectorized. + If VEC_STMT is also passed, vectorize STMT_INFO: create a vectorized + comparison, put it in VEC_STMT, and insert it at GSI. + + Return true if STMT_INFO is vectorizable in this way. */ + +static bool +vectorizable_comparison (vec_info *vinfo, + stmt_vec_info stmt_info, gimple_stmt_iterator *gsi, + gimple **vec_stmt, + slp_tree slp_node, stmt_vector_for_cost *cost_vec) +{ + bb_vec_info bb_vinfo = dyn_cast (vinfo); + + if (!STMT_VINFO_RELEVANT_P (stmt_info) && !bb_vinfo) + return false; + + if (STMT_VINFO_DEF_TYPE (stmt_info) != vect_internal_def) + return false; + + gassign *stmt = dyn_cast (stmt_info->stmt); + if (!stmt) + return false; + + enum tree_code code = gimple_assign_rhs_code (stmt); + tree vectype = STMT_VINFO_VECTYPE (stmt_info); + if (!vectorizable_comparison_1 (vinfo, vectype, stmt_info, code, gsi, + vec_stmt, slp_node, cost_vec)) + return false; + + if (!vec_stmt) + STMT_VINFO_TYPE (stmt_info) = comparison_vec_info_type; + + return true; +} + /* If SLP_NODE is nonnull, return true if vectorizable_live_operation can handle all live statements in the node. Otherwise return true if STMT_INFO is not live or if vectorizable_live_operation can handle it.