From patchwork Wed Jun 28 13:43:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 113888 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp8937679vqr; Wed, 28 Jun 2023 06:45:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4RZNjBJ05GRYeBpCINYK975QqM844zuctdQ+gD3NXTmDgz99y32M3z1mSGfXU4VWF9ZI1q X-Received: by 2002:a17:907:16a9:b0:98c:e3a1:dbb4 with SMTP id hc41-20020a17090716a900b0098ce3a1dbb4mr20032450ejc.68.1687959930456; Wed, 28 Jun 2023 06:45:30 -0700 (PDT) Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id gg18-20020a170906e29200b00988ceb28006si5741724ejb.754.2023.06.28.06.45.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jun 2023 06:45:30 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=gMnSHkMD; arc=fail (signature failed); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 0CD4B3856DCE for ; Wed, 28 Jun 2023 13:45:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0CD4B3856DCE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1687959905; bh=KlUK5zs44K+e6A8M/SegstZjuKi/yCGRI+AD+gbw9a0=; h=Date:To:Cc:Subject:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=gMnSHkMDP1M8twlLwiyV6wqaSCwkyHw1RAMJ8gPOwKEI9X57WGhWsGVzWjcD4tr6/ ohnhau0JSs7GuOlfee4DNN/m5cy7MBX1aVxfmNo2cIcefnGr/MS1FGArb2Ems+IoUf sB3EJPdxPevSG+/yiArM3cQ+K8sugxPlEIqbQC+w= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2054.outbound.protection.outlook.com [40.107.20.54]) by sourceware.org (Postfix) with ESMTPS id 4DDB83858C2D for ; Wed, 28 Jun 2023 13:44:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4DDB83858C2D Received: from DUZPR01CA0076.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::7) by DBBPR08MB6249.eurprd08.prod.outlook.com (2603:10a6:10:203::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.26; Wed, 28 Jun 2023 13:44:15 +0000 Received: from DBAEUR03FT053.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:46a:cafe::be) by DUZPR01CA0076.outlook.office365.com (2603:10a6:10:46a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.34 via Frontend Transport; Wed, 28 Jun 2023 13:44:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT053.mail.protection.outlook.com (100.127.142.121) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.20 via Frontend Transport; Wed, 28 Jun 2023 13:44:15 +0000 Received: ("Tessian outbound b11b8bb4dfe8:v142"); Wed, 28 Jun 2023 13:44:15 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 3452ead84983cf05 X-CR-MTA-TID: 64aa7808 Received: from ac284f754147.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 3E7F5CAB-5A96-4D9D-9D7D-0CCD857AC62B.1; Wed, 28 Jun 2023 13:44:09 +0000 Received: from EUR01-DB5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ac284f754147.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 28 Jun 2023 13:44:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XDP7T2BMnztSMsaJWBF20YbyJ6Ut56YEu43V/dcZHkNxtuUomFA0KC72EfBIa3BS7TdvUkJbDGrjxhUUkhuBhJySbDiVdSfuSq+BYv3FHa5jV7rNzRvuZtJQaJgSccFBTcUMXLmPzSpXzSl899UFGrQrbKaC5eNnpGg72oe7OK/ZxH4hYd8NsIw0y2v0GH9akJOYwO9GsB2YvZpOMgwjGSZT6zIdQlZZfaL4N9SiwhM2gB7C6nQKzdh2LXbkaUa0rYw6OOtCM+XZoFRD3w5OxJGk4hYHcFCDdK+8CJFDOS6m6obMp2QE9U6LqAH4tilj2WlvYTU9oit1xxb3wISq0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KlUK5zs44K+e6A8M/SegstZjuKi/yCGRI+AD+gbw9a0=; b=AkSRncNNBohAmumYNZNWslb9YyTwN2rJodVGBEX6S0QaimlUXcy7VZSEHPIpd/smjyEEZihMCaRAvk9v2MjDZQ09aVuHVZieNgbY/52EHqNYizOAEpYjvEZ4kh7kkpKA326ad6hJKPF3qz5WBD/c8PstjGDVBrIXDqUEPFLm5kwQaqDTzwC7fujA8upngb3w3idDmRs17mjK0GFiZfQBiuV6f6LNd643PVTUIL8GXqdOn0xe4oDiu83wP84IJtkDLTGqw8XHt5rvh6r+50wAl27+msmwHFyr52LTDmAH8SQw0OMmNL7ipka+C7zwlP2trem7LeP78JMEVTXg2zC1kQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by GV2PR08MB8172.eurprd08.prod.outlook.com (2603:10a6:150:7c::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.24; Wed, 28 Jun 2023 13:44:05 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::2301:1cde:cfe7:eaf0%6]) with mapi id 15.20.6521.026; Wed, 28 Jun 2023 13:44:05 +0000 Date: Wed, 28 Jun 2023 14:43:58 +0100 To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH 5/19]middle-end: Enable bit-field vectorization to work correctly when we're vectoring inside conds Message-ID: Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: SA1P222CA0146.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:3c2::28) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|GV2PR08MB8172:EE_|DBAEUR03FT053:EE_|DBBPR08MB6249:EE_ X-MS-Office365-Filtering-Correlation-Id: 7a13280d-df32-4b4e-c2f0-08db77ddc3c0 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: qbWQgkwT3M4yl4dGPJP/ql+65byyIpaNgX8Uf//ijOJkJBBmbfSWTM0fFufhJLAuAXBJP5NXMiWXx6NiqfCWYnZy67JbTdq3WtMkB8OMLdTrF6zPCLBjZj81hYDzQA/UBbpggD/u9X7aDqLotmebbqK5Qb/JOGzC/LWitFzax8GE8l7kXQMZ2F7yEc/XQirbdXf4sb1y2uw5lStpN2gOAuye4Adpq2e9wdQM10IJJx9PMNdy4rDegEEK7rcXTgocjSUlJWmOSTMJuahnEfABLP2d/8R5d1csXuZpFFte92/VlfUyHlQ0IRiat2m68yYh/1KYuE9aEqUWAvaf8fVL6EdLNlQE1hakbbMfwIYElAT+ZTLANl7JVKV0Hw5I7aHb+Dl4YMbS/FsXKCDHyVARYy+t4zE879TCYGy+GonFZd3UlWHiW3nPPGnMbeEa0dGYOB1US+Os68nv9wkdIkYE6crECjCvuWe4jlMX/mjU0313qtf1KlhrsuwEGjZwo8ZPLHP9t/JRD+NLGJDFANgDprTy+SPhZCTDcToyC3mUi5nhekSRsR+LxeSrJvdes3GYVXXPbpDOvEV5ksgoj/QoQ9DQRpxuBUxCfqw7wG8LRKZonKAsto5X4OcxFz7TXe8l X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(136003)(346002)(366004)(396003)(376002)(451199021)(2906002)(6486002)(33964004)(2616005)(4743002)(44144004)(6666004)(83380400001)(38100700002)(186003)(6512007)(86362001)(26005)(41300700001)(478600001)(36756003)(66556008)(66946007)(316002)(4326008)(6916009)(66476007)(6506007)(44832011)(5660300002)(235185007)(8936002)(8676002)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB8172 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT053.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 5f5c0347-5c42-4eab-328f-08db77ddbd6f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UI1ft/fTcEEzLNg6RwMGC38TdPWdb8Vs4P00+AipHR48d4qTdF5IkOw4c3p8B7vk5N/isnr7vKMfP5Z2gOIaGOjN0VPIw1HLaCzM4i2/fhQ8MV6WE+lb3yfQb9v2smZ1hHWKbsFTwjd60Q5bPo18hRWK9bQqqgcvJAEVXqG4iP/nl4qTw3avbs3o/vjrv83cXmcQf3WscNrQB5BujzHIP5rZ7cPyGJ0y5pi/0I84hBd0N9NI/jkool7xRMCzaqKbuY8HRnypF5BtS403sNjHjqxbKByPP8dXvCs8uAKLstM+VOEbu6Hhn8mrNdUUEqmHu+SDlcoK6EIgdiCMkHMrnzcUHk7bAb+acBOWKv/bdQUBb4sT4FC4pZ9b0aYTDEa5zePtAuaAQb7sqGjHdhLgQlmiXzWse2+SrkDJ4nALZ6iRWMPSLcyIJu7fd5OxhkU1aDHEmjfZX5mn9xLgbZoWGVH6qecIQOpggSvvrmpeKgj1W0Y03lLbRXVRac/Htrqg9vpSjzzr0KWUVEIXNwPORbFboZthHmTsmc/cRTowL0SL7jUBtYazPiucaE1q6paiUHnDEY248RYzU4vWabhdeOTHFHS/EBqkmIxIgCa0joKWwRGl2gJtk10hpjC+iftsaZJeRWHkX+Ye/moOG64KyAHUVsBMqff2VHJDFiR2uFYloVQtClutqIs3ptWMI9pDb8GcRqMfk5CwVTHRE4U+jGWbOjrdiB+lZk3xw1c1QPBsvuZ2ReRBrJztDhs+dze2DYvE4UoFPQa0pmOWOQDN0w== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(376002)(136003)(396003)(39860400002)(451199021)(36840700001)(46966006)(40470700004)(82310400005)(41300700001)(36860700001)(70206006)(107886003)(6486002)(47076005)(26005)(4743002)(6512007)(44144004)(6666004)(83380400001)(2616005)(336012)(478600001)(6506007)(33964004)(186003)(2906002)(356005)(44832011)(235185007)(36756003)(40460700003)(5660300002)(70586007)(81166007)(82740400003)(40480700001)(4326008)(316002)(6916009)(8676002)(8936002)(86362001)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2023 13:44:15.8724 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7a13280d-df32-4b4e-c2f0-08db77ddc3c0 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT053.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6249 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769954272056728411?= X-GMAIL-MSGID: =?utf-8?q?1769954272056728411?= Hi All, The bitfield vectorization support does not currently recognize bitfields inside gconds. This means they can't be used as conditions for early break vectorization which is a functionality we require. This adds support for them by explicitly matching and handling gcond as a source. Testcases are added in the testsuite update patch as the only way to get there is with the early break vectorization. See tests: - vect-early-break_20.c - vect-early-break_21.c Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-patterns.cc (vect_init_pattern_stmt): Copy STMT_VINFO_TYPE from original statement. (vect_recog_bitfield_ref_pattern): Support bitfields in gcond. Co-Authored-By: Andre Vieira --- inline copy of patch -- diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 60bc9be6819af9bd28a81430869417965ba9d82d..c221b1d64449ce3b6c8864bbec4b17ddf938c2d6 100644 --- diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 60bc9be6819af9bd28a81430869417965ba9d82d..c221b1d64449ce3b6c8864bbec4b17ddf938c2d6 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -128,6 +128,7 @@ vect_init_pattern_stmt (vec_info *vinfo, gimple *pattern_stmt, STMT_VINFO_RELATED_STMT (pattern_stmt_info) = orig_stmt_info; STMT_VINFO_DEF_TYPE (pattern_stmt_info) = STMT_VINFO_DEF_TYPE (orig_stmt_info); + STMT_VINFO_TYPE (pattern_stmt_info) = STMT_VINFO_TYPE (orig_stmt_info); if (!STMT_VINFO_VECTYPE (pattern_stmt_info)) { gcc_assert (!vectype @@ -2488,27 +2489,37 @@ static gimple * vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, tree *type_out) { - gassign *first_stmt = dyn_cast (stmt_info->stmt); + gassign *conv_stmt = dyn_cast (stmt_info->stmt); + gcond *cond_stmt = dyn_cast (stmt_info->stmt); - if (!first_stmt) - return NULL; - - gassign *bf_stmt; - if (CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (first_stmt)) - && TREE_CODE (gimple_assign_rhs1 (first_stmt)) == SSA_NAME) + gimple *bf_stmt = NULL; + tree cond_cst = NULL_TREE; + if (cond_stmt) { - gimple *second_stmt - = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (first_stmt)); - bf_stmt = dyn_cast (second_stmt); - if (!bf_stmt - || gimple_assign_rhs_code (bf_stmt) != BIT_FIELD_REF) + tree op = gimple_cond_lhs (cond_stmt); + if (TREE_CODE (op) != SSA_NAME) + return NULL; + bf_stmt = dyn_cast (SSA_NAME_DEF_STMT (op)); + cond_cst = gimple_cond_rhs (cond_stmt); + if (TREE_CODE (cond_cst) != INTEGER_CST) return NULL; } - else + else if (conv_stmt + && CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (conv_stmt)) + && TREE_CODE (gimple_assign_rhs1 (conv_stmt)) == SSA_NAME) + { + gimple *second_stmt = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (conv_stmt)); + bf_stmt = dyn_cast (second_stmt); + } + + if (!bf_stmt + || gimple_assign_rhs_code (bf_stmt) != BIT_FIELD_REF) return NULL; tree bf_ref = gimple_assign_rhs1 (bf_stmt); tree container = TREE_OPERAND (bf_ref, 0); + tree ret_type = cond_cst ? TREE_TYPE (container) + : TREE_TYPE (gimple_assign_lhs (conv_stmt)); if (!bit_field_offset (bf_ref).is_constant () || !bit_field_size (bf_ref).is_constant () @@ -2522,8 +2533,6 @@ vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, gimple *use_stmt, *pattern_stmt; use_operand_p use_p; - tree ret = gimple_assign_lhs (first_stmt); - tree ret_type = TREE_TYPE (ret); bool shift_first = true; tree container_type = TREE_TYPE (container); tree vectype = get_vectype_for_scalar_type (vinfo, container_type); @@ -2560,7 +2569,8 @@ vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, /* If the only use of the result of this BIT_FIELD_REF + CONVERT is a PLUS_EXPR then do the shift last as some targets can combine the shift and add into a single instruction. */ - if (single_imm_use (gimple_assign_lhs (first_stmt), &use_p, &use_stmt)) + if (conv_stmt + && single_imm_use (gimple_assign_lhs (conv_stmt), &use_p, &use_stmt)) { if (gimple_code (use_stmt) == GIMPLE_ASSIGN && gimple_assign_rhs_code (use_stmt) == PLUS_EXPR) @@ -2620,7 +2630,21 @@ vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, NOP_EXPR, result); } - *type_out = STMT_VINFO_VECTYPE (stmt_info); + if (cond_cst) + { + append_pattern_def_seq (vinfo, stmt_info, pattern_stmt, vectype); + pattern_stmt + = gimple_build_cond (gimple_cond_code (cond_stmt), + gimple_get_lhs (pattern_stmt), + fold_convert (ret_type, cond_cst), + gimple_cond_true_label (cond_stmt), + gimple_cond_false_label (cond_stmt)); + *type_out = STMT_VINFO_VECTYPE (stmt_info); + } + else + *type_out + = get_vectype_for_scalar_type (vinfo, + TREE_TYPE (gimple_get_lhs (pattern_stmt))); vect_pattern_detected ("bitfield_ref pattern", stmt_info->stmt); return pattern_stmt; --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -128,6 +128,7 @@ vect_init_pattern_stmt (vec_info *vinfo, gimple *pattern_stmt, STMT_VINFO_RELATED_STMT (pattern_stmt_info) = orig_stmt_info; STMT_VINFO_DEF_TYPE (pattern_stmt_info) = STMT_VINFO_DEF_TYPE (orig_stmt_info); + STMT_VINFO_TYPE (pattern_stmt_info) = STMT_VINFO_TYPE (orig_stmt_info); if (!STMT_VINFO_VECTYPE (pattern_stmt_info)) { gcc_assert (!vectype @@ -2488,27 +2489,37 @@ static gimple * vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, tree *type_out) { - gassign *first_stmt = dyn_cast (stmt_info->stmt); + gassign *conv_stmt = dyn_cast (stmt_info->stmt); + gcond *cond_stmt = dyn_cast (stmt_info->stmt); - if (!first_stmt) - return NULL; - - gassign *bf_stmt; - if (CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (first_stmt)) - && TREE_CODE (gimple_assign_rhs1 (first_stmt)) == SSA_NAME) + gimple *bf_stmt = NULL; + tree cond_cst = NULL_TREE; + if (cond_stmt) { - gimple *second_stmt - = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (first_stmt)); - bf_stmt = dyn_cast (second_stmt); - if (!bf_stmt - || gimple_assign_rhs_code (bf_stmt) != BIT_FIELD_REF) + tree op = gimple_cond_lhs (cond_stmt); + if (TREE_CODE (op) != SSA_NAME) + return NULL; + bf_stmt = dyn_cast (SSA_NAME_DEF_STMT (op)); + cond_cst = gimple_cond_rhs (cond_stmt); + if (TREE_CODE (cond_cst) != INTEGER_CST) return NULL; } - else + else if (conv_stmt + && CONVERT_EXPR_CODE_P (gimple_assign_rhs_code (conv_stmt)) + && TREE_CODE (gimple_assign_rhs1 (conv_stmt)) == SSA_NAME) + { + gimple *second_stmt = SSA_NAME_DEF_STMT (gimple_assign_rhs1 (conv_stmt)); + bf_stmt = dyn_cast (second_stmt); + } + + if (!bf_stmt + || gimple_assign_rhs_code (bf_stmt) != BIT_FIELD_REF) return NULL; tree bf_ref = gimple_assign_rhs1 (bf_stmt); tree container = TREE_OPERAND (bf_ref, 0); + tree ret_type = cond_cst ? TREE_TYPE (container) + : TREE_TYPE (gimple_assign_lhs (conv_stmt)); if (!bit_field_offset (bf_ref).is_constant () || !bit_field_size (bf_ref).is_constant () @@ -2522,8 +2533,6 @@ vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, gimple *use_stmt, *pattern_stmt; use_operand_p use_p; - tree ret = gimple_assign_lhs (first_stmt); - tree ret_type = TREE_TYPE (ret); bool shift_first = true; tree container_type = TREE_TYPE (container); tree vectype = get_vectype_for_scalar_type (vinfo, container_type); @@ -2560,7 +2569,8 @@ vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, /* If the only use of the result of this BIT_FIELD_REF + CONVERT is a PLUS_EXPR then do the shift last as some targets can combine the shift and add into a single instruction. */ - if (single_imm_use (gimple_assign_lhs (first_stmt), &use_p, &use_stmt)) + if (conv_stmt + && single_imm_use (gimple_assign_lhs (conv_stmt), &use_p, &use_stmt)) { if (gimple_code (use_stmt) == GIMPLE_ASSIGN && gimple_assign_rhs_code (use_stmt) == PLUS_EXPR) @@ -2620,7 +2630,21 @@ vect_recog_bitfield_ref_pattern (vec_info *vinfo, stmt_vec_info stmt_info, NOP_EXPR, result); } - *type_out = STMT_VINFO_VECTYPE (stmt_info); + if (cond_cst) + { + append_pattern_def_seq (vinfo, stmt_info, pattern_stmt, vectype); + pattern_stmt + = gimple_build_cond (gimple_cond_code (cond_stmt), + gimple_get_lhs (pattern_stmt), + fold_convert (ret_type, cond_cst), + gimple_cond_true_label (cond_stmt), + gimple_cond_false_label (cond_stmt)); + *type_out = STMT_VINFO_VECTYPE (stmt_info); + } + else + *type_out + = get_vectype_for_scalar_type (vinfo, + TREE_TYPE (gimple_get_lhs (pattern_stmt))); vect_pattern_detected ("bitfield_ref pattern", stmt_info->stmt); return pattern_stmt;