From patchwork Mon Nov 6 07:40:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 161904 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp2532781vqu; Mon, 6 Nov 2023 01:17:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IHTSMlcRkiMNegliLZyRPgz1YTKUQ3o7mVrKZhE3sULGuqChPGyoPN8jGDja3dfe5UfqQZw X-Received: by 2002:a05:622a:50b:b0:418:fed:be4 with SMTP id l11-20020a05622a050b00b004180fed0be4mr33542293qtx.21.1699262230257; Mon, 06 Nov 2023 01:17:10 -0800 (PST) ARC-Seal: i=4; a=rsa-sha256; t=1699262230; cv=pass; d=google.com; s=arc-20160816; b=zH6UykVhA9Z2b8UQldhZecb1G/YVVHU719UTLoNCF4ek+zH46EEyBZW5DVZWrJxBuh I6KIgPsB616mPB4+OQCsbta7zR5jLDH1uT+Zn31MzEcDEg/pKe3MEEl5VzbNf7UENYMH 2fA5ZbB9FCHCfQvHY3jpRWyq0hFGKYyNwBDIfFRAar9cVg2aLpUwf+N7ikRTH3Bm18xj kKbWXWoP8nRvoJUQqQDAQ728OP74fTf2Hrz616AS1oPishYxpWm1A3DJ9f0kxHnVJfqV SuonO2nJKwITLQJ2sh8fyHYq6vCI+L/ON70UQi8NHaFsRXL0ynroqMCW+RmHQfX+alns Q6hQ== ARC-Message-Signature: i=4; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:original-authentication-results :nodisclaimer:mime-version:in-reply-to:content-disposition :message-id:subject:cc:to:from:date:authentication-results-original :dkim-signature:dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=eohhDm7ly8MmKqEMPVki3IrMIFHpt1vOGt+6FcHsgA4=; fh=6lLuKcPp5JwcfhLsQ40FQN1vJS1KxlKbf2GiViQwbCM=; b=m8PjwAl2585HhQw2Qwf05yDtvOfFmRNstVxEmAb6nDPUWeHdbS6kWaU5e41r8B5mWG ySgKapy5ymI3plkUgTp1FjQeOFA/Mw1qja17+h/IrEEjudgTzINHiKZHFXb5/oIYnJIq oLearA4K6e1omwdAechfU3fTh+ZYy7Wma6klS//Z7Lrl+VhY85ysvAF+hkbEl7rV6BT0 6YwnSpZs0O9g0AJw1aSUbV1vmZZeN8Ro16TmofATTzOaw5thJAwPifsPolZLAtyKruVh RlGowPyAr/4pNETETdirri/X98wUIZY5pgtEDCW73ix+zO3Y0eiSSBnLn/9vpwTcXboN YVwg== ARC-Authentication-Results: i=4; mx.google.com; dkim=pass header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=OMcWLwYf; dkim=pass header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=OMcWLwYf; arc=pass (i=3); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id p19-20020a05622a13d300b0041cba84cc09si5256034qtk.357.2023.11.06.01.17.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Nov 2023 01:17:10 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=OMcWLwYf; dkim=pass header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=OMcWLwYf; arc=pass (i=3); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DFEE03861890 for ; Mon, 6 Nov 2023 07:41:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2088.outbound.protection.outlook.com [40.107.8.88]) by sourceware.org (Postfix) with ESMTPS id 379B93861878 for ; Mon, 6 Nov 2023 07:40:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 379B93861878 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 379B93861878 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.8.88 ARC-Seal: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1699256454; cv=pass; b=pqVcj872OaF/sjqh7QPBPRH+yM65mWnWGKR3U+2eN8AzEZSGLiCFFjrubb/FGcjuYG5pkjEuaXWmaFm80O6P8bxqD9gx7AbZlwx6jTJ4eTDqAxTfGTVfQii4j2Ph8n6ZKtxULZi5aZnhJY0VJHjnxh+SRDULWUBp0mh9mv5sjG0= ARC-Message-Signature: i=3; a=rsa-sha256; d=sourceware.org; s=key; t=1699256454; c=relaxed/simple; bh=mHQzvHr6neRPvoWETd7lbb6NAthP4yUZNp9BtScjMuA=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:Message-ID: MIME-Version; b=n0p5G9NQdQFrT78zQQIdklbtPfOPwgiJb+aYcqd8fbeAtWZPFPa/Tqd05L0fOLmes4lQmprDt5XsrM8LqKLbky4Ou3P15lJcEr+zUUU2VmMZ9qhdvkZ7aBARb8MVYkT6RsTfQMihs6H97ta/AEqcCQkT4K1pwq8oAO9FbFCD7OQ= ARC-Authentication-Results: i=3; server2.sourceware.org ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=OND51fT9+HX2LEFaH8ycu3iztGvyPSEAveE96GzDa0FZ62NlSr14blEOSXxND2GgC4PR+Mkn9wKMXPC3VY+Tsgc3PN6b88PNHBl0JU7Js5yEWZL3KPGVzBdI3DTXz3GKsFaJxfIl8DPxs4AWNVrwkuXxTPrmbX/TiltJjKbpToefYW4l2bLIC9EIq1SnquLgux4DQ3Wdc8GImS7h9tW2ZShZpIBbapAjnnc8xZmJzeN9x0ieKzlE60U7oW+t8l7VQBS7lrv8gQiPe5M2G25GOKdFOToV1/c8tR89cwBLdTY0yAUc6pl56qzn03tkgUhyy0zaOODwBkhgKfLn4x+4sA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eohhDm7ly8MmKqEMPVki3IrMIFHpt1vOGt+6FcHsgA4=; b=OSMSOuoOHIytlMjliL4p+ASSpyQP5AjtxzOdkoPFHZOKdPmbkz2PntU+/059K4TWe/axcIebiCmUPqtz1vYW04/umT6rJQYfKvqy8+9xSUO+EYp3cSiwCMWmaxL2+Sdug11+f7s5jyvH5mLA91FltyZVch/sD7SukINEoht+b8OtsWHcEZYs/ghtDtIsYwWCEZD6V41/fzZADkVSKAwwYgsvbaFHgKOkaoC4yywM/BwgEL5Ipo2ZfXzFM27sAHx1BEF9+FkLTfZax4U1dRE4FUcUJtT8oR2OcH1QcvS94+goHQlq2aQEqhBdituRhvrTN2DFM1eQzHXGTL7ke5nTwg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=gcc.gnu.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eohhDm7ly8MmKqEMPVki3IrMIFHpt1vOGt+6FcHsgA4=; b=OMcWLwYfDFoqNmGfyz43AI3K1++bkl3nPxFhvdMIUVy34PwCfTwWn3nqzkDZJTwjPjO8cjXHx8kbLfh+jC/JNYiiB1IipMR4q46lIX+ZZOdGrCcTUKtAr763wePK41pv7eUYDEQicG4riJPcWTDHOoJ4lcDrRDSZKzwpeKcdEzo= Received: from AM6P192CA0015.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::28) by DBBPR08MB6057.eurprd08.prod.outlook.com (2603:10a6:10:1f5::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6933.24; Mon, 6 Nov 2023 07:40:48 +0000 Received: from AM4PEPF00025F95.EURPRD83.prod.outlook.com (2603:10a6:209:83:cafe::e3) by AM6P192CA0015.outlook.office365.com (2603:10a6:209:83::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.27 via Frontend Transport; Mon, 6 Nov 2023 07:40:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM4PEPF00025F95.mail.protection.outlook.com (10.167.16.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.1 via Frontend Transport; Mon, 6 Nov 2023 07:40:46 +0000 Received: ("Tessian outbound 8289ea11ec17:v228"); Mon, 06 Nov 2023 07:40:46 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: acbbd8afa2bb2887 X-CR-MTA-TID: 64aa7808 Received: from 473de764631f.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id B077078C-FFA4-476F-93C3-D83C469D3DF6.1; Mon, 06 Nov 2023 07:40:39 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 473de764631f.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 06 Nov 2023 07:40:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=II92lZUvvzV5XL7cnQSYUpYh31ZDv8g5OtjrPd/Tyez1ws7Bl+Jyf0PUlG4Gj9OLV5zNeHQOD5HRS2KsJuubAkGl72xIzRiqXRTGxUnmTI1bVU6RXGyCZKIIwvtosI2eWBjUNcb2GzQUp8E8U5KX6dFVSe1/9UVkcyDJTd2M8a7gohcSH/nVD8bsfyUKCVZPwa5+8/0clBarE4lN7u3myEXIrwA/agjmuKXv4+j2xf+e4+QqowElAHnYvCpRN5eNva0PH+pSWS52YSqoXmdi2rwfPNcYx/tjvfDJaPcYEtrOYtnnwhpBGjA01/PpTLQRkLiwh+KXJYgAz7GsPipHGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eohhDm7ly8MmKqEMPVki3IrMIFHpt1vOGt+6FcHsgA4=; b=SoBk/flRqYMSLnHxgDPqhDjjHyPCJNwDoPjGYdKn76jHDSsKkubzBz24uEjWHiJ3s13zavR9dpfMvcST1xgWNwfS9PMTqyeCe5+0Bl0QtIw4pA429uOF18XEkfinG9pepyPd5yqOeOweo5wihaNyakZNU7juBCmFT3Cr6niHbvf9bwl8Z4D8QHbakxa8C8kQWdMG0vhNlun1Hs6F2USFb7EwEZYtUxRYlQw2gPDsbx6xehaDvPXF3TLzeDBAIxxrDQo/8Dt/mbwBR1HCk1gNXF/E3ZJk1eUvA5PcnW45jyh2BD0bCRmZmx4N0pOAwyRWOKKX80Zw1PvNiZaNwoA4XA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eohhDm7ly8MmKqEMPVki3IrMIFHpt1vOGt+6FcHsgA4=; b=OMcWLwYfDFoqNmGfyz43AI3K1++bkl3nPxFhvdMIUVy34PwCfTwWn3nqzkDZJTwjPjO8cjXHx8kbLfh+jC/JNYiiB1IipMR4q46lIX+ZZOdGrCcTUKtAr763wePK41pv7eUYDEQicG4riJPcWTDHOoJ4lcDrRDSZKzwpeKcdEzo= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by DBBPR08MB10676.eurprd08.prod.outlook.com (2603:10a6:10:530::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.28; Mon, 6 Nov 2023 07:40:36 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::26aa:efdd:a74a:27d0]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::26aa:efdd:a74a:27d0%5]) with mapi id 15.20.6954.028; Mon, 6 Nov 2023 07:40:35 +0000 Date: Mon, 6 Nov 2023 07:40:32 +0000 From: Tamar Christina To: gcc-patches@gcc.gnu.org Cc: nd@arm.com, rguenther@suse.de, jlaw@ventanamicro.com Subject: [PATCH 12/21]middle-end: Add remaining changes to peeling and vectorizer to support early breaks Message-ID: Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: LO3P123CA0004.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:ba::9) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI1PR08MB5325:EE_|DBBPR08MB10676:EE_|AM4PEPF00025F95:EE_|DBBPR08MB6057:EE_ X-MS-Office365-Filtering-Correlation-Id: c8e5a443-ad9d-4110-b0f0-08dbde9bb077 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: /aVcOTaHMZh9oURpRGVZPDb+lW0XqSGolkGvvhNTOI9pPhc9j9SlUeGfpxJuTYacUV7Afh3C1a6crsT9StUg00GZvGmvYYNHkueeYPtUtH7xxpMYXjGtgUG9h80/pM+rjMpxIleVyypfn4b6tlP1BovPPNVGcUs1a8qdZYY0hRrfHNrdTygTTTWFSIx6jdYtmK0Zhgo1Pxz2UUAxQFmG9HqexAZohSieXfOH3ccWFO6tJIbFcIoSVsjMiQTA12TdXnytwYEOVGhqIJCs9LCr/Ed8XTLMRnPPb35PGNAXteb1M1V6S1h1NXI7DwlcQ1ZutX/zuYhABtvk8QFFrx+lO33phlbGSJ9QBcgsydYiCexfcqHbqNNTbG/6h2oJhP+aaaMuLe2y+oyyo3cNl6GU44Z6iOgxeFwYxCX35nMoBULb7YqPaYmDlKT0sXpUfoe01simjxLD0u0yw5rMfGtyV/VyP9Ei/mQbZXcOGgi9c0CMiyOuku8C92BbVmFjJk1g3qKHj085GhOI1MuMT96nftLuJ8fIODduEOxtzgJJi6p12awmfypT0GVBZkY+TD4W2OqNCBcuiIZbdLXnPEPb9povb9BZwLzbGQZwXfXpbfbiH9GdYk52icG7AMJADNrY X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(396003)(376002)(366004)(346002)(39860400002)(230922051799003)(186009)(64100799003)(1800799009)(451199024)(6916009)(4326008)(8676002)(8936002)(41300700001)(36756003)(2906002)(235185007)(38100700002)(5660300002)(6486002)(66556008)(6512007)(316002)(66476007)(86362001)(44832011)(4743002)(66946007)(83380400001)(26005)(2616005)(6666004)(6506007)(478600001)(33964004)(44144004)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB10676 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM4PEPF00025F95.EURPRD83.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 5c13d68e-7da5-43ef-575e-08dbde9baa11 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4JANw2ik1pl8YZwHtgNAvKQ5wggAuHykRnqEAqN5ikb8uQGqzngWpkri9CuwAUTvVKxN8ttgQ7fVf4G0iFrma4QmHyAZ+JRwgGRHsDTr4Jed5pS68LM1HNqP91MpSc5SNyB/LCmrE8PyCPJtc3ZArYpiom8yZIvUOkNekPMxWaj7eyLwijurGWxiOJVgDYILe4f/hEn12pxWH6IVx54IcjSopp34litIHX6d+ttCMt3eU+9mpX7I6idovQAqn+RZy2I0cFeDeDzrt13l0u65xxaqdSlgyf6udkTIcDqnb2HemgWAE+a0M13fmXvo92CZjKnMY5hp4BmMjKN5D8Ha5PU5ba7ATFNn17YJWbg7AIsOEGNba9lzDWadvOKpIDHCQh9bUCX7zsuSHPs6dYi5x//23vmpLHMtqn8wblUH2R+LOJMwWS0rQpU17tWhcTibOZ7TNosEXpdMjjburkPPv7+XkLCpa0p4VZzzAB7cerqvqEmHfY0gdPmpI9TAoP1z6s4RASp2INl/qqYmgg+EmKnB5lMAtTSvxBpKno2cHgMGZ3qD1pc9gUmHMrJoGjVkU0lMg9hlPxlKYyVBq427L5911n9qmD71Gyv1959VsqCAef4RHbI/xPTmMjLsorr9q9le2uYdgKFhq07m3U0bAU27wXoBudUR90lqvsJuv7C8tb5fO6UUha1Baf/ghqNLAfdmfdIAV8a0+n86JnH/sR3tyxRn+ayhZASMY4DQlmSk6PSca9SrUYnxlvtLfrP7faIpEYtO+K9b5VlzMfHJyA== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230031)(4636009)(346002)(376002)(136003)(396003)(39860400002)(230922051799003)(186009)(64100799003)(82310400011)(1800799009)(451199024)(40470700004)(46966006)(36840700001)(86362001)(8676002)(5660300002)(41300700001)(44832011)(235185007)(8936002)(2906002)(336012)(316002)(70586007)(4743002)(70206006)(81166007)(83380400001)(356005)(4326008)(6916009)(6486002)(33964004)(6506007)(36860700001)(44144004)(40480700001)(6666004)(478600001)(40460700003)(47076005)(2616005)(82740400003)(107886003)(6512007)(36756003)(26005)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Nov 2023 07:40:46.4363 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c8e5a443-ad9d-4110-b0f0-08dbde9bb077 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00025F95.EURPRD83.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6057 X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781805592527157461 X-GMAIL-MSGID: 1781805592527157461 Hi All, This finishes wiring that didn't fit in any of the other patches. Essentially just adding related changes so peeling for early break works. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_set_loop_condition_normal, vect_do_peeling): Support early breaks. * tree-vect-loop.cc (vect_need_peeling_or_partial_vectors_p): Likewise. * tree-vectorizer.cc (pass_vectorize::execute): Check all exits. --- inline copy of patch -- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index eef2bb50c1505f5cf802d5d80300affc2cbe69f6..9c1405d79fd8fe8689007df3b7605b7a3d3ecdd7 100644 --- diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index eef2bb50c1505f5cf802d5d80300affc2cbe69f6..9c1405d79fd8fe8689007df3b7605b7a3d3ecdd7 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -1187,7 +1187,7 @@ vect_set_loop_condition_partial_vectors_avx512 (class loop *loop, loop handles exactly VF scalars per iteration. */ static gcond * -vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge exit_edge, +vect_set_loop_condition_normal (loop_vec_info loop_vinfo, edge exit_edge, class loop *loop, tree niters, tree step, tree final_iv, bool niters_maybe_zero, gimple_stmt_iterator loop_cond_gsi) @@ -1296,7 +1296,8 @@ vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge exit_edge, gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT); /* Record the number of latch iterations. */ - if (limit == niters) + if (limit == niters + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) /* Case A: the loop iterates NITERS times. Subtract one to get the latch count. */ loop->nb_iterations = fold_build2 (MINUS_EXPR, niters_type, niters, @@ -3242,6 +3243,16 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, bound_epilog += vf - 1; if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) bound_epilog += 1; + + /* For early breaks the scalar loop needs to execute at most VF times + to find the element that caused the break. */ + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) + { + bound_epilog = vf; + /* Force a scalar epilogue as we can't vectorize the index finding. */ + vect_epilogues = false; + } + bool epilog_peeling = maybe_ne (bound_epilog, 0U); poly_uint64 bound_scalar = bound_epilog; @@ -3376,14 +3387,23 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, bound_prolog + bound_epilog) : (!LOOP_REQUIRES_VERSIONING (loop_vinfo) || vect_epilogues)); + + /* We only support early break vectorization on known bounds at this time. + This means that if the vector loop can't be entered then we won't generate + it at all. So for now force skip_vector off because the additional control + flow messes with the BB exits and we've already analyzed them. */ + skip_vector = skip_vector && !LOOP_VINFO_EARLY_BREAKS (loop_vinfo); + /* Epilog loop must be executed if the number of iterations for epilog loop is known at compile time, otherwise we need to add a check at the end of vector loop and skip to the end of epilog loop. */ bool skip_epilog = (prolog_peeling < 0 || !LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) || !vf.is_constant ()); - /* PEELING_FOR_GAPS is special because epilog loop must be executed. */ - if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) + /* PEELING_FOR_GAPS and peeling for early breaks are special because epilog + loop must be executed. */ + if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) skip_epilog = false; class loop *scalar_loop = LOOP_VINFO_SCALAR_LOOP (loop_vinfo); diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 55d6aee3d29151e6b528f6fdde15c693e5bdd847..51a054c5b035ac80dfbbf3b5ba2f6da82fda91f6 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -1236,6 +1236,14 @@ vect_need_peeling_or_partial_vectors_p (loop_vec_info loop_vinfo) th = LOOP_VINFO_COST_MODEL_THRESHOLD (LOOP_VINFO_ORIG_LOOP_INFO (loop_vinfo)); + /* When we have multiple exits and VF is unknown, we must require partial + vectors because the loop bounds is not a minimum but a maximum. That is to + say we cannot unpredicate the main loop unless we peel or use partial + vectors in the epilogue. */ + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo) + && !LOOP_VINFO_VECT_FACTOR (loop_vinfo).is_constant ()) + return true; + if (LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) && LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo) >= 0) { @@ -3149,7 +3157,8 @@ start_over: /* If an epilogue loop is required make sure we can create one. */ if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) - || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo)) + || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo) + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) { if (dump_enabled_p ()) dump_printf_loc (MSG_NOTE, vect_location, "epilog loop required\n"); diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc index d97e2b54c25ac60378935392aa7b73476efed74b..8b495fc7ca137109fc0097c77837233c660e032a 100644 --- a/gcc/tree-vectorizer.cc +++ b/gcc/tree-vectorizer.cc @@ -1381,7 +1381,9 @@ pass_vectorize::execute (function *fun) predicates that need to be shared for optimal predicate usage. However reassoc will re-order them and prevent CSE from working as it should. CSE only the loop body, not the entry. */ - bitmap_set_bit (exit_bbs, single_exit (loop)->dest->index); + auto_vec exits = get_loop_exit_edges (loop); + for (edge exit : exits) + bitmap_set_bit (exit_bbs, exit->dest->index); edge entry = EDGE_PRED (loop_preheader_edge (loop)->src, 0); do_rpo_vn (fun, entry, exit_bbs); --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -1187,7 +1187,7 @@ vect_set_loop_condition_partial_vectors_avx512 (class loop *loop, loop handles exactly VF scalars per iteration. */ static gcond * -vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge exit_edge, +vect_set_loop_condition_normal (loop_vec_info loop_vinfo, edge exit_edge, class loop *loop, tree niters, tree step, tree final_iv, bool niters_maybe_zero, gimple_stmt_iterator loop_cond_gsi) @@ -1296,7 +1296,8 @@ vect_set_loop_condition_normal (loop_vec_info /* loop_vinfo */, edge exit_edge, gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT); /* Record the number of latch iterations. */ - if (limit == niters) + if (limit == niters + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) /* Case A: the loop iterates NITERS times. Subtract one to get the latch count. */ loop->nb_iterations = fold_build2 (MINUS_EXPR, niters_type, niters, @@ -3242,6 +3243,16 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, bound_epilog += vf - 1; if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) bound_epilog += 1; + + /* For early breaks the scalar loop needs to execute at most VF times + to find the element that caused the break. */ + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) + { + bound_epilog = vf; + /* Force a scalar epilogue as we can't vectorize the index finding. */ + vect_epilogues = false; + } + bool epilog_peeling = maybe_ne (bound_epilog, 0U); poly_uint64 bound_scalar = bound_epilog; @@ -3376,14 +3387,23 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, bound_prolog + bound_epilog) : (!LOOP_REQUIRES_VERSIONING (loop_vinfo) || vect_epilogues)); + + /* We only support early break vectorization on known bounds at this time. + This means that if the vector loop can't be entered then we won't generate + it at all. So for now force skip_vector off because the additional control + flow messes with the BB exits and we've already analyzed them. */ + skip_vector = skip_vector && !LOOP_VINFO_EARLY_BREAKS (loop_vinfo); + /* Epilog loop must be executed if the number of iterations for epilog loop is known at compile time, otherwise we need to add a check at the end of vector loop and skip to the end of epilog loop. */ bool skip_epilog = (prolog_peeling < 0 || !LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) || !vf.is_constant ()); - /* PEELING_FOR_GAPS is special because epilog loop must be executed. */ - if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo)) + /* PEELING_FOR_GAPS and peeling for early breaks are special because epilog + loop must be executed. */ + if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) skip_epilog = false; class loop *scalar_loop = LOOP_VINFO_SCALAR_LOOP (loop_vinfo); diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 55d6aee3d29151e6b528f6fdde15c693e5bdd847..51a054c5b035ac80dfbbf3b5ba2f6da82fda91f6 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -1236,6 +1236,14 @@ vect_need_peeling_or_partial_vectors_p (loop_vec_info loop_vinfo) th = LOOP_VINFO_COST_MODEL_THRESHOLD (LOOP_VINFO_ORIG_LOOP_INFO (loop_vinfo)); + /* When we have multiple exits and VF is unknown, we must require partial + vectors because the loop bounds is not a minimum but a maximum. That is to + say we cannot unpredicate the main loop unless we peel or use partial + vectors in the epilogue. */ + if (LOOP_VINFO_EARLY_BREAKS (loop_vinfo) + && !LOOP_VINFO_VECT_FACTOR (loop_vinfo).is_constant ()) + return true; + if (LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) && LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo) >= 0) { @@ -3149,7 +3157,8 @@ start_over: /* If an epilogue loop is required make sure we can create one. */ if (LOOP_VINFO_PEELING_FOR_GAPS (loop_vinfo) - || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo)) + || LOOP_VINFO_PEELING_FOR_NITER (loop_vinfo) + || LOOP_VINFO_EARLY_BREAKS (loop_vinfo)) { if (dump_enabled_p ()) dump_printf_loc (MSG_NOTE, vect_location, "epilog loop required\n"); diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc index d97e2b54c25ac60378935392aa7b73476efed74b..8b495fc7ca137109fc0097c77837233c660e032a 100644 --- a/gcc/tree-vectorizer.cc +++ b/gcc/tree-vectorizer.cc @@ -1381,7 +1381,9 @@ pass_vectorize::execute (function *fun) predicates that need to be shared for optimal predicate usage. However reassoc will re-order them and prevent CSE from working as it should. CSE only the loop body, not the entry. */ - bitmap_set_bit (exit_bbs, single_exit (loop)->dest->index); + auto_vec exits = get_loop_exit_edges (loop); + for (edge exit : exits) + bitmap_set_bit (exit_bbs, exit->dest->index); edge entry = EDGE_PRED (loop_preheader_edge (loop)->src, 0); do_rpo_vn (fun, entry, exit_bbs);