From patchwork Tue Jan 16 13:59:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 188530 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:42cf:b0:101:a8e8:374 with SMTP id q15csp275587dye; Tue, 16 Jan 2024 06:05:00 -0800 (PST) X-Google-Smtp-Source: AGHT+IHmKAlkaLOsZ8ommkBnO4Q7/j7E+uSP8ZoaNJAER2+Bqc874pYBNNo6JLaq1cJLWMS2BLD3 X-Received: by 2002:a05:6214:258e:b0:681:246b:1bb9 with SMTP id fq14-20020a056214258e00b00681246b1bb9mr7660475qvb.22.1705413900548; Tue, 16 Jan 2024 06:05:00 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1705413900; cv=pass; d=google.com; s=arc-20160816; b=rbB+XDdtdUUk6/qaPolL8jT0gk7pnM5t76aemgy7HG5XoA5DKAhzb/I4YglZrd8ixe UnhAtkMWI+QTlqmmbyRj5+f7wtsH5gd/ybvIaQGPVVLFYauAhgcuUMSBJ3YdtYGBST1r GPaChEFdqWBKTlYQIrC6Rg8Qzhe2qG4K7DyC56wj6oCX3SVXqjVXeKIXDQcP97kubYux ICBinyUgn0uZBQ/ZFYuV/+jKn1Bqkjl1ND4C+e2NWxBriwoNcZKEc8mwVFcS4I+loz5V NWvjEMD+KJ/wobB1oYPRrjA8ybu98t5FM4G/V/rGkpeebYZAs/dFC4zBfE+aqxb+OJpz Zvdg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:errors-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:mime-version :subject:cc:to:from:date:dkim-signature:dkim-signature :dkim-signature:dkim-signature:arc-filter:dmarc-filter:delivered-to; bh=VR7hZrd7AGQDaXlMwfLXJUB7rbPQeuQcHKflWyn5g9g=; fh=iwBcjOQuJQeC9NGqXlwJbGDynxCKlwwYbxsdwpHBsEc=; b=kbyiZWP0qzKcHO69aTa1ysQ+uajiZuqVtTyYNaJQQN6VRY4duf7FZZYryI7g2Sqoku /vhI7f7JLtz6uz/YvXj1hg+zhlqZVpnhzB8g7fjLHNlZ3IgbZ+o+yIc+w4xyeXf2V5CV gtDYYzJpxmIrBfkyS7n7bth9SRFy/kFXzqlehUPxfvZH0jAdJ1kQtwDhxWCjHlz4QY3L IEpcuMSKqe6Up8tS3HeyhZtz4SV940vsfOwkHCU5fry7oaCMGimuESve8gkXtD3RsYCn Ras3O3p8O+kRL+9lOv/uhZ+8yl4iEuVJzOB+jH1suR8mwy22v5HB/Huh0dg0GwlPOPA/ //ZA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=inw6bNFi; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=fGfWowYy; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id j10-20020a0cae8a000000b0068081592c46si9729609qvd.145.2024.01.16.06.05.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jan 2024 06:05:00 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=inw6bNFi; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=fGfWowYy; dkim=neutral (no key) header.i=@suse.de header.s=susede2_ed25519; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 43D0D3858C29 for ; Tue, 16 Jan 2024 14:05:00 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2a07:de40:b251:101:10:150:64:1]) by sourceware.org (Postfix) with ESMTPS id 0880D3858D1E for ; Tue, 16 Jan 2024 14:04:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0880D3858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0880D3858D1E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a07:de40:b251:101:10:150:64:1 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705413860; cv=none; b=FnD/v1iZiEvORcK+OEq1Q3iyKqQ6AgXueov03sUS5EShERzoJnDsl4FUk8sx3O+uE6Ln6HYSDiXNO+YGPEWaI1w09UuZTtaUQH0IlpRZPEMwW1mbh4vaVAc4mMJ0xTMDsqgQ5Lb9rxwkhnw4vxuklxpgD/FjBw/YdhihlvAezCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705413860; c=relaxed/simple; bh=+JmPcDSXHWWFzpesE+eQYgnYIzXwxrU4QyVsuR1TDWE=; h=DKIM-Signature:DKIM-Signature:DKIM-Signature:DKIM-Signature:Date: From:To:Subject:MIME-Version; b=c7L/KljZ0F+LKa7AJVwXgQ+1XYISHRX4E5/OoxWty9AQleyNmnkVCJO8PqLOnTTnDZXdMzQKC0XdCQYnhMh+leAMhH3VzGcxmI3Rxk+qY56LReO8IMpseQnVNiGx9qREE7mc5ZFJdzMUDiaMANwOqj21qhiUEhxwOipNN+YdgPg= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from [10.168.4.150] (unknown [10.168.4.150]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 13469220F3; Tue, 16 Jan 2024 14:04:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1705413849; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=VR7hZrd7AGQDaXlMwfLXJUB7rbPQeuQcHKflWyn5g9g=; b=inw6bNFi1as47vgAa21zkeQeA2l3Mc627p/i15J6/8qQPolI7iWgyWtflo4AJtPUfHgtjC 22/k/mshg1JBX7gZTHgTpfGNQfTas54VPWFXgwRWJ0lbgFm47c1sW3WRiI+bn3JQmYgLTU N/jRioWcmT+kWrTDl3TOeWld6M2hePM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1705413849; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=VR7hZrd7AGQDaXlMwfLXJUB7rbPQeuQcHKflWyn5g9g=; b=V/TpYZrIC/h6BLyhBf+Ibxp5hE8Ny+Okz/Q5BKoFPCtjRjb2qUcDExQqW0r+s8tToU5o9c 5rqkCuG1CUfflNBw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1705413847; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=VR7hZrd7AGQDaXlMwfLXJUB7rbPQeuQcHKflWyn5g9g=; b=fGfWowYyHetyZdDyZxOCls8Iye5HqBneiDLba66TaxeiFWWD4pU2dwRzqpWd4fFgNTfAGR 9272MKKml/JdJyBd/Rf0OM06VBVuivB2HuFu71t2J2vKLyUhIlgHrWcJ/DREzT9tA/1lWo d7148IdGzMkxkw0lZwDnDGsJTKm+k04= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1705413847; h=from:from:reply-to:date:date:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=VR7hZrd7AGQDaXlMwfLXJUB7rbPQeuQcHKflWyn5g9g=; b=aruzD8E5fbXAUaPnyvrR+xYd9Cm/8MjtEmVU9cRf4GO9l3IaeAW60IwqCxFaoXBWzFdYu/ K2XsgJZ+0KagjjBA== Date: Tue, 16 Jan 2024 14:59:00 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org cc: tamar.christina@arm.com Subject: [PATCH] tree-optimization/113371 - avoid prologue peeling for peeled early exits MIME-Version: 1.0 Authentication-Results: smtp-out1.suse.de; none X-Spamd-Result: default: False [-0.60 / 50.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; MISSING_MID(2.50)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[tree-vect-loop-manip.cc:url,tree-vect-data-refs.cc:url]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; BAYES_HAM(-3.00)[100.00%] X-Spam-Level: X-Spam-Score: -0.60 X-Spam-Status: No, score=-10.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Message-Id: <20240116140500.43D0D3858C29@sourceware.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788256086162034937 X-GMAIL-MSGID: 1788256086162034937 The following avoids prologue peeling when doing early exit vectorization with the IV exit before the early exit. That's because we it invalidates the invariant that the effective latch of the loop is empty causing wrong continuation to the main loop. In particular this is prone to break virtual SSA form. Bootstrapped and tested on x86_64-unknown-linux-gnu, will push if the linaro CI is happy. Richard. PR tree-optimization/113371 * tree-vect-data-refs.cc (vect_enhance_data_refs_alignment): Do not peel when LOOP_VINFO_EARLY_BREAKS_VECT_PEELED. * tree-vect-loop-manip.cc (vect_do_peeling): Assert we do not perform prologue peeling when LOOP_VINFO_EARLY_BREAKS_VECT_PEELED. * gcc.dg/vect/pr113371.c: New testcase. --- gcc/testsuite/gcc.dg/vect/pr113371.c | 19 +++++++++++++++++++ gcc/tree-vect-data-refs.cc | 5 +++-- gcc/tree-vect-loop-manip.cc | 3 ++- 3 files changed, 24 insertions(+), 3 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/pr113371.c diff --git a/gcc/testsuite/gcc.dg/vect/pr113371.c b/gcc/testsuite/gcc.dg/vect/pr113371.c new file mode 100644 index 00000000000..46c4deb0db9 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/pr113371.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-O" } */ +/* { dg-additional-options "-march=silvermont" { target { x86_64-*-* i?86-*-* } } } */ + +long *BN_uadd_ap; + +void +BN_uadd (int dif, long t1) +{ + long *rp; + while (dif) + { + dif--; + t1 = *BN_uadd_ap; + *rp++ = t1; + if (t1) + break; + } +} diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index 5e86da39468..0495842b350 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -2354,8 +2354,9 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo) /* Check if we can possibly peel the loop. */ if (!vect_can_advance_ivs_p (loop_vinfo) || !slpeel_can_duplicate_loop_p (loop, LOOP_VINFO_IV_EXIT (loop_vinfo), - LOOP_VINFO_IV_EXIT (loop_vinfo)) - || loop->inner) + loop_preheader_edge (loop)) + || loop->inner + || LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)) do_peeling = false; struct _vect_peel_extended_info peel_for_known_alignment; diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index c7e73f65155..8aa9224e1a9 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -3262,7 +3262,8 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1, { e = loop_preheader_edge (loop); edge exit_e = LOOP_VINFO_IV_EXIT (loop_vinfo); - gcc_checking_assert (slpeel_can_duplicate_loop_p (loop, exit_e, e)); + gcc_checking_assert (slpeel_can_duplicate_loop_p (loop, exit_e, e) + && !LOOP_VINFO_EARLY_BREAKS_VECT_PEELED (loop_vinfo)); /* Peel prolog and put it on preheader edge of loop. */ edge scalar_e = LOOP_VINFO_SCALAR_IV_EXIT (loop_vinfo);