From patchwork Fri Nov 10 13:16:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Biener X-Patchwork-Id: 163797 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b129:0:b0:403:3b70:6f57 with SMTP id q9csp1107606vqs; Fri, 10 Nov 2023 05:17:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IEOCPlDdmbtFx6lCENRtY+mqdo7Dkhsj9sDj0vCFKmCM2V5EuI8sdoF1Ngr6xsWnBTqMhTE X-Received: by 2002:ac8:5f89:0:b0:421:aedb:def9 with SMTP id j9-20020ac85f89000000b00421aedbdef9mr2096207qta.8.1699622243157; Fri, 10 Nov 2023 05:17:23 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699622243; cv=pass; d=google.com; s=arc-20160816; b=OCo/tWcmAZP5qxImAhK6OmzuIjaz8gg1eqOlCPZB/G2WX7GdDpXy1pF4b7KIHbR5Kn 6b37HB4nAPUhamS/lJZQ4R0zpiskdgZB2flwQjlFN4xK4hYHExJvPKC84aXiYcbvlWQF DXGCjXFEU3xLNOY8LlcvdEVuV4DPOMWa6Hv+JPIfaxly408UGwbAcZvMLjSSNlLZL1Vq CqBsjicW/AhZmfPLXFCWLKD+hiJ/NOUR363Q8lcguyLJL48zEGKIgbHjoOD3RbsRf+T0 18xpONRopdEGD1VYId02YbnlRNz5UGJ5CTWrvhkh1xsmBD5LhrY1D4vYcETgyPqCqPkX 28xQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:message-id:mime-version:subject :to:from:date:dkim-signature:dkim-signature:arc-filter:dmarc-filter :delivered-to; bh=Nch1YrtEGpgkM3bGP8kk+Id9X2wOwyXbjY0SrSSFK9Y=; fh=hPrbWPhweUx4V0GV9uXJqbyAzg2ABmTz7kczrAQqMmM=; b=J72/tUij5Z2kCv4FC7/fw/3S3B/L0U/dlv8Pj5whjqHCzd+epB3UHILaE7pIyzDcuy Wtdpvr39aPx32xx3JOneMXsJIEMR7MgI7pFgqMQc9y+HWG8pLGCmodvmGVFPffGvV3qt OHEG9DFh3esmwKx83iexnNHkm2itIc8AIYQi3YQFLRTQcEkc+5jPD8QsVvUcU5cWdBmp +nX75H3WF5vNp35KjZ9Ma2F4pdaLQ85TF4CtejSXvuoBFrh1e+4zqf1vwpSMRMGZeWm8 EHlwy0KJrFx+hl/Qkvlr7Qxo3J+N5dsqQ+k8oI56H2e5ZIYCIP6GyHF4ee2o+XFrJP3s oGHw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=xOjF9kke; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id z19-20020ac87f93000000b004197ed64cf6si4538658qtj.623.2023.11.10.05.17.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Nov 2023 05:17:23 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=xOjF9kke; dkim=neutral (no key) header.i=@suse.de; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 652503858415 for ; Fri, 10 Nov 2023 13:17:22 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by sourceware.org (Postfix) with ESMTPS id E3A5C3858C41 for ; Fri, 10 Nov 2023 13:16:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E3A5C3858C41 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.de ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E3A5C3858C41 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=195.135.220.28 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699622220; cv=none; b=lqJMOisk59BBm1aFzX+o7zIdYt6XCSN50Z0kCXP+5H/tE+Mvu6pH67pi3GP1Smr9j1FDyGBq0bu/GfffwlNh87i8WpwE8I+yUYLLdPM3pDHQw27sxF+TmaVRp/453AlypNuDEhjiVlMXDFAEGNPjdmDTtbW9rjj9Ca6IawVmHAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699622220; c=relaxed/simple; bh=sIKEx1aIulKMuQPjNYyfWT+xr0jb8lzFfyVNdSEL6wA=; h=DKIM-Signature:DKIM-Signature:Date:From:To:Subject:MIME-Version: Message-Id; b=j6dLHQ6xxn6HKhjDftNbrSOhfk1SpgmgmziA1Prmzd0lmptuYHv5CsadPM+QNAL8m5d/kBoe+Mei6VqlFoKyhkDEIpBVn4Sn6tcWaekf0/TfCdZ4Zcgl2WsZTAhNPIVlKgjw2wN7ETmeVwq/rfkLrkPS1Pukvc9qmTsPztwdgPM= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1D25C2187C for ; Fri, 10 Nov 2023 13:16:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1699622218; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=Nch1YrtEGpgkM3bGP8kk+Id9X2wOwyXbjY0SrSSFK9Y=; b=xOjF9kkeeA8ALEzOttb2cjzvtjqTGu3Dp5NueWkUqQpcjeaVdqnSEj0bLvyg1EsGmH6w4f tnQjsE5guyUo+l7evxWrY4ucJHQtK/okpeL+xxlaSLtjVJdSFMIJIcfxbr/1fepfOyxu8j STAtb9gpYyBsqDo1scnGciZtpyi3SW8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1699622218; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type; bh=Nch1YrtEGpgkM3bGP8kk+Id9X2wOwyXbjY0SrSSFK9Y=; b=sa4eI8Z9Zx2vMW419R6KLVtaAVdza6fDHC/EH8KEvfdMIeSrw9wFSHbccc+6fTn5mQq6PL l6lVcxAcIl8GsnBQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 09A5D13398 for ; Fri, 10 Nov 2023 13:16:58 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id n68TAUotTmVCNgAAMHmgww (envelope-from ) for ; Fri, 10 Nov 2023 13:16:58 +0000 Date: Fri, 10 Nov 2023 14:16:57 +0100 (CET) From: Richard Biener To: gcc-patches@gcc.gnu.org Subject: [PATCH] tree-optimization/110221 - SLP and loop mask/len MIME-Version: 1.0 Message-Id: <20231110131658.09A5D13398@imap2.suse-dmz.suse.de> X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782183092956463764 X-GMAIL-MSGID: 1782183092956463764 The following fixes the issue that when SLP stmts are internal defs but appear invariant because they end up only using invariant defs then they get scheduled outside of the loop. This nice optimization breaks down when loop masks or lens are applied since those are not explicitly tracked as dependences. The following makes sure to never schedule internal defs outside of the vectorized loop when the loop uses masks/lens. Bootstrapped and tested on x86_64-unknown-linux-gnu, pushed. PR tree-optimization/110221 * tree-vect-slp.cc (vect_schedule_slp_node): When loop masking / len is applied make sure to not schedule intenal defs outside of the loop. * gfortran.dg/pr110221.f: New testcase. --- gcc/testsuite/gfortran.dg/pr110221.f | 17 +++++++++++++++++ gcc/tree-vect-slp.cc | 10 ++++++++++ 2 files changed, 27 insertions(+) create mode 100644 gcc/testsuite/gfortran.dg/pr110221.f diff --git a/gcc/testsuite/gfortran.dg/pr110221.f b/gcc/testsuite/gfortran.dg/pr110221.f new file mode 100644 index 00000000000..8b57384313a --- /dev/null +++ b/gcc/testsuite/gfortran.dg/pr110221.f @@ -0,0 +1,17 @@ +C PR middle-end/68146 +C { dg-do compile } +C { dg-options "-O2 -w" } +C { dg-additional-options "-mavx512f --param vect-partial-vector-usage=2" { target avx512f } } + SUBROUTINE CJYVB(V,Z,V0,CBJ,CDJ,CBY,CYY) + IMPLICIT DOUBLE PRECISION (A,B,G,O-Y) + IMPLICIT COMPLEX*16 (C,Z) + DIMENSION CBJ(0:*),CDJ(0:*),CBY(0:*) + N=INT(V) + CALL GAMMA2(VG,GA) + DO 65 K=1,N + CBY(K)=CYY +65 CONTINUE + CDJ(0)=V0/Z*CBJ(0)-CBJ(1) + DO 70 K=1,N +70 CDJ(K)=-(K+V0)/Z*CBJ(K)+CBJ(K-1) + END diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index 3e5814c3a31..80e279d8f50 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -9081,6 +9081,16 @@ vect_schedule_slp_node (vec_info *vinfo, /* Emit other stmts after the children vectorized defs which is earliest possible. */ gimple *last_stmt = NULL; + if (auto loop_vinfo = dyn_cast (vinfo)) + if (LOOP_VINFO_FULLY_MASKED_P (loop_vinfo) + || LOOP_VINFO_FULLY_WITH_LENGTH_P (loop_vinfo)) + { + /* But avoid scheduling internal defs outside of the loop when + we might have only implicitly tracked loop mask/len defs. */ + gimple_stmt_iterator si + = gsi_after_labels (LOOP_VINFO_LOOP (loop_vinfo)->header); + last_stmt = *si; + } bool seen_vector_def = false; FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (node), i, child) if (SLP_TREE_DEF_TYPE (child) == vect_internal_def)