From patchwork Fri Jul 22 08:16:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Pan2 via Gcc-patches" X-Patchwork-Id: 119 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f503:0:0:0:0:0 with SMTP id q3csp124333wro; Fri, 22 Jul 2022 01:18:05 -0700 (PDT) X-Google-Smtp-Source: AGRyM1ujl6y47Kznj4YMNeoyAj41MKnMgYNX3VMYNlmvehvejXknUaClwwgVqtDVtdv+wwqyJjKg X-Received: by 2002:a17:907:1c0c:b0:72e:dd1f:a230 with SMTP id nc12-20020a1709071c0c00b0072edd1fa230mr2279581ejc.8.1658477885038; Fri, 22 Jul 2022 01:18:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658477885; cv=none; d=google.com; s=arc-20160816; b=rOiLsCAJGoxAXcqQy1/lWdpkuAvvr/muc6zfOiFmSANwzYdPW0ioUpdqg5bXAMRVc6 ZIPXC+1p6FdVKNqwyPdF7oplV7tW7XQdmRlPr3lxdMhMfmU0M39WbmSvTVyTKlmapAa6 mbjujXQTgzsQlEHd5KMgDJ57uZhFL2YpFjcRAlv6FSHrhsYGJA3vSfN5nryv/O0vAk9U GoNIdbbV8ocLJEWMGUKEihNoKDCzm+t4PkA+wh/7ASmO9q9gYJa+hbrW6csJFOAQ6yKl Z1A6bam+gnaSII3xVOAeJ8eqdxdGijkLNZOp+KvDT176MkvAoO3bv4ahcvEWfPxGQ/z7 CAIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:sender:errors-to:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mime-version:user-agent:subject:to:date:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=vfk9xRl5whTNbp+ZEE8mZrla3C3IqjHBRCld5XSCgIM=; b=GCSdkhNSVuKHFs8R5ZGYN2zN24veLrXp63OtscItAr7jnZN6UrVG2xM7WWdsxwvMXp Th/xIBFn/hxWtVSW01xQxAddkV+NRc2ewrOXzeK7/SIa5rBEbGDSJlZuund4ee/M9QA5 LjCQpOFWsFwOt7y7/x0P4IOX65aq3sFiYQ1U9vnq1RvyA7RXP/xuTU7DjFfMHFrU6IZK YjROPns29G+cmS1m4PeKDvqzQiOvsnf4WLQ0DbdbX+tiSoIxOQypcQbH8Lz0QP9kc7u4 0ECkfZ5RdH37MOuiLBDjaDBB7XMo2Ak3PaIsjTsh61ovSIQdwtUrIwzNb+Z9ZP9A1GIq LCuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=QfsV7PVf; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id u9-20020a05640207c900b0043b644f0175si4523046edy.88.2022.07.22.01.18.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jul 2022 01:18:05 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=QfsV7PVf; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C61073835696 for ; Fri, 22 Jul 2022 08:18:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C61073835696 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1658477883; bh=vfk9xRl5whTNbp+ZEE8mZrla3C3IqjHBRCld5XSCgIM=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=QfsV7PVfzDZ+NR+GlH0Goa9Nepq69jhMQ2HvHivusMS8xayZCIsLAUHD2Yo1ga56E UrMaiCqRq6KTarDGROCV9jNRVmGDz/sxnn1F/76OO+Y+5nWNXnoMLmv2OUi8Gfpj0T QuqekctRjNhY5/TBVstSAJFPX6xAlF0ydcoGwgZU= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [IPv6:2001:67c:2178:6::1d]) by sourceware.org (Postfix) with ESMTPS id B80EB38356B4 for ; Fri, 22 Jul 2022 08:16:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org B80EB38356B4 Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id D850D1F9BE for ; Fri, 22 Jul 2022 08:16:33 +0000 (UTC) Received: from wotan.suse.de (wotan.suse.de [10.160.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id C80B12C153 for ; Fri, 22 Jul 2022 08:16:33 +0000 (UTC) Date: Fri, 22 Jul 2022 08:16:33 +0000 (UTC) To: gcc-patches@gcc.gnu.org Subject: [PATCH] tree-optimization/106397 - array prefetch and LC SSA User-Agent: Alpine 2.22 (LSU 394 2020-01-19) MIME-Version: 1.0 X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, MISSING_MID, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Biener via Gcc-patches From: "Li, Pan2 via Gcc-patches" Reply-To: Richard Biener Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" Message-Id: <20220722081803.C61073835696@sourceware.org> X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1739040106980656266?= X-GMAIL-MSGID: =?utf-8?q?1739040106980656266?= The following fixes maintaining LC SSA when array prefetch inserts mfence instructions on loop exits that do not use memory. It also fixes the latent issue that it might split exit edges for this which will break LC SSA for non-virtuals as well. It should also make the process cheaper by accumulating the required (LC) SSA update until the end of the pass. Bootstrap and regtest running on x86_64-unknown-linux-gnu. PR tree-optimization/106397 * tree-ssa-loop-prefetch.cc (emit_mfence_after_loop): Do not update SSA form here. (mark_nontemporal_stores): Return whether we marked any non-temporal stores and inserted mfence. (loop_prefetch_arrays): Note when we need to update SSA. (tree_ssa_prefetch_arrays): Perform required (LC) SSA update at the end of the pass. * gcc.dg/pr106397.c: New testcase. --- gcc/testsuite/gcc.dg/pr106397.c | 17 +++++++++++++++++ gcc/tree-ssa-loop-prefetch.cc | 27 +++++++++++++++++---------- 2 files changed, 34 insertions(+), 10 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/pr106397.c diff --git a/gcc/testsuite/gcc.dg/pr106397.c b/gcc/testsuite/gcc.dg/pr106397.c new file mode 100644 index 00000000000..a6b2e913346 --- /dev/null +++ b/gcc/testsuite/gcc.dg/pr106397.c @@ -0,0 +1,17 @@ +/* { dg-do compile } */ +/* { dg-options "-O3 -fprefetch-loop-arrays --param l2-cache-size=0 --param prefetch-latency=3 -fprefetch-loop-arrays" } */ + +int +bar (void) +{ + /* No return statement. */ +} + +__attribute__ ((simd)) int +foo (void) +{ + if (bar ()) + return 0; + + __builtin_unreachable (); +} diff --git a/gcc/tree-ssa-loop-prefetch.cc b/gcc/tree-ssa-loop-prefetch.cc index 8f190ae469b..b6690b0e805 100644 --- a/gcc/tree-ssa-loop-prefetch.cc +++ b/gcc/tree-ssa-loop-prefetch.cc @@ -1308,8 +1308,6 @@ emit_mfence_after_loop (class loop *loop) gsi_insert_before (&bsi, call, GSI_NEW_STMT); } - - update_ssa (TODO_update_ssa_only_virtuals); } /* Returns true if we can use storent in loop, false otherwise. */ @@ -1340,23 +1338,27 @@ may_use_storent_in_loop_p (class loop *loop) } /* Marks nontemporal stores in LOOP. GROUPS contains the description of memory - references in the loop. */ + references in the loop. Returns whether we inserted any mfence call. */ -static void +static bool mark_nontemporal_stores (class loop *loop, struct mem_ref_group *groups) { struct mem_ref *ref; bool any = false; if (!may_use_storent_in_loop_p (loop)) - return; + return false; for (; groups; groups = groups->next) for (ref = groups->refs; ref; ref = ref->next) any |= mark_nontemporal_store (ref); if (any && FENCE_FOLLOWING_MOVNT != NULL_TREE) - emit_mfence_after_loop (loop); + { + emit_mfence_after_loop (loop); + return true; + } + return false; } /* Determines whether we can profitably unroll LOOP FACTOR times, and if @@ -1874,10 +1876,11 @@ insn_to_prefetch_ratio_too_small_p (unsigned ninsns, unsigned prefetch_count, /* Issue prefetch instructions for array references in LOOP. Returns - true if the LOOP was unrolled. */ + true if the LOOP was unrolled and updates NEED_LC_SSA_UPDATE if we need + to update SSA for virtual operands and LC SSA for a split edge. */ static bool -loop_prefetch_arrays (class loop *loop) +loop_prefetch_arrays (class loop *loop, bool &need_lc_ssa_update) { struct mem_ref_group *refs; unsigned ahead, ninsns, time, unroll_factor; @@ -1952,7 +1955,7 @@ loop_prefetch_arrays (class loop *loop) unroll_factor)) goto fail; - mark_nontemporal_stores (loop, refs); + need_lc_ssa_update |= mark_nontemporal_stores (loop, refs); /* Step 4: what to prefetch? */ if (!schedule_prefetches (refs, unroll_factor, ahead)) @@ -1980,6 +1983,7 @@ unsigned int tree_ssa_prefetch_arrays (void) { bool unrolled = false; + bool need_lc_ssa_update = false; int todo_flags = 0; if (!targetm.have_prefetch () @@ -2028,12 +2032,15 @@ tree_ssa_prefetch_arrays (void) if (dump_file && (dump_flags & TDF_DETAILS)) fprintf (dump_file, "Processing loop %d:\n", loop->num); - unrolled |= loop_prefetch_arrays (loop); + unrolled |= loop_prefetch_arrays (loop, need_lc_ssa_update); if (dump_file && (dump_flags & TDF_DETAILS)) fprintf (dump_file, "\n\n"); } + if (need_lc_ssa_update) + rewrite_into_loop_closed_ssa (NULL, TODO_update_ssa_only_virtuals); + if (unrolled) { scev_reset ();