From patchwork Thu Jun 1 04:36:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 101702 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp46138vqr; Wed, 31 May 2023 21:37:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6VDDaKjIifnN25gW05EOW6+XJoMdAhY3WYN6qdpuheWtrxsYnIgcu9hwQW6/QR579sDVhf X-Received: by 2002:aa7:c301:0:b0:504:a2e5:d951 with SMTP id l1-20020aa7c301000000b00504a2e5d951mr5783528edq.13.1685594219818; Wed, 31 May 2023 21:36:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685594219; cv=none; d=google.com; s=arc-20160816; b=Joap1pxr/ql3uaXLLliGxs5zGcboRim3lKftHf8vZCQV9s3OZK2kLMrXW9iI+qIPeR G7LsLxD+5n1QOTf0vangUAv1yv7JVkYv/ZjVNE1Nqr6a0HG6HiSVgFYMJhCaaAYcXdvA jAfE9fTiQ8V4JJRqaBdwPTmB+Eq47Ngm6rj5BtMccLO7Bv8K4kIUw3a0+WVVMOfMp5jE YEXDbhbx8yuQvr24oxasjfUf49wwD30exAUVF5yuRPnsvMAcWv/Uox35h9lRT4CavOyS B4DbGFBpWOhM+rU3Xe3ww8QvkZvpIGJoLXldF5qcRuXHRlEaCNUc0/U1ayOsnZNVBgq9 NITw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=MB0RbrQKHHYQCD+pNS2xLOrUt5JszEEQ9B1sxzbKAhs=; b=BuS2YrH7RMRdiIwkNknURZPAwVVGPjykTlosHa8HmCgxhR5G9Prm8p/1887jWB0m/b 0+YHwvkPjm6UlXISWZel1hcA3cqswC44cYDPAwnbqXMN9wKD074ahAIlvc2rFDEQxyh2 PO61EZT/72G3ScnUuMxKsyiRIVFPbrO0KZT5KQEKyEB3eaXfltFIrFQz7XYNAb29iAeh X0C4biGdlhHjQuGOnndpHpEO+kYb9m9Bh3gcxtFg0I41f2h0Gg91II60zmcuho0B+iPl 8V+I+tBFnvD5tFF2g4yOvc78DsPN8MFIW3GwE6c/29nO6sNowOyf10jHI1XkeJh8eOFH AkMQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id s18-20020a056402165200b0050c08405e71si3390269edx.613.2023.05.31.21.36.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 May 2023 21:36:59 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 631973857733 for ; Thu, 1 Jun 2023 04:36:55 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg156.qq.com (smtpbg156.qq.com [15.184.82.18]) by sourceware.org (Postfix) with ESMTPS id 430743858D20 for ; Thu, 1 Jun 2023 04:36:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 430743858D20 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp64t1685594180t0o8n2a8 Received: from server1.localdomain ( [58.60.1.22]) by bizesmtp.qq.com (ESMTP) with id ; Thu, 01 Jun 2023 12:36:19 +0800 (CST) X-QQ-SSF: 01400000000000F0R000000A0000000 X-QQ-FEAT: 1eo3oJ4iu3BKPJe5uBs1wn0CITmWcQT/mpkyk2k+6Qunn7a7mhS6Rcq/KT7Ei 9OKOEfX+2gXPbVe2LtkYUGqOKxTHc7uyKNoBST4c9Pgrf39b1FXBNROfyCuqlMs2xSf28iB rgtatmim/6o1W2V48fnLVyAzdjzvFMoUVMxg4AVDOD4F74YmgXZu72co/xXAjyeaV09bVX1 LPw2Bf1WmWmnEyYqAhi4wSK8jw5CbLAc+Uio0rREGknWaWN6919ZcPMmdcKXdjBY6D2FcDL HoE0gUn4b4ag0TD+ketBKKS06MFOzYHjysHE379e1lCQGOcPmdkaVcE9MB2OgpTRdlS0f9m kwRWraF2M2pUlEQwz515bMNDyx1ndHyt8QNlUt8Yhddzneut9UVt19s7rJAxdijRCaZpvHX 72k673wiZqs= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 12377322005757806926 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: richard.sandiford@arm.com, rguenther@suse.de, linkw@linux.ibm.com, Ju-Zhe Zhong Subject: [PATCH V3] VECT: Change flow of decrement IV Date: Thu, 1 Jun 2023 12:36:17 +0800 Message-Id: <20230601043617.173986-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H5, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767473644626804738?= X-GMAIL-MSGID: =?utf-8?q?1767473644626804738?= From: Ju-Zhe Zhong Follow Richi's suggestion, I change current decrement IV flow from: do { remain -= MIN (vf, remain); } while (remain != 0); into: do { old_remain = remain; len = MIN (vf, remain); remain -= vf; } while (old_remain >= vf); to enhance SCEV. Include fixes from kewen. This patch will need to wait for Kewen's test feedback. Testing on X86 is on-going Co-Authored by: Kewen Lin PR tree-optimization/109971 gcc/ChangeLog: * tree-vect-loop-manip.cc (vect_set_loop_controls_directly): Change decrement IV flow. (vect_set_loop_condition_partial_vectors): Ditto. --- gcc/tree-vect-loop-manip.cc | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index acf3642ceb2..3f735945e67 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -483,7 +483,7 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, gimple_stmt_iterator loop_cond_gsi, rgroup_controls *rgc, tree niters, tree niters_skip, bool might_wrap_p, - tree *iv_step) + tree *iv_step, tree *compare_step) { tree compare_type = LOOP_VINFO_RGROUP_COMPARE_TYPE (loop_vinfo); tree iv_type = LOOP_VINFO_RGROUP_IV_TYPE (loop_vinfo); @@ -538,9 +538,9 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, ... vect__4.8_28 = .LEN_LOAD (_17, 32B, _36, 0); ... - ivtmp_35 = ivtmp_9 - _36; + ivtmp_35 = ivtmp_9 - POLY_INT_CST [4, 4]; ... - if (ivtmp_35 != 0) + if (ivtmp_9 > POLY_INT_CST [4, 4]) goto ; [83.33%] else goto ; [16.67%] @@ -549,13 +549,15 @@ vect_set_loop_controls_directly (class loop *loop, loop_vec_info loop_vinfo, tree step = rgc->controls.length () == 1 ? rgc->controls[0] : make_ssa_name (iv_type); /* Create decrement IV. */ - create_iv (nitems_total, MINUS_EXPR, step, NULL_TREE, loop, &incr_gsi, - insert_after, &index_before_incr, &index_after_incr); + create_iv (nitems_total, MINUS_EXPR, nitems_step, NULL_TREE, loop, + &incr_gsi, insert_after, &index_before_incr, + &index_after_incr); gimple_seq_add_stmt (header_seq, gimple_build_assign (step, MIN_EXPR, index_before_incr, nitems_step)); *iv_step = step; - return index_after_incr; + *compare_step = nitems_step; + return index_before_incr; } /* Create increment IV. */ @@ -825,6 +827,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop, arbitrarily pick the last. */ tree test_ctrl = NULL_TREE; tree iv_step = NULL_TREE; + tree compare_step = NULL_TREE; rgroup_controls *rgc; rgroup_controls *iv_rgc = nullptr; unsigned int i; @@ -861,7 +864,7 @@ vect_set_loop_condition_partial_vectors (class loop *loop, &preheader_seq, &header_seq, loop_cond_gsi, rgc, niters, niters_skip, might_wrap_p, - &iv_step); + &iv_step, &compare_step); iv_rgc = rgc; } @@ -884,10 +887,21 @@ vect_set_loop_condition_partial_vectors (class loop *loop, /* Get a boolean result that tells us whether to iterate. */ edge exit_edge = single_exit (loop); - tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR; - tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl)); - gcond *cond_stmt = gimple_build_cond (code, test_ctrl, zero_ctrl, - NULL_TREE, NULL_TREE); + gcond *cond_stmt; + if (LOOP_VINFO_USING_DECREMENTING_IV_P (loop_vinfo)) + { + gcc_assert (compare_step); + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? LE_EXPR : GT_EXPR; + cond_stmt = gimple_build_cond (code, test_ctrl, compare_step, NULL_TREE, + NULL_TREE); + } + else + { + tree_code code = (exit_edge->flags & EDGE_TRUE_VALUE) ? EQ_EXPR : NE_EXPR; + tree zero_ctrl = build_zero_cst (TREE_TYPE (test_ctrl)); + cond_stmt + = gimple_build_cond (code, test_ctrl, zero_ctrl, NULL_TREE, NULL_TREE); + } gsi_insert_before (&loop_cond_gsi, cond_stmt, GSI_SAME_STMT); /* The loop iterates (NITERS - 1) / VF + 1 times.