From patchwork Thu Jul 6 06:16:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "juzhe.zhong@rivai.ai" X-Patchwork-Id: 116522 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp2346371vqx; Wed, 5 Jul 2023 23:17:13 -0700 (PDT) X-Google-Smtp-Source: APBJJlE71WYdOq4Qw7PxwpbXU3pgRNEhJXDESWM8utothtnnFXw/jILpSGSPdeUgTRwKzolAqen3 X-Received: by 2002:aa7:d74f:0:b0:51d:9905:6f60 with SMTP id a15-20020aa7d74f000000b0051d99056f60mr600026eds.41.1688624233116; Wed, 05 Jul 2023 23:17:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688624233; cv=none; d=google.com; s=arc-20160816; b=HcwGXwzA6KAUV2jZ3yeeg9tWtvO6yf5k/J3YfbR245b4DGbLRsmSWtb4tBYQMEWHL8 b/TmQ0VcajeKXYkbpOYp1JCbXi4mtPhlLX9BpXrAG4z05lwirmoj+gDjpZJPzjQGq/5N UraM+gJjIBnXkMUpPLS7eBuan8YQduYvLCVqGE69RlxWlVsRSyRdKHXWqjYZNaLsNsot DnNp5akT2Ln/jJ9HyLai6efpMtbPU5L7SK/j59wbq4huZ4tLBjbkH2lWiVT3IEC60gyH SqdCoAWFgA9sZrHoDRpzpfHlBD/cBTxhvFJ5U3Mg1J4gFxo3x40bskfx/nnKsLFAp5JX bEbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dmarc-filter:delivered-to; bh=OtXjyDN/mKefMLguOo8JMTM++rY7CGg5K8RwtuEthmk=; fh=CHbn33ss3MXGFqXGZpS89+qfBQv2oFkoJCJQmHw6RIo=; b=iaTGMdwHHeRcuejd+tqSxHvl2+9/BSoNDYpUwJAcYPy4aeYSWWOnBClWSq0q4vqUL+ BzC4+++kDciEXkFZWxReY7CX2+v2g2TDfW9kT8dFqvdpg7XzweyqxZs9D9S4teL4M34A qGCMVclo6wWn7NmsSmv9949d9tdkatCBoe3w0ZUDkuj3aUQPS0VCQgLbaKrTchgx/I+j 9Y1MFdbitFozHkrVl99XrYoggBFxcgw5QmGMx9MVpFfBIhyuP6z+lTks44SYCcgzo4F9 NBiOQbVBcuPNrIIgw2n1Nn21XEyM78Q/PP8NXuwzwvRl5g64FQcUsgLWd7XWi8hPMKOJ QSmQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id m5-20020a50ef05000000b0051e10955abdsi485630eds.214.2023.07.05.23.17.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Jul 2023 23:17:13 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 377743858D20 for ; Thu, 6 Jul 2023 06:17:05 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgsg2.qq.com (smtpbgsg2.qq.com [54.254.200.128]) by sourceware.org (Postfix) with ESMTPS id 779EA3858D20 for ; Thu, 6 Jul 2023 06:16:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 779EA3858D20 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp81t1688624188t0wgc1qa Received: from rios-cad5.localdomain ( [58.60.1.11]) by bizesmtp.qq.com (ESMTP) with id ; Thu, 06 Jul 2023 14:16:27 +0800 (CST) X-QQ-SSF: 01400000000000G0T000000A0000000 X-QQ-FEAT: C46Rb8GPIEdgU90fHLK9P5GvpEAsimAlGo05zZ/qMN53yyyzx9ZVt/Q9jqkNq tgKy3YZjjTN/t0sJjN5QInAv58eH13r8J6LMYXmPJUULiFu0rOMdr0Xx1dly6MZwBIlfH/v A7d3/2Pz195oxNbLdiEpZCytv0Zqjd83ytLmfvs23IXl5cgNp/oXiHzJRBD1TOkNGtXo1wy 0Nj7VzB8C584+3DnOpjtV7TuGD1VyKUmBcpbnSNQ6cU59btXKZl2jDUf6yshEI8GMk4LghV X/FHGXze6fxu0qNWZQm8GCfEN8kbmIpiYzQgkcG5eG51sYkFO9Kw2MB3DswbqmoVYAek3e6 BjpAhrhekEqzW8i3pEWvfTBmvtzUHgfkQuDu24CExfDn3dwB4qWIng6MqFFNQ== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 15439108886426024843 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: richard.sandiford@arm.com, rguenther@suse.de, Ju-Zhe Zhong Subject: [PATCH] VECT: Fix ICE of variable stride on strieded load/store with SELECT_VL loop control. Date: Thu, 6 Jul 2023 14:16:26 +0800 Message-Id: <20230706061626.3420739-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-12.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770650844162852916?= X-GMAIL-MSGID: =?utf-8?q?1770650844162852916?= From: Ju-Zhe Zhong Hi, Richi. Sorry for making mistake on LEN_MASK_GATHER_LOAD/LEN_MASK_SCATTER_STORE with SELECT_VL loop control. Consider this following case: #define TEST_LOOP(DATA_TYPE, BITS) \ void __attribute__ ((noinline, noclone)) \ f_##DATA_TYPE##_##BITS (DATA_TYPE *restrict dest, DATA_TYPE *restrict src, \ INDEX##BITS stride, INDEX##BITS n) \ { \ for (INDEX##BITS i = 0; i < n; ++i) \ dest[i] += src[i * stride]; \ } When "stride" is a constant, current flow works fine. However, when "stride" is a variable. It causes an ICE: # vectp_src.67_85 = PHI ... _96 = .SELECT_VL (ivtmp_94, 4); ... ivtmp_78 = ((sizetype) _39 * (sizetype) _96) * 4; vect__11.69_87 = .LEN_MASK_GATHER_LOAD (vectp_src.67_85, _84, 4, { 0, 0, 0, 0 }, { -1, -1, -1, -1 }, _96, 0); ... vectp_src.67_86 = vectp_src.67_85 + ivtmp_78; Becase the IR: ivtmp_78 = ((sizetype) _39 * (sizetype) _96) * 4; Instead, I split the IR into: step_stride = _39 step = step_stride * 4 ivtmp_78 = step * _96 I don't think this patch's code is elegant enough, could you help me refine these codes? Thanks. gcc/ChangeLog: * tree-vect-stmts.cc (vect_get_strided_load_store_ops): Fix ICE. --- gcc/tree-vect-stmts.cc | 38 +++++++++++++++++++++++++++++++++----- 1 file changed, 33 insertions(+), 5 deletions(-) diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc index c10a4be60eb..12d1b0f1ac0 100644 --- a/gcc/tree-vect-stmts.cc +++ b/gcc/tree-vect-stmts.cc @@ -3172,12 +3172,40 @@ vect_get_strided_load_store_ops (stmt_vec_info stmt_info, vectp_a.9_26 = vectp_a.9_7 + ivtmp_8; */ tree loop_len = vect_get_loop_len (loop_vinfo, gsi, loop_lens, 1, vectype, 0, 0); - tree tmp - = fold_build2 (MULT_EXPR, sizetype, - fold_convert (sizetype, unshare_expr (DR_STEP (dr))), - loop_len); + tree tmp; + gassign *assign; + + if (TREE_CODE (DR_STEP (dr)) == INTEGER_CST) + tmp = fold_build2 (MULT_EXPR, sizetype, + fold_convert (sizetype, unshare_expr (DR_STEP (dr))), + loop_len); + else + { + /* If DR_STEP = (unsigned int) _37 * 4; + Extract _37 and 4, explicit MULT_EXPR. */ + + /* 1. step_stride = (unsigned int) _37. */ + tree step_stride = make_ssa_name (create_tmp_var (sizetype)); + assign = gimple_build_assign ( + step_stride, TREE_OPERAND (TREE_OPERAND (DR_STEP (dr), 0), 0)); + gsi_insert_before (gsi, assign, GSI_SAME_STMT); + + /* 2. step = step_stride * 4. */ + tree step_align = TREE_OPERAND (TREE_OPERAND (DR_STEP (dr), 0), 1); + tree step = make_ssa_name (create_tmp_var (sizetype)); + assign + = gimple_build_assign (step, fold_build2 (MULT_EXPR, sizetype, + step_stride, step_align)); + gsi_insert_before (gsi, assign, GSI_SAME_STMT); + + /* 3. tmp = step * loop_len. */ + tmp = make_ssa_name (create_tmp_var (sizetype)); + assign = gimple_build_assign (tmp, fold_build2 (MULT_EXPR, sizetype, + step, loop_len)); + gsi_insert_before (gsi, assign, GSI_SAME_STMT); + } tree bump = make_temp_ssa_name (sizetype, NULL, "ivtmp"); - gassign *assign = gimple_build_assign (bump, tmp); + assign = gimple_build_assign (bump, tmp); gsi_insert_before (gsi, assign, GSI_SAME_STMT); *dataref_bump = bump; }