From patchwork Tue Oct 17 11:34:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 154106 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4071476vqb; Tue, 17 Oct 2023 04:43:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF4kRonCyxEZsbnEtWowHziAdrT+72pD3hIKgJ/N2c/ltYKhWKZdhqk+AGJP7Rj6vEUePGS X-Received: by 2002:a05:622a:506:b0:410:9111:4a0b with SMTP id l6-20020a05622a050600b0041091114a0bmr2272075qtx.13.1697542995856; Tue, 17 Oct 2023 04:43:15 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1697542995; cv=pass; d=google.com; s=arc-20160816; b=kRBOANyC4TuUyLBqUAihSPzIIblj0oieq9zSQHiejpxirWDYvoj46meuUcDopnXpMK GbM8Cmz+Ujv4rkwpFs6M01etdBj2Ncs1KpQfWym6bbRC4s9ZNEHoKu1VxnFCV3TCRLhB HZMjNYZg8mKHnkR6KMOTKjDSE0cqZGZPvRvXxq3nblG5qWuwWgKKR7ZEwEkG1FbcJ9FI wUHqq4DLiAlFa4ermxSB8/ztrM+RMY9ZT2Nj0+1oYm6LFNFv8uhb4OE/2xi2xAzZijgi Co3vTSrw0U2BzRekvrj+5OSRRxqwfolbkZcaM6D2wUUHh3VzvPd2KM87tY2jC/fBVMYc t8Mg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-filter:dmarc-filter :delivered-to; bh=Oz3gH2VftRVgsI+hAcL9Xeo1vB8b+EUt8fn+NjmFGTo=; fh=x/Q0OlwHuvCZ3FpkiZPiUSvevOYVxUAi4aNnf76mUPQ=; b=Sl2qNVGL/ocDPVziqx3nGPQ5lxOUKFd5OR6YG/B3nBgxtKUt6P+6fhJnjw8Byel1h6 2aM8+deUl2EaDTI/FN/iOuBRXOIPAzAl7QQtdud0gRqjwV9mYUcjrZsE+qd9oB4wmnyw EU2pCYBSQXhf4YzJ7z7NrpEhwPi2XtK3DahQII14Mf64TCndg6ybY3S5K1PfM+RwpytU NQvE/Mi9pHa5M6i1WjMazdJj0Vaz5Ctm1LKvlpFAUjIhO5rkbWk3yOpMCvPMxPsly6Af p0NODwdfDtDHXkgONfIlMSjPlAVjnWtUyy4fGa/C4ox4CvUFgoUUdwTPzqYx6uGQQYwS 2t1w== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id a8-20020a05622a064800b004181028650esi943304qtb.606.2023.10.17.04.43.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 04:43:15 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 15C973895FCC for ; Tue, 17 Oct 2023 11:42:03 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbguseast3.qq.com (smtpbguseast3.qq.com [54.243.244.52]) by sourceware.org (Postfix) with ESMTPS id 2848B385734B for ; Tue, 17 Oct 2023 11:35:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2848B385734B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2848B385734B Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.243.244.52 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697542545; cv=none; b=Y/t1iNpJMCfrcD32Z1JzLVtLPDlOYp3g0pD7RpheTnd7YESKYDvqjnMvANJiOAXF+A7PhQaWR8YwtFAU6pdM3BAnwULOsDf3WIPexvxi2B219wUa/QEYci6SvBr4oiJkDOZ/DXcN+L1agOam0W2AaFhruGmwtQkmcKvCzKkg6wE= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1697542545; c=relaxed/simple; bh=KQOeeobWPhxanl3Otbknv/wv57MnIDmfcgTjaPkY4Ys=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=WQ3CWWbE9r+tRjdZ9qJrkUpSe+VYIXBnKext50NgXczYt34AN/dAJ1k8M0g4/b84eA+tOMiaOfJkvPGsXZCmxcCtynKdFq9vsCoK0YWi5wo/di4FPPhL08LSdArvid4dusKAj3nQOdRP6QL37XUkffz5eAXYxpmDidT6apJswqk= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp62t1697542535tbu8x43s Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Tue, 17 Oct 2023 19:35:33 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: lkL5M32tl2BQO7lMla6Pn7RbmTRjN/AZadQg4CHTR1rdrPYSPRkrU0vw4f3mo YEj9vje4c084SroERX6XwdLmbFJwEitQBPcVZEYIMoLYDMooAe5xgq587sM+w70QKtjgm2K jLgzm42mXdzsOqhNxGMdeErnY/UtbKq3dhB7ZL7P4Va7Suciy8g9yU2147OsLjX57uqOy5r bwAhEQSA6wioEvgMrMr7aRhXmxWalqC3Gmu1fz5IOfpKAyycrUVqIkqxTF5/CIJVwhQWbbG 6keTxSnPR/GtaYZA1fiwrSYEjB9v8iwPs0gVOuScj/E2l7zVrkk8aZAnTpJODuaPFoVbNiI coo0uZLLVqnaoxov3SbvTShqHCry7NV48zlLyFmpUDo4rl8TLHlH43/vqMJI6/TxgecBD27 FcLtx0C88sc= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 15639861675832196300 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: juzhe.zhong@rivai.ai, kito.cheng@gmail.com, rdapp.gcc@gmail.com, palmer@rivosinc.com, jeffreyalaw@gmail.com, lehua.ding@rivai.ai Subject: [PATCH V2 08/14] RISC-V: P8: Unified insert and delete of vsetvl insn into Phase 4 Date: Tue, 17 Oct 2023 19:34:54 +0800 Message-Id: <20231017113500.1160997-9-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20231017113500.1160997-1-lehua.ding@rivai.ai> References: <20231017113500.1160997-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_NUMSUBJECT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780002844432204044 X-GMAIL-MSGID: 1780002844432204044 This sub-patch move the modification of rtl codes from pass_vsetvl into pre_vsetvl class. gcc/ChangeLog: * config/riscv/riscv-vsetvl.cc (pre_vsetvl::emit_vsetvl): New. (pass_vsetvl::can_refine_vsetvl_p): Removed. (pass_vsetvl::refine_vsetvls): Removed. (pass_vsetvl::cleanup_vsetvls): Removed. (pass_vsetvl::commit_vsetvls): Removed. --- gcc/config/riscv/riscv-vsetvl.cc | 389 +++++++++++-------------------- 1 file changed, 134 insertions(+), 255 deletions(-) -- 2.36.3 diff --git a/gcc/config/riscv/riscv-vsetvl.cc b/gcc/config/riscv/riscv-vsetvl.cc index a112895a283..5d84d290e9e 100644 --- a/gcc/config/riscv/riscv-vsetvl.cc +++ b/gcc/config/riscv/riscv-vsetvl.cc @@ -3658,6 +3658,140 @@ pre_vsetvl::pre_global_vsetvl_info () } } +void +pre_vsetvl::emit_vsetvl () +{ + bool need_commit = false; + + for (const bb_info *bb : crtl->ssa->bbs ()) + { + for (const auto &curr_info : get_block_info (bb).infos) + { + insn_info *insn = curr_info.get_insn (); + if (curr_info.ignore_p ()) + { + if (vsetvl_insn_p (insn->rtl ())) + eliminate_insn (insn->rtl ()); + continue; + } + else if (curr_info.valid_p ()) + { + if (vsetvl_insn_p (insn->rtl ())) + { + const vsetvl_info temp = vsetvl_info (insn); + if (!(curr_info == temp)) + { + if (dump_file) + { + fprintf (dump_file, "\n Change vsetvl info from: "); + temp.dump (dump_file, " "); + fprintf (dump_file, " to: "); + curr_info.dump (dump_file, " "); + } + change_vsetvl_insn (insn, curr_info); + } + } + else + { + if (dump_file) + { + fprintf (dump_file, + "\n Insert vsetvl info before insn %d: ", + insn->uid ()); + curr_info.dump (dump_file, " "); + } + insert_vsetvl (EMIT_BEFORE, insn->rtl (), curr_info); + } + } + } + } + + for (const vsetvl_info &item : delete_list) + { + gcc_assert (vsetvl_insn_p (item.get_insn ()->rtl ())); + eliminate_insn (item.get_insn ()->rtl ()); + } + + /* Insert vsetvl as LCM suggest. */ + for (int ed = 0; ed < NUM_EDGES (edges); ed++) + { + edge eg = INDEX_EDGE (edges, ed); + sbitmap i = insert[ed]; + if (bitmap_count_bits (i) < 1) + continue; + + if (bitmap_count_bits (i) > 1) + /* For code with infinite loop (e.g. pr61634.c), The data flow is + completely wrong. */ + continue; + + gcc_assert (bitmap_count_bits (i) == 1); + unsigned expr_index = bitmap_first_set_bit (i); + const vsetvl_info &info = *exprs[expr_index]; + gcc_assert (info.valid_p ()); + if (dump_file) + { + fprintf (dump_file, + "\n Insert vsetvl info at edge(bb %u -> bb %u): ", + eg->src->index, eg->dest->index); + info.dump (dump_file, " "); + } + rtl_profile_for_edge (eg); + start_sequence (); + + insn_info *insn = info.get_insn (); + insert_vsetvl (EMIT_DIRECT, insn->rtl (), info); + rtx_insn *rinsn = get_insns (); + end_sequence (); + default_rtl_profile (); + + /* We should not get an abnormal edge here. */ + gcc_assert (!(eg->flags & EDGE_ABNORMAL)); + need_commit = true; + insert_insn_on_edge (rinsn, eg); + } + + /* Insert vsetvl info that was not deleted after lift up. */ + for (const bb_info *bb : crtl->ssa->bbs ()) + { + const vsetvl_block_info &block_info = get_block_info (bb); + if (!block_info.has_info ()) + continue; + + const vsetvl_info &footer_info = block_info.get_footer_info (); + insn_info *insn = footer_info.get_insn (); + + if (footer_info.ignore_p ()) + continue; + + edge eg; + edge_iterator eg_iterator; + FOR_EACH_EDGE (eg, eg_iterator, bb->cfg_bb ()->succs) + { + gcc_assert (!(eg->flags & EDGE_ABNORMAL)); + if (dump_file) + { + fprintf ( + dump_file, + "\n Insert missed vsetvl info at edge(bb %u -> bb %u): ", + eg->src->index, eg->dest->index); + footer_info.dump (dump_file, " "); + } + start_sequence (); + insert_vsetvl (EMIT_DIRECT, insn->rtl (), footer_info); + rtx_insn *rinsn = get_insns (); + end_sequence (); + default_rtl_profile (); + insert_insn_on_edge (rinsn, eg); + need_commit = true; + } + } + + if (need_commit) + commit_edge_insertions (); +} + + const pass_data pass_data_vsetvl = { RTL_PASS, /* type */ "vsetvl", /* name */ @@ -3790,261 +3924,6 @@ make_pass_vsetvl (gcc::context *ctxt) return new pass_vsetvl (ctxt); } - -/* Return true if VSETVL in the block can be refined as vsetvl zero,zero. */ -bool -pass_vsetvl::can_refine_vsetvl_p (const basic_block cfg_bb, - const vector_insn_info &info) const -{ - if (!m_vector_manager->all_same_ratio_p ( - m_vector_manager->vector_avin[cfg_bb->index])) - return false; - - if (!m_vector_manager->all_same_avl_p ( - cfg_bb, m_vector_manager->vector_avin[cfg_bb->index])) - return false; - - size_t expr_id - = bitmap_first_set_bit (m_vector_manager->vector_avin[cfg_bb->index]); - if (!m_vector_manager->vector_exprs[expr_id]->same_vlmax_p (info)) - return false; - if (!m_vector_manager->vector_exprs[expr_id]->compatible_avl_p (info)) - return false; - - edge e; - edge_iterator ei; - bool all_valid_p = true; - FOR_EACH_EDGE (e, ei, cfg_bb->preds) - { - if (bitmap_empty_p (m_vector_manager->vector_avout[e->src->index])) - { - all_valid_p = false; - break; - } - } - - if (!all_valid_p) - return false; - return true; -} - -/* Optimize athe case like this: - - bb 0: - vsetvl 0 a5,zero,e8,mf8 - insn 0 (demand SEW + LMUL) - bb 1: - vsetvl 1 a5,zero,e16,mf4 - insn 1 (demand SEW + LMUL) - - In this case, we should be able to refine - vsetvl 1 into vsetvl zero, zero according AVIN. */ -void -pass_vsetvl::refine_vsetvls (void) const -{ - basic_block cfg_bb; - FOR_EACH_BB_FN (cfg_bb, cfun) - { - auto info = get_block_info (cfg_bb).local_dem; - insn_info *insn = info.get_insn (); - if (!info.valid_p ()) - continue; - - rtx_insn *rinsn = insn->rtl (); - if (!can_refine_vsetvl_p (cfg_bb, info)) - continue; - - /* We can't refine user vsetvl into vsetvl zero,zero since the dest - will be used by the following instructions. */ - if (vector_config_insn_p (rinsn)) - { - m_vector_manager->to_refine_vsetvls.add (rinsn); - continue; - } - - /* If all incoming edges to a block have a vector state that is compatbile - with the block. In such a case we need not emit a vsetvl in the current - block. */ - - gcc_assert (has_vtype_op (insn->rtl ())); - rinsn = PREV_INSN (insn->rtl ()); - gcc_assert (vector_config_insn_p (PREV_INSN (insn->rtl ()))); - if (m_vector_manager->all_avail_in_compatible_p (cfg_bb)) - { - size_t id = m_vector_manager->get_expr_id (info); - if (bitmap_bit_p (m_vector_manager->vector_del[cfg_bb->index], id)) - continue; - eliminate_insn (rinsn); - } - else - { - rtx new_pat - = gen_vsetvl_pat (VSETVL_VTYPE_CHANGE_ONLY, info, NULL_RTX); - change_insn (rinsn, new_pat); - } - } -} - -void -pass_vsetvl::cleanup_vsetvls () -{ - basic_block cfg_bb; - FOR_EACH_BB_FN (cfg_bb, cfun) - { - auto &info = get_block_info (cfg_bb).reaching_out; - gcc_assert (m_vector_manager->expr_set_num ( - m_vector_manager->vector_del[cfg_bb->index]) - <= 1); - for (size_t i = 0; i < m_vector_manager->vector_exprs.length (); i++) - { - if (bitmap_bit_p (m_vector_manager->vector_del[cfg_bb->index], i)) - { - if (info.dirty_p ()) - info.set_unknown (); - else - { - const auto dem = get_block_info (cfg_bb).local_dem; - gcc_assert (dem == *m_vector_manager->vector_exprs[i]); - insn_info *insn = dem.get_insn (); - gcc_assert (insn && insn->rtl ()); - rtx_insn *rinsn; - /* We can't eliminate user vsetvl since the dest will be used - * by the following instructions. */ - if (vector_config_insn_p (insn->rtl ())) - { - m_vector_manager->to_delete_vsetvls.add (insn->rtl ()); - continue; - } - - gcc_assert (has_vtype_op (insn->rtl ())); - rinsn = PREV_INSN (insn->rtl ()); - gcc_assert (vector_config_insn_p (PREV_INSN (insn->rtl ()))); - eliminate_insn (rinsn); - } - } - } - } -} - -bool -pass_vsetvl::commit_vsetvls (void) -{ - bool need_commit = false; - - for (int ed = 0; ed < NUM_EDGES (m_vector_manager->vector_edge_list); ed++) - { - for (size_t i = 0; i < m_vector_manager->vector_exprs.length (); i++) - { - edge eg = INDEX_EDGE (m_vector_manager->vector_edge_list, ed); - if (bitmap_bit_p (m_vector_manager->vector_insert[ed], i)) - { - const vector_insn_info *require - = m_vector_manager->vector_exprs[i]; - gcc_assert (require->valid_or_dirty_p ()); - rtl_profile_for_edge (eg); - start_sequence (); - - insn_info *insn = require->get_insn (); - vector_insn_info prev_info = vector_insn_info (); - sbitmap bitdata = m_vector_manager->vector_avout[eg->src->index]; - if (m_vector_manager->all_same_ratio_p (bitdata) - && m_vector_manager->all_same_avl_p (eg->dest, bitdata)) - { - size_t first = bitmap_first_set_bit (bitdata); - prev_info = *m_vector_manager->vector_exprs[first]; - } - - insert_vsetvl (EMIT_DIRECT, insn->rtl (), *require, prev_info); - rtx_insn *rinsn = get_insns (); - end_sequence (); - default_rtl_profile (); - - /* We should not get an abnormal edge here. */ - gcc_assert (!(eg->flags & EDGE_ABNORMAL)); - need_commit = true; - insert_insn_on_edge (rinsn, eg); - - if (dump_file) - { - fprintf (dump_file, - "\nInsert vsetvl insn %d at edge %d from to " - ":\n", - INSN_UID (rinsn), ed, eg->src->index, - eg->dest->index); - print_rtl_single (dump_file, rinsn); - } - } - } - } - - for (const bb_info *bb : crtl->ssa->bbs ()) - { - basic_block cfg_bb = bb->cfg_bb (); - const auto reaching_out = get_block_info (cfg_bb).reaching_out; - if (!reaching_out.dirty_p ()) - continue; - - rtx new_pat; - if (!reaching_out.demand_p (DEMAND_AVL)) - { - vl_vtype_info new_info = reaching_out; - new_info.set_avl_info (avl_info (const0_rtx, nullptr)); - new_pat = gen_vsetvl_pat (VSETVL_DISCARD_RESULT, new_info, NULL_RTX); - } - else if (can_refine_vsetvl_p (cfg_bb, reaching_out)) - new_pat - = gen_vsetvl_pat (VSETVL_VTYPE_CHANGE_ONLY, reaching_out, NULL_RTX); - else if (vlmax_avl_p (reaching_out.get_avl ())) - { - rtx vl = reaching_out.get_avl_or_vl_reg (); - new_pat = gen_vsetvl_pat (VSETVL_NORMAL, reaching_out, vl); - } - else - new_pat - = gen_vsetvl_pat (VSETVL_DISCARD_RESULT, reaching_out, NULL_RTX); - - edge eg; - edge_iterator eg_iterator; - FOR_EACH_EDGE (eg, eg_iterator, cfg_bb->succs) - { - /* We should not get an abnormal edge here. */ - gcc_assert (!(eg->flags & EDGE_ABNORMAL)); - /* We failed to optimize this case in Phase 3 (earliest fusion): - - bb 2: vsetvl a5, a3 ... - goto bb 4 - bb 3: vsetvl a5, a2 ... - goto bb 4 - bb 4: vsetvli zero, a5 ---> Redundant, should be elided. - - Since "a5" value can come from either bb 2 or bb 3, we can't make - it optimized in Phase 3 which will make phase 3 so complicated. - Now, we do post optimization here to elide the redundant VSETVL - insn in bb4. */ - if (m_vector_manager->vsetvl_dominated_by_all_preds_p (cfg_bb, - reaching_out)) - continue; - - start_sequence (); - emit_insn (copy_rtx (new_pat)); - rtx_insn *rinsn = get_insns (); - end_sequence (); - - insert_insn_on_edge (rinsn, eg); - need_commit = true; - if (dump_file) - { - fprintf (dump_file, - "\nInsert vsetvl insn %d from to :\n", - INSN_UID (rinsn), cfg_bb->index, eg->dest->index); - print_rtl_single (dump_file, rinsn); - } - } - } - - return need_commit; -} - /* Some instruction can not be accessed in RTL_SSA when we don't re-init the new RTL_SSA framework but it is definetely at the END of the block.