From patchwork Sun Nov 12 12:08:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 164248 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b909:0:b0:403:3b70:6f57 with SMTP id t9csp659908vqg; Sun, 12 Nov 2023 04:10:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IHmRvfkcxA+6a4EJs+7O7NhWv360Zh4I+txuIVxTYSLMDetz86j4qMxyyw6aSL6lDJriNSD X-Received: by 2002:a25:dc52:0:b0:daf:11cd:d54a with SMTP id y79-20020a25dc52000000b00daf11cdd54amr3116773ybe.36.1699791005096; Sun, 12 Nov 2023 04:10:05 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699791005; cv=pass; d=google.com; s=arc-20160816; b=EwalEnxOm3xgx3AtPblHjzcHnd7e4B6tWtpUjKN7JZgGEVsS3OuLWZQrPhl9WEqMgN APB2pru5Zw6ojwLAhlZF3rhbj6JNMvJdv370n92xn9YL4Do1i5nVISAggPV/2TTo3NX/ K9EhKQwWl6g1u2w1WBDYZZxJGHs4UGV4Zx05izySZs9ZFBwfhB5PbCFErx1JKxQ8/DkB gR3b5GVymRGubrEIjRpV07/EpB2IEOwZRVA5MwoUSLllEVOV/9YaBl3QY+VQICpm5MJn G3CTxMJ/l34xlKvrvRe5UhCk4E2Lby/dfeBk0y67ssTFGw8+4RSrm5Gm3Lnt1Ug0WIBM OCxA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-filter:dmarc-filter :delivered-to; bh=qBx8EBX5OpTeFT1zJnLqMSQBpjvnVFPyCoqj+m1YLM0=; fh=9Ok8HNl3eD0lUFF4nhUPZJmQfyAUbHnIPw/rSVNIfK0=; b=F76Y7akJUJyypbvEfsurEzd9OwjRLcxEd6CJBuVhbu7jpF0a8WeILAnMRaq7ehyeUo CIrqLD+lHw4twdmUadWTlRM+cCstC/oLkgenDuh0VWfN4paPSVO+gCrgwDCjo40uFI8o n3gUbDZfD2aKDx8KLyeZpAYSGsoAOXuAjQESoNkd+0pq6tXYemHOja7FnFZHGHi5A/H0 8828Yh1ovV+KDjgCnzzB4jvX6+CLEnWaJ5AYRP2iOMD60UsztObAn08AePOzf+x3E/54 dXA7mr5UCdolkNIOSjhpvW2wQIJRI4Fs2IM/KHOHjY/7H4DmSLhVbbl6iLRMAsD1Bz/g Xqjw== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id q10-20020ad4402a000000b0065e0125a6besi2787764qvp.410.2023.11.12.04.10.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Nov 2023 04:10:05 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3D537385C8B0 for ; Sun, 12 Nov 2023 12:09:42 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgbr1.qq.com (smtpbgbr1.qq.com [54.207.19.206]) by sourceware.org (Postfix) with ESMTPS id AB1323857709 for ; Sun, 12 Nov 2023 12:08:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AB1323857709 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AB1323857709 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.207.19.206 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699790932; cv=none; b=cAyIFPEXE4zikNE6CCXv8NAwgBKI+GOhOOYou59vxGMQOBITdqOcgW5ZNewd/ZLScYHLCLNkP9vfU5W+oJxy7F2FDCf3sde+hp/s+Bwnk70OWkYoB6JYlQCfv1itM+h8OGCWzPwVvnpsNFAhV7mIXcxb+LK1p1FMrcdvnfq9sv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699790932; c=relaxed/simple; bh=lqtsuFbvEONTwu99teuaxg+5qUkd/ZhJ9j1kQB7JOA0=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=mAj8mENcYE74foNehZvUUGC/8iMztmImQH2i9PhEp5RJlcgaJtSNlXXuT4lYX1mMK17MOcxNo4HEqPE9fiDvfu0mWgJBGyt/oLoW+CufhSyQBLTM7i3egwPniBXqy5HtK43wWaZRrBixcTCvrQaSjPvAalvHGuZKyQG0qKGf+ro= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp67t1699790919tsmkb273 Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Sun, 12 Nov 2023 20:08:39 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: rZJGTgY0+YMtKCLiOlXOQnFeDhyDNA6qp8+DteOx6cQMhjFSmN5iQagK88lj9 MrqwMTQbznJmQMy4ffWiuxWp7utE79Ecg2ImUScRIJWZp3ES9C6v5XGL1SGUH2xpBjp0qSm Wu3vddiMjJzGSCCwK663Z2hpnxxG4yfwW9BjNfk1AOiX0qrKRt7LubAG9rl4J629bn6CLfF DTWA5bYT2ngAzcR4ZArtzJ4j8eJ6nUF21B1rGE6jYW2O89ebuEGSeM8B/v+KkVH1IBszPrc M91D00lq6fttlb/v/zirqtHxF9doXG0jL1xVD3G+ElkISDsAAtB3kCHCgtINJdvlcXjIZy4 J7DjemfN86eQppYSQSHugkJjhAwv1dXy/hnlfkxeAflrTZiNQmImqV0w3D5iVSR58pl8hk2 HD68wzpBq0w= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 18255706897759880586 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH V3 6/7] lra: Switch to live_subreg data flow Date: Sun, 12 Nov 2023 20:08:16 +0800 Message-Id: <20231112120817.2635864-7-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20231112120817.2635864-1-lehua.ding@rivai.ai> References: <20231112120817.2635864-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782360052588988562 X-GMAIL-MSGID: 1782360052588988562 This patch switches the live_reg data in lra to live_subreg data, and the situation will be more complicated than in ira because this part of the data is modified in lra also and the live_subreg data will be recalculated. gcc/ChangeLog: * lra-coalesce.cc (update_live_info): Adjust to new live subreg data. (lra_coalesce): Ditto. * lra-constraints.cc (update_ebb_live_info): Ditto. (get_live_on_other_edges): Ditto. (inherit_in_ebb): Ditto. (lra_inheritance): Ditto. (fix_bb_live_info): Ditto. (remove_inheritance_pseudos): Ditto. * lra-int.h (GCC_LRA_INT_H): Ditto. * lra-lives.cc (class bb_data_pseudos): Ditto. (make_hard_regno_live): Ditto. (make_hard_regno_dead): Ditto. (mark_regno_live): Ditto. (mark_regno_dead): Ditto. (live_trans_fun): Ditto. (live_con_fun_0): Ditto. (live_con_fun_n): Ditto. (initiate_live_solver): Ditto. (finish_live_solver): Ditto. (process_bb_lives): Ditto. (lra_create_live_ranges_1): Ditto. * lra-remat.cc (dump_candidates_and_remat_bb_data): Ditto. (calculate_livein_cands): Ditto. (do_remat): Ditto. * lra-spills.cc (spill_pseudos): Ditto. --- gcc/lra-coalesce.cc | 20 ++- gcc/lra-constraints.cc | 93 +++++++++--- gcc/lra-int.h | 2 + gcc/lra-lives.cc | 328 ++++++++++++++++++++++++++++++++--------- gcc/lra-remat.cc | 13 +- gcc/lra-spills.cc | 22 ++- 6 files changed, 374 insertions(+), 104 deletions(-) diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc index 04a5bbd714b..abfc54f1cc2 100644 --- a/gcc/lra-coalesce.cc +++ b/gcc/lra-coalesce.cc @@ -188,19 +188,25 @@ static bitmap_head used_pseudos_bitmap; /* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info bitmap). */ static void -update_live_info (bitmap lr_bitmap) +update_live_info (bitmap all, bitmap full, bitmap partial) { unsigned int j; bitmap_iterator bi; bitmap_clear (&used_pseudos_bitmap); - EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap, + EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all, FIRST_PSEUDO_REGISTER, j, bi) bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]); if (! bitmap_empty_p (&used_pseudos_bitmap)) { - bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap); - bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap); + bitmap_and_compl_into (all, &coalesced_pseudos_bitmap); + bitmap_ior_into (all, &used_pseudos_bitmap); + + bitmap_and_compl_into (full, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial); + + bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full); } } @@ -303,8 +309,10 @@ lra_coalesce (void) bitmap_initialize (&used_pseudos_bitmap, ®_obstack); FOR_EACH_BB_FN (bb, cfun) { - update_live_info (df_get_live_in (bb)); - update_live_info (df_get_live_out (bb)); + update_live_info (DF_LIVE_SUBREG_IN (bb), DF_LIVE_SUBREG_FULL_IN (bb), + DF_LIVE_SUBREG_PARTIAL_IN (bb)); + update_live_info (DF_LIVE_SUBREG_OUT (bb), DF_LIVE_SUBREG_FULL_OUT (bb), + DF_LIVE_SUBREG_PARTIAL_OUT (bb)); FOR_BB_INSNS_SAFE (bb, insn, next) if (INSN_P (insn) && bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn))) diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc index 0607c8be7cb..c3ad846b97b 100644 --- a/gcc/lra-constraints.cc +++ b/gcc/lra-constraints.cc @@ -6571,34 +6571,75 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail) { if (prev_bb != NULL) { - /* Update df_get_live_in (prev_bb): */ + /* Update subreg live (prev_bb): */ + bitmap subreg_all_in = DF_LIVE_SUBREG_IN (prev_bb); + bitmap subreg_full_in = DF_LIVE_SUBREG_FULL_IN (prev_bb); + bitmap subreg_partial_in = DF_LIVE_SUBREG_PARTIAL_IN (prev_bb); + subregs_live *range_in = DF_LIVE_SUBREG_RANGE_IN (prev_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) if (bitmap_bit_p (&live_regs, j)) - bitmap_set_bit (df_get_live_in (prev_bb), j); - else - bitmap_clear_bit (df_get_live_in (prev_bb), j); + { + bitmap_set_bit (subreg_all_in, j); + bitmap_set_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_live (j); + } + } + else if (bitmap_bit_p (subreg_all_in, j)) + { + bitmap_clear_bit (subreg_all_in, j); + bitmap_clear_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_live (j); + } + } } + bitmap subreg_all_out = DF_LIVE_SUBREG_OUT (curr_bb); if (curr_bb != last_bb) { - /* Update df_get_live_out (curr_bb): */ + /* Update subreg live (curr_bb): */ + bitmap subreg_all_out = DF_LIVE_SUBREG_OUT (curr_bb); + bitmap subreg_full_out = DF_LIVE_SUBREG_FULL_OUT (curr_bb); + bitmap subreg_partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (curr_bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (curr_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) { live_p = bitmap_bit_p (&live_regs, j); if (! live_p) FOR_EACH_EDGE (e, ei, curr_bb->succs) - if (bitmap_bit_p (df_get_live_in (e->dest), j)) + if (bitmap_bit_p (DF_LIVE_SUBREG_IN (e->dest), j)) { live_p = true; break; } if (live_p) - bitmap_set_bit (df_get_live_out (curr_bb), j); - else - bitmap_clear_bit (df_get_live_out (curr_bb), j); + { + bitmap_set_bit (subreg_all_out, j); + bitmap_set_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_live (j); + } + } + else if (bitmap_bit_p (subreg_all_out, j)) + { + bitmap_clear_bit (subreg_all_out, j); + bitmap_clear_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_live (j); + } + } } } prev_bb = curr_bb; - bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb)); + bitmap_and (&live_regs, &check_only_regs, subreg_all_out); } if (! NONDEBUG_INSN_P (curr_insn)) continue; @@ -6715,7 +6756,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res) bitmap_clear (res); FOR_EACH_EDGE (e, ei, from->succs) if (e->dest != to) - bitmap_ior_into (res, df_get_live_in (e->dest)); + bitmap_ior_into (res, DF_LIVE_SUBREG_IN (e->dest)); last = get_last_insertion_point (from); if (! JUMP_P (last)) return; @@ -6787,7 +6828,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We are at the end of BB. Add qualified living pseudos for potential splitting. */ - to_process = df_get_live_out (curr_bb); + to_process = DF_LIVE_SUBREG_OUT (curr_bb); if (last_processed_bb != NULL) { /* We are somewhere in the middle of EBB. */ @@ -7159,7 +7200,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We reached the beginning of the current block -- do rest of spliting in the current BB. */ - to_process = df_get_live_in (curr_bb); + to_process = DF_LIVE_SUBREG_IN (curr_bb); if (BLOCK_FOR_INSN (head) != curr_bb) { /* We are somewhere in the middle of EBB. */ @@ -7236,7 +7277,7 @@ lra_inheritance (void) fprintf (lra_dump_file, "EBB"); /* Form a EBB starting with BB. */ bitmap_clear (&ebb_global_regs); - bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb)); + bitmap_ior_into (&ebb_global_regs, DF_LIVE_SUBREG_IN (bb)); for (;;) { if (lra_dump_file != NULL) @@ -7252,7 +7293,7 @@ lra_inheritance (void) break; bb = bb->next_bb; } - bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb)); + bitmap_ior_into (&ebb_global_regs, DF_LIVE_SUBREG_OUT (bb)); if (lra_dump_file != NULL) fprintf (lra_dump_file, "\n"); if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb))) @@ -7281,15 +7322,23 @@ int lra_undo_inheritance_iter; /* Fix BB live info LIVE after removing pseudos created on pass doing inheritance/split which are REMOVED_PSEUDOS. */ static void -fix_bb_live_info (bitmap live, bitmap removed_pseudos) +fix_bb_live_info (bitmap all, bitmap full, bitmap partial, + bitmap removed_pseudos) { unsigned int regno; bitmap_iterator bi; EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi) - if (bitmap_clear_bit (live, regno) - && REG_P (lra_reg_info[regno].restore_rtx)) - bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx)); + { + if (bitmap_clear_bit (all, regno) + && REG_P (lra_reg_info[regno].restore_rtx)) + { + bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx)); + bitmap_clear_bit (full, regno); + bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx)); + gcc_assert (!bitmap_bit_p (partial, regno)); + } + } } /* Return regno of the (subreg of) REG. Otherwise, return a negative @@ -7355,8 +7404,10 @@ remove_inheritance_pseudos (bitmap remove_pseudos) constraint pass. */ FOR_EACH_BB_FN (bb, cfun) { - fix_bb_live_info (df_get_live_in (bb), remove_pseudos); - fix_bb_live_info (df_get_live_out (bb), remove_pseudos); + fix_bb_live_info (DF_LIVE_SUBREG_IN (bb), DF_LIVE_SUBREG_FULL_IN (bb), + DF_LIVE_SUBREG_PARTIAL_IN (bb), remove_pseudos); + fix_bb_live_info (DF_LIVE_SUBREG_OUT (bb), DF_LIVE_SUBREG_FULL_OUT (bb), + DF_LIVE_SUBREG_PARTIAL_OUT (bb), remove_pseudos); FOR_BB_INSNS_REVERSE (bb, curr_insn) { if (! INSN_P (curr_insn)) diff --git a/gcc/lra-int.h b/gcc/lra-int.h index d0752c2ae50..678377d9ec6 100644 --- a/gcc/lra-int.h +++ b/gcc/lra-int.h @@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. If not see #ifndef GCC_LRA_INT_H #define GCC_LRA_INT_H +#include "subreg-live-range.h" + #define lra_assert(c) gcc_checking_assert (c) /* The parameter used to prevent infinite reloading for an insn. Each diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc index f60e564da82..d93921ad302 100644 --- a/gcc/lra-lives.cc +++ b/gcc/lra-lives.cc @@ -272,8 +272,26 @@ update_pseudo_point (int regno, int point, enum point_type type) } } -/* The corresponding bitmaps of BB currently being processed. */ -static bitmap bb_killed_pseudos, bb_gen_pseudos; +/* Structure describing local BB data used for pseudo + live-analysis. */ +class bb_data_pseudos : public basic_block_subreg_live_info +{ +public: + /* Basic block about which the below data are. */ + basic_block bb; +}; + +/* Array for all BB data. Indexed by the corresponding BB index. */ +typedef class bb_data_pseudos *bb_data_t; + +/* All basic block data are referred through the following array. */ +static bb_data_t bb_data; + +/* The corresponding basic block info of BB currently being processed. */ +static bb_data_t curr_bb_info; + +/* Flag mean curr function has subreg ref need be tracked. */ +static bool has_subreg_live_p; /* Record hard register REGNO as now being live. It updates living hard regs and START_LIVING. */ @@ -287,7 +305,7 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); sparseset_set_bit (start_living, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); } /* Process the definition of hard register REGNO. This updates @@ -310,8 +328,8 @@ make_hard_regno_dead (int regno) sparseset_set_bit (start_dying, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) { - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); } } @@ -355,7 +373,9 @@ mark_regno_live (int regno, machine_mode mode) else { mark_pseudo_live (regno); - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_use, regno)); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_def, regno)); } } @@ -375,8 +395,10 @@ mark_regno_dead (int regno, machine_mode mode) else { mark_pseudo_dead (regno); - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_use, regno)); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_def, regno)); } } @@ -387,23 +409,6 @@ mark_regno_dead (int regno, machine_mode mode) border. That might be a consequence of some global transformations in LRA, e.g. PIC pseudo reuse or rematerialization. */ -/* Structure describing local BB data used for pseudo - live-analysis. */ -class bb_data_pseudos -{ -public: - /* Basic block about which the below data are. */ - basic_block bb; - bitmap_head killed_pseudos; /* pseudos killed in the BB. */ - bitmap_head gen_pseudos; /* pseudos generated in the BB. */ -}; - -/* Array for all BB data. Indexed by the corresponding BB index. */ -typedef class bb_data_pseudos *bb_data_t; - -/* All basic block data are referred through the following array. */ -static bb_data_t bb_data; - /* Two small functions for access to the bb data. */ static inline bb_data_t get_bb_data (basic_block bb) @@ -430,13 +435,93 @@ static bool live_trans_fun (int bb_index) { basic_block bb = get_bb_data_by_index (bb_index)->bb; - bitmap bb_liveout = df_get_live_out (bb); - bitmap bb_livein = df_get_live_in (bb); + bitmap full_out = DF_LIVE_SUBREG_FULL_OUT (bb); + bitmap full_in = DF_LIVE_SUBREG_FULL_IN (bb); + bitmap partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (bb); + bitmap partial_in = DF_LIVE_SUBREG_PARTIAL_IN (bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (bb); + subregs_live *range_in = DF_LIVE_SUBREG_RANGE_IN (bb); bb_data_t bb_info = get_bb_data (bb); - bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap); - return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos, - &temp_bitmap, &bb_info->killed_pseudos); + if (!has_subreg_live_p) + { + bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap); + return bitmap_ior_and_compl (full_in, &bb_info->full_use, &temp_bitmap, + &bb_info->full_def); + } + + /* If there has subreg live need be tracked. */ + unsigned int regno; + bitmap_iterator bi; + bool changed = false; + bitmap_head temp_full_out; + bitmap_head temp_partial_out; + bitmap_head temp_partial_be_full_out; + bitmap_head all_def; + subregs_live temp_range_out; + bitmap_initialize (&temp_full_out, ®_obstack); + bitmap_initialize (&temp_partial_out, ®_obstack); + bitmap_initialize (&temp_partial_be_full_out, ®_obstack); + bitmap_initialize (&all_def, ®_obstack); + + bitmap_and_compl (&temp_full_out, full_out, &all_hard_regs_bitmap); + + bitmap_ior (&all_def, &bb_info->full_def, &bb_info->partial_def); + + bitmap_and (&temp_partial_out, &temp_full_out, &bb_info->partial_def); + EXECUTE_IF_SET_IN_BITMAP (&temp_partial_out, FIRST_PSEUDO_REGISTER, regno, bi) + { + subreg_ranges temp (bb_info->range_def->lives.at (regno).max); + temp.make_full (); + temp.remove_ranges (bb_info->range_def->lives.at (regno)); + temp_range_out.add_ranges (regno, temp); + } + bitmap_ior_and_compl_into (&temp_partial_out, partial_out, &all_def); + EXECUTE_IF_AND_COMPL_IN_BITMAP (partial_out, &all_def, FIRST_PSEUDO_REGISTER, + regno, bi) + { + temp_range_out.add_ranges (regno, range_out->lives.at (regno)); + } + EXECUTE_IF_AND_IN_BITMAP (partial_out, &bb_info->partial_def, 0, regno, bi) + { + subreg_ranges temp = range_out->lives.at (regno); + temp.remove_ranges (bb_info->range_def->lives.at (regno)); + if (!temp.empty_p ()) + { + bitmap_set_bit (&temp_partial_out, regno); + temp_range_out.add_ranges (regno, temp); + } + } + + temp_range_out.add_lives (*bb_info->range_use); + EXECUTE_IF_AND_IN_BITMAP (&temp_partial_out, &bb_info->partial_use, 0, regno, + bi) + { + subreg_ranges temp = temp_range_out.lives.at (regno); + temp.add_ranges (bb_info->range_use->lives.at (regno)); + if (temp.full_p ()) + { + bitmap_set_bit (&temp_partial_be_full_out, regno); + temp_range_out.remove_live (regno); + } + } + + bitmap_ior_and_compl_into (&temp_partial_be_full_out, &temp_full_out, + &all_def); + changed + |= bitmap_ior (full_in, &temp_partial_be_full_out, &bb_info->full_use); + + bitmap_ior_into (&temp_partial_out, &bb_info->partial_use); + changed |= bitmap_and_compl (partial_in, &temp_partial_out, + &temp_partial_be_full_out); + changed |= range_in->copy_lives (temp_range_out); + + bitmap_clear (&temp_full_out); + bitmap_clear (&temp_partial_out); + bitmap_clear (&temp_partial_be_full_out); + bitmap_clear (&all_def); + + return changed; } /* The confluence function used by the DF equation solver to set up @@ -444,7 +529,9 @@ live_trans_fun (int bb_index) static void live_con_fun_0 (basic_block bb) { - bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap); + bitmap_and_into (DF_LIVE_SUBREG_OUT (bb), &all_hard_regs_bitmap); + bitmap_and_into (DF_LIVE_SUBREG_FULL_OUT (bb), &all_hard_regs_bitmap); + bitmap_and_into (DF_LIVE_SUBREG_PARTIAL_OUT (bb), &all_hard_regs_bitmap); } /* The confluence function used by the DF equation solver to propagate @@ -456,13 +543,77 @@ live_con_fun_0 (basic_block bb) static bool live_con_fun_n (edge e) { - basic_block bb = e->src; - basic_block dest = e->dest; - bitmap bb_liveout = df_get_live_out (bb); - bitmap dest_livein = df_get_live_in (dest); + class df_live_subreg_bb_info *src_bb_info + = df_live_subreg_get_bb_info (e->src->index); + class df_live_subreg_bb_info *dest_bb_info + = df_live_subreg_get_bb_info (e->dest->index); + + if (!has_subreg_live_p) + { + return bitmap_ior_and_compl_into (&src_bb_info->full_out, + &dest_bb_info->full_in, + &all_hard_regs_bitmap); + } + + /* If there has subreg live need be tracked. Calculation formula: + temp_full mean: + 1. partial in out/in, full in other in/out + 2. partial in out and in, and mrege range is full + temp_range mean: + the range of regno which partial live + src_bb_info->partial_out = (src_bb_info->partial_out | + dest_bb_info->partial_in) & ~temp_full src_bb_info->range_out = copy + (temp_range) src_bb_info->full_out |= dest_bb_info->full_in | temp_full + */ + subregs_live temp_range; + temp_range.add_lives (*src_bb_info->range_out); + temp_range.add_lives (*dest_bb_info->range_in); + + bitmap_head temp_partial_all; + bitmap_initialize (&temp_partial_all, &bitmap_default_obstack); + bitmap_ior (&temp_partial_all, &src_bb_info->partial_out, + &dest_bb_info->partial_in); + + bitmap_head temp_full; + bitmap_initialize (&temp_full, &bitmap_default_obstack); + + /* Collect regno that become full after merge src_bb_info->partial_out + and dest_bb_info->partial_in. */ + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (&temp_partial_all, FIRST_PSEUDO_REGISTER, regno, bi) + { + if (bitmap_bit_p (&src_bb_info->full_out, regno) + || bitmap_bit_p (&dest_bb_info->full_in, regno)) + { + bitmap_set_bit (&temp_full, regno); + temp_range.remove_live (regno); + continue; + } + else if (!bitmap_bit_p (&src_bb_info->partial_out, regno) + || !bitmap_bit_p (&dest_bb_info->partial_in, regno)) + continue; + + subreg_ranges temp = src_bb_info->range_out->lives.at (regno); + temp.add_ranges (dest_bb_info->range_in->lives.at (regno)); + if (temp.full_p ()) + { + bitmap_set_bit (&temp_full, regno); + temp_range.remove_live (regno); + } + } + + /* Calculating src_bb_info->partial_out and src_bb_info->range_out. */ + bool changed = bitmap_and_compl (&src_bb_info->partial_out, &temp_partial_all, + &temp_full); + changed |= src_bb_info->range_out->copy_lives (temp_range); - return bitmap_ior_and_compl_into (bb_liveout, - dest_livein, &all_hard_regs_bitmap); + /* Calculating src_bb_info->full_out. */ + bitmap_ior_and_compl_into (&temp_full, &dest_bb_info->full_in, + &all_hard_regs_bitmap); + changed |= bitmap_ior_into (&src_bb_info->full_out, &temp_full); + + return changed; } /* Indexes of all function blocks. */ @@ -483,8 +634,12 @@ initiate_live_solver (void) { bb_data_t bb_info = get_bb_data (bb); bb_info->bb = bb; - bitmap_initialize (&bb_info->killed_pseudos, ®_obstack); - bitmap_initialize (&bb_info->gen_pseudos, ®_obstack); + bitmap_initialize (&bb_info->full_def, ®_obstack); + bitmap_initialize (&bb_info->partial_def, ®_obstack); + bitmap_initialize (&bb_info->full_use, ®_obstack); + bitmap_initialize (&bb_info->partial_use, ®_obstack); + bb_info->range_def = new subregs_live (); + bb_info->range_use = new subregs_live (); bitmap_set_bit (&all_blocks, bb->index); } } @@ -499,8 +654,12 @@ finish_live_solver (void) FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap_clear (&bb_info->killed_pseudos); - bitmap_clear (&bb_info->gen_pseudos); + bitmap_clear (&bb_info->full_def); + bitmap_clear (&bb_info->partial_def); + bitmap_clear (&bb_info->full_use); + bitmap_clear (&bb_info->partial_use); + delete bb_info->range_def; + delete bb_info->range_use; } free (bb_data); bitmap_clear (&all_hard_regs_bitmap); @@ -663,7 +822,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) /* Only has a meaningful value once we've seen a call. */ function_abi last_call_abi = default_function_abi; - reg_live_out = df_get_live_out (bb); + reg_live_out = DF_LIVE_SUBREG_OUT (bb); sparseset_clear (pseudos_live); sparseset_clear (pseudos_live_through_calls); sparseset_clear (pseudos_live_through_setjumps); @@ -675,10 +834,13 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_live (j); } - bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos; - bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos; - bitmap_clear (bb_gen_pseudos); - bitmap_clear (bb_killed_pseudos); + curr_bb_info = get_bb_data (bb); + bitmap_clear (&curr_bb_info->full_use); + bitmap_clear (&curr_bb_info->partial_use); + bitmap_clear (&curr_bb_info->full_def); + bitmap_clear (&curr_bb_info->partial_def); + curr_bb_info->range_use->clear (); + curr_bb_info->range_def->clear (); freq = REG_FREQ_FROM_BB (bb); if (lra_dump_file != NULL) @@ -1101,16 +1263,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) bool live_change_p = false; /* Check if bb border live info was changed. */ unsigned int live_pseudos_num = 0; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), - FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, j, + bi) { live_pseudos_num++; - if (! sparseset_bit_p (pseudos_live, j)) + if (!sparseset_bit_p (pseudos_live, j)) { live_change_p = true; if (lra_dump_file != NULL) - fprintf (lra_dump_file, - " r%d is removed as live at bb%d start\n", j, bb->index); + fprintf (lra_dump_file, " r%d is removed as live at bb%d start\n", + j, bb->index); break; } } @@ -1120,9 +1282,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) live_change_p = true; if (lra_dump_file != NULL) EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j) - if (! bitmap_bit_p (df_get_live_in (bb), j)) - fprintf (lra_dump_file, - " r%d is added to live at bb%d start\n", j, bb->index); + if (!bitmap_bit_p (DF_LIVE_SUBREG_IN (bb), j)) + fprintf (lra_dump_file, " r%d is added to live at bb%d start\n", j, + bb->index); } /* See if we'll need an increment at the end of this basic block. An increment is needed if the PSEUDOS_LIVE set is not empty, @@ -1135,8 +1297,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_dead (i); } - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi) - { + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, j, + bi) + { if (sparseset_cardinality (pseudos_live_through_calls) == 0) break; if (sparseset_bit_p (pseudos_live_through_calls, j)) @@ -1151,7 +1314,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i)) continue; - if (bitmap_bit_p (df_get_live_in (bb), i)) + if (bitmap_bit_p (DF_LIVE_SUBREG_IN (bb), i)) continue; live_change_p = true; @@ -1159,7 +1322,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) fprintf (lra_dump_file, " hard reg r%d is added to live at bb%d start\n", i, bb->index); - bitmap_set_bit (df_get_live_in (bb), i); + bitmap_set_bit (DF_LIVE_SUBREG_IN (bb), i); + bitmap_set_bit (DF_LIVE_SUBREG_FULL_IN (bb), i); } if (need_curr_point_incr) @@ -1425,10 +1589,24 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) disappear, e.g. pseudos with used equivalences. */ FOR_EACH_BB_FN (bb, cfun) { - bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_FULL_IN (bb), + FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); - bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (DF_LIVE_SUBREG_PARTIAL_IN (bb), + FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_OUT (bb), FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_FULL_OUT (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_PARTIAL_OUT (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + DF_LIVE_SUBREG_RANGE_IN (bb)->clear (); + DF_LIVE_SUBREG_RANGE_OUT (bb)->clear (); } /* As we did not change CFG since LRA start we can use DF-infrastructure solver to solve live data flow problem. */ @@ -1441,6 +1619,8 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) (DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n, live_trans_fun, &all_blocks, df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD)); + df_live_subreg_finalize (&all_blocks); + if (lra_dump_file != NULL) { fprintf (lra_dump_file, @@ -1449,16 +1629,28 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) FOR_EACH_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap bb_livein = df_get_live_in (bb); - bitmap bb_liveout = df_get_live_out (bb); fprintf (lra_dump_file, "\nBB %d:\n", bb->index); - lra_dump_bitmap_with_title (" gen:", - &bb_info->gen_pseudos, bb->index); - lra_dump_bitmap_with_title (" killed:", - &bb_info->killed_pseudos, bb->index); - lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index); - lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index); + lra_dump_bitmap_with_title (" full use", &bb_info->full_use, + bb->index); + lra_dump_bitmap_with_title (" partial use", + &bb_info->partial_use, bb->index); + lra_dump_bitmap_with_title (" full def", &bb_info->full_def, + bb->index); + lra_dump_bitmap_with_title (" partial def", + &bb_info->partial_def, bb->index); + lra_dump_bitmap_with_title (" live in full", + DF_LIVE_SUBREG_FULL_IN (bb), + bb->index); + lra_dump_bitmap_with_title (" live in partial", + DF_LIVE_SUBREG_PARTIAL_IN (bb), + bb->index); + lra_dump_bitmap_with_title (" live out full", + DF_LIVE_SUBREG_FULL_OUT (bb), + bb->index); + lra_dump_bitmap_with_title (" live out partial", + DF_LIVE_SUBREG_PARTIAL_OUT (bb), + bb->index); } } } diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc index 681dcf36331..26d3da07b00 100644 --- a/gcc/lra-remat.cc +++ b/gcc/lra-remat.cc @@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void) fprintf (lra_dump_file, "\nBB %d:\n", bb->index); /* Livein */ fprintf (lra_dump_file, " register live in:"); - dump_regset (df_get_live_in (bb), lra_dump_file); + dump_regset (DF_LIVE_SUBREG_IN (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Liveout */ fprintf (lra_dump_file, " register live out:"); - dump_regset (df_get_live_out (bb), lra_dump_file); + dump_regset (DF_LIVE_SUBREG_OUT (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Changed/dead regs: */ fprintf (lra_dump_file, " changed regs:"); @@ -727,7 +727,7 @@ calculate_livein_cands (void) FOR_EACH_BB_FN (bb, cfun) { - bitmap livein_regs = df_get_live_in (bb); + bitmap livein_regs = DF_LIVE_SUBREG_IN (bb); bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands; for (unsigned int i = 0; i < cands_num; i++) { @@ -1064,11 +1064,10 @@ do_remat (void) FOR_EACH_BB_FN (bb, cfun) { CLEAR_HARD_REG_SET (live_hard_regs); - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi) + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_IN (bb), 0, regno, bi) { - int hard_regno = regno < FIRST_PSEUDO_REGISTER - ? regno - : reg_renumber[regno]; + int hard_regno + = regno < FIRST_PSEUDO_REGISTER ? regno : reg_renumber[regno]; if (hard_regno >= 0) SET_HARD_REG_BIT (live_hard_regs, hard_regno); } diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc index a663a1931e3..d38a2ffe2a7 100644 --- a/gcc/lra-spills.cc +++ b/gcc/lra-spills.cc @@ -566,8 +566,26 @@ spill_pseudos (void) "Debug insn #%u is reset because it referenced " "removed pseudo\n", INSN_UID (insn)); } - bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos); - bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos); + unsigned int regno; + bitmap_iterator bi; + + bitmap_and_compl_into (DF_LIVE_SUBREG_IN (bb), spilled_pseudos); + bitmap_and_compl_into (DF_LIVE_SUBREG_FULL_IN (bb), spilled_pseudos); + bitmap partial_in = DF_LIVE_SUBREG_PARTIAL_IN (bb); + subregs_live *range_in = DF_LIVE_SUBREG_RANGE_IN (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_in->remove_live (regno); + bitmap_and_compl_into (partial_in, spilled_pseudos); + + bitmap_and_compl_into (DF_LIVE_SUBREG_OUT (bb), spilled_pseudos); + bitmap_and_compl_into (DF_LIVE_SUBREG_FULL_OUT (bb), spilled_pseudos); + bitmap partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_out->remove_live (regno); + bitmap_and_compl_into (partial_out, spilled_pseudos); } } }