From patchwork Sun Nov 12 09:58:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 164242 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b909:0:b0:403:3b70:6f57 with SMTP id t9csp621977vqg; Sun, 12 Nov 2023 02:00:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IH/um21JVBwMZMPWUOtppMFORlDumUaNRQI22hRs5qPO6wQYomJ2WNgsoyFNqIHo7AHn8nE X-Received: by 2002:a05:620a:27ca:b0:76c:ba10:4973 with SMTP id i10-20020a05620a27ca00b0076cba104973mr4824358qkp.37.1699783207350; Sun, 12 Nov 2023 02:00:07 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699783207; cv=pass; d=google.com; s=arc-20160816; b=ovq7p3HLpGa/euFJRFTlrbyNn8fVN111n4JBdKYqFbx/Gq74YvT+2Nf1AT/+pshVps awHAkgIoxC1myLc0zgYuk7Q/ytorjvNeqph6HTrtBv1DfnAeZp8IwrviHXjDfPHOqgkc rncSp/hX9xgi3He3QqkVYoGA9yS4j09XsXLNOnB2wcoZ5Hsc1Mrk1QpsRwm8TwezCy4S FKBvXtURrS80hwT4OhMuWtiJioEd8/3nJPJQJU3YLFIkWLASA+f2BUmtR6abHG+6g8HA EXPPG22JrInmzER5rD734R1pk7erFzEE4BFAw0C92KNZYe+pN/ZEG6uv4fNFlR8KKjWt TAUg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:feedback-id :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-filter:dmarc-filter :delivered-to; bh=/3+FOTfdNhAMMUmX/g6c9LRiZXG6tZ9F0/oIyXGQBzI=; fh=9Ok8HNl3eD0lUFF4nhUPZJmQfyAUbHnIPw/rSVNIfK0=; b=bQsyR/XXDeopbNcmd+pStlgqHE58NJqLNpQHBLkEJ9r4YHJ640rJjum+Gfakugc8TB 58tiXyvBjLQjgWz8sZKMp0d/fS6xrHB+TmMnsojrObAYt52K1HEkNPdGCbJVFKRkjMoR j0rw6wVLp+UCMpJkyFcVRsjCOXTiAtMuPEtK00MZvrmSCsoTaqphTDp6PrcDNuWmXuY0 /aUlnmowvS7P8yQ18PtevkwpGFg38ecHhsbjsy1IFxLDJRvjwwbdMbgqrA1r5bg67X6Z zgz97ptIlsq+SgM6pMrcKY6dIuSP3h98SpThYGPOo9p1FkN3S40Uqh65BSjf9o8EBYoM bneA== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id s22-20020a05620a0bd600b0076daa46b2a0si2626141qki.87.2023.11.12.02.00.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 12 Nov 2023 02:00:07 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6E9723858418 for ; Sun, 12 Nov 2023 10:00:02 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbguseast2.qq.com (smtpbguseast2.qq.com [54.204.34.130]) by sourceware.org (Postfix) with ESMTPS id 1F5883858D32 for ; Sun, 12 Nov 2023 09:59:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 1F5883858D32 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 1F5883858D32 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.204.34.130 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699783161; cv=none; b=r7TxnUSeZ3adULY/XVW3nwdaVREda0/r0+Q1OtdQeEC5Cz/yMvfPtlC3Pom3o+pZ2pua76tECI7BDwEjcDaMqXVvjbnKm9+QtED7/H+0+aVXOGSrIhIbag8p/LWqY+ANBKy6KSVCgYBjHlS9TLjxH5anYCdltal+x2U1UkQ+wyo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699783161; c=relaxed/simple; bh=y0Wlssn1XBmUHi20jwshRDhpavwlmjIiRwvzMudTlDU=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=m/rXqdEFa1ZuCKEbH288pxi9o/V6QVUkkhFZ1OzIpJlRTVbws74IwTu1B7Raxko+qyTYYX40bPgyLH7Vx40gON4S/27sjYY+t1m2xjLmTCnkBzjg9256IK26Lig7yaTW5VQHErWfrOzTRdYveKx4FWAoXWwMv5i1XJLCFqXJLBc= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp83t1699783149t42q4bms Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Sun, 12 Nov 2023 17:59:09 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: eTp2eCgHrZkTwiG2UY6eAgC+MhhPPKjdcLTVq1/YkBgQjo2XJ+/4RoAIMsay+ fXC8jfM9sHcJnYSCWTYWZwVkYzsyhJqWtFOdk0/Bd7mmbUl3BB69RTG7BN9iICbpN0osnOW U9SrHOCwpiZ/MOGd7yk02e73z2szegeGadoNMssU7kJPl/Yqg1Xk/k8+lAAPZJpbvmGKo/0 ynVcJrDML3nFFwsWK266o7MhPoY/+OdPmqDpAGWxeaQDaSGLjYw8yrlLTR7hLCOKadSbBnm 6qykIcNwRfGGglg3v3+hNE9JIbkOQPm5wWZzJRG9NR4zG103xnV5JBkDA2XU8Yvf7mSbqyj hXa/MQpTfKMgNWiBlZLCouQXVRL+1kP4HVKuXC3mYon91KPASU4rRj2aVRMvkWiFbk2SnhL X-QQ-GoodBg: 2 X-BIZMAIL-ID: 13116745521209702665 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH V2 3/7] ira: Support subreg live range track Date: Sun, 12 Nov 2023 17:58:54 +0800 Message-Id: <20231112095858.3669003-4-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20231112095858.3669003-1-lehua.ding@rivai.ai> References: <20231112095858.3669003-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, URIBL_SBL_A autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782351876156658215 X-GMAIL-MSGID: 1782351876156658215 This patch supports tracking subreg liveness. It first extends ira_object_t objects[2] to std::vector objects, which can hold more than one object, and is used to collect all access via subreg in program and the partial_in and partial_out of the basic block live in/out. Then there is a modification to the way conflicts between registers are detected, for example, if a object conflicts with b object, then the offset and size of the object relative to the allocno it belongs to need to be taken into account to compute the conflict registers between allocno and allocno. gcc/ChangeLog: * hard-reg-set.h (struct HARD_REG_SET): New shift operator. * ira-build.cc (ira_create_object): Adjust. (find_object): New. (find_object_anyway): New. (ira_create_allocno): Adjust. (get_range): New. (ira_copy_allocno_objects): New. (merge_hard_reg_conflicts): Adjust copy. (create_cap_allocno): Adjust. (find_subreg_p): New. (add_subregs): New. (create_insn_allocnos): Collect subreg. (create_bb_allocnos): Ditto. (move_allocno_live_ranges): Adjust. (copy_allocno_live_ranges): Adjust. (setup_min_max_allocno_live_range_point): Adjust. * ira-color.cc (INCLUDE_MAP): include map. (setup_left_conflict_sizes_p): Adjust conflict size. (setup_profitable_hard_regs): Adjust. (get_conflict_and_start_profitable_regs): Adjust. (check_hard_reg_p): Adjust conflict check. (assign_hard_reg): Adjust. (push_allocno_to_stack): Adjust conflict size. (improve_allocation): Adjust. * ira-conflicts.cc (record_object_conflict): Simplify. (build_object_conflicts): Adjust. (build_conflicts): Adjust. (print_allocno_conflicts): Adjust. * ira-emit.cc (modify_move_list): Adjust. * ira-int.h (struct ira_object): Adjust struct. (struct ira_allocno): Adjust struct. (ALLOCNO_NUM_OBJECTS): New accessor. (ALLOCNO_UNIT_SIZE): Ditto. (ALLOCNO_TRACK_SUBREG_P): Ditto. (ALLOCNO_NREGS): Ditto. (OBJECT_SUBWORD): Ditto. (OBJECT_INDEX): Ditto. (OBJECT_START): Ditto. (OBJECT_NREGS): Ditto. (find_object): Exported. (find_object_anyway): Ditto. (ira_copy_allocno_objects): Ditto. (has_subreg_object_p): Ditto. (get_full_object): Ditto. * ira-lives.cc (INCLUDE_VECTOR): Include vector. (add_onflict_hard_regs): New. (add_onflict_hard_reg): New. (make_hard_regno_dead): Adjust. (make_object_live): Adjust. (update_allocno_pressure_excess_length): Adjust. (make_object_dead): Adjust. (mark_pseudo_regno_live): Adjust. (add_subreg_point): New. (mark_pseudo_object_live): Adjust. (mark_pseudo_regno_subword_live): Adjust. (mark_pseudo_regno_subreg_live): Adjust. (mark_pseudo_regno_subregs_live): Adjust. (mark_pseudo_reg_live): Adjust. (mark_pseudo_regno_dead): Adjust. (mark_pseudo_object_dead): Adjust. (mark_pseudo_regno_subword_dead): Adjust. (mark_pseudo_regno_subreg_dead): Adjust. (mark_pseudo_reg_dead): Adjust. (process_single_reg_class_operands): Adjust. (process_out_of_region_eh_regs): Adjust. (add_conflict_from_region_landing_pads): Adjust. (process_bb_node_lives): Adjust. (class subreg_live_item): New class. (create_subregs_live_ranges): New function. (ira_create_allocno_live_ranges): Adjust. * ira.cc (check_allocation): Adjust. --- gcc/hard-reg-set.h | 33 +++ gcc/ira-build.cc | 235 +++++++++++++++++--- gcc/ira-color.cc | 302 +++++++++++++++++--------- gcc/ira-conflicts.cc | 48 ++--- gcc/ira-emit.cc | 2 +- gcc/ira-int.h | 57 ++++- gcc/ira-lives.cc | 500 ++++++++++++++++++++++++++++++++----------- gcc/ira.cc | 52 ++--- 8 files changed, 907 insertions(+), 322 deletions(-) diff --git a/gcc/hard-reg-set.h b/gcc/hard-reg-set.h index b0bb9bce074..760eadba186 100644 --- a/gcc/hard-reg-set.h +++ b/gcc/hard-reg-set.h @@ -113,6 +113,39 @@ struct HARD_REG_SET return !operator== (other); } + HARD_REG_SET + operator>> (unsigned int shift_amount) const + { + if (shift_amount == 0) + return *this; + + HARD_REG_SET res; + unsigned int total_bits = sizeof (HARD_REG_ELT_TYPE) * 8; + if (shift_amount >= total_bits) + { + unsigned int n_elt = shift_amount % total_bits; + shift_amount -= n_elt * total_bits; + for (unsigned int i = 0; i < ARRAY_SIZE (elts) - n_elt - 1; i += 1) + res.elts[i] = elts[i + n_elt]; + /* clear upper n_elt elements. */ + for (unsigned int i = 0; i < n_elt; i += 1) + res.elts[ARRAY_SIZE (elts) - 1 - i] = 0; + } + + if (shift_amount > 0) + { + /* The left bits of an element be shifted. */ + HARD_REG_ELT_TYPE left = 0; + /* Total bits of an element. */ + for (int i = ARRAY_SIZE (elts); i >= 0; --i) + { + res.elts[i] = (elts[i] >> shift_amount) | left; + left = elts[i] << (total_bits - shift_amount); + } + } + return res; + } + HARD_REG_ELT_TYPE elts[HARD_REG_SET_LONGS]; }; typedef const HARD_REG_SET &const_hard_reg_set; diff --git a/gcc/ira-build.cc b/gcc/ira-build.cc index f931c6e304c..a32693e69e4 100644 --- a/gcc/ira-build.cc +++ b/gcc/ira-build.cc @@ -29,10 +29,12 @@ along with GCC; see the file COPYING3. If not see #include "insn-config.h" #include "regs.h" #include "memmodel.h" +#include "tm_p.h" #include "ira.h" #include "ira-int.h" #include "sparseset.h" #include "cfgloop.h" +#include "subreg-live-range.h" static ira_copy_t find_allocno_copy (ira_allocno_t, ira_allocno_t, rtx_insn *, ira_loop_tree_node_t); @@ -442,13 +444,12 @@ initiate_allocnos (void) /* Create and return an object corresponding to a new allocno A. */ static ira_object_t -ira_create_object (ira_allocno_t a, int subword) +ira_create_object (ira_allocno_t a, int start, int nregs) { enum reg_class aclass = ALLOCNO_CLASS (a); ira_object_t obj = object_pool.allocate (); OBJECT_ALLOCNO (obj) = a; - OBJECT_SUBWORD (obj) = subword; OBJECT_CONFLICT_ID (obj) = ira_objects_num; OBJECT_CONFLICT_VEC_P (obj) = false; OBJECT_CONFLICT_ARRAY (obj) = NULL; @@ -460,12 +461,75 @@ ira_create_object (ira_allocno_t a, int subword) OBJECT_MIN (obj) = INT_MAX; OBJECT_MAX (obj) = -1; OBJECT_LIVE_RANGES (obj) = NULL; + OBJECT_START (obj) = start; + OBJECT_NREGS (obj) = nregs; + OBJECT_INDEX (obj) = ALLOCNO_NUM_OBJECTS (a); ira_object_id_map_vec.safe_push (obj); ira_object_id_map = ira_object_id_map_vec.address (); ira_objects_num = ira_object_id_map_vec.length (); + a->objects.push_back (obj); + + return obj; +} + +/* Return the object in allocno A which match START & NREGS. */ +ira_object_t +find_object (ira_allocno_t a, int start, int nregs) +{ + for (ira_object_t obj : a->objects) + { + if (OBJECT_START (obj) == start && OBJECT_NREGS (obj) == nregs) + return obj; + } + return NULL; +} + +ira_object_t +find_object (ira_allocno_t a, poly_int64 offset, poly_int64 size) +{ + enum reg_class aclass = ALLOCNO_CLASS (a); + machine_mode mode = ALLOCNO_MODE (a); + int nregs = ira_reg_class_max_nregs[aclass][mode]; + + if (!has_subreg_object_p (a) + || maybe_eq (GET_MODE_SIZE (ALLOCNO_MODE (a)), size)) + return find_object (a, 0, nregs); + + gcc_assert (maybe_lt (size, GET_MODE_SIZE (ALLOCNO_MODE (a))) + && maybe_le (offset + size, GET_MODE_SIZE (ALLOCNO_MODE (a)))); + + int subreg_start = -1; + int subreg_nregs = -1; + for (int i = 0; i < nregs; i += 1) + { + poly_int64 right = ALLOCNO_UNIT_SIZE (a) * (i + 1); + if (subreg_start < 0 && maybe_lt (offset, right)) + { + subreg_start = i; + } + if (subreg_nregs < 0 && maybe_le (offset + size, right)) + { + subreg_nregs = i + 1 - subreg_start; + break; + } + } + gcc_assert (subreg_start >= 0 && subreg_nregs > 0); + return find_object (a, subreg_start, subreg_nregs); +} + +/* Return the object in allocno A which match START & NREGS. Create when not + found. */ +ira_object_t +find_object_anyway (ira_allocno_t a, int start, int nregs) +{ + ira_object_t obj = find_object (a, start, nregs); + if (obj == NULL && ALLOCNO_TRACK_SUBREG_P (a)) + obj = ira_create_object (a, start, nregs); + + gcc_assert (obj != NULL); return obj; } @@ -525,7 +589,8 @@ ira_create_allocno (int regno, bool cap_p, ALLOCNO_MEMORY_COST (a) = 0; ALLOCNO_UPDATED_MEMORY_COST (a) = 0; ALLOCNO_EXCESS_PRESSURE_POINTS_NUM (a) = 0; - ALLOCNO_NUM_OBJECTS (a) = 0; + ALLOCNO_UNIT_SIZE (a) = 0; + ALLOCNO_TRACK_SUBREG_P (a) = false; ALLOCNO_ADD_DATA (a) = NULL; allocno_vec.safe_push (a); @@ -549,6 +614,51 @@ ira_set_allocno_class (ira_allocno_t a, enum reg_class aclass) OBJECT_CONFLICT_HARD_REGS (obj) |= ~reg_class_contents[aclass]; OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= ~reg_class_contents[aclass]; } + + if (aclass == NO_REGS) + return; + /* SET the unit_size of one register. */ + machine_mode mode = ALLOCNO_MODE (a); + int nregs = ira_reg_class_max_nregs[aclass][mode]; + if (nregs == 2 && maybe_eq (GET_MODE_SIZE (mode), nregs * UNITS_PER_WORD)) + { + ALLOCNO_UNIT_SIZE (a) = UNITS_PER_WORD; + ALLOCNO_TRACK_SUBREG_P (a) = true; + return; + } +} + +/* Return the subreg range of rtx SUBREG. */ +static subreg_range +get_range (rtx subreg) +{ + gcc_assert (read_modify_subreg_p (subreg)); + rtx reg = SUBREG_REG (subreg); + machine_mode reg_mode = GET_MODE (reg); + + machine_mode subreg_mode = GET_MODE (subreg); + int nblocks = get_nblocks (reg_mode); + poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode); + + poly_int64 offset = SUBREG_BYTE (subreg); + poly_int64 left = offset + GET_MODE_SIZE (subreg_mode); + + int subreg_start = -1; + int subreg_nblocks = -1; + for (int i = 0; i < nblocks; i += 1) + { + poly_int64 right = unit_size * (i + 1); + if (subreg_start < 0 && maybe_lt (offset, right)) + subreg_start = i; + if (subreg_nblocks < 0 && maybe_le (left, right)) + { + subreg_nblocks = i + 1 - subreg_start; + break; + } + } + gcc_assert (subreg_start >= 0 && subreg_nblocks > 0); + + return subreg_range (subreg_start, subreg_start + subreg_nblocks); } /* Determine the number of objects we should associate with allocno A @@ -558,15 +668,37 @@ ira_create_allocno_objects (ira_allocno_t a) { machine_mode mode = ALLOCNO_MODE (a); enum reg_class aclass = ALLOCNO_CLASS (a); - int n = ira_reg_class_max_nregs[aclass][mode]; - int i; + int nregs = ira_reg_class_max_nregs[aclass][mode]; - if (n != 2 || maybe_ne (GET_MODE_SIZE (mode), n * UNITS_PER_WORD)) - n = 1; + ira_create_object (a, 0, nregs); - ALLOCNO_NUM_OBJECTS (a) = n; - for (i = 0; i < n; i++) - ALLOCNO_OBJECT (a, i) = ira_create_object (a, i); + if (aclass == NO_REGS || !ALLOCNO_TRACK_SUBREG_P (a) || a->subregs.empty ()) + return; + + int nblocks = get_nblocks (ALLOCNO_MODE (a)); + int times = nblocks / ALLOCNO_NREGS (a); + gcc_assert (times >= 1 && nblocks % ALLOCNO_NREGS (a) == 0); + + for (const auto &range : a->subregs) + { + int start = range.start / times; + int end = CEIL (range.end, times); + if (find_object (a, start, end - start) != NULL) + continue; + ira_create_object (a, start, end - start); + } + + a->subregs.clear (); +} + +/* Copy the objects from FROM to TO. */ +void +ira_copy_allocno_objects (ira_allocno_t to, ira_allocno_t from) +{ + ira_allocno_object_iterator oi; + ira_object_t obj; + FOR_EACH_ALLOCNO_OBJECT (from, obj, oi) + ira_create_object (to, OBJECT_START (obj), OBJECT_NREGS (obj)); } /* For each allocno, set ALLOCNO_NUM_OBJECTS and create the @@ -590,11 +722,11 @@ merge_hard_reg_conflicts (ira_allocno_t from, ira_allocno_t to, bool total_only) { int i; - gcc_assert (ALLOCNO_NUM_OBJECTS (to) == ALLOCNO_NUM_OBJECTS (from)); - for (i = 0; i < ALLOCNO_NUM_OBJECTS (to); i++) + for (i = 0; i < ALLOCNO_NUM_OBJECTS (from); i++) { ira_object_t from_obj = ALLOCNO_OBJECT (from, i); - ira_object_t to_obj = ALLOCNO_OBJECT (to, i); + ira_object_t to_obj = find_object_anyway (to, OBJECT_START (from_obj), + OBJECT_NREGS (from_obj)); if (!total_only) OBJECT_CONFLICT_HARD_REGS (to_obj) @@ -888,7 +1020,7 @@ create_cap_allocno (ira_allocno_t a) ALLOCNO_WMODE (cap) = ALLOCNO_WMODE (a); aclass = ALLOCNO_CLASS (a); ira_set_allocno_class (cap, aclass); - ira_create_allocno_objects (cap); + ira_copy_allocno_objects (cap, a); ALLOCNO_CAP_MEMBER (cap) = a; ALLOCNO_CAP (a) = cap; ALLOCNO_CLASS_COST (cap) = ALLOCNO_CLASS_COST (a); @@ -1830,6 +1962,26 @@ ira_traverse_loop_tree (bool bb_p, ira_loop_tree_node_t loop_node, /* The basic block currently being processed. */ static basic_block curr_bb; +/* Return true if A's subregs has a subreg with same SIZE and OFFSET. */ +static bool +find_subreg_p (ira_allocno_t a, const subreg_range &r) +{ + for (const auto &item : a->subregs) + if (item.start == r.start && item.end == r.end) + return true; + return false; +} + +/* Return start and nregs subregs from DF_LIVE_SUBREG. */ +static void +add_subregs (ira_allocno_t a, const subreg_ranges &sr) +{ + gcc_assert (get_nblocks (ALLOCNO_MODE (a)) == (unsigned) sr.max); + for (const subreg_range &r : sr.ranges) + if (!find_subreg_p (a, r)) + a->subregs.push_back (r); +} + /* This recursive function creates allocnos corresponding to pseudo-registers containing in X. True OUTPUT_P means that X is an lvalue. OUTER corresponds to the parent expression of X. */ @@ -1859,6 +2011,14 @@ create_insn_allocnos (rtx x, rtx outer, bool output_p) } } + /* Collect subreg reference. */ + if (outer != NULL && read_modify_subreg_p (outer)) + { + const subreg_range r = get_range (outer); + if (!find_subreg_p (a, r)) + a->subregs.push_back (r); + } + ALLOCNO_NREFS (a)++; ALLOCNO_FREQ (a) += REG_FREQ_FROM_BB (curr_bb); if (output_p) @@ -1919,10 +2079,28 @@ create_bb_allocnos (ira_loop_tree_node_t bb_node) create_insn_allocnos (PATTERN (insn), NULL, false); /* It might be a allocno living through from one subloop to another. */ - EXECUTE_IF_SET_IN_REG_SET (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, + EXECUTE_IF_SET_IN_REG_SET (DF_LIVE_SUBREG_FULL_IN (bb), FIRST_PSEUDO_REGISTER, i, bi) if (ira_curr_regno_allocno_map[i] == NULL) ira_create_allocno (i, false, ira_curr_loop_tree_node); + + EXECUTE_IF_SET_IN_REG_SET (DF_LIVE_SUBREG_PARTIAL_IN (bb), + FIRST_PSEUDO_REGISTER, i, bi) + { + if (ira_curr_regno_allocno_map[i] == NULL) + ira_create_allocno (i, false, ira_curr_loop_tree_node); + add_subregs (ira_curr_regno_allocno_map[i], + DF_LIVE_SUBREG_RANGE_IN (bb)->lives.at (i)); + } + + EXECUTE_IF_SET_IN_REG_SET (DF_LIVE_SUBREG_PARTIAL_OUT (bb), + FIRST_PSEUDO_REGISTER, i, bi) + { + if (ira_curr_regno_allocno_map[i] == NULL) + ira_create_allocno (i, false, ira_curr_loop_tree_node); + add_subregs (ira_curr_regno_allocno_map[i], + DF_LIVE_SUBREG_RANGE_OUT (bb)->lives.at (i)); + } } /* Create allocnos corresponding to pseudo-registers living on edge E @@ -2137,20 +2315,20 @@ move_allocno_live_ranges (ira_allocno_t from, ira_allocno_t to) int i; int n = ALLOCNO_NUM_OBJECTS (from); - gcc_assert (n == ALLOCNO_NUM_OBJECTS (to)); - for (i = 0; i < n; i++) { ira_object_t from_obj = ALLOCNO_OBJECT (from, i); - ira_object_t to_obj = ALLOCNO_OBJECT (to, i); + ira_object_t to_obj = find_object_anyway (to, OBJECT_START (from_obj), + OBJECT_NREGS (from_obj)); live_range_t lr = OBJECT_LIVE_RANGES (from_obj); if (internal_flag_ira_verbose > 4 && ira_dump_file != NULL) { fprintf (ira_dump_file, - " Moving ranges of a%dr%d to a%dr%d: ", + " Moving ranges of a%dr%d_obj%d to a%dr%d_obj%d: ", ALLOCNO_NUM (from), ALLOCNO_REGNO (from), - ALLOCNO_NUM (to), ALLOCNO_REGNO (to)); + OBJECT_INDEX (from_obj), ALLOCNO_NUM (to), + ALLOCNO_REGNO (to), OBJECT_INDEX (to_obj)); ira_print_live_range_list (ira_dump_file, lr); } change_object_in_range_list (lr, to_obj); @@ -2166,12 +2344,11 @@ copy_allocno_live_ranges (ira_allocno_t from, ira_allocno_t to) int i; int n = ALLOCNO_NUM_OBJECTS (from); - gcc_assert (n == ALLOCNO_NUM_OBJECTS (to)); - for (i = 0; i < n; i++) { ira_object_t from_obj = ALLOCNO_OBJECT (from, i); - ira_object_t to_obj = ALLOCNO_OBJECT (to, i); + ira_object_t to_obj = find_object_anyway (to, OBJECT_START (from_obj), + OBJECT_NREGS (from_obj)); live_range_t lr = OBJECT_LIVE_RANGES (from_obj); if (internal_flag_ira_verbose > 4 && ira_dump_file != NULL) @@ -2783,15 +2960,17 @@ setup_min_max_allocno_live_range_point (void) ira_assert (OBJECT_LIVE_RANGES (obj) == NULL); OBJECT_MAX (obj) = 0; OBJECT_MIN (obj) = 1; - continue; } ira_assert (ALLOCNO_CAP_MEMBER (a) == NULL); /* Accumulation of range info. */ if (ALLOCNO_CAP (a) != NULL) { - for (cap = ALLOCNO_CAP (a); cap != NULL; cap = ALLOCNO_CAP (cap)) + for (cap = ALLOCNO_CAP (a); cap != NULL; + cap = ALLOCNO_CAP (cap)) { - ira_object_t cap_obj = ALLOCNO_OBJECT (cap, j); + ira_object_t cap_obj = find_object (cap, OBJECT_START (obj), + OBJECT_NREGS (obj)); + gcc_assert (cap_obj != NULL); if (OBJECT_MAX (cap_obj) < OBJECT_MAX (obj)) OBJECT_MAX (cap_obj) = OBJECT_MAX (obj); if (OBJECT_MIN (cap_obj) > OBJECT_MIN (obj)) @@ -2802,7 +2981,9 @@ setup_min_max_allocno_live_range_point (void) if ((parent = ALLOCNO_LOOP_TREE_NODE (a)->parent) == NULL) continue; parent_a = parent->regno_allocno_map[i]; - parent_obj = ALLOCNO_OBJECT (parent_a, j); + parent_obj + = find_object (parent_a, OBJECT_START (obj), OBJECT_NREGS (obj)); + gcc_assert (parent_obj != NULL); if (OBJECT_MAX (parent_obj) < OBJECT_MAX (obj)) OBJECT_MAX (parent_obj) = OBJECT_MAX (obj); if (OBJECT_MIN (parent_obj) > OBJECT_MIN (obj)) diff --git a/gcc/ira-color.cc b/gcc/ira-color.cc index 4aa3e316282..8aed25144b9 100644 --- a/gcc/ira-color.cc +++ b/gcc/ira-color.cc @@ -19,6 +19,7 @@ along with GCC; see the file COPYING3. If not see . */ #include "config.h" +#define INCLUDE_MAP #include "system.h" #include "coretypes.h" #include "backend.h" @@ -852,18 +853,17 @@ setup_left_conflict_sizes_p (ira_allocno_t a) node_preorder_num = node->preorder_num; node_set = node->hard_regs->set; node_check_tick++; + /* Collect conflict objects. */ + std::map allocno_conflict_regs; for (k = 0; k < nobj; k++) { ira_object_t obj = ALLOCNO_OBJECT (a, k); ira_object_t conflict_obj; ira_object_conflict_iterator oci; - + FOR_EACH_OBJECT_CONFLICT (obj, conflict_obj, oci) { - int size; - ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); - allocno_hard_regs_node_t conflict_node, temp_node; - HARD_REG_SET conflict_node_set; + ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); allocno_color_data_t conflict_data; conflict_data = ALLOCNO_COLOR_DATA (conflict_a); @@ -872,6 +872,24 @@ setup_left_conflict_sizes_p (ira_allocno_t a) conflict_data ->profitable_hard_regs)) continue; + int num = ALLOCNO_NUM (conflict_a); + if (allocno_conflict_regs.count (num) == 0) + allocno_conflict_regs.insert ({num, ira_allocate_bitmap ()}); + bitmap_head temp; + bitmap_initialize (&temp, ®_obstack); + bitmap_set_range (&temp, OBJECT_START (conflict_obj), + OBJECT_NREGS (conflict_obj)); + bitmap_and_compl_into (&temp, allocno_conflict_regs.at (num)); + int size = bitmap_count_bits (&temp); + bitmap_clear (&temp); + if (size == 0) + continue; + + bitmap_set_range (allocno_conflict_regs.at (num), + OBJECT_START (conflict_obj), + OBJECT_NREGS (conflict_obj)); + allocno_hard_regs_node_t conflict_node, temp_node; + HARD_REG_SET conflict_node_set; conflict_node = conflict_data->hard_regs_node; conflict_node_set = conflict_node->hard_regs->set; if (hard_reg_set_subset_p (node_set, conflict_node_set)) @@ -886,14 +904,13 @@ setup_left_conflict_sizes_p (ira_allocno_t a) temp_node->check = node_check_tick; temp_node->conflict_size = 0; } - size = (ira_reg_class_max_nregs - [ALLOCNO_CLASS (conflict_a)][ALLOCNO_MODE (conflict_a)]); - if (ALLOCNO_NUM_OBJECTS (conflict_a) > 1) - /* We will deal with the subwords individually. */ - size = 1; temp_node->conflict_size += size; } } + /* Setup conflict nregs of ALLOCNO. */ + for (auto &kv : allocno_conflict_regs) + ira_free_bitmap (kv.second); + for (i = 0; i < data->hard_regs_subnodes_num; i++) { allocno_hard_regs_node_t temp_node; @@ -1031,7 +1048,7 @@ static void setup_profitable_hard_regs (void) { unsigned int i; - int j, k, nobj, hard_regno, nregs, class_size; + int j, k, nobj, hard_regno, class_size; ira_allocno_t a; bitmap_iterator bi; enum reg_class aclass; @@ -1076,7 +1093,6 @@ setup_profitable_hard_regs (void) || (hard_regno = ALLOCNO_HARD_REGNO (a)) < 0) continue; mode = ALLOCNO_MODE (a); - nregs = hard_regno_nregs (hard_regno, mode); nobj = ALLOCNO_NUM_OBJECTS (a); for (k = 0; k < nobj; k++) { @@ -1088,24 +1104,39 @@ setup_profitable_hard_regs (void) { ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); - /* We can process the conflict allocno repeatedly with - the same result. */ - if (nregs == nobj && nregs > 1) + if (!has_subreg_object_p (a)) { - int num = OBJECT_SUBWORD (conflict_obj); - - if (REG_WORDS_BIG_ENDIAN) - CLEAR_HARD_REG_BIT - (ALLOCNO_COLOR_DATA (conflict_a)->profitable_hard_regs, - hard_regno + nobj - num - 1); - else - CLEAR_HARD_REG_BIT - (ALLOCNO_COLOR_DATA (conflict_a)->profitable_hard_regs, - hard_regno + num); + ALLOCNO_COLOR_DATA (conflict_a)->profitable_hard_regs + &= ~ira_reg_mode_hard_regset[hard_regno][mode]; + continue; + } + + /* Clear all hard regs occupied by obj. */ + if (REG_WORDS_BIG_ENDIAN) + { + int start_regno + = hard_regno + ALLOCNO_NREGS (a) - 1 - OBJECT_START (obj); + for (int i = 0; i < OBJECT_NREGS (obj); i += 1) + { + int regno = start_regno - i; + if (regno >= 0 && regno < FIRST_PSEUDO_REGISTER) + CLEAR_HARD_REG_BIT ( + ALLOCNO_COLOR_DATA (conflict_a)->profitable_hard_regs, + regno); + } } else - ALLOCNO_COLOR_DATA (conflict_a)->profitable_hard_regs - &= ~ira_reg_mode_hard_regset[hard_regno][mode]; + { + int start_regno = hard_regno + OBJECT_START (obj); + for (int i = 0; i < OBJECT_NREGS (obj); i += 1) + { + int regno = start_regno + i; + if (regno >= 0 && regno < FIRST_PSEUDO_REGISTER) + CLEAR_HARD_REG_BIT ( + ALLOCNO_COLOR_DATA (conflict_a)->profitable_hard_regs, + regno); + } + } } } } @@ -1677,18 +1708,25 @@ update_conflict_hard_regno_costs (int *costs, enum reg_class aclass, aligned. */ static inline void get_conflict_and_start_profitable_regs (ira_allocno_t a, bool retry_p, - HARD_REG_SET *conflict_regs, + HARD_REG_SET *start_conflict_regs, HARD_REG_SET *start_profitable_regs) { int i, nwords; ira_object_t obj; nwords = ALLOCNO_NUM_OBJECTS (a); - for (i = 0; i < nwords; i++) - { - obj = ALLOCNO_OBJECT (a, i); - conflict_regs[i] = OBJECT_TOTAL_CONFLICT_HARD_REGS (obj); - } + CLEAR_HARD_REG_SET (*start_conflict_regs); + if (has_subreg_object_p (a)) + for (i = 0; i < nwords; i++) + { + obj = ALLOCNO_OBJECT (a, i); + for (int j = 0; j < OBJECT_NREGS (obj); j += 1) + *start_conflict_regs |= OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) + >> (OBJECT_START (obj) + j); + } + else + *start_conflict_regs + = OBJECT_TOTAL_CONFLICT_HARD_REGS (get_full_object (a)); if (retry_p) *start_profitable_regs = (reg_class_contents[ALLOCNO_CLASS (a)] @@ -1702,9 +1740,9 @@ get_conflict_and_start_profitable_regs (ira_allocno_t a, bool retry_p, PROFITABLE_REGS and whose objects have CONFLICT_REGS. */ static inline bool check_hard_reg_p (ira_allocno_t a, int hard_regno, - HARD_REG_SET *conflict_regs, HARD_REG_SET profitable_regs) + HARD_REG_SET start_conflict_regs, + HARD_REG_SET profitable_regs) { - int j, nwords, nregs; enum reg_class aclass; machine_mode mode; @@ -1716,28 +1754,17 @@ check_hard_reg_p (ira_allocno_t a, int hard_regno, /* Checking only profitable hard regs. */ if (! TEST_HARD_REG_BIT (profitable_regs, hard_regno)) return false; - nregs = hard_regno_nregs (hard_regno, mode); - nwords = ALLOCNO_NUM_OBJECTS (a); - for (j = 0; j < nregs; j++) + + if (has_subreg_object_p (a)) + return !TEST_HARD_REG_BIT (start_conflict_regs, hard_regno); + else { - int k; - int set_to_test_start = 0, set_to_test_end = nwords; - - if (nregs == nwords) - { - if (REG_WORDS_BIG_ENDIAN) - set_to_test_start = nwords - j - 1; - else - set_to_test_start = j; - set_to_test_end = set_to_test_start + 1; - } - for (k = set_to_test_start; k < set_to_test_end; k++) - if (TEST_HARD_REG_BIT (conflict_regs[k], hard_regno + j)) - break; - if (k != set_to_test_end) - break; + int nregs = hard_regno_nregs (hard_regno, mode); + for (int i = 0; i < nregs; i += 1) + if (TEST_HARD_REG_BIT (start_conflict_regs, hard_regno + i)) + return false; + return true; } - return j == nregs; } /* Return number of registers needed to be saved and restored at @@ -1945,7 +1972,7 @@ spill_soft_conflicts (ira_allocno_t a, bitmap allocnos_to_spill, static bool assign_hard_reg (ira_allocno_t a, bool retry_p) { - HARD_REG_SET conflicting_regs[2], profitable_hard_regs; + HARD_REG_SET start_conflicting_regs, profitable_hard_regs; int i, j, hard_regno, best_hard_regno, class_size; int cost, mem_cost, min_cost, full_cost, min_full_cost, nwords, word; int *a_costs; @@ -1962,8 +1989,7 @@ assign_hard_reg (ira_allocno_t a, bool retry_p) HARD_REG_SET soft_conflict_regs = {}; ira_assert (! ALLOCNO_ASSIGNED_P (a)); - get_conflict_and_start_profitable_regs (a, retry_p, - conflicting_regs, + get_conflict_and_start_profitable_regs (a, retry_p, &start_conflicting_regs, &profitable_hard_regs); aclass = ALLOCNO_CLASS (a); class_size = ira_class_hard_regs_num[aclass]; @@ -2041,7 +2067,6 @@ assign_hard_reg (ira_allocno_t a, bool retry_p) (hard_regno, ALLOCNO_MODE (conflict_a), reg_class_contents[aclass]))) { - int n_objects = ALLOCNO_NUM_OBJECTS (conflict_a); int conflict_nregs; mode = ALLOCNO_MODE (conflict_a); @@ -2076,24 +2101,95 @@ assign_hard_reg (ira_allocno_t a, bool retry_p) note_conflict (r); } } + else if (has_subreg_object_p (a)) + { + /* Set start_conflicting_regs if that cause obj and + conflict_obj overlap. the overlap position: + +--------------+ + | conflict_obj | + +--------------+ + + +-----------+ +-----------+ + | obj | ... | obj | + +-----------+ +-----------+ + + Point: A B C + + the hard regs from A to C point will cause overlap. + For REG_WORDS_BIG_ENDIAN: + A = hard_regno + ALLOCNO_NREGS (conflict_a) - 1 + - OBJECT_START (conflict_obj) + - OBJECT_NREGS (obj) + 1 + C = A + OBJECT_NREGS (obj) + + OBJECT_NREGS (conflict_obj) - 2 + For !REG_WORDS_BIG_ENDIAN: + A = hard_regno + OBJECT_START (conflict_obj) + - OBJECT_NREGS (obj) + 1 + C = A + OBJECT_NREGS (obj) + + OBJECT_NREGS (conflict_obj) - 2 + */ + int start_regno; + int conflict_allocno_nregs, conflict_object_nregs, + conflict_object_start; + if (has_subreg_object_p (conflict_a)) + { + conflict_allocno_nregs = ALLOCNO_NREGS (conflict_a); + conflict_object_nregs = OBJECT_NREGS (conflict_obj); + conflict_object_start = OBJECT_START (conflict_obj); + } + else + { + conflict_allocno_nregs = conflict_object_nregs + = hard_regno_nregs (hard_regno, mode); + conflict_object_start = 0; + } + if (REG_WORDS_BIG_ENDIAN) + { + int A = hard_regno + conflict_allocno_nregs - 1 + - conflict_object_start - OBJECT_NREGS (obj) + + 1; + start_regno = A + OBJECT_NREGS (obj) - 1 + + OBJECT_START (obj) - ALLOCNO_NREGS (a) + + 1; + } + else + { + int A = hard_regno + conflict_object_start + - OBJECT_NREGS (obj) + 1; + start_regno = A - OBJECT_START (obj); + } + + for (int i = 0; + i <= OBJECT_NREGS (obj) + conflict_object_nregs - 2; + i += 1) + { + int regno = start_regno + i; + if (regno >= 0 && regno < FIRST_PSEUDO_REGISTER) + SET_HARD_REG_BIT (start_conflicting_regs, regno); + } + if (hard_reg_set_subset_p (profitable_hard_regs, + start_conflicting_regs)) + goto fail; + } else { - if (conflict_nregs == n_objects && conflict_nregs > 1) + if (has_subreg_object_p (conflict_a)) { - int num = OBJECT_SUBWORD (conflict_obj); - - if (REG_WORDS_BIG_ENDIAN) - SET_HARD_REG_BIT (conflicting_regs[word], - hard_regno + n_objects - num - 1); - else - SET_HARD_REG_BIT (conflicting_regs[word], - hard_regno + num); + int start_hard_regno + = REG_WORDS_BIG_ENDIAN + ? hard_regno + ALLOCNO_NREGS (conflict_a) + - OBJECT_START (conflict_obj) + : hard_regno + OBJECT_START (conflict_obj); + for (int i = 0; i < OBJECT_NREGS (conflict_obj); + i += 1) + SET_HARD_REG_BIT (start_conflicting_regs, + start_hard_regno + i); } else - conflicting_regs[word] + start_conflicting_regs |= ira_reg_mode_hard_regset[hard_regno][mode]; if (hard_reg_set_subset_p (profitable_hard_regs, - conflicting_regs[word])) + start_conflicting_regs)) goto fail; } } @@ -2160,8 +2256,8 @@ assign_hard_reg (ira_allocno_t a, bool retry_p) && FIRST_STACK_REG <= hard_regno && hard_regno <= LAST_STACK_REG) continue; #endif - if (! check_hard_reg_p (a, hard_regno, - conflicting_regs, profitable_hard_regs)) + if (!check_hard_reg_p (a, hard_regno, start_conflicting_regs, + profitable_hard_regs)) continue; cost = costs[i]; full_cost = full_costs[i]; @@ -2667,21 +2763,16 @@ push_allocno_to_stack (ira_allocno_t a) { enum reg_class aclass; allocno_color_data_t data, conflict_data; - int size, i, n = ALLOCNO_NUM_OBJECTS (a); - + int i, n = ALLOCNO_NUM_OBJECTS (a); + data = ALLOCNO_COLOR_DATA (a); data->in_graph_p = false; allocno_stack_vec.safe_push (a); aclass = ALLOCNO_CLASS (a); if (aclass == NO_REGS) return; - size = ira_reg_class_max_nregs[aclass][ALLOCNO_MODE (a)]; - if (n > 1) - { - /* We will deal with the subwords individually. */ - gcc_assert (size == ALLOCNO_NUM_OBJECTS (a)); - size = 1; - } + /* Already collect conflict objects. */ + std::map allocno_conflict_regs; for (i = 0; i < n; i++) { ira_object_t obj = ALLOCNO_OBJECT (a, i); @@ -2706,6 +2797,21 @@ push_allocno_to_stack (ira_allocno_t a) continue; ira_assert (bitmap_bit_p (coloring_allocno_bitmap, ALLOCNO_NUM (conflict_a))); + + int num = ALLOCNO_NUM (conflict_a); + if (allocno_conflict_regs.count (num) == 0) + allocno_conflict_regs.insert ({num, ira_allocate_bitmap ()}); + bitmap_head temp; + bitmap_initialize (&temp, ®_obstack); + bitmap_set_range (&temp, OBJECT_START (obj), OBJECT_NREGS (obj)); + bitmap_and_compl_into (&temp, allocno_conflict_regs.at (num)); + int size = bitmap_count_bits (&temp); + bitmap_clear (&temp); + if (size == 0) + continue; + + bitmap_set_range (allocno_conflict_regs.at (num), OBJECT_START (obj), + OBJECT_NREGS (obj)); if (update_left_conflict_sizes_p (conflict_a, a, size)) { delete_allocno_from_bucket @@ -2721,6 +2827,9 @@ push_allocno_to_stack (ira_allocno_t a) } } + + for (auto &kv : allocno_conflict_regs) + ira_free_bitmap (kv.second); } /* Put ALLOCNO onto the coloring stack and remove it from its bucket. @@ -3154,7 +3263,7 @@ improve_allocation (void) machine_mode mode; int *allocno_costs; int costs[FIRST_PSEUDO_REGISTER]; - HARD_REG_SET conflicting_regs[2], profitable_hard_regs; + HARD_REG_SET start_conflicting_regs, profitable_hard_regs; ira_allocno_t a; bitmap_iterator bi; int saved_nregs; @@ -3193,7 +3302,7 @@ improve_allocation (void) - allocno_copy_cost_saving (a, hregno)); try_p = false; get_conflict_and_start_profitable_regs (a, false, - conflicting_regs, + &start_conflicting_regs, &profitable_hard_regs); class_size = ira_class_hard_regs_num[aclass]; mode = ALLOCNO_MODE (a); @@ -3202,8 +3311,8 @@ improve_allocation (void) for (j = 0; j < class_size; j++) { hregno = ira_class_hard_regs[aclass][j]; - if (! check_hard_reg_p (a, hregno, - conflicting_regs, profitable_hard_regs)) + if (!check_hard_reg_p (a, hregno, start_conflicting_regs, + profitable_hard_regs)) continue; ira_assert (ira_class_hard_reg_index[aclass][hregno] == j); k = allocno_costs == NULL ? 0 : j; @@ -3287,16 +3396,15 @@ improve_allocation (void) } conflict_nregs = hard_regno_nregs (conflict_hregno, ALLOCNO_MODE (conflict_a)); - auto note_conflict = [&](int r) - { - if (check_hard_reg_p (a, r, - conflicting_regs, profitable_hard_regs)) - { - if (spill_a) - SET_HARD_REG_BIT (soft_conflict_regs, r); - costs[r] += spill_cost; - } - }; + auto note_conflict = [&] (int r) { + if (check_hard_reg_p (a, r, start_conflicting_regs, + profitable_hard_regs)) + { + if (spill_a) + SET_HARD_REG_BIT (soft_conflict_regs, r); + costs[r] += spill_cost; + } + }; for (r = conflict_hregno; r >= 0 && (int) end_hard_regno (mode, r) > conflict_hregno; r--) @@ -3314,8 +3422,8 @@ improve_allocation (void) for (j = 0; j < class_size; j++) { hregno = ira_class_hard_regs[aclass][j]; - if (check_hard_reg_p (a, hregno, - conflicting_regs, profitable_hard_regs) + if (check_hard_reg_p (a, hregno, start_conflicting_regs, + profitable_hard_regs) && min_cost > costs[hregno]) { best = hregno; diff --git a/gcc/ira-conflicts.cc b/gcc/ira-conflicts.cc index a4d93c8d734..0585ad10043 100644 --- a/gcc/ira-conflicts.cc +++ b/gcc/ira-conflicts.cc @@ -60,23 +60,8 @@ static IRA_INT_TYPE **conflicts; static void record_object_conflict (ira_object_t obj1, ira_object_t obj2) { - ira_allocno_t a1 = OBJECT_ALLOCNO (obj1); - ira_allocno_t a2 = OBJECT_ALLOCNO (obj2); - int w1 = OBJECT_SUBWORD (obj1); - int w2 = OBJECT_SUBWORD (obj2); - int id1, id2; - - /* Canonicalize the conflict. If two identically-numbered words - conflict, always record this as a conflict between words 0. That - is the only information we need, and it is easier to test for if - it is collected in each allocno's lowest-order object. */ - if (w1 == w2 && w1 > 0) - { - obj1 = ALLOCNO_OBJECT (a1, 0); - obj2 = ALLOCNO_OBJECT (a2, 0); - } - id1 = OBJECT_CONFLICT_ID (obj1); - id2 = OBJECT_CONFLICT_ID (obj2); + int id1 = OBJECT_CONFLICT_ID (obj1); + int id2 = OBJECT_CONFLICT_ID (obj2); SET_MINMAX_SET_BIT (conflicts[id1], id2, OBJECT_MIN (obj1), OBJECT_MAX (obj1)); @@ -606,8 +591,8 @@ build_object_conflicts (ira_object_t obj) if (parent_a == NULL) return; ira_assert (ALLOCNO_CLASS (a) == ALLOCNO_CLASS (parent_a)); - ira_assert (ALLOCNO_NUM_OBJECTS (a) == ALLOCNO_NUM_OBJECTS (parent_a)); - parent_obj = ALLOCNO_OBJECT (parent_a, OBJECT_SUBWORD (obj)); + parent_obj + = find_object_anyway (parent_a, OBJECT_START (obj), OBJECT_NREGS (obj)); parent_num = OBJECT_CONFLICT_ID (parent_obj); parent_min = OBJECT_MIN (parent_obj); parent_max = OBJECT_MAX (parent_obj); @@ -616,7 +601,6 @@ build_object_conflicts (ira_object_t obj) { ira_object_t another_obj = ira_object_id_map[i]; ira_allocno_t another_a = OBJECT_ALLOCNO (another_obj); - int another_word = OBJECT_SUBWORD (another_obj); ira_assert (ira_reg_classes_intersect_p [ALLOCNO_CLASS (a)][ALLOCNO_CLASS (another_a)]); @@ -627,11 +611,11 @@ build_object_conflicts (ira_object_t obj) ira_assert (ALLOCNO_NUM (another_parent_a) >= 0); ira_assert (ALLOCNO_CLASS (another_a) == ALLOCNO_CLASS (another_parent_a)); - ira_assert (ALLOCNO_NUM_OBJECTS (another_a) - == ALLOCNO_NUM_OBJECTS (another_parent_a)); SET_MINMAX_SET_BIT (conflicts[parent_num], - OBJECT_CONFLICT_ID (ALLOCNO_OBJECT (another_parent_a, - another_word)), + OBJECT_CONFLICT_ID ( + find_object_anyway (another_parent_a, + OBJECT_START (another_obj), + OBJECT_NREGS (another_obj))), parent_min, parent_max); } } @@ -659,9 +643,10 @@ build_conflicts (void) build_object_conflicts (obj); for (cap = ALLOCNO_CAP (a); cap != NULL; cap = ALLOCNO_CAP (cap)) { - ira_object_t cap_obj = ALLOCNO_OBJECT (cap, j); - gcc_assert (ALLOCNO_NUM_OBJECTS (cap) == ALLOCNO_NUM_OBJECTS (a)); - build_object_conflicts (cap_obj); + ira_object_t cap_obj + = find_object_anyway (cap, OBJECT_START (obj), + OBJECT_NREGS (obj)); + build_object_conflicts (cap_obj); } } } @@ -736,7 +721,8 @@ print_allocno_conflicts (FILE * file, bool reg_p, ira_allocno_t a) } if (n > 1) - fprintf (file, "\n;; subobject %d:", i); + fprintf (file, "\n;; subobject s%d,n%d,f%d:", OBJECT_START (obj), + OBJECT_NREGS (obj), ALLOCNO_NREGS (a)); FOR_EACH_OBJECT_CONFLICT (obj, conflict_obj, oci) { ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); @@ -746,8 +732,10 @@ print_allocno_conflicts (FILE * file, bool reg_p, ira_allocno_t a) { fprintf (file, " a%d(r%d", ALLOCNO_NUM (conflict_a), ALLOCNO_REGNO (conflict_a)); - if (ALLOCNO_NUM_OBJECTS (conflict_a) > 1) - fprintf (file, ",w%d", OBJECT_SUBWORD (conflict_obj)); + if (has_subreg_object_p (conflict_a)) + fprintf (file, ",s%d,n%d,f%d", OBJECT_START (conflict_obj), + OBJECT_NREGS (conflict_obj), + ALLOCNO_NREGS (conflict_a)); if ((bb = ALLOCNO_LOOP_TREE_NODE (conflict_a)->bb) != NULL) fprintf (file, ",b%d", bb->index); else diff --git a/gcc/ira-emit.cc b/gcc/ira-emit.cc index 84ed482e568..9dc7f3c655e 100644 --- a/gcc/ira-emit.cc +++ b/gcc/ira-emit.cc @@ -854,7 +854,7 @@ modify_move_list (move_t list) ALLOCNO_MODE (new_allocno) = ALLOCNO_MODE (set_move->to); ira_set_allocno_class (new_allocno, ALLOCNO_CLASS (set_move->to)); - ira_create_allocno_objects (new_allocno); + ira_copy_allocno_objects (new_allocno, set_move->to); ALLOCNO_ASSIGNED_P (new_allocno) = true; ALLOCNO_HARD_REGNO (new_allocno) = -1; ALLOCNO_EMIT_DATA (new_allocno)->reg diff --git a/gcc/ira-int.h b/gcc/ira-int.h index 0685e1f4e8d..9095a8227f7 100644 --- a/gcc/ira-int.h +++ b/gcc/ira-int.h @@ -23,6 +23,8 @@ along with GCC; see the file COPYING3. If not see #include "recog.h" #include "function-abi.h" +#include +#include "subreg-live-range.h" /* To provide consistency in naming, all IRA external variables, functions, common typedefs start with prefix ira_. */ @@ -222,11 +224,13 @@ extern int ira_max_point; extern live_range_t *ira_start_point_ranges, *ira_finish_point_ranges; /* A structure representing conflict information for an allocno - (or one of its subwords). */ + (or one of its subregs). */ struct ira_object { /* The allocno associated with this record. */ ira_allocno_t allocno; + /* Index in allocno->objects array */ + unsigned int index; /* Vector of accumulated conflicting conflict_redords with NULL end marker (if OBJECT_CONFLICT_VEC_P is true) or conflict bit vector otherwise. */ @@ -236,10 +240,9 @@ struct ira_object ranges in the list are not intersected and ordered by decreasing their program points*. */ live_range_t live_ranges; - /* The subword within ALLOCNO which is represented by this object. - Zero means the lowest-order subword (or the entire allocno in case - it is not being tracked in subwords). */ - int subword; + /* Reprensent OBJECT occupied [start, start + nregs) registers of it's + ALLOCNO. */ + int start, nregs; /* Allocated size of the conflicts array. */ unsigned int conflicts_array_size; /* A unique number for every instance of this structure, which is used @@ -295,6 +298,11 @@ struct ira_allocno reload (at this point pseudo-register has only one allocno) which did not get stack slot yet. */ signed int hard_regno : 16; + /* Unit size of one register that allocate for the allocno. Only use to + compute the start and nregs of subreg which be tracked. */ + poly_int64 unit_size; + /* Flag means need track subreg live range for the allocno. */ + bool track_subreg_p; /* A bitmask of the ABIs used by calls that occur while the allocno is live. */ unsigned int crossed_calls_abis : NUM_ABI_IDS; @@ -353,8 +361,6 @@ struct ira_allocno register class living at the point than number of hard-registers of the class available for the allocation. */ int excess_pressure_points_num; - /* The number of objects tracked in the following array. */ - int num_objects; /* Accumulated frequency of calls which given allocno intersects. */ int call_freq; @@ -387,8 +393,11 @@ struct ira_allocno /* An array of structures describing conflict information and live ranges for each object associated with the allocno. There may be more than one such object in cases where the allocno represents a - multi-word register. */ - ira_object_t objects[2]; + multi-hardreg pesudo. */ + std::vector objects; + /* An array of structures decribing the subreg mode start and subreg end for + this allocno. */ + std::vector subregs; /* Registers clobbered by intersected calls. */ HARD_REG_SET crossed_calls_clobbered_regs; /* Array of usage costs (accumulated and the one updated during @@ -468,8 +477,12 @@ struct ira_allocno #define ALLOCNO_EXCESS_PRESSURE_POINTS_NUM(A) \ ((A)->excess_pressure_points_num) #define ALLOCNO_OBJECT(A,N) ((A)->objects[N]) -#define ALLOCNO_NUM_OBJECTS(A) ((A)->num_objects) +#define ALLOCNO_NUM_OBJECTS(A) ((int) (A)->objects.size ()) #define ALLOCNO_ADD_DATA(A) ((A)->add_data) +#define ALLOCNO_UNIT_SIZE(A) ((A)->unit_size) +#define ALLOCNO_TRACK_SUBREG_P(A) ((A)->track_subreg_p) +#define ALLOCNO_NREGS(A) \ + (ira_reg_class_max_nregs[ALLOCNO_CLASS (A)][ALLOCNO_MODE (A)]) /* Typedef for pointer to the subsequent structure. */ typedef struct ira_emit_data *ira_emit_data_t; @@ -511,7 +524,7 @@ allocno_emit_reg (ira_allocno_t a) } #define OBJECT_ALLOCNO(O) ((O)->allocno) -#define OBJECT_SUBWORD(O) ((O)->subword) +#define OBJECT_INDEX(O) ((O)->index) #define OBJECT_CONFLICT_ARRAY(O) ((O)->conflicts_array) #define OBJECT_CONFLICT_VEC(O) ((ira_object_t *)(O)->conflicts_array) #define OBJECT_CONFLICT_BITVEC(O) ((IRA_INT_TYPE *)(O)->conflicts_array) @@ -524,6 +537,8 @@ allocno_emit_reg (ira_allocno_t a) #define OBJECT_MAX(O) ((O)->max) #define OBJECT_CONFLICT_ID(O) ((O)->id) #define OBJECT_LIVE_RANGES(O) ((O)->live_ranges) +#define OBJECT_START(O) ((O)->start) +#define OBJECT_NREGS(O) ((O)->nregs) /* Map regno -> allocnos with given regno (see comments for allocno member `next_regno_allocno'). */ @@ -1041,6 +1056,12 @@ extern void ira_free_cost_vector (int *, reg_class_t); extern void ira_flattening (int, int); extern bool ira_build (void); extern void ira_destroy (void); +extern ira_object_t +find_object (ira_allocno_t, int, int); +extern ira_object_t find_object (ira_allocno_t, poly_int64, poly_int64); +ira_object_t +find_object_anyway (ira_allocno_t a, int start, int nregs); +extern void ira_copy_allocno_objects (ira_allocno_t, ira_allocno_t); /* ira-costs.cc */ extern void ira_init_costs_once (void); @@ -1708,4 +1729,18 @@ ira_caller_save_loop_spill_p (ira_allocno_t a, ira_allocno_t subloop_a, return call_cost && call_cost >= spill_cost; } +/* Return true if allocno A has subreg object. */ +inline bool +has_subreg_object_p (ira_allocno_t a) +{ + return ALLOCNO_NUM_OBJECTS (a) > 1; +} + +/* Return the full object of allocno A. */ +inline ira_object_t +get_full_object (ira_allocno_t a) +{ + return find_object (a, 0, ALLOCNO_NREGS (a)); +} + #endif /* GCC_IRA_INT_H */ diff --git a/gcc/ira-lives.cc b/gcc/ira-lives.cc index 05e2be12a26..9ca9e5548da 100644 --- a/gcc/ira-lives.cc +++ b/gcc/ira-lives.cc @@ -19,6 +19,7 @@ along with GCC; see the file COPYING3. If not see . */ #include "config.h" +#define INCLUDE_VECTOR #include "system.h" #include "coretypes.h" #include "backend.h" @@ -35,6 +36,7 @@ along with GCC; see the file COPYING3. If not see #include "sparseset.h" #include "function-abi.h" #include "except.h" +#include "subreg-live-range.h" /* The code in this file is similar to one in global but the code works on the allocno basis and creates live ranges instead of @@ -91,6 +93,9 @@ static alternative_mask preferred_alternatives; we should not add a conflict with the copy's destination operand. */ static rtx ignore_reg_for_conflicts; +/* Store def/use point of has_subreg_object_p register. */ +static class subregs_live_points *subreg_live_points; + /* Record hard register REGNO as now being live. */ static void make_hard_regno_live (int regno) @@ -98,6 +103,33 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); } +/* Update conflict hard regs of ALLOCNO a for current live part. */ +static void +add_onflict_hard_regs (ira_allocno_t a, HARD_REG_SET regs) +{ + gcc_assert (has_subreg_object_p (a)); + + if (subreg_live_points->subreg_live_ranges.count (ALLOCNO_NUM (a)) == 0) + return; + + for (const subreg_range &r : + subreg_live_points->subreg_live_ranges.at (ALLOCNO_NUM (a)).ranges) + { + ira_object_t obj = find_object_anyway (a, r.start, r.end - r.start); + OBJECT_CONFLICT_HARD_REGS (obj) |= regs; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= regs; + } +} + +static void +add_onflict_hard_reg (ira_allocno_t a, unsigned int regno) +{ + HARD_REG_SET set; + CLEAR_HARD_REG_SET (set); + SET_HARD_REG_BIT (set, regno); + add_onflict_hard_regs (a, set); +} + /* Process the definition of hard register REGNO. This updates hard_regs_live and hard reg conflict information for living allocnos. */ static void @@ -113,8 +145,13 @@ make_hard_regno_dead (int regno) == (unsigned int) ALLOCNO_REGNO (OBJECT_ALLOCNO (obj))) continue; - SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); - SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); + if (has_subreg_object_p (OBJECT_ALLOCNO (obj))) + add_onflict_hard_reg (OBJECT_ALLOCNO (obj), regno); + else + { + SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); + SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); + } } CLEAR_HARD_REG_BIT (hard_regs_live, regno); } @@ -127,9 +164,29 @@ make_object_live (ira_object_t obj) sparseset_set_bit (objects_live, OBJECT_CONFLICT_ID (obj)); live_range_t lr = OBJECT_LIVE_RANGES (obj); - if (lr == NULL - || (lr->finish != curr_point && lr->finish + 1 != curr_point)) - ira_add_live_range_to_object (obj, curr_point, -1); + if (lr == NULL || (lr->finish != curr_point && lr->finish + 1 != curr_point)) + { + ira_add_live_range_to_object (obj, curr_point, -1); + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) + { + fprintf (ira_dump_file, + " add new live_range for a%d(r%d): [%d...-1]\n", + ALLOCNO_NUM (OBJECT_ALLOCNO (obj)), + ALLOCNO_REGNO (OBJECT_ALLOCNO (obj)), curr_point); + } + } + else + { + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) + { + fprintf ( + ira_dump_file, + " use old live_range for a%d(r%d): [%d...%d], curr: %d\n", + ALLOCNO_NUM (OBJECT_ALLOCNO (obj)), + ALLOCNO_REGNO (OBJECT_ALLOCNO (obj)), lr->start, lr->finish, + curr_point); + } + } } /* Update ALLOCNO_EXCESS_PRESSURE_POINTS_NUM for the allocno @@ -140,7 +197,6 @@ update_allocno_pressure_excess_length (ira_object_t obj) ira_allocno_t a = OBJECT_ALLOCNO (obj); int start, i; enum reg_class aclass, pclass, cl; - live_range_t p; aclass = ALLOCNO_CLASS (a); pclass = ira_pressure_class_translate[aclass]; @@ -152,10 +208,18 @@ update_allocno_pressure_excess_length (ira_object_t obj) continue; if (high_pressure_start_point[cl] < 0) continue; - p = OBJECT_LIVE_RANGES (obj); - ira_assert (p != NULL); - start = (high_pressure_start_point[cl] > p->start - ? high_pressure_start_point[cl] : p->start); + int start_point; + if (has_subreg_object_p (a)) + start_point = subreg_live_points->get_start_point (ALLOCNO_NUM (a)); + else + { + live_range_t p = OBJECT_LIVE_RANGES (obj); + ira_assert (p != NULL); + start_point = p->start; + } + start = (high_pressure_start_point[cl] > start_point + ? high_pressure_start_point[cl] + : start_point); ALLOCNO_EXCESS_PRESSURE_POINTS_NUM (a) += curr_point - start + 1; } } @@ -201,6 +265,14 @@ make_object_dead (ira_object_t obj) CLEAR_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); lr = OBJECT_LIVE_RANGES (obj); + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) + { + fprintf (ira_dump_file, + " finish a live_range a%d(r%d): [%d...%d] => [%d...%d]\n", + ALLOCNO_NUM (OBJECT_ALLOCNO (obj)), + ALLOCNO_REGNO (OBJECT_ALLOCNO (obj)), lr->start, lr->finish, + lr->start, curr_point); + } ira_assert (lr != NULL); lr->finish = curr_point; update_allocno_pressure_excess_length (obj); @@ -295,77 +367,144 @@ pseudo_regno_single_word_and_live_p (int regno) return sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj)); } -/* Mark the pseudo register REGNO as live. Update all information about - live ranges and register pressure. */ +/* Collect the point which the OBJ be def/use. */ static void -mark_pseudo_regno_live (int regno) +add_subreg_point (ira_object_t obj, bool is_def, bool is_dec = true) { - ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - enum reg_class pclass; - int i, n, nregs; - - if (a == NULL) - return; + ira_allocno_t a = OBJECT_ALLOCNO (obj); + if (is_def) + { + OBJECT_CONFLICT_HARD_REGS (obj) |= hard_regs_live; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= hard_regs_live; + if (is_dec) + { + enum reg_class pclass + = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + dec_register_pressure (pclass, ALLOCNO_NREGS (a)); + } + update_allocno_pressure_excess_length (obj); + } + else + { + enum reg_class pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + inc_register_pressure (pclass, ALLOCNO_NREGS (a)); + } - /* Invalidate because it is referenced. */ - allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + subreg_range r = subreg_range ( + {OBJECT_START (obj), OBJECT_START (obj) + OBJECT_NREGS (obj)}); + subreg_live_points->add_point (ALLOCNO_NUM (a), ALLOCNO_NREGS (a), r, is_def, + curr_point); - n = ALLOCNO_NUM_OBJECTS (a); - pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; - if (n > 1) + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) { - /* We track every subobject separately. */ - gcc_assert (nregs == n); - nregs = 1; + fprintf (ira_dump_file, " %s a%d(r%d", is_def ? "def" : "use", + ALLOCNO_NUM (a), ALLOCNO_REGNO (a)); + if (ALLOCNO_CLASS (a) != NO_REGS + && ALLOCNO_NREGS (a) != OBJECT_NREGS (obj)) + fprintf (ira_dump_file, " [subreg: start %d, nregs %d]", + OBJECT_START (obj), OBJECT_NREGS (obj)); + else + fprintf (ira_dump_file, " [full: nregs %d]", OBJECT_NREGS (obj)); + fprintf (ira_dump_file, ") at point %d\n", curr_point); } - for (i = 0; i < n; i++) - { - ira_object_t obj = ALLOCNO_OBJECT (a, i); + gcc_assert (has_subreg_object_p (a)); + gcc_assert (subreg_live_points->subreg_live_ranges.count (ALLOCNO_NUM (a)) + != 0); + + const subreg_ranges &sr + = subreg_live_points->subreg_live_ranges.at (ALLOCNO_NUM (a)); + ira_object_t main_obj = find_object (a, 0, ALLOCNO_NREGS (a)); + gcc_assert (main_obj != NULL); + if (sr.empty_p () + && sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (main_obj))) + sparseset_clear_bit (objects_live, OBJECT_CONFLICT_ID (main_obj)); + else if (!sr.empty_p () + && !sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (main_obj))) + sparseset_set_bit (objects_live, OBJECT_CONFLICT_ID (main_obj)); +} + +/* Mark the object OBJ as live. */ +static void +mark_pseudo_object_live (ira_allocno_t a, ira_object_t obj) +{ + /* Invalidate because it is referenced. */ + allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + if (has_subreg_object_p (a)) + add_subreg_point (obj, false); + else + { if (sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) - continue; + return; - inc_register_pressure (pclass, nregs); + enum reg_class pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + inc_register_pressure (pclass, ALLOCNO_NREGS (a)); make_object_live (obj); } } +/* Mark the pseudo register REGNO as live. Update all information about + live ranges and register pressure. */ +static void +mark_pseudo_regno_live (int regno) +{ + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; + + if (a == NULL) + return; + + int nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; + ira_object_t obj = find_object (a, 0, nregs); + gcc_assert (obj != NULL); + + mark_pseudo_object_live (a, obj); +} + /* Like mark_pseudo_regno_live, but try to only mark one subword of the pseudo as live. SUBWORD indicates which; a value of 0 indicates the low part. */ static void -mark_pseudo_regno_subword_live (int regno, int subword) +mark_pseudo_regno_subreg_live (int regno, rtx subreg) { ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - int n; - enum reg_class pclass; - ira_object_t obj; if (a == NULL) return; - /* Invalidate because it is referenced. */ - allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + ira_object_t obj + = find_object (a, SUBREG_BYTE (subreg), GET_MODE_SIZE (GET_MODE (subreg))); + gcc_assert (obj != NULL); + + mark_pseudo_object_live (a, obj); +} - n = ALLOCNO_NUM_OBJECTS (a); - if (n == 1) +/* Mark objects in subreg ranges SR as live. Update all information about + live ranges and register pressure. */ +static void +mark_pseudo_regno_subregs_live (int regno, const subreg_ranges &sr) +{ + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; + if (a == NULL) + return; + + if (!ALLOCNO_TRACK_SUBREG_P (a)) { mark_pseudo_regno_live (regno); return; } - pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - gcc_assert - (n == ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]); - obj = ALLOCNO_OBJECT (a, subword); - - if (sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) - return; - - inc_register_pressure (pclass, 1); - make_object_live (obj); + int times = sr.max / ALLOCNO_NREGS (a); + gcc_assert (sr.max >= ALLOCNO_NREGS (a) + && times * ALLOCNO_NREGS (a) == sr.max); + for (const subreg_range &range : sr.ranges) + { + int start = range.start / times; + int end = CEIL (range.end, times); + ira_object_t obj = find_object (a, start, end - start); + gcc_assert (obj != NULL); + mark_pseudo_object_live (a, obj); + } } /* Mark the register REG as live. Store a 1 in hard_regs_live for @@ -403,10 +542,7 @@ static void mark_pseudo_reg_live (rtx orig_reg, unsigned regno) { if (read_modify_subreg_p (orig_reg)) - { - mark_pseudo_regno_subword_live (regno, - subreg_lowpart_p (orig_reg) ? 0 : 1); - } + mark_pseudo_regno_subreg_live (regno, orig_reg); else mark_pseudo_regno_live (regno); } @@ -427,72 +563,59 @@ mark_ref_live (df_ref ref) mark_hard_reg_live (reg); } -/* Mark the pseudo register REGNO as dead. Update all information about - live ranges and register pressure. */ +/* Mark object as dead. */ static void -mark_pseudo_regno_dead (int regno) +mark_pseudo_object_dead (ira_allocno_t a, ira_object_t obj) { - ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - int n, i, nregs; - enum reg_class cl; - - if (a == NULL) - return; - /* Invalidate because it is referenced. */ allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; - n = ALLOCNO_NUM_OBJECTS (a); - cl = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; - if (n > 1) - { - /* We track every subobject separately. */ - gcc_assert (nregs == n); - nregs = 1; - } - for (i = 0; i < n; i++) + if (has_subreg_object_p (a)) + add_subreg_point (obj, true); + else { - ira_object_t obj = ALLOCNO_OBJECT (a, i); if (!sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) - continue; + return; - dec_register_pressure (cl, nregs); + enum reg_class cl = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + dec_register_pressure (cl, ALLOCNO_NREGS (a)); make_object_dead (obj); } } -/* Like mark_pseudo_regno_dead, but called when we know that only part of the - register dies. SUBWORD indicates which; a value of 0 indicates the low part. */ +/* Mark the pseudo register REGNO as dead. Update all information about + live ranges and register pressure. */ static void -mark_pseudo_regno_subword_dead (int regno, int subword) +mark_pseudo_regno_dead (int regno) { ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - int n; - enum reg_class cl; - ira_object_t obj; if (a == NULL) return; - /* Invalidate because it is referenced. */ - allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + int nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; + ira_object_t obj = find_object (a, 0, nregs); + gcc_assert (obj != NULL); - n = ALLOCNO_NUM_OBJECTS (a); - if (n == 1) - /* The allocno as a whole doesn't die in this case. */ - return; + mark_pseudo_object_dead (a, obj); +} - cl = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - gcc_assert - (n == ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]); +/* Like mark_pseudo_regno_dead, but called when we know that only part of the + register dies. SUBWORD indicates which; a value of 0 indicates the low part. + */ +static void +mark_pseudo_regno_subreg_dead (int regno, rtx subreg) +{ + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - obj = ALLOCNO_OBJECT (a, subword); - if (!sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) + if (a == NULL) return; - dec_register_pressure (cl, 1); - make_object_dead (obj); + ira_object_t obj + = find_object (a, SUBREG_BYTE (subreg), GET_MODE_SIZE (GET_MODE (subreg))); + gcc_assert (obj != NULL); + + mark_pseudo_object_dead (a, obj); } /* Process the definition of hard register REG. This updates hard_regs_live @@ -528,10 +651,7 @@ static void mark_pseudo_reg_dead (rtx orig_reg, unsigned regno) { if (read_modify_subreg_p (orig_reg)) - { - mark_pseudo_regno_subword_dead (regno, - subreg_lowpart_p (orig_reg) ? 0 : 1); - } + mark_pseudo_regno_subreg_dead (regno, orig_reg); else mark_pseudo_regno_dead (regno); } @@ -1059,8 +1179,15 @@ process_single_reg_class_operands (bool in_p, int freq) /* We could increase costs of A instead of making it conflicting with the hard register. But it works worse because it will be spilled in reload in anyway. */ - OBJECT_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; - OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; + if (has_subreg_object_p (a)) + add_onflict_hard_regs (OBJECT_ALLOCNO (obj), + reg_class_contents[cl]); + else + { + OBJECT_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) + |= reg_class_contents[cl]; + } } } } @@ -1198,17 +1325,15 @@ process_out_of_region_eh_regs (basic_block bb) bi) { ira_allocno_t a = ira_curr_regno_allocno_map[i]; - for (int n = ALLOCNO_NUM_OBJECTS (a) - 1; n >= 0; n--) + ira_object_t obj = find_object (a, 0, ALLOCNO_NREGS (a)); + for (int k = 0;; k++) { - ira_object_t obj = ALLOCNO_OBJECT (a, n); - for (int k = 0; ; k++) - { - unsigned int regno = EH_RETURN_DATA_REGNO (k); - if (regno == INVALID_REGNUM) - break; - SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); - SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); - } + unsigned int regno = EH_RETURN_DATA_REGNO (k); + if (regno == INVALID_REGNUM) + break; + + SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); + SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); } } } @@ -1234,8 +1359,13 @@ add_conflict_from_region_landing_pads (eh_region region, ira_object_t obj, { HARD_REG_SET new_conflict_regs = callee_abi.mode_clobbers (ALLOCNO_MODE (a)); - OBJECT_CONFLICT_HARD_REGS (obj) |= new_conflict_regs; - OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= new_conflict_regs; + if (has_subreg_object_p (a)) + add_onflict_hard_regs (a, new_conflict_regs); + else + { + OBJECT_CONFLICT_HARD_REGS (obj) |= new_conflict_regs; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= new_conflict_regs; + } return; } } @@ -1260,6 +1390,10 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) bb = loop_tree_node->bb; if (bb != NULL) { + if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL) + fprintf (ira_dump_file, "\n BB exit(l%d): point = %d\n", + loop_tree_node->parent->loop_num, curr_point); + for (i = 0; i < ira_pressure_classes_num; i++) { curr_reg_pressure[ira_pressure_classes[i]] = 0; @@ -1268,6 +1402,7 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) curr_bb_node = loop_tree_node; reg_live_out = DF_LIVE_SUBREG_OUT (bb); sparseset_clear (objects_live); + subreg_live_points->clear_live_ranges (); REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_out); hard_regs_live &= ~(eliminable_regset | ira_no_alloc_regs); for (i = 0; i < FIRST_PSEUDO_REGISTER; i++) @@ -1291,9 +1426,17 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) <= ira_class_hard_regs_num[cl]); } } - EXECUTE_IF_SET_IN_BITMAP (reg_live_out, FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_FULL_OUT (bb), + FIRST_PSEUDO_REGISTER, j, bi) mark_pseudo_regno_live (j); + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_PARTIAL_OUT (bb), + FIRST_PSEUDO_REGISTER, j, bi) + { + mark_pseudo_regno_subregs_live ( + j, DF_LIVE_SUBREG_RANGE_OUT (bb)->lives.at (j)); + } + #ifdef EH_RETURN_DATA_REGNO process_out_of_region_eh_regs (bb); #endif @@ -1408,8 +1551,18 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) && (find_reg_note (insn, REG_SETJMP, NULL_RTX) != NULL_RTX))) { - SET_HARD_REG_SET (OBJECT_CONFLICT_HARD_REGS (obj)); - SET_HARD_REG_SET (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj)); + if (has_subreg_object_p (a)) + { + HARD_REG_SET regs; + SET_HARD_REG_SET (regs); + add_onflict_hard_regs (a, regs); + } + else + { + SET_HARD_REG_SET (OBJECT_CONFLICT_HARD_REGS (obj)); + SET_HARD_REG_SET ( + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj)); + } } eh_region r; if (can_throw_internal (insn) @@ -1455,7 +1608,14 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) /* Mark each used value as live. */ FOR_EACH_INSN_USE (use, insn) - mark_ref_live (use); + { + unsigned regno = DF_REF_REGNO (use); + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; + if (a && has_subreg_object_p (a) + && DF_REF_FLAGS (use) & (DF_REF_READ_WRITE | DF_REF_SUBREG)) + continue; + mark_ref_live (use); + } process_single_reg_class_operands (true, freq); @@ -1485,6 +1645,10 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) } ignore_reg_for_conflicts = NULL_RTX; + if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL) + fprintf (ira_dump_file, "\n BB head(l%d): point = %d\n", + loop_tree_node->parent->loop_num, curr_point); + if (bb_has_eh_pred (bb)) for (j = 0; ; ++j) { @@ -1538,10 +1702,15 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) } EXECUTE_IF_SET_IN_SPARSESET (objects_live, i) - make_object_dead (ira_object_id_map[i]); + { + ira_object_t obj = ira_object_id_map[i]; + if (has_subreg_object_p (OBJECT_ALLOCNO (obj))) + add_subreg_point (obj, true, false); + else + make_object_dead (obj); + } curr_point++; - } /* Propagate register pressure to upper loop tree nodes. */ if (loop_tree_node != ira_loop_tree_root) @@ -1742,6 +1911,86 @@ ira_debug_live_ranges (void) print_live_ranges (stderr); } +class subreg_live_item +{ +public: + subreg_ranges subreg; + int start, finish; +}; + +/* Create subreg live ranges from objects def/use point info. */ +static void +create_subregs_live_ranges () +{ + for (const auto &subreg_point_it : subreg_live_points->subreg_points) + { + unsigned int allocno_num = subreg_point_it.first; + const class live_points &points = subreg_point_it.second; + ira_allocno_t a = ira_allocnos[allocno_num]; + std::vector temps; + gcc_assert (has_subreg_object_p (a)); + for (const auto &point_it : points.points) + { + int point = point_it.first; + const live_point ®s = point_it.second; + gcc_assert (temps.empty () || temps.back ().finish <= point); + if (!regs.use_reg.empty_p ()) + { + if (temps.empty ()) + temps.push_back ({regs.use_reg, point, -1}); + else if (temps.back ().finish == -1) + { + if (!temps.back ().subreg.same_p (regs.use_reg)) + { + if (temps.back ().start == point) + temps.back ().subreg.add_ranges (regs.use_reg); + else + { + temps.back ().finish = point - 1; + + subreg_ranges temp = regs.use_reg; + temp.add_ranges (temps.back ().subreg); + temps.push_back ({temp, point, -1}); + } + } + } + else if (temps.back ().subreg.same_p (regs.use_reg) + && (temps.back ().finish == point + || temps.back ().finish + 1 == point)) + temps.back ().finish = -1; + else + temps.push_back ({regs.use_reg, point, -1}); + } + if (!regs.def_reg.empty_p ()) + { + gcc_assert (!temps.empty ()); + if (regs.def_reg.include_ranges_p (temps.back ().subreg)) + temps.back ().finish = point; + else if (temps.back ().subreg.include_ranges_p (regs.def_reg)) + { + temps.back ().finish = point; + + subreg_ranges diff = temps.back ().subreg; + diff.remove_ranges (regs.def_reg); + temps.push_back ({diff, point + 1, -1}); + } + else + gcc_unreachable (); + } + } + for (const subreg_live_item &item : temps) + for (const subreg_range &r : item.subreg.ranges) + { + ira_object_t obj = find_object_anyway (a, r.start, r.end - r.start); + live_range_t lr = OBJECT_LIVE_RANGES (obj); + if (lr != NULL && lr->finish + 1 == item.start) + lr->finish = item.finish; + else + ira_add_live_range_to_object (obj, item.start, item.finish); + } + } +} + /* The main entry function creates live ranges, set up CONFLICT_HARD_REGS and TOTAL_CONFLICT_HARD_REGS for objects, and calculate register pressure info. */ @@ -1755,13 +2004,20 @@ ira_create_allocno_live_ranges (void) allocno_saved_at_call = (int *) ira_allocate (ira_allocnos_num * sizeof (int)); memset (allocno_saved_at_call, 0, ira_allocnos_num * sizeof (int)); + subreg_live_points = new subregs_live_points (); ira_traverse_loop_tree (true, ira_loop_tree_root, NULL, process_bb_node_lives); ira_max_point = curr_point; + create_subregs_live_ranges (); create_start_finish_chains (); if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL) - print_live_ranges (ira_dump_file); + { + fprintf (ira_dump_file, ";; subreg live points:\n"); + subreg_live_points->dump (ira_dump_file); + print_live_ranges (ira_dump_file); + } /* Clean up. */ + delete subreg_live_points; ira_free (allocno_saved_at_call); sparseset_free (objects_live); sparseset_free (allocnos_processed); diff --git a/gcc/ira.cc b/gcc/ira.cc index c7f27b17002..9ea57d3b1ea 100644 --- a/gcc/ira.cc +++ b/gcc/ira.cc @@ -2623,7 +2623,7 @@ static void check_allocation (void) { ira_allocno_t a; - int hard_regno, nregs, conflict_nregs; + int hard_regno; ira_allocno_iterator ai; FOR_EACH_ALLOCNO (a, ai) @@ -2634,28 +2634,18 @@ check_allocation (void) if (ALLOCNO_CAP_MEMBER (a) != NULL || (hard_regno = ALLOCNO_HARD_REGNO (a)) < 0) continue; - nregs = hard_regno_nregs (hard_regno, ALLOCNO_MODE (a)); - if (nregs == 1) - /* We allocated a single hard register. */ - n = 1; - else if (n > 1) - /* We allocated multiple hard registers, and we will test - conflicts in a granularity of single hard regs. */ - nregs = 1; for (i = 0; i < n; i++) { ira_object_t obj = ALLOCNO_OBJECT (a, i); ira_object_t conflict_obj; ira_object_conflict_iterator oci; - int this_regno = hard_regno; - if (n > 1) - { - if (REG_WORDS_BIG_ENDIAN) - this_regno += n - i - 1; - else - this_regno += i; - } + int this_regno; + if (REG_WORDS_BIG_ENDIAN) + this_regno = hard_regno + ALLOCNO_NREGS (a) - 1 - OBJECT_START (obj) + - OBJECT_NREGS (obj) + 1; + else + this_regno = hard_regno + OBJECT_START (obj); FOR_EACH_OBJECT_CONFLICT (obj, conflict_obj, oci) { ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); @@ -2665,24 +2655,18 @@ check_allocation (void) if (ira_soft_conflict (a, conflict_a)) continue; - conflict_nregs = hard_regno_nregs (conflict_hard_regno, - ALLOCNO_MODE (conflict_a)); - - if (ALLOCNO_NUM_OBJECTS (conflict_a) > 1 - && conflict_nregs == ALLOCNO_NUM_OBJECTS (conflict_a)) - { - if (REG_WORDS_BIG_ENDIAN) - conflict_hard_regno += (ALLOCNO_NUM_OBJECTS (conflict_a) - - OBJECT_SUBWORD (conflict_obj) - 1); - else - conflict_hard_regno += OBJECT_SUBWORD (conflict_obj); - conflict_nregs = 1; - } + if (REG_WORDS_BIG_ENDIAN) + conflict_hard_regno = conflict_hard_regno + + ALLOCNO_NREGS (conflict_a) - 1 + - OBJECT_START (conflict_obj) + - OBJECT_NREGS (conflict_obj) + 1; + else + conflict_hard_regno + = conflict_hard_regno + OBJECT_START (conflict_obj); - if ((conflict_hard_regno <= this_regno - && this_regno < conflict_hard_regno + conflict_nregs) - || (this_regno <= conflict_hard_regno - && conflict_hard_regno < this_regno + nregs)) + if (!(this_regno + OBJECT_NREGS (obj) <= conflict_hard_regno + || conflict_hard_regno + OBJECT_NREGS (conflict_obj) + <= this_regno)) { fprintf (stderr, "bad allocation for %d and %d\n", ALLOCNO_REGNO (a), ALLOCNO_REGNO (conflict_a));