From patchwork Tue Oct 18 02:57:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Takayuki 'January June' Suwa X-Patchwork-Id: 3913 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp1748067wrs; Mon, 17 Oct 2022 19:58:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6/ZfD+w0MWyVMX6onsGx5uKg0hrBj3gtsjtYtq74Es5k8LIqyWFy1+qGVdDngof8JRDrjG X-Received: by 2002:a17:907:2cf1:b0:78d:c7a1:172d with SMTP id hz17-20020a1709072cf100b0078dc7a1172dmr596056ejc.51.1666061910106; Mon, 17 Oct 2022 19:58:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666061910; cv=none; d=google.com; s=arc-20160816; b=oUmVM2e3NoewUCdbQPJwJNBOl75rmeIqaDWsKov0+w5+oprhF6rgaJa7jjxZtG0+9J gJ/YU2Qk70l43mwDikA9TXbco+XEE2l8nRgiVO8cwiInbeJjt+BOktlt4uHT8SGSQXNk jEE+HzrfRxYE5qxketvTQ1Kx2KfKinBihJghrXbXmdpHAIK/EBeBC65lz11nVMr+/44l /Zcea94zrZggoEGUZHKxtt+JBEX0LkyJ4eQX7EMMUgfCjJB5ECF+oSHL35H83Eklojzg 5528m3jeS0x7CMuRZhcB5dfMspLZyeTLm/EXzUps3nwbepaxe4IsUo0qrgahNSmqBETr 9MBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:from:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence :content-transfer-encoding:in-reply-to:references:to:subject :user-agent:mime-version:date:message-id:dmarc-filter:delivered-to :dkim-signature:dkim-filter; bh=olzDeYLWhMdyEli9O+5Gaaz6zelgrYtTu5HWI0eD+Oo=; b=GXnmdSNvOy5X2KsqowXZsBFKiIvO9rhR6kDTT7MIX6NebvoTVIPvCZSNU+IsOx7Oi8 JnAwAZN3KFdfxjV+77kOd9Qwb+HBl/8gtNgIwsPLex5ehkNf/+gdQsSsmxOex8wH7ZOG NC7qPeGMD51PaeqZpmnexDNx+Csm53ZucF6wqR9SiMr0l5CLiX6r0a94A3We2frzwR8s bSKpRWfa9Dft+b6HCQedBx8IjYqJDRvt12Y7TFb1m1cQ5K2jYYyBnt1OzceifbkvJYYS ulDyb9JpLiuhFHB6lkl5wnzqP+OzGCu2/iN9ZnfjVvVD5iGHSW12inAhNF4S5dPfyA4b qa+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=O7HRbcZe; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id e5-20020a170906844500b007836227b96fsi8362193ejy.888.2022.10.17.19.58.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Oct 2022 19:58:30 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=O7HRbcZe; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C440F3858005 for ; Tue, 18 Oct 2022 02:58:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C440F3858005 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1666061908; bh=olzDeYLWhMdyEli9O+5Gaaz6zelgrYtTu5HWI0eD+Oo=; h=Date:Subject:To:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=O7HRbcZeR62EBCkloH7GtPlKlxF3TljB9Pcp9ejRiCaVd6Sm9P8jz31JA0Do53J4e rU3kC842D0CjW+hnooTIHEOchqOlYoBX+2uRZEETdPL3dQ0b5Eq1cerRlYfuRjLBiG F3qdwQGeIXETsrSnK5wcwK07OPgZAFk5bwISyDkc= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from sonicconh6001.mail.ssk.yahoo.co.jp (sonicconh6001.mail.ssk.yahoo.co.jp [182.22.37.8]) by sourceware.org (Postfix) with ESMTPS id 864513858D32 for ; Tue, 18 Oct 2022 02:57:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 864513858D32 X-YMail-OSG: liYMYfgVM1lpEUncKsz05Z12Ou0bp67OXOoDXSnizE1QPK9e_.Ai07arRG6j.x4 ZT7rYSMuzc42Z.7EqVeSyBizhkvgD2zLMuHvwIBx7_2EJH5v.VSqdyow16rKiOnJfforik5XNITS 4BouvL7hJzJEyXdXfmgBwsnrgGqLfTAa1WOq.0g9K7kxHzFUQYdIbOvA_xyTdRyAkYfDPiDDuXsh 50tlLe2xPBEPU0IYKQkIS0bj1ch5WqWEb7bp4968Dv1CXvzDeHIr5gX2_TxXo2dcoNRjwqf7jdbQ JKyDHDi5hUfZf0oCt8MRToBtHVTEnBn.GwYUu5x6NfIuG9z9Fdi7064ccttZEVjIQQ2C6_B9XW8I g8fOFpczGxZtYCHFmS0UiH36a4C4Od6QhBTWxVDSsWCZwrkGJysOiGhwifLOtO8BLUGVINOVEvlg .bIVDki8k0lPbIcsMQEDBpJhsm2uwJu5wUcsBcVbdbpqyJR_lZoQiAV7nkZPhZ6vcCqjfgpiR6kM wIPRUsOQQa4FkPX4I614WE86ByFn6lZcNtEmh0qLXxYv78rxxWLWuJpPe.1dLe.25YheQuichjrL Yk_mNj4yU.8aCFwTwbTFGz4dxVQbbiazSuovvDMDY3gw0uEU9leuVsjKiLsKu0y0YkUvH.ice1ju cxpPV_SkMYXh_7b25hXfygZq14U5oYaZFUrx7io9iSVksJBxvcsWrDsTsT1kGeZqJRlo2feXN1LT rz00PWD4pnjlqpxv5rHwGt45q2UuEe2DPGurzOH9ySItNJMTVoOTvt7D9yJMe5sA0p_30z6qCysl pZs5X0Y7sdiHQSSFGs2cY8EUAKWEt7gMb.5DWL.IjD4bUFD5s_833eNEduSkE_LTWSwCvBuEHzkp 2RI8Aeb90Ru3kMpdMlqkZQAkrvlvyvCSAdZvX9krV8sgnFU2kXFpFrqoVDdxXW9OHW84TNYIb9ET qo34mZSI- Received: from sonicgw.mail.yahoo.co.jp by sonicconh6001.mail.ssk.yahoo.co.jp with HTTP; Tue, 18 Oct 2022 02:57:38 +0000 Received: by smtphe5001.mail.kks.ynwp.yahoo.co.jp (YJ Hermes SMTP Server) with ESMTPA ID 54baf41941ddefe00a3bea4878642569; Tue, 18 Oct 2022 11:57:33 +0900 (JST) Message-ID: Date: Tue, 18 Oct 2022 11:57:31 +0900 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.3.3 Subject: [PATCH v2] xtensa: Prepare the transition from Reload to LRA To: GCC Patches References: <7e3fe210-6dbc-fc29-dbb8-b951e89cf7e9@yahoo.co.jp> <87f124f0-8a10-6c3b-6b12-cabf855e2e4b@yahoo.co.jp> <3296b387-083a-40cf-1bb5-40269e804f52@yahoo.co.jp> In-Reply-To: X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, WEIRD_PORT autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Takayuki 'January June' Suwa via Gcc-patches From: Takayuki 'January June' Suwa Reply-To: Takayuki 'January June' Suwa Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746661695533076011?= X-GMAIL-MSGID: =?utf-8?q?1746992533796801615?= On 2022/10/16 14:03, Max Filippov wrote: > Hi Suwa-san, Hi! > This change results in a few new regressions in the following tests caused by ICE even when running without -mlra option: > > +FAIL: gcc.c-torture/execute/pr92904.c -O1 (internal compiler error: in extract_insn, at recog.cc:2791) > > The backtraces look like this in all of them: > > gcc/gcc/testsuite/gcc.c-torture/execute/pr92904.c:395:1: error: > unrecognizable insn: > (insn 10501 7 10502 2 (set (reg:SI 5913) > (const_int 1431655765 [0x55555555])) > "gcc/gcc/testsuite/gcc.c-torture/execute/pr92904.c":239:9 -1 > (nil)) > during RTL pass: subreg3 > gcc/gcc/testsuite/gcc.c-torture/execute/pr92904.c:395:1: internal compiler error: in extract_insn, at recog.cc:2791 "expand" pass generates the below from referencing to the struct: ;; MEM [(union Y *)&u] = 6148914691236517205; (set (reg:DI X) (mem:DI (symbol_ref:SI ("*.LC_u")))) and then "fwprop1" transforms it by dereference: (set (reg:DI X) (const_int 0x5555555555555555)) finally "subreg3" (but not "split1") splits it into the two that don't satisfy the constraint: (set (reg:SI X0) (const_int 0x55555555)) (set (reg:SI X1) (const_int 0x55555555)) > There's also the following runtime failures, but only on call0 configuration: > > +FAIL: gcc.c-torture/execute/20010122-1.c -O1 execution test > +FAIL: gcc.c-torture/execute/20010122-1.c -O2 execution test > +FAIL: gcc.c-torture/execute/20010122-1.c -O3 -g execution test > +FAIL: gcc.c-torture/execute/20010122-1.c -Os execution test > +FAIL: gcc.c-torture/execute/20010122-1.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test both assembler outputs with and without this patch are identical on my side, but perhaps it can break runtime init and/or libraries due to my silly mistake: -+ if (HARD_REGISTER_P (x) ++ if (! HARD_REGISTER_P (x) === This patch provides the first step in the transition from Reload to LRA in Xtensa. gcc/ChangeLog: * config/xtensa/xtensa-proto.h (xtensa_split1_finished_p, xtensa_split_DI_reg_imm): New prototypes. * config/xtensa/xtensa.cc (xtensa_split1_finished_p, xtensa_split_DI_reg_imm, xtensa_lra_p): New functions. (TARGET_LRA_P): Replace the dummy hook with xtensa_lra_p. (xt_true_regnum): Rework. * gcc/config/xtensa/xtensa.h (CALL_REALLY_USED_REGISTERS): Rename from CALL_USED_REGISTERS, and remove what correspond to FIXED_REGISTERS. * gcc/config/xtensa/constraints.md (Y): Use !xtensa_split1_finished_p() instead of can_create_pseudo_p(). * gcc/config/xtensa/predicates.md (move_operand): Ditto. * gcc/config/xtensa/xtensa.md: Add two new split patterns: - splits DImode immediate load into two SImode ones - puts out-of-constraint SImode constants into the constant pool * gcc/config/xtensa/xtensa.opt (-mlra): New target-specific option for testing purpose. --- gcc/config/xtensa/constraints.md | 2 +- gcc/config/xtensa/predicates.md | 2 +- gcc/config/xtensa/xtensa-protos.h | 2 + gcc/config/xtensa/xtensa.cc | 69 ++++++++++++++++++++++++++----- gcc/config/xtensa/xtensa.h | 6 +-- gcc/config/xtensa/xtensa.md | 36 ++++++++++++---- gcc/config/xtensa/xtensa.opt | 4 ++ 7 files changed, 98 insertions(+), 23 deletions(-) diff --git a/gcc/config/xtensa/constraints.md b/gcc/config/xtensa/constraints.md index e4c314b267c..cd200d6d15a 100644 --- a/gcc/config/xtensa/constraints.md +++ b/gcc/config/xtensa/constraints.md @@ -121,7 +121,7 @@ (ior (and (match_code "const_int,const_double,const,symbol_ref,label_ref") (match_test "TARGET_AUTO_LITPOOLS")) (and (match_code "const_int") - (match_test "can_create_pseudo_p ()")))) + (match_test "! xtensa_split1_finished_p ()")))) ;; Memory constraints. Do not use define_memory_constraint here. Doing so ;; causes reload to force some constants into the constant pool, but since diff --git a/gcc/config/xtensa/predicates.md b/gcc/config/xtensa/predicates.md index 0590c0f81a9..c11e8634dbe 100644 --- a/gcc/config/xtensa/predicates.md +++ b/gcc/config/xtensa/predicates.md @@ -149,7 +149,7 @@ (ior (and (match_code "const_int") (match_test "(GET_MODE_CLASS (mode) == MODE_INT && xtensa_simm12b (INTVAL (op))) - || can_create_pseudo_p ()")) + || ! xtensa_split1_finished_p ()")) (and (match_code "const_int,const_double,const,symbol_ref,label_ref") (match_test "(TARGET_CONST16 || TARGET_AUTO_LITPOOLS) && CONSTANT_P (op) diff --git a/gcc/config/xtensa/xtensa-protos.h b/gcc/config/xtensa/xtensa-protos.h index 459e2aac9fc..bc75ad9698a 100644 --- a/gcc/config/xtensa/xtensa-protos.h +++ b/gcc/config/xtensa/xtensa-protos.h @@ -58,6 +58,8 @@ extern char *xtensa_emit_call (int, rtx *); extern char *xtensa_emit_sibcall (int, rtx *); extern bool xtensa_tls_referenced_p (rtx); extern enum rtx_code xtensa_shlrd_which_direction (rtx, rtx); +extern bool xtensa_split1_finished_p (void); +extern void xtensa_split_DI_reg_imm (rtx *); #ifdef TREE_CODE extern void init_cumulative_args (CUMULATIVE_ARGS *, int); diff --git a/gcc/config/xtensa/xtensa.cc b/gcc/config/xtensa/xtensa.cc index 828c7642b7c..950eb5a59be 100644 --- a/gcc/config/xtensa/xtensa.cc +++ b/gcc/config/xtensa/xtensa.cc @@ -56,6 +56,7 @@ along with GCC; see the file COPYING3. If not see #include "hw-doloop.h" #include "rtl-iter.h" #include "insn-attr.h" +#include "tree-pass.h" /* This file should be included last. */ #include "target-def.h" @@ -199,6 +200,7 @@ static void xtensa_output_mi_thunk (FILE *file, tree thunk ATTRIBUTE_UNUSED, HOST_WIDE_INT delta, HOST_WIDE_INT vcall_offset, tree function); +static bool xtensa_lra_p (void); static rtx xtensa_delegitimize_address (rtx); @@ -295,7 +297,7 @@ static rtx xtensa_delegitimize_address (rtx); #define TARGET_CANNOT_FORCE_CONST_MEM xtensa_cannot_force_const_mem #undef TARGET_LRA_P -#define TARGET_LRA_P hook_bool_void_false +#define TARGET_LRA_P xtensa_lra_p #undef TARGET_LEGITIMATE_ADDRESS_P #define TARGET_LEGITIMATE_ADDRESS_P xtensa_legitimate_address_p @@ -492,21 +494,30 @@ xtensa_mask_immediate (HOST_WIDE_INT v) int xt_true_regnum (rtx x) { - if (GET_CODE (x) == REG) + if (REG_P (x)) { - if (reg_renumber - && REGNO (x) >= FIRST_PSEUDO_REGISTER - && reg_renumber[REGNO (x)] >= 0) + if (! HARD_REGISTER_P (x) + && reg_renumber + && (lra_in_progress || reg_renumber[REGNO (x)] >= 0)) return reg_renumber[REGNO (x)]; return REGNO (x); } - if (GET_CODE (x) == SUBREG) + if (SUBREG_P (x)) { int base = xt_true_regnum (SUBREG_REG (x)); - if (base >= 0 && base < FIRST_PSEUDO_REGISTER) - return base + subreg_regno_offset (REGNO (SUBREG_REG (x)), - GET_MODE (SUBREG_REG (x)), - SUBREG_BYTE (x), GET_MODE (x)); + + if (base >= 0 + && HARD_REGISTER_NUM_P (base)) + { + struct subreg_info info; + + subreg_get_info (lra_in_progress + ? (unsigned) base : REGNO (SUBREG_REG (x)), + GET_MODE (SUBREG_REG (x)), + SUBREG_BYTE (x), GET_MODE (x), &info); + if (info.representable_p) + return base + info.offset; + } } return -1; } @@ -2477,6 +2488,36 @@ xtensa_shlrd_which_direction (rtx op0, rtx op1) } +/* Return true after "split1" pass has been finished. */ + +bool +xtensa_split1_finished_p (void) +{ + return cfun && (cfun->curr_properties & PROP_rtl_split_insns); +} + + +/* Split a DImode pair of reg (operand[0]) and const_int (operand[1]) into + two SImode pairs, the low-part (operands[0] and [1]) and the high-part + (operands[2] and [3]). */ + +void +xtensa_split_DI_reg_imm (rtx *operands) +{ + rtx lowpart, highpart; + + if (WORDS_BIG_ENDIAN) + split_double (operands[1], &highpart, &lowpart); + else + split_double (operands[1], &lowpart, &highpart); + + operands[3] = highpart; + operands[2] = gen_highpart (SImode, operands[0]); + operands[1] = lowpart; + operands[0] = gen_lowpart (SImode, operands[0]); +} + + /* Implement TARGET_CANNOT_FORCE_CONST_MEM. */ static bool @@ -5119,4 +5160,12 @@ xtensa_delegitimize_address (rtx op) return op; } +/* Implement TARGET_LRA_P. */ + +static bool +xtensa_lra_p (void) +{ + return TARGET_LRA; +} + #include "gt-xtensa.h" diff --git a/gcc/config/xtensa/xtensa.h b/gcc/config/xtensa/xtensa.h index 16e3d55e896..6b60e596062 100644 --- a/gcc/config/xtensa/xtensa.h +++ b/gcc/config/xtensa/xtensa.h @@ -242,10 +242,10 @@ along with GCC; see the file COPYING3. If not see Proper values are computed in TARGET_CONDITIONAL_REGISTER_USAGE. */ -#define CALL_USED_REGISTERS \ +#define CALL_REALLY_USED_REGISTERS \ { \ - 1, 1, 4, 4, 4, 4, 4, 4, 1, 1, 1, 1, 2, 2, 2, 2, \ - 1, 1, 1, \ + 0, 0, 4, 4, 4, 4, 4, 4, 1, 1, 1, 1, 2, 2, 2, 2, \ + 0, 0, 1, \ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, \ 1, \ } diff --git a/gcc/config/xtensa/xtensa.md b/gcc/config/xtensa/xtensa.md index 608110c20bc..2e7f76ada5c 100644 --- a/gcc/config/xtensa/xtensa.md +++ b/gcc/config/xtensa/xtensa.md @@ -940,14 +940,9 @@ because of offering further optimization opportunities. */ if (register_operand (operands[0], DImode)) { - rtx lowpart, highpart; - - if (TARGET_BIG_ENDIAN) - split_double (operands[1], &highpart, &lowpart); - else - split_double (operands[1], &lowpart, &highpart); - emit_insn (gen_movsi (gen_lowpart (SImode, operands[0]), lowpart)); - emit_insn (gen_movsi (gen_highpart (SImode, operands[0]), highpart)); + xtensa_split_DI_reg_imm (operands); + emit_move_insn (operands[0], operands[1]); + emit_move_insn (operands[2], operands[3]); DONE; } @@ -981,6 +976,19 @@ } }) +(define_split + [(set (match_operand:DI 0 "register_operand") + (match_operand:DI 1 "const_int_operand"))] + "!TARGET_CONST16 && !TARGET_AUTO_LITPOOLS + && ! xtensa_split1_finished_p ()" + [(set (match_dup 0) + (match_dup 1)) + (set (match_dup 2) + (match_dup 3))] +{ + xtensa_split_DI_reg_imm (operands); +}) + ;; 32-bit Integer moves (define_expand "movsi" @@ -1017,6 +1025,18 @@ (set_attr "mode" "SI") (set_attr "length" "2,2,2,2,2,2,3,3,3,3,6,3,3,3,3,3")]) +(define_split + [(set (match_operand:SI 0 "register_operand") + (match_operand:SI 1 "const_int_operand"))] + "!TARGET_CONST16 && !TARGET_AUTO_LITPOOLS + && ! xtensa_split1_finished_p () + && ! xtensa_simm12b (INTVAL (operands[1]))" + [(set (match_dup 0) + (match_dup 1))] +{ + operands[1] = force_const_mem (SImode, operands[1]); +}) + (define_split [(set (match_operand:SI 0 "register_operand") (match_operand:SI 1 "constantpool_operand"))] diff --git a/gcc/config/xtensa/xtensa.opt b/gcc/config/xtensa/xtensa.opt index 08338e39060..00d2db4eae1 100644 --- a/gcc/config/xtensa/xtensa.opt +++ b/gcc/config/xtensa/xtensa.opt @@ -34,6 +34,10 @@ mextra-l32r-costs= Target RejectNegative Joined UInteger Var(xtensa_extra_l32r_costs) Init(0) Set extra memory access cost for L32R instruction, in clock-cycle units. +mlra +Target Mask(LRA) +Use LRA instead of reload (transitional). + mtarget-align Target Automatically align branch targets to reduce branch penalties.