From patchwork Wed Oct 26 05:52:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Wang X-Patchwork-Id: 11070 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp74715wru; Tue, 25 Oct 2022 22:53:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7l1Utv+hhFqMMekohjIQdXVEu4Ql/nKDbiVsil0xlEW9MWU1huJCSuhK7GdZ/dK3FZR7Me X-Received: by 2002:a17:906:fd82:b0:770:7e61:3707 with SMTP id xa2-20020a170906fd8200b007707e613707mr34705107ejb.143.1666763639692; Tue, 25 Oct 2022 22:53:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666763639; cv=none; d=google.com; s=arc-20160816; b=GX5F5F9pEZqu2Rz2dbiGfiC5CtbVocwz13Hwm0EIAaabK2qFLmYRK/3ADQJidJ4Slt /hwakc7CmrbL5Sj47usLEzGtX8/n6FuHW0U2uZEQJsCRHpQdgLQjThdfMk3BoJWQfhHQ Ia34nUXNWJCK1S9+nZgga+7wzuB4Vhyr9/wNxR+3BwAtYkehfclXPtBcFKCyYRJRN7Qz VzA9ebDOTi9Zp+I0HkEesa+4Ge//wws+z0F/NK5lHFGUdMY8fcxZnygtuKAbx6bIL33Q n/B0MCQRUQoArlHjWSHzt4FWtp562LM+Zwl0JirFxL6XcBv+BJtqA6XSe8E5IQm6psVT FlSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:reply-to:from:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :message-id:date:subject:to:dmarc-filter:delivered-to:dkim-signature :dkim-filter; bh=F4phL5OiFa5qV1vMN5KTc/SQuZM5fx4mSFRXCpwGg1g=; b=qZ8SA6vDmzVOVppv1BGf7Pp/ahKtkF5yqeUxvOWPoFVGtGsFu2dtVyHIjgfNfFxIlM aHu/JsZKpKxtOjoIWtMyq/TSHLtDJNOxo4BlYd4Ae9UNnbSEgsTswzx4AGcOwyz0aqMZ nDEaPQGGjtXsrJODoP+eBykWo5Avj0dViO4ICBWvDbgbf9Mfd+A7MjDG/ApjAA1rpH3F KTKFrC4gT7dc5IvfBa0IgxaOGjiK67xX+PcpsDkth6gwrUVdbeVaQ0nOP/WIXGBPnFWY C4u9je2fS9dJVW3536DXmf7b7KgeXuzSIsXJWyBGn+Woyr+l/uvV5RM2LRoJzLyuxIQk kEmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=bulOsflj; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id go15-20020a1709070d8f00b007392f9ad702si6055394ejc.741.2022.10.25.22.53.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Oct 2022 22:53:59 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=bulOsflj; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gnu.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5C5FD385E038 for ; Wed, 26 Oct 2022 05:53:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5C5FD385E038 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1666763638; bh=F4phL5OiFa5qV1vMN5KTc/SQuZM5fx4mSFRXCpwGg1g=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=bulOsfljQusad9HgqZrY8ziFw4gfoZGToGd01PhhR6ah3Y/Z0dwpO+IarZ7E8LB0m cakLjd8WdnjVclRDU9Ynhnp2koIQ4mTc4FG094VvBnJ5oGrkOF3nYntSO2CBftwygx ozYgUZO98H9jRV1RUNJIBuIyQIQwWp2rBvL55ffY= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by sourceware.org (Postfix) with ESMTPS id 7567D385843E for ; Wed, 26 Oct 2022 05:52:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7567D385843E X-IronPort-AV: E=McAfee;i="6500,9779,10511"; a="291169437" X-IronPort-AV: E=Sophos;i="5.95,213,1661842800"; d="scan'208";a="291169437" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2022 22:52:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10511"; a="665170560" X-IronPort-AV: E=Sophos;i="5.95,213,1661842800"; d="scan'208";a="665170560" Received: from shvmail03.sh.intel.com ([10.239.245.20]) by orsmga001.jf.intel.com with ESMTP; 25 Oct 2022 22:52:49 -0700 Received: from shliclel320.sh.intel.com (shliclel320.sh.intel.com [10.239.240.127]) by shvmail03.sh.intel.com (Postfix) with ESMTP id 8E37010056B1; Wed, 26 Oct 2022 13:52:48 +0800 (CST) To: gcc-patches@gcc.gnu.org Subject: [PATCH] i386: Enable small loop unrolling for O2 Date: Wed, 26 Oct 2022 13:52:48 +0800 Message-Id: <20221026055248.94100-1-hongyu.wang@intel.com> X-Mailer: git-send-email 2.18.1 X-Spam-Status: No, score=-10.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_NUMSUBJECT, KAM_SHORT, SPF_HELO_NONE, SPF_SOFTFAIL, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Hongyu Wang via Gcc-patches From: Hongyu Wang Reply-To: Hongyu Wang Cc: hongtao.liu@intel.com Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747728350214909590?= X-GMAIL-MSGID: =?utf-8?q?1747728350214909590?= Hi, Inspired by rs6000 and s390 port changes, this patch enables loop unrolling for small size loop at O2 by default. The default behavior is to unroll loop with unknown trip-count and less than 4 insns by 1 time. This improves 548.exchange2 by 3.5% on icelake and 6% on zen3 with 1.2% codesize increment. For other benchmarks the variants are minor and overall codesize increased by 0.2%. The kernel image size increased by 0.06%, and no impact on eembc. Bootstrapped & regrtested on x86_64-pc-linux-gnu. Ok for trunk? gcc/ChangeLog: * common/config/i386/i386-common.cc (ix86_optimization_table): Enable loop unroll and small loop unroll at O2 by default. * config/i386/i386-options.cc (ix86_override_options_after_change): Disable small loop unroll when funroll-loops enabled, reset cunroll_grow_size when it is not explicitly enabled. (ix86_option_override_internal): Call ix86_override_options_after_change instead of calling ix86_recompute_optlev_based_flags and ix86_default_align separately. * config/i386/i386.cc (ix86_loop_unroll_adjust): Adjust unroll factor if -munroll-only-small-loops enabled. * config/i386/i386.opt: Add -munroll-only-small-loops, -param=x86-small-unroll-ninsns= for loop insn limit, -param=x86-small-unroll-factor= for unroll factor. * doc/invoke.texi: Document -munroll-only-small-loops, x86-small-unroll-ninsns and x86-small-unroll-factor. gcc/testsuite/ChangeLog: * gcc.target/i386/pr86270.c: Add -mno-unroll-only-small-loops. * gcc.target/i386/pr93002.c: Likewise. --- gcc/common/config/i386/i386-common.cc | 6 ++++ gcc/config/i386/i386-options.cc | 40 ++++++++++++++++++++++--- gcc/config/i386/i386.cc | 13 ++++++++ gcc/config/i386/i386.opt | 13 ++++++++ gcc/doc/invoke.texi | 14 +++++++++ gcc/testsuite/gcc.target/i386/pr86270.c | 2 +- gcc/testsuite/gcc.target/i386/pr93002.c | 2 +- 7 files changed, 84 insertions(+), 6 deletions(-) diff --git a/gcc/common/config/i386/i386-common.cc b/gcc/common/config/i386/i386-common.cc index d6a68dc9b1d..0e580b39d14 100644 --- a/gcc/common/config/i386/i386-common.cc +++ b/gcc/common/config/i386/i386-common.cc @@ -1686,6 +1686,12 @@ static const struct default_options ix86_option_optimization_table[] = /* The STC algorithm produces the smallest code at -Os, for x86. */ { OPT_LEVELS_2_PLUS, OPT_freorder_blocks_algorithm_, NULL, REORDER_BLOCKS_ALGORITHM_STC }, + { OPT_LEVELS_2_PLUS_SPEED_ONLY, OPT_funroll_loops, NULL, 1 }, + { OPT_LEVELS_2_PLUS_SPEED_ONLY, OPT_munroll_only_small_loops, NULL, 1 }, + /* Turns off -frename-registers and -fweb which are enabled by + funroll-loops. */ + { OPT_LEVELS_ALL, OPT_frename_registers, NULL, 0 }, + { OPT_LEVELS_ALL, OPT_fweb, NULL, 0 }, /* Turn off -fschedule-insns by default. It tends to make the problem with not enough registers even worse. */ { OPT_LEVELS_ALL, OPT_fschedule_insns, NULL, 0 }, diff --git a/gcc/config/i386/i386-options.cc b/gcc/config/i386/i386-options.cc index acb2291e70f..6ea347c32e1 100644 --- a/gcc/config/i386/i386-options.cc +++ b/gcc/config/i386/i386-options.cc @@ -1819,8 +1819,43 @@ ix86_recompute_optlev_based_flags (struct gcc_options *opts, void ix86_override_options_after_change (void) { + /* Default align_* from the processor table. */ ix86_default_align (&global_options); + ix86_recompute_optlev_based_flags (&global_options, &global_options_set); + + /* Disable unrolling small loops when there's explicit + -f{,no}unroll-loop. */ + if ((OPTION_SET_P (flag_unroll_loops)) + || (OPTION_SET_P (flag_unroll_all_loops) + && flag_unroll_all_loops)) + { + if (!OPTION_SET_P (ix86_unroll_only_small_loops)) + ix86_unroll_only_small_loops = 0; + /* Re-enable -frename-registers and -fweb if funroll-loops + enabled. */ + if (!OPTION_SET_P (flag_web)) + flag_web = flag_unroll_loops; + if (!OPTION_SET_P (flag_rename_registers)) + flag_rename_registers = flag_unroll_loops; + if (!OPTION_SET_P (flag_cunroll_grow_size)) + flag_cunroll_grow_size = flag_unroll_loops + || flag_peel_loops + || optimize >= 3; + } + else + { + if (!OPTION_SET_P (flag_cunroll_grow_size)) + flag_cunroll_grow_size = flag_peel_loops || optimize >= 3; + /* Disables loop unrolling if -mno-unroll-only-small-loops is + explicitly set and -funroll-loops is not enabled. */ + if (OPTION_SET_P (ix86_unroll_only_small_loops) + && !ix86_unroll_only_small_loops + && !(OPTION_SET_P (flag_unroll_loops) + || OPTION_SET_P (flag_unroll_all_loops))) + flag_unroll_loops = flag_unroll_all_loops = 0; + } + } /* Clear stack slot assignments remembered from previous functions. @@ -2332,7 +2367,7 @@ ix86_option_override_internal (bool main_args_p, set_ix86_tune_features (opts, ix86_tune, opts->x_ix86_dump_tunes); - ix86_recompute_optlev_based_flags (opts, opts_set); + ix86_override_options_after_change (); ix86_tune_cost = processor_cost_table[ix86_tune]; /* TODO: ix86_cost should be chosen at instruction or function granuality @@ -2363,9 +2398,6 @@ ix86_option_override_internal (bool main_args_p, || TARGET_64BIT_P (opts->x_ix86_isa_flags)) opts->x_ix86_regparm = REGPARM_MAX; - /* Default align_* from the processor table. */ - ix86_default_align (opts); - /* Provide default for -mbranch-cost= value. */ SET_OPTION_IF_UNSET (opts, opts_set, ix86_branch_cost, ix86_tune_cost->branch_cost); diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc index 480db35f6cd..75829a5d0f4 100644 --- a/gcc/config/i386/i386.cc +++ b/gcc/config/i386/i386.cc @@ -23820,6 +23820,19 @@ ix86_loop_unroll_adjust (unsigned nunroll, class loop *loop) unsigned i; unsigned mem_count = 0; + /* Unroll small size loop when unroll factor is not explicitly + specified. */ + if (ix86_unroll_only_small_loops && !loop->unroll) + { + int small_unroll = 0; + if (loop->ninsns <= (unsigned) ix86_small_unroll_ninsns) + small_unroll = MIN ((unsigned) ix86_small_unroll_factor, + nunroll); + else + small_unroll = 1; + return small_unroll; + } + if (!TARGET_ADJUST_UNROLL) return nunroll; diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt index 0dbaacb57ed..a724c73c0c4 100644 --- a/gcc/config/i386/i386.opt +++ b/gcc/config/i386/i386.opt @@ -1214,3 +1214,16 @@ Do not use GOT to access external symbols. -param=x86-stlf-window-ninsns= Target Joined UInteger Var(x86_stlf_window_ninsns) Init(64) Param Instructions number above which STFL stall penalty can be compensated. + +munroll-only-small-loops +Target Var(ix86_unroll_only_small_loops) Init(0) Save +Enable conservative small loop unrolling. + +-param=x86-small-unroll-ninsns= +Target Joined UInteger Var(ix86_small_unroll_ninsns) Init(4) Param +Insturctions number limit for loop to be unrolled under +-munroll-only-small-loops. + +-param=x86-small-unroll-factor= +Target Joined UInteger Var(ix86_small_unroll_factor) Init(2) Param +Unroll factor for -munroll-only-small-loops. diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index cd4d3c1d72c..b6fa79eccc3 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -15779,6 +15779,14 @@ The following choices of @var{name} are available on i386 and x86_64 targets: @item x86-stlf-window-ninsns Instructions number above which STFL stall penalty can be compensated. +@item x86-small-unroll-ninsns +If -munroll-only-small-loops is enabled, only unroll loops with instruction +count less than this parameter. The default value is 4. + +@item x86-small-unroll-factor +If -munroll-only-small-loops is enabled, reset the unroll factor with this +value. The default value is 2 which means the loop will be unrolled once. + @end table @end table @@ -25186,6 +25194,12 @@ environments where no dynamic link is performed, like firmwares, OS kernels, executables linked with @option{-static} or @option{-static-pie}. @option{-mdirect-extern-access} is not compatible with @option{-fPIC} or @option{-fpic}. + +@item -munroll-only-small-loops +@itemx -mno-unroll-only-small-loops +@opindex munroll-only-small-loops +Controls conservative small loop unrolling. It is default enbaled by +O2, and unrolls loop with less than 4 insns by 1 time. @end table @node M32C Options diff --git a/gcc/testsuite/gcc.target/i386/pr86270.c b/gcc/testsuite/gcc.target/i386/pr86270.c index 81841ef5bd7..cbc9fbb0450 100644 --- a/gcc/testsuite/gcc.target/i386/pr86270.c +++ b/gcc/testsuite/gcc.target/i386/pr86270.c @@ -1,5 +1,5 @@ /* { dg-do compile } */ -/* { dg-options "-O2" } */ +/* { dg-options "-O2 -mno-unroll-only-small-loops" } */ int *a; long len; diff --git a/gcc/testsuite/gcc.target/i386/pr93002.c b/gcc/testsuite/gcc.target/i386/pr93002.c index 0248fcc00a5..f75a847f75d 100644 --- a/gcc/testsuite/gcc.target/i386/pr93002.c +++ b/gcc/testsuite/gcc.target/i386/pr93002.c @@ -1,6 +1,6 @@ /* PR target/93002 */ /* { dg-do compile } */ -/* { dg-options "-O2" } */ +/* { dg-options "-O2 -mno-unroll-only-small-loops" } */ /* { dg-final { scan-assembler-not "cmp\[^\n\r]*-1" } } */ volatile int sink;