From patchwork Fri Oct 14 02:32:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: chenglulu X-Patchwork-Id: 2465 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp581083wrs; Thu, 13 Oct 2022 19:33:56 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7IAOa23Dx8onV5VU+0thkBSxaOu4utkU5eMv6b68KLx7VpQg5vFcAbQg2Gcu/0PhZKZisR X-Received: by 2002:a17:907:6d27:b0:78d:46f6:c59e with SMTP id sa39-20020a1709076d2700b0078d46f6c59emr1939270ejc.30.1665714835930; Thu, 13 Oct 2022 19:33:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665714835; cv=none; d=google.com; s=arc-20160816; b=klN3aJzWIJSI+LeaknrQgR4wDZuZuGTI84FpPRhatVNf1SocEKYoJGU80pxvW1nQv6 E8vAGKTRXOZsFjmNKO1EMgzSaXM4w2XL42bJZrQmrxNtpjlo81AKflCpSfdDcRttGNbI 7/SP407kdBun8tp+8tEobRe0rzyjGzo32PGZd6zfiiYSnjhYDsYuPSDQ6g7lgiQ/CIfK lizwIwZJBPxtMAisZBC8KArEDBNdYM4lsLsAWadLogk+IjWwTg9EHZo/nUtw1iN3fLzd d6NeF0s1c/cE7k1tIUGOsgQy398JBvP3ONneVipSxln7yxx17UyN0wtH6e1a0NykCnqs dKYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:to:from:dmarc-filter :delivered-to; bh=lPJFnij6Ez+5/58vfcVZfW9dBMdgpDPFOyhPkl3UPIQ=; b=UjYLX3+Vd5xQ1ZPyEarCwGNKDCD/0nAIEp8xOGzavcnBSAHP+Crb03zP5LEiwiyfTQ CJ3f1BYWhrPgMYmmuNv5JCQvNy0mjnmhRZu7GjwWwE2YhUU3zRV31/YMVMEsx7zmftsz b8liS/+EkC1Yue0nrCiUqvsCdQCXn3p739W0rRL2GammAucLcm4glLozhf7pMgCEfwlm ApD+a4ykP8D20pvPYDOEmGD650xDBhZkEg32iy0WG5jOaBgMPxF1WcLkOT+Mk0lbqvQL URVAlfb+8JhrqYqauVEBd0Om0/gm3KmNyELDE4Wx5sdlp3A9/kTa7MArA51dvrLZpRx7 3Srw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id j11-20020a05640211cb00b0045878af0adbsi1382086edw.393.2022.10.13.19.33.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Oct 2022 19:33:55 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E67C3385355B for ; Fri, 14 Oct 2022 02:33:43 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by sourceware.org (Postfix) with ESMTP id 5D8913858C39 for ; Fri, 14 Oct 2022 02:32:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5D8913858C39 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=loongson.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=loongson.cn Received: from 5.5.5 (unknown [10.2.5.5]) by localhost.localdomain (Coremail) with SMTP id AQAAf8CxbWtOykhjrcstAA--.51347S2; Fri, 14 Oct 2022 10:32:50 +0800 (CST) From: Lulu Cheng To: gcc-patches@gcc.gnu.org, mliska@suse.cz, dmalcolm@redhat.com Subject: [PATCH v2] LoongArch: Optimize the implementation of stack check. Date: Fri, 14 Oct 2022 10:32:20 +0800 Message-Id: <20221014023219.1395533-1-chenglulu@loongson.cn> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-CM-TRANSID: AQAAf8CxbWtOykhjrcstAA--.51347S2 X-Coremail-Antispam: 1UD129KBjvAXoWfZrWrWr1kCF1fAw13GrWruFg_yoW5ZF4Uto ZakF47u34xGrySkr4UKwsxXrWUGw10yr43ZFZIk34UCF1DA345Cayagw1fury3GrnFqw4k AryakFW2yr97AF95n29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UjIYCTnIWjp_UUUYe7AC8VAFwI0_Gr0_Xr1l1xkIjI8I6I8E6xAIw20EY4v20xva j40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2 x7M28EF7xvwVC0I7IYx2IY67AKxVW5JVW7JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8 Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI 0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrVC2j2Wl Yx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbV WUJVW8JwACjcxG0xvY0x0EwIxGrwACjI8F5VA0II8E6IAqYI8I648v4I1lc2xSY4AK6svP MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr 0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUAVWUtwCIc40Y0x0E wIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVWUJV W8JwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAI cVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUbXdbUUUUUU== X-CM-SenderInfo: xfkh0wpoxo3qxorr0wxvrqhubq/ X-Spam-Status: No, score=-12.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: xuchenghua@loongson.cn, Lulu Cheng , i@xen0n.name Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746628599771466972?= X-GMAIL-MSGID: =?utf-8?q?1746628599771466972?= The old stack stack was performed before the stack was dropped, which would cause the detection tool to report a memory leak. The current stack check scheme is as follows: '-fstack-clash-protection': 1. When the frame->total_size is smaller than the guard page size, the stack is dropped according to the original scheme, and there is no need to perform stack detection in the prologue. 2. When frame->total_size is greater than or equal to guard page size, the first step to drop the stack is to drop the space required by the caller-save registers. This space needs to save the caller-save registers, so an implicit stack check is performed. So just need to check the rest of the stack space. '-fstack-check': There is no one-time stack drop and then page-by-page detection as described in the document. It is also the same as '-fstack-clash-protection', which is detected immediately after page drop. It is judged that when frame->total_size is not 0, only the size required to save the s register is dropped for the first stack down. The test cases are referenced from aarch64. gcc/ChangeLog: * config/loongarch/linux.h (STACK_CHECK_MOVING_SP): Define this macro to 1. * config/loongarch/loongarch.cc (loongarch_first_stack_step): Return the size of the first drop stack according to whether stack checking is performed (loongarch_emit_probe_stack_range): Adjust the method of stack checking in prologue. (loongarch_output_probe_stack_range): Delete useless code. (loongarch_expand_prologue): Adjust the method of stack checking in prologue. (loongarch_option_override_internal): Enforce that interval is the same size as size so the mid-end does the right thing. * config/loongarch/loongarch.h (STACK_CLASH_MAX_UNROLL_PAGES): New macro decide whether to loop stack detection. gcc/testsuite/ChangeLog: * lib/target-supports.exp: Add loongarch support for stack_clash_protection. * gcc.target/loongarch/stack-check-alloca-1.c: New test. * gcc.target/loongarch/stack-check-alloca-2.c: New test. * gcc.target/loongarch/stack-check-alloca-3.c: New test. * gcc.target/loongarch/stack-check-alloca-4.c: New test. * gcc.target/loongarch/stack-check-alloca-5.c: New test. * gcc.target/loongarch/stack-check-alloca-6.c: New test. * gcc.target/loongarch/stack-check-alloca.h: New test. * gcc.target/loongarch/stack-check-cfa-1.c: New test. * gcc.target/loongarch/stack-check-cfa-2.c: New test. * gcc.target/loongarch/stack-check-prologue-1.c: New test. * gcc.target/loongarch/stack-check-prologue-2.c: New test. * gcc.target/loongarch/stack-check-prologue-3.c: New test. * gcc.target/loongarch/stack-check-prologue-4.c: New test. * gcc.target/loongarch/stack-check-prologue-5.c: New test. * gcc.target/loongarch/stack-check-prologue-6.c: New test. * gcc.target/loongarch/stack-check-prologue-7.c: New test. * gcc.target/loongarch/stack-check-prologue.h: New test. --- gcc/config/loongarch/linux.h | 3 + gcc/config/loongarch/loongarch.cc | 249 +++++++++++------- gcc/config/loongarch/loongarch.h | 4 + .../loongarch/stack-check-alloca-1.c | 15 ++ .../loongarch/stack-check-alloca-2.c | 12 + .../loongarch/stack-check-alloca-3.c | 12 + .../loongarch/stack-check-alloca-4.c | 12 + .../loongarch/stack-check-alloca-5.c | 13 + .../loongarch/stack-check-alloca-6.c | 13 + .../gcc.target/loongarch/stack-check-alloca.h | 15 ++ .../gcc.target/loongarch/stack-check-cfa-1.c | 12 + .../gcc.target/loongarch/stack-check-cfa-2.c | 12 + .../loongarch/stack-check-prologue-1.c | 11 + .../loongarch/stack-check-prologue-2.c | 11 + .../loongarch/stack-check-prologue-3.c | 11 + .../loongarch/stack-check-prologue-4.c | 11 + .../loongarch/stack-check-prologue-5.c | 12 + .../loongarch/stack-check-prologue-6.c | 11 + .../loongarch/stack-check-prologue-7.c | 12 + .../loongarch/stack-check-prologue.h | 5 + gcc/testsuite/lib/target-supports.exp | 7 +- 21 files changed, 362 insertions(+), 101 deletions(-) create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca-1.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca-2.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca-3.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca-4.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca-5.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca-6.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-alloca.h create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-cfa-1.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-cfa-2.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-1.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-2.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-3.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-4.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-5.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-6.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue-7.c create mode 100644 gcc/testsuite/gcc.target/loongarch/stack-check-prologue.h diff --git a/gcc/config/loongarch/linux.h b/gcc/config/loongarch/linux.h index 110d0fab93d..00039ac1871 100644 --- a/gcc/config/loongarch/linux.h +++ b/gcc/config/loongarch/linux.h @@ -48,3 +48,6 @@ along with GCC; see the file COPYING3. If not see #define STACK_CHECK_PROTECT (TARGET_64BIT ? 16 * 1024 : 12 * 1024) #define TARGET_ASM_FILE_END file_end_indicate_exec_stack + +/* The stack pointer needs to be moved while checking the stack. */ +#define STACK_CHECK_MOVING_SP 1 diff --git a/gcc/config/loongarch/loongarch.cc b/gcc/config/loongarch/loongarch.cc index e9ba3374e35..755dec6a629 100644 --- a/gcc/config/loongarch/loongarch.cc +++ b/gcc/config/loongarch/loongarch.cc @@ -63,6 +63,7 @@ along with GCC; see the file COPYING3. If not see #include "context.h" #include "builtins.h" #include "rtl-iter.h" +#include "opts.h" /* This file should be included last. */ #include "target-def.h" @@ -257,6 +258,10 @@ const char *const loongarch_fp_conditions[16]= {LARCH_FP_CONDITIONS (STRINGIFY)}; #undef STRINGIFY +/* Size of guard page. */ +#define STACK_CLASH_PROTECTION_GUARD_SIZE \ + (1 << param_stack_clash_protection_guard_size) + /* Implement TARGET_FUNCTION_ARG_BOUNDARY. Every parameter gets at least PARM_BOUNDARY bits of alignment, but will be given anything up to PREFERRED_STACK_BOUNDARY bits if the type requires it. */ @@ -1069,11 +1074,20 @@ loongarch_restore_reg (rtx reg, rtx mem) static HOST_WIDE_INT loongarch_first_stack_step (struct loongarch_frame_info *frame) { + HOST_WIDE_INT min_first_step + = LARCH_STACK_ALIGN (frame->total_size - frame->fp_sp_offset); + + /* When stack checking is required, if the sum of frame->total_size + and stack_check_protect is greater than stack clash protection guard + size, then return min_first_step. */ + if (flag_stack_check == STATIC_BUILTIN_STACK_CHECK + || (flag_stack_clash_protection + && frame->total_size > STACK_CLASH_PROTECTION_GUARD_SIZE)) + return min_first_step; + if (IMM12_OPERAND (frame->total_size)) return frame->total_size; - HOST_WIDE_INT min_first_step - = LARCH_STACK_ALIGN (frame->total_size - frame->fp_sp_offset); HOST_WIDE_INT max_first_step = IMM_REACH / 2 - PREFERRED_STACK_BOUNDARY / 8; HOST_WIDE_INT min_second_step = frame->total_size - max_first_step; gcc_assert (min_first_step <= max_first_step); @@ -1106,103 +1120,109 @@ loongarch_emit_stack_tie (void) static void loongarch_emit_probe_stack_range (HOST_WIDE_INT first, HOST_WIDE_INT size) { - /* See if we have a constant small number of probes to generate. If so, - that's the easy case. */ - if ((TARGET_64BIT && (first + size <= 32768)) - || (!TARGET_64BIT && (first + size <= 2048))) - { - HOST_WIDE_INT i; + HOST_WIDE_INT rounded_size; + HOST_WIDE_INT interval; - /* Probe at FIRST + N * PROBE_INTERVAL for values of N from 1 until - it exceeds SIZE. If only one probe is needed, this will not - generate any code. Then probe at FIRST + SIZE. */ - for (i = PROBE_INTERVAL; i < size; i += PROBE_INTERVAL) - emit_stack_probe (plus_constant (Pmode, stack_pointer_rtx, - -(first + i))); + if (flag_stack_clash_protection) + interval = STACK_CLASH_PROTECTION_GUARD_SIZE; + else + interval = PROBE_INTERVAL; - emit_stack_probe (plus_constant (Pmode, stack_pointer_rtx, - -(first + size))); - } + rtx r12 = LARCH_PROLOGUE_TEMP2 (Pmode); + rtx r14 = LARCH_PROLOGUE_TEMP3 (Pmode); - /* Otherwise, do the same as above, but in a loop. Note that we must be - extra careful with variables wrapping around because we might be at - the very top (or the very bottom) of the address space and we have - to be able to handle this case properly; in particular, we use an - equality test for the loop condition. */ - else - { - HOST_WIDE_INT rounded_size; - rtx r13 = LARCH_PROLOGUE_TEMP (Pmode); - rtx r12 = LARCH_PROLOGUE_TEMP2 (Pmode); - rtx r14 = LARCH_PROLOGUE_TEMP3 (Pmode); + size = size + first; - /* Sanity check for the addressing mode we're going to use. */ - gcc_assert (first <= 16384); + /* Sanity check for the addressing mode we're going to use. */ + gcc_assert (first <= 16384); + /* Step 1: round SIZE to the previous multiple of the interval. */ - /* Step 1: round SIZE to the previous multiple of the interval. */ + rounded_size = ROUND_DOWN (size, interval); - rounded_size = ROUND_DOWN (size, PROBE_INTERVAL); + /* Step 2: compute initial and final value of the loop counter. */ - /* TEST_ADDR = SP + FIRST */ - if (first != 0) - { - emit_move_insn (r14, GEN_INT (first)); - emit_insn (gen_rtx_SET (r13, gen_rtx_MINUS (Pmode, - stack_pointer_rtx, - r14))); - } - else - emit_move_insn (r13, stack_pointer_rtx); + emit_move_insn (r14, GEN_INT (interval)); + + /* If rounded_size is zero, it means that the space requested by + the local variable is less than the interval, and there is no + need to display and detect the allocated space. */ + if (rounded_size != 0) + { + /* Step 3: the loop + + do + { + TEST_ADDR = TEST_ADDR + PROBE_INTERVAL + probe at TEST_ADDR + } + while (TEST_ADDR != LAST_ADDR) - /* Step 2: compute initial and final value of the loop counter. */ + probes at FIRST + N * PROBE_INTERVAL for values of N from 1 + until it is equal to ROUNDED_SIZE. */ - emit_move_insn (r14, GEN_INT (PROBE_INTERVAL)); - /* LAST_ADDR = SP + FIRST + ROUNDED_SIZE. */ - if (rounded_size == 0) - emit_move_insn (r12, r13); + if (rounded_size <= STACK_CLASH_MAX_UNROLL_PAGES * interval) + { + for (HOST_WIDE_INT i = 0; i < rounded_size; i += interval) + { + emit_insn (gen_rtx_SET (stack_pointer_rtx, + gen_rtx_MINUS (Pmode, + stack_pointer_rtx, + r14))); + emit_move_insn (gen_rtx_MEM (Pmode, + gen_rtx_PLUS (Pmode, + stack_pointer_rtx, + const0_rtx)), + const0_rtx); + emit_insn (gen_blockage ()); + } + dump_stack_clash_frame_info (PROBE_INLINE, size != rounded_size); + } else { emit_move_insn (r12, GEN_INT (rounded_size)); - emit_insn (gen_rtx_SET (r12, gen_rtx_MINUS (Pmode, r13, r12))); - /* Step 3: the loop - - do - { - TEST_ADDR = TEST_ADDR + PROBE_INTERVAL - probe at TEST_ADDR - } - while (TEST_ADDR != LAST_ADDR) - - probes at FIRST + N * PROBE_INTERVAL for values of N from 1 - until it is equal to ROUNDED_SIZE. */ - - emit_insn (gen_probe_stack_range (Pmode, r13, r13, r12, r14)); + emit_insn (gen_rtx_SET (r12, + gen_rtx_MINUS (Pmode, + stack_pointer_rtx, + r12))); + + emit_insn (gen_probe_stack_range (Pmode, stack_pointer_rtx, + stack_pointer_rtx, r12, r14)); + emit_insn (gen_blockage ()); + dump_stack_clash_frame_info (PROBE_LOOP, size != rounded_size); } + } + else + dump_stack_clash_frame_info (NO_PROBE_SMALL_FRAME, true); - /* Step 4: probe at FIRST + SIZE if we cannot assert at compile-time - that SIZE is equal to ROUNDED_SIZE. */ - if (size != rounded_size) + /* Step 4: probe at FIRST + SIZE if we cannot assert at compile-time + that SIZE is equal to ROUNDED_SIZE. */ + + if (size != rounded_size) + { + if (size - rounded_size >= 2048) { - if (TARGET_64BIT) - emit_stack_probe (plus_constant (Pmode, r12, rounded_size - size)); - else - { - HOST_WIDE_INT i; - for (i = 2048; i < (size - rounded_size); i += 2048) - { - emit_stack_probe (plus_constant (Pmode, r12, -i)); - emit_insn (gen_rtx_SET (r12, - plus_constant (Pmode, r12, -2048))); - } - rtx r1 = plus_constant (Pmode, r12, - -(size - rounded_size - i + 2048)); - emit_stack_probe (r1); - } + emit_move_insn (r14, GEN_INT (size - rounded_size)); + emit_insn (gen_rtx_SET (stack_pointer_rtx, + gen_rtx_MINUS (Pmode, + stack_pointer_rtx, + r14))); } + else + emit_insn (gen_rtx_SET (stack_pointer_rtx, + gen_rtx_PLUS (Pmode, + stack_pointer_rtx, + GEN_INT (rounded_size - size)))); } + if (first) + { + emit_move_insn (r12, GEN_INT (first)); + emit_insn (gen_rtx_SET (stack_pointer_rtx, + gen_rtx_PLUS (Pmode, + stack_pointer_rtx, r12))); + } /* Make sure nothing is scheduled before we are done. */ emit_insn (gen_blockage ()); } @@ -1223,7 +1243,6 @@ loongarch_output_probe_stack_range (rtx reg1, rtx reg2, rtx reg3) /* TEST_ADDR = TEST_ADDR + PROBE_INTERVAL. */ xops[0] = reg1; - xops[1] = GEN_INT (-PROBE_INTERVAL); xops[2] = reg3; if (TARGET_64BIT) output_asm_insn ("sub.d\t%0,%0,%2", xops); @@ -1249,28 +1268,11 @@ loongarch_expand_prologue (void) { struct loongarch_frame_info *frame = &cfun->machine->frame; HOST_WIDE_INT size = frame->total_size; - HOST_WIDE_INT tmp; rtx insn; if (flag_stack_usage_info) current_function_static_stack_size = size; - if (flag_stack_check == STATIC_BUILTIN_STACK_CHECK - || flag_stack_clash_protection) - { - if (crtl->is_leaf && !cfun->calls_alloca) - { - if (size > PROBE_INTERVAL && size > get_stack_check_protect ()) - { - tmp = size - get_stack_check_protect (); - loongarch_emit_probe_stack_range (get_stack_check_protect (), - tmp); - } - } - else if (size > 0) - loongarch_emit_probe_stack_range (get_stack_check_protect (), size); - } - /* Save the registers. */ if ((frame->mask | frame->fmask) != 0) { @@ -1283,7 +1285,6 @@ loongarch_expand_prologue (void) loongarch_for_each_saved_reg (size, loongarch_save_reg); } - /* Set up the frame pointer, if we're using one. */ if (frame_pointer_needed) { @@ -1294,7 +1295,45 @@ loongarch_expand_prologue (void) loongarch_emit_stack_tie (); } - /* Allocate the rest of the frame. */ + if (flag_stack_check == STATIC_BUILTIN_STACK_CHECK + || flag_stack_clash_protection) + { + HOST_WIDE_INT first = get_stack_check_protect (); + + if (frame->total_size == 0) + { + /* do nothing. */ + dump_stack_clash_frame_info (NO_PROBE_NO_FRAME, false); + return; + } + + if (crtl->is_leaf && !cfun->calls_alloca) + { + HOST_WIDE_INT interval; + + if (flag_stack_clash_protection) + interval = STACK_CLASH_PROTECTION_GUARD_SIZE; + else + interval = PROBE_INTERVAL; + + if (size > interval && size > first) + loongarch_emit_probe_stack_range (first, size - first); + else + loongarch_emit_probe_stack_range (first, size); + } + else + loongarch_emit_probe_stack_range (first, size); + + if (size > 0) + { + /* Describe the effect of the previous instructions. */ + insn = plus_constant (Pmode, stack_pointer_rtx, -size); + insn = gen_rtx_SET (stack_pointer_rtx, insn); + loongarch_set_frame_expr (insn); + } + return; + } + if (size > 0) { if (IMM12_OPERAND (-size)) @@ -1305,7 +1344,8 @@ loongarch_expand_prologue (void) } else { - loongarch_emit_move (LARCH_PROLOGUE_TEMP (Pmode), GEN_INT (-size)); + loongarch_emit_move (LARCH_PROLOGUE_TEMP (Pmode), + GEN_INT (-size)); emit_insn (gen_add3_insn (stack_pointer_rtx, stack_pointer_rtx, LARCH_PROLOGUE_TEMP (Pmode))); @@ -6127,6 +6167,15 @@ loongarch_option_override_internal (struct gcc_options *opts) gcc_unreachable (); } + /* Validate the guard size. */ + int guard_size = param_stack_clash_protection_guard_size; + + /* Enforce that interval is the same size as size so the mid-end does the + right thing. */ + SET_OPTION_IF_UNSET (opts, &global_options_set, + param_stack_clash_protection_probe_interval, + guard_size); + loongarch_init_print_operand_punct (); /* Set up array to map GCC register number to debug register number. diff --git a/gcc/config/loongarch/loongarch.h b/gcc/config/loongarch/loongarch.h index f4a9c329fef..a402d3ba35a 100644 --- a/gcc/config/loongarch/loongarch.h +++ b/gcc/config/loongarch/loongarch.h @@ -668,6 +668,10 @@ enum reg_class #define STACK_BOUNDARY (TARGET_ABI_LP64 ? 128 : 64) +/* This value controls how many pages we manually unroll the loop for when + generating stack clash probes. */ +#define STACK_CLASH_MAX_UNROLL_PAGES 4 + /* Symbolic macros for the registers used to return integer and floating point values. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-1.c b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-1.c new file mode 100644 index 00000000000..6ee589c4b3d --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-1.c @@ -0,0 +1,15 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-require-effective-target alloca } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE y +#include "stack-check-alloca.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r\d{1,2},-8} 1 } } */ +/* { dg-final { scan-assembler-times {stx\.d\t\$r0,\$r3,\$r12} 1 } } */ + +/* Dynamic alloca, expect loop, and 1 probes with top at sp. + 1st probe is inside the loop for the full guard-size allocations, second + probe is for the case where residual is zero. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-2.c b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-2.c new file mode 100644 index 00000000000..8deaa587305 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-2.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-require-effective-target alloca } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 0 +#include "stack-check-alloca.h" + +/* { dg-final { scan-assembler-not {stp*t*r*\.d\t\$r0,\$r3,4088} } } */ + +/* Alloca of 0 should emit no probes, boundary condition. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-3.c b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-3.c new file mode 100644 index 00000000000..e326ba9a092 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-3.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-require-effective-target alloca } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 100 +#include "stack-check-alloca.h" + +/* { dg-final { scan-assembler-times {st\.d\t\$r0,\$r3,104} 1 } } */ + +/* Alloca is less than guard-size, 1 probe at the top of the new allocation. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-4.c b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-4.c new file mode 100644 index 00000000000..b9f7572dedc --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-4.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-require-effective-target alloca } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 64 * 1024 +#include "stack-check-alloca.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r\d{1,2},-8} 1 } } */ + +/* Alloca is exactly one guard-size, 1 probe expected at top. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-5.c b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-5.c new file mode 100644 index 00000000000..0ff6e493fec --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-5.c @@ -0,0 +1,13 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-require-effective-target alloca } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 65 * 1024 +#include "stack-check-alloca.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r\d{1,2},-8} 1 } } */ +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r3,1016} 1 } } */ + +/* Alloca is more than one guard-page. 2 probes expected. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-6.c b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-6.c new file mode 100644 index 00000000000..c5cf74fcb4f --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca-6.c @@ -0,0 +1,13 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-require-effective-target alloca } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 127 * 64 * 1024 +#include "stack-check-alloca.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r\d{1,2},-8} 1 } } */ + +/* Large alloca of a constant amount which is a multiple of a guard-size. + Loop expected with top probe. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-alloca.h b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca.h new file mode 100644 index 00000000000..8c75f6c0f70 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-alloca.h @@ -0,0 +1,15 @@ + +/* Avoid inclusion of alloca.h, unavailable on some systems. */ +#define alloca __builtin_alloca + +__attribute__((noinline, noipa)) +void g (char* ptr, int y) +{ + ptr[y] = '\0'; +} + +void f_caller (int y) +{ + char* pStr = alloca(SIZE); + g (pStr, y); +} diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-cfa-1.c b/gcc/testsuite/gcc.target/loongarch/stack-check-cfa-1.c new file mode 100644 index 00000000000..f0c6877fc25 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-cfa-1.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16 -funwind-tables" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 128*1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {\.cfi_def_cfa_offset 131088} 1 } } */ +/* { dg-final { scan-assembler-times {\.cfi_def_cfa_offset 0} 1 } } */ + +/* Checks that the CFA notes are correct for every sp adjustment. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-cfa-2.c b/gcc/testsuite/gcc.target/loongarch/stack-check-cfa-2.c new file mode 100644 index 00000000000..c6e07bc561a --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-cfa-2.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16 -funwind-tables" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 1280*1024 + 512 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {\.cfi_def_cfa_offset 1311248} 1 } } */ +/* { dg-final { scan-assembler-times {\.cfi_def_cfa_offset 0} 1 } } */ + +/* Checks that the CFA notes are correct for every sp adjustment. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-1.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-1.c new file mode 100644 index 00000000000..351bc1f61fd --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-1.c @@ -0,0 +1,11 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 128 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r3,0} 0 } } */ + +/* SIZE is smaller than guard-size so no probe expected. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-2.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-2.c new file mode 100644 index 00000000000..6bba659a36d --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-2.c @@ -0,0 +1,11 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 63 * 1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*.d\t\$r0,\$r3,0} 0 } } */ + +/* SIZE is smaller than guard-size so no probe expected. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-3.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-3.c new file mode 100644 index 00000000000..164956c3747 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-3.c @@ -0,0 +1,11 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 64 * 1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r3,0} 1 } } */ + +/* SIZE is equal to guard-size, 1 probe expected, boundary condition. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-4.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-4.c new file mode 100644 index 00000000000..f53da6b0de9 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-4.c @@ -0,0 +1,11 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 65 * 1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r3,0} 1 } } */ + +/* SIZE is more than guard-size 1 probe expected. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-5.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-5.c new file mode 100644 index 00000000000..c092317ea94 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-5.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 127 * 1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r3,0} 1 } } */ + +/* SIZE is more than 1x guard-size and remainder small than guard-size, + 1 probe expected, unrolled, no loop. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-6.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-6.c new file mode 100644 index 00000000000..70a2f53f6d7 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-6.c @@ -0,0 +1,11 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 128 * 1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*\.d\t\$r0,\$r3,0} 2 } } */ + +/* SIZE is more than 2x guard-size and no remainder, unrolled, no loop. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-7.c b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-7.c new file mode 100644 index 00000000000..e2df89acc77 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue-7.c @@ -0,0 +1,12 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -fstack-clash-protection --param stack-clash-protection-guard-size=16" } */ +/* { dg-require-effective-target supports_stack_clash_protection } */ +/* { dg-skip-if "" { *-*-* } { "-fstack-check" } { "" } } */ + +#define SIZE 6 * 64 * 1024 +#include "stack-check-prologue.h" + +/* { dg-final { scan-assembler-times {stp*t*r*.d\t\$r0,\$r3,0} 1 } } */ + +/* SIZE is more than 4x guard-size and no remainder, 1 probe expected in a loop + and no residual probe. */ diff --git a/gcc/testsuite/gcc.target/loongarch/stack-check-prologue.h b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue.h new file mode 100644 index 00000000000..b7e06aedb81 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/stack-check-prologue.h @@ -0,0 +1,5 @@ +int f_test (int x) +{ + char arr[SIZE]; + return arr[x]; +} diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp index fdd88e6a516..8ce879468e8 100644 --- a/gcc/testsuite/lib/target-supports.exp +++ b/gcc/testsuite/lib/target-supports.exp @@ -11377,7 +11377,8 @@ proc check_effective_target_supports_stack_clash_protection { } { if { [istarget x86_64-*-*] || [istarget i?86-*-*] || [istarget powerpc*-*-*] || [istarget rs6000*-*-*] - || [istarget aarch64*-**] || [istarget s390*-*-*] } { + || [istarget aarch64*-**] || [istarget s390*-*-*] + || [istarget loongarch64*-**] } { return 1 } return 0 @@ -11428,6 +11429,10 @@ proc check_effective_target_caller_implicit_probes { } { return 1; } + if { [istarget loongarch64*-*-*] } { + return 1; + } + return 0 }