From patchwork Wed Sep 6 16:07:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_M=C3=BCllner?= X-Patchwork-Id: 137586 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ab0a:0:b0:3f2:4152:657d with SMTP id m10csp2410442vqo; Wed, 6 Sep 2023 09:10:02 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEeRd96KTA7/UxTPsiOFSP5knCEzajERbzFhq78dbJhoN3eAX7Uwzu1qINDu3tjGkM4BGlM X-Received: by 2002:a17:906:2191:b0:9a6:7fee:8258 with SMTP id 17-20020a170906219100b009a67fee8258mr2678843eju.19.1694016602263; Wed, 06 Sep 2023 09:10:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694016602; cv=none; d=google.com; s=arc-20160816; b=r6Uil7DBFHe2mzqzDQW67htO73ExK3XRLGMmpeNhFmgMXCwT8ZPGOIayUf8Iy9MhG1 Nde9mvGzKC+wmXtBWqoxf+qAIcpW0HL8Wf86VQ4ubP86UMSEmd/o4RcjaPkXIpA32lAK vnIr8MBAXTaoq8roL/7wPqP+FmoiSRpPLbn3aM7MfN6Lcfjccrwa4hClgccNlCHahUvb 6H4BYmCzN7FFrKIyZWQ0dOnufA1nhydGMpmImwMe0b6KGWwGrP+BFNHOlZOL7gLGBpYv 5zdPyCMS0hwGDEaPx19qG/r8mo9ueYGrGGndoM0Ahn3DUcIF5+omB+Jl7l9BSv1nokqP 5bXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature:dmarc-filter:delivered-to; bh=S7g2SH7nESXSqLvh3EC2FqpUjgymENliQ+7+7lMxR7k=; fh=+Tzd+TqnZdlHygD2c2HRoq08AoaISebVb4juE/jMCyU=; b=jo7PAgtC++Nlm81qSnMyjrQhm7/uWUGc+5+v3vzVLyPIKTcP7YNuf1aGnAYylPnoEL 28SI2kaxelkQ2JtKfUlDyy73Z9XVJqN5AWP9FovC0jkU/tuGcFfktKYJmHOvI7omQ/X2 rpY9Sj2TXPOZHAPuaBEtTBF5hKIe1QrEg5Yx6QJMyi8yDPwX4Dw8J+rDW9X5UUtf6sac tCMJiPm5finLDlf8po9xUcRPz/aHw+Z7iF3FUD5nxE5L1vzyn4e7HzgMriPNTAbTxxpj JMVvqyu/3R7mh0IOc7ednEvq33soV+MghdQLVMZGfuUWwJ0SvcxN3xJfUmvo4X/b95ue 6PCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@vrull.eu header.s=google header.b=TrrYfi+C; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (server2.sourceware.org. [2620:52:3:1:0:246e:9693:128c]) by mx.google.com with ESMTPS id dk13-20020a170906f0cd00b0098d373fa9e4si9113964ejb.1007.2023.09.06.09.10.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Sep 2023 09:10:02 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) client-ip=2620:52:3:1:0:246e:9693:128c; Authentication-Results: mx.google.com; dkim=fail header.i=@vrull.eu header.s=google header.b=TrrYfi+C; spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 2620:52:3:1:0:246e:9693:128c as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org" Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5D9CB38319D0 for ; Wed, 6 Sep 2023 16:08:57 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com [IPv6:2a00:1450:4864:20::229]) by sourceware.org (Postfix) with ESMTPS id 4C019385783F for ; Wed, 6 Sep 2023 16:07:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4C019385783F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=vrull.eu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=vrull.eu Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-2b9d07a8d84so467161fa.3 for ; Wed, 06 Sep 2023 09:07:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1694016466; x=1694621266; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S7g2SH7nESXSqLvh3EC2FqpUjgymENliQ+7+7lMxR7k=; b=TrrYfi+CxplYQ1Cc90YbKG77FptycZR+Mb7OaLOa6GL7Hgfp6Ac2mq7iA8QWVeUjoK qKw0ulwwpYpeku4tWdC9cK0O24UYHkCMQVWd0cN9eM3tf0L4wfZOJnbP+sLrkLNZTjzj sfmu/Nj/ttJZM3SOtgpmX8IvDpSLOEMv57fVYGa9SLMqIvXrLdF8G0JM8zxDHc9YFJDB iZtf3kf/hSbFDaRJ2DHA7+GvQV3fMO5+2lE0xO40jMATGpS9hctHFgX0np1NnmNUyu0Q NJjsj5bhCK5fEhGq2BoCdmpG6IBHTb6+zkLa0A9q3h9HDZTMV3/j380Y74Wk9E8562dg nj6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694016466; x=1694621266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S7g2SH7nESXSqLvh3EC2FqpUjgymENliQ+7+7lMxR7k=; b=QRC4i9rIV1KQZybVX8XaXLDi68tH/AHVg3O7XXvhqXv3S9padE85DAy76AcNsDkxFQ H7wU28UDUm7lx4xU7f/kTKVrAyylIr/tNz+Au3LVEgOOIUEFjsT3cuIEXNY+fd47f8U0 y2noNzsgPoeMCf5TstGEhWcrPFJR07Jv86+kuYVeDARIHv4zGAZ2dRbY919WKAqtnnwE b4eE7YVegRQEfSU2zOtT6Chp7Ks2cbhECEvwoHqKxo8niLWfUxykNsE39jIHYZYHaaqo o4i3qSOcpyU7Kxrbo351ZJsl6K5sDkAScdjF7qnP8bndpw1042Je3EUv01aRQfZIB8/M epZg== X-Gm-Message-State: AOJu0YxTqKh/VUqE8cX+pU14SOLqizz/qPP5tH7UabAqq0iP9abAPILq 8UqLdP3WE+N20dra7bBX57sO7p1AOx1JXxouE6I= X-Received: by 2002:a2e:9044:0:b0:2bd:10b4:c3e1 with SMTP id n4-20020a2e9044000000b002bd10b4c3e1mr2895079ljg.19.1694016465990; Wed, 06 Sep 2023 09:07:45 -0700 (PDT) Received: from beast.fritz.box (62-178-148-172.cable.dynamic.surfer.at. [62.178.148.172]) by smtp.gmail.com with ESMTPSA id oz19-20020a170906cd1300b0098e42bef736sm9330351ejb.176.2023.09.06.09.07.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Sep 2023 09:07:45 -0700 (PDT) From: Christoph Muellner To: gcc-patches@gcc.gnu.org, Kito Cheng , Jim Wilson , Palmer Dabbelt , Andrew Waterman , Philipp Tomsich , Jeff Law , Vineet Gupta Subject: [PATCH v2 1/2] riscv: Add support for strlen inline expansion Date: Wed, 6 Sep 2023 18:07:33 +0200 Message-ID: <20230906160734.2422522-2-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230906160734.2422522-1-christoph.muellner@vrull.eu> References: <20230906160734.2422522-1-christoph.muellner@vrull.eu> MIME-Version: 1.0 X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_MANYTO, KAM_SHORT, LIKELY_SPAM_BODY, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org Sender: "Gcc-patches" X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776305153005607613 X-GMAIL-MSGID: 1776305153005607613 From: Christoph Müllner This patch implements the expansion of the strlen builtin for RV32/RV64 for xlen-aligned aligned strings if Zbb or XTheadBb instructions are available. The inserted sequences are: rv32gc_zbb (RV64 is similar): add a3,a0,4 li a4,-1 .L1: lw a5,0(a0) add a0,a0,4 orc.b a5,a5 beq a5,a4,.L1 not a5,a5 ctz a5,a5 srl a5,a5,0x3 add a0,a0,a5 sub a0,a0,a3 rv64gc_xtheadbb (RV32 is similar): add a4,a0,8 .L2: ld a5,0(a0) add a0,a0,8 th.tstnbz a5,a5 beqz a5,.L2 th.rev a5,a5 th.ff1 a5,a5 srl a5,a5,0x3 add a0,a0,a5 sub a0,a0,a4 This allows to inline calls to strlen(), with optimized code for xlen-aligned strings, resulting in the following benefits over a call to libc: * no call/ret instructions * no stack frame allocation * no register saving/restoring * no alignment test The inlining mechanism is gated by a new switch ('-minline-strlen') and by the variable 'optimize_size'. Tested using the glibc string tests. Signed-off-by: Christoph Müllner gcc/ChangeLog: * config.gcc: Add new object riscv-string.o. riscv-string.cc. * config/riscv/riscv-protos.h (riscv_expand_strlen): New function. * config/riscv/riscv.md (strlen): New expand INSN. * config/riscv/riscv.opt: New flag 'minline-strlen'. * config/riscv/t-riscv: Add new object riscv-string.o. * config/riscv/thead.md (th_rev2): Export INSN name. (th_rev2): Likewise. (th_tstnbz2): New INSN. * doc/invoke.texi: Document '-minline-strlen'. * emit-rtl.cc (emit_likely_jump_insn): New helper function. (emit_unlikely_jump_insn): Likewise. * rtl.h (emit_likely_jump_insn): New prototype. (emit_unlikely_jump_insn): Likewise. * config/riscv/riscv-string.cc: New file. gcc/testsuite/ChangeLog: * gcc.target/riscv/xtheadbb-strlen-unaligned.c: New test. * gcc.target/riscv/xtheadbb-strlen.c: New test. * gcc.target/riscv/zbb-strlen-disabled-2.c: New test. * gcc.target/riscv/zbb-strlen-disabled.c: New test. * gcc.target/riscv/zbb-strlen-unaligned.c: New test. * gcc.target/riscv/zbb-strlen.c: New test. --- gcc/config.gcc | 3 +- gcc/config/riscv/riscv-protos.h | 3 + gcc/config/riscv/riscv-string.cc | 183 ++++++++++++++++++ gcc/config/riscv/riscv.md | 28 +++ gcc/config/riscv/riscv.opt | 4 + gcc/config/riscv/t-riscv | 6 + gcc/config/riscv/thead.md | 9 +- gcc/doc/invoke.texi | 11 +- gcc/emit-rtl.cc | 24 +++ gcc/rtl.h | 2 + .../riscv/xtheadbb-strlen-unaligned.c | 14 ++ .../gcc.target/riscv/xtheadbb-strlen.c | 19 ++ .../gcc.target/riscv/zbb-strlen-disabled-2.c | 15 ++ .../gcc.target/riscv/zbb-strlen-disabled.c | 15 ++ .../gcc.target/riscv/zbb-strlen-unaligned.c | 14 ++ gcc/testsuite/gcc.target/riscv/zbb-strlen.c | 19 ++ 16 files changed, 366 insertions(+), 3 deletions(-) create mode 100644 gcc/config/riscv/riscv-string.cc create mode 100644 gcc/testsuite/gcc.target/riscv/xtheadbb-strlen-unaligned.c create mode 100644 gcc/testsuite/gcc.target/riscv/xtheadbb-strlen.c create mode 100644 gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled-2.c create mode 100644 gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled.c create mode 100644 gcc/testsuite/gcc.target/riscv/zbb-strlen-unaligned.c create mode 100644 gcc/testsuite/gcc.target/riscv/zbb-strlen.c diff --git a/gcc/config.gcc b/gcc/config.gcc index b2fe7c7ceef..aff6b6a5601 100644 --- a/gcc/config.gcc +++ b/gcc/config.gcc @@ -530,7 +530,8 @@ pru-*-*) ;; riscv*) cpu_type=riscv - extra_objs="riscv-builtins.o riscv-c.o riscv-sr.o riscv-shorten-memrefs.o riscv-selftests.o riscv-v.o riscv-vsetvl.o riscv-vector-costs.o" + extra_objs="riscv-builtins.o riscv-c.o riscv-sr.o riscv-shorten-memrefs.o riscv-selftests.o riscv-string.o" + extra_objs="${extra_objs} riscv-v.o riscv-vsetvl.o riscv-vector-costs.o" extra_objs="${extra_objs} riscv-vector-builtins.o riscv-vector-builtins-shapes.o riscv-vector-builtins-bases.o" extra_objs="${extra_objs} thead.o" d_target_objs="riscv-d.o" diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h index 6dbf6b9f943..b060d047f01 100644 --- a/gcc/config/riscv/riscv-protos.h +++ b/gcc/config/riscv/riscv-protos.h @@ -517,6 +517,9 @@ const unsigned int RISCV_BUILTIN_SHIFT = 1; /* Mask that selects the riscv_builtin_class part of a function code. */ const unsigned int RISCV_BUILTIN_CLASS = (1 << RISCV_BUILTIN_SHIFT) - 1; +/* Routines implemented in riscv-string.cc. */ +extern bool riscv_expand_strlen (rtx, rtx, rtx, rtx); + /* Routines implemented in thead.cc. */ extern bool th_mempair_operands_p (rtx[4], bool, machine_mode); extern void th_mempair_order_operands (rtx[4], bool, machine_mode); diff --git a/gcc/config/riscv/riscv-string.cc b/gcc/config/riscv/riscv-string.cc new file mode 100644 index 00000000000..086900a6083 --- /dev/null +++ b/gcc/config/riscv/riscv-string.cc @@ -0,0 +1,183 @@ +/* Subroutines used to expand string operations for RISC-V. + Copyright (C) 2023 Free Software Foundation, Inc. + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published + by the Free Software Foundation; either version 3, or (at your + option) any later version. + + GCC is distributed in the hope that it will be useful, but WITHOUT + ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public + License for more details. + + You should have received a copy of the GNU General Public License + along with GCC; see the file COPYING3. If not see + . */ + +#define IN_TARGET_CODE 1 + +#include "config.h" +#include "system.h" +#include "coretypes.h" +#include "backend.h" +#include "rtl.h" +#include "tree.h" +#include "memmodel.h" +#include "tm_p.h" +#include "ira.h" +#include "print-tree.h" +#include "varasm.h" +#include "explow.h" +#include "expr.h" +#include "output.h" +#include "target.h" +#include "predict.h" +#include "optabs.h" + +/* Emit proper instruction depending on mode of dest. */ + +#define GEN_EMIT_HELPER2(name) \ +static rtx_insn * \ +do_## name ## 2(rtx dest, rtx src) \ +{ \ + rtx_insn *insn; \ + if (GET_MODE (dest) == DImode) \ + insn = emit_insn (gen_ ## name ## di2 (dest, src)); \ + else \ + insn = emit_insn (gen_ ## name ## si2 (dest, src)); \ + return insn; \ +} + +/* Emit proper instruction depending on mode of dest. */ + +#define GEN_EMIT_HELPER3(name) \ +static rtx_insn * \ +do_## name ## 3(rtx dest, rtx src1, rtx src2) \ +{ \ + rtx_insn *insn; \ + if (GET_MODE (dest) == DImode) \ + insn = emit_insn (gen_ ## name ## di3 (dest, src1, src2)); \ + else \ + insn = emit_insn (gen_ ## name ## si3 (dest, src1, src2)); \ + return insn; \ +} + +GEN_EMIT_HELPER3(add) /* do_add3 */ +GEN_EMIT_HELPER2(clz) /* do_clz2 */ +GEN_EMIT_HELPER2(ctz) /* do_ctz2 */ +GEN_EMIT_HELPER3(lshr) /* do_lshr3 */ +GEN_EMIT_HELPER2(orcb) /* do_orcb2 */ +GEN_EMIT_HELPER2(one_cmpl) /* do_one_cmpl2 */ +GEN_EMIT_HELPER3(sub) /* do_sub3 */ +GEN_EMIT_HELPER2(th_rev) /* do_th_rev2 */ +GEN_EMIT_HELPER2(th_tstnbz) /* do_th_tstnbz2 */ +GEN_EMIT_HELPER2(zero_extendqi) /* do_zero_extendqi2 */ + +#undef GEN_EMIT_HELPER2 +#undef GEN_EMIT_HELPER3 + +/* Helper function to load a byte or a Pmode register. + + MODE is the mode to use for the load (QImode or Pmode). + DEST is the destination register for the data. + ADDR_REG is the register that holds the address. + ADDR is the address expression to load from. + + This function returns an rtx containing the register, + where the ADDR is stored. */ + +static rtx +do_load_from_addr (machine_mode mode, rtx dest, rtx addr_reg, rtx addr) +{ + rtx mem = gen_rtx_MEM (mode, addr_reg); + MEM_COPY_ATTRIBUTES (mem, addr); + set_mem_size (mem, GET_MODE_SIZE (mode)); + + if (mode == QImode) + do_zero_extendqi2 (dest, mem); + else if (mode == Xmode) + emit_move_insn (dest, mem); + else + gcc_unreachable (); + + return addr_reg; +} + +/* If the provided string is aligned, then read XLEN bytes + in a loop and use orc.b to find NUL-bytes. */ + +static bool +riscv_expand_strlen_scalar (rtx result, rtx src, rtx align) +{ + rtx testval, addr, addr_plus_regsz, word, zeros; + rtx loop_label, cond; + + gcc_assert (TARGET_ZBB || TARGET_XTHEADBB); + + /* The alignment needs to be known and big enough. */ + if (!CONST_INT_P (align) || UINTVAL (align) < GET_MODE_SIZE (Xmode)) + return false; + + testval = gen_reg_rtx (Xmode); + addr = copy_addr_to_reg (XEXP (src, 0)); + addr_plus_regsz = gen_reg_rtx (Pmode); + word = gen_reg_rtx (Xmode); + zeros = gen_reg_rtx (Xmode); + + if (TARGET_ZBB) + emit_insn (gen_rtx_SET (testval, constm1_rtx)); + else + emit_insn (gen_rtx_SET (testval, const0_rtx)); + + do_add3 (addr_plus_regsz, addr, GEN_INT (UNITS_PER_WORD)); + + loop_label = gen_label_rtx (); + emit_label (loop_label); + + /* Load a word and use orc.b/th.tstnbz to find a zero-byte. */ + do_load_from_addr (Xmode, word, addr, src); + do_add3 (addr, addr, GEN_INT (UNITS_PER_WORD)); + if (TARGET_ZBB) + do_orcb2 (word, word); + else + do_th_tstnbz2 (word, word); + cond = gen_rtx_EQ (VOIDmode, word, testval); + emit_unlikely_jump_insn (gen_cbranch4 (Xmode, cond, word, testval, loop_label)); + + /* Calculate the return value by counting zero-bits. */ + if (TARGET_ZBB) + do_one_cmpl2 (word, word); + if (TARGET_BIG_ENDIAN) + do_clz2 (zeros, word); + else if (TARGET_ZBB) + do_ctz2 (zeros, word); + else + { + do_th_rev2 (word, word); + do_clz2 (zeros, word); + } + + do_lshr3 (zeros, zeros, GEN_INT (exact_log2 (BITS_PER_UNIT))); + do_add3 (addr, addr, zeros); + do_sub3 (result, addr, addr_plus_regsz); + + return true; +} + +/* Expand a strlen operation and return true if successful. + Return false if we should let the compiler generate normal + code, probably a strlen call. */ + +bool +riscv_expand_strlen (rtx result, rtx src, rtx search_char, rtx align) +{ + gcc_assert (search_char == const0_rtx); + + if (TARGET_ZBB || TARGET_XTHEADBB) + return riscv_expand_strlen_scalar (result, src, align); + + return false; +} diff --git a/gcc/config/riscv/riscv.md b/gcc/config/riscv/riscv.md index 9da2a9f1c42..e078ebc43cb 100644 --- a/gcc/config/riscv/riscv.md +++ b/gcc/config/riscv/riscv.md @@ -82,6 +82,9 @@ (define_c_enum "unspec" [ ;; the calling convention of callee UNSPEC_CALLEE_CC + + ;; String unspecs + UNSPEC_STRLEN ]) (define_c_enum "unspecv" [ @@ -3500,6 +3503,31 @@ (define_expand "msubhisi4" "TARGET_XTHEADMAC" ) +;; Search character in string (generalization of strlen). +;; Argument 0 is the resulting offset +;; Argument 1 is the string +;; Argument 2 is the search character +;; Argument 3 is the alignment + +(define_expand "strlen" + [(set (match_operand:X 0 "register_operand") + (unspec:X [(match_operand:BLK 1 "general_operand") + (match_operand:SI 2 "const_int_operand") + (match_operand:SI 3 "const_int_operand")] + UNSPEC_STRLEN))] + "riscv_inline_strlen && !optimize_size && (TARGET_ZBB || TARGET_XTHEADBB)" +{ + rtx search_char = operands[2]; + + if (search_char != const0_rtx) + FAIL; + + if (riscv_expand_strlen (operands[0], operands[1], operands[2], operands[3])) + DONE; + else + FAIL; +}) + (include "bitmanip.md") (include "crypto.md") (include "sync.md") diff --git a/gcc/config/riscv/riscv.opt b/gcc/config/riscv/riscv.opt index 98f342348b7..2491b335aef 100644 --- a/gcc/config/riscv/riscv.opt +++ b/gcc/config/riscv/riscv.opt @@ -278,6 +278,10 @@ minline-atomics Target Var(TARGET_INLINE_SUBWORD_ATOMIC) Init(1) Always inline subword atomic operations. +minline-strlen +Target Bool Var(riscv_inline_strlen) Init(0) +Inline strlen calls if possible. + Enum Name(riscv_autovec_preference) Type(enum riscv_autovec_preference_enum) Valid arguments to -param=riscv-autovec-preference=: diff --git a/gcc/config/riscv/t-riscv b/gcc/config/riscv/t-riscv index b1f80d1d87c..c012ac0cf33 100644 --- a/gcc/config/riscv/t-riscv +++ b/gcc/config/riscv/t-riscv @@ -91,6 +91,12 @@ riscv-selftests.o: $(srcdir)/config/riscv/riscv-selftests.cc \ $(COMPILE) $< $(POSTCOMPILE) +riscv-string.o: $(srcdir)/config/riscv/riscv-string.cc \ + $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TARGET_H) backend.h $(RTL_H) \ + memmodel.h $(EMIT_RTL_H) poly-int.h output.h + $(COMPILE) $< + $(POSTCOMPILE) + riscv-v.o: $(srcdir)/config/riscv/riscv-v.cc \ $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) $(RTL_H) \ $(TM_P_H) $(TARGET_H) memmodel.h insn-codes.h $(OPTABS_H) $(RECOG_H) \ diff --git a/gcc/config/riscv/thead.md b/gcc/config/riscv/thead.md index 29f98dec3a8..982b048cb65 100644 --- a/gcc/config/riscv/thead.md +++ b/gcc/config/riscv/thead.md @@ -110,7 +110,7 @@ (define_insn "*th_clz2" [(set_attr "type" "bitmanip") (set_attr "mode" "")]) -(define_insn "*th_rev2" +(define_insn "th_rev2" [(set (match_operand:GPR 0 "register_operand" "=r") (bswap:GPR (match_operand:GPR 1 "register_operand" "r")))] "TARGET_XTHEADBB && (TARGET_64BIT || mode == SImode)" @@ -121,6 +121,13 @@ (define_insn "*th_rev2" [(set_attr "type" "bitmanip") (set_attr "mode" "")]) +(define_insn "th_tstnbz2" + [(set (match_operand:X 0 "register_operand" "=r") + (unspec:X [(match_operand:X 1 "register_operand" "r")] UNSPEC_ORC_B))] + "TARGET_XTHEADBB" + "th.tstnbz\t%0,%1" + [(set_attr "type" "bitmanip")]) + ;; XTheadBs (define_insn "*th_tst3" diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 33befee7d6b..4a9e385d009 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -1236,7 +1236,8 @@ See RS/6000 and PowerPC Options. -mstack-protector-guard=@var{guard} -mstack-protector-guard-reg=@var{reg} -mstack-protector-guard-offset=@var{offset} -mcsr-check -mno-csr-check --minline-atomics -mno-inline-atomics} +-minline-atomics -mno-inline-atomics +-minline-strlen -mno-inline-strlen} @emph{RL78 Options} @gccoptlist{-msim -mmul=none -mmul=g13 -mmul=g14 -mallregs @@ -29359,6 +29360,14 @@ Do or don't use smaller but slower subword atomic emulation code that uses libatomic function calls. The default is to use fast inline subword atomics that do not require libatomic. +@opindex minline-strlen +@item -minline-strlen +@itemx -mno-inline-strlen +Do or do not attempt to inline strlen calls if possible. +Inlining will only be done if the string is properly aligned +and instructions for accelerated processing are available. +The default is to not inline strlen calls. + @opindex mshorten-memrefs @item -mshorten-memrefs @itemx -mno-shorten-memrefs diff --git a/gcc/emit-rtl.cc b/gcc/emit-rtl.cc index f6276a2d0b6..8bd623dcd0e 100644 --- a/gcc/emit-rtl.cc +++ b/gcc/emit-rtl.cc @@ -5168,6 +5168,30 @@ emit_jump_insn (rtx x) return last; } +/* Make an insn of code JUMP_INSN with pattern X, + add a REG_BR_PROB note that indicates very likely probability, + and add it to the end of the doubly-linked list. */ + +rtx_insn * +emit_likely_jump_insn (rtx x) +{ + rtx_insn *jump = emit_jump_insn (x); + add_reg_br_prob_note (jump, profile_probability::very_likely ()); + return jump; +} + +/* Make an insn of code JUMP_INSN with pattern X, + add a REG_BR_PROB note that indicates very unlikely probability, + and add it to the end of the doubly-linked list. */ + +rtx_insn * +emit_unlikely_jump_insn (rtx x) +{ + rtx_insn *jump = emit_jump_insn (x); + add_reg_br_prob_note (jump, profile_probability::very_unlikely ()); + return jump; +} + /* Make an insn of code CALL_INSN with pattern X and add it to the end of the doubly-linked list. */ diff --git a/gcc/rtl.h b/gcc/rtl.h index 0e9491b89b4..102ad9b57a6 100644 --- a/gcc/rtl.h +++ b/gcc/rtl.h @@ -3347,6 +3347,8 @@ extern rtx_note *emit_note_after (enum insn_note, rtx_insn *); extern rtx_insn *emit_insn (rtx); extern rtx_insn *emit_debug_insn (rtx); extern rtx_insn *emit_jump_insn (rtx); +extern rtx_insn *emit_likely_jump_insn (rtx); +extern rtx_insn *emit_unlikely_jump_insn (rtx); extern rtx_insn *emit_call_insn (rtx); extern rtx_code_label *emit_label (rtx); extern rtx_jump_table_data *emit_jump_table_data (rtx); diff --git a/gcc/testsuite/gcc.target/riscv/xtheadbb-strlen-unaligned.c b/gcc/testsuite/gcc.target/riscv/xtheadbb-strlen-unaligned.c new file mode 100644 index 00000000000..57a6b5ea66a --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/xtheadbb-strlen-unaligned.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-minline-strlen -march=rv32gc_xtheadbb" { target { rv32 } } } */ +/* { dg-options "-minline-strlen -march=rv64gc_xtheadbb" { target { rv64 } } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-Os" "-Og" "-Oz" } } */ + +typedef long unsigned int size_t; + +size_t +my_str_len (const char *s) +{ + return __builtin_strlen (s); +} + +/* { dg-final { scan-assembler-not "th.tstnbz\t" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/xtheadbb-strlen.c b/gcc/testsuite/gcc.target/riscv/xtheadbb-strlen.c new file mode 100644 index 00000000000..dbc8d1e7da7 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/xtheadbb-strlen.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* { dg-options "-minline-strlen -march=rv32gc_xtheadbb" { target { rv32 } } } */ +/* { dg-options "-minline-strlen -march=rv64gc_xtheadbb" { target { rv64 } } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-Os" "-Og" "-Oz" } } */ + +typedef long unsigned int size_t; + +size_t +my_str_len (const char *s) +{ + s = __builtin_assume_aligned (s, 4096); + return __builtin_strlen (s); +} + +/* { dg-final { scan-assembler "th.tstnbz\t" } } */ +/* { dg-final { scan-assembler-not "jalr" } } */ +/* { dg-final { scan-assembler-not "call" } } */ +/* { dg-final { scan-assembler-not "jr" } } */ +/* { dg-final { scan-assembler-not "tail" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled-2.c b/gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled-2.c new file mode 100644 index 00000000000..a481068aa0c --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled-2.c @@ -0,0 +1,15 @@ +/* { dg-do compile } */ +/* { dg-options "-march=rv32gc_zbb" { target { rv32 } } } */ +/* { dg-options "-march=rv64gc_zbb" { target { rv64 } } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-Os" "-Og" "-Oz" } } */ + +typedef long unsigned int size_t; + +size_t +my_str_len (const char *s) +{ + s = __builtin_assume_aligned (s, 4096); + return __builtin_strlen (s); +} + +/* { dg-final { scan-assembler-not "orc.b\t" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled.c b/gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled.c new file mode 100644 index 00000000000..1295aeb0086 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/zbb-strlen-disabled.c @@ -0,0 +1,15 @@ +/* { dg-do compile } */ +/* { dg-options "-mno-inline-strlen -march=rv32gc_zbb" { target { rv32 } } } */ +/* { dg-options "-mno-inline-strlen -march=rv64gc_zbb" { target { rv64 } } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-Os" "-Og" "-Oz" } } */ + +typedef long unsigned int size_t; + +size_t +my_str_len (const char *s) +{ + s = __builtin_assume_aligned (s, 4096); + return __builtin_strlen (s); +} + +/* { dg-final { scan-assembler-not "orc.b\t" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/zbb-strlen-unaligned.c b/gcc/testsuite/gcc.target/riscv/zbb-strlen-unaligned.c new file mode 100644 index 00000000000..326fef885d8 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/zbb-strlen-unaligned.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-minline-strlen -march=rv32gc_zbb" { target { rv32 } } } */ +/* { dg-options "-minline-strlen -march=rv64gc_zbb" { target { rv64 } } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-Os" "-Og" "-Oz" } } */ + +typedef long unsigned int size_t; + +size_t +my_str_len (const char *s) +{ + return __builtin_strlen (s); +} + +/* { dg-final { scan-assembler-not "orc.b\t" } } */ diff --git a/gcc/testsuite/gcc.target/riscv/zbb-strlen.c b/gcc/testsuite/gcc.target/riscv/zbb-strlen.c new file mode 100644 index 00000000000..19ebfaef16f --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/zbb-strlen.c @@ -0,0 +1,19 @@ +/* { dg-do compile } */ +/* { dg-options "-minline-strlen -march=rv32gc_zbb" { target { rv32 } } } */ +/* { dg-options "-minline-strlen -march=rv64gc_zbb" { target { rv64 } } } */ +/* { dg-skip-if "" { *-*-* } { "-O0" "-Os" "-Og" "-Oz" } } */ + +typedef long unsigned int size_t; + +size_t +my_str_len (const char *s) +{ + s = __builtin_assume_aligned (s, 4096); + return __builtin_strlen (s); +} + +/* { dg-final { scan-assembler "orc.b\t" } } */ +/* { dg-final { scan-assembler-not "jalr" } } */ +/* { dg-final { scan-assembler-not "call" } } */ +/* { dg-final { scan-assembler-not "jr" } } */ +/* { dg-final { scan-assembler-not "tail" } } */