From patchwork Tue Nov 14 21:52:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xi Ruoyao X-Patchwork-Id: 165101 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:6358:a59:b0:164:83eb:24d7 with SMTP id 25csp2226264rwb; Tue, 14 Nov 2023 13:56:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IHgCki5wHK/md2Knwq/+xOiLrmPHdi5Z/FTjRhghkdTc8MlIS/PUgxWrybfoeCrBYlry+va X-Received: by 2002:a25:74cc:0:b0:d9a:ba4b:44ab with SMTP id p195-20020a2574cc000000b00d9aba4b44abmr9439768ybc.61.1699998989255; Tue, 14 Nov 2023 13:56:29 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1699998989; cv=pass; d=google.com; s=arc-20160816; b=HWzBTmCBrC+0yHjB3H4F8c8npeNPIgZQtwjDxz4kymc45L3DOdG2vu05owTW3pUC4r 4qm9eE2xXEM95pJAn/qnykiYiKMhPvk/T1RlyXxpENfl/dAkLxVttUAeewxxtnU+1jDh hvaZFO7R4WIvY9/flJojsC5pBZKuOE/D4EYnR44bFKGrRHTdaz9ru9yBUJCR5HrN9CUq PG40cisVYCJkg+odxoDQi5mOt28qFLRVc8X8SJ+qcSvt6UojQRjze2N0U+pKXfhTTQX/ y5V6Nzod9vRn7x5KadjaahzxiUpH8fmFFlbWI2e5eK+zzXuwXIprZZM+2IHpgtxo+MlN MQvg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:cc:to:from:dkim-signature :arc-filter:dmarc-filter:delivered-to; bh=vPAcIoqO0ktf7slzazuFkm1+Nnjg9b8CbQzXzxmwAkg=; fh=oUCfM/eMlWtMCtZZKY1bglzxCo7b3kw9D5LTFFWuz38=; b=Sp5qDpbvm0hXXah134qRclWsrbJNOwaZvkoS7xbeXm7LFAZTUai8BxlL+/djAwleSm YW3xkT/SiuQQctetz37ZBFn6smmH6FSETpIa7QR57ct60khhOrDMkzV0AhOHWhCTXkPw Fov8YcNPjdmcWmmwuVSvlNLex/pRQrGBOSqnRkz2cDaXhy9u6rAJv6L7cJ+g1PDiuKP4 GP8XKmUy4dfKG8x/8p4qwDaL8GhEjqGI2u1FKbFvLM/PCcqn8HCRXDMG9NJ1GkaYJw+z CEkBaARGLdezOoQ2eR9peHDi+xA5DK6KKFq+cYfu1DQyt3jmWpiPfR9pLEmtbOsddyMx 4MZA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@xry111.site header.s=default header.b=ElkzZcw5; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=xry111.site Received: from server2.sourceware.org (server2.sourceware.org. [8.43.85.97]) by mx.google.com with ESMTPS id eu20-20020ad44f54000000b0066d127c27fcsi7498610qvb.553.2023.11.14.13.56.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Nov 2023 13:56:29 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) client-ip=8.43.85.97; Authentication-Results: mx.google.com; dkim=pass header.i=@xry111.site header.s=default header.b=ElkzZcw5; arc=pass (i=1); spf=pass (google.com: domain of gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org designates 8.43.85.97 as permitted sender) smtp.mailfrom="gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=xry111.site Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 04DA93858C5F for ; Tue, 14 Nov 2023 21:56:29 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from xry111.site (xry111.site [89.208.246.23]) by sourceware.org (Postfix) with ESMTPS id E1E9B3858D20 for ; Tue, 14 Nov 2023 21:56:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E1E9B3858D20 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=xry111.site Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=xry111.site ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E1E9B3858D20 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=89.208.246.23 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699998965; cv=none; b=tOIchPFJBhQq84NjGIVOxULkbPw1pxCLMBXgmdE78qbLeeciD0CdNQb2d8SA4a0FxY90wlIh2b23VZRNnrlB4LYOWWtDE+/xojUeHvX8V6uv13JI0rHx+KmvLShP4u8i9rFzWSSHmPUnzqJ0gTqWQCli7WshiNQ7uSz+NKJl/7M= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699998965; c=relaxed/simple; bh=QNZRHO+wRRnY0sA+eyvS92st+Mvmazusgi3SS+wQbjc=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=ox0cHjEBPvfGiDZkgQnryzrcfPNV6u8+fwFLjn6SDTRQppn/IFQwUHuJNEOYM09NCekz2C875e4e6NRiog64R4Bhf9m9nULseOVfhH54psumDKCSBAb/58+4yDRMMSxkmDAJjU8jcAxFgHsa506FoXC5bYRI0P4bcVvpr6NxjWo= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=xry111.site; s=default; t=1699998960; bh=QNZRHO+wRRnY0sA+eyvS92st+Mvmazusgi3SS+wQbjc=; h=From:To:Cc:Subject:Date:From; b=ElkzZcw5l9T8nMa4MMR6pj+tQBloFkgJpnKud2nOQOJ/0CYPC9MKW0sBS+r36sDxD /NEq4iKHybabWeAFb+j6iWliYYHLFEBeX+IhjRoXwYV4NOEayMSEmFlyT6gdl3OGus fxV27DqZ+Js0q4eCxYVjIwb23slBWlotBUi9/v1c= Received: from stargazer.. (unknown [IPv6:240e:358:11dd:4400:dc73:854d:832e:2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) (Authenticated sender: xry111@xry111.site) by xry111.site (Postfix) with ESMTPSA id 919BB66A03; Tue, 14 Nov 2023 16:55:57 -0500 (EST) From: Xi Ruoyao To: gcc-patches@gcc.gnu.org Cc: chenglulu , i@xen0n.name, xuchenghua@loongson.cn, Xi Ruoyao Subject: [PATCH v2] LoongArch: Remove redundant barrier instructions before LL-SC loops Date: Wed, 15 Nov 2023 05:52:28 +0800 Message-ID: <20231114215404.163508-2-xry111@xry111.site> X-Mailer: git-send-email 2.42.1 MIME-Version: 1.0 X-Spam-Status: No, score=-8.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, LIKELY_SPAM_FROM, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+ouuuleilei=gmail.com@gcc.gnu.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782578139974323713 X-GMAIL-MSGID: 1782578139974323713 This is isomorphic to the LLVM changes [1-2]. On LoongArch, the LL and SC instructions has memory barrier semantics: - LL: + - SC: + But the compare and swap operation is allowed to fail, and if it fails the SC instruction is not executed, thus the guarantee of acquiring semantics cannot be ensured. Therefore, an acquire barrier needs to be generated when failure_memorder includes an acquire operation. On CPUs implementing LoongArch v1.10 or later, "dbar 0b10100" is an acquire barrier; on CPUs implementing LoongArch v1.00, it is a full barrier. So it's always enough for acquire semantics. OTOH if an acquire semantic is not needed, we still needs the "dbar 0x700" as the load-load barrier like all LL-SC loops. [1]:https://github.com/llvm/llvm-project/pull/67391 [2]:https://github.com/llvm/llvm-project/pull/69339 gcc/ChangeLog: * config/loongarch/loongarch.cc (loongarch_memmodel_needs_release_fence): Remove. (loongarch_cas_failure_memorder_needs_acquire): New static function. (loongarch_print_operand): Redefine 'G' for the barrier on CAS failure. * config/loongarch/sync.md (atomic_cas_value_strong): Remove the redundant barrier before the LL instruction, and emit an acquire barrier on failure if needed by failure_memorder. (atomic_cas_value_cmp_and_7_): Likewise. (atomic_cas_value_add_7_): Remove the unnecessary barrier before the LL instruction. (atomic_cas_value_sub_7_): Likewise. (atomic_cas_value_and_7_): Likewise. (atomic_cas_value_xor_7_): Likewise. (atomic_cas_value_or_7_): Likewise. (atomic_cas_value_nand_7_): Likewise. (atomic_cas_value_exchange_7_): Likewise. gcc/testsuite/ChangeLog: * gcc.target/loongarch/cas-acquire.c: New test. --- v1: https://gcc.gnu.org/pipermail/gcc-patches/2023-November/635304.html Changes from v1: - Remove yet not reachable "case MEMMODEL_CONSUME". - Fix the test case so it almost always fail with (buggy) GCC 13 and LA664. Bootstrapped and regtested on loongarch64-linux-gnu running on LA664. Ok for trunk? gcc/config/loongarch/loongarch.cc | 30 ++++--- gcc/config/loongarch/sync.md | 49 +++++------ .../gcc.target/loongarch/cas-acquire.c | 82 +++++++++++++++++++ 3 files changed, 119 insertions(+), 42 deletions(-) create mode 100644 gcc/testsuite/gcc.target/loongarch/cas-acquire.c diff --git a/gcc/config/loongarch/loongarch.cc b/gcc/config/loongarch/loongarch.cc index 2998bf740d4..738911661d7 100644 --- a/gcc/config/loongarch/loongarch.cc +++ b/gcc/config/loongarch/loongarch.cc @@ -5833,27 +5833,27 @@ loongarch_memmodel_needs_rel_acq_fence (enum memmodel model) } } -/* Return true if a FENCE should be emitted to before a memory access to - implement the release portion of memory model MODEL. */ +/* Return true if a FENCE should be emitted after a failed CAS to + implement the acquire semantic of failure_memorder. */ static bool -loongarch_memmodel_needs_release_fence (enum memmodel model) +loongarch_cas_failure_memorder_needs_acquire (enum memmodel model) { - switch (model) + switch (memmodel_base (model)) { + case MEMMODEL_ACQUIRE: case MEMMODEL_ACQ_REL: case MEMMODEL_SEQ_CST: - case MEMMODEL_SYNC_SEQ_CST: - case MEMMODEL_RELEASE: - case MEMMODEL_SYNC_RELEASE: return true; - case MEMMODEL_ACQUIRE: - case MEMMODEL_CONSUME: - case MEMMODEL_SYNC_ACQUIRE: case MEMMODEL_RELAXED: + case MEMMODEL_RELEASE: return false; + /* MEMMODEL_CONSUME is deliberately not handled because it's always + replaced by MEMMODEL_ACQUIRE as at now. If you see an ICE caused by + MEMMODEL_CONSUME, read the change (re)introducing it carefully and + decide what to do. See PR 59448 and get_memmodel in builtins.cc. */ default: gcc_unreachable (); } @@ -5966,7 +5966,8 @@ loongarch_print_operand_reloc (FILE *file, rtx op, bool hi64_part, 'd' Print CONST_INT OP in decimal. 'E' Print CONST_INT OP element 0 of a replicated CONST_VECTOR in decimal. 'F' Print the FPU branch condition for comparison OP. - 'G' Print a DBAR insn if the memory model requires a release. + 'G' Print a DBAR insn for CAS failure (with an acquire semantic if + needed, otherwise a simple load-load barrier). 'H' Print address 52-61bit relocation associated with OP. 'h' Print the high-part relocation associated with OP. 'i' Print i if the operand is not a register. @@ -6057,8 +6058,11 @@ loongarch_print_operand (FILE *file, rtx op, int letter) break; case 'G': - if (loongarch_memmodel_needs_release_fence ((enum memmodel) INTVAL (op))) - fputs ("dbar\t0", file); + if (loongarch_cas_failure_memorder_needs_acquire ( + memmodel_from_int (INTVAL (op)))) + fputs ("dbar\t0b10100", file); + else + fputs ("dbar\t0x700", file); break; case 'h': diff --git a/gcc/config/loongarch/sync.md b/gcc/config/loongarch/sync.md index 1ad0c63e0d9..c112091a60f 100644 --- a/gcc/config/loongarch/sync.md +++ b/gcc/config/loongarch/sync.md @@ -162,19 +162,18 @@ (define_insn "atomic_cas_value_strong" (clobber (match_scratch:GPR 6 "=&r"))] "" { - return "%G5\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "bne\\t%0,%z2,2f\\n\\t" "or%i3\\t%6,$zero,%3\\n\\t" "sc.\\t%6,%1\\n\\t" - "beq\\t$zero,%6,1b\\n\\t" + "beqz\\t%6,1b\\n\\t" "b\\t3f\\n\\t" "2:\\n\\t" - "dbar\\t0x700\\n\\t" + "%G5\\n\\t" "3:\\n\\t"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_expand "atomic_compare_and_swap" [(match_operand:SI 0 "register_operand" "") ;; bool output @@ -267,8 +266,7 @@ (define_insn "atomic_cas_value_cmp_and_7_" (clobber (match_scratch:GPR 7 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%2\\n\\t" "bne\\t%7,%z4,2f\\n\\t" @@ -278,10 +276,10 @@ (define_insn "atomic_cas_value_cmp_and_7_" "beq\\t$zero,%7,1b\\n\\t" "b\\t3f\\n\\t" "2:\\n\\t" - "dbar\\t0x700\\n\\t" + "%G6\\n\\t" "3:\\n\\t"; } - [(set (attr "length") (const_int 40))]) + [(set (attr "length") (const_int 36))]) (define_expand "atomic_compare_and_swap" [(match_operand:SI 0 "register_operand" "") ;; bool output @@ -336,8 +334,7 @@ (define_insn "atomic_cas_value_add_7_" (clobber (match_scratch:GPR 8 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%3\\n\\t" "add.w\\t%8,%0,%z5\\n\\t" @@ -347,7 +344,7 @@ (define_insn "atomic_cas_value_add_7_" "beq\\t$zero,%7,1b"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_insn "atomic_cas_value_sub_7_" [(set (match_operand:GPR 0 "register_operand" "=&r") ;; res @@ -363,8 +360,7 @@ (define_insn "atomic_cas_value_sub_7_" (clobber (match_scratch:GPR 8 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%3\\n\\t" "sub.w\\t%8,%0,%z5\\n\\t" @@ -373,7 +369,7 @@ (define_insn "atomic_cas_value_sub_7_" "sc.\\t%7,%1\\n\\t" "beq\\t$zero,%7,1b"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_insn "atomic_cas_value_and_7_" [(set (match_operand:GPR 0 "register_operand" "=&r") ;; res @@ -389,8 +385,7 @@ (define_insn "atomic_cas_value_and_7_" (clobber (match_scratch:GPR 8 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%3\\n\\t" "and\\t%8,%0,%z5\\n\\t" @@ -399,7 +394,7 @@ (define_insn "atomic_cas_value_and_7_" "sc.\\t%7,%1\\n\\t" "beq\\t$zero,%7,1b"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_insn "atomic_cas_value_xor_7_" [(set (match_operand:GPR 0 "register_operand" "=&r") ;; res @@ -415,8 +410,7 @@ (define_insn "atomic_cas_value_xor_7_" (clobber (match_scratch:GPR 8 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%3\\n\\t" "xor\\t%8,%0,%z5\\n\\t" @@ -426,7 +420,7 @@ (define_insn "atomic_cas_value_xor_7_" "beq\\t$zero,%7,1b"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_insn "atomic_cas_value_or_7_" [(set (match_operand:GPR 0 "register_operand" "=&r") ;; res @@ -442,8 +436,7 @@ (define_insn "atomic_cas_value_or_7_" (clobber (match_scratch:GPR 8 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%3\\n\\t" "or\\t%8,%0,%z5\\n\\t" @@ -453,7 +446,7 @@ (define_insn "atomic_cas_value_or_7_" "beq\\t$zero,%7,1b"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_insn "atomic_cas_value_nand_7_" [(set (match_operand:GPR 0 "register_operand" "=&r") ;; res @@ -469,8 +462,7 @@ (define_insn "atomic_cas_value_nand_7_" (clobber (match_scratch:GPR 8 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%3\\n\\t" "and\\t%8,%0,%z5\\n\\t" @@ -479,7 +471,7 @@ (define_insn "atomic_cas_value_nand_7_" "sc.\\t%7,%1\\n\\t" "beq\\t$zero,%7,1b"; } - [(set (attr "length") (const_int 32))]) + [(set (attr "length") (const_int 28))]) (define_insn "atomic_cas_value_exchange_7_" [(set (match_operand:GPR 0 "register_operand" "=&r") @@ -494,8 +486,7 @@ (define_insn "atomic_cas_value_exchange_7_" (clobber (match_scratch:GPR 7 "=&r"))] "" { - return "%G6\\n\\t" - "1:\\n\\t" + return "1:\\n\\t" "ll.\\t%0,%1\\n\\t" "and\\t%7,%0,%z3\\n\\t" "or%i5\\t%7,%7,%5\\n\\t" diff --git a/gcc/testsuite/gcc.target/loongarch/cas-acquire.c b/gcc/testsuite/gcc.target/loongarch/cas-acquire.c new file mode 100644 index 00000000000..ff7ba866f32 --- /dev/null +++ b/gcc/testsuite/gcc.target/loongarch/cas-acquire.c @@ -0,0 +1,82 @@ +/* { dg-do run } */ +/* { dg-require-effective-target c99_runtime } */ +/* { dg-require-effective-target pthread } */ +/* { dg-options "-std=c99 -pthread" } */ + +/* https://github.com/llvm/llvm-project/pull/67391#issuecomment-1752403934 + reported that this had failed with GCC and 3A6000. */ + +#include +#include +#include +#include + +static unsigned int tags[32]; +static unsigned int vals[32]; + +static void * +writer_entry (void *data) +{ + atomic_uint *pt = (atomic_uint *)tags; + atomic_uint *pv = (atomic_uint *)vals; + + for (unsigned int n = 1; n < 10000; n++) + { + atomic_store_explicit (&pv[n & 31], n, memory_order_release); + atomic_store_explicit (&pt[n & 31], n, memory_order_release); + } + + return NULL; +} + +static void * +reader_entry (void *data) +{ + atomic_uint *pt = (atomic_uint *)tags; + atomic_uint *pv = (atomic_uint *)vals; + int i; + + for (;;) + { + for (i = 0; i < 32; i++) + { + unsigned int tag = 0; + bool res; + + res = atomic_compare_exchange_weak_explicit ( + &pt[i], &tag, 0, memory_order_acquire, memory_order_acquire); + if (!res) + { + unsigned int val; + + val = atomic_load_explicit (&pv[i], memory_order_relaxed); + if (val < tag) + __builtin_trap (); + } + } + } + + return NULL; +} + +int +main (int argc, char *argv[]) +{ + pthread_t writer; + pthread_t reader; + int res; + + res = pthread_create (&writer, NULL, writer_entry, NULL); + if (res < 0) + __builtin_trap (); + + res = pthread_create (&reader, NULL, reader_entry, NULL); + if (res < 0) + __builtin_trap (); + + res = pthread_join (writer, NULL); + if (res < 0) + __builtin_trap (); + + return 0; +}