From patchwork Mon Oct 9 17:49:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 150247 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:403:3b70:6f57 with SMTP id x8csp2030289vqo; Mon, 9 Oct 2023 10:49:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEnnGinB6oKKCQRpXuoTBadiyl57DJGfokDOFIBH2XEXYWL/YwSOI+ZxFPtbh2sPjV6r/q+ X-Received: by 2002:a17:902:e5d0:b0:1c5:6157:f073 with SMTP id u16-20020a170902e5d000b001c56157f073mr18798492plf.11.1696873783510; Mon, 09 Oct 2023 10:49:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696873783; cv=none; d=google.com; s=arc-20160816; b=pKbO/0e+H45G+rYgJs8eJakTSAo94X14w7ZZoWUnIiDHze0TM7Tp7ZxBwbnD+a1iLD b7dazQbBMBmE/G+htA/kDFu2YW/Dp/HC2BwwsUVwEergJIHEX/U/hTI8gugrsgVuBpGN 2uwkvnQUUDML7tTzFXDyBSS6rbA1RZlg9WxuCVIsXi7KeF5mjS17lfsZoYnkpuDaIXm3 WBKDTWy6kjaTk+Nyc8OeE2UXO65W7OfIB3PNPc3BgZWIRQFCCBseKkdYugdtxTDHcPza rKhOjmDGkC6Dc1kGu5PxTUErJ1DBFukQtETPHO67y4QauglVpS1LAjT3K7WJYSM2jpv6 BN5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:cc:subject:to:reply-to:sender:from :dkim-signature:dkim-signature:date; bh=AWNPjlwwdK1lk4c+xJcA60ZLb6nJMrrz1GEMuSFR7G8=; fh=TY/Nqd9UB5ZonvZXTnCDtuzCcyJZxfUvtNY2gYP401I=; b=C7ipFOWcEN8zJtCh+AfE3l3xIJfCHh1HkLcr8t/Kmg71M9e6EwHHGMH2QbkZ/V0/v0 7SVH8826GIHm9aplZOhrljvm6XPxbe8j+4A/f2yZ7gI1GorGE/6ViD4jtMldZu3j9DwI SpAv41xW9bliQ1X5PEFjMHrLq3/M2I9jmxxfGLbiEE96qXeFfcFQg4b+yKNhQ1xh4GX+ gN/mBi14k93OFqcwTfxOYd9ZHL87OXxszJvJnxV7JISNDR2PGU77mOi1wt5x49D0J/Rt qwYh6n3ljo+ByAC20Pz1qybCn+1rZWL9yIZJgH6tv6tI8t97exEH+CmlICfzs98cbZIx W+Ig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=ZHscMiMQ; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id k4-20020a170902d58400b001c73d73f163si9919792plh.505.2023.10.09.10.49.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 10:49:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=ZHscMiMQ; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 7156E81C809C; Mon, 9 Oct 2023 10:49:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377964AbjJIRtV (ORCPT + 18 others); Mon, 9 Oct 2023 13:49:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1377937AbjJIRtO (ORCPT ); Mon, 9 Oct 2023 13:49:14 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21450A3; Mon, 9 Oct 2023 10:49:11 -0700 (PDT) Date: Mon, 09 Oct 2023 17:49:08 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1696873749; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=AWNPjlwwdK1lk4c+xJcA60ZLb6nJMrrz1GEMuSFR7G8=; b=ZHscMiMQnF93wFaJzBe56PlLgnZrp9ihOJzU+UIDK3HXiAAfWx4nOVRuSTsRguGjoKBXDE 9Qmk2wtk6IL9zrH5e5qWJ6frQQ+QUeNFPZkg1mE6veGesYWEwIbGmlQhITZdXz9hn6qhN4 4xrTZWKcb2c3g8LSPbMAwk7QOm7sZmj1jvKypKDxUxsZ+IvrVgUhfSHt0sgJd0JkDqwcoT Wnh6RvewZ3pUAGyuaOPBVb5tDcZZZaIZOSyxEg9Fq74NfSQ49t6lqnS0BfOH18RaTRetvs S3NHt9DXItG63sCQ57KmYAPjpoHbXBXS6VeAJND1N9F5r27JQB3ei1pYlvfcVQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1696873749; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=AWNPjlwwdK1lk4c+xJcA60ZLb6nJMrrz1GEMuSFR7G8=; b=7tTIJ6OTk7Ky84tUX3lp1hT/76pM7LGerab4LXyQvaPTaTC37UroLbpzIUrVt4XVYTOaWq 2cVdMo0p5JnWtBDQ== From: "tip-bot2 for Uros Bizjak" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: locking/core] locking/atomic: Add generic support for sync_try_cmpxchg() and its fallback Cc: Uros Bizjak , Ingo Molnar , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Linus Torvalds , linux-kernel@vger.kernel.org, x86@kernel.org MIME-Version: 1.0 Message-ID: <169687374861.3135.9482250979449813044.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=2.7 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 09 Oct 2023 10:49:40 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779301124353785397 X-GMAIL-MSGID: 1779301124353785397 The following commit has been merged into the locking/core branch of tip: Commit-ID: e01cc1e8c2ad73cebb980878ede5584e0f2688f7 Gitweb: https://git.kernel.org/tip/e01cc1e8c2ad73cebb980878ede5584e0f2688f7 Author: Uros Bizjak AuthorDate: Mon, 25 Sep 2023 16:50:23 +02:00 Committer: Ingo Molnar CommitterDate: Mon, 09 Oct 2023 18:14:15 +02:00 locking/atomic: Add generic support for sync_try_cmpxchg() and its fallback Provide the generic sync_try_cmpxchg() function from the raw_ prefixed version, also adding explicit instrumentation. The patch amends existing scripts to generate sync_try_cmpxchg() locking primitive and its raw_sync_try_cmpxchg() fallback, while leaving existing macros from the try_cmpxchg() family unchanged. The target can define its own arch_sync_try_cmpxchg() to override the generic version of raw_sync_try_cmpxchg(). This allows the target to generate more optimal assembly than the generic version. Additionally, the patch renames two scripts to better reflect whet they really do. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Cc: Will Deacon Cc: Peter Zijlstra Cc: Boqun Feng Cc: Mark Rutland Cc: Linus Torvalds Cc: linux-kernel@vger.kernel.org --- include/linux/atomic/atomic-arch-fallback.h | 15 ++++++++- include/linux/atomic/atomic-instrumented.h | 10 +++++- scripts/atomic/gen-atomic-fallback.sh | 33 ++++++++++---------- scripts/atomic/gen-atomic-instrumented.sh | 3 +- 4 files changed, 43 insertions(+), 18 deletions(-) diff --git a/include/linux/atomic/atomic-arch-fallback.h b/include/linux/atomic/atomic-arch-fallback.h index b83ef19..5e95faa 100644 --- a/include/linux/atomic/atomic-arch-fallback.h +++ b/include/linux/atomic/atomic-arch-fallback.h @@ -428,6 +428,19 @@ extern void raw_cmpxchg128_relaxed_not_implemented(void); #define raw_sync_cmpxchg arch_sync_cmpxchg +#ifdef arch_sync_try_cmpxchg +#define raw_sync_try_cmpxchg arch_sync_try_cmpxchg +#else +#define raw_sync_try_cmpxchg(_ptr, _oldp, _new) \ +({ \ + typeof(*(_ptr)) *___op = (_oldp), ___o = *___op, ___r; \ + ___r = raw_sync_cmpxchg((_ptr), ___o, (_new)); \ + if (unlikely(___r != ___o)) \ + *___op = ___r; \ + likely(___r == ___o); \ +}) +#endif + /** * raw_atomic_read() - atomic load with relaxed ordering * @v: pointer to atomic_t @@ -4649,4 +4662,4 @@ raw_atomic64_dec_if_positive(atomic64_t *v) } #endif /* _LINUX_ATOMIC_FALLBACK_H */ -// 2fdd6702823fa842f9cea57a002e6e4476ae780c +// eec048affea735b8464f58e6d96992101f8f85f1 diff --git a/include/linux/atomic/atomic-instrumented.h b/include/linux/atomic/atomic-instrumented.h index d401b40..54d7bbe 100644 --- a/include/linux/atomic/atomic-instrumented.h +++ b/include/linux/atomic/atomic-instrumented.h @@ -4998,6 +4998,14 @@ atomic_long_dec_if_positive(atomic_long_t *v) raw_try_cmpxchg128_local(__ai_ptr, __ai_oldp, __VA_ARGS__); \ }) +#define sync_try_cmpxchg(ptr, ...) \ +({ \ + typeof(ptr) __ai_ptr = (ptr); \ + kcsan_mb(); \ + instrument_atomic_read_write(__ai_ptr, sizeof(*__ai_ptr)); \ + raw_sync_try_cmpxchg(__ai_ptr, __VA_ARGS__); \ +}) + #endif /* _LINUX_ATOMIC_INSTRUMENTED_H */ -// 1568f875fef72097413caab8339120c065a39aa4 +// 2cc4bc990fef44d3836ec108f11b610f3f438184 diff --git a/scripts/atomic/gen-atomic-fallback.sh b/scripts/atomic/gen-atomic-fallback.sh index a45154c..f80d69c 100755 --- a/scripts/atomic/gen-atomic-fallback.sh +++ b/scripts/atomic/gen-atomic-fallback.sh @@ -223,14 +223,15 @@ gen_xchg_fallbacks() gen_try_cmpxchg_fallback() { + local prefix="$1"; shift local cmpxchg="$1"; shift; - local order="$1"; shift; + local suffix="$1"; shift; cat <