From patchwork Fri Sep 15 11:25:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 140559 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp1196995vqi; Fri, 15 Sep 2023 10:09:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF47qDoR8vf1XfmF4T65T0/35BTrNlCv3DfDxhANwU8waJL04FGXVEaIUGbHAvE8QDIR+/T X-Received: by 2002:a17:90b:3003:b0:263:f4cc:a988 with SMTP id hg3-20020a17090b300300b00263f4cca988mr2149350pjb.5.1694797763477; Fri, 15 Sep 2023 10:09:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694797763; cv=none; d=google.com; s=arc-20160816; b=hQlKhcKuoALFlPxqkiEamw3u4r9xoijM8/j2hcr2WYXf4dBRlgN8QrPU2COPMa5KGR FCRnx4cEa8vqfsYzI/xFN5cLPlHqn/ZeuOPqxPkdOSFacA5NQyAYtCB6sl1PtLhgHlBx DgRYJRnD/nzDSY+j9bMWebj/k3nuAAseMcWa4ZnK+mM7MbJn09rtR+Q8c8zRRRcrWr3D qrJxzQiq8v95SPm7goqfYqjpLl1qbIcpzWdTr3FQhvt+5HV+f8HlNX04WuBZ73yIgaTb lZjRODYR88Y5AHTg9jmE9BEPJmhD1N/jb+TY7pdBv6+KTQFFSxR0vwDAegVpTOHXM6FW i/uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=u/ouAHMy0z5FMxazVqTNkA5KNv/eBXdXO68JiIIQxpg=; fh=lC3L+tLu9jMrTwLXdci/D+zTDmYl6oRylGrfIkeVy60=; b=m7hSjXEht54nwcI9GTrQSCDQ6489Wi81/WsYpn4OcsutM3P847r8/+JNgnfWqdwgEC 6IpDBx7GL6Ykqm4IOOtf1VQKcv7iFdst2sI7FLmHcgti+54biUNVRhvnl/Q6Z1p0zVMe 00ygPbt43cUMuEDrV7Q5N6GTpAYUAmhMR0Ty9313M6gklrEP2U2mCezMEKku+fKiI0F1 t4mBMmMB9hu2BGcFeOq+5AcliNjT1I9CtCWGQlX1dS2/7B+/7vCe1ggGcoez0K0SDADX t/co0j8zRDC6jnFxeDHSt0GxZc/IKxFdgjO/EkA9IadRAKQeO6TqP1V1wzyf0o+geZld 7Xlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=W+gunOOw; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id ck1-20020a17090afe0100b0026b51ae4574si5692565pjb.36.2023.09.15.10.09.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 10:09:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=W+gunOOw; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 0095183B2D58; Fri, 15 Sep 2023 04:25:25 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234343AbjIOLZS (ORCPT + 32 others); Fri, 15 Sep 2023 07:25:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234347AbjIOLZL (ORCPT ); Fri, 15 Sep 2023 07:25:11 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C9E71B1; Fri, 15 Sep 2023 04:25:04 -0700 (PDT) Date: Fri, 15 Sep 2023 11:25:01 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1694777102; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u/ouAHMy0z5FMxazVqTNkA5KNv/eBXdXO68JiIIQxpg=; b=W+gunOOwKsaIQU99f1bUR8e7wi9zChtZ5RY1JsA84oQRM6bwwZ/dqvvQ38MHMkKDKxMKcK d7i5fGlgZStkk5e5pNj+ZsrWJzDj4HYr99Hht40B7hJUQVAdm5g0QEpdHYA8UIjMrqb4n8 b2m/r1KSgx96HRpmZnSy3DBCtkQVU2ccTjsD7X+clZSoi16q/P1dm+TWYDrug1kE0nxHT4 LOlj5UQVZ24GKgcWVEmGPeq3ezRcLO4xhQmugKT03aMLY0pFpZl3L7aEzw0akpu/kVdAND Ese4+IdK3RmR1mEgsxTQwJX5CSdi6bUoIfnoHL/iP13EvNVmCD8jP7d2vlHe9w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1694777102; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u/ouAHMy0z5FMxazVqTNkA5KNv/eBXdXO68JiIIQxpg=; b=uqm+XEA91+TIWX8qsKH5kjjcpb8JafPnHYDbW2RTI6B4Ml8u6ssgnoCFMvWgUY39+rv0du Ow4pgxuEZffQ/AAQ== From: "tip-bot2 for Uros Bizjak" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/asm] x86/percpu: Define raw_cpu_try_cmpxchg and this_cpu_try_cmpxchg() Cc: Uros Bizjak , Ingo Molnar , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20230830151623.3900-1-ubizjak@gmail.com> References: <20230830151623.3900-1-ubizjak@gmail.com> MIME-Version: 1.0 Message-ID: <169477710197.27769.9358331675930285966.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Fri, 15 Sep 2023 04:25:25 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777124259401725737 X-GMAIL-MSGID: 1777124259401725737 The following commit has been merged into the x86/asm branch of tip: Commit-ID: 5f863897d964e834a0da35b1e483b5bb8faca522 Gitweb: https://git.kernel.org/tip/5f863897d964e834a0da35b1e483b5bb8faca522 Author: Uros Bizjak AuthorDate: Wed, 30 Aug 2023 17:13:56 +02:00 Committer: Ingo Molnar CommitterDate: Fri, 15 Sep 2023 13:18:23 +02:00 x86/percpu: Define raw_cpu_try_cmpxchg and this_cpu_try_cmpxchg() Define target-specific raw_cpu_try_cmpxchg_N() and this_cpu_try_cmpxchg_N() macros. These definitions override the generic fallback definitions and enable target-specific optimized implementations. Signed-off-by: Uros Bizjak Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20230830151623.3900-1-ubizjak@gmail.com --- arch/x86/include/asm/percpu.h | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index 4c36419..a87db61 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -210,6 +210,25 @@ do { \ (typeof(_var))(unsigned long) pco_old__; \ }) +#define percpu_try_cmpxchg_op(size, qual, _var, _ovalp, _nval) \ +({ \ + bool success; \ + __pcpu_type_##size *pco_oval__ = (__pcpu_type_##size *)(_ovalp); \ + __pcpu_type_##size pco_old__ = *pco_oval__; \ + __pcpu_type_##size pco_new__ = __pcpu_cast_##size(_nval); \ + asm qual (__pcpu_op2_##size("cmpxchg", "%[nval]", \ + __percpu_arg([var])) \ + CC_SET(z) \ + : CC_OUT(z) (success), \ + [oval] "+a" (pco_old__), \ + [var] "+m" (_var) \ + : [nval] __pcpu_reg_##size(, pco_new__) \ + : "memory"); \ + if (unlikely(!success)) \ + *pco_oval__ = pco_old__; \ + likely(success); \ +}) + #if defined(CONFIG_X86_32) && !defined(CONFIG_UML) #define percpu_cmpxchg64_op(size, qual, _var, _oval, _nval) \ ({ \ @@ -410,6 +429,9 @@ do { \ #define raw_cpu_cmpxchg_1(pcp, oval, nval) percpu_cmpxchg_op(1, , pcp, oval, nval) #define raw_cpu_cmpxchg_2(pcp, oval, nval) percpu_cmpxchg_op(2, , pcp, oval, nval) #define raw_cpu_cmpxchg_4(pcp, oval, nval) percpu_cmpxchg_op(4, , pcp, oval, nval) +#define raw_cpu_try_cmpxchg_1(pcp, ovalp, nval) percpu_try_cmpxchg_op(1, , pcp, ovalp, nval) +#define raw_cpu_try_cmpxchg_2(pcp, ovalp, nval) percpu_try_cmpxchg_op(2, , pcp, ovalp, nval) +#define raw_cpu_try_cmpxchg_4(pcp, ovalp, nval) percpu_try_cmpxchg_op(4, , pcp, ovalp, nval) #define this_cpu_add_return_1(pcp, val) percpu_add_return_op(1, volatile, pcp, val) #define this_cpu_add_return_2(pcp, val) percpu_add_return_op(2, volatile, pcp, val) @@ -417,6 +439,9 @@ do { \ #define this_cpu_cmpxchg_1(pcp, oval, nval) percpu_cmpxchg_op(1, volatile, pcp, oval, nval) #define this_cpu_cmpxchg_2(pcp, oval, nval) percpu_cmpxchg_op(2, volatile, pcp, oval, nval) #define this_cpu_cmpxchg_4(pcp, oval, nval) percpu_cmpxchg_op(4, volatile, pcp, oval, nval) +#define this_cpu_try_cmpxchg_1(pcp, ovalp, nval) percpu_try_cmpxchg_op(1, volatile, pcp, ovalp, nval) +#define this_cpu_try_cmpxchg_2(pcp, ovalp, nval) percpu_try_cmpxchg_op(2, volatile, pcp, ovalp, nval) +#define this_cpu_try_cmpxchg_4(pcp, ovalp, nval) percpu_try_cmpxchg_op(4, volatile, pcp, ovalp, nval) /* * Per cpu atomic 64 bit operations are only available under 64 bit. @@ -431,6 +456,7 @@ do { \ #define raw_cpu_add_return_8(pcp, val) percpu_add_return_op(8, , pcp, val) #define raw_cpu_xchg_8(pcp, nval) raw_percpu_xchg_op(pcp, nval) #define raw_cpu_cmpxchg_8(pcp, oval, nval) percpu_cmpxchg_op(8, , pcp, oval, nval) +#define raw_cpu_try_cmpxchg_8(pcp, ovalp, nval) percpu_try_cmpxchg_op(8, , pcp, ovalp, nval) #define this_cpu_read_8(pcp) percpu_from_op(8, volatile, "mov", pcp) #define this_cpu_write_8(pcp, val) percpu_to_op(8, volatile, "mov", (pcp), val) @@ -440,6 +466,7 @@ do { \ #define this_cpu_add_return_8(pcp, val) percpu_add_return_op(8, volatile, pcp, val) #define this_cpu_xchg_8(pcp, nval) percpu_xchg_op(8, volatile, pcp, nval) #define this_cpu_cmpxchg_8(pcp, oval, nval) percpu_cmpxchg_op(8, volatile, pcp, oval, nval) +#define this_cpu_try_cmpxchg_8(pcp, ovalp, nval) percpu_try_cmpxchg_op(8, volatile, pcp, ovalp, nval) #endif static __always_inline bool x86_this_cpu_constant_test_bit(unsigned int nr,