From patchwork Thu Aug 10 04:03:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133694 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp204864vqi; Wed, 9 Aug 2023 22:46:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF5U01dgzlGlsnQyJf/n0wnbXLOiw/ElYfH4kdpoEsi+eT8sqxXpRg4z2EY4QpiWb71uM41 X-Received: by 2002:a17:906:3107:b0:99b:f392:10b0 with SMTP id 7-20020a170906310700b0099bf39210b0mr1135719ejx.37.1691646359850; Wed, 09 Aug 2023 22:45:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691646359; cv=none; d=google.com; s=arc-20160816; b=MFYsKPH3fzbLpEQIqFYt9ctE4ROdjFUsX6bdhcYseaDN5IqKaufJ9MYqbQwFUDTyZP 6ojGaY5+pcDdYmC/9xuRnr46a6gJU2ggTnX88GIh1y2oXjr9Uv2kkN+Kti0QeLB9751U wNwjGnJ/Bm+7/nqxbRJ6DOHuaBx8RF6Wk1k/cCL/V/aY5/dMbpjQFzn+/qUrcTwxj4Ra 50ErfBXTDpUzLdlHTgeO6FlecXd9DZOi64JU+u0ViHFl//4KXVd9bMXCqA94RsVNbTe+ fGXlvIcFEyrcaN8+6zLiOI/g4BdLvB+/XyIR9WdkKoT1kRXal09WKRiHtda1GlqY93hR N5vQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OScrv20Vk5PZ28CoHB13qKk09OrWPyCgb0XsKA59wTM=; fh=/HAhUCSgceMdCiMxlR/A+InTwiEDI9nNuAOlAQnSzJk=; b=Tf5MtURp9MRbckqhuVrqFUaAeXp/Tha51e8KEghnbOz2sq+f1iDnt6cU7uwKHtdTsb tiq2UG85mc/DJAtVlqwFTWZRoj3KqzjZ22KyN95O3pkB+w/rt3YdGUSwX9/ryz/fTkJz NJwuFjc8ZxkWCl3qNuw6utSmOZ94W1gAJVD9+XFiHdERPtFxF0DrE8CsUYgmvwIarK+7 3Y8M00366wLD3l85jcMmchZf4HMC7o8sqF9nF1SgMyE5HZ2WTTnQoKikvUaxA8zpQiRy fmZr2X7GEEY/peTJ3LBcNGyy2xm/cHeeWvYUwI5bpVyUAGGalcvucD3WVPcmUAT9ZFca T/GQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=AEL+gLRI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i27-20020a170906265b00b0099b41f2c40fsi798311ejc.924.2023.08.09.22.45.36; Wed, 09 Aug 2023 22:45:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=AEL+gLRI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230123AbjHJEFA (ORCPT + 99 others); Thu, 10 Aug 2023 00:05:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229518AbjHJEE5 (ORCPT ); Thu, 10 Aug 2023 00:04:57 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF3E311F for ; Wed, 9 Aug 2023 21:04:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640255; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OScrv20Vk5PZ28CoHB13qKk09OrWPyCgb0XsKA59wTM=; b=AEL+gLRIL2frkBTT92O94PYXeOZ+cSM5Lck2r7LtOXD2fya+DRNYvrSMoUjWt+fGX2Dpfy DwsihkBtRh6K9B0SoHY4Ov0Od4dCZ5furricmebcca/oLV53jNI3iYQs8lAv6ViObm43m/ psr1+Ouu4IxoY+yoPtxE6krypvZBXwg= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-422-VSBmhtDHOgqGYPWzzHMhkg-1; Thu, 10 Aug 2023 00:04:13 -0400 X-MC-Unique: VSBmhtDHOgqGYPWzzHMhkg-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a78c2cc1adso549844b6e.3 for ; Wed, 09 Aug 2023 21:04:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640252; x=1692245052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OScrv20Vk5PZ28CoHB13qKk09OrWPyCgb0XsKA59wTM=; b=J0PTUww3Yiuw45xQWAkE7kITleVtL4WMXPhiaBC4GMmo0Z21q7FkzEeAHDXIaAEvXB TrV6IGijhq3l5sb4pMBZDyA27r1Ud07s+60xhH/PGUhWfL/bb1D78Imza9a8piYcNBcb zqBB9aCJ7awxZ9bCANtfx9PyYYNckUrYWMfM+fNXifvnShjgrhvNRopYm36mHxNn3ERI pKq70mlEep2U5vy4M30usqHWCzKUwcTH3oWFROjvL+BZELFDZ04CnlfU5HMDWuR5UNmY QKV9OiMiIJiXvW0Wp+gPTSBqrKzZJV9eHAO9Jb7wNf+WcELvYW5MIbaUb3mfqsidJLrt Pq3w== X-Gm-Message-State: AOJu0YxR7ghQwjea+ZF26s7nFs5GCsRw/UUYSvZ8uwn0uxL+6kVh49LB bhzYIWq+7XroQ33Mqw4h5n4QSMS0iRjBeZ9JlT9afkiaovw/Oslw4k3Vx5n6Ma/iT5LcCbSQVX3 wNKGVygGSr1vQP5MtZfOlcqIL X-Received: by 2002:a05:6808:16a8:b0:3a7:65ae:9ccf with SMTP id bb40-20020a05680816a800b003a765ae9ccfmr1751538oib.22.1691640252583; Wed, 09 Aug 2023 21:04:12 -0700 (PDT) X-Received: by 2002:a05:6808:16a8:b0:3a7:65ae:9ccf with SMTP id bb40-20020a05680816a800b003a765ae9ccfmr1751518oib.22.1691640252341; Wed, 09 Aug 2023 21:04:12 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:11 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Date: Thu, 10 Aug 2023 01:03:43 -0300 Message-ID: <20230810040349.92279-3-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773819773373174027 X-GMAIL-MSGID: 1773819773373174027 In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_xchg* generation. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 138 ++++++------------------------- 1 file changed, 23 insertions(+), 115 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 2f4726d3cfcc..48478a8eecee 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,140 +11,48 @@ #include #include -#define __xchg_relaxed(ptr, new, size) \ +#define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=r" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) -#define __xchg_acquire(ptr, new, size) \ +#define _arch_xchg(ptr, new, sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(*(__ptr)) __new = (new); \ + __typeof__(*(__ptr)) __ret; \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \ } \ - __ret; \ + (__typeof__(*(__ptr)))__ret; \ }) -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed(ptr, x) \ + _arch_xchg(ptr, x, "", "", "") -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire(ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) #define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") #define arch_xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "") #define xchg32(ptr, x) \ ({ \ From patchwork Thu Aug 10 04:03:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133695 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp204964vqi; Wed, 9 Aug 2023 22:46:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGNjPZYyqQ5YR0ZahZA3+pZNqAGpHfBfakDjZCEjQTxJo51D8wkz5Z3LRYGkeaE8qh5pZZN X-Received: by 2002:a17:90a:648e:b0:268:220a:7080 with SMTP id h14-20020a17090a648e00b00268220a7080mr1474733pjj.2.1691646377895; Wed, 09 Aug 2023 22:46:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691646377; cv=none; d=google.com; s=arc-20160816; b=T/XupWXq4LDVqhyk4TyDI6dW4VflixBxm1+yZyPAb8oIdVYibDURpknttxD5ljYYDe xovYourwaodtW/Itw/AF5Ez5MyE8pT3xl54JdOA7wQu6vubf6gIr0uyTGSqPLJ0s+4Cp OJMHNzVgOq/QVfhdEgmXhf67Cn+vht3284IeLH6zgW+bQnyVa/RsxQUC/XqeTkMWDXxH /ahWKg3/te/uqZ90gyyVa8xp3kvdJQ5e7mu+PElcWq2/t88OA8Ym+wqFaDKucupSe7A6 Gw0piO/m2N/cJagc51C84R6He1JL1uIB1tCt/rMGAmd9s9A4YKwnJeywOz7fqeAh9Oo1 AlpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WcBubYngvVAQlkkTX8jFLD9nX24FtmcCz/1wvQAzhyI=; fh=/HAhUCSgceMdCiMxlR/A+InTwiEDI9nNuAOlAQnSzJk=; b=B83VtujPJTcTI+kzMbFBi0XqzsbF6AFJV+2+SC2lLo4WLX2ub3e8O0XnLyME9CfRk6 51IA713zf75pwb1AuwXHYKc6RYfV0AyjfKVGCHRJFGYdBuC4XEuCT17oN43Vgaql82Hs /sr3fye1GEe5DhXSL7mvn1ojZLO0+xdqzs7ULagbONEcHFdCv3ZcCVD/g2t2/Ltm7eyW WCpFgfnOSkmhXGSIJsey76EUhl/0FDpgoW39XC4N9QVfaS0n4zRofAb7zR96lGl64aSR TnM9qcKhD2SPlA6nSeRbQip12qPtUIEQe9VAjg4HrDyFm5sRtsad+OrZl9k2ul2MiRjP b/Dw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=cTZvnkiY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q22-20020a17090aa01600b00268178130a2si900840pjp.60.2023.08.09.22.46.05; Wed, 09 Aug 2023 22:46:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=cTZvnkiY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232589AbjHJEFF (ORCPT + 99 others); Thu, 10 Aug 2023 00:05:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232363AbjHJEE7 (ORCPT ); Thu, 10 Aug 2023 00:04:59 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2180125 for ; Wed, 9 Aug 2023 21:04:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WcBubYngvVAQlkkTX8jFLD9nX24FtmcCz/1wvQAzhyI=; b=cTZvnkiYYwwOpSbpPxaZmn29lguBQ7qfGV3cONZr8skO6qpNC/8WDBtn101ckVda9A7YdI lZJ6kEDHzpP1McDsHyKIAAkNhn3pZcWowAk5xlO6U33rN3yWMZSwJxHPL5HtFg1/jSGi3L lMBGgY/m6jvDcFCETue+rfCVtkyf3/A= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-29-oUqOPZcyM-KfO_CCY-vZPA-1; Thu, 10 Aug 2023 00:04:18 -0400 X-MC-Unique: oUqOPZcyM-KfO_CCY-vZPA-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a7d4302c71so47670b6e.0 for ; Wed, 09 Aug 2023 21:04:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640258; x=1692245058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WcBubYngvVAQlkkTX8jFLD9nX24FtmcCz/1wvQAzhyI=; b=T4Mq/buAdRDFmBzjnnsaD5KUr0IfrBCR2cWo1VwiAyiVbq6CP//RhaFM41R4K01Mje 8qtppI+5664kUtFIMMhju2ozshQhlyewiIAV0+ORWy1+TsaYlTVKpb1M0fwTgSr3UzZ8 pSYDQZkw8FkrKptswJXQGGQMEWWye36jyqC9eXoxuKTkgE+oTfaTaMQchEG/nyM44Ixc w2/GzMWAKkcJv2WZ+Ifm/yPSgXThOGg+dP34lScjFeM/IJNNeEBFttc+ACFOUW27k2B4 wkxys20403B0E8hdZ03I5VgJVtcNL4cfxEaELhqT3JUAQB6XdZuSNyUc7c6lr9CA0oLs nb1A== X-Gm-Message-State: AOJu0Yx3MY1/YLnhXb6RTtRA/ioFGoXKg2cPocTu80WbGzfxKgLjsDMT qdSemwdt6/TdZZg+ANOUZMcU73OoWikRL6CUc/VoxOYSAlzcBblJUNJFRMYN17EAwd4oLSadw0w rc248I5o4eQ/zcZeL6WmSwVac X-Received: by 2002:a05:6808:1514:b0:3a2:5d31:b299 with SMTP id u20-20020a056808151400b003a25d31b299mr594885oiw.28.1691640257834; Wed, 09 Aug 2023 21:04:17 -0700 (PDT) X-Received: by 2002:a05:6808:1514:b0:3a2:5d31:b299 with SMTP id u20-20020a056808151400b003a25d31b299mr594877oiw.28.1691640257554; Wed, 09 Aug 2023 21:04:17 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:17 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Date: Thu, 10 Aug 2023 01:03:44 -0300 Message-ID: <20230810040349.92279-4-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773819792293137925 X-GMAIL-MSGID: 1773819792293137925 In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_cmpxchg* generation (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 195 ++++++------------------------- 1 file changed, 33 insertions(+), 162 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 48478a8eecee..e3e0ac7ba061 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -71,190 +71,61 @@ * store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) -#define arch_cmpxchg_relaxed(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) -#define __cmpxchg_acquire(ptr, old, new, size) \ +#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=&r" (r), "=&r" (__rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) -#define __cmpxchg_release(ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ + __typeof__(*(__ptr)) __old = (old); \ + __typeof__(*(__ptr)) __new = (new); \ + __typeof__(*(__ptr)) __ret; \ + \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \ } \ - __ret; \ + (__typeof__(*(__ptr)))__ret; \ }) -#define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") -#define __cmpxchg(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) + +#define arch_cmpxchg_release(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "") #define arch_cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n") #define arch_cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n)) #define arch_cmpxchg64(ptr, o, n) \ ({ \ From patchwork Thu Aug 10 04:03:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133686 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp175681vqi; Wed, 9 Aug 2023 21:20:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEWGMnKKqmtq13dT6tG79tDXCzUSDnJF03DeD0Vg+bP0isi518EZExpXKEyq+Z63iKyv+78 X-Received: by 2002:a19:6914:0:b0:4fe:8b5:588a with SMTP id e20-20020a196914000000b004fe08b5588amr586610lfc.57.1691641244997; Wed, 09 Aug 2023 21:20:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691641244; cv=none; d=google.com; s=arc-20160816; b=LZz+lVPNaDMX/pGQCAczdKRljf7Eu3tPLaecmPb1qruTVGUGeRLixJaJAbpui+MPPZ 8rWtkVYpJV7ajLtIKfK71mOpdWESVTRen/Ue83MEZpJlyeHBdIEq3FzR5+yEGGVQLsjb wNSO6zJRKMC3SGO5dRSCisCXEXuDzG1/zyY2B8cXp69LskXLELVp/APCJ4IPIqIwbNqx 3nKkZZq4Pu9+TedDKsjTPyswq8uGLh4AFMyoufOCTqBxDLxH4ahJyZ4hh48w42ZdbFhC LGB7adXNuNH0qjGuydH0Z+j21cbiAXLks79YiQ9ddyD2NYcid9uH9Gou3dY2SPzCClif 3P4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; fh=/HAhUCSgceMdCiMxlR/A+InTwiEDI9nNuAOlAQnSzJk=; b=gmRO9PDwCQXe8LtvrH0aztRojq59L3bh6KPOaM0zmSqJh439COZFsNkOtrrfvVWAhG fZmSYsarlL2V/VJ4izGR/YfxYc4NkCVCSMIAcdkxDxFeE1w1hA31MMzfidYxMJed5BQf 2EoBOMudWD4vH8zACzgtpLR1ZdWY9U9rcZ4J6R2N0pACv8dkR7jUws7fnY0D+3WGp4wZ B5/N6Ls0DhvB/Dl6JW0N0EjnT6FIJDGcMrZurJFfHb1aqp/7ie5VtMQFNQcbzq8vMEho ctRnR3K4phrtEWDF/h7J8Kwyav2tBqn/cLn/QpsfvIvz3AmD0SzOHDfhLpPDZMv74fQ5 2FZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=J9bu9AmF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f15-20020a056402160f00b005216d397e43si589090edv.494.2023.08.09.21.20.21; Wed, 09 Aug 2023 21:20:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=J9bu9AmF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232456AbjHJEFO (ORCPT + 99 others); Thu, 10 Aug 2023 00:05:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232583AbjHJEFF (ORCPT ); Thu, 10 Aug 2023 00:05:05 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E413120 for ; Wed, 9 Aug 2023 21:04:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=J9bu9AmFw8sltFqVpctH3UpjTbiNn38xvm5YfyPzVj0NAPZF94PD4yai4uMVrAkx+SgvE0 mtztcBZ1Vt30AURyOgf2v2XIvsDcV6FHp1xYRt6ybra+UEuorEUacbOxP0nvpxIn/kvmLu zW6Rtua2lrppq8isvjhKcIiRptVlh98= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-263-Z3_cq6pZP9W6V-MrCu7dNg-1; Thu, 10 Aug 2023 00:04:23 -0400 X-MC-Unique: Z3_cq6pZP9W6V-MrCu7dNg-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a7d402fc85so77409b6e.3 for ; Wed, 09 Aug 2023 21:04:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640263; x=1692245063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=DYteQqfF1lXhZZ27UZGPGl+y6u8CWctTL8ocNYq+jNk1O0mkVkkMfPvGVDZPUcpYhl djcmC97Jt8jBTZfwwrm4IRQrVahsc62NL/jzGRfTMeTkSPcfqX09RABH6vUUVDoZFeVV QeDK3HXpxYycxFPoJNYJrGOZwbpm4hKWwM3k9yHoJ2ryo/nbeuBD52ckMGGI5MOiq47A Yj3luTsw+1mVWk6b4XcdQuO9dtO835JiWTravi4YOAKPad+Wc1Dj6Aauqs4PIugVpx3J XvemBpye4wHcLt/l5wdXFspJOXGpExw0bO9trVExF7Xuyr5cciW2130wJwX2EIHMC2PC +KyA== X-Gm-Message-State: AOJu0YySb1a5FRCXEm6sC7/P5uTs0ViVaexzlContkfSFbf07F5Y9pl+ 9Mz157A95PVptOUrvM3AvDbrhaXks8mzgUEsbB/btsgXRN30IpyW0RKSOGp8cfSRKYVqG0G8Tkn RjZjL0sPesSx21+7jIZAYNAvt X-Received: by 2002:a05:6808:998:b0:3a7:540f:ca71 with SMTP id a24-20020a056808099800b003a7540fca71mr1397765oic.53.1691640262935; Wed, 09 Aug 2023 21:04:22 -0700 (PDT) X-Received: by 2002:a05:6808:998:b0:3a7:540f:ca71 with SMTP id a24-20020a056808099800b003a7540fca71mr1397752oic.53.1691640262690; Wed, 09 Aug 2023 21:04:22 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:22 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Date: Thu, 10 Aug 2023 01:03:45 -0300 Message-ID: <20230810040349.92279-5-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773814409854176476 X-GMAIL-MSGID: 1773814409854176476 Some functions use mostly the same asm for 32-bit and 64-bit versions. Make a macro that is generic enough and avoid code duplication. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Guo Ren Reviewed-by: Andrea Parri --- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h index f5dfef6c2153..80cca7ac16fd 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i) #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN +#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) + /* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w"); + return prev; } #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless @@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_unless(atomic64_t *v, s64 a, s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d"); + return prev; } #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless #endif +#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "w"); + return !(prev < 0); } #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative +#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "w"); + return !(prev > 0); } #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive +#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "w"); + return prev - 1; } @@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_negative(atomic64_t *v) s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "d"); + return !(prev < 0); } @@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_positive(atomic64_t *v) s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "d"); + return !(prev > 0); } @@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positive(atomic64_t *v) s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "d"); + return prev - 1; } From patchwork Thu Aug 10 04:03:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133696 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp206593vqi; Wed, 9 Aug 2023 22:51:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGJ7unhvWNG6TXdbVN7sFTEXrMT0fFRsbct8MjzNegAJmb4tZu77WHQhi7yH3sMpZj6RDTE X-Received: by 2002:a17:907:2d07:b0:98f:450e:fc20 with SMTP id gs7-20020a1709072d0700b0098f450efc20mr1257819ejc.17.1691646691156; Wed, 09 Aug 2023 22:51:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691646691; cv=none; d=google.com; s=arc-20160816; b=M+peygW/pkRiZIrg7rzg6bMIGxtvOdIGi/NaETP9q7QqGGgalP60pN9MVwleRXPkOM Jf2gR9CR+qgox+mofW19Yxb2j5jvOG+cmz0U9MwTU3cO4u55PW2jdlhIeksRFIc9oT/b nCldYwO0BZio0CtakO+eX8MSUtxW+t/efdPWjDpG/9d+fvVEWp4SiF6EyLscShCgCZJO b4n7ovvjnzYvXU0cjvCdfE3HVeZuZ+Lsz0sMErH+0e/w11ONqsUPR4Nwv/ZoPMvSZ8CP XevET8zCChChZaOMDSHBPeqczTDgiOcNS2HOOtE7x13adnO7wsTiUQUEGL41syMcFULH w11A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VLwtkHrR4Hnv0SG2htyTx6O/VUdZ3YGDzB9YQqYciKw=; fh=/HAhUCSgceMdCiMxlR/A+InTwiEDI9nNuAOlAQnSzJk=; b=quLep7CGjI+da4CpSkoVLsPnUAt6YETL0UJ0iJ2ZNmyaFlfO/bduI5P7AxqigKVO3P 6af51Kk1t/geReTl93xNGPBAQqWSV68RHpQQHQYZgkg0ry6z+AD9aX1ykt0ZDZ+ZDkLl NobmknNqPN6d+m7P5yES+RLQ6jrJ/i5eUq/93Xc1yR96dI5MYRJKn3hTVr8VFhpzyfc6 Sq5FG+67VnsuNYJkHbU0/FFkjOlspTQhg9b3tOW2AznUNTjRcljUnE5HEf1K3q48tQBZ y8l/Q5gxa4mc6nkhzwj/+B8lWd9bB84gJNzWX3Ks+zeOMryenDifn6BAQSqfdGnLlnxr L11A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=a7fZc30n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i20-20020aa7dd14000000b0052338b19bb7si682683edv.54.2023.08.09.22.51.07; Wed, 09 Aug 2023 22:51:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=a7fZc30n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232584AbjHJEFZ (ORCPT + 99 others); Thu, 10 Aug 2023 00:05:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232675AbjHJEFQ (ORCPT ); Thu, 10 Aug 2023 00:05:16 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34328E5B for ; Wed, 9 Aug 2023 21:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VLwtkHrR4Hnv0SG2htyTx6O/VUdZ3YGDzB9YQqYciKw=; b=a7fZc30nn5A0fu6C+UEEAnHvDKtTYklJnVxbW8MFMub1FRC78+JKbkGFm9ZrM3oS5ZS/af 3ffBd5L+OGY+swKWFLcR8GCWjeXI2gIypR9UNHiMnwKYAcxXeiTeSGj6ptZZrUe25nEVnK YO8XjPNqGbWmLLvrQ4eEEcKzs7Iwp+E= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-436-Euk6dC1sOp2ePg5o-fUyVQ-1; Thu, 10 Aug 2023 00:04:29 -0400 X-MC-Unique: Euk6dC1sOp2ePg5o-fUyVQ-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a7697e580fso597222b6e.1 for ; Wed, 09 Aug 2023 21:04:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640268; x=1692245068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VLwtkHrR4Hnv0SG2htyTx6O/VUdZ3YGDzB9YQqYciKw=; b=FWEQ9nZdtJXGNUh6AM/klGNeSQpsDYcv510oRhXpOVtOn0m/4ULAZQi4uXkrYugk7O NfiQ17tR5YIYwNJV+gsR3p/Q9iYpoGMoujF60BMOle7uQgb4EalWhm+UYqo5x46//dIk oRrK8Ra/ZgiNbtqNLXjV5FnszjENBonXVNQ0N7vxVHplLCFfKMqIBgZVbN6dhOlGv1Gk pbMc+eong9++mLNF2Sj+0DS70Ti3PXzCbveqgLUkKUeb8wRxqq0maAPT88qmgggMZH/7 +OlaRV22eikSsdaCNnODt068dvkUngPeBkPDb5DTU1J53AzgOLCMeBDLYQumJTTCoep/ iRKQ== X-Gm-Message-State: AOJu0Yw1OGdDwXxohznE3nVi9BWvTCtxPDO4UclenRsX3rSNwEwCS0k7 8dvomI62iBq7vSfQfZmXtWvZTyqy1OTrsYRtCHZ7ZcyHyQYxFBs0BdXe1AsH3LW1+vEzdiQVaVo rxHCaW/VB2RMrA0E0ARHTCR9X X-Received: by 2002:a05:6808:10cc:b0:3a3:d7de:5cfa with SMTP id s12-20020a05680810cc00b003a3d7de5cfamr1819128ois.11.1691640268304; Wed, 09 Aug 2023 21:04:28 -0700 (PDT) X-Received: by 2002:a05:6808:10cc:b0:3a3:d7de:5cfa with SMTP id s12-20020a05680810cc00b003a3d7de5cfamr1819092ois.11.1691640268064; Wed, 09 Aug 2023 21:04:28 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:27 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Date: Thu, 10 Aug 2023 01:03:46 -0300 Message-ID: <20230810040349.92279-6-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773820120706168692 X-GMAIL-MSGID: 1773820120706168692 cmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index e3e0ac7ba061..ac9d0eeb74e6 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -72,6 +72,35 @@ * indicated by comparing RETURN with OLD. */ +#define __arch_cmpxchg_masked(sc_sfx, prepend, append, r, p, o, n) \ +({ \ + u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ + ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx = (ulong)(n) << __s; \ + ulong __oldx = (ulong)(o) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z5\n" \ + " bne %1, %z3, 1f\n" \ + " and %1, %0, %z6\n" \ + " or %1, %1, %z4\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" ((long)__oldx), "rJ" (__newx), \ + "rJ" (__mask), "rJ" (~__mask) \ + : "memory"); \ + \ + r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ @@ -98,6 +127,11 @@ __typeof__(*(__ptr)) __ret; \ \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_cmpxchg_masked(sc_sfx, prepend, append, \ + __ret, __ptr, __old, __new); \ + break; \ case 4: \ __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ __ret, __ptr, (long), __old, __new); \ From patchwork Thu Aug 10 04:03:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133698 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp214089vqi; Wed, 9 Aug 2023 23:09:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGIFoXtfB5MIV97frX30sgchHNhdyNlU3PiFydaGf8mUxI7R5UStwEGsK2Ynb/9upXQn8f4 X-Received: by 2002:a17:906:2ce:b0:992:2f67:cd34 with SMTP id 14-20020a17090602ce00b009922f67cd34mr1454930ejk.22.1691647783942; Wed, 09 Aug 2023 23:09:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691647783; cv=none; d=google.com; s=arc-20160816; b=IVAtshcdCMWXDoFRdIYmP3HtX4vbovfOP7DlrSNj7W37sWIG52Gfyzsan0iYGc80Nk kztED8IQqsfmpMjPmQa+hMR85f68pS13PaOzfCngPumStJJI4NhhDoe78ZyTjJ2GyBWx 1zebFNh/9I0pegWIyGMRqp2v+9ca3+LnGCZFLXLcwkKMnINHglWCMlIxupLY5XlmRTPV oMsrcAXU3hNAGCwZVzYbHPVeXYE/Hrx86SOwJJrqrtoFTN4dFtukOdARBJ6o+irjbioR s1bvJfZd9lWsmtXJ4cfHKxiDSuQ4fpwWVw0QTxOdT9kyz0v6Rf5jJvLExka/T56xokEw wT7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=S9RDXJ5aWrysjU+rhwlqeMwMOsK56lpvgf+6LxPUCTg=; fh=/HAhUCSgceMdCiMxlR/A+InTwiEDI9nNuAOlAQnSzJk=; b=WKugZKU+fAwBUkkwh6gRqIKmSy9vWwybq11E6aw2QM3cI4XPv5sWeabWFOu8KL7p+H O8clatzOuRmKwJntPJX1HLC9TSShXkCxjTEml9vfNqk947UTk9isKY3RvViwGueO1SBd ru/4X5NKOuCkXjWuyBJpURgra4sVJu59KE06k3oecqKRHd76wbRs4FAc7nNWwjwTY5rz yizHFOI07PKJrtJiVmp3smalHVKKT4fYAmhdjEHXAMKwWcThLsXBsoKz1wjWAfVgCsV+ MxJ65J8adnQG3/+guWq1NKALoJMl+bY1IfwOsDwhcPbd4skgAHl08kySHiyQiK8fvSJ7 /I/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=IRHbNtSc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x25-20020a1709060a5900b009930ef7f05dsi817327ejf.908.2023.08.09.23.09.19; Wed, 09 Aug 2023 23:09:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=IRHbNtSc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232651AbjHJEF2 (ORCPT + 99 others); Thu, 10 Aug 2023 00:05:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232641AbjHJEFU (ORCPT ); Thu, 10 Aug 2023 00:05:20 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43FD0E69 for ; Wed, 9 Aug 2023 21:04:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691640276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S9RDXJ5aWrysjU+rhwlqeMwMOsK56lpvgf+6LxPUCTg=; b=IRHbNtScy0/Nrm7nV9r/y/jDk0kJK6cOENHOsCDLgqyMBAi4IIpvX9ov2xnbrD/akhZIK8 mHqHZOooBlGmgdisaSzREFkzPQr1WrS1myiJ40xZSsvnlGdAVUJhvHP8/1wnd50b2ko17m BLREvk0Csn+s2rLNZXYk0NViE+DvLIQ= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-283-Pk3jcTnrM8y507xMyNPyBQ-1; Thu, 10 Aug 2023 00:04:34 -0400 X-MC-Unique: Pk3jcTnrM8y507xMyNPyBQ-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3a5ab57036fso584576b6e.3 for ; Wed, 09 Aug 2023 21:04:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691640273; x=1692245073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S9RDXJ5aWrysjU+rhwlqeMwMOsK56lpvgf+6LxPUCTg=; b=AVHSK3m9lea5FMgXD2ccnwiFZZy4Z/kiHwvRVWDLZGzKwP8b/7bkEpCH/guNS+xikv Dh0C145D27s90BNR2srMaOU1d8i9b/r7O/FGmtQYn/FlI6T+jvgDHiNPNb6JB6pWdNxE DGIvmAFfDm383XezPuWfh2WIgTjFstkvY57FJG4Q0duahSAwh8kXzF67AFZ1mex+HAHV 1ltq0Esv5Qttq+CeBJvR42/EnJ5SLlfD1yproadMVzADKn9I3x3oRO0TCqMakfvchmhQ ++EiKymOSch2tVu6mZCzz69a5gds8AvtFB27s4AmZ4hLlkUqu292cGWbFppUR5YE2X/X phzQ== X-Gm-Message-State: AOJu0YyswhD9dX4SULSfce5BwzVDPYYbM1TQyMxxM+RAbD9txfu9A8zm dEMIYucBfRAFJrNZp5e56iao2sWlDi1cQyjEbytZY/O8A1VissYbI5CoEVz/Tn/8tGOl8ty4DW6 ZXyyAE6cFKOnZS+GPP+BxdHjz X-Received: by 2002:a05:6808:2124:b0:3a7:82e8:8fd1 with SMTP id r36-20020a056808212400b003a782e88fd1mr1804195oiw.20.1691640273706; Wed, 09 Aug 2023 21:04:33 -0700 (PDT) X-Received: by 2002:a05:6808:2124:b0:3a7:82e8:8fd1 with SMTP id r36-20020a056808212400b003a782e88fd1mr1804180oiw.20.1691640273540; Wed, 09 Aug 2023 21:04:33 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id b12-20020aca674c000000b003a7b5193909sm310087oiy.19.2023.08.09.21.04.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Aug 2023 21:04:33 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Andrzej Hajda , Arnd Bergmann , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v5 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2 Date: Thu, 10 Aug 2023 01:03:47 -0300 Message-ID: <20230810040349.92279-7-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230810040349.92279-2-leobras@redhat.com> References: <20230810040349.92279-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773821266444485956 X-GMAIL-MSGID: 1773821266444485956 xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index ac9d0eeb74e6..26cea2395aae 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,6 +11,31 @@ #include #include +#define __arch_xchg_masked(prepend, append, r, p, n) \ +({ \ + u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ + ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx = (ulong)(n) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z4\n" \ + " or %1, %1, %z3\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" (__newx), "rJ" (~__mask) \ + : "memory"); \ + \ + r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) + #define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ __asm__ __volatile__ ( \ @@ -27,7 +52,13 @@ __typeof__(ptr) __ptr = (ptr); \ __typeof__(*(__ptr)) __new = (new); \ __typeof__(*(__ptr)) __ret; \ + \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_xchg_masked(prepend, append, \ + __ret, __ptr, __new); \ + break; \ case 4: \ __arch_xchg(".w" sfx, prepend, append, \ __ret, __ptr, __new); \