From patchwork Wed Aug 9 02:13:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133038 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp2533655vqr; Tue, 8 Aug 2023 20:11:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFMQTTUnfh25XEGI7N/7H6ff6cOr7DFe5nZhL1ba+GZxydt8HMO6ndPrpSjbET1zYCLhVTL X-Received: by 2002:a17:90b:1115:b0:263:fc43:5f39 with SMTP id gi21-20020a17090b111500b00263fc435f39mr1146389pjb.13.1691550718919; Tue, 08 Aug 2023 20:11:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691550718; cv=none; d=google.com; s=arc-20160816; b=ik8xzoZvXQpZlZNn386AIlBQnl5UYjdCM/ZOylDlr1Q5orwleHrSsRM1FdmIdkZ9ZU 0BIsm2SSOKvVNt8+MUiOdhsotqNzviF80alz5N4/NOrbCfMcbII89Ri5cpecazgkl+k5 V+/EX/08xBMu9YwJz1iEq/LHumIM141PKDgeK1hyHjv/L4F49xALbqCvzH4OHU9EQrdt s0X5u+A7NZkj1u1e5BUwzd/IQvQoQYcFzOMwSYm6hMu45rPKhdYAdlSoHyt8nsJQxcAj i9TuppGW5vrSFdjzlaXaay9tv3M3rgt1ZpQTZfxZOGMwEZ5fCT7oolKsyw02JmE4Wq1j +BoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; fh=RC6bsfN8bPPp/iDUCpnJMEtdiqa5XC4G6XtB4MmAVnA=; b=uSp6CUXlGhiBDjj9wJDFrkBcbtSFqC2tmjBCeOrZv2FUpPZXJkXaFGaWJBT3iUlunb okzDXuZEId9lV5/5g9XOP0WyiKZXWoJRwry7s6xabVd0njqrvxuZihZtJ5eDTggLkKCx rxDaDnlT0ypdkeajV/vGELBIxEtGd9GSmx3a17aDSIGSjvvNGrqT5kX5khhVO8pABx1G yyD1A13Ibf9WNYLDNWNYRI1wYAA4/cI/PfgEh4ahyeV5a9LC8stH3EGDqdCyeV2rPAaf 7Mxu0D7OKmyrPjJQkhdtTWEfonucDRT4pApBvdzN0bZdjjPiercLNTSaIGmuVCVd5pyr vtpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=XRqnmWPy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id me18-20020a17090b17d200b00267bb4cfaa7si522127pjb.88.2023.08.08.20.11.46; Tue, 08 Aug 2023 20:11:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=XRqnmWPy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230341AbjHICOp (ORCPT + 99 others); Tue, 8 Aug 2023 22:14:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229450AbjHICOl (ORCPT ); Tue, 8 Aug 2023 22:14:41 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93954FE for ; Tue, 8 Aug 2023 19:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547239; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; b=XRqnmWPyTZYG31ea3SCYHl3lm2SMlU82Zqp23COwH9n6Ot0i1EYPjLdvcZcbjSjjxcNFkU MWfQ25+nVqNa5hzksLlQjSEMPBkUk68AybqxFPQujSTSsOXsJxOPUvCh0IJIjaxuBM/VWF ndBbvaPhP294EiqjVupEGHNrvpZumyE= Received: from mail-ot1-f70.google.com (mail-ot1-f70.google.com [209.85.210.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-278-WdnLxjy4MlijCfrkKkqyEg-1; Tue, 08 Aug 2023 22:13:58 -0400 X-MC-Unique: WdnLxjy4MlijCfrkKkqyEg-1 Received: by mail-ot1-f70.google.com with SMTP id 46e09a7af769-6b9cf208fb5so11856780a34.3 for ; Tue, 08 Aug 2023 19:13:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547238; x=1692152038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cG8axZDwBuWJQ+k2FsV81xz+WKLWi2QNSuhJ6eUeaqw=; b=hxbStUsIy3XFvyjQ23gqWiVmpxGsN7VuiCPA3O7voJmbbITR7gRTSt3TFkJh9PTjxZ R7oHdXQ5aklQFEdQshksQKMgUJhDp4HLTjtOutS7w0Ua1dAS9/06wOHL+C2YyJ2IE4g9 ZiHHsMvWWaxyUZ0XNH+2j8NB0C00LUBYMbHhKHlYKL9NTIawhNeay353JDktoISGjmO9 SKnEe6MgsJCYep9CzEH2oh5yFbY7ZVt3Se+NKepRyoD6Q09ZjOSdwD531gPeLb/nz+Hz aHuLZZwTt8T6atZXlw7lg3yOQIxa1faFUBnyn0VjcRuP+ued3QVItKd4TXJLx0dKjax5 pgRw== X-Gm-Message-State: AOJu0Yzd9Pw841GNPyovIeApTBNI67orY/FRClY+V5Q/lLEZSPUz2FRu zIH+kAKz4gEzEoFAd5xESAfcKYF+H5NRatTPI3GKFxAu2WKQwYb6PfbpeNCQCbMOd5HyFj8MjYh ANKvGw1MNUXoWUSLLgyKkOT6/ X-Received: by 2002:a05:6870:c084:b0:1bf:4f5e:55e2 with SMTP id c4-20020a056870c08400b001bf4f5e55e2mr1543844oad.53.1691547237749; Tue, 08 Aug 2023 19:13:57 -0700 (PDT) X-Received: by 2002:a05:6870:c084:b0:1bf:4f5e:55e2 with SMTP id c4-20020a056870c08400b001bf4f5e55e2mr1543823oad.53.1691547237498; Tue, 08 Aug 2023 19:13:57 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.13.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:13:57 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 1/5] riscv/cmpxchg: Deduplicate xchg() asm functions Date: Tue, 8 Aug 2023 23:13:06 -0300 Message-ID: <20230809021311.1390578-3-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773719486644162430 X-GMAIL-MSGID: 1773719486644162430 In this header every xchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_xchg* generation. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 136 +++++-------------------------- 1 file changed, 22 insertions(+), 114 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 2f4726d3cfcc..ec4ea4f3f908 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,60 +11,30 @@ #include #include -#define __xchg_relaxed(ptr, new, size) \ +#define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_relaxed(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_relaxed((ptr), \ - _x_, sizeof(*(ptr))); \ + __asm__ __volatile__ ( \ + prepend \ + " amoswap" sfx " %0, %2, %1\n" \ + append \ + : "=r" (r), "+A" (*(p)) \ + : "r" (n) \ + : "memory"); \ }) -#define __xchg_acquire(ptr, new, size) \ +#define _arch_xchg(ptr, new, sfx, prepend, append) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ + __typeof__(*(ptr)) __new = (new); \ __typeof__(*(ptr)) __ret; \ - switch (size) { \ + __typeof__(ptr) __ptr = (ptr); \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".w" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ + __arch_xchg(".d" sfx, prepend, append, \ + __ret, __ptr, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -72,79 +42,17 @@ __ret; \ }) -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) +#define arch_xchg_relaxed(ptr, x) \ + _arch_xchg(ptr, x, "", "", "") -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_xchg_acquire(ptr, x) \ + _arch_xchg(ptr, x, "", "", RISCV_ACQUIRE_BARRIER) #define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __arch_xchg(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w.aqrl %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d.aqrl %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) + _arch_xchg(ptr, x, "", RISCV_RELEASE_BARRIER, "") #define arch_xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __arch_xchg((ptr), _x_, sizeof(*(ptr))); \ -}) + _arch_xchg(ptr, x, ".aqrl", "", "") #define xchg32(ptr, x) \ ({ \ From patchwork Wed Aug 9 02:13:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133043 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp2539980vqr; Tue, 8 Aug 2023 20:30:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFH2GtI+/UENsTZfVnKQeQtMzmoA8L9xppfsFlrDgf5m6d3BaiyM0hrR4pPkJWvkQN4IS4A X-Received: by 2002:a17:90a:300e:b0:269:85d:2aef with SMTP id g14-20020a17090a300e00b00269085d2aefmr1283689pjb.20.1691551846899; Tue, 08 Aug 2023 20:30:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691551846; cv=none; d=google.com; s=arc-20160816; b=GH7+U1B39bEdW9HfHMCjsMImQiqCpk6VHprh2v9aGIXML6/OTn2y+2VAbHQndynRbr GuJIJ9HiDtDU4llvD/h0jM6JGEPCtlN4r0cuVpUCSZxBq4+ozHDvrJ6uTFaj6p86Faai bFrSPoNsdwBQDo98VLSoJmq5bJSTSZUk0dpqj+8pfy6XXoa5Mq6+QV+ynUjjwJQEOun7 Tq+sHAcNWV2hlTXsHceFlQq6fJBodIDrWCbDiIDxK6+IM+7ujxYocvECDPm0Ru1XCE86 MWyayFZnd6jqCiSm1VWbs3ovSW14/lNds17z2ej+oBREnbJ3c4EpE/MedZrIgk4pwDUt LFFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; fh=RC6bsfN8bPPp/iDUCpnJMEtdiqa5XC4G6XtB4MmAVnA=; b=KYbteXGdm+dL4S0AlhnbwqwsshZDMpcTjQ48xOOL1rUcMRxboCvZxdGe4dGORxopR4 ECCFDqpyidFID5Fc1WibHhvCaru5MuRK9uIOw7n48SnlGcYKnlUftoh0Bga7VfVksaqj BKRBFH6k1cBNQ4Y1Q24TojQrxz4DhSHkO3fIFrA9UZ7TQYebsu/MaR1Hrk9H+TJtGEtP vqvh9LvlhidrqJsKD5dxmwfxKUGT2+/Qfx0N1oRcTKhglR8UxdjKi6FU2s9ZAPenZXkj 7TCgtvF7nW1plS6UXK44XchdwiPyuSQUhQNopZAwb5ETFi2ZCpwqzBFKcegYmrt1DZAl sn8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Hlj7cYC6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ci21-20020a17090afc9500b00262f99a851asi511469pjb.96.2023.08.08.20.30.33; Tue, 08 Aug 2023 20:30:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Hlj7cYC6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230354AbjHICO4 (ORCPT + 99 others); Tue, 8 Aug 2023 22:14:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230404AbjHICOw (ORCPT ); Tue, 8 Aug 2023 22:14:52 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4B0E1BCF for ; Tue, 8 Aug 2023 19:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547245; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; b=Hlj7cYC62XGhxBJoJ0YtfPFKlAuiW63TigiC/kyWTFkPKRvL6+ustHNXYvnb8auYlqBoPh siP/j9Wjb4s+9LgK5bD01j+qm/SYF/yRr3DsZkuGBqVRXK5dpagmRavJoBavZVFKM1BNw3 EK50kPoBnv4Pl1qjxbLwQeKZ0gosoeI= Received: from mail-oo1-f72.google.com (mail-oo1-f72.google.com [209.85.161.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-635-81yYtrFvMh22bvAcvpLuhA-1; Tue, 08 Aug 2023 22:14:03 -0400 X-MC-Unique: 81yYtrFvMh22bvAcvpLuhA-1 Received: by mail-oo1-f72.google.com with SMTP id 006d021491bc7-56c7404fafdso10497303eaf.1 for ; Tue, 08 Aug 2023 19:14:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547243; x=1692152043; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=68droSgtD1yUeWujN9KsDS4jg7ssA0NCkb8PZ8Tb+SA=; b=bhOny3s7e/3/RvwBMR7SgD2WFHZLgIuTsSv0UltWQdBukNDP39SHc80adaL8HwsTHy YJFjUti1RswKto6DUAi2bP9kJx1z+VMpQLI0bmTNESW97/qBDmKoX6GvYR6yEOITJnW0 CUQ2E2y0PQYeFIuSYA6vCmTzZXByVhaowrJjl1XmTEmoLao/NWZUlhFm1DbaBs6uhYY+ eX+uCLQFdRYOX5GdR1azcAdllfU+trwNq3wWVgdBHP/40KzUwUFuUny//HDUJtbaRoQ6 Zdq3qfb1cOgHNzczhMYwdqYvyGrDDWxz/1ZGEh/zOULZye4eteSYc62fXVX+ok9HDypp Glrg== X-Gm-Message-State: AOJu0YwYNDaZsKLVj9sWlSlbxZK9D8zeyuJvflRXXgJZfKfSkvG7GEeG +eWCHMqMN96nIR4Hzo9O7tWu4ujcw0O8AfyO7xvPPYRzMG9+RNoiAKLpAYd2xk4+A/3zeS/+jS3 weDz1MCYpyQwv7oRwvBdy7pKf X-Received: by 2002:a4a:3005:0:b0:56c:aa7a:eaeb with SMTP id q5-20020a4a3005000000b0056caa7aeaebmr1376286oof.4.1691547243031; Tue, 08 Aug 2023 19:14:03 -0700 (PDT) X-Received: by 2002:a4a:3005:0:b0:56c:aa7a:eaeb with SMTP id q5-20020a4a3005000000b0056caa7aeaebmr1376278oof.4.1691547242715; Tue, 08 Aug 2023 19:14:02 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:02 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 2/5] riscv/cmpxchg: Deduplicate cmpxchg() asm and macros Date: Tue, 8 Aug 2023 23:13:07 -0300 Message-ID: <20230809021311.1390578-4-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773720668963677385 X-GMAIL-MSGID: 1773720668963677385 In this header every cmpxchg define (_relaxed, _acquire, _release, vanilla) contain it's own asm file, both for 4-byte variables an 8-byte variables, on a total of 8 versions of mostly the same asm. This is usually bad, as it means any change may be done in up to 8 different places. Unify those versions by creating a new define with enough parameters to generate any version of the previous 8. Then unify the result under a more general define, and simplify arch_cmpxchg* generation (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Andrea Parri --- arch/riscv/include/asm/cmpxchg.h | 187 +++++-------------------------- 1 file changed, 29 insertions(+), 158 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index ec4ea4f3f908..5a07646fae65 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -71,127 +71,40 @@ * store NEW in MEM. Return the initial value in MEM. Success is * indicated by comparing RETURN with OLD. */ -#define __cmpxchg_relaxed(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) -#define arch_cmpxchg_relaxed(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_relaxed((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) -#define __cmpxchg_acquire(ptr, old, new, size) \ +#define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - RISCV_ACQUIRE_BARRIER \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr" lr_sfx " %0, %2\n" \ + " bne %0, %z3, 1f\n" \ + " sc" sc_sfx " %1, %z4, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=&r" (r), "=&r" (__rc), "+A" (*(p)) \ + : "rJ" (co o), "rJ" (n) \ + : "memory"); \ }) -#define __cmpxchg_release(ptr, old, new, size) \ +#define _arch_cmpxchg(ptr, old, new, sc_sfx, prepend, append) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ __typeof__(*(ptr)) __old = (old); \ __typeof__(*(ptr)) __new = (new); \ __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ + \ + switch (sizeof(*__ptr)) { \ case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ + __ret, __ptr, (long), __old, __new); \ break; \ case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ + __arch_cmpxchg(".d", ".d" sc_sfx, prepend, append, \ + __ret, __ptr, /**/, __old, __new); \ break; \ default: \ BUILD_BUG(); \ @@ -199,62 +112,20 @@ __ret; \ }) -#define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) +#define arch_cmpxchg_relaxed(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", "") -#define __cmpxchg(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d.rl %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - " fence rw, rw\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) +#define arch_cmpxchg_acquire(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", "", RISCV_ACQUIRE_BARRIER) + +#define arch_cmpxchg_release(ptr, o, n) \ + _arch_cmpxchg((ptr), (o), (n), "", RISCV_RELEASE_BARRIER, "") #define arch_cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) + _arch_cmpxchg((ptr), (o), (n), ".rl", "", " fence rw, rw\n") #define arch_cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) + arch_cmpxchg_relaxed((ptr), (o), (n)) #define arch_cmpxchg64(ptr, o, n) \ ({ \ From patchwork Wed Aug 9 02:13:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133055 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp2554486vqr; Tue, 8 Aug 2023 21:16:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF1E4Eo09tDjdkaPEZ1MhCFrSg2frX8kLoZCnF5HAEUGnLh0mg5Eh5HM2NQZZiWz2YmEe3v X-Received: by 2002:a05:6830:33f8:b0:6bc:9c9b:e787 with SMTP id i24-20020a05683033f800b006bc9c9be787mr1792677otu.8.1691554617530; Tue, 08 Aug 2023 21:16:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691554617; cv=none; d=google.com; s=arc-20160816; b=s0oWBMyPF4f0Aq9D1XYLMfflKkI3xkHUR5ZOfG5EKrsUyI8610z0KVN66a+QvCDpth OgFTy7F+PYYvcripmUslN9vYdSi211AGBgLl6bJUj9wgRgFDZ4a+SkVSTEHjyoz57OPj MSvQMg7jZ/ISC5OGUXAiKCd0n833aGZ249X69zwfZwWqGOXiBcDOy6XLeAV0bhs8OMI0 gB+wWLMv6KkWp+mb3I5WFr2RKFrsYAoZdARIJ1jGmYiblplFqMVct5lRWabE3yIZ4ga1 8uQb4Aw09QSKla7qVCSLUysvdaoht1fHBPw4A2C7yY5d1zO6VOjifmkPZZyD+Mf44kQX 2GYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; fh=RC6bsfN8bPPp/iDUCpnJMEtdiqa5XC4G6XtB4MmAVnA=; b=lF42FzupD3isTKrR2DMgClMBGgsWYgHVWEqnEWPuZ6cRq5dF5RvhHN3MwJRH+Bm7dt EwIPJLbR0BbmYrpgCUWUlNOQZEemWAHoHhLlS8x6ywhBcBw29OO0BEU3TxBgxvGPEp7K 7TsTTXu3S9ggP3PFOUTNz5AatmPHyx2PVO12w9ToSWLaBM7w8ZY2ftFuJuVX2PAZ/ZkM o8vK/LQpkttII52FUqrE+mtL5KPIgKVJojh9JV2Hp3X+qtohe8A1AM+W6SyKJzkZ2mDK tu6S2xcJRAiaoApaGv7LFeWU+vI38cShpOhtd/2nXoevQ840FNh29kh0Medpi0TaAHhu dlSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=VTf0kHbf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 18-20020a631252000000b00563e93dfc8dsi7285003pgs.375.2023.08.08.21.16.42; Tue, 08 Aug 2023 21:16:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=VTf0kHbf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230299AbjHICO5 (ORCPT + 99 others); Tue, 8 Aug 2023 22:14:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230359AbjHICOy (ORCPT ); Tue, 8 Aug 2023 22:14:54 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA7C01BCD for ; Tue, 8 Aug 2023 19:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547251; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=VTf0kHbf7syq69H4/SQEYMO9GkiNYyaTPxkSTZu+98gfHMc7u8FW4BPIxfYo1tKRp5P3Ah MoH0q9QGqh0dhWyduypETmfPdlS0ewAek7a22/afqNGywudfH0TC/QFyZPwZsZZbL2Bqcj gM8yJDNLy2kI7XZCOUm6LZGOsy8WhiM= Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-59-CDIct0agM8-iaz0Gjnwp2g-1; Tue, 08 Aug 2023 22:14:09 -0400 X-MC-Unique: CDIct0agM8-iaz0Gjnwp2g-1 Received: by mail-oi1-f197.google.com with SMTP id 5614622812f47-3a73fbef692so12356399b6e.2 for ; Tue, 08 Aug 2023 19:14:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547249; x=1692152049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UzQb2VuQCXwMdF/Rvu48wxhx8ntZSb1KliAOBLDDWgc=; b=LHXAc9nM6zHt59Gqpc09SlFcQ6zr4302QgGsqUh+NHYPLqec5IW3Vung9dR3yyiPZN Wa2/0XDCI+p/pzPy65mFjN49uKhls8vFRJ24HVH0DQJhJJf8B3AD6UMScN8z5psXlTbX +p6JsQDcDhkOaT3c6WTWwX/GGGDAyI3+HVzdQkbw9JuVDEo/SXAb44m7otbbBTvhAxSk 05Ik0BDqE7hR2MGJfzMKOvwD51JyLqOeKDwHpnukYIB4kLM76AtAEQ3yJE+TtmYnkTHT qbSYLftnTi1M4bOWmydnke4+mXR3n2Vi/r6gly2QG8W+wwiPa97jTk5fFXXb73fjFwXd QWjQ== X-Gm-Message-State: AOJu0YzjMoxFMm4U/HnbOLATkI5lCSRVDsAywgXj2xYFTsRN65huRs9Q JsRXwKCvCNq/tqE3dKHC0c0364/sfFOHIAx//Tb/9o37DNbwE0jndow/keDlabPfqUoe7p2zd81 wqRfRdOjqfEEgWEA/3bztVuax X-Received: by 2002:a05:6870:c082:b0:1bf:74cc:c815 with SMTP id c2-20020a056870c08200b001bf74ccc815mr1491705oad.19.1691547248946; Tue, 08 Aug 2023 19:14:08 -0700 (PDT) X-Received: by 2002:a05:6870:c082:b0:1bf:74cc:c815 with SMTP id c2-20020a056870c08200b001bf74ccc815mr1491689oad.19.1691547248647; Tue, 08 Aug 2023 19:14:08 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:08 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 3/5] riscv/atomic.h : Deduplicate arch_atomic.* Date: Tue, 8 Aug 2023 23:13:08 -0300 Message-ID: <20230809021311.1390578-5-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773723574578798728 X-GMAIL-MSGID: 1773723574578798728 Some functions use mostly the same asm for 32-bit and 64-bit versions. Make a macro that is generic enough and avoid code duplication. (This did not cause any change in generated asm) Signed-off-by: Leonardo Bras Reviewed-by: Guo Ren Reviewed-by: Andrea Parri --- arch/riscv/include/asm/atomic.h | 164 +++++++++++++++----------------- 1 file changed, 76 insertions(+), 88 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h index f5dfef6c2153..80cca7ac16fd 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -196,22 +196,28 @@ ATOMIC_OPS(xor, xor, i) #undef ATOMIC_FETCH_OP #undef ATOMIC_OP_RETURN +#define _arch_atomic_fetch_add_unless(_prev, _rc, counter, _a, _u, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " beq %[p], %[u], 1f\n" \ + " add %[rc], %[p], %[a]\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : [a]"r" (_a), [u]"r" (_u) \ + : "memory"); \ +}) + /* This is required to provide a full barrier on success. */ static __always_inline int arch_atomic_fetch_add_unless(atomic_t *v, int a, int u) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "w"); + return prev; } #define arch_atomic_fetch_add_unless arch_atomic_fetch_add_unless @@ -222,77 +228,86 @@ static __always_inline s64 arch_atomic64_fetch_add_unless(atomic64_t *v, s64 a, s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " beq %[p], %[u], 1f\n" - " add %[rc], %[p], %[a]\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : [a]"r" (a), [u]"r" (u) - : "memory"); + _arch_atomic_fetch_add_unless(prev, rc, v->counter, a, u, "d"); + return prev; } #define arch_atomic64_fetch_add_unless arch_atomic64_fetch_add_unless #endif +#define _arch_atomic_inc_unless_negative(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bltz %[p], 1f\n" \ + " addi %[rc], %[p], 1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_inc_unless_negative(atomic_t *v) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "w"); + return !(prev < 0); } #define arch_atomic_inc_unless_negative arch_atomic_inc_unless_negative +#define _arch_atomic_dec_unless_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " bgtz %[p], 1f\n" \ + " addi %[rc], %[p], -1\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline bool arch_atomic_dec_unless_positive(atomic_t *v) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "w"); + return !(prev > 0); } #define arch_atomic_dec_unless_positive arch_atomic_dec_unless_positive +#define _arch_atomic_dec_if_positive(_prev, _rc, counter, sfx) \ +({ \ + __asm__ __volatile__ ( \ + "0: lr." sfx " %[p], %[c]\n" \ + " addi %[rc], %[p], -1\n" \ + " bltz %[rc], 1f\n" \ + " sc." sfx ".rl %[rc], %[rc], %[c]\n" \ + " bnez %[rc], 0b\n" \ + " fence rw, rw\n" \ + "1:\n" \ + : [p]"=&r" (_prev), [rc]"=&r" (_rc), [c]"+A" (counter) \ + : \ + : "memory"); \ +}) + static __always_inline int arch_atomic_dec_if_positive(atomic_t *v) { int prev, rc; - __asm__ __volatile__ ( - "0: lr.w %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.w.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "w"); + return prev - 1; } @@ -304,17 +319,8 @@ static __always_inline bool arch_atomic64_inc_unless_negative(atomic64_t *v) s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bltz %[p], 1f\n" - " addi %[rc], %[p], 1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_inc_unless_negative(prev, rc, v->counter, "d"); + return !(prev < 0); } @@ -325,17 +331,8 @@ static __always_inline bool arch_atomic64_dec_unless_positive(atomic64_t *v) s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " bgtz %[p], 1f\n" - " addi %[rc], %[p], -1\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_unless_positive(prev, rc, v->counter, "d"); + return !(prev > 0); } @@ -346,17 +343,8 @@ static __always_inline s64 arch_atomic64_dec_if_positive(atomic64_t *v) s64 prev; long rc; - __asm__ __volatile__ ( - "0: lr.d %[p], %[c]\n" - " addi %[rc], %[p], -1\n" - " bltz %[rc], 1f\n" - " sc.d.rl %[rc], %[rc], %[c]\n" - " bnez %[rc], 0b\n" - " fence rw, rw\n" - "1:\n" - : [p]"=&r" (prev), [rc]"=&r" (rc), [c]"+A" (v->counter) - : - : "memory"); + _arch_atomic_dec_if_positive(prev, rc, v->counter, "d"); + return prev - 1; } From patchwork Wed Aug 9 02:13:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133036 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp2531637vqr; Tue, 8 Aug 2023 20:06:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEOlZZ2+wbRRkKE98XwUeZwsVZcupWyQPyuZABzUXozmoHp5zG2vlW730BcgcXGGQ2pBzLl X-Received: by 2002:a05:6a21:1f1b:b0:140:6979:2952 with SMTP id ry27-20020a056a211f1b00b0014069792952mr1123237pzb.47.1691550378439; Tue, 08 Aug 2023 20:06:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691550378; cv=none; d=google.com; s=arc-20160816; b=Hebb1Xrz4auZv7M/uwWYqaAo5LD5uGyQNT6y63LoyPvXCe81+WEFzCmJeedXR4fuKx MA6N8W9YYTW1WI1Op15AgxPmYGxFeGqg3pbYUauG97o0lGSAc18DMQXUeLUGPmeBRIex 7dBnIv5BP3gU8iTpyNk6TyWZ06D6LcLyBjv0njNRAZtEKPupbP4I2DKqi0xWlPImMx5R zwrR+S2pNhzOWa6Wvq5reToEzAg9sS4k6Y5uGgM121Mubl8NLgbgQDFsQCb0t6796vOd NVqWRBt9FAp9c3TAYiowsqOLPGVISQ+eNwZPyWD45U2oOSdJVk8e+dRPKVcVJBZV/Vqq Q9Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yR44ogH6UDfncEstCVw9mSlc1d7ANRRQAVF648wbdjc=; fh=RC6bsfN8bPPp/iDUCpnJMEtdiqa5XC4G6XtB4MmAVnA=; b=qJMI3CLF4EOSxy9Y4pPxTljDWm54LF1glE6gLENxrZWtqoi6D9V4v8l+sksZTBLXWD FbDUaOVaVBKAdIeGwMnYMdzytxSj36qLphal0JNGNp4Vn9YbrBU1Pzk7vaYxI+jUzm1a tACTx9pB7/JNBIPK4uMFtI26a7N/pyzV1yRPmR88YAg0S0toPj7xj136G7mw71zfRwLM G+f5UmKkGtizBje8xU85+KwzbV82+4A+OhU7m+Qn92UWWdsCdZBYSVwRn67rZFDSfFdW YTCPhvAlnNoSeNnV0NKWSreljxfQf0ZsgVzIVF0vZerK6d59fCQChuzs0riZtW10i2Zd WG7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ccKJCTSe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bs71-20020a63284a000000b00563e8299dd7si8561479pgb.387.2023.08.08.20.06.05; Tue, 08 Aug 2023 20:06:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ccKJCTSe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230321AbjHICPF (ORCPT + 99 others); Tue, 8 Aug 2023 22:15:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230455AbjHICPB (ORCPT ); Tue, 8 Aug 2023 22:15:01 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3560B1BD3 for ; Tue, 8 Aug 2023 19:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yR44ogH6UDfncEstCVw9mSlc1d7ANRRQAVF648wbdjc=; b=ccKJCTSeZLh+a3usv3PTHRAW1GYDOfgwiEoyYbrIag47rmQpz9f6ANzaB4shd4GJVSk63Q fmR/wiZbRxMAFeMFdtTJGB1/SQD+Ozzzbk7W7ssn3ArZmkExC3xYSOE9tsUhMNroCSaTHr 1hCJFm+8Oy6W81cOxw3kTSUoFd8Mzqs= Received: from mail-ot1-f72.google.com (mail-ot1-f72.google.com [209.85.210.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-161-xq9A6TIuN7mbAS6ZXW-eYQ-1; Tue, 08 Aug 2023 22:14:15 -0400 X-MC-Unique: xq9A6TIuN7mbAS6ZXW-eYQ-1 Received: by mail-ot1-f72.google.com with SMTP id 46e09a7af769-6bb31e13a13so11085595a34.1 for ; Tue, 08 Aug 2023 19:14:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547254; x=1692152054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yR44ogH6UDfncEstCVw9mSlc1d7ANRRQAVF648wbdjc=; b=WFiIawWKMOs16siPhz4KO0owLIAclMJBxA5zlm54/ilVeFGJc/Urs2i1SjFuNKaWZx KocSGB7NTdTlQkZd9yGLY62da/PNDVWKp8ty867b/K3nZEvCggE8j2XkNT3d63qcx9Ag MC5zlRInJJpKF1+N8lGtOPAkLm96bNcAZKCm2Ob6PGbVFFKi131ZDoR90ptaXPCWVzGh +J6brggIxd5u+lKFEqVYOncnZzPATFJtHTV02essNB+aRByctGyxe2v9iLY9YID8s9v+ 09Liv0B4ObRUmIijIPCQEV830+quotBmPafqBOkofu5a61Ve7HigCh98sv8z9B1mmUFk iEJQ== X-Gm-Message-State: AOJu0Yw3lTOkMRgf1pNw1cX1KvEfChaXGuCTqFE/i7yZ7VsGQlmAmhh8 HnZcPwG1oMrjj7eOordVgq0YZFinJ0zj4NplNsbUj14Aek4lgEKr3gaJoHiOBg7sKYIt1Sp+Q2p UclwjycqICpAAMqJK/1JLqx+K X-Received: by 2002:a05:6870:40d1:b0:1be:d3a1:fd9 with SMTP id l17-20020a05687040d100b001bed3a10fd9mr1815530oal.9.1691547254601; Tue, 08 Aug 2023 19:14:14 -0700 (PDT) X-Received: by 2002:a05:6870:40d1:b0:1be:d3a1:fd9 with SMTP id l17-20020a05687040d100b001bed3a10fd9mr1815505oal.9.1691547254170; Tue, 08 Aug 2023 19:14:14 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.14.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:13 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 4/5] riscv/cmpxchg: Implement cmpxchg for variables of size 1 and 2 Date: Tue, 8 Aug 2023 23:13:09 -0300 Message-ID: <20230809021311.1390578-6-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773719129426821405 X-GMAIL-MSGID: 1773719129426821405 cmpxchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes cmpxchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 5a07646fae65..cfada8a7cfd2 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -72,6 +72,35 @@ * indicated by comparing RETURN with OLD. */ +#define __arch_cmpxchg_masked(sc_sfx, prepend, append, r, p, o, n) \ +({ \ + u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ + ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx = (ulong)(n) << __s; \ + ulong __oldx = (ulong)(o) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z5\n" \ + " bne %1, %z3, 1f\n" \ + " and %1, %0, %z6\n" \ + " or %1, %1, %z4\n" \ + " sc.w" sc_sfx " %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + "1:\n" \ + : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" ((long)__oldx), "rJ" (__newx), \ + "rJ" (__mask), "rJ" (~__mask) \ + : "memory"); \ + \ + r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) #define __arch_cmpxchg(lr_sfx, sc_sfx, prepend, append, r, p, co, o, n) \ ({ \ @@ -98,6 +127,11 @@ __typeof__(*(ptr)) __ret; \ \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_cmpxchg_masked(sc_sfx, prepend, append, \ + __ret, __ptr, __old, __new); \ + break; \ case 4: \ __arch_cmpxchg(".w", ".w" sc_sfx, prepend, append, \ __ret, __ptr, (long), __old, __new); \ From patchwork Wed Aug 9 02:13:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras Soares Passos X-Patchwork-Id: 133044 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp2542537vqr; Tue, 8 Aug 2023 20:38:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGMUIhwD07Sy4Nb/dI3B6L/70HWEvnWNJ8RdzFpVLnVqzz0X+ogY8LAr7MKKOLwZdAH7KIh X-Received: by 2002:a17:902:e74f:b0:1b8:948b:41b6 with SMTP id p15-20020a170902e74f00b001b8948b41b6mr15124902plf.10.1691552336440; Tue, 08 Aug 2023 20:38:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691552336; cv=none; d=google.com; s=arc-20160816; b=rjtsrxEpW+hv7J9pem+vBSYGKNTyScH2hBn/94nFywRcp2XZfDe7nJxPjE/jR9JH9L vWxnMS65RX3lM6DgtwOPiDCD6MdnCFE/FvDL0KO88Y3RjI1Q4UV4pGeCWEt2WFj6nMkM PThhMKZV8kxHxB6k45lOsh8I81vX81TRNF176j2DApk1scn+m9HBIYh5AYUdI4YklH7E HcPNuTo+LVwW81FA8N0rpguNkq7+N6K+G4/g/hWTcaYhn28feZE4SNCyom86gRg7qT2h 0yezlA5icQDX/gRhFmZUJa5undtkzdfy1tbKD6w7GyoKsegngCl5uEo9vfkWF/xeT4dt YWlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QSwkRG8bcSleBdtSsjCnyBmp/8Y+sHuiXOKQQOlQtH4=; fh=RC6bsfN8bPPp/iDUCpnJMEtdiqa5XC4G6XtB4MmAVnA=; b=WNczu+wfxMowFPf51e7a9/mire85FE4c74ZGA4fKYxBEy4KhkW2vZPyPoFl0aPgBRt ojmt5sPXY6UykgqcNNxHMgbHhO6oJV08/ra1zjnLVhvRKIkDIS53u0PkoqHKf/fmdV85 JEpMw0CCNVlzrKfMNoUU4lYNISQRp3OVRzAAR9WP1OEUvXr30ksVTnTRAvm0e7vUXIN/ HdnD0W130vwdU+jbt/+OrMHtxmEsfm/LziuSKxx6igu+fv4hfY8svCLWNo8rAifKHxJ5 QX0AYqG1eLZjxhSQMV2FgZHhP2EyqgGwJc2c3a6F8FqOQx44SdTonGTGZ0xtPaGNxFqI yHKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=WnzLQHsd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id li5-20020a170903294500b001bb162e944dsi8362975plb.51.2023.08.08.20.38.43; Tue, 08 Aug 2023 20:38:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=WnzLQHsd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229666AbjHICPN (ORCPT + 99 others); Tue, 8 Aug 2023 22:15:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229540AbjHICPI (ORCPT ); Tue, 8 Aug 2023 22:15:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA68E1BDA for ; Tue, 8 Aug 2023 19:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691547263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QSwkRG8bcSleBdtSsjCnyBmp/8Y+sHuiXOKQQOlQtH4=; b=WnzLQHsd7uNV/mInjNmcgWwQEaaqMqkhL+Q107HLGkQxwVk+E14r9nXvBDSaoKZqojPxDe bBNsRJqoffpb6Jizzm1e8sJSbD5PzZGvcLIfGXyRBI/JhxsbdJP8Klkiw4P+vk0kjVQl0r +dkCqWxK7VGWw0LjZUJ6ab14NaHa8/k= Received: from mail-oa1-f69.google.com (mail-oa1-f69.google.com [209.85.160.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-371-5fwvUZTtMJeHDZxbD_W5lg-1; Tue, 08 Aug 2023 22:14:21 -0400 X-MC-Unique: 5fwvUZTtMJeHDZxbD_W5lg-1 Received: by mail-oa1-f69.google.com with SMTP id 586e51a60fabf-1b0812d43a0so597717fac.0 for ; Tue, 08 Aug 2023 19:14:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691547261; x=1692152061; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QSwkRG8bcSleBdtSsjCnyBmp/8Y+sHuiXOKQQOlQtH4=; b=dFc0LrQprMs9CqGnkc3Wbbm2DVSObRM1LEPTfAs7XL03ee32d3gBqsivnQdLqWe26R qdB2KWtFZMTOYsbcyLtH9Flpz88ELwpi63B95tr6E3n+YkAat9lLPWBw9AsQOB0Jm2Yc 9NgVctX+dNVJD3zMUjmG6LdQ0tP1ziWazSpt7bOoczSuCz93aPqV63H+xSW02zkFm4Mg t50d7I+nutIAp03kSVOlCZHGCNDLlXqH0o0miTVJn+Y19AuhFnFwtaY0+lNa4zhaf2Us tfeyCWF0/8GoL2VW9p5fHGrEggJg06X9L8LFW1kkCGOfwTujn8Y/SzJPhf0T3EI5pzkc NP2Q== X-Gm-Message-State: AOJu0YyEgv2SRvyg14A8BBynm4qyKFGSkdZ33GKCEC5njkl/7pumrmTY BzrZFMbFmo6ngyc6tf6fS8fs1W15HqPjRftyQ8c1TJINpGSdeo2+2meIRlGD7Tjjj1s/sY46D4y qN5H2/W8MjztJxxRl/bBY05R1 X-Received: by 2002:a05:6870:b6a9:b0:1bf:1c49:749c with SMTP id cy41-20020a056870b6a900b001bf1c49749cmr8730778oab.13.1691547260770; Tue, 08 Aug 2023 19:14:20 -0700 (PDT) X-Received: by 2002:a05:6870:b6a9:b0:1bf:1c49:749c with SMTP id cy41-20020a056870b6a900b001bf1c49749cmr8730768oab.13.1691547260529; Tue, 08 Aug 2023 19:14:20 -0700 (PDT) Received: from localhost.localdomain ([2804:431:c7ec:e667:6b7d:ed55:c363:a088]) by smtp.gmail.com with ESMTPSA id v10-20020a4a8c4a000000b0054fba751207sm6475518ooj.47.2023.08.08.19.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Aug 2023 19:14:20 -0700 (PDT) From: Leonardo Bras To: Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Leonardo Bras , Andrea Parri , Ingo Molnar , Geert Uytterhoeven , Andrzej Hajda , Palmer Dabbelt , Guo Ren Cc: linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org Subject: [RFC PATCH v4 5/5] riscv/cmpxchg: Implement xchg for variables of size 1 and 2 Date: Tue, 8 Aug 2023 23:13:10 -0300 Message-ID: <20230809021311.1390578-7-leobras@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230809021311.1390578-2-leobras@redhat.com> References: <20230809021311.1390578-2-leobras@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773721183053431730 X-GMAIL-MSGID: 1773721183053431730 xchg for variables of size 1-byte and 2-bytes is not yet available for riscv, even though its present in other architectures such as arm64 and x86. This could lead to not being able to implement some locking mechanisms or requiring some rework to make it work properly. Implement 1-byte and 2-bytes xchg in order to achieve parity with other architectures. Signed-off-by: Leonardo Bras --- arch/riscv/include/asm/cmpxchg.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index cfada8a7cfd2..1640308f3c4d 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,6 +11,31 @@ #include #include +#define __arch_xchg_masked(prepend, append, r, p, n) \ +({ \ + u32 *__ptr32b = (u32 *)((ulong)(p) & ~0x3); \ + ulong __s = ((ulong)(p) & (0x4 - sizeof(*p))) * BITS_PER_BYTE; \ + ulong __mask = GENMASK(((sizeof(*p)) * BITS_PER_BYTE) - 1, 0) \ + << __s; \ + ulong __newx = (ulong)(n) << __s; \ + ulong __retx; \ + ulong __rc; \ + \ + __asm__ __volatile__ ( \ + prepend \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z4\n" \ + " or %1, %1, %z3\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + append \ + : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ + : "rJ" (__newx), "rJ" (~__mask) \ + : "memory"); \ + \ + r = (__typeof__(*(p)))((__retx & __mask) >> __s); \ +}) + #define __arch_xchg(sfx, prepend, append, r, p, n) \ ({ \ __asm__ __volatile__ ( \ @@ -27,7 +52,13 @@ __typeof__(*(ptr)) __new = (new); \ __typeof__(*(ptr)) __ret; \ __typeof__(ptr) __ptr = (ptr); \ + \ switch (sizeof(*__ptr)) { \ + case 1: \ + case 2: \ + __arch_xchg_masked(prepend, append, \ + __ret, __ptr, __new); \ + break; \ case 4: \ __arch_xchg(".w" sfx, prepend, append, \ __ret, __ptr, __new); \