From patchwork Sun Dec 31 08:29:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 184097 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp3418740dyb; Sun, 31 Dec 2023 00:31:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IHThCln5rFuluW4oSSnSkYHaJMCYhrcSgAk+JFCK/9lCr6wYcgu6829LTXsKRWRaiTN/2uW X-Received: by 2002:a05:6e02:1c28:b0:35f:7532:59d5 with SMTP id m8-20020a056e021c2800b0035f753259d5mr22560826ilh.31.1704011470860; Sun, 31 Dec 2023 00:31:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704011470; cv=none; d=google.com; s=arc-20160816; b=Fw2mg3EkKtEH+Rwhlz4kcEplDTTtB0Hg2TfTrgt/geGXF6Ih4NqohuJJftnNGclopB tt1nWd5XVdEFCy3oMAGz6NEhG39f+FecVFIJ+qlCrER9UPsFbFFF+KacFpwUfiQAtv8e DSFJlGlFn0C9B7x69l0EEy0Pa4W9/MQZFrpuC/EpnV2YdxWQ/7gJtOICmgeeKKMtkMgD cSVB3ZT3X3QACRvppx5JqzMdxPNP/Q6TAtNXV6jtKSyFjfuwS0KQ3ggwJ3iXOiy1Fn+r 09A7uLk56mssWF4UtcdkIXUq4eY/19BtTsinClJLy7Rfr2OF5V1u969sVlsKiUdKMltA tq9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=xGexFNeU+RSEavUfPJ6h7cdkTTwkqRt9Q3zroa8aWGY=; fh=8az2WX9seVnlqIwzdvRsfDVjT91R5QAYA+cOijWi0GY=; b=jsZDlc5FzF2t6aQXUFpeK76G+9E3RkCM76T7XVRLjtHND7UDxYchuw6oREe6s6C3gq wdj+gfzf5N/D4DjfP0+wUbh1AyWLnip4zh9wGDdvlQ45f5di7QsgWE0PAyv5HYhveVUK ciTfgTLIB/ptvPH4wwPSAgBr2hZ5tDpQGxL46Cav9vIB/9Dw4vuQAobC6wgrKDWg3pus G3q6ePqoMD3PX6w2iHGxwIIAsMyJ8rkQsaeJsgBOjyrw2IoiZ6swRNDnP/vPRgokwq2X X1I28xIFkR5FGQWTZUv3EpWKePdL6MYLSRmQncMotjc44znXZ+TcbJCtFE9nmGT2/ccq jplg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=l74aSVZn; spf=pass (google.com: domain of linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id u5-20020a17090282c500b001d1db5e39b4si16864544plz.172.2023.12.31.00.31.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 31 Dec 2023 00:31:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=l74aSVZn; spf=pass (google.com: domain of linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 8D6F4282B51 for ; Sun, 31 Dec 2023 08:31:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BBAFF11C85; Sun, 31 Dec 2023 08:30:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="l74aSVZn" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31F79F9FA for ; Sun, 31 Dec 2023 08:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E492C433C9; Sun, 31 Dec 2023 08:30:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704011424; bh=pGyWv9VDcODy3haubtquFUnBMC/QK1gpzpez9PuUJg8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l74aSVZnKCkxsU7iG5U39K6snGA5u0zORTgPF1sZDwcJ0bzyhtgB5uUiyzw0HVVsw ek4vZ6ge7ZUHDNrjVflanVu296zbSASdogcEg8Tb6frJ30w/0BZHKH5UJQoV7dXJ97 GeKq2Ox/1nR8xHE2+sEscfLFV5ut83FCw7ogbacrRHQq/Sa6Yp0wMU31UFkt+TVHfO u7t5+I8bIvXQjLwJOkgzlanYNs2WAr6AaUEXlmLudcTrbCGUW57Je2lm1kCJ8R/N/F 6bJ5FuvlLyeRYYASv8mvFG0yR+qcAblM9nRS23ll9s/RHLDqOvR9fgn3rkJwG5sq7P uFhh6mMktlIiQ== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, panqinglin2020@iscas.ac.cn, bjorn@rivosinc.com, conor.dooley@microchip.com, leobras@redhat.com, peterz@infradead.org, keescook@chromium.org, wuwei2016@iscas.ac.cn, xiaoguang.xing@sophgo.com, chao.wei@sophgo.com, unicorn_wang@outlook.com, uwu@icenowy.me, jszhang@kernel.org, wefu@redhat.com, atishp@atishpatra.org, ajones@ventanamicro.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [PATCH V2 3/3] riscv: xchg: Prefetch the destination word for sc.w Date: Sun, 31 Dec 2023 03:29:53 -0500 Message-Id: <20231231082955.16516-4-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231231082955.16516-1-guoren@kernel.org> References: <20231231082955.16516-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786785531953146404 X-GMAIL-MSGID: 1786785531953146404 From: Guo Ren The cost of changing a cacheline from shared to exclusive state can be significant, especially when this is triggered by an exclusive store, since it may result in having to retry the transaction. This patch makes use of prefetch.w to prefetch cachelines for write prior to lr/sc loops when using the xchg_small atomic routine. This patch is inspired by commit: 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word for write prior to stxr"). Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/include/asm/cmpxchg.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 26cea2395aae..d7b9d7951f08 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -10,6 +10,7 @@ #include #include +#include #define __arch_xchg_masked(prepend, append, r, p, n) \ ({ \ @@ -23,6 +24,7 @@ \ __asm__ __volatile__ ( \ prepend \ + PREFETCHW_ASM(%5) \ "0: lr.w %0, %2\n" \ " and %1, %0, %z4\n" \ " or %1, %1, %z3\n" \ @@ -30,7 +32,7 @@ " bnez %1, 0b\n" \ append \ : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ - : "rJ" (__newx), "rJ" (~__mask) \ + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \ : "memory"); \ \ r = (__typeof__(*(p)))((__retx & __mask) >> __s); \