From patchwork Sun Dec 31 08:29:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 184096 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp3418568dyb; Sun, 31 Dec 2023 00:30:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IFWf69z2mTwOFqiub1jc7N8PD3Uqrft0oor3XugisEP8J2qkhHti4kE6za3zjDj3tXNDwx+ X-Received: by 2002:a50:9514:0:b0:554:236b:d391 with SMTP id u20-20020a509514000000b00554236bd391mr6839936eda.9.1704011441432; Sun, 31 Dec 2023 00:30:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704011441; cv=none; d=google.com; s=arc-20160816; b=NV4C0X2CRSoQ3S9cgliP8rjRIMUR68wlfvwi1LuWUJExJMrjrCRn7Z2X/Ba9wMGu8W NeBempT6CwG/lFaR+x52fOYZFwo9tsPas2uphIVJ+WG7r+dc8yc10oINsgJCY+3AJDJi 6m0/m7xVLQDPtEnEXB3g3hWRjBzZmZoeuHY4ZD771ovA0VTuFRSgwKTca/NgSyRPu9rT K6f+UFyKiTiBgG0dnjx14YWLFbIl1pPCsTzOIWwiBcn8GKB5VYF770SNRRd5pR0s16s4 cAwJWQoIDO7yHPPI+gOJvYqyWLiHPVbVpvFzUsO4vNooYglhf719yhnMut3P7ZkxNKFx 42Fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=DNjTeVBddpFarDQUADavTh4ltNbVyQna6I+33dFAQRw=; fh=8az2WX9seVnlqIwzdvRsfDVjT91R5QAYA+cOijWi0GY=; b=KU8BXQsLYaUOWeczUbrOgFxx2oBYoNEU7cpkxf+WL/y7rJue6bm+LUY/23bs7k/kES XWOtXp/XsfOWS4ecnv/LSW4Co6RyHZTh7Eq5hjvN2uGb4dHPIBgnw+5anPGPFRBBltv3 oFJ8VpRBxhs36bUc6EBjnT4dq6sVWUCI2mNIlUhKzUb6I5I+/NJSGn2NmaXmpxTzQlL/ FnxexoXrNHnq+96WgOPVY0IzTzW2QfLtOVcfxbfouTDJtBcqHQHNmcAI6WxP6KnC3MT1 9IAMkAPWiq16PU0n+du7+8l6hbqz4F+fMq1HP1aeDePmMsJ0UHe900rGACFMvmWLS2yu hNFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=AgacJY7U; spf=pass (google.com: domain of linux-kernel+bounces-13652-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13652-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id an14-20020a17090656ce00b00a27cc46d6c3si792159ejc.310.2023.12.31.00.30.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 31 Dec 2023 00:30:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-13652-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=AgacJY7U; spf=pass (google.com: domain of linux-kernel+bounces-13652-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13652-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 11B811F21BC0 for ; Sun, 31 Dec 2023 08:30:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8E367BA45; Sun, 31 Dec 2023 08:30:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AgacJY7U" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1B0D9455 for ; Sun, 31 Dec 2023 08:30:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DE04DC433C8; Sun, 31 Dec 2023 08:30:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704011412; bh=lqr8kVrKoF7VekfM2/bpKrD4rWRk1NZ9sSJM5YGYlHk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AgacJY7UMsYUR5+lF8G56Nw0hW1kukPYIRTmm57xTr82tFnG+Q/7J6aMKzZO7FN/n 3GD4NM3cEEyAkmcdr5GQ6tx+DGenEEXRlptQmYVN3OB0TLtd6v+9T8Qih4UuQLIhHB 7rG2M0yRtvMsD3qnDQd5V6ZTE4/aZP8z0QL9SSyX/gHOQg2BkbkcQ+bcHI2ugQBfd4 03fyqm5bdN9TUJWcUlAnTupFArh0H8dKdfYo921qeadTiErrEgwRi1Ea0PvcV1B0qB x5ttO069TYDVY+YchUOzKd0CbRqeOwkKscTnI0yvRHQGOdin/irNujeJCtF+3gkf5P Q9zkeHLiX+wOg== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, panqinglin2020@iscas.ac.cn, bjorn@rivosinc.com, conor.dooley@microchip.com, leobras@redhat.com, peterz@infradead.org, keescook@chromium.org, wuwei2016@iscas.ac.cn, xiaoguang.xing@sophgo.com, chao.wei@sophgo.com, unicorn_wang@outlook.com, uwu@icenowy.me, jszhang@kernel.org, wefu@redhat.com, atishp@atishpatra.org, ajones@ventanamicro.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [PATCH V2 1/3] riscv: Add Zicbop instruction definitions & cpufeature Date: Sun, 31 Dec 2023 03:29:51 -0500 Message-Id: <20231231082955.16516-2-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231231082955.16516-1-guoren@kernel.org> References: <20231231082955.16516-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786785501169846868 X-GMAIL-MSGID: 1786785501169846868 From: Guo Ren Cache-block prefetch instructions are HINTs to the hardware to indicate that software intends to perform a particular type of memory access in the near future. This patch adds prefetch.i, prefetch.r and prefetch.w instruction definitions by RISCV_ISA_EXT_ZICBOP cpufeature. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/Kconfig | 15 ++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/insn-def.h | 60 +++++++++++++++++++++++++++++++ arch/riscv/kernel/cpufeature.c | 1 + 4 files changed, 77 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 24c1799e2ec4..fcbd417d65ea 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -579,6 +579,21 @@ config RISCV_ISA_ZICBOZ If you don't know what to do here, say Y. +config RISCV_ISA_ZICBOP + bool "Zicbop extension support for cache block prefetch" + depends on MMU + depends on RISCV_ALTERNATIVE + default y + help + Adds support to dynamically detect the presence of the ZICBOP + extension (Cache Block Prefetch Operations) and enable its + usage. + + The Zicbop extension can be used to prefetch cache block for + read/write fetch. + + If you don't know what to do here, say Y. + config TOOLCHAIN_HAS_ZIHINTPAUSE bool default y diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 06d30526ef3b..77d3b6ee25ab 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -57,6 +57,7 @@ #define RISCV_ISA_EXT_ZIHPM 42 #define RISCV_ISA_EXT_SMSTATEEN 43 #define RISCV_ISA_EXT_ZICOND 44 +#define RISCV_ISA_EXT_ZICBOP 45 #define RISCV_ISA_EXT_MAX 64 diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h index e27179b26086..bbda350a63bf 100644 --- a/arch/riscv/include/asm/insn-def.h +++ b/arch/riscv/include/asm/insn-def.h @@ -18,6 +18,13 @@ #define INSN_I_RD_SHIFT 7 #define INSN_I_OPCODE_SHIFT 0 +#define INSN_S_SIMM7_SHIFT 25 +#define INSN_S_RS2_SHIFT 20 +#define INSN_S_RS1_SHIFT 15 +#define INSN_S_FUNC3_SHIFT 12 +#define INSN_S_SIMM5_SHIFT 7 +#define INSN_S_OPCODE_SHIFT 0 + #ifdef __ASSEMBLY__ #ifdef CONFIG_AS_HAS_INSN @@ -30,6 +37,10 @@ .insn i \opcode, \func3, \rd, \rs1, \simm12 .endm + .macro insn_s, opcode, func3, rs2, simm12, rs1 + .insn s \opcode, \func3, \rs2, \simm12(\rs1) + .endm + #else #include @@ -51,10 +62,20 @@ (\simm12 << INSN_I_SIMM12_SHIFT)) .endm + .macro insn_s, opcode, func3, rs2, simm12, rs1 + .4byte ((\opcode << INSN_S_OPCODE_SHIFT) | \ + (\func3 << INSN_S_FUNC3_SHIFT) | \ + (.L__gpr_num_\rs2 << INSN_S_RS2_SHIFT) | \ + (.L__gpr_num_\rs1 << INSN_S_RS1_SHIFT) | \ + ((\simm12 & 0x1f) << INSN_S_SIMM5_SHIFT) | \ + (((\simm12 >> 5) & 0x7f) << INSN_S_SIMM7_SHIFT)) + .endm + #endif #define __INSN_R(...) insn_r __VA_ARGS__ #define __INSN_I(...) insn_i __VA_ARGS__ +#define __INSN_S(...) insn_s __VA_ARGS__ #else /* ! __ASSEMBLY__ */ @@ -66,6 +87,9 @@ #define __INSN_I(opcode, func3, rd, rs1, simm12) \ ".insn i " opcode ", " func3 ", " rd ", " rs1 ", " simm12 "\n" +#define __INSN_S(opcode, func3, rs2, simm12, rs1) \ + ".insn s " opcode ", " func3 ", " rs2 ", " simm12 "(" rs1 ")\n" + #else #include @@ -92,12 +116,26 @@ " (\\simm12 << " __stringify(INSN_I_SIMM12_SHIFT) "))\n" \ " .endm\n" +#define DEFINE_INSN_S \ + __DEFINE_ASM_GPR_NUMS \ +" .macro insn_s, opcode, func3, rs2, simm12, rs1\n" \ +" .4byte ((\\opcode << " __stringify(INSN_S_OPCODE_SHIFT) ") |" \ +" (\\func3 << " __stringify(INSN_S_FUNC3_SHIFT) ") |" \ +" (.L__gpr_num_\\rs2 << " __stringify(INSN_S_RS2_SHIFT) ") |" \ +" (.L__gpr_num_\\rs1 << " __stringify(INSN_S_RS1_SHIFT) ") |" \ +" ((\\simm12 & 0x1f) << " __stringify(INSN_S_SIMM5_SHIFT) ") |" \ +" (((\\simm12 >> 5) & 0x7f) << " __stringify(INSN_S_SIMM7_SHIFT) "))\n" \ +" .endm\n" + #define UNDEFINE_INSN_R \ " .purgem insn_r\n" #define UNDEFINE_INSN_I \ " .purgem insn_i\n" +#define UNDEFINE_INSN_S \ +" .purgem insn_s\n" + #define __INSN_R(opcode, func3, func7, rd, rs1, rs2) \ DEFINE_INSN_R \ "insn_r " opcode ", " func3 ", " func7 ", " rd ", " rs1 ", " rs2 "\n" \ @@ -108,6 +146,11 @@ "insn_i " opcode ", " func3 ", " rd ", " rs1 ", " simm12 "\n" \ UNDEFINE_INSN_I +#define __INSN_S(opcode, func3, rs2, simm12, rs1) \ + DEFINE_INSN_S \ + "insn_s " opcode ", " func3 ", " rs2 ", " simm12 ", " rs1 "\n" \ + UNDEFINE_INSN_S + #endif #endif /* ! __ASSEMBLY__ */ @@ -120,6 +163,10 @@ __INSN_I(RV_##opcode, RV_##func3, RV_##rd, \ RV_##rs1, RV_##simm12) +#define INSN_S(opcode, func3, rs2, simm12, rs1) \ + __INSN_S(RV_##opcode, RV_##func3, RV_##rs2, \ + RV_##simm12, RV_##rs1) + #define RV_OPCODE(v) __ASM_STR(v) #define RV_FUNC3(v) __ASM_STR(v) #define RV_FUNC7(v) __ASM_STR(v) @@ -133,6 +180,7 @@ #define RV___RS2(v) __RV_REG(v) #define RV_OPCODE_MISC_MEM RV_OPCODE(15) +#define RV_OPCODE_OP_IMM RV_OPCODE(19) #define RV_OPCODE_SYSTEM RV_OPCODE(115) #define HFENCE_VVMA(vaddr, asid) \ @@ -196,4 +244,16 @@ INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \ RS1(base), SIMM12(4)) +#define CBO_PREFETCH_I(base, offset) \ + INSN_S(OPCODE_OP_IMM, FUNC3(6), __RS2(0), \ + SIMM12(offset), RS1(base)) + +#define CBO_PREFETCH_R(base, offset) \ + INSN_S(OPCODE_OP_IMM, FUNC3(6), __RS2(1), \ + SIMM12(offset), RS1(base)) + +#define CBO_PREFETCH_W(base, offset) \ + INSN_S(OPCODE_OP_IMM, FUNC3(6), __RS2(3), \ + SIMM12(offset), RS1(base)) + #endif /* __ASM_INSN_DEF_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index b3785ffc1570..bdb02b066041 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -168,6 +168,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h), __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ), + __RISCV_ISA_EXT_DATA(zicbop, RISCV_ISA_EXT_ZICBOP), __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR), __RISCV_ISA_EXT_DATA(zicond, RISCV_ISA_EXT_ZICOND), __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR), From patchwork Sun Dec 31 08:29:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 184098 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp3418830dyb; Sun, 31 Dec 2023 00:31:30 -0800 (PST) X-Google-Smtp-Source: AGHT+IH1ZNuOfXiIZDuFWTMQr6leCMVAgHtsL24E5U9OfUs9Kc2xqThMYuKaUOMRc27Dp8BZ8er2 X-Received: by 2002:a17:903:110e:b0:1d4:83d5:a0cf with SMTP id n14-20020a170903110e00b001d483d5a0cfmr7152184plh.60.1704011490115; Sun, 31 Dec 2023 00:31:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704011490; cv=none; d=google.com; s=arc-20160816; b=UIP43Dpf/i2tac/vwflR1vJjsZRNYozYeaay12xc8DqyTQE6+Y2/VwK6tUBqaHaOaw Bwb9UhpKLbHacM7/eQtAMHXZiy16cmsBfEtLGwWctTiCkuldzf2FQ7oykh1HkHFIDWDy Z6kvmkVjKcUCajX8GFOrrDg4PvYqeP2rdZzxR16gkbi5XwdT+tD+PxoLb3cWSjTTwn9b zUxuL+GaXETkopYG0hNuOPoCDhtG9XJSb5AfVZVtGJhszx+ByBXAarwAsu9cy5ceuoXM EjzVPOXVmse0mGRC+7vc3iIiPZB/8n5ROrR7N1VI8RmEe2ZJ1caZbBqlfT708fblVF1O hVeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Nz7CIG6pduiVe24ADMOGO4P+ttlMk1mFOo1jePLewb0=; fh=8az2WX9seVnlqIwzdvRsfDVjT91R5QAYA+cOijWi0GY=; b=BW8ZTNdketZZNGxwBijA/qr/JaMcrxA7SLzZQ5cnIbOIj255WS3NdjEEchB+1Mn8sg JD2mu0ab4P2Ct9kosVXEf6cq4C06yhZ7GDUKM5WCRbreQP0cDhVoDPJ6DFpz5W2eddPs RZacxt0yXjnWy+SfWxg3s3MZtuZ8yQDalrb/oFOFwy+CI2fNm0oDMDKEUj6gC/kRvcBl J7kZRgfjnVlPw60HHHDKgTfMu2w+9BmxFBAFFU41a4roNFlX5PHrVtIOc4ysewnhIPYZ d9brc2McAPCpe2UTuL2hJgQaAjfwo7yofjNhosFvxjdliyRMtzD/X0Sp3ASiRSkyz2yM LzMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="pV/cfNnl"; spf=pass (google.com: domain of linux-kernel+bounces-13653-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13653-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id jw22-20020a170903279600b001d4a179e686si3309583plb.457.2023.12.31.00.31.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 31 Dec 2023 00:31:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-13653-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="pV/cfNnl"; spf=pass (google.com: domain of linux-kernel+bounces-13653-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13653-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 3AFC6B219D9 for ; Sun, 31 Dec 2023 08:30:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9A019E552; Sun, 31 Dec 2023 08:30:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pV/cfNnl" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1388FC8D5 for ; Sun, 31 Dec 2023 08:30:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0B8B8C433C7; Sun, 31 Dec 2023 08:30:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704011418; bh=D6VXuRwAKv9dO+mNmqMLcBG3Fo8hoXZOx0qi+4o2+wM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pV/cfNnlutV5oJo3GVEF+qPX7SM5xqctJKNU2amJ7TJU4IOiq3JWRgdUT3zEA+zi6 yA+d+w+GWovK1mBnqsKKPAxRFaGoEXNANQwL9/HeB2MfY2USxVQhZJ8gxA11tPd/ue ATpwZ1PQR5TOJg2o5rIlKdnu3JGH35NPy7eQ+GPyknwR0xYo7c8BwjD5fehnihnKCX T02m2Cg1NosuM5VC+Kz+A4jl4He+Eq8Jc6iqdT4m572jiZKEeAE+S8eMqhezNpsHe8 dHtKfAfC7klMoG5W7b00l4uXr43yVa8nE9paBEU8WFl2azCTQsVJosZ64QKwWKcfAX iJLa9KiKs6u5w== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, panqinglin2020@iscas.ac.cn, bjorn@rivosinc.com, conor.dooley@microchip.com, leobras@redhat.com, peterz@infradead.org, keescook@chromium.org, wuwei2016@iscas.ac.cn, xiaoguang.xing@sophgo.com, chao.wei@sophgo.com, unicorn_wang@outlook.com, uwu@icenowy.me, jszhang@kernel.org, wefu@redhat.com, atishp@atishpatra.org, ajones@ventanamicro.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [PATCH V2 2/3] riscv: Add ARCH_HAS_PRETCHW support with Zibop Date: Sun, 31 Dec 2023 03:29:52 -0500 Message-Id: <20231231082955.16516-3-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231231082955.16516-1-guoren@kernel.org> References: <20231231082955.16516-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786785552322449242 X-GMAIL-MSGID: 1786785552322449242 From: Guo Ren Enable Linux prefetchw primitive with Zibop cpufeature, which preloads cache line into L1 cache for the next write operation. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/include/asm/processor.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index f19f861cda54..8d3a2ab37678 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -13,6 +13,9 @@ #include #include +#include +#include +#include #ifdef CONFIG_64BIT #define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1)) @@ -106,6 +109,19 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset, #define KSTK_EIP(tsk) (task_pt_regs(tsk)->epc) #define KSTK_ESP(tsk) (task_pt_regs(tsk)->sp) +#ifdef CONFIG_RISCV_ISA_ZICBOP +#define ARCH_HAS_PREFETCHW + +#define PREFETCHW_ASM(x) \ + ALTERNATIVE(__nops(1), CBO_PREFETCH_W(x, 0), 0, \ + RISCV_ISA_EXT_ZICBOP, CONFIG_RISCV_ISA_ZICBOP) + + +static inline void prefetchw(const void *x) +{ + __asm__ __volatile__(PREFETCHW_ASM(%0) : : "r" (x) : "memory"); +} +#endif /* CONFIG_RISCV_ISA_ZICBOP */ /* Do necessary setup to start up a newly executed thread. */ extern void start_thread(struct pt_regs *regs, From patchwork Sun Dec 31 08:29:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 184097 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp3418740dyb; Sun, 31 Dec 2023 00:31:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IHThCln5rFuluW4oSSnSkYHaJMCYhrcSgAk+JFCK/9lCr6wYcgu6829LTXsKRWRaiTN/2uW X-Received: by 2002:a05:6e02:1c28:b0:35f:7532:59d5 with SMTP id m8-20020a056e021c2800b0035f753259d5mr22560826ilh.31.1704011470860; Sun, 31 Dec 2023 00:31:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704011470; cv=none; d=google.com; s=arc-20160816; b=Fw2mg3EkKtEH+Rwhlz4kcEplDTTtB0Hg2TfTrgt/geGXF6Ih4NqohuJJftnNGclopB tt1nWd5XVdEFCy3oMAGz6NEhG39f+FecVFIJ+qlCrER9UPsFbFFF+KacFpwUfiQAtv8e DSFJlGlFn0C9B7x69l0EEy0Pa4W9/MQZFrpuC/EpnV2YdxWQ/7gJtOICmgeeKKMtkMgD cSVB3ZT3X3QACRvppx5JqzMdxPNP/Q6TAtNXV6jtKSyFjfuwS0KQ3ggwJ3iXOiy1Fn+r 09A7uLk56mssWF4UtcdkIXUq4eY/19BtTsinClJLy7Rfr2OF5V1u969sVlsKiUdKMltA tq9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=xGexFNeU+RSEavUfPJ6h7cdkTTwkqRt9Q3zroa8aWGY=; fh=8az2WX9seVnlqIwzdvRsfDVjT91R5QAYA+cOijWi0GY=; b=jsZDlc5FzF2t6aQXUFpeK76G+9E3RkCM76T7XVRLjtHND7UDxYchuw6oREe6s6C3gq wdj+gfzf5N/D4DjfP0+wUbh1AyWLnip4zh9wGDdvlQ45f5di7QsgWE0PAyv5HYhveVUK ciTfgTLIB/ptvPH4wwPSAgBr2hZ5tDpQGxL46Cav9vIB/9Dw4vuQAobC6wgrKDWg3pus G3q6ePqoMD3PX6w2iHGxwIIAsMyJ8rkQsaeJsgBOjyrw2IoiZ6swRNDnP/vPRgokwq2X X1I28xIFkR5FGQWTZUv3EpWKePdL6MYLSRmQncMotjc44znXZ+TcbJCtFE9nmGT2/ccq jplg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=l74aSVZn; spf=pass (google.com: domain of linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id u5-20020a17090282c500b001d1db5e39b4si16864544plz.172.2023.12.31.00.31.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 31 Dec 2023 00:31:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=l74aSVZn; spf=pass (google.com: domain of linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13654-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 8D6F4282B51 for ; Sun, 31 Dec 2023 08:31:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BBAFF11C85; Sun, 31 Dec 2023 08:30:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="l74aSVZn" X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31F79F9FA for ; Sun, 31 Dec 2023 08:30:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2E492C433C9; Sun, 31 Dec 2023 08:30:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704011424; bh=pGyWv9VDcODy3haubtquFUnBMC/QK1gpzpez9PuUJg8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l74aSVZnKCkxsU7iG5U39K6snGA5u0zORTgPF1sZDwcJ0bzyhtgB5uUiyzw0HVVsw ek4vZ6ge7ZUHDNrjVflanVu296zbSASdogcEg8Tb6frJ30w/0BZHKH5UJQoV7dXJ97 GeKq2Ox/1nR8xHE2+sEscfLFV5ut83FCw7ogbacrRHQq/Sa6Yp0wMU31UFkt+TVHfO u7t5+I8bIvXQjLwJOkgzlanYNs2WAr6AaUEXlmLudcTrbCGUW57Je2lm1kCJ8R/N/F 6bJ5FuvlLyeRYYASv8mvFG0yR+qcAblM9nRS23ll9s/RHLDqOvR9fgn3rkJwG5sq7P uFhh6mMktlIiQ== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, panqinglin2020@iscas.ac.cn, bjorn@rivosinc.com, conor.dooley@microchip.com, leobras@redhat.com, peterz@infradead.org, keescook@chromium.org, wuwei2016@iscas.ac.cn, xiaoguang.xing@sophgo.com, chao.wei@sophgo.com, unicorn_wang@outlook.com, uwu@icenowy.me, jszhang@kernel.org, wefu@redhat.com, atishp@atishpatra.org, ajones@ventanamicro.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [PATCH V2 3/3] riscv: xchg: Prefetch the destination word for sc.w Date: Sun, 31 Dec 2023 03:29:53 -0500 Message-Id: <20231231082955.16516-4-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231231082955.16516-1-guoren@kernel.org> References: <20231231082955.16516-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786785531953146404 X-GMAIL-MSGID: 1786785531953146404 From: Guo Ren The cost of changing a cacheline from shared to exclusive state can be significant, especially when this is triggered by an exclusive store, since it may result in having to retry the transaction. This patch makes use of prefetch.w to prefetch cachelines for write prior to lr/sc loops when using the xchg_small atomic routine. This patch is inspired by commit: 0ea366f5e1b6 ("arm64: atomics: prefetch the destination word for write prior to stxr"). Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/include/asm/cmpxchg.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 26cea2395aae..d7b9d7951f08 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -10,6 +10,7 @@ #include #include +#include #define __arch_xchg_masked(prepend, append, r, p, n) \ ({ \ @@ -23,6 +24,7 @@ \ __asm__ __volatile__ ( \ prepend \ + PREFETCHW_ASM(%5) \ "0: lr.w %0, %2\n" \ " and %1, %0, %z4\n" \ " or %1, %1, %z3\n" \ @@ -30,7 +32,7 @@ " bnez %1, 0b\n" \ append \ : "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \ - : "rJ" (__newx), "rJ" (~__mask) \ + : "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \ : "memory"); \ \ r = (__typeof__(*(p)))((__retx & __mask) >> __s); \