Message ID | 20231220-optimize_checksum-v13-2-a73547e1cad8@rivosinc.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-7638-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp77038dyi; Wed, 20 Dec 2023 15:39:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IEHJL6cWgobkzg50pUJJiBbw7O4o6SsG9SEQhRi9ws9Ei9AL+x3900tC38XkssKHiUkeHHO X-Received: by 2002:a05:6808:3309:b0:3ba:231:cd4d with SMTP id ca9-20020a056808330900b003ba0231cd4dmr28809394oib.103.1703115563551; Wed, 20 Dec 2023 15:39:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703115563; cv=none; d=google.com; s=arc-20160816; b=x0k4r4PeKCpPK1luc5UKZwRfLU4TdvwseEung26y2cefP3LB+/Z8IwQB6PhB+prxx+ kJTGKlQGjsuz4PfRA7KJ7MOJLQoFBzs/6Ownk5KuKR8zvdx9X7FukC4XMSb6xOwFkfcP Rw1xJoEdehtmozHP/QbS5iSVhZbJGu7t3GTe5jNWtrZQSLBrHNqwgrc5feXni+wFxkSU +YBs9TcgLuQ+nHrb9T2Qc+Mf6X1wy0nrD/FHp1C0yfC/aVUihVAaXizXxeKfXue0xYH7 qwAt1h69wa2LMcMDC7uuj70rwZJ+baE40LzFBx0bXW+80eapQfp4u25fo6PEIQC8W2Hc PHpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=hqZmArtiXcDp/s2cDCE6VomUeBtSGPPb+28mpDKaCFA=; fh=NvPEvYYb1ljjPtQroLU0Yeq/uN2Mxoa3J4vK70VhuM0=; b=ggrYCifhzclecQumgHZhh7tNT+NvWG8n5nPuftXom2EAZwOjM1s7kNawouuwVkI+o6 8PuzsBJ7gLgV+Oxtc8STQ8ml8xqTWe9BwXv6pcfO1eUDnkss4RJed66We2eqnzPcuR2t C6rfzhznZfAGlJrOrserLgw83mEqBSFrJlXretKSetq1rs757s+SlWKqiwyCPrgz2mLG Vk6vdLPFMPWFfNjLl64DF3pmiNdUhDagU+1LajhOA6uYPt42k7s0SAkShPoKng4j3vSS sdOmmkpqlR4rQLHKk7XxAcRAaTvkDZhD1uAoavVmyLF5eHbxK3Zp7k0CWDEektzzsK/B Dnfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=06r2xLwo; spf=pass (google.com: domain of linux-kernel+bounces-7638-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-7638-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id y4-20020a056a001c8400b006d931c4966asi500884pfw.67.2023.12.20.15.39.23 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Dec 2023 15:39:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-7638-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=06r2xLwo; spf=pass (google.com: domain of linux-kernel+bounces-7638-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-7638-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 46703B22A3C for <ouuuleilei@gmail.com>; Wed, 20 Dec 2023 23:38:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C8C084C3CE; Wed, 20 Dec 2023 23:37:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="06r2xLwo" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-oa1-f46.google.com (mail-oa1-f46.google.com [209.85.160.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7BC94B142 for <linux-kernel@vger.kernel.org>; Wed, 20 Dec 2023 23:37:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Received: by mail-oa1-f46.google.com with SMTP id 586e51a60fabf-2042964b502so99936fac.3 for <linux-kernel@vger.kernel.org>; Wed, 20 Dec 2023 15:37:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1703115465; x=1703720265; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hqZmArtiXcDp/s2cDCE6VomUeBtSGPPb+28mpDKaCFA=; b=06r2xLwoQSA8lns02rRsdnIdDCnQG9524FvY6rw+4MoiXbJ49sl8k1TxRap88AC0ER bmcazPHa7dee+9pT9CMxmpxzoaHSbtO3dGS8bhfAHKKpu5Yrwwu9qjGAFfv7LSV6Skko ZqNdin08aVnG/RLZFuNxbm74iHDvF7B8nUrnSwXehKXtBB+exdjWK35uwUjxSn+/b6Mn n53tZuYbn56e6dd5u3Exk0ux+seFxddIUwr2G0jYkzNLRq41FtpZcx6RxpRZqxNPTAXg 9WQQqKcGi9pBnT71zwxu4J+IuqW7NayiLF+3wkIDHHp0/FsHjn35JN7E3+hM3ELwNKqT BkqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703115465; x=1703720265; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hqZmArtiXcDp/s2cDCE6VomUeBtSGPPb+28mpDKaCFA=; b=j8I7lm+mE8QTeR+aPoM52b3Qx0ZYnM4q9y25Gk+2IBF0Z7GF1CPVEKze7L4VN9wV+C nTU8MO3L1NiTs1B3OzTOKPYmAqNXJFvuN2JRCIIAn4jKiMNBd4KmfJ8BI+fgJJOQ99CK f4JqHT500scqdAjYUBNIYuEtdKqT2j/zX2k/pymW/jfGYBfp8C9cFaOVeCsgXWyeoMh+ l9ZGvgIju312N4U79lH4kVJZanTQmxKa6zzAM/ddqO9+/1VhPMxJcinmnbLWGPYuL9Dk CZZ3+3ptsdZ6njUVIrzRu/20jky82erKFnQmwQEcdni9iQnLSEZvEvYCnynCiHISmI1Z ftPg== X-Gm-Message-State: AOJu0Yy/yuTdXGtY7tZQUnAkUBWjA9blR6EBvBEwEdIZO/WXHP1iaE0Z uylxH1nrjNhI23FsGVeSmpnDIA== X-Received: by 2002:a05:6870:970f:b0:203:bcc7:8fa2 with SMTP id n15-20020a056870970f00b00203bcc78fa2mr672978oaq.5.1703115464850; Wed, 20 Dec 2023 15:37:44 -0800 (PST) Received: from charlie.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id k5-20020a056830150500b006d87e38f91asm132834otp.56.2023.12.20.15.37.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Dec 2023 15:37:44 -0800 (PST) From: Charlie Jenkins <charlie@rivosinc.com> Date: Wed, 20 Dec 2023 15:37:40 -0800 Subject: [PATCH v13 2/5] riscv: Add static key for misaligned accesses Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20231220-optimize_checksum-v13-2-a73547e1cad8@rivosinc.com> References: <20231220-optimize_checksum-v13-0-a73547e1cad8@rivosinc.com> In-Reply-To: <20231220-optimize_checksum-v13-0-a73547e1cad8@rivosinc.com> To: Charlie Jenkins <charlie@rivosinc.com>, Palmer Dabbelt <palmer@dabbelt.com>, Conor Dooley <conor@kernel.org>, Samuel Holland <samuel.holland@sifive.com>, David Laight <David.Laight@aculab.com>, Xiao Wang <xiao.w.wang@intel.com>, Evan Green <evan@rivosinc.com>, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org Cc: Paul Walmsley <paul.walmsley@sifive.com>, Albert Ou <aou@eecs.berkeley.edu>, Arnd Bergmann <arnd@arndb.de> X-Mailer: b4 0.12.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1703115460; l=2270; i=charlie@rivosinc.com; s=20231120; h=from:subject:message-id; bh=zAvCCM1AIoiNlx7AQwltNFw7ekzVRWkGRkhZ8BNL3iE=; b=/aoALA0sPu/mQrg3wwtXeXAI2bR1ghCTU4z4qItLaJnMLYVXoeBeX/8KZsrRPbIY0Vt0eIUCZ CCU5N5pjCbdA2zEuvQCl+CIPeGSez/Ug3N2nd09YxVNQiBASexBl8Fz X-Developer-Key: i=charlie@rivosinc.com; a=ed25519; pk=t4RSWpMV1q5lf/NWIeR9z58bcje60/dbtxxmoSfBEcs= X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785846104861055592 X-GMAIL-MSGID: 1785846104861055592 |
Series |
riscv: Add fine-tuned checksum functions
|
|
Commit Message
Charlie Jenkins
Dec. 20, 2023, 11:37 p.m. UTC
Support static branches depending on the value of misaligned accesses.
This will be used by a later patch in the series. All cpus must be
considered "fast" for this static branch to be flipped.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
arch/riscv/include/asm/cpufeature.h | 2 ++
arch/riscv/kernel/cpufeature.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 32 insertions(+)
Comments
On Wed, Dec 20, 2023 at 3:37 PM Charlie Jenkins <charlie@rivosinc.com> wrote: > > Support static branches depending on the value of misaligned accesses. > This will be used by a later patch in the series. All cpus must be > considered "fast" for this static branch to be flipped. > > Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> You didn't pick up my tag from the last spin, so here it is again: Reviewed-by: Evan Green <evan@rivosinc.com>
On Thu, Dec 21, 2023 at 7:38 AM Charlie Jenkins <charlie@rivosinc.com> wrote: > > Support static branches depending on the value of misaligned accesses. > This will be used by a later patch in the series. All cpus must be > considered "fast" for this static branch to be flipped. > > Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> > --- > arch/riscv/include/asm/cpufeature.h | 2 ++ > arch/riscv/kernel/cpufeature.c | 30 ++++++++++++++++++++++++++++++ > 2 files changed, 32 insertions(+) > > diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h > index a418c3112cd6..7b129e5e2f07 100644 > --- a/arch/riscv/include/asm/cpufeature.h > +++ b/arch/riscv/include/asm/cpufeature.h > @@ -133,4 +133,6 @@ static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsi > return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); > } > > +DECLARE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); > + > #endif > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c > index b3785ffc1570..095eb6ebdcaa 100644 > --- a/arch/riscv/kernel/cpufeature.c > +++ b/arch/riscv/kernel/cpufeature.c > @@ -10,6 +10,7 @@ > #include <linux/bitmap.h> > #include <linux/cpuhotplug.h> > #include <linux/ctype.h> > +#include <linux/jump_label.h> > #include <linux/log2.h> > #include <linux/memory.h> > #include <linux/module.h> > @@ -728,6 +729,35 @@ void riscv_user_isa_enable(void) > csr_set(CSR_SENVCFG, ENVCFG_CBZE); > } > > +DEFINE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); > + > +static int set_unaligned_access_static_branches(void) > +{ > + /* > + * This will be called after check_unaligned_access_all_cpus so the > + * result of unaligned access speed for all cpus will be available. > + */ > + > + int cpu; > + bool fast_misaligned_access_speed = true; > + > + for_each_online_cpu(cpu) { Each online_cpu? Is there any offline_cpu that is no fast_misaligned_access_speed? Move into your riscv_online_cpu for each CPU, and use stop_machine for synchronization. > + int this_perf = per_cpu(misaligned_access_speed, cpu); > + > + if (this_perf != RISCV_HWPROBE_MISALIGNED_FAST) { > + fast_misaligned_access_speed = false; > + break; > + } > + } > + > + if (fast_misaligned_access_speed) > + static_branch_enable(&fast_misaligned_access_speed_key); > + > + return 0; > +} > + > +arch_initcall_sync(set_unaligned_access_static_branches); > + > #ifdef CONFIG_RISCV_ALTERNATIVE > /* > * Alternative patch sites consider 48 bits when determining when to patch > > -- > 2.43.0 > >
On Fri, Dec 22, 2023 at 08:33:18AM +0800, Guo Ren wrote: > On Thu, Dec 21, 2023 at 7:38 AM Charlie Jenkins <charlie@rivosinc.com> wrote: > > > > Support static branches depending on the value of misaligned accesses. > > This will be used by a later patch in the series. All cpus must be > > considered "fast" for this static branch to be flipped. > > > > Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> > > --- > > arch/riscv/include/asm/cpufeature.h | 2 ++ > > arch/riscv/kernel/cpufeature.c | 30 ++++++++++++++++++++++++++++++ > > 2 files changed, 32 insertions(+) > > > > diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h > > index a418c3112cd6..7b129e5e2f07 100644 > > --- a/arch/riscv/include/asm/cpufeature.h > > +++ b/arch/riscv/include/asm/cpufeature.h > > @@ -133,4 +133,6 @@ static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsi > > return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); > > } > > > > +DECLARE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); > > + > > #endif > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c > > index b3785ffc1570..095eb6ebdcaa 100644 > > --- a/arch/riscv/kernel/cpufeature.c > > +++ b/arch/riscv/kernel/cpufeature.c > > @@ -10,6 +10,7 @@ > > #include <linux/bitmap.h> > > #include <linux/cpuhotplug.h> > > #include <linux/ctype.h> > > +#include <linux/jump_label.h> > > #include <linux/log2.h> > > #include <linux/memory.h> > > #include <linux/module.h> > > @@ -728,6 +729,35 @@ void riscv_user_isa_enable(void) > > csr_set(CSR_SENVCFG, ENVCFG_CBZE); > > } > > > > +DEFINE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); > > + > > +static int set_unaligned_access_static_branches(void) > > +{ > > + /* > > + * This will be called after check_unaligned_access_all_cpus so the > > + * result of unaligned access speed for all cpus will be available. > > + */ > > + > > + int cpu; > > + bool fast_misaligned_access_speed = true; > > + > > + for_each_online_cpu(cpu) { > Each online_cpu? Is there any offline_cpu that is no > fast_misaligned_access_speed? I think instead of checking offline cpus, it would make more sense to adjust the static branch when offline cpus come online. Since riscv_online_cpu is called when a new CPU comes online, I can update the static branch inside of that function. > > Move into your riscv_online_cpu for each CPU, and use stop_machine for > synchronization. > I do not understand what you mean by "Move into your riscv_online_cpu for each CPU", but I am assuming you are referring to updating the static branch inside of riscv_online_cpu. I believe any race condition that could be solved by stop_machine will become irrelevent by ensuring that the static branch is updated when a new cpu comes online. - Charlie > > + int this_perf = per_cpu(misaligned_access_speed, cpu); > > + > > + if (this_perf != RISCV_HWPROBE_MISALIGNED_FAST) { > > + fast_misaligned_access_speed = false; > > + break; > > + } > > + } > > + > > + if (fast_misaligned_access_speed) > > + static_branch_enable(&fast_misaligned_access_speed_key); > > + > > + return 0; > > +} > > + > > +arch_initcall_sync(set_unaligned_access_static_branches); > > + > > #ifdef CONFIG_RISCV_ALTERNATIVE > > /* > > * Alternative patch sites consider 48 bits when determining when to patch > > > > -- > > 2.43.0 > > > > > > > -- > Best Regards > Guo Ren
On Fri, Dec 22, 2023 at 9:37 AM Charlie Jenkins <charlie@rivosinc.com> wrote: > > On Fri, Dec 22, 2023 at 08:33:18AM +0800, Guo Ren wrote: > > On Thu, Dec 21, 2023 at 7:38 AM Charlie Jenkins <charlie@rivosinc.com> wrote: > > > > > > Support static branches depending on the value of misaligned accesses. > > > This will be used by a later patch in the series. All cpus must be > > > considered "fast" for this static branch to be flipped. > > > > > > Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> > > > --- > > > arch/riscv/include/asm/cpufeature.h | 2 ++ > > > arch/riscv/kernel/cpufeature.c | 30 ++++++++++++++++++++++++++++++ > > > 2 files changed, 32 insertions(+) > > > > > > diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h > > > index a418c3112cd6..7b129e5e2f07 100644 > > > --- a/arch/riscv/include/asm/cpufeature.h > > > +++ b/arch/riscv/include/asm/cpufeature.h > > > @@ -133,4 +133,6 @@ static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsi > > > return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); > > > } > > > > > > +DECLARE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); > > > + > > > #endif > > > diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c > > > index b3785ffc1570..095eb6ebdcaa 100644 > > > --- a/arch/riscv/kernel/cpufeature.c > > > +++ b/arch/riscv/kernel/cpufeature.c > > > @@ -10,6 +10,7 @@ > > > #include <linux/bitmap.h> > > > #include <linux/cpuhotplug.h> > > > #include <linux/ctype.h> > > > +#include <linux/jump_label.h> > > > #include <linux/log2.h> > > > #include <linux/memory.h> > > > #include <linux/module.h> > > > @@ -728,6 +729,35 @@ void riscv_user_isa_enable(void) > > > csr_set(CSR_SENVCFG, ENVCFG_CBZE); > > > } > > > > > > +DEFINE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); > > > + > > > +static int set_unaligned_access_static_branches(void) > > > +{ > > > + /* > > > + * This will be called after check_unaligned_access_all_cpus so the > > > + * result of unaligned access speed for all cpus will be available. > > > + */ > > > + > > > + int cpu; > > > + bool fast_misaligned_access_speed = true; > > > + > > > + for_each_online_cpu(cpu) { > > Each online_cpu? Is there any offline_cpu that is no > > fast_misaligned_access_speed? > > I think instead of checking offline cpus, it would make more sense to > adjust the static branch when offline cpus come online. Since > riscv_online_cpu is called when a new CPU comes online, I can update the > static branch inside of that function. > > > > > Move into your riscv_online_cpu for each CPU, and use stop_machine for > > synchronization. > > > > I do not understand what you mean by "Move into your riscv_online_cpu > for each CPU", but I am assuming you are referring to updating the > static branch inside of riscv_online_cpu. I mean in: arch/riscv/kernel/cpufeature.c: riscv_online_cpu() Yes,"adjust the static branch when offline cpus come online ..." > > I believe any race condition that could be solved by stop_machine will > become irrelevent by ensuring that the static branch is updated when a > new cpu comes online. Em... stop_machine may be not necessary. > > - Charlie > > > > + int this_perf = per_cpu(misaligned_access_speed, cpu); > > > + > > > + if (this_perf != RISCV_HWPROBE_MISALIGNED_FAST) { > > > + fast_misaligned_access_speed = false; > > > + break; > > > + } > > > + } > > > + > > > + if (fast_misaligned_access_speed) > > > + static_branch_enable(&fast_misaligned_access_speed_key); > > > + > > > + return 0; > > > +} > > > + > > > +arch_initcall_sync(set_unaligned_access_static_branches); > > > + > > > #ifdef CONFIG_RISCV_ALTERNATIVE > > > /* > > > * Alternative patch sites consider 48 bits when determining when to patch > > > > > > -- > > > 2.43.0 > > > > > > > > > > > > -- > > Best Regards > > Guo Ren
diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index a418c3112cd6..7b129e5e2f07 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -133,4 +133,6 @@ static __always_inline bool riscv_cpu_has_extension_unlikely(int cpu, const unsi return __riscv_isa_extension_available(hart_isa[cpu].isa, ext); } +DECLARE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); + #endif diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index b3785ffc1570..095eb6ebdcaa 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -10,6 +10,7 @@ #include <linux/bitmap.h> #include <linux/cpuhotplug.h> #include <linux/ctype.h> +#include <linux/jump_label.h> #include <linux/log2.h> #include <linux/memory.h> #include <linux/module.h> @@ -728,6 +729,35 @@ void riscv_user_isa_enable(void) csr_set(CSR_SENVCFG, ENVCFG_CBZE); } +DEFINE_STATIC_KEY_FALSE(fast_misaligned_access_speed_key); + +static int set_unaligned_access_static_branches(void) +{ + /* + * This will be called after check_unaligned_access_all_cpus so the + * result of unaligned access speed for all cpus will be available. + */ + + int cpu; + bool fast_misaligned_access_speed = true; + + for_each_online_cpu(cpu) { + int this_perf = per_cpu(misaligned_access_speed, cpu); + + if (this_perf != RISCV_HWPROBE_MISALIGNED_FAST) { + fast_misaligned_access_speed = false; + break; + } + } + + if (fast_misaligned_access_speed) + static_branch_enable(&fast_misaligned_access_speed_key); + + return 0; +} + +arch_initcall_sync(set_unaligned_access_static_branches); + #ifdef CONFIG_RISCV_ALTERNATIVE /* * Alternative patch sites consider 48 bits when determining when to patch