Message ID | 20230306095934.609589-1-chenhuacai@loongson.cn |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1760862wrd; Mon, 6 Mar 2023 02:25:59 -0800 (PST) X-Google-Smtp-Source: AK7set/+gKLyZQqq1bf9HoXDl+R7KdcYsIOa95VN8Bpirs947UQrB4vw6LsBTMF5wby6yXQ9SIMu X-Received: by 2002:a17:907:608b:b0:873:1b57:b27f with SMTP id ht11-20020a170907608b00b008731b57b27fmr13407696ejc.61.1678098358950; Mon, 06 Mar 2023 02:25:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678098358; cv=none; d=google.com; s=arc-20160816; b=KNCsuEJmtarengX4uDez2qdfnkzj5OSDKEl1Au++sQq2MRdos/sobpgQz1PXHxZW+x B6zQ2n0IoSo2bfVwdB/kOAzK/avYbPqxHgZT9YvrMTxWQ+ZvjDGDxGsWuZiSFk5qeglu IXceFGvCwncM+rfrV53B5hTQsmyaobz6vWvIhNkZ50LmptZSXGo6sx5d8xAndKcJaR3W o2eZXySz/tb625D7yYPMiwHpsldFCM1VyIpTAGFMgghz9AEUw33gEN7Am9iYDs94MB5+ 6eMgRn2Z7R+6PHmhEcmxbovhsOig3X6RekaHHYJThc5Oq0n4jSEZSRDGYD8tHa1r8Lwp 88JQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=ndx6WHcRcM6Ev2nhqCa6QNHetPBUu97wGS/I6gAOc3c=; b=RmTIJYJkqh6yYNFoQgDDegzbaP4KALBgUJ1wNlxW1g0YXPQWglLTimmmuJhAQeNj7M AUbhwH8qexNj+JGhCEQYGNencYhuxso5MUKSFD+uO4eAHhfpbwjMtrAgIvLeoudgkOKv TIzRz1vLBeXw7kajZR5b3jvqyzD5kUjxISwYo1d4dZTQVxEp3glsMjptWY8DwSGpITcK Q46V1KoDd4qsMvoosx2zg2ZyEtLU4iTKMy//FcX/fdtWmhn9aIZg4WlGWOhGP6rpJL5a kxIHYPAjSj+kMi0tvOLT1atM/C9tt6/HI8ODHsRt7uIDbeOv9jg+dw99nGeMH0whhUug reeg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f24-20020a056402069800b004acd2bb0015si3101774edy.346.2023.03.06.02.25.19; Mon, 06 Mar 2023 02:25:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230173AbjCFKAI (ORCPT <rfc822;toshivichauhan@gmail.com> + 99 others); Mon, 6 Mar 2023 05:00:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230206AbjCFJ7y (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 6 Mar 2023 04:59:54 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 224441C593; Mon, 6 Mar 2023 01:59:46 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 84900B80D76; Mon, 6 Mar 2023 09:59:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 91C72C433D2; Mon, 6 Mar 2023 09:59:41 +0000 (UTC) From: Huacai Chen <chenhuacai@loongson.cn> To: Arnd Bergmann <arnd@arndb.de>, Huacai Chen <chenhuacai@kernel.org> Cc: loongarch@lists.linux.dev, linux-arch@vger.kernel.org, Xuefeng Li <lixuefeng@loongson.cn>, Guo Ren <guoren@kernel.org>, Xuerui Wang <kernel@xen0n.name>, Jiaxun Yang <jiaxun.yang@flygoat.com>, linux-kernel@vger.kernel.org, loongson-kernel@lists.loongnix.cn, Huacai Chen <chenhuacai@loongson.cn> Subject: [PATCH V3] LoongArch: Provide kernel fpu functions Date: Mon, 6 Mar 2023 17:59:34 +0800 Message-Id: <20230306095934.609589-1-chenhuacai@loongson.cn> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759613664674442509?= X-GMAIL-MSGID: =?utf-8?q?1759613664674442509?= |
Series |
[V3] LoongArch: Provide kernel fpu functions
|
|
Commit Message
Huacai Chen
March 6, 2023, 9:59 a.m. UTC
Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself to use fpu. They can be used by some other kernel components, e.g., the AMDGPU graphic driver for DCN. Reported-by: WANG Xuerui <kernel@xen0n.name> Tested-by: WANG Xuerui <kernel@xen0n.name> Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> --- V2: Use non-GPL exports and update commit messages. V3: Add spaces for coding style. arch/loongarch/include/asm/fpu.h | 3 +++ arch/loongarch/kernel/Makefile | 2 +- arch/loongarch/kernel/kfpu.c | 41 ++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 arch/loongarch/kernel/kfpu.c
Comments
在 2023/3/6 17:59, Huacai Chen 写道: > Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself > to use fpu. They can be used by some other kernel components, e.g., the > AMDGPU graphic driver for DCN. Since kernel is compiled with -msoft-float, I guess hw fpu will not be used in kernel by present:). However it is deserved to try. > > Reported-by: WANG Xuerui <kernel@xen0n.name> > Tested-by: WANG Xuerui <kernel@xen0n.name> > Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> > --- > V2: Use non-GPL exports and update commit messages. > V3: Add spaces for coding style. > > arch/loongarch/include/asm/fpu.h | 3 +++ > arch/loongarch/kernel/Makefile | 2 +- > arch/loongarch/kernel/kfpu.c | 41 ++++++++++++++++++++++++++++++++ > 3 files changed, 45 insertions(+), 1 deletion(-) > create mode 100644 arch/loongarch/kernel/kfpu.c > > diff --git a/arch/loongarch/include/asm/fpu.h b/arch/loongarch/include/asm/fpu.h > index 358b254d9c1d..192f8e35d912 100644 > --- a/arch/loongarch/include/asm/fpu.h > +++ b/arch/loongarch/include/asm/fpu.h > @@ -21,6 +21,9 @@ > > struct sigcontext; > > +extern void kernel_fpu_begin(void); > +extern void kernel_fpu_end(void); > + > extern void _init_fpu(unsigned int); > extern void _save_fp(struct loongarch_fpu *); > extern void _restore_fp(struct loongarch_fpu *); > diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile > index 78d4e3384305..9a72d91cd104 100644 > --- a/arch/loongarch/kernel/Makefile > +++ b/arch/loongarch/kernel/Makefile > @@ -13,7 +13,7 @@ obj-y += head.o cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \ > obj-$(CONFIG_ACPI) += acpi.o > obj-$(CONFIG_EFI) += efi.o > > -obj-$(CONFIG_CPU_HAS_FPU) += fpu.o > +obj-$(CONFIG_CPU_HAS_FPU) += fpu.o kfpu.o > > obj-$(CONFIG_ARCH_STRICT_ALIGN) += unaligned.o > > diff --git a/arch/loongarch/kernel/kfpu.c b/arch/loongarch/kernel/kfpu.c > new file mode 100644 > index 000000000000..cd2a18fecdcc > --- /dev/null > +++ b/arch/loongarch/kernel/kfpu.c > @@ -0,0 +1,41 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (C) 2023 Loongson Technology Corporation Limited > + */ > + > +#include <linux/cpu.h> > +#include <linux/init.h> > +#include <asm/fpu.h> > +#include <asm/smp.h> > + > +static DEFINE_PER_CPU(bool, in_kernel_fpu); > + > +void kernel_fpu_begin(void) > +{ > + if (this_cpu_read(in_kernel_fpu)) > + return; > + > + preempt_disable(); > + this_cpu_write(in_kernel_fpu, true); > + > + if (!is_fpu_owner()) > + enable_fpu(); > + else > + _save_fp(¤t->thread.fpu); Do we need initialize fcsr rather than using random fcsr value of other processes? There may be fpu exception enabled by other tasks. Regards Bibo,mao > +} > +EXPORT_SYMBOL(kernel_fpu_begin); > + > +void kernel_fpu_end(void) > +{ > + if (!this_cpu_read(in_kernel_fpu)) > + return; > + > + if (!is_fpu_owner()) > + disable_fpu(); > + else > + _restore_fp(¤t->thread.fpu); > + > + this_cpu_write(in_kernel_fpu, false); > + preempt_enable(); > +} > +EXPORT_SYMBOL(kernel_fpu_end);
On Mon, 2023-03-06 at 20:03 +0800, maobibo wrote: > 在 2023/3/6 17:59, Huacai Chen 写道: > > Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself > > to use fpu. They can be used by some other kernel components, e.g., the > > AMDGPU graphic driver for DCN. > Since kernel is compiled with -msoft-float, I guess hw fpu will not be > used in kernel by present:). However it is deserved to try. See the draft AMD DCN support patch: https://github.com/loongson/linux/commit/0ee299095c963938a7626c3121a8feef32251301
在 2023/3/6 20:09, Xi Ruoyao 写道: > On Mon, 2023-03-06 at 20:03 +0800, maobibo wrote: >> 在 2023/3/6 17:59, Huacai Chen 写道: >>> Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself >>> to use fpu. They can be used by some other kernel components, e.g., the >>> AMDGPU graphic driver for DCN. >> Since kernel is compiled with -msoft-float, I guess hw fpu will not be >> used in kernel by present:). However it is deserved to try. > > See the draft AMD DCN support patch: > https://github.com/loongson/linux/commit/0ee299095c963938a7626c3121a8feef32251301 Got it, thanks for pointing it out, good job:) Regards Bibo,Mao >
Hi, On 2023/3/6 20:03, maobibo wrote: > > > 在 2023/3/6 17:59, Huacai Chen 写道: >> Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself >> to use fpu. They can be used by some other kernel components, e.g., the >> AMDGPU graphic driver for DCN. > Since kernel is compiled with -msoft-float, I guess hw fpu will not be > used in kernel by present:). However it is deserved to try. This has been explained by Ruoyao, but we'd need such support anyway when we have LSX/LASX support mainlined in the future and want to accelerate various algorithms with those instructions. Maybe at that time you'll want to port some of those too ;-)
On Mon, Mar 6, 2023 at 8:03 PM maobibo <maobibo@loongson.cn> wrote: > > > > 在 2023/3/6 17:59, Huacai Chen 写道: > > Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself > > to use fpu. They can be used by some other kernel components, e.g., the > > AMDGPU graphic driver for DCN. > Since kernel is compiled with -msoft-float, I guess hw fpu will not be > used in kernel by present:). However it is deserved to try. > > > > Reported-by: WANG Xuerui <kernel@xen0n.name> > > Tested-by: WANG Xuerui <kernel@xen0n.name> > > Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> > > --- > > V2: Use non-GPL exports and update commit messages. > > V3: Add spaces for coding style. > > > > arch/loongarch/include/asm/fpu.h | 3 +++ > > arch/loongarch/kernel/Makefile | 2 +- > > arch/loongarch/kernel/kfpu.c | 41 ++++++++++++++++++++++++++++++++ > > 3 files changed, 45 insertions(+), 1 deletion(-) > > create mode 100644 arch/loongarch/kernel/kfpu.c > > > > diff --git a/arch/loongarch/include/asm/fpu.h b/arch/loongarch/include/asm/fpu.h > > index 358b254d9c1d..192f8e35d912 100644 > > --- a/arch/loongarch/include/asm/fpu.h > > +++ b/arch/loongarch/include/asm/fpu.h > > @@ -21,6 +21,9 @@ > > > > struct sigcontext; > > > > +extern void kernel_fpu_begin(void); > > +extern void kernel_fpu_end(void); > > + > > extern void _init_fpu(unsigned int); > > extern void _save_fp(struct loongarch_fpu *); > > extern void _restore_fp(struct loongarch_fpu *); > > diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile > > index 78d4e3384305..9a72d91cd104 100644 > > --- a/arch/loongarch/kernel/Makefile > > +++ b/arch/loongarch/kernel/Makefile > > @@ -13,7 +13,7 @@ obj-y += head.o cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \ > > obj-$(CONFIG_ACPI) += acpi.o > > obj-$(CONFIG_EFI) += efi.o > > > > -obj-$(CONFIG_CPU_HAS_FPU) += fpu.o > > +obj-$(CONFIG_CPU_HAS_FPU) += fpu.o kfpu.o > > > > obj-$(CONFIG_ARCH_STRICT_ALIGN) += unaligned.o > > > > diff --git a/arch/loongarch/kernel/kfpu.c b/arch/loongarch/kernel/kfpu.c > > new file mode 100644 > > index 000000000000..cd2a18fecdcc > > --- /dev/null > > +++ b/arch/loongarch/kernel/kfpu.c > > @@ -0,0 +1,41 @@ > > +// SPDX-License-Identifier: GPL-2.0-only > > +/* > > + * Copyright (C) 2023 Loongson Technology Corporation Limited > > + */ > > + > > +#include <linux/cpu.h> > > +#include <linux/init.h> > > +#include <asm/fpu.h> > > +#include <asm/smp.h> > > + > > +static DEFINE_PER_CPU(bool, in_kernel_fpu); > > + > > +void kernel_fpu_begin(void) > > +{ > > + if (this_cpu_read(in_kernel_fpu)) > > + return; > > + > > + preempt_disable(); > > + this_cpu_write(in_kernel_fpu, true); > > + > > + if (!is_fpu_owner()) > > + enable_fpu(); > > + else > > + _save_fp(¤t->thread.fpu); > Do we need initialize fcsr rather than using random fcsr value > of other processes? There may be fpu exception enabled by > other tasks. Emm, I think initialize fcsr to 0 is better here. Huacai > > Regards > Bibo,mao > > +} > > +EXPORT_SYMBOL(kernel_fpu_begin); > > + > > +void kernel_fpu_end(void) > > +{ > > + if (!this_cpu_read(in_kernel_fpu)) > > + return; > > + > > + if (!is_fpu_owner()) > > + disable_fpu(); > > + else > > + _restore_fp(¤t->thread.fpu); > > + > > + this_cpu_write(in_kernel_fpu, false); > > + preempt_enable(); > > +} > > +EXPORT_SYMBOL(kernel_fpu_end); > >
On Mon, 2023-03-06 at 20:49 +0800, Huacai Chen wrote: > > > + if (!is_fpu_owner()) > > > + enable_fpu(); > > > + else > > > + _save_fp(¤t->thread.fpu); > > Do we need initialize fcsr rather than using random fcsr value > > of other processes? There may be fpu exception enabled by > > other tasks. > Emm, I think initialize fcsr to 0 is better here. I guess it's necessary: if we use a "dirty" FSCR0 with some exceptions enabled (esp. inaccurate exception which is expected as disabled by most developers), we may end up oops with a kernel FPE...
From: Huacai Chen > Sent: 06 March 2023 10:00 > > Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself > to use fpu. They can be used by some other kernel components, e.g., the > AMDGPU graphic driver for DCN. > ... > +void kernel_fpu_end(void) > +{ > + if (!this_cpu_read(in_kernel_fpu)) > + return; Shouldn't it be better for that to be a counter? Not sure what anyone else does. > + > + if (!is_fpu_owner()) > + disable_fpu(); > + else > + _restore_fp(¤t->thread.fpu); Does that actual do the restore? You really don't need to do it until 'return to user'. That will speed up a subsequent kernel_fpu_begin(). If fact, couldn't kernel_fpu_end() just be preemt_enable()? David > + > + this_cpu_write(in_kernel_fpu, false); > + preempt_enable(); > +} > +EXPORT_SYMBOL(kernel_fpu_end); > -- > 2.39.1 - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
From: Huacai Chen > Sent: 06 March 2023 10:00 > > Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself > to use fpu. They can be used by some other kernel components, e.g., the > AMDGPU graphic driver for DCN. > ... > diff --git a/arch/loongarch/kernel/kfpu.c b/arch/loongarch/kernel/kfpu.c > new file mode 100644 > index 000000000000..cd2a18fecdcc > --- /dev/null > +++ b/arch/loongarch/kernel/kfpu.c > @@ -0,0 +1,41 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (C) 2023 Loongson Technology Corporation Limited > + */ > + > +#include <linux/cpu.h> > +#include <linux/init.h> > +#include <asm/fpu.h> > +#include <asm/smp.h> > + > +static DEFINE_PER_CPU(bool, in_kernel_fpu); > + > +void kernel_fpu_begin(void) > +{ > + if (this_cpu_read(in_kernel_fpu)) > + return; Isn't this check entirely broken? It absolutely needs to be inside the preempt_disable(). If there are nested requests then fpu use is disabled by the first kernel_fpu_end() call. > + > + preempt_disable(); > + this_cpu_write(in_kernel_fpu, true); > + > + if (!is_fpu_owner()) > + enable_fpu(); > + else > + _save_fp(¤t->thread.fpu); > +} More interestingly, unless the kernel is doing the kind of 'lazy fpu switch' that x86 used to do (not sure it still does in Linux) where the fpu registers can contain values for a different process isn't it actually enough for kernel_fpu_begin() to just be: preempt_disable(); if (current->fpu_regs_live) __save_fp(current); preempt_enable(); and for kernel_fpu_end() to basically be a nop. Then rely on the 'return to user' path to pick up the live fpu registers from the save area. After all, you pretty much don't want to load the fpu regs every time a process wakes up and goes back to sleep without returning to user. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Hi, David, On Tue, Mar 7, 2023 at 1:17 AM David Laight <David.Laight@aculab.com> wrote: > > From: Huacai Chen > > Sent: 06 March 2023 10:00 > > > > Provide kernel_fpu_begin()/kernel_fpu_end() to allow the kernel itself > > to use fpu. They can be used by some other kernel components, e.g., the > > AMDGPU graphic driver for DCN. > > > ... > > diff --git a/arch/loongarch/kernel/kfpu.c b/arch/loongarch/kernel/kfpu.c > > new file mode 100644 > > index 000000000000..cd2a18fecdcc > > --- /dev/null > > +++ b/arch/loongarch/kernel/kfpu.c > > @@ -0,0 +1,41 @@ > > +// SPDX-License-Identifier: GPL-2.0-only > > +/* > > + * Copyright (C) 2023 Loongson Technology Corporation Limited > > + */ > > + > > +#include <linux/cpu.h> > > +#include <linux/init.h> > > +#include <asm/fpu.h> > > +#include <asm/smp.h> > > + > > +static DEFINE_PER_CPU(bool, in_kernel_fpu); > > + > > +void kernel_fpu_begin(void) > > +{ > > + if (this_cpu_read(in_kernel_fpu)) > > + return; > > Isn't this check entirely broken? > It absolutely needs to be inside the preempt_disable(). Yes, you are right, this check should be after preempt_disable(). > If there are nested requests then fpu use is disabled by the first > kernel_fpu_end() call. This check should be changed to WARN_ON() as x86 does since nested requests are unexpected use cases. > > > + > > + preempt_disable(); > > + this_cpu_write(in_kernel_fpu, true); > > + > > + if (!is_fpu_owner()) > > + enable_fpu(); > > + else > > + _save_fp(¤t->thread.fpu); > > +} > > More interestingly, unless the kernel is doing the kind of > 'lazy fpu switch' that x86 used to do (not sure it still does in Linux) > where the fpu registers can contain values for a different process > isn't it actually enough for kernel_fpu_begin() to just be: > > preempt_disable(); > if (current->fpu_regs_live) I think this condition is the same as is_fpu_owner(). Moreover, LoongArch doesn't allow fpu-disabled exception occured in kernel, so we should make sure fpu is enabled at kernel_fpu_begin(). > __save_fp(current); > preempt_enable(); > > and for kernel_fpu_end() to basically be a nop. > > Then rely on the 'return to user' path to pick up the > live fpu registers from the save area. > > After all, you pretty much don't want to load the fpu regs > every time a process wakes up and goes back to sleep without > returning to user. I think this is not a common case, so it isn't worthy to modify many files to optimize for now. Huacai > > David > > - > Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK > Registration No: 1397386 (Wales) >
diff --git a/arch/loongarch/include/asm/fpu.h b/arch/loongarch/include/asm/fpu.h index 358b254d9c1d..192f8e35d912 100644 --- a/arch/loongarch/include/asm/fpu.h +++ b/arch/loongarch/include/asm/fpu.h @@ -21,6 +21,9 @@ struct sigcontext; +extern void kernel_fpu_begin(void); +extern void kernel_fpu_end(void); + extern void _init_fpu(unsigned int); extern void _save_fp(struct loongarch_fpu *); extern void _restore_fp(struct loongarch_fpu *); diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile index 78d4e3384305..9a72d91cd104 100644 --- a/arch/loongarch/kernel/Makefile +++ b/arch/loongarch/kernel/Makefile @@ -13,7 +13,7 @@ obj-y += head.o cpu-probe.o cacheinfo.o env.o setup.o entry.o genex.o \ obj-$(CONFIG_ACPI) += acpi.o obj-$(CONFIG_EFI) += efi.o -obj-$(CONFIG_CPU_HAS_FPU) += fpu.o +obj-$(CONFIG_CPU_HAS_FPU) += fpu.o kfpu.o obj-$(CONFIG_ARCH_STRICT_ALIGN) += unaligned.o diff --git a/arch/loongarch/kernel/kfpu.c b/arch/loongarch/kernel/kfpu.c new file mode 100644 index 000000000000..cd2a18fecdcc --- /dev/null +++ b/arch/loongarch/kernel/kfpu.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 Loongson Technology Corporation Limited + */ + +#include <linux/cpu.h> +#include <linux/init.h> +#include <asm/fpu.h> +#include <asm/smp.h> + +static DEFINE_PER_CPU(bool, in_kernel_fpu); + +void kernel_fpu_begin(void) +{ + if (this_cpu_read(in_kernel_fpu)) + return; + + preempt_disable(); + this_cpu_write(in_kernel_fpu, true); + + if (!is_fpu_owner()) + enable_fpu(); + else + _save_fp(¤t->thread.fpu); +} +EXPORT_SYMBOL(kernel_fpu_begin); + +void kernel_fpu_end(void) +{ + if (!this_cpu_read(in_kernel_fpu)) + return; + + if (!is_fpu_owner()) + disable_fpu(); + else + _restore_fp(¤t->thread.fpu); + + this_cpu_write(in_kernel_fpu, false); + preempt_enable(); +} +EXPORT_SYMBOL(kernel_fpu_end);