From patchwork Sat Dec 3 00:35:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 29161 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1141893wrr; Fri, 2 Dec 2022 16:38:49 -0800 (PST) X-Google-Smtp-Source: AA0mqf7lDIjFOStp8XHoJWEO9h9qA/onxeIKGs9XaSldE4HSMxxDGD3djh/FZpHUsMKCWhzObZn8 X-Received: by 2002:a17:906:6dd5:b0:78d:a633:b55 with SMTP id j21-20020a1709066dd500b0078da6330b55mr64804560ejt.106.1670027929048; Fri, 02 Dec 2022 16:38:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670027929; cv=none; d=google.com; s=arc-20160816; b=ypsY/ZP7a6oZ4wJdfsONcRAQruVvdZ6MxTHpxgnlMApjMf0HsKcTMjPB0mOuWRM2zy w5rYevZTTj0z/8phRuSc6LXAVBo+N8tCI9m08XdybeIXT4qOx1xeRfrAyjZl017IQQEp M6gfvIXAZ07XuIHVFo8gzBbqtrJ6KIayoLv7qhsuNPxLI6KZ1eO52MOoABgUELphd0tB nCOalmVKRD94A/ZrqcZq203FNweBYwLocGGG4FTlSq3jgOLoi2RFppmUvl2RPFsQzvqD 9uOIximt9e3sU9l8vkLMRPO/tdpjkn0eN7iqbXCuH3IMtfbTLZAR1Pgvv5BQT30ujMhs bTPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=holp8FG1KuTwaCPNW2nBFLzGbV+C0253qOrO5vA4Zb4=; b=SW3o2O19i2OtoyjqD4Wp5VrRlazRMMwelBh6Qbc6/CQJsK3jRAsBEZ85dPdSPXn3bv iIQMqjtW9O0BIyVnDw+Av+x4fKYdOzW4T+yQLUzgdrvPkNJLoJsjfrOZv3B13QDeICPY rP4RRNZW0UfNPAwU+Otc4OffBD8Fyc0DiImt7ZZ4SV40jzNgcU3CaVqgg0JC9RUJJMZ5 XI0a46Sg4lhLQHNlC7NBbJEkMUjBlfPjHycehVzo4D98OvnbiblNP4N2/vwKCCUfnhKf 8JXUEGXgxav2FSyxnSNG6FGztEh2G4Vn2mb5x5lVQOTb7g2E/1OIoQuH/opQ2FKo41Jz d8oQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=OTcDoUA3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ji22-20020a170907981600b007c0a4c149bfsi6663427ejc.403.2022.12.02.16.38.26; Fri, 02 Dec 2022 16:38:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=OTcDoUA3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235015AbiLCAhS (ORCPT + 99 others); Fri, 2 Dec 2022 19:37:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234980AbiLCAgv (ORCPT ); Fri, 2 Dec 2022 19:36:51 -0500 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CED7AF8886; Fri, 2 Dec 2022 16:36:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1670027801; x=1701563801; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=oIjfcMVeiff8rS378Rozcj4vL/fH6nDkOB3vcy9Qe2A=; b=OTcDoUA3ZYsajgH2zoCxNoHUO3BCfHT43C5oTkx2cZwT2VScfvyBTOZe WC5SdIjAlFSHyR8maDYasa8p4X/NSsbhCPW1P92P98o9B3Ovoc+W0Jxu/ 1ULF5xMDEJlmpuYstLOPT6dRV5VbwZLergScR+3NYW2s57T6PGi8VCoI3 cMxnf0KaIjAGLoY/mF+zeCrhq2C4GJiiD0lI1EPMV4DyshQRUDo1qGREm JQLaRrjRok8eZWksRdV4nB7ZE7Qi74hK2QAw0gZ3aYQ1c0WPBSIjezMx/ aHOwOhlmlt2beXG0jeUKQERargoNdppU0Bu66nwvfWD9B6Tw9EQvaxCtX Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10549"; a="313710775" X-IronPort-AV: E=Sophos;i="5.96,213,1665471600"; d="scan'208";a="313710775" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2022 16:36:41 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10549"; a="787479781" X-IronPort-AV: E=Sophos;i="5.96,213,1665471600"; d="scan'208";a="787479781" Received: from bgordon1-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.212.211.211]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2022 16:36:39 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v4 06/39] x86/fpu: Add helper for modifying xstate Date: Fri, 2 Dec 2022 16:35:33 -0800 Message-Id: <20221203003606.6838-7-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221203003606.6838-1-rick.p.edgecombe@intel.com> References: <20221203003606.6838-1-rick.p.edgecombe@intel.com> X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751151205355114319?= X-GMAIL-MSGID: =?utf-8?q?1751151205355114319?= Just like user xfeatures, supervisor xfeatures can be active in the registers or present in the task FPU buffer. If the registers are active, the registers can be modified directly. If the registers are not active, the modification must be performed on the task FPU buffer. When the state is not active, the kernel could perform modifications directly to the buffer. But in order for it to do that, it needs to know where in the buffer the specific state it wants to modify is located. Doing this is not robust against optimizations that compact the FPU buffer, as each access would require computing where in the buffer it is. The easiest way to modify supervisor xfeature data is to force restore the registers and write directly to the MSRs. Often times this is just fine anyway as the registers need to be restored before returning to userspace. Do this for now, leaving buffer writing optimizations for the future. Add a new function fpregs_lock_and_load() that can simultaneously call fpregs_lock() and do this restore. Also perform some extra sanity checks in this function since this will be used in non-fpu focused code. Tested-by: Pengfei Xu Tested-by: John Allen Suggested-by: Thomas Gleixner Signed-off-by: Rick Edgecombe Reviewed-by: Kees Cook --- v3: - Rename to fpregs_lock_and_load() to match the unlocking fpregs_unlock(). (Kees) - Elaborate in comment about helper. (Dave) v2: - Drop optimization of writing directly the buffer, and change API accordingly. - fpregs_lock_and_load() suggested by tglx - Some commit log verbiage from dhansen v1: - New patch. arch/x86/include/asm/fpu/api.h | 9 +++++++++ arch/x86/kernel/fpu/core.c | 19 +++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 503a577814b2..aadc6893dcaa 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -82,6 +82,15 @@ static inline void fpregs_unlock(void) preempt_enable(); } +/* + * FPU state gets lazily restored before returning to userspace. So when in the + * kernel, the valid FPU state may be kept in the buffer. This function will force + * restore all the fpu state to the registers early if needed, and lock them from + * being automatically saved/restored. Then FPU state can be modified safely in the + * registers, before unlocking with fpregs_unlock(). + */ +void fpregs_lock_and_load(void); + #ifdef CONFIG_X86_DEBUG_FPU extern void fpregs_assert_state_consistent(void); #else diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 9baa89a8877d..9af78e9d92a0 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -753,6 +753,25 @@ void switch_fpu_return(void) } EXPORT_SYMBOL_GPL(switch_fpu_return); +void fpregs_lock_and_load(void) +{ + /* + * fpregs_lock() only disables preemption (mostly). So modifing state + * in an interrupt could screw up some in progress fpregs operation, + * but appear to work. Warn about it. + */ + WARN_ON_ONCE(!irq_fpu_usable()); + WARN_ON_ONCE(current->flags & PF_KTHREAD); + + fpregs_lock(); + + fpregs_assert_state_consistent(); + + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + fpregs_restore_userregs(); +} +EXPORT_SYMBOL_GPL(fpregs_lock_and_load); + #ifdef CONFIG_X86_DEBUG_FPU /* * If current FPU state according to its tracking (loaded FPU context on this