From patchwork Tue Jun 13 00:10:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 106977 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp222035vqr; Mon, 12 Jun 2023 17:44:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7JUVvKKnuvMkDf9It0oV67wpQQRSZ413W30R8n3O7RuPxe3oFOyjyATkZ5Ojx7ErV6w577 X-Received: by 2002:aa7:cd6d:0:b0:518:7232:e63a with SMTP id ca13-20020aa7cd6d000000b005187232e63amr697511edb.36.1686617051762; Mon, 12 Jun 2023 17:44:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686617051; cv=none; d=google.com; s=arc-20160816; b=JJKVR2x0XpdX5dPxqSG2gLH4F/lHyvGDyO8yhHJj6b4mX5xMHZCsbaFPKoKyuO3YdJ aHP/iaxicnipRsdlsV5twyX5ul36AlNh/4feg/QchvVwkf8WpcayRggkmn3ivuORVS0o FHG7FIt9TXxBwnyoLOs/T9I31TLGJzyeeynG1ZBPI33VzLxjfaPn0RyxcEleI/ubtCSw mb8FCgk15UpJiHaUDeSgz1DPITrx55qxGHzdU90XmQmE/+516mkyOOqUal6u96X6Ysyg XH6AaMUUqebi2O/t4ZXjfYayLR0iAif4o7Ord5RNW9ojFUE/kwNYUzHua+rDpwUhUu5D eTYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=F2RW2ABIPgY9zcBNGEtdjwLLlBLz/ha0Ddgc0k6oYH0=; b=TjUcd37hAu84qiE27/zuIi0+76ckVNHKQKWwSDyVhh7Aacoed+CM9+UFiyycuytD0Q bxerC2GT4LianSAscHvTUG19/t668qS8nfA9B8qvJ588ORvpjFMup29iuFYzlcUPq4CL KnJ8Q6q0VpnY9JJX7SlzAlxd5tDdQDjje3gySC88f+6HoRquJhsAfKwvZYG04jAQmPUK G3rHHysXQWTK3PxP2dXMTD05WHiHR5qZzJA+bw2Pln30d08gh4TOFaEP4oVQElutP2cQ hhukqmvVGASxENmTScD2EhIGq4UREZBwBlTtCD4HlbOYOXGOOrcHepvdWhxvgnSYJez0 T7QA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=m2hAgSCI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e4-20020a056402088400b0050489449e77si2706010edy.569.2023.06.12.17.43.48; Mon, 12 Jun 2023 17:44:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=m2hAgSCI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239374AbjFMAQc (ORCPT + 99 others); Mon, 12 Jun 2023 20:16:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238914AbjFMAPq (ORCPT ); Mon, 12 Jun 2023 20:15:46 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E03EC269D; Mon, 12 Jun 2023 17:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686615202; x=1718151202; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L2W4szzCW+qvvywxGcp6uHsWCFDTi4NqV9MxxSUxhGY=; b=m2hAgSCIvSKQe7x+o4sP0P0iMTZ8eLiENIOoIE8RAIkvuslNZ2qYuvTF +yfZfxnyqMLDiH4LSV3hezfYsQoTKxUx6OVJEpST/sv1yMeeZUSiGbz2L liCpQVq4v6XmO4n91rEpMHUONW3w2v2cBAJlmAzK3VGeUU/a6TL4wB5gb ug+NDKdeVH6SdRMYAyMthq34o7fifKHJDOZaF/lN1h3yBiDbNYIu+g4mV MoBhjFnDP3SimoTpcO8FCtbBqqcFiK4hR+I4jDd38dlxZKTPERcvx7Zz2 J6wII4D/38ew+5WTtadegZ8G7XCiuYOj7sByjMp2SlLrqP80BdLwTT8K+ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="361557243" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="361557243" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="835671072" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="835671072" Received: from almeisch-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.amr.corp.intel.com) ([10.209.42.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:28 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, broonie@kernel.org Cc: rick.p.edgecombe@intel.com, Pengfei Xu Subject: [PATCH v9 25/42] x86/fpu: Add helper for modifying xstate Date: Mon, 12 Jun 2023 17:10:51 -0700 Message-Id: <20230613001108.3040476-26-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> References: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768546161739402773?= X-GMAIL-MSGID: =?utf-8?q?1768546161739402773?= Just like user xfeatures, supervisor xfeatures can be active in the registers or present in the task FPU buffer. If the registers are active, the registers can be modified directly. If the registers are not active, the modification must be performed on the task FPU buffer. When the state is not active, the kernel could perform modifications directly to the buffer. But in order for it to do that, it needs to know where in the buffer the specific state it wants to modify is located. Doing this is not robust against optimizations that compact the FPU buffer, as each access would require computing where in the buffer it is. The easiest way to modify supervisor xfeature data is to force restore the registers and write directly to the MSRs. Often times this is just fine anyway as the registers need to be restored before returning to userspace. Do this for now, leaving buffer writing optimizations for the future. Add a new function fpregs_lock_and_load() that can simultaneously call fpregs_lock() and do this restore. Also perform some extra sanity checks in this function since this will be used in non-fpu focused code. Suggested-by: Thomas Gleixner Signed-off-by: Rick Edgecombe Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook --- arch/x86/include/asm/fpu/api.h | 9 +++++++++ arch/x86/kernel/fpu/core.c | 18 ++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 503a577814b2..aadc6893dcaa 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -82,6 +82,15 @@ static inline void fpregs_unlock(void) preempt_enable(); } +/* + * FPU state gets lazily restored before returning to userspace. So when in the + * kernel, the valid FPU state may be kept in the buffer. This function will force + * restore all the fpu state to the registers early if needed, and lock them from + * being automatically saved/restored. Then FPU state can be modified safely in the + * registers, before unlocking with fpregs_unlock(). + */ +void fpregs_lock_and_load(void); + #ifdef CONFIG_X86_DEBUG_FPU extern void fpregs_assert_state_consistent(void); #else diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index caf33486dc5e..f851558b673f 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -753,6 +753,24 @@ void switch_fpu_return(void) } EXPORT_SYMBOL_GPL(switch_fpu_return); +void fpregs_lock_and_load(void) +{ + /* + * fpregs_lock() only disables preemption (mostly). So modifying state + * in an interrupt could screw up some in progress fpregs operation. + * Warn about it. + */ + WARN_ON_ONCE(!irq_fpu_usable()); + WARN_ON_ONCE(current->flags & PF_KTHREAD); + + fpregs_lock(); + + fpregs_assert_state_consistent(); + + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + fpregs_restore_userregs(); +} + #ifdef CONFIG_X86_DEBUG_FPU /* * If current FPU state according to its tracking (loaded FPU context on this