Message ID | 20230319001535.23210-7-rick.p.edgecombe@intel.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp530328wrt; Sat, 18 Mar 2023 18:28:55 -0700 (PDT) X-Google-Smtp-Source: AK7set/KmK/xc+Y6PD2QEfpM5nVnbAO8KEZziDSkG1cOpR5KCv7cS5LgYNa4/v2/kXJUARtJXb+o X-Received: by 2002:a17:902:ec8b:b0:19e:b5d3:170f with SMTP id x11-20020a170902ec8b00b0019eb5d3170fmr15094715plg.4.1679189334727; Sat, 18 Mar 2023 18:28:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1679189334; cv=none; d=google.com; s=arc-20160816; b=Uh97wIuUQKZ6dvC8n0zctnS74yAQGSV47ahNnrIh1f7bLSHIxXJZ4om2lsl4KVp1VW 2f62mJSpmSk6hztVnOUiKjorsri2bh07CSABdoRjyhHbV71Zcpget50V0mFMzbcXSvPS eQVvJOdAs/ZPRTQ0qngg6OyqOSwBN6vBVyI2nEVZVMOh9XCJ9rEhMd/G0m+KIn0FmJC+ oWK38pV491+hobQx7bFJQvX3QVN3Wutq99lGWDJLDravcqvf9aOjLJHrMScqwQKn277M ZrwtkiSPoYvnhQ8K3U8uSs9l4udew7edr+LITaiMMBV4d1M5T0ojNz64GKvMJvQSf5gR jsEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=+zckbUWRWZvfPROGysXYXzeczc+/ZvxVdCvyMsXY99o=; b=pMAsW5sFJW86odQ0n9rrMz32+d/RMhjrcuMG23EbFY8zLW3Tj/g2ujqwp6S64LBp0B t3ngXVmC3WUIE4ybqtSyyTW9VTZOQ1ab+nQbUo+kDSXvB52E0nq5ISe3e/BaErc0hwRI MkCV0BPBjaw5OQNg7SkDadkt9MncPGcz26c8OFNu8Mtv+t6Z2m0dImAprqyC2Q4yKiZ8 TY4g/kwXUdNQtIMAtdSlkTJ8/hpI6G7dZQx/PjK7RpEsEWG56+rqrma0BdgQs909qXky hInJJc0gEjdA++GawB+1zWXGV5vXalRj4hDTnUFr9lXX+bfn4+ejis/7NTP6EQ6+DVWR EEJA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QPuaQR5V; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x6-20020a1709028ec600b0019a71e14c19si6531045plo.320.2023.03.18.18.28.42; Sat, 18 Mar 2023 18:28:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QPuaQR5V; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229911AbjCSAQX (ORCPT <rfc822;pusanteemu@gmail.com> + 99 others); Sat, 18 Mar 2023 20:16:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229774AbjCSAQG (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Sat, 18 Mar 2023 20:16:06 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C41528E74; Sat, 18 Mar 2023 17:16:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1679184965; x=1710720965; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=JJ3ES6pKzLeV3jf7I3FnSvx/TVTiB5ybDZUP+4xGp5E=; b=QPuaQR5Vi4d3Z/x3CyUxK59vRUKuToq9j9Djl3/AZNL5VFrnUhj3+wUS V+e8hk9uYSdvuoi7snnZcabq/H97SMjgC/94Oiyhn4OBrSpsOjSsWFMKD 0sx6AGCXVvti9Z3q2ksWZj5JnmwWuILMgcnTDdsGO6Y/xCr4bHZxJazG9 Si0JexYLGLmk68X+e2DuZPa7ZDHZe7y1nuq4P4aAdoZjLBy1S+uSgzatH cy0tvjF7m8G+QX3T6zROWHoVh+MfCc4WON3GU8QSrgZULpwTQRKx3AAo1 BFJUin+YZv+Z8HBnlSKGexLK6XJkxCKnAtpOWNI39avew12wcnf7Z4WZs A==; X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="338490848" X-IronPort-AV: E=Sophos;i="5.98,272,1673942400"; d="scan'208";a="338490848" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2023 17:16:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10653"; a="749672788" X-IronPort-AV: E=Sophos;i="5.98,272,1673942400"; d="scan'208";a="749672788" Received: from bmahatwo-mobl1.gar.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.135.34.5]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2023 17:16:02 -0700 From: Rick Edgecombe <rick.p.edgecombe@intel.com> To: x86@kernel.org, "H . Peter Anvin" <hpa@zytor.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann <arnd@arndb.de>, Andy Lutomirski <luto@kernel.org>, Balbir Singh <bsingharora@gmail.com>, Borislav Petkov <bp@alien8.de>, Cyrill Gorcunov <gorcunov@gmail.com>, Dave Hansen <dave.hansen@linux.intel.com>, Eugene Syromiatnikov <esyr@redhat.com>, Florian Weimer <fweimer@redhat.com>, "H . J . Lu" <hjl.tools@gmail.com>, Jann Horn <jannh@google.com>, Jonathan Corbet <corbet@lwn.net>, Kees Cook <keescook@chromium.org>, Mike Kravetz <mike.kravetz@oracle.com>, Nadav Amit <nadav.amit@gmail.com>, Oleg Nesterov <oleg@redhat.com>, Pavel Machek <pavel@ucw.cz>, Peter Zijlstra <peterz@infradead.org>, Randy Dunlap <rdunlap@infradead.org>, Weijiang Yang <weijiang.yang@intel.com>, "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>, John Allen <john.allen@amd.com>, kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v8 06/40] x86/fpu: Add helper for modifying xstate Date: Sat, 18 Mar 2023 17:15:01 -0700 Message-Id: <20230319001535.23210-7-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230319001535.23210-1-rick.p.edgecombe@intel.com> References: <20230319001535.23210-1-rick.p.edgecombe@intel.com> X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760757635982486618?= X-GMAIL-MSGID: =?utf-8?q?1760757635982486618?= |
Series |
Shadow stacks for userspace
|
|
Commit Message
Edgecombe, Rick P
March 19, 2023, 12:15 a.m. UTC
Just like user xfeatures, supervisor xfeatures can be active in the registers or present in the task FPU buffer. If the registers are active, the registers can be modified directly. If the registers are not active, the modification must be performed on the task FPU buffer. When the state is not active, the kernel could perform modifications directly to the buffer. But in order for it to do that, it needs to know where in the buffer the specific state it wants to modify is located. Doing this is not robust against optimizations that compact the FPU buffer, as each access would require computing where in the buffer it is. The easiest way to modify supervisor xfeature data is to force restore the registers and write directly to the MSRs. Often times this is just fine anyway as the registers need to be restored before returning to userspace. Do this for now, leaving buffer writing optimizations for the future. Add a new function fpregs_lock_and_load() that can simultaneously call fpregs_lock() and do this restore. Also perform some extra sanity checks in this function since this will be used in non-fpu focused code. Suggested-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Mike Rapoport (IBM) <rppt@kernel.org> Tested-by: Pengfei Xu <pengfei.xu@intel.com> Tested-by: John Allen <john.allen@amd.com> Tested-by: Kees Cook <keescook@chromium.org> --- v6: - Drop "but appear to work" (Boris) v5: - Fix spelling error (Boris) - Don't export fpregs_lock_and_load() (Boris) v3: - Rename to fpregs_lock_and_load() to match the unlocking fpregs_unlock(). (Kees) - Elaborate in comment about helper. (Dave) v2: - Drop optimization of writing directly the buffer, and change API accordingly. - fpregs_lock_and_load() suggested by tglx - Some commit log verbiage from dhansen --- arch/x86/include/asm/fpu/api.h | 9 +++++++++ arch/x86/kernel/fpu/core.c | 18 ++++++++++++++++++ 2 files changed, 27 insertions(+)
diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h index 503a577814b2..aadc6893dcaa 100644 --- a/arch/x86/include/asm/fpu/api.h +++ b/arch/x86/include/asm/fpu/api.h @@ -82,6 +82,15 @@ static inline void fpregs_unlock(void) preempt_enable(); } +/* + * FPU state gets lazily restored before returning to userspace. So when in the + * kernel, the valid FPU state may be kept in the buffer. This function will force + * restore all the fpu state to the registers early if needed, and lock them from + * being automatically saved/restored. Then FPU state can be modified safely in the + * registers, before unlocking with fpregs_unlock(). + */ +void fpregs_lock_and_load(void); + #ifdef CONFIG_X86_DEBUG_FPU extern void fpregs_assert_state_consistent(void); #else diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index caf33486dc5e..f851558b673f 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -753,6 +753,24 @@ void switch_fpu_return(void) } EXPORT_SYMBOL_GPL(switch_fpu_return); +void fpregs_lock_and_load(void) +{ + /* + * fpregs_lock() only disables preemption (mostly). So modifying state + * in an interrupt could screw up some in progress fpregs operation. + * Warn about it. + */ + WARN_ON_ONCE(!irq_fpu_usable()); + WARN_ON_ONCE(current->flags & PF_KTHREAD); + + fpregs_lock(); + + fpregs_assert_state_consistent(); + + if (test_thread_flag(TIF_NEED_FPU_LOAD)) + fpregs_restore_userregs(); +} + #ifdef CONFIG_X86_DEBUG_FPU /* * If current FPU state according to its tracking (loaded FPU context on this