From patchwork Fri Oct 27 14:38:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 158966 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp652286vqb; Fri, 27 Oct 2023 07:39:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGat+ttpoxqodONLJfsR4fw6mzspw2Y/BrUfB+ueNfS7z1WOdz88SlfDxmniJ7JwRbvIOlu X-Received: by 2002:a5b:e88:0:b0:d9c:aa17:2ae3 with SMTP id z8-20020a5b0e88000000b00d9caa172ae3mr2707967ybr.64.1698417568041; Fri, 27 Oct 2023 07:39:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698417568; cv=none; d=google.com; s=arc-20160816; b=tD3Tjpc/lazGyok1rciHY9oFF97Qb1nY3KAbDPHen6shXQSMAQUZKfUa+oqRnawSBe OREKXPsL+5HVDu7aqsSKcf+9XJxLBEDhgLYf+jathn2yz0cwpTd1UOtOHAjyh+8NF8mr 394nr1M5dfb2uUn9tbMRlIs2XTislmAvQyjlggVf9cgEBI/sbU8NqhRie5UmETxf1oP6 Yp2ZTv1hVKXQREjcA21pTDHDOCc5r+ougzA8ScO7Lb1GWwKI+7XeIKPRg39fE5Ip7Anx I2wSkTJROBzI9ij3C3Y/U68zSwCsrK3X4qHeoO/x0vW0J0VeSDuBfBFWe5bsW/J3pEDh 89uQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=JXW6ncWciaghnfyTdoACNbmv3XQBwz+zUjQClT1iM5A=; fh=CwSpNuzJZ4SYflAY8X/5Th1WavevelKKrCkLrndW6AM=; b=BmMVRykCcY97NmZtOOK2Qg+ZgpqPkwUmDRBypREqL16yvLIcHW6hMddWhASSehEDWf f+pBGw05GFdmhZgpPlkXUqObBsgGc9RACirAkj3uZt6OUNjiEbWFU22gy6IRJA9XZaVf w1Up7cO5fo0PB34Vjf/XwhnYDaw4QyWlpA0+NYIHw1e13oVXcOm1lPS6plgiPLQ1cFFw Psyy4uF+7yULYT+94vtXWcwL8XutfxFV/uoCWra1JNSFAPUWFh3qj+jMrurhaIs1cMnf dWXz37XYkqb1h3dFaERZjCFrRLqMPnJPzw2XjTL3HdVJJlaS2yld+T+vxVBgSxN3+OCE olVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Cuxy/yew"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id p205-20020a2542d6000000b00da0c581666bsi2821722yba.524.2023.10.27.07.39.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Oct 2023 07:39:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Cuxy/yew"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 37803837B2E0; Fri, 27 Oct 2023 07:39:26 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346049AbjJ0OjM (ORCPT + 25 others); Fri, 27 Oct 2023 10:39:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345977AbjJ0OjC (ORCPT ); Fri, 27 Oct 2023 10:39:02 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3D89E3; Fri, 27 Oct 2023 07:38:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698417540; x=1729953540; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=KI9R1ykioS10T+zkw0Cw1ATpow7yvlFUWGh3i+Y0oPk=; b=Cuxy/yewMll0iq1rxeyEa6vjsqrhzGCpXCTYzXRhZhOD1CrjXYM+QQDo xWzOk8Q6zbTI4v7MBgRvrCOp16n4TiGOZ9TGkn4Qn/aKhzzrTGZ1iZ0vH hqY9zn2GVYvUEHAVDA837Q1mlr6wZb6ElKbF1q7+3uakaij2vNqYhgeJF WN2ZRtXcmwnuRLw0xJswtxGFKXMgCYWl4PauGNp/0IcAY22WuQUJgQuPM 5KXcKHVSlD8eKcbxm6xLADsgj9Pu6dWP6pcE41nigDwHYCfeW4sIlUfmU sILB/8SxCoaPzmEqgtIY1peQdQQXvxpPWJgWY1FJ3/5Wyh11hSOXt+qZR w==; X-IronPort-AV: E=McAfee;i="6600,9927,10876"; a="386670135" X-IronPort-AV: E=Sophos;i="6.03,256,1694761200"; d="scan'208";a="386670135" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2023 07:38:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.03,256,1694761200"; d="scan'208";a="812623" Received: from dmnassar-mobl.amr.corp.intel.com (HELO desk) ([10.212.203.39]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2023 07:38:12 -0700 Date: Fri, 27 Oct 2023 07:38:47 -0700 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com, Andrew Cooper , Nikolay Borisov Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com, Greg Kroah-Hartman , Pawan Gupta , Dave Hansen Subject: [PATCH v4 2/6] x86/entry_64: Add VERW just before userspace transition Message-ID: <20231027-delay-verw-v4-2-9a3622d4bcf7@linux.intel.com> X-Mailer: b4 0.12.3 References: <20231027-delay-verw-v4-0-9a3622d4bcf7@linux.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231027-delay-verw-v4-0-9a3622d4bcf7@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 27 Oct 2023 07:39:26 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780919899656758159 X-GMAIL-MSGID: 1780919899656758159 Mitigation for MDS is to use VERW instruction to clear any secrets in CPU Buffers. Any memory accesses after VERW execution can still remain in CPU buffers. It is safer to execute VERW late in return to user path to minimize the window in which kernel data can end up in CPU buffers. There are not many kernel secrets to be had after SWITCH_TO_USER_CR3. Add support for deploying VERW mitigation after user register state is restored. This helps minimize the chances of kernel data ending up into CPU buffers after executing VERW. Note that the mitigation at the new location is not yet enabled. Corner case not handled ======================= Interrupts returning to kernel don't clear CPUs buffers since the exit-to-user path is expected to do that anyways. But, there could be a case when an NMI is generated in kernel after the exit-to-user path has cleared the buffers. This case is not handled and NMI returning to kernel don't clear CPU buffers because: 1. It is rare to get an NMI after VERW, but before returning to userspace. 2. For an unprivileged user, there is no known way to make that NMI less rare or target it. 3. It would take a large number of these precisely-timed NMIs to mount an actual attack. There's presumably not enough bandwidth. 4. The NMI in question occurs after a VERW, i.e. when user state is restored and most interesting data is already scrubbed. Whats left is only the data that NMI touches, and that may or may not be of any interest. Suggested-by: Dave Hansen Signed-off-by: Pawan Gupta --- arch/x86/entry/entry_64.S | 11 +++++++++++ arch/x86/entry/entry_64_compat.S | 1 + 2 files changed, 12 insertions(+) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 43606de22511..9f97a8bd11e8 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -223,6 +223,7 @@ syscall_return_via_sysret: SYM_INNER_LABEL(entry_SYSRETQ_unsafe_stack, SYM_L_GLOBAL) ANNOTATE_NOENDBR swapgs + CLEAR_CPU_BUFFERS sysretq SYM_INNER_LABEL(entry_SYSRETQ_end, SYM_L_GLOBAL) ANNOTATE_NOENDBR @@ -663,6 +664,7 @@ SYM_INNER_LABEL(swapgs_restore_regs_and_return_to_usermode, SYM_L_GLOBAL) /* Restore RDI. */ popq %rdi swapgs + CLEAR_CPU_BUFFERS jmp .Lnative_iret @@ -774,6 +776,8 @@ native_irq_return_ldt: */ popq %rax /* Restore user RAX */ + CLEAR_CPU_BUFFERS + /* * RSP now points to an ordinary IRET frame, except that the page * is read-only and RSP[31:16] are preloaded with the userspace @@ -1502,6 +1506,12 @@ nmi_restore: std movq $0, 5*8(%rsp) /* clear "NMI executing" */ + /* + * Skip CLEAR_CPU_BUFFERS here, since it only helps in rare cases like + * NMI in kernel after user state is restored. For an unprivileged user + * these conditions are hard to meet. + */ + /* * iretq reads the "iret" frame and exits the NMI stack in a * single instruction. We are returning to kernel mode, so this @@ -1520,6 +1530,7 @@ SYM_CODE_START(ignore_sysret) UNWIND_HINT_END_OF_STACK ENDBR mov $-ENOSYS, %eax + CLEAR_CPU_BUFFERS sysretl SYM_CODE_END(ignore_sysret) #endif diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S index 70150298f8bd..245697eb8485 100644 --- a/arch/x86/entry/entry_64_compat.S +++ b/arch/x86/entry/entry_64_compat.S @@ -271,6 +271,7 @@ SYM_INNER_LABEL(entry_SYSRETL_compat_unsafe_stack, SYM_L_GLOBAL) xorl %r9d, %r9d xorl %r10d, %r10d swapgs + CLEAR_CPU_BUFFERS sysretl SYM_INNER_LABEL(entry_SYSRETL_compat_end, SYM_L_GLOBAL) ANNOTATE_NOENDBR