From patchwork Tue Oct 24 08:08:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 157281 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce89:0:b0:403:3b70:6f57 with SMTP id p9csp1788634vqx; Tue, 24 Oct 2023 01:09:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHOBXN3guUbQfS5Q0DO5dhisPsGE+nteYh/tZNXunFvvUzQsMdIXS/9OF4w4XaqxP5NxlS7 X-Received: by 2002:a05:6a20:1606:b0:14d:d9f8:83f8 with SMTP id l6-20020a056a20160600b0014dd9f883f8mr2289919pzj.1.1698134994469; Tue, 24 Oct 2023 01:09:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698134994; cv=none; d=google.com; s=arc-20160816; b=tFObeVOH5aCOm+FkKKn82KfQvb4/rnrMhPrMtTWiqhV88j3B90jTlgAnrvUKij8zMN 8Yf8+ACkA/iweLAukpHCRrN6eN2DhtrMQ2phIFG7uRC5DFPIcDbIotlFestnSUl+AZMX rvk24Ioon+RtfogistEOwk3fr6wPEOOGpFakopdIDL48tM0pZ8VOJcV/pS6fjB3QneM4 RQh8Sk1I1bZ+tlhVhsl3XXhnDDQFXz+lOe7TE/urzzT1vic38FvWjPlwWpKvLsVRhWR7 CD81dQWfmuDmuJfWhnhxjwKhZLgUG5Blwl6rJHzPNkBZrOtHZj21qhmvXfIZ+vMYC1jz 7VJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=kqZ9boRD/BEfpEh7BxFSfdv7uSvpYLo6kClECWofZX8=; fh=lbJsPO+ZRDN9baDq/BZnnNH6CGOpZtNT0YCi3hapTHQ=; b=KUgODfIjf+xf6X54qSI8M1FLXoyH0MZjqsUwX3sNDbmbfy7AG+HVKb72a4LMUzBqha FeL8XIs6yTeSq/tjKFWW2k1TozZk1ky8TRgjHWr81D6jBsRwI2lAyCrimQs4oO/a5tVE 72xucI77piTnp18dmaXSoSNWBpfiJJiXiRxT+GSsQpkYcNIpCNDMUlFfJITXK6GFcnIv v1Zzyv61rFxFDxaRBKTZe30UwuXgSWAfMwWOTyLdkxf9NqRaxxIgK8HTzB4kRN36J7R7 UIWLuisz8geALH4qBjxfyrHhc6628w8SMXrNCAYbZsNKRC7rHIjHAhf2GXKdZ+BVYdiw BT6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JaNgVV+m; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id d5-20020a170902c18500b001b3bd85f54bsi7751178pld.35.2023.10.24.01.09.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 01:09:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JaNgVV+m; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id ECC3A80B87CC; Tue, 24 Oct 2023 01:09:51 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233848AbjJXIJh (ORCPT + 26 others); Tue, 24 Oct 2023 04:09:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233856AbjJXIJ2 (ORCPT ); Tue, 24 Oct 2023 04:09:28 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BC6E10C8; Tue, 24 Oct 2023 01:09:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698134959; x=1729670959; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=X1LhqsYhA19G67EeSSUcnUhNcMpaMfsi2uhvRqZyFK8=; b=JaNgVV+mVS3E/gC9V9jnPUj72Awj35a3QsPZvmA81cRdutuIBa/QtKwx Lxp+dwYCttDSP+9ru8iyqrcdudezssSvB2D6lRSxWmDpDgb5Oaw77ktRK 9uK5GBjkhztW0+jmSlThDFcfIf1zVbWNzRlDsv8jEuM9c20G3DE1cxHlm CFB9SzRogl4X71ocvqQcjLQTMwr+RxhfCFuTwWWVvKqFoDRGhjXN4oza5 IQkgmRxz7p1RkxL6Bqvj3qK/rderWcKQeuEJm0oNtuZS3vK+hGhBZTx/v DYn/Dhr6cc4cmYI6jdIi7PuWChoq80RcZDMSd+RJNCelQOvFVXQ3D9lYQ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10872"; a="366353032" X-IronPort-AV: E=Sophos;i="6.03,247,1694761200"; d="scan'208";a="366353032" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Oct 2023 01:08:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10872"; a="787708849" X-IronPort-AV: E=Sophos;i="6.03,247,1694761200"; d="scan'208";a="787708849" Received: from zijianw1-mobl.amr.corp.intel.com (HELO desk) ([10.209.109.187]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Oct 2023 01:08:54 -0700 Date: Tue, 24 Oct 2023 01:08:53 -0700 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com, Pawan Gupta Subject: [PATCH v2 6/6] KVM: VMX: Move VERW closer to VMentry for MDS mitigation Message-ID: <20231024-delay-verw-v2-6-f1881340c807@linux.intel.com> X-Mailer: b4 0.12.3 References: <20231024-delay-verw-v2-0-f1881340c807@linux.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231024-delay-verw-v2-0-f1881340c807@linux.intel.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 24 Oct 2023 01:09:52 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780623600166082569 X-GMAIL-MSGID: 1780623600166082569 During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Signed-off-by: Pawan Gupta --- arch/x86/kvm/vmx/vmenter.S | 4 ++++ arch/x86/kvm/vmx/vmx.c | 10 +++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index b3b13ec04bac..c566035938cc 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include #include +#include #include #include #include @@ -161,6 +162,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 24e8694b83fc..e2234c0643e9 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7226,13 +7226,17 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed inspite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) + /* MMIO mitigation is mutually exclusive to MDS mitigation later in asm */ mds_clear_cpu_buffers(); vmx_disable_fb_clear(vmx);