From patchwork Thu Jan 11 08:56:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 187254 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp1316112dyi; Thu, 11 Jan 2024 01:01:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IHAHhHYW7gm4oFWXG/N3zD7azNwQA9eYF9hxy738rcqMvoFPTY7nI9BnxlhVwU5XghDpdtn X-Received: by 2002:a05:6a20:271e:b0:19a:260b:ec26 with SMTP id u30-20020a056a20271e00b0019a260bec26mr581045pze.61.1704963694174; Thu, 11 Jan 2024 01:01:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704963694; cv=none; d=google.com; s=arc-20160816; b=PxBSuDvJw1kMqgCLKiI0eTRW/uln22Z+n3SOr+yGxPlC/FWMMNA/FInklyBnTYECab jWlQx+JVYvD7vNI0S78JtoQa5zAlL/nXxxlFxO9plmn6wveOXs14Y7YIsJy7zx0gS509 R5pPrJeL8LOaC37ukRPYeoZC+neowdxqnpgeoy6XClB2UizS6qYnnaAozm+2ftR39McN CGv+gubiVG9cGAuFyyvB6C5v9N+7Q4IhHWibkAhd0G/E0jKLIqW9NwGhlC4MFJOjDyvV aztylvIQi9ijDFbiFfy01A9X837v83vHAJuoLbgjVp5Vn6ssRlhbjc/4AbT/dzk61dqE D4Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature; bh=6gkq0m3ZW4hQGEKSGvfJPClu+To7TEynIn/wij5+DxA=; fh=ER2H6g+88FnZd3JyhT6ewHH56pdK2QWfWr77z0IHoNE=; b=Dvx2Xtb39E6GeXPgm89mxfQyCMIv42ZXb01Ag3ekZPqSeThKN37GWByTYQQpmHusHs z0ylk62pg/CxM1gZxIBwKtGqgiD5bUn+0K/ycGtJ58Mogt2FOFAGM4nqeQae26THqiBm F+qMd97kHw8VQXPdsb6skVZj8QrOE/VHFtkTSrc1qp/grJUuf1eahzXHY8LYnNcG1pc1 KF3zpF6x7y6Eq09lIvRVecH4f29pjfq6e2hsNkegn6rLQzqL2pB19qFtPYKxJy81mZDK pFZtNOdNFH8d5SENxWyzPYDeqJ7FaDDmxULoevwNBjW5VKIpTPtZQzN9/UFFNSmJEehj Mslg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PDSj3YIs; spf=pass (google.com: domain of linux-kernel+bounces-23267-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-23267-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id k128-20020a628486000000b006dae558afa8si643396pfd.60.2024.01.11.01.01.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jan 2024 01:01:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-23267-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PDSj3YIs; spf=pass (google.com: domain of linux-kernel+bounces-23267-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-23267-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 2A5E5282730 for ; Thu, 11 Jan 2024 09:00:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 58C95125CA; Thu, 11 Jan 2024 08:57:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PDSj3YIs" Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44747FBFA; Thu, 11 Jan 2024 08:56:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704963418; x=1736499418; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=UUWlIy1VxeScjKQ6UmhSu9xVQRv3RDvrQHXl6p2NOP0=; b=PDSj3YIscoybxYQTFyj6cCLIm1/V92LXdVhNyscGV+swhxytjswRqkT/ IldtAGVivK7vYAddyNx3PCnAinONfgcs6/anBo4bjdZqCUkzWp2ECv2pG Pc0mq1PCYdnMdTY4Zmaac1nzThCqfoTq6m9nZLrGMAAs5sD3dhCntCubf KL2/FWXWHcotSaZ5AbYyDp7XFBaOcXMr795JQkEtlPO8bEdfWHM0jlhX2 4kiKG1h9i8iqbt2RFVnFNYB9Knsecvo9NOZx0cMrtkSO1T/MzercEJJQU FNMYuUSj4OkeIikbFq61nf8c2xW5tk2h0WOHGofFz1gKnHkchmhORRFwf Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10949"; a="429967807" X-IronPort-AV: E=Sophos;i="6.04,185,1695711600"; d="scan'208";a="429967807" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2024 00:56:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10949"; a="785916920" X-IronPort-AV: E=Sophos;i="6.04,185,1695711600"; d="scan'208";a="785916920" Received: from ericwong-mobl2.amr.corp.intel.com (HELO desk) ([10.209.43.169]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2024 00:56:55 -0800 Date: Thu, 11 Jan 2024 00:56:55 -0800 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com, Andrew Cooper , Nikolay Borisov Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com, Pawan Gupta Subject: [PATCH v5 6/6] KVM: VMX: Move VERW closer to VMentry for MDS mitigation Message-ID: <20240111-delay-verw-v5-6-a3b234933ea6@linux.intel.com> X-Mailer: b4 0.12.3 References: <20240111-delay-verw-v5-0-a3b234933ea6@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240111-delay-verw-v5-0-a3b234933ea6@linux.intel.com> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787784010810247809 X-GMAIL-MSGID: 1787784010810247809 During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Signed-off-by: Pawan Gupta --- arch/x86/kvm/vmx/vmenter.S | 3 +++ arch/x86/kvm/vmx/vmx.c | 20 +++++++++++++++++--- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index b3b13ec04bac..139960deb736 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -161,6 +161,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index bdcf2c041e0c..8defba8e417b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -387,6 +387,17 @@ static __always_inline void vmx_enable_fb_clear(struct vcpu_vmx *vmx) static void vmx_update_fb_clear_dis(struct kvm_vcpu *vcpu, struct vcpu_vmx *vmx) { + /* + * FB_CLEAR_CTRL is to optimize VERW latency in guests when host is + * affected by MMIO Stale Data, but not by MDS/TAA. When + * X86_FEATURE_CLEAR_CPU_BUF is enabled, system is likely affected by + * MDS/TAA. Skip the optimization for such a case. + */ + if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) { + vmx->disable_fb_clear = false; + return; + } + vmx->disable_fb_clear = (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && !boot_cpu_has_bug(X86_BUG_MDS) && !boot_cpu_has_bug(X86_BUG_TAA); @@ -7226,11 +7237,14 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed in spite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) mds_clear_cpu_buffers();