From patchwork Wed Feb 14 02:22:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 200828 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp1025568dyb; Tue, 13 Feb 2024 22:39:52 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW6Wg1tzR6W6i1lOdZiFO+K946nOY2GBa3YA7/n1pcyIfE6xO9WRSy+8mcJ0ujFSYsGfpW7asN8KPq4Awlr/ekOLoWFWQ== X-Google-Smtp-Source: AGHT+IEXfhH1a3ForXpAd1qnKUkoRN7tgq6d6kx/dxYGsuvtJR+JGtCZd5ivO6XrUJVdq6kDUlFo X-Received: by 2002:a17:902:f54e:b0:1da:2122:be72 with SMTP id h14-20020a170902f54e00b001da2122be72mr1968274plf.62.1707892791919; Tue, 13 Feb 2024 22:39:51 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707892791; cv=pass; d=google.com; s=arc-20160816; b=TeItfqVwewlC9WUrG2E95XhjQc5Nz2AIJYfRtc/bDt4+40xLx2FNHlevkSAqYLNhcu LJSsWdT07Zm9ANpygu2nrr+4hgVrIs6avNv7eEapZ+o2ac2nczGnNIj44mKFdSKj1FBw shwJFcF2dnV9FczzQKaWMTuY2XS/PRPKm1TnYhbE5gw4tvcGn9ZAzjzhOMo+GBN+YfLS oi4b0J3nxhwo6Q8VjkOHjcuLyvCUf3OgIZo5Ot5Go46Qs7bmsOtOga9YDbxRbZ1vimOX OhGUUaSPst6LrXen7ejAnPPKKmc4t6FMwpVy8PZxc0NBPm4KqErfxl3cd+oBGsnO53gf 5GQA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature; bh=/7UK3x3tKrFCfYJG7YwyIIYKLN4qC7RJ/CJ7sQXwahs=; fh=hG4UUp1oC+LwLbUDqxfiPve8iqQ6gt9x8REnzohJjCA=; b=AxQuMwXvei4gqxgFY8BvLb5k40p7+VZqkBF70A8yEmbZJDmUWxyvXdad20Pl4WdLkR oAQRilLmMxRtIp7NqAhZub2RBBG456/wO/bH8E76gZm7OieUh0LhPCs9xHYFUZLytN65 vK6kIz3z0krhrzixXxBsE1KxOHYkoPINhqbFhK7k6Z+Pa89htomhPsXfDF6wgir9ajNZ IDYGo3luvWLvzsDwshHestS5UW6Mnz9Dry3UL0f1FKNp6GL2yUB7onkE0eyw4IArGGnx n/EUaJGQBFssp0pslrI3wVmf/nM7P5ojqqfmqN2HuAMQLNNC7EKxEfQceDHgMPnW7ZpK r2Xg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=XHiS2cTA; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-64690-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64690-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Forwarded-Encrypted: i=2; AJvYcCV7VCvNcVIcdYd+7UfjTWFOV/3r833ljbHfEKjwv/LyrWCe0xEeRtjdzV8+S3ZDtX2SYa+BlfD7WlsfRhQcCVqsc1xHGw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id l18-20020a170902d35200b001d981d92655si3255740plk.308.2024.02.13.22.39.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 22:39:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-64690-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=XHiS2cTA; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-64690-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-64690-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EBA3528F274 for ; Wed, 14 Feb 2024 02:24:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8D325D30B; Wed, 14 Feb 2024 02:23:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XHiS2cTA" Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BEF7BA22; Wed, 14 Feb 2024 02:22:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707877380; cv=none; b=EZTEpkuDSk6x+IrnK5ASQWBw6ve/cCQYy5ZMpIrceTz+t8TFXluSXSkNu/rDv4A3mlJ9hFIvAg3l+LpWCzq+eHMgz9RvEwClu3zWNeaEUnMnQuOjuXnWYc0OuEW5hMaQzN9sI4i5ch7Nro4ErBMhgX3aRrVW25HiTYeVKyNpMgk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707877380; c=relaxed/simple; bh=mYU4BqeqoU+8q8eBooFhBgcQd6WMmusiCtW20ZTS80g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IOZr2ZF3Pa3mUNvMgMWkd2TMmO9l2dJNBLLdePVoLtcuxKGvVG8+S6CLpFC3BZ/ot2SS3FuBHkZLXoyEUTzNTleej59UZe/MhUtbC5hXMAoYt9/N0v2/FzxM3CDu2r4amwFz8lAfDW0ain9Ubgxv1VDiTN7uE08IQnN0xHdMFV0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XHiS2cTA; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1707877378; x=1739413378; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=mYU4BqeqoU+8q8eBooFhBgcQd6WMmusiCtW20ZTS80g=; b=XHiS2cTAOLhWqZY7Ze4kn/BbNSLHUBmbzJ1Nq1Kdb8ecU1J58aCzZls3 bRduvcxXRe11axdwF6e1B1oVpCccvVMhOKKxRTpF35vQ13ufR3QDAQC+y 1SMCDO2TSO5yHAHEXoZIxi2ADvIl6k04QrQEk5sb5wr+X/UlIr/kY+I6j oRP2miooBHaGm+iRWe+cz+96TbEHiELc/rl4EglTsgIIf/0755Vzy8brz Cj2g4J6mqw9uJxP8emuj0ZIn781WFVSs+x+GdMyt26tss1P9fd4HMGfEf A46yGohgmaoylMX1FG8SrDjYBdUxGA5XyGyIaDmMppayC8TTDrluQlpO9 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10982"; a="27359354" X-IronPort-AV: E=Sophos;i="6.06,158,1705392000"; d="scan'208";a="27359354" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Feb 2024 18:22:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,158,1705392000"; d="scan'208";a="26229202" Received: from diegoavi-mobl.amr.corp.intel.com (HELO desk) ([10.255.230.185]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Feb 2024 18:22:56 -0800 Date: Tue, 13 Feb 2024 18:22:56 -0800 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com, Andrew Cooper , Nikolay Borisov Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com Subject: [PATCH v8 6/6] KVM: VMX: Move VERW closer to VMentry for MDS mitigation Message-ID: <20240213-delay-verw-v8-6-a6216d83edb7@linux.intel.com> X-Mailer: b4 0.12.3 References: <20240213-delay-verw-v8-0-a6216d83edb7@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240213-delay-verw-v8-0-a6216d83edb7@linux.intel.com> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790855392392217449 X-GMAIL-MSGID: 1790855392392217449 During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Cc: stable@kernel.org Signed-off-by: Pawan Gupta Acked-by: Sean Christopherson --- arch/x86/kvm/vmx/vmenter.S | 3 +++ arch/x86/kvm/vmx/vmx.c | 20 ++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index ef7cfbad4d57..2bfbf758d061 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -161,6 +161,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 40594eae2cd3..305237dcba88 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -389,7 +389,16 @@ static __always_inline void vmx_enable_fb_clear(struct vcpu_vmx *vmx) static void vmx_update_fb_clear_dis(struct kvm_vcpu *vcpu, struct vcpu_vmx *vmx) { - vmx->disable_fb_clear = (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && + /* + * Disable VERW's behavior of clearing CPU buffers for the guest if the + * CPU isn't affected by MDS/TAA, and the host hasn't forcefully enabled + * the mitigation. Disabling the clearing behavior provides a + * performance boost for guests that aren't aware that manually clearing + * CPU buffers is unnecessary, at the cost of MSR accesses on VM-Entry + * and VM-Exit. + */ + vmx->disable_fb_clear = !cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF) && + (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && !boot_cpu_has_bug(X86_BUG_MDS) && !boot_cpu_has_bug(X86_BUG_TAA); @@ -7227,11 +7236,14 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed in spite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) mds_clear_cpu_buffers();