From patchwork Wed Jan 24 07:42:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 191433 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp861133dyi; Wed, 24 Jan 2024 01:02:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IFh3th44h1APCINeYtLBQG8DVJsOxAAWnAaCYtfjgJVPCJn9608at+bvCxxrNOGvixsFIHM X-Received: by 2002:a05:6a20:4da6:b0:199:9c7e:a58b with SMTP id gj38-20020a056a204da600b001999c7ea58bmr552278pzb.118.1706086936839; Wed, 24 Jan 2024 01:02:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706086936; cv=pass; d=google.com; s=arc-20160816; b=RKYuQUCW4Ikk/8xXVYKjskPKum5kYqvDuiO/UfxKctAeDQfx2EhCLIcVkQDjdsKzzW lOQAd7clKe3Y7a7R8PMKPZjPdZeznvahIEQ/sN5Yoob725TU5NS6gaf/748/4GNAZtmo zj4YlJo3OsdI05YZgpADHBx4fI/KPN91Y4e22iU7hcDQtgxfaMd5sPGiY5TqLf14Ag9e LEcRl9rol3qyLARXSqo1PO/0EdoD4Yf5SaKNw3zivcz55zGJ6inD2T8917wgJP63w4KK GmCQikE2Res17FMfKm7f94H6RCm1wUdrm3krBwQ4b68kPe2K3hq2u2kmjU+boZmpQGoG xuHw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature; bh=Ya5x+74v6CArtc8dDax8Evc0luoPHJvtMq3RFVrUxcM=; fh=ER2H6g+88FnZd3JyhT6ewHH56pdK2QWfWr77z0IHoNE=; b=jI8AO2vlk1ivymai+eX0oTJUgEhzD6qsA9WhgqjgGuqZ7fLU9PSQsOWisvlrMF7xM7 26jh/bTDb/Su6BVgHq9mFXDv+buH4UFRXKir2NX9C8/HIZ3zP+P7XrKR2ZGEudw4sOzM kQnMVTb7FqGP6qUSTkSIFMwkeIbmFBxch3E/NdWLhUWwjNxN8T/Pv7a/CRtRTvQSvE5/ mDOHYg7mSzxHi5zHK+KGBYUynYQkY7F3gjTMmks/8cqUObOsP+svkyxUAXmOeOdhiU0K OtbRUt7R4GvQP8MwRhW7sBcGUP/rbl+lkcSWSmJfQ+R5Yy0Q6+OIR2y0wDddj3tseSL5 plSw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=DOtWu88t; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-36580-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-36580-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id bj7-20020a17090b088700b0029058af1c92si8541501pjb.166.2024.01.24.01.02.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 01:02:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-36580-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=DOtWu88t; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-36580-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-36580-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 791B228F2C1 for ; Wed, 24 Jan 2024 07:45:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E89FA18041; Wed, 24 Jan 2024 07:42:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DOtWu88t" Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EACA168D8; Wed, 24 Jan 2024 07:42:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=134.134.136.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706082156; cv=none; b=e9mGXpopZ71b7EvzJ+uGIc7Mf9/dCkJs71jpWLyCfYdCQ7tvgLewxQCgSuWpUmt+vGx5lu+0TlZECTzI3OjVjKqZrdf259hODLtX0mxDefa9DnKopEONiYAzliu2KPIQ8xF8p322yArwR8PcicQWeXsmXXFycW9RV0CY4M3Uy9A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706082156; c=relaxed/simple; bh=l7St/8Yz8St+HFZatHx/yUdB2U84CrVMB3Tw9Oi74Cc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=JWVIWTDTFe0LCvMmbN7RdbhDl1ScPui530V8HfqKT7qgCNDzna970aqxORnPDJxXdA36KqeNZFEfYiP69pHmYjv6WuOqJXy5Qlyn/CCSeLQtbzbTYnht3//qIxgq9BMtjf6qHFqWZ4UtVLC0AjL+z6Y4fJrg3VPBCmnhIBkRDwI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DOtWu88t; arc=none smtp.client-ip=134.134.136.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706082154; x=1737618154; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=l7St/8Yz8St+HFZatHx/yUdB2U84CrVMB3Tw9Oi74Cc=; b=DOtWu88tu3vGy9cJdvUSE00qbFEPC5uGZSXU6m7wpFShq7obmJ6EZMO6 PIvZ6p8gfnthph6CSJYGWmUA0eHLcVK4U4lm2B0C45GiPJN8DNTd6skpy nOxeDfXkLdnlIDrj53+D6SlxlqxmgYMrefKWsz2Cvqme/cryoxhhliQ50 iUDuz8CvRImW90vxc+Pl7UwUOn9guH2MVlQ1VKNsKF4vk4PuP4zDIeycs k5S7oLOIMXDjXN37RNeKjK0qbNOqHg9SgOeYrC5W+FmotTc17MUzE7S6u mgmKrWMHxwxJTnpeWyG8mt4tlQZoaDimI/Nxlrlb4oYTo0fH6t/k0zr6e g==; X-IronPort-AV: E=McAfee;i="6600,9927,10962"; a="392184770" X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="392184770" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 23:42:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,216,1701158400"; d="scan'208";a="1819198" Received: from bbaidya-mobl.amr.corp.intel.com (HELO desk) ([10.209.53.134]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jan 2024 23:42:31 -0800 Date: Tue, 23 Jan 2024 23:42:29 -0800 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com, Andrew Cooper , Nikolay Borisov Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com, Pawan Gupta Subject: [PATCH v6 6/6] KVM: VMX: Move VERW closer to VMentry for MDS mitigation Message-ID: <20240123-delay-verw-v6-6-a8206baca7d3@linux.intel.com> X-Mailer: b4 0.12.3 References: <20240123-delay-verw-v6-0-a8206baca7d3@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240123-delay-verw-v6-0-a8206baca7d3@linux.intel.com> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788961815682296046 X-GMAIL-MSGID: 1788961815682296046 During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Signed-off-by: Pawan Gupta Acked-by: Sean Christopherson --- arch/x86/kvm/vmx/vmenter.S | 3 +++ arch/x86/kvm/vmx/vmx.c | 20 ++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index b3b13ec04bac..139960deb736 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -161,6 +161,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index bdcf2c041e0c..0190e7584ffd 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -387,7 +387,16 @@ static __always_inline void vmx_enable_fb_clear(struct vcpu_vmx *vmx) static void vmx_update_fb_clear_dis(struct kvm_vcpu *vcpu, struct vcpu_vmx *vmx) { - vmx->disable_fb_clear = (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && + /* + * Disable VERW's behavior of clearing CPU buffers for the guest if the + * CPU isn't affected by MDS/TAA, and the host hasn't forcefully enabled + * the mitigation. Disabling the clearing behavior provides a + * performance boost for guests that aren't aware that manually clearing + * CPU buffers is unnecessary, at the cost of MSR accesses on VM-Entry + * and VM-Exit. + */ + vmx->disable_fb_clear = !cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF) && + (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && !boot_cpu_has_bug(X86_BUG_MDS) && !boot_cpu_has_bug(X86_BUG_TAA); @@ -7226,11 +7235,14 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed in spite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) mds_clear_cpu_buffers();