From patchwork Tue Feb 20 01:02:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 203361 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp124313dyc; Mon, 19 Feb 2024 17:02:42 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUs94RB2MM+dWBoHDxPvzCF6djEjFOegJ5Z4elBjgYzjegvnURqG2X7eQZtiXDIqmz6FDIwGtiL+QT3D5duvue/0O2YMQ== X-Google-Smtp-Source: AGHT+IF/zrsvWj4qN/+VcRHH1pOpZf9SkQFrTuhfAttkRorU+yb7Bz4/8Qriry+vCu+AX36068RF X-Received: by 2002:a05:6808:1286:b0:3c1:53d2:dfa8 with SMTP id a6-20020a056808128600b003c153d2dfa8mr7090140oiw.2.1708390962264; Mon, 19 Feb 2024 17:02:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708390962; cv=pass; d=google.com; s=arc-20160816; b=BDZ8vOhRnv0n8QiFboZjduuiflhfJUDxqIxuGO+arkNzI/jPenrIGca3S41UEh6naH wpdFaNMAU2hViTYltkdUlwUzazeul3/Bt+vtuOjpF1fONnMBj8wGxlyNO0WvjssOM1h+ ON02bZcaqeD1BZWJ4eVTMuKLIzWUd60cV2PyCbjL79VZ1DV7324Mi6HI0dMhPf1tGhpc OcjkvSln+ooZLs8UtSbi7gQEvyVD2N0Z0DF7k5pp8C8FzZYgiihnI0LycpqqyOk04lRG ZmoKyHh5VayFEcastJG/n/YQCsHVtvMkVvSjodIRfrS2Mvi3jU5sOy4qwcnuZw68YBLl IpDw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:precedence:robot-unsubscribe:robot-id :message-id:mime-version:list-unsubscribe:list-subscribe:list-id :precedence:cc:subject:to:reply-to:sender:from:dkim-signature :dkim-signature:date; bh=Qtzotdyhh5BYmI4EgnPhbEDmKfEaLL07NkNOMUC/9Hw=; fh=4wK8zps4K4EcESrQYUEBGR6VCG66xd82b19sKhpsycA=; b=bX4887DfcB8VFIKY65wk0iOh45ESBKu4iZpE5mBhiyBBD8uYsLK/pXyUOEOgkXWMiM nFI9La7vZP2l93uRmdl9g30AjCz5Fyj/S2sBcLKsZCygfsRjl0mL/jcei/zyFdylrQOG hLzC4/RqA0bGayWaUGX+Y+ig2A6xf3BirnNJXS7G4LxAbFhCINCZHx75T2GlhANQiWur 3dMqG5ehZ5x+9sB6pv6xsW6nUD4Z6VzZsNycmUmeAM0zSxooPqTsYgFR4V7ckwKR4Ljn SMTFldE/h4pcsHPXa7OpQABHOxtY2oBJw24zI+FiVSc5za2iyS+o6M7AS/GVG0bWI1gz HdZA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=mCPUEp80; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="r6+C/78m"; arc=pass (i=1 spf=pass spfdomain=linutronix.de dkim=pass dkdomain=linutronix.de dmarc=pass fromdomain=linutronix.de); spf=pass (google.com: domain of linux-kernel+bounces-72159-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-72159-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id w10-20020a05622a190a00b0042bfe8c5a23si8287379qtc.137.2024.02.19.17.02.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Feb 2024 17:02:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-72159-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=mCPUEp80; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="r6+C/78m"; arc=pass (i=1 spf=pass spfdomain=linutronix.de dkim=pass dkdomain=linutronix.de dmarc=pass fromdomain=linutronix.de); spf=pass (google.com: domain of linux-kernel+bounces-72159-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-72159-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0D5AB1C22567 for ; Tue, 20 Feb 2024 01:02:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 43E7622EEA; Tue, 20 Feb 2024 01:02:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="mCPUEp80"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="r6+C/78m" Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D2368465; Tue, 20 Feb 2024 01:02:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708390929; cv=none; b=tUzB22hjvwLCP1BlJFp70eI2w2AifoIUsco4fRWmJrGOddMBHv3LL73gFFlwpyot8aaEN5uaxSQQgMDz8O0SGLFX9U1MM63enVCz21rpu05995sGZZo+oJc4g3DfPs63eQJGhYnfq5JhdYyvE1qLUrEL5CLn0+saxLuB/VMIGt4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708390929; c=relaxed/simple; bh=nmFRqokI3uNyzTKS+VUD0/4ACgZDgXDwdWyPGQVLfXg=; h=Date:From:To:Subject:Cc:MIME-Version:Message-ID:Content-Type; b=kpN1WehwlSjdVrf/tKmQGaC7mvYkO2NJSOERZRYnzjVrbqqs1h3V/YhiZnVXM+us1isH/WLA0r/qfqCYQZmw3hLiLsHcsrTrOW53+D9uKIxeCS3QRaB23ey/rFWyNIsaa/XImJvg0PogUhhLMWKmXq0Jz5ytKsPlSGYPybNr1zM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=mCPUEp80; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=r6+C/78m; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Date: Tue, 20 Feb 2024 01:02:04 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1708390925; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Qtzotdyhh5BYmI4EgnPhbEDmKfEaLL07NkNOMUC/9Hw=; b=mCPUEp80jeo6IMza0HIZAoGdX6+BHgM92UcPqIvoAebVSvv4eDs2Pj6B+HbvP1kGe75IFN S3/ju8z/FJE4o/Ow5LiB+b/wV3Vukm/CVDQPzdK0N6SZOBv9b0d9aSIvpBFxzebPm4wbBQ kz3FIqLM9FZ1RFai1Djq9W1hXPlNx6/CVGjIlb+fsfDSReQl2QA0kHY77+7sj/hwHeB8XW if+MApSe8fB/pZNA3YPBquJGjjE1FfDAOEMT8BtQKoYHBGKvSFKZzJOCEHlc3mCVLopiOx ycxUM2kAy+isSECL79pRGi4G5ZSgTAPTJjVbcOgVVh3QM6sIq3iTk2CNgQo1OA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1708390925; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Qtzotdyhh5BYmI4EgnPhbEDmKfEaLL07NkNOMUC/9Hw=; b=r6+C/78mV9p5EMw9Ts3N3xAG0IPqM96fYtB1l2iQbGQefp4Y6Cd55OGcYIzmE75fSnzrxE FlBsxoXGMprDI3Cw== From: "tip-bot2 for Pawan Gupta" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/urgent] KVM/VMX: Move VERW closer to VMentry for MDS mitigation Cc: Pawan Gupta , Dave Hansen , Sean Christopherson , x86@kernel.org, linux-kernel@vger.kernel.org Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <170839092466.398.5378682060812783539.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791377761943196404 X-GMAIL-MSGID: 1791377761943196404 The following commit has been merged into the x86/urgent branch of tip: Commit-ID: 43fb862de8f628c5db5e96831c915b9aebf62d33 Gitweb: https://git.kernel.org/tip/43fb862de8f628c5db5e96831c915b9aebf62d33 Author: Pawan Gupta AuthorDate: Tue, 13 Feb 2024 18:22:56 -08:00 Committer: Dave Hansen CommitterDate: Mon, 19 Feb 2024 16:31:59 -08:00 KVM/VMX: Move VERW closer to VMentry for MDS mitigation During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Signed-off-by: Pawan Gupta Signed-off-by: Dave Hansen Acked-by: Sean Christopherson Link: https://lore.kernel.org/all/20240213-delay-verw-v8-6-a6216d83edb7%40linux.intel.com --- arch/x86/kvm/vmx/vmenter.S | 3 +++ arch/x86/kvm/vmx/vmx.c | 20 ++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index ef7cfba..2bfbf75 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -161,6 +161,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index db8a5fe..88a4ff2 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -388,7 +388,16 @@ static __always_inline void vmx_enable_fb_clear(struct vcpu_vmx *vmx) static void vmx_update_fb_clear_dis(struct kvm_vcpu *vcpu, struct vcpu_vmx *vmx) { - vmx->disable_fb_clear = (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && + /* + * Disable VERW's behavior of clearing CPU buffers for the guest if the + * CPU isn't affected by MDS/TAA, and the host hasn't forcefully enabled + * the mitigation. Disabling the clearing behavior provides a + * performance boost for guests that aren't aware that manually clearing + * CPU buffers is unnecessary, at the cost of MSR accesses on VM-Entry + * and VM-Exit. + */ + vmx->disable_fb_clear = !cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF) && + (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && !boot_cpu_has_bug(X86_BUG_MDS) && !boot_cpu_has_bug(X86_BUG_TAA); @@ -7224,11 +7233,14 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed in spite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) mds_clear_cpu_buffers();