From patchwork Mon Feb 12 18:32:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 199984 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp97163dyb; Mon, 12 Feb 2024 10:33:18 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUH47D33Qii3ul5R+xEVr9OGK3nHuWsFZ9FRhKLdtWVrjxtE5wrOAc0a1OV93TF19DxTd35GtEee4qRwZPR1xcIsZJP+Q== X-Google-Smtp-Source: AGHT+IGozw8aWnAPQie6/7R8E21A13fEtI49bglfXu2OtU4md841cQWQKFlXxbEOUjWVHqWb6fwP X-Received: by 2002:a17:903:1246:b0:1d9:8820:f413 with SMTP id u6-20020a170903124600b001d98820f413mr7869750plh.38.1707762798510; Mon, 12 Feb 2024 10:33:18 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707762798; cv=pass; d=google.com; s=arc-20160816; b=Phog+SyEwGXebv2Dme8Y8aJq++6Tbs/ZzGFii+3V/gVjURpE56H9ORshJc2MRRuT2c tnnayDdTGw4GaK1ymSlPxFdJao7KYVFJEbhov4qFA/dGeF466SEGQ+kFlYFkEgkBDnvU iMIHM1GcPXqUXvROCMPUt4m/AxM6UZ54uugGEJitClPxBnpKPorxxjkVrfmmWaJsMhQt ocbxZXGAWZ8v0XAxjVZ5TuAUzj1Vr+7Nf808teT1Fc89c20GuNmZXn03fyqP9/nO4uXp IU2jcp+ZFv7c9EMN04NEWszH/EHefnPIKZggw/AjzItH8SBU2EFIqgG2EH51FwSNDiao HkaA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:precedence:robot-unsubscribe:robot-id :message-id:mime-version:list-unsubscribe:list-subscribe:list-id :precedence:cc:subject:to:reply-to:sender:from:dkim-signature :dkim-signature:date; bh=ZnDMX9SB2+uRk6/SPP2GaGDjf0ovHnqT/3el3m6ZSyk=; fh=R7Ej8SrpREGR03gK6K/yG4ongSL9CCbN3tvouNFG4Pw=; b=j6obgQuisMcFGbQXbrEwFYEnKXzjXqXeXNeEHBo7VQeal7zmz9fmIn3XiUC4Y//FlF W5zAhNKTBVlKSIGArVjmw02XdVLupisd8djSM8EvRT/qdttD2GS0lbMoIBnOvoNTrW4P L/18Wu2K9EIlYfAP2yJ04adjg6ED4C44YV5u/TkoBa4X8McWQySfVjUZCfF19wgkRj8E 1aGaoO5/etygMybUtmU2AmRB6caa0NC1oc8M9108lyVDGU1xu63UKksWmnHpaVVCSrS2 94/tNjKsK31otOqFouwEvJnS8OA530wcsLDIWX/6mGidpEN4HOO9kPGtxwMxaZS6KjYo nQvA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=CChodpHV; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="DY//IUtp"; arc=pass (i=1 spf=pass spfdomain=linutronix.de dkim=pass dkdomain=linutronix.de dmarc=pass fromdomain=linutronix.de); spf=pass (google.com: domain of linux-kernel+bounces-62186-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62186-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de X-Forwarded-Encrypted: i=2; AJvYcCWz8usHsUi4RAlh07PK5+Pb/KstE2SlUKU3lhXw4hV4hvnNBPVHvWrsC9HzYT4uuJk76+1kB6I3LO6LTyr29eLie5dd9Q== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id mf6-20020a170902fc8600b001d7313028e5si603885plb.430.2024.02.12.10.33.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 10:33:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62186-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=CChodpHV; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="DY//IUtp"; arc=pass (i=1 spf=pass spfdomain=linutronix.de dkim=pass dkdomain=linutronix.de dmarc=pass fromdomain=linutronix.de); spf=pass (google.com: domain of linux-kernel+bounces-62186-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62186-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3F6952822D3 for ; Mon, 12 Feb 2024 18:33:18 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A6B8746450; Mon, 12 Feb 2024 18:32:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="CChodpHV"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="DY//IUtp" Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF6CAEEC5; Mon, 12 Feb 2024 18:32:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707762750; cv=none; b=Q2wtszxcexojSWjtxEv4FKEKSG/H3Cjfo9ZdacwhTVSFYeS3c3uTA3FUUZh5xa+sipLtF/GLHEnMLKfmces5VKoKGaAg9FnDk8s7UgAE1kOqXRStMcangTM2+9ATJZB165Lz8NQeoa9/ion7CaRArPjXFYEw9ebrWglcc32tq10= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707762750; c=relaxed/simple; bh=YJ1kfquzfEg3xMlggxVXNEPPrLvtBdFrSEj/fs42Plk=; h=Date:From:To:Subject:Cc:MIME-Version:Message-ID:Content-Type; b=q/B+gC5G9PwrSvU+c5e+bpTfSQR63xCn8cUtPNl7H8KM1MeoSlH+PNF3b1Uy5UfYHwh2k3Ku3T2uQ2ceLG121F+jFmprh5tOkfjX1dz22+lCUmkaxmntfLL6XqJp+19n2s4Mn2RwaoQppn6im5TgdjrU3w8COahov+9psbocETc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=CChodpHV; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=DY//IUtp; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Date: Mon, 12 Feb 2024 18:32:25 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1707762746; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZnDMX9SB2+uRk6/SPP2GaGDjf0ovHnqT/3el3m6ZSyk=; b=CChodpHVxpFDbzeOT5nU7B3qeO7DCXsGl6gl4ez8GYh8d7pPFwqR+GSBExZ/CksIDYkOnF R26yS+flRl0UsLlKOcihdvPt1er04y4jo7Tbj05INmkLRdZwi0sMB5rOnzTcmHI145P/NZ oH8Fh5FJ+eyKf9WhTlcqWm25rbt4S2xhLI5r4uM4JUH5qJNRJ1eJApFjckb2bahxOvFxoN HVGPeHKb0fsjusSXP4gttX5K0N83lFfyYD3rUA661pUzPXrAAMFQdJMru1X4ZsX/1lYRVB ES8RkOhAT9NtBwLA+VQSmEn5c2iEBEPo4X+0e8UTwyYini6SQQazpD3a5hZy/g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1707762746; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZnDMX9SB2+uRk6/SPP2GaGDjf0ovHnqT/3el3m6ZSyk=; b=DY//IUtpQEqzpmBXQfQez7SuhGI3ds9UkfnrdM4Y+PEMw4FtOGAi5PownLr+hVmFyKArb3 K2KzIdhBV2OP3ABA== From: "tip-bot2 for Pawan Gupta" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/urgent] KVM/VMX: Move VERW closer to VMentry for MDS mitigation Cc: Pawan Gupta , Dave Hansen , Sean Christopherson , x86@kernel.org, linux-kernel@vger.kernel.org Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <170776274528.398.3511446889501764521.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790719083803279653 X-GMAIL-MSGID: 1790719083803279653 The following commit has been merged into the x86/urgent branch of tip: Commit-ID: f1423564fd426c134f00c77dd0f97e350f1d2f41 Gitweb: https://git.kernel.org/tip/f1423564fd426c134f00c77dd0f97e350f1d2f41 Author: Pawan Gupta AuthorDate: Sun, 04 Feb 2024 23:20:23 -08:00 Committer: Dave Hansen CommitterDate: Mon, 12 Feb 2024 10:25:22 -08:00 KVM/VMX: Move VERW closer to VMentry for MDS mitigation During VMentry VERW is executed to mitigate MDS. After VERW, any memory access like register push onto stack may put host data in MDS affected CPU buffers. A guest can then use MDS to sample host data. Although likelihood of secrets surviving in registers at current VERW callsite is less, but it can't be ruled out. Harden the MDS mitigation by moving the VERW mitigation late in VMentry path. Note that VERW for MMIO Stale Data mitigation is unchanged because of the complexity of per-guest conditional VERW which is not easy to handle that late in asm with no GPRs available. If the CPU is also affected by MDS, VERW is unconditionally executed late in asm regardless of guest having MMIO access. Signed-off-by: Pawan Gupta Signed-off-by: Dave Hansen Acked-by: Sean Christopherson Link: https://lore.kernel.org/all/20240204-delay-verw-v7-6-59be2d704cb2%40linux.intel.com --- arch/x86/kvm/vmx/vmenter.S | 3 +++ arch/x86/kvm/vmx/vmx.c | 20 ++++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S index ef7cfba..2bfbf75 100644 --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -161,6 +161,9 @@ SYM_FUNC_START(__vmx_vcpu_run) /* Load guest RAX. This kills the @regs pointer! */ mov VCPU_RAX(%_ASM_AX), %_ASM_AX + /* Clobbers EFLAGS.ZF */ + CLEAR_CPU_BUFFERS + /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */ jnc .Lvmlaunch diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index b551de3..0ec71f9 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -388,7 +388,16 @@ static __always_inline void vmx_enable_fb_clear(struct vcpu_vmx *vmx) static void vmx_update_fb_clear_dis(struct kvm_vcpu *vcpu, struct vcpu_vmx *vmx) { - vmx->disable_fb_clear = (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && + /* + * Disable VERW's behavior of clearing CPU buffers for the guest if the + * CPU isn't affected by MDS/TAA, and the host hasn't forcefully enabled + * the mitigation. Disabling the clearing behavior provides a + * performance boost for guests that aren't aware that manually clearing + * CPU buffers is unnecessary, at the cost of MSR accesses on VM-Entry + * and VM-Exit. + */ + vmx->disable_fb_clear = !cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF) && + (host_arch_capabilities & ARCH_CAP_FB_CLEAR_CTRL) && !boot_cpu_has_bug(X86_BUG_MDS) && !boot_cpu_has_bug(X86_BUG_TAA); @@ -7224,11 +7233,14 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_enter_irqoff(); - /* L1D Flush includes CPU buffer clear to mitigate MDS */ + /* + * L1D Flush includes CPU buffer clear to mitigate MDS, but VERW + * mitigation for MDS is done late in VMentry and is still + * executed in spite of L1D Flush. This is because an extra VERW + * should not matter much after the big hammer L1D Flush. + */ if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); - else if (cpu_feature_enabled(X86_FEATURE_CLEAR_CPU_BUF)) - mds_clear_cpu_buffers(); else if (static_branch_unlikely(&mmio_stale_data_clear) && kvm_arch_has_assigned_device(vcpu->kvm)) mds_clear_cpu_buffers();