Message ID | 20221129193717.513824-12-mlevitsk@redhat.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp537272wrr; Tue, 29 Nov 2022 11:53:25 -0800 (PST) X-Google-Smtp-Source: AA0mqf794FQBR3U7Lwh3BhCAqrgQJzoBa8GdrE2srQ/A38oClW/cvaswIUJ4B8M7/28yRV2Pfc8N X-Received: by 2002:aa7:d558:0:b0:461:eea0:514c with SMTP id u24-20020aa7d558000000b00461eea0514cmr43537804edr.296.1669751605114; Tue, 29 Nov 2022 11:53:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669751605; cv=none; d=google.com; s=arc-20160816; b=uuNoJIFeecFxcU/kkHQdbhcHvpHViuPFpCIsh3W6qRJQtCxiJ6WUDe13N+k14ptolh iQRDprFERzyoLh9eoCO1eDYAXKeKJFKXGjBym0wHmT4pKbb1fTaszefYFufmAXZgGdC/ hxZ3JvZGFhTR1EtVMHthA48H+sLaGKmdNTf742IWC0KT1WACddblj+tdAHCYG+Xctqkf 26LhTQ/yGLwkScpG8DMVCgJst1Lx/PcXo3U54m5T0PCEgwrJ5NBUrWyvBRvr5M6j4C/4 W82vmYXPGmAvptMN1hc6Axeus3154jLygRwVlM8knuDYDiV8pTzeRLPwdrvPbJwpwoVj Mgjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zbNm6u7VwMLnnR5d0F1qjMdQ/5J2+Ysx2Y9gYe/zZwQ=; b=FoenIj4i45VW3OD1v3y5plUexVv/t08N642WptelXI7eQvloWxxnVzcJOZzbUDKbEq IniPG5jhuYPvbUCeiMDNJWqIpxguMjvRWLQFM2Ri4GnrTF1KkXTs+Od3B23dv1ItbTYi hFFmAD7jVyZ7JseHTvgxDZ4ZfM/lydsOsw5506/7T6VeW/UKOd1/tq/SItLVWzWBM3/L UPZWIzLXhucmitY1PGe6PO+TIEKoVZcV/IkNP2JeGqZzSMxhn5fq2MNv4tYMbZPtzgWm 0ArZvSns0we/hEBSbVef3LLydfVA2crAoHFomuHWLXC6KhooObhv/6yQwtiv3NZL2OWT x9RA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=X0JnGBW7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hc33-20020a17090716a100b007ae9f42f878si13763290ejc.354.2022.11.29.11.52.59; Tue, 29 Nov 2022 11:53:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=X0JnGBW7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236375AbiK2TmV (ORCPT <rfc822;rbbytesnap@gmail.com> + 99 others); Tue, 29 Nov 2022 14:42:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236944AbiK2TlS (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 29 Nov 2022 14:41:18 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A4B761760 for <linux-kernel@vger.kernel.org>; Tue, 29 Nov 2022 11:38:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669750699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zbNm6u7VwMLnnR5d0F1qjMdQ/5J2+Ysx2Y9gYe/zZwQ=; b=X0JnGBW7aYkVjciKNKX1OlDqKuO+9sn3mygQypANNhSkdvb/nwZs3XC6dH4qLxEk00U3oa v67MQnbvBnXis/gy9xtMFBdR4JIBgjD7TcT6EempLooYMDKvmtg9H/wit7IfOjYfJf6b7h 9xs+jJXwSFnIuAgpp5SxgC0rbTKiQrY= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-275-UxmNabJBMqKYQKTIqYI29A-1; Tue, 29 Nov 2022 14:38:10 -0500 X-MC-Unique: UxmNabJBMqKYQKTIqYI29A-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CF1CB3C0F22F; Tue, 29 Nov 2022 19:38:09 +0000 (UTC) Received: from localhost.localdomain (unknown [10.35.206.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 80E782027061; Tue, 29 Nov 2022 19:38:05 +0000 (UTC) From: Maxim Levitsky <mlevitsk@redhat.com> To: kvm@vger.kernel.org Cc: Sandipan Das <sandipan.das@amd.com>, Paolo Bonzini <pbonzini@redhat.com>, Jim Mattson <jmattson@google.com>, Peter Zijlstra <peterz@infradead.org>, Dave Hansen <dave.hansen@linux.intel.com>, Borislav Petkov <bp@alien8.de>, Pawan Gupta <pawan.kumar.gupta@linux.intel.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, Josh Poimboeuf <jpoimboe@kernel.org>, Daniel Sneddon <daniel.sneddon@linux.intel.com>, Jiaxi Chen <jiaxi.chen@linux.intel.com>, Babu Moger <babu.moger@amd.com>, linux-kernel@vger.kernel.org, Jing Liu <jing2.liu@intel.com>, Wyes Karny <wyes.karny@amd.com>, x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>, Sean Christopherson <seanjc@google.com>, Maxim Levitsky <mlevitsk@redhat.com>, Santosh Shukla <santosh.shukla@amd.com> Subject: [PATCH v2 11/11] KVM: nSVM: implement support for nested VNMI Date: Tue, 29 Nov 2022 21:37:17 +0200 Message-Id: <20221129193717.513824-12-mlevitsk@redhat.com> In-Reply-To: <20221129193717.513824-1-mlevitsk@redhat.com> References: <20221129193717.513824-1-mlevitsk@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750861458831431672?= X-GMAIL-MSGID: =?utf-8?q?1750861458831431672?= |
Series |
SVM: vNMI (with my fixes)
|
|
Commit Message
Maxim Levitsky
Nov. 29, 2022, 7:37 p.m. UTC
This patch allows L1 to use vNMI to accelerate its injection of NMIs to L2 by passing through vNMI int_ctl bits from vmcb12 to/from vmcb02. While L2 runs, L1's vNMI is inhibited, and L1's NMIs are injected normally. In order to support nested VNMI requires saving and restoring the VNMI bits during nested entry and exit. In case of L1 and L2 both using VNMI- Copy VNMI bits from vmcb12 to vmcb02 during entry and vice-versa during exit. And in case of L1 uses VNMI and L2 doesn't- Copy VNMI bits from vmcb01 to vmcb02 during entry and vice-versa during exit. Tested with the KVM-unit-test and Nested Guest scenario. Signed-off-by: Santosh Shukla <santosh.shukla@amd.com> Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> --- arch/x86/kvm/svm/nested.c | 13 ++++++++++++- arch/x86/kvm/svm/svm.c | 5 +++++ arch/x86/kvm/svm/svm.h | 6 ++++++ 3 files changed, 23 insertions(+), 1 deletion(-)
Comments
On 11/30/2022 1:07 AM, Maxim Levitsky wrote: > This patch allows L1 to use vNMI to accelerate its injection > of NMIs to L2 by passing through vNMI int_ctl bits from vmcb12 > to/from vmcb02. > > While L2 runs, L1's vNMI is inhibited, and L1's NMIs are injected > normally. > > In order to support nested VNMI requires saving and restoring the VNMI > bits during nested entry and exit. > In case of L1 and L2 both using VNMI- Copy VNMI bits from vmcb12 to > vmcb02 during entry and vice-versa during exit. > And in case of L1 uses VNMI and L2 doesn't- Copy VNMI bits from vmcb01 to > vmcb02 during entry and vice-versa during exit. > > Tested with the KVM-unit-test and Nested Guest scenario. > > > Signed-off-by: Santosh Shukla <santosh.shukla@amd.com> > Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> > --- > arch/x86/kvm/svm/nested.c | 13 ++++++++++++- > arch/x86/kvm/svm/svm.c | 5 +++++ > arch/x86/kvm/svm/svm.h | 6 ++++++ > 3 files changed, 23 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c > index 5bea672bf8b12d..81346665058e26 100644 > --- a/arch/x86/kvm/svm/nested.c > +++ b/arch/x86/kvm/svm/nested.c > @@ -278,6 +278,11 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu, > if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl))) > return false; > > + if (CC((control->int_ctl & V_NMI_ENABLE) && > + !vmcb12_is_intercept(control, INTERCEPT_NMI))) { > + return false; > + } > + > return true; > } > > @@ -427,6 +432,9 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm) > if (nested_vgif_enabled(svm)) > mask |= V_GIF_MASK; > > + if (nested_vnmi_enabled(svm)) > + mask |= V_NMI_MASK | V_NMI_PENDING; > + > svm->nested.ctl.int_ctl &= ~mask; > svm->nested.ctl.int_ctl |= svm->vmcb->control.int_ctl & mask; > } > @@ -682,8 +690,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, > else > int_ctl_vmcb01_bits |= (V_GIF_MASK | V_GIF_ENABLE_MASK); > > - if (vnmi) > + if (vnmi) { To avoid above change, I think we should move nested bits from 10/11 to this i.e.. move function (nested_svm_save_vnmi and nested_svm_restore_vnmi) to this patch. make sense? Thanks, Santosh > nested_svm_save_vnmi(svm); > + if (nested_vnmi_enabled(svm)) > + int_ctl_vmcb12_bits |= (V_NMI_PENDING | V_NMI_ENABLE | V_NMI_MASK); > + } > > /* Copied from vmcb01. msrpm_base can be overwritten later. */ > vmcb02->control.nested_ctl = vmcb01->control.nested_ctl; > diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c > index bf10adcf3170a8..fb203f536d2f9b 100644 > --- a/arch/x86/kvm/svm/svm.c > +++ b/arch/x86/kvm/svm/svm.c > @@ -4214,6 +4214,8 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) > > svm->vgif_enabled = vgif && guest_cpuid_has(vcpu, X86_FEATURE_VGIF); > > + svm->vnmi_enabled = vnmi && guest_cpuid_has(vcpu, X86_FEATURE_AMD_VNMI); > + > svm_recalc_instruction_intercepts(vcpu, svm); > > /* For sev guests, the memory encryption bit is not reserved in CR3. */ > @@ -4967,6 +4969,9 @@ static __init void svm_set_cpu_caps(void) > if (vgif) > kvm_cpu_cap_set(X86_FEATURE_VGIF); > > + if (vnmi) > + kvm_cpu_cap_set(X86_FEATURE_AMD_VNMI); > + > /* Nested VM can receive #VMEXIT instead of triggering #GP */ > kvm_cpu_cap_set(X86_FEATURE_SVME_ADDR_CHK); > } > diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h > index 0b7e1790fadde1..8fb2085188c5ac 100644 > --- a/arch/x86/kvm/svm/svm.h > +++ b/arch/x86/kvm/svm/svm.h > @@ -271,6 +271,7 @@ struct vcpu_svm { > bool pause_filter_enabled : 1; > bool pause_threshold_enabled : 1; > bool vgif_enabled : 1; > + bool vnmi_enabled : 1; > > u32 ldr_reg; > u32 dfr_reg; > @@ -545,6 +546,11 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm) > return svm->nested.ctl.nested_ctl & SVM_NESTED_CTL_NP_ENABLE; > } > > +static inline bool nested_vnmi_enabled(struct vcpu_svm *svm) > +{ > + return svm->vnmi_enabled && (svm->nested.ctl.int_ctl & V_NMI_ENABLE); > +} > + > static inline bool is_x2apic_msrpm_offset(u32 offset) > { > /* 4 msrs per u8, and 4 u8 in u32 */
On Mon, 2022-12-05 at 22:44 +0530, Santosh Shukla wrote: > On 11/30/2022 1:07 AM, Maxim Levitsky wrote: > > This patch allows L1 to use vNMI to accelerate its injection > > of NMIs to L2 by passing through vNMI int_ctl bits from vmcb12 > > to/from vmcb02. > > > > While L2 runs, L1's vNMI is inhibited, and L1's NMIs are injected > > normally. > > > > In order to support nested VNMI requires saving and restoring the VNMI > > bits during nested entry and exit. > > In case of L1 and L2 both using VNMI- Copy VNMI bits from vmcb12 to > > vmcb02 during entry and vice-versa during exit. > > And in case of L1 uses VNMI and L2 doesn't- Copy VNMI bits from vmcb01 to > > vmcb02 during entry and vice-versa during exit. > > > > Tested with the KVM-unit-test and Nested Guest scenario. > > > > > > Signed-off-by: Santosh Shukla <santosh.shukla@amd.com> > > Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> > > --- > > arch/x86/kvm/svm/nested.c | 13 ++++++++++++- > > arch/x86/kvm/svm/svm.c | 5 +++++ > > arch/x86/kvm/svm/svm.h | 6 ++++++ > > 3 files changed, 23 insertions(+), 1 deletion(-) > > > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c > > index 5bea672bf8b12d..81346665058e26 100644 > > --- a/arch/x86/kvm/svm/nested.c > > +++ b/arch/x86/kvm/svm/nested.c > > @@ -278,6 +278,11 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu, > > if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl))) > > return false; > > > > + if (CC((control->int_ctl & V_NMI_ENABLE) && > > + !vmcb12_is_intercept(control, INTERCEPT_NMI))) { > > + return false; > > + } > > + > > return true; > > } > > > > @@ -427,6 +432,9 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm) > > if (nested_vgif_enabled(svm)) > > mask |= V_GIF_MASK; > > > > + if (nested_vnmi_enabled(svm)) > > + mask |= V_NMI_MASK | V_NMI_PENDING; > > + > > svm->nested.ctl.int_ctl &= ~mask; > > svm->nested.ctl.int_ctl |= svm->vmcb->control.int_ctl & mask; > > } > > @@ -682,8 +690,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, > > else > > int_ctl_vmcb01_bits |= (V_GIF_MASK | V_GIF_ENABLE_MASK); > > > > - if (vnmi) > > + if (vnmi) { > > To avoid above change, I think we should move nested bits from 10/11 to this i.e.. move function > (nested_svm_save_vnmi and nested_svm_restore_vnmi) to this patch. > > make sense? This is done on purpose: For each optional SVM feature there are two parts in regard to nesting. First part is the nesting co-existance, meaning that KVM should still work while a nested guest runs, and the second part is letting the nested hypervisor use the feature. First part is mandatory, as otherwise KVM will be broken while a nested guest runs. I squashed all of the vNMI support including nested co-existance in the patch 10, and that includes the 'nested_svm_save_vnmi' and 'nested_svm_restore_vnmi' Now this patch adds the actual nested vNMI, meaning that the nested hypervisor can use it to speed up the delivery of NMI, it would like to inject to L2. Best regards, Maxim Levitsky > > Thanks, > Santosh > > > nested_svm_save_vnmi(svm); > > + if (nested_vnmi_enabled(svm)) > > + int_ctl_vmcb12_bits |= (V_NMI_PENDING | V_NMI_ENABLE | V_NMI_MASK); > > + } > > > > /* Copied from vmcb01. msrpm_base can be overwritten later. */ > > vmcb02->control.nested_ctl = vmcb01->control.nested_ctl; > > diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c > > index bf10adcf3170a8..fb203f536d2f9b 100644 > > --- a/arch/x86/kvm/svm/svm.c > > +++ b/arch/x86/kvm/svm/svm.c > > @@ -4214,6 +4214,8 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) > > > > svm->vgif_enabled = vgif && guest_cpuid_has(vcpu, X86_FEATURE_VGIF); > > > > + svm->vnmi_enabled = vnmi && guest_cpuid_has(vcpu, X86_FEATURE_AMD_VNMI); > > + > > svm_recalc_instruction_intercepts(vcpu, svm); > > > > /* For sev guests, the memory encryption bit is not reserved in CR3. */ > > @@ -4967,6 +4969,9 @@ static __init void svm_set_cpu_caps(void) > > if (vgif) > > kvm_cpu_cap_set(X86_FEATURE_VGIF); > > > > + if (vnmi) > > + kvm_cpu_cap_set(X86_FEATURE_AMD_VNMI); > > + > > /* Nested VM can receive #VMEXIT instead of triggering #GP */ > > kvm_cpu_cap_set(X86_FEATURE_SVME_ADDR_CHK); > > } > > diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h > > index 0b7e1790fadde1..8fb2085188c5ac 100644 > > --- a/arch/x86/kvm/svm/svm.h > > +++ b/arch/x86/kvm/svm/svm.h > > @@ -271,6 +271,7 @@ struct vcpu_svm { > > bool pause_filter_enabled : 1; > > bool pause_threshold_enabled : 1; > > bool vgif_enabled : 1; > > + bool vnmi_enabled : 1; > > > > u32 ldr_reg; > > u32 dfr_reg; > > @@ -545,6 +546,11 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm) > > return svm->nested.ctl.nested_ctl & SVM_NESTED_CTL_NP_ENABLE; > > } > > > > +static inline bool nested_vnmi_enabled(struct vcpu_svm *svm) > > +{ > > + return svm->vnmi_enabled && (svm->nested.ctl.int_ctl & V_NMI_ENABLE); > > +} > > + > > static inline bool is_x2apic_msrpm_offset(u32 offset) > > { > > /* 4 msrs per u8, and 4 u8 in u32 */
On 12/6/2022 5:49 PM, Maxim Levitsky wrote: > On Mon, 2022-12-05 at 22:44 +0530, Santosh Shukla wrote: >> On 11/30/2022 1:07 AM, Maxim Levitsky wrote: >>> This patch allows L1 to use vNMI to accelerate its injection >>> of NMIs to L2 by passing through vNMI int_ctl bits from vmcb12 >>> to/from vmcb02. >>> >>> While L2 runs, L1's vNMI is inhibited, and L1's NMIs are injected >>> normally. >>> >>> In order to support nested VNMI requires saving and restoring the VNMI >>> bits during nested entry and exit. >>> In case of L1 and L2 both using VNMI- Copy VNMI bits from vmcb12 to >>> vmcb02 during entry and vice-versa during exit. >>> And in case of L1 uses VNMI and L2 doesn't- Copy VNMI bits from vmcb01 to >>> vmcb02 during entry and vice-versa during exit. >>> >>> Tested with the KVM-unit-test and Nested Guest scenario. >>> >>> >>> Signed-off-by: Santosh Shukla <santosh.shukla@amd.com> >>> Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> >>> --- >>> arch/x86/kvm/svm/nested.c | 13 ++++++++++++- >>> arch/x86/kvm/svm/svm.c | 5 +++++ >>> arch/x86/kvm/svm/svm.h | 6 ++++++ >>> 3 files changed, 23 insertions(+), 1 deletion(-) >>> >>> diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c >>> index 5bea672bf8b12d..81346665058e26 100644 >>> --- a/arch/x86/kvm/svm/nested.c >>> +++ b/arch/x86/kvm/svm/nested.c >>> @@ -278,6 +278,11 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu, >>> if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl))) >>> return false; >>> >>> + if (CC((control->int_ctl & V_NMI_ENABLE) && >>> + !vmcb12_is_intercept(control, INTERCEPT_NMI))) { >>> + return false; >>> + } >>> + >>> return true; >>> } >>> >>> @@ -427,6 +432,9 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm) >>> if (nested_vgif_enabled(svm)) >>> mask |= V_GIF_MASK; >>> >>> + if (nested_vnmi_enabled(svm)) >>> + mask |= V_NMI_MASK | V_NMI_PENDING; >>> + >>> svm->nested.ctl.int_ctl &= ~mask; >>> svm->nested.ctl.int_ctl |= svm->vmcb->control.int_ctl & mask; >>> } >>> @@ -682,8 +690,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, >>> else >>> int_ctl_vmcb01_bits |= (V_GIF_MASK | V_GIF_ENABLE_MASK); >>> >>> - if (vnmi) >>> + if (vnmi) { >> >> To avoid above change, I think we should move nested bits from 10/11 to this i.e.. move function >> (nested_svm_save_vnmi and nested_svm_restore_vnmi) to this patch. >> >> make sense? > > > This is done on purpose: > > For each optional SVM feature there are two parts in regard to nesting. > > First part is the nesting co-existance, meaning that KVM should still work > while a nested guest runs, and the second part is letting the nested hypervisor > use the feature. > > First part is mandatory, as otherwise KVM will be broken while a nested > guest runs. > Ok!,. > I squashed all of the vNMI support including nested co-existance in the patch 10, > and that includes the 'nested_svm_save_vnmi' and 'nested_svm_restore_vnmi' > > Now this patch adds the actual nested vNMI, meaning that the nested hypervisor can > use it to speed up the delivery of NMI, it would like to inject to L2. > Ok, Make sense to me. Thank-you for the explanation. Regards, Santosh > Best regards, > Maxim Levitsky > >> >> Thanks, >> Santosh >> >>> nested_svm_save_vnmi(svm); >>> + if (nested_vnmi_enabled(svm)) >>> + int_ctl_vmcb12_bits |= (V_NMI_PENDING | V_NMI_ENABLE | V_NMI_MASK); >>> + } >>> >>> /* Copied from vmcb01. msrpm_base can be overwritten later. */ >>> vmcb02->control.nested_ctl = vmcb01->control.nested_ctl; >>> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c >>> index bf10adcf3170a8..fb203f536d2f9b 100644 >>> --- a/arch/x86/kvm/svm/svm.c >>> +++ b/arch/x86/kvm/svm/svm.c >>> @@ -4214,6 +4214,8 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) >>> >>> svm->vgif_enabled = vgif && guest_cpuid_has(vcpu, X86_FEATURE_VGIF); >>> >>> + svm->vnmi_enabled = vnmi && guest_cpuid_has(vcpu, X86_FEATURE_AMD_VNMI); >>> + >>> svm_recalc_instruction_intercepts(vcpu, svm); >>> >>> /* For sev guests, the memory encryption bit is not reserved in CR3. */ >>> @@ -4967,6 +4969,9 @@ static __init void svm_set_cpu_caps(void) >>> if (vgif) >>> kvm_cpu_cap_set(X86_FEATURE_VGIF); >>> >>> + if (vnmi) >>> + kvm_cpu_cap_set(X86_FEATURE_AMD_VNMI); >>> + >>> /* Nested VM can receive #VMEXIT instead of triggering #GP */ >>> kvm_cpu_cap_set(X86_FEATURE_SVME_ADDR_CHK); >>> } >>> diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h >>> index 0b7e1790fadde1..8fb2085188c5ac 100644 >>> --- a/arch/x86/kvm/svm/svm.h >>> +++ b/arch/x86/kvm/svm/svm.h >>> @@ -271,6 +271,7 @@ struct vcpu_svm { >>> bool pause_filter_enabled : 1; >>> bool pause_threshold_enabled : 1; >>> bool vgif_enabled : 1; >>> + bool vnmi_enabled : 1; >>> >>> u32 ldr_reg; >>> u32 dfr_reg; >>> @@ -545,6 +546,11 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm) >>> return svm->nested.ctl.nested_ctl & SVM_NESTED_CTL_NP_ENABLE; >>> } >>> >>> +static inline bool nested_vnmi_enabled(struct vcpu_svm *svm) >>> +{ >>> + return svm->vnmi_enabled && (svm->nested.ctl.int_ctl & V_NMI_ENABLE); >>> +} >>> + >>> static inline bool is_x2apic_msrpm_offset(u32 offset) >>> { >>> /* 4 msrs per u8, and 4 u8 in u32 */ > >
On Tue, Nov 29, 2022, Maxim Levitsky wrote: > This patch allows L1 to use vNMI to accelerate its injection > of NMIs to L2 by passing through vNMI int_ctl bits from vmcb12 > to/from vmcb02. > > While L2 runs, L1's vNMI is inhibited, and L1's NMIs are injected > normally. Same feedback on stating the change as a command instead of describing the net effects. > In order to support nested VNMI requires saving and restoring the VNMI > bits during nested entry and exit. Again, avoid saving+restoring. And it's not just for terminology, it's not a true save/restore, e.g. a pending vNMI for L1 needs to be recognized and trigger a nested VM-Exit. I.e. KVM can't simply stash the state and restore it later, KVM needs to actively process the pending NMI. > In case of L1 and L2 both using VNMI- Copy VNMI bits from vmcb12 to > vmcb02 during entry and vice-versa during exit. > And in case of L1 uses VNMI and L2 doesn't- Copy VNMI bits from vmcb01 to > vmcb02 during entry and vice-versa during exit. > > Tested with the KVM-unit-test and Nested Guest scenario. > > > Signed-off-by: Santosh Shukla <santosh.shukla@amd.com> > Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Same SoB issues. > --- > arch/x86/kvm/svm/nested.c | 13 ++++++++++++- > arch/x86/kvm/svm/svm.c | 5 +++++ > arch/x86/kvm/svm/svm.h | 6 ++++++ > 3 files changed, 23 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c > index 5bea672bf8b12d..81346665058e26 100644 > --- a/arch/x86/kvm/svm/nested.c > +++ b/arch/x86/kvm/svm/nested.c > @@ -278,6 +278,11 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu, > if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl))) > return false; > > + if (CC((control->int_ctl & V_NMI_ENABLE) && > + !vmcb12_is_intercept(control, INTERCEPT_NMI))) { Align indentation. if (CC((control->int_ctl & V_NMI_ENABLE) && !vmcb12_is_intercept(control, INTERCEPT_NMI))) { return false; } > + return false; > + } > + > return true; > } > > diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h > index 0b7e1790fadde1..8fb2085188c5ac 100644 > --- a/arch/x86/kvm/svm/svm.h > +++ b/arch/x86/kvm/svm/svm.h > @@ -271,6 +271,7 @@ struct vcpu_svm { > bool pause_filter_enabled : 1; > bool pause_threshold_enabled : 1; > bool vgif_enabled : 1; > + bool vnmi_enabled : 1; > > u32 ldr_reg; > u32 dfr_reg; > @@ -545,6 +546,11 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm) > return svm->nested.ctl.nested_ctl & SVM_NESTED_CTL_NP_ENABLE; > } > > +static inline bool nested_vnmi_enabled(struct vcpu_svm *svm) > +{ > + return svm->vnmi_enabled && (svm->nested.ctl.int_ctl & V_NMI_ENABLE); Gah, the "nested" flags in vcpu_svm are super confusing. I initially read this as "if vNMI is enabled in L1 and vmcb12". I have a series that I originally prepped for the architectural LBRs series that will allow turning this into return guest_can_use(vcpu, X86_FEATURE_VNMI) && (svm->nested.ctl.int_ctl & V_NMI_ENABLE); I'll get that series posted. Nothing to do on your end, just an FYI. I'll sort out conflicts if/when they happen.
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 5bea672bf8b12d..81346665058e26 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -278,6 +278,11 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu, if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl))) return false; + if (CC((control->int_ctl & V_NMI_ENABLE) && + !vmcb12_is_intercept(control, INTERCEPT_NMI))) { + return false; + } + return true; } @@ -427,6 +432,9 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm) if (nested_vgif_enabled(svm)) mask |= V_GIF_MASK; + if (nested_vnmi_enabled(svm)) + mask |= V_NMI_MASK | V_NMI_PENDING; + svm->nested.ctl.int_ctl &= ~mask; svm->nested.ctl.int_ctl |= svm->vmcb->control.int_ctl & mask; } @@ -682,8 +690,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, else int_ctl_vmcb01_bits |= (V_GIF_MASK | V_GIF_ENABLE_MASK); - if (vnmi) + if (vnmi) { nested_svm_save_vnmi(svm); + if (nested_vnmi_enabled(svm)) + int_ctl_vmcb12_bits |= (V_NMI_PENDING | V_NMI_ENABLE | V_NMI_MASK); + } /* Copied from vmcb01. msrpm_base can be overwritten later. */ vmcb02->control.nested_ctl = vmcb01->control.nested_ctl; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index bf10adcf3170a8..fb203f536d2f9b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4214,6 +4214,8 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) svm->vgif_enabled = vgif && guest_cpuid_has(vcpu, X86_FEATURE_VGIF); + svm->vnmi_enabled = vnmi && guest_cpuid_has(vcpu, X86_FEATURE_AMD_VNMI); + svm_recalc_instruction_intercepts(vcpu, svm); /* For sev guests, the memory encryption bit is not reserved in CR3. */ @@ -4967,6 +4969,9 @@ static __init void svm_set_cpu_caps(void) if (vgif) kvm_cpu_cap_set(X86_FEATURE_VGIF); + if (vnmi) + kvm_cpu_cap_set(X86_FEATURE_AMD_VNMI); + /* Nested VM can receive #VMEXIT instead of triggering #GP */ kvm_cpu_cap_set(X86_FEATURE_SVME_ADDR_CHK); } diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 0b7e1790fadde1..8fb2085188c5ac 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -271,6 +271,7 @@ struct vcpu_svm { bool pause_filter_enabled : 1; bool pause_threshold_enabled : 1; bool vgif_enabled : 1; + bool vnmi_enabled : 1; u32 ldr_reg; u32 dfr_reg; @@ -545,6 +546,11 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm) return svm->nested.ctl.nested_ctl & SVM_NESTED_CTL_NP_ENABLE; } +static inline bool nested_vnmi_enabled(struct vcpu_svm *svm) +{ + return svm->vnmi_enabled && (svm->nested.ctl.int_ctl & V_NMI_ENABLE); +} + static inline bool is_x2apic_msrpm_offset(u32 offset) { /* 4 msrs per u8, and 4 u8 in u32 */