[RFC,02/11] KVM: x86: hyper-v: Move Hyper-V partition assist page out of Hyper-V emulation context
Message ID | 20231010160300.1136799-3-vkuznets@redhat.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp310305vqb; Tue, 10 Oct 2023 09:04:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFXIY99sMc2ifAWqLK1fjSNhM4T42oOROpZhuWxfPiifwR6IQ9F3bVMyvpH2ESePY9koRSk X-Received: by 2002:a17:902:c404:b0:1c9:9fa6:ce5b with SMTP id k4-20020a170902c40400b001c99fa6ce5bmr8250195plk.16.1696953882721; Tue, 10 Oct 2023 09:04:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696953882; cv=none; d=google.com; s=arc-20160816; b=SvWlhKzdaoNO+YxzgXIXxf6d22Pyu/gAduHdWSnA+8wdlG42SCo/KHDYQvRovuF21q CY5diUmPbuXb4tWfBWqeteR63FTeTSPg80WyCwpN4wpV8bprhDQ2teaPGxTWcYcRzua9 WIUNrG+KIt5Mz/o3eS5FCzk01IWGwJE17pygW1tBYoxxd14X2cG4Thir8wDJLwW8v/MB NAx7ESqEL8OcV7rauT+6K7DEwmSxv+qzIddFhyyZa9v3/3+LbnT0bEoAtlqRCtDcpfy7 S4BoGSK1LYGl4bo2iaJQCpAPoGWvhu8EgUMYNyjTzMHlWJ5yHIDAjDB7qyHf4H3DmlkX CONw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xyOFdvwjZ+xxDeHG2VVNVrJrkXBiFcGsu54eo0pRt4I=; fh=SJRVlyvhIPEM+1ZkklEUy6ba01ZQIQ4PgkK8h+im/FM=; b=0Y6OoqHhn8T/CeGp/46Ahl2Iy757Fy4lFwOtT2n/nm2L2+2KBQLniqJkE/NRUaTooa UYIqDxf9IP9x1v6HDgttk+H30o1ib1hcJ7yMmSPk98I/7a6lRvvZon332Bfwk06j0uoH WCv8AixwfxPCtOGF5h3uctNe8MMiz7mQinHhESjzSklwiTRy3ZVCvhPmxGVgqTqqvMgp PKENlASfptSUqKd55VoWDW5dqeuD7Xpr6YWokEha2gLDevG229yjOApO0x11xbfmBuGr wSFrtqWayYyhJIF8jmQlRCZW+mNwKKPYgoXvmmIhRielC570dNhTJBMCj/HadrLxRWyk Cm5w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=RXw2s1y7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id r2-20020a170902c60200b001c3ea2bbebcsi11254566plr.322.2023.10.10.09.04.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Oct 2023 09:04:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=RXw2s1y7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 9B92882DAFE5; Tue, 10 Oct 2023 09:04:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233889AbjJJQER (ORCPT <rfc822;rua109.linux@gmail.com> + 20 others); Tue, 10 Oct 2023 12:04:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233767AbjJJQEF (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 10 Oct 2023 12:04:05 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB187DE for <linux-kernel@vger.kernel.org>; Tue, 10 Oct 2023 09:03:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696953791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xyOFdvwjZ+xxDeHG2VVNVrJrkXBiFcGsu54eo0pRt4I=; b=RXw2s1y7K17Kg5wrktRwrj+QoiZevHTomdoN59k/Ww2gl9+4Fg/WCpRisgfAT/u7tRSJMj yvZcx7xNi8plvPWoe2mI9JvjqmDH+VU9OwMQ3d5NnwY+dwCoyQ1TQL8sd8cMSq3rQOb4Fq mX3sZeJfFwtKXa50CxGaARFkJzcMDko= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-150-YROgcB0TOFG-LQf36gX9NA-1; Tue, 10 Oct 2023 12:03:04 -0400 X-MC-Unique: YROgcB0TOFG-LQf36gX9NA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5A72D1019C88; Tue, 10 Oct 2023 16:03:04 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8AEFF1B94; Tue, 10 Oct 2023 16:03:03 +0000 (UTC) From: Vitaly Kuznetsov <vkuznets@redhat.com> To: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>, Sean Christopherson <seanjc@google.com> Cc: linux-kernel@vger.kernel.org Subject: [PATCH RFC 02/11] KVM: x86: hyper-v: Move Hyper-V partition assist page out of Hyper-V emulation context Date: Tue, 10 Oct 2023 18:02:51 +0200 Message-ID: <20231010160300.1136799-3-vkuznets@redhat.com> In-Reply-To: <20231010160300.1136799-1-vkuznets@redhat.com> References: <20231010160300.1136799-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 10 Oct 2023 09:04:37 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779385114592929717 X-GMAIL-MSGID: 1779385114592929717 |
Series |
KVM: x86: Make Hyper-V emulation optional (AKA introduce CONFIG_KVM_HYPERV)
|
|
Commit Message
Vitaly Kuznetsov
Oct. 10, 2023, 4:02 p.m. UTC
Hyper-V partition assist page is used when KVM runs on top of Hyper-V and
is not used for Windows/Hyper-V guests on KVM, this means that 'hv_pa_pg'
placement in 'struct kvm_hv' is unfortunate. As a preparation to making
Hyper-V emulation optional, move 'hv_pa_pg' to 'struct kvm_arch' and put it
under CONFIG_HYPERV.
No functional change intended.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/svm/svm_onhyperv.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 2 +-
arch/x86/kvm/x86.c | 4 +++-
4 files changed, 6 insertions(+), 4 deletions(-)
Comments
У вт, 2023-10-10 у 18:02 +0200, Vitaly Kuznetsov пише: > Hyper-V partition assist page is used when KVM runs on top of Hyper-V and > is not used for Windows/Hyper-V guests on KVM, this means that 'hv_pa_pg' > placement in 'struct kvm_hv' is unfortunate. As a preparation to making > Hyper-V emulation optional, move 'hv_pa_pg' to 'struct kvm_arch' and put it > under CONFIG_HYPERV. It took me a while to realize that this parition assist page is indeed something that L0, running above KVM consumes. (what a confusing name Microsoft picked...) As far as I know currently the partition assist page has only one shared memory variable which allows L1 to be notified of direct TLB flushes that L0 does for L2, but since KVM doesn't need it, it never touches this variable/page, but specs still demand that L1 does allocate that page. If you agree, it would be great to add a large comment to the code, explaining the above, and fact that the partition assist page is something L1 exposes to L0. I don't know though where to put the comment because hv_enable_l2_tlb_flush is duplicated between SVM and VMX. It might be a good idea to have a helper function to allocate the partition assist page, which will both reduce the code duplication slightly and allow us to put this comment there. Best regards, Maxim Levitsky > > No functional change intended. > > Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com> > --- > arch/x86/include/asm/kvm_host.h | 2 +- > arch/x86/kvm/svm/svm_onhyperv.c | 2 +- > arch/x86/kvm/vmx/vmx.c | 2 +- > arch/x86/kvm/x86.c | 4 +++- > 4 files changed, 6 insertions(+), 4 deletions(-) > > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h > index e5d4b8a44630..711dc880a9f0 100644 > --- a/arch/x86/include/asm/kvm_host.h > +++ b/arch/x86/include/asm/kvm_host.h > @@ -1115,7 +1115,6 @@ struct kvm_hv { > */ > unsigned int synic_auto_eoi_used; > > - struct hv_partition_assist_pg *hv_pa_pg; > struct kvm_hv_syndbg hv_syndbg; > }; > > @@ -1436,6 +1435,7 @@ struct kvm_arch { > #if IS_ENABLED(CONFIG_HYPERV) > hpa_t hv_root_tdp; > spinlock_t hv_root_tdp_lock; > + struct hv_partition_assist_pg *hv_pa_pg; > #endif > /* > * VM-scope maximum vCPU ID. Used to determine the size of structures > diff --git a/arch/x86/kvm/svm/svm_onhyperv.c b/arch/x86/kvm/svm/svm_onhyperv.c > index 7af8422d3382..d19666f9b9ac 100644 > --- a/arch/x86/kvm/svm/svm_onhyperv.c > +++ b/arch/x86/kvm/svm/svm_onhyperv.c > @@ -19,7 +19,7 @@ int svm_hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) > { > struct hv_vmcb_enlightenments *hve; > struct hv_partition_assist_pg **p_hv_pa_pg = > - &to_kvm_hv(vcpu->kvm)->hv_pa_pg; > + &vcpu->kvm->arch.hv_pa_pg; > > if (!*p_hv_pa_pg) > *p_hv_pa_pg = kzalloc(PAGE_SIZE, GFP_KERNEL); > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > index 72e3943f3693..b7dc7acf14be 100644 > --- a/arch/x86/kvm/vmx/vmx.c > +++ b/arch/x86/kvm/vmx/vmx.c > @@ -524,7 +524,7 @@ static int hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) > { > struct hv_enlightened_vmcs *evmcs; > struct hv_partition_assist_pg **p_hv_pa_pg = > - &to_kvm_hv(vcpu->kvm)->hv_pa_pg; > + &vcpu->kvm->arch.hv_pa_pg; > /* > * Synthetic VM-Exit is not enabled in current code and so All > * evmcs in singe VM shares same assist page. > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 9f18b06bbda6..e273ce8e0b3f 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -12291,7 +12291,9 @@ void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) > > void kvm_arch_free_vm(struct kvm *kvm) > { > - kfree(to_kvm_hv(kvm)->hv_pa_pg); > +#if IS_ENABLED(CONFIG_HYPERV) > + kfree(kvm->arch.hv_pa_pg); > +#endif > __kvm_arch_free_vm(kvm); > } >
Maxim Levitsky <mlevitsk@redhat.com> writes: > У вт, 2023-10-10 у 18:02 +0200, Vitaly Kuznetsov пише: >> Hyper-V partition assist page is used when KVM runs on top of Hyper-V and >> is not used for Windows/Hyper-V guests on KVM, this means that 'hv_pa_pg' >> placement in 'struct kvm_hv' is unfortunate. As a preparation to making >> Hyper-V emulation optional, move 'hv_pa_pg' to 'struct kvm_arch' and put it >> under CONFIG_HYPERV. > > It took me a while to realize that this parition assist page is indeed something that L0, > running above KVM consumes. > (what a confusing name Microsoft picked...) > > As far as I know currently the partition assist page has only > one shared memory variable which allows L1 to be notified of direct TLB flushes that L0 does for L2, > but since KVM doesn't need it, it > never touches this variable/page, > but specs still demand that L1 does allocate that page. > Yes, KVM doesn't ask L0 (Hyper-V) to deliver synthetic vmexits but the page needs to be allocated. I'm not sure whether this is done to follow the spec ("The partition assist page is a page-size aligned page-size region of memory that the L1 hypervisor must allocate and zero before direct flush hypercalls can be used.") or if anyone has ever tried writing '0' to the corresponding field to see what happens with various Hyper-V versions but even if it happens to work today, there's no guarantee for the future. > > If you agree, it would be great to add a large comment to the code, > explaining the above, There' this in vmx.c: /* * Synthetic VM-Exit is not enabled in current code and so All * evmcs in singe VM shares same assist page. */ but this can certainly get extended. Moreover, it seems that hv_enable_l2_tlb_flush() should go vmx_onhyperv.c to make that fact that it's for KVM-on-Hyper-V 'more obvious'. > and fact that the partition assist page > is something L1 exposes to L0. > > I don't know though where to put the comment > because hv_enable_l2_tlb_flush is duplicated between SVM and VMX. > > It might be a good idea to have a helper function to allocate the partition assist page, > which will both reduce the code duplication slightly and allow us to > put this comment there. OK. > > > Best regards, > Maxim Levitsky > >> >> No functional change intended. >> >> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com> >> --- >> arch/x86/include/asm/kvm_host.h | 2 +- >> arch/x86/kvm/svm/svm_onhyperv.c | 2 +- >> arch/x86/kvm/vmx/vmx.c | 2 +- >> arch/x86/kvm/x86.c | 4 +++- >> 4 files changed, 6 insertions(+), 4 deletions(-) >> >> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h >> index e5d4b8a44630..711dc880a9f0 100644 >> --- a/arch/x86/include/asm/kvm_host.h >> +++ b/arch/x86/include/asm/kvm_host.h >> @@ -1115,7 +1115,6 @@ struct kvm_hv { >> */ >> unsigned int synic_auto_eoi_used; >> >> - struct hv_partition_assist_pg *hv_pa_pg; >> struct kvm_hv_syndbg hv_syndbg; >> }; >> >> @@ -1436,6 +1435,7 @@ struct kvm_arch { >> #if IS_ENABLED(CONFIG_HYPERV) >> hpa_t hv_root_tdp; >> spinlock_t hv_root_tdp_lock; >> + struct hv_partition_assist_pg *hv_pa_pg; >> #endif >> /* >> * VM-scope maximum vCPU ID. Used to determine the size of structures >> diff --git a/arch/x86/kvm/svm/svm_onhyperv.c b/arch/x86/kvm/svm/svm_onhyperv.c >> index 7af8422d3382..d19666f9b9ac 100644 >> --- a/arch/x86/kvm/svm/svm_onhyperv.c >> +++ b/arch/x86/kvm/svm/svm_onhyperv.c >> @@ -19,7 +19,7 @@ int svm_hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) >> { >> struct hv_vmcb_enlightenments *hve; >> struct hv_partition_assist_pg **p_hv_pa_pg = >> - &to_kvm_hv(vcpu->kvm)->hv_pa_pg; >> + &vcpu->kvm->arch.hv_pa_pg; >> >> if (!*p_hv_pa_pg) >> *p_hv_pa_pg = kzalloc(PAGE_SIZE, GFP_KERNEL); >> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c >> index 72e3943f3693..b7dc7acf14be 100644 >> --- a/arch/x86/kvm/vmx/vmx.c >> +++ b/arch/x86/kvm/vmx/vmx.c >> @@ -524,7 +524,7 @@ static int hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) >> { >> struct hv_enlightened_vmcs *evmcs; >> struct hv_partition_assist_pg **p_hv_pa_pg = >> - &to_kvm_hv(vcpu->kvm)->hv_pa_pg; >> + &vcpu->kvm->arch.hv_pa_pg; >> /* >> * Synthetic VM-Exit is not enabled in current code and so All >> * evmcs in singe VM shares same assist page. >> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >> index 9f18b06bbda6..e273ce8e0b3f 100644 >> --- a/arch/x86/kvm/x86.c >> +++ b/arch/x86/kvm/x86.c >> @@ -12291,7 +12291,9 @@ void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) >> >> void kvm_arch_free_vm(struct kvm *kvm) >> { >> - kfree(to_kvm_hv(kvm)->hv_pa_pg); >> +#if IS_ENABLED(CONFIG_HYPERV) >> + kfree(kvm->arch.hv_pa_pg); >> +#endif >> __kvm_arch_free_vm(kvm); >> } >> > > > >
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e5d4b8a44630..711dc880a9f0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1115,7 +1115,6 @@ struct kvm_hv { */ unsigned int synic_auto_eoi_used; - struct hv_partition_assist_pg *hv_pa_pg; struct kvm_hv_syndbg hv_syndbg; }; @@ -1436,6 +1435,7 @@ struct kvm_arch { #if IS_ENABLED(CONFIG_HYPERV) hpa_t hv_root_tdp; spinlock_t hv_root_tdp_lock; + struct hv_partition_assist_pg *hv_pa_pg; #endif /* * VM-scope maximum vCPU ID. Used to determine the size of structures diff --git a/arch/x86/kvm/svm/svm_onhyperv.c b/arch/x86/kvm/svm/svm_onhyperv.c index 7af8422d3382..d19666f9b9ac 100644 --- a/arch/x86/kvm/svm/svm_onhyperv.c +++ b/arch/x86/kvm/svm/svm_onhyperv.c @@ -19,7 +19,7 @@ int svm_hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) { struct hv_vmcb_enlightenments *hve; struct hv_partition_assist_pg **p_hv_pa_pg = - &to_kvm_hv(vcpu->kvm)->hv_pa_pg; + &vcpu->kvm->arch.hv_pa_pg; if (!*p_hv_pa_pg) *p_hv_pa_pg = kzalloc(PAGE_SIZE, GFP_KERNEL); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 72e3943f3693..b7dc7acf14be 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -524,7 +524,7 @@ static int hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) { struct hv_enlightened_vmcs *evmcs; struct hv_partition_assist_pg **p_hv_pa_pg = - &to_kvm_hv(vcpu->kvm)->hv_pa_pg; + &vcpu->kvm->arch.hv_pa_pg; /* * Synthetic VM-Exit is not enabled in current code and so All * evmcs in singe VM shares same assist page. diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9f18b06bbda6..e273ce8e0b3f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12291,7 +12291,9 @@ void kvm_arch_sched_in(struct kvm_vcpu *vcpu, int cpu) void kvm_arch_free_vm(struct kvm *kvm) { - kfree(to_kvm_hv(kvm)->hv_pa_pg); +#if IS_ENABLED(CONFIG_HYPERV) + kfree(kvm->arch.hv_pa_pg); +#endif __kvm_arch_free_vm(kvm); }