From patchwork Fri Sep 22 15:00:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 143727 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp5936033vqi; Fri, 22 Sep 2023 16:57:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHuFh0xvhLUV8CevYoEwXEhWm+w664KltbPqbA6NB0x+EVyDvzRNQSXXwPkpio4ufPkzQJ6 X-Received: by 2002:a17:902:7049:b0:1c5:cd1c:46fa with SMTP id h9-20020a170902704900b001c5cd1c46famr838879plt.36.1695427068994; Fri, 22 Sep 2023 16:57:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695427068; cv=none; d=google.com; s=arc-20160816; b=HNZGHv1sYu/Iw9LjSEXyYhcnVud+tgEXNumUnp4FvMzQecAo3ockd0FLIz6GuUHLlz Gpi1lgJZwry6h4ZEdHohbIzGdH2544SSQWeg3el52yDkC/9oTFpL+S4KZAqx46A3phPD thDr7xZABAYLNsCuW2WMhcUPkISd+1WU/Wj+dzR1+Q1K6Wp41ZyAZBUL0kdkZxG1bGXf +hokXJn1SL7sPR1CSAJSTSY3tpn0jmif4B+ui6Ccf8ePy7UT8dEYa4WMfOCq0M10c6tO di27iurmZX6oY3908B6r+FgK+iyFb8u9ACA1gp7QvP2vPIYr2+52YlVKaemE/bTlpI8M elmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=7Xg3VsT+niHE2xRvzgMpdBgGz0Csn+Ig0JZVwwD1HsY=; fh=8AvbwpPKF0Qe5WwS5eyABOVDUChKGxIrkPaOnjrOqKo=; b=ndNTv9GVAAL/TtGa8frqA/hmGvhp4iIYYaPTtwd5T7No5rUX+lVFs0EYwCIxp51r5M tJXsBJdjA6Z8q5r32odpXBtOu21YgHhFSAqXUG1wHZ4fL7N/APQgdWLtPvzDs5L74DHe 0d1p9VN5AoSfP6H+DE0lPYXtTepnHaU3gduYzFJ6T24jt8rcb8WBQUCWaNIrGBIFVdC0 MLXR6T2GwCXUri0bEilzDVCru8AkiAyB+0Q5e5frSEUbMSsXjWMNhxgGfvMA6FN9zvZ2 s56/ZSEQ6S06P6/wIpfzZQql/1AJ7tGb1Kh9CyfXqDu4Neym9LEy5ElUSVQAZ/l4nnlt LbNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@xen.org header.s=20200302mail header.b=q2fnifBG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id l16-20020a170902f69000b001c5e0672f53si3623164plg.466.2023.09.22.16.57.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 16:57:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@xen.org header.s=20200302mail header.b=q2fnifBG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 38373822E565; Fri, 22 Sep 2023 08:02:09 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232624AbjIVPBD (ORCPT + 29 others); Fri, 22 Sep 2023 11:01:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232327AbjIVPAk (ORCPT ); Fri, 22 Sep 2023 11:00:40 -0400 Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E95F194; Fri, 22 Sep 2023 08:00:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:Content-Type:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=7Xg3VsT+niHE2xRvzgMpdBgGz0Csn+Ig0JZVwwD1HsY=; b=q2fnifBGofysyROgqfxrNhFc+T HPOPa4mspgeB7R2y7jM7SmMA9nwm9TO7b/RdaTGQvFZ7/RChzUyUvB+VbHVEg7o1QAlHp5g73xNj2 xkgoG7nKDyl7Et8Qcn5wemKrV9zieH3IhAc1JOIIJlu0ZIy2pz5SL8RjmSii4txgGFtg=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qjhdt-000451-La; Fri, 22 Sep 2023 15:00:29 +0000 Received: from ec2-63-33-11-17.eu-west-1.compute.amazonaws.com ([63.33.11.17] helo=REM-PW02S00X.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qjhdt-0005y2-Dh; Fri, 22 Sep 2023 15:00:29 +0000 From: Paul Durrant To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paul Durrant , David Woodhouse , David Woodhouse , Sean Christopherson , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH v5 06/10] KVM: xen: allow shared_info to be mapped by fixed HVA Date: Fri, 22 Sep 2023 15:00:05 +0000 Message-Id: <20230922150009.3319-7-paul@xen.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230922150009.3319-1-paul@xen.org> References: <20230922150009.3319-1-paul@xen.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Fri, 22 Sep 2023 08:02:09 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777784134262774930 X-GMAIL-MSGID: 1777784134262774930 From: Paul Durrant The shared_info page is not guest memory as such. It is a dedicated page allocated by the VMM and overlaid onto guest memory in a GFN chosen by the guest. The guest may even request that shared_info be moved from one GFN to another, but the HVA is never going to change. Thus it makes much more sense to map the shared_info page in kernel once using this fixed HVA. Hence add a new KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA attribute type for this purpose and a KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA flag to advertize its availability. Don't actually advertize it yet though. That will be done in a subsequent patch, which will also add tests for the new attribute type. Also update the KVM API documentation with the new attribute and also fix it up to consistently refer to 'shared_info' (with the underscore). NOTE: The change of the kvm_xen_hvm_attr shared_info from struct to union is technically an ABI change but it's entirely compatible with existing users. Signed-off-by: Paul Durrant Reviewed-by: David Woodhouse --- Cc: David Woodhouse Cc: Sean Christopherson Cc: Paolo Bonzini Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: "H. Peter Anvin" Cc: x86@kernel.org v2: - Define the new attribute and capability but don't advertize the capability yet. - Add API documentation. --- Documentation/virt/kvm/api.rst | 25 +++++++++++++++++++------ arch/x86/kvm/xen.c | 28 ++++++++++++++++++++++------ include/uapi/linux/kvm.h | 6 +++++- 3 files changed, 46 insertions(+), 13 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 21a7578142a1..e9df4df6fe48 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -353,7 +353,7 @@ The bits in the dirty bitmap are cleared before the ioctl returns, unless KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 is enabled. For more information, see the description of the capability. -Note that the Xen shared info page, if configured, shall always be assumed +Note that the Xen shared_info page, if configured, shall always be assumed to be dirty. KVM will not explicitly mark it such. @@ -5408,8 +5408,9 @@ KVM_PV_ASYNC_CLEANUP_PERFORM __u8 long_mode; __u8 vector; __u8 runstate_update_flag; - struct { + union { __u64 gfn; + __u64 hva; } shared_info; struct { __u32 send_port; @@ -5437,10 +5438,10 @@ type values: KVM_XEN_ATTR_TYPE_LONG_MODE Sets the ABI mode of the VM to 32-bit or 64-bit (long mode). This - determines the layout of the shared info pages exposed to the VM. + determines the layout of the shared_info page exposed to the VM. KVM_XEN_ATTR_TYPE_SHARED_INFO - Sets the guest physical frame number at which the Xen "shared info" + Sets the guest physical frame number at which the Xen shared_info page resides. Note that although Xen places vcpu_info for the first 32 vCPUs in the shared_info page, KVM does not automatically do so and instead requires that KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO be used @@ -5449,7 +5450,7 @@ KVM_XEN_ATTR_TYPE_SHARED_INFO not be aware of the Xen CPU id which is used as the index into the vcpu_info[] array, so may know the correct default location. - Note that the shared info page may be constantly written to by KVM; + Note that the shared_info page may be constantly written to by KVM; it contains the event channel bitmap used to deliver interrupts to a Xen guest, amongst other things. It is exempt from dirty tracking mechanisms — KVM will not explicitly mark the page as dirty each @@ -5458,9 +5459,21 @@ KVM_XEN_ATTR_TYPE_SHARED_INFO any vCPU has been running or any event channel interrupts can be routed to the guest. - Setting the gfn to KVM_XEN_INVALID_GFN will disable the shared info + Setting the gfn to KVM_XEN_INVALID_GFN will disable the shared_info page. +KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA + If the KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA flag is also set in the + Xen capabilities, then this attribute may be used to set the + userspace address at which the shared_info page resides, which + will always be fixed in the VMM regardless of where it is mapped + in guest physical address space. This attribute should be used in + preference to KVM_XEN_ATTR_TYPE_SHARED_INFO as it avoids + unnecessary invalidation of an internal cache when the page is + re-mapped in guest physcial address space. + + Setting the hva to zero will disable the shared_info page. + KVM_XEN_ATTR_TYPE_UPCALL_VECTOR Sets the exception vector used to deliver Xen event channel upcalls. This is the HVM-wide vector injected directly by the hypervisor diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 8e6fdcd7bb6e..1abb4547642a 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -34,24 +34,27 @@ static bool kvm_xen_hcall_evtchn_send(struct kvm_vcpu *vcpu, u64 param, u64 *r); DEFINE_STATIC_KEY_DEFERRED_FALSE(kvm_xen_enabled, HZ); -static int kvm_xen_shared_info_init(struct kvm *kvm, gfn_t gfn) +static int kvm_xen_shared_info_init(struct kvm *kvm, u64 addr, bool addr_is_gfn) { struct gfn_to_pfn_cache *gpc = &kvm->arch.xen.shinfo_cache; struct pvclock_wall_clock *wc; - gpa_t gpa = gfn_to_gpa(gfn); u32 *wc_sec_hi; u32 wc_version; u64 wall_nsec; int ret = 0; int idx = srcu_read_lock(&kvm->srcu); - if (gfn == KVM_XEN_INVALID_GFN) { + if ((addr_is_gfn && addr == KVM_XEN_INVALID_GFN) || + (!addr_is_gfn && addr == 0)) { kvm_gpc_deactivate(gpc); goto out; } do { - ret = kvm_gpc_activate(gpc, gpa, PAGE_SIZE); + if (addr_is_gfn) + ret = kvm_gpc_activate(gpc, gfn_to_gpa(addr), PAGE_SIZE); + else + ret = kvm_gpc_activate_hva(gpc, addr, PAGE_SIZE); if (ret) goto out; @@ -604,7 +607,6 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) { int r = -ENOENT; - switch (data->type) { case KVM_XEN_ATTR_TYPE_LONG_MODE: if (!IS_ENABLED(CONFIG_64BIT) && data->u.long_mode) { @@ -619,7 +621,13 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) case KVM_XEN_ATTR_TYPE_SHARED_INFO: mutex_lock(&kvm->arch.xen.xen_lock); - r = kvm_xen_shared_info_init(kvm, data->u.shared_info.gfn); + r = kvm_xen_shared_info_init(kvm, data->u.shared_info.gfn, true); + mutex_unlock(&kvm->arch.xen.xen_lock); + break; + + case KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA: + mutex_lock(&kvm->arch.xen.xen_lock); + r = kvm_xen_shared_info_init(kvm, data->u.shared_info.hva, false); mutex_unlock(&kvm->arch.xen.xen_lock); break; @@ -684,6 +692,14 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r = 0; break; + case KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA: + if (kvm->arch.xen.shinfo_cache.active) + data->u.shared_info.hva = kvm_gpc_hva(&kvm->arch.xen.shinfo_cache); + else + data->u.shared_info.hva = 0; + r = 0; + break; + case KVM_XEN_ATTR_TYPE_UPCALL_VECTOR: data->u.vector = kvm->arch.xen.upcall_vector; r = 0; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 13065dd96132..062bfa14b4d9 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1282,6 +1282,7 @@ struct kvm_x86_mce { #define KVM_XEN_HVM_CONFIG_EVTCHN_2LEVEL (1 << 4) #define KVM_XEN_HVM_CONFIG_EVTCHN_SEND (1 << 5) #define KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG (1 << 6) +#define KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA (1 << 7) struct kvm_xen_hvm_config { __u32 flags; @@ -1793,9 +1794,10 @@ struct kvm_xen_hvm_attr { __u8 long_mode; __u8 vector; __u8 runstate_update_flag; - struct { + union { __u64 gfn; #define KVM_XEN_INVALID_GFN ((__u64)-1) + __u64 hva; } shared_info; struct { __u32 send_port; @@ -1837,6 +1839,8 @@ struct kvm_xen_hvm_attr { #define KVM_XEN_ATTR_TYPE_XEN_VERSION 0x4 /* Available with KVM_CAP_XEN_HVM / KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG */ #define KVM_XEN_ATTR_TYPE_RUNSTATE_UPDATE_FLAG 0x5 +/* Available with KVM_CAP_XEN_HVM / KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA */ +#define KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA 0x6 /* Per-vCPU Xen attributes */ #define KVM_XEN_VCPU_GET_ATTR _IOWR(KVMIO, 0xca, struct kvm_xen_vcpu_attr)