From patchwork Fri Sep 22 15:00:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 143500 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp5652088vqi; Fri, 22 Sep 2023 08:09:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF+H7KCkco4IjdXh3ntSj873mSc4eTi0uRWxQdfEBNBqQvQS4eBWwl9y2LEh4byxmMndogk X-Received: by 2002:a17:90a:888:b0:274:78:4f0e with SMTP id v8-20020a17090a088800b0027400784f0emr8563761pjc.20.1695395390048; Fri, 22 Sep 2023 08:09:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695395390; cv=none; d=google.com; s=arc-20160816; b=jTx84fMLPsagWRPCeh3AHsauH/ads+miPXlrauo+SkiSSC2wQzDQoslX6SWh/VChbx aka7QTZOZM1dfH4qN/QBKr3fDl37+QSwx9W98Ik0g/e1IcbZbmYTH5Er7XnPKIWGhNN/ /gld4gEnOAP2DeKRYE+QRf7B/WchRdR7oGtrI2oYzmFtKToYJ2wnBIRjwnJ3En9oN+yk mB/z2lG9MWGHrvtQhES0xhdEdKEP+kyzwkmi1fUJ5fT2Tqe+sF+xLiX0dp2Nt01Edk2V PhA7nEqti0FwF0sUeIv3OOqaP4IZRcXTJF1eAVk3mdoTT5dMquD9dTFwNLpWan01QD0C Ez+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uR3RY44FOYUdZ/yYeXcGIkRXrRmwuyl6HvHad0MhhqE=; fh=lKhNn3P8ZvksS1fElybz1iP8zlw6JVVQ9gjuN2JwMdc=; b=wjFDWlhNOKhzqq4810usJBZgeNsOGMsXoN8MuFXMgYDi5LEJ3AlMVYuhk1RmNwRyDs s/SnE7KdD6BwxDRFjAcuVRaVWE8iJm2StVn8JADwWYOP1uADli5EWhQlvNAwt7gkX4JA DBR88CUK+jU6FUFbtMv154E9krmPAeLeNSkeCSPTKqBnzquGt+I4ENo5eW5sXFNyhAGt Hi60qMRnmlDRbfiZVT+qY/fWmWAa3M+eG8zzcM7f4VSBdGmVediX3wOIXlQDn0QZUYw2 75e/wAkMFG7KoFhKVePm2lvD4ISKkvl4YVE0dHbs1QpKCUO1/uikbaVAP3Nu+NBwYQwW 0H8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@xen.org header.s=20200302mail header.b=vD0eENGb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id h13-20020a17090acf0d00b00263fa42fdcbsi6332509pju.107.2023.09.22.08.09.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 08:09:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@xen.org header.s=20200302mail header.b=vD0eENGb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 3861A80C2E16; Fri, 22 Sep 2023 08:01:32 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232605AbjIVPAy (ORCPT + 29 others); Fri, 22 Sep 2023 11:00:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232297AbjIVPAi (ORCPT ); Fri, 22 Sep 2023 11:00:38 -0400 Received: from mail.xenproject.org (mail.xenproject.org [104.130.215.37]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C16119B; Fri, 22 Sep 2023 08:00:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=xen.org; s=20200302mail; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=uR3RY44FOYUdZ/yYeXcGIkRXrRmwuyl6HvHad0MhhqE=; b=vD0eENGb99cUaUCNbJj0KzFlDU /ZYbDr5kGnibuiGqh3JpTpQuBRYw+tQKYPWqNMsCPw8U8OCAbdVFN+7dpSOOc+Z/iPrPb7AswuhhD 4zgkuyiUZ/p2yk0X/DeYzKHIuF8Sn6XCgsd7HU1oyyZcvcmCEcqLppNm1IkOYt+Pfx5o=; Received: from xenbits.xenproject.org ([104.239.192.120]) by mail.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qjhdr-00044n-Ve; Fri, 22 Sep 2023 15:00:27 +0000 Received: from ec2-63-33-11-17.eu-west-1.compute.amazonaws.com ([63.33.11.17] helo=REM-PW02S00X.ant.amazon.com) by xenbits.xenproject.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qjhdr-0005y2-Ny; Fri, 22 Sep 2023 15:00:27 +0000 From: Paul Durrant To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paul Durrant , David Woodhouse , Sean Christopherson , Paolo Bonzini , David Woodhouse Subject: [PATCH v5 05/10] KVM: pfncache: allow a cache to be activated with a fixed (userspace) HVA Date: Fri, 22 Sep 2023 15:00:04 +0000 Message-Id: <20230922150009.3319-6-paul@xen.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230922150009.3319-1-paul@xen.org> References: <20230922150009.3319-1-paul@xen.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Fri, 22 Sep 2023 08:01:32 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777750916725463597 X-GMAIL-MSGID: 1777750916725463597 From: Paul Durrant Some cached pages may actually be overlays on guest memory that have a fixed HVA within the VMM. It's pointless to invalidate such cached mappings if the overlay is moved so allow a cache to be activated directly with the HVA to cater for such cases. A subsequent patch will make use of this facility. Signed-off-by: Paul Durrant Reviewed-by: David Woodhouse --- Cc: Sean Christopherson Cc: Paolo Bonzini Cc: David Woodhouse --- include/linux/kvm_host.h | 29 ++++++++++++++++ include/linux/kvm_types.h | 3 +- virt/kvm/pfncache.c | 73 ++++++++++++++++++++++++++++----------- 3 files changed, 84 insertions(+), 21 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4d8027fe9928..6823bae5c66c 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1321,6 +1321,22 @@ void kvm_gpc_init(struct gfn_to_pfn_cache *gpc, struct kvm *kvm, */ int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len); +/** + * kvm_gpc_activate_hva - prepare a cached kernel mapping and HPA for a given HVA. + * + * @gpc: struct gfn_to_pfn_cache object. + * @hva: userspace virtual address to map. + * @len: sanity check; the range being access must fit a single page. + * + * @return: 0 for success. + * -EINVAL for a mapping which would cross a page boundary. + * -EFAULT for an untranslatable guest physical address. + * + * The semantics of this function are the same as those of kvm_gpc_activate(). It + * merely bypasses a layer of address translation. + */ +int kvm_gpc_activate_hva(struct gfn_to_pfn_cache *gpc, unsigned long hva, unsigned long len); + /** * kvm_gpc_check - check validity of a gfn_to_pfn_cache. * @@ -1378,9 +1394,22 @@ void kvm_gpc_mark_dirty(struct gfn_to_pfn_cache *gpc); * kvm_gpc_gpa - retrieve the guest physical address of a cached mapping * * @gpc: struct gfn_to_pfn_cache object. + * + * @return: If the cache was activated with a fixed HVA then INVALID_GPA + * will be returned. */ gpa_t kvm_gpc_gpa(struct gfn_to_pfn_cache *gpc); +/** + * kvm_gpc_hva - retrieve the fixed host physical address of a cached mapping + * + * @gpc: struct gfn_to_pfn_cache object. + * + * @return: If the cache was activated with a guest physical address then + * 0 will be returned. + */ +unsigned long kvm_gpc_hva(struct gfn_to_pfn_cache *gpc); + void kvm_sigset_activate(struct kvm_vcpu *vcpu); void kvm_sigset_deactivate(struct kvm_vcpu *vcpu); diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h index 6f4737d5046a..d49946ee7ae3 100644 --- a/include/linux/kvm_types.h +++ b/include/linux/kvm_types.h @@ -64,7 +64,7 @@ struct gfn_to_hva_cache { struct gfn_to_pfn_cache { u64 generation; - gpa_t gpa; + u64 addr; unsigned long uhva; struct kvm_memory_slot *memslot; struct kvm *kvm; @@ -77,6 +77,7 @@ struct gfn_to_pfn_cache { enum pfn_cache_usage usage; bool active; bool valid; + bool addr_is_gpa; }; #ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 37bcb4399780..b3e3f7e38410 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -83,7 +83,7 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len) if (!gpc->active) return false; - if (gpc->generation != slots->generation) + if (gpc->addr_is_gpa && gpc->generation != slots->generation) return false; if (kvm_is_error_hva(gpc->uhva)) @@ -229,7 +229,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) gpc->valid = true; gpc->pfn = new_pfn; - gpc->khva = new_khva + (gpc->gpa & ~PAGE_MASK); + gpc->khva = new_khva + (gpc->addr & ~PAGE_MASK); /* * Put the reference to the _new_ pfn. The pfn is now tracked by the @@ -246,11 +246,11 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) return -EFAULT; } -static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, - unsigned long len) +static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, u64 addr, + unsigned long len, bool addr_is_gpa) { struct kvm_memslots *slots = kvm_memslots(gpc->kvm); - unsigned long page_offset = gpa & ~PAGE_MASK; + unsigned long page_offset = addr & ~PAGE_MASK; bool unmap_old = false; unsigned long old_uhva; kvm_pfn_t old_pfn; @@ -282,22 +282,34 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, old_khva = gpc->khva - offset_in_page(gpc->khva); old_uhva = gpc->uhva; - /* If the userspace HVA is invalid, refresh that first */ - if (gpc->gpa != gpa || gpc->generation != slots->generation || + /* + * If the address has changed, switched from guest to host (or vice + * versa), or it's a guest address and the memory slots have been + * updated, we need to refresh the userspace HVA. + */ + if (gpc->addr != addr || + gpc->addr_is_gpa != addr_is_gpa || + (addr_is_gpa && gpc->generation != slots->generation) || kvm_is_error_hva(gpc->uhva)) { - gfn_t gfn = gpa_to_gfn(gpa); + gpc->addr = addr; + gpc->addr_is_gpa = addr_is_gpa; - gpc->gpa = gpa; - gpc->generation = slots->generation; - gpc->memslot = __gfn_to_memslot(slots, gfn); - gpc->uhva = gfn_to_hva_memslot(gpc->memslot, gfn); + if (addr_is_gpa) { + gfn_t gfn = gpa_to_gfn(addr); - if (kvm_is_error_hva(gpc->uhva)) { - ret = -EFAULT; - goto out; + gpc->generation = slots->generation; + gpc->memslot = __gfn_to_memslot(slots, gfn); + gpc->uhva = gfn_to_hva_memslot(gpc->memslot, gfn); + } else { + gpc->uhva = addr & PAGE_MASK; } } + if (kvm_is_error_hva(gpc->uhva)) { + ret = -EFAULT; + goto out; + } + /* * If the userspace HVA changed or the PFN was already invalid, * drop the lock and do the HVA to PFN lookup again. @@ -343,7 +355,7 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, int kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, unsigned long len) { - return __kvm_gpc_refresh(gpc, gpc->gpa, len); + return __kvm_gpc_refresh(gpc, gpc->addr, len, gpc->addr_is_gpa); } EXPORT_SYMBOL_GPL(kvm_gpc_refresh); @@ -364,7 +376,8 @@ void kvm_gpc_init(struct gfn_to_pfn_cache *gpc, struct kvm *kvm, } EXPORT_SYMBOL_GPL(kvm_gpc_init); -int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len) +static int __kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, u64 addr, unsigned long len, + bool addr_is_gpa) { struct kvm *kvm = gpc->kvm; @@ -385,19 +398,39 @@ int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len) gpc->active = true; write_unlock_irq(&gpc->lock); } - return __kvm_gpc_refresh(gpc, gpa, len); + return __kvm_gpc_refresh(gpc, addr, len, addr_is_gpa); +} + +int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len) +{ + return __kvm_gpc_activate(gpc, gpa, len, true); } EXPORT_SYMBOL_GPL(kvm_gpc_activate); gpa_t kvm_gpc_gpa(struct gfn_to_pfn_cache *gpc) { - return gpc->gpa; + return gpc->addr_is_gpa ? gpc->addr : INVALID_GPA; } EXPORT_SYMBOL_GPL(kvm_gpc_gpa); +int kvm_gpc_activate_hva(struct gfn_to_pfn_cache *gpc, unsigned long hva, unsigned long len) +{ + return __kvm_gpc_activate(gpc, hva, len, false); +} +EXPORT_SYMBOL_GPL(kvm_gpc_activate_hva); + +unsigned long kvm_gpc_hva(struct gfn_to_pfn_cache *gpc) +{ + return !gpc->addr_is_gpa ? gpc->addr : 0; +} +EXPORT_SYMBOL_GPL(kvm_gpc_hva); + void kvm_gpc_mark_dirty(struct gfn_to_pfn_cache *gpc) { - mark_page_dirty_in_slot(gpc->kvm, gpc->memslot, gpc->gpa >> PAGE_SHIFT); + if (!gpc->addr_is_gpa) + return; + + mark_page_dirty_in_slot(gpc->kvm, gpc->memslot, gpc->addr >> PAGE_SHIFT); } EXPORT_SYMBOL_GPL(kvm_gpc_mark_dirty);