From patchwork Wed Dec 21 22:24:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Gardon X-Patchwork-Id: 35537 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:8188:b0:89:790f:f786 with SMTP id m8csp4945405dye; Wed, 21 Dec 2022 14:26:29 -0800 (PST) X-Google-Smtp-Source: AMrXdXv60bl5CU4GudkJkTBEI8nlzTMKdNQlCRHgvoH1ugRtnZLFTlFfGvEOmWoA42LMiGZoEE8N X-Received: by 2002:a17:906:a18c:b0:7c1:5467:39b1 with SMTP id s12-20020a170906a18c00b007c1546739b1mr3034113ejy.72.1671661588891; Wed, 21 Dec 2022 14:26:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671661588; cv=none; d=google.com; s=arc-20160816; b=cXWGGtBU0Tncp+IU8z0MGIXpXd04W/emom34PnDz6N14p3Id+GL/Oa5xA1zHiyzJtF Dsh8vKRaGhDU2oEWIx1OfIqZWD8jFR4gjjQY76SPCSEaRROoFtWNAv19H8zUV0udRMfC n3O8VbklJMU4pKwaValn3ap4zGdi3B7+eTA7uMas+BVexAS93wXc5u1QPt3DIG1pfHY/ DEfGXlvxZRklvQikiAEYN7asUGz1hN48qQ06DUScrSLgITP9jLuBVbF9m+2mqIJjQmJa S39hwgRNdrqRNwhqTnUyAa5/GITJVX2MkLIHKXPkuL45mFMHuxn2/K8hZBjR/3Wbe/vM 9zqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=6Y1uFW2wjlbXMFly7tpXboV7VZ3E/dcx8Q2Gfsts8Tk=; b=0oCsHCVl7bSVeXtJU0NRqeEqDsbtZekyy9FjONpqMjwQh1shB9Pb866r5iPhbFRpNH bCZgq034C3XlOIzPX+l83V3RMMvIrQwdX9BttrqHQGkYiZcIeykZCTSVZG8JSBDHu34c 60H0IrkWfps6Kg3j0T+VmYBxsWB+58Em82vv2RKAnLeM4pltDH74UMIgtfhFpVuf6Rm3 mQO943qp8jpnSbk4ARtCxkYWRWJyKWu3K8LHwsoSCOsMqgAEaJfW/tSCCeuGl5cqD9YO SLSNCK680Eiwt/+UoztzqqtD2OkZqDGpT38GKBtxVggnQowe8cDHZ/Pzzq38Kyhv/PoF 4n/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=j6BY0U68; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nd19-20020a170907629300b007c10a0c6f44si16516262ejc.623.2022.12.21.14.26.04; Wed, 21 Dec 2022 14:26:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=j6BY0U68; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235053AbiLUWZP (ORCPT + 99 others); Wed, 21 Dec 2022 17:25:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234897AbiLUWYn (ORCPT ); Wed, 21 Dec 2022 17:24:43 -0500 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5655275E2 for ; Wed, 21 Dec 2022 14:24:34 -0800 (PST) Received: by mail-pg1-x549.google.com with SMTP id r126-20020a632b84000000b004393806c06eso115391pgr.4 for ; Wed, 21 Dec 2022 14:24:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6Y1uFW2wjlbXMFly7tpXboV7VZ3E/dcx8Q2Gfsts8Tk=; b=j6BY0U68byYLgySr44DngXyEm134/1hFKLFShEVTVJ/IuhmuHUhPj2IFs5+alWpjes FnmgU/a+mWQ9lDW3CaNzgmNQp5HsZoHpHcyV/b1il72r6Xzaj9qXk5NXsYY0j+JXwNjk FZejHNVrbE5LJsqGqrP4ZPA/zp5q0qfmZqF/ezW2x9qxFl5GNHdMhzaOZVSwWHoYKYqH 08xYqohGPLH5zNwDVBBmvSSBKQHJMlkqT5i+Wtxx9V3Dvv+kdODLKmik7SKFpsmBhFQW cfjubSaMmD3KdtMWQs+eq26B7yPez7mGSsQDiVaiRL+/oXIuPIBq7yXEg4mMa1UUUuq9 /2Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6Y1uFW2wjlbXMFly7tpXboV7VZ3E/dcx8Q2Gfsts8Tk=; b=1riDbpmR3PBsfWaXdbhmHltNixPuiKvXqKUUA9YckAmohbhrrU6LWWM1MF3prMLQbh TOjWput9CrUesudEYch2F7krfS6G05Wuz6jjbnJ+YTD3RBkYpqvKP42uPDkxJefhkF0J BlwdC4Q4Yx759DBUrNf4bTNWjfgD3t80iCP64iRLS0nBMUXdos2XINyWUsNUOR+ZUXXY Ck7fMLsxnjDg+cr5diUMaWUjLFeizpKi5uwxI9Psrbgh1NZS47GoBXalcyluTJfNClwF xSV6ObAsJeE7cSoegtPT+5qs0euZi/Ax6e747jTjBMNZlYpECcBqBLI3l/j9NcCkPAR0 CvUg== X-Gm-Message-State: AFqh2kp8+5Q2MY4oziACu6vj5Bo3yy2aq8XMzYjx/n0QaDLZ1mERZGI4 c6HNIAGNejb1+iUBVuA58vttaKASGdSiA9le4RzDaU/TWs1ioCUK6iId/Km6CBT5c76Zfx+9iUu PYxqoG4T3oNZu0wX0H+8Po1gkZn1asiQhhqZfwbZ4AFR46IyMOcGYljn/NYwKXyrBgVmKLWkZ X-Received: from sweer.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:e45]) (user=bgardon job=sendgmr) by 2002:a17:90a:8183:b0:219:9874:c7d3 with SMTP id e3-20020a17090a818300b002199874c7d3mr320888pjn.221.1671661474119; Wed, 21 Dec 2022 14:24:34 -0800 (PST) Date: Wed, 21 Dec 2022 22:24:11 +0000 In-Reply-To: <20221221222418.3307832-1-bgardon@google.com> Mime-Version: 1.0 References: <20221221222418.3307832-1-bgardon@google.com> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog Message-ID: <20221221222418.3307832-8-bgardon@google.com> Subject: [RFC 07/14] KVM: x86/MMU: Cleanup shrinker interface with Shadow MMU From: Ben Gardon To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Peter Xu , Sean Christopherson , David Matlack , Vipin Sharma , Nagareddy Reddy , Ben Gardon X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752864222163575817?= X-GMAIL-MSGID: =?utf-8?q?1752864222163575817?= The MMU shrinker currently only operates on the Shadow MMU, but having the entire implemenatation in shadow_mmu.c is awkward since much of the function isn't Shadow MMU specific. There has also been talk of changing the target of the shrinker to the MMU caches rather than already allocated page tables. As a result, it makes sense to move some of the implementation back to mmu.c. No functional change intended. Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 43 ++++++++++++++++++++++++ arch/x86/kvm/mmu/shadow_mmu.c | 62 ++++++++--------------------------- arch/x86/kvm/mmu/shadow_mmu.h | 3 +- 3 files changed, 58 insertions(+), 50 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index dd97e346c786..4c45a5b63356 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3147,6 +3147,49 @@ static unsigned long mmu_shrink_count(struct shrinker *shrink, return percpu_counter_read_positive(&kvm_total_used_mmu_pages); } +unsigned long mmu_shrink_scan(struct shrinker *shrink, struct shrink_control *sc) +{ + struct kvm *kvm; + int nr_to_scan = sc->nr_to_scan; + unsigned long freed = 0; + + mutex_lock(&kvm_lock); + + list_for_each_entry(kvm, &vm_list, vm_list) { + /* + * Never scan more than sc->nr_to_scan VM instances. + * Will not hit this condition practically since we do not try + * to shrink more than one VM and it is very unlikely to see + * !n_used_mmu_pages so many times. + */ + if (!nr_to_scan--) + break; + + /* + * n_used_mmu_pages is accessed without holding kvm->mmu_lock + * here. We may skip a VM instance errorneosly, but we do not + * want to shrink a VM that only started to populate its MMU + * anyway. + */ + if (!kvm->arch.n_used_mmu_pages && + !kvm_shadow_mmu_has_zapped_obsolete_pages(kvm)) + continue; + + freed = kvm_shadow_mmu_shrink_scan(kvm, sc->nr_to_scan); + + /* + * unfair on small ones + * per-vm shrinkers cry out + * sadness comes quickly + */ + list_move_tail(&kvm->vm_list, &vm_list); + break; + } + + mutex_unlock(&kvm_lock); + return freed; +} + static struct shrinker mmu_shrinker = { .count_objects = mmu_shrink_count, .scan_objects = mmu_shrink_scan, diff --git a/arch/x86/kvm/mmu/shadow_mmu.c b/arch/x86/kvm/mmu/shadow_mmu.c index 090b4788f7de..1259c4a3b140 100644 --- a/arch/x86/kvm/mmu/shadow_mmu.c +++ b/arch/x86/kvm/mmu/shadow_mmu.c @@ -3147,7 +3147,7 @@ void kvm_zap_obsolete_pages(struct kvm *kvm) kvm_mmu_commit_zap_page(kvm, &kvm->arch.zapped_obsolete_pages); } -static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm) +bool kvm_shadow_mmu_has_zapped_obsolete_pages(struct kvm *kvm) { return unlikely(!list_empty_careful(&kvm->arch.zapped_obsolete_pages)); } @@ -3416,60 +3416,24 @@ void kvm_rmap_zap_collapsible_sptes(struct kvm *kvm, kvm_arch_flush_remote_tlbs_memslot(kvm, slot); } -unsigned long mmu_shrink_scan(struct shrinker *shrink, struct shrink_control *sc) +unsigned long kvm_shadow_mmu_shrink_scan(struct kvm *kvm, int pages_to_free) { - struct kvm *kvm; - int nr_to_scan = sc->nr_to_scan; unsigned long freed = 0; + int idx; - mutex_lock(&kvm_lock); - - list_for_each_entry(kvm, &vm_list, vm_list) { - int idx; - LIST_HEAD(invalid_list); - - /* - * Never scan more than sc->nr_to_scan VM instances. - * Will not hit this condition practically since we do not try - * to shrink more than one VM and it is very unlikely to see - * !n_used_mmu_pages so many times. - */ - if (!nr_to_scan--) - break; - /* - * n_used_mmu_pages is accessed without holding kvm->mmu_lock - * here. We may skip a VM instance errorneosly, but we do not - * want to shrink a VM that only started to populate its MMU - * anyway. - */ - if (!kvm->arch.n_used_mmu_pages && - !kvm_has_zapped_obsolete_pages(kvm)) - continue; - - idx = srcu_read_lock(&kvm->srcu); - write_lock(&kvm->mmu_lock); - - if (kvm_has_zapped_obsolete_pages(kvm)) { - kvm_mmu_commit_zap_page(kvm, - &kvm->arch.zapped_obsolete_pages); - goto unlock; - } + idx = srcu_read_lock(&kvm->srcu); + write_lock(&kvm->mmu_lock); - freed = kvm_mmu_zap_oldest_mmu_pages(kvm, sc->nr_to_scan); + if (kvm_shadow_mmu_has_zapped_obsolete_pages(kvm)) { + kvm_mmu_commit_zap_page(kvm, &kvm->arch.zapped_obsolete_pages); + goto out; + } -unlock: - write_unlock(&kvm->mmu_lock); - srcu_read_unlock(&kvm->srcu, idx); + freed = kvm_mmu_zap_oldest_mmu_pages(kvm, pages_to_free); - /* - * unfair on small ones - * per-vm shrinkers cry out - * sadness comes quickly - */ - list_move_tail(&kvm->vm_list, &vm_list); - break; - } +out: + write_unlock(&kvm->mmu_lock); + srcu_read_unlock(&kvm->srcu, idx); - mutex_unlock(&kvm_lock); return freed; } diff --git a/arch/x86/kvm/mmu/shadow_mmu.h b/arch/x86/kvm/mmu/shadow_mmu.h index 20c65a0ea52c..9952aa1e86cf 100644 --- a/arch/x86/kvm/mmu/shadow_mmu.h +++ b/arch/x86/kvm/mmu/shadow_mmu.h @@ -99,7 +99,8 @@ void kvm_shadow_mmu_try_split_huge_pages(struct kvm *kvm, void kvm_rmap_zap_collapsible_sptes(struct kvm *kvm, const struct kvm_memory_slot *slot); -unsigned long mmu_shrink_scan(struct shrinker *shrink, struct shrink_control *sc); +bool kvm_shadow_mmu_has_zapped_obsolete_pages(struct kvm *kvm); +unsigned long kvm_shadow_mmu_shrink_scan(struct kvm *kvm, int pages_to_free); /* Exports from paging_tmpl.h */ gpa_t paging32_gva_to_gpa(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,