From patchwork Tue Dec 6 17:35:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Gardon X-Patchwork-Id: 30463 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2959396wrr; Tue, 6 Dec 2022 09:42:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf5vEY2kvd+YDIp5cNW455LgDQAFh+jaQdADMTI65Q2yy8DDy2JmyyxJGNFSx/NavYgLvCgc X-Received: by 2002:aa7:d14c:0:b0:46c:f8af:b331 with SMTP id r12-20020aa7d14c000000b0046cf8afb331mr5131801edo.390.1670348522209; Tue, 06 Dec 2022 09:42:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670348522; cv=none; d=google.com; s=arc-20160816; b=b0QgE7VD+6dpLlbW8CGCH7DYTq3Sr8dHF28wireQED0BU8HsH6CB2BBOThOw1r0t1Y ifXqHFiwwlN/i3K1A8bqJCdhd6J6ei2bJ1Rl344KstQPBMef1M1eF5UiiAfSQkI1uUMO B56QRqn4mqMnGIu2eFntcmk8GvPdSnHwAJTTjamMNlz5BaGzpsRDqpOyNtvQ53QBY2aI 6IhFb+vu7umuj7Qkmxq5YfdNMjL5OM/s7yeJ2l2gKzpuqEEdZzzZIqgDd2nxx3wdQiT1 ogUgVQ4g+5kgS26QYIxE6wXkmayHS3t5AfPOXFAm2wbY/yBuaL1326ZTf1Xb1SO7hg83 TbFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=/TVyB4/pSme/cFmmgXmwyjFM6o5GyBUdN87C0tWnxAo=; b=NA2TMXxRpkH3IExWBH+9K5drc220wIIKPrZhsWc5jY4aiHSt/u+xyIfXHu+sLNSDxV 0NFL//UYlhXSm+IDQ48pfrJ4mcTzB6FIOIQLjwBu33fJB7tJZ7g3RjdBtNKErvMg+/r6 bPSEL/I5X/uz84RQHrdtSLT8ufIWcCV1nzgZGEnBN5iF0VqwzLhA13ELvNlWr0QTH/eg FeGoXKT2oPYzQ18KxFEastM4AITSiAmhHeR6wibxK9kR02ZpXaLUWrK6anfULcae9Zp0 idV5uW4jmVjX+uyQ/RWCphudkr58pLE+FdYRAGsiPISXBSATI0oXaJNH3F3hZ75IVYQK r86w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=oVPJ2nG7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ho10-20020a1709070e8a00b007c0ae1af6dcsi3355629ejc.256.2022.12.06.09.41.39; Tue, 06 Dec 2022 09:42:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=oVPJ2nG7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235573AbiLFRgh (ORCPT + 99 others); Tue, 6 Dec 2022 12:36:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235309AbiLFRgS (ORCPT ); Tue, 6 Dec 2022 12:36:18 -0500 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E52623B9C3 for ; Tue, 6 Dec 2022 09:36:14 -0800 (PST) Received: by mail-pf1-x449.google.com with SMTP id b13-20020a056a000a8d00b0057348c50123so13531014pfl.18 for ; Tue, 06 Dec 2022 09:36:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/TVyB4/pSme/cFmmgXmwyjFM6o5GyBUdN87C0tWnxAo=; b=oVPJ2nG7PFgq3bksWsgionXC79t3Kk/k1kgfLgV8FmLQwSFfrzXfhIv7LJ4yoS4zML KxXcvA273d0cpEacTsrs3CPwZ0QeGzc/wrtfBDP8vEEiwzvY9qAx/YXCee4fbzCDlyPA wf12/QEcjIC8x4ZQyEhI4RVPSyH0ittgNyouIntMcJOj9jEe20vMURToED9sTUyrn/4z 9oQV3HNShZ/sB9f4KGupI+YqgXr8EERfZU+z+oZiaGH+zBRcZ0PjIAAJsyAjMUzSm7Vs 4g6VdBhmU7ZsH3z3DVifdx/0F/p8Z+JwDN7p3vpkEeG1Qde7kp/1dlDnT2BnY/1VIB4d dowA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/TVyB4/pSme/cFmmgXmwyjFM6o5GyBUdN87C0tWnxAo=; b=0G/cwGlhqMZPkjZE5N0Mckc7JvXYU7h37H7im+5IO7I/ygV1CCrkchSIZcB/HVyR8V f9ZhILh2/LrKpqIzBKcZKe+xjI6qanJsDT3B+iMRuYDsbQ4XXwu5VBept/rK8DJKb8bU dbYRNhlcZQvR90RMxW36X3JzE1R/Kodii2LU4WNpdzGSJYMrbBCUq6FMA0CTbtZ36loM i/8tyQ/yjdqGF09+yFk8ZU0qi5IEwis4P/AAD9kj3hkzAArBhq1Ikb5Wo4czh0dtGdWA iYd8icycoie0lUaewD8UpBpvSRbEK+Jr7kXrCdAAAXZR5kfJguNcmT8C7DZggiqS+jaL TpAg== X-Gm-Message-State: ANoB5pnhGngahw7OZmt9ezfRq7kqBxiSZhgPag1eG6X7XlY5Al8VqKD3 JavBCwPXoaasE2UCpGUiNFhMzr9YpCKgX9i8aWuG7bwwrEVOX3Wy19qWFpH/Snn2+5lvKx0921w T/09Vr+22iV5ifPUzlo68FglQKZhOQoUD+qfWLV2YxCGxOYlI2VWQxXJ3u8he3EqGZ55g9BfW X-Received: from sweer.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:e45]) (user=bgardon job=sendgmr) by 2002:a17:90a:8b03:b0:213:16d2:4d4c with SMTP id y3-20020a17090a8b0300b0021316d24d4cmr96765107pjn.70.1670348174449; Tue, 06 Dec 2022 09:36:14 -0800 (PST) Date: Tue, 6 Dec 2022 17:35:59 +0000 In-Reply-To: <20221206173601.549281-1-bgardon@google.com> Mime-Version: 1.0 References: <20221206173601.549281-1-bgardon@google.com> X-Mailer: git-send-email 2.39.0.rc0.267.gcb52ba06e7-goog Message-ID: <20221206173601.549281-6-bgardon@google.com> Subject: [PATCH 5/7] KVM: x86/MMU: Move the rmap walk iterator out of mmu.c From: Ben Gardon To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Peter Xu , Sean Christopherson , David Matlack , Vipin Sharma , Ben Gardon X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751487372143654762?= X-GMAIL-MSGID: =?utf-8?q?1751487372143654762?= Move slot_rmap_walk_iterator and its associated functions out of mmu.c to rmap.(c|h). No functional change intended. Signed-off-by: Ben Gardon --- arch/x86/kvm/mmu/mmu.c | 73 ----------------------------------------- arch/x86/kvm/mmu/rmap.c | 43 ++++++++++++++++++++++++ arch/x86/kvm/mmu/rmap.h | 36 ++++++++++++++++++++ 3 files changed, 79 insertions(+), 73 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 52e487d89d54..88da2abc2375 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1198,79 +1198,6 @@ static bool kvm_set_pte_rmap(struct kvm *kvm, struct kvm_rmap_head *rmap_head, return need_flush; } -struct slot_rmap_walk_iterator { - /* input fields. */ - const struct kvm_memory_slot *slot; - gfn_t start_gfn; - gfn_t end_gfn; - int start_level; - int end_level; - - /* output fields. */ - gfn_t gfn; - struct kvm_rmap_head *rmap; - int level; - - /* private field. */ - struct kvm_rmap_head *end_rmap; -}; - -static void -rmap_walk_init_level(struct slot_rmap_walk_iterator *iterator, int level) -{ - iterator->level = level; - iterator->gfn = iterator->start_gfn; - iterator->rmap = gfn_to_rmap(iterator->gfn, level, iterator->slot); - iterator->end_rmap = gfn_to_rmap(iterator->end_gfn, level, iterator->slot); -} - -static void -slot_rmap_walk_init(struct slot_rmap_walk_iterator *iterator, - const struct kvm_memory_slot *slot, int start_level, - int end_level, gfn_t start_gfn, gfn_t end_gfn) -{ - iterator->slot = slot; - iterator->start_level = start_level; - iterator->end_level = end_level; - iterator->start_gfn = start_gfn; - iterator->end_gfn = end_gfn; - - rmap_walk_init_level(iterator, iterator->start_level); -} - -static bool slot_rmap_walk_okay(struct slot_rmap_walk_iterator *iterator) -{ - return !!iterator->rmap; -} - -static void slot_rmap_walk_next(struct slot_rmap_walk_iterator *iterator) -{ - while (++iterator->rmap <= iterator->end_rmap) { - iterator->gfn += (1UL << KVM_HPAGE_GFN_SHIFT(iterator->level)); - - if (iterator->rmap->val) - return; - } - - if (++iterator->level > iterator->end_level) { - iterator->rmap = NULL; - return; - } - - rmap_walk_init_level(iterator, iterator->level); -} - -#define for_each_slot_rmap_range(_slot_, _start_level_, _end_level_, \ - _start_gfn, _end_gfn, _iter_) \ - for (slot_rmap_walk_init(_iter_, _slot_, _start_level_, \ - _end_level_, _start_gfn, _end_gfn); \ - slot_rmap_walk_okay(_iter_); \ - slot_rmap_walk_next(_iter_)) - -typedef bool (*rmap_handler_t)(struct kvm *kvm, struct kvm_rmap_head *rmap_head, - struct kvm_memory_slot *slot, gfn_t gfn, - int level, pte_t pte); - static __always_inline bool kvm_handle_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range, rmap_handler_t handler) diff --git a/arch/x86/kvm/mmu/rmap.c b/arch/x86/kvm/mmu/rmap.c index 6833676aa9ea..91af5b32cffb 100644 --- a/arch/x86/kvm/mmu/rmap.c +++ b/arch/x86/kvm/mmu/rmap.c @@ -240,3 +240,46 @@ void rmap_remove(struct kvm *kvm, u64 *spte) pte_list_remove(spte, rmap_head); } + +void rmap_walk_init_level(struct slot_rmap_walk_iterator *iterator, int level) +{ + iterator->level = level; + iterator->gfn = iterator->start_gfn; + iterator->rmap = gfn_to_rmap(iterator->gfn, level, iterator->slot); + iterator->end_rmap = gfn_to_rmap(iterator->end_gfn, level, iterator->slot); +} + +void slot_rmap_walk_init(struct slot_rmap_walk_iterator *iterator, + const struct kvm_memory_slot *slot, int start_level, + int end_level, gfn_t start_gfn, gfn_t end_gfn) +{ + iterator->slot = slot; + iterator->start_level = start_level; + iterator->end_level = end_level; + iterator->start_gfn = start_gfn; + iterator->end_gfn = end_gfn; + + rmap_walk_init_level(iterator, iterator->start_level); +} + +bool slot_rmap_walk_okay(struct slot_rmap_walk_iterator *iterator) +{ + return !!iterator->rmap; +} + +void slot_rmap_walk_next(struct slot_rmap_walk_iterator *iterator) +{ + while (++iterator->rmap <= iterator->end_rmap) { + iterator->gfn += (1UL << KVM_HPAGE_GFN_SHIFT(iterator->level)); + + if (iterator->rmap->val) + return; + } + + if (++iterator->level > iterator->end_level) { + iterator->rmap = NULL; + return; + } + + rmap_walk_init_level(iterator, iterator->level); +} diff --git a/arch/x86/kvm/mmu/rmap.h b/arch/x86/kvm/mmu/rmap.h index 81df186ba3c3..dc4bf7e609ec 100644 --- a/arch/x86/kvm/mmu/rmap.h +++ b/arch/x86/kvm/mmu/rmap.h @@ -54,4 +54,40 @@ struct kvm_rmap_head *gfn_to_rmap(gfn_t gfn, int level, bool rmap_can_add(struct kvm_vcpu *vcpu); void rmap_remove(struct kvm *kvm, u64 *spte); + +struct slot_rmap_walk_iterator { + /* input fields. */ + const struct kvm_memory_slot *slot; + gfn_t start_gfn; + gfn_t end_gfn; + int start_level; + int end_level; + + /* output fields. */ + gfn_t gfn; + struct kvm_rmap_head *rmap; + int level; + + /* private field. */ + struct kvm_rmap_head *end_rmap; +}; + +void rmap_walk_init_level(struct slot_rmap_walk_iterator *iterator, int level); +void slot_rmap_walk_init(struct slot_rmap_walk_iterator *iterator, + const struct kvm_memory_slot *slot, int start_level, + int end_level, gfn_t start_gfn, gfn_t end_gfn); +bool slot_rmap_walk_okay(struct slot_rmap_walk_iterator *iterator); +void slot_rmap_walk_next(struct slot_rmap_walk_iterator *iterator); + +#define for_each_slot_rmap_range(_slot_, _start_level_, _end_level_, \ + _start_gfn, _end_gfn, _iter_) \ + for (slot_rmap_walk_init(_iter_, _slot_, _start_level_, \ + _end_level_, _start_gfn, _end_gfn); \ + slot_rmap_walk_okay(_iter_); \ + slot_rmap_walk_next(_iter_)) + +typedef bool (*rmap_handler_t)(struct kvm *kvm, struct kvm_rmap_head *rmap_head, + struct kvm_memory_slot *slot, gfn_t gfn, + int level, pte_t pte); + #endif /* __KVM_X86_MMU_RMAP_H */