From patchwork Mon Nov 14 20:11:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 20024 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2344489wru; Mon, 14 Nov 2022 12:17:11 -0800 (PST) X-Google-Smtp-Source: AA0mqf4W/oYEG9reUjU/3ENxuifZCvRG8K/CilEszJLf9Z2ZmAR4ZVq6Yn5w05PJY64MjIJqUQJ1 X-Received: by 2002:a05:6402:4501:b0:461:ca0f:affc with SMTP id ez1-20020a056402450100b00461ca0faffcmr12165300edb.169.1668457031543; Mon, 14 Nov 2022 12:17:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668457031; cv=none; d=google.com; s=arc-20160816; b=b9JDQQquKPlj2tjKyep7bSjo4A2dnHidUp+DDC6vglEsC9oaQfXM1SX56bjy3VtMF4 9SlEINBP/umxqdiFEEQj2ka33Fu0KAhTHUV3iNr/3FwNCv9F18nqpMH6fZr8pOEEIkJq 7Wo1CShzLQGwEMqomYgmgGm5qc5DKIFLtwtT/BvEQgil9RiEro4xSDYopTE7ZBXs8CoW ioGUsnYhTfUH5h3NlkfOjKwXsuYxeip+H9fezeCZNWIgaRJlTzYU7554utSLtVFv6Bn6 M+kujwU+BIiqe2Iev3V9AsLE7jBJaz2TRh0JtjXcQMRb7XJBffvcS8e9/ab4zs2NlgSM OTQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mglCJONHjZaMkBoIgcjixSkCCSFrzRBKYTMx418wVb4=; b=VnaBH0XbM2RLSotHyfhUkutRRDDa76NvyPFrc2GRxmjZlaNMct6yRPHIQU0bkf4cpU jd8yJG3wLTUsuU5p3jZ1JsLLXTBKG0FaCuijLbRi6qx5kT9BXsqSfjhvhA+A5P4sVT4F EESf3VZFjKO1zF/z5wL1c+amo/OGAA8Tz3090pbBJPpLIxJg0hWIBf5ptfQrrZI3xwSj y7zdLV00qCTdA9eXa0BOrp04RGpeZdLoAZxb9u8o8iDMRhjxSMoIwMDq8rolwQcgVAqJ a2s0ItceODkpLvPqRRAjtBKcbupoRiMl2USeZaSGBlAgpeaD3x5Fx52EIgYIwJ2tCNQ9 KK7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=oja4AWIi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f17-20020a50d551000000b0046453c385e4si8698183edj.365.2022.11.14.12.16.47; Mon, 14 Nov 2022 12:17:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=oja4AWIi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237536AbiKNULv (ORCPT + 99 others); Mon, 14 Nov 2022 15:11:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237459AbiKNULo (ORCPT ); Mon, 14 Nov 2022 15:11:44 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF9E91BEB3; Mon, 14 Nov 2022 12:11:42 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668456701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mglCJONHjZaMkBoIgcjixSkCCSFrzRBKYTMx418wVb4=; b=oja4AWIi6mYdgYeWVFJ67Cqf9vP4TIUr4x/wU74OS60f/fSGWh3gGuvD4GPrnpZnx90l/i VttLBxAj+U4AeyZ91bChgEKiB0+OiIzyb2uhL8ZQ+9QFn96jNJQx0u/v6/1iAHZ/62fWMD c/OEw66RkQ10zTcxDtKRoUdPA010VyQ= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 1/1] KVM: arm64: Use a separate function for hyp stage-1 walks Date: Mon, 14 Nov 2022 20:11:27 +0000 Message-Id: <20221114201127.1814794-2-oliver.upton@linux.dev> In-Reply-To: <20221114201127.1814794-1-oliver.upton@linux.dev> References: <20221114201127.1814794-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749503999955978630?= X-GMAIL-MSGID: =?utf-8?q?1749503999955978630?= A subsequent change to the page table walkers adds RCU protection for walking stage-2 page tables. KVM uses a global lock to serialize hyp stage-1 walks, meaning RCU protection is quite meaningless for protecting hyp stage-1 walkers. Add a new helper, kvm_pgtable_hyp_walk(), for use when walking hyp stage-1 tables. Call directly into __kvm_pgtable_walk() as table concatenation is not a supported feature at stage-1. No functional change intended. Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_pgtable.h | 24 ++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- arch/arm64/kvm/hyp/pgtable.c | 18 +++++++++++++++--- 3 files changed, 40 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index a874ce0ce7b5..43b2f1882e11 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -596,6 +596,30 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size); int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, struct kvm_pgtable_walker *walker); +/** + * kvm_pgtable_hyp_walk() - Walk a hyp stage-1 page-table. + * @pgt: Page-table structure initialized by kvm_pgtable_hyp_init(). + * @addr: Input address for the start of the walk. + * @size: Size of the range to walk. + * @walker: Walker callback description. + * + * The offset of @addr within a page is ignored and @size is rounded-up to + * the next page boundary. + * + * The walker will walk the page-table entries corresponding to the input + * address range specified, visiting entries according to the walker flags. + * Invalid entries are treated as leaf entries. Leaf entries are reloaded + * after invoking the walker callback, allowing the walker to descend into + * a newly installed table. + * + * Returning a negative error code from the walker callback function will + * terminate the walk immediately with the same error code. + * + * Return: 0 on success, negative error code on failure. + */ +int kvm_pgtable_hyp_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, + struct kvm_pgtable_walker *walker); + /** * kvm_pgtable_get_leaf() - Walk a page-table and retrieve the leaf entry * with its level. diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 1068338d77f3..55eeb3ed1891 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -246,7 +246,7 @@ static int finalize_host_mappings(void) struct memblock_region *reg = &hyp_memory[i]; u64 start = (u64)hyp_phys_to_virt(reg->base); - ret = kvm_pgtable_walk(&pkvm_pgtable, start, reg->size, &walker); + ret = kvm_pgtable_hyp_walk(&pkvm_pgtable, start, reg->size, &walker); if (ret) return ret; } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 5bca9610d040..385fa1051b5d 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -335,6 +335,18 @@ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, return ret; } +int kvm_pgtable_hyp_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, + struct kvm_pgtable_walker *walker) +{ + struct kvm_pgtable_walk_data data = { + .walker = walker, + .addr = ALIGN_DOWN(addr, PAGE_SIZE), + .end = PAGE_ALIGN(addr + size), + }; + + return __kvm_pgtable_walk(&data, pgt->mm_ops, pgt->pgd, pgt->start_level); +} + struct hyp_map_data { u64 phys; kvm_pte_t attr; @@ -454,7 +466,7 @@ int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, if (ret) return ret; - ret = kvm_pgtable_walk(pgt, addr, size, &walker); + ret = kvm_pgtable_hyp_walk(pgt, addr, size, &walker); dsb(ishst); isb(); return ret; @@ -512,7 +524,7 @@ u64 kvm_pgtable_hyp_unmap(struct kvm_pgtable *pgt, u64 addr, u64 size) if (!pgt->mm_ops->page_count) return 0; - kvm_pgtable_walk(pgt, addr, size, &walker); + kvm_pgtable_hyp_walk(pgt, addr, size, &walker); return unmapped; } @@ -557,7 +569,7 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt) .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, }; - WARN_ON(kvm_pgtable_walk(pgt, 0, BIT(pgt->ia_bits), &walker)); + WARN_ON(kvm_pgtable_hyp_walk(pgt, 0, BIT(pgt->ia_bits), &walker)); pgt->mm_ops->put_page(kvm_dereference_pteref(pgt->pgd, false)); pgt->pgd = NULL; }