From patchwork Mon Dec 18 13:58:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Ene X-Patchwork-Id: 180451 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp1267874dyi; Mon, 18 Dec 2023 06:12:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IHgyjLxsJBIuAcFShUDDZZmklzjco/pFxKfmGskCVGvDIkl488QGr/B8SKDg7iQkOApN0IY X-Received: by 2002:a17:906:54f:b0:a23:54fe:a9a6 with SMTP id k15-20020a170906054f00b00a2354fea9a6mr899540eja.8.1702908725551; Mon, 18 Dec 2023 06:12:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702908725; cv=none; d=google.com; s=arc-20160816; b=BHMKqjYy1Ee+BszLPrSvkRUh8iZ29BfrJrp2s/TKg3UxKmGCcAEdJ/d3N6kGHhIh8M FM5rok0ovxe14tlLjUURZGAhVeIxYx92j4KYL0dAzolmSqhlmCZ1z2dvJv6tebKsl6JA wUPXYmG+nXQp2fMnrAfYxD1+NdJhUaZgaGyi5xADRGK8mzQeqUSVpVF0DygT71V1EJZL 4HKmv6WNsIUuobU3mTsolOdl23udP9hlWNsNlGPax76mMr0K23blemFtVZCWkAwHG+gS C4mvgREnQq6rXNqXiM8Ti8EzsLxrphbh7I7F3LIPANdPtkfn5PaCMRITA22krck9ABhX NKvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=LD/nzFY+bvR1rFs9NYLk8iA9qjOdN0RioRORlh5RIvY=; fh=4RXM1FFjr/2FDx2f50K7N78eRirWaG15JMQkQwGVT00=; b=AoQOdkCl9lUsE7y5B6U8SkV2SzdXO6ccV/668A6UsS4eubmGE8Fpc1Ddu+92W1JtOJ lFmcKxJ/7SSwimYl8XMP50fhEMUDao0XpV7pRF0OLSIGo12T9av5EzOQ1L/L5GifaNmr 2Tk5T+NSOkJDEqbX5l/AFSFePvMcQfn4T5gM/gmhLk5kRdlKmxd8X3YXxLLl7ApzpEya OaJnS9JYdEuM0YIUa3S/bEFGTYkEjt3u1Q/yt/CEiHR0BVEYaM/uKpQBjuNTrJPRc/L9 deqNvsDqJW/jqCMCk4VoYEUH54e1wLYbLduJODkg9fUlY0zZ27Q6yhC1GAf0cPy93Bki br/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=vdpVYKCD; spf=pass (google.com: domain of linux-kernel+bounces-3789-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-3789-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id a10-20020a1709064a4a00b00a23537017acsi1096107ejv.703.2023.12.18.06.12.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Dec 2023 06:12:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-3789-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=vdpVYKCD; spf=pass (google.com: domain of linux-kernel+bounces-3789-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-3789-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 59BF91F25CF1 for ; Mon, 18 Dec 2023 14:00:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 797733A1D8; Mon, 18 Dec 2023 13:59:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vdpVYKCD" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B94537884 for ; Mon, 18 Dec 2023 13:59:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sebastianene.bounces.google.com Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3333c009305so2212322f8f.1 for ; Mon, 18 Dec 2023 05:59:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702907993; x=1703512793; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LD/nzFY+bvR1rFs9NYLk8iA9qjOdN0RioRORlh5RIvY=; b=vdpVYKCDfLhlgKQGMiVZJAA4KWBzuycpZ4G8H68n84/tISFGb/wkhKLxkXNVLrd+cQ lFceHAM7f4jZyQjFkAKtTG9lClSVpDh/GB+jSw3s4mAYSR7x8c/BzTwO764/TOdV1Neu aBHljePq8AR/BBYvbbvp3/FinedUKiO6eFJGpLt1qjNW6iQjnQIrSWbNPP3uEvWKF6Ij NVAOejGCu5DIszaxcXtSsY6DkRscW4lwuJ7i0fo6POG2hdMOI6ChSbPgVAhkw9YLVeCy amjNd+tPG2UnLGD3dgQFXHE/Sf/OuDMGbBzbhcC3cz2auSiVdeKaIhrYgP2WvliNZ9nH ZfkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702907993; x=1703512793; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LD/nzFY+bvR1rFs9NYLk8iA9qjOdN0RioRORlh5RIvY=; b=ljtdIEQqbo9pactMWU6a05FRT3a3Kubu1ljgrad/q0mT2F2AwwnRZtDNRZO6iJNUtu HFF12UbiiEb2Vv/0VUmk9mblFqEQ1s3oLmS7bFtbKhGb8HpIlAcsvAXR2kn/1EIgnInn rvPDJ6dm0oHTr6IiXTiiGN/puVrYGeaaftfzE659ApGct4cEhE/I2N750GAsI0wnVQ4F qpyib2PoUMUGsYeItSOGObrMRqii2kOIoaw3EJO1nYYKyU6GvU1+GJbMGUvRdonHPEwG jLyl5KF8OU2auW+FitEtdRJ71gTrLjiQUDc4FIX2hdM6CtMO7NhK0zhlt0rFhCUcaZp+ ISkQ== X-Gm-Message-State: AOJu0YxH142tbZ1+Z/1OdSXvE6m/Ez5t0ns31tdEqpWNT9zV3tCiFSG3 KK9B+eE+Ik84rgIepZia7BZZ8MxpXzF8nQ4JKeE= X-Received: from sebkvm.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:cd5]) (user=sebastianene job=sendgmr) by 2002:a05:6000:1ac9:b0:336:4b35:ba52 with SMTP id i9-20020a0560001ac900b003364b35ba52mr32388wry.9.1702907993532; Mon, 18 Dec 2023 05:59:53 -0800 (PST) Date: Mon, 18 Dec 2023 13:58:53 +0000 In-Reply-To: <20231218135859.2513568-2-sebastianene@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231218135859.2513568-2-sebastianene@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231218135859.2513568-5-sebastianene@google.com> Subject: [PATCH v4 03/10] KVM: arm64: Invoke the snapshot interface for the host stage-2 pagetable From: Sebastian Ene To: will@kernel.org, Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , catalin.marinas@arm.com, mark.rutland@arm.com, akpm@linux-foundation.org, maz@kernel.org Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kernel-team@android.com, vdonnefort@google.com, qperret@google.com, smostafa@google.com, Sebastian Ene X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785629219689043319 X-GMAIL-MSGID: 1785629219689043319 Allocate memory for the snapshot by creating a memory cache with empty pages that will be used by the hypervisor during the page table copy. Get the required size of the PGD and allocate physically contiguous memory for it. Allocate contiguous memory for an array that is used to keep track of the pages used from the memcache. Call the snapshot interface and release the memory for the snapshot. Signed-off-by: Sebastian Ene --- arch/arm64/kvm/ptdump.c | 107 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/arch/arm64/kvm/ptdump.c b/arch/arm64/kvm/ptdump.c index 5816fc632..e99bab427 100644 --- a/arch/arm64/kvm/ptdump.c +++ b/arch/arm64/kvm/ptdump.c @@ -25,6 +25,9 @@ static int kvm_ptdump_open(struct inode *inode, struct file *file); static int kvm_ptdump_release(struct inode *inode, struct file *file); static int kvm_ptdump_show(struct seq_file *m, void *); +static phys_addr_t get_host_pa(void *addr); +static void *get_host_va(phys_addr_t pa); + static const struct file_operations kvm_ptdump_fops = { .open = kvm_ptdump_open, .read = seq_read, @@ -32,6 +35,11 @@ static const struct file_operations kvm_ptdump_fops = { .release = kvm_ptdump_release, }; +static struct kvm_pgtable_mm_ops ptdump_host_mmops = { + .phys_to_virt = get_host_va, + .virt_to_phys = get_host_pa, +}; + static int kvm_ptdump_open(struct inode *inode, struct file *file) { struct kvm_ptdump_register *reg = inode->i_private; @@ -78,11 +86,110 @@ static void kvm_ptdump_debugfs_register(struct kvm_ptdump_register *reg, static struct kvm_ptdump_register host_reg; +static size_t host_stage2_get_pgd_len(void) +{ + u32 phys_shift = get_kvm_ipa_limit(); + u64 vtcr = kvm_get_vtcr(read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1), + read_sanitised_ftr_reg(SYS_ID_AA64MMFR1_EL1), + phys_shift); + return (kvm_pgtable_stage2_pgd_size(vtcr) >> PAGE_SHIFT); +} + +static phys_addr_t get_host_pa(void *addr) +{ + return __pa(addr); +} + +static void *get_host_va(phys_addr_t pa) +{ + return __va(pa); +} + +static void kvm_host_put_ptdump_info(void *snap) +{ + void *mc_page; + size_t i; + struct kvm_pgtable_snapshot *snapshot; + + if (!snap) + return; + + snapshot = snap; + while ((mc_page = pop_hyp_memcache(&snapshot->mc, get_host_va)) != NULL) + free_page((unsigned long)mc_page); + + if (snapshot->pgd_hva) + free_pages_exact(snapshot->pgd_hva, snapshot->pgd_pages); + + if (snapshot->used_pages_hva) { + for (i = 0; i < snapshot->used_pages_indx; i++) { + mc_page = get_host_va(snapshot->used_pages_hva[i]); + free_page((unsigned long)mc_page); + } + + free_pages_exact(snapshot->used_pages_hva, snapshot->num_used_pages); + } + + free_page((unsigned long)snapshot); +} + +static void *kvm_host_get_ptdump_info(struct kvm_ptdump_register *reg) +{ + int i, ret; + void *mc_page; + struct kvm_pgtable_snapshot *snapshot; + size_t memcache_len; + + snapshot = (void *)__get_free_page(GFP_KERNEL_ACCOUNT); + if (!snapshot) + return NULL; + + memset(snapshot, 0, sizeof(struct kvm_pgtable_snapshot)); + + snapshot->pgd_pages = host_stage2_get_pgd_len(); + snapshot->pgd_hva = alloc_pages_exact(snapshot->pgd_pages, GFP_KERNEL_ACCOUNT); + if (!snapshot->pgd_hva) + goto err; + + memcache_len = (size_t)reg->priv; + for (i = 0; i < memcache_len; i++) { + mc_page = (void *)__get_free_page(GFP_KERNEL_ACCOUNT); + if (!mc_page) + goto err; + + push_hyp_memcache(&snapshot->mc, mc_page, get_host_pa); + } + + snapshot->num_used_pages = DIV_ROUND_UP(sizeof(phys_addr_t) * memcache_len, + PAGE_SIZE); + snapshot->used_pages_hva = alloc_pages_exact(snapshot->num_used_pages, + GFP_KERNEL_ACCOUNT); + if (!snapshot->used_pages_hva) + goto err; + + ret = kvm_call_hyp_nvhe(__pkvm_host_stage2_snapshot, snapshot); + if (ret) { + pr_err("ERROR %d snapshot host pagetables\n", ret); + goto err; + } + + snapshot->pgtable.pgd = get_host_va((phys_addr_t)snapshot->pgtable.pgd); + snapshot->pgtable.mm_ops = &ptdump_host_mmops; + + return snapshot; +err: + kvm_host_put_ptdump_info(snapshot); + return NULL; +} + void kvm_ptdump_register_host(void) { if (!is_protected_kvm_enabled()) return; + host_reg.get_ptdump_info = kvm_host_get_ptdump_info; + host_reg.put_ptdump_info = kvm_host_put_ptdump_info; + kvm_ptdump_debugfs_register(&host_reg, "host_page_tables", kvm_debugfs_dir); }