From patchwork Fri May 12 14:57:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93228 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5175346vqo; Fri, 12 May 2023 08:05:22 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6dcBHFbxefChzpQPYU+aqNAIaZEZMOGde2IrHJ51fST0L/0wtf2ip6DJnH0tI9KMxkV6jb X-Received: by 2002:a05:6a21:78a0:b0:100:5082:611d with SMTP id bf32-20020a056a2178a000b001005082611dmr24344398pzc.32.1683903921939; Fri, 12 May 2023 08:05:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683903921; cv=none; d=google.com; s=arc-20160816; b=V4sg2+kT5mhMvk3ljq/sD/ZCNc9hrnqPImRqlcnkfmmpXg5ps08Xo4QFei+QxULBz+ hjs3W781CYvSgvHABsAPW80hv2NjnwIHsmanULmFtmcpkwXhTmx2dWl1Tz8AU6uGD6RQ gdwkRzmEQlCFErwoZASIehI/ONCm6hz/y9OMAp82tQzoO9LLrQ7wbUX+HAFnJMqa7MEb HqmcVf6qZ5XO3DLVHC1a5b5B6eF1kky52yMabxX6enztw/6YAzS2uDgZXEHDq7w/j+sx R0pJH4MOHUxD4ylFTDq/ljQx+D8IpWq87ZVmNY6XJnzx3UZ3txPYLe9cHs26Bc7EJJx+ t2iA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bXJFTabP/eR2shGXG2ZpQufidqoIkTMUZda4PVMUggs=; b=h/HMT+V0xGC3UoPa+DoY/hGUBJ5SpEtFPbiWeadVnIDjGG5FCh8KuQY20Nz41rJA9c KogwA6zL28sYgckwM/WvaT6uhjiIN4Qp5X7iLTZVz3EymOsS1c9Wd0TlUGlAECcALJko g8/EuVb7eyj2+7LLiH/sCAjh3R61Ec+3ppCCerP7Pt0l+pA/fdYTUVJo05zkrsM0L/Aj HAhVWHYAOgKltk1wP5RTwblK2psxMbpZdpMm+SvekzLgJC7oRW19ofr3AvxOm0uUicxf mVxn2n2T+oYxaQCWVbLttE9S+6eQiBHViRl/HSwLw9X7bozEIv2jybHZyhRsyYL6J7Yx Gehg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=UkPLrlyF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u189-20020a6385c6000000b00530114f3d06si9033137pgd.605.2023.05.12.08.05.05; Fri, 12 May 2023 08:05:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=UkPLrlyF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241558AbjELO6M (ORCPT + 99 others); Fri, 12 May 2023 10:58:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241613AbjELO6D (ORCPT ); Fri, 12 May 2023 10:58:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 645B73A86 for ; Fri, 12 May 2023 07:58:00 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 09FCE65735 for ; Fri, 12 May 2023 14:58:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9CEA8C4339B; Fri, 12 May 2023 14:57:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903479; bh=nJDJ9NE+3mcyJNnGkZl+cobDBvPDump2PA8ld/F5o/w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UkPLrlyFezs3jORrA3aROLm9iXC6Kr9vZjNIJzD6TiCZIqDJsxH49o0Pu+gEOK/gU VQeKq8IgWZHG4gxyuGcDrQwf0pp5Ti0KFT8is/rChRqO0i3zkfDEZmL9fQi/Yrx6Wj gHPD02kEB1754XbX7Dra6eV9gfEWk+l0+gyV+8AtChmvojBDWyWK4jNMnQ6wv8Ytdv W7YJLyXvLt9mx73HO6vwf+V0/FvXaM4esp+YR7xgUlVMxUGXnkUQ0kiaiaLLtHbtN/ QB9T/Gofz00zxK9cWCo+6XUlba0SMqDf2WOJ/jm6328ZydxnvV7MKyLnlDsJQZIFjQ M+xECs9iopGtA== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 1/7] riscv: mm: Pre-allocate PGD leaves to avoid synchronization Date: Fri, 12 May 2023 16:57:31 +0200 Message-Id: <20230512145737.985671-2-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701238807704491?= X-GMAIL-MSGID: =?utf-8?q?1765701238807704491?= From: Björn Töpel The RISC-V port copies PGD from init_mm to all userland pages-tables, which means that when the PGD level of the init_mm table is changed, other page-tables has to be updated. One way to avoid synchronizing page-tables is to pre-allocate the pages that are copied (need to be synchronized). For memory hotswapping builds, prefer to waste some pages, rather than do explicit synchronization. Prepare the RISC-V port for memory add/remove, by getting rid of PGD synchronization. Pre-allocate vmemmap, and direct map pages. This will roughly waste ~128 worth of 4K pages. Note that this is only done for memory hotswap enabled configuration. Signed-off-by: Björn Töpel --- arch/riscv/include/asm/kasan.h | 4 +- arch/riscv/mm/init.c | 86 ++++++++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/kasan.h b/arch/riscv/include/asm/kasan.h index 0b85e363e778..e6a0071bdb56 100644 --- a/arch/riscv/include/asm/kasan.h +++ b/arch/riscv/include/asm/kasan.h @@ -6,8 +6,6 @@ #ifndef __ASSEMBLY__ -#ifdef CONFIG_KASAN - /* * The following comment was copied from arm64: * KASAN_SHADOW_START: beginning of the kernel virtual addresses. @@ -34,6 +32,8 @@ */ #define KASAN_SHADOW_START ((KASAN_SHADOW_END - KASAN_SHADOW_SIZE) & PGDIR_MASK) #define KASAN_SHADOW_END MODULES_LOWEST_VADDR + +#ifdef CONFIG_KASAN #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) void kasan_init(void); diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 747e5b1ef02d..d2595cc33a1c 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include "../kernel/head.h" @@ -156,6 +157,90 @@ static void __init print_vm_layout(void) static void print_vm_layout(void) { } #endif /* CONFIG_DEBUG_VM */ +#ifdef CONFIG_MEMORY_HOTPLUG +/* + * Pre-allocates page-table pages for a specific area in the kernel + * page-table. Only the level which needs to be synchronized between + * all page-tables is allocated because the synchronization can be + * expensive. + */ +static void __init preallocate_pgd_pages_range(unsigned long start, unsigned long end, + const char *area) +{ + unsigned long addr; + const char *lvl; + + for (addr = start; addr < end; addr = ALIGN(addr + 1, PGDIR_SIZE)) { + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + + lvl = "p4d"; + p4d = p4d_alloc(&init_mm, pgd, addr); + if (!p4d) + goto failed; + + if (pgtable_l5_enabled) + continue; + + /* + * The goal here is to allocate all possibly required + * hardware page tables pointed to by the top hardware + * level. + * + * On 4-level systems, the P4D layer is folded away + * and the above code does no preallocation. Below, + * go down to the pud _software_ level to ensure the + * second hardware level is allocated on 4-level + * systems too. + */ + lvl = "pud"; + pud = pud_alloc(&init_mm, p4d, addr); + if (!pud) + goto failed; + + if (pgtable_l4_enabled) + continue; + /* + * The goal here is to allocate all possibly required + * hardware page tables pointed to by the top hardware + * level. + * + * On 3-level systems, the PUD layer is folded away + * and the above code does no preallocation. Below, + * go down to the pmd _software_ level to ensure the + * second hardware level is allocated on 3-level + * systems too. + */ + lvl = "pmd"; + pmd = pmd_alloc(&init_mm, pud, addr); + if (!pmd) + goto failed; + } + + return; + +failed: + + /* + * The pages have to be there now or they will be missing in + * process page-tables later. + */ + panic("Failed to pre-allocate %s pages for %s area\n", lvl, area); +} + +#define PAGE_END KASAN_SHADOW_START +#endif + +static void __init prepare_memory_hotplug(void) +{ +#ifdef CONFIG_MEMORY_HOTPLUG + preallocate_pgd_pages_range(VMEMMAP_START, VMEMMAP_END, "vmemmap"); + preallocate_pgd_pages_range(PAGE_OFFSET, PAGE_END, "direct map"); +#endif +} + void __init mem_init(void) { #ifdef CONFIG_FLATMEM @@ -164,6 +249,7 @@ void __init mem_init(void) swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE); memblock_free_all(); + prepare_memory_hotplug(); print_vm_layout(); } From patchwork Fri May 12 14:57:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93230 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5176293vqo; Fri, 12 May 2023 08:06:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ70yFEpTL4CBY7CtsH13CzipKXmhxtSNSDLr7AYdPMI2f+dX0g+UNIk5zsHNkBH4nl8fi6x X-Received: by 2002:a05:6a21:78a9:b0:102:a593:a158 with SMTP id bf41-20020a056a2178a900b00102a593a158mr13193388pzc.47.1683903985218; Fri, 12 May 2023 08:06:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683903985; cv=none; d=google.com; s=arc-20160816; b=dhHAQsNzsEgqJu0EvqgI40lFXXtmX44SBVajH5F2eBs1E7J2Vp833Y1ahByvX/yWlf qlPXWFig6Jhwh5rjbE0+pOue639rUcfRUIHTusho6mvQ77r8gr28gf1CdzO77Ah3d9V3 yf2Ga19s3xlASfR0dEzili/mfjLo6Fr/ooQsMMVsgrNjTEX+sFIezfpUZKiyif/YTYwW aUr6TNl3SpZDuz0YjpwUZh8tpBUMC+im19c9XA1yjpSYUL7bidmG0+eZ3bOY2PUd9ct9 u+eTJJ/qrqSrsh7aZ9WM9/fW1kVVolzO82gqIcHM+bKGuyWZoW1/BnsHxWbcnIE1o0Hx M3SA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kH8QXAEm7/YCy5ID7Hw0LisT0Mphin+ZXa6tQSzWqm8=; b=I9sp4At22z42PjM2WBhF+X95LvC5Gq/OFwKu5iGpJze7tC57kGirD6kqdKQ2CJ3bQX +1JABQwyzpIyaFabBOKjgezDhVTAdrw9X3LF/sC/4eFW18UNGyBGbc7A33Q8BgcELbEg oPC/qJmEJ22ErQnE1KbX9k/dGuyofy/OMDIrIj+Cl4ox2Vv2wAmouXF5qfU/+4alOdvt e7jAHKEbmQnPxJuoDTR0gOuiYQ0D/PGIRQ51ceABoe5n6dwRpEAQHqTpfV8XXREhwF7o gnoYFkcVNUNOVet8evg2W3MXpJxN/cEgNhnUqZ1SJ3DaMNjOpbeZc6nxi5Uw44p4LOzJ g/cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=rWaW1yO4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w68-20020a626247000000b0063b61b49a29si4150069pfb.251.2023.05.12.08.06.10; Fri, 12 May 2023 08:06:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=rWaW1yO4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241620AbjELO6T (ORCPT + 99 others); Fri, 12 May 2023 10:58:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241616AbjELO6F (ORCPT ); Fri, 12 May 2023 10:58:05 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB7FF3A86 for ; Fri, 12 May 2023 07:58:03 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 54A3561137 for ; Fri, 12 May 2023 14:58:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 057B2C433EF; Fri, 12 May 2023 14:57:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903482; bh=TbHgG2w0XhFAPXkR+9W7hs3AnwbEfk/0KY6kqBQIJeI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rWaW1yO4zoG2gWhzY8P7BBZ0myq/alPhT7G9vRjXjVAcXidr8fdCS6X5CaHFGusk7 qjVS7b8XygHKs5JocjZwRQDxE+2oIuGJdZK09eCTMn0jLx8sOjkhFdpD3WL7S+BBgX r4z2SQVi2ZV24FFoPCpEtzWv+8VokXA3/LhdoSwVI7mcsZQJ/GkKl2yHLw+2PwlzvI 8GE3RPwss0CYTl9KkCRlDemA4GHpeMZ15fFn4z7MXRJg1TGizz45LBk1tMwq4mOHkO 9ILUr0Iz6q6JnHnIL/zxOMu9dQ/j4NYpc9GZAgm4PtOIR5ohWNiccDnmE/1e2KtNht 3fAZL7Z6pcaJA== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 2/7] riscv: mm: Change attribute from __init to __meminit for page functions Date: Fri, 12 May 2023 16:57:32 +0200 Message-Id: <20230512145737.985671-3-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701305566794984?= X-GMAIL-MSGID: =?utf-8?q?1765701305566794984?= From: Björn Töpel Prepare for memory hot add/remove support by changing from __init to __meminit for the page-table functions that are used by the upcoming arch specific callbacks. Changing the __init attribute to __meminit, avoids that the functions are removed after init. __meminit keeps the functions after init, if memory hotplugging is enabled for the build. Signed-off-by: Björn Töpel --- arch/riscv/include/asm/mmu.h | 2 +- arch/riscv/include/asm/pgtable.h | 2 +- arch/riscv/mm/init.c | 49 ++++++++++++++------------------ 3 files changed, 24 insertions(+), 29 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 0099dc116168..9e5d4f37ba2e 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -22,7 +22,7 @@ typedef struct { #endif } mm_context_t; -void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, +void __meminit create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 2258b27173b0..a4cdcb689959 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -147,7 +147,7 @@ struct pt_alloc_ops { #endif }; -extern struct pt_alloc_ops pt_ops __initdata; +extern struct pt_alloc_ops pt_ops __meminitdata; #ifdef CONFIG_MMU /* Number of PGD entries that a user-mode program can use */ diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index d2595cc33a1c..e974ff6ef036 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -356,7 +356,7 @@ static void __init setup_bootmem(void) } #ifdef CONFIG_MMU -struct pt_alloc_ops pt_ops __initdata; +struct pt_alloc_ops pt_ops __meminitdata; pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss; pgd_t trampoline_pg_dir[PTRS_PER_PGD] __page_aligned_bss; @@ -418,7 +418,7 @@ static inline pte_t *__init get_pte_virt_fixmap(phys_addr_t pa) return (pte_t *)set_fixmap_offset(FIX_PTE, pa); } -static inline pte_t *__init get_pte_virt_late(phys_addr_t pa) +static inline pte_t *__meminit get_pte_virt_late(phys_addr_t pa) { return (pte_t *) __va(pa); } @@ -437,7 +437,7 @@ static inline phys_addr_t __init alloc_pte_fixmap(uintptr_t va) return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE); } -static phys_addr_t __init alloc_pte_late(uintptr_t va) +static phys_addr_t __meminit alloc_pte_late(uintptr_t va) { unsigned long vaddr; @@ -447,9 +447,8 @@ static phys_addr_t __init alloc_pte_late(uintptr_t va) return __pa(vaddr); } -static void __init create_pte_mapping(pte_t *ptep, - uintptr_t va, phys_addr_t pa, - phys_addr_t sz, pgprot_t prot) +static void __meminit create_pte_mapping(pte_t *ptep, uintptr_t va, phys_addr_t pa, phys_addr_t sz, + pgprot_t prot) { uintptr_t pte_idx = pte_index(va); @@ -503,7 +502,7 @@ static pmd_t *__init get_pmd_virt_fixmap(phys_addr_t pa) return (pmd_t *)set_fixmap_offset(FIX_PMD, pa); } -static pmd_t *__init get_pmd_virt_late(phys_addr_t pa) +static pmd_t *__meminit get_pmd_virt_late(phys_addr_t pa) { return (pmd_t *) __va(pa); } @@ -520,7 +519,7 @@ static phys_addr_t __init alloc_pmd_fixmap(uintptr_t va) return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE); } -static phys_addr_t __init alloc_pmd_late(uintptr_t va) +static phys_addr_t __meminit alloc_pmd_late(uintptr_t va) { unsigned long vaddr; @@ -530,9 +529,8 @@ static phys_addr_t __init alloc_pmd_late(uintptr_t va) return __pa(vaddr); } -static void __init create_pmd_mapping(pmd_t *pmdp, - uintptr_t va, phys_addr_t pa, - phys_addr_t sz, pgprot_t prot) +static void __meminit create_pmd_mapping(pmd_t *pmdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, + pgprot_t prot) { pte_t *ptep; phys_addr_t pte_phys; @@ -568,7 +566,7 @@ static pud_t *__init get_pud_virt_fixmap(phys_addr_t pa) return (pud_t *)set_fixmap_offset(FIX_PUD, pa); } -static pud_t *__init get_pud_virt_late(phys_addr_t pa) +static pud_t *__meminit get_pud_virt_late(phys_addr_t pa) { return (pud_t *)__va(pa); } @@ -586,7 +584,7 @@ static phys_addr_t __init alloc_pud_fixmap(uintptr_t va) return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE); } -static phys_addr_t alloc_pud_late(uintptr_t va) +static phys_addr_t __meminit alloc_pud_late(uintptr_t va) { unsigned long vaddr; @@ -606,7 +604,7 @@ static p4d_t *__init get_p4d_virt_fixmap(phys_addr_t pa) return (p4d_t *)set_fixmap_offset(FIX_P4D, pa); } -static p4d_t *__init get_p4d_virt_late(phys_addr_t pa) +static p4d_t *__meminit get_p4d_virt_late(phys_addr_t pa) { return (p4d_t *)__va(pa); } @@ -624,7 +622,7 @@ static phys_addr_t __init alloc_p4d_fixmap(uintptr_t va) return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE); } -static phys_addr_t alloc_p4d_late(uintptr_t va) +static phys_addr_t __meminit alloc_p4d_late(uintptr_t va) { unsigned long vaddr; @@ -633,9 +631,8 @@ static phys_addr_t alloc_p4d_late(uintptr_t va) return __pa(vaddr); } -static void __init create_pud_mapping(pud_t *pudp, - uintptr_t va, phys_addr_t pa, - phys_addr_t sz, pgprot_t prot) +static void __meminit create_pud_mapping(pud_t *pudp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, + pgprot_t prot) { pmd_t *nextp; phys_addr_t next_phys; @@ -660,9 +657,8 @@ static void __init create_pud_mapping(pud_t *pudp, create_pmd_mapping(nextp, va, pa, sz, prot); } -static void __init create_p4d_mapping(p4d_t *p4dp, - uintptr_t va, phys_addr_t pa, - phys_addr_t sz, pgprot_t prot) +static void __meminit create_p4d_mapping(p4d_t *p4dp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, + pgprot_t prot) { pud_t *nextp; phys_addr_t next_phys; @@ -718,9 +714,8 @@ static void __init create_p4d_mapping(p4d_t *p4dp, #define create_pmd_mapping(__pmdp, __va, __pa, __sz, __prot) do {} while(0) #endif /* __PAGETABLE_PMD_FOLDED */ -void __init create_pgd_mapping(pgd_t *pgdp, - uintptr_t va, phys_addr_t pa, - phys_addr_t sz, pgprot_t prot) +void __meminit create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, + pgprot_t prot) { pgd_next_t *nextp; phys_addr_t next_phys; @@ -745,7 +740,7 @@ void __init create_pgd_mapping(pgd_t *pgdp, create_pgd_next_mapping(nextp, va, pa, sz, prot); } -static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) +static uintptr_t __meminit best_map_size(phys_addr_t base, phys_addr_t size) { if (!(base & (PGDIR_SIZE - 1)) && size >= PGDIR_SIZE) return PGDIR_SIZE; @@ -778,7 +773,7 @@ asmlinkage void __init __copy_data(void) #endif #ifdef CONFIG_STRICT_KERNEL_RWX -static __init pgprot_t pgprot_from_va(uintptr_t va) +static __meminit pgprot_t pgprot_from_va(uintptr_t va) { if (is_va_kernel_text(va)) return PAGE_KERNEL_READ_EXEC; @@ -805,7 +800,7 @@ void mark_rodata_ro(void) debug_checkwx(); } #else -static __init pgprot_t pgprot_from_va(uintptr_t va) +static __meminit pgprot_t pgprot_from_va(uintptr_t va) { if (IS_ENABLED(CONFIG_64BIT) && !is_kernel_mapping(va)) return PAGE_KERNEL; From patchwork Fri May 12 14:57:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93231 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5180167vqo; Fri, 12 May 2023 08:10:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7KP4X3x4Y2lt554FXsc6J8zytGWM63K14oTD6cc20u7o5aHwdHxLtHqTf1cZ8Gz+PZBdxJ X-Received: by 2002:a17:90a:af86:b0:250:5194:b135 with SMTP id w6-20020a17090aaf8600b002505194b135mr20650706pjq.34.1683904233835; Fri, 12 May 2023 08:10:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683904233; cv=none; d=google.com; s=arc-20160816; b=i3O5rMvqON0Ukv1BAqHMBgN9g5HAl/slwRWRveJo2b9+KloRQHUsPJk7NrlvUmaeD+ 34xHYMYJv+TBaxKeo+2lAwQ22vX6EtFPPsbKYB3++juXaMgx2fHIbhSUU2Z608DVGWpy X+RuUnhHQesDwyK8j13uERNKGFtlQ4zlB/8PEQTec0Vd2h4Tj3wdCKjuQI/COy5q6RN+ yDZ2atuuJq2X2bCAlHH/xjQeeBSak1v1MmWD0H0e0Ncv7bN/tftK9Mvyyc1qeMN/ro24 5n/ugp6EOy4KIzRp9PjvOavG3ClUV3p1zeYJthpA3qeqvw1JFmMHrfmumzJjZvIdR1tF tgBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HpZgKr5fu0P/l7DuXAWJOQwuEwXmxH+7XtfSyg24V+Y=; b=nDIbFiGJzrbJ6OBpiff2/LLf8OLQ0S+7EN0DVYw0h+MtrytJ+KzDdaIHQPR2zoycA2 8YhYj8alSfLD3dMgTfU90xQER92cm6MHXKgQlgHvoEg85M0nzliO3tFL80YK21oUfY3i 51XfB7GtdqkaZd/SH8XDl23bSVTIJsE6OGs8sxKIYijfI5Fke5IpY6tEluz/nB0yVRM2 +v5Z55Tr46loA03KXeUECXNusZA/BM7qwjnG0WgeUWotWW0ixEhFsnG/rDXRsYlV4X3N /msw6BnBld3GWcFRuSZjwwc04TKNrrn4r1/RzZ6zy3WokaLkU8GoN6l6AoPw3EEYu9KQ oYkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=YPQSWSrA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j184-20020a6380c1000000b0050f736a8d3dsi10465637pgd.64.2023.05.12.08.10.21; Fri, 12 May 2023 08:10:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=YPQSWSrA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241437AbjELO6X (ORCPT + 99 others); Fri, 12 May 2023 10:58:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241623AbjELO6H (ORCPT ); Fri, 12 May 2023 10:58:07 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5A3E35B3 for ; Fri, 12 May 2023 07:58:06 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 629A461137 for ; Fri, 12 May 2023 14:58:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F83CC433D2; Fri, 12 May 2023 14:58:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903485; bh=aaVyrjFt4Z24qTU7IkdzOXPq++vuBvyBWDRYOQbS9VA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YPQSWSrA4888qRlom2V9n0E4yFAcDlwO6kLEPA1Z3/jN+kNGhPXLeY8aFL0ESQTul RMyaIs6qUO82q50vSJrCQWrwL5FL7Zd2It0w/k09t7jeuI9/8zofE8JzaXyKCEZlUs 6vZOfEid0lLW22Wa0vb8u5t7SQjIJphqI/J9HwOCpQIywm81oMLE6fnM89OUtnUkFn j8XQGA0U7Xt+BTOHhoHC7wsFBB8N8mbrfH+4xlCjWl8xkta+DzSiBxsCwtLgzfktYM MaJ2hhSLpdhYY4k+7RXHrTEDITwtEA6t4tJs5cfx65BsquOpd3IWQp5793KvR6uCq6 t5vc9nXQmqCFQ== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 3/7] riscv: mm: Refactor create_linear_mapping_range() for hot add Date: Fri, 12 May 2023 16:57:33 +0200 Message-Id: <20230512145737.985671-4-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701565742645769?= X-GMAIL-MSGID: =?utf-8?q?1765701565742645769?= From: Björn Töpel Add a parameter to the direct map setup function, so it can be used in arch_add_memory() later. Signed-off-by: Björn Töpel --- arch/riscv/mm/init.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index e974ff6ef036..aea8ccb3f4ae 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -1247,18 +1247,19 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) pt_ops_set_fixmap(); } -static void __init create_linear_mapping_range(phys_addr_t start, - phys_addr_t end) +static void __meminit create_linear_mapping_range(phys_addr_t start, phys_addr_t end, + struct mhp_params *params) { phys_addr_t pa; uintptr_t va, map_size; for (pa = start; pa < end; pa += map_size) { + pgprot_t pgprot; + va = (uintptr_t)__va(pa); + pgprot = params ? params->pgprot : pgprot_from_va(va); map_size = best_map_size(pa, end - pa); - - create_pgd_mapping(swapper_pg_dir, va, pa, map_size, - pgprot_from_va(va)); + create_pgd_mapping(swapper_pg_dir, va, pa, map_size, pgprot); } } @@ -1288,13 +1289,12 @@ static void __init create_linear_mapping_page_table(void) if (end >= __pa(PAGE_OFFSET) + memory_limit) end = __pa(PAGE_OFFSET) + memory_limit; - create_linear_mapping_range(start, end); + create_linear_mapping_range(start, end, NULL); } #ifdef CONFIG_STRICT_KERNEL_RWX - create_linear_mapping_range(ktext_start, ktext_start + ktext_size); - create_linear_mapping_range(krodata_start, - krodata_start + krodata_size); + create_linear_mapping_range(ktext_start, ktext_start + ktext_size, NULL); + create_linear_mapping_range(krodata_start, krodata_start + krodata_size, NULL); memblock_clear_nomap(ktext_start, ktext_size); memblock_clear_nomap(krodata_start, krodata_size); From patchwork Fri May 12 14:57:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93229 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5175940vqo; Fri, 12 May 2023 08:06:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7A40lZ4rwiXcxRI/bp03K1ondFiSbb2M5+KlQsFxCBuUxgfg1Smoemr2U90tja4o/FF8gG X-Received: by 2002:a17:902:b904:b0:1ac:4bf1:a29f with SMTP id bf4-20020a170902b90400b001ac4bf1a29fmr23539063plb.59.1683903961899; Fri, 12 May 2023 08:06:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683903961; cv=none; d=google.com; s=arc-20160816; b=XEoODe+iyOF+aHXT+fYv2//WNgdt6gWUlrLj1H7UenhXNsftzNRErU+SXYr6Rsi0tx eUvWPKRKqsGFSyQseTfY0nT/EVgzvITGgFBy93MDJiljdRb5UWkdxdsVcDjT6Fvpb1Eo d3xAyggiz2rDHIPkj2hcJSUFKz4r6SaF/zqPM5vV6nVgYtN/NK0NScUHbHZvfkMT1GZg CAdVYPWdYriWqxeipSJ+P6Vr05NfMmW6crhBw9Xz7DkuWOl0iCaFwe/PzcQA7VxeyrFQ Z/nxFwLCb2FU2ZhUYQfBfLm4fGmAJ8AVl1eBHa3Yhd3Wk4+aMluSeHPCVRdZlTzD1gqq 3IBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=e1oESrIyym/GlTQwMzdbC00z/SKfXrvdk/+pJfjHr38=; b=IQTBdCCGuIq1tY1MtQOd/dAAhNVmMnC9epDDplYBwCryMzk0bhtTHdbZbpsolP7+Si /E1FJYE4ZUFgNv1xH2QrfjvpoyPnJWDssUyd1jfjFN1UX1MpOtG+MM1cF6RSS0ZkH3Sm FhK+jcjtQSA6wm+/ME8A2MZWQsuOCp0pCfSo6h0r/ynxCewmi37YACoBaOdFVj8+Cs2r aY6uOtOWQ2iOwuxZo4Jqg7VsMjiK969mOqv4xZ0xMsoDKoYvfZLR/JG7YXQmSFzO4PM9 l5R/JQMx5BgVOwBJCwxJJ16/R0DFoQN4s6BrM/dS6XZKeG3spKKaLWicc428mTgqIUEd hrew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Os5JiE+9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n8-20020a170902d2c800b001a6ce2cdb20si9725862plc.244.2023.05.12.08.05.46; Fri, 12 May 2023 08:06:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Os5JiE+9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241629AbjELO60 (ORCPT + 99 others); Fri, 12 May 2023 10:58:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241592AbjELO6L (ORCPT ); Fri, 12 May 2023 10:58:11 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E13603A86 for ; Fri, 12 May 2023 07:58:09 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7BED961542 for ; Fri, 12 May 2023 14:58:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 414DCC433EF; Fri, 12 May 2023 14:58:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903488; bh=2Hg7N6ebdPoIJZTj/Z2XT0CsZmR9YcBL0bRVObnrrco=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Os5JiE+94pc+yBVvOpqjMSDiGSPKKDvcZURYVV7tm3zq/Llfx7r+svdbETNcWHa9V Li9vVWnnYPg4WmY1beATZukfwTwxRLiQJpQmEs/GjF49Mf4C7W1sm0VTzGAs3LvJ4V fiW7zPjP32r2q8vcrAiansu0eVnkhKadKCYOX+a3UFnS4f2mBOnAYzv6slNn4OKi0T BBktSF+bxWWWpHNDAtW4wOAZ8gQ/LY6hbDNeK7wW+z40xpd+VuFNYevwb5gOiYtXJB 0A/zrgC1iRCSbGLV+DiScyv8mJHKg3G9Tak/03DMMiegzuW6pG8j5kuNpkGbWaRLoJ nuEOcUj9yW5Aw== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 4/7] riscv: mm: Add memory hot add/remove support Date: Fri, 12 May 2023 16:57:34 +0200 Message-Id: <20230512145737.985671-5-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701280952325685?= X-GMAIL-MSGID: =?utf-8?q?1765701280952325685?= From: Björn Töpel From an arch perspective, a couple of callbacks needs to be implemented to support hotplugging: arch_add_memory() This callback is responsible for updating the linear/direct map, and call into the memory hotplugging generic code via __add_pages(). arch_remove_memory() In this callback the linear/direct map is tore down. vmemmap_free() The function tears down the vmemmap mappings (if CONFIG_SPARSEMEM_VMEMMAP is in-use), and also deallocates the backing vmemmap pages. Note that for persistent memory, an alternative allocator for the backing pages can be used -- the vmem_altmap. This means that when the backing pages are cleared, extra care is needed so that the correct deallocation method is used. Note that RISC-V populates the vmemmap using vmemmap_populate_basepages(), so currently no hugepages are used for the backing store. The page table unmap/teardown functions are heavily based (copied!) from the x86 tree. The same remove_pgd_mapping() is used in both vmemmap_free() and arch_remove_memory(), but in the latter function the backing pages are not removed. Signed-off-by: Björn Töpel --- arch/riscv/mm/init.c | 233 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 233 insertions(+) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index aea8ccb3f4ae..a468708d1e1c 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -1444,3 +1444,236 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, return vmemmap_populate_basepages(start, end, node, NULL); } #endif + +#ifdef CONFIG_MEMORY_HOTPLUG +static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd) +{ + pte_t *pte; + int i; + + for (i = 0; i < PTRS_PER_PTE; i++) { + pte = pte_start + i; + if (!pte_none(*pte)) + return; + } + + free_pages((unsigned long)page_address(pmd_page(*pmd)), 0); + pmd_clear(pmd); +} + +static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud) +{ + pmd_t *pmd; + int i; + + for (i = 0; i < PTRS_PER_PMD; i++) { + pmd = pmd_start + i; + if (!pmd_none(*pmd)) + return; + } + + free_pages((unsigned long)page_address(pud_page(*pud)), 0); + pud_clear(pud); +} + +static void __meminit free_pud_table(pud_t *pud_start, p4d_t *p4d) +{ + pud_t *pud; + int i; + + for (i = 0; i < PTRS_PER_PUD; i++) { + pud = pud_start + i; + if (!pud_none(*pud)) + return; + } + + free_pages((unsigned long)page_address(p4d_page(*p4d)), 0); + p4d_clear(p4d); +} + +static void __meminit free_vmemmap_storage(struct page *page, size_t size, + struct vmem_altmap *altmap) +{ + if (altmap) + vmem_altmap_free(altmap, size >> PAGE_SHIFT); + else + free_pages((unsigned long)page_address(page), get_order(size)); +} + +static void __meminit remove_pte_mapping(pte_t *pte_base, unsigned long addr, unsigned long end, + bool is_vmemmap, struct vmem_altmap *altmap) +{ + unsigned long next; + pte_t *ptep, pte; + + for (; addr < end; addr = next) { + next = (addr + PAGE_SIZE) & PAGE_MASK; + if (next > end) + next = end; + + ptep = pte_base + pte_index(addr); + pte = READ_ONCE(*ptep); + + if (!pte_present(*ptep)) + continue; + + pte_clear(&init_mm, addr, ptep); + if (is_vmemmap) + free_vmemmap_storage(pte_page(pte), PAGE_SIZE, altmap); + } +} + +static void __meminit remove_pmd_mapping(pmd_t *pmd_base, unsigned long addr, unsigned long end, + bool is_vmemmap, struct vmem_altmap *altmap) +{ + unsigned long next; + pte_t *pte_base; + pmd_t *pmdp, pmd; + + for (; addr < end; addr = next) { + next = pmd_addr_end(addr, end); + pmdp = pmd_base + pmd_index(addr); + pmd = READ_ONCE(*pmdp); + + if (!pmd_present(pmd)) + continue; + + if (pmd_leaf(pmd)) { + pmd_clear(pmdp); + if (is_vmemmap) + free_vmemmap_storage(pmd_page(pmd), PMD_SIZE, altmap); + continue; + } + + pte_base = (pte_t *)pmd_page_vaddr(*pmdp); + remove_pte_mapping(pte_base, addr, next, is_vmemmap, altmap); + free_pte_table(pte_base, pmdp); + } +} + +static void __meminit remove_pud_mapping(pud_t *pud_base, unsigned long addr, unsigned long end, + bool is_vmemmap, struct vmem_altmap *altmap) +{ + unsigned long next; + pud_t *pudp, pud; + pmd_t *pmd_base; + + for (; addr < end; addr = next) { + next = pud_addr_end(addr, end); + pudp = pud_base + pud_index(addr); + pud = READ_ONCE(*pudp); + + if (!pud_present(pud)) + continue; + + if (pud_leaf(pud)) { + if (pgtable_l4_enabled) { + pud_clear(pudp); + if (is_vmemmap) + free_vmemmap_storage(pud_page(pud), PUD_SIZE, altmap); + } + continue; + } + + pmd_base = pmd_offset(pudp, 0); + remove_pmd_mapping(pmd_base, addr, next, is_vmemmap, altmap); + + if (pgtable_l4_enabled) + free_pmd_table(pmd_base, pudp); + } +} + +static void __meminit remove_p4d_mapping(p4d_t *p4d_base, unsigned long addr, unsigned long end, + bool is_vmemmap, struct vmem_altmap *altmap) +{ + unsigned long next; + p4d_t *p4dp, p4d; + pud_t *pud_base; + + for (; addr < end; addr = next) { + next = p4d_addr_end(addr, end); + p4dp = p4d_base + p4d_index(addr); + p4d = READ_ONCE(*p4dp); + + if (!p4d_present(p4d)) + continue; + + if (p4d_leaf(p4d)) { + if (pgtable_l5_enabled) { + p4d_clear(p4dp); + if (is_vmemmap) + free_vmemmap_storage(p4d_page(p4d), P4D_SIZE, altmap); + } + continue; + } + + pud_base = pud_offset(p4dp, 0); + remove_pud_mapping(pud_base, addr, next, is_vmemmap, altmap); + + if (pgtable_l5_enabled) + free_pud_table(pud_base, p4dp); + } +} + +static void __meminit remove_pgd_mapping(unsigned long va, unsigned long end, bool is_vmemmap, + struct vmem_altmap *altmap) +{ + unsigned long addr, next; + p4d_t *p4d_base; + pgd_t *pgd; + + for (addr = va; addr < end; addr = next) { + next = pgd_addr_end(addr, end); + pgd = pgd_offset_k(addr); + + if (!pgd_present(*pgd)) + continue; + + if (pgd_leaf(*pgd)) + continue; + + p4d_base = p4d_offset(pgd, 0); + remove_p4d_mapping(p4d_base, addr, next, is_vmemmap, altmap); + } + + flush_tlb_all(); +} + +static void __meminit remove_linear_mapping(phys_addr_t start, u64 size) +{ + unsigned long va = (unsigned long)__va(start); + unsigned long end = (unsigned long)__va(start + size); + + remove_pgd_mapping(va, end, false, NULL); +} + +int __ref arch_add_memory(int nid, u64 start, u64 size, struct mhp_params *params) +{ + int ret; + + create_linear_mapping_range(start, start + size, params); + flush_tlb_all(); + ret = __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT, params); + if (ret) { + remove_linear_mapping(start, size); + return ret; + } + + max_pfn = PFN_UP(start + size); + max_low_pfn = max_pfn; + return 0; +} + +void __ref arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) +{ + __remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT, altmap); + remove_linear_mapping(start, size); +} + +#ifdef CONFIG_SPARSEMEM_VMEMMAP +void __ref vmemmap_free(unsigned long start, unsigned long end, struct vmem_altmap *altmap) +{ + remove_pgd_mapping(start, end, true, altmap); +} +#endif /* CONFIG_SPARSEMEM_VMEMMAP */ +#endif /* CONFIG_MEMORY_HOTPLUG */ From patchwork Fri May 12 14:57:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93233 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5181720vqo; Fri, 12 May 2023 08:12:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ60lq/crqzZtdF1/fOm6L/CK6eQ1bTM2MvhYYu6tlBgOLzfw7/JsG1JO5+6MFn+Ah4kvUMt X-Received: by 2002:a17:90a:d252:b0:250:1d63:3227 with SMTP id o18-20020a17090ad25200b002501d633227mr25196480pjw.10.1683904346220; Fri, 12 May 2023 08:12:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683904346; cv=none; d=google.com; s=arc-20160816; b=MU0Talzueig2JtLThQ7A9HTl+crAPOmrwUI1gduCHwjUlsxEl85iv9/gmWxDkBvr1+ sqqmY/JKft30i278N3lDeqlInLggYf6PvtqITEQkMgjfl9yGNPgZQ+7yxAg3LHPWOv0o Qn86Xta45Px3GuzFAgchl1FnXHEyUyTUMaACJ/lTaT32Csiw4MPYwc1aud+eYj5tGoA2 wK1jr7HAAu4ldiujfYfIA23CyGI4zp4FAa1Dki8ijWb/yv6yQECOLJD2Q2n+yzVOxIQZ kohL0pK569bWM61nMpwkaJtHBnaNgIURZod+mgD3jRjFetORf2sxYLLe5rq5+kj2A7kx nJfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RHoSFbyGi0eW5u3jgCSI5Cav0nRvNf1gitJMGy1VIBE=; b=NY4u/dDDKwU1sn7R5tmdaZyVyw/uB08kcbzeSdGuXtaGV0pplBSfOELZ6Dtn4rG0lL Q9Ne2/SugUKdTU++GHgrwPXHnrfy2wrzJCAp4zz1SuL2qdQP03yMPmsMmaAk1D6AIMGw oWUYqshZC/Qus+MSRLaoxAXQ9WscmTmgnyrJd72NCmQqdK7TYOhOBefS7N/xcoSNNcVS yBycmHYqNynbkpFlcvLu2gcRTFRfdcDdMMNhMCx1yu7S4PLpKDa2kLo5/OYBCkyWeLJD 6hO4YvV0ftyQOl//GENwUZ+xcieurU7c1ceNqf7rb73AGj/c1fwPeUWttn0mbOXXJGzl HGNA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qQzflYqn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mq18-20020a17090b381200b0024e1c1b3004si29971631pjb.78.2023.05.12.08.12.12; Fri, 12 May 2023 08:12:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qQzflYqn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241600AbjELO6a (ORCPT + 99 others); Fri, 12 May 2023 10:58:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241605AbjELO6O (ORCPT ); Fri, 12 May 2023 10:58:14 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32C209022 for ; Fri, 12 May 2023 07:58:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B715F655F5 for ; Fri, 12 May 2023 14:58:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5593AC4339E; Fri, 12 May 2023 14:58:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903492; bh=Pna6hgXZPMsGHenSDkIIzlTOUDQrq5sDRKKv7+K0xpY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qQzflYqn6X1XwTYqDskdaUvkozpqmlKiTZ7sJZMcSXHaRn188NF1w4O4Gcg9IS09k oKwEhmh46ngeDtbZCpX71wFNrPl+kK3Y4p+ewFov1/UkgDjuFqVOTty0tiMMSElTNb yMuyw+QZmlU6JyZOeqC3QUttWJyV6GkHm73I62awiyhzXWWbPf3Lm1PjNuZoNrq0QB VApf+wgzHLHZ2xAEvvSCNJwzRHGvl/v5NT9pDPzmfOPYPJjDHZOW346ZF/PUDBnppQ xqqHnvoiJdxwAchWZf8t4aSIlDXEgYY06mtzf3klWqYHbjBSQG5Jhc3+1nVtzW5hkE X+9KbCvtUB98A== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 5/7] riscv: Enable memory hot add/remove arch kbuild support Date: Fri, 12 May 2023 16:57:35 +0200 Message-Id: <20230512145737.985671-6-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701683961656137?= X-GMAIL-MSGID: =?utf-8?q?1765701683961656137?= From: Björn Töpel Enable ARCH_ENABLE_MEMORY_HOTPLUG and ARCH_ENABLE_MEMORY_HOTREMOVE for RISC-V. Signed-off-by: Björn Töpel --- arch/riscv/Kconfig | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 348c0fa1fc8c..81b3f188f396 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -14,6 +14,8 @@ config RISCV def_bool y select ARCH_DMA_DEFAULT_COHERENT select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION + select ARCH_ENABLE_MEMORY_HOTPLUG if SPARSEMEM && 64BIT && MMU + select ARCH_ENABLE_MEMORY_HOTREMOVE if SPARSEMEM && 64BIT && MMU select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_BINFMT_FLAT From patchwork Fri May 12 14:57:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93232 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5180548vqo; Fri, 12 May 2023 08:11:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4E8o37/dqET5adoMxP04LXzuQIUWwi0YmCsMeaGdpaOGMH+t/l8ayuk1I2aX+i6S17TFCN X-Received: by 2002:a05:6a20:7d84:b0:100:d88c:4a23 with SMTP id v4-20020a056a207d8400b00100d88c4a23mr21479275pzj.52.1683904259880; Fri, 12 May 2023 08:10:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683904259; cv=none; d=google.com; s=arc-20160816; b=h1Vsi1GN8gz+AWT4q/YyX3ENnwelP85290HcvyanC5mTloMl242igZb2VyONz5KZai +vWdEbrmNdWPQdGxtfGGz3dKUWuHYGvwmLTCGJLyy1dGqiP/+9Ea1r0zqw9suuAGlu03 dUFKfFcfUPs0/kjb+Gz6FpxK/IrEQNFsVeo2DBbkEieb52jQrTxuZ80WymCgRg5YVm6z gvlxBvrU+69xhdBognJIR/ARiP4tJCZ28hCnMK+n/zDneN4cn0cGGsvaXGleo3E9fGxw giPwCiGjCDT+tA3hbX33qeUoe9fsjsTrnmThXqP3+UJQ3cqhsP9rRbTPKOyRhESQzOAx NMJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QELtVjSMwEIWESkCCY1X5XuG9zF4GxpJdhvIwi0XLec=; b=x3ylcy+PIZ/0+feNLkHeTYsippY67CX3P/ysTC5XZozwZDqfFjJeZZgn/TV9sU80Ph j5Ew84RwYssctrPEyIJxvT/orIfx/cWtw6T/uKBY6FoKk302vW6HIq4+lAaWw+FYSX01 6cAu/4e+7DK8yPrM1orRTtXCHI32wA8qiq4kph/tj5qRGziCQAWGlm2twWMHQ/zh+AZw aRLwmRs9QckC8PKEhnuDZL0jNA5/ob+7qhe1dtricY+rImbgdHQ9WV8mcvtJRqJglLoO DzV1egqrh5u0L/h+uyTT3yN6djBVXul8O9HJgt8Y1ilxgZRS5QzUk1jbgzTlJ4yU53Jc yrCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=NcLbHLWx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s4-20020a63b404000000b0053009ff9f39si9393175pgf.713.2023.05.12.08.10.36; Fri, 12 May 2023 08:10:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=NcLbHLWx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241647AbjELO6q (ORCPT + 99 others); Fri, 12 May 2023 10:58:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241641AbjELO6a (ORCPT ); Fri, 12 May 2023 10:58:30 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A518DC58 for ; Fri, 12 May 2023 07:58:16 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0A6A86576E for ; Fri, 12 May 2023 14:58:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF583C4339B; Fri, 12 May 2023 14:58:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903495; bh=OlUUIHh8F/ikEn4ruOXdEkA3wElniYzLmKIr7MhQ0Hw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NcLbHLWxqI2bumvNceUzMAPv+C0Y746k1Jp/7tXA1GgJqday/8MB/23BL0VcKD2EF l7sOoE8BAouxMF63X6G55suALJoO0H7hzI5XVSs9OXLwTPoBkriEnruif/KXQ3aW+/ SFMo23v3zgADZZc+8aPweKC+kX6VdWxTcQh0IOUvIyRD43+WepPIyPmdzGCt6wUJWx R49ozGhEMEXMJZEifDbl4mVjzatCqLEVcObGNbLsH/wJ2aox1/STvfILMriazGRm51 mnOGBU56o6Scmd5rVB7pPYs1z+zlGd1OdAwUcUAL4VkEWtJlONEsexD186+SZnZ1pr dXjChETIBrCjA== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 6/7] virtio-mem: Enable virtio-mem for RISC-V Date: Fri, 12 May 2023 16:57:36 +0200 Message-Id: <20230512145737.985671-7-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701593244737605?= X-GMAIL-MSGID: =?utf-8?q?1765701593244737605?= From: Björn Töpel Now that RISC-V has memory hot add/remove support, virtio-mem can be used on the platform. Signed-off-by: Björn Töpel --- drivers/virtio/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig index 0a53a61231c2..358e79ece169 100644 --- a/drivers/virtio/Kconfig +++ b/drivers/virtio/Kconfig @@ -117,7 +117,7 @@ config VIRTIO_BALLOON config VIRTIO_MEM tristate "Virtio mem driver" - depends on X86_64 || ARM64 + depends on X86_64 || ARM64 || RISCV depends on VIRTIO depends on MEMORY_HOTPLUG depends on MEMORY_HOTREMOVE From patchwork Fri May 12 14:57:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93234 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5181943vqo; Fri, 12 May 2023 08:12:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6y8whxxFw+r+gSBl64SbVaJBIeWno0L6OZ80H7TG/TOh0SFWCx7VFYZI55BRk0NzAxg9TF X-Received: by 2002:a05:6a20:3d1a:b0:101:5f22:b46d with SMTP id y26-20020a056a203d1a00b001015f22b46dmr17831765pzi.29.1683904363876; Fri, 12 May 2023 08:12:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683904363; cv=none; d=google.com; s=arc-20160816; b=iT29a4p4gal/4XyECmETcVOfFksQ5DSm/zqv/TrR9tNtgm0AKQPbctsXBkwgnnLf+6 o8yZrU450kBHo216jtlJfqEhHr0lUbnfABiFphyWoazjIXF14OUVHczR+lwkGWRx5pAt l6+HIKrmc4cbkx6TorVmiRZFi10r+6CoR7vZae8eD4coRgjgyHlEtI3WmjdkYPMWsuMG S1nt5Wph3y6CoOOzUwhNJ4BY90PxCZsaiN6Hnb5rZLQaGnvo+7ho31MEDrdDXBmnaI6N do/39N/KDLe8tjK2juzlaeUqGtWa1HVZkqO7Lyaf/3SuomZiA0ILUb+tHRU+eUBAWEQz joKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aj6GQIswVFDjikg1aHibrD6lxBUr92Ih68mWiYBsEIA=; b=rg7W0hx/c4g/bYCwUZ/Jsd09xw21JNuCo4vD8OpHu+P7mQUMAd4r2Ub9QgEC7AVRlW LSaWHlHI+WP+YfWaOJ+g7AXnY01rd5CxC6yUk+rbmCDm/qkn55erOozjpMIp/J9IC8xA t8OO8dtMeoQfULsOu/TVD+kvl4GTondWhxkU5rjRPz0Q/mA+SUIfkl0KVgztIJjU2DzW pvJGpGzm+uVlZVQLyT5uvBnevx7Jz+CMFc1IrCj/FeagGmxFppBdKTqZblPr8qNnnAs5 rHdIEBkFdScCEsdpGllCIjcRKbTh1a0iVtUSYDosrDLEXnQQkyPk0Dj4NuorpfABlLHr 2B8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lFJE9ofA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s4-20020a63b404000000b0053009ff9f39si9393175pgf.713.2023.05.12.08.12.28; Fri, 12 May 2023 08:12:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lFJE9ofA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241660AbjELO6w (ORCPT + 99 others); Fri, 12 May 2023 10:58:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241676AbjELO6d (ORCPT ); Fri, 12 May 2023 10:58:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80BBEDD8C for ; Fri, 12 May 2023 07:58:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4F19465735 for ; Fri, 12 May 2023 14:58:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9BF1C4339C; Fri, 12 May 2023 14:58:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903498; bh=lRwO3YfIMF9fmZSTE+/WO7BqWpiO2x+8QNF4La5hiEE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lFJE9ofAvx/W7GMthxymGs+b9O5oFO1BwdKIGcHaAN62YeWePDQ0N/sK71knTdSoF FVBIIqU4AmmF6IG79I2XtGg0JavzeJwFrHvwtyzSV76oVsE2HRceOMMnAX1c65IAP/ wzRAeUAfR4ibQ4s6pSnjOcUPEJrI0Uwkbd762zC/EYqhScoqAKaEuy84NEiMva3S8O cUjmxdLyvrJYLCFOB3H2n8gsVJcGT3aBKjaVsXZO/B0iymi2R+/sIoE+4yZ4U/slfd k0/FpzS+nNiLNCLb4BGq3Qd2jSem/r0rZlEnE2QEYIKn4OVGCS3P4sJXIb6FK2umT0 TTdE0MNOfw90w== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 7/7] riscv: mm: Pre-allocate vmalloc PGD leaves Date: Fri, 12 May 2023 16:57:37 +0200 Message-Id: <20230512145737.985671-8-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701702015998618?= X-GMAIL-MSGID: =?utf-8?q?1765701702015998618?= From: Björn Töpel Instead of relying on vmalloc_fault() to synchronize the page-tables, pre-allocate the PGD leaves of the vmalloc area. This is only enabled if memory hot/add is enabled by the build. Signed-off-by: Björn Töpel --- arch/riscv/mm/fault.c | 7 ++++++- arch/riscv/mm/init.c | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 8685f85a7474..b61e279acd50 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -233,12 +233,17 @@ void handle_page_fault(struct pt_regs *regs) * Fault-in kernel-space virtual memory on-demand. * The 'reference' page table is init_mm.pgd. * + * For memory hotplug enabled systems, the PGD entries are + * pre-allocated, which avoids the need to synchronize + * pgd/fault-in. + * * NOTE! We MUST NOT take any locks for this case. We may * be in an interrupt or a critical region, and should * only copy the information from the master page table, * nothing more. */ - if (unlikely((addr >= VMALLOC_START) && (addr < VMALLOC_END))) { + if (unlikely(!IS_ENABLED(CONFIG_MEMORY_HOTPLUG) && + (addr >= VMALLOC_START) && (addr < VMALLOC_END))) { vmalloc_fault(regs, code, addr); return; } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index a468708d1e1c..fd5a6d3fe182 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -236,6 +236,7 @@ static void __init preallocate_pgd_pages_range(unsigned long start, unsigned lon static void __init prepare_memory_hotplug(void) { #ifdef CONFIG_MEMORY_HOTPLUG + preallocate_pgd_pages_range(VMALLOC_START, VMALLOC_END, "vmalloc"); preallocate_pgd_pages_range(VMEMMAP_START, VMEMMAP_END, "vmemmap"); preallocate_pgd_pages_range(PAGE_OFFSET, PAGE_END, "direct map"); #endif