From patchwork Fri May 12 14:57:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 93234 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5181943vqo; Fri, 12 May 2023 08:12:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6y8whxxFw+r+gSBl64SbVaJBIeWno0L6OZ80H7TG/TOh0SFWCx7VFYZI55BRk0NzAxg9TF X-Received: by 2002:a05:6a20:3d1a:b0:101:5f22:b46d with SMTP id y26-20020a056a203d1a00b001015f22b46dmr17831765pzi.29.1683904363876; Fri, 12 May 2023 08:12:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683904363; cv=none; d=google.com; s=arc-20160816; b=iT29a4p4gal/4XyECmETcVOfFksQ5DSm/zqv/TrR9tNtgm0AKQPbctsXBkwgnnLf+6 o8yZrU450kBHo216jtlJfqEhHr0lUbnfABiFphyWoazjIXF14OUVHczR+lwkGWRx5pAt l6+HIKrmc4cbkx6TorVmiRZFi10r+6CoR7vZae8eD4coRgjgyHlEtI3WmjdkYPMWsuMG S1nt5Wph3y6CoOOzUwhNJ4BY90PxCZsaiN6Hnb5rZLQaGnvo+7ho31MEDrdDXBmnaI6N do/39N/KDLe8tjK2juzlaeUqGtWa1HVZkqO7Lyaf/3SuomZiA0ILUb+tHRU+eUBAWEQz joKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aj6GQIswVFDjikg1aHibrD6lxBUr92Ih68mWiYBsEIA=; b=rg7W0hx/c4g/bYCwUZ/Jsd09xw21JNuCo4vD8OpHu+P7mQUMAd4r2Ub9QgEC7AVRlW LSaWHlHI+WP+YfWaOJ+g7AXnY01rd5CxC6yUk+rbmCDm/qkn55erOozjpMIp/J9IC8xA t8OO8dtMeoQfULsOu/TVD+kvl4GTondWhxkU5rjRPz0Q/mA+SUIfkl0KVgztIJjU2DzW pvJGpGzm+uVlZVQLyT5uvBnevx7Jz+CMFc1IrCj/FeagGmxFppBdKTqZblPr8qNnnAs5 rHdIEBkFdScCEsdpGllCIjcRKbTh1a0iVtUSYDosrDLEXnQQkyPk0Dj4NuorpfABlLHr 2B8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lFJE9ofA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s4-20020a63b404000000b0053009ff9f39si9393175pgf.713.2023.05.12.08.12.28; Fri, 12 May 2023 08:12:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lFJE9ofA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241660AbjELO6w (ORCPT + 99 others); Fri, 12 May 2023 10:58:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241676AbjELO6d (ORCPT ); Fri, 12 May 2023 10:58:33 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80BBEDD8C for ; Fri, 12 May 2023 07:58:20 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4F19465735 for ; Fri, 12 May 2023 14:58:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E9BF1C4339C; Fri, 12 May 2023 14:58:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1683903498; bh=lRwO3YfIMF9fmZSTE+/WO7BqWpiO2x+8QNF4La5hiEE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lFJE9ofAvx/W7GMthxymGs+b9O5oFO1BwdKIGcHaAN62YeWePDQ0N/sK71knTdSoF FVBIIqU4AmmF6IG79I2XtGg0JavzeJwFrHvwtyzSV76oVsE2HRceOMMnAX1c65IAP/ wzRAeUAfR4ibQ4s6pSnjOcUPEJrI0Uwkbd762zC/EYqhScoqAKaEuy84NEiMva3S8O cUjmxdLyvrJYLCFOB3H2n8gsVJcGT3aBKjaVsXZO/B0iymi2R+/sIoE+4yZ4U/slfd k0/FpzS+nNiLNCLb4BGq3Qd2jSem/r0rZlEnE2QEYIKn4OVGCS3P4sJXIb6FK2umT0 TTdE0MNOfw90w== From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Hildenbrand , Oscar Salvador , virtualization@lists.linux-foundation.org, linux@rivosinc.com, Alexandre Ghiti Subject: [PATCH 7/7] riscv: mm: Pre-allocate vmalloc PGD leaves Date: Fri, 12 May 2023 16:57:37 +0200 Message-Id: <20230512145737.985671-8-bjorn@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512145737.985671-1-bjorn@kernel.org> References: <20230512145737.985671-1-bjorn@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765701702015998618?= X-GMAIL-MSGID: =?utf-8?q?1765701702015998618?= From: Björn Töpel Instead of relying on vmalloc_fault() to synchronize the page-tables, pre-allocate the PGD leaves of the vmalloc area. This is only enabled if memory hot/add is enabled by the build. Signed-off-by: Björn Töpel --- arch/riscv/mm/fault.c | 7 ++++++- arch/riscv/mm/init.c | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 8685f85a7474..b61e279acd50 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -233,12 +233,17 @@ void handle_page_fault(struct pt_regs *regs) * Fault-in kernel-space virtual memory on-demand. * The 'reference' page table is init_mm.pgd. * + * For memory hotplug enabled systems, the PGD entries are + * pre-allocated, which avoids the need to synchronize + * pgd/fault-in. + * * NOTE! We MUST NOT take any locks for this case. We may * be in an interrupt or a critical region, and should * only copy the information from the master page table, * nothing more. */ - if (unlikely((addr >= VMALLOC_START) && (addr < VMALLOC_END))) { + if (unlikely(!IS_ENABLED(CONFIG_MEMORY_HOTPLUG) && + (addr >= VMALLOC_START) && (addr < VMALLOC_END))) { vmalloc_fault(regs, code, addr); return; } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index a468708d1e1c..fd5a6d3fe182 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -236,6 +236,7 @@ static void __init preallocate_pgd_pages_range(unsigned long start, unsigned lon static void __init prepare_memory_hotplug(void) { #ifdef CONFIG_MEMORY_HOTPLUG + preallocate_pgd_pages_range(VMALLOC_START, VMALLOC_END, "vmalloc"); preallocate_pgd_pages_range(VMEMMAP_START, VMEMMAP_END, "vmemmap"); preallocate_pgd_pages_range(PAGE_OFFSET, PAGE_END, "direct map"); #endif