From patchwork Thu Mar 16 13:17:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 70792 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp487962wrt; Thu, 16 Mar 2023 06:39:58 -0700 (PDT) X-Google-Smtp-Source: AK7set83bkafUl1ecbIFRh0WmHV39Br22qCU9WtC/0IkHWOFeEwj3iG2uKhwZ5WqTz+uapUcUV2f X-Received: by 2002:aa7:9422:0:b0:625:89d5:59db with SMTP id y2-20020aa79422000000b0062589d559dbmr2878806pfo.28.1678973998157; Thu, 16 Mar 2023 06:39:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678973998; cv=none; d=google.com; s=arc-20160816; b=x41bjAyHgUb1hlm8+NcebCdmceZ93JlIJTnf2m3KjX1WGBgTxJRfDg1RDUw0v6klHU abZf3rWAUzWBSCPfjgDuySq5+kCIWs98a9Kwe2enqawz/n3D4UgAsDSZxjY5T5XjywRq qgTSHFRfnU7w9zNQlugujaiA2UTMi8QftirJe6uAI0+FAQKVOPepp73wOj3ingmNdcwp +rhngvW2SI0aJiFzSRZ2LHbCGI5ED+rWUd+tC1AMr/3cTGDhvE9K7urlh9YA/Z7VIZPN oCIyF2Qr8At9hgi6U4aS5KauMMZfShm9r/UFkMCUqO/Yz2/G1lJXN7DyGsp6oSkE8QTS K94g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Tp3HoPN4Gc82+RijS+v6sxY9JgB+vr3DxCDQErkACNI=; b=efO5GPvOCPrhgoT1gW4cROTi6yE5+u2DEPCYYPCtZhufpAGnFNz7LfSywirpEg7e2A eQsy03czqfASCn+6n/8X86a0bIh9Eq9xeNjd47CbP5NpS/hojwJfobXXvBoirVkJo7tp KkcedUeM9Y48+HuLlNC4vD3PJW5EZn09/RgyxMvaPG7+q7VJfvmAayZYtfR7zUSybQLg sreir+r6gvOTMHMBQEeTx7Mfl0S7g4ImTGXkXKT18XvPtMVAjFSvJf+X3/y9ljDrws6R gnDn7G1ZVW9KrQcQ/M+BM+jttviIE1R4xZ8Ci6wAXNO8xfwojczdk5aCfytuiwWtgYfS ngKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=PToYMu0c; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m22-20020a656a16000000b005023dfa3c68si8629388pgu.145.2023.03.16.06.39.43; Thu, 16 Mar 2023 06:39:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=PToYMu0c; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230147AbjCPNS6 (ORCPT + 99 others); Thu, 16 Mar 2023 09:18:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230011AbjCPNS4 (ORCPT ); Thu, 16 Mar 2023 09:18:56 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D87BCD654 for ; Thu, 16 Mar 2023 06:18:20 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id m35so1230479wms.4 for ; Thu, 16 Mar 2023 06:18:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1678972698; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tp3HoPN4Gc82+RijS+v6sxY9JgB+vr3DxCDQErkACNI=; b=PToYMu0cKs8JYZ6vE0jZ7YqptxvvNxzJLVidWaPVHOytoFMIFwdTLHn0duEcW9msRA 1DEp4a1ec5G0nWISGMpEGtVdHHRcqLIy32AocIm6p96VGsWXu1VqxQPHrribt2ncNfPa IT7NvAy3f1XFXxF4ab7MvjC7CER1BZR7JNhxXrN5STyfV/hVjikL5OIksSxcZfM5NZ7b p+sKuSERVfHhmhbBauv0yFqr6stSOvpGururNP0b9AutYpWisTZOOvIyYvb+WKc3PTc5 kSNH7H862sCAjFRcT5NtWiledbRkxgFXVqgQFo/OSB2M+TbcKKYrTnLSa0NrQWaJraNx warg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678972698; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tp3HoPN4Gc82+RijS+v6sxY9JgB+vr3DxCDQErkACNI=; b=lXGb49+JCQ1wX3tCspu78+KsHs5ywRRKlK4lE7kjn35gsDhBFIKrYQsQdygs6Udpxb Wt/t3eCKykALCglUmZsF4sO5th+iVw6gL2qhtKGT2vqSy8eHh88flukf/mwi+oUGq4Y2 yAZdRw3FItvEOzfbRf43CdacXP/zXDcKCQEZRd7QJap0Rm8xxRf1aP22xvuESR6ymUXk kORhSVAiQ1Wv53+DKtNzCMDYj9fRUSuhl44J1GZEPuq7iYfxcVBwdOL4y+FnqpeIHweL BGLcl8+WVxbN54ZaYXkpQGgPr3Hjt3yVreZzkDVlkMgkfgDlqcHFrk2ZCFE+3GjpFXnv UjQg== X-Gm-Message-State: AO0yUKV4NvWYU8a5Dj0FfB6i5ogUww7DfgWeO5+rQ08MELdAbsxV3fpL uJcqhbVxobr/5wCv8cJliy8vVg== X-Received: by 2002:a05:600c:a46:b0:3ec:4621:680b with SMTP id c6-20020a05600c0a4600b003ec4621680bmr17906976wmq.14.1678972698428; Thu, 16 Mar 2023 06:18:18 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (238.174.185.81.rev.sfr.net. [81.185.174.238]) by smtp.gmail.com with ESMTPSA id c16-20020adffb50000000b002d2f0e23acbsm24648wrs.12.2023.03.16.06.18.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Mar 2023 06:18:18 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Rob Herring , Frank Rowand , Mike Rapoport , Andrew Morton , Anup Patel , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-mm@kvack.org Cc: Alexandre Ghiti , Andrew Jones Subject: [PATCH v8 1/4] riscv: Get rid of riscv_pfn_base variable Date: Thu, 16 Mar 2023 14:17:08 +0100 Message-Id: <20230316131711.1284451-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230316131711.1284451-1-alexghiti@rivosinc.com> References: <20230316131711.1284451-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760531839126445199?= X-GMAIL-MSGID: =?utf-8?q?1760531839126445199?= Use directly phys_ram_base instead, riscv_pfn_base is just the pfn of the address contained in phys_ram_base. Even if there is no functional change intended in this patch, actually setting phys_ram_base that early changes the behaviour of kernel_mapping_pa_to_va during the early boot: phys_ram_base used to be zero before this patch and now it is set to the physical start address of the kernel. But it does not break the conversion of a kernel physical address into a virtual address since kernel_mapping_pa_to_va should only be used on kernel physical addresses, i.e. addresses greater than the physical start address of the kernel. Signed-off-by: Alexandre Ghiti Reviewed-by: Andrew Jones Reviewed-by: Anup Patel Tested-by: Anup Patel --- arch/riscv/include/asm/page.h | 3 +-- arch/riscv/mm/init.c | 6 +----- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 7fed7c431928..8dc686f549b6 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -91,8 +91,7 @@ typedef struct page *pgtable_t; #endif #ifdef CONFIG_MMU -extern unsigned long riscv_pfn_base; -#define ARCH_PFN_OFFSET (riscv_pfn_base) +#define ARCH_PFN_OFFSET (PFN_DOWN((unsigned long)phys_ram_base)) #else #define ARCH_PFN_OFFSET (PAGE_OFFSET >> PAGE_SHIFT) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 87f6a5d475a6..cc558d94559a 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -271,9 +271,6 @@ static void __init setup_bootmem(void) #ifdef CONFIG_MMU struct pt_alloc_ops pt_ops __initdata; -unsigned long riscv_pfn_base __ro_after_init; -EXPORT_SYMBOL(riscv_pfn_base); - pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss; pgd_t trampoline_pg_dir[PTRS_PER_PGD] __page_aligned_bss; static pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss; @@ -285,7 +282,6 @@ static pmd_t __maybe_unused early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAG #ifdef CONFIG_XIP_KERNEL #define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops)) -#define riscv_pfn_base (*(unsigned long *)XIP_FIXUP(&riscv_pfn_base)) #define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir)) #define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) #define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) @@ -985,7 +981,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) kernel_map.va_pa_offset = PAGE_OFFSET - kernel_map.phys_addr; kernel_map.va_kernel_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr; - riscv_pfn_base = PFN_DOWN(kernel_map.phys_addr); + phys_ram_base = kernel_map.phys_addr; /* * The default maximal physical memory size is KERN_VIRT_SIZE for 32-bit From patchwork Thu Mar 16 13:17:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 70789 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp485618wrt; Thu, 16 Mar 2023 06:35:14 -0700 (PDT) X-Google-Smtp-Source: AK7set8HZJ7lX4a6v0/J9GWudUxbKPDF1EvcL8Q74cJhjlC1IqoNc7EJC9o/TvPqe519blhhh0fV X-Received: by 2002:a17:902:d4d0:b0:19f:30be:ea06 with SMTP id o16-20020a170902d4d000b0019f30beea06mr4695699plg.50.1678973714336; Thu, 16 Mar 2023 06:35:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678973714; cv=none; d=google.com; s=arc-20160816; b=a1L4Kmb2VOF7QS0BdDjSbkFCCfpcmmXer0ewl4t6RMSZd7dhoxvx0PqMXALI/OJCoF NKzsy08YK0eh7LJzHD1cop6NkfTFsno8HUjvHNR7RcBnLxBwkQ4ZlokwiD9K4Pd+EVg1 AXpeZ+iRY+ooFjyivgLvG8aSIjt1dbe/EbAk7+sD/hwYfLPbtTI9EH0Yjkvi219cogpI j8Rkuhu2cgBbPloYQA+Bdtug7XlDjPYpnXVldYYfKIzUCMLRJMWWv9up7LcyE+lq0RJm SXugQFdS/Sm3dmPO7coFRmXDtbih7Xg48SrGbLADs6pb5cvVbN/BV7zpqRdg3OBk3oWm 3WYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yXNicKwHPUURyALO36PEegj/jGCRXjX7/YBezTaryXs=; b=ZNMd/2dpvQ6BaMicRx/3unpfdwfEhB7t2Lx8LobLzLD7Ug3v9N7dwnmKAccI8I07U1 RynrJ5kslvAAcypIcw05h2oaKC8GEWq4+mOAEFxUzxTJfwxXN4yLKneMvjZ0+GgKThNn 60ZtFWiYQ8przo3qKgRr/GIfMDv1VI4S8VLHKctoKR7lqlMg1TpMMnJa1Ye0kxJcz9hU CgOV9ecLmHvlLNz1DYSxd/kFUkU+ZSUco3qoMmCSqOYtlzq4KMDSPFlzWTHVxOacra1N GBLSu1+7OWxvNHhd1yofZyVh/88PD8qqQDpQFlncOBHBotgUNis3V5OMzNanSCK/G9at j7ag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=67doldxY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n4-20020a170903110400b00189f58ecb7asi8712505plh.151.2023.03.16.06.34.59; Thu, 16 Mar 2023 06:35:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=67doldxY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229716AbjCPNUF (ORCPT + 99 others); Thu, 16 Mar 2023 09:20:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230075AbjCPNUC (ORCPT ); Thu, 16 Mar 2023 09:20:02 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA731C0803 for ; Thu, 16 Mar 2023 06:19:22 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id l12so1504476wrm.10 for ; Thu, 16 Mar 2023 06:19:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1678972760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yXNicKwHPUURyALO36PEegj/jGCRXjX7/YBezTaryXs=; b=67doldxYgYKlU/XZ0kZu3+NOAqQ/2WbnP4mwdaTGI0McuaMhRPDAE/s74Wq3t6tF/V J0nRxluNtH5jyXjG6gwranylrEnecoP/DUWgaO9JZ4gg6xFffDGE08IwIc2KOUXaspal HHuTUIg4II4i6Oa5sRMFXG0qwnNOZQwEH5YClaHtl1ekxMRL+d17IM5dSSKvN8etH4o4 Gryhf8z+9DXYE61ol4wGcZdxrAPW2qatjZ5Q8iOQkHFCUmxFt43/+ApFfD1rBG+HZP3t j+M2PnF2oTfQycLOuf/WXwZevaBF66GgvGXiPTY9lyNMUBSV3fX9CsUSbPa1JK7AlIvp vKjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678972760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yXNicKwHPUURyALO36PEegj/jGCRXjX7/YBezTaryXs=; b=yrx8mJNUc9+fkBJ8kREkaWFHaUzshiJKiOnk4nOqUpUSvMcvSNdzax6fHqs8PAxS9Y 64BB5FJk9Mc4A1XxWIDFR+xDO9sLjCLoqL2Ry0QI0iUS3RDywAuKD/AavVglQept6pv9 v6XT1ZfkBwGPT2L6g4cZA0WvMJERMsGaDJcE9rA4VZIVYbTGbedq6dxBiioDW4yBfaDi 4LQiaeQTfzttu12hT6rajxSklGwp8++Jb4hRCGaTXRIjIm3cc86FAQaMcEktmC9rFl7U 4fHze9yyMAF9RW3drJZuEsCLMaqsw5bIFZmghPE16Jh/M/K/fMt6lWP+FhvTp0HR66Fw dNZA== X-Gm-Message-State: AO0yUKXrRA7CF4GbKM7QYcOpGRs28SiGuLEx7ZM+9ueg4ULYUt20WblN 72+Fds1KZP6FRO/2viYHLM8U7w== X-Received: by 2002:a5d:49cc:0:b0:2c7:57e:633e with SMTP id t12-20020a5d49cc000000b002c7057e633emr4666191wrs.49.1678972760162; Thu, 16 Mar 2023 06:19:20 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (238.174.185.81.rev.sfr.net. [81.185.174.238]) by smtp.gmail.com with ESMTPSA id h5-20020a5d5485000000b002c5a1bd5280sm7210070wrv.95.2023.03.16.06.19.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Mar 2023 06:19:20 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Rob Herring , Frank Rowand , Mike Rapoport , Andrew Morton , Anup Patel , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v8 2/4] mm: Introduce memblock_isolate_memory Date: Thu, 16 Mar 2023 14:17:09 +0100 Message-Id: <20230316131711.1284451-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230316131711.1284451-1-alexghiti@rivosinc.com> References: <20230316131711.1284451-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760531541180151195?= X-GMAIL-MSGID: =?utf-8?q?1760531541180151195?= This function allows to split a region in memblock.memory and will be useful when setting up the linear mapping with STRICT_KERNEL_RWX: it allows to isolate the kernel text/rodata and then avoid to map those regions with a PUD/P4D/PGD. Signed-off-by: Alexandre Ghiti Reviewed-by: Anup Patel Tested-by: Anup Patel Acked-by: Mike Rapoport (IBM) --- include/linux/memblock.h | 1 + mm/memblock.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/include/linux/memblock.h b/include/linux/memblock.h index 50ad19662a32..2f7ef97c0da7 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -125,6 +125,7 @@ int memblock_clear_hotplug(phys_addr_t base, phys_addr_t size); int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); +int memblock_isolate_memory(phys_addr_t base, phys_addr_t size); void memblock_free_all(void); void memblock_free(void *ptr, size_t size); diff --git a/mm/memblock.c b/mm/memblock.c index 25fd0626a9e7..e8c651a37012 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -805,6 +805,26 @@ static int __init_memblock memblock_isolate_range(struct memblock_type *type, return 0; } +/** + * memblock_isolate_memory - isolate given range in memblock.memory + * @base: base of range to isolate + * @size: size of range to isolate + * + * Isolates the given range in memblock.memory so that it does not share any + * region with other ranges. + * + * Return: + * 0 on success, -errno on failure. + */ + +int __init_memblock memblock_isolate_memory(phys_addr_t base, phys_addr_t size) +{ + int start_rgn, end_rgn; + + return memblock_isolate_range(&memblock.memory, base, size, + &start_rgn, &end_rgn); +} + static int __init_memblock memblock_remove_range(struct memblock_type *type, phys_addr_t base, phys_addr_t size) { From patchwork Thu Mar 16 13:17:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 70788 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp481254wrt; Thu, 16 Mar 2023 06:26:37 -0700 (PDT) X-Google-Smtp-Source: AK7set/s28uo4PdS5LE89xCrU2tAtcOzVtQ/AAWnCMszEMPOGForKbJD1zHeH6t4XLyyz/n/A9q5 X-Received: by 2002:aa7:9e5b:0:b0:626:6a3:6b81 with SMTP id z27-20020aa79e5b000000b0062606a36b81mr898911pfq.15.1678973197511; Thu, 16 Mar 2023 06:26:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678973197; cv=none; d=google.com; s=arc-20160816; b=bX3OT2+Mn7d08nzeWlmz7LAL8E8+KXCYdrASZri+BjJXYTMAC6Y5zLeBrfqNKd36aB CcIrjcBrA29CAj3NNryaBTDLyke9hw5OAGkb8u55heQ2u74xhB0/uaJ6B8CieVrr1Zo8 Q/WWjqLZ2EJ0//cuFXeT4vVbJqutAD+CH8n1851kK4X5MPKIYt0ATjQ6KtzscIwoXQxY wMHewW8gUD0+8B8xClMkog8MFpIotDctlWKEx6ga0PbFH+ssPQNnbOLUt54YNk5oSymB cC2Xzb5naqqv9i9aQR+zTle0JDgf3SYHECSG+ockC8WmPQdwhtcn35IiRsNAhef9qOzJ +/hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=cNtcjIvt7Jcu6BbJu0k6G/sx5AHflabOU3bxX5prrUE=; b=trrFgxIqlDpKnEqDeIRwKuUnyyJyAR0l6QHNQ5AknKtCiIdvyWsH68eoqCdDRXERHI qK996aQS0Laeybp/+pSk5RhUKERhl9O4mwwCaWD22iHdQ1vg2fOfcQJTdL00Ys3XIHxs 5ulTfCtX/6EgeKOjcK3Ac7dBRecRDXWumIspgRiiCyeqEjIQxgGcIsg69ZmvuyFqX+OB JwZjuESW5zajBa7kBQaecBSPnX8QOySa0Bzp++EDoYnWpcwFDyA1wM6XXEZ6yV5mf2G1 4Qn/mZw8k9yq5cNzkHlC1tPXdmNYmrVmkTWcam/4BruGgDhbGNruldTU5BgNGI50njxP G70A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=L6eAT+sm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w14-20020a62820e000000b00625d6ac65b7si3335881pfd.372.2023.03.16.06.26.24; Thu, 16 Mar 2023 06:26:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=L6eAT+sm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230130AbjCPNUr (ORCPT + 99 others); Thu, 16 Mar 2023 09:20:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230063AbjCPNUo (ORCPT ); Thu, 16 Mar 2023 09:20:44 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E643DCA1D4 for ; Thu, 16 Mar 2023 06:20:23 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id l12so1507308wrm.10 for ; Thu, 16 Mar 2023 06:20:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1678972822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cNtcjIvt7Jcu6BbJu0k6G/sx5AHflabOU3bxX5prrUE=; b=L6eAT+smuQGoJO4DezL+IdwC0T7yTp/6MdgBgpSYGnzsE3GMtH0OsHZCA5KvkK/jsL SCQbzTNJRG+h5gG55AwIc3rp3qvPJ5HvHBpBdMMvum+UhNDX3K0d4mevTt2us+FFR9fP /SkYJ3PFgwjlEeZJ4QPID/NaQDyD2lSqhEWFJsMLe0iYwjDfErv7L30D/hLJYYVSFfx7 jrT7h87/rKRPpMs9Aij8oJj9DBxmwR0E5LN3DULWfc0aFVYbJwTggaKYwjOlxm9f5rbi T0csEj0sYOg1wDUvvVEpshdAj9QA/fAZ/IJurP8+1WuNoMR3+2zSN2/cqzpGfR2FuvNh EyZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678972822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cNtcjIvt7Jcu6BbJu0k6G/sx5AHflabOU3bxX5prrUE=; b=tmKeKYigbgURGNsubAQfWnuDTKd4UzjL33wdM7m7E44syBN9TAmdqgziwmd8ZSrITD MJX3Oi02KKTEhQ0eXxhwcrgEkoWhvzxu0Z2GYV4SVX9lWgSaqKqAP7L1zRqqiiF+AY+u piX75WAth8pENzWD34HdvzzgQxNm57DdJMnOW6OgO8ToFRdKmiJHSAoFE2xud2gb7wFu p7nwuHyqE0tJ/WQ76ipPPNwNV8Ejx270M2O5aeOCM0mDijtJWfZO0Z1+TKcafaQZ6sHy ceOUPbegqEjvYNL6qysqF8ctWredN2Lpq1YAPdN9mFTR3TniNYMkOtrQeXXS/Orvze3w Lf/w== X-Gm-Message-State: AO0yUKW5DduOGII0baV9ZKAiZjQAUj/lZm6/DHSdUOwOP3vQpeTGPwuf +5gYIXKzFWbRkTLShQb99c0hwg== X-Received: by 2002:adf:ce11:0:b0:2cf:e3d0:2a43 with SMTP id p17-20020adfce11000000b002cfe3d02a43mr4536359wrn.4.1678972821966; Thu, 16 Mar 2023 06:20:21 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (238.174.185.81.rev.sfr.net. [81.185.174.238]) by smtp.gmail.com with ESMTPSA id z4-20020a5d6544000000b002c56013c07fsm7218162wrv.109.2023.03.16.06.20.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Mar 2023 06:20:21 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Rob Herring , Frank Rowand , Mike Rapoport , Andrew Morton , Anup Patel , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH v8 3/4] arm64: Make use of memblock_isolate_memory for the linear mapping Date: Thu, 16 Mar 2023 14:17:10 +0100 Message-Id: <20230316131711.1284451-4-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230316131711.1284451-1-alexghiti@rivosinc.com> References: <20230316131711.1284451-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760530999837224107?= X-GMAIL-MSGID: =?utf-8?q?1760530999837224107?= In order to isolate the kernel text mapping and the crash kernel region, we used some sort of hack to isolate thoses ranges which consisted in marking them as not mappable with memblock_mark_nomap. Simply use the newly introduced memblock_isolate_memory function which does exactly the same but does not uselessly mark the region as not mappable. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/mmu.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 6f9d8898a025..387c2a065a09 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -548,19 +548,18 @@ static void __init map_mem(pgd_t *pgdp) /* * Take care not to create a writable alias for the - * read-only text and rodata sections of the kernel image. - * So temporarily mark them as NOMAP to skip mappings in - * the following for-loop + * read-only text and rodata sections of the kernel image so isolate + * those regions and map them after the for loop. */ - memblock_mark_nomap(kernel_start, kernel_end - kernel_start); + memblock_isolate_memory(kernel_start, kernel_end - kernel_start); #ifdef CONFIG_KEXEC_CORE if (crash_mem_map) { if (defer_reserve_crashkernel()) flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; else if (crashk_res.end) - memblock_mark_nomap(crashk_res.start, - resource_size(&crashk_res)); + memblock_isolate_memory(crashk_res.start, + resource_size(&crashk_res)); } #endif @@ -568,6 +567,17 @@ static void __init map_mem(pgd_t *pgdp) for_each_mem_range(i, &start, &end) { if (start >= end) break; + + if (start == kernel_start) + continue; + +#ifdef CONFIG_KEXEC_CORE + if (start == crashk_res.start && + crash_mem_map && !defer_reserve_crashkernel() && + crashk_res.end) + continue; +#endif + /* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as @@ -589,7 +599,6 @@ static void __init map_mem(pgd_t *pgdp) */ __map_memblock(pgdp, kernel_start, kernel_end, PAGE_KERNEL, NO_CONT_MAPPINGS); - memblock_clear_nomap(kernel_start, kernel_end - kernel_start); /* * Use page-level mappings here so that we can shrink the region @@ -603,8 +612,6 @@ static void __init map_mem(pgd_t *pgdp) crashk_res.end + 1, PAGE_KERNEL, NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS); - memblock_clear_nomap(crashk_res.start, - resource_size(&crashk_res)); } } #endif From patchwork Thu Mar 16 13:17:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 70790 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:604a:0:0:0:0:0 with SMTP id j10csp486702wrt; Thu, 16 Mar 2023 06:37:18 -0700 (PDT) X-Google-Smtp-Source: AK7set9/XlcSoiK80AJaSRQI61SiMgJdk2WdK2n+oJKFjfpbqjSOdaowJ0ZXW/DU+IaNcCF09vVO X-Received: by 2002:a17:903:2452:b0:1a1:818b:8ce4 with SMTP id l18-20020a170903245200b001a1818b8ce4mr3720972pls.62.1678973838694; Thu, 16 Mar 2023 06:37:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678973838; cv=none; d=google.com; s=arc-20160816; b=dRfqouYxjROk9c4/3CCSea+uws5pM2DR7sTH6RUzmPGJUiEOuBSWmfUNwwbXm1bCXh jFlZLJHnZqPLDUhanpVgtQIfBBuVyS3XKI3Pu499VXf00AFsolWtQkmVpXWFA0DX5PKy uTnTBu7Timy2u4WyQ40UjQ5Fwy6U4s9QRG1wykxiVXnfDAwEGM5Ta8O/p+HPvFz/Rz17 1mETdPbSe6bayvwbBn3JuSzdVS1FvyN1tLcygDPhC3oRUYe9UDgxtOooMTdVFWVvIXZ5 66ormGGSoibEvAc5jxEjKB9DbkEkEc2NDZVxh1XVBty524Snm2kFigi6M/E08UBs5jpF HbAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xI+mOo9w6N5hnNZASu9KEqOIyOdQmEOvWrAnFqrzLH0=; b=BsZJNyLQpwxoQ8fJxSLEYOF+zpBn0WdQ6qR41jQgj/vlw0h4Lr3wXoK3tOEHWBv2/X ZS7LulM0vOHZao+k5M6+A4X6ihGwjc+cPuNDGkCxCV/oFBAxkKdehm/TKHElQqQTWs/C h+6B0jNWYnFEPNjjg+XhFYjw302GaRSv4HgnUc5aZCZo+jep7b+dY3Ca1xMzi+6xS1jf H8pX8itau9l9Epnp8v4dwXF0F6lCtAjRcDmHEM3nwtfjt5ovPry9VIHPqUaVtQl2kMNB p2JCsqpfZb7HZU656Ll7zcKfHfT+joq399wpe9jq4sJOoybysNfy89T1SG3NhyHdZa7k gJIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=6NbDDiWh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n11-20020a170902968b00b0019e30e3066bsi7924276plp.286.2023.03.16.06.37.01; Thu, 16 Mar 2023 06:37:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@rivosinc-com.20210112.gappssmtp.com header.s=20210112 header.b=6NbDDiWh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230096AbjCPNV3 (ORCPT + 99 others); Thu, 16 Mar 2023 09:21:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230075AbjCPNV1 (ORCPT ); Thu, 16 Mar 2023 09:21:27 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5785655BD for ; Thu, 16 Mar 2023 06:21:25 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id bg16-20020a05600c3c9000b003eb34e21bdfso3178661wmb.0 for ; Thu, 16 Mar 2023 06:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1678972884; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xI+mOo9w6N5hnNZASu9KEqOIyOdQmEOvWrAnFqrzLH0=; b=6NbDDiWhj5FJ6JbELI1WQQGmivylRUUFCQr2jFMHhqG/H6CtJsjXNtyc/MNtMKKoVv xNB1dFC+2oad1flg1qOi4UVFqUCTvnDr4neGcEsd6Gx7qxLwY6WW38zuiMQgr72VOh/c t0pD2CZidNEjriYDCG4ILalKpNijLp1i4EakDKUuOA/uMJNNMARP4vdpQL0ie9eT9TaK FB7+STFcpor4Uyq8zP9spP6k2sofE7b5RQ1mp8Dd1/xCDutwkiESvyN2IUr7tw6tBZ4I 6HDWXhqFFeBcFDQ/6kCk6FSMl60WIR0lOtwbZSSMYwd+ZsVTzbBkd1UysrboB+TGL/IH j8cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678972884; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xI+mOo9w6N5hnNZASu9KEqOIyOdQmEOvWrAnFqrzLH0=; b=XwB8QSvLKEUgXITn64QG7rR5vVZEPKh9AWFhp3wga5S2hCRl6yNELk0IWyanmVwGsR nOIAZwp8zLPzr39quXDk3EYEDxj0C8p5DpyqVUh21U8RRLtD/1ajBXlhZ2e5DsA94U/o 9X0HWXi5oH+4vC/IYOZcVbfVMBctJqlwYxfY6YE3+FOhuaGS67+JqwPQ7MjPnIURfQIW PzK1RO1TXXMGCMlliw4h/WZG/6SsVHBDUdj13voDtrBxA+SwJ3NFW2rnudelTa1uABbV ktxq2/+ARRl8C+lXafQsMg3k+f7y460gy6f8N2fkVtMnZq56TWT63+eD7av3gxquuNQP lGUw== X-Gm-Message-State: AO0yUKXRJsSvtb9+zFYZXaUuzXfwkxk690MAwAEvzoiO8tF/2S1vzc5m cm0/CKbMxjd3Ybp/hpyNRldx6A== X-Received: by 2002:a05:600c:4453:b0:3eb:389d:156c with SMTP id v19-20020a05600c445300b003eb389d156cmr22345321wmn.37.1678972883803; Thu, 16 Mar 2023 06:21:23 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (238.174.185.81.rev.sfr.net. [81.185.174.238]) by smtp.gmail.com with ESMTPSA id bh10-20020a05600c3d0a00b003ed23e9e03bsm4993248wmb.46.2023.03.16.06.21.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Mar 2023 06:21:23 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , Rob Herring , Frank Rowand , Mike Rapoport , Andrew Morton , Anup Patel , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-mm@kvack.org Cc: Alexandre Ghiti , Rob Herring , Andrew Jones Subject: [PATCH v8 4/4] riscv: Use PUD/P4D/PGD pages for the linear mapping Date: Thu, 16 Mar 2023 14:17:11 +0100 Message-Id: <20230316131711.1284451-5-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230316131711.1284451-1-alexghiti@rivosinc.com> References: <20230316131711.1284451-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760531176129429604?= X-GMAIL-MSGID: =?utf-8?q?1760531671932873845?= During the early page table creation, we used to set the mapping for PAGE_OFFSET to the kernel load address: but the kernel load address is always offseted by PMD_SIZE which makes it impossible to use PUD/P4D/PGD pages as this physical address is not aligned on PUD/P4D/PGD size (whereas PAGE_OFFSET is). But actually we don't have to establish this mapping (ie set va_pa_offset) that early in the boot process because: - first, setup_vm installs a temporary kernel mapping and among other things, discovers the system memory, - then, setup_vm_final creates the final kernel mapping and takes advantage of the discovered system memory to create the linear mapping. During the first phase, we don't know the start of the system memory and then until the second phase is finished, we can't use the linear mapping at all and phys_to_virt/virt_to_phys translations must not be used because it would result in a different translation from the 'real' one once the final mapping is installed. So here we simply delay the initialization of va_pa_offset to after the system memory discovery. But to make sure noone uses the linear mapping before, we add some guard in the DEBUG_VIRTUAL config. Finally we can use PUD/P4D/PGD hugepages when possible, which will result in a better TLB utilization. Note that: - this does not apply to rv32 as the kernel mapping lies in the linear mapping. - we rely on the firmware to protect itself using PMP. Signed-off-by: Alexandre Ghiti Acked-by: Rob Herring # DT bits Reviewed-by: Andrew Jones Reviewed-by: Anup Patel Tested-by: Anup Patel --- arch/riscv/include/asm/page.h | 16 ++++++++++++ arch/riscv/mm/init.c | 49 ++++++++++++++++++++++++++++++----- arch/riscv/mm/physaddr.c | 16 ++++++++++++ drivers/of/fdt.c | 11 ++++---- 4 files changed, 81 insertions(+), 11 deletions(-) diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 8dc686f549b6..ea1a0e237211 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -90,6 +90,14 @@ typedef struct page *pgtable_t; #define PTE_FMT "%08lx" #endif +#ifdef CONFIG_64BIT +/* + * We override this value as its generic definition uses __pa too early in + * the boot process (before kernel_map.va_pa_offset is set). + */ +#define MIN_MEMBLOCK_ADDR 0 +#endif + #ifdef CONFIG_MMU #define ARCH_PFN_OFFSET (PFN_DOWN((unsigned long)phys_ram_base)) #else @@ -121,7 +129,11 @@ extern phys_addr_t phys_ram_base; #define is_linear_mapping(x) \ ((x) >= PAGE_OFFSET && (!IS_ENABLED(CONFIG_64BIT) || (x) < PAGE_OFFSET + KERN_VIRT_SIZE)) +#ifndef CONFIG_DEBUG_VIRTUAL #define linear_mapping_pa_to_va(x) ((void *)((unsigned long)(x) + kernel_map.va_pa_offset)) +#else +void *linear_mapping_pa_to_va(unsigned long x); +#endif #define kernel_mapping_pa_to_va(y) ({ \ unsigned long _y = (unsigned long)(y); \ (IS_ENABLED(CONFIG_XIP_KERNEL) && _y < phys_ram_base) ? \ @@ -130,7 +142,11 @@ extern phys_addr_t phys_ram_base; }) #define __pa_to_va_nodebug(x) linear_mapping_pa_to_va(x) +#ifndef CONFIG_DEBUG_VIRTUAL #define linear_mapping_va_to_pa(x) ((unsigned long)(x) - kernel_map.va_pa_offset) +#else +phys_addr_t linear_mapping_va_to_pa(unsigned long x); +#endif #define kernel_mapping_va_to_pa(y) ({ \ unsigned long _y = (unsigned long)(y); \ (IS_ENABLED(CONFIG_XIP_KERNEL) && _y < kernel_map.virt_addr + XIP_OFFSET) ? \ diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index cc558d94559a..7af7cd201a9c 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -213,6 +213,14 @@ static void __init setup_bootmem(void) phys_ram_end = memblock_end_of_DRAM(); if (!IS_ENABLED(CONFIG_XIP_KERNEL)) phys_ram_base = memblock_start_of_DRAM(); + + /* + * In 64-bit, any use of __va/__pa before this point is wrong as we + * did not know the start of DRAM before. + */ + if (IS_ENABLED(CONFIG_64BIT)) + kernel_map.va_pa_offset = PAGE_OFFSET - phys_ram_base; + /* * memblock allocator is not aware of the fact that last 4K bytes of * the addressable memory can not be mapped because of IS_ERR_VALUE @@ -667,9 +675,16 @@ void __init create_pgd_mapping(pgd_t *pgdp, static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) { - /* Upgrade to PMD_SIZE mappings whenever possible */ - base &= PMD_SIZE - 1; - if (!base && size >= PMD_SIZE) + if (!(base & (PGDIR_SIZE - 1)) && size >= PGDIR_SIZE) + return PGDIR_SIZE; + + if (!(base & (P4D_SIZE - 1)) && size >= P4D_SIZE) + return P4D_SIZE; + + if (!(base & (PUD_SIZE - 1)) && size >= PUD_SIZE) + return PUD_SIZE; + + if (!(base & (PMD_SIZE - 1)) && size >= PMD_SIZE) return PMD_SIZE; return PAGE_SIZE; @@ -978,11 +993,22 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) set_satp_mode(); #endif - kernel_map.va_pa_offset = PAGE_OFFSET - kernel_map.phys_addr; + /* + * In 64-bit, we defer the setup of va_pa_offset to setup_bootmem, + * where we have the system memory layout: this allows us to align + * the physical and virtual mappings and then make use of PUD/P4D/PGD + * for the linear mapping. This is only possible because the kernel + * mapping lies outside the linear mapping. + * In 32-bit however, as the kernel resides in the linear mapping, + * setup_vm_final can not change the mapping established here, + * otherwise the same kernel addresses would get mapped to different + * physical addresses (if the start of dram is different from the + * kernel physical address start). + */ + kernel_map.va_pa_offset = IS_ENABLED(CONFIG_64BIT) ? + 0UL : PAGE_OFFSET - kernel_map.phys_addr; kernel_map.va_kernel_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr; - phys_ram_base = kernel_map.phys_addr; - /* * The default maximal physical memory size is KERN_VIRT_SIZE for 32-bit * kernel, whereas for 64-bit kernel, the end of the virtual address @@ -1097,6 +1123,17 @@ static void __init setup_vm_final(void) __pa_symbol(fixmap_pgd_next), PGDIR_SIZE, PAGE_TABLE); +#ifdef CONFIG_STRICT_KERNEL_RWX + /* + * Isolate the kernel text and rodata linear so they don't + * get mapped with a PUD in the linear mapping. + */ + memblock_isolate_memory(__pa_symbol(_start), + __init_data_begin - _start); + memblock_isolate_memory(__pa_symbol(__start_rodata), + __start_rodata - _data); +#endif + /* Map all memory banks in the linear mapping */ for_each_mem_range(i, &start, &end) { if (start >= end) diff --git a/arch/riscv/mm/physaddr.c b/arch/riscv/mm/physaddr.c index 9b18bda74154..18706f457da7 100644 --- a/arch/riscv/mm/physaddr.c +++ b/arch/riscv/mm/physaddr.c @@ -33,3 +33,19 @@ phys_addr_t __phys_addr_symbol(unsigned long x) return __va_to_pa_nodebug(x); } EXPORT_SYMBOL(__phys_addr_symbol); + +phys_addr_t linear_mapping_va_to_pa(unsigned long x) +{ + BUG_ON(!kernel_map.va_pa_offset); + + return ((unsigned long)(x) - kernel_map.va_pa_offset); +} +EXPORT_SYMBOL(linear_mapping_va_to_pa); + +void *linear_mapping_pa_to_va(unsigned long x) +{ + BUG_ON(!kernel_map.va_pa_offset); + + return ((void *)((unsigned long)(x) + kernel_map.va_pa_offset)); +} +EXPORT_SYMBOL(linear_mapping_pa_to_va); diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c index d1a68b6d03b3..d14735a81301 100644 --- a/drivers/of/fdt.c +++ b/drivers/of/fdt.c @@ -887,12 +887,13 @@ const void * __init of_flat_dt_match_machine(const void *default_match, static void __early_init_dt_declare_initrd(unsigned long start, unsigned long end) { - /* ARM64 would cause a BUG to occur here when CONFIG_DEBUG_VM is - * enabled since __va() is called too early. ARM64 does make use - * of phys_initrd_start/phys_initrd_size so we can skip this - * conversion. + /* + * __va() is not yet available this early on some platforms. In that + * case, the platform uses phys_initrd_start/phys_initrd_size instead + * and does the VA conversion itself. */ - if (!IS_ENABLED(CONFIG_ARM64)) { + if (!IS_ENABLED(CONFIG_ARM64) && + !(IS_ENABLED(CONFIG_RISCV) && IS_ENABLED(CONFIG_64BIT))) { initrd_start = (unsigned long)__va(start); initrd_end = (unsigned long)__va(end); initrd_below_start_ok = 1;