From patchwork Thu Jun 1 10:12:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 101869 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp201865vqr; Thu, 1 Jun 2023 03:33:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ooQ1JhSWFPtgrS1J+pQaK6P344fkVWtYOvdojqOQkZMRmwmI8iItcl9fJVO8onRLFLoW9 X-Received: by 2002:a05:6870:98b4:b0:19a:ce25:6d06 with SMTP id eg52-20020a05687098b400b0019ace256d06mr6389456oab.56.1685615591456; Thu, 01 Jun 2023 03:33:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685615591; cv=none; d=google.com; s=arc-20160816; b=M/FX9WPk6uEQ9eIcpI1oR1LD1Bhf+9MyAE1O2h64Fzr5QPWWHvEr8Je2168Kvx3eTu 5a6fo7AGQC2FQTtBIZSQ7J7lPIpmHi/e4F4ZNSBTgz/rM4kIN5P2472IrQCh33koLkXl oNnEA8UX5ajj6N5NnQVVR25vfGF5LivSLRh1cX2a8oyG1rmlThhFXAhngP3K/L1ueAJZ T+Y1q5hxDgUdaE6PYvHHyS2XOrCefgrR3lFoRaVFp33b7+5MO8aDLmUxsi5bcdsk6o2R vINnUo13DZ5vi1+HKXk9dGWo6mP4EAisZHUorGHDjh5R9wMLqHLhRPVH4sR84isStz3z 5aog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=d2h3+FaSiO2MUuUPt6RKROhs8tOU+V9XaKlBroOkRKI=; b=EJG8cwRCEO8nrLsR10IfKKOCMLzXmfiYw0s6zpe2hGpHUx/zakXT4GyMzKqVtbF8u6 N8nu+AqlKvDLy8vqlHfqcJc474z8oUuo5a4aj6RkrtjQPW8pr+Uf8Zp55GsmEx29K1Yl +4BypD4z2H1rv9wmSlTn7/SWXWuC+igQUAbKoSdJyBYQLKxL2AbAAeY8ns7cMdbKe4z9 LTLxSm1dQCzbWWbBErcOxPeHdboqEV/xWvgPQdB1XHtmuzplfAd5PMy+Gi8GJwO9d7gP 4ktCjGxHW5w1G4lWb8ouBCKCg6EY0SmXNqgF5+4H7DNOGWLHbvhukMr8mN2RzENWfglt kLhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fRH3Dr0w; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e68-20020a636947000000b0053eef685a0csi2544857pgc.852.2023.06.01.03.32.57; Thu, 01 Jun 2023 03:33:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fRH3Dr0w; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233443AbjFAKSV (ORCPT + 99 others); Thu, 1 Jun 2023 06:18:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233530AbjFAKRv (ORCPT ); Thu, 1 Jun 2023 06:17:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8AF25E4C; Thu, 1 Jun 2023 03:15:23 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 86D6B6431B; Thu, 1 Jun 2023 10:14:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 75981C433AA; Thu, 1 Jun 2023 10:14:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1685614468; bh=K7wvO7ElwhoAy7Oh2IpfLgKs2YHT7c7jqjInBLBuDTg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fRH3Dr0wJXXqyiX4DpPIKcmgXGzVC4gxNGWC6LITX/cuu1fgtqI+DGd/9maCsR1YG n02Pkrgsj/Vf+omfNPMLkDIo5AN0DpSJtbXnHd8HZYRidz4tixxp+zVAey7pSOXiL2 cko5pQEh79FuGCFjyg1UUYvNTHWmqFfd0GoIcCkM0Zc2qiZnUklG6CFkw/wVmFZbjc SNrDkttPHy4gt7c/8IG464jkTRJMmL6XvHlOQ637slnEy6SnCyf+T1hQ+VMOoEny+t HOm3wN3FqH/AR42jxZ+6LMUF0Xt0ozqmMyjhXT2Z8ufKOXgnAqLmo+fVYAvP698VE6 fxfx5BUrh+Gtw== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Michael Ellerman , Mike Rapoport , "Naveen N. Rao" , Palmer Dabbelt , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH 08/13] arch: make jitalloc setup available regardless of CONFIG_MODULES Date: Thu, 1 Jun 2023 13:12:52 +0300 Message-Id: <20230601101257.530867-9-rppt@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20230601101257.530867-1-rppt@kernel.org> References: <20230601101257.530867-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767496054008274649?= X-GMAIL-MSGID: =?utf-8?q?1767496054008274649?= From: "Mike Rapoport (IBM)" jitalloc does not depend on modules, on the contrary modules use jitalloc. To make jitalloc available when CONFIG_MODULES=n, for instance for kprobes, split jit_alloc_params initialization out from arch/kernel/module.c and compile it when CONFIG_JIT_ALLOC=y Signed-off-by: Mike Rapoport (IBM) --- arch/arm/kernel/module.c | 32 -------------------------- arch/arm/mm/init.c | 35 ++++++++++++++++++++++++++++ arch/arm64/kernel/module.c | 40 -------------------------------- arch/arm64/mm/init.c | 42 ++++++++++++++++++++++++++++++++++ arch/loongarch/kernel/module.c | 14 ------------ arch/loongarch/mm/init.c | 16 +++++++++++++ arch/mips/kernel/module.c | 17 -------------- arch/mips/mm/init.c | 19 +++++++++++++++ arch/parisc/kernel/module.c | 17 -------------- arch/parisc/mm/init.c | 21 ++++++++++++++++- arch/powerpc/kernel/module.c | 39 ------------------------------- arch/powerpc/mm/mem.c | 41 +++++++++++++++++++++++++++++++++ arch/riscv/kernel/module.c | 16 ------------- arch/riscv/mm/init.c | 18 +++++++++++++++ arch/s390/kernel/module.c | 32 -------------------------- arch/s390/mm/init.c | 35 ++++++++++++++++++++++++++++ arch/sparc/kernel/module.c | 19 --------------- arch/sparc/mm/Makefile | 2 ++ arch/sparc/mm/jitalloc.c | 21 +++++++++++++++++ 19 files changed, 249 insertions(+), 227 deletions(-) create mode 100644 arch/sparc/mm/jitalloc.c diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c index 83ccbf98164f..054e799e7091 100644 --- a/arch/arm/kernel/module.c +++ b/arch/arm/kernel/module.c @@ -16,44 +16,12 @@ #include #include #include -#include #include #include #include #include -#ifdef CONFIG_XIP_KERNEL -/* - * The XIP kernel text is mapped in the module area for modules and - * some other stuff to work without any indirect relocations. - * MODULES_VADDR is redefined here and not in asm/memory.h to avoid - * recompiling the whole kernel when CONFIG_XIP_KERNEL is turned on/off. - */ -#undef MODULES_VADDR -#define MODULES_VADDR (((unsigned long)_exiprom + ~PMD_MASK) & PMD_MASK) -#endif - -#ifdef CONFIG_MMU -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, - .text.start = MODULES_VADDR, - .text.end = MODULES_END, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.pgprot = PAGE_KERNEL_EXEC; - - if (IS_ENABLED(CONFIG_ARM_MODULE_PLTS)) { - jit_alloc_params.text.fallback_start = VMALLOC_START; - jit_alloc_params.text.fallback_end = VMALLOC_END; - } - - return &jit_alloc_params; -} -#endif - bool module_init_section(const char *name) { return strstarts(name, ".init") || diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c index ce64bdb55a16..e492625b7f3d 100644 --- a/arch/arm/mm/init.c +++ b/arch/arm/mm/init.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include @@ -486,3 +487,37 @@ void free_initrd_mem(unsigned long start, unsigned long end) free_reserved_area((void *)start, (void *)end, -1, "initrd"); } #endif + +#ifdef CONFIG_JIT_ALLOC +#ifdef CONFIG_XIP_KERNEL +/* + * The XIP kernel text is mapped in the module area for modules and + * some other stuff to work without any indirect relocations. + * MODULES_VADDR is redefined here and not in asm/memory.h to avoid + * recompiling the whole kernel when CONFIG_XIP_KERNEL is turned on/off. + */ +#undef MODULES_VADDR +#define MODULES_VADDR (((unsigned long)_exiprom + ~PMD_MASK) & PMD_MASK) +#endif + +#ifdef CONFIG_MMU +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, + .text.start = MODULES_VADDR, + .text.end = MODULES_END, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.pgprot = PAGE_KERNEL_EXEC; + + if (IS_ENABLED(CONFIG_ARM_MODULE_PLTS)) { + jit_alloc_params.text.fallback_start = VMALLOC_START; + jit_alloc_params.text.fallback_end = VMALLOC_END; + } + + return &jit_alloc_params; +} +#endif + +#endif /* CONFIG_JIT_ALLOC */ diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 91ffcff5a44c..6d09b29fe9db 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -17,51 +17,11 @@ #include #include #include -#include #include #include #include #include -static struct jit_alloc_params jit_alloc_params = { - .alignment = JIT_ALLOC_ALIGN, - .flags = JIT_ALLOC_KASAN_SHADOW, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - u64 module_alloc_end = module_alloc_base + MODULES_VSIZE; - - if (IS_ENABLED(CONFIG_KASAN_GENERIC) || - IS_ENABLED(CONFIG_KASAN_SW_TAGS)) - /* don't exceed the static module region - see below */ - module_alloc_end = MODULES_END; - - jit_alloc_params.text.pgprot = PAGE_KERNEL; - jit_alloc_params.text.start = module_alloc_base; - jit_alloc_params.text.end = module_alloc_end; - - /* - * KASAN without KASAN_VMALLOC can only deal with module - * allocations being served from the reserved module region, - * since the remainder of the vmalloc region is already - * backed by zero shadow pages, and punching holes into it - * is non-trivial. Since the module region is not randomized - * when KASAN is enabled without KASAN_VMALLOC, it is even - * less likely that the module region gets exhausted, so we - * can simply omit this fallback in that case. - */ - if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) && - (IS_ENABLED(CONFIG_KASAN_VMALLOC) || - (!IS_ENABLED(CONFIG_KASAN_GENERIC) && - !IS_ENABLED(CONFIG_KASAN_SW_TAGS)))) { - jit_alloc_params.text.fallback_start = module_alloc_base; - jit_alloc_params.text.fallback_end = module_alloc_base + SZ_2G; - } - - return &jit_alloc_params; -} - enum aarch64_reloc_op { RELOC_OP_NONE, RELOC_OP_ABS, diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 66e70ca47680..a4463a35b3c5 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include @@ -493,3 +494,44 @@ void dump_mem_limit(void) pr_emerg("Memory Limit: none\n"); } } + +#ifdef CONFIG_JIT_ALLOC +static struct jit_alloc_params jit_alloc_params = { + .alignment = JIT_ALLOC_ALIGN, + .flags = JIT_ALLOC_KASAN_SHADOW, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + u64 module_alloc_end = module_alloc_base + MODULES_VSIZE; + + if (IS_ENABLED(CONFIG_KASAN_GENERIC) || + IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + /* don't exceed the static module region - see below */ + module_alloc_end = MODULES_END; + + jit_alloc_params.text.pgprot = PAGE_KERNEL; + jit_alloc_params.text.start = module_alloc_base; + jit_alloc_params.text.end = module_alloc_end; + + /* + * KASAN without KASAN_VMALLOC can only deal with module + * allocations being served from the reserved module region, + * since the remainder of the vmalloc region is already + * backed by zero shadow pages, and punching holes into it + * is non-trivial. Since the module region is not randomized + * when KASAN is enabled without KASAN_VMALLOC, it is even + * less likely that the module region gets exhausted, so we + * can simply omit this fallback in that case. + */ + if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) && + (IS_ENABLED(CONFIG_KASAN_VMALLOC) || + (!IS_ENABLED(CONFIG_KASAN_GENERIC) && + !IS_ENABLED(CONFIG_KASAN_SW_TAGS)))) { + jit_alloc_params.text.fallback_start = module_alloc_base; + jit_alloc_params.text.fallback_end = module_alloc_base + SZ_2G; + } + + return &jit_alloc_params; +} +#endif diff --git a/arch/loongarch/kernel/module.c b/arch/loongarch/kernel/module.c index 1d5e00874ae7..181b5f8b09f1 100644 --- a/arch/loongarch/kernel/module.c +++ b/arch/loongarch/kernel/module.c @@ -18,7 +18,6 @@ #include #include #include -#include #include #include @@ -470,19 +469,6 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, return 0; } -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, - .text.pgprot = PAGE_KERNEL, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.start = MODULES_VADDR; - jit_alloc_params.text.end = MODULES_END; - - return &jit_alloc_params; -} - static void module_init_ftrace_plt(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *mod) { diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index 3b7d8129570b..30ca8e497377 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -274,3 +275,18 @@ EXPORT_SYMBOL(invalid_pmd_table); #endif pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss; EXPORT_SYMBOL(invalid_pte_table); + +#ifdef CONFIG_JIT_ALLOC +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, + .text.pgprot = PAGE_KERNEL, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.start = MODULES_VADDR; + jit_alloc_params.text.end = MODULES_END; + + return &jit_alloc_params; +} +#endif diff --git a/arch/mips/kernel/module.c b/arch/mips/kernel/module.c index f762c697ab9c..dba78c7a4a88 100644 --- a/arch/mips/kernel/module.c +++ b/arch/mips/kernel/module.c @@ -20,7 +20,6 @@ #include #include #include -#include extern void jump_label_apply_nops(struct module *mod); @@ -33,22 +32,6 @@ struct mips_hi16 { static LIST_HEAD(dbe_list); static DEFINE_SPINLOCK(dbe_lock); -#ifdef MODULE_START - -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, - .text.start = MODULE_START, - .text.end = MODULE_END, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.pgprot = PAGE_KERNEL; - - return &jit_alloc_params; -} -#endif - static void apply_r_mips_32(u32 *location, u32 base, Elf_Addr v) { *location = base + v; diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c index 5a8002839550..1fd1bea78fdc 100644 --- a/arch/mips/mm/init.c +++ b/arch/mips/mm/init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include @@ -568,3 +569,21 @@ EXPORT_SYMBOL_GPL(invalid_pmd_table); #endif pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss; EXPORT_SYMBOL(invalid_pte_table); + +#ifdef CONFIG_JIT_ALLOC +#ifdef MODULE_START + +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, + .text.start = MODULE_START, + .text.end = MODULE_END, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.pgprot = PAGE_KERNEL; + + return &jit_alloc_params; +} +#endif +#endif diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c index 49fdf741fd24..3cb0b2c72d85 100644 --- a/arch/parisc/kernel/module.c +++ b/arch/parisc/kernel/module.c @@ -49,7 +49,6 @@ #include #include #include -#include #include #include @@ -174,22 +173,6 @@ static inline int reassemble_22(int as22) ((as22 & 0x0003ff) << 3)); } -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, - /* using RWX means less protection for modules, but it's - * easier than trying to map the text, data, init_text and - * init_data correctly */ - .text.pgprot = PAGE_KERNEL_RWX, - .text.end = VMALLOC_END, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.start = VMALLOC_START; - - return &jit_alloc_params; -} - #ifndef CONFIG_64BIT static inline unsigned long count_gots(const Elf_Rela *rela, unsigned long n) { diff --git a/arch/parisc/mm/init.c b/arch/parisc/mm/init.c index b0c43f3b0a5f..1601519486fa 100644 --- a/arch/parisc/mm/init.c +++ b/arch/parisc/mm/init.c @@ -24,6 +24,7 @@ #include /* for node_online_map */ #include /* for release_pages */ #include +#include #include #include @@ -479,7 +480,7 @@ void free_initmem(void) /* finally dump all the instructions which were cached, since the * pages are no-longer executable */ flush_icache_range(init_begin, init_end); - + free_initmem_default(POISON_FREE_INITMEM); /* set up a new led state on systems shipped LED State panel */ @@ -891,3 +892,21 @@ static const pgprot_t protection_map[16] = { [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_RWX }; DECLARE_VM_GET_PAGE_PROT + +#ifdef CONFIG_JIT_ALLOC +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, + /* using RWX means less protection for modules, but it's + * easier than trying to map the text, data, init_text and + * init_data correctly */ + .text.pgprot = PAGE_KERNEL_RWX, + .text.end = VMALLOC_END, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.start = VMALLOC_START; + + return &jit_alloc_params; +} +#endif diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c index b58af61e90c0..b30e00964a60 100644 --- a/arch/powerpc/kernel/module.c +++ b/arch/powerpc/kernel/module.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include @@ -89,41 +88,3 @@ int module_finalize(const Elf_Ehdr *hdr, return 0; } - -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - /* - * BOOK3S_32 and 8xx define MODULES_VADDR for text allocations and - * allow allocating data in the entire vmalloc space - */ -#ifdef MODULES_VADDR - pgprot_t prot = strict_module_rwx_enabled() ? PAGE_KERNEL : PAGE_KERNEL_EXEC; - unsigned long limit = (unsigned long)_etext - SZ_32M; - - jit_alloc_params.text.pgprot = prot; - - /* First try within 32M limit from _etext to avoid branch trampolines */ - if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit) { - jit_alloc_params.text.start = limit; - jit_alloc_params.text.end = MODULES_END; - jit_alloc_params.text.fallback_start = MODULES_VADDR; - jit_alloc_params.text.fallback_end = MODULES_END; - } else { - jit_alloc_params.text.start = MODULES_VADDR; - jit_alloc_params.text.end = MODULES_END; - } - - jit_alloc_params.data.pgprot = PAGE_KERNEL; - jit_alloc_params.data.start = VMALLOC_START; - jit_alloc_params.data.end = VMALLOC_END; -#else - jit_alloc_params.text.start = VMALLOC_START; - jit_alloc_params.text.end = VMALLOC_END; -#endif - - return &jit_alloc_params; -} diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 8b121df7b08f..de970988119f 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -406,3 +407,43 @@ int devmem_is_allowed(unsigned long pfn) * the EHEA driver. Drop this when drivers/net/ethernet/ibm/ehea is removed. */ EXPORT_SYMBOL_GPL(walk_system_ram_range); + +#ifdef CONFIG_JIT_ALLOC +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + /* + * BOOK3S_32 and 8xx define MODULES_VADDR for text allocations and + * allow allocating data in the entire vmalloc space + */ +#ifdef MODULES_VADDR + pgprot_t prot = strict_module_rwx_enabled() ? PAGE_KERNEL : PAGE_KERNEL_EXEC; + unsigned long limit = (unsigned long)_etext - SZ_32M; + + jit_alloc_params.text.pgprot = prot; + + /* First try within 32M limit from _etext to avoid branch trampolines */ + if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit) { + jit_alloc_params.text.start = limit; + jit_alloc_params.text.end = MODULES_END; + jit_alloc_params.text.fallback_start = MODULES_VADDR; + jit_alloc_params.text.fallback_end = MODULES_END; + } else { + jit_alloc_params.text.start = MODULES_VADDR; + jit_alloc_params.text.end = MODULES_END; + } + + jit_alloc_params.data.pgprot = PAGE_KERNEL; + jit_alloc_params.data.start = VMALLOC_START; + jit_alloc_params.data.end = VMALLOC_END; +#else + jit_alloc_params.text.start = VMALLOC_START; + jit_alloc_params.text.end = VMALLOC_END; +#endif + + return &jit_alloc_params; +} +#endif diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c index 731255654c94..8af08d5449bf 100644 --- a/arch/riscv/kernel/module.c +++ b/arch/riscv/kernel/module.c @@ -11,7 +11,6 @@ #include #include #include -#include #include #include @@ -436,21 +435,6 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, return 0; } -#if defined(CONFIG_MMU) && defined(CONFIG_64BIT) -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, - .text.pgprot = PAGE_KERNEL, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.start = MODULES_VADDR; - jit_alloc_params.text.end = MODULES_END; - - return &jit_alloc_params; -} -#endif - int module_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *me) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 747e5b1ef02d..5b87f83ef810 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -23,6 +23,7 @@ #ifdef CONFIG_RELOCATABLE #include #endif +#include #include #include @@ -1363,3 +1364,20 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, return vmemmap_populate_basepages(start, end, node, NULL); } #endif + +#ifdef CONFIG_JIT_ALLOC +#if defined(CONFIG_MMU) && defined(CONFIG_64BIT) +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, + .text.pgprot = PAGE_KERNEL, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.start = MODULES_VADDR; + jit_alloc_params.text.end = MODULES_END; + + return &jit_alloc_params; +} +#endif +#endif diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c index 3f85cf1e7c4e..0a4f4f32ef49 100644 --- a/arch/s390/kernel/module.c +++ b/arch/s390/kernel/module.c @@ -37,38 +37,6 @@ #define PLT_ENTRY_SIZE 22 -static unsigned long get_module_load_offset(void) -{ - static DEFINE_MUTEX(module_kaslr_mutex); - static unsigned long module_load_offset; - - if (!kaslr_enabled()) - return 0; - /* - * Calculate the module_load_offset the first time this code - * is called. Once calculated it stays the same until reboot. - */ - mutex_lock(&module_kaslr_mutex); - if (!module_load_offset) - module_load_offset = get_random_u32_inclusive(1, 1024) * PAGE_SIZE; - mutex_unlock(&module_kaslr_mutex); - return module_load_offset; -} - -static struct jit_alloc_params jit_alloc_params = { - .alignment = JIT_ALLOC_ALIGN, - .flags = JIT_ALLOC_KASAN_SHADOW, - .text.pgprot = PAGE_KERNEL, -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.start = MODULES_VADDR + get_module_load_offset(); - jit_alloc_params.text.end = MODULES_END; - - return &jit_alloc_params; -} - #ifdef CONFIG_FUNCTION_TRACER void module_arch_cleanup(struct module *mod) { diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c index 8d94e29adcdb..6e428e0f3215 100644 --- a/arch/s390/mm/init.c +++ b/arch/s390/mm/init.c @@ -50,6 +50,7 @@ #include #include #include +#include pgd_t swapper_pg_dir[PTRS_PER_PGD] __section(".bss..swapper_pg_dir"); pgd_t invalid_pg_dir[PTRS_PER_PGD] __section(".bss..invalid_pg_dir"); @@ -311,3 +312,37 @@ void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) vmem_remove_mapping(start, size); } #endif /* CONFIG_MEMORY_HOTPLUG */ + +#ifdef CONFIG_JIT_ALLOC +static unsigned long get_module_load_offset(void) +{ + static DEFINE_MUTEX(module_kaslr_mutex); + static unsigned long module_load_offset; + + if (!kaslr_enabled()) + return 0; + /* + * Calculate the module_load_offset the first time this code + * is called. Once calculated it stays the same until reboot. + */ + mutex_lock(&module_kaslr_mutex); + if (!module_load_offset) + module_load_offset = get_random_u32_inclusive(1, 1024) * PAGE_SIZE; + mutex_unlock(&module_kaslr_mutex); + return module_load_offset; +} + +static struct jit_alloc_params jit_alloc_params = { + .alignment = JIT_ALLOC_ALIGN, + .flags = JIT_ALLOC_KASAN_SHADOW, + .text.pgprot = PAGE_KERNEL, +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.start = MODULES_VADDR + get_module_load_offset(); + jit_alloc_params.text.end = MODULES_END; + + return &jit_alloc_params; +} +#endif diff --git a/arch/sparc/kernel/module.c b/arch/sparc/kernel/module.c index 03f0de693b4d..9edbd0372add 100644 --- a/arch/sparc/kernel/module.c +++ b/arch/sparc/kernel/module.c @@ -14,7 +14,6 @@ #include #include #include -#include #ifdef CONFIG_SPARC64 #include @@ -26,24 +25,6 @@ #include "entry.h" -static struct jit_alloc_params jit_alloc_params = { - .alignment = 1, -#ifdef CONFIG_SPARC64 - .text.start = MODULES_VADDR, - .text.end = MODULES_END, -#else - .text.start = VMALLOC_START, - .text.end = VMALLOC_END, -#endif -}; - -struct jit_alloc_params *jit_alloc_arch_params(void) -{ - jit_alloc_params.text.pgprot = PAGE_KERNEL; - - return &jit_alloc_params; -} - /* Make generic code ignore STT_REGISTER dummy undefined symbols. */ int module_frob_arch_sections(Elf_Ehdr *hdr, Elf_Shdr *sechdrs, diff --git a/arch/sparc/mm/Makefile b/arch/sparc/mm/Makefile index 871354aa3c00..95ede0fd851a 100644 --- a/arch/sparc/mm/Makefile +++ b/arch/sparc/mm/Makefile @@ -15,3 +15,5 @@ obj-$(CONFIG_SPARC32) += leon_mm.o # Only used by sparc64 obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o + +obj-$(CONFIG_JIT_ALLOC) += jitalloc.o diff --git a/arch/sparc/mm/jitalloc.c b/arch/sparc/mm/jitalloc.c new file mode 100644 index 000000000000..6b407a8e85ef --- /dev/null +++ b/arch/sparc/mm/jitalloc.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +static struct jit_alloc_params jit_alloc_params = { + .alignment = 1, +#ifdef CONFIG_SPARC64 + .text.start = MODULES_VADDR, + .text.end = MODULES_END, +#else + .text.start = VMALLOC_START, + .text.end = VMALLOC_END, +#endif +}; + +struct jit_alloc_params *jit_alloc_arch_params(void) +{ + jit_alloc_params.text.pgprot = PAGE_KERNEL; + + return &jit_alloc_params; +}