From patchwork Mon Sep 18 07:29:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141289 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2498107vqi; Mon, 18 Sep 2023 01:07:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFGMzoOOCKqxupsH2jtCZKjwHAeRMZ8v08D0mBDdkjQ8HQr4Yn5o+WHISLy5fc3udUDP9md X-Received: by 2002:a05:6808:3ce:b0:3a7:459d:61bf with SMTP id o14-20020a05680803ce00b003a7459d61bfmr9015040oie.0.1695024438545; Mon, 18 Sep 2023 01:07:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695024438; cv=none; d=google.com; s=arc-20160816; b=s3u2hk7t5Z0ekZam3ph5kKGmaPXCW6XPy+P18UlKI14bsFS/0omRiSwa+pT3pkHYTb hRXW3RJ/5TkV0fOqrP7bP9/x5nJQRGTWCVS4r+eaPkwopznovgk6HJwoaGMJwkcpV4TM ZQ/I+UR4+5AaDe9z+rSXAF4Dyqx1/8eHLS0maQgLX2qMgBI3AtbHie/9Y/YTTQKn1h5H wBt0LWWFMch22gfOlMuQJpnsq8VwnPGxtWqYftGptH7RNwZxlC3yR5DkZvkcUZaEQU7D MfSFF1glmVM8wHMvibelLNUnq6F+Wivo+HbTUjX6dd2uYekT2AJLnGX0WR77WQvft29p UV7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=IkPztNd7bnD91GoOX9u3X5sY7LI8HpPP33/j7zBq9yU=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=UIwi/GAg4qgNQesB3+O3le3xHdZMQTm7iUeckOQbzxe66V6GM8uh7R7mrpuI5bxWk8 46EFQfhoQQ6aOlT0KQx+k2DMdwx5JLETt6NTaPXA4fmKjFKrXTPWeY7ybxQ9cOGYhyIC 83MGU5Sni9sFSTmLW+spyia98C0el50VIWUFW74OyZUVJUJXo6TMGXnnq75YoDf/7CVd 6oMfr1IXM/rVmv0Oty9gBBjtWDqf3hbcnPOCn9aQiXlHS8N9OTGB0yqDhT5FDoNTmpQp HVt5VXI/1RQq3jv5AUk0zuC4YoezmneEKvgwYrVJb9GJxc/MB9fUAaoe0Q8lGdXYjW3X AbHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=n1BsBmpf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id s196-20020a632ccd000000b00573ffd25b44si7646800pgs.182.2023.09.18.01.07.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 01:07:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=n1BsBmpf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id CA018808E649; Mon, 18 Sep 2023 00:52:23 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240275AbjIRHav (ORCPT + 27 others); Mon, 18 Sep 2023 03:30:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240329AbjIRHak (ORCPT ); Mon, 18 Sep 2023 03:30:40 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA38C19A; Mon, 18 Sep 2023 00:30:24 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 55C0EC43397; Mon, 18 Sep 2023 07:30:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022224; bh=KSPIxmcvM6pw+gN3UxNfSb/hLNccmDr75av1M8zdRQI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n1BsBmpfW3QKK3NJROVE3ZzVUFsmFfDjPKEBn27xuClR0kSquHorXzHaU80koKHjq oZZubDbrJshf2OzAsSW/lYZeiCxCVkNHUyRKf0J25B3oyQqT68RZdj7eYC/N0Cta3F s7s8WnXGl08TzUW7RJcoZTxms2v0q/MjRq68IGaI1IRBqPpLAZ6vaNChPkgshwIF5h 3KnstVmy4jP0LDvFd3KogmLI2XFZUpl0AY5gcnG4Aq2Zy+SxUhV393i/W7f5/3shGG k9UrFYiH9x7//wdEvOIFolwWa2pPRCHkauyGN/FXYwMai04Ht7kCMou6e5Iw71dZtp HuRwsr2VE4kwA== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 01/13] nios2: define virtual address space for modules Date: Mon, 18 Sep 2023 10:29:43 +0300 Message-Id: <20230918072955.2507221-2-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:52:23 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777361945749020837 X-GMAIL-MSGID: 1777361945749020837 From: "Mike Rapoport (IBM)" nios2 uses kmalloc() to implement module_alloc() because CALL26/PCREL26 cannot reach all of vmalloc address space. Define module space as 32MiB below the kernel base and switch nios2 to use vmalloc for module allocations. Suggested-by: Thomas Gleixner Acked-by: Dinh Nguyen Acked-by: Song Liu Signed-off-by: Mike Rapoport (IBM) --- arch/nios2/include/asm/pgtable.h | 5 ++++- arch/nios2/kernel/module.c | 19 ++++--------------- 2 files changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/nios2/include/asm/pgtable.h b/arch/nios2/include/asm/pgtable.h index 5144506dfa69..d2fb42fb6db8 100644 --- a/arch/nios2/include/asm/pgtable.h +++ b/arch/nios2/include/asm/pgtable.h @@ -25,7 +25,10 @@ #include #define VMALLOC_START CONFIG_NIOS2_KERNEL_MMU_REGION_BASE -#define VMALLOC_END (CONFIG_NIOS2_KERNEL_REGION_BASE - 1) +#define VMALLOC_END (CONFIG_NIOS2_KERNEL_REGION_BASE - SZ_32M - 1) + +#define MODULES_VADDR (CONFIG_NIOS2_KERNEL_REGION_BASE - SZ_32M) +#define MODULES_END (CONFIG_NIOS2_KERNEL_REGION_BASE - 1) struct mm_struct; diff --git a/arch/nios2/kernel/module.c b/arch/nios2/kernel/module.c index 76e0a42d6e36..9c97b7513853 100644 --- a/arch/nios2/kernel/module.c +++ b/arch/nios2/kernel/module.c @@ -21,23 +21,12 @@ #include -/* - * Modules should NOT be allocated with kmalloc for (obvious) reasons. - * But we do it for now to avoid relocation issues. CALL26/PCREL26 cannot reach - * from 0x80000000 (vmalloc area) to 0xc00000000 (kernel) (kmalloc returns - * addresses in 0xc0000000) - */ void *module_alloc(unsigned long size) { - if (size == 0) - return NULL; - return kmalloc(size, GFP_KERNEL); -} - -/* Free memory returned from module_alloc */ -void module_memfree(void *module_region) -{ - kfree(module_region); + return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, + GFP_KERNEL, PAGE_KERNEL_EXEC, + VM_FLUSH_RESET_PERMS, NUMA_NO_NODE, + __builtin_return_address(0)); } int apply_relocate_add(Elf32_Shdr *sechdrs, const char *strtab, From patchwork Mon Sep 18 07:29:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141270 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2487616vqi; Mon, 18 Sep 2023 00:37:32 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHWcrPw02fZuJEwkLQpv2M+VrE9nU/0v94xG4AWH6y/oQMvQvrwAjL2FgR6bJcknJeRqbie X-Received: by 2002:a05:6358:341d:b0:140:ecf2:4c7e with SMTP id h29-20020a056358341d00b00140ecf24c7emr8007776rwd.1.1695022652022; Mon, 18 Sep 2023 00:37:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695022651; cv=none; d=google.com; s=arc-20160816; b=UMXODgKDJP/Bfu19BR5P2zbh+fPcTpF3SOPN9fm1GLxZWOdkJVbDQIWi3eB3OuqFyx UGySF62bxhcW7HZHqqboO0YeNdgiLRho5EH8kKXWjyZ6yYuVgXhLcotIsx1dD2zlfijP cBNAUsIjDrylSR7GBZ5E4fOTqKE+kdULYf5s0B3c5ENxy87nVMkWNo/Bxdfqo7OQwGEx x64HgjMeJ29M33D6t3hs/QnJ7C9r3oCwuKPMscqRy6LrZmBjR8H2g1EaLuLgTclbsr/P 36L4nhk0lRRKlWVxIA6Db1wWpbcmZ5LM0Bz6YaHiRh9WGacULkSVeqZ1GA6cK5uJTR6d Hw0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=W/azGQGI+uP/W0OvBaLCbphxVCzvPSm9O0Qscmq5+HA=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=DiW/Y5l/eXPBDdMSycN3WgSjFJbxyEd/KT/oTz1XUSMiIpUOdfDzKrE9FlwkaKSC7w 4Lact6EuaTMOtmN9g5fH4WCgWQ3RPSzm1Vzv0nWwyz30r91dMB89e4y9SbCv4kBYyNl3 gu+8pHD2jAqF/+1cYjd2JNmZmndGiQhERtiLStvvwQjnIjmZsEf6layLQAGNShWp5cAK I8td8TNAcq9ZsegzDsMXLE/zitQBpRiVZn1NvdGbt+uAAVaxbg1JsR0/ChRRh4oAdZIZ wrPsHlXygAwTnxBGgJTumgMjG2V6MYI58V2gsJTKu905xg8KpP1QYRVSkHCXXvW1kQZ8 dG/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TBLQcR1l; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id w20-20020a63fb54000000b00577ac67a31dsi7361567pgj.757.2023.09.18.00.37.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:37:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TBLQcR1l; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id E804B818C799; Mon, 18 Sep 2023 00:32:04 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239915AbjIRHbb (ORCPT + 27 others); Mon, 18 Sep 2023 03:31:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240252AbjIRHas (ORCPT ); Mon, 18 Sep 2023 03:30:48 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC96A120; Mon, 18 Sep 2023 00:30:34 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D537C433C8; Mon, 18 Sep 2023 07:30:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022234; bh=bvwcP2LqH7M0LqBQS8gC0daDhK3BG4ji8BhLvEjeq+s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TBLQcR1lAmU6KOalSsJ/oR1o5zZoazG/YXTfJUDqgJb0o6vPktmqTK+vjgO53nDe2 wRRsB421t+7jR7di9+tRCC1Vc5UL9ISE3HJXSgRpGXB2AcnDWCxIzabugyKHowlQRM dVtPneEfqW56D4trpvlIHzeEr1S//A3oRTZGBXCHfaluSiZJrflI+JHArvH50+TYYA RgIwhEVFloUsg24G4ybg8nt2Ea/Mr7bBH7Fum/wXH0+jrq62z5TRAralvtFuyV85hy FhD+mpWRLiJSUQyEBvo38Sy93YKxiF+tl1IvR4PCSFVan844bfOzzMlfFZccF2EytU +u0ufqp2nMUIg== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 02/13] mm: introduce execmem_text_alloc() and execmem_free() Date: Mon, 18 Sep 2023 10:29:44 +0300 Message-Id: <20230918072955.2507221-3-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:32:04 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360072636626817 X-GMAIL-MSGID: 1777360072636626817 From: "Mike Rapoport (IBM)" module_alloc() is used everywhere as a mean to allocate memory for code. Beside being semantically wrong, this unnecessarily ties all subsystems that need to allocate code, such as ftrace, kprobes and BPF to modules and puts the burden of code allocation to the modules code. Several architectures override module_alloc() because of various constraints where the executable memory can be located and this causes additional obstacles for improvements of code allocation. Start splitting code allocation from modules by introducing execmem_text_alloc() and execmem_free() APIs. Initially, execmem_text_alloc() is a wrapper for module_alloc() and execmem_free() is a replacement of module_memfree() to allow updating all call sites to use the new APIs. Since architectures define different restrictions on placement, permissions, alignment and other parameters for memory that can be used by different subsystems that allocate executable memory, execmem_text_alloc() takes a type argument, that will be used to identify the calling subsystem and to allow architectures define parameters for ranges suitable for that subsystem. The name execmem_text_alloc() emphasizes that the allocated memory is for executable code, the allocations of the associated data, like data sections of a module will use execmem_data_alloc() interface that will be added later. Signed-off-by: Mike Rapoport (IBM) --- arch/powerpc/kernel/kprobes.c | 4 +-- arch/s390/kernel/ftrace.c | 4 +-- arch/s390/kernel/kprobes.c | 4 +-- arch/s390/kernel/module.c | 5 +-- arch/sparc/net/bpf_jit_comp_32.c | 8 ++--- arch/x86/kernel/ftrace.c | 6 ++-- arch/x86/kernel/kprobes/core.c | 4 +-- include/linux/execmem.h | 56 ++++++++++++++++++++++++++++++++ include/linux/moduleloader.h | 3 -- kernel/bpf/core.c | 6 ++-- kernel/kprobes.c | 8 ++--- kernel/module/Kconfig | 1 + kernel/module/main.c | 25 +++++--------- mm/Kconfig | 3 ++ mm/Makefile | 1 + mm/execmem.c | 26 +++++++++++++++ 16 files changed, 120 insertions(+), 44 deletions(-) create mode 100644 include/linux/execmem.h create mode 100644 mm/execmem.c diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c index b20ee72e873a..62228c7072a2 100644 --- a/arch/powerpc/kernel/kprobes.c +++ b/arch/powerpc/kernel/kprobes.c @@ -19,8 +19,8 @@ #include #include #include -#include #include +#include #include #include #include @@ -130,7 +130,7 @@ void *alloc_insn_page(void) { void *page; - page = module_alloc(PAGE_SIZE); + page = execmem_text_alloc(EXECMEM_KPROBES, PAGE_SIZE); if (!page) return NULL; diff --git a/arch/s390/kernel/ftrace.c b/arch/s390/kernel/ftrace.c index c46381ea04ec..4052e10eb6a4 100644 --- a/arch/s390/kernel/ftrace.c +++ b/arch/s390/kernel/ftrace.c @@ -7,13 +7,13 @@ * Author(s): Martin Schwidefsky */ -#include #include #include #include #include #include #include +#include #include #include #include @@ -220,7 +220,7 @@ static int __init ftrace_plt_init(void) { const char *start, *end; - ftrace_plt = module_alloc(PAGE_SIZE); + ftrace_plt = execmem_text_alloc(EXECMEM_FTRACE, PAGE_SIZE); if (!ftrace_plt) panic("cannot allocate ftrace plt\n"); diff --git a/arch/s390/kernel/kprobes.c b/arch/s390/kernel/kprobes.c index d4b863ed0aa7..48928460dcb9 100644 --- a/arch/s390/kernel/kprobes.c +++ b/arch/s390/kernel/kprobes.c @@ -9,7 +9,6 @@ #define pr_fmt(fmt) "kprobes: " fmt -#include #include #include #include @@ -21,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -38,7 +38,7 @@ void *alloc_insn_page(void) { void *page; - page = module_alloc(PAGE_SIZE); + page = execmem_text_alloc(EXECMEM_KPROBES, PAGE_SIZE); if (!page) return NULL; set_memory_rox((unsigned long)page, 1); diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c index 42215f9404af..db5561d0c233 100644 --- a/arch/s390/kernel/module.c +++ b/arch/s390/kernel/module.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -76,7 +77,7 @@ void *module_alloc(unsigned long size) #ifdef CONFIG_FUNCTION_TRACER void module_arch_cleanup(struct module *mod) { - module_memfree(mod->arch.trampolines_start); + execmem_free(mod->arch.trampolines_start); } #endif @@ -510,7 +511,7 @@ static int module_alloc_ftrace_hotpatch_trampolines(struct module *me, size = FTRACE_HOTPATCH_TRAMPOLINES_SIZE(s->sh_size); numpages = DIV_ROUND_UP(size, PAGE_SIZE); - start = module_alloc(numpages * PAGE_SIZE); + start = execmem_text_alloc(EXECMEM_FTRACE, numpages * PAGE_SIZE); if (!start) return -ENOMEM; set_memory_rox((unsigned long)start, numpages); diff --git a/arch/sparc/net/bpf_jit_comp_32.c b/arch/sparc/net/bpf_jit_comp_32.c index a74e5004c6c8..5fa9c45fba0a 100644 --- a/arch/sparc/net/bpf_jit_comp_32.c +++ b/arch/sparc/net/bpf_jit_comp_32.c @@ -1,10 +1,10 @@ // SPDX-License-Identifier: GPL-2.0 -#include #include #include #include #include #include +#include #include #include @@ -713,7 +713,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; if (unlikely(proglen + ilen > oldproglen)) { pr_err("bpb_jit_compile fatal error\n"); kfree(addrs); - module_memfree(image); + execmem_free(image); return; } memcpy(image + proglen, temp, ilen); @@ -736,7 +736,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; break; } if (proglen == oldproglen) { - image = module_alloc(proglen); + image = execmem_text_alloc(EXECMEM_BPF, proglen); if (!image) goto out; } @@ -758,7 +758,7 @@ cond_branch: f_offset = addrs[i + filter[i].jf]; void bpf_jit_free(struct bpf_prog *fp) { if (fp->jited) - module_memfree(fp->bpf_func); + execmem_free(fp->bpf_func); bpf_prog_unlock_free(fp); } diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 12df54ff0e81..ae56d79a6a74 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -25,6 +25,7 @@ #include #include #include +#include #include @@ -261,15 +262,14 @@ void arch_ftrace_update_code(int command) #ifdef CONFIG_X86_64 #ifdef CONFIG_MODULES -#include /* Module allocation simplifies allocating memory for code */ static inline void *alloc_tramp(unsigned long size) { - return module_alloc(size); + return execmem_text_alloc(EXECMEM_FTRACE, size); } static inline void tramp_free(void *tramp) { - module_memfree(tramp); + execmem_free(tramp); } #else /* Trampolines can only be created if modules are supported */ diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index e8babebad7b8..c4f58e893efd 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -40,12 +40,12 @@ #include #include #include -#include #include #include #include #include #include +#include #include #include @@ -448,7 +448,7 @@ void *alloc_insn_page(void) { void *page; - page = module_alloc(PAGE_SIZE); + page = execmem_text_alloc(EXECMEM_KPROBES, PAGE_SIZE); if (!page) return NULL; diff --git a/include/linux/execmem.h b/include/linux/execmem.h new file mode 100644 index 000000000000..3491bf7e9714 --- /dev/null +++ b/include/linux/execmem.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_EXECMEM_ALLOC_H +#define _LINUX_EXECMEM_ALLOC_H + +#include + +/** + * enum execmem_type - types of executable memory ranges + * + * There are several subsystems that allocate executable memory. + * Architectures define different restrictions on placement, + * permissions, alignment and other parameters for memory that can be used + * by these subsystems. + * Types in this enum identify subsystems that allocate executable memory + * and let architectures define parameters for ranges suitable for + * allocations by each subsystem. + * + * @EXECMEM_DEFAULT: default parameters that would be used for types that + * are not explcitly defined. + * @EXECMEM_MODULE_TEXT: parameters for module text sections + * @EXECMEM_KPROBES: parameters for kprobes + * @EXECMEM_FTRACE: parameters for ftrace + * @EXECMEM_BPF: parameters for BPF + * @EXECMEM_TYPE_MAX: + */ +enum execmem_type { + EXECMEM_DEFAULT, + EXECMEM_MODULE_TEXT = EXECMEM_DEFAULT, + EXECMEM_KPROBES, + EXECMEM_FTRACE, + EXECMEM_BPF, + EXECMEM_TYPE_MAX, +}; + +/** + * execmem_text_alloc - allocate executable memory + * @type: type of the allocation + * @size: how many bytes of memory are required + * + * Allocates memory that will contain executable code, either generated or + * loaded from kernel modules. + * + * The memory will have protections defined by architecture for executable + * region of the @type. + * + * Return: a pointer to the allocated memory or %NULL + */ +void *execmem_text_alloc(enum execmem_type type, size_t size); + +/** + * execmem_free - free executable memory + * @ptr: pointer to the memory that should be freed + */ +void execmem_free(void *ptr); + +#endif /* _LINUX_EXECMEM_ALLOC_H */ diff --git a/include/linux/moduleloader.h b/include/linux/moduleloader.h index 001b2ce83832..a23718aa2f4d 100644 --- a/include/linux/moduleloader.h +++ b/include/linux/moduleloader.h @@ -29,9 +29,6 @@ unsigned int arch_mod_section_prepend(struct module *mod, unsigned int section); sections. Returns NULL on failure. */ void *module_alloc(unsigned long size); -/* Free memory returned from module_alloc. */ -void module_memfree(void *module_region); - /* Determines if the section name is an init section (that is only used during * module loading). */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 4e3ce0542e31..75249f2d9f77 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -22,7 +22,6 @@ #include #include #include -#include #include #include #include @@ -37,6 +36,7 @@ #include #include #include +#include #include #include @@ -1007,12 +1007,12 @@ void bpf_jit_uncharge_modmem(u32 size) void *__weak bpf_jit_alloc_exec(unsigned long size) { - return module_alloc(size); + return execmem_text_alloc(EXECMEM_BPF, size); } void __weak bpf_jit_free_exec(void *addr) { - module_memfree(addr); + execmem_free(addr); } struct bpf_binary_header * diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 0c6185aefaef..0ccb4d2ec9a2 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -26,7 +26,6 @@ #include #include #include -#include #include #include #include @@ -39,6 +38,7 @@ #include #include #include +#include #include #include @@ -113,17 +113,17 @@ enum kprobe_slot_state { void __weak *alloc_insn_page(void) { /* - * Use module_alloc() so this page is within +/- 2GB of where the + * Use execmem_text_alloc() so this page is within +/- 2GB of where the * kernel image and loaded module images reside. This is required * for most of the architectures. * (e.g. x86-64 needs this to handle the %rip-relative fixups.) */ - return module_alloc(PAGE_SIZE); + return execmem_text_alloc(EXECMEM_KPROBES, PAGE_SIZE); } static void free_insn_page(void *page) { - module_memfree(page); + execmem_free(page); } struct kprobe_insn_cache kprobe_insn_slots = { diff --git a/kernel/module/Kconfig b/kernel/module/Kconfig index 33a2e991f608..813e116bdee6 100644 --- a/kernel/module/Kconfig +++ b/kernel/module/Kconfig @@ -2,6 +2,7 @@ menuconfig MODULES bool "Enable loadable module support" modules + select EXECMEM help Kernel modules are small pieces of compiled code which can be inserted in the running kernel, rather than being diff --git a/kernel/module/main.c b/kernel/module/main.c index 98fedfdb8db5..4ec982cc943c 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -57,6 +57,7 @@ #include #include #include +#include #include #include "internal.h" @@ -1179,16 +1180,6 @@ resolve_symbol_wait(struct module *mod, return ksym; } -void __weak module_memfree(void *module_region) -{ - /* - * This memory may be RO, and freeing RO memory in an interrupt is not - * supported by vmalloc. - */ - WARN_ON(in_interrupt()); - vfree(module_region); -} - void __weak module_arch_cleanup(struct module *mod) { } @@ -1207,7 +1198,7 @@ static void *module_memory_alloc(unsigned int size, enum mod_mem_type type) { if (mod_mem_use_vmalloc(type)) return vzalloc(size); - return module_alloc(size); + return execmem_text_alloc(EXECMEM_MODULE_TEXT, size); } static void module_memory_free(void *ptr, enum mod_mem_type type) @@ -1215,7 +1206,7 @@ static void module_memory_free(void *ptr, enum mod_mem_type type) if (mod_mem_use_vmalloc(type)) vfree(ptr); else - module_memfree(ptr); + execmem_free(ptr); } static void free_mod_mem(struct module *mod) @@ -2479,9 +2470,9 @@ static void do_free_init(struct work_struct *w) llist_for_each_safe(pos, n, list) { initfree = container_of(pos, struct mod_initfree, node); - module_memfree(initfree->init_text); - module_memfree(initfree->init_data); - module_memfree(initfree->init_rodata); + execmem_free(initfree->init_text); + execmem_free(initfree->init_data); + execmem_free(initfree->init_rodata); kfree(initfree); } } @@ -2584,10 +2575,10 @@ static noinline int do_init_module(struct module *mod) * We want to free module_init, but be aware that kallsyms may be * walking this with preempt disabled. In all the failure paths, we * call synchronize_rcu(), but we don't want to slow down the success - * path. module_memfree() cannot be called in an interrupt, so do the + * path. execmem_free() cannot be called in an interrupt, so do the * work and call synchronize_rcu() in a work queue. * - * Note that module_alloc() on most architectures creates W+X page + * Note that execmem_text_alloc() on most architectures creates W+X page * mappings which won't be cleaned up until do_free_init() runs. Any * code such as mark_rodata_ro() which depends on those mappings to * be cleaned up needs to sync with the queued work - ie diff --git a/mm/Kconfig b/mm/Kconfig index 264a2df5ecf5..fb12931238e8 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1258,6 +1258,9 @@ config LOCK_MM_AND_FIND_VMA bool depends on !STACK_GROWSUP +config EXECMEM + bool + source "mm/damon/Kconfig" endmenu diff --git a/mm/Makefile b/mm/Makefile index ec65984e2ade..2e5fec94f09c 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -138,3 +138,4 @@ obj-$(CONFIG_IO_MAPPING) += io-mapping.o obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o +obj-$(CONFIG_EXECMEM) += execmem.o diff --git a/mm/execmem.c b/mm/execmem.c new file mode 100644 index 000000000000..638dc2b26a81 --- /dev/null +++ b/mm/execmem.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include +#include + +static void *execmem_alloc(size_t size) +{ + return module_alloc(size); +} + +void *execmem_text_alloc(enum execmem_type type, size_t size) +{ + return execmem_alloc(size); +} + +void execmem_free(void *ptr) +{ + /* + * This memory may be RO, and freeing RO memory in an interrupt is not + * supported by vmalloc. + */ + WARN_ON(in_interrupt()); + vfree(ptr); +} From patchwork Mon Sep 18 07:29:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141292 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2500242vqi; Mon, 18 Sep 2023 01:12:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGOQI8dWBxgmnTL16ODzfqNMl8VNd00gSt4R/1A41msDMJYqh5MIzR5zLMNvGlWegw/D3zi X-Received: by 2002:a17:903:1251:b0:1c3:8230:30d8 with SMTP id u17-20020a170903125100b001c3823030d8mr8838520plh.38.1695024764424; Mon, 18 Sep 2023 01:12:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695024764; cv=none; d=google.com; s=arc-20160816; b=tlR9lWX0Cjj2n7NselWTgkYwJjNg+Oa60VtImPIz3L9gSjU10loNFP5Na+lrp2pyhb hP8JImZvO0zqZaLYB26vv5YT8hD/v2ZzNRwj0FYoKleXoeD68lVrGdL7xTsCCaog8Rja jpWTo/MfZT/gzRU7tvzmWqDNqUZ6EODdb1Wangl+2S8FufvJfESUrVgpF35+nbmkvWhC 1dTNxM+ZiIoolymX2s5UoDTQ7QjemCp5zrR/2mafXl10yG90thU+OSC5f9VQ+H5QsLhQ QrhXDUDLNah+guR8vr0gruV/cQCIXjyjcBN3GxPap1+1XOsSFT6MHkiPL3OlABgLHXY5 WQvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QiFG+H2gwHWRM2G3A0YywAW1M0fMEUih/EbPX4rtawY=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=tepq/02bp3G8uwe9VD0EScS23c6F0zAjqlsfPCq94JMVv1kb+sL9e6FP2lJBMGi2St CM+GMydNHfUoSZceTkBXzm1BHcc9oJ2EFKNACPRR3QNFoegXhCyHH8ht+dev1zgWk4Wh V53ewTKaNtexuaJeK97GhEuph+iRwAN7c4q2goZmIT7607LDq6BYIBy+11tzmCn8owGx XCh8+sGwtfK913zsToob7kE4nMYUCj0t+VPHZxU2M0MocBKV5/cYTV/G9UUbUJwRmriP TlT1RuFkgmIGUNqvSssCf7/y5Xwp/icgsmmPbVhDRXv9cus6JcdLyPDCH0zmh6GfxRbO gokQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sSCiLbiN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id k7-20020a170902c40700b001b3c63eba76si7914510plk.492.2023.09.18.01.12.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 01:12:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sSCiLbiN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 322F581829DD; Mon, 18 Sep 2023 00:53:42 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240311AbjIRHwe (ORCPT + 27 others); Mon, 18 Sep 2023 03:52:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238367AbjIRHa4 (ORCPT ); Mon, 18 Sep 2023 03:30:56 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FEE2189; Mon, 18 Sep 2023 00:30:44 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F1667C433AB; Mon, 18 Sep 2023 07:30:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022244; bh=sRqJHHj6sN+zVuxAlEgaERQMpk9iFfs+P0v/5bsjzqU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sSCiLbiNncF007b8yg/PCaamnCR3IktSTan43cauE8/7iFy9m2xixDIpEVuIJReLP jrggUxEzdIvYURXnGZUsrjawe2u27YGN2kA82YiX1l3NRKgcmaNnhRD4rJ3lwcYZFH kcsyJaoBejSzi/mjeTbKUXDpZatwwEE68FNOT2kvW74EcuAw7/7pEI72l1dj86JMJG tO7yZMu8nliG1qmp1eySQMtAnE8tCUfbmYaCkFhY/eq4QY0IuzpNmP6jmRXX4716Qz r28qd7FAGP4eyZIrKeBmxc2Zjbl6SHjAiWIHvssIlogTeakI1PofbFTxkyfvNtlbRA f8aCB2/C6EHLw== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 03/13] mm/execmem, arch: convert simple overrides of module_alloc to execmem Date: Mon, 18 Sep 2023 10:29:45 +0300 Message-Id: <20230918072955.2507221-4-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:53:42 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777362286915237452 X-GMAIL-MSGID: 1777362286915237452 From: "Mike Rapoport (IBM)" Several architectures override module_alloc() only to define address range for code allocations different than VMALLOC address space. Provide a generic implementation in execmem that uses the parameters for address space ranges, required alignment and page protections provided by architectures. The architectures must fill execmem_params structure and implement execmem_arch_params() that returns a pointer to that structure. This way the execmem initialization won't be called from every architecture, but rather from a central place, namely initialization of the core memory management. The execmem provides execmem_text_alloc() API that wraps __vmalloc_node_range() with the parameters defined by the architectures. If an architecture does not implement execmem_arch_params(), execmem_text_alloc() will fall back to module_alloc(). The name execmem_text_alloc() emphasizes that the allocated memory is for executable code, the allocations of the associated data, like data sections of a module will use execmem_data_alloc() interface that will be added later. Signed-off-by: Mike Rapoport (IBM) --- arch/loongarch/kernel/module.c | 18 ++++++++-- arch/mips/kernel/module.c | 19 +++++++--- arch/nios2/kernel/module.c | 19 +++++++--- arch/parisc/kernel/module.c | 23 +++++++----- arch/riscv/kernel/module.c | 20 ++++++++--- arch/sparc/kernel/module.c | 44 +++++++++++------------ include/linux/execmem.h | 44 +++++++++++++++++++++++ mm/execmem.c | 66 ++++++++++++++++++++++++++++++++-- mm/mm_init.c | 2 ++ 9 files changed, 203 insertions(+), 52 deletions(-) diff --git a/arch/loongarch/kernel/module.c b/arch/loongarch/kernel/module.c index b8b86088b2dd..a1d8fe9796fa 100644 --- a/arch/loongarch/kernel/module.c +++ b/arch/loongarch/kernel/module.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -469,10 +470,21 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, return 0; } -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .pgprot = PAGE_KERNEL, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, - GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0)); + execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + + return &execmem_params; } static void module_init_ftrace_plt(const Elf_Ehdr *hdr, diff --git a/arch/mips/kernel/module.c b/arch/mips/kernel/module.c index 0c936cbf20c5..1c959074b35f 100644 --- a/arch/mips/kernel/module.c +++ b/arch/mips/kernel/module.c @@ -20,6 +20,7 @@ #include #include #include +#include extern void jump_label_apply_nops(struct module *mod); @@ -33,11 +34,21 @@ static LIST_HEAD(dbe_list); static DEFINE_SPINLOCK(dbe_lock); #ifdef MODULE_START -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .start = MODULE_START, + .end = MODULE_END, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - return __vmalloc_node_range(size, 1, MODULE_START, MODULE_END, - GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, - __builtin_return_address(0)); + execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; + + return &execmem_params; } #endif diff --git a/arch/nios2/kernel/module.c b/arch/nios2/kernel/module.c index 9c97b7513853..5a8df4f9c04e 100644 --- a/arch/nios2/kernel/module.c +++ b/arch/nios2/kernel/module.c @@ -18,15 +18,24 @@ #include #include #include +#include #include -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .start = MODULES_VADDR, + .end = MODULES_END, + .pgprot = PAGE_KERNEL_EXEC, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, - GFP_KERNEL, PAGE_KERNEL_EXEC, - VM_FLUSH_RESET_PERMS, NUMA_NO_NODE, - __builtin_return_address(0)); + return &execmem_params; } int apply_relocate_add(Elf32_Shdr *sechdrs, const char *strtab, diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c index d214bbe3c2af..0c6dfd1daef3 100644 --- a/arch/parisc/kernel/module.c +++ b/arch/parisc/kernel/module.c @@ -49,6 +49,7 @@ #include #include #include +#include #include #include @@ -173,15 +174,21 @@ static inline int reassemble_22(int as22) ((as22 & 0x0003ff) << 3)); } -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .pgprot = PAGE_KERNEL_RWX, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - /* using RWX means less protection for modules, but it's - * easier than trying to map the text, data, init_text and - * init_data correctly */ - return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, - GFP_KERNEL, - PAGE_KERNEL_RWX, 0, NUMA_NO_NODE, - __builtin_return_address(0)); + execmem_params.ranges[EXECMEM_DEFAULT].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_DEFAULT].end = VMALLOC_END; + + return &execmem_params; } #ifndef CONFIG_64BIT diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c index 7c651d55fcbd..343a0edfb6dd 100644 --- a/arch/riscv/kernel/module.c +++ b/arch/riscv/kernel/module.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -436,12 +437,21 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, } #if defined(CONFIG_MMU) && defined(CONFIG_64BIT) -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .pgprot = PAGE_KERNEL, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - return __vmalloc_node_range(size, 1, MODULES_VADDR, - MODULES_END, GFP_KERNEL, - PAGE_KERNEL, 0, NUMA_NO_NODE, - __builtin_return_address(0)); + execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + + return &execmem_params; } #endif diff --git a/arch/sparc/kernel/module.c b/arch/sparc/kernel/module.c index 66c45a2764bc..1d8d1fba95b9 100644 --- a/arch/sparc/kernel/module.c +++ b/arch/sparc/kernel/module.c @@ -14,6 +14,10 @@ #include #include #include +#include +#ifdef CONFIG_SPARC64 +#include +#endif #include #include @@ -21,34 +25,26 @@ #include "entry.h" +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { #ifdef CONFIG_SPARC64 - -#include - -static void *module_map(unsigned long size) -{ - if (PAGE_ALIGN(size) > MODULES_LEN) - return NULL; - return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, - GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, - __builtin_return_address(0)); -} + .start = MODULES_VADDR, + .end = MODULES_END, #else -static void *module_map(unsigned long size) -{ - return vmalloc(size); -} -#endif /* CONFIG_SPARC64 */ - -void *module_alloc(unsigned long size) + .start = VMALLOC_START, + .end = VMALLOC_END, +#endif + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - void *ret; - - ret = module_map(size); - if (ret) - memset(ret, 0, size); + execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; - return ret; + return &execmem_params; } /* Make generic code ignore STT_REGISTER dummy undefined symbols. */ diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 3491bf7e9714..44e213625053 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -32,6 +32,44 @@ enum execmem_type { EXECMEM_TYPE_MAX, }; +/** + * struct execmem_range - definition of a memory range suitable for code and + * related data allocations + * @start: address space start + * @end: address space end (inclusive) + * @pgprot: permissions for memory in this address space + * @alignment: alignment required for text allocations + */ +struct execmem_range { + unsigned long start; + unsigned long end; + pgprot_t pgprot; + unsigned int alignment; +}; + +/** + * struct execmem_params - architecture parameters for code allocations + * @ranges: array of ranges defining architecture specific parameters for + * each type of executable memory allocations + */ +struct execmem_params { + struct execmem_range ranges[EXECMEM_TYPE_MAX]; +}; + +/** + * execmem_arch_params - supply parameters for allocations of executable memory + * + * A hook for architectures to define parameters for allocations of + * executable memory described by struct execmem_params + * + * For architectures that do not implement this method a default set of + * parameters will be used + * + * Return: a structure defining architecture parameters and restrictions + * for allocations of executable memory + */ +struct execmem_params *execmem_arch_params(void); + /** * execmem_text_alloc - allocate executable memory * @type: type of the allocation @@ -53,4 +91,10 @@ void *execmem_text_alloc(enum execmem_type type, size_t size); */ void execmem_free(void *ptr); +#ifdef CONFIG_EXECMEM +void execmem_init(void); +#else +static inline void execmem_init(void) {} +#endif + #endif /* _LINUX_EXECMEM_ALLOC_H */ diff --git a/mm/execmem.c b/mm/execmem.c index 638dc2b26a81..f25a5e064886 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -5,14 +5,26 @@ #include #include -static void *execmem_alloc(size_t size) +static struct execmem_params execmem_params; + +static void *execmem_alloc(size_t size, struct execmem_range *range) { - return module_alloc(size); + unsigned long start = range->start; + unsigned long end = range->end; + unsigned int align = range->alignment; + pgprot_t pgprot = range->pgprot; + + return __vmalloc_node_range(size, align, start, end, + GFP_KERNEL, pgprot, VM_FLUSH_RESET_PERMS, + NUMA_NO_NODE, __builtin_return_address(0)); } void *execmem_text_alloc(enum execmem_type type, size_t size) { - return execmem_alloc(size); + if (!execmem_params.ranges[type].start) + return module_alloc(size); + + return execmem_alloc(size, &execmem_params.ranges[type]); } void execmem_free(void *ptr) @@ -24,3 +36,51 @@ void execmem_free(void *ptr) WARN_ON(in_interrupt()); vfree(ptr); } + +struct execmem_params * __weak execmem_arch_params(void) +{ + return NULL; +} + +static bool execmem_validate_params(struct execmem_params *p) +{ + struct execmem_range *r = &p->ranges[EXECMEM_DEFAULT]; + + if (!r->alignment || !r->start || !r->end || !pgprot_val(r->pgprot)) { + pr_crit("Invalid parameters for execmem allocator, module loading will fail"); + return false; + } + + return true; +} + +static void execmem_init_missing(struct execmem_params *p) +{ + struct execmem_range *default_range = &p->ranges[EXECMEM_DEFAULT]; + + for (int i = EXECMEM_DEFAULT + 1; i < EXECMEM_TYPE_MAX; i++) { + struct execmem_range *r = &p->ranges[i]; + + if (!r->start) { + r->pgprot = default_range->pgprot; + r->alignment = default_range->alignment; + r->start = default_range->start; + r->end = default_range->end; + } + } +} + +void __init execmem_init(void) +{ + struct execmem_params *p = execmem_arch_params(); + + if (!p) + return; + + if (!execmem_validate_params(p)) + return; + + execmem_init_missing(p); + + execmem_params = *p; +} diff --git a/mm/mm_init.c b/mm/mm_init.c index 50f2f34745af..7c002b36da21 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -2797,4 +2798,5 @@ void __init mm_core_init(void) pti_init(); kmsan_init_runtime(); mm_cache_init(); + execmem_init(); } From patchwork Mon Sep 18 07:29:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141269 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2486453vqi; Mon, 18 Sep 2023 00:34:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IErOA5LLpIhR8UNEgtioJ20gXCeW/CQAcvtP13NS6kE1oP4XPDBqowzmBkVghwssz87hrTi X-Received: by 2002:a9d:4e90:0:b0:6be:e1cd:7326 with SMTP id v16-20020a9d4e90000000b006bee1cd7326mr10112357otk.4.1695022441533; Mon, 18 Sep 2023 00:34:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695022441; cv=none; d=google.com; s=arc-20160816; b=R7SpDf2bwlrwKxEjYBtmlBg7bkBUbkfg+FJyumMXmkjtyzhGiKbd1A+1JdkrIhaxfW If4e7jCKuWo9JUHCFw1Y5+ohIfaqW65t8c6Rss6KeLXq5k8r7DgTk3TB/FpXT+9cmZZt 7CJ5iQqQi4dw6k8xEQxKOpfo9YntRvDypu2VGwx3OK/75XBMVSDsVOT/r86/MyfRsFfK FDQTxgD8WvxRgSZYCRWo0tk413OWGjX5ptmjgTTQcLgUKHoCcjpwkVLQSRQcZGih8hNl JEjotWrbjPKiDkxLWjmi02zvNSTJcwnSrBJ84GespOqyptiRHbCKs+XWvpdcKjiHt1jh U/cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pR/DurLeWvhG1FKfSYFkfT2vHvRwZ8tEPH/eAXER1RU=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=Xuq8grsQkcV7vUVgAj2BP+kb3uxXnr2i28+C7+vJ2JK2e4oZlhPVZo+ZJw+pkQzWil jo3skokAHayp4QIfhQWaWa+Uc3NSRb9EjKG/vY+X8APAvR2daLZwIJZ4Pqpx7Wy2qnDn fnvyLcW6DIgMhEozbhcHut4LBStYtIZT60CtZRyaSWjqQhr9SiY8LrQlY6iXOSO4oq81 mKqvS6UMehTZvbdX3oURfn5tx1WUX0BX5pAVnfRYxViS0dXbWx3IEm/LTpo6q0WPrd4z lE56f+IQZb6DwSEaWv+S4o2qugO4sKiUJoJK9DaFuktelTscAqX9onPk1vXRVoBLwoYB eoaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=B3MpqZd8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id t17-20020a63d251000000b0055c81ab9a9fsi7542922pgi.582.2023.09.18.00.34.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:34:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=B3MpqZd8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id C3F6481F330C; Mon, 18 Sep 2023 00:32:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234872AbjIRHbf (ORCPT + 27 others); Mon, 18 Sep 2023 03:31:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240366AbjIRHbM (ORCPT ); Mon, 18 Sep 2023 03:31:12 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 492AD187; Mon, 18 Sep 2023 00:30:54 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFDC9C433B9; Mon, 18 Sep 2023 07:30:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022253; bh=Ab3JxL0vUtd+Dgg9Evr/24yil9x7cJIlVNJ+mIHZ+Fw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B3MpqZd8kSnshGSIZk32sAvbfxOLIcvRzx/L6e6twO1GyIkCPbyDJ8YFZK7YV3DYj XGhAnNQsgAWy/xSCUlo5IqNl0WBTUqkmBh0vCS1bt48EfgT1+dVwPuJNvN1ivOBOMn gvH0utW4rdMOUN0nD6KFxNUfq8geoRPWIJ6VKmk1aH2z09LsV+3CM1A5GN8hUlJfXJ ASdeswxXtYTc0LaHjCD2b/Jj+5rTLeYIx1gwiLh7ZEjRZ1XJfx7lSWfp5ovjLk7FgH t/b+tjGlJaBG8XvWXgVzie1/pkYA3m8DAD1W+ztdCKD9/UtdKJD1oiqOVjKdWg5YrB ItW+yxqIqOhzA== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 04/13] mm/execmem, arch: convert remaining overrides of module_alloc to execmem Date: Mon, 18 Sep 2023 10:29:46 +0300 Message-Id: <20230918072955.2507221-5-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:32:49 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777359851783421950 X-GMAIL-MSGID: 1777359851783421950 From: "Mike Rapoport (IBM)" Extend execmem parameters to accommodate more complex overrides of module_alloc() by architectures. This includes specification of a fallback range required by arm, arm64 and powerpc and support for allocation of KASAN shadow required by arm64, s390 and x86. The core implementation of execmem_alloc() takes care of suppressing warnings when the initial allocation fails but there is a fallback range defined. Signed-off-by: Mike Rapoport (IBM) --- arch/arm/kernel/module.c | 38 ++++++++++++--------- arch/arm64/kernel/module.c | 57 ++++++++++++++------------------ arch/powerpc/kernel/module.c | 52 ++++++++++++++--------------- arch/s390/kernel/module.c | 52 +++++++++++------------------ arch/x86/kernel/module.c | 64 +++++++++++------------------------- include/linux/execmem.h | 14 ++++++++ mm/execmem.c | 43 ++++++++++++++++++++++-- 7 files changed, 167 insertions(+), 153 deletions(-) diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c index e74d84f58b77..2c7651a2d84c 100644 --- a/arch/arm/kernel/module.c +++ b/arch/arm/kernel/module.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -34,23 +35,28 @@ #endif #ifdef CONFIG_MMU -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .start = MODULES_VADDR, + .end = MODULES_END, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - gfp_t gfp_mask = GFP_KERNEL; - void *p; - - /* Silence the initial allocation */ - if (IS_ENABLED(CONFIG_ARM_MODULE_PLTS)) - gfp_mask |= __GFP_NOWARN; - - p = __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, - gfp_mask, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, - __builtin_return_address(0)); - if (!IS_ENABLED(CONFIG_ARM_MODULE_PLTS) || p) - return p; - return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, - GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, - __builtin_return_address(0)); + struct execmem_range *r = &execmem_params.ranges[EXECMEM_DEFAULT]; + + r->pgprot = PAGE_KERNEL_EXEC; + + if (IS_ENABLED(CONFIG_ARM_MODULE_PLTS)) { + r->fallback_start = VMALLOC_START; + r->fallback_end = VMALLOC_END; + } + + return &execmem_params; } #endif diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index dd851297596e..cd6320de1c54 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -108,46 +109,38 @@ static int __init module_init_limits(void) return 0; } -subsys_initcall(module_init_limits); -void *module_alloc(unsigned long size) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .flags = EXECMEM_KASAN_SHADOW, + .alignment = MODULE_ALIGN, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - void *p = NULL; + struct execmem_range *r = &execmem_params.ranges[EXECMEM_DEFAULT]; - /* - * Where possible, prefer to allocate within direct branch range of the - * kernel such that no PLTs are necessary. - */ - if (module_direct_base) { - p = __vmalloc_node_range(size, MODULE_ALIGN, - module_direct_base, - module_direct_base + SZ_128M, - GFP_KERNEL | __GFP_NOWARN, - PAGE_KERNEL, 0, NUMA_NO_NODE, - __builtin_return_address(0)); - } + module_init_limits(); - if (!p && module_plt_base) { - p = __vmalloc_node_range(size, MODULE_ALIGN, - module_plt_base, - module_plt_base + SZ_2G, - GFP_KERNEL | __GFP_NOWARN, - PAGE_KERNEL, 0, NUMA_NO_NODE, - __builtin_return_address(0)); - } + r->pgprot = PAGE_KERNEL; - if (!p) { - pr_warn_ratelimited("%s: unable to allocate memory\n", - __func__); - } + if (module_direct_base) { + r->start = module_direct_base; + r->end = module_direct_base + SZ_128M; - if (p && (kasan_alloc_module_shadow(p, size, GFP_KERNEL) < 0)) { - vfree(p); - return NULL; + if (module_plt_base) { + r->fallback_start = module_plt_base; + r->fallback_end = module_plt_base + SZ_2G; + } + } else if (module_plt_base) { + r->start = module_plt_base; + r->end = module_plt_base + SZ_2G; } - /* Memory is intended to be executable, reset the pointer tag. */ - return kasan_reset_tag(p); + return &execmem_params; } enum aarch64_reloc_op { diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c index f6d6ae0a1692..f4dd26f693a3 100644 --- a/arch/powerpc/kernel/module.c +++ b/arch/powerpc/kernel/module.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -89,39 +90,38 @@ int module_finalize(const Elf_Ehdr *hdr, return 0; } -static __always_inline void * -__module_alloc(unsigned long size, unsigned long start, unsigned long end, bool nowarn) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { pgprot_t prot = strict_module_rwx_enabled() ? PAGE_KERNEL : PAGE_KERNEL_EXEC; - gfp_t gfp = GFP_KERNEL | (nowarn ? __GFP_NOWARN : 0); - - /* - * Don't do huge page allocations for modules yet until more testing - * is done. STRICT_MODULE_RWX may require extra work to support this - * too. - */ - return __vmalloc_node_range(size, 1, start, end, gfp, prot, - VM_FLUSH_RESET_PERMS, - NUMA_NO_NODE, __builtin_return_address(0)); -} + struct execmem_range *range = &execmem_params.ranges[EXECMEM_DEFAULT]; -void *module_alloc(unsigned long size) -{ #ifdef MODULES_VADDR unsigned long limit = (unsigned long)_etext - SZ_32M; - void *ptr = NULL; - - BUILD_BUG_ON(TASK_SIZE > MODULES_VADDR); /* First try within 32M limit from _etext to avoid branch trampolines */ - if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit) - ptr = __module_alloc(size, limit, MODULES_END, true); - - if (!ptr) - ptr = __module_alloc(size, MODULES_VADDR, MODULES_END, false); - - return ptr; + if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit) { + range->start = limit; + range->end = MODULES_END; + range->fallback_start = MODULES_VADDR; + range->fallback_end = MODULES_END; + } else { + range->start = MODULES_VADDR; + range->end = MODULES_END; + } #else - return __module_alloc(size, VMALLOC_START, VMALLOC_END, false); + range->start = VMALLOC_START; + range->end = VMALLOC_END; #endif + + range->pgprot = prot; + + return &execmem_params; } diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c index db5561d0c233..538d5f24af66 100644 --- a/arch/s390/kernel/module.c +++ b/arch/s390/kernel/module.c @@ -37,41 +37,29 @@ #define PLT_ENTRY_SIZE 22 -static unsigned long get_module_load_offset(void) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .flags = EXECMEM_KASAN_SHADOW, + .alignment = MODULE_ALIGN, + .pgprot = PAGE_KERNEL, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) { - static DEFINE_MUTEX(module_kaslr_mutex); - static unsigned long module_load_offset; - - if (!kaslr_enabled()) - return 0; - /* - * Calculate the module_load_offset the first time this code - * is called. Once calculated it stays the same until reboot. - */ - mutex_lock(&module_kaslr_mutex); - if (!module_load_offset) + unsigned long module_load_offset = 0; + unsigned long start; + + if (kaslr_enabled()) module_load_offset = get_random_u32_inclusive(1, 1024) * PAGE_SIZE; - mutex_unlock(&module_kaslr_mutex); - return module_load_offset; -} -void *module_alloc(unsigned long size) -{ - gfp_t gfp_mask = GFP_KERNEL; - void *p; - - if (PAGE_ALIGN(size) > MODULES_LEN) - return NULL; - p = __vmalloc_node_range(size, MODULE_ALIGN, - MODULES_VADDR + get_module_load_offset(), - MODULES_END, gfp_mask, PAGE_KERNEL, - VM_FLUSH_RESET_PERMS | VM_DEFER_KMEMLEAK, - NUMA_NO_NODE, __builtin_return_address(0)); - if (p && (kasan_alloc_module_shadow(p, size, gfp_mask) < 0)) { - vfree(p); - return NULL; - } - return p; + start = MODULES_VADDR + module_load_offset; + execmem_params.ranges[EXECMEM_DEFAULT].start = start; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + + return &execmem_params; } #ifdef CONFIG_FUNCTION_TRACER diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 5f71a0cf4399..9d37375e2f05 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -36,55 +37,30 @@ do { \ } while (0) #endif -#ifdef CONFIG_RANDOMIZE_BASE -static unsigned long module_load_offset; +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .flags = EXECMEM_KASAN_SHADOW, + .alignment = MODULE_ALIGN, + }, + }, +}; -/* Mutex protects the module_load_offset. */ -static DEFINE_MUTEX(module_kaslr_mutex); - -static unsigned long int get_module_load_offset(void) -{ - if (kaslr_enabled()) { - mutex_lock(&module_kaslr_mutex); - /* - * Calculate the module_load_offset the first time this - * code is called. Once calculated it stays the same until - * reboot. - */ - if (module_load_offset == 0) - module_load_offset = - get_random_u32_inclusive(1, 1024) * PAGE_SIZE; - mutex_unlock(&module_kaslr_mutex); - } - return module_load_offset; -} -#else -static unsigned long int get_module_load_offset(void) -{ - return 0; -} -#endif - -void *module_alloc(unsigned long size) +struct execmem_params __init *execmem_arch_params(void) { - gfp_t gfp_mask = GFP_KERNEL; - void *p; - - if (PAGE_ALIGN(size) > MODULES_LEN) - return NULL; + unsigned long module_load_offset = 0; + unsigned long start; - p = __vmalloc_node_range(size, MODULE_ALIGN, - MODULES_VADDR + get_module_load_offset(), - MODULES_END, gfp_mask, PAGE_KERNEL, - VM_FLUSH_RESET_PERMS | VM_DEFER_KMEMLEAK, - NUMA_NO_NODE, __builtin_return_address(0)); + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && kaslr_enabled()) + module_load_offset = + get_random_u32_inclusive(1, 1024) * PAGE_SIZE; - if (p && (kasan_alloc_module_shadow(p, size, gfp_mask) < 0)) { - vfree(p); - return NULL; - } + start = MODULES_VADDR + module_load_offset; + execmem_params.ranges[EXECMEM_DEFAULT].start = start; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; - return p; + return &execmem_params; } #ifdef CONFIG_X86_32 diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 44e213625053..806ad1a0088d 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -32,19 +32,33 @@ enum execmem_type { EXECMEM_TYPE_MAX, }; +/** + * enum execmem_module_flags - options for executable memory allocations + * @EXECMEM_KASAN_SHADOW: allocate kasan shadow + */ +enum execmem_range_flags { + EXECMEM_KASAN_SHADOW = (1 << 0), +}; + /** * struct execmem_range - definition of a memory range suitable for code and * related data allocations * @start: address space start * @end: address space end (inclusive) + * @fallback_start: start of the range for fallback allocations + * @fallback_end: end of the range for fallback allocations (inclusive) * @pgprot: permissions for memory in this address space * @alignment: alignment required for text allocations + * @flags: options for memory allocations for this range */ struct execmem_range { unsigned long start; unsigned long end; + unsigned long fallback_start; + unsigned long fallback_end; pgprot_t pgprot; unsigned int alignment; + enum execmem_range_flags flags; }; /** diff --git a/mm/execmem.c b/mm/execmem.c index f25a5e064886..a8c2f44d0133 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -11,12 +11,46 @@ static void *execmem_alloc(size_t size, struct execmem_range *range) { unsigned long start = range->start; unsigned long end = range->end; + unsigned long fallback_start = range->fallback_start; + unsigned long fallback_end = range->fallback_end; unsigned int align = range->alignment; pgprot_t pgprot = range->pgprot; + bool kasan = range->flags & EXECMEM_KASAN_SHADOW; + unsigned long vm_flags = VM_FLUSH_RESET_PERMS; + bool fallback = !!fallback_start; + gfp_t gfp_flags = GFP_KERNEL; + void *p; - return __vmalloc_node_range(size, align, start, end, - GFP_KERNEL, pgprot, VM_FLUSH_RESET_PERMS, - NUMA_NO_NODE, __builtin_return_address(0)); + if (PAGE_ALIGN(size) > (end - start)) + return NULL; + + if (kasan) + vm_flags |= VM_DEFER_KMEMLEAK; + + if (fallback) + gfp_flags |= __GFP_NOWARN; + + p = __vmalloc_node_range(size, align, start, end, gfp_flags, + pgprot, vm_flags, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (!p && fallback) { + start = fallback_start; + end = fallback_end; + gfp_flags = GFP_KERNEL; + + p = __vmalloc_node_range(size, align, start, end, gfp_flags, + pgprot, vm_flags, NUMA_NO_NODE, + __builtin_return_address(0)); + } + + if (p && kasan && + (kasan_alloc_module_shadow(p, size, GFP_KERNEL) < 0)) { + vfree(p); + return NULL; + } + + return kasan_reset_tag(p); } void *execmem_text_alloc(enum execmem_type type, size_t size) @@ -66,6 +100,9 @@ static void execmem_init_missing(struct execmem_params *p) r->alignment = default_range->alignment; r->start = default_range->start; r->end = default_range->end; + r->flags = default_range->flags; + r->fallback_start = default_range->fallback_start; + r->fallback_end = default_range->fallback_end; } } } From patchwork Mon Sep 18 07:29:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141276 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2490227vqi; Mon, 18 Sep 2023 00:45:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG3T4sh+dkgGx9+7VtXqiCjkRqJuO9bPdTqxKQIbRkr6lwNXTRg7Fie+OBF0npynmhP4iPZ X-Received: by 2002:a05:6870:fb8e:b0:1bb:99fe:6ad1 with SMTP id kv14-20020a056870fb8e00b001bb99fe6ad1mr10094115oab.6.1695023137236; Mon, 18 Sep 2023 00:45:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695023137; cv=none; d=google.com; s=arc-20160816; b=epTQiZ6gTivhzmPuIfppX+Gs4nDyuPc8d2Mg5yFRUtRfQ4RKozFQC09mMlBwNxKgdO tt4dyERkaNg3c3kUuBAnq6AS4P251wLSNYO66C272ib2K/mJeUefiXUnX14TapuYsjSL dgcYRXZmZLZQoFD/HrypSWSsQswofANxvlA6eWQLBDAqN2eBPi2P/CrvP8bl3himJXGJ SfMruD3UIgJgKxGI/0uE2oireWeDPsV3WjD7J3WJqePCgGPZ8j/wY+wLm6z8FHNUCs2B K7BWAbfXxpiPGzgrk7xiQxS4oabj5lweW78ttwDDvk9cJyRuv+0ES6VyHKrHdv8dRS5f 12Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D67hK1WK/glEPsWwRha4zs/WTd3TgpZNhVD7Yn+lGNM=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=jQ9T11jxuhioKSNpbD6+y2E6K/Xa2oW9fHSSu5rpCZSj7bAUtS9ncjuxu+atW5V2vY HfEcWorQSpYaH0G9M8eMnbMD9eCIq/3E4jxpcmFYPSY76wIMHU77p6AM1lnDkT9awiTJ 28/F6wUhEDIQvSTfC/11pVV+dhgh4kv7Ke52ao8UdKwnJYdpjeNUYJ1nBUMwLELuwzpy uagY2HTROk3/cNd0NeSQbLdn2FqTOB1Md9ODkt9mqPQL3RLMKXnD4LdKkRGUyqkkJkGi 1D1oxB+0G2RIXhUSLlJ1YQSvAIGVtKup7UEFGRMH4yXfH0WFvM6dwUoLPAITJTgHyde5 gomA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ojfgNgsu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id j10-20020a17090a734a00b002680e08a877si9547355pjs.186.2023.09.18.00.45.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:45:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ojfgNgsu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 41C538043AD6; Mon, 18 Sep 2023 00:32:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240323AbjIRHcE (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240215AbjIRHbY (ORCPT ); Mon, 18 Sep 2023 03:31:24 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF01A19A; Mon, 18 Sep 2023 00:31:03 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 68ED7C433C8; Mon, 18 Sep 2023 07:30:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022263; bh=mbiiTE+Ax/Si/BZxcJVEdVXi15nThNmQlUqiyILb0zo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ojfgNgsup07S+iaZErkHdPn6pFXvUxpfd0rqcJ0g/fFjdgDTdp5RCwZChtRwqs+hU 358zdLhiRZXyw3aIFuKwPq3ciyzug59B+euqQg31hDJF2IYAsE+6D264chrRpy0JCv lIJDaEg0ezBhP+IOLDXi95uXUolbh37QjiMgyCp6vToq0/niAQX3TSUJ4+A3chXFuR Fz6pQnaNxbhFzVVMJL+/JI6h0Va38Wr9Y8Zzq0rWzkuGOl0+j0c7QpIR3g6g0WJ4vh Na6tomyrzTQLlYxetqXYeBzvBjSJ3J7TMO8PQeFVdg4iyOutejtYTgrDjMGzC7l/J1 0uZomhqve3rWw== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 05/13] modules, execmem: drop module_alloc Date: Mon, 18 Sep 2023 10:29:47 +0300 Message-Id: <20230918072955.2507221-6-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:32:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360581173824690 X-GMAIL-MSGID: 1777360581173824690 From: "Mike Rapoport (IBM)" Define default parameters for address range for code allocations using the current values in module_alloc() and make execmem_text_alloc() use these defaults when an architecture does not supply its specific parameters. With this, execmem_text_alloc() implements memory allocation in a way compatible with module_alloc() and can be used as a replacement for module_alloc(). Signed-off-by: Mike Rapoport (IBM) --- include/linux/execmem.h | 8 ++++++++ include/linux/moduleloader.h | 12 ------------ kernel/module/main.c | 7 ------- mm/execmem.c | 12 ++++++++---- 4 files changed, 16 insertions(+), 23 deletions(-) diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 806ad1a0088d..519bdfdca595 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -4,6 +4,14 @@ #include +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ + !defined(CONFIG_KASAN_VMALLOC) +#include +#define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT) +#else +#define MODULE_ALIGN PAGE_SIZE +#endif + /** * enum execmem_type - types of executable memory ranges * diff --git a/include/linux/moduleloader.h b/include/linux/moduleloader.h index a23718aa2f4d..8c81f389117d 100644 --- a/include/linux/moduleloader.h +++ b/include/linux/moduleloader.h @@ -25,10 +25,6 @@ int module_frob_arch_sections(Elf_Ehdr *hdr, /* Additional bytes needed by arch in front of individual sections */ unsigned int arch_mod_section_prepend(struct module *mod, unsigned int section); -/* Allocator used for allocating struct module, core sections and init - sections. Returns NULL on failure. */ -void *module_alloc(unsigned long size); - /* Determines if the section name is an init section (that is only used during * module loading). */ @@ -118,12 +114,4 @@ void module_arch_cleanup(struct module *mod); /* Any cleanup before freeing mod->module_init */ void module_arch_freeing_init(struct module *mod); -#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ - !defined(CONFIG_KASAN_VMALLOC) -#include -#define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT) -#else -#define MODULE_ALIGN PAGE_SIZE -#endif - #endif diff --git a/kernel/module/main.c b/kernel/module/main.c index 4ec982cc943c..c4146bfcd0a7 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1601,13 +1601,6 @@ static void free_modinfo(struct module *mod) } } -void * __weak module_alloc(unsigned long size) -{ - return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, - GFP_KERNEL, PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, - NUMA_NO_NODE, __builtin_return_address(0)); -} - bool __weak module_init_section(const char *name) { return strstarts(name, ".init"); diff --git a/mm/execmem.c b/mm/execmem.c index a8c2f44d0133..abcbd07e05ac 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -55,9 +55,6 @@ static void *execmem_alloc(size_t size, struct execmem_range *range) void *execmem_text_alloc(enum execmem_type type, size_t size) { - if (!execmem_params.ranges[type].start) - return module_alloc(size); - return execmem_alloc(size, &execmem_params.ranges[type]); } @@ -111,8 +108,15 @@ void __init execmem_init(void) { struct execmem_params *p = execmem_arch_params(); - if (!p) + if (!p) { + p = &execmem_params; + p->ranges[EXECMEM_DEFAULT].start = VMALLOC_START; + p->ranges[EXECMEM_DEFAULT].end = VMALLOC_END; + p->ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL_EXEC; + p->ranges[EXECMEM_DEFAULT].alignment = 1; + return; + } if (!execmem_validate_params(p)) return; From patchwork Mon Sep 18 07:29:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141277 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2491073vqi; Mon, 18 Sep 2023 00:48:06 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHSSz7m0xke/PSMKkD9mLWxu19UW8Loz0+eH5WUYQwyUmCguTO1HigRosFwP6/324DVuFi1 X-Received: by 2002:a17:903:2345:b0:1b2:5d5b:e871 with SMTP id c5-20020a170903234500b001b25d5be871mr7925556plh.59.1695023286570; Mon, 18 Sep 2023 00:48:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695023286; cv=none; d=google.com; s=arc-20160816; b=BNsQoiecRpIGHnjWmllijs44590UDC3BDLmQvp000xQjYxlzCy9V+Qcy4BBzVeYWdy jOZBUnhRgt2yQoHddXBQU9JfTjH+lQdsORW6iUqeMiDJkQaUtrapfXegn3Tjv9o8RxGe I45jiE5+6qk0J05UJEm/OhXSRDb4dLoxONgHI2oDPI++vzbVy0awBiEk24S7alscbx2o tn7cU6BbTV94vpfGKR3LtafMkqye7NDSm1Mhm5+0u1/EVWfdSGgYja+5P+BL6Vq6sDtu 8rLzB42lbigYNdvKJ/6r42y4jjSYivZvsHH6l2tc0TUfGK5Ri21zFGd6Gi2By5T9UCub I97A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=rYcSVBLgotk26OnoL1AjLxqTJ0/L7cxqyELacD3vUwM=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=mnLsvZQ9L2B4kmofVqINconPlfHdmHjRX11DGSZ7+nKO2vOAYVaaDcxjyf0kvROxeh LIBWAjNJg8MUkRCViJaE2IvU1775t1Dbf4U6b/P5oI8Rp7uryIpVssf8iw53aUPgLQfN q0O+Sc4fvBtnOe5Ndz0arzVlye7zFfizbBfr1x7weyLdlwd5HbwuCvCEpv+maCRdLQ6Y xJ6vs71s51WojW/xgW2Z6v1qRSyy9NmfhhMW10YFLLe5lh80wIkfHKs3aQC+C0v9QTBJ QpYZYh6kA9elPpTlUIK/pvoLZao7fcQRJO1qeN7309NOLYciSy3JE1O386UgZ7Ewil91 OJDQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=L26X5Mym; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id q5-20020a170902a3c500b001b9e2ce5723si7771110plb.495.2023.09.18.00.48.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:48:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=L26X5Mym; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 910C980C9A62; Mon, 18 Sep 2023 00:32:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240408AbjIRHcR (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240282AbjIRHbh (ORCPT ); Mon, 18 Sep 2023 03:31:37 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B078F114; Mon, 18 Sep 2023 00:31:13 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2285FC433AD; Mon, 18 Sep 2023 07:31:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022273; bh=GrzrJESS7vx7QyV0O3w+oX12p1W5MbqUcsXcqyR+rMA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L26X5Mym5jbDCDCi9pBttJiCUmGDJK7QCrrwON0wsqLsaMRtQtziMfk+8vmq102wK apB9eMngz4FDoZOnyfOnpxsYAYV+s6z6uy/A7GUFANPsDrtCidzVD8sw9PZ9EeZeSS BBZbnyT3UkLAeKFTSJ+WQXJ44CEr3Yz7/npPDOeCiIyuKPQdJ6wFULwDSI42C+21i2 TtK4M26N760ml2Bd8ec+c+dtMjxHoi8aQYpVD0iQ48GL/MfHLj1xfBm+sXCRSzVgn2 IlOo26svfqfJVEKxvnTHvne8XyU+XB4i1UAyBnP2cTSXuXl9ONzyCPVFbHoWx7LKOg fqTz/6KsDW2hQ== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 06/13] mm/execmem: introduce execmem_data_alloc() Date: Mon, 18 Sep 2023 10:29:48 +0300 Message-Id: <20230918072955.2507221-7-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:32:41 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360737499640433 X-GMAIL-MSGID: 1777360737499640433 From: "Mike Rapoport (IBM)" Data related to code allocations, such as module data section, need to comply with architecture constraints for its placement and its allocation right now was done using execmem_text_alloc(). Create a dedicated API for allocating data related to code allocations and allow architectures to define address ranges for data allocations. Since currently this is only relevant for powerpc variants that use the VMALLOC address space for module data allocations, automatically reuse address ranges defined for text unless address range for data is explicitly defined by an architecture. With separation of code and data allocations, data sections of the modules are now mapped as PAGE_KERNEL rather than PAGE_KERNEL_EXEC which was a default on many architectures. Signed-off-by: Mike Rapoport (IBM) --- arch/powerpc/kernel/module.c | 12 ++++++++++++ include/linux/execmem.h | 19 +++++++++++++++++++ kernel/module/main.c | 15 +++------------ mm/execmem.c | 17 ++++++++++++++++- 4 files changed, 50 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c index f4dd26f693a3..824d9541a310 100644 --- a/arch/powerpc/kernel/module.c +++ b/arch/powerpc/kernel/module.c @@ -95,6 +95,9 @@ static struct execmem_params execmem_params __ro_after_init = { [EXECMEM_DEFAULT] = { .alignment = 1, }, + [EXECMEM_MODULE_DATA] = { + .alignment = 1, + }, }, }; @@ -103,7 +106,12 @@ struct execmem_params __init *execmem_arch_params(void) pgprot_t prot = strict_module_rwx_enabled() ? PAGE_KERNEL : PAGE_KERNEL_EXEC; struct execmem_range *range = &execmem_params.ranges[EXECMEM_DEFAULT]; + /* + * BOOK3S_32 and 8xx define MODULES_VADDR for text allocations and + * allow allocating data in the entire vmalloc space + */ #ifdef MODULES_VADDR + struct execmem_range *data = &execmem_params.ranges[EXECMEM_MODULE_DATA]; unsigned long limit = (unsigned long)_etext - SZ_32M; /* First try within 32M limit from _etext to avoid branch trampolines */ @@ -116,6 +124,10 @@ struct execmem_params __init *execmem_arch_params(void) range->start = MODULES_VADDR; range->end = MODULES_END; } + data->start = VMALLOC_START; + data->end = VMALLOC_END; + data->pgprot = PAGE_KERNEL; + data->alignment = 1; #else range->start = VMALLOC_START; range->end = VMALLOC_END; diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 519bdfdca595..09d45ac786e9 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -29,6 +29,7 @@ * @EXECMEM_KPROBES: parameters for kprobes * @EXECMEM_FTRACE: parameters for ftrace * @EXECMEM_BPF: parameters for BPF + * @EXECMEM_MODULE_DATA: parameters for module data sections * @EXECMEM_TYPE_MAX: */ enum execmem_type { @@ -37,6 +38,7 @@ enum execmem_type { EXECMEM_KPROBES, EXECMEM_FTRACE, EXECMEM_BPF, + EXECMEM_MODULE_DATA, EXECMEM_TYPE_MAX, }; @@ -107,6 +109,23 @@ struct execmem_params *execmem_arch_params(void); */ void *execmem_text_alloc(enum execmem_type type, size_t size); +/** + * execmem_data_alloc - allocate memory for data coupled to code + * @type: type of the allocation + * @size: how many bytes of memory are required + * + * Allocates memory that will contain data coupled with executable code, + * like data sections in kernel modules. + * + * The memory will have protections defined by architecture. + * + * The allocated memory will reside in an area that does not impose + * restrictions on the addressing modes. + * + * Return: a pointer to the allocated memory or %NULL + */ +void *execmem_data_alloc(enum execmem_type type, size_t size); + /** * execmem_free - free executable memory * @ptr: pointer to the memory that should be freed diff --git a/kernel/module/main.c b/kernel/module/main.c index c4146bfcd0a7..2ae83a6abf66 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1188,25 +1188,16 @@ void __weak module_arch_freeing_init(struct module *mod) { } -static bool mod_mem_use_vmalloc(enum mod_mem_type type) -{ - return IS_ENABLED(CONFIG_ARCH_WANTS_MODULES_DATA_IN_VMALLOC) && - mod_mem_type_is_core_data(type); -} - static void *module_memory_alloc(unsigned int size, enum mod_mem_type type) { - if (mod_mem_use_vmalloc(type)) - return vzalloc(size); + if (mod_mem_type_is_data(type)) + return execmem_data_alloc(EXECMEM_MODULE_DATA, size); return execmem_text_alloc(EXECMEM_MODULE_TEXT, size); } static void module_memory_free(void *ptr, enum mod_mem_type type) { - if (mod_mem_use_vmalloc(type)) - vfree(ptr); - else - execmem_free(ptr); + execmem_free(ptr); } static void free_mod_mem(struct module *mod) diff --git a/mm/execmem.c b/mm/execmem.c index abcbd07e05ac..aeff85261360 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -53,11 +53,23 @@ static void *execmem_alloc(size_t size, struct execmem_range *range) return kasan_reset_tag(p); } +static inline bool execmem_range_is_data(enum execmem_type type) +{ + return type == EXECMEM_MODULE_DATA; +} + void *execmem_text_alloc(enum execmem_type type, size_t size) { return execmem_alloc(size, &execmem_params.ranges[type]); } +void *execmem_data_alloc(enum execmem_type type, size_t size) +{ + WARN_ON_ONCE(!execmem_range_is_data(type)); + + return execmem_alloc(size, &execmem_params.ranges[type]); +} + void execmem_free(void *ptr) { /* @@ -93,7 +105,10 @@ static void execmem_init_missing(struct execmem_params *p) struct execmem_range *r = &p->ranges[i]; if (!r->start) { - r->pgprot = default_range->pgprot; + if (execmem_range_is_data(i)) + r->pgprot = PAGE_KERNEL; + else + r->pgprot = default_range->pgprot; r->alignment = default_range->alignment; r->start = default_range->start; r->end = default_range->end; From patchwork Mon Sep 18 07:29:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141268 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2486432vqi; Mon, 18 Sep 2023 00:33:58 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGWBzFEKF623S7z85pRry17uPPFw1P8gy3YpwJ2stuB7TAGa2Y4b8bzl4AJ/qFL6dAkmPUV X-Received: by 2002:a05:6808:48f:b0:3ac:b0ff:ab44 with SMTP id z15-20020a056808048f00b003acb0ffab44mr10166741oid.14.1695022438356; Mon, 18 Sep 2023 00:33:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695022438; cv=none; d=google.com; s=arc-20160816; b=y6J1KIXJzIibFP/39wOLe8pScd4LPhYBn6xaE8lTJJn+7N0HZ2Kv9dHB8/QgKi4dN6 CkF7iAS0dZimtqIAkoF87wxP9+ZDBzLHsn7887YQxlOLqnA3j3fiRsmQlTfIqS+46hJo oQvvWo9Y259oAqvT04zgPd7kFOW7+c5+V7sUA09gRQiIxOiCD2kmq+IBfdcCVXgx698V eU2TZaL1XbnRJnz+qBNrVFCC7ED+5aBhFSbBvv3ODlehjPyyVRc/NVpUPGKG8PWGwEsH vVvUY4LPc/feyL3mjp5+z7tFVOzj9LN5pMDTTZXI933UyieKzP7aN7nhrAU02xTY34bM gnkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Qo37PvQ23r50OzMpHp40Xs5RSpcqAH908YBD31i86Ss=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=pNghaLtgaoC39rNz53bQsyLIW0yOCGEdcnltzXhCv4WYE4uGAuiXuWMDdTzBgmgXD5 U1LZTHSWNw8aV+GYNT4E+KliKNsC4ezu+949u1GIfMFai9e2ni6ccNgsji0x7s/95gBR Lxg+99EfeUtIz0+NpgyIDPtwOMqJ6jLxh0gjdgjE0/ACoDdYRoJwXtHi+UpnBXztoG1l IwyTuAjxEX8e/c1BqAkp9/O+SCU0GtXIC6KijhJOwqAzK8F2kXi9RrInTRlbbb3AUPC9 EiXpm/Q3nTfwq/z9/13OoaRsWPqFbfc1rWwb98BjN4cI7u96+ZF82qPrqCVznU/DTWKe f2wQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=QwbKIwCm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id w8-20020a63f508000000b00565eb0b4f33si7665294pgh.224.2023.09.18.00.33.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:33:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=QwbKIwCm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 1B11081F330D; Mon, 18 Sep 2023 00:32:38 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240352AbjIRHcP (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240322AbjIRHbi (ORCPT ); Mon, 18 Sep 2023 03:31:38 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66DEB138; Mon, 18 Sep 2023 00:31:23 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3F8CC433BF; Mon, 18 Sep 2023 07:31:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022283; bh=uhUpg7LkgDj87OPKWv+2ZQQTqeXwsgqGQfVhMMOlf6I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QwbKIwCmsi/m+QIPzGWyDvn+Tk81CaEzaNlQaj1OrbfY6aH7VjNTOMfyqPjfbO3ZI IrekkL8ga776lr+J3x8Ygyh2wVMU8QELU6aweeJeBpGEbBfrtJxHf+rNjrvDsHVs7+ 6bm1TS2tPNGOkx/h7R28N2r8hennFfDFz6aE6FuiYGkO4GZPqz8JGrUJkS1iup7wLd Isg0JWVRjOAmJTjw/rUuKi838+cqznX9dO82PejgtCEzU2RQuKCDDPVyOOKasyX6kb 8fpNnivohYt0YSrYt8tzLAAtD+1zq8pRlzkl8JhZ7kRPGcY2/LRuqibiB20CPobXX4 xLjJAnEDRG4hQ== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 07/13] arm64, execmem: extend execmem_params for generated code allocations Date: Mon, 18 Sep 2023 10:29:49 +0300 Message-Id: <20230918072955.2507221-8-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:32:38 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777359848165384084 X-GMAIL-MSGID: 1777359848165384084 From: "Mike Rapoport (IBM)" The memory allocations for kprobes and BPF on arm64 can be placed anywhere in vmalloc address space and currently this is implemented with overrides of alloc_insn_page() and bpf_jit_alloc_exec() in arm64. Define EXECMEM_KPROBES and EXECMEM_BPF ranges in arm64::execmem_params and drop overrides of alloc_insn_page() and bpf_jit_alloc_exec(). Signed-off-by: Mike Rapoport (IBM) Acked-by: Will Deacon --- arch/arm64/kernel/module.c | 13 +++++++++++++ arch/arm64/kernel/probes/kprobes.c | 7 ------- arch/arm64/net/bpf_jit_comp.c | 11 ----------- 3 files changed, 13 insertions(+), 18 deletions(-) diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index cd6320de1c54..d27db168d2a2 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -116,6 +116,16 @@ static struct execmem_params execmem_params __ro_after_init = { .flags = EXECMEM_KASAN_SHADOW, .alignment = MODULE_ALIGN, }, + [EXECMEM_KPROBES] = { + .start = VMALLOC_START, + .end = VMALLOC_END, + .alignment = 1, + }, + [EXECMEM_BPF] = { + .start = VMALLOC_START, + .end = VMALLOC_END, + .alignment = 1, + }, }, }; @@ -140,6 +150,9 @@ struct execmem_params __init *execmem_arch_params(void) r->end = module_plt_base + SZ_2G; } + execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_ROX; + execmem_params.ranges[EXECMEM_BPF].pgprot = PAGE_KERNEL; + return &execmem_params; } diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index 70b91a8c6bb3..6fccedd02b2a 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -129,13 +129,6 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p) return 0; } -void *alloc_insn_page(void) -{ - return __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START, VMALLOC_END, - GFP_KERNEL, PAGE_KERNEL_ROX, VM_FLUSH_RESET_PERMS, - NUMA_NO_NODE, __builtin_return_address(0)); -} - /* arm kprobe: install breakpoint in text */ void __kprobes arch_arm_kprobe(struct kprobe *p) { diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 150d1c6543f7..3a7590f828d1 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -1687,17 +1687,6 @@ u64 bpf_jit_alloc_exec_limit(void) return VMALLOC_END - VMALLOC_START; } -void *bpf_jit_alloc_exec(unsigned long size) -{ - /* Memory is intended to be executable, reset the pointer tag. */ - return kasan_reset_tag(vmalloc(size)); -} - -void bpf_jit_free_exec(void *addr) -{ - return vfree(addr); -} - /* Indicate the JIT backend supports mixing bpf2bpf and tailcalls. */ bool bpf_jit_supports_subprog_tailcalls(void) { From patchwork Mon Sep 18 07:29:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141273 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2488563vqi; Mon, 18 Sep 2023 00:40:32 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFs3n9hnzBouuzLJOr468vpj2F1XyAEINc8GIcSCo1fi20YL6bhcSD4hV7z1xDu7NJDPOSp X-Received: by 2002:a05:6830:1d51:b0:6c0:f0e6:a325 with SMTP id p17-20020a0568301d5100b006c0f0e6a325mr8430995oth.16.1695022832051; Mon, 18 Sep 2023 00:40:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695022832; cv=none; d=google.com; s=arc-20160816; b=fOIsfYza8uU3tbFvGwCSPeO1T8K9TQ/eEP6RXOxtmIjqXyJXVf2nNkzbLxGaLdUNrs JFurWhjmJzpWoPu94neHE5dHEs9jMbTyQ21kJb5gdoW8Okc32SSVnUHDEyWlRYZtpiK/ xSkyyGygjNsPgDRnY8kl3ciWHzG4BApaUJR2iIUaWJdzBfDz8hqup/h0j4mPaAiVzFKx jqfJ8VIrxSTx4xkLmbTi/Ou8bctA/UY+0svT8ZWKnAvHdpx9ryQw7wZkXqdpbDa9JSBN 9Tv9Q99++SJZYkfrfVlBzZ86H7fTqBR/VrMpRur16gtsBScgb/qieKwaPRLoJauRGo1g zBgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=r7WH6k3cQ1gcDKVDqjiXKIrMyc70EktDzeRRqm44eao=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=i2SX012BPI1yFOd3QTaaaSmc5tZjWnSpjKLRvhzOY2PnsnpHzpWXZ4diJmPOoCiQeF 7j+JiL2+3jxfPQHSrW+dahgus6AmCeShdZJDOHO6tAvwYNdg4BVPAmQ31U+w71NBVd85 sFf81o/eDu3AYVT3SAJIJSTcXCstIA7pzILzxo39IVlwXc4FjXQ+edjMCzic3Ck46GoJ TKMLl7qKJwomIaZ7XaxvZZ/f8zUE5+premxY+3OictiMqH8qnWT29PSgsJcwZNgutqm9 JogrVqu5u/36Xp7vNtTtCf1qOTqQAjkif78jWZUaB+8WAtE9VT5ZXfFqHdhA9Zbg2y7x PaJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sUl4tcVd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id a35-20020a631a63000000b00569374c7c71si7345640pgm.857.2023.09.18.00.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:40:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=sUl4tcVd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 1E0E0811ADBF; Mon, 18 Sep 2023 00:32:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240427AbjIRHcV (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240336AbjIRHbp (ORCPT ); Mon, 18 Sep 2023 03:31:45 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41F92122; Mon, 18 Sep 2023 00:31:33 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90D66C4160E; Mon, 18 Sep 2023 07:31:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022292; bh=Bxy7NNRI6l4akC8Uk0JGSyZsyMn1J934EmvkFA96qyo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sUl4tcVd+CE1RIfDd+iHzBuo/TtqH+sjUX4AKoidjsBhiPE9N8u5sBFCrh0U2OnnW 02VNXIu/PasFZvJmqp4LQS/fRSyOWf9W0w4R1mThqbVlQHxfh9deE8Gen6PrybSKrA HQTfOkMmKWxzdY89y0l1tlT8/WpmYk4oOHqiKiyXMFacErx4oC2oV4nkojcJ6bpZun 9S9TLaBh1Kth+hdxlxcmXjxDVU8y/jq3PgPfSmOZxNeFoZYJdyBqlLfSCGOoTzYYQr s02+6pJi35T4N9EWcJQvWIbsw2teHNSr/sLV1vIIuBQWC8WuxxKTSKFPi24i20hJWy PMgDbwaP/8Fzw== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 08/13] riscv: extend execmem_params for generated code allocations Date: Mon, 18 Sep 2023 10:29:50 +0300 Message-Id: <20230918072955.2507221-9-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:32:36 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360260757018011 X-GMAIL-MSGID: 1777360260757018011 From: "Mike Rapoport (IBM)" The memory allocations for kprobes and BPF on RISC-V are not placed in the modules area and these custom allocations are implemented with overrides of alloc_insn_page() and bpf_jit_alloc_exec(). Slightly reorder execmem_params initialization to support both 32 and 64 bit variants, define EXECMEM_KPROBES and EXECMEM_BPF ranges in riscv::execmem_params and drop overrides of alloc_insn_page() and bpf_jit_alloc_exec(). Signed-off-by: Mike Rapoport (IBM) Reviewed-by: Alexandre Ghiti --- arch/riscv/kernel/module.c | 21 ++++++++++++++++++++- arch/riscv/kernel/probes/kprobes.c | 10 ---------- arch/riscv/net/bpf_jit_core.c | 13 ------------- 3 files changed, 20 insertions(+), 24 deletions(-) diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c index 343a0edfb6dd..31505ecb5c72 100644 --- a/arch/riscv/kernel/module.c +++ b/arch/riscv/kernel/module.c @@ -436,20 +436,39 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, return 0; } -#if defined(CONFIG_MMU) && defined(CONFIG_64BIT) +#ifdef CONFIG_MMU static struct execmem_params execmem_params __ro_after_init = { .ranges = { [EXECMEM_DEFAULT] = { .pgprot = PAGE_KERNEL, .alignment = 1, }, + [EXECMEM_KPROBES] = { + .pgprot = PAGE_KERNEL_READ_EXEC, + .alignment = 1, + }, + [EXECMEM_BPF] = { + .pgprot = PAGE_KERNEL, + .alignment = 1, + }, }, }; struct execmem_params __init *execmem_arch_params(void) { +#ifdef CONFIG_64BIT execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; +#else + execmem_params.ranges[EXECMEM_DEFAULT].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_DEFAULT].end = VMALLOC_END; +#endif + + execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_KPROBES].end = VMALLOC_END; + + execmem_params.ranges[EXECMEM_BPF].start = BPF_JIT_REGION_START; + execmem_params.ranges[EXECMEM_BPF].end = BPF_JIT_REGION_END; return &execmem_params; } diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index 2f08c14a933d..e64f2f3064eb 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -104,16 +104,6 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p) return 0; } -#ifdef CONFIG_MMU -void *alloc_insn_page(void) -{ - return __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START, VMALLOC_END, - GFP_KERNEL, PAGE_KERNEL_READ_EXEC, - VM_FLUSH_RESET_PERMS, NUMA_NO_NODE, - __builtin_return_address(0)); -} -#endif - /* install breakpoint in text */ void __kprobes arch_arm_kprobe(struct kprobe *p) { diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c index 7b70ccb7fec3..c8a758f0882b 100644 --- a/arch/riscv/net/bpf_jit_core.c +++ b/arch/riscv/net/bpf_jit_core.c @@ -218,19 +218,6 @@ u64 bpf_jit_alloc_exec_limit(void) return BPF_JIT_REGION_SIZE; } -void *bpf_jit_alloc_exec(unsigned long size) -{ - return __vmalloc_node_range(size, PAGE_SIZE, BPF_JIT_REGION_START, - BPF_JIT_REGION_END, GFP_KERNEL, - PAGE_KERNEL, 0, NUMA_NO_NODE, - __builtin_return_address(0)); -} - -void bpf_jit_free_exec(void *addr) -{ - return vfree(addr); -} - void *bpf_arch_text_copy(void *dst, void *src, size_t len) { int ret; From patchwork Mon Sep 18 07:29:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141278 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2491347vqi; Mon, 18 Sep 2023 00:49:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH/TkJT+mjHW3N2Z/YohXclPwnQftyot3YCN4TVGaS/JOBzQWrB3Hi285CYVj7ViS4A3PQ/ X-Received: by 2002:a05:6a20:9699:b0:148:4c5:9714 with SMTP id hp25-20020a056a20969900b0014804c59714mr7052975pzc.13.1695023348864; Mon, 18 Sep 2023 00:49:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695023348; cv=none; d=google.com; s=arc-20160816; b=PrmsukZozurKlu1H9fvxQJm+azagBSg9dSyJBgy3LwjwpPkuP7BT4JeN0dKXkFVYiT lUScB/s/pXqVBCW1Jy4YwnJKBr4CgH0nFgFFMV4BEdqg4v8X5VCquk1j/skRUyBrP9Rz 1Io7ZiKxyATkCn+P3kwwsNX2o/jNBGm6FhH8BH4PSnNbU6Tf78+HJvguw+6xY+G2CBgw pSnSI8SorJRGV8Aqdh/hVDGmmIeIR/AjNcHSgPs0mo95sgdKD8V639xkrbnHj7J11ray 5oyufY+btehf58/50B9U6BG85OMTtn1jWfMl5zZjjBiZgOE4ViPrXRraeVwCZvYuSOaC 3mww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EJSx7dTPSl55UPeMrpomdQNp6HYHj5K4Qaz1intYvHs=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=cgimPfSVoQ8qshahlN4W0muIOlUIBi8C/xWb2v67HxI8EBMlUbnavP/60Zq+LiE3Xn KveO/bwqT/fG3vD+5oDmpywg3tNrzEB7jqwl4Fx5WmQYVwFQb0jYI8rs/xnTzeffBo7G FINaOEfZ8JGJS6tmVWZdLKWkVtQx2V0szCA2PZDsNN7apgHfKVgQjd6FZjFeBPE/mdGq XTBpqIvJW/Vh7qtLdiilpJMabmupd3M7KSdew+jyD5tTDb+RXPpbVvynawc2upC1TL29 O1tYx6OhTDicfMQrxsKEY9NlBrN6hgjCHIoxnO7sJFHGPn+vLeyNVLrcsHbFHwXetvw+ 6y+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=d2NNM60P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id u17-20020a17090341d100b001b8af930e3dsi8075202ple.487.2023.09.18.00.49.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:49:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=d2NNM60P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id C008A80BC3ED; Mon, 18 Sep 2023 00:33:16 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238569AbjIRHc3 (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240385AbjIRHb4 (ORCPT ); Mon, 18 Sep 2023 03:31:56 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03D8118C; Mon, 18 Sep 2023 00:31:42 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5155AC433B7; Mon, 18 Sep 2023 07:31:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022302; bh=35ZlmvV7rKqwzQaOmbBijH380/EeVtPirZbGWiToTW8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d2NNM60PB58jmuhS2dGN8NLbq1KB4ygYvRR0t6DZZJS7qBVZpROHao20O2uQV3bli 0QvXp0s5+eLFeJhc/w10Fn3KC9lksq3uhoYzEZC5CWp+6gjai+j/d0a89/bBUhyjJy FfXXBFOSn88hPGWgJbSxUtSr8ULAUympoyAWJKP5uJmt4lykdt9r5kJb2u9np9CG9l pAU2vChu/4/QGlbV5wO6V/EhzFjCpLIlsY1oLhcdgzQ+bitYphtILzsTup5ZLrX357 OmssgGZ5JmBAamSicGtU177ClYYdO2+YHkslBuFOroSFhEYDOcBWDp8NAr0VRv3fiX mkBw/0g11HpDQ== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 09/13] powerpc: extend execmem_params for kprobes allocations Date: Mon, 18 Sep 2023 10:29:51 +0300 Message-Id: <20230918072955.2507221-10-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:33:16 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360803448765803 X-GMAIL-MSGID: 1777360803448765803 From: "Mike Rapoport (IBM)" powerpc overrides kprobes::alloc_insn_page() to remove writable permissions when STRICT_MODULE_RWX is on. Add definition of EXECMEM_KRPOBES to execmem_params to allow using the generic kprobes::alloc_insn_page() with the desired permissions. As powerpc uses breakpoint instructions to inject kprobes, it does not need to constrain kprobe allocations to the modules area and can use the entire vmalloc address space. Signed-off-by: Mike Rapoport (IBM) --- arch/powerpc/kernel/kprobes.c | 14 -------------- arch/powerpc/kernel/module.c | 11 +++++++++++ 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c index 62228c7072a2..14c5ddec3056 100644 --- a/arch/powerpc/kernel/kprobes.c +++ b/arch/powerpc/kernel/kprobes.c @@ -126,20 +126,6 @@ kprobe_opcode_t *arch_adjust_kprobe_addr(unsigned long addr, unsigned long offse return (kprobe_opcode_t *)(addr + offset); } -void *alloc_insn_page(void) -{ - void *page; - - page = execmem_text_alloc(EXECMEM_KPROBES, PAGE_SIZE); - if (!page) - return NULL; - - if (strict_module_rwx_enabled()) - set_memory_rox((unsigned long)page, 1); - - return page; -} - int arch_prepare_kprobe(struct kprobe *p) { int ret = 0; diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c index 824d9541a310..bf2c62aef628 100644 --- a/arch/powerpc/kernel/module.c +++ b/arch/powerpc/kernel/module.c @@ -95,6 +95,9 @@ static struct execmem_params execmem_params __ro_after_init = { [EXECMEM_DEFAULT] = { .alignment = 1, }, + [EXECMEM_KPROBES] = { + .alignment = 1, + }, [EXECMEM_MODULE_DATA] = { .alignment = 1, }, @@ -135,5 +138,13 @@ struct execmem_params __init *execmem_arch_params(void) range->pgprot = prot; + execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_END; + + if (strict_module_rwx_enabled()) + execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_ROX; + else + execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_EXEC; + return &execmem_params; } From patchwork Mon Sep 18 07:29:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141271 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2487900vqi; Mon, 18 Sep 2023 00:38:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH+oh/Q8c2a3a6fGz9gGJS5p+CCRy9gQQg/wLCgXYKkP826Bs2IP6Ay/PRhegkyDovVeo0x X-Received: by 2002:a17:902:c1cc:b0:1bb:25bd:d09c with SMTP id c12-20020a170902c1cc00b001bb25bdd09cmr6420951plc.1.1695022711662; Mon, 18 Sep 2023 00:38:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695022711; cv=none; d=google.com; s=arc-20160816; b=Tw1BPbd/7ttanrmpG6ussxgTNsEyWP9BPfTDIisA/4xhRIvLdNu0vFTvSk9KIWr6HY uD30Crh52WG/O36j5M2pRn/SYdYFRxX0n3clOQpGny82hzCHvRMeNICMPWLQ1BmCV6Mk qb+mWtr8U1nposkH+EywERpbJX/OytQ2rYndEiBS1DeY356MmrBCYJ5ref8jgyOxOjXT 8DEY4ReNAhrOk5ZvRnnLmiPoLnx2CwnXdZpTEvxvTJwkBvU8XgR3VgEFG1P2N4w2Z332 pA49mWRu0/XmT8SiG/i3v0C7g0YlYi3guHId0qodX12NhTL7P62kr5NhswKSJj2TXH9M v06w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6g26NJg24TXt6CFg+UEJLVlOMObq4F61xl8/MmH9ZEU=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=FASzIYI/WUkBCWG9YrNyOcsFUlZNEvlL/RbHropntDgLGFMZG+jlw1qKS7q+JhvslD V5IY5UirzMypVcZNY8W/RNGiWCBNvrYyNq7laRZAz3WLMKmYselW8T2GLY52HFbhog7y XwOdy4kw6ajhQ05Vusl2YbZiX+Mzp8qUhb+6/F9KOujrfWSVdqgKGn/881Vshr75kjoq oObNuPzvN0+3OBn6SiDCioXmMR3YjLtDVMuVtMHnY7zjD/vdle7Nl2GVocT/vPn3hChB qRS/KU2nx/5Da6NV4Ddfqo3IZN2nXNMlfD/r+GCF7wqjgxyw3dztN7cr6V3kEnkZw21X JSWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=BpYN3pG2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id f18-20020a170902ce9200b001bbd0358ef7si7934195plg.518.2023.09.18.00.38.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:38:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=BpYN3pG2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 145668191659; Mon, 18 Sep 2023 00:33:14 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240342AbjIRHcf (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239892AbjIRHcU (ORCPT ); Mon, 18 Sep 2023 03:32:20 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2DDD102; Mon, 18 Sep 2023 00:31:52 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0D35BC433CD; Mon, 18 Sep 2023 07:31:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022312; bh=/lkcmGgfZ0h40rnIgrlGtkFY3fqKKa7LS9K2y40s078=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BpYN3pG2mJV/rqYXUts/nj29qV9EPMjt0zx/SMPu/4jisp6o2O8xaHc0tuU0dozmC EkX/reooL7FObsZ0D9hJ8qVr3s0jxZwQDniBogZP502PMqTzmb9zf8IC8v48ADq9JJ vhZzaS3bIiYR4KpLC0gQ9XGiX8Sg8U3ZgIlfxPbUxlyBunRPZGtKxFV4rwutmG1U2Q xNiJqoyc2QgzMbsZ7fvm26WevVyonKdQpLbr248krCbEnUGRO38cNr6hKTgmskgkoC M+BAIpPXGnu21jR6jiTi0lpov13TwKfc4fKKAJMozA7dQLpxyNDE6MCIa6xWY0rBPk BD29RoWukxl2Q== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 10/13] arch: make execmem setup available regardless of CONFIG_MODULES Date: Mon, 18 Sep 2023 10:29:52 +0300 Message-Id: <20230918072955.2507221-11-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:33:14 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360134866189388 X-GMAIL-MSGID: 1777360134866189388 From: "Mike Rapoport (IBM)" execmem does not depend on modules, on the contrary modules use execmem. To make execmem available when CONFIG_MODULES=n, for instance for kprobes, split execmem_params initialization out from arch/kernel/module.c and compile it when CONFIG_EXECMEM=y Signed-off-by: Mike Rapoport (IBM) --- arch/arm/kernel/module.c | 38 ---------- arch/arm/mm/init.c | 38 ++++++++++ arch/arm64/kernel/module.c | 130 -------------------------------- arch/arm64/mm/init.c | 132 +++++++++++++++++++++++++++++++++ arch/loongarch/kernel/module.c | 18 ----- arch/loongarch/mm/init.c | 20 +++++ arch/mips/kernel/module.c | 19 ----- arch/mips/mm/init.c | 20 +++++ arch/parisc/kernel/module.c | 17 ----- arch/parisc/mm/init.c | 22 +++++- arch/powerpc/kernel/module.c | 60 --------------- arch/powerpc/mm/mem.c | 62 ++++++++++++++++ arch/riscv/kernel/module.c | 39 ---------- arch/riscv/mm/init.c | 39 ++++++++++ arch/s390/kernel/module.c | 25 ------- arch/s390/mm/init.c | 28 +++++++ arch/sparc/kernel/module.c | 23 ------ arch/sparc/mm/Makefile | 2 + arch/sparc/mm/execmem.c | 25 +++++++ arch/x86/kernel/module.c | 27 ------- arch/x86/mm/init.c | 29 ++++++++ 21 files changed, 416 insertions(+), 397 deletions(-) create mode 100644 arch/sparc/mm/execmem.c diff --git a/arch/arm/kernel/module.c b/arch/arm/kernel/module.c index 2c7651a2d84c..3282f304f6b1 100644 --- a/arch/arm/kernel/module.c +++ b/arch/arm/kernel/module.c @@ -16,50 +16,12 @@ #include #include #include -#include #include #include #include #include -#ifdef CONFIG_XIP_KERNEL -/* - * The XIP kernel text is mapped in the module area for modules and - * some other stuff to work without any indirect relocations. - * MODULES_VADDR is redefined here and not in asm/memory.h to avoid - * recompiling the whole kernel when CONFIG_XIP_KERNEL is turned on/off. - */ -#undef MODULES_VADDR -#define MODULES_VADDR (((unsigned long)_exiprom + ~PMD_MASK) & PMD_MASK) -#endif - -#ifdef CONFIG_MMU -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .start = MODULES_VADDR, - .end = MODULES_END, - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - struct execmem_range *r = &execmem_params.ranges[EXECMEM_DEFAULT]; - - r->pgprot = PAGE_KERNEL_EXEC; - - if (IS_ENABLED(CONFIG_ARM_MODULE_PLTS)) { - r->fallback_start = VMALLOC_START; - r->fallback_end = VMALLOC_END; - } - - return &execmem_params; -} -#endif - bool module_init_section(const char *name) { return strstarts(name, ".init") || diff --git a/arch/arm/mm/init.c b/arch/arm/mm/init.c index a42e4cd11db2..c0b536e398b4 100644 --- a/arch/arm/mm/init.c +++ b/arch/arm/mm/init.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include @@ -486,3 +487,40 @@ void free_initrd_mem(unsigned long start, unsigned long end) free_reserved_area((void *)start, (void *)end, -1, "initrd"); } #endif + +#ifdef CONFIG_XIP_KERNEL +/* + * The XIP kernel text is mapped in the module area for modules and + * some other stuff to work without any indirect relocations. + * MODULES_VADDR is redefined here and not in asm/memory.h to avoid + * recompiling the whole kernel when CONFIG_XIP_KERNEL is turned on/off. + */ +#undef MODULES_VADDR +#define MODULES_VADDR (((unsigned long)_exiprom + ~PMD_MASK) & PMD_MASK) +#endif + +#if defined(CONFIG_MMU) && defined(CONFIG_EXECMEM) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .start = MODULES_VADDR, + .end = MODULES_END, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + struct execmem_range *r = &execmem_params.ranges[EXECMEM_DEFAULT]; + + r->pgprot = PAGE_KERNEL_EXEC; + + if (IS_ENABLED(CONFIG_ARM_MODULE_PLTS)) { + r->fallback_start = VMALLOC_START; + r->fallback_end = VMALLOC_END; + } + + return &execmem_params; +} +#endif diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index d27db168d2a2..eb1505128b75 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -20,142 +20,12 @@ #include #include #include -#include #include #include #include #include -static u64 module_direct_base __ro_after_init = 0; -static u64 module_plt_base __ro_after_init = 0; - -/* - * Choose a random page-aligned base address for a window of 'size' bytes which - * entirely contains the interval [start, end - 1]. - */ -static u64 __init random_bounding_box(u64 size, u64 start, u64 end) -{ - u64 max_pgoff, pgoff; - - if ((end - start) >= size) - return 0; - - max_pgoff = (size - (end - start)) / PAGE_SIZE; - pgoff = get_random_u32_inclusive(0, max_pgoff); - - return start - pgoff * PAGE_SIZE; -} - -/* - * Modules may directly reference data and text anywhere within the kernel - * image and other modules. References using PREL32 relocations have a +/-2G - * range, and so we need to ensure that the entire kernel image and all modules - * fall within a 2G window such that these are always within range. - * - * Modules may directly branch to functions and code within the kernel text, - * and to functions and code within other modules. These branches will use - * CALL26/JUMP26 relocations with a +/-128M range. Without PLTs, we must ensure - * that the entire kernel text and all module text falls within a 128M window - * such that these are always within range. With PLTs, we can expand this to a - * 2G window. - * - * We chose the 128M region to surround the entire kernel image (rather than - * just the text) as using the same bounds for the 128M and 2G regions ensures - * by construction that we never select a 128M region that is not a subset of - * the 2G region. For very large and unusual kernel configurations this means - * we may fall back to PLTs where they could have been avoided, but this keeps - * the logic significantly simpler. - */ -static int __init module_init_limits(void) -{ - u64 kernel_end = (u64)_end; - u64 kernel_start = (u64)_text; - u64 kernel_size = kernel_end - kernel_start; - - /* - * The default modules region is placed immediately below the kernel - * image, and is large enough to use the full 2G relocation range. - */ - BUILD_BUG_ON(KIMAGE_VADDR != MODULES_END); - BUILD_BUG_ON(MODULES_VSIZE < SZ_2G); - - if (!kaslr_enabled()) { - if (kernel_size < SZ_128M) - module_direct_base = kernel_end - SZ_128M; - if (kernel_size < SZ_2G) - module_plt_base = kernel_end - SZ_2G; - } else { - u64 min = kernel_start; - u64 max = kernel_end; - - if (IS_ENABLED(CONFIG_RANDOMIZE_MODULE_REGION_FULL)) { - pr_info("2G module region forced by RANDOMIZE_MODULE_REGION_FULL\n"); - } else { - module_direct_base = random_bounding_box(SZ_128M, min, max); - if (module_direct_base) { - min = module_direct_base; - max = module_direct_base + SZ_128M; - } - } - - module_plt_base = random_bounding_box(SZ_2G, min, max); - } - - pr_info("%llu pages in range for non-PLT usage", - module_direct_base ? (SZ_128M - kernel_size) / PAGE_SIZE : 0); - pr_info("%llu pages in range for PLT usage", - module_plt_base ? (SZ_2G - kernel_size) / PAGE_SIZE : 0); - - return 0; -} - -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .flags = EXECMEM_KASAN_SHADOW, - .alignment = MODULE_ALIGN, - }, - [EXECMEM_KPROBES] = { - .start = VMALLOC_START, - .end = VMALLOC_END, - .alignment = 1, - }, - [EXECMEM_BPF] = { - .start = VMALLOC_START, - .end = VMALLOC_END, - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - struct execmem_range *r = &execmem_params.ranges[EXECMEM_DEFAULT]; - - module_init_limits(); - - r->pgprot = PAGE_KERNEL; - - if (module_direct_base) { - r->start = module_direct_base; - r->end = module_direct_base + SZ_128M; - - if (module_plt_base) { - r->fallback_start = module_plt_base; - r->fallback_end = module_plt_base + SZ_2G; - } - } else if (module_plt_base) { - r->start = module_plt_base; - r->end = module_plt_base + SZ_2G; - } - - execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_ROX; - execmem_params.ranges[EXECMEM_BPF].pgprot = PAGE_KERNEL; - - return &execmem_params; -} - enum aarch64_reloc_op { RELOC_OP_NONE, RELOC_OP_ABS, diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 8a0f8604348b..9b7716b4d84c 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include @@ -547,3 +548,134 @@ void dump_mem_limit(void) pr_emerg("Memory Limit: none\n"); } } + +#ifdef CONFIG_EXECMEM +static u64 module_direct_base __ro_after_init = 0; +static u64 module_plt_base __ro_after_init = 0; + +/* + * Choose a random page-aligned base address for a window of 'size' bytes which + * entirely contains the interval [start, end - 1]. + */ +static u64 __init random_bounding_box(u64 size, u64 start, u64 end) +{ + u64 max_pgoff, pgoff; + + if ((end - start) >= size) + return 0; + + max_pgoff = (size - (end - start)) / PAGE_SIZE; + pgoff = get_random_u32_inclusive(0, max_pgoff); + + return start - pgoff * PAGE_SIZE; +} + +/* + * Modules may directly reference data and text anywhere within the kernel + * image and other modules. References using PREL32 relocations have a +/-2G + * range, and so we need to ensure that the entire kernel image and all modules + * fall within a 2G window such that these are always within range. + * + * Modules may directly branch to functions and code within the kernel text, + * and to functions and code within other modules. These branches will use + * CALL26/JUMP26 relocations with a +/-128M range. Without PLTs, we must ensure + * that the entire kernel text and all module text falls within a 128M window + * such that these are always within range. With PLTs, we can expand this to a + * 2G window. + * + * We chose the 128M region to surround the entire kernel image (rather than + * just the text) as using the same bounds for the 128M and 2G regions ensures + * by construction that we never select a 128M region that is not a subset of + * the 2G region. For very large and unusual kernel configurations this means + * we may fall back to PLTs where they could have been avoided, but this keeps + * the logic significantly simpler. + */ +static int __init module_init_limits(void) +{ + u64 kernel_end = (u64)_end; + u64 kernel_start = (u64)_text; + u64 kernel_size = kernel_end - kernel_start; + + /* + * The default modules region is placed immediately below the kernel + * image, and is large enough to use the full 2G relocation range. + */ + BUILD_BUG_ON(KIMAGE_VADDR != MODULES_END); + BUILD_BUG_ON(MODULES_VSIZE < SZ_2G); + + if (!kaslr_enabled()) { + if (kernel_size < SZ_128M) + module_direct_base = kernel_end - SZ_128M; + if (kernel_size < SZ_2G) + module_plt_base = kernel_end - SZ_2G; + } else { + u64 min = kernel_start; + u64 max = kernel_end; + + if (IS_ENABLED(CONFIG_RANDOMIZE_MODULE_REGION_FULL)) { + pr_info("2G module region forced by RANDOMIZE_MODULE_REGION_FULL\n"); + } else { + module_direct_base = random_bounding_box(SZ_128M, min, max); + if (module_direct_base) { + min = module_direct_base; + max = module_direct_base + SZ_128M; + } + } + + module_plt_base = random_bounding_box(SZ_2G, min, max); + } + + pr_info("%llu pages in range for non-PLT usage", + module_direct_base ? (SZ_128M - kernel_size) / PAGE_SIZE : 0); + pr_info("%llu pages in range for PLT usage", + module_plt_base ? (SZ_2G - kernel_size) / PAGE_SIZE : 0); + + return 0; +} + +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .flags = EXECMEM_KASAN_SHADOW, + .alignment = MODULE_ALIGN, + }, + [EXECMEM_KPROBES] = { + .start = VMALLOC_START, + .end = VMALLOC_END, + .alignment = 1, + }, + [EXECMEM_BPF] = { + .start = VMALLOC_START, + .end = VMALLOC_END, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + struct execmem_range *r = &execmem_params.ranges[EXECMEM_DEFAULT]; + + module_init_limits(); + + r->pgprot = PAGE_KERNEL; + + if (module_direct_base) { + r->start = module_direct_base; + r->end = module_direct_base + SZ_128M; + + if (module_plt_base) { + r->fallback_start = module_plt_base; + r->fallback_end = module_plt_base + SZ_2G; + } + } else if (module_plt_base) { + r->start = module_plt_base; + r->end = module_plt_base + SZ_2G; + } + + execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_ROX; + execmem_params.ranges[EXECMEM_BPF].pgprot = PAGE_KERNEL; + + return &execmem_params; +} +#endif diff --git a/arch/loongarch/kernel/module.c b/arch/loongarch/kernel/module.c index a1d8fe9796fa..181b5f8b09f1 100644 --- a/arch/loongarch/kernel/module.c +++ b/arch/loongarch/kernel/module.c @@ -18,7 +18,6 @@ #include #include #include -#include #include #include @@ -470,23 +469,6 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, return 0; } -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .pgprot = PAGE_KERNEL, - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; - execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; - - return &execmem_params; -} - static void module_init_ftrace_plt(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *mod) { diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c index f3fe8c06ba4d..26b10a51309c 100644 --- a/arch/loongarch/mm/init.c +++ b/arch/loongarch/mm/init.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -247,3 +248,22 @@ EXPORT_SYMBOL(invalid_pmd_table); #endif pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss; EXPORT_SYMBOL(invalid_pte_table); + +#ifdef CONFIG_EXECMEM +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .pgprot = PAGE_KERNEL, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + + return &execmem_params; +} +#endif diff --git a/arch/mips/kernel/module.c b/arch/mips/kernel/module.c index 1c959074b35f..ebf9496f5db0 100644 --- a/arch/mips/kernel/module.c +++ b/arch/mips/kernel/module.c @@ -33,25 +33,6 @@ struct mips_hi16 { static LIST_HEAD(dbe_list); static DEFINE_SPINLOCK(dbe_lock); -#ifdef MODULE_START -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .start = MODULE_START, - .end = MODULE_END, - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; - - return &execmem_params; -} -#endif - static void apply_r_mips_32(u32 *location, u32 base, Elf_Addr v) { *location = base + v; diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c index 5dcb525a8995..55e7869d03f2 100644 --- a/arch/mips/mm/init.c +++ b/arch/mips/mm/init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include @@ -573,3 +574,22 @@ EXPORT_SYMBOL_GPL(invalid_pmd_table); #endif pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss; EXPORT_SYMBOL(invalid_pte_table); + +#if defined(CONFIG_EXECMEM) && defined(MODULE_START) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .start = MODULE_START, + .end = MODULE_END, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; + + return &execmem_params; +} +#endif diff --git a/arch/parisc/kernel/module.c b/arch/parisc/kernel/module.c index 0c6dfd1daef3..fecd2760b7a6 100644 --- a/arch/parisc/kernel/module.c +++ b/arch/parisc/kernel/module.c @@ -174,23 +174,6 @@ static inline int reassemble_22(int as22) ((as22 & 0x0003ff) << 3)); } -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .pgprot = PAGE_KERNEL_RWX, - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - execmem_params.ranges[EXECMEM_DEFAULT].start = VMALLOC_START; - execmem_params.ranges[EXECMEM_DEFAULT].end = VMALLOC_END; - - return &execmem_params; -} - #ifndef CONFIG_64BIT static inline unsigned long count_gots(const Elf_Rela *rela, unsigned long n) { diff --git a/arch/parisc/mm/init.c b/arch/parisc/mm/init.c index a088c243edea..c87fed38e38e 100644 --- a/arch/parisc/mm/init.c +++ b/arch/parisc/mm/init.c @@ -24,6 +24,7 @@ #include /* for node_online_map */ #include /* for release_pages */ #include +#include #include #include @@ -479,7 +480,7 @@ void free_initmem(void) /* finally dump all the instructions which were cached, since the * pages are no-longer executable */ flush_icache_range(init_begin, init_end); - + free_initmem_default(POISON_FREE_INITMEM); /* set up a new led state on systems shipped LED State panel */ @@ -919,3 +920,22 @@ static const pgprot_t protection_map[16] = { [VM_SHARED | VM_EXEC | VM_WRITE | VM_READ] = PAGE_RWX }; DECLARE_VM_GET_PAGE_PROT + +#ifdef CONFIG_EXECMEM +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .pgprot = PAGE_KERNEL_RWX, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + execmem_params.ranges[EXECMEM_DEFAULT].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_DEFAULT].end = VMALLOC_END; + + return &execmem_params; +} +#endif diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c index bf2c62aef628..b30e00964a60 100644 --- a/arch/powerpc/kernel/module.c +++ b/arch/powerpc/kernel/module.c @@ -10,7 +10,6 @@ #include #include #include -#include #include #include #include @@ -89,62 +88,3 @@ int module_finalize(const Elf_Ehdr *hdr, return 0; } - -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .alignment = 1, - }, - [EXECMEM_KPROBES] = { - .alignment = 1, - }, - [EXECMEM_MODULE_DATA] = { - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - pgprot_t prot = strict_module_rwx_enabled() ? PAGE_KERNEL : PAGE_KERNEL_EXEC; - struct execmem_range *range = &execmem_params.ranges[EXECMEM_DEFAULT]; - - /* - * BOOK3S_32 and 8xx define MODULES_VADDR for text allocations and - * allow allocating data in the entire vmalloc space - */ -#ifdef MODULES_VADDR - struct execmem_range *data = &execmem_params.ranges[EXECMEM_MODULE_DATA]; - unsigned long limit = (unsigned long)_etext - SZ_32M; - - /* First try within 32M limit from _etext to avoid branch trampolines */ - if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit) { - range->start = limit; - range->end = MODULES_END; - range->fallback_start = MODULES_VADDR; - range->fallback_end = MODULES_END; - } else { - range->start = MODULES_VADDR; - range->end = MODULES_END; - } - data->start = VMALLOC_START; - data->end = VMALLOC_END; - data->pgprot = PAGE_KERNEL; - data->alignment = 1; -#else - range->start = VMALLOC_START; - range->end = VMALLOC_END; -#endif - - range->pgprot = prot; - - execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_START; - execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_END; - - if (strict_module_rwx_enabled()) - execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_ROX; - else - execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_EXEC; - - return &execmem_params; -} diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 8b121df7b08f..06f4bb6fb780 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -406,3 +407,64 @@ int devmem_is_allowed(unsigned long pfn) * the EHEA driver. Drop this when drivers/net/ethernet/ibm/ehea is removed. */ EXPORT_SYMBOL_GPL(walk_system_ram_range); + +#ifdef CONFIG_EXECMEM +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .alignment = 1, + }, + [EXECMEM_KPROBES] = { + .alignment = 1, + }, + [EXECMEM_MODULE_DATA] = { + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + pgprot_t prot = strict_module_rwx_enabled() ? PAGE_KERNEL : PAGE_KERNEL_EXEC; + struct execmem_range *range = &execmem_params.ranges[EXECMEM_DEFAULT]; + + /* + * BOOK3S_32 and 8xx define MODULES_VADDR for text allocations and + * allow allocating data in the entire vmalloc space + */ +#ifdef MODULES_VADDR + struct execmem_range *data = &execmem_params.ranges[EXECMEM_MODULE_DATA]; + unsigned long limit = (unsigned long)_etext - SZ_32M; + + /* First try within 32M limit from _etext to avoid branch trampolines */ + if (MODULES_VADDR < PAGE_OFFSET && MODULES_END > limit) { + range->start = limit; + range->end = MODULES_END; + range->fallback_start = MODULES_VADDR; + range->fallback_end = MODULES_END; + } else { + range->start = MODULES_VADDR; + range->end = MODULES_END; + } + data->start = VMALLOC_START; + data->end = VMALLOC_END; + data->pgprot = PAGE_KERNEL; + data->alignment = 1; +#else + range->start = VMALLOC_START; + range->end = VMALLOC_END; +#endif + + range->pgprot = prot; + + execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_END; + + if (strict_module_rwx_enabled()) + execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_ROX; + else + execmem_params.ranges[EXECMEM_KPROBES].pgprot = PAGE_KERNEL_EXEC; + + return &execmem_params; +} +#endif diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c index 31505ecb5c72..8af08d5449bf 100644 --- a/arch/riscv/kernel/module.c +++ b/arch/riscv/kernel/module.c @@ -11,7 +11,6 @@ #include #include #include -#include #include #include @@ -436,44 +435,6 @@ int apply_relocate_add(Elf_Shdr *sechdrs, const char *strtab, return 0; } -#ifdef CONFIG_MMU -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .pgprot = PAGE_KERNEL, - .alignment = 1, - }, - [EXECMEM_KPROBES] = { - .pgprot = PAGE_KERNEL_READ_EXEC, - .alignment = 1, - }, - [EXECMEM_BPF] = { - .pgprot = PAGE_KERNEL, - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ -#ifdef CONFIG_64BIT - execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; - execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; -#else - execmem_params.ranges[EXECMEM_DEFAULT].start = VMALLOC_START; - execmem_params.ranges[EXECMEM_DEFAULT].end = VMALLOC_END; -#endif - - execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_START; - execmem_params.ranges[EXECMEM_KPROBES].end = VMALLOC_END; - - execmem_params.ranges[EXECMEM_BPF].start = BPF_JIT_REGION_START; - execmem_params.ranges[EXECMEM_BPF].end = BPF_JIT_REGION_END; - - return &execmem_params; -} -#endif - int module_finalize(const Elf_Ehdr *hdr, const Elf_Shdr *sechdrs, struct module *me) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 0798bd861dcb..b0f7848f39e3 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -24,6 +24,7 @@ #include #endif #include +#include #include #include @@ -1564,3 +1565,41 @@ void __init pgtable_cache_init(void) preallocate_pgd_pages_range(MODULES_VADDR, MODULES_END, "bpf/modules"); } #endif + +#if defined(CONFIG_MMU) && defined(CONFIG_EXECMEM) +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .pgprot = PAGE_KERNEL, + .alignment = 1, + }, + [EXECMEM_KPROBES] = { + .pgprot = PAGE_KERNEL_READ_EXEC, + .alignment = 1, + }, + [EXECMEM_BPF] = { + .pgprot = PAGE_KERNEL, + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ +#ifdef CONFIG_64BIT + execmem_params.ranges[EXECMEM_DEFAULT].start = MODULES_VADDR; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; +#else + execmem_params.ranges[EXECMEM_DEFAULT].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_DEFAULT].end = VMALLOC_END; +#endif + + execmem_params.ranges[EXECMEM_KPROBES].start = VMALLOC_START; + execmem_params.ranges[EXECMEM_KPROBES].end = VMALLOC_END; + + execmem_params.ranges[EXECMEM_BPF].start = BPF_JIT_REGION_START; + execmem_params.ranges[EXECMEM_BPF].end = BPF_JIT_REGION_END; + + return &execmem_params; +} +#endif diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c index 538d5f24af66..81a8d92ca092 100644 --- a/arch/s390/kernel/module.c +++ b/arch/s390/kernel/module.c @@ -37,31 +37,6 @@ #define PLT_ENTRY_SIZE 22 -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .flags = EXECMEM_KASAN_SHADOW, - .alignment = MODULE_ALIGN, - .pgprot = PAGE_KERNEL, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - unsigned long module_load_offset = 0; - unsigned long start; - - if (kaslr_enabled()) - module_load_offset = get_random_u32_inclusive(1, 1024) * PAGE_SIZE; - - start = MODULES_VADDR + module_load_offset; - execmem_params.ranges[EXECMEM_DEFAULT].start = start; - execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; - - return &execmem_params; -} - #ifdef CONFIG_FUNCTION_TRACER void module_arch_cleanup(struct module *mod) { diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c index 8b94d2212d33..2e6d6512fc5f 100644 --- a/arch/s390/mm/init.c +++ b/arch/s390/mm/init.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -311,3 +312,30 @@ void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) vmem_remove_mapping(start, size); } #endif /* CONFIG_MEMORY_HOTPLUG */ + +#ifdef CONFIG_EXECMEM +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .flags = EXECMEM_KASAN_SHADOW, + .alignment = MODULE_ALIGN, + .pgprot = PAGE_KERNEL, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + unsigned long module_load_offset = 0; + unsigned long start; + + if (kaslr_enabled()) + module_load_offset = get_random_u32_inclusive(1, 1024) * PAGE_SIZE; + + start = MODULES_VADDR + module_load_offset; + execmem_params.ranges[EXECMEM_DEFAULT].start = start; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + + return &execmem_params; +} +#endif diff --git a/arch/sparc/kernel/module.c b/arch/sparc/kernel/module.c index 1d8d1fba95b9..dff1d85ba202 100644 --- a/arch/sparc/kernel/module.c +++ b/arch/sparc/kernel/module.c @@ -14,7 +14,6 @@ #include #include #include -#include #ifdef CONFIG_SPARC64 #include #endif @@ -25,28 +24,6 @@ #include "entry.h" -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { -#ifdef CONFIG_SPARC64 - .start = MODULES_VADDR, - .end = MODULES_END, -#else - .start = VMALLOC_START, - .end = VMALLOC_END, -#endif - .alignment = 1, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; - - return &execmem_params; -} - /* Make generic code ignore STT_REGISTER dummy undefined symbols. */ int module_frob_arch_sections(Elf_Ehdr *hdr, Elf_Shdr *sechdrs, diff --git a/arch/sparc/mm/Makefile b/arch/sparc/mm/Makefile index 871354aa3c00..87e2cf7efb5b 100644 --- a/arch/sparc/mm/Makefile +++ b/arch/sparc/mm/Makefile @@ -15,3 +15,5 @@ obj-$(CONFIG_SPARC32) += leon_mm.o # Only used by sparc64 obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o + +obj-$(CONFIG_EXECMEM) += execmem.o diff --git a/arch/sparc/mm/execmem.c b/arch/sparc/mm/execmem.c new file mode 100644 index 000000000000..fb53a859869a --- /dev/null +++ b/arch/sparc/mm/execmem.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { +#ifdef CONFIG_SPARC64 + .start = MODULES_VADDR, + .end = MODULES_END, +#else + .start = VMALLOC_START, + .end = VMALLOC_END, +#endif + .alignment = 1, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; + + return &execmem_params; +} diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 9d37375e2f05..c52d591c0f3f 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -19,7 +19,6 @@ #include #include #include -#include #include #include @@ -37,32 +36,6 @@ do { \ } while (0) #endif -static struct execmem_params execmem_params __ro_after_init = { - .ranges = { - [EXECMEM_DEFAULT] = { - .flags = EXECMEM_KASAN_SHADOW, - .alignment = MODULE_ALIGN, - }, - }, -}; - -struct execmem_params __init *execmem_arch_params(void) -{ - unsigned long module_load_offset = 0; - unsigned long start; - - if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && kaslr_enabled()) - module_load_offset = - get_random_u32_inclusive(1, 1024) * PAGE_SIZE; - - start = MODULES_VADDR + module_load_offset; - execmem_params.ranges[EXECMEM_DEFAULT].start = start; - execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; - execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; - - return &execmem_params; -} - #ifdef CONFIG_X86_32 int apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 679893ea5e68..022af7ab50f9 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -1099,3 +1100,31 @@ unsigned long arch_max_swapfile_size(void) return pages; } #endif + +#ifdef CONFIG_EXECMEM +static struct execmem_params execmem_params __ro_after_init = { + .ranges = { + [EXECMEM_DEFAULT] = { + .flags = EXECMEM_KASAN_SHADOW, + .alignment = MODULE_ALIGN, + }, + }, +}; + +struct execmem_params __init *execmem_arch_params(void) +{ + unsigned long module_load_offset = 0; + unsigned long start; + + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && kaslr_enabled()) + module_load_offset = + get_random_u32_inclusive(1, 1024) * PAGE_SIZE; + + start = MODULES_VADDR + module_load_offset; + execmem_params.ranges[EXECMEM_DEFAULT].start = start; + execmem_params.ranges[EXECMEM_DEFAULT].end = MODULES_END; + execmem_params.ranges[EXECMEM_DEFAULT].pgprot = PAGE_KERNEL; + + return &execmem_params; +} +#endif /* CONFIG_EXECMEM */ From patchwork Mon Sep 18 07:29:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141280 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2491545vqi; Mon, 18 Sep 2023 00:49:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IET/Rs9zT+scCw5iZW3QHKL+ARsVndFhgoSmpIrfaEqY47n0syK3i1BpdLWOOmmdBk9/t6q X-Received: by 2002:a05:6a00:1956:b0:68f:cf6f:e21e with SMTP id s22-20020a056a00195600b0068fcf6fe21emr10212464pfk.28.1695023387923; Mon, 18 Sep 2023 00:49:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695023387; cv=none; d=google.com; s=arc-20160816; b=mLqb8Il7DHWXsrDtDgHp3he4R0hcKjVbGwHmKAhYG9XzWCxCkqb5sN542/4eqHaKAj MnInbUBVTnuqvxbHyEtxV0tLJpNtCkdFvnNVr8FCm29ONgQv/Uut+ULMBfkZh2clrFdB Vy1ZdBoqs29fPgNwdBh38QGZF+kOpSGll3kmw5ZDKudWZwx8AlfW4OdcO1umyDnxqEhF 4UqKLtMbzWFYINZHTicxCYEzEqzmwiqe9XO71g9r+XKgBnoGOtlWfciLgDTEEQphZgQ5 WhFnltvGI31E7rE+L3SFYU9oD8liNhMAekyUwezYH72GpJYkkcvKyX8XMk+ObyUsBnG9 Vvmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pZR7i50VIMpGPwB+fYLRiSUppIV7vmV9jffC49cC9qk=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=W0fXi6XBeHhfBjTA3xV3FD0Sn/L4meTAvoznQFfOQcyfwLqQnzobik/ifUXe9KtAvu vWxEKU2UphZnKZvvSBzuKtQB1O13QyySjrOAkSdXqPoV/jyRjOc6Bzee95kBEu61y3Ko g6Ubnc0LMBFUXd5ZgsZ/hHPoKsxgi7Q2Cw7SNrxizw9tpL0o4i7oRvJWo0vTXINGygp9 cmjX/TU6RFxNEbDQK2snQV820VZp+NasVh8eZG3O5iY6fca5u9I9/xc+gOJyY1r/6QQx sAoUv4q2BxGurqLBB/Lhba/D80UyyGzSZcFiZ+GefWdsCyO6xkMOsot8SMiXmWkJmRcR +MCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jOnWnURB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id k184-20020a6384c1000000b0057039bdf391si7764763pgd.69.2023.09.18.00.49.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:49:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=jOnWnURB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 590DE8029A85; Mon, 18 Sep 2023 00:33:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240372AbjIRHcj (ORCPT + 27 others); Mon, 18 Sep 2023 03:32:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240456AbjIRHc0 (ORCPT ); Mon, 18 Sep 2023 03:32:26 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D3421B9; Mon, 18 Sep 2023 00:32:02 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE792C433BC; Mon, 18 Sep 2023 07:31:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022322; bh=1UL8tl9eANdNOFG3ZTSYgBTww7VCSn8pNWCEuJfnTK4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jOnWnURBWcrt92iQ1E5Vr3mXWyQ6RAJ4i9B8tsY7FXOUVwWaNtFLbRVsEzVfJ0Nfg zcF2T76bilI3SPkaUjpMzEfs1sQN/JzTRMR83DnXh+Ov+MJ2AuKwZwF8Fnu/AObC0k QAdcrio7/1JZevOAFEj9aMF9cUGQUD6UCX9K1G2jVF2I2HWuIYbnj0wy/Rm4EXVNom HeAH3TaXGuMZDe4Y3BnUwv/DnuIZbNW8G+v45crhACvScxYPJb49nAjf6MNnqIIZFe chFFMgGweSzKG7hh/3g3keI9zbSw1HmwgbPcUexA3rwgKV2tInhKeo5IsMuopaavQ2 VqOvvHTW8VtZw== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 11/13] x86/ftrace: enable dynamic ftrace without CONFIG_MODULES Date: Mon, 18 Sep 2023 10:29:53 +0300 Message-Id: <20230918072955.2507221-12-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:33:53 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360843892159006 X-GMAIL-MSGID: 1777360843892159006 From: "Mike Rapoport (IBM)" Dynamic ftrace must allocate memory for code and this was impossible without CONFIG_MODULES. With execmem separated from the modules code, execmem_text_alloc() is available regardless of CONFIG_MODULES. Remove dependency of dynamic ftrace on CONFIG_MODULES and make CONFIG_DYNAMIC_FTRACE select CONFIG_EXECMEM in Kconfig. Signed-off-by: Mike Rapoport (IBM) --- arch/x86/Kconfig | 1 + arch/x86/kernel/ftrace.c | 10 ---------- 2 files changed, 1 insertion(+), 10 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 982b777eadc7..cc7c4a0a8c16 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -35,6 +35,7 @@ config X86_64 select SWIOTLB select ARCH_HAS_ELFCORE_COMPAT select ZONE_DMA32 + select EXECMEM if DYNAMIC_FTRACE config FORCE_DYNAMIC_FTRACE def_bool y diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index ae56d79a6a74..7ed7e8297ba3 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -261,8 +261,6 @@ void arch_ftrace_update_code(int command) /* Currently only x86_64 supports dynamic trampolines */ #ifdef CONFIG_X86_64 -#ifdef CONFIG_MODULES -/* Module allocation simplifies allocating memory for code */ static inline void *alloc_tramp(unsigned long size) { return execmem_text_alloc(EXECMEM_FTRACE, size); @@ -271,14 +269,6 @@ static inline void tramp_free(void *tramp) { execmem_free(tramp); } -#else -/* Trampolines can only be created if modules are supported */ -static inline void *alloc_tramp(unsigned long size) -{ - return NULL; -} -static inline void tramp_free(void *tramp) { } -#endif /* Defined as markers to the end of the ftrace default trampolines */ extern void ftrace_regs_caller_end(void); From patchwork Mon Sep 18 07:29:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141279 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2491541vqi; Mon, 18 Sep 2023 00:49:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH0KFY45BY4GE9XD1VQh+EtGcpjrqdvO3C/msMJXMn76ALb3d735NotJ6rMk3VH/kFzUG0b X-Received: by 2002:a17:90a:a891:b0:273:ec82:5554 with SMTP id h17-20020a17090aa89100b00273ec825554mr11244112pjq.9.1695023387297; Mon, 18 Sep 2023 00:49:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695023387; cv=none; d=google.com; s=arc-20160816; b=YfO30/9r3MiOsaF1RU86gzOvuStc9neaLPoXgDd6PU9SLhGehhYXzGfOIYzM2fxQAV Y/bwd0KHdRJWnpJ4FAi8q6HcBmffdO1/5vZPQLbzDYJw4D/3TC29kAwGBXxMLzQGf88P K14GmcMNrSQo7B+G1bIc3vPMrD9y3YkWXPzwwYOtaQwrSDIAJjZG4uSUlUG4trk/bCH+ WIYHh6dDxfQKJVEP46YQJHZ+O4t4eFP7S7QcdcYQSNtfzeb+OeQQfsKClC2uYklGeVWq X1Bn2i5KRjtNHiw8Gyo2BT3648uWiCeLjNQPATdx0zl3t5WR80iZQqzkjdibY4vuzbbm PFoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dqCPvIMXOTJlhRnOJhYfwq6l5xz7OPueXxaVIBOdmGc=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=XOZ432E83HaaaZzDiwIjX6tRTNqyBzr9vMnULZ1IQyeMlgJRKZhNpKBYEg9lIHTBe5 Jr/KkBqf+aRGLOBmrH/E2HDagVUTlWpoEWqgYwcs22BpJmbsJ59p12gusIcZggOaUwCs ucRMGMw8AIuuGEV0BUc2B/7YxCNMyOh9LXZbgDctjY2ctIcrgHryJumQWkASL+9z6x8A 1f5no4ynksW09nFCJCd2S/VPNhfYqGH4xyXbu3AsYnNRnXNA8FkhlCksg4Ue5Cjj1NO2 A5udhnDlfe+lg6xA5py/lgkA7LRQMzd7msPqOrjhGMUUGwDyNQk4PFXTDiAXJk/KbaE8 08hA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=kT3NctsE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id ha11-20020a17090af3cb00b002747da1ef70si6837583pjb.16.2023.09.18.00.49.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:49:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=kT3NctsE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 129948029A87; Mon, 18 Sep 2023 00:33:48 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240350AbjIRHdD (ORCPT + 27 others); Mon, 18 Sep 2023 03:33:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240354AbjIRHcg (ORCPT ); Mon, 18 Sep 2023 03:32:36 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4389ED2; Mon, 18 Sep 2023 00:32:12 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B212CC433A9; Mon, 18 Sep 2023 07:32:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022331; bh=g2ysQzPrupxvZBEk8R4nz6nGqIwxox2mdTADaK2KSBo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kT3NctsEzR46ZMcvQmZYEzpOY6ldmIA+UpjJflzhcN6xMD3eDfrH5TshpxPSzqYKZ 4RtqZhzZqqMGjQFg0TiMJ4pIA+4pgayZMlfPQIIGjFZSqPjnRRF0/SACLSULT/WzJX e5w4J3wzseysRp3HjruUqlEklzGHP8/6ICjr9417gwdOyxSuUT0n9ca5FaDmjgMb4N wh+TlKoXNolf6tuh2rWFnkZ3l88wMsKsT34tZyIMrsdUBBtjJC/AKV44QLF3klepDN XUU8NCM4DCe8D1KxoJHEl8Op3szJMcdX1IE3cH+lAoAp79l+6uCu78MbUgn4aNR/c+ j8TJxIqTZhi6w== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 12/13] kprobes: remove dependency on CONFIG_MODULES Date: Mon, 18 Sep 2023 10:29:54 +0300 Message-Id: <20230918072955.2507221-13-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:33:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360843244442167 X-GMAIL-MSGID: 1777360843244442167 From: "Mike Rapoport (IBM)" kprobes depended on CONFIG_MODULES because it has to allocate memory for code. Since code allocations are now implemented with execmem, kprobes can be enabled in non-modular kernels. Add #ifdef CONFIG_MODULE guards for the code dealing with kprobes inside modules, make CONFIG_KPROBES select CONFIG_EXECMEM and drop the dependency of CONFIG_KPROBES on CONFIG_MODULES. Signed-off-by: Mike Rapoport (IBM) --- arch/Kconfig | 2 +- kernel/kprobes.c | 43 +++++++++++++++++++++---------------- kernel/trace/trace_kprobe.c | 11 ++++++++++ 3 files changed, 37 insertions(+), 19 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 12d51495caec..c52a600b63ca 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -52,9 +52,9 @@ config GENERIC_ENTRY config KPROBES bool "Kprobes" - depends on MODULES depends on HAVE_KPROBES select KALLSYMS + select EXECMEM select TASKS_RCU if PREEMPTION help Kprobes allows you to trap at almost any kernel address and diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 0ccb4d2ec9a2..c95d0088f966 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1580,6 +1580,7 @@ static int check_kprobe_address_safe(struct kprobe *p, goto out; } +#ifdef CONFIG_MODULES /* Check if 'p' is probing a module. */ *probed_mod = __module_text_address((unsigned long) p->addr); if (*probed_mod) { @@ -1603,6 +1604,8 @@ static int check_kprobe_address_safe(struct kprobe *p, ret = -ENOENT; } } +#endif + out: preempt_enable(); jump_label_unlock(); @@ -2495,24 +2498,6 @@ int kprobe_add_area_blacklist(unsigned long start, unsigned long end) return 0; } -/* Remove all symbols in given area from kprobe blacklist */ -static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end) -{ - struct kprobe_blacklist_entry *ent, *n; - - list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) { - if (ent->start_addr < start || ent->start_addr >= end) - continue; - list_del(&ent->list); - kfree(ent); - } -} - -static void kprobe_remove_ksym_blacklist(unsigned long entry) -{ - kprobe_remove_area_blacklist(entry, entry + 1); -} - int __weak arch_kprobe_get_kallsym(unsigned int *symnum, unsigned long *value, char *type, char *sym) { @@ -2577,6 +2562,25 @@ static int __init populate_kprobe_blacklist(unsigned long *start, return ret ? : arch_populate_kprobe_blacklist(); } +#ifdef CONFIG_MODULES +/* Remove all symbols in given area from kprobe blacklist */ +static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end) +{ + struct kprobe_blacklist_entry *ent, *n; + + list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) { + if (ent->start_addr < start || ent->start_addr >= end) + continue; + list_del(&ent->list); + kfree(ent); + } +} + +static void kprobe_remove_ksym_blacklist(unsigned long entry) +{ + kprobe_remove_area_blacklist(entry, entry + 1); +} + static void add_module_kprobe_blacklist(struct module *mod) { unsigned long start, end; @@ -2678,6 +2682,7 @@ static struct notifier_block kprobe_module_nb = { .notifier_call = kprobes_module_callback, .priority = 0 }; +#endif void kprobe_free_init_mem(void) { @@ -2737,8 +2742,10 @@ static int __init init_kprobes(void) err = arch_init_kprobes(); if (!err) err = register_die_notifier(&kprobe_exceptions_nb); +#ifdef CONFIG_MODULES if (!err) err = register_module_notifier(&kprobe_module_nb); +#endif kprobes_initialized = (err == 0); kprobe_sysctls_init(); diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c index 3d7a180a8427..25a5293a80c0 100644 --- a/kernel/trace/trace_kprobe.c +++ b/kernel/trace/trace_kprobe.c @@ -111,6 +111,7 @@ static nokprobe_inline bool trace_kprobe_within_module(struct trace_kprobe *tk, return strncmp(module_name(mod), name, len) == 0 && name[len] == ':'; } +#ifdef CONFIG_MODULES static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk) { char *p; @@ -129,6 +130,12 @@ static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk) return ret; } +#else +static inline bool trace_kprobe_module_exist(struct trace_kprobe *tk) +{ + return false; +} +#endif static bool trace_kprobe_is_busy(struct dyn_event *ev) { @@ -670,6 +677,7 @@ static int register_trace_kprobe(struct trace_kprobe *tk) return ret; } +#ifdef CONFIG_MODULES /* Module notifier call back, checking event on the module */ static int trace_kprobe_module_callback(struct notifier_block *nb, unsigned long val, void *data) @@ -704,6 +712,7 @@ static struct notifier_block trace_kprobe_module_nb = { .notifier_call = trace_kprobe_module_callback, .priority = 1 /* Invoked after kprobe module callback */ }; +#endif static int __trace_kprobe_create(int argc, const char *argv[]) { @@ -1810,8 +1819,10 @@ static __init int init_kprobe_trace_early(void) if (ret) return ret; +#ifdef CONFIG_MODULES if (register_module_notifier(&trace_kprobe_module_nb)) return -EINVAL; +#endif return 0; } From patchwork Mon Sep 18 07:29:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 141272 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp2488081vqi; Mon, 18 Sep 2023 00:39:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE0S27tWbOhgDqm2aG038dD0ZZK2i8hmL1TBHCnDdfK3tiae/XlHnbmelBNaClVKf1Ci+TK X-Received: by 2002:a05:6a20:841c:b0:149:700e:f50a with SMTP id c28-20020a056a20841c00b00149700ef50amr16635908pzd.29.1695022744772; Mon, 18 Sep 2023 00:39:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695022744; cv=none; d=google.com; s=arc-20160816; b=LK1Wg60y1GJKcrWJzmlelx236A80oVMnJ2rzFGniAhxLBP7d1xa5rDnB74/N+gQAyr nu6WmHlb/FuyeZS8Bn07FbwFlQfkQj2XokbuONXfsiJMlPVfPSBg5XwheNei5aHq/BCN l5tPqSnZvCETFNCvNPazSckg8VnsaPbv0wh6dRIQYoeI31zoRrGD6BScJD4IVHlkV5O3 izf1b2HvvFjZPh9WjeTgDMxEydHBhBeFz2Mmo2VC3Qi0guiwVv6CFXib4WAtzxsBr0jB B3TCZFjPixgCCqGtPjPySHYbUpy0f8pU3NL15HHiQwDx0SYB7mCKtdTfsseQqmX6Sp+e jv3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hpRvplrQS5IqpGgGr3tvfF4hOei026V7oQnE9P+m2j0=; fh=mYytDhFxh0BOqNSa1Iv610hxG4sclAxBmwUIWE9B5gw=; b=GeAu9xVLE8kqRBt3klf+dUGktBxzXBpAlfFsaKvYI1eZIcDYwSjGXzvnbD7tqhofGk cQzoNVadO1mWPUIthIthvfyIOQ9LsXp1VOeeVY5BU3F82GghMVt9jyVDNWx9bJCXvbFo lRiVAlXRG6UUJwlfSy6vJPLzr3h3oyiSns9tmFM8A+/se+0iYmoURK2fOToeyct1fFJm Kq0dBKCxgpjOmPq9+BzVBqKghTU8gzu36+EDyQdKA0CAF58a0ekIWkRcs1gfv8ZFt7b0 tYCtIBtgBcwx8d5+qjBpfGCzzc5HvSXuCoqJ2hrLZPBELmuEAImNVUjx5AGRlX8sJOWY b0kA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Vnbzf3Qh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id s1-20020a637701000000b00573faff7d3esi7382188pgc.712.2023.09.18.00.39.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 00:39:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Vnbzf3Qh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id A250081F1D9E; Mon, 18 Sep 2023 00:34:09 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240202AbjIRHde (ORCPT + 27 others); Mon, 18 Sep 2023 03:33:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240252AbjIRHc5 (ORCPT ); Mon, 18 Sep 2023 03:32:57 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBEE8186; Mon, 18 Sep 2023 00:32:21 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6BC40C433B9; Mon, 18 Sep 2023 07:32:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022341; bh=07Jp3LphXzl4EkHja0JdIUA/xjHpNGiZuguIXloPztM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vnbzf3QhQuVmwYQAf2Mt0WSZBefd5lhLSFtkXhwMC+n+X7N4ki+6T0boDwsgMr0XN AAdXg1yYsIa/lFW8JGjF+xmiLYY63eHqaMwytJIlEVwOkPrMrojuYJZhQpE797YZju GZcg8J0n+fEArc0GhUtiTV4LpXaeVrImgv2qDiS2pZ01DSA9wJwPYqPfL7KPbaX6bE QYQH9HSRdQuGBz3raOjQKyLrYnfzs7cZoMWOYQ1LzctZYuLHDNxMU/K31vWL6RTmKL zi7xoAjilllhk4DMDBlwvUkOVzwZl6SwuVwOf6rk+tbCRJ2s6/D0PIr6IMDdrgSqtE mxvGlqxn371yQ== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 13/13] bpf: remove CONFIG_BPF_JIT dependency on CONFIG_MODULES of Date: Mon, 18 Sep 2023 10:29:55 +0300 Message-Id: <20230918072955.2507221-14-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230918072955.2507221-1-rppt@kernel.org> References: <20230918072955.2507221-1-rppt@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 18 Sep 2023 00:34:09 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777360169498761198 X-GMAIL-MSGID: 1777360169498761198 From: "Mike Rapoport (IBM)" BPF just-in-time compiler depended on CONFIG_MODULES because it used module_alloc() to allocate memory for the generated code. Since code allocations are now implemented with execmem, drop dependency of CONFIG_BPF_JIT on CONFIG_MODULES and make it select CONFIG_EXECMEM. Suggested-by: Björn Töpel Signed-off-by: Mike Rapoport (IBM) --- kernel/bpf/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/bpf/Kconfig b/kernel/bpf/Kconfig index 6a906ff93006..5be11c906f93 100644 --- a/kernel/bpf/Kconfig +++ b/kernel/bpf/Kconfig @@ -42,7 +42,7 @@ config BPF_JIT bool "Enable BPF Just In Time compiler" depends on BPF depends on HAVE_CBPF_JIT || HAVE_EBPF_JIT - depends on MODULES + select EXECMEM help BPF programs are normally handled by a BPF interpreter. This option allows the kernel to generate native code when a program is loaded