From patchwork Mon Jan 9 09:07:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Youling Tang X-Patchwork-Id: 40737 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp2052519wrt; Mon, 9 Jan 2023 01:13:18 -0800 (PST) X-Google-Smtp-Source: AMrXdXtpv5uOh/SuaEPMSZ3skcWioVBNm+6DvO6tup6O1mydwUsF2xA7hiKB9owvT0qD/BeJLZlX X-Received: by 2002:a17:902:ce82:b0:192:fb92:327 with SMTP id f2-20020a170902ce8200b00192fb920327mr23176427plg.58.1673255598213; Mon, 09 Jan 2023 01:13:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673255598; cv=none; d=google.com; s=arc-20160816; b=yHX/FwOPAJOoCVBB2eEnJPmol70dTmxuH4z2SLw4+zpMBT63QTojzoJY2jzq3fYeC+ XzMxOvOQXybwPgv7t0J8zDQk8Ex7IfNAI2SU2GZIVwENbVfWF/aXIcJwiWKFZX2J70MV PB/NNUcdFErnazUs5Bk4L7dZ5sWzw2SNLsZ9VTvwK4TWLqnIYSBhlh6pK4vLgjUPE+MH zLBIZdrUjFeHFMeoDz+eQozKrELQwpCX2XZ6gMJgtSiFei03GRY3cOaE+w37PO7zHCry r4wH6eb06pjf7vfuEFifmUjXKr1Ubf1/bj6iU4s1tsk5uCHGXKrgQA4men8IhF9spYSL flJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=lyn6A2odJNHZZqzi5+aJ9GDAfnaIGdaKYSwLkhhZtMU=; b=J0JvCkWoL0hpApJRr6Am8FlVagJhi8AoVfrJi8x5HCf5on2flwnjteRJ2CqkUZXhL3 UXl83Y36X2yA+zzR2RcB/rgOvIFflXZg+APfRYBzk1A9sxfOXBQhDZBSfN3IC0kHbs71 ks0TJHg/CZAjQH/j5XLCSyOBj5y1EMT3pG6SbLZUT9Xzonw1kvUwZg0Mz2Te+LJ7RaEj xoKTibp+/z05rUhNFgGpxEiHfeJZnNNwm0CsQ3DqiIOxs5QONRzVfuwiCw+UD0OGUzAg pjBvJ9oWJNHqj8chXg5D31DBGnjb5f8AbiOLSUbxxYI9kwpaVMDgKD+ucxwI66vY8+hl GQQg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x12-20020a1709027c0c00b0018666611f85si8030163pll.508.2023.01.09.01.13.05; Mon, 09 Jan 2023 01:13:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233475AbjAIJMf (ORCPT + 99 others); Mon, 9 Jan 2023 04:12:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236996AbjAIJLu (ORCPT ); Mon, 9 Jan 2023 04:11:50 -0500 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7399A17E23 for ; Mon, 9 Jan 2023 01:08:14 -0800 (PST) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8AxrvB82btjgHUAAA--.1723S3; Mon, 09 Jan 2023 17:08:12 +0800 (CST) Received: from bogon.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxTuRw2btj_6sWAA--.4042S3; Mon, 09 Jan 2023 17:08:11 +0800 (CST) From: Youling Tang To: Huacai Chen , Xi Ruoyao , Jinyang He Cc: Xuerui Wang , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 1/4] LoongArch: Use trampoline for exception handlers and kill la.abs Date: Mon, 9 Jan 2023 17:07:51 +0800 Message-Id: <1673255274-18238-2-git-send-email-tangyouling@loongson.cn> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> References: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> X-CM-TRANSID: AQAAf8DxTuRw2btj_6sWAA--.4042S3 X-CM-SenderInfo: 5wdqw5prxox03j6o00pqjv00gofq/ X-Coremail-Antispam: 1Uk129KBjvJXoW3GF1rXry3tFWfXFW3CrWDtwb_yoW3WF17pw nxAr4DJa10gFn5Za4UKw1UurW5JwnFga1aga17CrWru3W2vF18X34vy397uF9rKay8XF40 9F1rJw4IvFWUJwUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU b7AYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_Jrv_JF1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwA2z4 x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l e2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4CE44I27wAqx4xG64xvF2 IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_JF0_Jw1lYx0Ex4A2jsIE14v26r1j6r4U McvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwIxGrwCFx2 IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v2 6r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67 AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1lIxAIcVCF04k26cxKx2IY s7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Jr 0_GrUvcSsGvfC2KfnxnUUI43ZEXa7IU82g43UUUUU== X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754535662307489880?= X-GMAIL-MSGID: =?utf-8?q?1754535662307489880?= From: Xi Ruoyao Use a trampoline as an exception handlers, which can kill some use of la.abs in preparation for the subsequent support of the PIE kernel. Signed-off-by: Xi Ruoyao Signed-off-by: Youling Tang --- arch/loongarch/include/asm/stackframe.h | 8 +++---- arch/loongarch/include/asm/uaccess.h | 1 - arch/loongarch/kernel/entry.S | 6 +++--- arch/loongarch/kernel/genex.S | 20 +++++++++--------- arch/loongarch/kernel/head.S | 2 +- arch/loongarch/kernel/traps.c | 4 +++- arch/loongarch/mm/tlbex.S | 28 +++++++------------------ 7 files changed, 29 insertions(+), 40 deletions(-) diff --git a/arch/loongarch/include/asm/stackframe.h b/arch/loongarch/include/asm/stackframe.h index 4ca953062b5b..96c94035b5d0 100644 --- a/arch/loongarch/include/asm/stackframe.h +++ b/arch/loongarch/include/asm/stackframe.h @@ -76,8 +76,8 @@ * kernelsp array for it. It stores the current sp in t0 and loads the * new value in sp. */ - .macro get_saved_sp docfi=0 - la.abs t1, kernelsp + .macro get_saved_sp docfi=0 + la.pcrel t1, kernelsp #ifdef CONFIG_SMP csrrd t0, PERCPU_BASE_KS LONG_ADD t1, t1, t0 @@ -89,8 +89,8 @@ LONG_L sp, t1, 0 .endm - .macro set_saved_sp stackp temp temp2 - la.abs \temp, kernelsp + .macro set_saved_sp stackp temp temp2 + la.pcrel \temp, kernelsp #ifdef CONFIG_SMP LONG_ADD \temp, \temp, u0 #endif diff --git a/arch/loongarch/include/asm/uaccess.h b/arch/loongarch/include/asm/uaccess.h index 255899d4a7c3..0d22991ae430 100644 --- a/arch/loongarch/include/asm/uaccess.h +++ b/arch/loongarch/include/asm/uaccess.h @@ -22,7 +22,6 @@ extern u64 __ua_limit; #define __UA_ADDR ".dword" -#define __UA_LA "la.abs" #define __UA_LIMIT __ua_limit /* diff --git a/arch/loongarch/kernel/entry.S b/arch/loongarch/kernel/entry.S index d53b631c9022..ca01afdbec3f 100644 --- a/arch/loongarch/kernel/entry.S +++ b/arch/loongarch/kernel/entry.S @@ -18,9 +18,9 @@ .text .cfi_sections .debug_frame .align 5 -SYM_FUNC_START(handle_syscall) +SYM_FUNC_START(handle_sys) csrrd t0, PERCPU_BASE_KS - la.abs t1, kernelsp + la.pcrel t1, kernelsp add.d t1, t1, t0 move t2, sp ld.d sp, t1, 0 @@ -66,7 +66,7 @@ SYM_FUNC_START(handle_syscall) bl do_syscall RESTORE_ALL_AND_RET -SYM_FUNC_END(handle_syscall) +SYM_FUNC_END(handle_sys) SYM_CODE_START(ret_from_fork) bl schedule_tail # a0 = struct task_struct *prev diff --git a/arch/loongarch/kernel/genex.S b/arch/loongarch/kernel/genex.S index 75e5be807a0d..d3df0fa725a2 100644 --- a/arch/loongarch/kernel/genex.S +++ b/arch/loongarch/kernel/genex.S @@ -32,9 +32,8 @@ SYM_FUNC_START(__arch_cpu_idle) SYM_FUNC_END(__arch_cpu_idle) SYM_FUNC_START(handle_vint) - BACKUP_T0T1 SAVE_ALL - la.abs t1, __arch_cpu_idle + la.pcrel t1, __arch_cpu_idle LONG_L t0, sp, PT_ERA /* 32 byte rollback region */ ori t0, t0, 0x1f @@ -43,8 +42,7 @@ SYM_FUNC_START(handle_vint) LONG_S t0, sp, PT_ERA 1: move a0, sp move a1, sp - la.abs t0, do_vint - jirl ra, t0, 0 + bl do_vint RESTORE_ALL_AND_RET SYM_FUNC_END(handle_vint) @@ -67,12 +65,10 @@ SYM_FUNC_END(except_vec_cex) .macro BUILD_HANDLER exception handler prep .align 5 SYM_FUNC_START(handle_\exception) - BACKUP_T0T1 SAVE_ALL build_prep_\prep move a0, sp - la.abs t0, do_\handler - jirl ra, t0, 0 + bl do_\handler RESTORE_ALL_AND_RET SYM_FUNC_END(handle_\exception) .endm @@ -89,7 +85,11 @@ SYM_FUNC_END(except_vec_cex) BUILD_HANDLER watch watch none BUILD_HANDLER reserved reserved none /* others */ -SYM_FUNC_START(handle_sys) - la.abs t0, handle_syscall +SYM_FUNC_START(handler_trampoline) + csrwr t0, EXCEPTION_KS0 + csrwr t1, EXCEPTION_KS1 + pcaddi t0, 0 + ld.d t0, t0, 16 jr t0 -SYM_FUNC_END(handle_sys) + nop +SYM_FUNC_END(handler_trampoline) diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index 57bada6b4e93..aa6181714ec3 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -117,7 +117,7 @@ SYM_CODE_START(smpboot_entry) li.w t0, 0x00 # FPE=0, SXE=0, ASXE=0, BTE=0 csrwr t0, LOONGARCH_CSR_EUEN - la.abs t0, cpuboot_data + la.pcrel t0, cpuboot_data ld.d sp, t0, CPU_BOOT_STACK ld.d tp, t0, CPU_BOOT_TINFO diff --git a/arch/loongarch/kernel/traps.c b/arch/loongarch/kernel/traps.c index 7ea62faeeadb..0e8faaca3679 100644 --- a/arch/loongarch/kernel/traps.c +++ b/arch/loongarch/kernel/traps.c @@ -61,6 +61,7 @@ extern asmlinkage void handle_lasx(void); extern asmlinkage void handle_reserved(void); extern asmlinkage void handle_watch(void); extern asmlinkage void handle_vint(void); +extern asmlinkage void handler_trampoline(void); static void show_backtrace(struct task_struct *task, const struct pt_regs *regs, const char *loglvl, bool user) @@ -716,7 +717,8 @@ void per_cpu_trap_init(int cpu) /* Install CPU exception handler */ void set_handler(unsigned long offset, void *addr, unsigned long size) { - memcpy((void *)(eentry + offset), addr, size); + memcpy((void *)(eentry + offset), &handler_trampoline, 24); + memcpy((void *)(eentry + offset + 24), &addr, 8); local_flush_icache_range(eentry + offset, eentry + offset + size); } diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S index 58781c6e4191..cfaacdac518c 100644 --- a/arch/loongarch/mm/tlbex.S +++ b/arch/loongarch/mm/tlbex.S @@ -24,8 +24,7 @@ move a0, sp REG_S a2, sp, PT_BVADDR li.w a1, \write - la.abs t0, do_page_fault - jirl ra, t0, 0 + bl do_page_fault RESTORE_ALL_AND_RET SYM_FUNC_END(tlb_do_page_fault_\write) .endm @@ -34,20 +33,16 @@ tlb_do_page_fault 1 SYM_FUNC_START(handle_tlb_protect) - BACKUP_T0T1 SAVE_ALL move a0, sp move a1, zero csrrd a2, LOONGARCH_CSR_BADV REG_S a2, sp, PT_BVADDR - la.abs t0, do_page_fault - jirl ra, t0, 0 + bl do_page_fault RESTORE_ALL_AND_RET SYM_FUNC_END(handle_tlb_protect) SYM_FUNC_START(handle_tlb_load) - csrwr t0, EXCEPTION_KS0 - csrwr t1, EXCEPTION_KS1 csrwr ra, EXCEPTION_KS2 /* @@ -116,7 +111,7 @@ smp_pgtable_change_load: #ifdef CONFIG_64BIT vmalloc_load: - la.abs t1, swapper_pg_dir + la.pcrel t1, swapper_pg_dir b vmalloc_done_load #endif @@ -187,13 +182,10 @@ tlb_huge_update_load: nopage_tlb_load: dbar 0 csrrd ra, EXCEPTION_KS2 - la.abs t0, tlb_do_page_fault_0 - jr t0 + b tlb_do_page_fault_0 SYM_FUNC_END(handle_tlb_load) SYM_FUNC_START(handle_tlb_store) - csrwr t0, EXCEPTION_KS0 - csrwr t1, EXCEPTION_KS1 csrwr ra, EXCEPTION_KS2 /* @@ -263,7 +255,7 @@ smp_pgtable_change_store: #ifdef CONFIG_64BIT vmalloc_store: - la.abs t1, swapper_pg_dir + la.pcrel t1, swapper_pg_dir b vmalloc_done_store #endif @@ -336,13 +328,10 @@ tlb_huge_update_store: nopage_tlb_store: dbar 0 csrrd ra, EXCEPTION_KS2 - la.abs t0, tlb_do_page_fault_1 - jr t0 + b tlb_do_page_fault_1 SYM_FUNC_END(handle_tlb_store) SYM_FUNC_START(handle_tlb_modify) - csrwr t0, EXCEPTION_KS0 - csrwr t1, EXCEPTION_KS1 csrwr ra, EXCEPTION_KS2 /* @@ -411,7 +400,7 @@ smp_pgtable_change_modify: #ifdef CONFIG_64BIT vmalloc_modify: - la.abs t1, swapper_pg_dir + la.pcrel t1, swapper_pg_dir b vmalloc_done_modify #endif @@ -483,8 +472,7 @@ tlb_huge_update_modify: nopage_tlb_modify: dbar 0 csrrd ra, EXCEPTION_KS2 - la.abs t0, tlb_do_page_fault_1 - jr t0 + b tlb_do_page_fault_1 SYM_FUNC_END(handle_tlb_modify) SYM_FUNC_START(handle_tlb_refill) From patchwork Mon Jan 9 09:07:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Youling Tang X-Patchwork-Id: 40736 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp2052484wrt; Mon, 9 Jan 2023 01:13:13 -0800 (PST) X-Google-Smtp-Source: AMrXdXun9NH725JA0PuNiDxTzuLpb4OgD9v6NcwrAhd4WLMgMGtHGVnPjdqSFfk/FPPfC6Bt9BRE X-Received: by 2002:a05:6a21:2d09:b0:a4:aa40:2253 with SMTP id tw9-20020a056a212d0900b000a4aa402253mr76483383pzb.21.1673255592967; Mon, 09 Jan 2023 01:13:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673255592; cv=none; d=google.com; s=arc-20160816; b=i8KhpADqMa2VFC9XcYOYYOPP1gag/qUJorNwn+D+HlviNr0uGsCoHaYAk/m6FYd4z6 RkluqCgUo/3gBcb8xSaMKBsvCaHZ/6h5BPeKkhGOFhl9TOAUcjkcuiI3uCFeLGkyBWjQ OzWPvc9zVvwokE4j6iCY5d3rDUcpV+JOv8Y1liaL/jtJJXCIMQp0os/VJ/IE1dJZlGW4 yn21tynKx/dy5N8YehXsSHh35ovP8bXAyz96JX4q676GWM8oJb1ao4vNIgYvcYxF3n8V Mos2DHEATka/ILMP4S/RTJ/BGmEVbRLZ67nVA300fY8EnpsXiIykoa8uHZu/xfzdzJ04 dqZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=cAHZ+Eokh1kaKk6pWU/wRMc2SbmYKqGAuEBuAaJul58=; b=ytjV/E1qwTcOAuGXA1fMQFr43K+3jk0NUepVprYoEMYD8kVL8PEUiHgAy3U4yN90Mp 8RgBxVqg00qH9and4My05uMZskIN66rFbP7dnpzmRin24l5N0ZvGRke3vsSvSpuTFON5 ulWkWF3SNuIqM2uupjoHTMSLZCtq78xB+ahTQ0nUOaJmEodw7dBRwz25badmNrvdklcU naOUMwCJZ2lt8z2jM8GU01xBm0+H31AUzoHCiZNpHFePn8crpDgwjacZo8SYUCIaHOrI qsTUzFoUkEV2J+CIyETSTtpKQiomSfgtOvVHb9NvTP1cWO64bL+Q7tGMKH2amDSFR+Nw Xkzg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d12-20020a63360c000000b0049ca23dd6d9si9641596pga.134.2023.01.09.01.13.00; Mon, 09 Jan 2023 01:13:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236390AbjAIJMa (ORCPT + 99 others); Mon, 9 Jan 2023 04:12:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236994AbjAIJLu (ORCPT ); Mon, 9 Jan 2023 04:11:50 -0500 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7391A17E22 for ; Mon, 9 Jan 2023 01:08:14 -0800 (PST) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8Dxi+p92btjhnUAAA--.1356S3; Mon, 09 Jan 2023 17:08:13 +0800 (CST) Received: from bogon.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxTuRw2btj_6sWAA--.4042S4; Mon, 09 Jan 2023 17:08:12 +0800 (CST) From: Youling Tang To: Huacai Chen , Xi Ruoyao , Jinyang He Cc: Xuerui Wang , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] LoongArch: Add JUMP_LINK_ADDR macro implementation to avoid using la.abs Date: Mon, 9 Jan 2023 17:07:52 +0800 Message-Id: <1673255274-18238-3-git-send-email-tangyouling@loongson.cn> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> References: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> X-CM-TRANSID: AQAAf8DxTuRw2btj_6sWAA--.4042S4 X-CM-SenderInfo: 5wdqw5prxox03j6o00pqjv00gofq/ X-Coremail-Antispam: 1Uk129KBjvJXoW7WF4kuw4rtr1rXw4xXr1DKFg_yoW8ur1fpr 90vr4kGFW8WFn3AFyUta4UurW5J3WkCa1aganrta4IkF43XFy8Xw1vkr1DXF1Utw4rJr40 93WfGw4agF1UJ3JanT9S1TB71UUUUUDqnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU b28YFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_JF0_JFyl8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwA2z4 x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l e2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4CE44I27wAqx4xG64xvF2 IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jw0_WrylYx0Ex4A2jsIE14v26r4j6F4U McvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwIxGrwCFx2 IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v2 6r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67 AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IY s7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr 0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07j5l1kUUUUU= X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754535656244685882?= X-GMAIL-MSGID: =?utf-8?q?1754535656244685882?= Add JUMP_LINK_ADDR macro implementation to avoid using la.abs. Signed-off-by: Youling Tang --- arch/loongarch/include/asm/stackframe.h | 8 ++++++++ arch/loongarch/kernel/head.S | 10 ++++------ arch/loongarch/power/suspend_asm.S | 5 ++--- 3 files changed, 14 insertions(+), 9 deletions(-) diff --git a/arch/loongarch/include/asm/stackframe.h b/arch/loongarch/include/asm/stackframe.h index 96c94035b5d0..f538b3a6d680 100644 --- a/arch/loongarch/include/asm/stackframe.h +++ b/arch/loongarch/include/asm/stackframe.h @@ -216,4 +216,12 @@ RESTORE_SP_AND_RET \docfi .endm +/* Jump to the virtual address of the link. */ + .macro JUMP_LINK_ADDR + li.d t0, CACHE_BASE + pcaddi t1, 0 + or t0, t0, t1 + jirl zero, t0, 0xc + .endm + #endif /* _ASM_STACKFRAME_H */ diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index aa6181714ec3..e8a4bf9d7599 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -52,9 +52,8 @@ SYM_CODE_START(kernel_entry) # kernel entry point /* We might not get launched at the address the kernel is linked to, so we jump there. */ - la.abs t0, 0f - jr t0 -0: + JUMP_LINK_ADDR + /* Enable PG */ li.w t0, 0xb0 # PLV=0, IE=0, PG=1 csrwr t0, LOONGARCH_CSR_CRMD @@ -106,9 +105,8 @@ SYM_CODE_START(smpboot_entry) li.d t0, CSR_DMW1_INIT # CA, PLV0 csrwr t0, LOONGARCH_CSR_DMWIN1 - la.abs t0, 0f - jr t0 -0: + JUMP_LINK_ADDR + /* Enable PG */ li.w t0, 0xb0 # PLV=0, IE=0, PG=1 csrwr t0, LOONGARCH_CSR_CRMD diff --git a/arch/loongarch/power/suspend_asm.S b/arch/loongarch/power/suspend_asm.S index eb2675642f9f..596a682a7924 100644 --- a/arch/loongarch/power/suspend_asm.S +++ b/arch/loongarch/power/suspend_asm.S @@ -78,9 +78,8 @@ SYM_INNER_LABEL(loongarch_wakeup_start, SYM_L_GLOBAL) li.d t0, CSR_DMW1_INIT # CA, PLV0 csrwr t0, LOONGARCH_CSR_DMWIN1 - la.abs t0, 0f - jr t0 -0: + JUMP_LINK_ADDR + la.pcrel t0, acpi_saved_sp ld.d sp, t0, 0 SETUP_WAKEUP From patchwork Mon Jan 9 09:07:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Youling Tang X-Patchwork-Id: 40739 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp2053579wrt; Mon, 9 Jan 2023 01:16:11 -0800 (PST) X-Google-Smtp-Source: AMrXdXvosDnfQXmU8L4Xmit8G6RrSuJODtRSfF3Kk3Ehax+GOGAIqELDxrsRDWR+HKKULYxQdOZS X-Received: by 2002:a05:6a21:32a1:b0:aa:6efd:1883 with SMTP id yt33-20020a056a2132a100b000aa6efd1883mr103575301pzb.37.1673255770896; Mon, 09 Jan 2023 01:16:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673255770; cv=none; d=google.com; s=arc-20160816; b=myRjVde2vs4z7wtUccJl1TXNqoWh4fzfcL8u5nJ9KECr01RnQB72zkbB2hDVMSzKoR pHfb9zgbbPua9PZq0BdBOHgtA/hvBlxBC/R+PHKt+TMfIKhMVs3hbgniL3pSocm6hU95 bJH6HqRseH150npec/80dYeNhAbb3tWD/hyKxx5prYR+DeTqi2rsVJir6JXMahaZMlmp 6THMMsFl7woDhWqXOxBfynpXxPRV/uz3q4ZikFP3yuLIlKglXyNu0wNu7QjLt+MXFaTy H4A0HgZ6D1dzqc8hoLMj16qLt/8lCqzAq9ghumA+pSes+2+Ty6GHPJBde67omPQt5UOE XpTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=yktS2W4aPn8yLvNMOHLSXALAm7X0RwrfFJsdNQzjhhA=; b=K2jKB4wax4wX9AwtkbqXEKNM9TaUqxdIYOI5B2ARPP6yYrewKeBQ7TG97hUVCX54OW fJivcZlKQyfIkx+D+wWrI3+0HWZbUzeWYsncdB1KSconUA18H0U4e7ohq9MUG565E1UO l1t43VnJQo8V3qSFSi1/G5SxhEmYSOdTUPFNDOxWIjqhqgdmyVoE1Y2gjPcRCvgboGpa rV3s5Hr7xSBAyzV/bj6FhtqU6LeqArIYG+fLq3YxaYJiBYolhP1roPeoqE4czLCM9r+g HGztRogm5j6suay6t+i3jY6dwlWalwK7M+HnGkEnxENfOh5cuh8J2B7EBx3Tnhi4fIsw By6g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n64-20020a632743000000b004aebc6eac92si6164092pgn.467.2023.01.09.01.15.57; Mon, 09 Jan 2023 01:16:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237044AbjAIJMk (ORCPT + 99 others); Mon, 9 Jan 2023 04:12:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237006AbjAIJLv (ORCPT ); Mon, 9 Jan 2023 04:11:51 -0500 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 3064C17E2A for ; Mon, 9 Jan 2023 01:08:15 -0800 (PST) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8AxrvB+2btjkHUAAA--.1724S3; Mon, 09 Jan 2023 17:08:14 +0800 (CST) Received: from bogon.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxTuRw2btj_6sWAA--.4042S5; Mon, 09 Jan 2023 17:08:13 +0800 (CST) From: Youling Tang To: Huacai Chen , Xi Ruoyao , Jinyang He Cc: Xuerui Wang , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] LoongArch: Add support for kernel relocation Date: Mon, 9 Jan 2023 17:07:53 +0800 Message-Id: <1673255274-18238-4-git-send-email-tangyouling@loongson.cn> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> References: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> X-CM-TRANSID: AQAAf8DxTuRw2btj_6sWAA--.4042S5 X-CM-SenderInfo: 5wdqw5prxox03j6o00pqjv00gofq/ X-Coremail-Antispam: 1Uk129KBjvJXoW3GrWxKF45uw4UZw4fWF18AFb_yoW3AF43pr W3ZrWkJr48GFnxJ3s2q348uryUZan7Ww43WanrKa4rCF1avF1rXw1vvr9rXFyjqws5Gr4F qFyrKasFva1UAaDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU b28YFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_JrI_Jryl8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW5JVW7JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwA2z4 x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l e2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4CE44I27wAqx4xG64xvF2 IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jw0_WrylYx0Ex4A2jsIE14v26r4j6F4U McvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwIxGrwCFx2 IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v2 6r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67 AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IY s7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr 0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jOdb8UUUUU= X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754535843183827353?= X-GMAIL-MSGID: =?utf-8?q?1754535843183827353?= arch/loongarch/kernel/relocate.c contains the functions necessary to relocate the kernel elsewhere in memory. The kernel makes a copy of itself at the new address. It uses the relocation table inserted by the relocs tool to fix symbol references within the new image. If copy/relocation is successful then the entry point of the new kernel is returned, otherwise fall back to starting the kernel in place. Signed-off-by: Youling Tang --- arch/loongarch/Kconfig | 15 +++++ arch/loongarch/Makefile | 5 ++ arch/loongarch/kernel/Makefile | 2 + arch/loongarch/kernel/head.S | 18 ++++++ arch/loongarch/kernel/relocate.c | 96 +++++++++++++++++++++++++++++ arch/loongarch/kernel/vmlinux.lds.S | 11 +++- 6 files changed, 145 insertions(+), 2 deletions(-) create mode 100644 arch/loongarch/kernel/relocate.c diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 9cc8b84f7eb0..089a4695b1b3 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -48,6 +48,7 @@ config LOONGARCH select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_HUGETLBFS select ARCH_SUPPORTS_NUMA_BALANCING + select SYS_SUPPORTS_RELOCATABLE select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_QUEUED_RWLOCKS @@ -229,6 +230,11 @@ config SCHED_OMIT_FRAME_POINTER config AS_HAS_EXPLICIT_RELOCS def_bool $(as-instr,x:pcalau12i \$t0$(comma)%pc_hi20(x)) +config SYS_SUPPORTS_RELOCATABLE + bool + help + Selected if the platform supports relocating the kernel. + menu "Kernel type and options" source "kernel/Kconfig.hz" @@ -474,6 +480,15 @@ config PHYSICAL_START specified in the "crashkernel=YM@XM" command line boot parameter passed to the panic-ed kernel). +config RELOCATABLE + bool "Relocatable kernel" + depends on SYS_SUPPORTS_RELOCATABLE + help + This builds the kernel as a Position Independent Executable (PIE), + which retains all relocation metadata required to relocate the + kernel binary at runtime to a different virtual address than the + address it was linked at. + config SECCOMP bool "Enable seccomp to safely compute untrusted bytecode" depends on PROC_FS diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile index 4402387d2755..27b5a70ff31c 100644 --- a/arch/loongarch/Makefile +++ b/arch/loongarch/Makefile @@ -71,6 +71,11 @@ KBUILD_AFLAGS_MODULE += -Wa,-mla-global-with-abs KBUILD_CFLAGS_MODULE += -fplt -Wa,-mla-global-with-abs,-mla-local-with-abs endif +ifeq ($(CONFIG_RELOCATABLE),y) +LDFLAGS_vmlinux += -static -pie --no-dynamic-linker -z notext +KBUILD_CFLAGS_KERNEL += -fPIE +endif + cflags-y += -ffreestanding cflags-y += $(call cc-option, -mno-check-zero-division) diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile index fcaa024a685e..33787d22e6f4 100644 --- a/arch/loongarch/kernel/Makefile +++ b/arch/loongarch/kernel/Makefile @@ -31,6 +31,8 @@ endif obj-$(CONFIG_MODULES) += module.o module-sections.o obj-$(CONFIG_STACKTRACE) += stacktrace.o +obj-$(CONFIG_RELOCATABLE) += relocate.o + obj-$(CONFIG_PROC_FS) += proc.o obj-$(CONFIG_SMP) += smp.o diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index e8a4bf9d7599..6db1549177ad 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -88,7 +88,25 @@ SYM_CODE_START(kernel_entry) # kernel entry point PTR_ADD sp, sp, tp set_saved_sp sp, t0, t1 +#ifdef CONFIG_RELOCATABLE + /* Copy kernel and apply the relocations */ + bl relocate_kernel + + /* Repoint the sp into the new kernel image */ + PTR_LI sp, (_THREAD_SIZE - 32 - PT_SIZE) + PTR_ADD sp, sp, tp + set_saved_sp sp, t0, t1 + PTR_ADDI sp, sp, -4 * SZREG # init stack pointer + + /* + * relocate_kernel returns the entry point either + * in the relocated kernel or the original if for + * some reason relocation failed. + */ + jr a0 +#else bl start_kernel +#endif ASM_BUG() SYM_CODE_END(kernel_entry) diff --git a/arch/loongarch/kernel/relocate.c b/arch/loongarch/kernel/relocate.c new file mode 100644 index 000000000000..a58551c0698d --- /dev/null +++ b/arch/loongarch/kernel/relocate.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Support for Kernel relocation at boot time + * + * Copyright (C) 2023 Loongson Technology Corporation Limited + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define RELOCATED(x) ((void *)((long)x + offset)) + +extern long __rela_dyn_start; +extern long __rela_dyn_end; + +/* + * Choose a new address for the kernel, for now we'll hard + * code the destination. + */ +static inline void __init *determine_relocation_address(void) +{ + return (void *)(CACHE_BASE + 0x02000000); +} + +static inline int __init relocation_addr_valid(void *loc_new) +{ + if ((unsigned long)loc_new & 0x0000ffff) { + /* Inappropriately aligned new location */ + return 0; + } + if ((unsigned long)loc_new < (unsigned long)_end) { + /* New location overlaps original kernel */ + return 0; + } + return 1; +} + +void *__init relocate_kernel(void) +{ + Elf64_Rela *rela, *rela_end; + void *loc_new; + unsigned long kernel_length; + long offset = 0; + int res = 1; + /* Default to original kernel entry point */ + void *kernel_entry = start_kernel; + + kernel_length = (long)(_end) - (long)(_text); + + loc_new = determine_relocation_address(); + + /* Sanity check relocation address */ + if (relocation_addr_valid(loc_new)) + offset = (unsigned long)loc_new - (unsigned long)(_text); + + if (offset) { + /* Copy the kernel to it's new location */ + memcpy(loc_new, _text, kernel_length); + + /* Sync the caches ready for execution of new kernel */ + __asm__ __volatile__ ( + "ibar 0 \t\n" + "dbar 0 \t\n"); + + rela = (Elf64_Rela *)RELOCATED(&__rela_dyn_start); + rela_end = (Elf64_Rela *)RELOCATED(&__rela_dyn_end); + + for ( ; rela < rela_end; rela++) { + Elf64_Addr addr = rela->r_offset; + Elf64_Addr relocated_addr = rela->r_addend; + + if (rela->r_info != R_LARCH_RELATIVE) + continue; + + if (relocated_addr >= VMLINUX_LOAD_ADDRESS) + relocated_addr = RELOCATED(relocated_addr); + + *(Elf64_Addr *)RELOCATED(addr) = relocated_addr; + + } + + /* The current thread is now within the relocated image */ + __current_thread_info = RELOCATED(__current_thread_info); + + /* Return the new kernel's entry point */ + kernel_entry = RELOCATED(start_kernel); + } +out: + return kernel_entry; +} diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S index 733b16e8d55d..aec0b6567d24 100644 --- a/arch/loongarch/kernel/vmlinux.lds.S +++ b/arch/loongarch/kernel/vmlinux.lds.S @@ -70,6 +70,8 @@ SECTIONS .plt : ALIGN(16) { *(.plt) } .got.plt : ALIGN(16) { *(.got.plt) } + .data.rel : { *(.data.rel*) } + . = ALIGN(PECOFF_SEGMENT_ALIGN); __init_begin = .; __inittext_begin = .; @@ -93,8 +95,6 @@ SECTIONS PERCPU_SECTION(1 << CONFIG_L1_CACHE_SHIFT) #endif - .rela.dyn : ALIGN(8) { *(.rela.dyn) *(.rela*) } - .init.bss : { *(.init.bss) } @@ -107,6 +107,12 @@ SECTIONS RO_DATA(4096) RW_DATA(1 << CONFIG_L1_CACHE_SHIFT, PAGE_SIZE, THREAD_SIZE) + .rela.dyn : ALIGN(8) { + __rela_dyn_start = .; + *(.rela.dyn) *(.rela*) + __rela_dyn_end = .; + } + .sdata : { *(.sdata) } @@ -133,6 +139,7 @@ SECTIONS DISCARDS /DISCARD/ : { + *(.dynamic .dynsym .dynstr .hash .gnu.hash) *(.gnu.attributes) *(.options) *(.eh_frame) From patchwork Mon Jan 9 09:07:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Youling Tang X-Patchwork-Id: 40738 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp2052835wrt; Mon, 9 Jan 2023 01:14:20 -0800 (PST) X-Google-Smtp-Source: AMrXdXupdoqrxrCuiJoj/JmP/OLUdW54F4dF1sCO7I9n8VO6Fys+3YVcyZOB8c6OhLu7W9n65TPQ X-Received: by 2002:aa7:c2c5:0:b0:498:2223:2df9 with SMTP id m5-20020aa7c2c5000000b0049822232df9mr7408044edp.4.1673255660321; Mon, 09 Jan 2023 01:14:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673255660; cv=none; d=google.com; s=arc-20160816; b=ijWaPQnJ6qAyOM2RmUSPYPxwydKLbu9H/BLLUaLRVdjGNUx/5AnkrsNzs2N/RV1tPZ 4fUyHqNiETf6dKIn9SNqZdwq6eJ6Np9N9D3mB4aXbB5yT8NaVTaPws85UAtOn/ijiEpK ufVahX/Ji2/7AUjbt3HX9R7wb0PAvFQ5dCE/qNJrklTptslb2704HCqOOXaRs/rZa/zL V6YhoX3QMeDyhiF5TyE4oXIIHYY/R1+D5+8QPWVpFMH0OX+U2HBiT8kkSRtsPLwONPhj KR9n4TyjxI8Ozo8AswpQo8rp/Ree3KUK9b8PLlC3lPznBTa3BCby44wHGstsLAHvGsSy xyJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=xL/SaYCmfFoVom3j8et5Xu7UY4g3qYLHVC4a4qLPBxA=; b=USxkeaHEbYn+EC2EYoMgKIHJ9UfkUBeHRenRGDy4tLzAQx3m9R9fBuelvR4O13JGpI CkI+Hip1I4kEnFinvzwhCPaNt+zZrIpjsiHWpdLzIk4B3C5ha4J4l3scMcWsy4dwqucD oTaO/Et5BohiIrtB01J7tXY52Xhbs6VzES167jcPqG0HCGhOdC04WqWG5kfsAB96vge2 cTPA7YZLHrHUGa1d4YhhkzyBLHX4MxJFCZgszxE8SEjsHtYV99CyzY/T6MZa4/77J1js yVmIlQ6hl0bf0COCmzB3eh89zW9MF0llPF5spkjQTYGFOwD3KF3zGcpQv67VsuHYd6TP qP3w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v17-20020a056402185100b00469fd4323a4si8038045edy.246.2023.01.09.01.13.56; Mon, 09 Jan 2023 01:14:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237061AbjAIJMo (ORCPT + 99 others); Mon, 9 Jan 2023 04:12:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234428AbjAIJLv (ORCPT ); Mon, 9 Jan 2023 04:11:51 -0500 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1752B17E39 for ; Mon, 9 Jan 2023 01:08:15 -0800 (PST) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8AxGup+2btjlnUAAA--.1306S3; Mon, 09 Jan 2023 17:08:14 +0800 (CST) Received: from bogon.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxTuRw2btj_6sWAA--.4042S6; Mon, 09 Jan 2023 17:08:14 +0800 (CST) From: Youling Tang To: Huacai Chen , Xi Ruoyao , Jinyang He Cc: Xuerui Wang , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] LoongArch: Add support for kernel address space layout randomization (KASLR) Date: Mon, 9 Jan 2023 17:07:54 +0800 Message-Id: <1673255274-18238-5-git-send-email-tangyouling@loongson.cn> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> References: <1673255274-18238-1-git-send-email-tangyouling@loongson.cn> X-CM-TRANSID: AQAAf8DxTuRw2btj_6sWAA--.4042S6 X-CM-SenderInfo: 5wdqw5prxox03j6o00pqjv00gofq/ X-Coremail-Antispam: 1Uk129KBjvJXoW3XFyxJr1kJF15uw13CF4fAFb_yoW3WryfpF 9rAw4DGF4fGF17JrsrX34Dury5Cws7G343WFsrK34rZFsFqF18ZFWkur9rXF18t39Yqr4S vFZ8JF129w4UAaUanT9S1TB71UUUUUDqnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU bIkYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_JF0_JFyl8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW5JVW7JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwA2z4 x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l e2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4CE44I27wAqx4xG64xvF2 IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jw0_WrylYx0Ex4A2jsIE14v26r4j6F4U McvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwIxGrwCFx2 IqxVCFs4IE7xkEbVWUJVW8JwCFI7km07C267AKxVWUAVWUtwC20s026c02F40E14v26r1j 6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64 vIr41lIxAIcVC0I7IYx2IY67AKxVW5JVW7JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_ Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0x vEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07j5l1kUUUUU= X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754535727197975070?= X-GMAIL-MSGID: =?utf-8?q?1754535727197975070?= This patch adds support for relocating the kernel to a random address. Entropy is derived from the banner, which will change every build and random_get_entropy() which should provide additional runtime entropy. The kernel is relocated by up to RANDOMIZE_BASE_MAX_OFFSET bytes from its link address. Because relocation happens so early in the kernel boot, the amount of physical memory has not yet been determined. This means the only way to limit relocation within the available memory is via Kconfig. Limit the maximum value of RANDOMIZE_BASE_MAX_OFFSET to 256M(0x10000000) because our memory layout has many holes. KERNELOFFSET (kaslr_offset) is added to vmcoreinfo in the future, for crash --kaslr support. Signed-off-by: Youling Tang --- arch/loongarch/Kconfig | 22 ++++++ arch/loongarch/include/asm/page.h | 6 ++ arch/loongarch/kernel/relocate.c | 114 ++++++++++++++++++++++++++++++ arch/loongarch/kernel/setup.c | 3 + 4 files changed, 145 insertions(+) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 089a4695b1b3..cdf0535764ae 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -489,6 +489,28 @@ config RELOCATABLE kernel binary at runtime to a different virtual address than the address it was linked at. +config RANDOMIZE_BASE + bool "Randomize the address of the kernel image (KASLR)" + depends on RELOCATABLE + help + Randomizes the physical and virtual address at which the + kernel image is loaded, as a security feature that + deters exploit attempts relying on knowledge of the location + of kernel internals. + + The kernel will be offset by up to RANDOMIZE_BASE_MAX_OFFSET. + + If unsure, say N. + +config RANDOMIZE_BASE_MAX_OFFSET + hex "Maximum KASLR offset" if EXPERT + depends on RANDOMIZE_BASE + range 0x0 0x10000000 if 64BIT + default "0x01000000" + help + When KASLR is active, this provides the maximum offset that will + be applied to the kernel image. + config SECCOMP bool "Enable seccomp to safely compute untrusted bytecode" depends on PROC_FS diff --git a/arch/loongarch/include/asm/page.h b/arch/loongarch/include/asm/page.h index 53f284a96182..6dda0d6271ca 100644 --- a/arch/loongarch/include/asm/page.h +++ b/arch/loongarch/include/asm/page.h @@ -106,6 +106,12 @@ extern int __virt_addr_valid(volatile void *kaddr); ((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0) | \ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) +extern unsigned long __kaslr_offset; +static inline unsigned long kaslr_offset(void) +{ + return __kaslr_offset; +} + #include #include diff --git a/arch/loongarch/kernel/relocate.c b/arch/loongarch/kernel/relocate.c index a58551c0698d..b5070a015dd4 100644 --- a/arch/loongarch/kernel/relocate.c +++ b/arch/loongarch/kernel/relocate.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -19,6 +20,70 @@ extern long __rela_dyn_start; extern long __rela_dyn_end; +#ifdef CONFIG_RANDOMIZE_BASE + +static inline __init unsigned long rotate_xor(unsigned long hash, + const void *area, size_t size) +{ + size_t i; + unsigned long *ptr = (unsigned long *)area; + + for (i = 0; i < size / sizeof(hash); i++) { + /* Rotate by odd number of bits and XOR. */ + hash = (hash << ((sizeof(hash) * 8) - 7)) | (hash >> 7); + hash ^= ptr[i]; + } + + return hash; +} + +static inline __init unsigned long get_random_boot(void) +{ + unsigned long entropy = random_get_entropy(); + unsigned long hash = 0; + + /* Attempt to create a simple but unpredictable starting entropy. */ + hash = rotate_xor(hash, linux_banner, strlen(linux_banner)); + + /* Add in any runtime entropy we can get */ + hash = rotate_xor(hash, &entropy, sizeof(entropy)); + + return hash; +} + +static inline __init bool kaslr_disabled(void) +{ + char *str; + + str = strstr(boot_command_line, "nokaslr"); + if (str == boot_command_line || (str > boot_command_line && *(str - 1) == ' ')) + return true; + + return false; +} + +/* Choose a new address for the kernel */ +static inline void __init *determine_relocation_address(void) +{ + unsigned long kernel_length; + void *dest = _text; + unsigned long offset; + + if (kaslr_disabled()) + return dest; + + kernel_length = (long)_end - (long)_text; + + offset = get_random_boot() << 16; + offset &= (CONFIG_RANDOMIZE_BASE_MAX_OFFSET - 1); + if (offset < kernel_length) + offset += ALIGN(kernel_length, 0xffff); + + return RELOCATED(dest); +} + +#else + /* * Choose a new address for the kernel, for now we'll hard * code the destination. @@ -28,6 +93,8 @@ static inline void __init *determine_relocation_address(void) return (void *)(CACHE_BASE + 0x02000000); } +#endif + static inline int __init relocation_addr_valid(void *loc_new) { if ((unsigned long)loc_new & 0x0000ffff) { @@ -41,6 +108,13 @@ static inline int __init relocation_addr_valid(void *loc_new) return 1; } +static inline void __init update_kaslr_offset(unsigned long *addr, long offset) +{ + unsigned long *new_addr = (unsigned long *)RELOCATED(addr); + + *new_addr = (unsigned long)offset; +} + void *__init relocate_kernel(void) { Elf64_Rela *rela, *rela_end; @@ -50,6 +124,10 @@ void *__init relocate_kernel(void) int res = 1; /* Default to original kernel entry point */ void *kernel_entry = start_kernel; + char *cmdline = early_ioremap(fw_arg1, COMMAND_LINE_SIZE); + + /* Boot command line was passed in fw_arg1 */ + strscpy(boot_command_line, cmdline, COMMAND_LINE_SIZE); kernel_length = (long)(_end) - (long)(_text); @@ -90,7 +168,43 @@ void *__init relocate_kernel(void) /* Return the new kernel's entry point */ kernel_entry = RELOCATED(start_kernel); + + /* Error may occur before, so keep it at last */ + update_kaslr_offset(&__kaslr_offset, offset); } out: return kernel_entry; } + +/* + * Show relocation information on panic. + */ +static void show_kernel_relocation(const char *level) +{ + if (__kaslr_offset > 0) { + printk(level); + pr_cont("Kernel relocated offset @ 0x%lx\n", __kaslr_offset); + pr_cont(" .text @ 0x%lx\n", (unsigned long)&_text); + pr_cont(" .data @ 0x%lx\n", (unsigned long)&_sdata); + pr_cont(" .bss @ 0x%lx\n", (unsigned long)&__bss_start); + } +} + +static int kernel_location_notifier_fn(struct notifier_block *self, + unsigned long v, void *p) +{ + show_kernel_relocation(KERN_EMERG); + return NOTIFY_DONE; +} + +static struct notifier_block kernel_location_notifier = { + .notifier_call = kernel_location_notifier_fn +}; + +static int __init register_kernel_offset_dumper(void) +{ + atomic_notifier_chain_register(&panic_notifier_list, + &kernel_location_notifier); + return 0; +} +__initcall(register_kernel_offset_dumper); diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index 4344502c0b31..b2ba2741945c 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -82,6 +82,9 @@ static struct resource code_resource = { .name = "Kernel code", }; static struct resource data_resource = { .name = "Kernel data", }; static struct resource bss_resource = { .name = "Kernel bss", }; +unsigned long __kaslr_offset __ro_after_init; +EXPORT_SYMBOL(__kaslr_offset); + const char *get_system_type(void) { return "generic-loongson-machine";