From patchwork Tue Apr 4 08:41:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Qing Zhang X-Patchwork-Id: 78927 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2869065vqo; Tue, 4 Apr 2023 01:43:02 -0700 (PDT) X-Google-Smtp-Source: AKy350YQ1Puzah+rTwEtuX0BLXVk1GqeGnhtr+c0VcP618jPLFKCC576Qk5npLiO0qLkvDK2cajU X-Received: by 2002:a05:6a20:791e:b0:d5:f7f5:85e4 with SMTP id b30-20020a056a20791e00b000d5f7f585e4mr1168758pzg.22.1680597782513; Tue, 04 Apr 2023 01:43:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680597782; cv=none; d=google.com; s=arc-20160816; b=XeotcCXzUQpiRgPXNAkPh+D8HtjnDnCr3ulNW36EVl9Clnk9V+epaf3KoT0AoMfObH X9a+PRePSn0vOx5dLECepupx9I/v8RDyhtO6HDgCjHqDiHVxamlguJCSeixaZrmcVyhT 3OoZQY7x4TIjPOXJbmnKl8UyeU5Tacu7+nnw8t6fH4Cb9fDxo0w8/Vbn9jgex2LJcdZE gPYUV/BJjLfQ2m+32jamIOk/N4pFes3RC42qaiQip07GGBR7gNV68zwFWzn8s0UlU7if Uwpivl3ExREyRBmI5HPvIzKu++2nWj4Mm7tQ+uX3gnYQO8Tzo/bZvtecNUpmwl3A+Dg0 c0Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=bK9gLMxjKISsmIbyAXMDAt9NQo6Xkgz6c17HqxCv4xE=; b=0vyw2eAkBJ6n8k7ocIVYikyjSkUpoM8uNnvzbjEBe5+3dXA6LkipHpi147xVv+VgVj Mx3FaEjc8AaiRdDQG6ueG7cpccOAf/CSmHClXBR+CBg0DUVoBEsScMnUkyPLgDrVDWBa sSOs9q9rrlQcV7E0FeC5LJ/OmDP3wgg3qLZyjWERokSHfukxPO+cPaXV4diR5PM4qTWS b30U9D3VAdVHI/kEXIxcF8E87Q7o0jWEehKFED6RADoiqRUCADsJeailSCRzXHxELTbP dbgYSE3RpMha6d3drAlSHuQ0T41G3z5tHYdnwLZeaaZRgbSQc0TuaUko7vX9QBbeoKP4 AR8Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x22-20020aa79ad6000000b00627fbc1cb02si10154689pfp.338.2023.04.04.01.42.49; Tue, 04 Apr 2023 01:43:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233699AbjDDImM (ORCPT + 99 others); Tue, 4 Apr 2023 04:42:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229693AbjDDImK (ORCPT ); Tue, 4 Apr 2023 04:42:10 -0400 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 74DC21FC0; Tue, 4 Apr 2023 01:42:03 -0700 (PDT) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8Bxok7a4itkQV0WAA--.22927S3; Tue, 04 Apr 2023 16:42:02 +0800 (CST) Received: from localhost.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxwOTW4itkYRYVAA--.55009S3; Tue, 04 Apr 2023 16:42:00 +0800 (CST) From: Qing Zhang To: Andrey Ryabinin , Jonathan Corbet , Huacai Chen , Andrew Morton Cc: Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , WANG Xuerui , Jiaxun Yang , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-mm@kvack.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 1/6] LoongArch: Simplified randomization layout after jump new kernel processing Date: Tue, 4 Apr 2023 16:41:43 +0800 Message-Id: <20230404084148.744-2-zhangqing@loongson.cn> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230404084148.744-1-zhangqing@loongson.cn> References: <20230404084148.744-1-zhangqing@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxwOTW4itkYRYVAA--.55009S3 X-CM-SenderInfo: x2kd0wptlqwqxorr0wxvrqhubq/ X-Coremail-Antispam: 1Uk129KBjvJXoWxCFy5JFyUAw4rur18urW7Arb_yoW5AF45pr y7Zw1kJr45Grs7J34qqa4Dury5XwnrWw1aganrK34rZr12qFy5Xw1kurnrWFWjq3yFgr4S qFyrKF9Iva1UJ3DanT9S1TB71UUUUb7qnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU bSxYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_JF0_JFyl8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW5JVW7JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwA2z4 x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l n4kS14v26r1q6r43M2AIxVAIcxkEcVAq07x20xvEncxIr21l57IF6xkI12xvs2x26I8E6x ACxx1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1q6rW5McIj6I8E 87Iv67AKxVW8JVWxJwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lc7CjxV Aaw2AFwI0_Jw0_GFyl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1l4IxY O2xFxVAFwI0_Jw0_GFylx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGV WUWwC2zVAF1VAY17CE14v26r4a6rW5MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_ Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rV WUJVWUCwCI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4U JbIYCTnIWIevJa73UjIFyTuYvjxUsw0eDUUUU X-Spam-Status: No, score=-0.0 required=5.0 tests=SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762234500251005223?= X-GMAIL-MSGID: =?utf-8?q?1762234500251005223?= Modified relocate_kernel is not returned directly new kernel's entry point， instead, we share start_kernel processing with the normal kernel, which avoids calling 'jr a0' directly and we can do other operations(eg: kasan_early_init) before start_kernel when CONFIG_RANDOMIZE_BASE is turned on. Signed-off-by: Qing Zhang --- arch/loongarch/include/asm/setup.h | 2 +- arch/loongarch/kernel/head.S | 10 +++++----- arch/loongarch/kernel/relocate.c | 8 ++------ 3 files changed, 8 insertions(+), 12 deletions(-) diff --git a/arch/loongarch/include/asm/setup.h b/arch/loongarch/include/asm/setup.h index be05c0e706a2..2dca0d1dd90a 100644 --- a/arch/loongarch/include/asm/setup.h +++ b/arch/loongarch/include/asm/setup.h @@ -33,7 +33,7 @@ extern long __la_abs_end; extern long __rela_dyn_begin; extern long __rela_dyn_end; -extern void * __init relocate_kernel(void); +extern unsigned long __init relocate_kernel(void); #endif diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index aa64b179744f..35c4a78614c3 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -95,12 +95,12 @@ SYM_CODE_START(kernel_entry) # kernel entry point PTR_LI sp, (_THREAD_SIZE - PT_SIZE) PTR_ADD sp, sp, tp set_saved_sp sp, t0, t1 -#endif - - /* relocate_kernel() returns the new kernel entry point */ - jr a0 - ASM_BUG() + /* Jump to new kernel: new_pc = current_pc + random_offset */ + pcaddi t0, 0 + add.d t0, t0, a0 + jirl zero, t0, 0xc +#endif #endif bl start_kernel diff --git a/arch/loongarch/kernel/relocate.c b/arch/loongarch/kernel/relocate.c index 01f94d1e3edf..6c3eff9af9fb 100644 --- a/arch/loongarch/kernel/relocate.c +++ b/arch/loongarch/kernel/relocate.c @@ -157,12 +157,11 @@ static inline void __init update_reloc_offset(unsigned long *addr, long random_o *new_addr = (unsigned long)reloc_offset; } -void * __init relocate_kernel(void) +unsigned long __init relocate_kernel(void) { unsigned long kernel_length; unsigned long random_offset = 0; void *location_new = _text; /* Default to original kernel start */ - void *kernel_entry = start_kernel; /* Default to original kernel entry point */ char *cmdline = early_ioremap(fw_arg1, COMMAND_LINE_SIZE); /* Boot command line is passed in fw_arg1 */ strscpy(boot_command_line, cmdline, COMMAND_LINE_SIZE); @@ -190,9 +189,6 @@ void * __init relocate_kernel(void) reloc_offset += random_offset; - /* Return the new kernel's entry point */ - kernel_entry = RELOCATED_KASLR(start_kernel); - /* The current thread is now within the relocated kernel */ __current_thread_info = RELOCATED_KASLR(__current_thread_info); @@ -204,7 +200,7 @@ void * __init relocate_kernel(void) relocate_absolute(random_offset); - return kernel_entry; + return random_offset; } /* From patchwork Tue Apr 4 08:41:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qing Zhang X-Patchwork-Id: 78935 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2874388vqo; Tue, 4 Apr 2023 01:56:23 -0700 (PDT) X-Google-Smtp-Source: AKy350bFrWyDrHWoruM5gmbKNmeZMCOqmmwFJm5JQVSUrAItcOzCAEwM0hZDYoGn9pufWYJlqQHx X-Received: by 2002:aa7:c44f:0:b0:500:58cb:3b05 with SMTP id n15-20020aa7c44f000000b0050058cb3b05mr1674840edr.27.1680598583497; Tue, 04 Apr 2023 01:56:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680598583; cv=none; d=google.com; s=arc-20160816; b=c04LiWPInIr2wLhs+eIVZWE4rgYWXnFplWfVpYgoG4WSf34YXzvU2N39pmYUF2M/hz 1cex2u8jqLpCUrY5qYldUTmlsMdoxRcJsQStnt6FjLBm3jXsN0xUSiCTkXA58WTeDUwJ sa8C6g67VVUL1UfOs0FKoUEfEPKgQO0AN/4cOSn+5WnPtx3gwB9bEDdqoVVGbanB3xGP XHrdGc9j2ka4K0p+q9YK2Z7dnESopNZGEoYpHgFpo5UpB08rBb1kGOOl8uSVVN6RJzqv 8Ze74q0d7aVZkqnX9m1e1xulaRkOu11k4SdPxumxAD3p26BgZXcZH15UAnmiu4AYHWJe v0eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=zUqPHNvm72b1Z7rJDfbqVQjdQ+2h29X6/VA4g8xsVbE=; b=0XboKIIE90QCwuuYAxOZ/qwIb2tkkCMGiPVO+Q1GbNTTdkIuE5Z2Rz9zT/VNmapdVE T9zlNDiSDRGKwXuDxqSsF1B7m4R1sCemG6QBLTtjyfewvBtUUy6zF6lILlccZUacg1Tp pnVf71VjMn/lytqFGbC2ADzvXv5NigwVXNNsnK4Az/PrP6Y7ve5FhCCcchMR1q1YXPwc tIJ3QiaxpYw9+z24MzdfcRq8vXVpxOgSgzM6ZujwU8RGyLjH1yzIQ39fOkZTct6ADnMx quSuK4zKtqNArbOLl5TE9twVBZPhLEziPjLkEyiC8LOZe7VOlmXqfHBQ1ygog88M60dz YVCw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f7-20020a50ee87000000b004fe5ea74b93si646810edr.349.2023.04.04.01.55.59; Tue, 04 Apr 2023 01:56:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233916AbjDDImP (ORCPT + 99 others); Tue, 4 Apr 2023 04:42:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229632AbjDDImK (ORCPT ); Tue, 4 Apr 2023 04:42:10 -0400 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 3C8261BE6; Tue, 4 Apr 2023 01:42:05 -0700 (PDT) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8Bxedna4itkT10WAA--.34526S3; Tue, 04 Apr 2023 16:42:02 +0800 (CST) Received: from localhost.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxwOTW4itkYRYVAA--.55009S4; Tue, 04 Apr 2023 16:42:02 +0800 (CST) From: Qing Zhang To: Andrey Ryabinin , Jonathan Corbet , Huacai Chen , Andrew Morton Cc: Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , WANG Xuerui , Jiaxun Yang , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-mm@kvack.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 2/6] LoongArch: Fix _CONST64_(x) as unsigned Date: Tue, 4 Apr 2023 16:41:44 +0800 Message-Id: <20230404084148.744-3-zhangqing@loongson.cn> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230404084148.744-1-zhangqing@loongson.cn> References: <20230404084148.744-1-zhangqing@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxwOTW4itkYRYVAA--.55009S4 X-CM-SenderInfo: x2kd0wptlqwqxorr0wxvrqhubq/ X-Coremail-Antispam: 1Uk129KBjvdXoWrtF4ktr4UAr4fXr4UJF4xZwb_yoW3WFX_Aa 97Ja1kur48AFW7Aws0y34rJw1Uu3ykJFn8uFnIqr9xAwn0kw45Jay8Wa4rZryakF4a9rs5 XFWvqr9Iy34UtjkaLaAFLSUrUUUUYb8apTn2vfkv8UJUUUU8wcxFpf9Il3svdxBIdaVrn0 xqx4xG64xvF2IEw4CE5I8CrVC2j2Jv73VFW2AGmfu7bjvjm3AaLaJ3UjIYCTnIWjp_UUUY Z7kC6x804xWl14x267AKxVWUJVW8JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0rVWrJVCq3w AFIxvE14AKwVWUXVWUAwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l84ACjcxK 6xIIjxv20xvE14v26ryj6F1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwA2z4 x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l n4kS14v26r1q6r43M2AIxVAIcxkEcVAq07x20xvEncxIr21l57IF6xkI12xvs2x26I8E6x ACxx1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26rWY6Fy7McIj6I8E 87Iv67AKxVW8JVWxJwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lc7CjxV Aaw2AFwI0_Jw0_GFyl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1l4IxY O2xFxVAFwI0_Jw0_GFylx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGV WUWwC2zVAF1VAY17CE14v26r4a6rW5MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_ Xr0_Ar1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rV WUJVWUCwCI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4U JbIYCTnIWIevJa73UjIFyTuYvjxU-KsUDUUUU X-Spam-Status: No, score=-0.0 required=5.0 tests=SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762235340279361547?= X-GMAIL-MSGID: =?utf-8?q?1762235340279361547?= Addresses should all be of unsigned type to avoid unnecessary conversions. Signed-off-by: Qing Zhang --- arch/loongarch/include/asm/addrspace.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/loongarch/include/asm/addrspace.h b/arch/loongarch/include/asm/addrspace.h index 8fb699b4d40a..5c9c03bdf915 100644 --- a/arch/loongarch/include/asm/addrspace.h +++ b/arch/loongarch/include/asm/addrspace.h @@ -71,9 +71,9 @@ extern unsigned long vm_map_base; #define _ATYPE32_ int #define _ATYPE64_ __s64 #ifdef CONFIG_64BIT -#define _CONST64_(x) x ## L +#define _CONST64_(x) x ## UL #else -#define _CONST64_(x) x ## LL +#define _CONST64_(x) x ## ULL #endif #endif From patchwork Tue Apr 4 08:41:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Qing Zhang X-Patchwork-Id: 78934 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2874171vqo; Tue, 4 Apr 2023 01:55:43 -0700 (PDT) X-Google-Smtp-Source: AKy350bj37QfAD7JoM8hQqJsYC50gqXeGixfJ3oZdyw08vB6/JluYf/7Dt4txbRGo5xPeLZjHFtx X-Received: by 2002:a17:906:c04b:b0:939:e7d6:7c52 with SMTP id bm11-20020a170906c04b00b00939e7d67c52mr1340718ejb.20.1680598542951; Tue, 04 Apr 2023 01:55:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680598542; cv=none; d=google.com; s=arc-20160816; b=ksbQZ7vWphtHpq4F6ud2gcbJw0+PJS8XZ22rCF3C/CNEenlPxIzYaPfLtrDDRCJv24 EGl+zueI16skuZ98ev/Pt3X/xiiH5D0szwcz8Z/7nE32SYXCNscM117/p7axrkC4dyKR gN00+cPh96H5iR3jY1I221bTMdZra/VZZlwV8vsTAU53C6tgXlKyuM5GbBdXzIqXRxG/ hrFu9b+a4DNnCpCjeEWVuXkSos2JY4YfQao264BeIRb+qJJb6B741f2m/SLTlUnIPRWW +9GoBDGy6KjhNpm9uMLmKh09w5tm8lfu3nFEgOjglPeKKZViegDE9AZtHsMuzYiVBlFL xrJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=jOQ3AYWQK/1sCEZyCTvYw6YoJnX4qY8pvzTbakVou/w=; b=f03a7cnYO232h0lQR+tyUkK0Loqy5DyE3rHjGRu8qwDduiuBZ1Qf9sh6HV3Q7bRKHh XQK3fbM/0Qj+3fup2uvZHe5Cz7Prd8x0le3UaFKgFyuopoKCe7zv4atPQ5hdWQ4/3jRb aNkS022rUP+kWLsKBGjBxjBivo6ZgwBuwxKIf4C+LVpr+IjAPWVwA7onwJB3yBLeKu+9 1kgFZ+UKnDNWRUv4klhs+psCxLizbu2oEwdY5DUI8Mc0FsJ/ZR7K9kZca22gJuPY9G4v vuuv+CXWI6XEmGDzlemEO1ADeHFywl+u1eiBTSurwNSJuDn/Bgv23ryoRF6hkslImW2R u7qQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m4-20020aa7c484000000b004ad0993e54esi1401883edq.487.2023.04.04.01.55.19; Tue, 04 Apr 2023 01:55:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234003AbjDDImX (ORCPT + 99 others); Tue, 4 Apr 2023 04:42:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233526AbjDDImL (ORCPT ); Tue, 4 Apr 2023 04:42:11 -0400 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 406091FC6; Tue, 4 Apr 2023 01:42:06 -0700 (PDT) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8AxEk7c4itkXF0WAA--.23064S3; Tue, 04 Apr 2023 16:42:04 +0800 (CST) Received: from localhost.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8DxwOTW4itkYRYVAA--.55009S5; Tue, 04 Apr 2023 16:42:03 +0800 (CST) From: Qing Zhang To: Andrey Ryabinin , Jonathan Corbet , Huacai Chen , Andrew Morton Cc: Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , WANG Xuerui , Jiaxun Yang , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-mm@kvack.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 3/6] LoongArch: Add kernel address sanitizer support Date: Tue, 4 Apr 2023 16:41:45 +0800 Message-Id: <20230404084148.744-4-zhangqing@loongson.cn> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230404084148.744-1-zhangqing@loongson.cn> References: <20230404084148.744-1-zhangqing@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: AQAAf8DxwOTW4itkYRYVAA--.55009S5 X-CM-SenderInfo: x2kd0wptlqwqxorr0wxvrqhubq/ X-Coremail-Antispam: 1Uk129KBjvAXoWfWr18GrW5WF1xCr1rKw17ZFb_yoW5JFy3Ao WYkF43K348Kw12k393Ww1UJFyUtr1qkF4fA3sFvF1fWF17ArW3G34UKFWSq343JrZYkr1f Wayvgrsav3s5trnxn29KB7ZKAUJUUUUk529EdanIXcx71UUUUU7KY7ZEXasCq-sGcSsGvf J3Ic02F40EFcxC0VAKzVAqx4xG6I80ebIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnRJU UUBSb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I20VC2zVCF04k26cxKx2IYs7xG6rWj6s 0DM7CIcVAFz4kK6r1Y6r17M28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UM2 8EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j6F4U JwAaw2AFwI0_Jw0_GFyle2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4 CE44I27wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Wrv_ZF1lYx0E x4A2jsIE14v26r4j6F4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCY1x 0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwCF I7km07C267AKxVWUtVW8ZwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r 106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AK xVW7JVWDJwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcI k0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4j 6r4UJbIYCTnIWIevJa73UjIFyTuYvjxU-KsUDUUUU X-Spam-Status: No, score=-0.0 required=5.0 tests=SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762235231485366252?= X-GMAIL-MSGID: =?utf-8?q?1762235297966606813?= 1/8 of kernel addresses reserved for shadow memory. But for LoongArch, There are a lot of holes between different segments and valid address space(256T available) is insufficient to map all these segments to kasan shadow memory with the common formula provided by kasan core, saying addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET So LoongArch has a ARCH specific mapping formula,different segments are mapped individually, and only limited length of space of that specific segment is mapped to shadow. At early boot stage the whole shadow region populated with just one physical page (kasan_early_shadow_page). Later, this page is reused as readonly zero shadow for some memory that Kasan currently don't track. After mapping the physical memory, pages for shadow memory are allocated and mapped. Functions like memset/memmove/memcpy do a lot of memory accesses. If bad pointer passed to one of these function it is important to catch this. Compiler's instrumentation cannot do this since these functions are written in assembly. KASan replaces memory functions with manually instrumented variants. Original functions declared as weak symbols so strong definitions in mm/kasan/kasan.c could replace them. Original functions have aliases with '__' prefix in name, so we could call non-instrumented variant if needed. Signed-off-by: Qing Zhang --- Documentation/dev-tools/kasan.rst | 4 +- .../features/debug/KASAN/arch-support.txt | 2 +- .../translations/zh_CN/dev-tools/kasan.rst | 2 +- arch/loongarch/Kconfig | 7 + arch/loongarch/include/asm/kasan.h | 125 +++++++++ arch/loongarch/include/asm/pgtable.h | 7 + arch/loongarch/include/asm/string.h | 16 ++ arch/loongarch/kernel/Makefile | 3 + arch/loongarch/kernel/head.S | 4 + arch/loongarch/kernel/setup.c | 4 + arch/loongarch/lib/memcpy.S | 4 +- arch/loongarch/lib/memmove.S | 13 +- arch/loongarch/lib/memset.S | 4 +- arch/loongarch/mm/Makefile | 2 + arch/loongarch/mm/kasan_init.c | 255 ++++++++++++++++++ arch/loongarch/vdso/Makefile | 4 + 16 files changed, 445 insertions(+), 11 deletions(-) create mode 100644 arch/loongarch/include/asm/kasan.h create mode 100644 arch/loongarch/mm/kasan_init.c diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index e66916a483cd..ee91f2872767 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -41,8 +41,8 @@ Support Architectures ~~~~~~~~~~~~~ -Generic KASAN is supported on x86_64, arm, arm64, powerpc, riscv, s390, and -xtensa, and the tag-based KASAN modes are supported only on arm64. +Generic KASAN is supported on x86_64, arm, arm64, powerpc, riscv, s390, xtensa, +and loongarch, and the tag-based KASAN modes are supported only on arm64. Compilers ~~~~~~~~~ diff --git a/Documentation/features/debug/KASAN/arch-support.txt b/Documentation/features/debug/KASAN/arch-support.txt index bf0124fae643..c4581c2edb28 100644 --- a/Documentation/features/debug/KASAN/arch-support.txt +++ b/Documentation/features/debug/KASAN/arch-support.txt @@ -13,7 +13,7 @@ | csky: | TODO | | hexagon: | TODO | | ia64: | TODO | - | loongarch: | TODO | + | loongarch: | ok | | m68k: | TODO | | microblaze: | TODO | | mips: | TODO | diff --git a/Documentation/translations/zh_CN/dev-tools/kasan.rst b/Documentation/translations/zh_CN/dev-tools/kasan.rst index 05ef904dbcfb..8fdb20c9665b 100644 --- a/Documentation/translations/zh_CN/dev-tools/kasan.rst +++ b/Documentation/translations/zh_CN/dev-tools/kasan.rst @@ -42,7 +42,7 @@ KASAN有三种模式: 体系架构 ~~~~~~~~ -在x86_64、arm、arm64、powerpc、riscv、s390和xtensa上支持通用KASAN， +在x86_64、arm、arm64、powerpc、riscv、s390、xtensa和loongarch上支持通用KASAN， 而基于标签的KASAN模式只在arm64上支持。 编译器 diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 72dd00f48b8c..61f883c51045 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -7,6 +7,7 @@ config LOONGARCH select ACPI_MCFG if ACPI select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI select ARCH_BINFMT_ELF_STATE + select ARCH_DISABLE_KASAN_INLINE select ARCH_ENABLE_MEMORY_HOTPLUG select ARCH_ENABLE_MEMORY_HOTREMOVE select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI @@ -83,6 +84,7 @@ config LOONGARCH select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_SECCOMP_FILTER + select HAVE_ARCH_KASAN if 64BIT select HAVE_ARCH_TRACEHOOK select HAVE_ARCH_TRANSPARENT_HUGEPAGE select HAVE_ASM_MODVERSIONS @@ -626,6 +628,11 @@ config ARCH_MMAP_RND_BITS_MIN config ARCH_MMAP_RND_BITS_MAX default 18 +config KASAN_SHADOW_OFFSET + hex + default 0x0 + depends on KASAN + menu "Power management options" config ARCH_SUSPEND_POSSIBLE diff --git a/arch/loongarch/include/asm/kasan.h b/arch/loongarch/include/asm/kasan.h new file mode 100644 index 000000000000..bc847df2de49 --- /dev/null +++ b/arch/loongarch/include/asm/kasan.h @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_H +#define __ASM_KASAN_H + +#ifndef __ASSEMBLY__ + +#include +#include +#include +#include +#include + +#define __HAVE_ARCH_SHADOW_MAP + +#define KASAN_SHADOW_SCALE_SHIFT 3 +#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) + +#define XRANGE_SHIFT (48) + +/* Valid address length */ +#define XRANGE_SHADOW_SHIFT (PGDIR_SHIFT + PAGE_SHIFT - 3) +/* Used for taking out the valid address */ +#define XRANGE_SHADOW_MASK GENMASK_ULL(XRANGE_SHADOW_SHIFT - 1, 0) +/* One segment whole address space size */ +#define XRANGE_SIZE (XRANGE_SHADOW_MASK + 1) + +/* 64-bit segment value. */ +#define XKPRANGE_UC_SEG (UNCACHE_BASE >> DMW_PABITS) +#define XKPRANGE_CC_SEG (CACHE_BASE >> DMW_PABITS) +#define XKVRANGE_VC_SEG (0xffff) + +/* Cached */ +#define XKPRANGE_CC_START CACHE_BASE +#define XKPRANGE_CC_SIZE XRANGE_SIZE +#define XKPRANGE_CC_KASAN_OFFSET (0) +#define XKPRANGE_CC_SHADOW_SIZE (XKPRANGE_CC_SIZE >> KASAN_SHADOW_SCALE_SHIFT) +#define XKPRANGE_CC_SHADOW_END (XKPRANGE_CC_KASAN_OFFSET + XKPRANGE_CC_SHADOW_SIZE) + +/* UnCached */ +#define XKPRANGE_UC_START UNCACHE_BASE +#define XKPRANGE_UC_SIZE XRANGE_SIZE +#define XKPRANGE_UC_KASAN_OFFSET XKPRANGE_CC_SHADOW_END +#define XKPRANGE_UC_SHADOW_SIZE (XKPRANGE_UC_SIZE >> KASAN_SHADOW_SCALE_SHIFT) +#define XKPRANGE_UC_SHADOW_END (XKPRANGE_UC_KASAN_OFFSET + XKPRANGE_UC_SHADOW_SIZE) + +/* VMALLOC (Cached or UnCached) */ +#define XKVRANGE_VC_START MODULES_VADDR +#define XKVRANGE_VC_SIZE round_up(VMEMMAP_END - MODULES_VADDR + 1, PGDIR_SIZE) +#define XKVRANGE_VC_KASAN_OFFSET XKPRANGE_UC_SHADOW_END +#define XKVRANGE_VC_SHADOW_SIZE (XKVRANGE_VC_SIZE >> KASAN_SHADOW_SCALE_SHIFT) +#define XKVRANGE_VC_SHADOW_END (XKVRANGE_VC_KASAN_OFFSET + XKVRANGE_VC_SHADOW_SIZE) + +/* Kasan shadow memory start right after vmalloc. */ +#define KASAN_SHADOW_START round_up(VMEMMAP_END, PGDIR_SIZE) +#define KASAN_SHADOW_SIZE (XKVRANGE_VC_SHADOW_END - XKPRANGE_CC_KASAN_OFFSET) +#define KASAN_SHADOW_END round_up(KASAN_SHADOW_START + KASAN_SHADOW_SIZE, PGDIR_SIZE) + +#define XKPRANGE_CC_SHADOW_OFFSET (KASAN_SHADOW_START + XKPRANGE_CC_KASAN_OFFSET) +#define XKPRANGE_UC_SHADOW_OFFSET (KASAN_SHADOW_START + XKPRANGE_UC_KASAN_OFFSET) +#define XKVRANGE_VC_SHADOW_OFFSET (KASAN_SHADOW_START + XKVRANGE_VC_KASAN_OFFSET) + +extern bool kasan_early_stage; +extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; + +static inline void *kasan_mem_to_shadow(const void *addr) +{ + if (kasan_early_stage) { + return (void *)(kasan_early_shadow_page); + } else { + unsigned long maddr = (unsigned long)addr; + unsigned long xrange = (maddr >> XRANGE_SHIFT) & 0xffff; + unsigned long offset = 0; + + maddr &= XRANGE_SHADOW_MASK; + switch (xrange) { + case XKPRANGE_CC_SEG: + offset = XKPRANGE_CC_SHADOW_OFFSET; + break; + case XKPRANGE_UC_SEG: + offset = XKPRANGE_UC_SHADOW_OFFSET; + break; + case XKVRANGE_VC_SEG: + offset = XKVRANGE_VC_SHADOW_OFFSET; + break; + default: + WARN_ON(1); + return NULL; + } + + return (void *)((maddr >> KASAN_SHADOW_SCALE_SHIFT) + offset); + } +} + +static inline const void *kasan_shadow_to_mem(const void *shadow_addr) +{ + unsigned long addr = (unsigned long)shadow_addr; + + if (unlikely(addr > KASAN_SHADOW_END) || + unlikely(addr < KASAN_SHADOW_START)) { + WARN_ON(1); + return NULL; + } + + if (addr >= XKVRANGE_VC_SHADOW_OFFSET) + return (void *)(((addr - XKVRANGE_VC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKVRANGE_VC_START); + else if (addr >= XKPRANGE_UC_SHADOW_OFFSET) + return (void *)(((addr - XKPRANGE_UC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKPRANGE_UC_START); + else if (addr >= XKPRANGE_CC_SHADOW_OFFSET) + return (void *)(((addr - XKPRANGE_CC_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT) + XKPRANGE_CC_START); + else { + WARN_ON(1); + return NULL; + } +} + +#ifdef CONFIG_KASAN +void kasan_init(void); +asmlinkage void kasan_early_init(void); +#else +static inline void kasan_early_init(void) { } +static inline void kasan_init(void) { } +#endif + +#endif +#endif diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h index d28fb9dbec59..5cfdf79b287e 100644 --- a/arch/loongarch/include/asm/pgtable.h +++ b/arch/loongarch/include/asm/pgtable.h @@ -86,9 +86,16 @@ extern unsigned long zero_page_mask; #define MODULES_END (MODULES_VADDR + SZ_256M) #define VMALLOC_START MODULES_END + +#ifndef CONFIG_KASAN #define VMALLOC_END \ (vm_map_base + \ min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE - VMEMMAP_SIZE) +#else +#define VMALLOC_END \ + (vm_map_base + \ + min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits) / 2) - PMD_SIZE - VMEMMAP_SIZE) +#endif #define vmemmap ((struct page *)((VMALLOC_END + PMD_SIZE) & PMD_MASK)) #define VMEMMAP_END ((unsigned long)vmemmap + VMEMMAP_SIZE - 1) diff --git a/arch/loongarch/include/asm/string.h b/arch/loongarch/include/asm/string.h index 7b29cc9c70aa..a6482abdc8b3 100644 --- a/arch/loongarch/include/asm/string.h +++ b/arch/loongarch/include/asm/string.h @@ -7,11 +7,27 @@ #define __HAVE_ARCH_MEMSET extern void *memset(void *__s, int __c, size_t __count); +extern void *__memset(void *__s, int __c, size_t __count); #define __HAVE_ARCH_MEMCPY extern void *memcpy(void *__to, __const__ void *__from, size_t __n); +extern void *__memcpy(void *__to, __const__ void *__from, size_t __n); #define __HAVE_ARCH_MEMMOVE extern void *memmove(void *__dest, __const__ void *__src, size_t __n); +extern void *__memmove(void *__dest, __const__ void *__src, size_t __n); + +#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__) + +/* + * For files that are not instrumented (e.g. mm/slub.c) we + * should use not instrumented version of mem* functions. + */ + +#define memset(s, c, n) __memset(s, c, n) +#define memcpy(dst, src, len) __memcpy(dst, src, len) +#define memmove(dst, src, len) __memmove(dst, src, len) + +#endif #endif /* _ASM_STRING_H */ diff --git a/arch/loongarch/kernel/Makefile b/arch/loongarch/kernel/Makefile index 9a72d91cd104..0055e7582e15 100644 --- a/arch/loongarch/kernel/Makefile +++ b/arch/loongarch/kernel/Makefile @@ -30,6 +30,9 @@ ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_perf_event.o = $(CC_FLAGS_FTRACE) endif +KASAN_SANITIZE_vdso.o := n +KASAN_SANITIZE_efi.o := n + obj-$(CONFIG_MODULES) += module.o module-sections.o obj-$(CONFIG_STACKTRACE) += stacktrace.o diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index 35c4a78614c3..b846445a9710 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -103,6 +103,10 @@ SYM_CODE_START(kernel_entry) # kernel entry point #endif #endif +#ifdef CONFIG_KASAN + bl kasan_early_init +#endif + bl start_kernel ASM_BUG() diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index 27f71f9531e1..18453f8cb9e8 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -610,4 +610,8 @@ void __init setup_arch(char **cmdline_p) #endif paging_init(); + +#if defined(CONFIG_KASAN) + kasan_init(); +#endif } diff --git a/arch/loongarch/lib/memcpy.S b/arch/loongarch/lib/memcpy.S index 3b7e1dec7109..db92ef7bef3a 100644 --- a/arch/loongarch/lib/memcpy.S +++ b/arch/loongarch/lib/memcpy.S @@ -10,16 +10,18 @@ #include #include -SYM_FUNC_START(memcpy) +SYM_FUNC_START_WEAK(memcpy) /* * Some CPUs support hardware unaligned access */ ALTERNATIVE "b __memcpy_generic", \ "b __memcpy_fast", CPU_FEATURE_UAL SYM_FUNC_END(memcpy) +SYM_FUNC_ALIAS(__memcpy, memcpy) _ASM_NOKPROBE(memcpy) EXPORT_SYMBOL(memcpy) +EXPORT_SYMBOL(__memcpy) /* * void *__memcpy_generic(void *dst, const void *src, size_t n) diff --git a/arch/loongarch/lib/memmove.S b/arch/loongarch/lib/memmove.S index b796c3d6da05..a2dec5899f5c 100644 --- a/arch/loongarch/lib/memmove.S +++ b/arch/loongarch/lib/memmove.S @@ -10,7 +10,7 @@ #include #include -SYM_FUNC_START(memmove) +SYM_FUNC_START_WEAK(memmove) blt a0, a1, 1f /* dst < src, memcpy */ blt a1, a0, 3f /* src < dst, rmemcpy */ jr ra /* dst == src, return */ @@ -19,27 +19,30 @@ SYM_FUNC_START(memmove) 1: ori a3, zero, 64 sub.d t0, a1, a0 blt t0, a3, 2f - b memcpy + b __memcpy 2: b __memcpy_generic /* if (dst - src) < 64, copy 1 byte at a time */ 3: ori a3, zero, 64 sub.d t0, a0, a1 blt t0, a3, 4f - b rmemcpy + b __rmemcpy 4: b __rmemcpy_generic SYM_FUNC_END(memmove) +SYM_FUNC_ALIAS(__memmove, memmove) _ASM_NOKPROBE(memmove) EXPORT_SYMBOL(memmove) +EXPORT_SYMBOL(__memmove) + +SYM_FUNC_START(__rmemcpy) -SYM_FUNC_START(rmemcpy) /* * Some CPUs support hardware unaligned access */ ALTERNATIVE "b __rmemcpy_generic", \ "b __rmemcpy_fast", CPU_FEATURE_UAL -SYM_FUNC_END(rmemcpy) +SYM_FUNC_END(__rmemcpy) _ASM_NOKPROBE(rmemcpy) /* diff --git a/arch/loongarch/lib/memset.S b/arch/loongarch/lib/memset.S index a9eb732ab2ad..b5cdbecba8ef 100644 --- a/arch/loongarch/lib/memset.S +++ b/arch/loongarch/lib/memset.S @@ -16,16 +16,18 @@ bstrins.d \r0, \r0, 63, 32 .endm -SYM_FUNC_START(memset) +SYM_FUNC_START_WEAK(memset) /* * Some CPUs support hardware unaligned access */ ALTERNATIVE "b __memset_generic", \ "b __memset_fast", CPU_FEATURE_UAL SYM_FUNC_END(memset) +SYM_FUNC_ALIAS(__memset, memset) _ASM_NOKPROBE(memset) EXPORT_SYMBOL(memset) +EXPORT_SYMBOL(__memset) /* * void *__memset_generic(void *s, int c, size_t n) diff --git a/arch/loongarch/mm/Makefile b/arch/loongarch/mm/Makefile index 8ffc6383f836..6e50cf6cf733 100644 --- a/arch/loongarch/mm/Makefile +++ b/arch/loongarch/mm/Makefile @@ -7,3 +7,5 @@ obj-y += init.o cache.o tlb.o tlbex.o extable.o \ fault.o ioremap.o maccess.o mmap.o pgtable.o page.o obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o +obj-$(CONFIG_KASAN) += kasan_init.o +KASAN_SANITIZE_kasan_init.o := n diff --git a/arch/loongarch/mm/kasan_init.c b/arch/loongarch/mm/kasan_init.c new file mode 100644 index 000000000000..fb3077f8d508 --- /dev/null +++ b/arch/loongarch/mm/kasan_init.c @@ -0,0 +1,255 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 Loongson Technology Corporation Limited + */ +#define pr_fmt(fmt) "kasan: " fmt +#include +#include +#include + +#include +#include +#include + +static pgd_t tmp_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE); + +static inline int __p4d_none(int early, p4d_t p4d) {return 0; } + +#ifndef __PAGETABLE_PUD_FOLDED +#define __p4d_none(early, p4d) (early ? (p4d_val(p4d) == 0) : \ +(__pa(p4d_val(p4d)) == (unsigned long)__pa(kasan_early_shadow_pud))) +#endif + +#define __pud_none(early, pud) (early ? (pud_val(pud) == 0) : \ +(__pa(pud_val(pud)) == (unsigned long)__pa(kasan_early_shadow_pmd))) + +#define __pmd_none(early, pmd) (early ? (pmd_val(pmd) == 0) : \ +(__pa(pmd_val(pmd)) == (unsigned long)__pa(kasan_early_shadow_pte))) + +#define __pte_none(early, pte) (early ? pte_none(pte) : \ +((pte_val(pte) & _PFN_MASK) == (unsigned long)__pa(kasan_early_shadow_page))) + +bool kasan_early_stage = true; + +/* + * Alloc memory for shadow memory page table. + */ +static phys_addr_t __init kasan_alloc_zeroed_page(int node) +{ + void *p = memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, node); + if (!p) + panic("%s: Failed to allocate %lu bytes align=0x%lx nid=%d from=%llx\n", + __func__, PAGE_SIZE, PAGE_SIZE, node, __pa(MAX_DMA_ADDRESS)); + return __pa(p); +} + +static pte_t *kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, + bool early) +{ + if (__pmd_none(early, READ_ONCE(*pmdp))) { + phys_addr_t pte_phys = early ? + __pa_symbol(kasan_early_shadow_pte) + : kasan_alloc_zeroed_page(node); + if (!early) + memcpy(__va(pte_phys), kasan_early_shadow_pte, + sizeof(kasan_early_shadow_pte)); + pmd_populate_kernel(NULL, pmdp, (pte_t *)__va(pte_phys)); + } + + return pte_offset_kernel(pmdp, addr); +} + +static inline void kasan_set_pgd(pgd_t *pgdp, pgd_t pgdval) +{ + WRITE_ONCE(*pgdp, pgdval); +} + +static pmd_t *kasan_pmd_offset(pud_t *pudp, unsigned long addr, int node, + bool early) +{ + if (__pud_none(early, READ_ONCE(*pudp))) { + phys_addr_t pmd_phys = early ? + __pa_symbol(kasan_early_shadow_pmd) + : kasan_alloc_zeroed_page(node); + if (!early) + memcpy(__va(pmd_phys), kasan_early_shadow_pmd, + sizeof(kasan_early_shadow_pmd)); + pud_populate(&init_mm, pudp, (pmd_t *)__va(pmd_phys)); + } + + return pmd_offset(pudp, addr); +} + +static pud_t *__init kasan_pud_offset(p4d_t *p4dp, unsigned long addr, int node, + bool early) +{ + if (__p4d_none(early, READ_ONCE(*p4dp))) { + phys_addr_t pud_phys = early ? + __pa_symbol(kasan_early_shadow_pud) + : kasan_alloc_zeroed_page(node); + if (!early) + memcpy(__va(pud_phys), kasan_early_shadow_pud, + sizeof(kasan_early_shadow_pud)); + p4d_populate(&init_mm, p4dp, (pud_t *)__va(pud_phys)); + } + + return pud_offset(p4dp, addr); +} + +static void kasan_pte_populate(pmd_t *pmdp, unsigned long addr, + unsigned long end, int node, bool early) +{ + unsigned long next; + pte_t *ptep = kasan_pte_offset(pmdp, addr, node, early); + + do { + phys_addr_t page_phys = early ? + __pa_symbol(kasan_early_shadow_page) + : kasan_alloc_zeroed_page(node); + next = addr + PAGE_SIZE; + set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); + } while (ptep++, addr = next, addr != end && __pte_none(early, READ_ONCE(*ptep))); +} + +static void kasan_pmd_populate(pud_t *pudp, unsigned long addr, + unsigned long end, int node, bool early) +{ + unsigned long next; + pmd_t *pmdp = kasan_pmd_offset(pudp, addr, node, early); + + do { + next = pmd_addr_end(addr, end); + kasan_pte_populate(pmdp, addr, next, node, early); + } while (pmdp++, addr = next, addr != end && __pmd_none(early, READ_ONCE(*pmdp))); +} + +static void __init kasan_pud_populate(p4d_t *p4dp, unsigned long addr, + unsigned long end, int node, bool early) +{ + unsigned long next; + pud_t *pudp = kasan_pud_offset(p4dp, addr, node, early); + + do { + next = pud_addr_end(addr, end); + kasan_pmd_populate(pudp, addr, next, node, early); + } while (pudp++, addr = next, addr != end); +} + +static void __init kasan_p4d_populate(pgd_t *pgdp, unsigned long addr, + unsigned long end, int node, bool early) +{ + unsigned long next; + p4d_t *p4dp = p4d_offset(pgdp, addr); + + do { + next = p4d_addr_end(addr, end); + kasan_pud_populate(p4dp, addr, next, node, early); + } while (p4dp++, addr = next, addr != end); +} + +static void __init kasan_pgd_populate(unsigned long addr, unsigned long end, + int node, bool early) +{ + unsigned long next; + pgd_t *pgdp; + + pgdp = pgd_offset_k(addr); + + do { + next = pgd_addr_end(addr, end); + kasan_p4d_populate(pgdp, addr, next, node, early); + } while (pgdp++, addr = next, addr != end); + +} + +asmlinkage void __init kasan_early_init(void) +{ + BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_START, PGDIR_SIZE)); + BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, PGDIR_SIZE)); +} + +/* Set up full kasan mappings, ensuring that the mapped pages are zeroed */ +static void __init kasan_map_populate(unsigned long start, unsigned long end, + int node) +{ + kasan_pgd_populate(start & PAGE_MASK, PAGE_ALIGN(end), node, false); +} + +static void __init clear_pgds(unsigned long start, unsigned long end) +{ + /* + * Remove references to kasan page tables from + * swapper_pg_dir. pgd_clear() can't be used + * here because it's nop on 2,3-level pagetable setups + */ + for (; start < end; start += PGDIR_SIZE) + kasan_set_pgd((pgd_t *)pgd_offset_k(start), __pgd(0)); +} + +void __init kasan_init(void) +{ + u64 i; + phys_addr_t pa_start, pa_end; + /* + * PGD was populated as invalid_pmd_table or invalid_pud_table + * in pagetable_init() which depends on how many levels of page + * table you are using, but we had to clean the gpd of kasan + * shadow memory, as the pgd value is none-zero. + * The assertion pgd_none is going to be false and the formal populate + * afterwards is not going to create any new pgd at all. + */ + memcpy(tmp_pg_dir, swapper_pg_dir, sizeof(tmp_pg_dir)); + __sync(); + csr_write64(__pa_symbol(tmp_pg_dir), LOONGARCH_CSR_PGDH); + local_flush_tlb_all(); + + clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END); + + /* Maps everything to a single page of zeroes */ + kasan_pgd_populate(KASAN_SHADOW_START, KASAN_SHADOW_END, + NUMA_NO_NODE, true); + + kasan_populate_early_shadow(kasan_mem_to_shadow((void *)MODULES_END), + kasan_mem_to_shadow((void *)VMEMMAP_END)); + + if (!IS_ENABLED(CONFIG_KASAN_VMALLOC)) + kasan_populate_early_shadow(kasan_mem_to_shadow((void *)VMALLOC_START), + kasan_mem_to_shadow((void *)VMALLOC_END)); + + kasan_early_stage = false; + + /* Populate the linear mapping */ + for_each_mem_range(i, &pa_start, &pa_end) { + void *start = (void *)phys_to_virt(pa_start); + void *end = (void *)phys_to_virt(pa_end); + + if (start >= end) + break; + + kasan_map_populate((unsigned long)kasan_mem_to_shadow(start), + (unsigned long)kasan_mem_to_shadow(end), NUMA_NO_NODE); + } + + /* Populate modules mapping */ + kasan_map_populate((unsigned long)kasan_mem_to_shadow((void *)MODULES_VADDR), + (unsigned long)kasan_mem_to_shadow((void *)MODULES_END), NUMA_NO_NODE); + /* + * Kasan may reuse the contents of kasan_early_shadow_pte directly, so we + * should make sure that it maps the zero page read-only. + */ + for (i = 0; i < PTRS_PER_PTE; i++) + set_pte(&kasan_early_shadow_pte[i], + pfn_pte(__phys_to_pfn(__pa_symbol(kasan_early_shadow_page)), + PAGE_KERNEL_RO)); + + memset(kasan_early_shadow_page, 0, PAGE_SIZE); + __sync(); + csr_write64(__pa_symbol(swapper_pg_dir), LOONGARCH_CSR_PGDH); + local_flush_tlb_all(); + + /* At this point kasan is fully initialized. Enable error messages */ + init_task.kasan_depth = 0; + pr_info("KernelAddressSanitizer initialized.\n"); +} diff --git a/arch/loongarch/vdso/Makefile b/arch/loongarch/vdso/Makefile index d89e2ac75f7b..df328cd92875 100644 --- a/arch/loongarch/vdso/Makefile +++ b/arch/loongarch/vdso/Makefile @@ -1,6 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 # Objects to go into the VDSO. +ifdef CONFIG_KASAN +KASAN_SANITIZE := n +endif + # Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before # the inclusion of generic Makefile. ARCH_REL_TYPE_ABS := R_LARCH_32|R_LARCH_64|R_LARCH_MARK_LA|R_LARCH_JUMP_SLOT From patchwork Tue Apr 4 08:43:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qing Zhang X-Patchwork-Id: 78929 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2871032vqo; Tue, 4 Apr 2023 01:48:05 -0700 (PDT) X-Google-Smtp-Source: AKy350Z1V5gA8zJhPshjDgXRlhAHVcshVWfjHXR9JRzJQcn9hkXfbpdk2DKEl9ZqfUe2R5d19QoC X-Received: by 2002:a17:906:38ca:b0:93e:9362:75fe with SMTP id r10-20020a17090638ca00b0093e936275femr1258166ejd.76.1680598084837; Tue, 04 Apr 2023 01:48:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680598084; cv=none; d=google.com; s=arc-20160816; b=PBS/InQMDeiAPW2zkHZI9YXslZtiyk933Bmy0uIbv4lg/1nM3gG+hML8sseIVk+Hci i2Q3ANSzrhzZo8HK+DkskFzFY+i+qS4riNPQZ1tzvw9dY/ZNpG9VLFnhUK1/ZlS53T2W Ef9H9oFm41Czm/2hEu1qKVBjM/QEHK5n56uL15MpM+EYklWSz/AJY4z93VLdxcUhIpWc NOHRVQUOLaWOOYAjUDdksz/d9GHCcvsa/XIxvgz/+VQHZhIU8tSfCHdr0/xQhG1sOqpq m/JRF4lk2mZAfEd1G6WdYow4VPY1aTjHvV2B8wmJ1cWh5duIk7wSUqPvok/WwJld+DQM mXFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=UVoDPEwl9zzWrReMqk0AQq31NzX8iUtFEhFKCcfSYL0=; b=Rv3dah/qMeYFYuk5ZokBOPBjky0lXTX7I8S9U3NKkKNHAXErgP5xPx8d4ceNJciPS5 mNK5j+CRPgK7scDKHKYdnLfbIaj9Kf8v8T1Vh3DzZuskIoA/zNcLJGkvJdzIpWneCDQH Ww6ZISh7KGsdgseQw4VRle3W12zyc6HeZlFXCFN0xoQo1jgt3qQO2I/wnPjVAWu2m8fF 9iSoRIStL3BdlDmoU2Jh7kFhajKLk6pMFLLTjGJfh6qe3yKrzNqSLF12WI5NZmMD7RpU U656H1kCL3NnJE9Duu8LV7U63VZxAt9I0hXXMYjeqBSU11yiNvWLcyLu7vhvCM8wiAtC 4YZg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y15-20020a170906070f00b00930645a3b08si502878ejb.19.2023.04.04.01.47.41; Tue, 04 Apr 2023 01:48:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234025AbjDDIn0 (ORCPT + 99 others); Tue, 4 Apr 2023 04:43:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233938AbjDDInX (ORCPT ); Tue, 4 Apr 2023 04:43:23 -0400 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A47162709; Tue, 4 Apr 2023 01:43:12 -0700 (PDT) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8Dx_5cf4ytkr10WAA--.34838S3; Tue, 04 Apr 2023 16:43:11 +0800 (CST) Received: from localhost.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxT+Qc4ytkChcVAA--.55041S2; Tue, 04 Apr 2023 16:43:09 +0800 (CST) From: Qing Zhang To: Andrey Ryabinin , Jonathan Corbet , Huacai Chen , Andrew Morton Cc: Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , WANG Xuerui , Jiaxun Yang , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-mm@kvack.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 4/6] kasan: Add __HAVE_ARCH_SHADOW_MAP to support arch specific mapping Date: Tue, 4 Apr 2023 16:43:06 +0800 Message-Id: <20230404084308.813-1-zhangqing@loongson.cn> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxT+Qc4ytkChcVAA--.55041S2 X-CM-SenderInfo: x2kd0wptlqwqxorr0wxvrqhubq/ X-Coremail-Antispam: 1Uk129KBjvJXoW7WryDtw1xWw18Xw1UAr17Awb_yoW8uw4UpF ZrGFyxtrs2qFy0ga43Cr4Uur15JrnaqF4ktrZIgw4rCFy5W3WvqF1q9F9Yyrn7Wr47tFyY vwn7ZFZxJr90q3DanT9S1TB71UUUUbUqnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU bhxYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_JrI_Jryl8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW7JVWDJwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2kKe7AKxVWUAVWUtwAS0I0E0xvYzxvE52x082IY62kv0487Mc804VCY07AIYIkI8VC2zV CFFI0UMc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWrXVW3AwAv7VC2 z280aVAFwI0_Cr0_Gr1UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCY1x 0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwCF I7km07C267AKxVWUAVWUtwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r 106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AK xVW7JVWDJwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcI k0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWxJVW8Jr1lIxAIcVC2z280aVCY1x0267AKxVW8 JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU08gA7UUUUU== X-Spam-Status: No, score=-0.0 required=5.0 tests=SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762234817274842365?= X-GMAIL-MSGID: =?utf-8?q?1762234817274842365?= Like the LoongArch, which has many holes between different segments and valid address space(256T available) is insufficient to map all these segments to kasan shadow memory with the common formula provided by kasan core, We need architecture specific mapping formula,different segments are mapped individually, and only limited length of space of that specific segment is mapped to shadow. Therefore, when the incoming address is converted to a shadow, we need to add a condition to determine whether it is valid. Signed-off-by: Qing Zhang Reviewed-by: Andrey Konovalov --- include/linux/kasan.h | 2 ++ mm/kasan/kasan.h | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index f7ef70661ce2..3b91b941873d 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -54,11 +54,13 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); +#ifndef __HAVE_ARCH_SHADOW_MAP static inline void *kasan_mem_to_shadow(const void *addr) { return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET; } +#endif int kasan_add_zero_shadow(void *start, unsigned long size); void kasan_remove_zero_shadow(void *start, unsigned long size); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a61eeee3095a..033335c13b25 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -291,16 +291,22 @@ struct kasan_stack_ring { #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) +#ifndef __HAVE_ARCH_SHADOW_MAP static inline const void *kasan_shadow_to_mem(const void *shadow_addr) { return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET) << KASAN_SHADOW_SCALE_SHIFT); } +#endif static __always_inline bool addr_has_metadata(const void *addr) { +#ifdef __HAVE_ARCH_SHADOW_MAP + return (kasan_mem_to_shadow((void *)addr) != NULL); +#else return (kasan_reset_tag(addr) >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +#endif } /** From patchwork Tue Apr 4 08:43:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qing Zhang X-Patchwork-Id: 78932 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2872830vqo; Tue, 4 Apr 2023 01:52:18 -0700 (PDT) X-Google-Smtp-Source: AKy350bKYRzrTx4A5DVaUNoST57Y9OD3p9XKkKg/ReVjrjYl2qe44i1cpG/iughpVMi8Z8VjJEhh X-Received: by 2002:a17:906:3448:b0:92d:44ca:1137 with SMTP id d8-20020a170906344800b0092d44ca1137mr2017472ejb.43.1680598338762; Tue, 04 Apr 2023 01:52:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680598338; cv=none; d=google.com; s=arc-20160816; b=O8dDM9F3LcTFMSl4o9SOJQX8aw+wd496MoHeDNlo4NC4ygAyn7Kptou2gwnIdrX6xr QmCZHIpNiss+rTB2MN3Fgb7591cJE3vSe/vEAhQvi43gnecuhSWx590MkmJCmaw2Kzq5 bkZimn51x6/lNwMNXThiga2GRnscyxDI5cVXy+QnxboMmq5AbS0yoExi+nuZ8cplTCB0 MksvyrJrSWDsZuJMyRxfpix0+AHMatCC1JICYKYrL4AqKmFKw925bcsMHay/9h/wJf+O Aoxd2fViT10b36vO2Pf6a4vkGgVPe9ZsqWoHp29ztdYBlEZYqoItqfiIghjELTMI4Rte 00Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=eraKDZXAVWRAoDpzslhZKrcjIAmbAugbyfi3AGnEHjo=; b=ixO2gHcYN2oGMqKaj+hXrMBOMDPD2rYs5qIyxkMCqT53Pv6ar1xubY8a6dqSQ06kU5 tQh+GP+UTprRW4lreYAgpizNqUpVvKH650sPd3YVuShs2/D92ARwfnKRTN7oy1Q6CV3p CJReLDaR6yrRt/kh8gx3jsnbh77xFuJL7GxukJ77CRBhGnQXteQIHOCmz7qzHDK4THYG 9Id6sSrvAWJXJgCzI5eOsmyW/zBwqYW/R0hfxN6dcEzjpsTy88QuVyr9uRdb4UQIds3Q rxq6wotBdudU1BXfeEo1MX1QNh53irPaeX+KfhTEgpka+MGtN//WEIwbp5qpf1mmmp0A fQKg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y4-20020a1709060a8400b0092fd56751ffsi6088303ejf.420.2023.04.04.01.51.54; Tue, 04 Apr 2023 01:52:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233915AbjDDInc (ORCPT + 99 others); Tue, 4 Apr 2023 04:43:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233526AbjDDInY (ORCPT ); Tue, 4 Apr 2023 04:43:24 -0400 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 16F722D4B; Tue, 4 Apr 2023 01:43:13 -0700 (PDT) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8Dxldgg4ytku10WAA--.39499S3; Tue, 04 Apr 2023 16:43:12 +0800 (CST) Received: from localhost.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxT+Qc4ytkChcVAA--.55041S3; Tue, 04 Apr 2023 16:43:11 +0800 (CST) From: Qing Zhang To: Andrey Ryabinin , Jonathan Corbet , Huacai Chen , Andrew Morton Cc: Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , WANG Xuerui , Jiaxun Yang , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-mm@kvack.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 5/6] kasan: Add (pmd|pud)_init for LoongArch zero_(pud|p4d)_populate process Date: Tue, 4 Apr 2023 16:43:07 +0800 Message-Id: <20230404084308.813-2-zhangqing@loongson.cn> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230404084308.813-1-zhangqing@loongson.cn> References: <20230404084308.813-1-zhangqing@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxT+Qc4ytkChcVAA--.55041S3 X-CM-SenderInfo: x2kd0wptlqwqxorr0wxvrqhubq/ X-Coremail-Antispam: 1Uk129KBjvJXoW7Zw4DXF1rtr1DZry7tF4fZrb_yoW8Ar17pF WUW3W0qw43Xa9rXws3Jr1vgry7Jan7K3W7Kay2kr1rA345XrWUXFy8Jr1q9r45AFWkZFyS yan3Gry3C3WDJaDanT9S1TB71UUUUb7qnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU bhkYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_Jrv_JF1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW7JVWDJwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2kKe7AKxVWUtVW8ZwAS0I0E0xvYzxvE52x082IY62kv0487Mc804VCY07AIYIkI8VC2zV CFFI0UMc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVW3AVW8Xw1lYx0E x4A2jsIE14v26F4j6r4UJwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lc7 CjxVAaw2AFwI0_Jw0_GFyl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1l 4IxYO2xFxVAFwI0_Jw0_GFylx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxV WUGVWUWwC2zVAF1VAY17CE14v26r4a6rW5MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAF wI0_Ar0_tr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY6xAIw20EY4v20x vaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Cr0_Gr1UMIIF0xvEx4A2jsIEc7CjxVAFwI0_ Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jDjjDUUUUU= X-Spam-Status: No, score=-0.0 required=5.0 tests=SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762235083803173003?= X-GMAIL-MSGID: =?utf-8?q?1762235083803173003?= Loongarch populate pmd/pud with invalid_pmd_table/invalid_pud_table in pagetable_init, So pmd_init/pud_init(p) is required, define them as __weak in mm/kasan/init.c, like mm/sparse-vmemmap.c. Signed-off-by: Qing Zhang Reviewed-by: Andrey Konovalov --- mm/kasan/init.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/mm/kasan/init.c b/mm/kasan/init.c index cc64ed6858c6..a7fa223b96e4 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -139,6 +139,10 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr, return 0; } +void __weak __meminit pmd_init(void *addr) +{ +} + static int __ref zero_pud_populate(p4d_t *p4d, unsigned long addr, unsigned long end) { @@ -166,8 +170,9 @@ static int __ref zero_pud_populate(p4d_t *p4d, unsigned long addr, if (!p) return -ENOMEM; } else { - pud_populate(&init_mm, pud, - early_alloc(PAGE_SIZE, NUMA_NO_NODE)); + p = early_alloc(PAGE_SIZE, NUMA_NO_NODE); + pmd_init(p); + pud_populate(&init_mm, pud, p); } } zero_pmd_populate(pud, addr, next); @@ -176,6 +181,10 @@ static int __ref zero_pud_populate(p4d_t *p4d, unsigned long addr, return 0; } +void __weak __meminit pud_init(void *addr) +{ +} + static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, unsigned long end) { @@ -207,8 +216,9 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, if (!p) return -ENOMEM; } else { - p4d_populate(&init_mm, p4d, - early_alloc(PAGE_SIZE, NUMA_NO_NODE)); + p = early_alloc(PAGE_SIZE, NUMA_NO_NODE); + pud_init(p); + p4d_populate(&init_mm, p4d, p); } } zero_pud_populate(p4d, addr, next); From patchwork Tue Apr 4 08:43:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qing Zhang X-Patchwork-Id: 78930 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2871033vqo; Tue, 4 Apr 2023 01:48:05 -0700 (PDT) X-Google-Smtp-Source: AKy350ag0W2GiZoyW98vx6ccqgqeLVrP70KlQVxKP+a2KQX3pWVbo8Mr+uNG5E7jwf85tlj64H8O X-Received: by 2002:a50:ec91:0:b0:4fd:c5e:79b8 with SMTP id e17-20020a50ec91000000b004fd0c5e79b8mr1722150edr.32.1680598084842; Tue, 04 Apr 2023 01:48:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680598084; cv=none; d=google.com; s=arc-20160816; b=TX+nLFzzmnKo08sVBJ9x5KYxANmmA7SUm3s42jsZ+WiOQS+5qVMJMycDAmDVN6vTCi y/OH9eSAoew0a4kBkA78BoURDP0aabBtKwzx4ZBd3/t3rGTGF4XDz1icRj+yAhmcUD7I 99Tjfkub2kL2FItmSkzZfzwP44BbHC8AUBJ1eCKEfG7kZ3oYvvFqO9YbKMIMTtZKnIzF 7TW1cZTw1y/iqIP/w3HY6Io8dPGOXj738kJfic/4/0leqsZh81qg9uI+ulCftLD2/5N5 u6pfQMVtNtEI+gfRwi93dtjXSSMGUAsJQAeUbMRmN3fL6n7QiuujMmHfq8A548a/4uY1 1DDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=ojSUF8/qqXE/bnfEFAt+zRgTWbIStxDciISKhrAb4TM=; b=SOjZLa1kS5JGhG0h+fU5M15tmZYYDs7oo9tWTCbf0qLh7lh0+5B2Ck///+fgHCm78p TKhPkbrIzIDces2T6MO20UBvwliokX8tRwXkq6ackT1Wnqdl+oFcKOACbAWpr8f0++3o 7zS+Dpbp1MKbjNUUscDiRqvZtmBhk6diJSELqOJu2YM9TbqjWYensenWP8Jhd5Sv/asg /nSA6H5PHb/M9YVRHi8lvsGkvhtJ+jCGB4CW83k9ZgQcpn0JgRaVfC1tljWNIWpO3VbW dmSKMQDEXS7/E1so3a3T8R90Rb28npBJPNu13P23Eo98G8bg+zMcs6AzYYyLoEw5+FQF TilA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v14-20020a056402184e00b004c5a8aa02fcsi272899edy.435.2023.04.04.01.47.41; Tue, 04 Apr 2023 01:48:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234037AbjDDInf (ORCPT + 99 others); Tue, 4 Apr 2023 04:43:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233983AbjDDInZ (ORCPT ); Tue, 4 Apr 2023 04:43:25 -0400 Received: from loongson.cn (mail.loongson.cn [114.242.206.163]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B6733211D; Tue, 4 Apr 2023 01:43:14 -0700 (PDT) Received: from loongson.cn (unknown [113.200.148.30]) by gateway (Coremail) with SMTP id _____8AxYeUh4ytkx10WAA--.34733S3; Tue, 04 Apr 2023 16:43:13 +0800 (CST) Received: from localhost.localdomain (unknown [113.200.148.30]) by localhost.localdomain (Coremail) with SMTP id AQAAf8AxT+Qc4ytkChcVAA--.55041S4; Tue, 04 Apr 2023 16:43:12 +0800 (CST) From: Qing Zhang To: Andrey Ryabinin , Jonathan Corbet , Huacai Chen , Andrew Morton Cc: Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , WANG Xuerui , Jiaxun Yang , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-mm@kvack.org, loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org Subject: [PATCH v2 6/6] LoongArch: Add ARCH_HAS_FORTIFY_SOURCE Date: Tue, 4 Apr 2023 16:43:08 +0800 Message-Id: <20230404084308.813-3-zhangqing@loongson.cn> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230404084308.813-1-zhangqing@loongson.cn> References: <20230404084308.813-1-zhangqing@loongson.cn> MIME-Version: 1.0 X-CM-TRANSID: AQAAf8AxT+Qc4ytkChcVAA--.55041S4 X-CM-SenderInfo: x2kd0wptlqwqxorr0wxvrqhubq/ X-Coremail-Antispam: 1Uk129KBjvJXoW7uF1xtF18KFW5Gr1ktw4DXFb_yoW8WFyrpF nrA3s5Jr48CFn7AFWjy34UWryUWF97Kr42gFyYya48AFy3XryDXrs2q3Z0vFy5Za1rG3yx uFyfWa4aqF4DX37anT9S1TB71UUUUb7qnTZGkaVYY2UrUUUUj1kv1TuYvTs0mT0YCTnIWj qI5I8CrVACY4xI64kE6c02F40Ex7xfYxn0WfASr-VFAUDa7-sFnT9fnUUIcSsGvfJTRUUU b6xYFVCjjxCrM7AC8VAFwI0_Jr0_Gr1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_Wr0E3s 1l1IIY67AEw4v_Jrv_JF1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM2 8EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j6F4U JwAaw2AFwI0_Jw0_GFyle2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4 CE44I27wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_ZF0_GryDMcIj 6I8E87Iv67AKxVWxJVW8Jr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JMx kF7I0En4kS14v26r1q6r43MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4U MxCIbckI1I0E14v26r1q6r43MI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI 0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE 14v26w1j6s0DMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI 8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWxJVW8Jr1lIxAIcVC2z280aVCY1x0267AK xVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x07jDjjDUUUUU= X-Spam-Status: No, score=-0.0 required=5.0 tests=SPF_HELO_PASS,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762234817246622369?= X-GMAIL-MSGID: =?utf-8?q?1762234817246622369?= FORTIFY_SOURCE could detect various overflows at compile and run time. ARCH_HAS_FORTIFY_SOURCE means that the architecture can be built and run with CONFIG_FORTIFY_SOURCE. Select it in LoongArch. See more about this feature from commit 6974f0c4555e ("include/linux/string.h: add the option of fortified string.h functions"). Signed-off-by: Qing Zhang --- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/string.h | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 61f883c51045..6c525e50bb7c 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -11,6 +11,7 @@ config LOONGARCH select ARCH_ENABLE_MEMORY_HOTPLUG select ARCH_ENABLE_MEMORY_HOTREMOVE select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI + select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_NMI_SAFE_THIS_CPU_OPS select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST diff --git a/arch/loongarch/include/asm/string.h b/arch/loongarch/include/asm/string.h index a6482abdc8b3..5bb5a90d2681 100644 --- a/arch/loongarch/include/asm/string.h +++ b/arch/loongarch/include/asm/string.h @@ -28,6 +28,10 @@ extern void *__memmove(void *__dest, __const__ void *__src, size_t __n); #define memcpy(dst, src, len) __memcpy(dst, src, len) #define memmove(dst, src, len) __memmove(dst, src, len) +#ifndef __NO_FORTIFY +#define __NO_FORTIFY /* FORTIFY_SOURCE uses __builtin_memcpy, etc. */ +#endif + #endif #endif /* _ASM_STRING_H */