From patchwork Fri Jan 27 13:05:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49397 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp831167wrn; Fri, 27 Jan 2023 05:24:09 -0800 (PST) X-Google-Smtp-Source: AMrXdXv8yguxl1ECvUT/WmwPUjNXSHKL2YqILjmVPY6moMMx1vjw6kXgWkw/ZO8HGUHfXdRsDBOZ X-Received: by 2002:a17:906:4ad5:b0:7ad:e67d:f15c with SMTP id u21-20020a1709064ad500b007ade67df15cmr52125204ejt.48.1674825849011; Fri, 27 Jan 2023 05:24:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825848; cv=none; d=google.com; s=arc-20160816; b=Bro+7wEJ/ZxjhN8v9DAY3cy+FgHQAvxhFgR8I7RJhqF+dTcKzHd5mRu6KtcF4PEh9h j2sjPdB++bvFzEKZd3kf3Ior+VGmhiWB32dwXp/vBrEJMQ9KFJV69Cw0CN9KTBL0mDW9 p8cr5CjJGcqSoJJ/BiZ2KlYCglGspITYfNBipZJ+ByxADrFME+/oq9oxO2w53OYHjkEY 0g+usEIqNErPRoNkI/V/JGSEN6bdM65CzZZxg5AQDm0H9etZNMTjMAx3IADYxBZmqOza QrsrRrK4fTrEQoRm9JABofSvhRUm5NMMyUTNnTg5fwRH2JEPtJxNX56S9hrHUaKp9Prg /OIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=O5aQsechXL5b13YPYs70vKGVXjFK+nHN8SftcINL0Ls=; b=K5MwiHihA0zMBLEQfmVi1os1tyeipBKYyZmvh6JR0FHOHvCPb0ybiPp6GkPx/BumjJ zMGaI6FPHoq+Rkn4S1hF0j9Y8+he1+f63yKYSIyuOpWuKDTVIsUdnluKg5tCt1mP34AX Mf3GbvoQsk8AvFN8sAkisTP4+0Br66ag65I6Aen6fiCqnK7Tup6e3yXpWHQxOoI/abFf tgZL4Dni1wWtz10LXomsznXheEQ5vhMqURReZbhf8F3yFtVg2t85bI8udRcUtXeatnca DP7o2DyEex/jAazSss2KHZREC4wmt7rTmKz8d/jUrvUNn/njPSoIimwE11Yah2o0SG3v XtWA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f17-20020a170906825100b008779e4f2561si5568505ejx.648.2023.01.27.05.23.44; Fri, 27 Jan 2023 05:24:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234831AbjA0NS7 (ORCPT + 99 others); Fri, 27 Jan 2023 08:18:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234681AbjA0NSc (ORCPT ); Fri, 27 Jan 2023 08:18:32 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B091165EF7 for ; Fri, 27 Jan 2023 05:18:01 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S5; Fri, 27 Jan 2023 21:06:08 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 01/13] riscv/kprobe: Prepare the skeleton to implement RISCV OPTPROBES Date: Fri, 27 Jan 2023 21:05:29 +0800 Message-Id: <20230127130541.1250865-2-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S5 X-Coremail-Antispam: 1UD129KBjvJXoWxtr48Gr4DZFy5ur45JryxZrb_yoW7Ar4DpF s5Cws5WrWrAas3G3yay3WkurWSqan5Ga17KrWDAry5Jr4xtr45Awn2grWUXrWxJrn0gryf WF1FvryYkry3X3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQj14x267AKxVW5JVWrJwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r1I6r4UM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYx C7M4IIrI8v6xkF7I0E8cxan2IY04v7MxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCF x2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14 v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY 67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2 IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_ Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUw2-5UUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiAwoEE2PTrr4gzgAAsQ X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756182189766192553?= X-GMAIL-MSGID: =?utf-8?q?1756182189766192553?= From: Liao Chang Prepare skeleton to implement optimized kprobe on RISCV, although some architecture specific functions are left blank, they do not change the correctness of existing kprobe code, on account of these functions just return zero. To avoid each patch being too complicated to review and test, these functions will be implemented incrementally. Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/kprobes.h | 32 ++++++++++++++ arch/riscv/kernel/probes/Makefile | 1 + arch/riscv/kernel/probes/opt.c | 51 +++++++++++++++++++++++ arch/riscv/kernel/probes/opt_trampoline.S | 12 ++++++ 5 files changed, 97 insertions(+) create mode 100644 arch/riscv/kernel/probes/opt.c create mode 100644 arch/riscv/kernel/probes/opt_trampoline.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 9c687da7756d..48a639c7c055 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -101,6 +101,7 @@ config RISCV select HAVE_KPROBES if !XIP_KERNEL select HAVE_KPROBES_ON_FTRACE if !XIP_KERNEL select HAVE_KRETPROBES if !XIP_KERNEL + select HAVE_OPTPROBES if !XIP_KERNEL select HAVE_RETHOOK if !XIP_KERNEL select HAVE_MOVE_PMD select HAVE_MOVE_PUD diff --git a/arch/riscv/include/asm/kprobes.h b/arch/riscv/include/asm/kprobes.h index e7882ccb0fd4..96cd36e67e2e 100644 --- a/arch/riscv/include/asm/kprobes.h +++ b/arch/riscv/include/asm/kprobes.h @@ -41,5 +41,37 @@ int kprobe_fault_handler(struct pt_regs *regs, unsigned int trapnr); bool kprobe_breakpoint_handler(struct pt_regs *regs); bool kprobe_single_step_handler(struct pt_regs *regs); +#ifdef CONFIG_OPTPROBES + +/* optinsn template addresses */ +extern __visible kprobe_opcode_t optprobe_template_entry[]; +extern __visible kprobe_opcode_t optprobe_template_end[]; + +#define MAX_OPTINSN_SIZE \ + ((unsigned long)optprobe_template_end - \ + (unsigned long)optprobe_template_entry) + +/* + * For RVI and RVC hybrid encoding kernel, although long jump just needs + * 2 RVI instructions(AUIPC/JALR), optimized instructions are 10 bytes long + * at most to ensure no RVI would be truncated actually, so it means four + * combinations: + * - 2 RVI + * - 4 RVC + * - 2 RVC + 1 RVI + * - 3 RVC + 1 RVI (truncated, need padding) + */ +#define MAX_COPIED_INSN 4 +#define MAX_OPTIMIZED_LENGTH 10 + +struct arch_optimized_insn { + kprobe_opcode_t copied_insn[MAX_COPIED_INSN]; + /* detour code buffer */ + kprobe_opcode_t *insn; + unsigned long length; + int rd; +}; + +#endif /* CONFIG_OPTPROBES */ #endif /* CONFIG_KPROBES */ #endif /* _ASM_RISCV_KPROBES_H */ diff --git a/arch/riscv/kernel/probes/Makefile b/arch/riscv/kernel/probes/Makefile index c40139e9ca47..3d837eb5f9be 100644 --- a/arch/riscv/kernel/probes/Makefile +++ b/arch/riscv/kernel/probes/Makefile @@ -3,4 +3,5 @@ obj-$(CONFIG_KPROBES) += kprobes.o decode-insn.o simulate-insn.o obj-$(CONFIG_RETHOOK) += rethook.o rethook_trampoline.o obj-$(CONFIG_KPROBES_ON_FTRACE) += ftrace.o obj-$(CONFIG_UPROBES) += uprobes.o decode-insn.o simulate-insn.o +obj-$(CONFIG_OPTPROBES) += opt.o opt_trampoline.o CFLAGS_REMOVE_simulate-insn.o = $(CC_FLAGS_FTRACE) diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c new file mode 100644 index 000000000000..56c8a227c857 --- /dev/null +++ b/arch/riscv/kernel/probes/opt.c @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Kernel Probes Jump Optimization (Optprobes) + * + * Copyright (C) Guokai Chen, 2022 + * Author: Guokai Chen chenguokai17@mails.ucas.ac.cn + */ + +#define pr_fmt(fmt) "optprobe: " fmt + +#include +#include + +int arch_prepared_optinsn(struct arch_optimized_insn *optinsn) +{ + return 0; +} + +int arch_check_optimized_kprobe(struct optimized_kprobe *op) +{ + return 0; +} + +int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, + struct kprobe *orig) +{ + return 0; +} + +void arch_remove_optimized_kprobe(struct optimized_kprobe *op) +{ +} + +void arch_optimize_kprobes(struct list_head *oplist) +{ +} + +void arch_unoptimize_kprobes(struct list_head *oplist, + struct list_head *done_list) +{ +} + +void arch_unoptimize_kprobe(struct optimized_kprobe *op) +{ +} + +int arch_within_optimized_kprobe(struct optimized_kprobe *op, + kprobe_opcode_t *addr) +{ + return 0; +} diff --git a/arch/riscv/kernel/probes/opt_trampoline.S b/arch/riscv/kernel/probes/opt_trampoline.S new file mode 100644 index 000000000000..16160c4367ff --- /dev/null +++ b/arch/riscv/kernel/probes/opt_trampoline.S @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2022 Guokai Chen + */ + +#include + +#incldue +#include + +SYM_ENTRY(optprobe_template_entry, SYM_L_GLOBAL, SYM_A_NONE) +SYM_ENTRY(optprobe_template_end, SYM_L_GLOBAL, SYM_A_NONE) From patchwork Fri Jan 27 13:05:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49380 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825844wrn; Fri, 27 Jan 2023 05:13:29 -0800 (PST) X-Google-Smtp-Source: AMrXdXv9OPD2hq7GXICjr8tixdF6538wk3k3uaYDmMrogZiVeOR1le0G/ddIhHPAo0fVsYNcVHPY X-Received: by 2002:a17:906:9f07:b0:877:60d7:3181 with SMTP id fy7-20020a1709069f0700b0087760d73181mr34928587ejc.17.1674825209665; Fri, 27 Jan 2023 05:13:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825209; cv=none; d=google.com; s=arc-20160816; b=wpAgKucUiJMEvI9Ph8GB+ZGLIILzYf9M2xPsJLzit09Oy+oLXmkxabflCwSUEOZaL/ BHg2pkiuSWg28E+EbDxKh49otNOFQlUFRqh8nw8EHpt86Sc7MOebWYhGG0SnHpMRNnQm Ngnqt4x/YWCRS/8+RT8xmC9qU9p7oBJx2tyUJ8e6jusNUaQmQK0SYJTYuptP4oT/YCID OEv1wUd5B5sbwugxRWKz7crOPBup6CPEK2vNGIXN1ho8eKQJycuRhFsUj7h9ci3KR1dA xUOB/I6DtWIq7nRP3ckK+UZGX+A0JJn+vwGNIuuC8LO+H4SHfNto6J8BBSZTPQTh/6to RV5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=m0bWMOxb3QBuF7pulRUyjksg1qiYGEZKNI2YN9WGi2k=; b=0/HarRvfV1pWKQJEs9WBigfQXThkeI7sePhIgeZkTvLDUNF6MfVPOXZbiXaXi4HC8k 4e4yM2fscuQhUGm09YgUXdGnK4Vwoe2jlqFswSuPcrXf25dqPWixl2dNmHnl5hjOepkC /4Blgj16tJCm0MLc5FIAGm3gPW+tJYzMpyj3odVk1Wc92bSxxbKgDBHyoFylNPOdBcol xaYxY26dPAZFD8O59hMsKeICziHe223fRcWUpQWOPfiEwKDd6m1bxtSvmYMd/wCjXXPb lSoe42B84hCBILEVqIrbU3jneUDn4cpO/KhHTKFAq2UBAqkSOJf6tCa3ThzOEfOcA5P1 JEwQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s26-20020a170906221a00b007c173f77500si5237947ejs.427.2023.01.27.05.13.05; Fri, 27 Jan 2023 05:13:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232456AbjA0NGm (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230458AbjA0NGb (ORCPT ); Fri, 27 Jan 2023 08:06:31 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A874B7D6C5 for ; Fri, 27 Jan 2023 05:06:28 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S6; Fri, 27 Jan 2023 21:06:08 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 02/13] riscv/kprobe: Allocate detour buffer from module region Date: Fri, 27 Jan 2023 21:05:30 +0800 Message-Id: <20230127130541.1250865-3-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S6 X-Coremail-Antispam: 1UD129KBjvJXoW7ur1kJFWrtw13AFyfAFWrKrg_yoW8uFyxpF 4DCr45XrZ8Aw13GrWayw1v9r9YqFykJr47KrW2vr10vw43tr47Aw4v934Fqr95JrWj9Fy5 Wr4q9ryYkFWUA3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQj14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYx C7M4IIrI8v6xkF7I0E8cxan2IY04v7MxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCF x2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14 v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY 67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2 IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_ Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0pREtCwUUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiCgUEE2PTtMIZwAABsG X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181518946929876?= X-GMAIL-MSGID: =?utf-8?q?1756181518946929876?= From: Liao Chang To address the limitation of PC-relative branch instruction on riscv architecture, detour buffer slot used for optprobes has to be allocated at virtual address that can access from kernel and modules text via AUIPC/JALR. For the time being, the vmalloc region is far from kernel/modules text, the distance between them is half of kernel address space [1], which can't transfer control to 32-bit pc-relative address, hence it needs to override the alloc_optinsn_page() to allocate detour buffer from module region. [1] Documentation/riscv/vm-layout.rst Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai --- arch/riscv/kernel/probes/kprobes.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index f21592d20306..e1856b04db04 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -84,6 +85,29 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p) } #ifdef CONFIG_MMU +#if defined(CONFIG_OPTPROBES) && defined(CONFIG_64BIT) +void *alloc_optinsn_page(void) +{ + void *page; + + page = __vmalloc_node_range(PAGE_SIZE, 1, MODULES_VADDR, + MODULES_END, GFP_KERNEL, + PAGE_KERNEL, 0, NUMA_NO_NODE, + __builtin_return_address(0)); + if (!page) + return NULL; + + set_vm_flush_reset_perms(page); + /* + * First make the page read-only, and only then make it executable to + * prevent it from being W+X in between. + */ + set_memory_rox((unsigned long)page, 1); + + return page; +} +#endif + void *alloc_insn_page(void) { return __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START, VMALLOC_END, From patchwork Fri Jan 27 13:05:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49379 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825787wrn; Fri, 27 Jan 2023 05:13:20 -0800 (PST) X-Google-Smtp-Source: AMrXdXuJ7cGpAq9OZEcJlPscwO9NAIofUZdzkIL5ekKL8zLgZgsDcQf8OLLHD7dT0aSqOkT0px6n X-Received: by 2002:a17:906:2b57:b0:86d:3c59:73f8 with SMTP id b23-20020a1709062b5700b0086d3c5973f8mr40426807ejg.34.1674825200161; Fri, 27 Jan 2023 05:13:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825200; cv=none; d=google.com; s=arc-20160816; b=ywFCjlVQlN58bC3H33plBwlilM1mCqWL3J/aNHDhTevjtjvssv2n2Mmltr/bu+Ukbt 5/tkINawUufEqa5ib/sZsZ9HHpGI+dFEAmvILb9iv+eDV8jib0LTEgwjl50EDuQ0WudB 1bN2IMwS31OcQ3pVFa0P2J7OZ1nnWW0AvWL2iHeW+gdhDYgeoO2adIlcA9Z/77RYbspt UBypL5dQHsAQNRkDx9T1GDPMabos+Jzd7bfzoephnDwBPaXs8JGgmNpGfwAAQRvTcsut hVZt2OUwlRRgJygtdxkQ76MeqFWJESNIBKJBw8Wd0hpOJbfdCtEqdb+jqdVfz7UDwgR7 vl2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=qLQFK2hxdYf8s7Cm4MUr/KP7sHi1l3PHbvvGLXAwE4U=; b=D2UzKFfMxORetd2XSZ1QoKo2db0Wwl6E2lXyVQ67boPASgE1A+SZfPQkKE2eTk7oBX 5IC8lZJZBaZRn3SnzwGGKoK7TpJy27upxRN9h9vhLZMeDKMCDE5s/rjqjF5cAJYCjMvp Z/Hn0z/bwta/fXL7iynNZsm0/UifB/M1Y8faA9UkpUJXnw+vMMG/KfE07mkoqezOUBl/ vuxJNXU2tXXlkIf73U2iHk0zBs7vmXALU+u7f3bqHvVR3/LbOBy5zVyekScvMy1tv0tv Dtnzy80ZxAkrrAaCFJg6WUBMQeIcPPk8BETBQNPd1+QCfp+wow5H7wQv3IHGNQnBaPsq HQrw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gc8-20020a1709072b0800b0087329ff5940si4549034ejc.770.2023.01.27.05.12.56; Fri, 27 Jan 2023 05:13:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233745AbjA0NGh (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47588 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234523AbjA0NGa (ORCPT ); Fri, 27 Jan 2023 08:06:30 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 6561A7CCB1 for ; Fri, 27 Jan 2023 05:06:27 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S7; Fri, 27 Jan 2023 21:06:09 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 03/13] riscv/kprobe: Add skeleton for preparing optimized kprobe Date: Fri, 27 Jan 2023 21:05:31 +0800 Message-Id: <20230127130541.1250865-4-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S7 X-Coremail-Antispam: 1UD129KBjvJXoWxuFyUWr4fGw18ZFy8ur1Dtrb_yoWrXryrpF s3Cw45trWDJa93urWfZwsYvrySyFsYqr47CrW7J345Zr17Jr90va97tr1YqF9xGrWF9rW3 JFsYyry5uFW3ZFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQj14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYx C7M4IIrI8v6xkF7I0E8cxan2IY04v7MxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCF x2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14 v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY 67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2 IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_ Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JU3DG5UUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiCggEE2PTtMIZxAAAsO X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181509476691088?= X-GMAIL-MSGID: =?utf-8?q?1756181509476691088?= From: Liao Chang The skeleton for preparing optprobe is consist of three major parts: - Check if kprobe satisfies the requirements of optimization. - Search two registers to form AUIPC/JALR instructions. - Prepare detour buffer for optimized kprobe. To avoid introducing too much code in single patch just add some dummy implementaion for compilation. Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai --- arch/riscv/kernel/probes/opt.c | 98 +++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c index 56c8a227c857..c03cdb1512a6 100644 --- a/arch/riscv/kernel/probes/opt.c +++ b/arch/riscv/kernel/probes/opt.c @@ -10,6 +10,53 @@ #include #include +#include + +static int in_auipc_jalr_range(long val) +{ +#ifdef CONFIG_ARCH_RV32I + return 1; +#else + /* + * Note that the set of address offsets that can be formed + * by pairing LUI with LD, AUIPC with JALR, etc. RV64I is + * [−2^31−2^11, 2^31−2^11−1]. + */ + return ((-(1L << 31) - (1L << 11)) <= val) && + (val < ((1L << 31) - (1L << 11))); +#endif +} + +/* + * Copy optprobe assembly code template into detour buffer and modify some + * instructions for each kprobe. + */ +static void prepare_detour_buffer(kprobe_opcode_t *code, kprobe_opcode_t *slot, + int rd, struct optimized_kprobe *op, + kprobe_opcode_t opcode) +{ +} + +/* + * In RISC-V ISA, AUIPC/JALR clobber one register to form target address, + * inspired by register renaming in OoO processor, this involves search + * backward that is not previously used as a source register and is used + * as a destination register before any branch or jump instruction. + */ +static void find_free_registers(struct kprobe *kp, struct optimized_kprobe *op, + int *rd, int *ra) +{ +} + +/* + * The kprobe based on breakpoint just requires the instrumented instruction + * supports execute out-of-line or simulation, besides that, optimized kprobe + * requires no near instruction jump to any instruction replaced by AUIPC/JALR. + */ +static bool can_optimize(unsigned long paddr, struct optimized_kprobe *op) +{ + return false; +} int arch_prepared_optinsn(struct arch_optimized_insn *optinsn) { @@ -24,7 +71,56 @@ int arch_check_optimized_kprobe(struct optimized_kprobe *op) int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *orig) { - return 0; + long rel; + int rd = 0, ra = 0, ret; + kprobe_opcode_t *code = NULL, *slot = NULL; + + if (!can_optimize((unsigned long)orig->addr, op)) + return -EILSEQ; + + code = kzalloc(MAX_OPTINSN_SIZE, GFP_KERNEL); + slot = get_optinsn_slot(); + if (!code || !slot) { + ret = -ENOMEM; + goto on_error; + } + + /* Check if the detour buffer is in the 32-bit pc-relative range. */ + rel = (unsigned long)slot - (unsigned long)orig->addr; + if (!in_auipc_jalr_range(rel)) { + ret = -ERANGE; + goto on_error; + } + + /* + * Search two free registers, rd is used to form AUIPC/JALR jumping + * to detour buffer, ra is used to form JR jumping back from detour + * buffer. + */ + find_free_registers(orig, op, &rd, &ra); + if (rd == 0 || ra == 0) { + ret = -EILSEQ; + goto on_error; + } + + op->optinsn.rd = rd; + prepare_detour_buffer(code, slot, ra, op, orig->opcode); + + ret = patch_text_nosync((void *)slot, code, MAX_OPTINSN_SIZE); + if (!ret) { + op->optinsn.insn = slot; + kfree(code); + return 0; + } + +on_error: + if (slot) { + free_optinsn_slot(slot, 0); + op->optinsn.insn = NULL; + op->optinsn.length = 0; + } + kfree(code); + return ret; } void arch_remove_optimized_kprobe(struct optimized_kprobe *op) From patchwork Fri Jan 27 13:05:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49383 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825934wrn; Fri, 27 Jan 2023 05:13:42 -0800 (PST) X-Google-Smtp-Source: AMrXdXtJDhsA7GMbnn16NvKXjM9rIdIC4P1udoe9p3CaCOCUDbjJLrQe8HKvxVSIsfBvxWU6Gni2 X-Received: by 2002:a17:906:df49:b0:84d:34fa:f1a4 with SMTP id if9-20020a170906df4900b0084d34faf1a4mr35684339ejc.60.1674825222315; Fri, 27 Jan 2023 05:13:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825222; cv=none; d=google.com; s=arc-20160816; b=wLGAeui9N3Cy39uZyk288TLeHh587NgnMKD9UiaMhNEPBDnSfFm52Unzhhmy2XNUK5 XlRHohU4CH5IzQ9Bmq00qs5OxfRuFstH0UqHSXBo0yEWvOsOWGKAwWPpsb3q8bv6oDrU jq6LW29lt+r0CzIaGivpDwBV4Y2u5wDJj3odiQA5b4CKCWB/sCttZatG3rx//6LgrukD SkZwCI05pPFxWTOGSRdHAAVwyhmcZFYoS9Aizqm2JwwDS0o7/RQNpf9vnD/EFKsMya3x FgRRX08Nt2dJhv625wDHxSa7zVr/zj4kMYz6TZHyaAQ+SKP3MzFmymr7+r9D0+wP4L7/ oagQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=s27kBeIxIWjp3bBnyZgDfFptHt3OLkeOQ/IC1VPEjGo=; b=YiIZeQYwi+kn5RqHsg7GAopGSnWwHzy8eTMCBNCiCMmSdIFgIyau9Z0iwIEhjb8+KG PYG+1C8Ehw5YdG5WSibtuMF+3gRguVST9WawXbW8oRPML8gQ/YfbPL2bzZuS8bumZoLg qi5eSwhZNmk+8hRnU/wRWpvj/RCw6ycQsfOSI+XGUg0q00gjY1ox7ukn/VY6Bw4ii3r6 dfg2xyy68Wm8P92PDVmdTkZ6Ha6keLTYOKlJldpzY1+tAvUAlgdWaGaseZopEsac1dFp EqPLnDs87X75K9qF4t4mZac0xshlrOtS2Ld+e2ZoXmysPjOd0H5B52ddh2v4B2jQFfy7 +BWw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id uk42-20020a170907ca2a00b0087877581bf9si4339105ejc.266.2023.01.27.05.13.18; Fri, 27 Jan 2023 05:13:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234582AbjA0NGu (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234540AbjA0NGc (ORCPT ); Fri, 27 Jan 2023 08:06:32 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 3D2F921A2C for ; Fri, 27 Jan 2023 05:06:28 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S8; Fri, 27 Jan 2023 21:06:09 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 04/13] riscv/kprobe: Add common RVI and RVC instruction decoder code Date: Fri, 27 Jan 2023 21:05:32 +0800 Message-Id: <20230127130541.1250865-5-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S8 X-Coremail-Antispam: 1UD129KBjvJXoW3XF47tr4rZFWfGrW7WF48WFg_yoW3Ar15pa n5Crn0v3ykGF93urZ7tr48CrWYqw4rGw4DKay0ga1ayF12qr4UXr9aqry3tF4kWFWFgr47 CFZ8JrWkG3y2y3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQ214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr 1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j 6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFc xC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_ Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2 IErcIFxwACI402YVCY1x02628vn2kIc2xKxwCY02Avz4vE14v_Xr4l42xK82IYc2Ij64vI r41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8Gjc xK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0 cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8V AvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E 14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfU0LvKUUUUU X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiAwcEE2PTrr4g0AAAsD X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181532370323733?= X-GMAIL-MSGID: =?utf-8?q?1756181532370323733?= From: Liao Chang These RVI and RVC instruction decoder are used in the free register searching algorithm, each instruction of instrumented function needs to decode and test if it contains a free register to form AUIPC/JALR. For RVI instruction format, the position and length of rs1/rs2/rd/opcode parts are uniform [1], but RVC instruction formats are complicated, so it addresses a series of functions to decode rs1/rs2/rd for RVC [1]. [1] https://github.com/riscv/riscv-isa-manual/releases Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai --- arch/riscv/include/asm/bug.h | 5 +- arch/riscv/kernel/probes/decode-insn.h | 148 +++++++++++++++++++++++ arch/riscv/kernel/probes/simulate-insn.h | 42 +++++++ 3 files changed, 194 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/bug.h b/arch/riscv/include/asm/bug.h index 1aaea81fb141..9c33d3b58225 100644 --- a/arch/riscv/include/asm/bug.h +++ b/arch/riscv/include/asm/bug.h @@ -19,11 +19,14 @@ #define __BUG_INSN_32 _UL(0x00100073) /* ebreak */ #define __BUG_INSN_16 _UL(0x9002) /* c.ebreak */ +#define RVI_INSN_LEN 4UL +#define RVC_INSN_LEN 2UL + #define GET_INSN_LENGTH(insn) \ ({ \ unsigned long __len; \ __len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? \ - 4UL : 2UL; \ + RVI_INSN_LEN : RVC_INSN_LEN; \ __len; \ }) diff --git a/arch/riscv/kernel/probes/decode-insn.h b/arch/riscv/kernel/probes/decode-insn.h index 42269a7d676d..785b023a62ea 100644 --- a/arch/riscv/kernel/probes/decode-insn.h +++ b/arch/riscv/kernel/probes/decode-insn.h @@ -3,6 +3,7 @@ #ifndef _RISCV_KERNEL_KPROBES_DECODE_INSN_H #define _RISCV_KERNEL_KPROBES_DECODE_INSN_H +#include #include #include @@ -15,4 +16,151 @@ enum probe_insn { enum probe_insn __kprobes riscv_probe_decode_insn(probe_opcode_t *addr, struct arch_probe_insn *asi); +#ifdef CONFIG_KPROBES + +static inline u16 rvi_rs1(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 15) & 0x1f); +} + +static inline u16 rvi_rs2(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 20) & 0x1f); +} + +static inline u16 rvi_rd(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 7) & 0x1f); +} + +static inline s32 rvi_branch_imme(kprobe_opcode_t opcode) +{ + u32 imme = 0; + + imme |= (((opcode >> 8) & 0xf) << 1) | + (((opcode >> 25) & 0x3f) << 5) | + (((opcode >> 7) & 0x1) << 11) | + (((opcode >> 31) & 0x1) << 12); + + return sign_extend32(imme, 13); +} + +static inline s32 rvi_jal_imme(kprobe_opcode_t opcode) +{ + u32 imme = 0; + + imme |= (((opcode >> 21) & 0x3ff) << 1) | + (((opcode >> 20) & 0x1) << 11) | + (((opcode >> 12) & 0xff) << 12) | + (((opcode >> 31) & 0x1) << 20); + + return sign_extend32(imme, 21); +} + +#ifdef CONFIG_RISCV_ISA_C +static inline u16 rvc_r_rs1(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 2) & 0x1f); +} + +static inline u16 rvc_r_rs2(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 2) & 0x1f); +} + +static inline u16 rvc_r_rd(kprobe_opcode_t opcode) +{ + return rvc_r_rs1(opcode); +} + +static inline u16 rvc_i_rs1(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 7) & 0x1f); +} + +static inline u16 rvc_i_rd(kprobe_opcode_t opcode) +{ + return rvc_i_rs1(opcode); +} + +static inline u16 rvc_ss_rs2(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 2) & 0x1f); +} + +static inline u16 rvc_l_rd(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 2) & 0x7); +} + +static inline u16 rvc_l_rs(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 7) & 0x7); +} + +static inline u16 rvc_s_rs2(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 2) & 0x7); +} + +static inline u16 rvc_s_rs1(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 7) & 0x7); +} + +static inline u16 rvc_a_rs2(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 2) & 0x7); +} + +static inline u16 rvc_a_rs1(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 7) & 0x7); +} + +static inline u16 rvc_a_rd(kprobe_opcode_t opcode) +{ + return rvc_a_rs1(opcode); +} + +static inline u16 rvc_b_rd(kprobe_opcode_t opcode) +{ + return (u16)((opcode >> 7) & 0x7); +} + +static inline u16 rvc_b_rs(kprobe_opcode_t opcode) +{ + return rvc_b_rd(opcode); +} + +static inline s32 rvc_branch_imme(kprobe_opcode_t opcode) +{ + u32 imme = 0; + + imme |= (((opcode >> 3) & 0x3) << 1) | + (((opcode >> 10) & 0x3) << 3) | + (((opcode >> 2) & 0x1) << 5) | + (((opcode >> 5) & 0x3) << 6) | + (((opcode >> 12) & 0x1) << 8); + + return sign_extend32(imme, 9); +} + +static inline s32 rvc_jal_imme(kprobe_opcode_t opcode) +{ + u32 imme = 0; + + imme |= (((opcode >> 3) & 0x3) << 1) | + (((opcode >> 11) & 0x1) << 4) | + (((opcode >> 2) & 0x1) << 5) | + (((opcode >> 7) & 0x1) << 6) | + (((opcode >> 6) & 0x1) << 7) | + (((opcode >> 9) & 0x3) << 8) | + (((opcode >> 8) & 0x1) << 10) | + (((opcode >> 12) & 0x1) << 11); + + return sign_extend32(imme, 12); +} +#endif /* CONFIG_KPROBES */ +#endif /* CONFIG_RISCV_ISA_C */ #endif /* _RISCV_KERNEL_KPROBES_DECODE_INSN_H */ diff --git a/arch/riscv/kernel/probes/simulate-insn.h b/arch/riscv/kernel/probes/simulate-insn.h index a19aaa0feb44..e89747dfabbb 100644 --- a/arch/riscv/kernel/probes/simulate-insn.h +++ b/arch/riscv/kernel/probes/simulate-insn.h @@ -28,4 +28,46 @@ bool simulate_branch(u32 opcode, unsigned long addr, struct pt_regs *regs); bool simulate_jal(u32 opcode, unsigned long addr, struct pt_regs *regs); bool simulate_jalr(u32 opcode, unsigned long addr, struct pt_regs *regs); +/* RVC(S) instructions contain rs1 and rs2 */ +__RISCV_INSN_FUNCS(c_sq, 0xe003, 0xa000); +__RISCV_INSN_FUNCS(c_sw, 0xe003, 0xc000); +__RISCV_INSN_FUNCS(c_sd, 0xe003, 0xe000); +/* RVC(A) instructions contain rs1 and rs2 */ +__RISCV_INSN_FUNCS(c_sub, 0xfc63, 0x8c01); +__RISCV_INSN_FUNCS(c_subw, 0xfc43, 0x9c01); +/* RVC(L) instructions contain rs1 */ +__RISCV_INSN_FUNCS(c_lq, 0xe003, 0x2000); +__RISCV_INSN_FUNCS(c_lw, 0xe003, 0x4000); +__RISCV_INSN_FUNCS(c_ld, 0xe003, 0x6000); +/* RVC(I) instructions contain rs1 */ +__RISCV_INSN_FUNCS(c_addi, 0xe003, 0x0001); +__RISCV_INSN_FUNCS(c_addiw, 0xe003, 0x2001); +__RISCV_INSN_FUNCS(c_addi16sp, 0xe183, 0x6101); +__RISCV_INSN_FUNCS(c_slli, 0xe003, 0x0002); +/* RVC(B) instructions contain rs1 */ +__RISCV_INSN_FUNCS(c_sri, 0xe803, 0x8001); +__RISCV_INSN_FUNCS(c_andi, 0xec03, 0x8801); +/* RVC(SS) instructions contain rs2 */ +__RISCV_INSN_FUNCS(c_sqsp, 0xe003, 0xa002); +__RISCV_INSN_FUNCS(c_swsp, 0xe003, 0xc002); +__RISCV_INSN_FUNCS(c_sdsp, 0xe003, 0xe002); +/* RVC(R) instructions contain rs2 and rd */ +__RISCV_INSN_FUNCS(c_mv, 0xf003, 0x8002); +/* RVC(I) instructions contain sp and rd */ +__RISCV_INSN_FUNCS(c_lqsp, 0xe003, 0x2002); +__RISCV_INSN_FUNCS(c_lwsp, 0xe003, 0x4002); +__RISCV_INSN_FUNCS(c_ldsp, 0xe003, 0x6002); +/* RVC(CW) instructions contain sp and rd */ +__RISCV_INSN_FUNCS(c_addi4spn, 0xe003, 0x0000); +/* RVC(I) instructions contain rd */ +__RISCV_INSN_FUNCS(c_li, 0xe003, 0x4001); +__RISCV_INSN_FUNCS(c_lui, 0xe003, 0x6001); + +__RISCV_INSN_FUNCS(arith_rr, 0x77, 0x33); +__RISCV_INSN_FUNCS(arith_ri, 0x77, 0x13); +__RISCV_INSN_FUNCS(lui, 0x7f, 0x37); +__RISCV_INSN_FUNCS(load, 0x7f, 0x03); +__RISCV_INSN_FUNCS(store, 0x7f, 0x23); +__RISCV_INSN_FUNCS(amo, 0x7f, 0x2f); + #endif /* _RISCV_KERNEL_PROBES_SIMULATE_INSN_H */ From patchwork Fri Jan 27 13:05:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49384 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp826242wrn; Fri, 27 Jan 2023 05:14:15 -0800 (PST) X-Google-Smtp-Source: AMrXdXtKvJ6nRAK8Oo9E2EZqccIuEPUJ4ZLyXRtUbKwUhRAftZIs8oj3KiBnYPYPcw7YQonjRldd X-Received: by 2002:a17:907:cca4:b0:7c0:f719:838d with SMTP id up36-20020a170907cca400b007c0f719838dmr44095977ejc.36.1674825254885; Fri, 27 Jan 2023 05:14:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825254; cv=none; d=google.com; s=arc-20160816; b=tN0kIwtc7u33+IThu258eEEWpgJTHzfxpy245LInm5Ml/ACdavVfMUFN+yI/LIkhuu yQaY2C0y/8FL/ZK7bf77alfXyCJlGX00wSpgzE1cnKsKXBelERNcDC8i109KZ5ndtPtx B12vjsTEhD3KLwsDlIIKn0F3W1apbDgJpoWAqshqm8VQRVsOSRNMVBVKjt16YxrKv0nK 0q7HyWm58akEO/7v2xs1GPGg7lQazS91sLkfDrywap1HnV1ncvaPgVynjTTtHkja8gdG j8VWhGtQrdgD1Fw8u96enPCniuCpGYEi05eZU+pCvVK2PyYDgKDe3qFs2mPoB634TxaC m+dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=kovyLoJJw+NwHDoBemt4xFYD3G9Hbi2Ja6hZ4ilfit8=; b=SWi6OuYqN/B6AhATQQfps6KEXmUXJ0HFGZGFlkGa0hAn30Hm/sRDnfWXOruhwHT9Bh Dhj5C0fXE9sKjjxDFSJdNkXlfO6Be+VFf4n3d7yCmh4loPrry1zlqkL8QwvQjTh4BqGl 150oZpZOp7M0MIeq1J+Rq1F3F9o++Ec76vaRgC/pbB6GDGe85HBDncCJpdRZbQmfy5OG 0u3GxZ0wjmJpEmNF7MN5ACR5ThMgpY2hjqzTpKoeb/jscfbXOrPf0rWs1rdHIAyaXxSa znMM9kh9Hu/quCWX4XwyNv6n+gh/L2BIjmiqHI/loN2y842fpma4inytmJlHI41pC6IC clug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ex8-20020a170907954800b007c17f750b75si4436660ejc.796.2023.01.27.05.13.50; Fri, 27 Jan 2023 05:14:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234193AbjA0NG5 (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233988AbjA0NGg (ORCPT ); Fri, 27 Jan 2023 08:06:36 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 6D1437CCA3 for ; Fri, 27 Jan 2023 05:06:29 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S9; Fri, 27 Jan 2023 21:06:09 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 05/13] riscv/kprobe: Introduce free register(s) searching algorithm Date: Fri, 27 Jan 2023 21:05:33 +0800 Message-Id: <20230127130541.1250865-6-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S9 X-Coremail-Antispam: 1UD129KBjvJXoW3WrWrKry5Kr43JF45Jr18Xwb_yoWfXw45pF ZxWw4FqF4UtFs5W3y3tr1kZrWSgFs3Grs8Cr15t3yUZr43G3ySqF4vga43Zr1DJF13Zr48 Gr4Y9rWI9r4DAFUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr 1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j 6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFc xC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_ Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2 IErcIFxwACI402YVCY1x02628vn2kIc2xKxwCY02Avz4vE14v_Xr4l42xK82IYc2Ij64vI r41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8Gjc xK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0 cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY6xAIw2 0EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x02 67AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUjTmh7UUUUU== X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiAwkEE2PTrr4g0QAAsM X-Spam-Status: No, score=1.2 required=5.0 tests=BAYES_00,DRUGS_ERECTILE, DRUGS_ERECTILE_OBFU,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181566188431546?= X-GMAIL-MSGID: =?utf-8?q?1756181566188431546?= From: Liao Chang To do jump optimization, it needs to clobber two integer GPRs, the first one is used to form AUIPC/JALR jumping to detour buffer, the second one is used to form JR in detour buffer. Since kprobe can be installed anywhere of kernel/module text, hence the register being clobbered needs to be chosen carefully to avoid changing the original logic. The algorithm for finding free register is inspired by the register renaming in modern processors. From the perspective of register renaming, a register could be represented as two different registers if two neighbor instructions both write to it but no one ever reads it. Extending this fact a register is considered to be free if it has never been read since the first write on it in the execution flow. Let's use the example below to explain how the algorithm work. Given kernel is RVI and RCV hybrid binary, and one kprobe is instrumented at the entry of function idle_dummy(). Before Optimized Detour buffer : ... #1 add sp,sp,-16 auipc a0, #? add sp,sp,-16 #2 sd s0,8(sp) sd s0,8(sp) #3 addi s0,sp,16 jalr a0, #?(a0) addi s0,sp,16 #4 ld s0,8(sp) ld s0,8(sp) #5 li a0,0 li a0,0 auipc a0, #? #6 addi sp,sp,16 addi sp,sp,16 jr x0, #?(a0) #7 ret ret To optimize kprobe, it used to patch the first 8 bytes with AUIPC/JALR, because from #1 to #7, a0 is the only register that satisfies condition: - Never been read before write - Never been updated in detour buffer So a0 will be chosen to form AUIPC/JALR and JR. Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai --- arch/riscv/kernel/probes/opt.c | 221 +++++++++++++++++++++++++++++++++ 1 file changed, 221 insertions(+) diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c index c03cdb1512a6..d38ed1a52c93 100644 --- a/arch/riscv/kernel/probes/opt.c +++ b/arch/riscv/kernel/probes/opt.c @@ -12,6 +12,9 @@ #include #include +#include "simulate-insn.h" +#include "decode-insn.h" + static int in_auipc_jalr_range(long val) { #ifdef CONFIG_ARCH_RV32I @@ -37,15 +40,233 @@ static void prepare_detour_buffer(kprobe_opcode_t *code, kprobe_opcode_t *slot, { } +/* Registers the first usage of which is the destination of instruction */ +#define WRITE_ON(reg) \ + (*write |= (((*read >> (reg)) ^ 1UL) & 1) << (reg)) +/* Registers the first usage of which is the source of instruction */ +#define READ_ON(reg) \ + (*read |= (((*write >> (reg)) ^ 1UL) & 1) << (reg)) + /* * In RISC-V ISA, AUIPC/JALR clobber one register to form target address, * inspired by register renaming in OoO processor, this involves search * backward that is not previously used as a source register and is used * as a destination register before any branch or jump instruction. */ +static void find_register(unsigned long start, unsigned long end, + unsigned long *write, unsigned long *read) +{ + kprobe_opcode_t insn; + unsigned long addr, offset = 0UL; + + for (addr = start; addr < end; addr += offset) { + insn = *(kprobe_opcode_t *)addr; + offset = GET_INSN_LENGTH(insn); + +#ifdef CONFIG_RISCV_ISA_C + if (offset == RVI_INSN_LEN) + goto is_rvi; + + insn &= __COMPRESSED_INSN_MASK; + /* Stop searching until any control transfer instruction */ + if (riscv_insn_is_c_ebreak(insn) || riscv_insn_is_c_j(insn)) + break; + + if (riscv_insn_is_c_jal(insn)) { + /* The rd of C.JAL is x1 by default */ + WRITE_ON(1); + break; + } + + if (riscv_insn_is_c_jr(insn)) { + READ_ON(rvc_r_rs1(insn)); + break; + } + + if (riscv_insn_is_c_jalr(insn)) { + READ_ON(rvc_r_rs1(insn)); + /* The rd of C.JALR is x1 by default */ + WRITE_ON(1); + break; + } + + if (riscv_insn_is_c_beqz(insn) || riscv_insn_is_c_bnez(insn)) { + READ_ON(rvc_b_rs(insn)); + break; + } + + /* + * Decode RVC instructions to find out some destination + * registers never be used as a source register. + */ + if (riscv_insn_is_c_sub(insn) || riscv_insn_is_c_subw(insn)) { + READ_ON(rvc_a_rs1(insn)); + READ_ON(rvc_a_rs2(insn)); + continue; + } else if (riscv_insn_is_c_sq(insn) || + riscv_insn_is_c_sw(insn) || + riscv_insn_is_c_sd(insn)) { + READ_ON(rvc_s_rs1(insn)); + READ_ON(rvc_s_rs2(insn)); + continue; + } else if (riscv_insn_is_c_addi16sp(insn) || + riscv_insn_is_c_addi(insn) || + riscv_insn_is_c_addiw(insn) || + riscv_insn_is_c_slli(insn)) { + READ_ON(rvc_i_rs1(insn)); + continue; + } else if (riscv_insn_is_c_sri(insn) || + riscv_insn_is_c_andi(insn)) { + READ_ON(rvc_b_rs(insn)); + continue; + } else if (riscv_insn_is_c_sqsp(insn) || + riscv_insn_is_c_swsp(insn) || + riscv_insn_is_c_sdsp(insn)) { + READ_ON(rvc_ss_rs2(insn)); + /* The rs2 of C.SQSP/SWSP/SDSP are x2 by default */ + READ_ON(2); + continue; + } else if (riscv_insn_is_c_mv(insn)) { + READ_ON(rvc_r_rs2(insn)); + WRITE_ON(rvc_r_rd(insn)); + } else if (riscv_insn_is_c_addi4spn(insn)) { + /* The rs of C.ADDI4SPN is x2 by default */ + READ_ON(2); + WRITE_ON(rvc_l_rd(insn)); + } else if (riscv_insn_is_c_lq(insn) || + riscv_insn_is_c_lw(insn) || + riscv_insn_is_c_ld(insn)) { + /* FIXME: c.lw/c.ld share opcode with c.flw/c.fld */ + READ_ON(rvc_l_rs(insn)); + WRITE_ON(rvc_l_rd(insn)); + } else if (riscv_insn_is_c_lqsp(insn) || + riscv_insn_is_c_lwsp(insn) || + riscv_insn_is_c_ldsp(insn)) { + /* + * FIXME: c.lwsp/c.ldsp share opcode with c.flwsp/c.fldsp + * The rs of C.LQSP/C.LWSP/C.LDSP is x2 by default. + */ + READ_ON(2); + WRITE_ON(rvc_i_rd(insn)); + } else if (riscv_insn_is_c_li(insn) || + riscv_insn_is_c_lui(insn)) { + WRITE_ON(rvc_i_rd(insn)); + } + + if ((*write > 1UL) && __builtin_ctzl(*write & ~1UL)) + return; +is_rvi: +#endif + /* Stop searching until any control transfer instruction */ + if (riscv_insn_is_branch(insn)) { + READ_ON(rvi_rs1(insn)); + READ_ON(rvi_rs2(insn)); + break; + } + + if (riscv_insn_is_jal(insn)) { + WRITE_ON(rvi_rd(insn)); + break; + } + + if (riscv_insn_is_jalr(insn)) { + READ_ON(rvi_rs1(insn)); + WRITE_ON(rvi_rd(insn)); + break; + } + + if (riscv_insn_is_system(insn)) { + /* csrrw, csrrs, csrrc */ + if (rvi_rs1(insn)) + READ_ON(rvi_rs1(insn)); + /* csrrwi, csrrsi, csrrci, csrrw, csrrs, csrrc */ + if (rvi_rd(insn)) + WRITE_ON(rvi_rd(insn)); + break; + } + + /* + * Decode RVI instructions to find out some destination + * registers never be used as a source register. + */ + if (riscv_insn_is_lui(insn) || riscv_insn_is_auipc(insn)) { + WRITE_ON(rvi_rd(insn)); + } else if (riscv_insn_is_arith_ri(insn) || + riscv_insn_is_load(insn)) { + READ_ON(rvi_rs1(insn)); + WRITE_ON(rvi_rd(insn)); + } else if (riscv_insn_is_arith_rr(insn) || + riscv_insn_is_store(insn) || + riscv_insn_is_amo(insn)) { + READ_ON(rvi_rs1(insn)); + READ_ON(rvi_rs2(insn)); + WRITE_ON(rvi_rd(insn)); + } + + if ((*write > 1UL) && __builtin_ctzl(*write & ~1UL)) + return; + } +} + static void find_free_registers(struct kprobe *kp, struct optimized_kprobe *op, int *rd, int *ra) { + unsigned long start, end; + /* + * Searching algorithm explanation: + * + * 1. Define two types of instruction areas firstly: + * + * +-----+ + * + + + * + + ---> instructions modified by optprobe, named 'O-Area'. + * + + + * +-----+ + * + + + * + + ---> instructions after optprobe, named 'K-Area'. + * + + + * + ~ + + * + * 2. There are two usages for each GPR in the given instruction area. + * + * - W: GPR is used as the RD oprand at first emergence. + * - R: GPR is used as the RS oprand at first emergence. + * + * Then there are 4 different usages for each GPR total: + * + * 1. Used as W in O-Area, Used as W in K-Area. + * 2. Used as W in O-Area, Used as R in K-Area. + * 3. Used as R in O-Area, Used as W in K-Area. + * 4. Used as R in O-Area, Used as R in K-Area. + * + * All registers satisfy #1 or #3 could be chosen to form 'AUIPC/JALR' + * jumping to detour buffer. + * + * All registers satisfy #1 or #2, could be chosen to form 'JR' jumping + * back from detour buffer. + */ + unsigned long kw = 0UL, kr = 0UL, ow = 0UL, or = 0UL; + + /* Search one free register used to form AUIPC/JALR */ + start = (unsigned long)&kp->opcode; + end = start + GET_INSN_LENGTH(kp->opcode); + find_register(start, end, &ow, &or); + + start = (unsigned long)kp->addr + GET_INSN_LENGTH(kp->opcode); + end = (unsigned long)kp->addr + op->optinsn.length; + find_register(start, end, &ow, &or); + + /* Search one free register used to form JR */ + find_register(end, (unsigned long)_end, &kw, &kr); + + if ((kw & ow) > 1UL) { + *rd = __builtin_ctzl((kw & ow) & ~1UL); + *ra = *rd; + return; + } + + *rd = ((kw | ow) == 1UL) ? 0 : __builtin_ctzl((kw | ow) & ~1UL); + *ra = (kw == 1UL) ? 0 : __builtin_ctzl(kw & ~1UL); } /* From patchwork Fri Jan 27 13:05:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49385 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp826451wrn; Fri, 27 Jan 2023 05:14:41 -0800 (PST) X-Google-Smtp-Source: AMrXdXvEly+6Yp+crp9XeRK+ogIohlpWAXV7c+cM7qyHnuB//Lv66KniveqiPZr2Yfk5GZoQSN6B X-Received: by 2002:a17:90a:b002:b0:229:932:a0f3 with SMTP id x2-20020a17090ab00200b002290932a0f3mr41681907pjq.27.1674825280931; Fri, 27 Jan 2023 05:14:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825280; cv=none; d=google.com; s=arc-20160816; b=OXX8ez6itbwqYkXPAIM5MriPYYrBNjXc2dAfCz74T2tjVMM8mwaxP0VRBPVn5PliTo dbtBoZarkps20uFmCrdsYOKf61TN8jJkHOaFNuqCamTTrjO0WD8LeOmatmbMGwOUy0Yp mfE+Qc11KSS5pkKIrKDfSwfxSRtVNh48NmOOa6YLJuUx0O7ShjbRa3ljd12cW/Bshx2o cjCUjWH+smhZOBFwywgZM7pjvXeVywgJUjT/Tkda6TfGkk2StEOb1JMESCI0dxCOJsLE IhJB2g55IIbcJFLXc+gFTDwpVSFyk1it9hmEl7s/0znkytZbPdCltFCcJ87vz0Nsfqom WFog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=RuvXXkQBPeHSE0mKE7gE6GKfzFfeKW/o2vBrLa0NQ+U=; b=WD0/hUBR6wzradDeRzxpr36djgmIfbWOVubzn54rqwf4mIL2EcKe/pZXFo17QdZsrk qtn1xXxu7DoSQu7SjlnEjvRHB8qsjr+Z1s/y++PpWBf9pylLIgLv0xEyvhWy+fFsCOvp k45YdzihObAhAOCrQxeun2PkPzxhnQSvTsgq/bRsfj4Iz3w9OXlY0V4gSkobJI2apthu a8eD6Mc/3Yg7ZXJPThZuNPAjyLscVoh0ngIfRGJDdUzbCA/ZSffVWgCefkH4ArSs4Gzf xD9eVk7E9NEa3s+WSDMl3o50e/wWU6rJDtrNNZI8UUp4+J62xvx6aVkgO6mM2+fgRQGs YUBA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m5-20020a17090a3f8500b00229ef929eaasi7499578pjc.181.2023.01.27.05.14.27; Fri, 27 Jan 2023 05:14:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234562AbjA0NGx (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234536AbjA0NGc (ORCPT ); Fri, 27 Jan 2023 08:06:32 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1C85A7D6DB for ; Fri, 27 Jan 2023 05:06:28 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S10; Fri, 27 Jan 2023 21:06:10 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 06/13] riscv/kprobe: Add code to check if kprobe can be optimized Date: Fri, 27 Jan 2023 21:05:34 +0800 Message-Id: <20230127130541.1250865-7-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S10 X-Coremail-Antispam: 1UD129KBjvJXoWxGr47Jw1rKFyUuw4kuF48Zwb_yoWrur1fpF s0ka4YqrWrJFZagrZayrs8Ar1ftws5Gr4xtFW7KrySyw12qr90gan2kr4YvwnxGF4F9r17 AF40yry5ur43ZrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr 1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j 6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFc xC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_ Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2 IErcIFxwACI402YVCY1x02628vn2kIc2xKxwCY02Avz4vE14v_Xr4l42xK82IYc2Ij64vI r41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8Gjc xK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0 cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY6xAIw2 0EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x02 67AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUjTmh7UUUUU== X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiCgUEE2PTtMIZwAADsE X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181593590278635?= X-GMAIL-MSGID: =?utf-8?q?1756181593590278635?= From: Liao Chang For the RVI and RVC hybrid encoding kernel, although AUIPC/JALR just occupy 8 bytes space, the patched code is 10 bytes at the worst case to ensure no RVI is truncated, so to check if kprobe satisfies the requirement of jump optimization, it has to find out an instruction window large enough to patch AUIPC/JALR(and padding C.NOP), and ensure no instruction nearby jumps into the patching window. Besides that, this series does not support the simulation of pc-relative instruction in optprobe handler yet, so the patching window should not includes pc-relative instruction. Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai Reviewed-by: Björn Töpel --- arch/riscv/kernel/probes/opt.c | 94 +++++++++++++++++++++++++++++++++- 1 file changed, 93 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c index d38ed1a52c93..d84aa1420fa2 100644 --- a/arch/riscv/kernel/probes/opt.c +++ b/arch/riscv/kernel/probes/opt.c @@ -269,6 +269,50 @@ static void find_free_registers(struct kprobe *kp, struct optimized_kprobe *op, *ra = (kw == 1UL) ? 0 : __builtin_ctzl(kw & ~1UL); } +static bool insn_jump_into_range(unsigned long addr, unsigned long start, + unsigned long end) +{ + kprobe_opcode_t insn = *(kprobe_opcode_t *)addr; + unsigned long target, offset = GET_INSN_LENGTH(insn); + +#ifdef CONFIG_RISCV_ISA_C + if (offset == RVC_INSN_LEN) { + if (riscv_insn_is_c_beqz(insn) || riscv_insn_is_c_bnez(insn)) + target = addr + rvc_branch_imme(insn); + else if (riscv_insn_is_c_jal(insn) || riscv_insn_is_c_j(insn)) + target = addr + rvc_jal_imme(insn); + else + target = 0; + return (target >= start) && (target < end); + } +#endif + + if (riscv_insn_is_branch(insn)) + target = addr + rvi_branch_imme(insn); + else if (riscv_insn_is_jal(insn)) + target = addr + rvi_jal_imme(insn); + else + target = 0; + return (target >= start) && (target < end); +} + +static int search_copied_insn(unsigned long paddr, struct optimized_kprobe *op) +{ + int i = 1; + struct arch_probe_insn api; + unsigned long offset = GET_INSN_LENGTH(*(kprobe_opcode_t *)paddr); + + while ((i++ < MAX_COPIED_INSN) && (offset < 2 * RVI_INSN_LEN)) { + if (riscv_probe_decode_insn((kprobe_opcode_t *)(paddr + offset), + &api) != INSN_GOOD) + return -1; + offset += GET_INSN_LENGTH(*(kprobe_opcode_t *)(paddr + offset)); + } + + op->optinsn.length = offset; + return 0; +} + /* * The kprobe based on breakpoint just requires the instrumented instruction * supports execute out-of-line or simulation, besides that, optimized kprobe @@ -276,7 +320,55 @@ static void find_free_registers(struct kprobe *kp, struct optimized_kprobe *op, */ static bool can_optimize(unsigned long paddr, struct optimized_kprobe *op) { - return false; + int ret; + struct arch_probe_insn api; + unsigned long addr, size = 0, offset = 0; + struct kprobe *kp = get_kprobe((kprobe_opcode_t *)paddr); + + /* + * Skip optimization if kprobe has been disarmed or instrumented + * instruction doest not support XOI. + */ + if (!kp || (riscv_probe_decode_insn(&kp->opcode, &api) != INSN_GOOD)) + return false; + + /* + * Find a instruction window large enough to contain a pair + * of AUIPC/JALR, and ensure each instruction in this window + * supports XOI. + */ + ret = search_copied_insn(paddr, op); + if (ret) + return false; + + if (!kallsyms_lookup_size_offset(paddr, &size, &offset)) + return false; + + /* Check there is enough space for relative jump(AUIPC/JALR) */ + if (size - offset <= op->optinsn.length) + return false; + + /* + * Decode instructions until function end, check any instruction + * don't jump into the window used to emit optprobe(AUIPC/JALR). + */ + addr = paddr - offset; + while (addr < paddr) { + if (insn_jump_into_range(addr, paddr + RVC_INSN_LEN, + paddr + op->optinsn.length)) + return false; + addr += GET_INSN_LENGTH(*(kprobe_opcode_t *)addr); + } + + addr = paddr + op->optinsn.length; + while (addr < paddr - offset + size) { + if (insn_jump_into_range(addr, paddr + RVC_INSN_LEN, + paddr + op->optinsn.length)) + return false; + addr += GET_INSN_LENGTH(*(kprobe_opcode_t *)addr); + } + + return true; } int arch_prepared_optinsn(struct arch_optimized_insn *optinsn) From patchwork Fri Jan 27 13:05:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49378 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825777wrn; Fri, 27 Jan 2023 05:13:19 -0800 (PST) X-Google-Smtp-Source: AK7set8Ex1M9SyYYlkqQJ5EGd4RyYyBHYbP9tPg8fZufj3X80CdDPq6P0awBaf84lHVW2q50GaM4 X-Received: by 2002:a17:906:8444:b0:879:ab3:2864 with SMTP id e4-20020a170906844400b008790ab32864mr2915240ejy.24.1674825199067; Fri, 27 Jan 2023 05:13:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825199; cv=none; d=google.com; s=arc-20160816; b=W2Fv81ysP7qtABl2cg2s/QzwXbWQUAtrR6DxMz32fnQmuxA80/9FHMCtU2hKiltfoY k6Fp8H8nU3MrVlRdvXAhFck9sdx4npxou5drRlSNugNcKRtsEEm7+sMiCKaQPRaL/k6w dJNnVMLHK3t9SAJLJOnxo4CfohcJgxTkozJU0twis84R4X6SpTK/uZdMerUL1eQxztcy xxV+3Ei9ARfi5bqDYRdvsWJkw02+QYMH8GtAVtJ1VrjRyeYpmf+wd6tOJ54d7nU0PKLT 05nl75Bflh7pAvYmbkYIlK1Pj+rwzn/f/rWTAkVPnxy2ckCP9vUfTpr7Vyp+GgelkAoP uuUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=fN3+lTf+y2QX6cXUzXoBHvVjvy12sW2lTA1dKhTisPg=; b=PPd9cZxxIBwaQSyj9BuaEG8f2gcdSwCm/R1+YScTZT/ikeRghohzaNMsaDA9P2IVlg j+2/ul4HHjwJFqx65U9jV0Wm8CGhYlJqpszh9SaNNRg2Lrs+T+MkrUtsksdqSh88RjoC g+eqXuw7mnuH5GM7kAYaRgUiwVmmLwkpcerVm8nDnRt2HSSv4KVVo80Jo13CJG2NRlQ1 oBrhiyRyLUPs8Q+6dt0/4oEKptc/ACpqowtm18vTL6OH22TEylv12ZbD2//8hdooWhyf NxfwE3EOw/ocPV8DdMHQ58FLH7dfLvP+b7mu2zR9ADUri8O8NlgH4XHp9gv9ogzLpdTd rY/Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id um20-20020a170907cb1400b00878ef86ce54si3282866ejc.449.2023.01.27.05.12.54; Fri, 27 Jan 2023 05:13:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234214AbjA0NGe (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234522AbjA0NGa (ORCPT ); Fri, 27 Jan 2023 08:06:30 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9245B7CCAD for ; Fri, 27 Jan 2023 05:06:26 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S11; Fri, 27 Jan 2023 21:06:10 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 07/13] riscv/kprobe: Prepare detour buffer for optimized kprobe Date: Fri, 27 Jan 2023 21:05:35 +0800 Message-Id: <20230127130541.1250865-8-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S11 X-Coremail-Antispam: 1UD129KBjvJXoWxKw13AFy3Xr1rGFy8Jr4Dtwb_yoWfZF43pa 9YkwnxXF4UZFn7Cry7t3Z5Jr4Sqws5Jws7Grs7GrW5A3Z2grW5Xw4Ig345Xr45G390gry3 ua4rJryUuF9xXrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr 1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j 6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFc xC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_ Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2 IErcIFxwACI402YVCY1x02628vn2kIc2xKxwCY02Avz4vE14v_Xr4l42xK82IYc2Ij64vI r41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8Gjc xK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0 cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY6xAIw2 0EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x02 67AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUjTmh7UUUUU== X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiCQkEE2PTt04WbgAAsm X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181507817614090?= X-GMAIL-MSGID: =?utf-8?q?1756181507817614090?= From: Liao Chang To avoid messing up the execution context calling optprobe handler, it needs to save and restore GPR/CSR context in the detour buffer. The payload of detour buffer for different optprobe have some differences, which derive from these reasons: - 'CALL optimized_callback', the relative offset for 'call' instruction is different for each detour buffer. - 'EXECUTE INSN OUT-OF-LINE'. - 'RETURN BACK', the chosen free register is reused here as the destination register of jumping back. So it also needs to customize the payload for each optimized kprobe. Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai --- arch/riscv/include/asm/kprobes.h | 16 +++ arch/riscv/kernel/probes/opt.c | 71 ++++++++++++ arch/riscv/kernel/probes/opt_trampoline.S | 125 ++++++++++++++++++++++ 3 files changed, 212 insertions(+) diff --git a/arch/riscv/include/asm/kprobes.h b/arch/riscv/include/asm/kprobes.h index 96cd36e67e2e..75ebd02be171 100644 --- a/arch/riscv/include/asm/kprobes.h +++ b/arch/riscv/include/asm/kprobes.h @@ -46,10 +46,26 @@ bool kprobe_single_step_handler(struct pt_regs *regs); /* optinsn template addresses */ extern __visible kprobe_opcode_t optprobe_template_entry[]; extern __visible kprobe_opcode_t optprobe_template_end[]; +extern __visible kprobe_opcode_t optprobe_template_save[]; +extern __visible kprobe_opcode_t optprobe_template_call[]; +extern __visible kprobe_opcode_t optprobe_template_insn[]; +extern __visible kprobe_opcode_t optprobe_template_return[]; #define MAX_OPTINSN_SIZE \ ((unsigned long)optprobe_template_end - \ (unsigned long)optprobe_template_entry) +#define DETOUR_SAVE_OFFSET \ + ((unsigned long)optprobe_template_save - \ + (unsigned long)optprobe_template_entry) +#define DETOUR_CALL_OFFSET \ + ((unsigned long)optprobe_template_call - \ + (unsigned long)optprobe_template_entry) +#define DETOUR_INSN_OFFSET \ + ((unsigned long)optprobe_template_insn - \ + (unsigned long)optprobe_template_entry) +#define DETOUR_RETURN_OFFSET \ + ((unsigned long)optprobe_template_return - \ + (unsigned long)optprobe_template_entry) /* * For RVI and RVC hybrid encoding kernel, although long jump just needs diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c index d84aa1420fa2..a47f7d2bf3a6 100644 --- a/arch/riscv/kernel/probes/opt.c +++ b/arch/riscv/kernel/probes/opt.c @@ -11,9 +11,32 @@ #include #include #include +#include #include "simulate-insn.h" #include "decode-insn.h" +#include "../../net/bpf_jit.h" + +static void optimized_callback(struct optimized_kprobe *op, + struct pt_regs *regs) +{ + if (kprobe_disabled(&op->kp)) + return; + + preempt_disable(); + if (kprobe_running()) { + kprobes_inc_nmissed_count(&op->kp); + } else { + __this_cpu_write(current_kprobe, &op->kp); + /* Save skipped registers */ + instruction_pointer_set(regs, (unsigned long)op->kp.addr); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + opt_pre_handler(&op->kp, regs); + __this_cpu_write(current_kprobe, NULL); + } + preempt_enable(); +} +NOKPROBE_SYMBOL(optimized_callback) static int in_auipc_jalr_range(long val) { @@ -30,6 +53,11 @@ static int in_auipc_jalr_range(long val) #endif } +#define DETOUR_ADDR(code, offs) \ + ((void *)((unsigned long)(code) + (offs))) +#define DETOUR_INSN(code, offs) \ + (*(kprobe_opcode_t *)((unsigned long)(code) + (offs))) + /* * Copy optprobe assembly code template into detour buffer and modify some * instructions for each kprobe. @@ -38,6 +66,49 @@ static void prepare_detour_buffer(kprobe_opcode_t *code, kprobe_opcode_t *slot, int rd, struct optimized_kprobe *op, kprobe_opcode_t opcode) { + long offs; + unsigned long data; + + memcpy(code, optprobe_template_entry, MAX_OPTINSN_SIZE); + + /* Step1: record optimized_kprobe pointer into detour buffer */ + memcpy(DETOUR_ADDR(code, DETOUR_SAVE_OFFSET), &op, sizeof(op)); + + /* + * Step2 + * auipc ra, 0 --> aupic ra, HI20.{optimized_callback - pc} + * jalr ra, 0(ra) --> jalr ra, LO12.{optimized_callback - pc}(ra) + */ + offs = (unsigned long)&optimized_callback - + (unsigned long)DETOUR_ADDR(slot, DETOUR_CALL_OFFSET); + DETOUR_INSN(code, DETOUR_CALL_OFFSET) = + rv_auipc(1, (offs + (1 << 11)) >> 12); + DETOUR_INSN(code, DETOUR_CALL_OFFSET + 0x4) = + rv_jalr(1, 1, offs & 0xFFF); + + /* Step3: copy replaced instructions into detour buffer */ + memcpy(DETOUR_ADDR(code, DETOUR_INSN_OFFSET), op->kp.addr, + op->optinsn.length); + memcpy(DETOUR_ADDR(code, DETOUR_INSN_OFFSET), &opcode, + GET_INSN_LENGTH(opcode)); + + /* Step4: record return address of long jump into detour buffer */ + data = (unsigned long)op->kp.addr + op->optinsn.length; + memcpy(DETOUR_ADDR(code, DETOUR_RETURN_OFFSET), &data, sizeof(data)); + + /* + * Step5 + * auipc ra, 0 --> auipc rd, 0 + * ld/w ra, -4(ra) --> ld/w rd, -8(rd) + * jalr x0, 0(ra) --> jalr x0, 0(rd) + */ + DETOUR_INSN(code, DETOUR_RETURN_OFFSET + 0x8) = rv_auipc(rd, 0); +#if __riscv_xlen == 32 + DETOUR_INSN(code, DETOUR_RETURN_OFFSET + 0xC) = rv_lw(rd, -8, rd); +#else + DETOUR_INSN(code, DETOUR_RETURN_OFFSET + 0xC) = rv_ld(rd, -8, rd); +#endif + DETOUR_INSN(code, DETOUR_RETURN_OFFSET + 0x10) = rv_jalr(0, rd, 0); } /* Registers the first usage of which is the destination of instruction */ diff --git a/arch/riscv/kernel/probes/opt_trampoline.S b/arch/riscv/kernel/probes/opt_trampoline.S index 16160c4367ff..5187e71d8e61 100644 --- a/arch/riscv/kernel/probes/opt_trampoline.S +++ b/arch/riscv/kernel/probes/opt_trampoline.S @@ -1,12 +1,137 @@ /* SPDX-License-Identifier: GPL-2.0-only */ /* * Copyright (C) 2022 Guokai Chen + * Copyright (C) 2022 Liao, Chang */ #include +#include #incldue #include SYM_ENTRY(optprobe_template_entry, SYM_L_GLOBAL, SYM_A_NONE) + addi sp, sp, -(PT_SIZE_ON_STACK) + REG_S x1, PT_RA(sp) + REG_S x2, PT_SP(sp) + REG_S x3, PT_GP(sp) + REG_S x4, PT_TP(sp) + REG_S x5, PT_T0(sp) + REG_S x6, PT_T1(sp) + REG_S x7, PT_T2(sp) + REG_S x8, PT_S0(sp) + REG_S x9, PT_S1(sp) + REG_S x10, PT_A0(sp) + REG_S x11, PT_A1(sp) + REG_S x12, PT_A2(sp) + REG_S x13, PT_A3(sp) + REG_S x14, PT_A4(sp) + REG_S x15, PT_A5(sp) + REG_S x16, PT_A6(sp) + REG_S x17, PT_A7(sp) + REG_S x18, PT_S2(sp) + REG_S x19, PT_S3(sp) + REG_S x20, PT_S4(sp) + REG_S x21, PT_S5(sp) + REG_S x22, PT_S6(sp) + REG_S x23, PT_S7(sp) + REG_S x24, PT_S8(sp) + REG_S x25, PT_S9(sp) + REG_S x26, PT_S10(sp) + REG_S x27, PT_S11(sp) + REG_S x28, PT_T3(sp) + REG_S x29, PT_T4(sp) + REG_S x30, PT_T5(sp) + REG_S x31, PT_T6(sp) + /* Update fp is friendly for stacktrace */ + addi s0, sp, (PT_SIZE_ON_STACK) + j 1f + +SYM_ENTRY(optprobe_template_save, SYM_L_GLOBAL, SYM_A_NONE) + /* + * Step1: + * Filled with the pointer to optimized_kprobe data + */ + .dword 0 +1: + /* Load optimize_kprobe pointer from .dword below */ + auipc a0, 0 + REG_L a0, -8(a0) + add a1, sp, x0 + +SYM_ENTRY(optprobe_template_call, SYM_L_GLOBAL, SYM_A_NONE) + /* + * Step2: + * of AUIPC/JALR are modified to the offset to optimized_callback + * jump target is loaded from above .dword. + */ + auipc ra, 0 + jalr ra, 0(ra) + + REG_L x1, PT_RA(sp) + REG_L x3, PT_GP(sp) + REG_L x4, PT_TP(sp) + REG_L x5, PT_T0(sp) + REG_L x6, PT_T1(sp) + REG_L x7, PT_T2(sp) + REG_L x8, PT_S0(sp) + REG_L x9, PT_S1(sp) + REG_L x10, PT_A0(sp) + REG_L x11, PT_A1(sp) + REG_L x12, PT_A2(sp) + REG_L x13, PT_A3(sp) + REG_L x14, PT_A4(sp) + REG_L x15, PT_A5(sp) + REG_L x16, PT_A6(sp) + REG_L x17, PT_A7(sp) + REG_L x18, PT_S2(sp) + REG_L x19, PT_S3(sp) + REG_L x20, PT_S4(sp) + REG_L x21, PT_S5(sp) + REG_L x22, PT_S6(sp) + REG_L x23, PT_S7(sp) + REG_L x24, PT_S8(sp) + REG_L x25, PT_S9(sp) + REG_L x26, PT_S10(sp) + REG_L x27, PT_S11(sp) + REG_L x28, PT_T3(sp) + REG_L x29, PT_T4(sp) + REG_L x30, PT_T5(sp) + REG_L x31, PT_T6(sp) + REG_L x2, PT_SP(sp) + addi sp, sp, (PT_SIZE_ON_STACK) + +SYM_ENTRY(optprobe_template_insn, SYM_L_GLOBAL, SYM_A_NONE) + /* + * Step3: + * NOPS will be replaced by the probed instruction, at worst case 3 RVC + * and 1 RVI instructions is about to execute out of line. + */ +#ifdef CONFIG_RISCV_ISA_C + c.addi zero, 0 + c.addi zero, 0 + c.addi zero, 0 + c.addi zero, 0 + c.addi zero, 0 +#else + addi zero, zero, 0 + addi zero, zero, 0 +#endif + j 2f + +SYM_ENTRY(optprobe_template_return, SYM_L_GLOBAL, SYM_A_NONE) + /* + * Step4: + * Filled with the return address of long jump(AUIPC/JALR) + */ + .dword 0 +2: + /* + * Step5: + * The of AUIPC/LD/JALR will be replaced for each kprobe, + * used to read return address saved in .dword above. + */ + auipc ra, 0 + REG_L ra, -8(ra) + jalr x0, 0(ra) SYM_ENTRY(optprobe_template_end, SYM_L_GLOBAL, SYM_A_NONE) From patchwork Fri Jan 27 13:05:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49382 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825897wrn; Fri, 27 Jan 2023 05:13:36 -0800 (PST) X-Google-Smtp-Source: AK7set/ZeSh6Dn/cJAkt7/w86JZIoVv/Iqq2yXRbi0bplYWsSESufBaHBGtV9VnwJ1wt17s4ygCH X-Received: by 2002:a17:906:6716:b0:878:652d:eb04 with SMTP id a22-20020a170906671600b00878652deb04mr6136152ejp.76.1674825216365; Fri, 27 Jan 2023 05:13:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825216; cv=none; d=google.com; s=arc-20160816; b=kamKhioluAYopqyOrwLraYluUI2AADJln0dhrWWziJ7nWamICitiP7Jcyw563ygtWG 7czb4dV610kSzjdqaHk9il4ommWw9GVWkGupLvipAYFPq2Fcld8c1VSLuJX3qIBscTXl 5HiKrJFgnXlFVP8wbqTsCcqAZZLY4ULFHsj38Yh6ieuf/0pEX6UTtGzqhmeldB57IoqJ kjZF7SuxxiwFhADB7Cl1UMrIf5RrJrUwknd7sOPPyz34CknWtwNsGyEx6WYNCTlzGY1M xpdZa6dObiWVS61tMLglDUdJ8Nfr3NROVwu1WikZ7nfIdIKYY6N4gxKg0nF6oZu8hurR PT6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=DKNfnChcvn1xl3YmyNcAfFf1rHf5VRznH6KivgvBiXg=; b=l76jWnLLEqSURfmEtEG0TYyTM5uTCEdmj/1Trsl3JPJoIr95zy5vVZQ83ow2MUm0Q+ pru0PU0pTzL8BtJS0RbFCbBGOLttfX7XDItZr4wfLqq4RpGB4GNc9NKYZyjaBIeznfAf MH8v/l55K0PjWeiNTswVsWp0/H2ih6nwdkf3Sq+puPBUeux/jOYrXdU549J+Meya1rYY A+DuOz5RxjZFQs1uEgc3pVShcNwwDck0L0gkCuLHH1HDedZ+ITlOBWq3fS+7k4AhUCrU yzTnI1igurAa9I3CNMwZVm/EJlJymIWq3iFr734LFxWHQhYmO8GRGagKhcebPfkW8epp eflA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id eq20-20020a170907291400b0087bd412d77csi1704268ejc.768.2023.01.27.05.13.12; Fri, 27 Jan 2023 05:13:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234569AbjA0NGr (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234541AbjA0NGc (ORCPT ); Fri, 27 Jan 2023 08:06:32 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C789C7D986 for ; Fri, 27 Jan 2023 05:06:29 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S12; Fri, 27 Jan 2023 21:06:10 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai Subject: [PATCH v6 08/13] riscv/kprobe: Patch AUIPC/JALR pair to optimize kprobe Date: Fri, 27 Jan 2023 21:05:36 +0800 Message-Id: <20230127130541.1250865-9-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S12 X-Coremail-Antispam: 1UD129KBjvJXoW3Ww1fZF4kXFyfGFWfur1fCrg_yoW7ZFWxpF s8C3ZxJrWYkFn2grZxAws5ur1rKwsYvay3K34DGFWfCrsrJrs8Wwn7Kwn8Zr15GF1Fgr13 ArnYkryruay7JrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr 1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j 6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFc xC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_ Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2 IErcIFxwACI402YVCY1x02628vn2kIc2xKxwCY02Avz4vE14v_Xr4l42xK82IYc2Ij64vI r41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8Gjc xK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0 cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY6xAIw2 0EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x02 67AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUjTmh7UUUUU== X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiBwUEE2PTsCcetQAAsZ X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181526035017485?= X-GMAIL-MSGID: =?utf-8?q?1756181526035017485?= From: Liao Chang There is race when replacing EBREAK with AUIPC/JALR pairs under SMP, so it needs to patch multiple instructions safely, this patch enhances patch_text_cb() to ensure no race occurs when patching AUIPC/JALR pairs. Signed-off-by: Liao Chang Co-developed-by: Chen Guokai Signed-off-by: Chen Guokai Reviewed-by: Björn Töpel --- arch/riscv/include/asm/patch.h | 1 + arch/riscv/kernel/patch.c | 23 +++++++++--- arch/riscv/kernel/probes/opt.c | 65 ++++++++++++++++++++++++++++++++-- 3 files changed, 83 insertions(+), 6 deletions(-) diff --git a/arch/riscv/include/asm/patch.h b/arch/riscv/include/asm/patch.h index 9a7d7346001e..ee31539de65f 100644 --- a/arch/riscv/include/asm/patch.h +++ b/arch/riscv/include/asm/patch.h @@ -8,5 +8,6 @@ int patch_text_nosync(void *addr, const void *insns, size_t len); int patch_text(void *addr, u32 insn); +int patch_text_batch(void *addr, const void *insn, size_t size); #endif /* _ASM_RISCV_PATCH_H */ diff --git a/arch/riscv/kernel/patch.c b/arch/riscv/kernel/patch.c index 765004b60513..ce324b6a6998 100644 --- a/arch/riscv/kernel/patch.c +++ b/arch/riscv/kernel/patch.c @@ -15,7 +15,8 @@ struct patch_insn { void *addr; - u32 insn; + const void *insn; + size_t size; atomic_t cpu_count; }; @@ -106,8 +107,7 @@ static int patch_text_cb(void *data) if (atomic_inc_return(&patch->cpu_count) == num_online_cpus()) { ret = - patch_text_nosync(patch->addr, &patch->insn, - GET_INSN_LENGTH(patch->insn)); + patch_text_nosync(patch->addr, patch->insn, patch->size); atomic_inc(&patch->cpu_count); } else { while (atomic_read(&patch->cpu_count) <= num_online_cpus()) @@ -123,7 +123,8 @@ int patch_text(void *addr, u32 insn) { struct patch_insn patch = { .addr = addr, - .insn = insn, + .insn = &insn, + .size = GET_INSN_LENGTH(insn), .cpu_count = ATOMIC_INIT(0), }; @@ -131,3 +132,17 @@ int patch_text(void *addr, u32 insn) &patch, cpu_online_mask); } NOKPROBE_SYMBOL(patch_text); + +int patch_text_batch(void *addr, const void *insn, size_t size) +{ + struct patch_insn patch = { + .addr = addr, + .insn = insn, + .size = size, + .cpu_count = ATOMIC_INIT(0), + }; + + return stop_machine_cpuslocked(patch_text_cb, &patch, cpu_online_mask); +} + +NOKPROBE_SYMBOL(patch_text_batch); diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c index a47f7d2bf3a6..c52d5bdc748c 100644 --- a/arch/riscv/kernel/probes/opt.c +++ b/arch/riscv/kernel/probes/opt.c @@ -8,6 +8,7 @@ #define pr_fmt(fmt) "optprobe: " fmt +#include #include #include #include @@ -444,11 +445,19 @@ static bool can_optimize(unsigned long paddr, struct optimized_kprobe *op) int arch_prepared_optinsn(struct arch_optimized_insn *optinsn) { - return 0; + return optinsn->length; } int arch_check_optimized_kprobe(struct optimized_kprobe *op) { + unsigned long i; + struct kprobe *p; + + for (i = RVC_INSN_LEN; i < op->optinsn.length; i += RVC_INSN_LEN) { + p = get_kprobe(op->kp.addr + i); + if (p && !kprobe_disabled(p)) + return -EEXIST; + } return 0; } @@ -509,23 +518,75 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, void arch_remove_optimized_kprobe(struct optimized_kprobe *op) { + if (op->optinsn.insn) { + free_optinsn_slot(op->optinsn.insn, 1); + op->optinsn.insn = NULL; + op->optinsn.length = 0; + } } void arch_optimize_kprobes(struct list_head *oplist) { + long offs; + kprobe_opcode_t insn[3]; + struct optimized_kprobe *op, *tmp; + + list_for_each_entry_safe(op, tmp, oplist, list) { + WARN_ON(kprobe_disabled(&op->kp)); + + /* Backup instructions which will be replaced by jump address */ + memcpy(op->optinsn.copied_insn, + DETOUR_ADDR(op->optinsn.insn, DETOUR_INSN_OFFSET), + op->optinsn.length); + + /* + * After patching, it should be: + * auipc free_register, %hi(detour_buffer) + * jalr free_register, free_register, %lo(detour_buffer) + * where free_register will eventually save the return address + */ + offs = (unsigned long)op->optinsn.insn - + (unsigned long)op->kp.addr; + insn[0] = rv_auipc(op->optinsn.rd, (offs + (1 << 11)) >> 12); + insn[1] = rv_jalr(op->optinsn.rd, op->optinsn.rd, offs & 0xFFF); + /* For 3 RVC + 1 RVI scenario, fill C.NOP for padding */ + if (op->optinsn.length > 2 * RVI_INSN_LEN) + insn[2] = rvc_addi(0, 0); + + patch_text_batch(op->kp.addr, insn, op->optinsn.length); + if (memcmp(op->kp.addr, insn, op->optinsn.length)) + continue; + + list_del_init(&op->list); + } } void arch_unoptimize_kprobes(struct list_head *oplist, struct list_head *done_list) { + struct optimized_kprobe *op, *tmp; + + list_for_each_entry_safe(op, tmp, oplist, list) { + arch_unoptimize_kprobe(op); + list_move(&op->list, done_list); + } } void arch_unoptimize_kprobe(struct optimized_kprobe *op) { + kprobe_opcode_t buf[MAX_COPIED_INSN]; + + memcpy(buf, op->optinsn.copied_insn, op->optinsn.length); + if (GET_INSN_LENGTH(op->kp.opcode) == RVI_INSN_LEN) + *(u32 *)buf = __BUG_INSN_32; + else + *(u16 *)buf = __BUG_INSN_16; + patch_text_batch(op->kp.addr, buf, op->optinsn.length); } int arch_within_optimized_kprobe(struct optimized_kprobe *op, kprobe_opcode_t *addr) { - return 0; + return (op->kp.addr <= addr && + op->kp.addr + op->optinsn.length > addr); } From patchwork Fri Jan 27 13:05:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49387 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp826640wrn; Fri, 27 Jan 2023 05:15:07 -0800 (PST) X-Google-Smtp-Source: AK7set/V8UfPLo8IrsXycD+1vDvITQCUfPGIS2+DefKUOHc1FC2vN60zhzAx/6FUPMn42ftlKIy4 X-Received: by 2002:a17:906:1443:b0:878:80f1:ada1 with SMTP id q3-20020a170906144300b0087880f1ada1mr3837594ejc.30.1674825306761; Fri, 27 Jan 2023 05:15:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825306; cv=none; d=google.com; s=arc-20160816; b=Jrl/vUj7t3IjvQTNPgsnF1EikSblSkKXPBW8Y4DeFZIy0gXjBBw/qQrgLpyCcX/1KV +WbV40E+WbjKD5DkLHnRo9IYtOBZfsVThyUvzOIib2nrJvEVwYDaGGGefgUx8EfBSVwe PVrkR5gRXbELj1wFnBA0BW9Fv9ATetQxNEhl8NQaX2N1oR1uL0EO6QELnEbdtON4Qw3T cQpeHLcDeDEYc9AcY1IFpxgIwNc6wArTkgx+RrsjGwsc0++djEVPusdCU/7rTVj0Y+M2 tTyw25Z/Nnf4CaxHma6DGnF0IoWw74WS2HBsIjI8I4x27WT48rIJvQQqj9Qmqkq0VnFx GPIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=tJDP99IKckiGBbQzRNRBGab86CwPPRenAtk3MXmT2J0=; b=t/VsMNTIVsp/pGStBk5Pv0oVaQzNhWEn8jMOYH15hU2X6jI/C91Y99tRva8wl2KhQ4 uuUk5k4bd1DatgvNQu7zxUcNZsX/v3a13sao5/Z3N7so4OZ4DE62XkaPhjStSkiyB07+ V4T526FWgnSgGQ3gvG1jRIAQyMfhmhUY/TjYIX4q9BrXosHfM4K/oZiPhHYjB/RWfYdx 6w/Y5ETHOMPuI/1+QKZgq301qP9d1DxIBqu3/BRjiLbVsVTTIfk8b9P7V/fIl0BIospN Y4IWCqj4OJb/ZnBRTUJlQugaFjF2YIf8oI6V2aPrNIk8xUa0j8342v5KkKP3GxTuOZ1f wyxA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id uo36-20020a170907cc2400b00879d3ea9603si2588301ejc.841.2023.01.27.05.14.42; Fri, 27 Jan 2023 05:15:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233351AbjA0NHA (ORCPT + 99 others); Fri, 27 Jan 2023 08:07:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234554AbjA0NGj (ORCPT ); Fri, 27 Jan 2023 08:06:39 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C45567CCB1 for ; Fri, 27 Jan 2023 05:06:32 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S13; Fri, 27 Jan 2023 21:06:10 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com, Chen Guokai , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= Subject: [PATCH v6 09/13] riscv/kprobe: Search free registers from unused caller-saved ones Date: Fri, 27 Jan 2023 21:05:37 +0800 Message-Id: <20230127130541.1250865-10-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S13 X-Coremail-Antispam: 1UD129KBjvJXoWxtF45Ary3JryUtrWkuryDZFb_yoWfXF15pF sxGw4FvrWUtFykCrZrAw1kGr4fKrs8KrW0vFyUtFyayr43X39IvFWvgF1aqr1DKF4jvr4x Ar45Cry8urW3AFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUQq14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26r4UJVWxJr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4U JVWxJr1lnxkEFVAIw20F6cxK64vIFxWle2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64 xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j 6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwACjI8F5VA0II8E6IAqYI 8I648v4I1lFIxGxcIEc7CjxVA2Y2ka0xkIwI1lc2xSY4AK67AK6ry8MxAIw28IcxkI7VAK I48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7 xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x0EwIxGrwCI42IY6xII jxv20xvE14v26r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UMIIF0xvE42 xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF 7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUjTmh7UUUUU== X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiBwUEE2PTsCcetQABsY X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181621001916071?= X-GMAIL-MSGID: =?utf-8?q?1756181621001916071?= This patch further allows optprobe to use caller-saved registers that is not used across the function being optimized as free registers. Signed-off-by: Chen Guokai Co-developed-by: Liao Chang Signed-off-by: Liao Chang Reported-by: Björn Töpel Reported-by: should be used for fixes. Please change to Suggested-by:, --- arch/riscv/include/asm/kprobes.h | 1 + arch/riscv/kernel/probes/decode-insn.h | 29 +++++++ arch/riscv/kernel/probes/opt.c | 116 ++++++++++++++++++++++--- 3 files changed, 134 insertions(+), 12 deletions(-) diff --git a/arch/riscv/include/asm/kprobes.h b/arch/riscv/include/asm/kprobes.h index 75ebd02be171..f7d33f6861c6 100644 --- a/arch/riscv/include/asm/kprobes.h +++ b/arch/riscv/include/asm/kprobes.h @@ -86,6 +86,7 @@ struct arch_optimized_insn { kprobe_opcode_t *insn; unsigned long length; int rd; + u32 free_reg; }; #endif /* CONFIG_OPTPROBES */ diff --git a/arch/riscv/kernel/probes/decode-insn.h b/arch/riscv/kernel/probes/decode-insn.h index 785b023a62ea..140f5b6a9886 100644 --- a/arch/riscv/kernel/probes/decode-insn.h +++ b/arch/riscv/kernel/probes/decode-insn.h @@ -13,6 +13,35 @@ enum probe_insn { INSN_GOOD, }; +#define NRREG 32 +#define ALL_REG_OCCUPIED 0xffffffffu +/* + * Register ABI Name Saver + * x0 zero -- + * x1 ra Caller + * x2 sp Callee + * x3 gp -- + * x4 tp -- + * x5-7 t0-2 Caller + * x8 so/fp Callee + * x9 so/fp Callee + * x10-11 a0-1 Caller + * x12-17 a2-7 Caller + * x18-27 s2-11 Callee + * x28-32 t3-6 Caller + * + * If register is not caller-saved, it is potentially unsafe to used + * as a free register to form AUIPC/JALR, then use one bitmask to filter + * out these registers. Because ra is used to record return address for + * function call, so mark ra as non-caller-saved register here. + * */ +#define NON_CALLER_SAVED_MASK \ + (1 << 0) | (1 << 1) | (1 << 2) | (1 << 3) | \ + (1 << 4) | (1 << 8) | (1 << 9) | (1 << 18) | \ + (1 << 19) | (1 << 20) | (1 << 21) | (1 << 22) | \ + (1 << 23) | (1 << 24) | (1 << 25) | (1 << 26) | \ + (1 << 27) + enum probe_insn __kprobes riscv_probe_decode_insn(probe_opcode_t *addr, struct arch_probe_insn *asi); diff --git a/arch/riscv/kernel/probes/opt.c b/arch/riscv/kernel/probes/opt.c index c52d5bdc748c..e151b1c60d6d 100644 --- a/arch/riscv/kernel/probes/opt.c +++ b/arch/riscv/kernel/probes/opt.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "simulate-insn.h" #include "decode-insn.h" @@ -126,7 +127,7 @@ static void prepare_detour_buffer(kprobe_opcode_t *code, kprobe_opcode_t *slot, * as a destination register before any branch or jump instruction. */ static void find_register(unsigned long start, unsigned long end, - unsigned long *write, unsigned long *read) + unsigned long *write, unsigned long *read) { kprobe_opcode_t insn; unsigned long addr, offset = 0UL; @@ -385,18 +386,101 @@ static int search_copied_insn(unsigned long paddr, struct optimized_kprobe *op) return 0; } +static void update_free_reg(unsigned long addr, uint32_t *used_reg) +{ + kprobe_opcode_t insn = *(kprobe_opcode_t *)addr; + unsigned long offset = GET_INSN_LENGTH(insn); + +#ifdef CONFIG_RISCV_ISA_C + if (offset == RVI_INSN_LEN) + goto is_rvi; + + insn &= __COMPRESSED_INSN_MASK; + if (riscv_insn_is_c_jal(insn)) { + *used_reg |= 1 << 1; + } else if (riscv_insn_is_c_jr(insn)) { + *used_reg |= 1 << rvc_r_rs1(insn); + } else if (riscv_insn_is_c_jalr(insn)) { + *used_reg |= 1 << rvc_r_rs1(insn); + } else if (riscv_insn_is_c_beqz(insn) || riscv_insn_is_c_bnez(insn)) { + *used_reg |= 1 << rvc_b_rs(insn); + } else if (riscv_insn_is_c_sub(insn) || riscv_insn_is_c_subw(insn)) { + *used_reg |= 1 << rvc_a_rs1(insn); + *used_reg |= 1 << rvc_a_rs2(insn); + } else if (riscv_insn_is_c_sq(insn) || riscv_insn_is_c_sw(insn) || + riscv_insn_is_c_sd(insn)) { + *used_reg |= 1 << rvc_s_rs1(insn); + *used_reg |= 1 << rvc_s_rs2(insn); + } else if (riscv_insn_is_c_addi16sp(insn) || riscv_insn_is_c_addi(insn) || + riscv_insn_is_c_addiw(insn) || + riscv_insn_is_c_slli(insn)) { + *used_reg |= 1 << rvc_i_rs1(insn); + } else if (riscv_insn_is_c_sri(insn) || + riscv_insn_is_c_andi(insn)) { + *used_reg |= 1 << rvc_b_rs(insn); + } else if (riscv_insn_is_c_sqsp(insn) || riscv_insn_is_c_swsp(insn) || + riscv_insn_is_c_sdsp(insn)) { + *used_reg |= 1 << rvc_ss_rs2(insn); + *used_reg |= 1 << 2; + } else if (riscv_insn_is_c_mv(insn)) { + *used_reg |= 1 << rvc_r_rs2(insn); + } else if (riscv_insn_is_c_addi4spn(insn)) { + *used_reg |= 1 << 2; + } else if (riscv_insn_is_c_lq(insn) || riscv_insn_is_c_lw(insn) || + riscv_insn_is_c_ld(insn)) { + *used_reg |= 1 << rvc_l_rs(insn); + } else if (riscv_insn_is_c_lqsp(insn) || riscv_insn_is_c_lwsp(insn) || + riscv_insn_is_c_ldsp(insn)) { + *used_reg |= 1 << 2; + } + /* li and lui does not have source reg */ + return; +is_rvi: +#endif + if (riscv_insn_is_arith_ri(insn) || riscv_insn_is_load(insn)) { + *used_reg |= 1 << rvi_rs1(insn); + } else if (riscv_insn_is_arith_rr(insn) || riscv_insn_is_store(insn) || + riscv_insn_is_amo(insn)) { + *used_reg |= 1 << rvi_rs1(insn); + *used_reg |= 1 << rvi_rs2(insn); + } else if (riscv_insn_is_branch(insn)) { + *used_reg |= 1 << rvi_rs1(insn); + *used_reg |= 1 << rvi_rs2(insn); + } else if (riscv_insn_is_jalr(insn)) { + *used_reg |= 1 << rvi_rs1(insn); + } +} + +static bool scan_code(unsigned long *addr, unsigned long paddr, + struct optimized_kprobe *op, uint32_t *used_reg) +{ + if (insn_jump_into_range(*addr, paddr + RVC_INSN_LEN, + paddr + op->optinsn.length)) + return false; + if (search_exception_tables(*addr)) + return false; + update_free_reg(*addr, used_reg); + *addr += GET_INSN_LENGTH(*(kprobe_opcode_t *)addr); + return true; +} + /* * The kprobe based on breakpoint just requires the instrumented instruction * supports execute out-of-line or simulation, besides that, optimized kprobe * requires no near instruction jump to any instruction replaced by AUIPC/JALR. */ -static bool can_optimize(unsigned long paddr, struct optimized_kprobe *op) +static bool can_optimize(unsigned long paddr, struct optimized_kprobe *op, uint32_t *used_reg) { int ret; struct arch_probe_insn api; unsigned long addr, size = 0, offset = 0; struct kprobe *kp = get_kprobe((kprobe_opcode_t *)paddr); + /* + * All callee + */ + *used_reg = NON_CALLER_SAVED_MASK; + /* * Skip optimization if kprobe has been disarmed or instrumented * instruction doest not support XOI. @@ -426,18 +510,14 @@ static bool can_optimize(unsigned long paddr, struct optimized_kprobe *op) */ addr = paddr - offset; while (addr < paddr) { - if (insn_jump_into_range(addr, paddr + RVC_INSN_LEN, - paddr + op->optinsn.length)) + if (!scan_code(&addr, paddr, op, used_reg)) return false; - addr += GET_INSN_LENGTH(*(kprobe_opcode_t *)addr); } - - addr = paddr + op->optinsn.length; + update_free_reg((unsigned long)&kp->opcode, used_reg); + addr = paddr + GET_INSN_LENGTH(*(kprobe_opcode_t *)&kp->opcode); while (addr < paddr - offset + size) { - if (insn_jump_into_range(addr, paddr + RVC_INSN_LEN, - paddr + op->optinsn.length)) + if (!scan_code(&addr, paddr, op, used_reg)) return false; - addr += GET_INSN_LENGTH(*(kprobe_opcode_t *)addr); } return true; @@ -466,10 +546,13 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, { long rel; int rd = 0, ra = 0, ret; + u32 used_reg; kprobe_opcode_t *code = NULL, *slot = NULL; - if (!can_optimize((unsigned long)orig->addr, op)) + if (!can_optimize((unsigned long)orig->addr, op, &used_reg)) { + op->optinsn.rd = -1; return -EILSEQ; + } code = kzalloc(MAX_OPTINSN_SIZE, GFP_KERNEL); slot = get_optinsn_slot(); @@ -490,7 +573,14 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, * to detour buffer, ra is used to form JR jumping back from detour * buffer. */ - find_free_registers(orig, op, &rd, &ra); + if (used_reg == ALL_REG_OCCUPIED) { + find_free_registers(orig, op, &rd, &ra); + } else { + /* Choose one unused caller-saved register. */ + rd = ffz(used_reg); + ra = rd; + } + if (rd == 0 || ra == 0) { ret = -EILSEQ; goto on_error; @@ -534,6 +624,8 @@ void arch_optimize_kprobes(struct list_head *oplist) list_for_each_entry_safe(op, tmp, oplist, list) { WARN_ON(kprobe_disabled(&op->kp)); + if (op->optinsn.rd < 0) + continue; /* Backup instructions which will be replaced by jump address */ memcpy(op->optinsn.copied_insn, DETOUR_ADDR(op->optinsn.insn, DETOUR_INSN_OFFSET), From patchwork Fri Jan 27 13:05:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49381 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825893wrn; Fri, 27 Jan 2023 05:13:36 -0800 (PST) X-Google-Smtp-Source: AMrXdXtmxsGM/P0GP6uIE3II4CnKrKqQOxu51IZDdmzld80sjsAHevYpG7CJZ+iRMYluNKoJGU2g X-Received: by 2002:a17:907:8b98:b0:84d:44dd:e03a with SMTP id tb24-20020a1709078b9800b0084d44dde03amr47035116ejc.57.1674825216108; Fri, 27 Jan 2023 05:13:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825216; cv=none; d=google.com; s=arc-20160816; b=zWFi4tpFNBCjm3uTmC6OTR0KP5Oz12r3/KHG+RIaRfIB3NTXQotPZC2iX88hJP38Ma 6D+kDC0nX77yG7IfGgcj4IIk7s3z+OyJxPASaPjo7i/3bULGnQr93du7XvQoqvzkEPh7 mUlH35tzRHdeuS/5nwfVitGgSNU5XC4DP0D+S/Oi25vC/B9eE78xITKvLpb/se0MJ5Ms lQ8eih8q8HG96Ea3SgmknxL7tFZfyHlZSW7fUFzRa3AP/nWcJ0Qqzj3PbdPjgk5vkfNr 9wUistUDusB31fF8162qFGBKLz/ez321JLf9LBheKyW/o1CFvXa80aMZL91/V10rAwq+ ewGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Ov1UwlLQS46MUcGKOfivdrMjN87yhnKQ9gjrNoMLUbM=; b=YYnx+gHi0zPReK0UaLQ1MqOPKYOiaioUV3fV2VfkIypQ5LX+/4YYY2fCyM6KRAdwIu laZfkBPOLC7+2/D+ruNP7S83ilTLi7qql/QqcPpjv5M3NPcZSH9TI4yxoJgMWy9xnlIY 5q2kHw24Z0V3s2FwMkpPqK0o1NheCOSBIgJ/Qn0rLRqTtSgHbm5vtwROD9sFehXTkL9d sZ5GPYPopdq2WiUlVEmMxEgrmCdM4fxD15Fc2IzPhlxdjnlK17V0XFeD0wgAZSDgvbZI JOhyVJwprxclc+pm0xPudOtRRjh3m/kela907F1SAy30rmqt30sMm9ayTGGKK/JESc2S siAg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id zu24-20020a17090708d800b00829cbd5f6f2si4259379ejb.498.2023.01.27.05.13.12; Fri, 27 Jan 2023 05:13:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234028AbjA0NGo (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234520AbjA0NGb (ORCPT ); Fri, 27 Jan 2023 08:06:31 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B3CFF7D6CA for ; Fri, 27 Jan 2023 05:06:28 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S14; Fri, 27 Jan 2023 21:06:10 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com Subject: [PATCH v6 10/13] riscv/kprobe: Add instruction boundary check for RVI/RVC hybrid kernel Date: Fri, 27 Jan 2023 21:05:38 +0800 Message-Id: <20230127130541.1250865-11-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S14 X-Coremail-Antispam: 1UD129KBjvJXoW7uF4rWF15GFyUuFWrXF1UWrg_yoW8Wry8pF s8Cw45JrWrXw47GrySyw48X34SvF4kXr4aqFW7GFyrG34UXr45Xana9rWUtF98Kr9Y9r13 ZF15try0kry7A37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPKb7Iv0xC_Cr1lb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI 8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28C jxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI 8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vE x4A2jsIEc7CjxVAFwI0_Gr1j6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52 x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWU GwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI4 8JMxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z2 80aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU 0xZFpf9x07jF_M3UUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiCgsEE2PTtMIZxQAAsM X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181525701044548?= X-GMAIL-MSGID: =?utf-8?q?1756181525701044548?= From: Liao Chang Add instruction boundary check to ensure kprobe doesn't truncate any RVI instruction, which leads to kernel crash. Signed-off-by: Liao Chang --- arch/riscv/kernel/probes/kprobes.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index e1856b04db04..91a6b46909cc 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -49,11 +49,33 @@ static void __kprobes arch_simulate_insn(struct kprobe *p, struct pt_regs *regs) post_kprobe_handler(p, kcb, regs); } +bool __kprobes riscv_insn_boundary_check(unsigned long paddr) +{ +#if defined(CONFIG_RISCV_ISA_C) + unsigned long size = 0, offs = 0, len = 0, entry = 0; + + if (!kallsyms_lookup_size_offset(paddr, &size, &offs)) + return false; + + /* + * Scan instructions from function entry ensure the kprobe address + * is aligned with RVI or RVC boundary. + */ + entry = paddr - offs; + while ((entry + len) < paddr) + len += GET_INSN_LENGTH(*(kprobe_opcode_t *)(entry + len)); + return (entry + len) == paddr; +#else + return true; +#endif +} + int __kprobes arch_prepare_kprobe(struct kprobe *p) { unsigned long probe_addr = (unsigned long)p->addr; - if (probe_addr & 0x1) + /* for RVI/RCV hybrid kernel, it needs instruction boundary check */ + if ((probe_addr & 0x1) || !riscv_insn_boundary_check(probe_addr)) return -EILSEQ; /* copy instruction */ From patchwork Fri Jan 27 13:05:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49377 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825761wrn; Fri, 27 Jan 2023 05:13:16 -0800 (PST) X-Google-Smtp-Source: AK7set/Ol6k4IWgUW40uCJE+WTyBSicFOoczaUq91wV6kuq3wMqTZ991wmHaVrMmj27Q7pnVWXOP X-Received: by 2002:a17:906:7851:b0:878:4edf:4e06 with SMTP id p17-20020a170906785100b008784edf4e06mr8143200ejm.62.1674825196654; Fri, 27 Jan 2023 05:13:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825196; cv=none; d=google.com; s=arc-20160816; b=NGIfhDaj79WC3R8dg/DHReSNvmXJXoYkCls6DXih5O19gbS9f24QnxlLSuvEK0v5Md eJF9Dj0AE5RPB8xAlKNdX9MECFvKRDav0ppRp19FMJ2YM+rGdSwIYeEu/EkbYG973IJM aKDjg5L8JMBzbfSUfG81M5+eqCy8wlEvx4xdwPBB87BjgP2Do6KgxMQZL+ZnazUvh0T2 MwOBj8H1/YegYlUPc29AY7+2tUgTuZbO+uoqqj4YFAeCIxbkOW5DVW96uiuwRRGxIKod HeQK0u49dDfW1WyddLuNHKvdjrxrWHjAwMh7KSTbUyCfC+pAIkStNMtC9iDLvO9VqFhN KdRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=V1QyBN2sOzPyPXk0kyELsOVD6ToAm/rizPer/oGsU5Q=; b=dn9eXRup+f0QkZTB5ZdZWDsiEeRZe/iUB8CGIY7AJGt1yggAmDEiEWfhckR/DYZBaZ 3HUdYVD/lyUjJRf7O5z5zZLO7fK5VXUcg+pxsUG2lkcYrx8WlLHK3AbnUuT6mScCNRQd oBbbD6OGyJk1GRrEip6XTlt5rCJgIxQXnvzSnQcBc7b89k9mwGqH1Ry3jL/He3ROO4gb U7WHV9X4CARy4vvlI+EFA9JV7Wkmqy0kz5AUBa61DnYWkt1wACaRcqba0FmXS0cTSrkH g37Bz48VUsVpSYXVznQIqB7rdlKUbbBlpVPiMUF81wokdre9VqcQZjLlxEoAx6czKOoW 0KRw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 16-20020a170906015000b0087bdb8e120fsi1355083ejh.369.2023.01.27.05.12.52; Fri, 27 Jan 2023 05:13:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234534AbjA0NGc (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233894AbjA0NG3 (ORCPT ); Fri, 27 Jan 2023 08:06:29 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A97317CCA3 for ; Fri, 27 Jan 2023 05:06:26 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S15; Fri, 27 Jan 2023 21:06:10 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com Subject: [PATCH v6 11/13] riscv/kprobe: Fix instruction simulation of JALR Date: Fri, 27 Jan 2023 21:05:39 +0800 Message-Id: <20230127130541.1250865-12-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S15 X-Coremail-Antispam: 1UD129KBjvJXoWxGw17AFyUCFy3trW8Jw13Jwb_yoW5Xw1Upr W5Jr1UCr40grn7Jay7Jr1rAr4UJr1jk3WxWF18Kw1FkF1UW34UXw18tF43AF1Dtr13Ca47 Jr1DJr40qr18GaUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPKb7Iv0xC_Cr1lb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI 8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28C jxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI 8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vE x4A2jsIEc7CjxVAFwI0_Gr1j6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52 x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWU GwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI4 8JMxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z2 80aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU 0xZFpf9x07jF_M3UUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiAwoEE2PTrr4g1wAAsJ X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181505428268253?= X-GMAIL-MSGID: =?utf-8?q?1756181505428268253?= From: Liao Chang Set kprobe at 'jalr 1140(ra)' of vfs_write results in the following crash: [ 32.092235] Unable to handle kernel access to user memory without uaccess routines at virtual address 00aaaaaad77b1170 [ 32.093115] Oops [#1] [ 32.093251] Modules linked in: [ 32.093626] CPU: 0 PID: 135 Comm: ftracetest Not tainted 6.2.0-rc2-00013-gb0aa5e5df0cb-dirty #16 [ 32.093985] Hardware name: riscv-virtio,qemu (DT) [ 32.094280] epc : ksys_read+0x88/0xd6 [ 32.094855] ra : ksys_read+0xc0/0xd6 [ 32.095016] epc : ffffffff801cda80 ra : ffffffff801cdab8 sp : ff20000000d7bdc0 [ 32.095227] gp : ffffffff80f14000 tp : ff60000080f9cb40 t0 : ffffffff80f13e80 [ 32.095500] t1 : ffffffff8000c29c t2 : ffffffff800dbc54 s0 : ff20000000d7be60 [ 32.095716] s1 : 0000000000000000 a0 : ffffffff805a64ae a1 : ffffffff80a83708 [ 32.095921] a2 : ffffffff80f160a0 a3 : 0000000000000000 a4 : f229b0afdb165300 [ 32.096171] a5 : f229b0afdb165300 a6 : ffffffff80eeebd0 a7 : 00000000000003ff [ 32.096411] s2 : ff6000007ff76800 s3 : fffffffffffffff7 s4 : 00aaaaaad77b1170 [ 32.096638] s5 : ffffffff80f160a0 s6 : ff6000007ff76800 s7 : 0000000000000030 [ 32.096865] s8 : 00ffffffc3d97be0 s9 : 0000000000000007 s10: 00aaaaaad77c9410 [ 32.097092] s11: 0000000000000000 t3 : ffffffff80f13e48 t4 : ffffffff8000c29c [ 32.097317] t5 : ffffffff8000c29c t6 : ffffffff800dbc54 [ 32.097505] status: 0000000200000120 badaddr: 00aaaaaad77b1170 cause: 000000000000000d [ 32.098011] [] ksys_write+0x6c/0xd6 [ 32.098222] [] sys_write+0x2a/0x38 [ 32.098405] [] ret_from_syscall+0x0/0x2 Since the rs1 and rd might be the same one, such as 'jalr 1140(ra)', hence it requires obtaining the target address from rs1 followed by updating rd. Fixes: c22b0bcb1dd0 ("riscv: Add kprobes supported") Signed-off-by: Liao Chang --- arch/riscv/kernel/probes/simulate-insn.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/arch/riscv/kernel/probes/simulate-insn.c b/arch/riscv/kernel/probes/simulate-insn.c index 7441ac8a6843..8402020010d5 100644 --- a/arch/riscv/kernel/probes/simulate-insn.c +++ b/arch/riscv/kernel/probes/simulate-insn.c @@ -75,13 +75,9 @@ bool __kprobes simulate_jalr(u32 opcode, unsigned long addr, struct pt_regs *reg if (!ret) return ret; - ret = rv_insn_reg_set_val(regs, rd_index, addr + 4); - if (!ret) - return ret; - instruction_pointer_set(regs, (base_addr + sign_extend32((imm), 11))&~1); - return ret; + return rv_insn_reg_set_val(regs, rd_index, addr + 4); } #define auipc_rd_idx(opcode) \ From patchwork Fri Jan 27 13:05:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49400 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp840668wrn; Fri, 27 Jan 2023 05:44:56 -0800 (PST) X-Google-Smtp-Source: AK7set+YWRSZMWurlkF5bWLlx4yc2hMJqlf+6/OZjc5ZkrPzcEBbrvtQcBYyyPcG+lvNQMxwr6vb X-Received: by 2002:a17:903:41cc:b0:196:37a4:775b with SMTP id u12-20020a17090341cc00b0019637a4775bmr9730483ple.33.1674827096148; Fri, 27 Jan 2023 05:44:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674827096; cv=none; d=google.com; s=arc-20160816; b=rj4n0e2K/HiaS3YuJ1IkxuK+RnaIymWBdMyH8kgL+/80FCZn1AZozzYEoYZ1uXFHLA i7rKIum9SJICYmDbQyM2wB8XGZ/1R6dQzNTVYSb36UoPqGRDD/dg2fSayuYAYED9sctT 5S2I3DMH2B+PiJVaCnkF858mb12EvSqvHAfyfPvtxkznj3ZIgNGTZJZS4EUXKugbT5Xr oALt39KkkxWKxqG2P9Z2r9FJBJfBpo7LRikQSxVnf1tbGYYfqZ+PQ05H4lIw8xy8+A6y oCWeNDWl1YTtTBnFa6Nid3+UVdzCkTBuy5YOhEsj6vm9J0rCPOFxZ4hMPYFPThY5pLU/ /uvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=MqhfdXTgtu30q1U/PQdFXcenAVNGicwfj+rqLk/0IiU=; b=pBH2nGljZcwHQdjKuJuekvmJzYIEmjyDBdg2fbfj8OintKs8km81sec7rh1WH7HtVb 5j46kfhsV8nnLwV5JrpDyUw60Xe0wVOnd6VQD4cyFaee7j3rXVNBdHNwTWuxLyeaS1Zu XELrlx7Yr5Sj8yOqKnms3I9o+2PZjE4niJj74G3XFfnF3M3LhXh7Wq7dUPz2q7wcsiCF SPBOGupUYbB3eQjGkYSyZqiHMTzAjjgPyXNdP89h3eX2tK2pn9HGuxU6iD57De6SvOYP zXLhrjzKXs+OwAqkCiJmrEvobU7kVDOPzZLEJfP1h1lyv9e6uhf7RwQ6E3YGMT3YVTf9 Kv5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j19-20020a633c13000000b004c697f2f114si4402384pga.190.2023.01.27.05.44.43; Fri, 27 Jan 2023 05:44:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234608AbjA0NSp (ORCPT + 99 others); Fri, 27 Jan 2023 08:18:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234704AbjA0NSY (ORCPT ); Fri, 27 Jan 2023 08:18:24 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5BF5C2B089 for ; Fri, 27 Jan 2023 05:17:56 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S16; Fri, 27 Jan 2023 21:06:11 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com Subject: [PATCH v6 12/13] riscv/kprobe: Move exception related symbols to .kprobe_blacklist Date: Fri, 27 Jan 2023 21:05:40 +0800 Message-Id: <20230127130541.1250865-13-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S16 X-Coremail-Antispam: 1UD129KBjvJXoW7uw45Xw18uF45WrWrKF18uFg_yoW8uw4kpF 4DCrn5WrWrWryfC34ayr1DuayxJwn5Ww4ayr1ruFWrAF42grW8Awn0v3yDXryIgF4xW3yr uF109rWYga4UX37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPKb7Iv0xC_tr1lb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI 8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28C jxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW8JVW5JwA2z4x0Y4vE2Ix0cI 8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vE x4A2jsIEc7CjxVAFwI0_Gr1j6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52 x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWU GwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI4 8JMxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z2 80aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU 0xZFpf9x07jzmhrUUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiAwkEE2PTrr4g2AAAsF X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756183497615935197?= X-GMAIL-MSGID: =?utf-8?q?1756183497615935197?= From: Liao Chang Due to the generic exception entry and exit code is part of the critical path for kprobe breakpoint and uprobe syscall entry, set a kprobe on the assembly symbols in entry.S result in kernel stack overflow crash, hence it has to explicitly blacklist it, requiring a new _ASM_NOKPROBE() asm helper. Signed-off-by: Liao Chang Reviewed-by: Björn Töpel --- arch/riscv/include/asm/asm.h | 10 ++++++++++ arch/riscv/kernel/entry.S | 12 ++++++++++++ arch/riscv/kernel/mcount.S | 1 + 3 files changed, 23 insertions(+) diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h index 816e753de636..5d9f13d8b809 100644 --- a/arch/riscv/include/asm/asm.h +++ b/arch/riscv/include/asm/asm.h @@ -81,6 +81,16 @@ .endr .endm +#ifdef CONFIG_KPROBES +#define _ASM_NOKPROBE(entry) \ + .pushsection "_kprobe_blacklist", "aw" ; \ + .balign SZREG ; \ + REG_ASM entry ; \ + .popsection +#else +#define _ASM_NOKPROBE(entry) +#endif + #endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_ASM_H */ diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 99d38fdf8b18..9e8882a78523 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -606,3 +606,15 @@ ENTRY(__user_rt_sigreturn) scall END(__user_rt_sigreturn) #endif + +_ASM_NOKPROBE(handle_exception) +_ASM_NOKPROBE(_restore_kernel_tpsp) +_ASM_NOKPROBE(_save_context) +_ASM_NOKPROBE(ret_from_exception) +_ASM_NOKPROBE(ret_from_syscall) +_ASM_NOKPROBE(__switch_to) +_ASM_NOKPROBE(ret_from_syscall_rejected) +_ASM_NOKPROBE(restore_all) +_ASM_NOKPROBE(resume_kernel) +_ASM_NOKPROBE(resume_userspace) +_ASM_NOKPROBE(check_syscall_nr) diff --git a/arch/riscv/kernel/mcount.S b/arch/riscv/kernel/mcount.S index 30102aadc4d7..7393b8895ef3 100644 --- a/arch/riscv/kernel/mcount.S +++ b/arch/riscv/kernel/mcount.S @@ -54,6 +54,7 @@ ENTRY(ftrace_stub) #endif ret ENDPROC(ftrace_stub) +_ASM_NOKPROBE(MCOUNT_NAME) #ifdef CONFIG_FUNCTION_GRAPH_TRACER ENTRY(return_to_handler) From patchwork Fri Jan 27 13:05:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xim X-Patchwork-Id: 49376 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp825295wrn; Fri, 27 Jan 2023 05:12:17 -0800 (PST) X-Google-Smtp-Source: AMrXdXt6o5sfA4CAzHGl8NUayYW2eXKmRzm7aE8Y/0nO9NBgGNCPx8fD5A+rySDc8u9cxFOXCZL9 X-Received: by 2002:a17:90b:4a0c:b0:22a:3c8:9d1 with SMTP id kk12-20020a17090b4a0c00b0022a03c809d1mr29866145pjb.32.1674825136895; Fri, 27 Jan 2023 05:12:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674825136; cv=none; d=google.com; s=arc-20160816; b=SHqXtxJsyparTnslJu0ee+N9ol87MiGng+/3L4cP6xccUggjfbSYnmopWmeLgTjEyG gPNvAS990Z2j1/YWTB3LmyJEffoZqeoa5l+fkbfjBfuwkVwJ1qnRUKu7xBYMt+kYJHFQ lfSBT9WzQ5E7OUOTOFsJVx2ISUmVqSYSn9h6PP8UvoEX81kbymYjksNk0EjFAqZCQlEK bnW36/GD43ctxLivhDA7s52m6P7TRLiBD2psBxDUTc6fiB6kviEb9GbKVYxw8RHUyqto 3Sst4qXpZKCUqKu62l4LWLVvkv4E8dOqh7sUFgCYGtWqJjM080VYDp0PE8aNgUn4kyZL JxRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=NPoM3f7T232lcSpFvVctGvWzD+Hh9O3vZLwaPnGjN9c=; b=Opmv0ghTNSuGpm1GnvQV6Yyyb19FSTXHXizsaScesZCfcpXsQnG+N31fv4RSSlyukv w3KtxCLORov6RDLmdojhvFTyn3IHmWCJfKUUQD3ZC8/ERHBprQRlvlDX6Q/5WBMNx2w1 9YwQ0N1YKqlO22++kbLAapOqPZ+vfUBa9uO/fWPmsUy7R7YI+ilcsqQO2HuMdwTBhNbZ HM5lheZ6jkdutvXqv30hs5IoEp9eVhKGPPn8fibbIlFi1/sxeGxGtwz6EMmuM8dWIgSh ca6UDAbLFXLzxkHDQDXHDFY8JkVenDOBPOzrmprqL//54++EoEO/WehKtchAiPFOrEgJ LuiQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m5-20020a17090a3f8500b00229ef929eaasi7499578pjc.181.2023.01.27.05.12.03; Fri, 27 Jan 2023 05:12:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234518AbjA0NGa (ORCPT + 99 others); Fri, 27 Jan 2023 08:06:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230458AbjA0NG2 (ORCPT ); Fri, 27 Jan 2023 08:06:28 -0500 Received: from cstnet.cn (smtp80.cstnet.cn [159.226.251.80]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id ED8A221A2C for ; Fri, 27 Jan 2023 05:06:25 -0800 (PST) Received: from cgk-Precision-3650-Tower.. (unknown [219.141.235.82]) by APP-01 (Coremail) with SMTP id qwCowADX3Eg9zNNjDyF3Ag--.52827S17; Fri, 27 Jan 2023 21:06:11 +0800 (CST) From: Chen Guokai To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, rostedt@goodmis.org, mingo@redhat.com, sfr@canb.auug.org.au Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, liaochang1@huawei.com Subject: [PATCH v6 13/13] selftest/kprobes: Add testcase for kprobe SYM[+offs] Date: Fri, 27 Jan 2023 21:05:41 +0800 Message-Id: <20230127130541.1250865-14-chenguokai17@mails.ucas.ac.cn> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> References: <20230127130541.1250865-1-chenguokai17@mails.ucas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADX3Eg9zNNjDyF3Ag--.52827S17 X-Coremail-Antispam: 1UD129KBjvJXoW7CF15GFW3Xw15CF47GryfWFg_yoW8urW3pa s8Cw15Kw4SqFW3K3s3Gw48Cr1Fy3W8Ar43Jw1DJr4a9a4Uur1jqan2grWUAas7ur4q9r13 Aw1YvFy7KFy7ZFDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPKb7Iv0xC_Cr1lb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI 8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28C jxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW5JVW7JwA2z4x0Y4vE2Ix0cI 8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_Gr1j6F4UJwA2z4x0Y4vE x4A2jsIEc7CjxVAFwI0_Gr1j6F4UJwAac4AC62xK8xCEY4vEwIxC4wAS0I0E0xvYzxvE52 x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWU GwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI4 8JMxkIecxEwVAFwVW5GwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z2 80aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU 0xZFpf9x07jF_M3UUUUU= X-Originating-IP: [219.141.235.82] X-CM-SenderInfo: xfkh0w5xrntxyrx6ztxlovh3xfdvhtffof0/1tbiCgcEE2PTtMIZyQAAsM X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756181442925235013?= X-GMAIL-MSGID: =?utf-8?q?1756181442925235013?= From: Liao Chang This testcase set multiple kprobes to the function that contains a series of complex opcode pattern, it helps discover some subtle bugs in the instruction decoder and kprobe jump optimization. Signed-off-by: Liao Chang --- .../ftrace/test.d/kprobe/kprobe_sym_offs.tc | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_sym_offs.tc diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_sym_offs.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_sym_offs.tc new file mode 100644 index 000000000000..0007bec48308 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_sym_offs.tc @@ -0,0 +1,49 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Kprobe dynamic event with offset +# requires: kprobe_events +TARGET_FUNC=vfs_write + +dec_addr() { # hexaddr + printf "%d" "0x"`echo $1 | tail -c 8` +} + +set_offs() { # target next + SYMADDR=$1 + ENDADDR=$2 + A1=`dec_addr $SYMADDR` + A2=`dec_addr $ENDADDR` + NEXT=`expr $A2 - $A1` # offset to previous symbol +} + +# Get the instruction number between two contiguous symbols +set_offs `grep -A1 -w ${TARGET_FUNC} /proc/kallsyms | cut -f 1 -d " " | xargs` + +# Instruction length depends on the machine architecute. +case `uname -m` in + arm64) LEN=4;; + riscv32|riscv64) LEN=2;; + *) LEN=2;; +esac + +N=0 +OFFS=0 +echo "Setup up kprobes on each instruction in function $TARGET_FUNC" +while true; do + N=$(($N+1)) + ! echo p ${TARGET_FUNC}+${OFFS} >> kprobe_events + OFFS=$(($OFFS+$LEN)) + test $OFFS -eq $NEXT && break +done + +L=`cat kprobe_events | wc -l` +echo "The number of kprobes events ($L) not $N in function $TARGET_FUNC" + +echo 1 > events/kprobes/enable +# Trigger vfs_write to test kprobes +cat kprobe_events >> $testlog +echo 0 > events/kprobes/enable +echo > kprobe_events +echo "Waiting for unoptimizing & freeing" +sleep 5 +echo "Done"