From patchwork Fri Apr 28 09:51:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hou Wenlong X-Patchwork-Id: 88579 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp825012vqo; Fri, 28 Apr 2023 03:19:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7JjQNvcjTq2XDJ8gyUPcKdzEMXbvRMvairocCwL3KaCVSPsYzQULjGYe5LotYsfIjD2N4t X-Received: by 2002:a17:90a:db8c:b0:249:822d:e5bc with SMTP id h12-20020a17090adb8c00b00249822de5bcmr4810594pjv.5.1682677151164; Fri, 28 Apr 2023 03:19:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682677151; cv=none; d=google.com; s=arc-20160816; b=Aad0qMEnjd1apNzMAD2yQZ1wG0L2IXiayO0JKGu5fdoix1e4HBLRFeqtElfPGa2EWz oipONxPuCJaKGX5b6ASP84dvIGlOTkXDxxuUmtSLXr5zzBIA5TPmRfhdDaBjmmmrziAM TrRn+juSXXqOMZftMqo4h2iKd7zGWVEkzKDGEiKBIFnMIi/kSPme/nNyYnbgyWpPkQMH pxQPTHUpcNi4ikFuNBnEMkpMw8mEGgJla+OgEO5cac1IG6fTZTlH5YwxOPSgF+K+0KCf 5/UBPewHe6wBYfYDbT7Fh6jXhUex1eFgOPqzxGlamJzfdqaX0xfLWJWcQxHaLgeZ3KHK kWWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=c1SgUbN7R7pQbFxp4okH420mq9fXavcx3ZccKD7ytlc=; b=zCZyoB7/JQGZFM3Mm7pveO82H/V/puEvnCjjRBbrZ4LqKZ2cXkegSLypqq/e00wNUO WJ4gsb37E2NQPEtIeakYDuepQPrBulgX4HYe/3BT6f63vswc51YoOuMF3sKyi5NK/CJc frwU4l3+Ot+r8gtUFI26GuWg/fEhZWsN9/5+K0Pgv/kTYwZRBC2utmtbC9kTMJ+fLbCc bGTgk1TR9FfZ8nH8HJKfpFMCZuMDm/pXyuE7YpjHEVx7sTnwTK8bcGvR+OSEIrxFRyZ5 pvTtaTCZcl7/xlw94TIFCQGtqtxz2pRxd64Dgn4KXli/Wv0ONU3Nlwn0bt8AOXMpF4yA p2Sg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=antgroup.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mn6-20020a17090b188600b0024714b2954csi1806659pjb.78.2023.04.28.03.18.56; Fri, 28 Apr 2023 03:19:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=antgroup.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346046AbjD1KCP (ORCPT + 99 others); Fri, 28 Apr 2023 06:02:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230341AbjD1KBv (ORCPT ); Fri, 28 Apr 2023 06:01:51 -0400 Received: from out0-214.mail.aliyun.com (out0-214.mail.aliyun.com [140.205.0.214]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84B1061BE; Fri, 28 Apr 2023 03:01:27 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018047187;MF=houwenlong.hwl@antgroup.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---.STCEPeI_1682675581; Received: from localhost(mailfrom:houwenlong.hwl@antgroup.com fp:SMTPD_---.STCEPeI_1682675581) by smtp.aliyun-inc.com; Fri, 28 Apr 2023 17:53:01 +0800 From: "Hou Wenlong" To: linux-kernel@vger.kernel.org Cc: "Thomas Garnier" , "Lai Jiangshan" , "Kees Cook" , "Hou Wenlong" , "Steven Rostedt" , "Masami Hiramatsu" , "Mark Rutland" , "Thomas Gleixner" , "Ingo Molnar" , "Borislav Petkov" , "Dave Hansen" , , "H. Peter Anvin" , "Huacai Chen" , "Qing Zhang" , Subject: [PATCH RFC 22/43] x86/ftrace: Adapt ftrace nop patching for PIE support Date: Fri, 28 Apr 2023 17:51:02 +0800 Message-Id: <38a5029cd2590e04209117740f8912db36eff58f.1682673543.git.houwenlong.hwl@antgroup.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1764414876267548878?= X-GMAIL-MSGID: =?utf-8?q?1764414876267548878?= From: Thomas Garnier From: Thomas Garnier When using PIE with function tracing, the compiler generates a call through the GOT (call *__fentry__@GOTPCREL). This instruction takes 6-bytes instead of 5-bytes with a relative call. And -mnop-mcount option is not implemented for -fPIE now. If PIE is enabled, replace the 6th byte of the GOT call by a 1-byte nop so ftrace can handle the previous 5-bytes as before. [Hou Wenlong: Adapt code change and fix wrong offset calculation in make_nop_x86()] Signed-off-by: Thomas Garnier Co-developed-by: Hou Wenlong Signed-off-by: Hou Wenlong Cc: Lai Jiangshan Cc: Kees Cook --- arch/x86/kernel/ftrace.c | 46 ++++++++++++++++++++++- scripts/recordmcount.c | 81 ++++++++++++++++++++++++++-------------- 2 files changed, 98 insertions(+), 29 deletions(-) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 5e7ead52cfdb..b795f9dde561 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -124,6 +124,50 @@ ftrace_modify_code_direct(unsigned long ip, const char *old_code, return 0; } +/* Bytes before call GOT offset */ +static const unsigned char got_call_preinsn[] = { 0xff, 0x15 }; + +static int __ref +ftrace_modify_initial_code(unsigned long ip, unsigned const char *old_code, + unsigned const char *new_code) +{ + unsigned char replaced[MCOUNT_INSN_SIZE + 1]; + + /* + * If PIE is not enabled default to the original approach to code + * modification. + */ + if (!IS_ENABLED(CONFIG_X86_PIE)) + return ftrace_modify_code_direct(ip, old_code, new_code); + + ftrace_expected = old_code; + + /* Ensure the instructions point to a call to the GOT */ + if (copy_from_kernel_nofault(replaced, (void *)ip, sizeof(replaced))) { + WARN_ONCE(1, "invalid function"); + return -EFAULT; + } + + if (memcmp(replaced, got_call_preinsn, sizeof(got_call_preinsn))) { + WARN_ONCE(1, "invalid function call"); + return -EINVAL; + } + + /* + * Build a nop slide with a 5-byte nop and 1-byte nop to keep the ftrace + * hooking algorithm working with the expected 5 bytes instruction. + */ + memset(replaced, x86_nops[1][0], sizeof(replaced)); + memcpy(replaced, new_code, MCOUNT_INSN_SIZE); + + /* replace the text with the new text */ + if (ftrace_poke_late) + text_poke_queue((void *)ip, replaced, MCOUNT_INSN_SIZE + 1, NULL); + else + text_poke_early((void *)ip, replaced, MCOUNT_INSN_SIZE + 1); + return 0; +} + int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr) { unsigned long ip = rec->ip; @@ -141,7 +185,7 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long ad * just modify the code directly. */ if (addr == MCOUNT_ADDR) - return ftrace_modify_code_direct(ip, old, new); + return ftrace_modify_initial_code(ip, old, new); /* * x86 overrides ftrace_replace_code -- this function will never be used diff --git a/scripts/recordmcount.c b/scripts/recordmcount.c index e30216525325..02783a29d428 100644 --- a/scripts/recordmcount.c +++ b/scripts/recordmcount.c @@ -218,36 +218,10 @@ static void *mmap_file(char const *fname) return file_map; } - -static unsigned char ideal_nop5_x86_64[5] = { 0x0f, 0x1f, 0x44, 0x00, 0x00 }; -static unsigned char ideal_nop5_x86_32[5] = { 0x3e, 0x8d, 0x74, 0x26, 0x00 }; -static unsigned char *ideal_nop; - static char rel_type_nop; - static int (*make_nop)(void *map, size_t const offset); -static int make_nop_x86(void *map, size_t const offset) -{ - uint32_t *ptr; - unsigned char *op; - - /* Confirm we have 0xe8 0x0 0x0 0x0 0x0 */ - ptr = map + offset; - if (*ptr != 0) - return -1; - - op = map + offset - 1; - if (*op != 0xe8) - return -1; - - /* convert to nop */ - if (ulseek(offset - 1, SEEK_SET) < 0) - return -1; - if (uwrite(ideal_nop, 5) < 0) - return -1; - return 0; -} +static unsigned char *ideal_nop; static unsigned char ideal_nop4_arm_le[4] = { 0x00, 0x00, 0xa0, 0xe1 }; /* mov r0, r0 */ static unsigned char ideal_nop4_arm_be[4] = { 0xe1, 0xa0, 0x00, 0x00 }; /* mov r0, r0 */ @@ -504,6 +478,50 @@ static void MIPS64_r_info(Elf64_Rel *const rp, unsigned sym, unsigned type) }).r_info; } +static unsigned char ideal_nop5_x86_64[5] = { 0x0f, 0x1f, 0x44, 0x00, 0x00 }; +static unsigned char ideal_nop6_x86_64[6] = { 0x66, 0x0f, 0x1f, 0x44, 0x00, 0x00 }; +static unsigned char ideal_nop5_x86_32[5] = { 0x3e, 0x8d, 0x74, 0x26, 0x00 }; +static size_t ideal_nop_x86_size; + +static unsigned char stub_default_x86[2] = { 0xe8, 0x00 }; /* call relative */ +static unsigned char stub_got_x86[3] = { 0xff, 0x15, 0x00 }; /* call .got */ +static unsigned char *stub_x86; +static size_t stub_x86_size; + +static int make_nop_x86(void *map, size_t const offset) +{ + uint32_t *ptr; + size_t stub_offset = offset + 1 - stub_x86_size; + + /* Confirm we have the expected stub */ + ptr = map + stub_offset; + if (memcmp(ptr, stub_x86, stub_x86_size)) + return -1; + + /* convert to nop */ + if (ulseek(stub_offset, SEEK_SET) < 0) + return -1; + if (uwrite(ideal_nop, ideal_nop_x86_size) < 0) + return -1; + return 0; +} + +/* Swap the stub and nop for a got call if the binary is built with PIE */ +static int is_fake_mcount_x86_x64(Elf64_Rel const *rp) +{ + if (ELF64_R_TYPE(rp->r_info) == R_X86_64_GOTPCREL) { + ideal_nop = ideal_nop6_x86_64; + ideal_nop_x86_size = sizeof(ideal_nop6_x86_64); + stub_x86 = stub_got_x86; + stub_x86_size = sizeof(stub_got_x86); + mcount_adjust_64 = 1 - stub_x86_size; + } + + /* Once the relocation was checked, rollback to default */ + is_fake_mcount64 = fn_is_fake_mcount64; + return is_fake_mcount64(rp); +} + static int do_file(char const *const fname) { unsigned int reltype = 0; @@ -568,6 +586,9 @@ static int do_file(char const *const fname) rel_type_nop = R_386_NONE; make_nop = make_nop_x86; ideal_nop = ideal_nop5_x86_32; + ideal_nop_x86_size = sizeof(ideal_nop5_x86_32); + stub_x86 = stub_default_x86; + stub_x86_size = sizeof(stub_default_x86); mcount_adjust_32 = -1; gpfx = 0; break; @@ -597,9 +618,13 @@ static int do_file(char const *const fname) case EM_X86_64: make_nop = make_nop_x86; ideal_nop = ideal_nop5_x86_64; + ideal_nop_x86_size = sizeof(ideal_nop5_x86_64); + stub_x86 = stub_default_x86; + stub_x86_size = sizeof(stub_default_x86); reltype = R_X86_64_64; rel_type_nop = R_X86_64_NONE; - mcount_adjust_64 = -1; + is_fake_mcount64 = is_fake_mcount_x86_x64; + mcount_adjust_64 = 1 - stub_x86_size; gpfx = 0; break; } /* end switch */