From patchwork Wed Nov 2 08:49:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14095 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494366wru; Wed, 2 Nov 2022 01:54:15 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ORr+mne/dJ4lXtc6mKcsd5wv1sUka+kgtgnJK5sKdPh1nozYBhq4rgt3Xga6cuFvYA29f X-Received: by 2002:a17:906:6a13:b0:7ad:b598:9a52 with SMTP id qw19-20020a1709066a1300b007adb5989a52mr21022689ejc.205.1667379255443; Wed, 02 Nov 2022 01:54:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379255; cv=none; d=google.com; s=arc-20160816; b=r5wiDkE33qb3sINrA3N0Fyw/B8DuTgyImVa10Sf1Hgh1KymWyOLrm2xyCaQsw4wrrh uyEslQnMMVit7ulgEyFo4dIg3NhblwQvRcfDMbANoKFH3J724Xjzv6jwK+fUpOa8dY/Y /bHSdU51gKIFF/PSK54cYoeFIr/lrRzpBQhmAllq78XsHRKF2X6LQXfdqXrlUq6+yZmj jo1nOyVLysDwq3qr/QCXCKAg9We+oNwCqtO3UZMbLsbVNlWg/FuyXgKdcGVceS7Dqavy kOiHq8fCJKGJIVzsBw4YF05qeovx2zd5NrZzC/WGHpuRQ9GlbG4qKEYNGMimS/FQyC8U D3oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=27ivNInO1WNZHSaFBcLCEgzvlo2LVYUDZnP7WEvjbqs=; b=FVoqFn9dxgFJEDHVJ4T72t1rR9raXlZcC4SMVE3xnODN0qkjecWj0iIlQ+wRv8WXVN /6BS2XUUW67cpw3HuXFdwC+1Yhy9BgzziD+5RpL9nn4hxMkb43BPVh14Zx0wwqGQDoia NLLPexLHaOvoPRpFm1c3qPS0HZKhAwAvBaFEK2YzoOLw/mFpZ8Edfb7CE6sO1NdGUHX5 hI6DriFlxZ+QUQF4+1hxu5fe6GXxIKj/0dOYmlaM0AJS0gNmXiOegSt2C1vgiRcp1JL2 b8ZmBXT/6H1h/J/vUzEIa3DdJkCWitwy64Lq0DYTLiEaAnMPuC/Gl31UhfEjnN3dGxQj QIaQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sg39-20020a170907a42700b00730870cb4b6si16489716ejc.621.2022.11.02.01.53.51; Wed, 02 Nov 2022 01:54:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230388AbiKBIub (ORCPT + 99 others); Wed, 2 Nov 2022 04:50:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229539AbiKBIu3 (ORCPT ); Wed, 2 Nov 2022 04:50:29 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05DC925E8C; Wed, 2 Nov 2022 01:50:27 -0700 (PDT) Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4N2LFB28ZwzmVZb; Wed, 2 Nov 2022 16:50:22 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:26 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:24 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 1/9] scripts/kallsyms: rename build_initial_tok_table() Date: Wed, 2 Nov 2022 16:49:13 +0800 Message-ID: <20221102084921.1615-2-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373870169606139?= X-GMAIL-MSGID: =?utf-8?q?1748373870169606139?= Except for the function build_initial_tok_table(), no token abbreviation is used elsewhere. $ cat scripts/kallsyms.c | grep tok | wc -l 33 $ cat scripts/kallsyms.c | grep token | wc -l 31 Here, it would be clearer to use the full name. Signed-off-by: Zhen Lei Reviewed-by: Petr Mladek --- scripts/kallsyms.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 03fa07ad45d95b8..ab105bdde4efe4f 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -573,7 +573,7 @@ static void forget_symbol(const unsigned char *symbol, int len) } /* do the initial token count */ -static void build_initial_tok_table(void) +static void build_initial_token_table(void) { unsigned int i; @@ -698,7 +698,7 @@ static void insert_real_symbols_in_table(void) static void optimize_token_table(void) { - build_initial_tok_table(); + build_initial_token_table(); insert_real_symbols_in_table(); From patchwork Wed Nov 2 08:49:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14098 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494554wru; Wed, 2 Nov 2022 01:54:50 -0700 (PDT) X-Google-Smtp-Source: AMsMyM55gTuRs6/FKnUuDRSpydKmBuE8Q+n+d4sDgfN5gSw4reOvuYxvkEPWYVSwZH0VsXAxpCpG X-Received: by 2002:a05:6402:538f:b0:444:c17b:1665 with SMTP id ew15-20020a056402538f00b00444c17b1665mr23752370edb.98.1667379290068; Wed, 02 Nov 2022 01:54:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379290; cv=none; d=google.com; s=arc-20160816; b=LUrxCrYd4ejX7CRmYP2PDnyLrLOEBlY/TL/ZbkWjvfWX5i0ZH1keYG87HqkQOD4OWQ ZU2vDJbkEs/JI1gJa85qVPRCgTViPkRbc4MbvQQUOiktmwX3DJd0Hl6RSa0TWo8QdhvB 5JVoUvGbHtT8Du82mdjtkZgEFfMRKBf6EyUmaEx257vMegFRDEgPmIJG/7aCDAAXBTFO UFgf9ejl130QDEpjrh+5ZDu06ctI9X8aQLzyL7QvLhh5pJNeEN1XGhenY4tT1w7PipRf ETa+ov7D0vwOYqH7SOtp+/thyqLE4gfryXi3pk7428KBzNPjelfIpDZcaXFSXM9PG42u P2bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=0hnrVhR7sWSNbldphcgfZEn7Hj+ySftk6R/da+/kWT0=; b=L6uVvYTxUrWjcwPJtYX408MhQWddOzyn8VgEN5DVzQukLqPmDT183Z1FoBd0LAIaFf pPHFT+um/jB0z1p4e6WWxeiqVviNZ/56/sEEmezJrt9xd/6crkxhedtjmV4U1p7Q8Rv4 F+ycmWDwZFXzcbCL52GJl+NFhFYAtcSso+9gr0IIIgQKgAkcnVDPEgZWoHW4oZbtjU0n Z1PRbJOPX3CFnykE5yX4HZQC/YIcILXAsTQuagLbhVL6Zi71+TDpN9ipdFokiVsm8UXG RLgd6LlWsChWg+clVgWtyR8O2S+nUxI7DSnH9XFeOCJMn4EIjmj0tmR+4GE+PWKLK+aT BhyQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nb15-20020a1709071c8f00b00780805b99ccsi17939550ejc.648.2022.11.02.01.54.26; Wed, 02 Nov 2022 01:54:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229975AbiKBIvh (ORCPT + 99 others); Wed, 2 Nov 2022 04:51:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231205AbiKBIu7 (ORCPT ); Wed, 2 Nov 2022 04:50:59 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEC4427DF1; Wed, 2 Nov 2022 01:50:55 -0700 (PDT) Received: from dggpemm500023.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4N2LFQ6RwhzHvbD; Wed, 2 Nov 2022 16:50:34 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500023.china.huawei.com (7.185.36.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:26 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:25 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 2/9] kallsyms: Improve the performance of kallsyms_lookup_name() Date: Wed, 2 Nov 2022 16:49:14 +0800 Message-ID: <20221102084921.1615-3-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373906490190812?= X-GMAIL-MSGID: =?utf-8?q?1748373906490190812?= Currently, to search for a symbol, we need to expand the symbols in 'kallsyms_names' one by one, and then use the expanded string for comparison. It's O(n). If we sort names in ascending order like addresses, we can also use binary search. It's O(log(n)). In order not to change the implementation of "/proc/kallsyms", the table kallsyms_names[] is still stored in a one-to-one correspondence with the address in ascending order. Add array kallsyms_seqs_of_names[], it's indexed by the sequence number of the sorted names, and the corresponding content is the sequence number of the sorted addresses. For example: Assume that the index of NameX in array kallsyms_seqs_of_names[] is 'i', the content of kallsyms_seqs_of_names[i] is 'k', then the corresponding address of NameX is kallsyms_addresses[k]. The offset in kallsyms_names[] is get_symbol_offset(k). Note that the memory usage will increase by (4 * kallsyms_num_syms) bytes, the next two patches will reduce (1 * kallsyms_num_syms) bytes and properly handle the case CONFIG_LTO_CLANG=y. Performance test results: (x86) Before: min=234, max=10364402, avg=5206926 min=267, max=11168517, avg=5207587 After: min=1016, max=90894, avg=7272 min=1014, max=93470, avg=7293 The average lookup performance of kallsyms_lookup_name() improved 715x. Signed-off-by: Zhen Lei --- kernel/kallsyms.c | 86 +++++++++++++++++++++++++++++++++----- kernel/kallsyms_internal.h | 1 + scripts/kallsyms.c | 37 ++++++++++++++++ 3 files changed, 113 insertions(+), 11 deletions(-) diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 60c20f301a6ba2c..ba351dfa109b6ac 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -187,26 +187,90 @@ static bool cleanup_symbol_name(char *s) return false; } +static int compare_symbol_name(const char *name, char *namebuf) +{ + int ret; + + ret = strcmp(name, namebuf); + if (!ret) + return ret; + + if (cleanup_symbol_name(namebuf) && !strcmp(name, namebuf)) + return 0; + + return ret; +} + +static int kallsyms_lookup_names(const char *name, + unsigned int *start, + unsigned int *end) +{ + int ret; + int low, mid, high; + unsigned int seq, off; + char namebuf[KSYM_NAME_LEN]; + + low = 0; + high = kallsyms_num_syms - 1; + + while (low <= high) { + mid = low + (high - low) / 2; + seq = kallsyms_seqs_of_names[mid]; + off = get_symbol_offset(seq); + kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); + ret = compare_symbol_name(name, namebuf); + if (ret > 0) + low = mid + 1; + else if (ret < 0) + high = mid - 1; + else + break; + } + + if (low > high) + return -ESRCH; + + low = mid; + while (low) { + seq = kallsyms_seqs_of_names[low - 1]; + off = get_symbol_offset(seq); + kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); + if (compare_symbol_name(name, namebuf)) + break; + low--; + } + *start = low; + + if (end) { + high = mid; + while (high < kallsyms_num_syms - 1) { + seq = kallsyms_seqs_of_names[high + 1]; + off = get_symbol_offset(seq); + kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); + if (compare_symbol_name(name, namebuf)) + break; + high++; + } + *end = high; + } + + return 0; +} + /* Lookup the address for this symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name(const char *name) { - char namebuf[KSYM_NAME_LEN]; - unsigned long i; - unsigned int off; + int ret; + unsigned int i; /* Skip the search for empty string. */ if (!*name) return 0; - for (i = 0, off = 0; i < kallsyms_num_syms; i++) { - off = kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); - - if (strcmp(namebuf, name) == 0) - return kallsyms_sym_address(i); + ret = kallsyms_lookup_names(name, &i, NULL); + if (!ret) + return kallsyms_sym_address(kallsyms_seqs_of_names[i]); - if (cleanup_symbol_name(namebuf) && strcmp(namebuf, name) == 0) - return kallsyms_sym_address(i); - } return module_kallsyms_lookup_name(name); } diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h index 2d0c6f2f0243a28..a04b7a5cb1e3eaf 100644 --- a/kernel/kallsyms_internal.h +++ b/kernel/kallsyms_internal.h @@ -26,5 +26,6 @@ extern const char kallsyms_token_table[] __weak; extern const u16 kallsyms_token_index[] __weak; extern const unsigned int kallsyms_markers[] __weak; +extern const unsigned int kallsyms_seqs_of_names[] __weak; #endif // LINUX_KALLSYMS_INTERNAL_H_ diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index ab105bdde4efe4f..df2d93fb0e8d095 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -49,6 +49,7 @@ _Static_assert( struct sym_entry { unsigned long long addr; unsigned int len; + unsigned int seq; unsigned int start_pos; unsigned int percpu_absolute; unsigned char sym[]; @@ -410,6 +411,35 @@ static int symbol_absolute(const struct sym_entry *s) return s->percpu_absolute; } +static int compare_names(const void *a, const void *b) +{ + int ret; + char sa_namebuf[KSYM_NAME_LEN]; + char sb_namebuf[KSYM_NAME_LEN]; + const struct sym_entry *sa = *(const struct sym_entry **)a; + const struct sym_entry *sb = *(const struct sym_entry **)b; + + expand_symbol(sa->sym, sa->len, sa_namebuf); + expand_symbol(sb->sym, sb->len, sb_namebuf); + ret = strcmp(&sa_namebuf[1], &sb_namebuf[1]); + if (!ret) { + if (sa->addr > sb->addr) + return 1; + else if (sa->addr < sb->addr) + return -1; + + /* keep old order */ + return (int)(sa->seq - sb->seq); + } + + return ret; +} + +static void sort_symbols_by_name(void) +{ + qsort(table, table_cnt, sizeof(table[0]), compare_names); +} + static void write_src(void) { unsigned int i, k, off; @@ -495,6 +525,7 @@ static void write_src(void) for (i = 0; i < table_cnt; i++) { if ((i & 0xFF) == 0) markers[i >> 8] = off; + table[i]->seq = i; /* There cannot be any symbol of length zero. */ if (table[i]->len == 0) { @@ -535,6 +566,12 @@ static void write_src(void) free(markers); + sort_symbols_by_name(); + output_label("kallsyms_seqs_of_names"); + for (i = 0; i < table_cnt; i++) + printf("\t.long\t%u\n", table[i]->seq); + printf("\n"); + output_label("kallsyms_token_table"); off = 0; for (i = 0; i < 256; i++) { From patchwork Wed Nov 2 08:49:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14096 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494435wru; Wed, 2 Nov 2022 01:54:29 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7pk0Ti8ak8l0DRY/rHx/hPTOuyl4MVcWtkU1koCIP01i/MlVIJLr3DnCe1+KyGzRxzEciZ X-Received: by 2002:aa7:de9a:0:b0:44d:8191:44c5 with SMTP id j26-20020aa7de9a000000b0044d819144c5mr22757824edv.232.1667379268807; Wed, 02 Nov 2022 01:54:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379268; cv=none; d=google.com; s=arc-20160816; b=xwA2vHd5s+fveTuF35J6ldfjqjDlUlHe374oSFQoP3pLMrJEbN6rDdA81n1c6M8ZGg 4rSL6ae17Rx5Od0xxVtHgYt0qg31u4n3PedAFVa2nnnWgXN/dTKnwy0PRY3aXqPqvxsG ZfVnKRd3Q8aBCGfdonzQLUmHWw/xunnZkU691uc9apv6W98Xduu9Wi4mkBl6hkM36sDg 8k5bSNQCvbyBInqmn9EiGDqcM3sasM/RoHHooHsAxCeSLj2RABVgSs5c8+ssY7SIhszP 9fcKXcode5sfhiRnlGycYYlxpJVArmspgCrkZUBUj163BDjU/wyyQ2AOvmhfJhB+XwFO AYxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Z7T1fx2hFsO667l2SwtXwgdAh7qWMMGem+Sz/mXUkeY=; b=BL9zNMTPGoKqwotZMe7ItgWDB438w5Pq2g+E09mG1fHvu8mf0MDsIy1fx2RnoeLCmJ v13ltiGkrO4yJ8ieJqYhxvOJsiMI038BpttVQNDVAbbczi8JIG34ojB445mBoNdTD+Co PLmSSA6I+4uIViNHCgsFQN3BeZMO/5AdWBmaeTCpvMf774X2eKKMkTUBQ+Ns63GdS1No PfkRtcsAA0oLXYcOmVs6oG8kbBhFt3DV2lxZ4W8x4T4BhqW720gHGvZvAVwmG2YVdUXW Ah/RchIDQTT1muFAiDEX3F47I5rMgVKVuCk1QsQoaGzyyqvSmgOmL5IOnmL/U+Cs/eSV wCdQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hd43-20020a17090796ab00b007a4feae7adesi14070217ejc.837.2022.11.02.01.54.04; Wed, 02 Nov 2022 01:54:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231167AbiKBIun (ORCPT + 99 others); Wed, 2 Nov 2022 04:50:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230308AbiKBIub (ORCPT ); Wed, 2 Nov 2022 04:50:31 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AB8B25289; Wed, 2 Nov 2022 01:50:29 -0700 (PDT) Received: from dggpemm500024.china.huawei.com (unknown [172.30.72.55]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4N2L9y05V0zJnKl; Wed, 2 Nov 2022 16:47:34 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500024.china.huawei.com (7.185.36.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:27 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:27 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 3/9] kallsyms: Correctly sequence symbols when CONFIG_LTO_CLANG=y Date: Wed, 2 Nov 2022 16:49:15 +0800 Message-ID: <20221102084921.1615-4-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373884221266456?= X-GMAIL-MSGID: =?utf-8?q?1748373884221266456?= LLVM appends various suffixes for local functions and variables, suffixes observed: - foo.llvm.[0-9a-f]+ - foo.[0-9a-f]+ Therefore, when CONFIG_LTO_CLANG=y, kallsyms_lookup_name() needs to truncate the suffix of the symbol name before comparing the local function or variable name. Old implementation code: - if (strcmp(namebuf, name) == 0) - return kallsyms_sym_address(i); - if (cleanup_symbol_name(namebuf) && strcmp(namebuf, name) == 0) - return kallsyms_sym_address(i); The preceding process is traversed by address from low to high. That is, for those with the same name after the suffix is removed, the one with the smallest address is returned first. Therefore, when sorting in the tool, if the raw names are the same, they should be sorted by address in ascending order. ASCII[.] = 2e ASCII[0-9] = 30,39 ASCII[A-Z] = 41,5a ASCII[_] = 5f ASCII[a-z] = 61,7a According to the preceding ASCII code values, the following sorting result is strictly followed. --------------------------------- | main-key | sub-key | |---------------------------------| | | addr_lowest | | | ... | | . | ... | | | addr_highest | |---------------------------------| | ? | | //? is [_A-Za-z0-9] --------------------------------- Signed-off-by: Zhen Lei --- scripts/kallsyms.c | 36 ++++++++++++++++++++++++++++++++++-- scripts/link-vmlinux.sh | 4 ++++ 2 files changed, 38 insertions(+), 2 deletions(-) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index df2d93fb0e8d095..07ecf7e5c49f616 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -78,6 +78,7 @@ static unsigned int table_size, table_cnt; static int all_symbols; static int absolute_percpu; static int base_relative; +static int lto_clang; static int token_profit[0x10000]; @@ -89,7 +90,7 @@ static unsigned char best_table_len[256]; static void usage(void) { fprintf(stderr, "Usage: kallsyms [--all-symbols] [--absolute-percpu] " - "[--base-relative] in.map > out.S\n"); + "[--base-relative] [--lto-clang] in.map > out.S\n"); exit(1); } @@ -411,6 +412,34 @@ static int symbol_absolute(const struct sym_entry *s) return s->percpu_absolute; } +static char * s_name(char *buf) +{ + /* Skip the symbol type */ + return buf + 1; +} + +static void cleanup_symbol_name(char *s) +{ + char *p; + + if (!lto_clang) + return; + + /* + * ASCII[.] = 2e + * ASCII[0-9] = 30,39 + * ASCII[A-Z] = 41,5a + * ASCII[_] = 5f + * ASCII[a-z] = 61,7a + * + * As above, replacing '.' with '\0' does not affect the main sorting, + * but it helps us with subsorting. + */ + p = strchr(s, '.'); + if (p) + *p = '\0'; +} + static int compare_names(const void *a, const void *b) { int ret; @@ -421,7 +450,9 @@ static int compare_names(const void *a, const void *b) expand_symbol(sa->sym, sa->len, sa_namebuf); expand_symbol(sb->sym, sb->len, sb_namebuf); - ret = strcmp(&sa_namebuf[1], &sb_namebuf[1]); + cleanup_symbol_name(s_name(sa_namebuf)); + cleanup_symbol_name(s_name(sb_namebuf)); + ret = strcmp(s_name(sa_namebuf), s_name(sb_namebuf)); if (!ret) { if (sa->addr > sb->addr) return 1; @@ -855,6 +886,7 @@ int main(int argc, char **argv) {"all-symbols", no_argument, &all_symbols, 1}, {"absolute-percpu", no_argument, &absolute_percpu, 1}, {"base-relative", no_argument, &base_relative, 1}, + {"lto-clang", no_argument, <o_clang, 1}, {}, }; diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 918470d768e9c7d..32e573943cf036b 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -156,6 +156,10 @@ kallsyms() kallsymopt="${kallsymopt} --base-relative" fi + if is_enabled CONFIG_LTO_CLANG; then + kallsymopt="${kallsymopt} --lto-clang" + fi + info KSYMS ${2} scripts/kallsyms ${kallsymopt} ${1} > ${2} } From patchwork Wed Nov 2 08:49:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14091 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494224wru; Wed, 2 Nov 2022 01:53:51 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5KyK/QH0nYtFU6bYndckg4Qy/vhkrOTyHAMD1ClPBu4ClaCFDVP2qLHFGByjncKF2BUmhN X-Received: by 2002:a50:ccc2:0:b0:460:4db2:5006 with SMTP id b2-20020a50ccc2000000b004604db25006mr23422781edj.369.1667379231128; Wed, 02 Nov 2022 01:53:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379231; cv=none; d=google.com; s=arc-20160816; b=qfL4v9gb+TjbM5oov0aLWh5iWNLDLfcy6YEWr3rGXUnFlU+gCNpn1PA0bbUAcWQKqb xLjgJ+x5QEDQ+F+h6+agT4v1485cEVVyWKBDSGEQ/BOavY61Nc/ugNACwsD44ua9hnc5 uVdhwXJLRTzIF8DbrbGfkrtVVcijfZaQxZcxvB3927m2Gqj4qbsJ27oGRO5W8iy33TWU W2bWyPEfcW8vP+JFUUIDUJ4TclIc2WjaNnzOy5e+3JfHijPrdA7zpuHgQ39WFMqP++BG gepoBai/YJ8m+0kzI4iZgFto5wnKyCzBXftCVXiA7BZP7z8kY7+KYV15SedEjTmDRvUg s7qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=6el2x93t/bjgtIpCiYZoOiK6Y/TNd6UspzQiI1Dw2qU=; b=XZKj0EMOiFutJDXhDfsrmSHDzYGPtfK1GD0XkJXTvwPaillUrjCKnW/xb/v33heIgr TrhO0tfCVowL/6+gjH6fg2WnK+jeCuoMR/1hieZas68mxfpzxKZdUMaRuovbfrfh6O57 xFTjfBSXux/k0TEduwuXf/KEAEgS6NWZ3NutawJJpP/rwPhuVRzOx4b7bmhEm0evqNiw 8x5hU4jtRBstHJ2OBGXuxZ+XASIqv91fW30BzRSNHeUhukW6sJad4aHIR6cYsJ/lfVxA 8m7uZ95GevZFtr0DVx3oR0xx2ltUx422bTi2q/OdxUaLJhTYLq3Dw12S79upB+91MJRE Lv+Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g12-20020a056402090c00b00453a0393deasi17543570edz.368.2022.11.02.01.53.27; Wed, 02 Nov 2022 01:53:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231254AbiKBIu7 (ORCPT + 99 others); Wed, 2 Nov 2022 04:50:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230424AbiKBIud (ORCPT ); Wed, 2 Nov 2022 04:50:33 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5150C27CFD; Wed, 2 Nov 2022 01:50:31 -0700 (PDT) Received: from dggpemm500020.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4N2LFG1g3fz15MF2; Wed, 2 Nov 2022 16:50:26 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500020.china.huawei.com (7.185.36.49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:28 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:28 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 4/9] kallsyms: Reduce the memory occupied by kallsyms_seqs_of_names[] Date: Wed, 2 Nov 2022 16:49:16 +0800 Message-ID: <20221102084921.1615-5-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373844312904523?= X-GMAIL-MSGID: =?utf-8?q?1748373844312904523?= kallsyms_seqs_of_names[] records the symbol index sorted by address, the maximum value in kallsyms_seqs_of_names[] is the number of symbols. And 2^24 = 16777216, which means that three bytes are enough to store the index. This can help us save (1 * kallsyms_num_syms) bytes of memory. Signed-off-by: Zhen Lei --- kernel/kallsyms.c | 18 ++++++++++++++---- kernel/kallsyms_internal.h | 2 +- scripts/kallsyms.c | 5 ++++- 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index ba351dfa109b6ac..48f36fd7e10b95e 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -201,6 +201,16 @@ static int compare_symbol_name(const char *name, char *namebuf) return ret; } +static unsigned int get_symbol_seq(int index) +{ + unsigned int i, seq = 0; + + for (i = 0; i < 3; i++) + seq = (seq << 8) | kallsyms_seqs_of_names[3 * index + i]; + + return seq; +} + static int kallsyms_lookup_names(const char *name, unsigned int *start, unsigned int *end) @@ -215,7 +225,7 @@ static int kallsyms_lookup_names(const char *name, while (low <= high) { mid = low + (high - low) / 2; - seq = kallsyms_seqs_of_names[mid]; + seq = get_symbol_seq(mid); off = get_symbol_offset(seq); kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); ret = compare_symbol_name(name, namebuf); @@ -232,7 +242,7 @@ static int kallsyms_lookup_names(const char *name, low = mid; while (low) { - seq = kallsyms_seqs_of_names[low - 1]; + seq = get_symbol_seq(low - 1); off = get_symbol_offset(seq); kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); if (compare_symbol_name(name, namebuf)) @@ -244,7 +254,7 @@ static int kallsyms_lookup_names(const char *name, if (end) { high = mid; while (high < kallsyms_num_syms - 1) { - seq = kallsyms_seqs_of_names[high + 1]; + seq = get_symbol_seq(high + 1); off = get_symbol_offset(seq); kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); if (compare_symbol_name(name, namebuf)) @@ -269,7 +279,7 @@ unsigned long kallsyms_lookup_name(const char *name) ret = kallsyms_lookup_names(name, &i, NULL); if (!ret) - return kallsyms_sym_address(kallsyms_seqs_of_names[i]); + return kallsyms_sym_address(get_symbol_seq(i)); return module_kallsyms_lookup_name(name); } diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h index a04b7a5cb1e3eaf..27fabdcc40f5793 100644 --- a/kernel/kallsyms_internal.h +++ b/kernel/kallsyms_internal.h @@ -26,6 +26,6 @@ extern const char kallsyms_token_table[] __weak; extern const u16 kallsyms_token_index[] __weak; extern const unsigned int kallsyms_markers[] __weak; -extern const unsigned int kallsyms_seqs_of_names[] __weak; +extern const u8 kallsyms_seqs_of_names[] __weak; #endif // LINUX_KALLSYMS_INTERNAL_H_ diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 07ecf7e5c49f616..04e04fbd9625caf 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -600,7 +600,10 @@ static void write_src(void) sort_symbols_by_name(); output_label("kallsyms_seqs_of_names"); for (i = 0; i < table_cnt; i++) - printf("\t.long\t%u\n", table[i]->seq); + printf("\t.byte 0x%02x, 0x%02x, 0x%02x\n", + (unsigned char)(table[i]->seq >> 16), + (unsigned char)(table[i]->seq >> 8), + (unsigned char)(table[i]->seq >> 0)); printf("\n"); output_label("kallsyms_token_table"); From patchwork Wed Nov 2 08:49:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14090 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494183wru; Wed, 2 Nov 2022 01:53:43 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4inT+MHAfMXiJzl9FpXwXRE78jqp9B/3XD1xTYpmZmUqDibFB1+gh8U190j7NmvYm5enLb X-Received: by 2002:a05:6402:5:b0:463:39ab:c1da with SMTP id d5-20020a056402000500b0046339abc1damr17419759edu.166.1667379223114; Wed, 02 Nov 2022 01:53:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379223; cv=none; d=google.com; s=arc-20160816; b=OV4Ts4Osp7pAJOJXOmhwbxGmeq2ke+54EOeUh4wG61sDPF4RXgxA0IemOCPaYmuQ8e UNPgof4GcEQdUNUrrZ2WxYHGsZ5wRiHzoFtOGDwbXbjyZko5BofH6CchxXB4Ht59/CxM 0Y6mfYHsPDZigLUfSPzGxdK3irwI44Y/PX+aoGgbqlrDYZFeIm/UrkgbBjShuzq3GHzU V4FI8WOvp8kame4324WbQVIQlweGLFOYrpI6UYr4iZtF6SCK+KgUt1uZDk5x2sAxa7Yi Uv72szL93dK59/jidwLCaZijr1kov44nnDelMn3c7Q9XeWxhPWNFKvqohr/TS/cs1b6Q Nopw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=gMcInFbUdT+ScSFrMiiii85OGS705dGZ5ymAGPc0ZqQ=; b=kTA7aLK3JOPXRcAbiFTxpABeDqNtg0buvct4crFCeopXF3+aKv/WuPJIMq5gQ3cKmh LXVUE/iH9A1WYHzhScuEu/abKuqE/tWqTe097K98/IQwD1/8Bnu41rjIR92r+R/6Rpwu m4JvbACMT7KqzVKXtQzMzS3dNhTbRsmmrRhmP7B7ctsix24WuoyQf2HdSiU669cXOwpf nWsgSQIQA9FFn3itD5ipO5IBOfvvMKw+aMQ2jb2dXpWdTErx4bpV/00NXo+WkgKXHMpK 3zH2tPe33NGeM4JpX2OL9ladyiiE5tlkgRJigy4CTGhH3tENyWbKq0eWf0ap9NfLnp9t X1LQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b10-20020a056402084a00b00462273b6b75si18299453edz.361.2022.11.02.01.53.19; Wed, 02 Nov 2022 01:53:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230492AbiKBIux (ORCPT + 99 others); Wed, 2 Nov 2022 04:50:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230503AbiKBIud (ORCPT ); Wed, 2 Nov 2022 04:50:33 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2306927DCD; Wed, 2 Nov 2022 01:50:32 -0700 (PDT) Received: from dggpemm500022.china.huawei.com (unknown [172.30.72.54]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4N2LB041PXzJnKw; Wed, 2 Nov 2022 16:47:36 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500022.china.huawei.com (7.185.36.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:30 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:29 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 5/9] kallsyms: Add helper kallsyms_on_each_match_symbol() Date: Wed, 2 Nov 2022 16:49:17 +0800 Message-ID: <20221102084921.1615-6-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373836499378936?= X-GMAIL-MSGID: =?utf-8?q?1748373836499378936?= Function kallsyms_on_each_symbol() traverses all symbols and submits each symbol to the hook 'fn' for judgment and processing. For some cases, the hook actually only handles the matched symbol, such as livepatch. Because all symbols are currently sorted by name, all the symbols with the same name are clustered together. Function kallsyms_lookup_names() gets the start and end positions of the set corresponding to the specified name. So we can easily and quickly traverse all the matches. The test results are as follows (twice): (x86) kallsyms_on_each_match_symbol: 7454, 7984 kallsyms_on_each_symbol : 11733809, 11785803 kallsyms_on_each_match_symbol() consumes only 0.066% of kallsyms_on_each_symbol()'s time. In other words, 1523x better performance. Signed-off-by: Zhen Lei --- include/linux/kallsyms.h | 8 ++++++++ kernel/kallsyms.c | 18 ++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index 649faac31ddb162..0cd33be7142ad0d 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -69,6 +69,8 @@ static inline void *dereference_symbol_descriptor(void *ptr) int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, unsigned long), void *data); +int kallsyms_on_each_match_symbol(int (*fn)(void *, unsigned long), + const char *name, void *data); /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name(const char *name); @@ -168,6 +170,12 @@ static inline int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct { return -EOPNOTSUPP; } + +static inline int kallsyms_on_each_match_symbol(int (*fn)(void *, unsigned long), + const char *name, void *data) +{ + return -EOPNOTSUPP; +} #endif /*CONFIG_KALLSYMS*/ static inline void print_ip_sym(const char *loglvl, unsigned long ip) diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 48f36fd7e10b95e..0008ada2b135bef 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -307,6 +307,24 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, return 0; } +int kallsyms_on_each_match_symbol(int (*fn)(void *, unsigned long), + const char *name, void *data) +{ + int ret; + unsigned int i, start, end; + + ret = kallsyms_lookup_names(name, &start, &end); + if (ret) + return 0; + + for (i = start; !ret && i <= end; i++) { + ret = fn(data, kallsyms_sym_address(get_symbol_seq(i))); + cond_resched(); + } + + return ret; +} + static unsigned long get_symbol_pos(unsigned long addr, unsigned long *symbolsize, unsigned long *offset) From patchwork Wed Nov 2 08:49:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14097 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494449wru; Wed, 2 Nov 2022 01:54:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5lJScjYGLN4Sg4fZ3lI0TwM5XsFoyZgCNMO4VdzDsFlm+myVGOngi3ULVnQGSetjwdw6JC X-Received: by 2002:a17:903:4d7:b0:178:8564:f754 with SMTP id jm23-20020a17090304d700b001788564f754mr23795299plb.60.1667379271096; Wed, 02 Nov 2022 01:54:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379271; cv=none; d=google.com; s=arc-20160816; b=Tov9gE+po5EE2SZ9HEScxHJLJj5rxdwxAIFC15Tvv23EnU9DnflVjFhetZqNy0+lnS IKKM2SPKmtFsH5ATEBNFh5hrH2ITe1CgSYcR2ltbfq1tUKmB4XJcCDDls0zuJ9RMxg64 Uza3V++IJGUIB1Y0tcrB6ziJdI+ORdUIuLkUvQF7D6mwza7q1ID8LWGuongx00PX98Ih CkD/9COf4m0Zc2gSqqc2QxpMCBShSjJ34D9ISOr9LbhDNE7og6IN9HMrzN1QBaG3dk3n TZ1ftIp3/10Jk3h5p1pSxeHzcLps0egQLqesn/6yQJYaw7t9/00m7MMG7sNaNZRHu28b d9+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=O4EM+Z9u9BAqy+uGwOyXQNwfT3jl2sLXypQlQV1N480=; b=IGx1U5PfYXDdho+55jR6fpww8ULjEcGLfjf9Bo9BUeTOW/i2zMu/aaYFRVO+oRIf05 XQ4gOmDUMxrFfyDHze2/cDkt1bXNwmDde1C/iMP1k0/Ow+95QbN2JH4Q0xGy1gD7ceK0 QUCLhQUnOityQ46lKOyPyCb32v8b/STNKhvEcCGZ7iYybRqa+9bbSaqdvz1CY/XRP5IF AXl2NOnqdWJyJTkEHO9B7kVvUVJvqSJKevy+iL5lZcmoBfk0xDUExioJ58k3BpIvtKFr axDQhliMMbuzvMHZ8hoGaoJZouEU3I3xOTEvNzg06W+1vkZAj6RJ+pGHxgNhNitMHa9f N+xA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s1-20020a170903200100b00172cb948c68si13054262pla.227.2022.11.02.01.54.18; Wed, 02 Nov 2022 01:54:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231244AbiKBIvc (ORCPT + 99 others); Wed, 2 Nov 2022 04:51:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231204AbiKBIu6 (ORCPT ); Wed, 2 Nov 2022 04:50:58 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEAE127DDF; Wed, 2 Nov 2022 01:50:55 -0700 (PDT) Received: from dggpemm500023.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4N2LFR329DzHvXx; Wed, 2 Nov 2022 16:50:35 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500023.china.huawei.com (7.185.36.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:31 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:30 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 6/9] livepatch: Use kallsyms_on_each_match_symbol() to improve performance Date: Wed, 2 Nov 2022 16:49:18 +0800 Message-ID: <20221102084921.1615-7-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373886459417505?= X-GMAIL-MSGID: =?utf-8?q?1748373886459417505?= Based on the test results of kallsyms_on_each_match_symbol() and kallsyms_on_each_symbol(), the average performance can be improved by more than 1500 times. Signed-off-by: Zhen Lei --- kernel/livepatch/core.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c index 9ada0bc5247be5d..50bfc3481a4ee38 100644 --- a/kernel/livepatch/core.c +++ b/kernel/livepatch/core.c @@ -153,6 +153,24 @@ static int klp_find_callback(void *data, const char *name, return 0; } +static int klp_match_callback(void *data, unsigned long addr) +{ + struct klp_find_arg *args = data; + + args->addr = addr; + args->count++; + + /* + * Finish the search when the symbol is found for the desired position + * or the position is not defined for a non-unique symbol. + */ + if ((args->pos && (args->count == args->pos)) || + (!args->pos && (args->count > 1))) + return 1; + + return 0; +} + static int klp_find_object_symbol(const char *objname, const char *name, unsigned long sympos, unsigned long *addr) { @@ -167,7 +185,7 @@ static int klp_find_object_symbol(const char *objname, const char *name, if (objname) module_kallsyms_on_each_symbol(klp_find_callback, &args); else - kallsyms_on_each_symbol(klp_find_callback, &args); + kallsyms_on_each_match_symbol(klp_match_callback, name, &args); /* * Ensure an address was found. If sympos is 0, ensure symbol is unique; From patchwork Wed Nov 2 08:49:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14092 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494258wru; Wed, 2 Nov 2022 01:53:57 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5fR2HYbkdpi4geF+TJOsmciYB7ylizQtRYaPYF6YSR01jY5ld6Ji2XcszMuI83rEfQ7NJi X-Received: by 2002:a17:907:60d4:b0:79e:8082:132d with SMTP id hv20-20020a17090760d400b0079e8082132dmr22906614ejc.9.1667379237391; Wed, 02 Nov 2022 01:53:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379237; cv=none; d=google.com; s=arc-20160816; b=WKLTaFpvMpI4PrhGCPFrzamfaMomWSxo9Wp0Dp30t6qUhww4U3W4MonmWvrcfmO5eS R4HFWFYnCEgR43jOzCo8xrxTgMje1RtWeeN7p/iKwmvTZVJy6tGo6IGQClf/z+qo8Xb8 0WP9jUe0E99ynt3BTT2qcC8G9RXEdBszVQ/NeoH4VDaUEzH3tVis1USom//RZrTArQpj OVtEuhmdmKtJaHh2ht9dXeGME17vJWzNd/7EukqMjPn3iiByeixo9ZafagFsR1zE1gBf UXr/7xG0qSivEA7ilvpg3wFDVAg8/i2KqWVAZO/WjzjOkDzQ7RbQ3HELLcqZhEJhp7Qe Pwpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=x+qI4LJ5JrJ3M7Q1W5gysaPj0SLVspAorDwDzXWWRbQ=; b=JNepflRrgTy5kN608htkELoEih8sHZwUnBkeIZx0yfBLSgpH+yD2njxzDmNOdCcTno +wGUrsnw0NqfHbnvx2o5j7KFB101bTdfRSsh1IZkZiy78BDgiXBsENpsXQV7sAKpYNxS 9tgZB4cqMy3qAXolpNr30Ng1xIDs1Z7PkkOdf1ntoMCv77SAJihJooi19C2L46rKod12 GBcUS5WOl+Pv5KdVhLrLTqFNZ8jkkjFwQJNJSAI9yQOYIYYdfojrzQe/YWerHDBMGfSg 3zuGLdOkQx1PhnYNx7oUhs9KoiisL/823DIWBrULwCuwKVGXjHTASWD7UDOi7CB/JfLT WEPg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v17-20020a509551000000b00461a8c8f155si13642251eda.161.2022.11.02.01.53.33; Wed, 02 Nov 2022 01:53:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230370AbiKBIvE (ORCPT + 99 others); Wed, 2 Nov 2022 04:51:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230056AbiKBIuu (ORCPT ); Wed, 2 Nov 2022 04:50:50 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79B7827CFA; Wed, 2 Nov 2022 01:50:34 -0700 (PDT) Received: from dggpemm500024.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4N2LFK2YjJz15MC9; Wed, 2 Nov 2022 16:50:29 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500024.china.huawei.com (7.185.36.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:32 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:31 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 7/9] livepatch: Improve the search performance of module_kallsyms_on_each_symbol() Date: Wed, 2 Nov 2022 16:49:19 +0800 Message-ID: <20221102084921.1615-8-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_FILL_THIS_FORM_SHORT autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373851414178149?= X-GMAIL-MSGID: =?utf-8?q?1748373851414178149?= Currently we traverse all symbols of all modules to find the specified function for the specified module. But in reality, we just need to find the given module and then traverse all the symbols in it. Let's add a new parameter 'const char *modname' to function module_kallsyms_on_each_symbol(), then we can compare the module names directly in this function and call hook 'fn' after matching. And the parameter 'struct module *' in the hook 'fn' can also be deleted. Phase1: mod1-->mod2..(subsequent modules do not need to be compared) | Phase2: -->f1-->f2-->f3 Signed-off-by: Zhen Lei --- include/linux/module.h | 4 ++-- kernel/livepatch/core.c | 13 ++----------- kernel/module/kallsyms.c | 15 ++++++++++++--- 3 files changed, 16 insertions(+), 16 deletions(-) diff --git a/include/linux/module.h b/include/linux/module.h index ec61fb53979a92a..0a3b44ff885a48c 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -879,8 +879,8 @@ static inline bool module_sig_ok(struct module *module) } #endif /* CONFIG_MODULE_SIG */ -int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, - struct module *, unsigned long), +int module_kallsyms_on_each_symbol(const char *modname, + int (*fn)(void *, const char *, unsigned long), void *data); #endif /* _LINUX_MODULE_H */ diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c index 50bfc3481a4ee38..d4fe2d1b0e562bc 100644 --- a/kernel/livepatch/core.c +++ b/kernel/livepatch/core.c @@ -118,27 +118,19 @@ static struct klp_object *klp_find_object(struct klp_patch *patch, } struct klp_find_arg { - const char *objname; const char *name; unsigned long addr; unsigned long count; unsigned long pos; }; -static int klp_find_callback(void *data, const char *name, - struct module *mod, unsigned long addr) +static int klp_find_callback(void *data, const char *name, unsigned long addr) { struct klp_find_arg *args = data; - if ((mod && !args->objname) || (!mod && args->objname)) - return 0; - if (strcmp(args->name, name)) return 0; - if (args->objname && strcmp(args->objname, mod->name)) - return 0; - args->addr = addr; args->count++; @@ -175,7 +167,6 @@ static int klp_find_object_symbol(const char *objname, const char *name, unsigned long sympos, unsigned long *addr) { struct klp_find_arg args = { - .objname = objname, .name = name, .addr = 0, .count = 0, @@ -183,7 +174,7 @@ static int klp_find_object_symbol(const char *objname, const char *name, }; if (objname) - module_kallsyms_on_each_symbol(klp_find_callback, &args); + module_kallsyms_on_each_symbol(objname, klp_find_callback, &args); else kallsyms_on_each_match_symbol(klp_match_callback, name, &args); diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c index f5c5c9175333df7..329cef573675d49 100644 --- a/kernel/module/kallsyms.c +++ b/kernel/module/kallsyms.c @@ -495,8 +495,8 @@ unsigned long module_kallsyms_lookup_name(const char *name) } #ifdef CONFIG_LIVEPATCH -int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, - struct module *, unsigned long), +int module_kallsyms_on_each_symbol(const char *modname, + int (*fn)(void *, const char *, unsigned long), void *data) { struct module *mod; @@ -510,6 +510,9 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, if (mod->state == MODULE_STATE_UNFORMED) continue; + if (strcmp(modname, mod->name)) + continue; + /* Use rcu_dereference_sched() to remain compliant with the sparse tool */ preempt_disable(); kallsyms = rcu_dereference_sched(mod->kallsyms); @@ -522,10 +525,16 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, continue; ret = fn(data, kallsyms_symbol_name(kallsyms, i), - mod, kallsyms_symbol_value(sym)); + kallsyms_symbol_value(sym)); if (ret != 0) goto out; } + + /* + * The given module is found, the subsequent modules do not + * need to be compared. + */ + break; } out: mutex_unlock(&module_mutex); From patchwork Wed Nov 2 08:49:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14094 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494289wru; Wed, 2 Nov 2022 01:54:01 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7vM3qgTID8MOZo7C/OfJV2zmZqRIO2sdG0qib8qKNRZYatuNcNWfHAD66pMd4SnYDcBjG3 X-Received: by 2002:a17:907:3f04:b0:741:4bf4:fe42 with SMTP id hq4-20020a1709073f0400b007414bf4fe42mr23856212ejc.664.1667379241776; Wed, 02 Nov 2022 01:54:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379241; cv=none; d=google.com; s=arc-20160816; b=c60gqyxN+8OXgMR9dIgz8EBl4HMn1Uyub4aynL8lilnR5xM9qmrx3Y/EnFTwaShQQW p/+nyssfHcgPSkIHCDHEibc2fodqGUhdpI0mereHalIiNNqlR6nRZY7G3dMZtdm6Ixob YPbChnzW4siBTIk2KaGjnJLuhACh1Pwz50Ka5S6wqlQnG5A3pRCn/cGzsGvIkuVM0TPL j9xzC0ymBO4U0kGmuBu3ooytas6cPxWbehI5Esye2y/w0xlVxi0+4SxJTYXuKecnAj6C i+lE0AVNPGQUkZaoEmuydjPmfUXp0VheruGMZEym+y+zB1jTsKfT/Br4WCF10eyRGPi2 nrrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=bS3hOnp1RwEadoRQ+/ijEap1N31XOvFGeSCgtywSrqs=; b=FO5+24nk/RrgMoPpclpI/doXCJMK223JIfW9aPo+8FTew87yn8y7LGM50nXmZSxck2 AMfoKdhsv7k6PiyOHR5FQE6G9Xmhek4+jtiM8yCVFxi2ohV3FeS3rhJg5CRpbNxEF6KQ BdLAsd7XnxjBfAo6inZ4gtWHw/is8BE/LNmHH+tkOmC/2ID+ptfh4f8pCsBMVu/AeSD7 jsCAAlw2zSFnIjJRXsjzCXiegFhZ+EW1UcicRPykri6eHcYfQUzNVW/EvtZlqEOye+Q0 cHi3aXMyvVtYQg0UyVCgo7YMeCgWsgSIDJiHhdVGYVDU0bltsh/t9vl4umxh5jpsGdiH wPfA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gs15-20020a1709072d0f00b0078e1246e7d6si16132101ejc.748.2022.11.02.01.53.37; Wed, 02 Nov 2022 01:54:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230375AbiKBIvQ (ORCPT + 99 others); Wed, 2 Nov 2022 04:51:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230300AbiKBIuu (ORCPT ); Wed, 2 Nov 2022 04:50:50 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B97527DE3; Wed, 2 Nov 2022 01:50:35 -0700 (PDT) Received: from dggpemm500021.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4N2LFK4TxVzmV7K; Wed, 2 Nov 2022 16:50:29 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500021.china.huawei.com (7.185.36.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:33 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:32 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 8/9] kallsyms: Delete an unused parameter related to kallsyms_on_each_symbol() Date: Wed, 2 Nov 2022 16:49:20 +0800 Message-ID: <20221102084921.1615-9-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373855558182005?= X-GMAIL-MSGID: =?utf-8?q?1748373855558182005?= The parameter 'struct module *' in the hook function associated with kallsyms_on_each_symbol() is no longer used. Delete it. Suggested-by: Petr Mladek Signed-off-by: Zhen Lei --- include/linux/kallsyms.h | 3 +-- kernel/kallsyms.c | 5 ++--- kernel/trace/ftrace.c | 3 +-- 3 files changed, 4 insertions(+), 7 deletions(-) diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index 0cd33be7142ad0d..5002ebe9dff5a0e 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -66,8 +66,7 @@ static inline void *dereference_symbol_descriptor(void *ptr) } #ifdef CONFIG_KALLSYMS -int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, - unsigned long), +int kallsyms_on_each_symbol(int (*fn)(void *, const char *, unsigned long), void *data); int kallsyms_on_each_match_symbol(int (*fn)(void *, unsigned long), const char *name, void *data); diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 0008ada2b135bef..5110d5501edeeb3 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -288,8 +288,7 @@ unsigned long kallsyms_lookup_name(const char *name) * Iterate over all symbols in vmlinux. For symbols from modules use * module_kallsyms_on_each_symbol instead. */ -int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, - unsigned long), +int kallsyms_on_each_symbol(int (*fn)(void *, const char *, unsigned long), void *data) { char namebuf[KSYM_NAME_LEN]; @@ -299,7 +298,7 @@ int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, for (i = 0, off = 0; i < kallsyms_num_syms; i++) { off = kallsyms_expand_symbol(off, namebuf, ARRAY_SIZE(namebuf)); - ret = fn(data, namebuf, NULL, kallsyms_sym_address(i)); + ret = fn(data, namebuf, kallsyms_sym_address(i)); if (ret != 0) return ret; cond_resched(); diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index fbf2543111c05c2..e3ef4f0defb2e37 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -8267,8 +8267,7 @@ struct kallsyms_data { size_t found; }; -static int kallsyms_callback(void *data, const char *name, - struct module *mod, unsigned long addr) +static int kallsyms_callback(void *data, const char *name, unsigned long addr) { struct kallsyms_data *args = data; const char **sym; From patchwork Wed Nov 2 08:49:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhen Lei X-Patchwork-Id: 14093 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp3494269wru; Wed, 2 Nov 2022 01:53:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4BiXXVIBetXxjkx+vQnObRMVBei1O7jccUkRGIHc+g5O2eI3yRfafY8aKRiKmwnN80pC+U X-Received: by 2002:a17:907:7faa:b0:7ad:e8e4:6ac7 with SMTP id qk42-20020a1709077faa00b007ade8e46ac7mr9363639ejc.21.1667379238895; Wed, 02 Nov 2022 01:53:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667379238; cv=none; d=google.com; s=arc-20160816; b=E9FQPb5GQHoMILlFUtEEp3A9Um+vGabeotkVpT4aZSKW5HTv36gpGqrDnG/WwhRnKF NfoZXez9InnT0jo7WwfrOJUjI173uv/SjuIhHWSnzEYMwH+386kmMOLovWUCMIRZkGyG cujnYoqgzmfyzKbc4uOTtiLiEbBzpY9OzjoGM43KdOX+AQjM67RuI9w5b6NXaUY/Y9ob l7CTcw19N9RYsECrF5tnO32QKpUGZ/PwxgrZLNuzkXZBIoZXQlsD7uoLfpo8nvLEVmx+ zNGr/dUbIC1YOBrSUHrUtE/t/yMlFfB5VijnaJwB+iGcu3XrYhn2bFJgS6iZQBE5Qn53 snXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=aWAda/y6bkJXmZm6QmmUOIg/kBElvR+98JAgKehXrTQ=; b=SCkygeN6HpNFqARNOocbdELcRqnp86S9oUl1Y3NvVyG+pIgQbZt+CeJOG3MA6KJHC/ 2Qz5IP7/+q70I6nSgCrnJ5OVsZlv89O8GBtezwzv95Xl1cpFPOiG/T0xVVm4imVatVcI VH6y9NIEFq5AhgnDaPhpJ+4FhUEqBjKZmWgB73dIndHWh+JaBZll2W6ryHFgVXSTWxUg XkVKmMfe9wfvkDiK9Ud096FIAzWgbUBPEavNmhx+dSCj2j46oh9/+TZKYEJSppFhwpMn DkptNE4v9RXiwun/xtFPmViQgHZ5VuPDNLzpg+k3KCKAlOoeKxGMqn6nPBj9VjdAJsUe LxHA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sa11-20020a1709076d0b00b007ae096f5de4si761328ejc.962.2022.11.02.01.53.35; Wed, 02 Nov 2022 01:53:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229598AbiKBIvL (ORCPT + 99 others); Wed, 2 Nov 2022 04:51:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231178AbiKBIuv (ORCPT ); Wed, 2 Nov 2022 04:50:51 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F97F27DF0; Wed, 2 Nov 2022 01:50:36 -0700 (PDT) Received: from dggpemm500020.china.huawei.com (unknown [172.30.72.56]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4N2LB44bxMzJnK2; Wed, 2 Nov 2022 16:47:40 +0800 (CST) Received: from dggpemm500006.china.huawei.com (7.185.36.236) by dggpemm500020.china.huawei.com (7.185.36.49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:34 +0800 Received: from thunder-town.china.huawei.com (10.174.178.55) by dggpemm500006.china.huawei.com (7.185.36.236) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 2 Nov 2022 16:50:33 +0800 From: Zhen Lei To: Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , , , Masahiro Yamada , Alexei Starovoitov , Jiri Olsa , Kees Cook , Andrew Morton , "Luis Chamberlain" , , "Steven Rostedt" , Ingo Molnar CC: Zhen Lei , David Laight Subject: [PATCH v8 9/9] kallsyms: Add self-test facility Date: Wed, 2 Nov 2022 16:49:21 +0800 Message-ID: <20221102084921.1615-10-thunder.leizhen@huawei.com> X-Mailer: git-send-email 2.37.3.windows.1 In-Reply-To: <20221102084921.1615-1-thunder.leizhen@huawei.com> References: <20221102084921.1615-1-thunder.leizhen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.178.55] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm500006.china.huawei.com (7.185.36.236) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748373852779302149?= X-GMAIL-MSGID: =?utf-8?q?1748373852779302149?= Added test cases for basic functions and performance of functions kallsyms_lookup_name(), kallsyms_on_each_symbol() and kallsyms_on_each_match_symbol(). It also calculates the compression rate of the kallsyms compression algorithm for the current symbol set. The basic functions test begins by testing a set of symbols whose address values are known. Then, traverse all symbol addresses and find the corresponding symbol name based on the address. It's impossible to determine whether these addresses are correct, but we can use the above three functions along with the addresses to test each other. Due to the traversal operation of kallsyms_on_each_symbol() is too slow, only 60 symbols can be tested in one second, so let it test on average once every 128 symbols. The other two functions validate all symbols. If the basic functions test is passed, print only performance test results. If the test fails, print error information, but do not perform subsequent performance tests. Start self-test automatically after system startup if CONFIG_KALLSYMS_SELFTEST=y. Example of output content: (prefix 'kallsyms_selftest:' is omitted) start --------------------------------------------------------- | nr_symbols | compressed size | original size | ratio(%) | |---------------------------------------------------------| | 174099 | 1960154 | 3750756 | 52.26 | --------------------------------------------------------- kallsyms_lookup_name() looked up 174099 symbols The time spent on each symbol is (ns): min=5250, max=726560, avg=302132 kallsyms_on_each_symbol() traverse all: 16659500 ns kallsyms_on_each_match_symbol() traverse all: 557400 ns finish Signed-off-by: Zhen Lei --- include/linux/kallsyms.h | 1 + init/Kconfig | 13 + kernel/Makefile | 1 + kernel/kallsyms.c | 2 +- kernel/kallsyms_selftest.c | 485 +++++++++++++++++++++++++++++++++++++ kernel/kallsyms_selftest.h | 13 + 6 files changed, 514 insertions(+), 1 deletion(-) create mode 100644 kernel/kallsyms_selftest.c create mode 100644 kernel/kallsyms_selftest.h diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index 5002ebe9dff5a0e..d4079b3d951d1ef 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -66,6 +66,7 @@ static inline void *dereference_symbol_descriptor(void *ptr) } #ifdef CONFIG_KALLSYMS +unsigned long kallsyms_sym_address(int idx); int kallsyms_on_each_symbol(int (*fn)(void *, const char *, unsigned long), void *data); int kallsyms_on_each_match_symbol(int (*fn)(void *, unsigned long), diff --git a/init/Kconfig b/init/Kconfig index abf65098f1b6bf1..c45935cd2f1f471 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1723,6 +1723,19 @@ config KALLSYMS symbolic stack backtraces. This increases the size of the kernel somewhat, as all symbols have to be loaded into the kernel image. +config KALLSYMS_SELFTEST + bool "Test the basic functions and performance of kallsyms" + depends on KALLSYMS + default n + help + Test the basic functions and performance of some interfaces, such as + kallsyms_lookup_name. It also calculates the compression rate of the + kallsyms compression algorithm for the current symbol set. + + Start self-test automatically after system startup. Suggest executing + "dmesg | grep kallsyms_selftest" to collect test results. "finish" is + displayed in the last line, indicating that the test is complete. + config KALLSYMS_ALL bool "Include all symbols in kallsyms" depends on DEBUG_KERNEL && KALLSYMS diff --git a/kernel/Makefile b/kernel/Makefile index d754e0be1176df3..e7fc37a6806979f 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -69,6 +69,7 @@ endif obj-$(CONFIG_UID16) += uid16.o obj-$(CONFIG_MODULE_SIG_FORMAT) += module_signature.o obj-$(CONFIG_KALLSYMS) += kallsyms.o +obj-$(CONFIG_KALLSYMS_SELFTEST) += kallsyms_selftest.o obj-$(CONFIG_BSD_PROCESS_ACCT) += acct.o obj-$(CONFIG_CRASH_CORE) += crash_core.o obj-$(CONFIG_KEXEC_CORE) += kexec_core.o diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 5110d5501edeeb3..77747391f49b66c 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -146,7 +146,7 @@ static unsigned int get_symbol_offset(unsigned long pos) return name - kallsyms_names; } -static unsigned long kallsyms_sym_address(int idx) +unsigned long kallsyms_sym_address(int idx) { if (!IS_ENABLED(CONFIG_KALLSYMS_BASE_RELATIVE)) return kallsyms_addresses[idx]; diff --git a/kernel/kallsyms_selftest.c b/kernel/kallsyms_selftest.c new file mode 100644 index 000000000000000..e8cc0b41c7230a2 --- /dev/null +++ b/kernel/kallsyms_selftest.c @@ -0,0 +1,485 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Test the function and performance of kallsyms + * + * Copyright (C) Huawei Technologies Co., Ltd., 2022 + * + * Authors: Zhen Lei Huawei + */ + +#define pr_fmt(fmt) "kallsyms_selftest: " fmt + +#include +#include +#include +#include +#include +#include +#include + +#include "kallsyms_internal.h" +#include "kallsyms_selftest.h" + + +#define MAX_NUM_OF_RECORDS 64 + +struct test_stat { + int min; + int max; + int save_cnt; + int real_cnt; + int perf; + u64 sum; + char *name; + unsigned long addr; + unsigned long addrs[MAX_NUM_OF_RECORDS]; +}; + +struct test_item { + char *name; + unsigned long addr; +}; + +#define ITEM_FUNC(s) \ + { \ + .name = #s, \ + .addr = (unsigned long)s, \ + } + +#define ITEM_DATA(s) \ + { \ + .name = #s, \ + .addr = (unsigned long)&s, \ + } + + +static int kallsyms_test_var_bss_static; +static int kallsyms_test_var_data_static = 1; +int kallsyms_test_var_bss; +int kallsyms_test_var_data = 1; + +static int kallsyms_test_func_static(void) +{ + kallsyms_test_var_bss_static++; + kallsyms_test_var_data_static++; + + return 0; +} + +int kallsyms_test_func(void) +{ + return kallsyms_test_func_static(); +} + +__weak int kallsyms_test_func_weak(void) +{ + kallsyms_test_var_bss++; + kallsyms_test_var_data++; + return 0; +} + +static struct test_item test_items[] = { + ITEM_FUNC(kallsyms_test_func_static), + ITEM_FUNC(kallsyms_test_func), + ITEM_FUNC(kallsyms_test_func_weak), + ITEM_FUNC(vmalloc), + ITEM_FUNC(vfree), +#ifdef CONFIG_KALLSYMS_ALL + ITEM_DATA(kallsyms_test_var_bss_static), + ITEM_DATA(kallsyms_test_var_data_static), + ITEM_DATA(kallsyms_test_var_bss), + ITEM_DATA(kallsyms_test_var_data), + ITEM_DATA(vmap_area_list), +#endif +}; + +static char stub_name[KSYM_NAME_LEN]; + +static int stat_symbol_len(void *data, const char *name, unsigned long addr) +{ + *(u32 *)data += strlen(name); + + return 0; +} + +static void test_kallsyms_compression_ratio(void) +{ + u32 pos, off, len, num; + u32 ratio, total_size, total_len = 0; + + kallsyms_on_each_symbol(stat_symbol_len, &total_len); + + /* + * A symbol name cannot start with a number. This stub name helps us + * traverse the entire symbol table without finding a match. It's used + * for subsequent performance tests, and its length is the average + * length of all symbol names. + */ + memset(stub_name, '4', sizeof(stub_name)); + pos = total_len / kallsyms_num_syms; + stub_name[pos] = 0; + + pos = 0; + num = 0; + off = 0; + while (pos < kallsyms_num_syms) { + len = kallsyms_names[off]; + num++; + off++; + pos++; + if ((len & 0x80) != 0) { + len = (len & 0x7f) | (kallsyms_names[off] << 7); + num++; + off++; + } + off += len; + }; + + /* + * 1. The length fields is not counted + * 2. The memory occupied by array kallsyms_token_table[] and + * kallsyms_token_index[] needs to be counted. + */ + total_size = off - num; + pos = kallsyms_token_index[0xff]; + total_size += pos + strlen(&kallsyms_token_table[pos]) + 1; + total_size += 0x100 * sizeof(u16); + + pr_info(" ---------------------------------------------------------\n"); + pr_info("| nr_symbols | compressed size | original size | ratio(%%) |\n"); + pr_info("|---------------------------------------------------------|\n"); + ratio = (u32)div_u64(10000ULL * total_size, total_len); + pr_info("| %10d | %10d | %10d | %2d.%-2d |\n", + kallsyms_num_syms, total_size, total_len, ratio / 100, ratio % 100); + pr_info(" ---------------------------------------------------------\n"); +} + +static int lookup_name(void *data, const char *name, unsigned long addr) +{ + u64 t0, t1, t; + unsigned long flags; + struct test_stat *stat = (struct test_stat *)data; + + local_irq_save(flags); + t0 = sched_clock(); + (void)kallsyms_lookup_name(name); + t1 = sched_clock(); + local_irq_restore(flags); + + t = t1 - t0; + if (t < stat->min) + stat->min = t; + + if (t > stat->max) + stat->max = t; + + stat->real_cnt++; + stat->sum += t; + + return 0; +} + +static void test_perf_kallsyms_lookup_name(void) +{ + struct test_stat stat; + + memset(&stat, 0, sizeof(stat)); + stat.min = INT_MAX; + kallsyms_on_each_symbol(lookup_name, &stat); + pr_info("kallsyms_lookup_name() looked up %d symbols\n", stat.real_cnt); + pr_info("The time spent on each symbol is (ns): min=%d, max=%d, avg=%lld\n", + stat.min, stat.max, div_u64(stat.sum, stat.real_cnt)); +} + +static bool match_cleanup_name(const char *s, const char *name) +{ + char *p; + int len; + + if (!IS_ENABLED(CONFIG_LTO_CLANG)) + return false; + + p = strchr(s, '.'); + if (!p) + return false; + + len = strlen(name); + if (p - s != len) + return false; + + return !strncmp(s, name, len); +} + +static int find_symbol(void *data, const char *name, unsigned long addr) +{ + struct test_stat *stat = (struct test_stat *)data; + + if (strcmp(name, stat->name) == 0 || + (!stat->perf && match_cleanup_name(name, stat->name))) { + stat->real_cnt++; + stat->addr = addr; + + if (stat->save_cnt < MAX_NUM_OF_RECORDS) { + stat->addrs[stat->save_cnt] = addr; + stat->save_cnt++; + } + + if (stat->real_cnt == stat->max) + return 1; + } + + return 0; +} + +static void test_perf_kallsyms_on_each_symbol(void) +{ + u64 t0, t1; + unsigned long flags; + struct test_stat stat; + + memset(&stat, 0, sizeof(stat)); + stat.max = INT_MAX; + stat.name = stub_name; + stat.perf = 1; + local_irq_save(flags); + t0 = sched_clock(); + kallsyms_on_each_symbol(find_symbol, &stat); + t1 = sched_clock(); + local_irq_restore(flags); + pr_info("kallsyms_on_each_symbol() traverse all: %lld ns\n", t1 - t0); +} + +static int match_symbol(void *data, unsigned long addr) +{ + struct test_stat *stat = (struct test_stat *)data; + + stat->real_cnt++; + stat->addr = addr; + + if (stat->save_cnt < MAX_NUM_OF_RECORDS) { + stat->addrs[stat->save_cnt] = addr; + stat->save_cnt++; + } + + if (stat->real_cnt == stat->max) + return 1; + + return 0; +} + +static void test_perf_kallsyms_on_each_match_symbol(void) +{ + u64 t0, t1; + unsigned long flags; + struct test_stat stat; + + memset(&stat, 0, sizeof(stat)); + stat.max = INT_MAX; + stat.name = stub_name; + local_irq_save(flags); + t0 = sched_clock(); + kallsyms_on_each_match_symbol(match_symbol, stat.name, &stat); + t1 = sched_clock(); + local_irq_restore(flags); + pr_info("kallsyms_on_each_match_symbol() traverse all: %lld ns\n", t1 - t0); +} + +static int test_kallsyms_basic_function(void) +{ + int i, j, ret; + int next = 0, nr_failed = 0; + char *prefix; + unsigned short rand; + unsigned long addr, lookup_addr; + char namebuf[KSYM_NAME_LEN]; + struct test_stat *stat, *stat2; + + stat = kmalloc(sizeof(*stat) * 2, GFP_KERNEL); + if (!stat) + return -ENOMEM; + stat2 = stat + 1; + + prefix = "kallsyms_lookup_name() for"; + for (i = 0; i < ARRAY_SIZE(test_items); i++) { + addr = kallsyms_lookup_name(test_items[i].name); + if (addr != test_items[i].addr) { + nr_failed++; + pr_info("%s %s failed: addr=%lx, expect %lx\n", + prefix, test_items[i].name, addr, test_items[i].addr); + } + } + + prefix = "kallsyms_on_each_symbol() for"; + for (i = 0; i < ARRAY_SIZE(test_items); i++) { + memset(stat, 0, sizeof(*stat)); + stat->max = INT_MAX; + stat->name = test_items[i].name; + kallsyms_on_each_symbol(find_symbol, stat); + if (stat->addr != test_items[i].addr || stat->real_cnt != 1) { + nr_failed++; + pr_info("%s %s failed: count=%d, addr=%lx, expect %lx\n", + prefix, test_items[i].name, + stat->real_cnt, stat->addr, test_items[i].addr); + } + } + + prefix = "kallsyms_on_each_match_symbol() for"; + for (i = 0; i < ARRAY_SIZE(test_items); i++) { + memset(stat, 0, sizeof(*stat)); + stat->max = INT_MAX; + stat->name = test_items[i].name; + kallsyms_on_each_match_symbol(match_symbol, test_items[i].name, stat); + if (stat->addr != test_items[i].addr || stat->real_cnt != 1) { + nr_failed++; + pr_info("%s %s failed: count=%d, addr=%lx, expect %lx\n", + prefix, test_items[i].name, + stat->real_cnt, stat->addr, test_items[i].addr); + } + } + + if (nr_failed) { + kfree(stat); + return -ESRCH; + } + + for (i = 0; i < kallsyms_num_syms; i++) { + addr = kallsyms_sym_address(i); + if (!is_ksym_addr(addr)) + continue; + + ret = lookup_symbol_name(addr, namebuf); + if (unlikely(ret)) { + namebuf[0] = 0; + goto failed; + } + + /* + * The first '.' may be the initial letter, in which case the + * entire symbol name will be truncated to an empty string in + * cleanup_symbol_name(). Do not test these symbols. + * + * For example: + * cat /proc/kallsyms | awk '{print $3}' | grep -E "^\." | head + * .E_read_words + * .E_leading_bytes + * .E_trailing_bytes + * .E_write_words + * .E_copy + * .str.292.llvm.12122243386960820698 + * .str.24.llvm.12122243386960820698 + * .str.29.llvm.12122243386960820698 + * .str.75.llvm.12122243386960820698 + * .str.99.llvm.12122243386960820698 + */ + if (IS_ENABLED(CONFIG_LTO_CLANG) && !namebuf[0]) + continue; + + lookup_addr = kallsyms_lookup_name(namebuf); + + memset(stat, 0, sizeof(*stat)); + stat->max = INT_MAX; + kallsyms_on_each_match_symbol(match_symbol, namebuf, stat); + + /* + * kallsyms_on_each_symbol() is too slow, randomly select some + * symbols for test. + */ + if (i >= next) { + memset(stat2, 0, sizeof(*stat2)); + stat2->max = INT_MAX; + stat2->name = namebuf; + kallsyms_on_each_symbol(find_symbol, stat2); + + /* + * kallsyms_on_each_symbol() and kallsyms_on_each_match_symbol() + * need to get the same traversal result. + */ + if (stat->addr != stat2->addr || + stat->real_cnt != stat2->real_cnt || + memcmp(stat->addrs, stat2->addrs, + stat->save_cnt * sizeof(stat->addrs[0]))) + goto failed; + + /* + * The average of random increments is 128, that is, one of + * them is tested every 128 symbols. + */ + get_random_bytes(&rand, sizeof(rand)); + next = i + (rand & 0xff) + 1; + } + + /* Need to be found at least once */ + if (!stat->real_cnt) + goto failed; + + /* + * kallsyms_lookup_name() returns the address of the first + * symbol found and cannot be NULL. + */ + if (!lookup_addr || lookup_addr != stat->addrs[0]) + goto failed; + + /* + * If the addresses of all matching symbols are recorded, the + * target address needs to be exist. + */ + if (stat->real_cnt <= MAX_NUM_OF_RECORDS) { + for (j = 0; j < stat->save_cnt; j++) { + if (stat->addrs[j] == addr) + break; + } + + if (j == stat->save_cnt) + goto failed; + } + } + + kfree(stat); + + return 0; + +failed: + pr_info("Test for %dth symbol failed: (%s) addr=%lx", i, namebuf, addr); + kfree(stat); + return -ESRCH; +} + +static int test_entry(void *p) +{ + int ret; + + do { + schedule_timeout(5 * HZ); + } while (system_state != SYSTEM_RUNNING); + + pr_info("start\n"); + ret = test_kallsyms_basic_function(); + if (ret) { + pr_info("abort\n"); + return 0; + } + + test_kallsyms_compression_ratio(); + test_perf_kallsyms_lookup_name(); + test_perf_kallsyms_on_each_symbol(); + test_perf_kallsyms_on_each_match_symbol(); + pr_info("finish\n"); + + return 0; +} + +static int __init kallsyms_test_init(void) +{ + struct task_struct *t; + + t = kthread_create(test_entry, NULL, "kallsyms_test"); + if (IS_ERR(t)) { + pr_info("Create kallsyms selftest task failed\n"); + return PTR_ERR(t); + } + kthread_bind(t, 0); + wake_up_process(t); + + return 0; +} +late_initcall(kallsyms_test_init); diff --git a/kernel/kallsyms_selftest.h b/kernel/kallsyms_selftest.h new file mode 100644 index 000000000000000..c0ca548e2a22599 --- /dev/null +++ b/kernel/kallsyms_selftest.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef LINUX_KALLSYMS_SELFTEST_H_ +#define LINUX_KALLSYMS_SELFTEST_H_ + +#include + +extern int kallsyms_test_var_bss; +extern int kallsyms_test_var_data; + +extern int kallsyms_test_func(void); +extern int kallsyms_test_func_weak(void); + +#endif // LINUX_KALLSYMS_SELFTEST_H_