Message ID | 20221227130701.124278-1-yangjihong1@huawei.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1390426wrt; Tue, 27 Dec 2022 05:27:18 -0800 (PST) X-Google-Smtp-Source: AMrXdXsJzSW/5312nEjkrANH4HTgSn7ESWd+D5FwGfcFh9eul4zv5M7jEpdajQ+jisNM9WsGop6G X-Received: by 2002:a17:90b:3597:b0:214:222:6ed3 with SMTP id mm23-20020a17090b359700b0021402226ed3mr24609860pjb.43.1672147638401; Tue, 27 Dec 2022 05:27:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672147638; cv=none; d=google.com; s=arc-20160816; b=qqP7HV4UQt3Ar4ym8nH0nWS+A3T3fobOAF2Kk9PjmJ3nVRDmQQ9GjGvjOph2L+RtmK 8YZKDhIkOyOckLnH3UfmiAk57WW0Dl4UiKC1UG4RqDSOekQn8vh4Nx7J5r2HD+KaJZ76 al0bIKli1J3wK5KyYFsar2HL53etPZ5PW9rpxdCez7mDeBQHfKMdeFBAD6ITE9k5o8wK skf1452rzfijgYZK9WvwczICiniH4nTV2yEBpJUzjIBn+Na6c4FpyO1QT5gqIUbcmcnk VR1QuYpkZq21MANEdkg/BuJoB74E3+MoLr74hSWvIdPxx0CzLG+80PK6bGEFjDKrrDQf I9YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=SDCDyHO2bpRwxtI3A9M/8tvTFkfId7bgcKmEnyJ/sMA=; b=W8EQAiQ7/TZd/lO2Nij3L3GbN/ViSE5DTVkYOzl6FMk6IWUbuZhgiiRHW/Mah9Qhcg qAmY9SRYLV96S9ruwqtAKdLjkEPnSFv19YIlciLl11tDlZCS3xwHaLEbUhWIab6hzupa xqXCHyfud9uG89zMks6+gxNXHJcwoUGEYdpLJItuvGTIWVJWMNM2SgNaLpIw2K/zvLcb dLZLYFqP05fkca7yS9pdbmrAhTynkBINfbAcjIoEoDoHsYhVfzkAe4AsllS4x0u2cGke Fj0KpWuqhh0eem2+f9lmRf9UCIBFLqZ6qMYqgNcmM7AtY8UED/fUJYJUbPiuoZpZOQK3 rWfA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z13-20020a17090a7b8d00b00225b5d6a297si12717134pjc.65.2022.12.27.05.27.06; Tue, 27 Dec 2022 05:27:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231527AbiL0NKu (ORCPT <rfc822;eddaouddi.ayoub@gmail.com> + 99 others); Tue, 27 Dec 2022 08:10:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231486AbiL0NKl (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 27 Dec 2022 08:10:41 -0500 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E11561F0; Tue, 27 Dec 2022 05:10:38 -0800 (PST) Received: from kwepemm600003.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4NhFNd18Wvz16LrY; Tue, 27 Dec 2022 21:09:21 +0800 (CST) Received: from localhost.localdomain (10.67.174.95) by kwepemm600003.china.huawei.com (7.193.23.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 27 Dec 2022 21:10:35 +0800 From: Yang Jihong <yangjihong1@huawei.com> To: <peterz@infradead.org>, <mingo@redhat.com>, <acme@kernel.org>, <mark.rutland@arm.com>, <alexander.shishkin@linux.intel.com>, <jolsa@kernel.org>, <namhyung@kernel.org>, <jiwei.sun@windriver.com>, <linux-perf-users@vger.kernel.org>, <linux-kernel@vger.kernel.org> CC: <yangjihong1@huawei.com> Subject: [PATCH] perf record: Fix coredump with --overwrite and --max-size Date: Tue, 27 Dec 2022 13:07:01 +0000 Message-ID: <20221227130701.124278-1-yangjihong1@huawei.com> X-Mailer: git-send-email 2.30.GIT MIME-Version: 1.0 Content-Transfer-Encoding: 7BIT Content-Type: text/plain; charset=US-ASCII X-Originating-IP: [10.67.174.95] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To kwepemm600003.china.huawei.com (7.193.23.202) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1753373881971073210?= X-GMAIL-MSGID: =?utf-8?q?1753373881971073210?= |
Series |
perf record: Fix coredump with --overwrite and --max-size
|
|
Commit Message
Yang Jihong
Dec. 27, 2022, 1:07 p.m. UTC
When --overwrite and --max-size options of perf record are used together,
a segmentation fault occurs. The following is an example:
# perf record -e sched:sched* --overwrite --max-size 1M -a -- sleep 1
[ perf record: Woken up 1 times to write data ]
perf: Segmentation fault
Obtained 1 stack frames.
[0xc4c67f]
Segmentation fault (core dumped)
backtrace of the core file is as follows:
#0 0x0000000000417990 in process_locked_synthesized_event (tool=0x0, event=0x15, sample=0x1de0, machine=0xf8) at builtin-record.c:630
#1 0x000000000057ee53 in perf_event__synthesize_threads (nr_threads_synthesize=21, mmap_data=<optimized out>, needs_mmap=<optimized out>, machine=0x17ad9b0, process=<optimized out>, tool=0x0) at util/synthetic-events.c:1950
#2 __machine__synthesize_threads (nr_threads_synthesize=0, data_mmap=<optimized out>, needs_mmap=<optimized out>, process=<optimized out>, threads=0x8, target=0x8, tool=0x0, machine=0x17ad9b0) at util/synthetic-events.c:1936
#3 machine__synthesize_threads (machine=0x17ad9b0, target=0x8, threads=0x8, needs_mmap=<optimized out>, data_mmap=<optimized out>, nr_threads_synthesize=0) at util/synthetic-events.c:1947
#4 0x000000000040165d in record__synthesize (tail=<optimized out>, rec=0xbe2520 <record>) at builtin-record.c:2010
#5 0x0000000000403989 in __cmd_record (argc=<optimized out>, argv=<optimized out>, rec=0xbe2520 <record>) at builtin-record.c:2810
#6 0x00000000004196ba in record__init_thread_user_masks (rec=0xbe2520 <record>, cpus=0x17a65f0) at builtin-record.c:3837
#7 record__init_thread_masks (rec=0xbe2520 <record>) at builtin-record.c:3938
#8 cmd_record (argc=1, argv=0x7ffdd692dc60) at builtin-record.c:4241
#9 0x00000000004b701d in pager_command_config (var=0x0, value=0x15 <error: Cannot access memory at address 0x15>, data=0x1de0) at perf.c:117
#10 0x00000000004b732b in get_leaf_frame_caller_aarch64 (sample=0xfffffffb, thread=0x0, usr_idx=<optimized out>) at util/arm64-frame-pointer-unwind-support.c:56
#11 0x0000000000406331 in execv_dashed_external (argv=0x7ffdd692d9e8) at perf.c:410
#12 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:431
#13 main (argc=<optimized out>, argv=0x7ffdd692d9e8) at perf.c:562
The reason is that record__bytes_written accesses the freed memory rec->thread_data,
The process is as follows:
__cmd_record
-> record__free_thread_data
-> zfree(&rec->thread_data) // free rec->thread_data
-> record__synthesize
-> perf_event__synthesize_id_index
-> process_synthesized_event
-> record__write
-> record__bytes_written // access rec->thread_data
In the overwrite scenario, to synthesize non-sample events,
we do not need to check perf size limit.
Fixes: 6d57581659f7 ("perf record: Add support for limit perf output file size")
Signed-off-by: Yang Jihong <yangjihong1@huawei.com>
---
tools/perf/builtin-record.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Comments
On Tue, Dec 27, 2022 at 5:10 AM Yang Jihong <yangjihong1@huawei.com> wrote: > > When --overwrite and --max-size options of perf record are used together, > a segmentation fault occurs. The following is an example: > > # perf record -e sched:sched* --overwrite --max-size 1M -a -- sleep 1 > [ perf record: Woken up 1 times to write data ] > perf: Segmentation fault > Obtained 1 stack frames. > [0xc4c67f] > Segmentation fault (core dumped) > > backtrace of the core file is as follows: > > #0 0x0000000000417990 in process_locked_synthesized_event (tool=0x0, event=0x15, sample=0x1de0, machine=0xf8) at builtin-record.c:630 > #1 0x000000000057ee53 in perf_event__synthesize_threads (nr_threads_synthesize=21, mmap_data=<optimized out>, needs_mmap=<optimized out>, machine=0x17ad9b0, process=<optimized out>, tool=0x0) at util/synthetic-events.c:1950 > #2 __machine__synthesize_threads (nr_threads_synthesize=0, data_mmap=<optimized out>, needs_mmap=<optimized out>, process=<optimized out>, threads=0x8, target=0x8, tool=0x0, machine=0x17ad9b0) at util/synthetic-events.c:1936 > #3 machine__synthesize_threads (machine=0x17ad9b0, target=0x8, threads=0x8, needs_mmap=<optimized out>, data_mmap=<optimized out>, nr_threads_synthesize=0) at util/synthetic-events.c:1947 > #4 0x000000000040165d in record__synthesize (tail=<optimized out>, rec=0xbe2520 <record>) at builtin-record.c:2010 > #5 0x0000000000403989 in __cmd_record (argc=<optimized out>, argv=<optimized out>, rec=0xbe2520 <record>) at builtin-record.c:2810 > #6 0x00000000004196ba in record__init_thread_user_masks (rec=0xbe2520 <record>, cpus=0x17a65f0) at builtin-record.c:3837 > #7 record__init_thread_masks (rec=0xbe2520 <record>) at builtin-record.c:3938 > #8 cmd_record (argc=1, argv=0x7ffdd692dc60) at builtin-record.c:4241 > #9 0x00000000004b701d in pager_command_config (var=0x0, value=0x15 <error: Cannot access memory at address 0x15>, data=0x1de0) at perf.c:117 > #10 0x00000000004b732b in get_leaf_frame_caller_aarch64 (sample=0xfffffffb, thread=0x0, usr_idx=<optimized out>) at util/arm64-frame-pointer-unwind-support.c:56 > #11 0x0000000000406331 in execv_dashed_external (argv=0x7ffdd692d9e8) at perf.c:410 > #12 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:431 > #13 main (argc=<optimized out>, argv=0x7ffdd692d9e8) at perf.c:562 I'm not sure this callstack is correct. > > The reason is that record__bytes_written accesses the freed memory rec->thread_data, > The process is as follows: > __cmd_record > -> record__free_thread_data > -> zfree(&rec->thread_data) // free rec->thread_data > -> record__synthesize > -> perf_event__synthesize_id_index > -> process_synthesized_event > -> record__write > -> record__bytes_written // access rec->thread_data > > In the overwrite scenario, to synthesize non-sample events, > we do not need to check perf size limit. Hmm.. I think we should prevent this kind of access after record__free_thread_data(). We may set nr_threads to 0 and save the bytes_written for threads separately. Thanks, Namhyung > > Fixes: 6d57581659f7 ("perf record: Add support for limit perf output file size") > Signed-off-by: Yang Jihong <yangjihong1@huawei.com> > --- > tools/perf/builtin-record.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c > index 29dcd454b8e2..c5f169150d63 100644 > --- a/tools/perf/builtin-record.c > +++ b/tools/perf/builtin-record.c > @@ -260,7 +260,7 @@ static int record__write(struct record *rec, struct mmap *map __maybe_unused, > else > rec->bytes_written += size; > > - if (record__output_max_size_exceeded(rec) && !done) { > + if (!rec->opts.tail_synthesize && record__output_max_size_exceeded(rec) && !done) { > fprintf(stderr, "[ perf record: perf size limit reached (%" PRIu64 " KB)," > " stopping session ]\n", > record__bytes_written(rec) >> 10); > -- > 2.17.1 >
Hello, On 2022/12/28 5:05, Namhyung Kim wrote: > On Tue, Dec 27, 2022 at 5:10 AM Yang Jihong <yangjihong1@huawei.com> wrote: >> >> When --overwrite and --max-size options of perf record are used together, >> a segmentation fault occurs. The following is an example: >> >> # perf record -e sched:sched* --overwrite --max-size 1M -a -- sleep 1 >> [ perf record: Woken up 1 times to write data ] >> perf: Segmentation fault >> Obtained 1 stack frames. >> [0xc4c67f] >> Segmentation fault (core dumped) >> >> backtrace of the core file is as follows: >> >> #0 0x0000000000417990 in process_locked_synthesized_event (tool=0x0, event=0x15, sample=0x1de0, machine=0xf8) at builtin-record.c:630 >> #1 0x000000000057ee53 in perf_event__synthesize_threads (nr_threads_synthesize=21, mmap_data=<optimized out>, needs_mmap=<optimized out>, machine=0x17ad9b0, process=<optimized out>, tool=0x0) at util/synthetic-events.c:1950 >> #2 __machine__synthesize_threads (nr_threads_synthesize=0, data_mmap=<optimized out>, needs_mmap=<optimized out>, process=<optimized out>, threads=0x8, target=0x8, tool=0x0, machine=0x17ad9b0) at util/synthetic-events.c:1936 >> #3 machine__synthesize_threads (machine=0x17ad9b0, target=0x8, threads=0x8, needs_mmap=<optimized out>, data_mmap=<optimized out>, nr_threads_synthesize=0) at util/synthetic-events.c:1947 >> #4 0x000000000040165d in record__synthesize (tail=<optimized out>, rec=0xbe2520 <record>) at builtin-record.c:2010 >> #5 0x0000000000403989 in __cmd_record (argc=<optimized out>, argv=<optimized out>, rec=0xbe2520 <record>) at builtin-record.c:2810 >> #6 0x00000000004196ba in record__init_thread_user_masks (rec=0xbe2520 <record>, cpus=0x17a65f0) at builtin-record.c:3837 >> #7 record__init_thread_masks (rec=0xbe2520 <record>) at builtin-record.c:3938 >> #8 cmd_record (argc=1, argv=0x7ffdd692dc60) at builtin-record.c:4241 >> #9 0x00000000004b701d in pager_command_config (var=0x0, value=0x15 <error: Cannot access memory at address 0x15>, data=0x1de0) at perf.c:117 >> #10 0x00000000004b732b in get_leaf_frame_caller_aarch64 (sample=0xfffffffb, thread=0x0, usr_idx=<optimized out>) at util/arm64-frame-pointer-unwind-support.c:56 >> #11 0x0000000000406331 in execv_dashed_external (argv=0x7ffdd692d9e8) at perf.c:410 >> #12 run_argv (argcp=<synthetic pointer>, argv=<synthetic pointer>) at perf.c:431 >> #13 main (argc=<optimized out>, argv=0x7ffdd692d9e8) at perf.c:562 > > I'm not sure this callstack is correct. This is the backtrace printed by using the gdb to debug the core file, which should be normal. The preceding example should trigger this problem as long as the perf file reaches max_size. > >> >> The reason is that record__bytes_written accesses the freed memory rec->thread_data, >> The process is as follows: >> __cmd_record >> -> record__free_thread_data >> -> zfree(&rec->thread_data) // free rec->thread_data >> -> record__synthesize >> -> perf_event__synthesize_id_index >> -> process_synthesized_event >> -> record__write >> -> record__bytes_written // access rec->thread_data >> >> In the overwrite scenario, to synthesize non-sample events, >> we do not need to check perf size limit. > > Hmm.. I think we should prevent this kind of access after > record__free_thread_data(). We may set nr_threads to 0 > and save the bytes_written for threads separately. Ok, will change in next version. > the value of done is 1 here, Therefore, we only need to check the value of done first. Thanks, Yang. > Thanks, > Namhyung > > >> >> Fixes: 6d57581659f7 ("perf record: Add support for limit perf output file size") >> Signed-off-by: Yang Jihong <yangjihong1@huawei.com> >> --- >> tools/perf/builtin-record.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c >> index 29dcd454b8e2..c5f169150d63 100644 >> --- a/tools/perf/builtin-record.c >> +++ b/tools/perf/builtin-record.c >> @@ -260,7 +260,7 @@ static int record__write(struct record *rec, struct mmap *map __maybe_unused, >> else >> rec->bytes_written += size; >> >> - if (record__output_max_size_exceeded(rec) && !done) { >> + if (!rec->opts.tail_synthesize && record__output_max_size_exceeded(rec) && !done) { >> fprintf(stderr, "[ perf record: perf size limit reached (%" PRIu64 " KB)," >> " stopping session ]\n", >> record__bytes_written(rec) >> 10); >> -- >> 2.17.1 >> > . >
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c index 29dcd454b8e2..c5f169150d63 100644 --- a/tools/perf/builtin-record.c +++ b/tools/perf/builtin-record.c @@ -260,7 +260,7 @@ static int record__write(struct record *rec, struct mmap *map __maybe_unused, else rec->bytes_written += size; - if (record__output_max_size_exceeded(rec) && !done) { + if (!rec->opts.tail_synthesize && record__output_max_size_exceeded(rec) && !done) { fprintf(stderr, "[ perf record: perf size limit reached (%" PRIu64 " KB)," " stopping session ]\n", record__bytes_written(rec) >> 10);