Message ID | 20240212233322.1855161-1-namhyung@kernel.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-62541-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:bc8a:b0:106:860b:bbdd with SMTP id dn10csp224994dyb; Mon, 12 Feb 2024 15:33:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU8tSJlsWzYBgAe0MEuiKsBx3/OMInGhLxZl/zFhXqYePUFcW5k5U1SLyc47uqKV1+Pb0uKRCLeFR44Lse28bHPad8CuQ== X-Google-Smtp-Source: AGHT+IHY4ejeqZ66MEQ+1EF+sc4HtyHujoWS+SheZaVrTAle6zAVw99I/fn6mpSPoHBMnwiFDBNT X-Received: by 2002:a17:902:e5c8:b0:1d9:2369:d0f8 with SMTP id u8-20020a170902e5c800b001d92369d0f8mr10107479plf.9.1707780826072; Mon, 12 Feb 2024 15:33:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707780826; cv=pass; d=google.com; s=arc-20160816; b=hI6oD3b1kjJi62GSbj4kXT0sgPtUX/S6zEcCUq2EmGe/T+jFglK9e6jcpXIHZW4Ysb AZ3EwQhfB2yF6tEKEveYaFM84mKYZEU1pR50W0AdqhD7r/hlOVAxNKNNMEmPvasoZKhl SswggKLQbn7FNeH+MN4uEjsXtbsg9YPDvwF5TPlDraok+/nKH/t+1sJInaERvB4CsRp7 q8lqf6MdsSdJBpyiFGwBV/0JjlQpqAdnzlEYWyqoyPDoQ+kHMUE8WQJVhor073jt0KjM SBv/m7JFwl479Nto5OSPJkbyIxury9dNetJXgLDC8PAgsgX9uWnU4uQ7n3v31SfOzMWC PzTg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=bRP88klJzxAt+kVvOlVZRMOyebtISu2xVf1f4LuZAGI=; fh=5pJYZiwPU3FfNGS6k35gtoTzvoSHg9jQFyRI/lzF028=; b=oYAoI3sJcaD0gMvyy9nARDqOpn8TkbqWoDMIbCstHB67XA1K9dA6yt5rzK80bKMitl 9mT+C9Dtai/Nf1K1smyR0uLKjPCHJSc+p9ATQJ+YIN5yuwLXieFMuyJ5x6d3vOoiYEAN whIYGgyUHK2A96IMbs+Jf3tfZqqnD9vYnKZ8KZ9D2lxnSJd4AE87zzkhUFVF3MxIWoIh VllYkKgXEVFbcmdMybbizPpGQtguOnQyDVeYt2VWIy94kADa8SugxZsBU3Lo5GhnBndA PRVvR9rFBefbxph2Hm2672sIbx4WL/LBAUIQNF9zrB7XLf/c+rLpTs2NuYiQGIWDg1Yq CFCA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=np8KwOMt; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-62541-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62541-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=2; AJvYcCVHsAa9l5Z/EEDrm2ZEF9OvKorHvPeU1hHtzhzibIOAm8Ub8KDoTJUftrSZyk6XSLXvtiLZwUZB4N5cVFSUmd5iz+voXQ== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id b8-20020a170902d50800b001d767748d26si187552plg.629.2024.02.12.15.33.45 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 15:33:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-62541-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=np8KwOMt; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-62541-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-62541-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 68EFDB21D78 for <ouuuleilei@gmail.com>; Mon, 12 Feb 2024 23:33:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D23050A70; Mon, 12 Feb 2024 23:33:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="np8KwOMt" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D10B84F5FE; Mon, 12 Feb 2024 23:33:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707780804; cv=none; b=qmXZAtfZnJBI8UySkwDoUodco4/jh19gYd7P+HNlzjbSCSLZnI97MC9xjak5Q++K99p9fOyuKvA8mP1D6vIFjdWKn6a338kaKMvOxj5DClLC40vg7Oh681sg39qHW4LX7TvtsxzQdf2fLRP3fhDzF6j7MK/9VZWQlf0hvcXrMjs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707780804; c=relaxed/simple; bh=f+WQOpFf8Q6j7gyAcUMEvgt2grnf1Lmgn20YVuov3JI=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Azys0bxMgl7JBqeLIyvd3YtjbzEv4tZh3eV8F0AaWGCecJNLuoiHqGUbiBMHORROAMPcnuYpK4U8g8QPPQxxkVjas5xrpfvG1Neqs06gR4gv+lj0mubv3q6Us7OHi+rJgBLZrkj2PtsZHhOGyp2Q3UDedlq++QdOUyG2nqjppEU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=np8KwOMt; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8032C433F1; Mon, 12 Feb 2024 23:33:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1707780804; bh=f+WQOpFf8Q6j7gyAcUMEvgt2grnf1Lmgn20YVuov3JI=; h=From:To:Cc:Subject:Date:From; b=np8KwOMtCy2fO+orYiTn3Ep1+jt5Bv2nJ58MKS4wd/+XvniMf2x6InAWrwKv3kcPf fA+e42TZXRLYYd16MifLWB9lfoJCuufMCUOn48Agy++a2zT/5yiJrTetG6BOJU+4Op k1LV0bxFSaG6YvJvF0XqG8ms1DEbjqg8R2vKofiXuOS0iljhplswKMRtGd5aZwguzh FMhAENPKcE4rRDHmxZVjYZqdTREcB7aC/vnbFpODSgFOTx8XMijy2hc2FoYPZnkr0a PwzO1/wx0B7oYnX37+lFfq9IJUaZhwpuHBJzAKBp2ctmSfTBN3jRgmDaONJ9P3RrWd rVTYejPs5+dZA== From: Namhyung Kim <namhyung@kernel.org> To: Arnaldo Carvalho de Melo <acme@kernel.org>, Ian Rogers <irogers@google.com> Cc: Jiri Olsa <jolsa@kernel.org>, Adrian Hunter <adrian.hunter@intel.com>, Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@kernel.org>, LKML <linux-kernel@vger.kernel.org>, linux-perf-users@vger.kernel.org, Leo Yan <leo.yan@linux.dev>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, John Garry <john.g.garry@oracle.com>, Mike Leach <mike.leach@linaro.org> Subject: [PATCH] perf tools: Fixup module symbol end address properly Date: Mon, 12 Feb 2024 15:33:22 -0800 Message-ID: <20240212233322.1855161-1-namhyung@kernel.org> X-Mailer: git-send-email 2.43.0.687.g38aa6559b0-goog Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790737987438254795 X-GMAIL-MSGID: 1790737987438254795 |
Series |
perf tools: Fixup module symbol end address properly
|
|
Commit Message
Namhyung Kim
Feb. 12, 2024, 11:33 p.m. UTC
I got a strange error on ARM to fail on processing FINISHED_ROUND
record. It turned out that it was failing in symbol__alloc_hist()
because the symbol size is too big.
When a sample is captured on a specific BPF program, it failed. I've
added a debug code and found the end address of the symbol is from
the next module which is placed far way.
ffff800008795778-ffff80000879d6d8: bpf_prog_1bac53b8aac4bc58_netcg_sock [bpf]
ffff80000879d6d8-ffff80000ad656b4: bpf_prog_76867454b5944e15_netcg_getsockopt [bpf]
ffff80000ad656b4-ffffd69b7af74048: bpf_prog_1d50286d2eb1be85_hn_egress [bpf] <---------- here
ffffd69b7af74048-ffffd69b7af74048: $x.5 [sha3_generic]
ffffd69b7af74048-ffffd69b7af740b8: crypto_sha3_init [sha3_generic]
ffffd69b7af740b8-ffffd69b7af741e0: crypto_sha3_update [sha3_generic]
The logic in symbols__fixup_end() just uses curr->start to update the
prev->end. But in this case, it won't work as it's too different.
I think ARM has a different kernel memory layout for modules and BPF
than on x86. Actually there's a logic to handle kernel and module
boundary. Let's do the same for symbols between different modules.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/util/symbol.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
Comments
On Mon, Feb 12, 2024 at 03:33:22PM -0800, Namhyung Kim wrote: > I got a strange error on ARM to fail on processing FINISHED_ROUND > record. It turned out that it was failing in symbol__alloc_hist() > because the symbol size is too big. > > When a sample is captured on a specific BPF program, it failed. I've > added a debug code and found the end address of the symbol is from > the next module which is placed far way. > > ffff800008795778-ffff80000879d6d8: bpf_prog_1bac53b8aac4bc58_netcg_sock [bpf] > ffff80000879d6d8-ffff80000ad656b4: bpf_prog_76867454b5944e15_netcg_getsockopt [bpf] > ffff80000ad656b4-ffffd69b7af74048: bpf_prog_1d50286d2eb1be85_hn_egress [bpf] <---------- here > ffffd69b7af74048-ffffd69b7af74048: $x.5 [sha3_generic] > ffffd69b7af74048-ffffd69b7af740b8: crypto_sha3_init [sha3_generic] > ffffd69b7af740b8-ffffd69b7af741e0: crypto_sha3_update [sha3_generic] > > The logic in symbols__fixup_end() just uses curr->start to update the > prev->end. But in this case, it won't work as it's too different. > > I think ARM has a different kernel memory layout for modules and BPF > than on x86. Actually there's a logic to handle kernel and module > boundary. Let's do the same for symbols between different modules. Even Arm32 and Arm64 kernel have different memory layout for modules and kernel image. eBPF program (JITed) should be allocated from the vmalloc region, for Arm64, see bpf_jit_alloc_exec() in arch/arm64/net/bpf_jit_comp.c. > Signed-off-by: Namhyung Kim <namhyung@kernel.org> > --- > tools/perf/util/symbol.c | 21 +++++++++++++++++++-- > 1 file changed, 19 insertions(+), 2 deletions(-) > > diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c > index 35975189999b..9ebdb8e13c0b 100644 > --- a/tools/perf/util/symbol.c > +++ b/tools/perf/util/symbol.c > @@ -248,14 +248,31 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms) > * segment is very big. Therefore do not fill this gap and do > * not assign it to the kernel dso map (kallsyms). > * > + * Also BPF code can be allocated separately from text segments > + * and modules. So the last entry in a module should not fill > + * the gap too. > + * > * In kallsyms, it determines module symbols using '[' character > * like in: > * ffffffffc1937000 T hdmi_driver_init [snd_hda_codec_hdmi] > */ > if (prev->end == prev->start) { > + const char *prev_mod; > + const char *curr_mod; > + > + if (!is_kallsyms) { > + prev->end = curr->start; > + continue; > + } > + > + prev_mod = strchr(prev->name, '['); > + curr_mod = strchr(curr->name, '['); > + > /* Last kernel/module symbol mapped to end of page */ > - if (is_kallsyms && (!strchr(prev->name, '[') != > - !strchr(curr->name, '['))) > + if (!prev_mod != !curr_mod) > + prev->end = roundup(prev->end + 4096, 4096); > + /* Last symbol in the previous module */ > + else if (prev_mod && strcmp(prev_mod, curr_mod)) Should two consecutive moudles fall into this case? I think we need to assign 'prev->end = curr->start' for two two consecutive moudles. If so, we should use a specific checking for eBPF program, e.g.: else if (prev_mod && strcmp(prev_mod, curr_mod) && (!strcmp(prev->name, "bpf") || !strcmp(curr->name, "bpf"))) Thanks, Leo > prev->end = roundup(prev->end + 4096, 4096); > else > prev->end = curr->start; > -- > 2.43.0.687.g38aa6559b0-goog >
Hi Leo, Thanks for your review! On Mon, Feb 12, 2024 at 7:40 PM Leo Yan <leo.yan@linux.dev> wrote: > > On Mon, Feb 12, 2024 at 03:33:22PM -0800, Namhyung Kim wrote: > > I got a strange error on ARM to fail on processing FINISHED_ROUND > > record. It turned out that it was failing in symbol__alloc_hist() > > because the symbol size is too big. > > > > When a sample is captured on a specific BPF program, it failed. I've > > added a debug code and found the end address of the symbol is from > > the next module which is placed far way. > > > > ffff800008795778-ffff80000879d6d8: bpf_prog_1bac53b8aac4bc58_netcg_sock [bpf] > > ffff80000879d6d8-ffff80000ad656b4: bpf_prog_76867454b5944e15_netcg_getsockopt [bpf] > > ffff80000ad656b4-ffffd69b7af74048: bpf_prog_1d50286d2eb1be85_hn_egress [bpf] <---------- here > > ffffd69b7af74048-ffffd69b7af74048: $x.5 [sha3_generic] > > ffffd69b7af74048-ffffd69b7af740b8: crypto_sha3_init [sha3_generic] > > ffffd69b7af740b8-ffffd69b7af741e0: crypto_sha3_update [sha3_generic] > > > > The logic in symbols__fixup_end() just uses curr->start to update the > > prev->end. But in this case, it won't work as it's too different. > > > > I think ARM has a different kernel memory layout for modules and BPF > > than on x86. Actually there's a logic to handle kernel and module > > boundary. Let's do the same for symbols between different modules. > > Even Arm32 and Arm64 kernel have different memory layout for modules > and kernel image. > > eBPF program (JITed) should be allocated from the vmalloc region, for > Arm64, see bpf_jit_alloc_exec() in arch/arm64/net/bpf_jit_comp.c. Ok, so chances are they can fall out far away right? > > > Signed-off-by: Namhyung Kim <namhyung@kernel.org> > > --- > > tools/perf/util/symbol.c | 21 +++++++++++++++++++-- > > 1 file changed, 19 insertions(+), 2 deletions(-) > > > > diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c > > index 35975189999b..9ebdb8e13c0b 100644 > > --- a/tools/perf/util/symbol.c > > +++ b/tools/perf/util/symbol.c > > @@ -248,14 +248,31 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms) > > * segment is very big. Therefore do not fill this gap and do > > * not assign it to the kernel dso map (kallsyms). > > * > > + * Also BPF code can be allocated separately from text segments > > + * and modules. So the last entry in a module should not fill > > + * the gap too. > > + * > > * In kallsyms, it determines module symbols using '[' character > > * like in: > > * ffffffffc1937000 T hdmi_driver_init [snd_hda_codec_hdmi] > > */ > > if (prev->end == prev->start) { > > + const char *prev_mod; > > + const char *curr_mod; > > + > > + if (!is_kallsyms) { > > + prev->end = curr->start; > > + continue; > > + } > > + > > + prev_mod = strchr(prev->name, '['); > > + curr_mod = strchr(curr->name, '['); > > + > > /* Last kernel/module symbol mapped to end of page */ > > - if (is_kallsyms && (!strchr(prev->name, '[') != > > - !strchr(curr->name, '['))) > > + if (!prev_mod != !curr_mod) > > + prev->end = roundup(prev->end + 4096, 4096); > > + /* Last symbol in the previous module */ > > + else if (prev_mod && strcmp(prev_mod, curr_mod)) > > Should two consecutive moudles fall into this case? I think we need to assign > 'prev->end = curr->start' for two two consecutive moudles. Yeah I thought about that case but I believe they would be on separate pages (hopefully there's a page gap between them). So I think it should not overlap. But if you really care we can check it explicitly like this: prev->end = min(roundup(...), curr->start); > > If so, we should use a specific checking for eBPF program, e.g.: > > else if (prev_mod && strcmp(prev_mod, curr_mod) && > (!strcmp(prev->name, "bpf") || > !strcmp(curr->name, "bpf"))) I suspect it can happen on any module boundary so better to handle it in a more general way. Thanks, Namhyung > > > prev->end = roundup(prev->end + 4096, 4096); > > else > > prev->end = curr->start; > > -- > > 2.43.0.687.g38aa6559b0-goog > >
On Tue, Feb 13, 2024 at 10:48:53AM -0800, Namhyung Kim wrote: > Hi Leo, > > Thanks for your review! > > On Mon, Feb 12, 2024 at 7:40???PM Leo Yan <leo.yan@linux.dev> wrote: > > > > On Mon, Feb 12, 2024 at 03:33:22PM -0800, Namhyung Kim wrote: > > > I got a strange error on ARM to fail on processing FINISHED_ROUND > > > record. It turned out that it was failing in symbol__alloc_hist() > > > because the symbol size is too big. > > > > > > When a sample is captured on a specific BPF program, it failed. I've > > > added a debug code and found the end address of the symbol is from > > > the next module which is placed far way. > > > > > > ffff800008795778-ffff80000879d6d8: bpf_prog_1bac53b8aac4bc58_netcg_sock [bpf] > > > ffff80000879d6d8-ffff80000ad656b4: bpf_prog_76867454b5944e15_netcg_getsockopt [bpf] > > > ffff80000ad656b4-ffffd69b7af74048: bpf_prog_1d50286d2eb1be85_hn_egress [bpf] <---------- here > > > ffffd69b7af74048-ffffd69b7af74048: $x.5 [sha3_generic] > > > ffffd69b7af74048-ffffd69b7af740b8: crypto_sha3_init [sha3_generic] > > > ffffd69b7af740b8-ffffd69b7af741e0: crypto_sha3_update [sha3_generic] > > > > > > The logic in symbols__fixup_end() just uses curr->start to update the > > > prev->end. But in this case, it won't work as it's too different. > > > > > > I think ARM has a different kernel memory layout for modules and BPF > > > than on x86. Actually there's a logic to handle kernel and module > > > boundary. Let's do the same for symbols between different modules. > > > > Even Arm32 and Arm64 kernel have different memory layout for modules > > and kernel image. > > > > eBPF program (JITed) should be allocated from the vmalloc region, for > > Arm64, see bpf_jit_alloc_exec() in arch/arm64/net/bpf_jit_comp.c. > > Ok, so chances are they can fall out far away right? Yes, this is my understanding. > > > Signed-off-by: Namhyung Kim <namhyung@kernel.org> > > > --- > > > tools/perf/util/symbol.c | 21 +++++++++++++++++++-- > > > 1 file changed, 19 insertions(+), 2 deletions(-) > > > > > > diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c > > > index 35975189999b..9ebdb8e13c0b 100644 > > > --- a/tools/perf/util/symbol.c > > > +++ b/tools/perf/util/symbol.c > > > @@ -248,14 +248,31 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms) > > > * segment is very big. Therefore do not fill this gap and do > > > * not assign it to the kernel dso map (kallsyms). > > > * > > > + * Also BPF code can be allocated separately from text segments > > > + * and modules. So the last entry in a module should not fill > > > + * the gap too. > > > + * > > > * In kallsyms, it determines module symbols using '[' character > > > * like in: > > > * ffffffffc1937000 T hdmi_driver_init [snd_hda_codec_hdmi] > > > */ > > > if (prev->end == prev->start) { > > > + const char *prev_mod; > > > + const char *curr_mod; > > > + > > > + if (!is_kallsyms) { > > > + prev->end = curr->start; > > > + continue; > > > + } > > > + > > > + prev_mod = strchr(prev->name, '['); > > > + curr_mod = strchr(curr->name, '['); > > > + > > > /* Last kernel/module symbol mapped to end of page */ > > > - if (is_kallsyms && (!strchr(prev->name, '[') != > > > - !strchr(curr->name, '['))) > > > + if (!prev_mod != !curr_mod) > > > + prev->end = roundup(prev->end + 4096, 4096); > > > + /* Last symbol in the previous module */ > > > + else if (prev_mod && strcmp(prev_mod, curr_mod)) > > > > Should two consecutive moudles fall into this case? I think we need to assign > > 'prev->end = curr->start' for two two consecutive moudles. > > Yeah I thought about that case but I believe they would be on > separate pages (hopefully there's a page gap between them). > So I think it should not overlap. But if you really care we can > check it explicitly like this: > > prev->end = min(roundup(...), curr->start); I am not concerned that to assign a bigger end value for the 'prev' symbol. With an exaggerate end region, it will not cause any difficulty for parsing symbols. On the other hand, I am a bit concern for a big function (e.g. its code size > 4KiB), we might fail to find symbols in this case with the change above. > > If so, we should use a specific checking for eBPF program, e.g.: > > > > else if (prev_mod && strcmp(prev_mod, curr_mod) && > > (!strcmp(prev->name, "bpf") || > > !strcmp(curr->name, "bpf"))) > > I suspect it can happen on any module boundary so better > to handle it in a more general way. I don't want to introduce over complexity at here. We can apply current patch as it is. A side topic, when I saw the code is hard coded for 4096 as the page size, this is not always true on Arm64 (the page size can be 4KiB, 16KiB or 64KiB). We need to consider to extend the environment for recording the system's page size. Thanks, Leo > Thanks, > Namhyung > > > > > > prev->end = roundup(prev->end + 4096, 4096); > > > else > > > prev->end = curr->start; > > > -- > > > 2.43.0.687.g38aa6559b0-goog > > >
On Wed, Feb 14, 2024 at 2:14 AM Leo Yan <leo.yan@linux.dev> wrote: > > On Tue, Feb 13, 2024 at 10:48:53AM -0800, Namhyung Kim wrote: > > Hi Leo, > > > > Thanks for your review! > > > > On Mon, Feb 12, 2024 at 7:40???PM Leo Yan <leo.yan@linux.dev> wrote: > > > > > > On Mon, Feb 12, 2024 at 03:33:22PM -0800, Namhyung Kim wrote: > > > > I got a strange error on ARM to fail on processing FINISHED_ROUND > > > > record. It turned out that it was failing in symbol__alloc_hist() > > > > because the symbol size is too big. > > > > > > > > When a sample is captured on a specific BPF program, it failed. I've > > > > added a debug code and found the end address of the symbol is from > > > > the next module which is placed far way. > > > > > > > > ffff800008795778-ffff80000879d6d8: bpf_prog_1bac53b8aac4bc58_netcg_sock [bpf] > > > > ffff80000879d6d8-ffff80000ad656b4: bpf_prog_76867454b5944e15_netcg_getsockopt [bpf] > > > > ffff80000ad656b4-ffffd69b7af74048: bpf_prog_1d50286d2eb1be85_hn_egress [bpf] <---------- here > > > > ffffd69b7af74048-ffffd69b7af74048: $x.5 [sha3_generic] > > > > ffffd69b7af74048-ffffd69b7af740b8: crypto_sha3_init [sha3_generic] > > > > ffffd69b7af740b8-ffffd69b7af741e0: crypto_sha3_update [sha3_generic] > > > > > > > > The logic in symbols__fixup_end() just uses curr->start to update the > > > > prev->end. But in this case, it won't work as it's too different. > > > > > > > > I think ARM has a different kernel memory layout for modules and BPF > > > > than on x86. Actually there's a logic to handle kernel and module > > > > boundary. Let's do the same for symbols between different modules. > > > > > > Even Arm32 and Arm64 kernel have different memory layout for modules > > > and kernel image. > > > > > > eBPF program (JITed) should be allocated from the vmalloc region, for > > > Arm64, see bpf_jit_alloc_exec() in arch/arm64/net/bpf_jit_comp.c. > > > > Ok, so chances are they can fall out far away right? > > Yes, this is my understanding. > > > > > Signed-off-by: Namhyung Kim <namhyung@kernel.org> > > > > --- > > > > tools/perf/util/symbol.c | 21 +++++++++++++++++++-- > > > > 1 file changed, 19 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c > > > > index 35975189999b..9ebdb8e13c0b 100644 > > > > --- a/tools/perf/util/symbol.c > > > > +++ b/tools/perf/util/symbol.c > > > > @@ -248,14 +248,31 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms) > > > > * segment is very big. Therefore do not fill this gap and do > > > > * not assign it to the kernel dso map (kallsyms). > > > > * > > > > + * Also BPF code can be allocated separately from text segments > > > > + * and modules. So the last entry in a module should not fill > > > > + * the gap too. > > > > + * > > > > * In kallsyms, it determines module symbols using '[' character > > > > * like in: > > > > * ffffffffc1937000 T hdmi_driver_init [snd_hda_codec_hdmi] > > > > */ > > > > if (prev->end == prev->start) { > > > > + const char *prev_mod; > > > > + const char *curr_mod; > > > > + > > > > + if (!is_kallsyms) { > > > > + prev->end = curr->start; > > > > + continue; > > > > + } > > > > + > > > > + prev_mod = strchr(prev->name, '['); > > > > + curr_mod = strchr(curr->name, '['); > > > > + > > > > /* Last kernel/module symbol mapped to end of page */ > > > > - if (is_kallsyms && (!strchr(prev->name, '[') != > > > > - !strchr(curr->name, '['))) > > > > + if (!prev_mod != !curr_mod) > > > > + prev->end = roundup(prev->end + 4096, 4096); > > > > + /* Last symbol in the previous module */ > > > > + else if (prev_mod && strcmp(prev_mod, curr_mod)) > > > > > > Should two consecutive moudles fall into this case? I think we need to assign > > > 'prev->end = curr->start' for two two consecutive moudles. > > > > Yeah I thought about that case but I believe they would be on > > separate pages (hopefully there's a page gap between them). > > So I think it should not overlap. But if you really care we can > > check it explicitly like this: > > > > prev->end = min(roundup(...), curr->start); > > I am not concerned that to assign a bigger end value for the 'prev' > symbol. With an exaggerate end region, it will not cause any > difficulty for parsing symbols. Right, but my problem was not in parsing. It failed to allocate memory for the symbol because it's too big. > On the other hand, I am a bit concern > for a big function (e.g. its code size > 4KiB), we might fail to find > symbols in this case with the change above. Yes, it's another problem. But it cannot know the exact size so it just assumes it fits in a page. > > > > If so, we should use a specific checking for eBPF program, e.g.: > > > > > > else if (prev_mod && strcmp(prev_mod, curr_mod) && > > > (!strcmp(prev->name, "bpf") || > > > !strcmp(curr->name, "bpf"))) > > > > I suspect it can happen on any module boundary so better > > to handle it in a more general way. > > I don't want to introduce over complexity at here. We can apply > current patch as it is. Good, can I get your Reviewed-by then? :) > > A side topic, when I saw the code is hard coded for 4096 as the page > size, this is not always true on Arm64 (the page size can be 4KiB, > 16KiB or 64KiB). We need to consider to extend the environment for > recording the system's page size. Sounds good. But until then, 4K would be the reasonable choice. Thanks, Namhyung > > > > > > > > > prev->end = roundup(prev->end + 4096, 4096); > > > > else > > > > prev->end = curr->start; > > > > -- > > > > 2.43.0.687.g38aa6559b0-goog > > > >
On Thu, Feb 15, 2024 at 09:19:51PM -0800, Namhyung Kim wrote: [...] > > On the other hand, I am a bit concern > > for a big function (e.g. its code size > 4KiB), we might fail to find > > symbols in this case with the change above. > > Yes, it's another problem. But it cannot know the exact size > so it just assumes it fits in a page. Agreed. > > > > If so, we should use a specific checking for eBPF program, e.g.: > > > > > > > > else if (prev_mod && strcmp(prev_mod, curr_mod) && > > > > (!strcmp(prev->name, "bpf") || > > > > !strcmp(curr->name, "bpf"))) > > > > > > I suspect it can happen on any module boundary so better > > > to handle it in a more general way. > > > > I don't want to introduce over complexity at here. We can apply > > current patch as it is. > > Good, can I get your Reviewed-by then? :) Yes. Reviewed-by: Leo Yan <leo.yan@linux.dev> > > A side topic, when I saw the code is hard coded for 4096 as the page > > size, this is not always true on Arm64 (the page size can be 4KiB, > > 16KiB or 64KiB). We need to consider to extend the environment for > > recording the system's page size. > > Sounds good. But until then, 4K would be the reasonable choice. This is fine for me. Thanks, Leo
On Mon, 12 Feb 2024 15:33:22 -0800, Namhyung Kim wrote: > I got a strange error on ARM to fail on processing FINISHED_ROUND > record. It turned out that it was failing in symbol__alloc_hist() > because the symbol size is too big. > > When a sample is captured on a specific BPF program, it failed. I've > added a debug code and found the end address of the symbol is from > the next module which is placed far way. > > [...] Applied to perf-tools-next, thanks! [1/1] perf tools: Fixup module symbol end address properly commit: bacefe0c7b77b7527a613e053b6d378412a8a779 Best regards,
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c index 35975189999b..9ebdb8e13c0b 100644 --- a/tools/perf/util/symbol.c +++ b/tools/perf/util/symbol.c @@ -248,14 +248,31 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms) * segment is very big. Therefore do not fill this gap and do * not assign it to the kernel dso map (kallsyms). * + * Also BPF code can be allocated separately from text segments + * and modules. So the last entry in a module should not fill + * the gap too. + * * In kallsyms, it determines module symbols using '[' character * like in: * ffffffffc1937000 T hdmi_driver_init [snd_hda_codec_hdmi] */ if (prev->end == prev->start) { + const char *prev_mod; + const char *curr_mod; + + if (!is_kallsyms) { + prev->end = curr->start; + continue; + } + + prev_mod = strchr(prev->name, '['); + curr_mod = strchr(curr->name, '['); + /* Last kernel/module symbol mapped to end of page */ - if (is_kallsyms && (!strchr(prev->name, '[') != - !strchr(curr->name, '['))) + if (!prev_mod != !curr_mod) + prev->end = roundup(prev->end + 4096, 4096); + /* Last symbol in the previous module */ + else if (prev_mod && strcmp(prev_mod, curr_mod)) prev->end = roundup(prev->end + 4096, 4096); else prev->end = curr->start;