Message ID | 20230130183549.85471-1-mathieu.desnoyers@efficios.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2339675wrn; Mon, 30 Jan 2023 10:48:33 -0800 (PST) X-Google-Smtp-Source: AK7set913ceY8rXZhrhSjZorjzsy8Nc7nnHoun5VbGLmQGrrRna9TljtzQewqLliNotdw68uypO+ X-Received: by 2002:a17:906:8444:b0:879:ab3:2864 with SMTP id e4-20020a170906844400b008790ab32864mr11835730ejy.24.1675104513472; Mon, 30 Jan 2023 10:48:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675104513; cv=none; d=google.com; s=arc-20160816; b=aAqP5ZvOvak2TJXtS4JXrVzEfmn46CeKa+YNcY3dgnwEFj4rMTKaergSIr2whoqukS BS6ZF2Sk7T+3zEMdNKLi+LenuEuusI9bbOjY68CV9WAe6DywZij2KKhRI8y4fnA8aqFG Rl1J4E3Xt0Hu7+Z6qgNTJ0TWxxdoJZ3Ae5Q2QgmnCOm2ApQ8Tt7x9kwqDXi0iVoRcvZo U6w/3fzditmXnjjZ0ku3xDF8suvDBJRcazNGSMavXLzJCplvBRfk4SWCUpDxPL8rvJHA t7B0CZvKsgxiJwrqDye4fAaA/3Vc6D/mYeGgUxEgO83t6Ivuu1x/+JCqiY+r42BteX1q itlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Jd8BXtd0ZhXaaw6+RrTJSnhbdzlvW/uHPpxfL7wg5N4=; b=d8GR3ck+rB16I7uSTAR2CmKRkV/vKCB5uUv0o3CuKmy7FpfB769ijsGWeQfccFYPvQ 1dNl3xX1r3ofVhSOkF1gqON3LAuEHvLuXNSg542C+Eg5IpBTRN0REOzyvoMOk5c7/B2Y mUn5+huR3ibHN2XC885T1RV1vc8WzdzFZC2nhJsJMg+TwYDnBWGeatx5ZMq+GQsocVBM xacBEqGILCZQ39Zh+1RENfanrhG+Cl+ZyDJ3setMNbLM1a+HUPJ5rzilgUCsf1Z/Cn22 nf0EVeUN1uVfUtj62b1m9+MiB3gbXmwZj0JPyJ27+SmMk8/FO4iTU4toTtEnRo/8HiDU LVig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@efficios.com header.s=smtpout1 header.b=NgUHyCRB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y2-20020a170906518200b008775fc0de22si16569530ejk.246.2023.01.30.10.48.10; Mon, 30 Jan 2023 10:48:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@efficios.com header.s=smtpout1 header.b=NgUHyCRB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238102AbjA3SgJ (ORCPT <rfc822;maxin.john@gmail.com> + 99 others); Mon, 30 Jan 2023 13:36:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237895AbjA3SgI (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 30 Jan 2023 13:36:08 -0500 Received: from smtpout.efficios.com (unknown [IPv6:2607:5300:203:b2ee::31e5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A0CE26AD; Mon, 30 Jan 2023 10:35:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1675103755; bh=AYGBgN50Jr1DUDnzCE+Pzynb8j9uxfAHOfkP82pWKaU=; h=From:To:Cc:Subject:Date:From; b=NgUHyCRBAip4aOgELlG0MTM/tlyiZ0/5JnsMFENUMRIaoj7CahT30UZ4Fne4We8lK iFcxEqINtnPr9hTS3Q7Ctu3WOTEdYwhyULhXSEKoiXjZ2kXrZf5AhHa+E5NEqJzrX7 X1ogrYuSABJt/4dPrtleVeNSuWQ5ZPkR2bQXnk8JHjmkUB4NXTiCSIR6fs+le0Hkl1 XgqWCwi+uWMLjx3BehePK63VNjwgXqFDywMK1bWHxeCiIhwqumXuXH3jv9ndpFwE70 mC4fnML9NV+ydck5aVL3LqXK1MV9g9OjwMGyCppV4OHHVDszwDtzIVqRwCtMrTRU9c 9RD27jZG1QrPQ== Received: from localhost.localdomain (192-222-180-24.qc.cable.ebox.net [192.222.180.24]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4P5H1k6F44zhmX; Mon, 30 Jan 2023 13:35:54 -0500 (EST) From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> To: Alexei Starovoitov <ast@kernel.org> Cc: linux-kernel@vger.kernel.org, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Shuah Khan <skhan@linuxfoundation.org>, bpf@vger.kernel.org, Shuah Khan <shuah@kernel.org>, linux-kselftest@vger.kernel.org, Ingo Molnar <mingo@redhat.com>, Daniel Borkmann <daniel@iogearbox.net>, Andrii Nakryiko <andrii@kernel.org>, Martin KaFai Lau <martin.lau@linux.dev>, Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>, John Fastabend <john.fastabend@gmail.com>, KP Singh <kpsingh@kernel.org>, Stanislav Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>, Jiri Olsa <jolsa@kernel.org>, Mykola Lysenko <mykolal@fb.com>, stable@vger.kernel.org Subject: [PATCH bpf-next 1/2] selftests: bpf: Fix incorrect kernel headers search path Date: Mon, 30 Jan 2023 13:35:48 -0500 Message-Id: <20230130183549.85471-1-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RDNS_NONE,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756474390383459955?= X-GMAIL-MSGID: =?utf-8?q?1756474390383459955?= |
Series |
[bpf-next,1/2] selftests: bpf: Fix incorrect kernel headers search path
|
|
Commit Message
Mathieu Desnoyers
Jan. 30, 2023, 6:35 p.m. UTC
Use $(KHDR_INCLUDES) as lookup path for kernel headers. This prevents building against kernel headers from the build environment in scenarios where kernel headers are installed into a specific output directory (O=...). Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Acked-by: Shuah Khan <skhan@linuxfoundation.org> Cc: <bpf@vger.kernel.org> Cc: Shuah Khan <shuah@kernel.org> Cc: linux-kselftest@vger.kernel.org Cc: Ingo Molnar <mingo@redhat.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Song Liu <song@kernel.org> Cc: Yonghong Song <yhs@fb.com> Cc: John Fastabend <john.fastabend@gmail.com> Cc: KP Singh <kpsingh@kernel.org> Cc: Stanislav Fomichev <sdf@google.com> Cc: Hao Luo <haoluo@google.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Mykola Lysenko <mykolal@fb.com> Cc: <stable@vger.kernel.org> [5.18+] --- tools/testing/selftests/bpf/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Comments
On Mon, Jan 30, 2023 at 10:36 AM Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > > Use $(KHDR_INCLUDES) as lookup path for kernel headers. This prevents > building against kernel headers from the build environment in scenarios > where kernel headers are installed into a specific output directory > (O=...). > > Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> > Acked-by: Shuah Khan <skhan@linuxfoundation.org> > Cc: <bpf@vger.kernel.org> > Cc: Shuah Khan <shuah@kernel.org> > Cc: linux-kselftest@vger.kernel.org > Cc: Ingo Molnar <mingo@redhat.com> > Cc: Alexei Starovoitov <ast@kernel.org> > Cc: Daniel Borkmann <daniel@iogearbox.net> > Cc: Andrii Nakryiko <andrii@kernel.org> > Cc: Martin KaFai Lau <martin.lau@linux.dev> > Cc: Song Liu <song@kernel.org> > Cc: Yonghong Song <yhs@fb.com> > Cc: John Fastabend <john.fastabend@gmail.com> > Cc: KP Singh <kpsingh@kernel.org> > Cc: Stanislav Fomichev <sdf@google.com> > Cc: Hao Luo <haoluo@google.com> > Cc: Jiri Olsa <jolsa@kernel.org> > Cc: Mykola Lysenko <mykolal@fb.com> > Cc: <stable@vger.kernel.org> [5.18+] > --- > tools/testing/selftests/bpf/Makefile | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile > index c22c43bbee19..6998c816afef 100644 > --- a/tools/testing/selftests/bpf/Makefile > +++ b/tools/testing/selftests/bpf/Makefile > @@ -327,7 +327,7 @@ endif > CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) > BPF_CFLAGS = -g -Werror -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) \ > -I$(INCLUDE_DIR) -I$(CURDIR) -I$(APIDIR) \ > - -I$(abspath $(OUTPUT)/../usr/include) > + $(KHDR_INCLUDES) It breaks the build: https://github.com/kernel-patches/bpf/actions/runs/4047075637/jobs/6960655246 make[1]: *** No rule to make target '/linux/bpf.h', needed by '/tmp/work/bpf/bpf/tools/testing/selftests/bpf/bpf-helpers.rst'. Stop.
On 2023-01-30 16:03, Alexei Starovoitov wrote: > On Mon, Jan 30, 2023 at 10:36 AM Mathieu Desnoyers > <mathieu.desnoyers@efficios.com> wrote: >> >> Use $(KHDR_INCLUDES) as lookup path for kernel headers. This prevents >> building against kernel headers from the build environment in scenarios >> where kernel headers are installed into a specific output directory >> (O=...). >> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> >> Acked-by: Shuah Khan <skhan@linuxfoundation.org> >> Cc: <bpf@vger.kernel.org> >> Cc: Shuah Khan <shuah@kernel.org> >> Cc: linux-kselftest@vger.kernel.org >> Cc: Ingo Molnar <mingo@redhat.com> >> Cc: Alexei Starovoitov <ast@kernel.org> >> Cc: Daniel Borkmann <daniel@iogearbox.net> >> Cc: Andrii Nakryiko <andrii@kernel.org> >> Cc: Martin KaFai Lau <martin.lau@linux.dev> >> Cc: Song Liu <song@kernel.org> >> Cc: Yonghong Song <yhs@fb.com> >> Cc: John Fastabend <john.fastabend@gmail.com> >> Cc: KP Singh <kpsingh@kernel.org> >> Cc: Stanislav Fomichev <sdf@google.com> >> Cc: Hao Luo <haoluo@google.com> >> Cc: Jiri Olsa <jolsa@kernel.org> >> Cc: Mykola Lysenko <mykolal@fb.com> >> Cc: <stable@vger.kernel.org> [5.18+] >> --- >> tools/testing/selftests/bpf/Makefile | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile >> index c22c43bbee19..6998c816afef 100644 >> --- a/tools/testing/selftests/bpf/Makefile >> +++ b/tools/testing/selftests/bpf/Makefile >> @@ -327,7 +327,7 @@ endif >> CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) >> BPF_CFLAGS = -g -Werror -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) \ >> -I$(INCLUDE_DIR) -I$(CURDIR) -I$(APIDIR) \ >> - -I$(abspath $(OUTPUT)/../usr/include) >> + $(KHDR_INCLUDES) > > It breaks the build: > https://github.com/kernel-patches/bpf/actions/runs/4047075637/jobs/6960655246 > > make[1]: *** No rule to make target '/linux/bpf.h', needed by > '/tmp/work/bpf/bpf/tools/testing/selftests/bpf/bpf-helpers.rst'. Stop. I finally figured out why I did not catch it in my own testing: it appears that the bpf selftest is special: it's not built when issuing "make" from tools/testing/selftests/. Perhaps that's also why Shuah did not see any issues with my bpf patches: tools/testing/selftests/Makefile: # User can optionally provide a TARGETS skiplist. By default we skip # BPF since it has cutting edge build time dependencies which require # more effort to install. SKIP_TARGETS ?= bpf It appears that the baseline bpf selftest on v6.2-rc6 does not build on my system: CLNG-BPF [test_maps] test_bpf_nf.bpf.o progs/test_bpf_nf.c:156:7: error: no member named 'mark' in 'struct nf_conn' ct->mark = 77; ~~ ^ progs/test_bpf_nf.c:160:42: error: use of undeclared identifier 'NF_NAT_MANIP_SRC' bpf_ct_set_nat_info(ct, &saddr, sport, NF_NAT_MANIP_SRC); ^ progs/test_bpf_nf.c:163:42: error: use of undeclared identifier 'NF_NAT_MANIP_DST' bpf_ct_set_nat_info(ct, &daddr, dport, NF_NAT_MANIP_DST); ^ progs/test_bpf_nf.c:187:38: error: no member named 'mark' in 'struct nf_conn' test_insert_lookup_mark = ct_lk->mark; ~~~~~ ^ progs/test_bpf_nf.c:189:12: error: use of undeclared identifier 'IPS_CONFIRMED' IPS_CONFIRMED | IPS_SEEN_REPLY); ^ progs/test_bpf_nf.c:189:28: error: use of undeclared identifier 'IPS_SEEN_REPLY' IPS_CONFIRMED | IPS_SEEN_REPLY); ^ progs/test_bpf_nf.c:209:11: error: no member named 'mark' in 'struct nf_conn' if (ct->mark == 42) { ~~ ^ progs/test_bpf_nf.c:210:8: error: no member named 'mark' in 'struct nf_conn' ct->mark++; ~~ ^ progs/test_bpf_nf.c:211:33: error: no member named 'mark' in 'struct nf_conn' test_exist_lookup_mark = ct->mark; ~~ ^ 9 errors generated. Based on this: https://lore.kernel.org/bpf/37649bee-5eb3-93a2-ac57-56eb375ef8cd@iogearbox.net/ It appears that NF_CONNTRACK_MARK should be set to =y. Perhaps it should be detected at selftest build time and skipped rather than failing to build ? It appears to be a case where a user-space selftest depends on internal kernel data structures: Ref. include/net/netfilter/nf_conntrack.h:struct nf_conn After setting this config option, it fails at: GEN-SKEL [test_progs] test_kfunc_dynptr_param.skel.h libbpf: sec '?lsm.s/bpf': corrupted program 'not_ptr_to_stack', offset 48, size 0 Error: failed to open BPF object file: Invalid argument make: *** [Makefile:546: /home/efficios/git/linux/tools/testing/selftests/bpf/test_kfunc_dynptr_param.skel.h] Error 234 make: *** Deleting file '/home/efficios/git/linux/tools/testing/selftests/bpf/test_kfunc_dynptr_param.skel.h' Based on this issue: https://github.com/libbpf/libbpf-bootstrap/issues/12 it appears that bpf selftests are only meant to be used against a set of pre-specified kernel configurations. README.rst confirms that. So let's use tools/testing/selftests/bpf/config.x86_64 with make olddefconfig for my kernel build then. It would have been less unexpected for the bpf selftests to report incorrect or missing kernel config options rather than limiting the supported set to specific configuration files provided by the bpf selftests. This limits the testing coverage to very few kernel configurations. It still does not work. I need to explicitly enable BPF_SYSCALL=y which is not present in the bpf selftests config.x86_64. Then I can explicitly reenable DEBUG_INFO_BTF=y. It appears that config.x86_64 is outdated in the bpf selftests. It also misses NF_CONNTRACK_MARK=y. So let's assume that config.x86_64 is outdated and broken and go with tools/testing/selftests/bpf/config followed by make olddefconfig. No more luck, this time the bpf kernel config misses the DEBUG_KERNEL=y, which is needed for DEBUG_INFO and DEBUG_INFO_BTF. Now it's: CLANG /home/efficios/git/linux/tools/testing/selftests/bpf/tools/build/bpftool/profiler.bpf.o skeleton/profiler.bpf.c:18:21: error: invalid application of 'sizeof' to an incomplete type 'struct bpf_perf_event_value' __uint(value_size, sizeof(struct bpf_perf_event_value)); [...] And that's where I give up. Has anyone else succeeded in building bpf selftests locally ? Thanks, Mathieu
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index c22c43bbee19..6998c816afef 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -327,7 +327,7 @@ endif CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) BPF_CFLAGS = -g -Werror -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) \ -I$(INCLUDE_DIR) -I$(CURDIR) -I$(APIDIR) \ - -I$(abspath $(OUTPUT)/../usr/include) + $(KHDR_INCLUDES) CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \ -Wno-compare-distinct-pointer-types