From patchwork Tue Oct 17 10:37:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 154056 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp4039938vqb; Tue, 17 Oct 2023 03:38:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGjui678Z6i3iMzW1/2+B0KCX6ORumoVta2HXjOGAODDaYE0VZnLkjLUziS2mjqDmmS8Fhe X-Received: by 2002:a05:6359:5e20:b0:166:d93e:339 with SMTP id pw32-20020a0563595e2000b00166d93e0339mr1791408rwb.0.1697539114961; Tue, 17 Oct 2023 03:38:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697539114; cv=none; d=google.com; s=arc-20160816; b=uRrTd5M8Del/NuyUmr3SyNQz8R3IgFPNz/EegZMraZGf0B/TxJVuQqNsL8FEg8MuLw 26d/Qf5hqIS93qaQTPRBab6hX60S7CT6XEGQjEc9bxVb7U9ry/4R8Tgtc/kqZSszDmcF Amw6Qhrfo//WDgH2p14uP+uGUy/mzWQhGSt+lc3loDtFuP5krLpHmJQucgevWxzHX50u xdbmcDYQglE0Eb9eA8qpMrmJLRslW9jfjvayRjR9lOJFelto2E0UafMJ4lLdh988Os+p ovee+S/B1Ps7M1COkEZ8Ps0tvfWqLq7nXuMilISqISWTS/nQnFEBGnWFT5iiw6bYI8uJ okug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ajzPYkPzS+JKnfhwdhQyXEGy3tVX8Us7yT8vlF3e3qc=; fh=fDkx6bVHh3l/rA0E4892gjuP6gNEs3Z88QlOHrZLqpA=; b=auctRIuwrW6/dKL21n7W++EsBF6VvxlDDGzKQOUv2dObg5oqSJiHZFwE+nW8v0Ru83 i64vEVNcdlrdVmtoB7vCKuOZ/HZUeNyvYHp8cnIoLMZFKvCJCXhkwetMe7JuLEHTXB5J Z/CtR/Q4UhGEbsJF4C6MAFBSEhpR9BQVU2IZ6jOTfbHHZmwZ21NqLW8Qw/rt7JEC47D2 SqG90nw8twyEin6BOQV4qefGEao9iJ6XN8g84ILwNvrx4ZcsW+kM5J8mzaLpJnsUFCv/ AiENrHMCZ3od/dZMFguYWYJrJYdFsUrA8X1ofxrZyztR3Y0hyvr7nZNtDVaQ3DoOSf6R oJNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=k2ytQAQi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id u191-20020a6385c8000000b005b2d044af30si1421947pgd.480.2023.10.17.03.38.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Oct 2023 03:38:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=k2ytQAQi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id A18C78036846; Tue, 17 Oct 2023 03:38:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343652AbjJQKiS (ORCPT + 19 others); Tue, 17 Oct 2023 06:38:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234594AbjJQKiQ (ORCPT ); Tue, 17 Oct 2023 06:38:16 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F6EBA2; Tue, 17 Oct 2023 03:38:15 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 71C4EC433CA; Tue, 17 Oct 2023 10:38:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697539094; bh=A6qLcCYxkwgWjCL+CxEKfQPWGv0e7JNIKXIZW89MeuU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k2ytQAQi1GJKjYWyS2S00/+UNBIvCCyZOAFkCE9TFt+3ZueDC4OLAjybipTBO++p7 lSznrtrnBtPgpvlmBNt6OM2fAUeOwQxj4dQNjqNvWKTX+DMCwS4qNiSkTiXcgF7iXX IVWyV/5vgO1OM9YzlILfH8VzLtz/pCA9hqXZrvwDjhxZ+F6gUUJP9/OIEhYH1ElMue dFHIhx2GPCHJTtAQeuqceZ02h0gP8+H7GgB8vFa1SINkFD1W4N0B3HEMOSRPXrmsYA 6ACuJiALmbS0dKkNEiRZKjV1i4jhjjf/Y7/5bJXdH8bcAESPGvcIcVX73YsDJgg5gO Wsuoqa5OLtQBQ== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Masahiro Yamada , Alex Gaynor , Alexei Starovoitov , Alice Ryhl , Andreas Hindborg , Andrii Nakryiko , Benno Lossin , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Boqun Feng , Daniel Borkmann , Gary Guo , Hao Luo , Jiri Olsa , John Fastabend , KP Singh , Martin KaFai Lau , Miguel Ojeda , Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Song Liu , Stanislav Fomichev , Wedson Almeida Filho , Yonghong Song , bpf@vger.kernel.org, rust-for-linux@vger.kernel.org Subject: [PATCH 2/4] kbuild: avoid too many execution of scripts/pahole-flags.sh Date: Tue, 17 Oct 2023 19:37:40 +0900 Message-Id: <20231017103742.130927-2-masahiroy@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231017103742.130927-1-masahiroy@kernel.org> References: <20231017103742.130927-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 17 Oct 2023 03:38:30 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779998775013847069 X-GMAIL-MSGID: 1779998775013847069 scripts/pahole-flags.sh is executed so many times. You can check how many times it is invoked during the build, as follows: $ cat <> scripts/pahole-flags.sh > echo "scripts/pahole-flags.sh was executed" >&2 > EOF $ make -s scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed scripts/pahole-flags.sh was executed [ lots of repeated lines suppressed... ] This scripts is exectuted more than 20 times during the kernel build because PAHOLE_FLAGS is a recursively expanded variable and exported to sub-processes. With the GNU Make >= 4.4, it is executed more than 60 times because exported variables are also passed to other $(shell ) invocations. Without careful coding, it is known to cause an exponential fork explosion. [1] The use of $(shell ) in an exported recursive variable is likely wrong because $(shell ) is always evaluated due to the 'export' keyword, and the evaluation can occur multiple times by the nature of recursive variables. Convert the shell script to a Makefile, which is included only when CONFIG_DEBUG_INFO_BTF=y. [1]: https://savannah.gnu.org/bugs/index.php?64746 Signed-off-by: Masahiro Yamada Reviewed-by: Alan Maguire Tested-by: Alan Maguire Reviewed-by: Nicolas Schier Tested-by: Miguel Ojeda Acked-by: Miguel Ojeda Reviewed-by: Martin Rodriguez Reboredo --- Makefile | 4 +--- scripts/Makefile.btf | 19 +++++++++++++++++++ scripts/pahole-flags.sh | 30 ------------------------------ 3 files changed, 20 insertions(+), 33 deletions(-) create mode 100644 scripts/Makefile.btf delete mode 100755 scripts/pahole-flags.sh diff --git a/Makefile b/Makefile index fed9a6cc3665..eaddec67e5e1 100644 --- a/Makefile +++ b/Makefile @@ -513,8 +513,6 @@ LZ4 = lz4c XZ = xz ZSTD = zstd -PAHOLE_FLAGS = $(shell PAHOLE=$(PAHOLE) $(srctree)/scripts/pahole-flags.sh) - CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF) NOSTDINC_FLAGS := @@ -605,7 +603,6 @@ export KBUILD_RUSTFLAGS RUSTFLAGS_KERNEL RUSTFLAGS_MODULE export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_RUSTFLAGS_MODULE KBUILD_LDFLAGS_MODULE export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL KBUILD_RUSTFLAGS_KERNEL -export PAHOLE_FLAGS # Files to ignore in find ... statements @@ -1002,6 +999,7 @@ KBUILD_CPPFLAGS += $(call cc-option,-fmacro-prefix-map=$(srctree)/=) # include additional Makefiles when needed include-y := scripts/Makefile.extrawarn include-$(CONFIG_DEBUG_INFO) += scripts/Makefile.debug +include-$(CONFIG_DEBUG_INFO_BTF)+= scripts/Makefile.btf include-$(CONFIG_KASAN) += scripts/Makefile.kasan include-$(CONFIG_KCSAN) += scripts/Makefile.kcsan include-$(CONFIG_KMSAN) += scripts/Makefile.kmsan diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf new file mode 100644 index 000000000000..82377e470aed --- /dev/null +++ b/scripts/Makefile.btf @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: GPL-2.0 + +pahole-ver := $(CONFIG_PAHOLE_VERSION) +pahole-flags-y := + +# pahole 1.18 through 1.21 can't handle zero-sized per-CPU vars +ifeq ($(call test-le, $(pahole-ver), 121),y) +pahole-flags-$(call test-ge, $(pahole-ver), 118) += --skip_encoding_btf_vars +endif + +pahole-flags-$(call test-ge, $(pahole-ver), 121) += --btf_gen_floats + +pahole-flags-$(call test-ge, $(pahole-ver), 122) += -j + +pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust + +pahole-flags-$(call test-ge, $(pahole-ver), 125) += --skip_encoding_btf_inconsistent_proto --btf_gen_optimized + +export PAHOLE_FLAGS := $(pahole-flags-y) diff --git a/scripts/pahole-flags.sh b/scripts/pahole-flags.sh deleted file mode 100755 index 728d55190d97..000000000000 --- a/scripts/pahole-flags.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 - -extra_paholeopt= - -if ! [ -x "$(command -v ${PAHOLE})" ]; then - exit 0 -fi - -pahole_ver=$($(dirname $0)/pahole-version.sh ${PAHOLE}) - -if [ "${pahole_ver}" -ge "118" ] && [ "${pahole_ver}" -le "121" ]; then - # pahole 1.18 through 1.21 can't handle zero-sized per-CPU vars - extra_paholeopt="${extra_paholeopt} --skip_encoding_btf_vars" -fi -if [ "${pahole_ver}" -ge "121" ]; then - extra_paholeopt="${extra_paholeopt} --btf_gen_floats" -fi -if [ "${pahole_ver}" -ge "122" ]; then - extra_paholeopt="${extra_paholeopt} -j" -fi -if [ "${pahole_ver}" -ge "124" ]; then - # see PAHOLE_HAS_LANG_EXCLUDE - extra_paholeopt="${extra_paholeopt} --lang_exclude=rust" -fi -if [ "${pahole_ver}" -ge "125" ]; then - extra_paholeopt="${extra_paholeopt} --skip_encoding_btf_inconsistent_proto --btf_gen_optimized" -fi - -echo ${extra_paholeopt}