From patchwork Sun Dec 11 03:10:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 32129 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1600966wrr; Sat, 10 Dec 2022 19:35:00 -0800 (PST) X-Google-Smtp-Source: AA0mqf5jOkRB1Muu1eGWuWSLFo+NfVbv6U9l4O8zAs8meHJKg1YqIrA5z2FDY1WDtlSoDbXsD5HS X-Received: by 2002:aa7:9289:0:b0:56c:318a:f83b with SMTP id j9-20020aa79289000000b0056c318af83bmr12272341pfa.13.1670729700520; Sat, 10 Dec 2022 19:35:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670729700; cv=none; d=google.com; s=arc-20160816; b=CnxfDqOpKwkURZLjFexxBYrbBEr2o/yXQV9Uca09kLTV06Vf09mO/LRC2+nWZJxIPN ZIAc8IjeYEKkq/gtQxI1rwAtH8nRrjdMrvJrsBlEyBpiN0CNJVIDm12mA9Pjvo7LBplU XN/9RJkSzBS1vm/lT0MTLikShIAJAxJCbLo+7A9aGYqK/NuFc+nNNlVqRjgm5ieHndWS 6mLjtfQtwcKN6m84m6GaLtK6GMCQX0JnQZAdDJjy4KdDM+XcJolZ7VT2ZXf3CLeXmwM7 jNvktSI+XmcuA/9htu93NjF3TLO7+tS+Qwoj55wYoMaWgHDItWai02+Gb+QhkPdTIhE7 xWxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=XBPs/6ZjcVU+cocuvAp9Nuc/5hji6VQfW67ilcoTbVM=; b=o7C/ILUv32ISBdY3b1VIsY+atmKBPyjIfjZN1dxwvlNR8u3nHE25rUB7bnOaPgrN1w ZXKDZCWKiRZBa+dxBYdQd0/0ula35205yByKJJVNx6ljzXV4SgOyelDP3ZUYFafKYvrp BnUVTl2dQbiiRjo3u1W8j5NI/j+Vzm7f+xj2VWKv0gQAPXfrg9neoZ+PwN0FF1kxF/Y9 f7YeQSpCKqRGM9a7PW2/7X/HeiVTruX4N6ymOn8TkFz0wBtU9Lawa+gF8jXeHc9TmpEo UzVs/2KZlM7innrlqAeov8WetwArPE0rK7ygYe32QIGFZJeezUQbFS89UtZ8iZaUJR+J RbHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=aJsAimix; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g21-20020a056a001a1500b00575ff07cb27si6239053pfv.219.2022.12.10.19.34.45; Sat, 10 Dec 2022 19:35:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=aJsAimix; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229876AbiLKDLT (ORCPT + 99 others); Sat, 10 Dec 2022 22:11:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229538AbiLKDLR (ORCPT ); Sat, 10 Dec 2022 22:11:17 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C50A10FD2; Sat, 10 Dec 2022 19:11:15 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 56E7DB8085D; Sun, 11 Dec 2022 03:11:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B50E5C433EF; Sun, 11 Dec 2022 03:11:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1670728272; bh=y6YN1xorqFcEkOtQF4dW8ivbVjRCnyP1cx4XYgFc950=; h=From:To:Cc:Subject:Date:From; b=aJsAimixltAg2cIsAs7NpzfJxVKzVk/ty2U+C2NvRw1K+7W5hBdfomjF8fjYGCml5 Nh8YRhVF5+LFF+KRDr6/9lMhgZgALxB5lIN5mfvmaNmHcQstMdp/6RuZzflTCGlpZq Pzsvdmvgsgk7zVkbe51MkKAccjU4uF+MMgE9X1de3SUI+L5zJUGCGzbWCHQSgOrf2N h0z3VMPqvrCs1af+YLqSbos0wd5HvDsw/xrwVEgMgPlatQAQTire7rQYLMPTGbodvl lI/GOQCVxng4p94+YnaooVHdcAj/eEWQ4uHP+qJQNiNmgJZgLsPPTOo7upNbziP8ng kvDKqPa2lJ5IQ== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Masahiro Yamada , Nathan Chancellor , Nick Desaulniers , Nicolas Schier Subject: [PATCH] kbuild: use .NOTINTERMEDIATE for future GNU Make versions Date: Sun, 11 Dec 2022 12:10:59 +0900 Message-Id: <20221211031059.2623781-1-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751887066628439184?= X-GMAIL-MSGID: =?utf-8?q?1751887066628439184?= In Kbuild, some files are generated by chains of pattern/implicit rules. For example, *.dtb.o files in drivers/of/unittest-data/Makefile are generated by the chain of 3 pattern rules, like this: %.dts -> %.dtb -> %.dtb.S -> %.dtb.o Here, %.dts is the real source, %.dtb.o is the final target. %.dtb and %.dtb.S are called "intermediate files". As GNU Make manual [1] says, intermediate files are treated differently in two ways: (a) The first difference is what happens if the intermediate file does not exist. If an ordinary file 'b' does not exist, and make considers a target that depends on 'b', it invariably creates 'b' and then updates the target from 'b'. But if 'b' is an intermediate file, then make can leave well enough alone: it won't create 'b' unless one of its prerequisites is out of date. This means the target depending on 'b' won't be rebuilt either, unless there is some other reason to update that target: for example the target doesn't exist or a different prerequisite is newer than the target. (b) The second difference is that if make does create 'b' in order to update something else, it deletes 'b' later on after it is no longer needed. Therefore, an intermediate file which did not exist before make also does not exist after make. make reports the deletion to you by printing a 'rm' command showing which file it is deleting. Actually, (b) is problematic for Kbuild because most of the build rules depend on FORCE and the if_changed* macros really determine if the target should be updated. So, all missing files, whether they are intermediate or not, are always rebuilt. To see why (b) is a problem, delete ".SECONDARY:" from scripts/Kbuild.include, and repeat this command: $ make allmodconfig drivers/of/unittest-data/ The intermediate files will be deleted, which results in rebuilding intermediate and final objects in the next run of make. In the old days, people suppressed (b) in inconsistent ways. As commit 54a702f70589 ("kbuild: mark $(targets) as .SECONDARY and remove .PRECIOUS markers") noted, you should not use .PRECIOUS because .PRECIOUS has the following behavior (c), which is not likely what you want. (c) If make is killed or interrupted during the execution of their recipes, the target is not deleted. Also, the target is not deleted on error even if .DELETE_ON_ERROR is specified. .SECONDARY is a much better way to disable (b), but a small problem is that .SECONDARY enables (a), which gives a side-effect to $?; prerequisites marked as .SECONDARY do not appear in $?. This is a drawback for Kbuild. I thought it was a bug and opened a bug report. As Paul, the GNU Make maintainer, concluded in [2], this is not a bug. A good news is that, GNU Make 4.4 added the perfect solution, .NOTINTERMEDIATE, which cancels both (a) and (b). For clarificaton, my understanding of .INTERMEDIATE, .SECONDARY, .PRECIOUS and .NOTINTERMEDIATE are as follows: (a) (b) (c) .INTERMEDIATE enable enable disable .SECONDARY enable disable disable .PRECIOUS disable disable enable .NOTINTERMEDIATE disable disable disable However, GNU Make 4.4 has a bug for the global .NOTINTERMEDIATE. [3] It was fixed by commit 6164608900ad ("[SV 63417] Ensure global .NOTINTERMEDIATE disables all intermediates"), and will be available in the next release of GNU Make. The following is the gain for .NOTINTERMEDIATE: [Current Make] $ make allnoconfig vmlinux [ full build ] $ rm include/linux/device.h $ make vmlinux CALL scripts/checksyscalls.sh Make does not notice the removal of . [Future Make] $ make-latest allnoconfig vmlinux [ full build ] $ rm include/linux/device.h $ make-latest vmlinux CC arch/x86/kernel/asm-offsets.s In file included from ./include/linux/writeback.h:13, from ./include/linux/memcontrol.h:22, from ./include/linux/swap.h:9, from ./include/linux/suspend.h:5, from arch/x86/kernel/asm-offsets.c:13: ./include/linux/blk_types.h:11:10: fatal error: linux/device.h: No such file or directory 11 | #include | ^~~~~~~~~~~~~~~~ compilation terminated. make-latest[1]: *** [scripts/Makefile.build:114: arch/x86/kernel/asm-offsets.s] Error 1 make-latest: *** [Makefile:1282: prepare0] Error 2 Make notices the removal of , and rebuilds objects that depended on . There exists a source file that includes , and it raises an error. To see detailed background information, refer to commit 2d3b1b8f0da7 ("kbuild: drop $(wildcard $^) check in if_changed* for faster rebuild"). [1]: https://www.gnu.org/software/make/manual/make.html#Chained-Rules [2]: https://savannah.gnu.org/bugs/?55532 [3]: https://savannah.gnu.org/bugs/?63417 Signed-off-by: Masahiro Yamada Reviewed-by: Nicolas Schier --- scripts/Kbuild.include | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include index abdc269a51da..555c68788d09 100644 --- a/scripts/Kbuild.include +++ b/scripts/Kbuild.include @@ -185,9 +185,6 @@ endif make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1))))) # Find any prerequisites that are newer than target or that do not exist. -# (This is not true for now; $? should contain any non-existent prerequisites, -# but it does not work as expected when .SECONDARY is present. This seems a bug -# of GNU Make.) # PHONY targets skipped in both cases. newer-prereqs = $(filter-out $(PHONY),$?) @@ -263,4 +260,14 @@ endif .DELETE_ON_ERROR: # do not delete intermediate files automatically +# +# .NOTINTERMEDIATE is more correct, but only available on newer Make versions. +# Make 4.4 introduced .NOTINTERMEDIATE, and it appears in .FEATURES, but the +# global .NOTINTERMEDIATE does not work. We can use it on Make > 4.4. +# Use .SECONDARY for older Make versions, but "newer-prereq" cannot detect +# deleted files. +ifneq ($(and $(filter notintermediate, $(.FEATURES)),$(filter-out 4.4,$(MAKE_VERSION))),) +.NOTINTERMEDIATE: +else .SECONDARY: +endif