From patchwork Fri Feb 10 18:18:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 55513 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1105031wrn; Fri, 10 Feb 2023 10:19:29 -0800 (PST) X-Google-Smtp-Source: AK7set9f/j3dWzV3+J7q5Vpv8nc7M+sLiDqFKTX+iNH9UqCAu2u6nDS/VQqGUT54GtQsxevPMUco X-Received: by 2002:a50:a44e:0:b0:4aa:b2d3:eb2f with SMTP id v14-20020a50a44e000000b004aab2d3eb2fmr17193821edb.38.1676053169345; Fri, 10 Feb 2023 10:19:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676053169; cv=none; d=google.com; s=arc-20160816; b=RAc4tsTuL518d2f87Uvkv8BbwOzJ7WZ/4i4HwRlkUassTxN6mYAMbIcV5D31zxpc2s mA2xdhOVYboN+JisyX0Je4wfAWxRLqJJHcwWaMf/kauJS0c1yyhB4wyueAcGgR5nbl8f kAsO3Sq3yDyGhTGR0X7SJ9TuTZAI+DL42zbeWLvm1b/JR7v9UQzTniyVU2PiA4moC85M jKr1PKcgiPvhfwCX/Fk4Dch3S4SjtNw8wmz6OUiaxpMgNd29Gme7Ehx1+GQJG4N+BjGk 8OPgu492AR1hdqVtETXiBc6ylMkroAblDwqMXq8eQdZIE8ipkOgCbK2a/9KtOLTZsOu/ mu1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=6TAaexQjwmKNxSgiNjl94ipRyS5vsXk80NfJ55OQYFs=; b=ghuJeLUn84NJTlmasDGlUf3dCCW2rExDXnsx54TlGJW4rRajvO6KRiAzndtZi/Ej5x orL6kOkdXWtf6QrLosdUh+id3HXlqRa0LOjhrsr8F1BLyKz3+TxYiXaeo6PW38znyGrh Z9jqhBzHokYvow38Pv/HsrJOSzIXhGmr4X2GDgXRrFRlr9qRlTdw/aPdC5/4IXSTRjfg RpFAUUhop61tS+O1po6XYu7V5T359n3C7JAmsCYtejYGc68oFcsvVkTE+n5IFkXURcpE 6cNSGCiEn42IEQpwfN05NS24ndg2P4tgMOmNGgFchhkZVLamvmkizP45BgSZT25ODBzT C0FQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=R03kje0j; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h25-20020a50ed99000000b004abfa2a3794si1105483edr.606.2023.02.10.10.19.05; Fri, 10 Feb 2023 10:19:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=R03kje0j; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232919AbjBJSSl (ORCPT + 99 others); Fri, 10 Feb 2023 13:18:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231792AbjBJSSk (ORCPT ); Fri, 10 Feb 2023 13:18:40 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E37A5A915; Fri, 10 Feb 2023 10:18:37 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EEE6761E6E; Fri, 10 Feb 2023 18:18:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D86E8C433EF; Fri, 10 Feb 2023 18:18:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053116; bh=BYLc+fB3U+S1UlWOqC0+Ue4mYooTlr2zEt8AqNbw2DM=; h=From:To:Cc:Subject:Date:From; b=R03kje0jlPdSUfTdh5Uhs6h8HADpKAfOohyLyI0LkD+Bt2YnqHxudVVAqiSNnli9s 8DdEv+0UoWlIr2ZMLRuJROcYyT1ZYmlUqIhpznh8tIW7rCkdHTlGPR79eWxAATCucJ GHntQyxzR1emKtaq26iE9timbCfPP4OBaBGLnTYMDYSufmCW9rD7ging+jb4JM1JcV YMNnW7V6dVE5Z4ntDeyFDC9uhJ4UyiTVXOSt05CZvqD3512TvxD2ZBdbFTf6K2DFX+ zy7J27kopWaF2+VZb+fh/UstQu3mtAdFM/PUhOevULPKgazEx425dKqCAPjDVmBJcN rmalAnIDQlvJg== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada Subject: [PATCH v5 1/7] kbuild: add a tool to list files ignored by git Date: Sat, 11 Feb 2023 03:18:22 +0900 Message-Id: <20230210181828.124765-1-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757469127974314947?= X-GMAIL-MSGID: =?utf-8?q?1757469127974314947?= In short, the motivation of this commit is to build a source package without cleaning the source tree. The deb-pkg and (src)rpm-pkg targets first run 'make clean' before creating a source tarball. Otherwise build artifacts such as *.o, *.a, etc. would be included in the tarball. Yet, the tarball ends up containing several garbage files since 'make clean' does not clean everything. Cleaning the tree every time is annoying since it makes the incremental build impossible. It is desirable to create a source tarball without cleaning the tree. In fact, there are some ways to achieve this. The easiest solution is 'git archive'. 'make perf-tar*-src-pkg' uses it, but I do not like it because it works only when the source tree is managed by git, and all files you want in the tarball must be committed in advance. I want to make it work without relying on git. We can do this. Files that are ignored by git are generated files, so should be excluded from the source tarball. We can list them out by parsing the .gitignore files. Of course, .gitignore does not cover all the cases, but it works well enough. tar(1) claims to support it: --exclude-vcs-ignores Exclude files that match patterns read from VCS-specific ignore files. Supported files are: .cvsignore, .gitignore, .bzrignore, and .hgignore. The best scenario would be to use 'tar --exclude-vcs-ignores', but this option does not work. --exclude-vcs-ignore does not understand any of the negation (!), preceding slash, following slash, etc.. So, this option is just useless. Hence, I wrote this gitignore parser. The previous version [1], written in Python, was so slow. This version is implemented in C, so it works much faster. I imported the code from git, so we get the same result. This tool traverses the source tree, parsing all .gitignore files, and prints file paths that are ignored by git. The output is similar to 'git ls-files --ignored --directory --others --exclude-per-directory=.gitignore', except [1] Not sorted [2] No trailing slash for directories [2] is intentional because tar's --exclude-from option cannot handle trailing slashes. [How to test this tool] $ git clean -dfx $ make -s -j$(nproc) defconfig all # or allmodconifg or whatever $ git archive -o ../linux1.tar --prefix=./ HEAD $ tar tf ../linux1.tar | LANG=C sort > ../file-list1 # files emitted by 'git archive' $ make scripts_package HOSTCC scripts/list-gitignored $ scripts/list-gitignored --prefix=./ -o ../exclude-list $ tar cf ../linux2.tar --exclude-from=../exclude-list . $ tar tf ../linux2.tar | LANG=C sort > ../file-list2 # files emitted by 'tar' $ diff ../file-list1 ../file-list2 | grep -E '^(<|>)' < ./Documentation/devicetree/bindings/.yamllint < ./drivers/clk/.kunitconfig < ./drivers/gpu/drm/tests/.kunitconfig < ./drivers/hid/.kunitconfig < ./fs/ext4/.kunitconfig < ./fs/fat/.kunitconfig < ./kernel/kcsan/.kunitconfig < ./lib/kunit/.kunitconfig < ./mm/kfence/.kunitconfig < ./tools/testing/selftests/arm64/tags/ < ./tools/testing/selftests/arm64/tags/.gitignore < ./tools/testing/selftests/arm64/tags/Makefile < ./tools/testing/selftests/arm64/tags/run_tags_test.sh < ./tools/testing/selftests/arm64/tags/tags_test.c < ./tools/testing/selftests/kvm/.gitignore < ./tools/testing/selftests/kvm/Makefile < ./tools/testing/selftests/kvm/config < ./tools/testing/selftests/kvm/settings The source tarball contains most of files that are tracked by git. You see some diffs, but it is just because some .gitignore files are wrong. $ git ls-files -i -c --exclude-per-directory=.gitignore Documentation/devicetree/bindings/.yamllint drivers/clk/.kunitconfig drivers/gpu/drm/tests/.kunitconfig drivers/hid/.kunitconfig fs/ext4/.kunitconfig fs/fat/.kunitconfig kernel/kcsan/.kunitconfig lib/kunit/.kunitconfig mm/kfence/.kunitconfig tools/testing/selftests/arm64/tags/.gitignore tools/testing/selftests/arm64/tags/Makefile tools/testing/selftests/arm64/tags/run_tags_test.sh tools/testing/selftests/arm64/tags/tags_test.c tools/testing/selftests/kvm/.gitignore tools/testing/selftests/kvm/Makefile tools/testing/selftests/kvm/config tools/testing/selftests/kvm/settings [1]: https://lore.kernel.org/all/20230128173843.765212-1-masahiroy@kernel.org/ Signed-off-by: Masahiro Yamada --- Changes in v5: - Import a lot of code from GIT - Fix a bug that the output file is relative to --rootdir - Check ENOENT when open() fails to open .gitignore - add more options, --ignore-case, --stat Changes in v3: - Various code refactoring: remove struct gitignore, remove next: label etc. - Support --extra-pattern option Changes in v2: - Reimplement in C Makefile | 7 +- scripts/.gitignore | 1 + scripts/Makefile | 2 +- scripts/list-gitignored.c | 1074 +++++++++++++++++++++++++++++++++++++ 4 files changed, 1082 insertions(+), 2 deletions(-) create mode 100644 scripts/list-gitignored.c diff --git a/Makefile b/Makefile index 2faf872b6808..7c726fd26540 100644 --- a/Makefile +++ b/Makefile @@ -274,7 +274,8 @@ no-dot-config-targets := $(clean-targets) \ cscope gtags TAGS tags help% %docs check% coccicheck \ $(version_h) headers headers_% archheaders archscripts \ %asm-generic kernelversion %src-pkg dt_binding_check \ - outputmakefile rustavailable rustfmt rustfmtcheck + outputmakefile rustavailable rustfmt rustfmtcheck \ + scripts_package # Installation targets should not require compiler. Unfortunately, vdso_install # is an exception where build artifacts may be updated. This must be fixed. no-compiler-targets := $(no-dot-config-targets) install dtbs_install \ @@ -1652,6 +1653,10 @@ distclean: mrproper %pkg: include/config/kernel.release FORCE $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.package $@ +PHONY += scripts_package +scripts_package: scripts_basic + $(Q)$(MAKE) $(build)=scripts scripts/list-gitignored + # Brief documentation of the typical targets used # --------------------------------------------------------------------------- diff --git a/scripts/.gitignore b/scripts/.gitignore index 6e9ce6720a05..feb43045d1b1 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -3,6 +3,7 @@ /generate_rust_target /insert-sys-cert /kallsyms +/list-gitignored /module.lds /recordmcount /sign-file diff --git a/scripts/Makefile b/scripts/Makefile index 32b6ba722728..e8917975905c 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -38,7 +38,7 @@ HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED endif # The following programs are only built on demand -hostprogs += unifdef +hostprogs += list-gitignored unifdef # The module linker script is preprocessed on demand targets += module.lds diff --git a/scripts/list-gitignored.c b/scripts/list-gitignored.c new file mode 100644 index 000000000000..0bdc776d7931 --- /dev/null +++ b/scripts/list-gitignored.c @@ -0,0 +1,1074 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +// Traverse the source tree, parsing all .gitignore files, and print file paths +// that are ignored by git. +// The output is suitable to the --exclude-from option of tar. +// This is useful until the --exclude-vcs-ignores option gets working correctly. +// +// Copyright (C) 2023 Masahiro Yamada +// (with a lot of code imported from GIT) + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +//---------------------------(IMPORT FROM GIT BEGIN)--------------------------- + +// Copied from environment.c + +static bool ignore_case; + +// Copied from git-compat-util.h + +/* Sane ctype - no locale, and works with signed chars */ +#undef isascii +#undef isspace +#undef isdigit +#undef isalpha +#undef isalnum +#undef isprint +#undef islower +#undef isupper +#undef tolower +#undef toupper +#undef iscntrl +#undef ispunct +#undef isxdigit + +static const unsigned char sane_ctype[256]; +#define GIT_SPACE 0x01 +#define GIT_DIGIT 0x02 +#define GIT_ALPHA 0x04 +#define GIT_GLOB_SPECIAL 0x08 +#define GIT_REGEX_SPECIAL 0x10 +#define GIT_PATHSPEC_MAGIC 0x20 +#define GIT_CNTRL 0x40 +#define GIT_PUNCT 0x80 +#define sane_istest(x,mask) ((sane_ctype[(unsigned char)(x)] & (mask)) != 0) +#define isascii(x) (((x) & ~0x7f) == 0) +#define isspace(x) sane_istest(x,GIT_SPACE) +#define isdigit(x) sane_istest(x,GIT_DIGIT) +#define isalpha(x) sane_istest(x,GIT_ALPHA) +#define isalnum(x) sane_istest(x,GIT_ALPHA | GIT_DIGIT) +#define isprint(x) ((x) >= 0x20 && (x) <= 0x7e) +#define islower(x) sane_iscase(x, 1) +#define isupper(x) sane_iscase(x, 0) +#define is_glob_special(x) sane_istest(x,GIT_GLOB_SPECIAL) +#define is_regex_special(x) sane_istest(x,GIT_GLOB_SPECIAL | GIT_REGEX_SPECIAL) +#define iscntrl(x) (sane_istest(x,GIT_CNTRL)) +#define ispunct(x) sane_istest(x, GIT_PUNCT | GIT_REGEX_SPECIAL | \ + GIT_GLOB_SPECIAL | GIT_PATHSPEC_MAGIC) +#define isxdigit(x) (hexval_table[(unsigned char)(x)] != -1) +#define tolower(x) sane_case((unsigned char)(x), 0x20) +#define toupper(x) sane_case((unsigned char)(x), 0) + +static inline int sane_case(int x, int high) +{ + if (sane_istest(x, GIT_ALPHA)) + x = (x & ~0x20) | high; + return x; +} + +static inline int sane_iscase(int x, int is_lower) +{ + if (!sane_istest(x, GIT_ALPHA)) + return 0; + + if (is_lower) + return (x & 0x20) != 0; + else + return (x & 0x20) == 0; +} + +// Copied from ctype.c + +enum { + S = GIT_SPACE, + A = GIT_ALPHA, + D = GIT_DIGIT, + G = GIT_GLOB_SPECIAL, /* *, ?, [, \\ */ + R = GIT_REGEX_SPECIAL, /* $, (, ), +, ., ^, {, | */ + P = GIT_PATHSPEC_MAGIC, /* other non-alnum, except for ] and } */ + X = GIT_CNTRL, + U = GIT_PUNCT, + Z = GIT_CNTRL | GIT_SPACE +}; + +static const unsigned char sane_ctype[256] = { + X, X, X, X, X, X, X, X, X, Z, Z, X, X, Z, X, X, /* 0.. 15 */ + X, X, X, X, X, X, X, X, X, X, X, X, X, X, X, X, /* 16.. 31 */ + S, P, P, P, R, P, P, P, R, R, G, R, P, P, R, P, /* 32.. 47 */ + D, D, D, D, D, D, D, D, D, D, P, P, P, P, P, G, /* 48.. 63 */ + P, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A, /* 64.. 79 */ + A, A, A, A, A, A, A, A, A, A, A, G, G, U, R, P, /* 80.. 95 */ + P, A, A, A, A, A, A, A, A, A, A, A, A, A, A, A, /* 96..111 */ + A, A, A, A, A, A, A, A, A, A, A, R, R, U, P, X, /* 112..127 */ + /* Nothing in the 128.. range */ +}; + +// Copied from hex.c + +static const signed char hexval_table[256] = { + -1, -1, -1, -1, -1, -1, -1, -1, /* 00-07 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 08-0f */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 10-17 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 18-1f */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 20-27 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 28-2f */ + 0, 1, 2, 3, 4, 5, 6, 7, /* 30-37 */ + 8, 9, -1, -1, -1, -1, -1, -1, /* 38-3f */ + -1, 10, 11, 12, 13, 14, 15, -1, /* 40-47 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 48-4f */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 50-57 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 58-5f */ + -1, 10, 11, 12, 13, 14, 15, -1, /* 60-67 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 68-67 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 70-77 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 78-7f */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 80-87 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 88-8f */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 90-97 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* 98-9f */ + -1, -1, -1, -1, -1, -1, -1, -1, /* a0-a7 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* a8-af */ + -1, -1, -1, -1, -1, -1, -1, -1, /* b0-b7 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* b8-bf */ + -1, -1, -1, -1, -1, -1, -1, -1, /* c0-c7 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* c8-cf */ + -1, -1, -1, -1, -1, -1, -1, -1, /* d0-d7 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* d8-df */ + -1, -1, -1, -1, -1, -1, -1, -1, /* e0-e7 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* e8-ef */ + -1, -1, -1, -1, -1, -1, -1, -1, /* f0-f7 */ + -1, -1, -1, -1, -1, -1, -1, -1, /* f8-ff */ +}; + +// Copied from wildmatch.h + +#define WM_CASEFOLD 1 +#define WM_PATHNAME 2 + +#define WM_NOMATCH 1 +#define WM_MATCH 0 +#define WM_ABORT_ALL -1 +#define WM_ABORT_TO_STARSTAR -2 + +// Copied from wildmatch.c + +typedef unsigned char uchar; + +/* What character marks an inverted character class? */ +#define NEGATE_CLASS '!' +#define NEGATE_CLASS2 '^' + +#define CC_EQ(class, len, litmatch) ((len) == sizeof (litmatch)-1 \ + && *(class) == *(litmatch) \ + && strncmp((char*)class, litmatch, len) == 0) + +#if defined STDC_HEADERS || !defined isascii +# define ISASCII(c) 1 +#else +# define ISASCII(c) isascii(c) +#endif + +#ifdef isblank +# define ISBLANK(c) (ISASCII(c) && isblank(c)) +#else +# define ISBLANK(c) ((c) == ' ' || (c) == '\t') +#endif + +#ifdef isgraph +# define ISGRAPH(c) (ISASCII(c) && isgraph(c)) +#else +# define ISGRAPH(c) (ISASCII(c) && isprint(c) && !isspace(c)) +#endif + +#define ISPRINT(c) (ISASCII(c) && isprint(c)) +#define ISDIGIT(c) (ISASCII(c) && isdigit(c)) +#define ISALNUM(c) (ISASCII(c) && isalnum(c)) +#define ISALPHA(c) (ISASCII(c) && isalpha(c)) +#define ISCNTRL(c) (ISASCII(c) && iscntrl(c)) +#define ISLOWER(c) (ISASCII(c) && islower(c)) +#define ISPUNCT(c) (ISASCII(c) && ispunct(c)) +#define ISSPACE(c) (ISASCII(c) && isspace(c)) +#define ISUPPER(c) (ISASCII(c) && isupper(c)) +#define ISXDIGIT(c) (ISASCII(c) && isxdigit(c)) + +/* Match pattern "p" against "text" */ +static int dowild(const uchar *p, const uchar *text, unsigned int flags) +{ + uchar p_ch; + const uchar *pattern = p; + + for ( ; (p_ch = *p) != '\0'; text++, p++) { + int matched, match_slash, negated; + uchar t_ch, prev_ch; + if ((t_ch = *text) == '\0' && p_ch != '*') + return WM_ABORT_ALL; + if ((flags & WM_CASEFOLD) && ISUPPER(t_ch)) + t_ch = tolower(t_ch); + if ((flags & WM_CASEFOLD) && ISUPPER(p_ch)) + p_ch = tolower(p_ch); + switch (p_ch) { + case '\\': + /* Literal match with following character. Note that the test + * in "default" handles the p[1] == '\0' failure case. */ + p_ch = *++p; + /* FALLTHROUGH */ + default: + if (t_ch != p_ch) + return WM_NOMATCH; + continue; + case '?': + /* Match anything but '/'. */ + if ((flags & WM_PATHNAME) && t_ch == '/') + return WM_NOMATCH; + continue; + case '*': + if (*++p == '*') { + const uchar *prev_p = p - 2; + while (*++p == '*') {} + if (!(flags & WM_PATHNAME)) + /* without WM_PATHNAME, '*' == '**' */ + match_slash = 1; + else if ((prev_p < pattern || *prev_p == '/') && + (*p == '\0' || *p == '/' || + (p[0] == '\\' && p[1] == '/'))) { + /* + * Assuming we already match 'foo/' and are at + * , just assume it matches + * nothing and go ahead match the rest of the + * pattern with the remaining string. This + * helps make foo/<*><*>/bar (<> because + * otherwise it breaks C comment syntax) match + * both foo/bar and foo/a/bar. + */ + if (p[0] == '/' && + dowild(p + 1, text, flags) == WM_MATCH) + return WM_MATCH; + match_slash = 1; + } else /* WM_PATHNAME is set */ + match_slash = 0; + } else + /* without WM_PATHNAME, '*' == '**' */ + match_slash = flags & WM_PATHNAME ? 0 : 1; + if (*p == '\0') { + /* Trailing "**" matches everything. Trailing "*" matches + * only if there are no more slash characters. */ + if (!match_slash) { + if (strchr((char *)text, '/')) + return WM_NOMATCH; + } + return WM_MATCH; + } else if (!match_slash && *p == '/') { + /* + * _one_ asterisk followed by a slash + * with WM_PATHNAME matches the next + * directory + */ + const char *slash = strchr((char*)text, '/'); + if (!slash) + return WM_NOMATCH; + text = (const uchar*)slash; + /* the slash is consumed by the top-level for loop */ + break; + } + while (1) { + if (t_ch == '\0') + break; + /* + * Try to advance faster when an asterisk is + * followed by a literal. We know in this case + * that the string before the literal + * must belong to "*". + * If match_slash is false, do not look past + * the first slash as it cannot belong to '*'. + */ + if (!is_glob_special(*p)) { + p_ch = *p; + if ((flags & WM_CASEFOLD) && ISUPPER(p_ch)) + p_ch = tolower(p_ch); + while ((t_ch = *text) != '\0' && + (match_slash || t_ch != '/')) { + if ((flags & WM_CASEFOLD) && ISUPPER(t_ch)) + t_ch = tolower(t_ch); + if (t_ch == p_ch) + break; + text++; + } + if (t_ch != p_ch) + return WM_NOMATCH; + } + if ((matched = dowild(p, text, flags)) != WM_NOMATCH) { + if (!match_slash || matched != WM_ABORT_TO_STARSTAR) + return matched; + } else if (!match_slash && t_ch == '/') + return WM_ABORT_TO_STARSTAR; + t_ch = *++text; + } + return WM_ABORT_ALL; + case '[': + p_ch = *++p; +#ifdef NEGATE_CLASS2 + if (p_ch == NEGATE_CLASS2) + p_ch = NEGATE_CLASS; +#endif + /* Assign literal 1/0 because of "matched" comparison. */ + negated = p_ch == NEGATE_CLASS ? 1 : 0; + if (negated) { + /* Inverted character class. */ + p_ch = *++p; + } + prev_ch = 0; + matched = 0; + do { + if (!p_ch) + return WM_ABORT_ALL; + if (p_ch == '\\') { + p_ch = *++p; + if (!p_ch) + return WM_ABORT_ALL; + if (t_ch == p_ch) + matched = 1; + } else if (p_ch == '-' && prev_ch && p[1] && p[1] != ']') { + p_ch = *++p; + if (p_ch == '\\') { + p_ch = *++p; + if (!p_ch) + return WM_ABORT_ALL; + } + if (t_ch <= p_ch && t_ch >= prev_ch) + matched = 1; + else if ((flags & WM_CASEFOLD) && ISLOWER(t_ch)) { + uchar t_ch_upper = toupper(t_ch); + if (t_ch_upper <= p_ch && t_ch_upper >= prev_ch) + matched = 1; + } + p_ch = 0; /* This makes "prev_ch" get set to 0. */ + } else if (p_ch == '[' && p[1] == ':') { + const uchar *s; + int i; + for (s = p += 2; (p_ch = *p) && p_ch != ']'; p++) {} /*SHARED ITERATOR*/ + if (!p_ch) + return WM_ABORT_ALL; + i = p - s - 1; + if (i < 0 || p[-1] != ':') { + /* Didn't find ":]", so treat like a normal set. */ + p = s - 2; + p_ch = '['; + if (t_ch == p_ch) + matched = 1; + continue; + } + if (CC_EQ(s,i, "alnum")) { + if (ISALNUM(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "alpha")) { + if (ISALPHA(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "blank")) { + if (ISBLANK(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "cntrl")) { + if (ISCNTRL(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "digit")) { + if (ISDIGIT(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "graph")) { + if (ISGRAPH(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "lower")) { + if (ISLOWER(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "print")) { + if (ISPRINT(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "punct")) { + if (ISPUNCT(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "space")) { + if (ISSPACE(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "upper")) { + if (ISUPPER(t_ch)) + matched = 1; + else if ((flags & WM_CASEFOLD) && ISLOWER(t_ch)) + matched = 1; + } else if (CC_EQ(s,i, "xdigit")) { + if (ISXDIGIT(t_ch)) + matched = 1; + } else /* malformed [:class:] string */ + return WM_ABORT_ALL; + p_ch = 0; /* This makes "prev_ch" get set to 0. */ + } else if (t_ch == p_ch) + matched = 1; + } while (prev_ch = p_ch, (p_ch = *++p) != ']'); + if (matched == negated || + ((flags & WM_PATHNAME) && t_ch == '/')) + return WM_NOMATCH; + continue; + } + } + + return *text ? WM_NOMATCH : WM_MATCH; +} + +/* Match the "pattern" against the "text" string. */ +static int wildmatch(const char *pattern, const char *text, unsigned int flags) +{ + // local modification: move WM_CASEFOLD here + if (ignore_case) + flags |= WM_CASEFOLD; + + return dowild((const uchar*)pattern, (const uchar*)text, flags); +} + +// copied from dir.h + +#define PATTERN_FLAG_NODIR 1 +#define PATTERN_FLAG_ENDSWITH 4 +#define PATTERN_FLAG_MUSTBEDIR 8 +#define PATTERN_FLAG_NEGATIVE 16 + +// Copied from dir.c + +static int fspathncmp(const char *a, const char *b, size_t count) +{ + return ignore_case ? strncasecmp(a, b, count) : strncmp(a, b, count); +} + +static int simple_length(const char *match) +{ + int len = -1; + + for (;;) { + unsigned char c = *match++; + len++; + if (c == '\0' || is_glob_special(c)) + return len; + } +} + +static int no_wildcard(const char *string) +{ + return string[simple_length(string)] == '\0'; +} + +static void parse_path_pattern(const char **pattern, + int *patternlen, + unsigned *flags, + int *nowildcardlen) +{ + const char *p = *pattern; + size_t i, len; + + *flags = 0; + if (*p == '!') { + *flags |= PATTERN_FLAG_NEGATIVE; + p++; + } + len = strlen(p); + if (len && p[len - 1] == '/') { + len--; + *flags |= PATTERN_FLAG_MUSTBEDIR; + } + for (i = 0; i < len; i++) { + if (p[i] == '/') + break; + } + if (i == len) + *flags |= PATTERN_FLAG_NODIR; + *nowildcardlen = simple_length(p); + /* + * we should have excluded the trailing slash from 'p' too, + * but that's one more allocation. Instead just make sure + * nowildcardlen does not exceed real patternlen + */ + if (*nowildcardlen > len) + *nowildcardlen = len; + if (*p == '*' && no_wildcard(p + 1)) + *flags |= PATTERN_FLAG_ENDSWITH; + *pattern = p; + *patternlen = len; +} + +static void trim_trailing_spaces(char *buf) +{ + char *p, *last_space = NULL; + + for (p = buf; *p; p++) + switch (*p) { + case ' ': + if (!last_space) + last_space = p; + break; + case '\\': + p++; + if (!*p) + return; + /* fallthrough */ + default: + last_space = NULL; + } + + if (last_space) + *last_space = '\0'; +} + +static int match_basename(const char *basename, int basenamelen, + const char *pattern, int prefix, int patternlen, + unsigned flags) +{ + if (prefix == patternlen) { + if (patternlen == basenamelen && + !fspathncmp(pattern, basename, basenamelen)) + return 1; + } else if (flags & PATTERN_FLAG_ENDSWITH) { + /* "*literal" matching against "fooliteral" */ + if (patternlen - 1 <= basenamelen && + !fspathncmp(pattern + 1, + basename + basenamelen - (patternlen - 1), + patternlen - 1)) + return 1; + } else { + // local modification: call wildmatch() directly + if (!wildmatch(pattern, basename, flags)) + return 1; + } + return 0; +} + +static int match_pathname(const char *pathname, int pathlen, + const char *base, int baselen, + const char *pattern, int prefix, int patternlen) +{ + // local modification: remove local variables + + /* + * match with FNM_PATHNAME; the pattern has base implicitly + * in front of it. + */ + if (*pattern == '/') { + pattern++; + patternlen--; + prefix--; + } + + /* + * baselen does not count the trailing slash. base[] may or + * may not end with a trailing slash though. + */ + if (pathlen < baselen + 1 || + (baselen && pathname[baselen] != '/') || + fspathncmp(pathname, base, baselen)) + return 0; + + pathname += baselen + 1; + pathlen -= baselen + 1; + + if (prefix) { + /* + * if the non-wildcard part is longer than the + * remaining pathname, surely it cannot match. + */ + if (prefix > pathlen) + return 0; + + if (fspathncmp(pattern, pathname, prefix)) + return 0; + pattern += prefix; + patternlen -= prefix; + pathname += prefix; + pathlen -= prefix; + + /* + * If the whole pattern did not have a wildcard, + * then our prefix match is all we need; we + * do not need to call fnmatch at all. + */ + if (!patternlen && !pathlen) + return 1; + } + + // local modification: call wildmatch() directly + return !wildmatch(pattern, pathname, WM_PATHNAME); +} + +// Copied from git/utf8.c + +static const char utf8_bom[] = "\357\273\277"; + +//----------------------------(IMPORT FROM GIT END)---------------------------- + +struct pattern { + unsigned int flags; + int nowildcardlen; + int patternlen; + int dirlen; + char pattern[]; +}; + +static struct pattern **pattern_list; +static int nr_patterns, alloced_patterns; + +// Remember the number of patterns at each directory level +static int *nr_patterns_at; +// Track the current/max directory level; +static int depth, max_depth; +static bool debug_on; +static FILE *out_fp, *stat_fp; +static char *prefix = ""; +static char *progname; + +static void __attribute__((noreturn)) perror_exit(const char *s) +{ + perror(s); + + exit(EXIT_FAILURE); +} + +static void __attribute__((noreturn)) error_exit(const char *fmt, ...) +{ + va_list args; + + fprintf(stderr, "%s: error: ", progname); + + va_start(args, fmt); + vfprintf(stderr, fmt, args); + va_end(args); + + exit(EXIT_FAILURE); +} + +static void debug(const char *fmt, ...) +{ + va_list args; + int i; + + if (!debug_on) + return; + + fprintf(stderr, "[DEBUG] "); + + for (i = 0; i < depth * 2; i++) + fputc(' ', stderr); + + va_start(args, fmt); + vfprintf(stderr, fmt, args); + va_end(args); +} + +static void *xrealloc(void *ptr, size_t size) +{ + ptr = realloc(ptr, size); + if (!ptr) + perror_exit(progname); + + return ptr; +} + +static void *xmalloc(size_t size) +{ + return xrealloc(NULL, size); +} + +// similar to last_matching_pattern_from_list() in GIT +static bool is_ignored(const char *path, int pathlen, int dirlen, bool is_dir) +{ + int i; + + // Search in the reverse order because the last matching pattern wins. + for (i = nr_patterns - 1; i >= 0; i--) { + struct pattern *p = pattern_list[i]; + unsigned int flags = p->flags; + const char *gitignore_dir = p->pattern + p->patternlen + 1; + bool ignored; + + if ((flags & PATTERN_FLAG_MUSTBEDIR) && !is_dir) + continue; + + if (flags & PATTERN_FLAG_NODIR) { + if (!match_basename(path + dirlen + 1, + pathlen - dirlen - 1, + p->pattern, + p->nowildcardlen, + p->patternlen, + p->flags)) + continue; + } else { + if (!match_pathname(path, pathlen, + gitignore_dir, p->dirlen, + p->pattern, + p->nowildcardlen, + p->patternlen)) + continue; + } + + debug("%s: matches %s%s%s (%s/.gitignore)\n", path, + flags & PATTERN_FLAG_NEGATIVE ? "!" : "", p->pattern, + flags & PATTERN_FLAG_MUSTBEDIR ? "/" : "", + gitignore_dir); + + ignored = (flags & PATTERN_FLAG_NEGATIVE) == 0; + if (ignored) + debug("Ignore: %s\n", path); + + return ignored; + } + + debug("%s: no match\n", path); + + return false; +} + +static void add_pattern(const char *string, const char *dir, int dirlen) +{ + struct pattern *p; + int patternlen, nowildcardlen; + unsigned int flags; + + parse_path_pattern(&string, &patternlen, &flags, &nowildcardlen); + + if (patternlen == 0) + return; + + p = xmalloc(sizeof(*p) + patternlen + dirlen + 2); + + memcpy(p->pattern, string, patternlen); + p->pattern[patternlen] = 0; + memcpy(p->pattern + patternlen + 1, dir, dirlen); + p->pattern[patternlen + 1 + dirlen] = 0; + + p->patternlen = patternlen; + p->nowildcardlen = nowildcardlen; + p->dirlen = dirlen; + p->flags = flags; + + debug("Add pattern: %s%s%s\n", + flags & PATTERN_FLAG_NEGATIVE ? "!" : "", p->pattern, + flags & PATTERN_FLAG_MUSTBEDIR ? "/" : ""); + + if (nr_patterns >= alloced_patterns) { + alloced_patterns += 128; + pattern_list = xrealloc(pattern_list, + sizeof(*pattern_list) * alloced_patterns); + } + + pattern_list[nr_patterns++] = p; +} + +// similar to add_patterns_from_buffer() +static void add_patterns_from_gitignore(const char *dir, int dirlen) +{ + struct stat st; + char path[PATH_MAX], *buf, *entry; + size_t size; + int fd, pathlen, i; + + pathlen = snprintf(path, sizeof(path), "%s/.gitignore", dir); + if (pathlen >= sizeof(path)) + error_exit("%s: too long path was truncated\n", path); + + fd = open(path, O_RDONLY | O_NOFOLLOW); + if (fd < 0) { + if (errno != ENOENT) + return perror_exit(path); + return; + } + + if (fstat(fd, &st) < 0) + perror_exit(path); + + size = st.st_size; + + buf = xmalloc(size + 1); + if (read(fd, buf, st.st_size) != st.st_size) + perror_exit(path); + + buf[st.st_size] = '\n'; + if (close(fd)) + perror_exit(path); + + debug("Parse %s\n", path); + + entry = buf; + + // skip utf8 bom + if (!strncmp(entry, utf8_bom, strlen(utf8_bom))) + entry += strlen(utf8_bom); + + for (i = entry - buf; i < size; i++) { + if (buf[i] == '\n') { + if (entry != buf + i && entry[0] != '#') { + buf[i - (i && buf[i-1] == '\r')] = 0; + trim_trailing_spaces(entry); + add_pattern(entry, dir, dirlen); + } + entry = buf + i + 1; + } + } + + free(buf); +} + +// Save the current number of patterns and increment the depth +static void increment_depth(void) +{ + if (depth >= max_depth) { + max_depth += 1; + nr_patterns_at = xrealloc(nr_patterns_at, + sizeof(*nr_patterns_at) * max_depth); + } + + nr_patterns_at[depth] = nr_patterns; + depth++; +} + +// Decrement the depth, and free up the patterns of this directory level. +static void decrement_depth(void) +{ + depth--; + assert(depth >= 0); + + while (nr_patterns > nr_patterns_at[depth]) + free(pattern_list[--nr_patterns]); +} + +static void print_path(const char *path) +{ + // The path always starts with "./" + assert(strlen(path) >= 2); + + // Replace the root directory with a preferred prefix. + // This is useful for the tar command. + fprintf(out_fp, "%s%s\n", prefix, path + 2); +} + +static void print_stat(const char *path, struct stat *st) +{ + if (!stat_fp) + return; + + if (!S_ISREG(st->st_mode) && !S_ISLNK(st->st_mode)) + return; + + assert(strlen(path) >= 2); + + fprintf(stat_fp, "%c %9ld %10ld %s\n", + S_ISLNK(st->st_mode) ? 'l' : '-', + st->st_size, st->st_mtim.tv_sec, path + 2); +} + +// Traverse the entire directory tree, parsing .gitignore files. +// Print file paths that are not tracked by git. +// +// Return true if all files under the directory are ignored, false otherwise. +static bool traverse_directory(const char *dir, int dirlen) +{ + bool all_ignored = true; + DIR *dirp; + + debug("Enter[%d]: %s\n", depth, dir); + increment_depth(); + + add_patterns_from_gitignore(dir, dirlen); + + dirp = opendir(dir); + if (!dirp) + perror_exit(dir); + + while (1) { + struct dirent *d; + struct stat st; + char path[PATH_MAX]; + int pathlen; + bool ignored; + + errno = 0; + d = readdir(dirp); + if (!d) { + if (errno) + perror_exit(dir); + break; + } + + if (!strcmp(d->d_name, "..") || !strcmp(d->d_name, ".")) + continue; + + pathlen = snprintf(path, sizeof(path), "%s/%s", dir, d->d_name); + if (pathlen >= sizeof(path)) + error_exit("%s: too long path was truncated\n", path); + + if (lstat(path, &st) < 0) + perror_exit(path); + + if ((!S_ISREG(st.st_mode) && !S_ISDIR(st.st_mode) && !S_ISLNK(st.st_mode)) || + is_ignored(path, pathlen, dirlen, S_ISDIR(st.st_mode))) { + ignored = true; + } else { + if (S_ISDIR(st.st_mode) && !S_ISLNK(st.st_mode)) + // If all the files in a directory are ignored, + // let's ignore that directory as well. This + // will avoid empty directories in the tarball. + ignored = traverse_directory(path, pathlen); + else + ignored = false; + } + + if (ignored) { + print_path(path); + } else { + print_stat(path, &st); + all_ignored = false; + } + } + + if (closedir(dirp)) + perror_exit(dir); + + decrement_depth(); + debug("Leave[%d]: %s\n", depth, dir); + + return all_ignored; +} + +static void usage(void) +{ + fprintf(stderr, + "usage: %s [options]\n" + "\n" + "Show files that are ignored by git\n" + "\n" + "options:\n" + " -d, --debug print debug messages to stderr\n" + " -e, --exclude PATTERN add the given exclude pattern\n" + " -h, --help show this help message and exit\n" + " -i, --ignore-case Ignore case differences the patterns and the files\n" + " -o, --output FILE output to a file (default: '-', i.e. stdout)\n" + " -p, --prefix PREFIX prefix added to each path (default: empty string)\n" + " -r, --rootdir DIR root of the source tree (default: current working directory)\n" + " -s, --stat FILE output the file stat of non-ignored files to a file\n", + progname); +} + +static void open_output(const char *pathname, FILE **fp) +{ + if (strcmp(pathname, "-")) { + *fp = fopen(pathname, "w"); + if (!*fp) + perror_exit(pathname); + } else { + *fp = stdout; + } +} + +static void close_output(const char *pathname, FILE *fp) +{ + fflush(fp); + + if (ferror(fp)) + error_exit("not all data was written to the output\n"); + + if (fclose(fp)) + perror_exit(pathname); +} + +int main(int argc, char *argv[]) +{ + const char *output = "-"; + const char *rootdir = "."; + const char *stat = NULL; + + progname = strrchr(argv[0], '/'); + if (progname) + progname++; + else + progname = argv[0]; + + while (1) { + static struct option long_options[] = { + {"debug", no_argument, NULL, 'd'}, + {"help", no_argument, NULL, 'h'}, + {"ignore-case", no_argument, NULL, 'i'}, + {"output", required_argument, NULL, 'o'}, + {"prefix", required_argument, NULL, 'p'}, + {"rootdir", required_argument, NULL, 'r'}, + {"stat", required_argument, NULL, 's'}, + {"exclude", required_argument, NULL, 'x'}, + {}, + }; + + int c = getopt_long(argc, argv, "dhino:p:r:s:x:", long_options, NULL); + + if (c == -1) + break; + + switch (c) { + case 'd': + debug_on = true; + break; + case 'h': + usage(); + exit(0); + case 'i': + ignore_case = true; + break; + case 'o': + output = optarg; + break; + case 'p': + prefix = optarg; + break; + case 'r': + rootdir = optarg; + break; + case 's': + stat = optarg; + break; + case 'x': + add_pattern(optarg, ".", strlen(".")); + break; + case '?': + usage(); + /* fallthrough */ + default: + exit(EXIT_FAILURE); + } + } + + open_output(output, &out_fp); + if (stat && stat[0]) + open_output(stat, &stat_fp); + + if (chdir(rootdir)) + perror_exit(rootdir); + + add_pattern(".git/", ".", strlen(".")); + + if (traverse_directory(".", strlen("."))) + print_path("./"); + + assert(depth == 0); + + while (nr_patterns > 0) + free(pattern_list[--nr_patterns]); + free(pattern_list); + free(nr_patterns_at); + + close_output(output, out_fp); + if (stat_fp) + close_output(stat, stat_fp); + + return 0; +} From patchwork Fri Feb 10 18:18:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 55514 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1105082wrn; Fri, 10 Feb 2023 10:19:35 -0800 (PST) X-Google-Smtp-Source: AK7set//YJe6b98eI85zL48+d0DXhiLz07SzZj0xjbk9KqXKa7RkHXT6+k7v8a+bZvVdfyO/018D X-Received: by 2002:a17:906:7816:b0:878:5976:5bb7 with SMTP id u22-20020a170906781600b0087859765bb7mr16083739ejm.8.1676053175140; Fri, 10 Feb 2023 10:19:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676053175; cv=none; d=google.com; s=arc-20160816; b=Yu3qPgsXlm/ixjzqFEWwln4il7wXoXsFAsinhFI/RwT6ISj36a4idXIwFDJsSEEOlf IlglHl7r9KfdbsQzJK4w0ToNqioHONbJsHwB3usOWxv21rum0K+2NwVK3Rdq9g/477+U Y19vYh3pOKwscOjuzlLUyOKEpELe15Zi5PigQ4cR9Dk6DE0KmUKgs1x5tNoI/SzRTp7x X0wzTlEHW2K7NO1IZ+GA9PAQ1D4TAtCYgR2nCB/rYh4XCrtgUyBOQCERFJdp1dvNz754 Q8v1CLKycOWjrSEVQiU+f+JuW6OppM9+dXbtKF26J7fKljGYD0zhzkDQh6veet+zISX5 Kf7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=68HgEqYdLCVEkJSlJTU4CDC/CA+BNtx6JDB5Ji+GTeQ=; b=SF6diYHnxcFbIemVp46xsafOJb9z2xrHssabv6yrqHDkNd6HwJa/WLueV91XSZJmkV r3N9hXhbWFt1v7fBAqS7cWX6WoJbCACqPlmpDkT8Je+RSDmsh2iknRkXcccYf0Pg6hs8 7ZPT/GeIx2jDPyWZDKWN3lrPjApFQg/cVLeJmTRApjZQLvPxc/azpLjgMZtxu70ewI4u Kh6cB7yB2As2SRl4dn5g1x2OZ7d+R7CgwzXjrq8AQF4TIzV4TleLOUdXeawVSp+AEUDF +cmCFfYInVpofXELdnVnrC1SNfhgcTnRu0MLZhTZJ2R6zUmbQiY/j/1CGzqkVBeS6iw1 MTjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TcnPILo3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fi3-20020a1709073ac300b0087761d8cff9si5301768ejc.996.2023.02.10.10.19.12; Fri, 10 Feb 2023 10:19:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TcnPILo3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232938AbjBJSSs (ORCPT + 99 others); Fri, 10 Feb 2023 13:18:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232916AbjBJSSm (ORCPT ); Fri, 10 Feb 2023 13:18:42 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3653C56EE7; Fri, 10 Feb 2023 10:18:41 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B9403B824DC; Fri, 10 Feb 2023 18:18:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A9687C4339E; Fri, 10 Feb 2023 18:18:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053118; bh=OOnQiLtiUJOqoDdzJaP936hrThLt6/kh3l2/k+KX7a4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TcnPILo3ifUOzSjnjIJD8HddzLBYjFKyGJWT2qDyb8TEg3hHJZhrbyMm5CifS1SOi 7wNojkMhvLvjxjNFQFCqrP/FsXzQ7X1JE3unugRnE6KNkRbXXLhgBx9SukXe0xNVSh YRRSa/KV40FwBltSA6l7JZCxOVIHyVPzBFW7uFy7tHEcC0UZWhqScxOkSOeLQIx6s1 vM9oV1lQjvATqH35a5Q2OzKoLIegya7YhzHbOLQWubjS8v3Ho+1pduf7XwOCnEVRZF gcs537M8ATNLtaJ/geUJR97AUp5ZKM2fq2OkfAi35j+IKL9YwM2mrEe8+iz9GaIjaS auZQpo86cSaJA== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada , Tom Rix , llvm@lists.linux.dev Subject: [PATCH v5 2/7] kbuild: deb-pkg: create source package without cleaning Date: Sat, 11 Feb 2023 03:18:23 +0900 Message-Id: <20230210181828.124765-2-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210181828.124765-1-masahiroy@kernel.org> References: <20230210181828.124765-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757469134027871967?= X-GMAIL-MSGID: =?utf-8?q?1757469134027871967?= If you run 'make deb-pkg', all objects are lost due to 'make clean', which makes the incremental builds impossible. Instead of cleaning, pass the exclude list to tar's --exclude-from option. Previously, *.diff.gz contained some check-in files such as .clang-format, .cocciconfig. With this commit, *.diff.gz will only contain the .config and debian/. The other source files will go into the .orig tarball. linux.tar.gz is rebuilt only when the source files that would go into the tarball are changed. Signed-off-by: Masahiro Yamada --- Changes in v5: - Avoid unneeded rebuild of the tarball when nothing in the source tree is changed. Changes in v4: - Fix a typo in comment Changes in v3: - Add --extra-pattern='*.rej' - Exclude symlinks at the toplevel - Add --sort=name tar option scripts/Makefile.package | 40 ++++++++++++++++++++++++++++++++++------ scripts/package/mkdebian | 24 ++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 6 deletions(-) diff --git a/scripts/Makefile.package b/scripts/Makefile.package index dfbf40454a99..0dda909f87a9 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -80,13 +80,12 @@ binrpm-pkg: $(UTS_MACHINE)-linux -bb $(objtree)/binkernel.spec PHONY += deb-pkg -deb-pkg: - $(MAKE) clean +deb-pkg: linux.tar.gz $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian - $(call cmd,src_tar,$(KDEB_SOURCENAME)) - origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\ - mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz - +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) --source-option=-sP -i.git -us -uc + $(Q)origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\ + cp linux.tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz + +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \ + --build=source,binary --source-option=-sP -nc -us -uc PHONY += bindeb-pkg bindeb-pkg: @@ -120,6 +119,32 @@ $(tar-pkgs): $(MAKE) -f $(srctree)/Makefile +$(CONFIG_SHELL) $(srctree)/scripts/package/buildtar $@ +# Source Tarball +# --------------------------------------------------------------------------- + +scripts/list-gitignored: FORCE + $(Q)$(MAKE) -f $(srctree)/Makefile scripts_package + +# 1f5d3a6b6532e25a5cdf1f311956b2b03d343a48 removed '*.rej' from .gitignore, +# but it is definitely a generated file. +filechk_filelist = \ + $< --exclude='*.rej' --output=$@_exclude --prefix=./ --rootdir=$(srctree) --stat=- + +.tmp_filelist: scripts/list-gitignored FORCE + $(call filechk,filelist) + +quiet_cmd_tar = TAR $@ + cmd_tar = tar $(compress-opt) -c -f $@ \ + --exclude=./$@ --exclude-from=$<_exclude \ + --owner=0 --group=0 --sort=name \ + --transform 's:^\.:$*:S' -C $(tar-rootdir) . + +tar-rootdir := $(srctree) + +%.tar.gz: compress-opt := -I $(KGZIP) +%.tar.gz: .tmp_filelist + $(call cmd,tar) + # perf-pkg - generate a source tarball with perf source # --------------------------------------------------------------------------- @@ -174,4 +199,7 @@ help: @echo ' perf-tarxz-src-pkg - Build $(perf-tar).tar.xz source tarball' @echo ' perf-tarzst-src-pkg - Build $(perf-tar).tar.zst source tarball' +PHONY += FORCE +FORCE: + .PHONY: $(PHONY) diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index c3bbef7a6754..68b13ef590ba 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -84,6 +84,8 @@ set_debarch() { fi } +rm -rf debian + # Some variables and settings used throughout the script version=$KERNELRELEASE if [ -n "$KDEB_PKGVERSION" ]; then @@ -135,6 +137,28 @@ fi mkdir -p debian/source/ echo "1.0" > debian/source/format +# Ugly: ignore anything except .config or debian/ +cat<<'EOF' > debian/source/local-options +diff-ignore + +extend-diff-ignore = ^[^.d] + +extend-diff-ignore = ^\.[^c] +extend-diff-ignore = ^\.c($|[^o]) +extend-diff-ignore = ^\.co($|[^n]) +extend-diff-ignore = ^\.con($|[^f]) +extend-diff-ignore = ^\.conf($|[^i]) +extend-diff-ignore = ^\.confi($|[^g]) +extend-diff-ignore = ^\.config. + +extend-diff-ignore = ^d($|[^e]) +extend-diff-ignore = ^de($|[^b]) +extend-diff-ignore = ^deb($|[^i]) +extend-diff-ignore = ^debi($|[^a]) +extend-diff-ignore = ^debia($|[^n]) +extend-diff-ignore = ^debian[^/] +EOF + echo $debarch > debian/arch extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)" extra_build_depends="$extra_build_depends, $(if_enabled_echo CONFIG_SYSTEM_TRUSTED_KEYRING libssl-dev:native)" From patchwork Fri Feb 10 18:18:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 55516 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1105528wrn; Fri, 10 Feb 2023 10:20:40 -0800 (PST) X-Google-Smtp-Source: AK7set8TpzGpNX5Gc+f21uv88/pkmksOP0P2PI4Dy8kLNyHIJCs6U2aITZbhcOrAxVZ7v+mtQVwy X-Received: by 2002:a17:906:ca2:b0:887:7871:2b2f with SMTP id k2-20020a1709060ca200b0088778712b2fmr16480900ejh.61.1676053240633; Fri, 10 Feb 2023 10:20:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676053240; cv=none; d=google.com; s=arc-20160816; b=FeTeQVwL6oyNk5r8wraBaNZYHQsmmPcMzqkwRdOvL2J3PXCxn4t+5VnN6vcoNxsr0C Cswv/jhGLQVF3lJ70U1lEgyLQvahPMHCgTkO5eYZ5C2k0QUakhp/xcW/fomo4z1lS89r E/Yi7c0jJ7Uu1crdvksipuijJ5XQtVSgGAMEewvCyVA1U3kFzTs1xSEr/UDppTLPFW9w aWJxcIG4pNB48rDBbUsxzml6nF9+Xh5xYaiokdEG6XmAnp1DZOVX8l1Vo2TcMSrwRsd5 rFk0Z7ZlfD2C1q8JL7SNsziqZV4kM1FcJBE+0E4AAboyQiWZYCd6Y56vJzBEHu0bXRAj O4BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=m8RX3byj3f/OJD76CoKVJHTe7RkRdMdb8CU7Zz/KOws=; b=qCLaw8IRRcKkByR0WI7OWxHyxL0EGzNN6yBDcCRETUJHEXWQ2eJZTu6mFz3YQBQygH X2LJZDnG0pMmL9n63eP5mHqTCHaNTjBXCJ6uS8VtatTTIcmMgQnbwrA8Ojdx9tzFyg4E 2wWiud+RZY/smMbr8toj2/6JOiN4hJyk8Zy8pVkV2ub4n+vD7fV8RmikF3rNDW+FJguX eGxEiRTNoK0G8N1rY4vEEEOVg1pLroQLg5X6KlBZSLYCDgi8WGnLnmS2WSPJ7y9+JepG Z8w4XZLMOhQvm2e8tf78Oqb7Jhz1zI6xebPL/qe5samfFQ48bD92QYV9L3J1wdnfoqO0 l6GA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=D75Ayfcg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id vw6-20020a170907058600b0087da417217dsi5039434ejb.728.2023.02.10.10.20.13; Fri, 10 Feb 2023 10:20:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=D75Ayfcg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232963AbjBJSTA (ORCPT + 99 others); Fri, 10 Feb 2023 13:19:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232934AbjBJSSp (ORCPT ); Fri, 10 Feb 2023 13:18:45 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85C3B5EBCC; Fri, 10 Feb 2023 10:18:43 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 238E0B825BC; Fri, 10 Feb 2023 18:18:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E93EAC433D2; Fri, 10 Feb 2023 18:18:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053120; bh=Dch3vAjHkuwS0Ip0OjVDZfKz/fFwFDY7bH9X+rd6/vM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D75Ayfcgyv/vP+xf7rDaxhN5QGC9/Op3ktbIFIty6m0MzlQBKO/H2em62RQiuL4s3 nDFqebwmpytnB58Oz/FLa1sUesGqsGqU+SGsRZH/D3MbJysb5vgfE2UwdGNVFLP1De +xWGxx8A0rnPeNKTNnHC/TWVZKPxuRZQ3/BwTCDEncWX4ud7S849ld6WsrV+NVATzz l98ilQzOcEh5T5o5FWw+IB6bqT7+mdEcqHOxDdlWTlrXn514itv5/ldjXl7IYFLhNy o1S6NZaHaituU7Fo5zRYH4gHHPmSkk2KyOqsv8n2TiUvhHnnWuQB60HG7k/xxAVNho gVOxQWzgdHdag== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada Subject: [PATCH v5 3/7] kbuild: rpm-pkg: build binary packages from source rpm Date: Sat, 11 Feb 2023 03:18:24 +0900 Message-Id: <20230210181828.124765-3-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210181828.124765-1-masahiroy@kernel.org> References: <20230210181828.124765-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757469202826053614?= X-GMAIL-MSGID: =?utf-8?q?1757469202826053614?= The build rules of rpm-pkg and srcrpm-pkg are almost the same. Remove the code duplication. Change rpm-pkg to build binary packages from the source package generated by srcrpm-pkg. This changes the output directory of the srpm generated by 'make rpm-pkg' because srcrpm-pkg overrides _srcrpmdir. Signed-off-by: Masahiro Yamada --- (no changes since v3) Changes in v3: - Explain that the source package location will be changed. scripts/Makefile.package | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/scripts/Makefile.package b/scripts/Makefile.package index 0dda909f87a9..0b66bd667212 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -53,11 +53,9 @@ tar -I $(KGZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \ # rpm-pkg # --------------------------------------------------------------------------- PHONY += rpm-pkg -rpm-pkg: - $(MAKE) clean - $(CONFIG_SHELL) $(MKSPEC) >$(objtree)/kernel.spec - $(call cmd,src_tar,$(KERNELPATH),kernel.spec) - +rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -ta $(KERNELPATH).tar.gz \ +rpm-pkg: srpm = $(shell rpmspec --srpm --query --queryformat='%{name}-%{VERSION}-%{RELEASE}.src.rpm' kernel.spec) +rpm-pkg: srcrpm-pkg + +rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -rb $(srpm) \ --define='_smp_mflags %{nil}' # srcrpm-pkg From patchwork Fri Feb 10 18:18:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 55515 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1105280wrn; Fri, 10 Feb 2023 10:20:09 -0800 (PST) X-Google-Smtp-Source: AK7set+h7KZG3VNkRldO8yR9jeDDuw9NGTIWFt3IBuaR59FE3Dj9Ne1lm77If6xAgg211mP7nRIo X-Received: by 2002:a17:906:3491:b0:87d:f3e2:7f4d with SMTP id g17-20020a170906349100b0087df3e27f4dmr16400169ejb.25.1676053208978; Fri, 10 Feb 2023 10:20:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676053208; cv=none; d=google.com; s=arc-20160816; b=jVvsDQDOmhh+nAi5P+5F/og1kKgDJPw8y/2F1rSZwLjuTxgr8wrKg508g3UVOBYtSL 8r8+n519RqK4z9p2CKdA4WgJs6Na/HyCx3B7h1scSwPjcM9eNCtBeH/OnMt3FW7rCkqf +cMXbgxyeNKs0vR4R9KMJHW9Bkr/Pp4H61qIxpNGvxYDkqNaykzvnJWaOiVUkCUd6e6a rQPDfoydY01r9pLxYMTCltHh5fOyBMIOXStrCRPVp1fAvez2Gi3P2I7PDGSY1UYOwmDN 1n/9bTBXlhxgjx59VRZwX0czd2n7KbHiY1TiDKK5MKEDd495xX2BD2SVTwNAfDtVsBkP yjHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PttIt93hBvaDl4rhN8G8I6j0VyAYzrJC00hVe1hrYUc=; b=Ngxl4wlTObkAkzj8Rp3yO3xf4xx50adD+MfLZMYlHp91FeZevZh3CGGu6s5wiQmiNE Ge4oZFAmnFj2QXGxaHXvRSVQP7VfhZgFpFwFLb5m/zwf31nSXTaMZf6HO1tHY6tHB3gX 0yUNJjmZAoocWH6O3iPNAsjiv6OUxbBvXWniXPo32w6ddD/E59dBzCBfXwzd6Abwp/lR etD8FVPTZDGw1hQS50rNYVO/tQ/bExS9b2eZ/aTWGr0KAc/jERJE6hxRy7XKVDCWD980 RQM3Y1/tE7FNeGuSEMAgDGohRih6boomoQW/XlEUgY5D9CCuwIi/UF2nH7c0SXjNEPfc wJ6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="S/yK6gJx"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e12-20020a50ec8c000000b0049defeaec6dsi5614817edr.530.2023.02.10.10.19.46; Fri, 10 Feb 2023 10:20:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="S/yK6gJx"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232979AbjBJSS5 (ORCPT + 99 others); Fri, 10 Feb 2023 13:18:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232932AbjBJSSo (ORCPT ); Fri, 10 Feb 2023 13:18:44 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63EB15DC37; Fri, 10 Feb 2023 10:18:43 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0089E61E6E; Fri, 10 Feb 2023 18:18:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 323C1C433A1; Fri, 10 Feb 2023 18:18:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053122; bh=4Qxghh6QU4N932aY/0tlyYMGET6IecDHbvbpkYpyZTo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S/yK6gJxX2E0Y+4e3Z47a9QDTM40XVFXSnhwq1fgcVSCvaoiNGRdrDc7dGw1AlFwG oQKE2o+qeNyMPdTs9n8B9ThgrCB4ZPDkJ577/NDAr7sqTCWfzFzxCZqQqw2QEW7FQA fI6ubo5YZKi5SaaVxNxodZzutRj9uikOq4J/bIoNZ1/peeOI8St1RMgTZiXCgI8rM/ slU7ll0DYEQ8TPBSJTOhzSZKoVLmErqkwAu/w5H3JHVS/bvb5I0psWEIxjA+KApM4s TkcYpoWgYEuHwDMbRVCCOWzYhTCATotRBFrPqKt1EjyzXkT5qSAd4SqnpY63FutTUm sWkvWQaHXgrig== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada Subject: [PATCH v5 4/7] kbuild: srcrpm-pkg: create source package without cleaning Date: Sat, 11 Feb 2023 03:18:25 +0900 Message-Id: <20230210181828.124765-4-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210181828.124765-1-masahiroy@kernel.org> References: <20230210181828.124765-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757469169556488534?= X-GMAIL-MSGID: =?utf-8?q?1757469169556488534?= If you run 'make (src)rpm-pkg', all objects are lost due to 'make clean', which makes the incremental builds impossible. Instead of cleaning, pass the exclude list to tar's --exclude-from option. Previously, the .config was contained in the source tarball. With this commit, the source rpm consists of separate linux.tar.gz and .config. Remove stale comments. Now, 'make (src)rpm-pkg' works with O= option. Signed-off-by: Masahiro Yamada --- (no changes since v4) Changes in v4: - Do not delete the old tar command because it is still used by snap-pkg although snap-pkg is broken, and it does not work at all. scripts/Makefile.package | 29 +++-------------------------- scripts/package/mkspec | 8 ++++---- 2 files changed, 7 insertions(+), 30 deletions(-) diff --git a/scripts/Makefile.package b/scripts/Makefile.package index 0b66bd667212..39f67f876092 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -3,27 +3,6 @@ include $(srctree)/scripts/Kbuild.include -# RPM target -# --------------------------------------------------------------------------- -# The rpm target generates two rpm files: -# /usr/src/packages/SRPMS/kernel-2.6.7rc2-1.src.rpm -# /usr/src/packages/RPMS/i386/kernel-2.6.7rc2-1..rpm -# The src.rpm files includes all source for the kernel being built -# The .rpm includes kernel configuration, modules etc. -# -# Process to create the rpm files -# a) clean the kernel -# b) Generate .spec file -# c) Build a tar ball, using symlink to make kernel version -# first entry in the path -# d) and pack the result to a tar.gz file -# e) generate the rpm files, based on kernel.spec -# - Use /. to avoid tar packing just the symlink - -# Note that the rpm-pkg target cannot be used with KBUILD_OUTPUT, -# but the binrpm-pkg target can; for some reason O= gets ignored. - -# Remove hyphens since they have special meaning in RPM filenames KERNELPATH := kernel-$(subst -,_,$(KERNELRELEASE)) KDEB_SOURCENAME ?= linux-upstream KBUILD_PKG_ROOTCMD ?="fakeroot -u" @@ -61,12 +40,10 @@ rpm-pkg: srcrpm-pkg # srcrpm-pkg # --------------------------------------------------------------------------- PHONY += srcrpm-pkg -srcrpm-pkg: - $(MAKE) clean +srcrpm-pkg: linux.tar.gz $(CONFIG_SHELL) $(MKSPEC) >$(objtree)/kernel.spec - $(call cmd,src_tar,$(KERNELPATH),kernel.spec) - +rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -ts $(KERNELPATH).tar.gz \ - --define='_smp_mflags %{nil}' --define='_srcrpmdir $(srctree)' + +rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -bs kernel.spec \ + --define='_smp_mflags %{nil}' --define='_sourcedir .' --define='_srcrpmdir .' # binrpm-pkg # --------------------------------------------------------------------------- diff --git a/scripts/package/mkspec b/scripts/package/mkspec index 108c0cb95436..83a64d9d7372 100755 --- a/scripts/package/mkspec +++ b/scripts/package/mkspec @@ -47,7 +47,8 @@ sed -e '/^DEL/d' -e 's/^\t*//' < X-Patchwork-Id: 55519 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1111958wrn; Fri, 10 Feb 2023 10:34:24 -0800 (PST) X-Google-Smtp-Source: AK7set8jGHR7LXhNBuyGqDZJ+O8jJiI0xli9/1oPZ5KHIrniXNAsHw3ouxN7YubUm+q0ozYmEIvN X-Received: by 2002:aa7:9988:0:b0:5a8:4bf8:1752 with SMTP id k8-20020aa79988000000b005a84bf81752mr7644486pfh.32.1676054064576; Fri, 10 Feb 2023 10:34:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676054064; cv=none; d=google.com; s=arc-20160816; b=wEZOQ5ijNbdKCK7tMml1fyDwIeS50vITCMkx6gACFpm1rLTb447lvBXrzNAl3ICsXw RhOjbwqlb7rPULDWb88usDy8Dv/62iAzwdpQQqyij5PCbX3tQ9PVs86AZ+UdV3JMLOBY v705D/56af0oUvNCQEjnYTcWNzOEQbivQEpF37Jc5EFCVaCvuavTPxMqxjDX0mlmWoif Hop7sMpOpv9RE2345oFfjGkZL5U0x9KfZAF2eBcgwfLZkJV/Qs8Ay38GJFyW3dSwu2gh qotrPgwSwBma5qhXpsxTZlBt+urJMj1xNxS8Hfg1gStpTUMFBRme5WIyHXU/pedbEz0N lhvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9n8z6/j+ZxeIMdTGv5ZSCwjHq2dFqRORWYq0G1IKQR0=; b=ajA5bXNZoOsefBY164D/vURT79GE8Dv3QspmM6XXWT2mQSnBb1eTTS1BQlMIimBYNJ mrpB9PHipuPZRmVMK6Gi/Y9X07ZtnkezovqhKj5z/JGw4dpDArxLbYD5acoAnq5e+JpO 2DUPVDnH6DpkWolnEhC9Dcz01O3N5ZkuWWiE9RSwFBx8bHcFn7M366o3aHc2wEpTjrg7 60zSkHX/MIh7Q69PQpH8Xqcem5nS/rzBoiyvjODPCBxwUjKm0BQPuODombqyoe/k/Jc6 0xjWMahpUXEDdHymNlAeRVm5iCAbnt/fa4QL1qfSVTrrF+NibebvSJWjS3fnag4H6Oz/ 8q1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=vQjeMU5R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 186-20020a6219c3000000b0059242ce2458si4915113pfz.342.2023.02.10.10.34.10; Fri, 10 Feb 2023 10:34:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=vQjeMU5R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233029AbjBJSTF (ORCPT + 99 others); Fri, 10 Feb 2023 13:19:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232916AbjBJSS4 (ORCPT ); Fri, 10 Feb 2023 13:18:56 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 164D15DC0C; Fri, 10 Feb 2023 10:18:47 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id AB163B825A9; Fri, 10 Feb 2023 18:18:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5430C433D2; Fri, 10 Feb 2023 18:18:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053124; bh=qGlkzx4kts7v08dgnhKLSWrqXLO9a0rN1+g/GlO6Wr4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vQjeMU5RMEO6p4sBNGm7WyCDrChwYmHd1BAZ3BjbRYWvYOZENCNGc319TUJGwZt0Q Cr32jFChEwdvhy+tCo+0ispSPJ85Xi+kHD7Jm/YZzxIwrBYB49oZirBixE/VTboOym rWvhWTpwQnXjLV6Wb1OJTKvB8JASqzPaCzYUzfkWGcwGv0ts5XZj3zhe4kl0wjK9qZ YC7wgemv9EM2RqlSZNN4gdG5vHoPVcjwTcl8RhcmEC7rKh2CublwwShlD3sT3aWXAD P0PX8yc4lIuJfxF8a6MvjZH/WeerqFWOkGwZ0UPrimFaL0JorFicPSSL8gF0rszcWO rtCBodF6MHOSQ== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada Subject: [PATCH v5 5/7] kbuild: deb-pkg: hide KDEB_SOURCENAME from Makefile Date: Sat, 11 Feb 2023 03:18:26 +0900 Message-Id: <20230210181828.124765-5-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210181828.124765-1-masahiroy@kernel.org> References: <20230210181828.124765-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757470066446718535?= X-GMAIL-MSGID: =?utf-8?q?1757470066446718535?= scripts/Makefile.package does not need to know the value of KDEB_SOURCENAME because the source name can be taken from debian/changelog by using dpkg-parsechangelog. Move the default of KDEB_SOURCENAME (i.e. linux-upstream) to scripts/package/mkdebian. Signed-off-by: Masahiro Yamada --- Changes in v5: - Copy linux.tar.gz to .orig tarball Changes in v3: - Move cmd_debianize Changes in v2: - New patch scripts/Makefile.package | 22 ++++++++++++++-------- scripts/package/mkdebian | 2 +- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/scripts/Makefile.package b/scripts/Makefile.package index 39f67f876092..b591f036e63f 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -4,9 +4,7 @@ include $(srctree)/scripts/Kbuild.include KERNELPATH := kernel-$(subst -,_,$(KERNELRELEASE)) -KDEB_SOURCENAME ?= linux-upstream KBUILD_PKG_ROOTCMD ?="fakeroot -u" -export KDEB_SOURCENAME # Include only those top-level files that are needed by make, plus the GPL copy TAR_CONTENT := Documentation LICENSES arch block certs crypto drivers fs \ include init io_uring ipc kernel lib mm net rust \ @@ -54,17 +52,25 @@ binrpm-pkg: +rpmbuild $(RPMOPTS) --define "_builddir $(objtree)" --target \ $(UTS_MACHINE)-linux -bb $(objtree)/binkernel.spec +quiet_cmd_debianize = GEN $@ + cmd_debianize = $(srctree)/scripts/package/mkdebian + +debian: FORCE + $(call cmd,debianize) + +PHONY += debian-orig +debian-orig: private source = $(shell dpkg-parsechangelog -S Source) +debian-orig: private version = $(shell dpkg-parsechangelog -S Version | sed 's/-[^-]*$$//') +debian-orig: linux.tar.gz debian + $(Q)cp $< ../$(source)_$(version).orig.tar.gz + PHONY += deb-pkg -deb-pkg: linux.tar.gz - $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian - $(Q)origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\ - cp linux.tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz +deb-pkg: debian-orig +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \ --build=source,binary --source-option=-sP -nc -us -uc PHONY += bindeb-pkg -bindeb-pkg: - $(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian +bindeb-pkg: debian +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -b -nc -uc PHONY += intdeb-pkg diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index 68b13ef590ba..534e9713761a 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -95,7 +95,7 @@ else revision=$($srctree/init/build-version) packageversion=$version-$revision fi -sourcename=$KDEB_SOURCENAME +sourcename=${KDEB_SOURCENAME:-linux-upstream} if [ "$ARCH" = "um" ] ; then packagename=user-mode-linux From patchwork Fri Feb 10 18:18:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 55517 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1106563wrn; Fri, 10 Feb 2023 10:22:39 -0800 (PST) X-Google-Smtp-Source: AK7set+9MZhUmo9R4MUIgL34hZu25uar0dM+pH10EeKPhT82OHZRbGcpvYyy+8cSCH2RDWn5/JeM X-Received: by 2002:a17:906:4913:b0:88c:9b39:23c6 with SMTP id b19-20020a170906491300b0088c9b3923c6mr16810806ejq.14.1676053359029; Fri, 10 Feb 2023 10:22:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676053359; cv=none; d=google.com; s=arc-20160816; b=Nv7Mwlu5j302Z7vD6IplI5Rzn5g46Z5FknkAh0XZauTXPrq7cNgYBpWpFnKnVL4i72 oKsgzHfbVrs/NiY54X58SIaBUc3srbLCdwR+sqLZhZea3Qpi5h97/6eoPmMHssSWrNHb HKPfnVYQK/8+HINqb3sT7VTh69r7Vi9RaG8Wet9v801S34QFLoN3FbHDRw5wpQliW6RH PHyV08pjeKde6odMhSYh8+N0nbH/TJ6SKQPMriFV9UHf0EensWC5g3FRRZKafyPHUEBH wHv5PWQG6Bd66tQ3z7+Efqnd57VQMQMNbvx2CScWX0QyT1/T1yTXmRNKeOzPgjfirhky HB5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WiMKaqTNiEev3lxBlYiTONdej8Iy0h4ebTQCRVRjz8E=; b=P3dIQU2T4Oiu3E9KnLY9kvujgc0EDIeOs8BILazxsCKLBR38NsNxOOIuoHRGcyMqCG 5KQV71e0DR5T7zmIRrxmadj8a8wLGMn3/F6nkCJoZs8K8r7XC1wFR0eK4sRDiKILEe8D LNv9Sun4xedxAi2EfBOONG3OC/vjoTZqs59LIrs4VobjR1eDyh1c6dtY3ULNgHQ/Plyl fkfS0ap6uNO5OWlQPvg7tspryAlOQg5iEDjf5ai5PaUZwjgyu0LboMaUd8aAR+1hMA4Y Pmo6E/e7XZ4Vo7CeOJpvQ67kl7obv9iPxtWnrRF4Eu8DzcrCE84WuSFze5stwubq/Z8w l1ZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="qiFF5L/R"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h8-20020a17090791c800b0088a35d495a2si5824984ejz.293.2023.02.10.10.22.14; Fri, 10 Feb 2023 10:22:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="qiFF5L/R"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233017AbjBJSTD (ORCPT + 99 others); Fri, 10 Feb 2023 13:19:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232944AbjBJSS4 (ORCPT ); Fri, 10 Feb 2023 13:18:56 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C66268AE5; Fri, 10 Feb 2023 10:18:47 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1627D61E00; Fri, 10 Feb 2023 18:18:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E1021C4339B; Fri, 10 Feb 2023 18:18:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053126; bh=S9ku+PHQJhdA2T48EMJ9NZiif3Y3/eyMvf3hs1erLEg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qiFF5L/RQYnfZjXuON5e0fb60QBvxGtAiCo7SuOvKohv4hCBzj0mk0H95sYKCiTkM GvRtJ6GsFZAjTEBhVKSfRuLgHImaGKEIC+5KfRHbHzESw1KFweNvAjWbKojVG1cQ/L 20WndPp2vEiGPfsKls86YzxoAxfMiW8KquBvQAJgmyJMNfByft0Vp8iQK4nVRxDu3P yRqsRJdPA+K0ZJZ+h0BBr2pBrOs4CiwG5zVwcRT37i39g+/a9SJGpfTcUtTSoNGl3u KpZIt0mPDpP7i0gWXgu8IUYo41S3UN5epj3sD0T3CYUvKma8aZVlCY3mgFMMzieVb3 CTFqnDLkykH1Q== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada Subject: [PATCH v5 6/7] kbuild: deb-pkg: switch over to source format 3.0 (quilt) Date: Sat, 11 Feb 2023 03:18:27 +0900 Message-Id: <20230210181828.124765-6-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210181828.124765-1-masahiroy@kernel.org> References: <20230210181828.124765-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757469326749200233?= X-GMAIL-MSGID: =?utf-8?q?1757469326749200233?= Change the source format from "1.0" to "3.0 (quilt)" because it works more cleanly. All files except .config and debian/ go into the .orig tarball. You can add a single patch, debian/patches/config, and delete the ugly extend-diff-ignore patterns. The debian tarball will be compressed into *.debian.tar.xz by default. If you like to use a different compression mode, you can pass the command line option, DPKG_FLAGS=-Zgzip, for example. The .orig tarball only supports gzip for now. The combination of gzip and xz is somewhat clumsy, but it is not a practical problem. Signed-off-by: Masahiro Yamada --- (no changes since v4) Changes in v4: - New patch scripts/Makefile.package | 2 +- scripts/package/mkdebian | 41 ++++++++++++++++++---------------------- 2 files changed, 19 insertions(+), 24 deletions(-) diff --git a/scripts/Makefile.package b/scripts/Makefile.package index b591f036e63f..55be31d0e040 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -67,7 +67,7 @@ debian-orig: linux.tar.gz debian PHONY += deb-pkg deb-pkg: debian-orig +dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \ - --build=source,binary --source-option=-sP -nc -us -uc + --build=source,binary -nc -us -uc PHONY += bindeb-pkg bindeb-pkg: debian diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index 534e9713761a..1ab4c6ee76d9 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -135,29 +135,24 @@ else fi mkdir -p debian/source/ -echo "1.0" > debian/source/format - -# Ugly: ignore anything except .config or debian/ -cat<<'EOF' > debian/source/local-options -diff-ignore - -extend-diff-ignore = ^[^.d] - -extend-diff-ignore = ^\.[^c] -extend-diff-ignore = ^\.c($|[^o]) -extend-diff-ignore = ^\.co($|[^n]) -extend-diff-ignore = ^\.con($|[^f]) -extend-diff-ignore = ^\.conf($|[^i]) -extend-diff-ignore = ^\.confi($|[^g]) -extend-diff-ignore = ^\.config. - -extend-diff-ignore = ^d($|[^e]) -extend-diff-ignore = ^de($|[^b]) -extend-diff-ignore = ^deb($|[^i]) -extend-diff-ignore = ^debi($|[^a]) -extend-diff-ignore = ^debia($|[^n]) -extend-diff-ignore = ^debian[^/] -EOF +echo "3.0 (quilt)" > debian/source/format + +{ + echo "diff-ignore" + echo "extend-diff-ignore = .*" +} > debian/source/local-options + +# Add .config as a patch +mkdir -p debian/patches +{ + echo "Subject: Add .config" + echo "Author: ${maintainer}" + echo + echo "--- /dev/null" + echo "+++ linux/.config" + diff -u /dev/null .config | tail -n +3 +} > debian/patches/config +echo config > debian/patches/series echo $debarch > debian/arch extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)" From patchwork Fri Feb 10 18:18:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 55518 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1106689wrn; Fri, 10 Feb 2023 10:22:57 -0800 (PST) X-Google-Smtp-Source: AK7set9JcFWGow4NY3XulEsxdPff70z7Nslg4TG5HSj1+S3cTI+EV4ayIOH8KBVeYH9oz2NDzhPv X-Received: by 2002:a17:907:9714:b0:8ab:7b03:9a03 with SMTP id jg20-20020a170907971400b008ab7b039a03mr14568483ejc.16.1676053377601; Fri, 10 Feb 2023 10:22:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676053377; cv=none; d=google.com; s=arc-20160816; b=EZi3j6Q8wGdDEmAjnTJYhkshiV+aEZ4pjSitTI939FLhp5IIfyGbnD8uNP4viIu5Bv mXFgHSuRm5CbTIuHvI0kfLrlKCOCWSfd2ZOAph7yKSb904oQXhs/V2hXn641qtfFmjoF WQct+rIhubKeekBmjNX1CBZbEy/YPYMX6u/RU8tL2HWFbEgQYcIpt46ykJQ9HaAlg5tq yE9z3zQQq1EwkqdDxGmeVXJUuyD6VbFckm9YrUqIbtHvDXkqPd4OjMgjZx7zE2jdJK0O 9y4FjImkDiuDM4BKKHuFTnO+nZMqt5KvcJpwrFgqat6WiOdWkvb/jF7V6XaddDq/MPf3 1gIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EzAEttIIDKk9P3Bk4BoIooUajOFpowPo4R+4kmwrBok=; b=DQrFH1eRTyGmsv7udEJSEZLepbZcdt5OBo/G9g0M07XI6m9asvfS5vhgcJiVL864Ja qMjd0JiMYBBT1grxee5M3ukaz61FRlfohKCw1BJkAQaeGqQz97cmdy6Y2e4MJeTjnF6x 5PDeXmnapgX2fZId97fQm/oAkMMWYwnCvskJp43PE8KQOz7lYNMWg32p/AwfxuS3giZN hsS340enNAJjS8zvqfs9GBVKlmmaZu3fOkLpowC5T/UMILW86zv3GUD306sw0IgCIG7i Ayj1GOiS+9z1nYtOCKkiZASYHdN7sRkIqWbn8D1FtIpGMIiSgNE1iD+cl4XBKkGNEbAZ tRBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ou+H9jRT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jr18-20020a170906515200b008718874c66asi5175548ejc.626.2023.02.10.10.22.33; Fri, 10 Feb 2023 10:22:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ou+H9jRT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233054AbjBJSTM (ORCPT + 99 others); Fri, 10 Feb 2023 13:19:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232974AbjBJSS5 (ORCPT ); Fri, 10 Feb 2023 13:18:57 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CC646F22C; Fri, 10 Feb 2023 10:18:51 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 778AEB80C8B; Fri, 10 Feb 2023 18:18:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ECC0DC4339E; Fri, 10 Feb 2023 18:18:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676053128; bh=fhbDbBrH4mf4Re3SJLCSFDHbs63IuiydRRp3WHTmHEw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ou+H9jRTWeKPlIqO+aRnnrKWsqXNrzAEbaFgslSmdUmkuPhfkKUc3/oktTS3yLMnQ vxtfo9MUzckjZFxThqwLTZUrnW0kPafvX6AJ6ihqvu0NGT10ZHzDd+xto+ulqnFYOC LLX00JcPf3WlFUFg6rYth2cBR7ABCPGBXuff9izRjsAb7arE+i+K1n6+GThvZmxjaS cP6OLmUglcnaSjPQLZ0VQQvmggth55bxWmKtkZ7GuBl1DZEXq9Qwz5Mum6cKGANHQ/ vLDhkzGN9EkwOaHRNC/cetIc/uL6ueBJ2bdTjOSRGlnkMY+0fElMhemK6Yj4R+jA/e EotUqLWd1kSDA== From: Masahiro Yamada To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor , Nick Desaulniers , Nicolas Schier , Ben Hutchings , Masahiro Yamada Subject: [PATCH v5 7/7] kbuild: make perf-tar*-src-pkg work without relying on git Date: Sat, 11 Feb 2023 03:18:28 +0900 Message-Id: <20230210181828.124765-7-masahiroy@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210181828.124765-1-masahiroy@kernel.org> References: <20230210181828.124765-1-masahiroy@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757469346807711432?= X-GMAIL-MSGID: =?utf-8?q?1757469346807711432?= Currently, perf-tar*-src-pkg only uses 'git archive', but it is better to make it work without relying on git. The file, HEAD, which saves the commit hash, will be included in the tarball only when the source tree is managed by git. The git tree is more precisely checked; it has been copied from scripts/setlocalversion. Signed-off-by: Masahiro Yamada --- Changes in v5: - New patch scripts/Makefile.package | 67 ++++++++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 27 deletions(-) diff --git a/scripts/Makefile.package b/scripts/Makefile.package index 55be31d0e040..e676d14fef23 100644 --- a/scripts/Makefile.package +++ b/scripts/Makefile.package @@ -122,40 +122,53 @@ quiet_cmd_tar = TAR $@ tar-rootdir := $(srctree) +%.tar: compress-opt := +%.tar: .tmp_filelist + $(call cmd,tar) + %.tar.gz: compress-opt := -I $(KGZIP) %.tar.gz: .tmp_filelist $(call cmd,tar) +%.tar.bz2: compress-opt := -I $(KBZIP2) +%.tar.bz2: .tmp_filelist + $(call cmd,tar) + +%.tar.xz: compress-opt := -I $(XZ) +%.tar.xz: .tmp_filelist + $(call cmd,tar) + +%.tar.zst: compress-opt := -I $(ZSTD) +%.tar.zst: .tmp_filelist + $(call cmd,tar) + # perf-pkg - generate a source tarball with perf source # --------------------------------------------------------------------------- -perf-tar=perf-$(KERNELVERSION) - -quiet_cmd_perf_tar = TAR - cmd_perf_tar = \ -git --git-dir=$(srctree)/.git archive --prefix=$(perf-tar)/ \ - HEAD^{tree} $$(cd $(srctree); \ - echo $$(cat tools/perf/MANIFEST)) \ - -o $(perf-tar).tar; \ -mkdir -p $(perf-tar); \ -git --git-dir=$(srctree)/.git rev-parse HEAD > $(perf-tar)/HEAD; \ -(cd $(srctree)/tools/perf; \ -util/PERF-VERSION-GEN $(CURDIR)/$(perf-tar)/); \ -tar rf $(perf-tar).tar $(perf-tar)/HEAD $(perf-tar)/PERF-VERSION-FILE; \ -rm -r $(perf-tar); \ -$(if $(findstring tar-src,$@),, \ -$(if $(findstring bz2,$@),$(KBZIP2), \ -$(if $(findstring gz,$@),$(KGZIP), \ -$(if $(findstring xz,$@),$(XZ), \ -$(if $(findstring zst,$@),$(ZSTD), \ -$(error unknown target $@))))) \ - -f -9 $(perf-tar).tar) - -perf-tar-pkgs := perf-tar-src-pkg perf-targz-src-pkg perf-tarbz2-src-pkg \ - perf-tarxz-src-pkg perf-tarzst-src-pkg -PHONY += $(perf-tar-pkgs) -$(perf-tar-pkgs): - $(call cmd,perf_tar) +perf-tar = perf-$(KERNELVERSION) + +perf-tar-phony = perf-$(subst .,,$(1))-src-pkg + +.tmp_perf: FORCE + $(Q)rm -rf $@ + $(Q)mkdir $@ + $(Q)tar -c -f - -C $(srctree) --files-from=$(srctree)/tools/perf/MANIFEST | tar -x -f - -C $@ + $(Q)if test -z "$(git -C $(srctree) rev-parse --show-cdup 2>/dev/null)" && \ + head=$$(git -C $(srctree) rev-parse --verify HEAD 2>/dev/null); then \ + echo $$head > $@/HEAD; \ + fi + $(Q)cd $(srctree)/tools/perf; util/PERF-VERSION-GEN $(abspath $@)/ + +define perf-pkg-rule +PHONY += $(perf-tar-phony) +$(perf-tar-phony): $(perf-tar).$(1) + @: + +$(perf-tar).$(1): private tar-rootdir := .tmp_perf +$(perf-tar).$(1): | .tmp_perf +endef + +$(foreach x, tar tar.gz tar.bz2 tar.xz tar.zst, $(eval $(call perf-pkg-rule,$(x)))) # Help text displayed when executing 'make help' # ---------------------------------------------------------------------------