diff mbox series

Makefile.debug: support for -gz=zstd

Message ID	20221020175655.1660864-1-ndesaulniers@google.com
State	New
Headers	Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Date: Thu, 20 Oct 2022 10:56:49 -0700 Mime-Version: 1.0 Message-ID: <20221020175655.1660864-1-ndesaulniers@google.com> Subject: [PATCH] Makefile.debug: support for -gz=zstd From: Nick Desaulniers <ndesaulniers@google.com> To: Masahiro Yamada <masahiroy@kernel.org> Cc: Fangrui Song <maskray@google.com>, Nick Desaulniers <ndesaulniers@google.com>, Sedat Dilek <sedat.dilek@dhl.com>, Michal Marek <michal.lkml@markovi.net>, Nick Terrell <terrelln@fb.com>, Nathan Chancellor <nathan@kernel.org>, Tom Rix <trix@redhat.com>, Andrew Morton <akpm@linux-foundation.org>, "Peter Zijlstra (Intel)" <peterz@infradead.org>, David Gow <davidgow@google.com>, Kees Cook <keescook@chromium.org>, Josh Poimboeuf <jpoimboe@kernel.org>, Dan Williams <dan.j.williams@intel.com>, Miguel Ojeda <ojeda@kernel.org>, Isabella Basso <isabbasso@riseup.net>, Vlastimil Babka <vbabka@suse.cz>, Rasmus Villemoes <linux@rasmusvillemoes.dk>, linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Precedence: bulk
Series	Makefile.debug: support for -gz=zstd \| Makefile.debug: support for -gz=zstd

Commit Message

Nick Desaulniers Oct. 20, 2022, 5:56 p.m. UTC

  Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
uses zstd.

Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
488M vmlinux
27.6%   136Mi   0.0%       0    .debug_info
 6.1%  30.2Mi   0.0%       0    .debug_str_offsets
 3.5%  17.2Mi   0.0%       0    .debug_line
 3.3%  16.3Mi   0.0%       0    .debug_loclists
 0.9%  4.62Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
385M vmlinux
21.8%  85.4Mi   0.0%       0    .debug_info
 2.1%  8.26Mi   0.0%       0    .debug_str_offsets
 2.1%  8.24Mi   0.0%       0    .debug_loclists
 1.9%  7.48Mi   0.0%       0    .debug_line
 0.5%  1.94Mi   0.0%       0    .debug_str

clang-16, x86_64 defconfig plus
CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
373M vmlinux
21.4%  81.4Mi   0.0%       0    .debug_info
 2.3%  8.85Mi   0.0%       0    .debug_loclists
 1.5%  5.71Mi   0.0%       0    .debug_line
 0.5%  1.95Mi   0.0%       0    .debug_str_offsets
 0.4%  1.62Mi   0.0%       0    .debug_str

That's only a 3.11% overall binary size savings over zlib, but at no
performance regression.

Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
Suggested-by: Sedat Dilek (DHL Supply Chain) <sedat.dilek@dhl.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
---
 lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
 scripts/Makefile.debug |  4 ++++
 2 files changed, 29 insertions(+), 1 deletion(-)

Comments

Nathan Chancellor Oct. 20, 2022, 6:39 p.m. UTC | #1

On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> uses zstd.
> 
> Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
> 
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> 488M vmlinux
> 27.6%   136Mi   0.0%       0    .debug_info
>  6.1%  30.2Mi   0.0%       0    .debug_str_offsets
>  3.5%  17.2Mi   0.0%       0    .debug_line
>  3.3%  16.3Mi   0.0%       0    .debug_loclists
>  0.9%  4.62Mi   0.0%       0    .debug_str
> 
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> 385M vmlinux
> 21.8%  85.4Mi   0.0%       0    .debug_info
>  2.1%  8.26Mi   0.0%       0    .debug_str_offsets
>  2.1%  8.24Mi   0.0%       0    .debug_loclists
>  1.9%  7.48Mi   0.0%       0    .debug_line
>  0.5%  1.94Mi   0.0%       0    .debug_str
> 
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> 373M vmlinux
> 21.4%  81.4Mi   0.0%       0    .debug_info
>  2.3%  8.85Mi   0.0%       0    .debug_loclists
>  1.5%  5.71Mi   0.0%       0    .debug_line
>  0.5%  1.95Mi   0.0%       0    .debug_str_offsets
>  0.4%  1.62Mi   0.0%       0    .debug_str
> 
> That's only a 3.11% overall binary size savings over zlib, but at no
> performance regression.
> 
> Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> Suggested-by: Sedat Dilek (DHL Supply Chain) <sedat.dilek@dhl.com>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

One small comment below.

> ---
>  lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
>  scripts/Makefile.debug |  4 ++++
>  2 files changed, 29 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 3fc7abffc7aa..4085ac77dc12 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
>  	  DEBUG_INFO build and compile times are reduced too.
>  	  Only works with newer gcc versions.
>  
> +choice
> +	prompt "Compressed Debug information"
> +	depends on DEBUG_KERNEL

I think you can drop this depends. The entire block is in an
'if DEBUG_INFO', which can only be true if CONFIG_DEBUG_KERNEL is set
because of the dependencies of the "Debug information" prompt above this
file, which is how CONFIG_DEBUG_INFO is selected.

> +	help
> +	  Compress the resulting debug info. Results in smaller debug info sections,
> +	  but requires that consumers are able to decompress the results.
> +
> +	  If unsure, choose DEBUG_INFO_UNCOMPRESSED.
> +
> +config DEBUG_INFO_UNCOMPRESSED
> +	bool "Don't compress debug information"
> +	help
> +	  Don't compress debug info sections.
> +
>  config DEBUG_INFO_COMPRESSED
> -	bool "Compressed debugging information"
> +	bool "Compress debugging information with zlib"
>  	depends on $(cc-option,-gz=zlib)
>  	depends on $(ld-option,--compress-debug-sections=zlib)
>  	help
> @@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
>  	  preferable to setting $KDEB_COMPRESS to "none" which would be even
>  	  larger.
>  
> +config DEBUG_INFO_COMPRESSED_ZSTD
> +	bool "Compress debugging information with zstd"
> +	depends on $(cc-option,-gz=zstd)
> +	depends on $(ld-option,--compress-debug-sections=zstd)
> +	help
> +	  Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
> +	  16.0+, binutils 2.40+, and zstd.
> +
> +endchoice # "Compressed Debug information"
> +
>  config DEBUG_INFO_SPLIT
>  	bool "Produce split debuginfo in .dwo files"
>  	depends on $(cc-option,-gsplit-dwarf)
> diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> index 332c486f705f..8ac3379d2255 100644
> --- a/scripts/Makefile.debug
> +++ b/scripts/Makefile.debug
> @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
>  DEBUG_CFLAGS	+= -gz=zlib
>  KBUILD_AFLAGS	+= -gz=zlib
>  KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
> +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> +DEBUG_CFLAGS	+= -gz=zstd
> +KBUILD_AFLAGS	+= -gz=zstd
> +KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
>  endif
>  
>  KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)
> -- 
> 2.38.0.135.g90850a2211-goog
>

Sedat Dilek Oct. 23, 2022, 3:45 p.m. UTC | #2

On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> > default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> > uses zstd.
> >
> > Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
> >
> > clang-16, x86_64 defconfig plus
> > CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> > 488M vmlinux
> > 27.6%   136Mi   0.0%       0    .debug_info
> >  6.1%  30.2Mi   0.0%       0    .debug_str_offsets
> >  3.5%  17.2Mi   0.0%       0    .debug_line
> >  3.3%  16.3Mi   0.0%       0    .debug_loclists
> >  0.9%  4.62Mi   0.0%       0    .debug_str
> >
> > clang-16, x86_64 defconfig plus
> > CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> > Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> > 385M vmlinux
> > 21.8%  85.4Mi   0.0%       0    .debug_info
> >  2.1%  8.26Mi   0.0%       0    .debug_str_offsets
> >  2.1%  8.24Mi   0.0%       0    .debug_loclists
> >  1.9%  7.48Mi   0.0%       0    .debug_line
> >  0.5%  1.94Mi   0.0%       0    .debug_str
> >
> > clang-16, x86_64 defconfig plus
> > CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> > 373M vmlinux
> > 21.4%  81.4Mi   0.0%       0    .debug_info
> >  2.3%  8.85Mi   0.0%       0    .debug_loclists
> >  1.5%  5.71Mi   0.0%       0    .debug_line
> >  0.5%  1.95Mi   0.0%       0    .debug_str_offsets
> >  0.4%  1.62Mi   0.0%       0    .debug_str
> >
> > That's only a 3.11% overall binary size savings over zlib, but at no
> > performance regression.
> >
> > Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> > Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> > Suggested-by: Sedat Dilek (DHL Supply Chain) <sedat.dilek@dhl.com>
> > Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
>
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
>
> One small comment below.
>
> > ---
> >  lib/Kconfig.debug      | 26 +++++++++++++++++++++++++-
> >  scripts/Makefile.debug |  4 ++++
> >  2 files changed, 29 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> > index 3fc7abffc7aa..4085ac77dc12 100644
> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -312,8 +312,22 @@ config DEBUG_INFO_REDUCED
> >         DEBUG_INFO build and compile times are reduced too.
> >         Only works with newer gcc versions.
> >
> > +choice
> > +     prompt "Compressed Debug information"
> > +     depends on DEBUG_KERNEL
>
> I think you can drop this depends. The entire block is in an
> 'if DEBUG_INFO', which can only be true if CONFIG_DEBUG_KERNEL is set
> because of the dependencies of the "Debug information" prompt above this
> file, which is how CONFIG_DEBUG_INFO is selected.
>

Yes, dropping this is good.

> > +     help
> > +       Compress the resulting debug info. Results in smaller debug info sections,
> > +       but requires that consumers are able to decompress the results.
> > +
> > +       If unsure, choose DEBUG_INFO_UNCOMPRESSED.
> > +
> > +config DEBUG_INFO_UNCOMPRESSED
> > +     bool "Don't compress debug information"
> > +     help
> > +       Don't compress debug info sections.
> > +
> >  config DEBUG_INFO_COMPRESSED

Also, I prefer...

DEBUG_INFO_COMPRESSED_NONE
DEBUG_INFO_COMPRESSED_ZLIB
DEBUG_INFO_COMPRESSED_ZSTD

Following...

$ ld.lld-16 -v
Debian LLD 16.0.0 (compatible with GNU linkers)

$ ld.lld-16 --help | grep compress-debug-sections
 --compress-debug-sections=[none,zlib,zstd]

> > -     bool "Compressed debugging information"
> > +     bool "Compress debugging information with zlib"
> >       depends on $(cc-option,-gz=zlib)
> >       depends on $(ld-option,--compress-debug-sections=zlib)
> >       help
> > @@ -327,6 +341,16 @@ config DEBUG_INFO_COMPRESSED
> >         preferable to setting $KDEB_COMPRESS to "none" which would be even
> >         larger.
> >

Anyone, checked $KDEB_COMPRESS - does this need changes?

-Sedat-

> > +config DEBUG_INFO_COMPRESSED_ZSTD
> > +     bool "Compress debugging information with zstd"
> > +     depends on $(cc-option,-gz=zstd)
> > +     depends on $(ld-option,--compress-debug-sections=zstd)
> > +     help
> > +       Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
> > +       16.0+, binutils 2.40+, and zstd.
> > +
> > +endchoice # "Compressed Debug information"
> > +
> >  config DEBUG_INFO_SPLIT
> >       bool "Produce split debuginfo in .dwo files"
> >       depends on $(cc-option,-gsplit-dwarf)
> > diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> > index 332c486f705f..8ac3379d2255 100644
> > --- a/scripts/Makefile.debug
> > +++ b/scripts/Makefile.debug
> > @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED
> >  DEBUG_CFLAGS += -gz=zlib
> >  KBUILD_AFLAGS        += -gz=zlib
> >  KBUILD_LDFLAGS       += --compress-debug-sections=zlib
> > +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> > +DEBUG_CFLAGS += -gz=zstd
> > +KBUILD_AFLAGS        += -gz=zstd
> > +KBUILD_LDFLAGS       += --compress-debug-sections=zstd
> >  endif
> >
> >  KBUILD_CFLAGS        += $(DEBUG_CFLAGS)
> > --
> > 2.38.0.135.g90850a2211-goog
> >

Nick Desaulniers Oct. 24, 2022, 5:44 p.m. UTC | #3

On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > +config DEBUG_INFO_UNCOMPRESSED
> > > +     bool "Don't compress debug information"
> > > +     help
> > > +       Don't compress debug info sections.
> > > +
> > >  config DEBUG_INFO_COMPRESSED
>
> Also, I prefer...
>
> DEBUG_INFO_COMPRESSED_NONE
> DEBUG_INFO_COMPRESSED_ZLIB
> DEBUG_INFO_COMPRESSED_ZSTD
>
> Following...
>
> $ ld.lld-16 -v
> Debian LLD 16.0.0 (compatible with GNU linkers)
>
> $ ld.lld-16 --help | grep compress-debug-sections
>  --compress-debug-sections=[none,zlib,zstd]

It's a good suggestion, though I deliberately did not modify the
identifier used for the existing DEBUG_INFO_COMPRESSED.  No defconfigs
set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with
changing the identifier of existing kconfigs is that upon upgrading to
newer kernel sources, if you run `make oldmodconfig` if you previously
had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has
been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently
disabled, and image sizes would increase due to uncompressed debug
info. That seems developer hostile.

Masahiro, what are your thoughts on how to proceed? Should I change
the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

Sedat Dilek (DHL Supply Chain) Oct. 25, 2022, 5:08 p.m. UTC | #4

-----Ursprüngliche Nachricht-----
Von: Nick Desaulniers <ndesaulniers@google.com> 
Gesendet: Montag, 24. Oktober 2022 19:44
An: sedat.dilek@gmail.com; Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>; Fangrui Song <maskray@google.com>; Sedat Dilek (DHL Supply Chain) <sedat.dilek@dhl.com>; Michal Marek <michal.lkml@markovi.net>; Nick Terrell <terrelln@fb.com>; Tom Rix <trix@redhat.com>; Andrew Morton <akpm@linux-foundation.org>; Peter Zijlstra (Intel) <peterz@infradead.org>; David Gow <davidgow@google.com>; Kees Cook <keescook@chromium.org>; Josh Poimboeuf <jpoimboe@kernel.org>; Dan Williams <dan.j.williams@intel.com>; Miguel Ojeda <ojeda@kernel.org>; Isabella Basso <isabbasso@riseup.net>; Vlastimil Babka <vbabka@suse.cz>; Rasmus Villemoes <linux@rasmusvillemoes.dk>; linux-kernel@vger.kernel.org; linux-kbuild@vger.kernel.org; llvm@lists.linux.dev
Betreff: Re: [PATCH] Makefile.debug: support for -gz=zstd

On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > +config DEBUG_INFO_UNCOMPRESSED
> > > +     bool "Don't compress debug information"
> > > +     help
> > > +       Don't compress debug info sections.
> > > +
> > >  config DEBUG_INFO_COMPRESSED
>
> Also, I prefer...
>
> DEBUG_INFO_COMPRESSED_NONE
> DEBUG_INFO_COMPRESSED_ZLIB
> DEBUG_INFO_COMPRESSED_ZSTD
>
> Following...
>
> $ ld.lld-16 -v
> Debian LLD 16.0.0 (compatible with GNU linkers)
>
> $ ld.lld-16 --help | grep compress-debug-sections  
> --compress-debug-sections=[none,zlib,zstd]

It's a good suggestion, though I deliberately did not modify the identifier used for the existing DEBUG_INFO_COMPRESSED.  No defconfigs set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with changing the identifier of existing kconfigs is that upon upgrading to newer kernel sources, if you run `make oldmodconfig` if you previously had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently disabled, and image sizes would increase due to uncompressed debug info. That seems developer hostile.

Masahiro, what are your thoughts on how to proceed? Should I change the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

***

Cannot say what happens when CONFIG_XXX changes - I have not played with your patch.

Perhaps, we can use CONFIG_ DEBUG_INFO_COMPRESSED="none|zlib|zstd" with default: "none"?

-Sedat-

Nick Desaulniers Oct. 31, 2022, 6:49 p.m. UTC | #5

On Mon, Oct 24, 2022 at 10:44 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > > +config DEBUG_INFO_UNCOMPRESSED
> > > > +     bool "Don't compress debug information"
> > > > +     help
> > > > +       Don't compress debug info sections.
> > > > +
> > > >  config DEBUG_INFO_COMPRESSED
> >
> > Also, I prefer...
> >
> > DEBUG_INFO_COMPRESSED_NONE
> > DEBUG_INFO_COMPRESSED_ZLIB
> > DEBUG_INFO_COMPRESSED_ZSTD
> >
> > Following...
> >
> > $ ld.lld-16 -v
> > Debian LLD 16.0.0 (compatible with GNU linkers)
> >
> > $ ld.lld-16 --help | grep compress-debug-sections
> >  --compress-debug-sections=[none,zlib,zstd]
>
> It's a good suggestion, though I deliberately did not modify the
> identifier used for the existing DEBUG_INFO_COMPRESSED.  No defconfigs
> set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with
> changing the identifier of existing kconfigs is that upon upgrading to
> newer kernel sources, if you run `make oldmodconfig` if you previously
> had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has
> been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently
> disabled, and image sizes would increase due to uncompressed debug
> info. That seems developer hostile.
>
> Masahiro, what are your thoughts on how to proceed? Should I change
> the pre-existing Kconfig identifier, or "let sleeping dogs lie?"

Hi Masahiro,
What are your thoughts on the above?

> --
> Thanks,
> ~Nick Desaulniers

Masahiro Yamada Nov. 1, 2022, 5:57 a.m. UTC | #6

On Tue, Nov 1, 2022 at 3:49 AM Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> On Mon, Oct 24, 2022 at 10:44 AM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > On Sun, Oct 23, 2022 at 8:46 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Thu, Oct 20, 2022 at 8:43 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > >
> > > > On Thu, Oct 20, 2022 at 10:56:49AM -0700, Nick Desaulniers wrote:
> > > > > +config DEBUG_INFO_UNCOMPRESSED
> > > > > +     bool "Don't compress debug information"
> > > > > +     help
> > > > > +       Don't compress debug info sections.
> > > > > +
> > > > >  config DEBUG_INFO_COMPRESSED
> > >
> > > Also, I prefer...
> > >
> > > DEBUG_INFO_COMPRESSED_NONE
> > > DEBUG_INFO_COMPRESSED_ZLIB
> > > DEBUG_INFO_COMPRESSED_ZSTD
> > >
> > > Following...
> > >
> > > $ ld.lld-16 -v
> > > Debian LLD 16.0.0 (compatible with GNU linkers)
> > >
> > > $ ld.lld-16 --help | grep compress-debug-sections
> > >  --compress-debug-sections=[none,zlib,zstd]
> >
> > It's a good suggestion, though I deliberately did not modify the
> > identifier used for the existing DEBUG_INFO_COMPRESSED.  No defconfigs
> > set DEBUG_INFO_COMPRESSED explicitly, but I think one issue with
> > changing the identifier of existing kconfigs is that upon upgrading to
> > newer kernel sources, if you run `make oldmodconfig` if you previously
> > had DEBUG_INFO_COMPRESSED=y, kconfig doesn't know that that option has
> > been renamed to DEBUG_INFO_COMPRESSED_ZLIB so it would be silently
> > disabled, and image sizes would increase due to uncompressed debug
> > info. That seems developer hostile.
> >
> > Masahiro, what are your thoughts on how to proceed? Should I change
> > the pre-existing Kconfig identifier, or "let sleeping dogs lie?"
>
> Hi Masahiro,
> What are your thoughts on the above?


We renamed CONFIG options from time to time.
We should strive to update in-tree defconfigs,
but we did not care about external users very much.
(we rely on "make oldconfig" showing new options
and asking users for their choice.)

So, I like Sedat's DEBUG_INFO_COMPRESSED_{ZLIB,ZSTD,NONE}.
(and it is consist with CONFIG_INITRAMFS_COMPRESSION_*)

diff mbox series

Patch

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 3fc7abffc7aa..4085ac77dc12 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -312,8 +312,22 @@  config DEBUG_INFO_REDUCED
 	  DEBUG_INFO build and compile times are reduced too.
 	  Only works with newer gcc versions.
 
+choice
+	prompt "Compressed Debug information"
+	depends on DEBUG_KERNEL
+	help
+	  Compress the resulting debug info. Results in smaller debug info sections,
+	  but requires that consumers are able to decompress the results.
+
+	  If unsure, choose DEBUG_INFO_UNCOMPRESSED.
+
+config DEBUG_INFO_UNCOMPRESSED
+	bool "Don't compress debug information"
+	help
+	  Don't compress debug info sections.
+
 config DEBUG_INFO_COMPRESSED
-	bool "Compressed debugging information"
+	bool "Compress debugging information with zlib"
 	depends on $(cc-option,-gz=zlib)
 	depends on $(ld-option,--compress-debug-sections=zlib)
 	help
@@ -327,6 +341,16 @@  config DEBUG_INFO_COMPRESSED
 	  preferable to setting $KDEB_COMPRESS to "none" which would be even
 	  larger.
 
+config DEBUG_INFO_COMPRESSED_ZSTD
+	bool "Compress debugging information with zstd"
+	depends on $(cc-option,-gz=zstd)
+	depends on $(ld-option,--compress-debug-sections=zstd)
+	help
+	  Compress the debug information using zstd.  Requires GCC 13.0+ or Clang
+	  16.0+, binutils 2.40+, and zstd.
+
+endchoice # "Compressed Debug information"
+
 config DEBUG_INFO_SPLIT
 	bool "Produce split debuginfo in .dwo files"
 	depends on $(cc-option,-gsplit-dwarf)
diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
index 332c486f705f..8ac3379d2255 100644
--- a/scripts/Makefile.debug
+++ b/scripts/Makefile.debug
@@ -31,6 +31,10 @@  ifdef CONFIG_DEBUG_INFO_COMPRESSED
 DEBUG_CFLAGS	+= -gz=zlib
 KBUILD_AFLAGS	+= -gz=zlib
 KBUILD_LDFLAGS	+= --compress-debug-sections=zlib
+else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
+DEBUG_CFLAGS	+= -gz=zstd
+KBUILD_AFLAGS	+= -gz=zstd
+KBUILD_LDFLAGS	+= --compress-debug-sections=zstd
 endif
 
 KBUILD_CFLAGS	+= $(DEBUG_CFLAGS)