[v1,4/8] selftests/nolibc: add extra config file customize support

Message ID ecd5cd252bc728df1becfebff772713e8edce1ec.1689713175.git.falcon@tinylab.org
State New
Headers
Series tools/nolibc: add 32/64-bit powerpc support |

Commit Message

Zhangjin Wu July 18, 2023, 9:14 p.m. UTC
  The default DEFCONFIG_<ARCH> may not always work for all architectures,
some architectures require to add extra kernel config options, this adds
a new 'extconfig' target for this requirement.

It allows to customize extra kernel config options via the common
common.config and the architecture specific <ARCH>.config, at last
trigger 'allnoconfig' to let them take effect with missing config
options as disabled.

The scripts/kconfig/merge_config.sh tool is used to merge the extra
config files.

Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
Link: https://lore.kernel.org/lkml/67eb70d4-c9ff-4afc-bac7-7f36cc2c81bc@t-8ch.de/
Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/Makefile | 7 +++++++
 1 file changed, 7 insertions(+)
  

Comments

Willy Tarreau July 22, 2023, noon UTC | #1
On Wed, Jul 19, 2023 at 05:14:07AM +0800, Zhangjin Wu wrote:
> The default DEFCONFIG_<ARCH> may not always work for all architectures,
> some architectures require to add extra kernel config options, this adds
> a new 'extconfig' target for this requirement.
> 
> It allows to customize extra kernel config options via the common
> common.config and the architecture specific <ARCH>.config, at last
> trigger 'allnoconfig' to let them take effect with missing config
> options as disabled.
> 
> The scripts/kconfig/merge_config.sh tool is used to merge the extra
> config files.
> 
> Suggested-by: Thomas Weißschuh <linux@weissschuh.net>
> Link: https://lore.kernel.org/lkml/67eb70d4-c9ff-4afc-bac7-7f36cc2c81bc@t-8ch.de/
> Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> ---
>  tools/testing/selftests/nolibc/Makefile | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> index f42adef87e12..08a5ca5f418b 100644
> --- a/tools/testing/selftests/nolibc/Makefile
> +++ b/tools/testing/selftests/nolibc/Makefile
> @@ -39,6 +39,9 @@ DEFCONFIG_s390       = defconfig
>  DEFCONFIG_loongarch  = defconfig
>  DEFCONFIG            = $(DEFCONFIG_$(ARCH))
>  
> +# extra kernel config files under configs/, include common + architecture specific
> +EXTCONFIG            = common.config $(ARCH).config
> +
>  # optional tests to run (default = all)
>  TEST =
>  
> @@ -162,6 +165,10 @@ initramfs: nolibc-test
>  defconfig:
>  	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
>  
> +extconfig:
> +	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> +	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
> +

Please also mention this entry in the "help" message.
Other than that, OK.

Willy
  
Willy Tarreau July 29, 2023, 7:45 a.m. UTC | #2
Hi Zhangjin,

On Tue, Jul 25, 2023 at 10:30:06PM +0800, Zhangjin Wu wrote:
> Hi, Willy
> 
> > On Wed, Jul 19, 2023 at 05:14:07AM +0800, Zhangjin Wu wrote:
> > > The default DEFCONFIG_<ARCH> may not always work for all architectures,
> > > some architectures require to add extra kernel config options, this adds
> > > a new 'extconfig' target for this requirement.
> > > 
> > > It allows to customize extra kernel config options via the common
> > > common.config and the architecture specific <ARCH>.config, at last
> > > trigger 'allnoconfig' to let them take effect with missing config
> > > options as disabled.
> > > 
> > > The scripts/kconfig/merge_config.sh tool is used to merge the extra
> > > config files.
> > > 
> > > Suggested-by: Thomas Wei?schuh <linux@weissschuh.net>
> > > Link: https://lore.kernel.org/lkml/67eb70d4-c9ff-4afc-bac7-7f36cc2c81bc@t-8ch.de/
> > > Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> > > ---
> > >  tools/testing/selftests/nolibc/Makefile | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> > > index f42adef87e12..08a5ca5f418b 100644
> > > --- a/tools/testing/selftests/nolibc/Makefile
> > > +++ b/tools/testing/selftests/nolibc/Makefile
> > > @@ -39,6 +39,9 @@ DEFCONFIG_s390       = defconfig
> > >  DEFCONFIG_loongarch  = defconfig
> > >  DEFCONFIG            = $(DEFCONFIG_$(ARCH))
> > >  
> > > +# extra kernel config files under configs/, include common + architecture specific
> > > +EXTCONFIG            = common.config $(ARCH).config
> > > +
> > >  # optional tests to run (default = all)
> > >  TEST =
> > >  
> > > @@ -162,6 +165,10 @@ initramfs: nolibc-test
> > >  defconfig:
> > >  	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
> > >  
> > > +extconfig:
> > > +	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> > > +	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
> > > +
> > 
> > Please also mention this entry in the "help" message.
> > Other than that, OK.
> >
> 
> Willy, as we discussed in another tinyconfig series, in order to avoid
> add more complexity to users, I plan to drop this standalone 'extconfig'
> target and move the extra config operations to defconfig target like
> this:
> 
>     defconfig:
>      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
>      	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
>      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
> 
> This extra config options are really critical to default boot and test, so, it
> should be a 'default' config action as the 'defconfig' target originally mean.
> 
> Will test carefully about this change, what's your idea?

Well, *iff* we need to have per-arch config settings, probably to better
support Qemu, the console output, etc, then indeed we'd rather add them
to all configs indeed. However the example above is bogus. First you
create a default config, then prepare the rest of the kernel, then
merge that config with the extensions (based on the arch and not the
sub-arch BTW) then erase everything to restart from an allnoconfig.

Also if you're using merge_config you'll need -Q to disable warning
about overridden options since you're starting from a defconfig which
contains plenty of them. Usually I just do defconfig, append the few
changes, then oldconfig and that's done. But merge_config can probably
be fine as well. Also make prepare should be done last, to make sure
that if it depends on anything in the config (e.g. 32 vs 64 bit etc)
it's up to date.

Willy
  
Zhangjin Wu July 29, 2023, 9:43 a.m. UTC | #3
> Hi Zhangjin,
> 
> On Tue, Jul 25, 2023 at 10:30:06PM +0800, Zhangjin Wu wrote:
> > Hi, Willy
> > 
> > > On Wed, Jul 19, 2023 at 05:14:07AM +0800, Zhangjin Wu wrote:
> > > > The default DEFCONFIG_<ARCH> may not always work for all architectures,
> > > > some architectures require to add extra kernel config options, this adds
> > > > a new 'extconfig' target for this requirement.
> > > > 
> > > > It allows to customize extra kernel config options via the common
> > > > common.config and the architecture specific <ARCH>.config, at last
> > > > trigger 'allnoconfig' to let them take effect with missing config
> > > > options as disabled.
> > > > 
> > > > The scripts/kconfig/merge_config.sh tool is used to merge the extra
> > > > config files.
> > > > 
> > > > Suggested-by: Thomas Wei?schuh <linux@weissschuh.net>
> > > > Link: https://lore.kernel.org/lkml/67eb70d4-c9ff-4afc-bac7-7f36cc2c81bc@t-8ch.de/
> > > > Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> > > > ---
> > > >  tools/testing/selftests/nolibc/Makefile | 7 +++++++
> > > >  1 file changed, 7 insertions(+)
> > > > 
> > > > diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> > > > index f42adef87e12..08a5ca5f418b 100644
> > > > --- a/tools/testing/selftests/nolibc/Makefile
> > > > +++ b/tools/testing/selftests/nolibc/Makefile
> > > > @@ -39,6 +39,9 @@ DEFCONFIG_s390       = defconfig
> > > >  DEFCONFIG_loongarch  = defconfig
> > > >  DEFCONFIG            = $(DEFCONFIG_$(ARCH))
> > > >  
> > > > +# extra kernel config files under configs/, include common + architecture specific
> > > > +EXTCONFIG            = common.config $(ARCH).config
> > > > +
> > > >  # optional tests to run (default = all)
> > > >  TEST =
> > > >  
> > > > @@ -162,6 +165,10 @@ initramfs: nolibc-test
> > > >  defconfig:
> > > >  	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
> > > >  
> > > > +extconfig:
> > > > +	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> > > > +	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
> > > > +
> > > 
> > > Please also mention this entry in the "help" message.
> > > Other than that, OK.
> > >
> > 
> > Willy, as we discussed in another tinyconfig series, in order to avoid
> > add more complexity to users, I plan to drop this standalone 'extconfig'
> > target and move the extra config operations to defconfig target like
> > this:
> > 
> >     defconfig:
> >      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
> >      	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
> >      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
> > 
> > This extra config options are really critical to default boot and test, so, it
> > should be a 'default' config action as the 'defconfig' target originally mean.
> > 
> > Will test carefully about this change, what's your idea?
> 
> Well, *iff* we need to have per-arch config settings, probably to better
> support Qemu, the console output, etc, then indeed we'd rather add them
> to all configs indeed. However the example above is bogus. First you
> create a default config, then prepare the rest of the kernel , then
> merge that config with the extensions (based on the arch and not the
> sub-arch BTW) then erase everything to restart from an allnoconfig.
>

allnoconfig is based on the "$(srctree)/.config" generated by defconfig
plus the extra ARCH specific config options, so, it should be additional?

Btw, what do you mean? "based on the arch and not the sub-arch BTW", in
reality, the ARCH will be changed to XARCH, so, this is sub-arch/variant
specific.

> Also if you're using merge_config you'll need -Q to disable warning
> about overridden options since you're starting from a defconfig which
> contains plenty of them.

Yeah, we do need this to silence some warnings.

> Usually I just do defconfig, append the few
> changes, then oldconfig and that's done.

I have used olddefconfig (similar to oldconfig but without prompts)
before, but allnoconfig is used to mainly to only compile the ones we
explicitly enable, with the new additional options as No, which may be
more deterministic, it should not touch the old configured ones in
.config? I have learned its usage from the merge_config.sh

> But merge_config can probably
> be fine as well. Also make prepare should be done last, to make sure
> that if it depends on anything in the config (e.g. 32 vs 64 bit etc)
> it's up to date.
>

Ok, let's move 'prepare' at the end of all above.

And also add the -Q option to silence the potential warnings:

    defconfig:
      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG)
      	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -Q -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
      	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) prepare

We may still need to clear the usage of allnoconfig, In my test, it does
work as I explained above. If I really misused it, let's change it back
to olddefconfig.

Thanks,
Zhangjin

> Willy
  

Patch

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index f42adef87e12..08a5ca5f418b 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -39,6 +39,9 @@  DEFCONFIG_s390       = defconfig
 DEFCONFIG_loongarch  = defconfig
 DEFCONFIG            = $(DEFCONFIG_$(ARCH))
 
+# extra kernel config files under configs/, include common + architecture specific
+EXTCONFIG            = common.config $(ARCH).config
+
 # optional tests to run (default = all)
 TEST =
 
@@ -162,6 +165,10 @@  initramfs: nolibc-test
 defconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
 
+extconfig:
+	$(Q)$(srctree)/scripts/kconfig/merge_config.sh -O "$(srctree)" -m "$(srctree)/.config" $(foreach c,$(EXTCONFIG),$(wildcard $(CURDIR)/configs/$c))
+	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG="$(srctree)/.config" allnoconfig
+
 kernel: initramfs
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs