[RFC,0/4] Introduce uts_release

  When hacking it is a waste of time and compute energy that we need to
rebuild much kernel code just for changing the head git commit, like this:

> touch include/generated/utsrelease.h 
> time make  -j3
mkdir -p /home/john/mnt_sda4/john/kernel-dev2/tools/objtool && make O=/home/john/mnt_sda4/john/kernel-dev2 subdir=tools/objtool --no-print-directory -C objtool 
  INSTALL libsubcmd_headers
  CALL    scripts/checksyscalls.sh
  CC      init/version.o
  AR      init/built-in.a
  CC      kernel/sys.o
  CC      kernel/module/main.o
  AR      kernel/module/built-in.a
  CC      drivers/base/firmware_loader/main.o
  CC      kernel/trace/trace.o
  AR      drivers/base/firmware_loader/built-in.a
  AR      drivers/base/built-in.a
  CC      net/ethtool/ioctl.o
  AR      kernel/trace/built-in.a
  AR      kernel/built-in.a
  AR      net/ethtool/built-in.a
  AR      net/built-in.a
  AR      drivers/built-in.a
  AR      built-in.a
  ...

Files like drivers/base/firmware_loader/main.c needs to be recompiled as
it includes generated/utsrelease.h for UTS_RELEASE macro, and utsrelease.h
is regenerated when the head commit changes.

Introduce global char uts_release[] in init/version.c, which this
mentioned code can use instead of UTS_RELEASE, meaning that we don't need
to rebuild for changing the head commit - only init/version.c needs to be
rebuilt. Whether all the references to UTS_RELEASE in the codebase are
proper is a different matter.

For an x86_64 defconfig build for this series on my old laptop, here is
before and after rebuild time:

before:
real    0m53.591s
user    1m1.842s
sys     0m9.161s

after:
real    0m37.481s
user    0m46.461s
sys     0m7.199s

Sending as an RFC as I need to test more of the conversions and I would
like to also convert more UTS_RELEASE users to prove this is proper
approach.

John Garry (4):
  init: Add uts_release
  tracing: Use uts_release
  net: ethtool: Use uts_release
  firmware_loader: Use uts_release

 drivers/base/firmware_loader/main.c | 39 +++++++++++++++++++++++------
 include/linux/utsname.h             |  1 +
 init/version.c                      |  3 +++
 kernel/trace/trace.c                |  4 +--
 net/ethtool/ioctl.c                 |  4 +--
 5 files changed, 39 insertions(+), 12 deletions(-)
  

On Wed, Jan 31, 2024 at 10:48:47AM +0000, John Garry wrote:
> When hacking it is a waste of time and compute energy that we need to
> rebuild much kernel code just for changing the head git commit, like this:
> 
> > touch include/generated/utsrelease.h 
> > time make  -j3
> mkdir -p /home/john/mnt_sda4/john/kernel-dev2/tools/objtool && make O=/home/john/mnt_sda4/john/kernel-dev2 subdir=tools/objtool --no-print-directory -C objtool 
>   INSTALL libsubcmd_headers
>   CALL    scripts/checksyscalls.sh
>   CC      init/version.o
>   AR      init/built-in.a
>   CC      kernel/sys.o
>   CC      kernel/module/main.o
>   AR      kernel/module/built-in.a
>   CC      drivers/base/firmware_loader/main.o
>   CC      kernel/trace/trace.o
>   AR      drivers/base/firmware_loader/built-in.a
>   AR      drivers/base/built-in.a
>   CC      net/ethtool/ioctl.o
>   AR      kernel/trace/built-in.a
>   AR      kernel/built-in.a
>   AR      net/ethtool/built-in.a
>   AR      net/built-in.a
>   AR      drivers/built-in.a
>   AR      built-in.a
>   ...
> 
> Files like drivers/base/firmware_loader/main.c needs to be recompiled as
> it includes generated/utsrelease.h for UTS_RELEASE macro, and utsrelease.h
> is regenerated when the head commit changes.
> 
> Introduce global char uts_release[] in init/version.c, which this
> mentioned code can use instead of UTS_RELEASE, meaning that we don't need
> to rebuild for changing the head commit - only init/version.c needs to be
> rebuilt. Whether all the references to UTS_RELEASE in the codebase are
> proper is a different matter.
> 
> For an x86_64 defconfig build for this series on my old laptop, here is
> before and after rebuild time:
> 
> before:
> real    0m53.591s
> user    1m1.842s
> sys     0m9.161s
> 
> after:
> real    0m37.481s
> user    0m46.461s
> sys     0m7.199s
> 
> Sending as an RFC as I need to test more of the conversions and I would
> like to also convert more UTS_RELEASE users to prove this is proper
> approach.

I like it, I also think that v4l2 includes this as well as all of those
drivers seem to rebuild when this changes, does that not happen for you
too?

Anyway, if the firmware changes work, I'm all for this, thanks for
taking it on!

thanks,

greg k-h
  

On 31/01/2024 16:22, Greg KH wrote:
>> before:
>> real    0m53.591s
>> user    1m1.842s
>> sys     0m9.161s
>>
>> after:
>> real    0m37.481s
>> user    0m46.461s
>> sys     0m7.199s
>>
>> Sending as an RFC as I need to test more of the conversions and I would
>> like to also convert more UTS_RELEASE users to prove this is proper
>> approach.
> I like it, I also think that v4l2 includes this as well as all of those
> drivers seem to rebuild when this changes, does that not happen for you
> too?

I didn't see that. Were you were building for arm64? I can see some v4l2 
configs enabled there for the vanilla defconfig (but none for x86-64).

> 
> Anyway, if the firmware changes work, I'm all for this, thanks for
> taking it on!

cheers. BTW, I just noticed that utsrelase.h might not be updated for my 
allmodconfig build when I change the head commit - I'll investigate why ...

John
  

On Wed, Jan 31, 2024 at 05:16:09PM +0000, John Garry wrote:
> On 31/01/2024 16:22, Greg KH wrote:
> > > before:
> > > real    0m53.591s
> > > user    1m1.842s
> > > sys     0m9.161s
> > > 
> > > after:
> > > real    0m37.481s
> > > user    0m46.461s
> > > sys     0m7.199s
> > > 
> > > Sending as an RFC as I need to test more of the conversions and I would
> > > like to also convert more UTS_RELEASE users to prove this is proper
> > > approach.
> > I like it, I also think that v4l2 includes this as well as all of those
> > drivers seem to rebuild when this changes, does that not happen for you
> > too?
> 
> I didn't see that. Were you were building for arm64? I can see some v4l2
> configs enabled there for the vanilla defconfig (but none for x86-64).

Building for x86, maybe it's one of the other LINUX_VERSION type defines
we have, sorry, can't remember, it's been a long time since I looked
into it.

thanks,

greg k-h
  

On Wed, Jan 31, 2024 at 7:49 PM John Garry <john.g.garry@oracle.com> wrote:
>
> When hacking it is a waste of time and compute energy that we need to
> rebuild much kernel code just for changing the head git commit, like this:
>
> > touch include/generated/utsrelease.h
> > time make  -j3
> mkdir -p /home/john/mnt_sda4/john/kernel-dev2/tools/objtool && make O=/home/john/mnt_sda4/john/kernel-dev2 subdir=tools/objtool --no-print-directory -C objtool
>   INSTALL libsubcmd_headers
>   CALL    scripts/checksyscalls.sh
>   CC      init/version.o
>   AR      init/built-in.a
>   CC      kernel/sys.o
>   CC      kernel/module/main.o
>   AR      kernel/module/built-in.a
>   CC      drivers/base/firmware_loader/main.o
>   CC      kernel/trace/trace.o
>   AR      drivers/base/firmware_loader/built-in.a
>   AR      drivers/base/built-in.a
>   CC      net/ethtool/ioctl.o
>   AR      kernel/trace/built-in.a
>   AR      kernel/built-in.a
>   AR      net/ethtool/built-in.a
>   AR      net/built-in.a
>   AR      drivers/built-in.a
>   AR      built-in.a
>   ...
>
> Files like drivers/base/firmware_loader/main.c needs to be recompiled as
> it includes generated/utsrelease.h for UTS_RELEASE macro, and utsrelease.h
> is regenerated when the head commit changes.
>
> Introduce global char uts_release[] in init/version.c, which this
> mentioned code can use instead of UTS_RELEASE, meaning that we don't need
> to rebuild for changing the head commit - only init/version.c needs to be
> rebuilt. Whether all the references to UTS_RELEASE in the codebase are
> proper is a different matter.
>
> For an x86_64 defconfig build for this series on my old laptop, here is
> before and after rebuild time:
>
> before:
> real    0m53.591s
> user    1m1.842s
> sys     0m9.161s
>
> after:
> real    0m37.481s
> user    0m46.461s
> sys     0m7.199s
>
> Sending as an RFC as I need to test more of the conversions and I would
> like to also convert more UTS_RELEASE users to prove this is proper
> approach.
>
> John Garry (4):
>   init: Add uts_release
>   tracing: Use uts_release
>   net: ethtool: Use uts_release
>   firmware_loader: Use uts_release
>
>  drivers/base/firmware_loader/main.c | 39 +++++++++++++++++++++++------
>  include/linux/utsname.h             |  1 +
>  init/version.c                      |  3 +++
>  kernel/trace/trace.c                |  4 +--
>  net/ethtool/ioctl.c                 |  4 +--
>  5 files changed, 39 insertions(+), 12 deletions(-)
>
> --
> 2.35.3
>





As you see, several drivers store UTS_RELEASE in their driver data,
and even print it in debug print.


I do not see why it is useful.
As you discussed in 3/4, if UTS_RELEASE is unneeded,
it is better to get rid of it.


If such version information is useful for drivers, the intention is
whether the version of the module, or the version of vmlinux.
That is a question.
They differ when CONFIG_MODVERSION.


When module developers intend to printk the git version
from which the module was compiled from,
presumably they want to use UTS_RELEASE, which
was expanded at the compile time of the module.

If you replace it with uts_release, it is the git version
of vmlinux.


Of course, the replacement is safe for always-builtin code.



Lastly, we can avoid using UTS_RELEASE without relying
on your patch.



For example, commit 3a3a11e6e5a2bc0595c7e36ae33c861c9e8c75b1
replaced  UTS_RELEASE with init_uts_ns.name.release


So, is your uts_release a shorthand of init_uts_ns.name.release?



I think what you can contribute are:

 - Explore the UTS_RELEASE users, and check if you can get rid of it.

 - Where UTS_RELEASE is useful, consider if it is possible
   to replace it with init_uts_ns.name.release
  

On Sat, 3 Feb 2024 00:01:26 +0900 Masahiro Yamada wrote:
> I do not see why it is useful.
> As you discussed in 3/4, if UTS_RELEASE is unneeded,
> it is better to get rid of it.

To be clear - the discussion on 3/4 was about the fact that netdev
already prints UTS_RELEASE into the version member of driver info
struct, as a default. So the drivers no longer have to. But there's 
no user-observable change there.
  

On 02/02/2024 15:01, Masahiro Yamada wrote:
>> --
>> 2.35.3
> 
> As you see, several drivers store UTS_RELEASE in their driver data,
> and even print it in debug print.
> 
> 
> I do not see why it is useful.

I would tend to agree, and mentioned that earlier.

> As you discussed in 3/4, if UTS_RELEASE is unneeded,
> it is better to get rid of it.

Jakub replied about this.

> 
> 
> If such version information is useful for drivers, the intention is
> whether the version of the module, or the version of vmlinux.
> That is a question.
> They differ when CONFIG_MODVERSION.
> 

I think often this information in UTS_RELEASE is shared as informative 
only, so the user can conveniently know the specific kernel git version.

> 
> When module developers intend to printk the git version
> from which the module was compiled from,
> presumably they want to use UTS_RELEASE, which
> was expanded at the compile time of the module.
> 
> If you replace it with uts_release, it is the git version
> of vmlinux.
> 
> 
> Of course, the replacement is safe for always-builtin code.
> 
> 
> 
> Lastly, we can avoid using UTS_RELEASE without relying
> on your patch.
> 
> 
> 
> For example, commit 3a3a11e6e5a2bc0595c7e36ae33c861c9e8c75b1
> replaced  UTS_RELEASE with init_uts_ns.name.release
> 
> 
> So, is your uts_release a shorthand of init_uts_ns.name.release?

Yes - well that both are strings containing UTS_RELEASE. Using a struct 
sub-member is bit ungainly, but I suppose that we should not be making 
life easy for people using this.

However we already have init_utsname in:

static inline struct new_utsname *init_utsname(void)
{
	return &init_uts_ns.name;
}

So could use init_utsname()->release, which is a bit nicer.

> 
> 
> 
> I think what you can contribute are:
> 
>   - Explore the UTS_RELEASE users, and check if you can get rid of it.

Unfortunately I expect resistance for this. I also expect places like FW 
loader it is necessary. And when this is used in sysfs, people will say 
that it is part of the ABI now.

How about I send the patch to update to use init_uts_ns and mention also 
that it would be better to not use at all, if possible? I can cc you.

> 
>   - Where UTS_RELEASE is useful, consider if it is possible
>     to replace it with init_uts_ns.name.release

ok, but, as above, could use init_utsname()->release also

Thanks,
John
  

On Mon, Feb 5, 2024 at 5:25 PM John Garry <john.g.garry@oracle.com> wrote:
>
> On 02/02/2024 15:01, Masahiro Yamada wrote:
> >> --
> >> 2.35.3
> >
> > As you see, several drivers store UTS_RELEASE in their driver data,
> > and even print it in debug print.
> >
> >
> > I do not see why it is useful.
>
> I would tend to agree, and mentioned that earlier.
>
> > As you discussed in 3/4, if UTS_RELEASE is unneeded,
> > it is better to get rid of it.
>
> Jakub replied about this.
>
> >
> >
> > If such version information is useful for drivers, the intention is
> > whether the version of the module, or the version of vmlinux.
> > That is a question.
> > They differ when CONFIG_MODVERSION.
> >
>
> I think often this information in UTS_RELEASE is shared as informative
> only, so the user can conveniently know the specific kernel git version.
>
> >
> > When module developers intend to printk the git version
> > from which the module was compiled from,
> > presumably they want to use UTS_RELEASE, which
> > was expanded at the compile time of the module.
> >
> > If you replace it with uts_release, it is the git version
> > of vmlinux.
> >
> >
> > Of course, the replacement is safe for always-builtin code.
> >
> >
> >
> > Lastly, we can avoid using UTS_RELEASE without relying
> > on your patch.
> >
> >
> >
> > For example, commit 3a3a11e6e5a2bc0595c7e36ae33c861c9e8c75b1
> > replaced  UTS_RELEASE with init_uts_ns.name.release
> >
> >
> > So, is your uts_release a shorthand of init_uts_ns.name.release?
>
> Yes - well that both are strings containing UTS_RELEASE. Using a struct
> sub-member is bit ungainly, but I suppose that we should not be making
> life easy for people using this.
>
> However we already have init_utsname in:
>
> static inline struct new_utsname *init_utsname(void)
> {
>         return &init_uts_ns.name;
> }
>
> So could use init_utsname()->release, which is a bit nicer.
>
> >
> >
> >
> > I think what you can contribute are:
> >
> >   - Explore the UTS_RELEASE users, and check if you can get rid of it.
>
> Unfortunately I expect resistance for this. I also expect places like FW
> loader it is necessary. And when this is used in sysfs, people will say
> that it is part of the ABI now.
>
> How about I send the patch to update to use init_uts_ns and mention also
> that it would be better to not use at all, if possible? I can cc you.


OK.


As I mentioned in the previous reply, the replacement is safe
for builtin code.

When you touch modular code, please pay a little more care,
because UTS_RELEASE and init_utsname()->release
may differ when CONFIG_MODVERSIONS=y.







>
> >
> >   - Where UTS_RELEASE is useful, consider if it is possible
> >     to replace it with init_uts_ns.name.release
>
> ok, but, as above, could use init_utsname()->release also


I am fine with it.


init_utsname()->release is more commonly used
(but less common than UTS_RELEASE)



$ git grep   'init_utsname()->release' | wc
     28      92    2065
$ git grep   'init_uts_ns.name.release' | wc
      7      34     587
$ git grep   'UTS_RELEASE' | wc
     57     304    4741
  

On 05/02/2024 23:10, Masahiro Yamada wrote:
>>> I think what you can contribute are:
>>>
>>>    - Explore the UTS_RELEASE users, and check if you can get rid of it.
>> Unfortunately I expect resistance for this. I also expect places like FW
>> loader it is necessary. And when this is used in sysfs, people will say
>> that it is part of the ABI now.
>>
>> How about I send the patch to update to use init_uts_ns and mention also
>> that it would be better to not use at all, if possible? I can cc you.
> 
> OK.
> 
> 
> As I mentioned in the previous reply, the replacement is safe
> for builtin code.
> 
> When you touch modular code, please pay a little more care,
> because UTS_RELEASE and init_utsname()->release
> may differ when CONFIG_MODVERSIONS=y.
> 

Are you saying that we may have a different release version kernel and 
module built with CONFIG_MODVERSIONS=y, and the module was using 
UTS_RELEASE for something? That something may be like setting some info 
in a sysfs file, like in this example:

static ssize_t target_core_item_version_show(struct config_item *item,
		char *page)
{
	return sprintf(page, "Target Engine Core ConfigFS Infrastructure %s"
		" on %s/%s on "UTS_RELEASE"\n", TARGET_CORE_VERSION,
		utsname()->sysname, utsname()->machine);
}

And the intention is to use the module codebase release version and not 
the kernel codebase release version. Hence utsname() is used for 
sysname and .machine, but not .release .

Thanks,
John
  

On 08/02/2024 10:08, John Garry wrote:
> On 05/02/2024 23:10, Masahiro Yamada wrote:
>>>> I think what you can contribute are:
>>>>
>>>>    - Explore the UTS_RELEASE users, and check if you can get rid of it.
>>> Unfortunately I expect resistance for this. I also expect places like FW
>>> loader it is necessary. And when this is used in sysfs, people will say
>>> that it is part of the ABI now.
>>>
>>> How about I send the patch to update to use init_uts_ns and mention also
>>> that it would be better to not use at all, if possible? I can cc you.
>>
>> OK.
>>
>>
>> As I mentioned in the previous reply, the replacement is safe
>> for builtin code.
>>
>> When you touch modular code, please pay a little more care,
>> because UTS_RELEASE and init_utsname()->release
>> may differ when CONFIG_MODVERSIONS=y.
>>
> 
> Are you saying that we may have a different release version kernel and 
> module built with CONFIG_MODVERSIONS=y, and the module was using 
> UTS_RELEASE for something? That something may be like setting some info 
> in a sysfs file, like in this example:
> 
> static ssize_t target_core_item_version_show(struct config_item *item,
>          char *page)
> {
>      return sprintf(page, "Target Engine Core ConfigFS Infrastructure %s"
>          " on %s/%s on "UTS_RELEASE"\n", TARGET_CORE_VERSION,
>          utsname()->sysname, utsname()->machine);
> }
> 
> And the intention is to use the module codebase release version and not 
> the kernel codebase release version. Hence utsname() is used for 
> .sysname and .machine, but not .release .

Hi Masahiro,

Can you comment on whether I am right about CONFIG_MODVERSIONS, above?

Thanks,
John
  

On Wed, Feb 21, 2024 at 6:01 PM John Garry <john.g.garry@oracle.com> wrote:
>
> On 08/02/2024 10:08, John Garry wrote:
> > On 05/02/2024 23:10, Masahiro Yamada wrote:
> >>>> I think what you can contribute are:
> >>>>
> >>>>    - Explore the UTS_RELEASE users, and check if you can get rid of it.
> >>> Unfortunately I expect resistance for this. I also expect places like FW
> >>> loader it is necessary. And when this is used in sysfs, people will say
> >>> that it is part of the ABI now.
> >>>
> >>> How about I send the patch to update to use init_uts_ns and mention also
> >>> that it would be better to not use at all, if possible? I can cc you.
> >>
> >> OK.
> >>
> >>
> >> As I mentioned in the previous reply, the replacement is safe
> >> for builtin code.
> >>
> >> When you touch modular code, please pay a little more care,
> >> because UTS_RELEASE and init_utsname()->release
> >> may differ when CONFIG_MODVERSIONS=y.
> >>
> >
> > Are you saying that we may have a different release version kernel and
> > module built with CONFIG_MODVERSIONS=y, and the module was using
> > UTS_RELEASE for something? That something may be like setting some info
> > in a sysfs file, like in this example:
> >
> > static ssize_t target_core_item_version_show(struct config_item *item,
> >          char *page)
> > {
> >      return sprintf(page, "Target Engine Core ConfigFS Infrastructure %s"
> >          " on %s/%s on "UTS_RELEASE"\n", TARGET_CORE_VERSION,
> >          utsname()->sysname, utsname()->machine);
> > }
> >
> > And the intention is to use the module codebase release version and not
> > the kernel codebase release version. Hence utsname() is used for
> > .sysname and .machine, but not .release .
>
> Hi Masahiro,
>
> Can you comment on whether I am right about CONFIG_MODVERSIONS, above?
>
> Thanks,
> John



Your understanding about CONFIG_MODVERSIONS is correct.