[00/12] Introduce CONFIG_SLUB_TINY and deprecate SLOB

  Hi,

this continues the discussion from [1]. Reasons to remove SLOB are
outlined there and no-one has objected so far. The last patch of this
series therefore deprecates CONFIG_SLOB and updates all the defconfigs
using CONFIG_SLOB=y in the tree.

There is a k210 board with 8MB RAM where switching to SLUB caused issues
[2] and the lkp bot wasn't also happy about code bloat [3]. To address
both, this series introduces CONFIG_SLUB_TINY to perform some rather
low-hanging fruit modifications to SLUB to reduce its memory overhead.
This seems to have been successful at least in the k210 case [4]. I
consider this as an acceptable tradeoff for getting rid of SLOB.

The series is also available in git:
https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slub-tiny-v1r2

[1] https://lore.kernel.org/all/b35c3f82-f67b-2103-7d82-7a7ba7521439@suse.cz/
[2] https://lore.kernel.org/all/a5bba3ca-da19-293c-c01b-a28291533466@opensource.wdc.com/
[3] https://lore.kernel.org/all/Y25E9cJbhDAKi1vd@99bb1221be19/
[4] https://lore.kernel.org/all/6a1883c4-4c3f-545a-90e8-2cd805bcf4ae@opensource.wdc.com/

Vlastimil Babka (12):
  mm, slab: ignore hardened usercopy parameters when disabled
  mm, slub: add CONFIG_SLUB_TINY
  mm, slub: disable SYSFS support with CONFIG_SLUB_TINY
  mm, slub: retain no free slabs on partial list with CONFIG_SLUB_TINY
  mm, slub: lower the default slub_max_order with CONFIG_SLUB_TINY
  mm, slub: don't create kmalloc-rcl caches with CONFIG_SLUB_TINY
  mm, slab: ignore SLAB_RECLAIM_ACCOUNT with CONFIG_SLUB_TINY
  mm, slub: refactor free debug processing
  mm, slub: split out allocations from pre/post hooks
  mm, slub: remove percpu slabs with CONFIG_SLUB_TINY
  mm, slub: don't aggressively inline with CONFIG_SLUB_TINY
  mm, slob: rename CONFIG_SLOB to CONFIG_SLOB_DEPRECATED

 arch/arm/configs/clps711x_defconfig           |   3 +-
 arch/arm/configs/collie_defconfig             |   3 +-
 arch/arm/configs/multi_v4t_defconfig          |   3 +-
 arch/arm/configs/omap1_defconfig              |   3 +-
 arch/arm/configs/pxa_defconfig                |   3 +-
 arch/arm/configs/tct_hammer_defconfig         |   3 +-
 arch/arm/configs/xcep_defconfig               |   3 +-
 arch/openrisc/configs/or1ksim_defconfig       |   3 +-
 arch/openrisc/configs/simple_smp_defconfig    |   3 +-
 arch/riscv/configs/nommu_k210_defconfig       |   3 +-
 .../riscv/configs/nommu_k210_sdcard_defconfig |   3 +-
 arch/riscv/configs/nommu_virt_defconfig       |   3 +-
 arch/sh/configs/rsk7201_defconfig             |   3 +-
 arch/sh/configs/rsk7203_defconfig             |   3 +-
 arch/sh/configs/se7206_defconfig              |   3 +-
 arch/sh/configs/shmin_defconfig               |   3 +-
 arch/sh/configs/shx3_defconfig                |   3 +-
 include/linux/slab.h                          |   8 +
 include/linux/slub_def.h                      |   6 +-
 kernel/configs/tiny.config                    |   5 +-
 mm/Kconfig                                    |  38 +-
 mm/Kconfig.debug                              |   2 +-
 mm/slab_common.c                              |  16 +-
 mm/slub.c                                     | 415 ++++++++++++------
 24 files changed, 377 insertions(+), 164 deletions(-)
  

On Mon, Nov 21, 2022, at 18:11, Vlastimil Babka wrote:
>
> this continues the discussion from [1]. Reasons to remove SLOB are
> outlined there and no-one has objected so far. The last patch of this
> series therefore deprecates CONFIG_SLOB and updates all the defconfigs
> using CONFIG_SLOB=y in the tree.
>
> There is a k210 board with 8MB RAM where switching to SLUB caused issues
> [2] and the lkp bot wasn't also happy about code bloat [3]. To address
> both, this series introduces CONFIG_SLUB_TINY to perform some rather
> low-hanging fruit modifications to SLUB to reduce its memory overhead.
> This seems to have been successful at least in the k210 case [4]. I
> consider this as an acceptable tradeoff for getting rid of SLOB.

I agree that this is a great success for replacing SLOB on the
smallest machines that have 32MB or less and have to run a
a highly customized kernel, and this is probably enough to
have a drop-in replacement without making any currently working
system worse.

On the other hand, I have the feeling that we may want something
a bit less aggressive than this for machines that are slightly
less constrained, in particular when a single kernel needs to
scale from 64MB to 512MB, which can happen e.g. on OpenWRT.
I have seen a number of reports over the years that suggest
that new kernels handle fragmentation and low memory worse than
old ones, and it would be great to improve that again.

I can imagine those machines wanting to use sysfs in general
but not for the slab caches, so having a separate knob to
configure out the sysfs stuff could be useful without having
to go all the way to SLUB_TINY.

For the options that trade off performance against lower
fragmentation (MIN/MAX_PARTIAL, KMALLOC_RECLAIM, percpu
slabs), I wonder if it's possible to have a boot time
default based on the amount of RAM per CPU to have a better
tuned system on most cases, rather than having to go
to one extreme or the other at compile time.

    Arnd

https://openwrt.org/toh/views/toh_standard_all?datasrt=target&dataflt%5B0%5D=availability_%3DAvailable%202021
  

On 11/22/22 17:33, Arnd Bergmann wrote:
> On Mon, Nov 21, 2022, at 18:11, Vlastimil Babka wrote:
>>
>> this continues the discussion from [1]. Reasons to remove SLOB are
>> outlined there and no-one has objected so far. The last patch of this
>> series therefore deprecates CONFIG_SLOB and updates all the defconfigs
>> using CONFIG_SLOB=y in the tree.
>>
>> There is a k210 board with 8MB RAM where switching to SLUB caused issues
>> [2] and the lkp bot wasn't also happy about code bloat [3]. To address
>> both, this series introduces CONFIG_SLUB_TINY to perform some rather
>> low-hanging fruit modifications to SLUB to reduce its memory overhead.
>> This seems to have been successful at least in the k210 case [4]. I
>> consider this as an acceptable tradeoff for getting rid of SLOB.
> 
> I agree that this is a great success for replacing SLOB on the
> smallest machines that have 32MB or less and have to run a
> a highly customized kernel, and this is probably enough to
> have a drop-in replacement without making any currently working
> system worse.
> 
> On the other hand, I have the feeling that we may want something
> a bit less aggressive than this for machines that are slightly
> less constrained, in particular when a single kernel needs to
> scale from 64MB to 512MB, which can happen e.g. on OpenWRT.
> I have seen a number of reports over the years that suggest
> that new kernels handle fragmentation and low memory worse than
> old ones, and it would be great to improve that again.

I see. That would need to study such reports and see if the problem there is
actually SLUB, or the page allocator or something else entirely.

> I can imagine those machines wanting to use sysfs in general
> but not for the slab caches, so having a separate knob to
> configure out the sysfs stuff could be useful without having
> to go all the way to SLUB_TINY.

Right, but AFAIK that wouldn't save much except some text size and kobjects,
so probably negligible for >32MB?

> For the options that trade off performance against lower
> fragmentation (MIN/MAX_PARTIAL, KMALLOC_RECLAIM, percpu
> slabs), I wonder if it's possible to have a boot time
> default based on the amount of RAM per CPU to have a better
> tuned system on most cases, rather than having to go
> to one extreme or the other at compile time.

Possible for some of these things, but for others that brings us back to the
question what are the actual observed issues. If it's low memory in absolute
number of pages, these can help, but if it's fragmentation (and the kind if
RAM sizes should have page grouping by mobility enabled), ditching e.g. the
KMALLOC_RECLAIM could make it worse. Unfortunately some of these tradeoffs
can be rather unpredictable.

Thanks,
Vlastimil

>     Arnd
> 
> https://openwrt.org/toh/views/toh_standard_all?datasrt=target&dataflt%5B0%5D=availability_%3DAvailable%202021
  

On Tue, Nov 22, 2022, at 17:59, Vlastimil Babka wrote:
> On 11/22/22 17:33, Arnd Bergmann wrote:
>> On Mon, Nov 21, 2022, at 18:11, Vlastimil Babka wrote:
>> I can imagine those machines wanting to use sysfs in general
>> but not for the slab caches, so having a separate knob to
>> configure out the sysfs stuff could be useful without having
>> to go all the way to SLUB_TINY.
>
> Right, but AFAIK that wouldn't save much except some text size and kobjects,
> so probably negligible for >32MB?

Makes sense, I assume you have a better idea of how much this
could save. I'm not at all worried about the .text size, but
my initial guess was that the metadata for sysfs would be
noticeable.

>> For the options that trade off performance against lower
>> fragmentation (MIN/MAX_PARTIAL, KMALLOC_RECLAIM, percpu
>> slabs), I wonder if it's possible to have a boot time
>> default based on the amount of RAM per CPU to have a better
>> tuned system on most cases, rather than having to go
>> to one extreme or the other at compile time.
>
> Possible for some of these things, but for others that brings us back to the
> question what are the actual observed issues. If it's low memory in absolute
> number of pages, these can help, but if it's fragmentation (and the kind if
> RAM sizes should have page grouping by mobility enabled), ditching e.g. the
> KMALLOC_RECLAIM could make it worse. Unfortunately some of these tradeoffs
> can be rather unpredictable.

Are there any obvious wins on memory uage? I would guess that it
would be safe to e.g. ditch percpu slabs when running with less
128MB per CPU, and the MIN/MAX_PARTIAL values could easily
be a function of the number of pages in total or per cpu,
whichever makes most sense. As a side-effect, those could also
grow slightly larger on huge systems by scaling them with
log2(totalpages).

     Arnd
  

On Mon, Nov 21, 2022 at 06:11:50PM +0100, Vlastimil Babka wrote:
> Hi,
> 
> this continues the discussion from [1]. Reasons to remove SLOB are
> outlined there and no-one has objected so far. The last patch of this
> series therefore deprecates CONFIG_SLOB and updates all the defconfigs
> using CONFIG_SLOB=y in the tree.
> 
> There is a k210 board with 8MB RAM where switching to SLUB caused issues
> [2] and the lkp bot wasn't also happy about code bloat [3]. To address
> both, this series introduces CONFIG_SLUB_TINY to perform some rather
> low-hanging fruit modifications to SLUB to reduce its memory overhead.
> This seems to have been successful at least in the k210 case [4]. I
> consider this as an acceptable tradeoff for getting rid of SLOB.
> 
> The series is also available in git:
> https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slub-tiny-v1r2
> 
> [1] https://lore.kernel.org/all/b35c3f82-f67b-2103-7d82-7a7ba7521439@suse.cz/
> [2] https://lore.kernel.org/all/a5bba3ca-da19-293c-c01b-a28291533466@opensource.wdc.com/
> [3] https://lore.kernel.org/all/Y25E9cJbhDAKi1vd@99bb1221be19/
> [4] https://lore.kernel.org/all/6a1883c4-4c3f-545a-90e8-2cd805bcf4ae@opensource.wdc.com/
> 
> Vlastimil Babka (12):
>   mm, slab: ignore hardened usercopy parameters when disabled
>   mm, slub: add CONFIG_SLUB_TINY
>   mm, slub: disable SYSFS support with CONFIG_SLUB_TINY
>   mm, slub: retain no free slabs on partial list with CONFIG_SLUB_TINY
>   mm, slub: lower the default slub_max_order with CONFIG_SLUB_TINY
>   mm, slub: don't create kmalloc-rcl caches with CONFIG_SLUB_TINY
>   mm, slab: ignore SLAB_RECLAIM_ACCOUNT with CONFIG_SLUB_TINY
>   mm, slub: refactor free debug processing
>   mm, slub: split out allocations from pre/post hooks
>   mm, slub: remove percpu slabs with CONFIG_SLUB_TINY
>   mm, slub: don't aggressively inline with CONFIG_SLUB_TINY
>   mm, slob: rename CONFIG_SLOB to CONFIG_SLOB_DEPRECATED
> 
>  arch/arm/configs/clps711x_defconfig           |   3 +-
>  arch/arm/configs/collie_defconfig             |   3 +-
>  arch/arm/configs/multi_v4t_defconfig          |   3 +-
>  arch/arm/configs/omap1_defconfig              |   3 +-
>  arch/arm/configs/pxa_defconfig                |   3 +-
>  arch/arm/configs/tct_hammer_defconfig         |   3 +-
>  arch/arm/configs/xcep_defconfig               |   3 +-
>  arch/openrisc/configs/or1ksim_defconfig       |   3 +-
>  arch/openrisc/configs/simple_smp_defconfig    |   3 +-
>  arch/riscv/configs/nommu_k210_defconfig       |   3 +-
>  .../riscv/configs/nommu_k210_sdcard_defconfig |   3 +-
>  arch/riscv/configs/nommu_virt_defconfig       |   3 +-
>  arch/sh/configs/rsk7201_defconfig             |   3 +-
>  arch/sh/configs/rsk7203_defconfig             |   3 +-
>  arch/sh/configs/se7206_defconfig              |   3 +-
>  arch/sh/configs/shmin_defconfig               |   3 +-
>  arch/sh/configs/shx3_defconfig                |   3 +-
>  include/linux/slab.h                          |   8 +
>  include/linux/slub_def.h                      |   6 +-
>  kernel/configs/tiny.config                    |   5 +-
>  mm/Kconfig                                    |  38 +-
>  mm/Kconfig.debug                              |   2 +-
>  mm/slab_common.c                              |  16 +-
>  mm/slub.c                                     | 415 ++++++++++++------
>  24 files changed, 377 insertions(+), 164 deletions(-)

For the series

Acked-by: Mike Rapoport <rppt@linux.ibm.com>