[v3,5/5] arm64: mte: add compression support to mteswap.c

  Define the internal mteswap.h interface:
 - _mte_alloc_and_save_tags()
 - _mte_free_saved_tags()
 - _mte_restore_tags()

, that encapsulates saving tags for a struct page (together with memory
allocation), restoring tags, and deleting the storage allocated for them.

These functions accept opaque pointers, which may point to 128-byte
tag buffers, as well as smaller buffers containing compressed tags, or
have compressed tags stored directly in them.

The existing code from mteswap.c operating with uncompressed tags is split
away into mteswap_nocomp.c, and the newly introduced mteswap_comp.c
provides compression with the EA0 algorithm. The latter implementation
is picked if CONFIG_ARM64_MTE_COMP=y.

Soon after booting Android, tag compression saves ~2.5x memory previously
spent by mteswap.c on tag allocations. With the growing uptime, the
savings reach 20x and even more.

Signed-off-by: Alexander Potapenko <glider@google.com>

---
 v3:
  - Addressed comments by Andy Shevchenko in another patch:
   - fixed includes order
   - replaced u64 with unsigned long
   - added MODULE_IMPORT_NS(MTECOMP)
---
 arch/arm64/mm/Makefile         |  5 ++++
 arch/arm64/mm/mteswap.c        | 20 ++++++-------
 arch/arm64/mm/mteswap.h        | 12 ++++++++
 arch/arm64/mm/mteswap_comp.c   | 52 ++++++++++++++++++++++++++++++++++
 arch/arm64/mm/mteswap_nocomp.c | 38 +++++++++++++++++++++++++
 5 files changed, 116 insertions(+), 11 deletions(-)
 create mode 100644 arch/arm64/mm/mteswap.h
 create mode 100644 arch/arm64/mm/mteswap_comp.c
 create mode 100644 arch/arm64/mm/mteswap_nocomp.c
  

On Mon, Jul 17, 2023 at 01:37:08PM +0200, Alexander Potapenko wrote:
> Define the internal mteswap.h interface:
>  - _mte_alloc_and_save_tags()
>  - _mte_free_saved_tags()
>  - _mte_restore_tags()
> 
> , that encapsulates saving tags for a struct page (together with memory
> allocation), restoring tags, and deleting the storage allocated for them.
> 
> These functions accept opaque pointers, which may point to 128-byte
> tag buffers, as well as smaller buffers containing compressed tags, or
> have compressed tags stored directly in them.
> 
> The existing code from mteswap.c operating with uncompressed tags is split
> away into mteswap_nocomp.c, and the newly introduced mteswap_comp.c
> provides compression with the EA0 algorithm. The latter implementation
> is picked if CONFIG_ARM64_MTE_COMP=y.
> 
> Soon after booting Android, tag compression saves ~2.5x memory previously
> spent by mteswap.c on tag allocations. With the growing uptime, the
> savings reach 20x and even more.

...

> +#ifndef ARCH_ARM64_MM_MTESWAP_H_
> +#define ARCH_ARM64_MM_MTESWAP_H_

> +#include <linux/mm_types.h>

But you actually don't use that.

struct page;

forward declaration is enough.

> +void *_mte_alloc_and_save_tags(struct page *page);
> +void _mte_free_saved_tags(void *tags);
> +void _mte_restore_tags(void *tags, struct page *page);
> +
> +#endif // ARCH_ARM64_MM_MTESWAP_H_

...

> +void _mte_free_saved_tags(void *storage)
> +{
> +	unsigned long handle = xa_to_value(storage);
> +	int size;
> +
> +	if (!handle)
> +		return;

Perhaps

	unsigned long handle;

	handle = xa_to_value(storage);
	if (!handle)
		return;

> +	size = ea0_storage_size(handle);
> +	ea0_release_handle(handle);
> +}

> +void _mte_restore_tags(void *tags, struct page *page)
> +{

As per above.

> +	if (try_page_mte_tagging(page)) {
> +		if (!ea0_decompress(handle, tags_decomp))
> +			return;
> +		mte_restore_page_tags(page_address(page), tags_decomp);
> +		set_page_mte_tagged(page);
> +	}

I think you may drop an indentation level by

	if (!try_page_mte_tagging(page))
		return;

> +}

...

> +void _mte_restore_tags(void *tags, struct page *page)
> +{
> +	if (try_page_mte_tagging(page)) {
> +		mte_restore_page_tags(page_address(page), tags);
> +		set_page_mte_tagged(page);
> +	}

Ditto.

> +}
  

> > +#include <linux/mm_types.h>
>
> But you actually don't use that.
>
> struct page;
>
> forward declaration is enough.
Fair enough.
>
> > +void *_mte_alloc_and_save_tags(struct page *page);
> > +void _mte_free_saved_tags(void *tags);
> > +void _mte_restore_tags(void *tags, struct page *page);
> > +
> > +#endif // ARCH_ARM64_MM_MTESWAP_H_
>
> ...
>
> > +void _mte_free_saved_tags(void *storage)
> > +{
> > +     unsigned long handle = xa_to_value(storage);
> > +     int size;
> > +
> > +     if (!handle)
> > +             return;
>
> Perhaps
>
>         unsigned long handle;
>
>         handle = xa_to_value(storage);
>         if (!handle)
>                 return;

I don't have a strong preference and am happy to change this, but, out
of curiosity, why do you think it is better?
This pattern (calling (even non-)trivial functions when declaring
variables) is widely used across the kernel.
Or is it just for consistency with how `handle` is used in the rest of the file?


> > +void _mte_restore_tags(void *tags, struct page *page)
> > +{
>
> As per above.

Ack

> > +     if (try_page_mte_tagging(page)) {
> > +             if (!ea0_decompress(handle, tags_decomp))
> > +                     return;
> > +             mte_restore_page_tags(page_address(page), tags_decomp);
> > +             set_page_mte_tagged(page);
> > +     }
>
> I think you may drop an indentation level by
>
>         if (!try_page_mte_tagging(page))
>                 return;
>
> > +}

Ack

> ...
>
> > +void _mte_restore_tags(void *tags, struct page *page)
> > +{
> > +     if (try_page_mte_tagging(page)) {
> > +             mte_restore_page_tags(page_address(page), tags);
> > +             set_page_mte_tagged(page);
> > +     }
>
> Ditto.
Thanks!
  

On Tue, Jul 18, 2023 at 12:48:00PM +0200, Alexander Potapenko wrote:

...

> > > +void _mte_free_saved_tags(void *storage)
> > > +{
> > > +     unsigned long handle = xa_to_value(storage);
> > > +     int size;
> > > +
> > > +     if (!handle)
> > > +             return;
> >
> > Perhaps
> >
> >         unsigned long handle;
> >
> >         handle = xa_to_value(storage);
> >         if (!handle)
> >                 return;
> 
> I don't have a strong preference and am happy to change this, but, out
> of curiosity, why do you think it is better?
> This pattern (calling (even non-)trivial functions when declaring
> variables) is widely used across the kernel.
> Or is it just for consistency with how `handle` is used in the rest of the file?

Ah, it's pure maintenance and error prone approach in case some code is sneezed
in between.

Imagine that you have


	foo = bar(x);
	...many lines that by some reason don't make one page on the screen...
	if (!foo)
		...do something...

Now if by unsuccessful rebase or by non-experienced developer we got

	foo = bar(x);
	...part 1 of many lines that by some reason don't make one page on the screen...
	baz(foo);
	...part 2 of many lines that by some reason don't make one page on the screen...
	if (!foo)
		...do something...

the compiler will eliminate the check — you got your mine on the nice minefield!

> > > +}