[05/30] selftests/rseq: Use ELF auxiliary vector for extensible rseq

  Use the ELF auxiliary vector AT_RSEQ_FEATURE_SIZE to detect the RSEQ
features supported by the kernel.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
 tools/testing/selftests/rseq/rseq-abi.h |  5 ++
 tools/testing/selftests/rseq/rseq.c     | 67 ++++++++++++++++++++++---
 tools/testing/selftests/rseq/rseq.h     | 18 +++++--
 3 files changed, 78 insertions(+), 12 deletions(-)
  

* Mathieu Desnoyers:

> +static
> +unsigned int get_rseq_feature_size(void)
> +{
> +	unsigned long auxv_rseq_feature_size, auxv_rseq_align;
> +
> +	auxv_rseq_align = getauxval(AT_RSEQ_ALIGN);
> +	assert(!auxv_rseq_align || auxv_rseq_align <= RSEQ_THREAD_AREA_ALLOC_SIZE);
> +
> +	auxv_rseq_feature_size = getauxval(AT_RSEQ_FEATURE_SIZE);
> +	assert(!auxv_rseq_feature_size || auxv_rseq_feature_size <= RSEQ_THREAD_AREA_ALLOC_SIZE);
> +	if (auxv_rseq_feature_size)
> +		return auxv_rseq_feature_size;
> +	else
> +		return ORIG_RSEQ_FEATURE_SIZE;
> +}

Do you intend to use the auxiliary vector as the userspace handshake
for glibc-managed rseq, too?  I don't think it works if the kernel
overtakes glibc.  Or is there some other approach shown in the series
that I missed?

Maybe we should just skip the existing padding and use it only for
some vaguely kernel-internal purpose (say through a vDSO helper), so
that it is less of an issue how to communicate the presence of these
fields to userspace.
  

On 2023-01-04 14:14, Florian Weimer wrote:
> * Mathieu Desnoyers:
> 
>> +static
>> +unsigned int get_rseq_feature_size(void)
>> +{
>> +	unsigned long auxv_rseq_feature_size, auxv_rseq_align;
>> +
>> +	auxv_rseq_align = getauxval(AT_RSEQ_ALIGN);
>> +	assert(!auxv_rseq_align || auxv_rseq_align <= RSEQ_THREAD_AREA_ALLOC_SIZE);
>> +
>> +	auxv_rseq_feature_size = getauxval(AT_RSEQ_FEATURE_SIZE);
>> +	assert(!auxv_rseq_feature_size || auxv_rseq_feature_size <= RSEQ_THREAD_AREA_ALLOC_SIZE);
>> +	if (auxv_rseq_feature_size)
>> +		return auxv_rseq_feature_size;
>> +	else
>> +		return ORIG_RSEQ_FEATURE_SIZE;
>> +}
> 
> Do you intend to use the auxiliary vector as the userspace handshake
> for glibc-managed rseq, too?

Yes.

I don't think it works if the kernel
> overtakes glibc.  Or is there some other approach shown in the series
> that I missed?

The handshake I am proposing is as follows:

1- libc init:

issues getauxval(AT_RSEQ_FEATURE_SIZE) to learn the rseq feature size supported
by the Linux kernel. It can be either:

a) getauxval(AT_RSEQ_FEATURE_SIZE) == 0, errno=ENOENT: pre-6.3 kernel or CONFIG_RSEQ=n.

Need to issue the rseq system call to figure out if rseq is implemented/available or not.
If rseq is indeed implemented, use a __rseq_size=32.

b) getauxval(AT_RSEQ_FEATURE_SIZE) > 0:

The kernel exposes its supported rseq feature size. libc either needs to register rseq with a rseq_len
of 32-byte (original size), or with a rseq_len larger than 32 bytes with enough space to hold all
features.

2- Now about applications (and libc) use of rseq fields:

Using both __rseq_size (from libc) and the result of getauxval(AT_RSEQ_FEATURE_SIZE),
a rseq user can figure which rseq fields can indeed be used. The important part is
how get_rseq_feature_size() is called in the rseq selftests:


                 rseq_feature_size = get_rseq_feature_size();
                 if (rseq_feature_size > rseq_size)
                         rseq_feature_size = rseq_size;

which basically sets rseq_feature_size to the feature size exposed by the kernel, except
if libc's __rseq_size is smaller than the feature size exposed by the kernel, in which case
it will truncate the rseq_feature_size to __rseq_size.

This allows rseq users to know which feature set is supported by the kernel and for which
libc has allocated enough space.

The only thing here is that rseq users cannot rely on libc's __rseq_size symbol to get the
feature size. But considering that this is a contract between the kernel and the rseq user
(libc is mostly just there to allocate per-thread memory), I don't think it's a concern to
request users to query getauxval(AT_RSEQ_FEATURE_SIZE) in addition to load __rseq_size.

> 
> Maybe we should just skip the existing padding and use it only for
> some vaguely kernel-internal purpose (say through a vDSO helper), so
> that it is less of an issue how to communicate the presence of these
> fields to userspace.

The fact that libc's __rseq_size is included the original struct rseq padding is unfortunate,
but I really see this as a purely userspace ABI concern, which should not dictate the layout
of the kernel ABI exposed to user-space, especially given that all the information required to
allow rseq users to know which fields can be used is readily available by combining the value
loaded from __rseq_size and the result of getauxval(AT_RSEQ_FEATURE_SIZE).

Thoughts ?

Thanks,

Mathieu
  

* Mathieu Desnoyers:

> 2- Now about applications (and libc) use of rseq fields:
>
> Using both __rseq_size (from libc) and the result of
> getauxval(AT_RSEQ_FEATURE_SIZE), a rseq user can figure which rseq
> fields can indeed be used. The important part is how
> get_rseq_feature_size() is called in the rseq selftests:
>
>
>                  rseq_feature_size = get_rseq_feature_size();
>                  if (rseq_feature_size > rseq_size)
>                          rseq_feature_size = rseq_size;
>
> which basically sets rseq_feature_size to the feature size exposed
> by the kernel, except if libc's __rseq_size is smaller than the
> feature size exposed by the kernel, in which case it will truncate
> the rseq_feature_size to __rseq_size.

Ahh, this happens to work because we pass 32 today from glibc, and
there is nothing left to do in glibc to enable these new fields.

If true, that really argues in favor of this approach.

>> Maybe we should just skip the existing padding and use it only for
>> some vaguely kernel-internal purpose (say through a vDSO helper), so
>> that it is less of an issue how to communicate the presence of these
>> fields to userspace.
>
> The fact that libc's __rseq_size is included the original struct
> rseq padding is unfortunate, but I really see this as a purely
> userspace ABI concern, which should not dictate the layout of the
> kernel ABI exposed to user-space, especially given that all the
> information required to allow rseq users to know which fields can be
> used is readily available by combining the value loaded from
> __rseq_size and the result of getauxval(AT_RSEQ_FEATURE_SIZE).

But we must pass size 32 to the kernel today, otherwise rseq
registration fails.  It's a kernel-mandated value, not something
that's purely a userspace concern.
  

On 2023-01-05 11:19, Florian Weimer wrote:
> * Mathieu Desnoyers:
> 
>> 2- Now about applications (and libc) use of rseq fields:
>>
>> Using both __rseq_size (from libc) and the result of
>> getauxval(AT_RSEQ_FEATURE_SIZE), a rseq user can figure which rseq
>> fields can indeed be used. The important part is how
>> get_rseq_feature_size() is called in the rseq selftests:
>>
>>
>>                   rseq_feature_size = get_rseq_feature_size();
>>                   if (rseq_feature_size > rseq_size)
>>                           rseq_feature_size = rseq_size;
>>
>> which basically sets rseq_feature_size to the feature size exposed
>> by the kernel, except if libc's __rseq_size is smaller than the
>> feature size exposed by the kernel, in which case it will truncate
>> the rseq_feature_size to __rseq_size.
> 
> Ahh, this happens to work because we pass 32 today from glibc, and
> there is nothing left to do in glibc to enable these new fields.
> 
> If true, that really argues in favor of this approach.

Yes, you are correct..

> 
>>> Maybe we should just skip the existing padding and use it only for
>>> some vaguely kernel-internal purpose (say through a vDSO helper), so
>>> that it is less of an issue how to communicate the presence of these
>>> fields to userspace.
>>
>> The fact that libc's __rseq_size is included the original struct
>> rseq padding is unfortunate, but I really see this as a purely
>> userspace ABI concern, which should not dictate the layout of the
>> kernel ABI exposed to user-space, especially given that all the
>> information required to allow rseq users to know which fields can be
>> used is readily available by combining the value loaded from
>> __rseq_size and the result of getauxval(AT_RSEQ_FEATURE_SIZE).
> 
> But we must pass size 32 to the kernel today, otherwise rseq
> registration fails.  It's a kernel-mandated value, not something
> that's purely a userspace concern.

What I mean when stating the "userspace concern" is the semantic of the libc's
__rseq_size symbol: whether it means allocated space (including padding)
or actively populated feature fields.

In terms of rseq registration rseq_len argument, here is the updated check
in the system call:

         /*
          * If there was no rseq previously registered, ensure the provided rseq
          * is properly aligned, as communcated to user-space through the ELF
          * auxiliary vector AT_RSEQ_ALIGN. If rseq_len is the original rseq
          * size, the required alignment is the original struct rseq alignment.
          *
          * In order to be valid, rseq_len is either the original rseq size, or
          * large enough to contain all supported fields, as communicated to
          * user-space through the ELF auxiliary vector AT_RSEQ_FEATURE_SIZE.
          */
         if (rseq_len < ORIG_RSEQ_SIZE ||
             (rseq_len == ORIG_RSEQ_SIZE && !IS_ALIGNED((unsigned long)rseq, ORIG_RSEQ_SIZE)) ||
             (rseq_len != ORIG_RSEQ_SIZE && (!IS_ALIGNED((unsigned long)rseq, __alignof__(*rseq)) ||
                                             rseq_len < offsetof(struct rseq, end))))
                 return -EINVAL;

Which keeps accepting rseq_len=32 (original ABI), else requires that enough space
is available to hold all supported feature fields (but never less than 32 bytes).

Do my explanations take care of your concerns, or are there still aspects that
you are uneasy with ?

Thanks,

Mathieu