[v6,2/3] random: introduce generic vDSO getrandom() implementation

  Provide a generic C vDSO getrandom() implementation, which operates on
an opaque state returned by vgetrandom_alloc() and produces random bytes
the same way as getrandom(). This has a the API signature:

  ssize_t vgetrandom(void *buffer, size_t len, unsigned int flags, void *opaque_state);

The return value and the first 3 arguments are the same as ordinary
getrandom(), while the last argument is a pointer to the opaque
allocated state. Were all four arguments passed to the getrandom()
syscall, nothing different would happen, and the functions would have
the exact same behavior.

The actual vDSO RNG algorithm implemented is the same one implemented by
drivers/char/random.c, using the same fast-erasure techniques as that.
Should the in-kernel implementation change, so too will the vDSO one.

It requires an implementation of ChaCha20 that does not use any stack,
in order to maintain forward secrecy, so this is left as an
architecture-specific fill-in. Stack-less ChaCha20 is an easy algorithm
to implement on a variety of architectures, so this shouldn't be too
onerous.

Initially, the state is keyless, and so the first call makes a
getrandom() syscall to generate that key, and then uses it for
subsequent calls. By keeping track of a generation counter, it knows
when its key is invalidated and it should fetch a new one using the
syscall. Later, more than just a generation counter might be used.

Since MADV_WIPEONFORK is set on the opaque state, the key and related
state is wiped during a fork(), so secrets don't roll over into new
processes, and the same state doesn't accidentally generate the same
random stream. The generation counter, as well, is always >0, so that
the 0 counter is a useful indication of a fork() or otherwise
uninitialized state.

If the kernel RNG is not yet initialized, then the vDSO always calls the
syscall, because that behavior cannot be emulated in userspace, but
fortunately that state is short lived and only during early boot. If it
has been initialized, then there is no need to inspect the `flags`
argument, because the behavior does not change post-initialization
regardless of the `flags` value.

Since the opaque state passed to it is mutated, vDSO getrandom() is not
reentrant, when used with the same opaque state, which libc should be
mindful of.

Together with the previous commit that introduces vgetrandom_alloc(),
this functionality is intended to be integrated into libc's thread
management. As an illustrative example, the following code might be used
to do the same outside of libc. All of the static functions are to be
considered implementation private, including the vgetrandom_alloc()
syscall wrapper, which generally shouldn't be exposed outside of libc,
with the non-static vgetrandom() function at the end being the exported
interface. The various pthread-isms are expected to be elided into libc
internals. This per-thread allocation scheme is very naive and does not
shrink; other implementations may choose to be more complex.

  static void *vgetrandom_alloc(size_t *num, size_t *size_per_each, unsigned int flags)
  {
    unsigned long ret = syscall(__NR_vgetrandom_alloc, num, size_per_each, flags);
    return ret > -4096UL ? NULL : (void *)ret;
  }

  static struct {
    pthread_mutex_t lock;
    void **states;
    size_t len, cap;
  } grnd_allocator = {
    .lock = PTHREAD_MUTEX_INITIALIZER
  };

  static void *vgetrandom_get_state(void)
  {
    void *state = NULL;

    pthread_mutex_lock(&grnd_allocator.lock);
    if (!grnd_allocator.len) {
      size_t new_cap, size_per_each, num = 16; /* Just a hint. */
      void *new_block = vgetrandom_alloc(&num, &size_per_each, 0), *new_states;

      if (!new_block)
        goto out;
      new_cap = grnd_allocator.cap + num;
      new_states = reallocarray(grnd_allocator.states, new_cap, sizeof(*grnd_allocator.states));
      if (!new_states) {
        munmap(new_block, num * size_per_each);
        goto out;
      }
      grnd_allocator.cap = new_cap;
      grnd_allocator.states = new_states;

      for (size_t i = 0; i < num; ++i) {
        grnd_allocator.states[i] = new_block;
        new_block += size_per_each;
      }
      grnd_allocator.len = num;
    }
    state = grnd_allocator.states[--grnd_allocator.len];

  out:
    pthread_mutex_unlock(&grnd_allocator.lock);
    return state;
  }

  static void vgetrandom_put_state(void *state)
  {
    if (!state)
      return;
    pthread_mutex_lock(&grnd_allocator.lock);
    grnd_allocator.states[grnd_allocator.len++] = state;
    pthread_mutex_unlock(&grnd_allocator.lock);
  }

  static struct {
    ssize_t(*fn)(void *buf, size_t len, unsigned long flags, void *state);
    pthread_key_t key;
    pthread_once_t initialized;
  } grnd_ctx = {
    .initialized = PTHREAD_ONCE_INIT
  };

  static void vgetrandom_init(void)
  {
    if (pthread_key_create(&grnd_ctx.key, vgetrandom_put_state) != 0)
      return;
    grnd_ctx.fn = __vdsosym("LINUX_2.6", "__vdso_getrandom");
  }

  ssize_t vgetrandom(void *buf, size_t len, unsigned long flags)
  {
    void *state;

    pthread_once(&grnd_ctx.initialized, vgetrandom_init);
    if (!grnd_ctx.fn)
      return getrandom(buf, len, flags);
    state = pthread_getspecific(grnd_ctx.key);
    if (!state) {
      state = vgetrandom_get_state();
      if (pthread_setspecific(grnd_ctx.key, state) != 0) {
        vgetrandom_put_state(state);
        state = NULL;
      }
      if (!state)
        return getrandom(buf, len, flags);
    }
    return grnd_ctx.fn(buf, len, flags, state);
  }

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
 MAINTAINERS             |   1 +
 drivers/char/random.c   |   9 ++++
 include/vdso/datapage.h |   6 +++
 lib/vdso/Kconfig        |   5 ++
 lib/vdso/getrandom.c    | 113 ++++++++++++++++++++++++++++++++++++++++
 5 files changed, 134 insertions(+)
 create mode 100644 lib/vdso/getrandom.c
  

On 21/11/2022 16.29, Jason A. Donenfeld wrote:

Cc += linux-api

> 
>       if (!new_block)
>         goto out;
>       new_cap = grnd_allocator.cap + num;
>       new_states = reallocarray(grnd_allocator.states, new_cap, sizeof(*grnd_allocator.states));
>       if (!new_states) {
>         munmap(new_block, num * size_per_each);

Hm. This does leak an implementation detail of vgetrandom_alloc(),
namely that it is based on mmap() of that size rounded up to page size.
Do we want to commit to this being the proper way of disposing of a
succesful vgetrandom_alloc(), or should there also be a
vgetrandom_free(void *states, long num, long size_per_each)?

And if so, what color should the bikeshed really have. I.e.,

- does it need to take that size_per_each parameter which the kernel knows

- should it rather take the product so it can for now be a simple alias
for munmap

- should it also have a flags argument just because that's what all
well-behaving syscalls have these days...

Also, should vgetrandom_alloc() take a void *hint argument that
would/could be passed through to mmap() to give userspace some control
over where the memory is located - possibly only in the future, i.e.
insist on it being NULL for now, but it could open the possibility for
adding e.g. VGRND_MAP_FIXED[_NOREPLACE] that would translate to the
corresponding MAP_ flags.

Rasmus
  

* Jason A. Donenfeld:

>   static void *vgetrandom_alloc(size_t *num, size_t *size_per_each, unsigned int flags)
>   {
>     unsigned long ret = syscall(__NR_vgetrandom_alloc, num, size_per_each, flags);
>     return ret > -4096UL ? NULL : (void *)ret;
>   }

The traditional syscall function returns -1 on error and set errors, so
using unsing long and the 4096 is quite misleading.

Thanks,
Florian
  

Hi Florian,

On Wed, Nov 23, 2022 at 11:48:06AM +0100, Florian Weimer wrote:
> * Jason A. Donenfeld:
> 
> >   static void *vgetrandom_alloc(size_t *num, size_t *size_per_each, unsigned int flags)
> >   {
> >     unsigned long ret = syscall(__NR_vgetrandom_alloc, num, size_per_each, flags);
> >     return ret > -4096UL ? NULL : (void *)ret;
> >   }
> 
> The traditional syscall function returns -1 on error and set errors, so
> using unsing long and the 4096 is quite misleading.

Not sure I have any idea at all whatsoever about what you're talking
about. Firstly, the function you quoted is from the "sample userspace
code" in the commit message, so it might not be code for the context you
have in mind.

Secondly, it's just doing the thing to figure out if the return value is
an error value or a pointer. Were we in glibc, we'd write this as:

    return INTERNAL_SYSCALL_ERROR_P(r) ? NULL : (void *) r;

Right? And if you look at the expansion of that glibc macro, it's just:

  #define INTERNAL_SYSCALL_ERROR_P(val) \
    ((unsigned long int) (val) > -4096UL)

So it looks like the same exact thing?

The only difference I could see is that I assign it to a `unsigned long
ret`, while glibc code tends to assign it to a `long r`? Is that the
difference you're pointing out? Except that clearly doesn't matter
because it just gets casted to unsigned by that macro anyway?

Confused.

Jason
  

Hi Rasmus,

On Wed, Nov 23, 2022 at 09:51:04AM +0100, Rasmus Villemoes wrote:
> On 21/11/2022 16.29, Jason A. Donenfeld wrote:
> 
> Cc += linux-api
> 
> > 
> >       if (!new_block)
> >         goto out;
> >       new_cap = grnd_allocator.cap + num;
> >       new_states = reallocarray(grnd_allocator.states, new_cap, sizeof(*grnd_allocator.states));
> >       if (!new_states) {
> >         munmap(new_block, num * size_per_each);
> 
> Hm. This does leak an implementation detail of vgetrandom_alloc(),
> namely that it is based on mmap() of that size rounded up to page size.
> Do we want to commit to this being the proper way of disposing of a
> succesful vgetrandom_alloc(), or should there also be a
> vgetrandom_free(void *states, long num, long size_per_each)?

Yes, this is intentional, and this is exactly what I wanted to do. There
are various wrappers of vm_mmap() throughout, mmap being one of them,
and they typically then resort to munmap to unmap it. This is how
userspace handles memory - maps, always maps. So I think doing that is
fine and consistent.

However, your point about it relying on it being a rounded up size isn't
correct. `munmap` will unmap the whole page if the size you pass lies
within a page. So `num * size_of_each` will always do the right thing,
without needing to have userspace code round anything up. (From the man
page: "The  address addr must be a multiple of the page size (but length
need not be). All pages containing a part of the indicated range are
unmapped.") And as you can see in my example code, nothing is rounded
up. So I don't know why you made that comment.

> And if so, what color should the bikeshed really have. I.e.,

No color, thanks.

> Also, should vgetrandom_alloc() take a void *hint argument that
> would/could be passed through to mmap() to give userspace some control
> over where the memory is located - possibly only in the future, i.e.
> insist on it being NULL for now, but it could open the possibility for
> adding e.g. VGRND_MAP_FIXED[_NOREPLACE] that would translate to the
> corresponding MAP_ flags.

I think adding more control is exactly what this is trying to avoid.
It's very intentionally *not* a general allocator function, but
something specific for vDSO getrandom(). However, it does already, in
this very patchset here, take a (currently unused) flags argument, in
case we have the need for later extension.

In the meantime, however, I'm not very interested in complicating this
interface into oblivion. Firstly, it ensures nothing will get done. But
moreover, this interface needs to be somewhat future-proof, yes, but it
does not need to be a general syscall; rather, it's a specific syscall
for a specific task.

Jason
  

* Jason A. Donenfeld:

> Hi Florian,
>
> On Wed, Nov 23, 2022 at 11:48:06AM +0100, Florian Weimer wrote:
>> * Jason A. Donenfeld:
>> 
>> >   static void *vgetrandom_alloc(size_t *num, size_t *size_per_each, unsigned int flags)
>> >   {
>> >     unsigned long ret = syscall(__NR_vgetrandom_alloc, num, size_per_each, flags);
>> >     return ret > -4096UL ? NULL : (void *)ret;
>> >   }
>> 
>> The traditional syscall function returns -1 on error and set errors, so
>> using unsing long and the 4096 is quite misleading.
>
> Not sure I have any idea at all whatsoever about what you're talking
> about. Firstly, the function you quoted is from the "sample userspace
> code" in the commit message, so it might not be code for the context you
> have in mind.

I'm talking about the syscall function that is available through
userspace via <sys/syscall.h>.

> Secondly, it's just doing the thing to figure out if the return value is
> an error value or a pointer. Were we in glibc, we'd write this as:
>
>     return INTERNAL_SYSCALL_ERROR_P(r) ? NULL : (void *) r;
>
> Right? And if you look at the expansion of that glibc macro, it's just:
>
>   #define INTERNAL_SYSCALL_ERROR_P(val) \
>     ((unsigned long int) (val) > -4096UL)
>
> So it looks like the same exact thing?

syscall already does internally (with a translation to -1, not NULL), so
the caller shouldn't do it again.  The userspace syscall function does
*not* return an error code.

Thanks,
Florian
  

Hi Florian,

On Thu, Nov 24, 2022 at 06:28:44AM +0100, Florian Weimer wrote:
> > Right? And if you look at the expansion of that glibc macro, it's just:
> >
> >   #define INTERNAL_SYSCALL_ERROR_P(val) \
> >     ((unsigned long int) (val) > -4096UL)
> >
> > So it looks like the same exact thing?
> 
> syscall already does internally (with a translation to -1, not NULL), so
> the caller shouldn't do it again.  The userspace syscall function does
> *not* return an error code.

Ahh, okay. Thanks. I'll fix up the example to assume those semantics.

Jason
  

On 24/11/2022 02.18, Jason A. Donenfeld wrote:
> Hi Rasmus,
> 
> On Wed, Nov 23, 2022 at 09:51:04AM +0100, Rasmus Villemoes wrote:
>> On 21/11/2022 16.29, Jason A. Donenfeld wrote:
>>
>> Cc += linux-api
>>
>>>
>>>       if (!new_block)
>>>         goto out;
>>>       new_cap = grnd_allocator.cap + num;
>>>       new_states = reallocarray(grnd_allocator.states, new_cap, sizeof(*grnd_allocator.states));
>>>       if (!new_states) {
>>>         munmap(new_block, num * size_per_each);
>>
>> Hm. This does leak an implementation detail of vgetrandom_alloc(),
>> namely that it is based on mmap() of that size rounded up to page size.
>> Do we want to commit to this being the proper way of disposing of a
>> succesful vgetrandom_alloc(), or should there also be a
>> vgetrandom_free(void *states, long num, long size_per_each)?
> 
> Yes, this is intentional, and this is exactly what I wanted to do. There
> are various wrappers of vm_mmap() throughout, mmap being one of them,
> and they typically then resort to munmap to unmap it. This is how
> userspace handles memory - maps, always maps. So I think doing that is
> fine and consistent.

OK. Perhaps for the benefit of future libc implementors drop a comment
somewhere as to how to dealloc the blob.

> However, your point about it relying on it being a rounded up size isn't
> correct. `munmap` will unmap the whole page if the size you pass lies
> within a page. So `num * size_of_each` will always do the right thing,
> without needing to have userspace code round anything up. (From the man
> page: "The  address addr must be a multiple of the page size (but length
> need not be). 

I know, and I never said userspace needed to round anything up.

All pages containing a part of the indicated range are
> unmapped.") And as you can see in my example code, nothing is rounded
> up. So I don't know why you made that comment.

I made that comment because it's clear from what this does that you get
something back that is _at least_ num*size_per_each in size, but what is
not clear is that the actual allocation is exactly and will always be
that size rounded up to a page size (and no more), so that
munmap(num*size_per_each), with its well-known and documented semantics,
will DTRT.

> I think adding more control is exactly what this is trying to avoid.
> It's very intentionally *not* a general allocator function, but
> something specific for vDSO getrandom(). However, it does already, in
> this very patchset here, take a (currently unused) flags argument, in
> case we have the need for later extension.

OK.

Perhaps you can spend a few more words on why this allocation _needs_ to
be MAP_LOCKED? That seems somewhat of a policy thing imposed by the
kernel, something that would be better left to the libc or distro or
whatnot to request via a flag. I could imagine applications that
currently run at the mlock limit start failing after a libc upgrade -
which could of course be considered a libc problem, and perhaps it's too
unlikely to worry about.

Rasmus