[v2,4/5] mm: Add new ptep_deref() helper to fully encapsulate pte_t

  There are many call sites that directly dereference a pte_t pointer.
This makes it very difficult to properly encapsulate a page table in the
arch code without having to allocate shadow page tables. ptep_deref()
aims to solve this by replacing all direct dereferences with a call to
this function.

The default implementation continues to just dereference the pointer
(*ptep), so generated code should be exactly the same. However, it is
possible for the architecture to override the default with their own
implementation, that can (e.g.) hide certain bits from the core code, or
determine young/dirty status by mixing in state from another source.

While ptep_get() and ptep_get_lockless() already exist, these are
implemented as atomic accesses (e.g. READ_ONCE() in the default case).
So rather than using ptep_get() and risking performance regressions,
introduce an new variant.

Call sites will be converted to use the accessor in future commits.

Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
 include/linux/pgtable.h | 7 +++++++
 1 file changed, 7 insertions(+)
  

On Thu, May 18, 2023 at 5:07 AM Ryan Roberts <ryan.roberts@arm.com> wrote:
>
> There are many call sites that directly dereference a pte_t pointer.
> This makes it very difficult to properly encapsulate a page table in the
> arch code without having to allocate shadow page tables. ptep_deref()
> aims to solve this by replacing all direct dereferences with a call to
> this function.
>
> The default implementation continues to just dereference the pointer
> (*ptep), so generated code should be exactly the same. However, it is
> possible for the architecture to override the default with their own
> implementation, that can (e.g.) hide certain bits from the core code, or
> determine young/dirty status by mixing in state from another source.
>
> While ptep_get() and ptep_get_lockless() already exist, these are
> implemented as atomic accesses (e.g. READ_ONCE() in the default case).
> So rather than using ptep_get() and risking performance regressions,
> introduce an new variant.

We should reuse ptep_get():
1. I don't think READ_ONCE() can cause measurable regressions in this case.
2. It's technically wrong without it.
  

On 18/05/2023 20:28, Yu Zhao wrote:
> On Thu, May 18, 2023 at 5:07 AM Ryan Roberts <ryan.roberts@arm.com> wrote:
>>
>> There are many call sites that directly dereference a pte_t pointer.
>> This makes it very difficult to properly encapsulate a page table in the
>> arch code without having to allocate shadow page tables. ptep_deref()
>> aims to solve this by replacing all direct dereferences with a call to
>> this function.
>>
>> The default implementation continues to just dereference the pointer
>> (*ptep), so generated code should be exactly the same. However, it is
>> possible for the architecture to override the default with their own
>> implementation, that can (e.g.) hide certain bits from the core code, or
>> determine young/dirty status by mixing in state from another source.
>>
>> While ptep_get() and ptep_get_lockless() already exist, these are
>> implemented as atomic accesses (e.g. READ_ONCE() in the default case).
>> So rather than using ptep_get() and risking performance regressions,
>> introduce an new variant.
> 
> We should reuse ptep_get():
> 1. I don't think READ_ONCE() can cause measurable regressions in this case.
> 2. It's technically wrong without it.

Can you clarify what you mean by technically wrong? Are you saying that the
current code that does direct dereferencing is buggy?

I previously convinced myself that the potential for the compiler generating
multiple loads was safe because the code in question is under the PTL so there
are no concurrent stores. And we shouldn't see any tearing for the same reason.

That said, if there is concensus that we can just use ptep_get() (==
READ_ONCE()) everywhere, then I agree that would be cleaner. Does anyone object?
  

On Thu, May 18, 2023 at 12:07:26PM +0100, Ryan Roberts wrote:
> There are many call sites that directly dereference a pte_t pointer.
> This makes it very difficult to properly encapsulate a page table in the
> arch code without having to allocate shadow page tables. ptep_deref()
> aims to solve this by replacing all direct dereferences with a call to
> this function.
> 
> The default implementation continues to just dereference the pointer
> (*ptep), so generated code should be exactly the same. However, it is
> possible for the architecture to override the default with their own
> implementation, that can (e.g.) hide certain bits from the core code, or
> determine young/dirty status by mixing in state from another source.
> 
> While ptep_get() and ptep_get_lockless() already exist, these are
> implemented as atomic accesses (e.g. READ_ONCE() in the default case).
> So rather than using ptep_get() and risking performance regressions,
> introduce an new variant.
> 
> Call sites will be converted to use the accessor in future commits.
> 
> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
> ---
>  include/linux/pgtable.h | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> index c5a51481bbb9..1161beab2492 100644
> --- a/include/linux/pgtable.h
> +++ b/include/linux/pgtable.h
> @@ -204,6 +204,13 @@ static inline int pudp_set_access_flags(struct vm_area_struct *vma,
>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>  #endif
>  
> +#ifndef ptep_deref
> +static inline pte_t ptep_deref(pte_t *ptep)
> +{
> +	return *(pte_t *)ptep;

Why do you need the casting here?

> +}
> +#endif
> +
>  #ifndef __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
>  static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
>  					    unsigned long address,
> -- 
> 2.25.1
> 
>
  

On 24/05/2023 20:06, Mike Rapoport wrote:
> On Thu, May 18, 2023 at 12:07:26PM +0100, Ryan Roberts wrote:
>> There are many call sites that directly dereference a pte_t pointer.
>> This makes it very difficult to properly encapsulate a page table in the
>> arch code without having to allocate shadow page tables. ptep_deref()
>> aims to solve this by replacing all direct dereferences with a call to
>> this function.
>>
>> The default implementation continues to just dereference the pointer
>> (*ptep), so generated code should be exactly the same. However, it is
>> possible for the architecture to override the default with their own
>> implementation, that can (e.g.) hide certain bits from the core code, or
>> determine young/dirty status by mixing in state from another source.
>>
>> While ptep_get() and ptep_get_lockless() already exist, these are
>> implemented as atomic accesses (e.g. READ_ONCE() in the default case).
>> So rather than using ptep_get() and risking performance regressions,
>> introduce an new variant.
>>
>> Call sites will be converted to use the accessor in future commits.
>>
>> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
>> ---
>>  include/linux/pgtable.h | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
>> index c5a51481bbb9..1161beab2492 100644
>> --- a/include/linux/pgtable.h
>> +++ b/include/linux/pgtable.h
>> @@ -204,6 +204,13 @@ static inline int pudp_set_access_flags(struct vm_area_struct *vma,
>>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>>  #endif
>>  
>> +#ifndef ptep_deref
>> +static inline pte_t ptep_deref(pte_t *ptep)
>> +{
>> +	return *(pte_t *)ptep;
> 
> Why do you need the casting here?

I don't - good spot. Will fix for v3.

This is some residue from one of the approaches I took to finding all the call
sites, where I globally did s/pte_t */pte_handle_t/ and typedef'ed pte_handle_t
as a void*. Then the compiler would error on any attempted dereferences, but I
had to explicitly cast in the places that could legitimately dereference.

Thanks for the reviews.

> 
>> +}
>> +#endif
>> +
>>  #ifndef __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
>>  static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
>>  					    unsigned long address,
>> -- 
>> 2.25.1
>>
>>
>
  

On 19/05/2023 10:12, Ryan Roberts wrote:
> On 18/05/2023 20:28, Yu Zhao wrote:
>> On Thu, May 18, 2023 at 5:07 AM Ryan Roberts <ryan.roberts@arm.com> wrote:
>>>
>>> There are many call sites that directly dereference a pte_t pointer.
>>> This makes it very difficult to properly encapsulate a page table in the
>>> arch code without having to allocate shadow page tables. ptep_deref()
>>> aims to solve this by replacing all direct dereferences with a call to
>>> this function.
>>>
>>> The default implementation continues to just dereference the pointer
>>> (*ptep), so generated code should be exactly the same. However, it is
>>> possible for the architecture to override the default with their own
>>> implementation, that can (e.g.) hide certain bits from the core code, or
>>> determine young/dirty status by mixing in state from another source.
>>>
>>> While ptep_get() and ptep_get_lockless() already exist, these are
>>> implemented as atomic accesses (e.g. READ_ONCE() in the default case).
>>> So rather than using ptep_get() and risking performance regressions,
>>> introduce an new variant.
>>
>> We should reuse ptep_get():
>> 1. I don't think READ_ONCE() can cause measurable regressions in this case.
>> 2. It's technically wrong without it.
> 
> Can you clarify what you mean by technically wrong? Are you saying that the
> current code that does direct dereferencing is buggy?
> 
> I previously convinced myself that the potential for the compiler generating
> multiple loads was safe because the code in question is under the PTL so there
> are no concurrent stores. And we shouldn't see any tearing for the same reason.
> 
> That said, if there is concensus that we can just use ptep_get() (==
> READ_ONCE()) everywhere, then I agree that would be cleaner. Does anyone object?

Hi all,

A politie bump: It would be great to hear opinions on this before I go ahead and
make the change.

Thanks,
Ryan
  

On Fri, May 19, 2023 at 3:12 AM Ryan Roberts <ryan.roberts@arm.com> wrote:
>
> On 18/05/2023 20:28, Yu Zhao wrote:
> > On Thu, May 18, 2023 at 5:07 AM Ryan Roberts <ryan.roberts@arm.com> wrote:
> >>
> >> There are many call sites that directly dereference a pte_t pointer.
> >> This makes it very difficult to properly encapsulate a page table in the
> >> arch code without having to allocate shadow page tables. ptep_deref()
> >> aims to solve this by replacing all direct dereferences with a call to
> >> this function.
> >>
> >> The default implementation continues to just dereference the pointer
> >> (*ptep), so generated code should be exactly the same. However, it is
> >> possible for the architecture to override the default with their own
> >> implementation, that can (e.g.) hide certain bits from the core code, or
> >> determine young/dirty status by mixing in state from another source.
> >>
> >> While ptep_get() and ptep_get_lockless() already exist, these are
> >> implemented as atomic accesses (e.g. READ_ONCE() in the default case).
> >> So rather than using ptep_get() and risking performance regressions,
> >> introduce an new variant.
> >
> > We should reuse ptep_get():
> > 1. I don't think READ_ONCE() can cause measurable regressions in this case.
> > 2. It's technically wrong without it.
>
> Can you clarify what you mean by technically wrong? Are you saying that the
> current code that does direct dereferencing is buggy?

Sorry for not being clear.

I think we can agree that *ptep is volatile. Not being treated as such
seems a bad idea to me. I don't think it'd cause any real problems --
most warnings KCSAN reported didn't either, but we fixed them anyway.
So should we fix this case as well while we are at it.

> I previously convinced myself that the potential for the compiler generating
> multiple loads was safe because the code in question is under the PTL so there
> are no concurrent stores. And we shouldn't see any tearing for the same reason.
>
> That said, if there is concensus that we can just use ptep_get() (==
> READ_ONCE()) everywhere, then I agree that would be cleaner. Does anyone object?

(No objection to NOT using it either. Just a recommendation, since
it's already there.)