[RFC,22/32] x86/fred: FRED initialization code

  From: "H. Peter Anvin (Intel)" <hpa@zytor.com>

The code to initialize FRED when it's available and _not_ disabled.

cpu_init_fred_exceptions() is the core function to initialize FRED,
which
  1. Sets up FRED entrypoints for events happening in ring 0 and 3.
  2. Sets up a default stack for event handling.
  3. Sets up dedicated event stacks for DB/NMI/MC/DF, equivalent to
     the IDT IST stacks.
  4. Forces 32-bit system calls to use "int $0x80" only.
  5. Enables FRED and invalidtes IDT.

When the FRED is used, cpu_init_exception_handling() initializes FRED
through calling cpu_init_fred_exceptions(), otherwise it sets up TSS
IST and loads IDT.

As FRED uses the ring 3 FRED entrypoint for SYSCALL and SYSENTER,
it skips setting up SYSCALL/SYSENTER related MSRs, e.g., MSR_LSTAR.

Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Co-developed-by: Xin Li <xin3.li@intel.com>
Signed-off-by: Xin Li <xin3.li@intel.com>
---
 arch/x86/include/asm/fred.h  | 14 +++++++
 arch/x86/include/asm/traps.h |  2 +
 arch/x86/kernel/Makefile     |  1 +
 arch/x86/kernel/cpu/common.c | 74 +++++++++++++++++++++++-------------
 arch/x86/kernel/fred.c       | 73 +++++++++++++++++++++++++++++++++++
 arch/x86/kernel/irqinit.c    |  7 +++-
 arch/x86/kernel/traps.c      | 16 +++++++-
 7 files changed, 157 insertions(+), 30 deletions(-)
 create mode 100644 arch/x86/kernel/fred.c
  

On Mon, Dec 19, 2022 at 10:36:48PM -0800, Xin Li wrote:

> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
> +	       FRED_STKLVL(X86_TRAP_DF,  3));
> +
> +	/* The FRED equivalents to IST stacks... */
> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));

Not quite.. IIRC fred only switches to another stack when the level of
the exception is higher. Specifically, if we trigger #DB while inside
#NMI we will not switch to the #DB stack (since 1 < 2).

Now, as mentioned elsewhere, it all nests a lot saner, but stack
exhaustion is still a thing, given the above, what happens when a #DB
hits an #NMI which tickles a #VE or something?

I don't think we've increased the exception stack size, but perhaps we
should for FRED?
  

On Tue, Dec 20, 2022 at 09:55:31AM +0000, Andrew Cooper wrote:
> On 20/12/2022 9:45 am, Peter Zijlstra wrote:
> > On Mon, Dec 19, 2022 at 10:36:48PM -0800, Xin Li wrote:
> >
> >> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
> >> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
> >> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
> >> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
> >> +	       FRED_STKLVL(X86_TRAP_DF,  3));
> >> +
> >> +	/* The FRED equivalents to IST stacks... */
> >> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
> >> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
> >> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
> > Not quite.. IIRC fred only switches to another stack when the level of
> > the exception is higher. Specifically, if we trigger #DB while inside
> > #NMI we will not switch to the #DB stack (since 1 < 2).
> 
> There needs to be a new stack for #DF, and just possibly one for #MC. 
> NMI and #DB do not need separate stacks under FRED.

True, there is very little need to use additional stacks with FRED.

> > Now, as mentioned elsewhere, it all nests a lot saner, but stack
> > exhaustion is still a thing, given the above, what happens when a #DB
> > hits an #NMI which tickles a #VE or something?
> >
> > I don't think we've increased the exception stack size, but perhaps we
> > should for FRED?
> 
> Not sure if it matters too much - it doesn't seem usefully different to
> IDT delivery.  #DB shouldn't get too deep, and NMI gets properly
> inhibited now.

Both #DB and #NMI can end up in perf, and all that goes quite deep :/
  

> > >> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
> > >> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
> > >> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
> > >> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
> > >> +	       FRED_STKLVL(X86_TRAP_DF,  3));
> > >> +
> > >> +	/* The FRED equivalents to IST stacks... */
> > >> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
> > >> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
> > >> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
> > > Not quite.. IIRC fred only switches to another stack when the level
> > > of the exception is higher. Specifically, if we trigger #DB while
> > > inside #NMI we will not switch to the #DB stack (since 1 < 2).

Yes, current stack level can only grow higher.

> >
> > There needs to be a new stack for #DF, and just possibly one for #MC.
> > NMI and #DB do not need separate stacks under FRED.
> 
> True, there is very little need to use additional stacks with FRED.

Pretty much.

#DB/NMI from a ring 3 context uses CSL 0, and their CSLs increase only
when happening from a ring 0 context.

> 
> > > Now, as mentioned elsewhere, it all nests a lot saner, but stack
> > > exhaustion is still a thing, given the above, what happens when a
> > > #DB hits an #NMI which tickles a #VE or something?
> > >
> > > I don't think we've increased the exception stack size, but perhaps
> > > we should for FRED?
> >
> > Not sure if it matters too much - it doesn't seem usefully different
> > to IDT delivery.  #DB shouldn't get too deep, and NMI gets properly
> > inhibited now.
> 
> Both #DB and #NMI can end up in perf, and all that goes quite deep :/

Can you please elaborate it a bit?
  

On December 20, 2022 9:28:52 PM PST, "Li, Xin3" <xin3.li@intel.com> wrote:
>> > >> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
>> > >> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
>> > >> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
>> > >> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
>> > >> +	       FRED_STKLVL(X86_TRAP_DF,  3));
>> > >> +
>> > >> +	/* The FRED equivalents to IST stacks... */
>> > >> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
>> > >> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
>> > >> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
>> > > Not quite.. IIRC fred only switches to another stack when the level
>> > > of the exception is higher. Specifically, if we trigger #DB while
>> > > inside #NMI we will not switch to the #DB stack (since 1 < 2).
>
>Yes, current stack level can only grow higher.
>
>> >
>> > There needs to be a new stack for #DF, and just possibly one for #MC.
>> > NMI and #DB do not need separate stacks under FRED.
>> 
>> True, there is very little need to use additional stacks with FRED.
>
>Pretty much.
>
>#DB/NMI from a ring 3 context uses CSL 0, and their CSLs increase only
>when happening from a ring 0 context.
>
>> 
>> > > Now, as mentioned elsewhere, it all nests a lot saner, but stack
>> > > exhaustion is still a thing, given the above, what happens when a
>> > > #DB hits an #NMI which tickles a #VE or something?
>> > >
>> > > I don't think we've increased the exception stack size, but perhaps
>> > > we should for FRED?
>> >
>> > Not sure if it matters too much - it doesn't seem usefully different
>> > to IDT delivery.  #DB shouldn't get too deep, and NMI gets properly
>> > inhibited now.
>> 
>> Both #DB and #NMI can end up in perf, and all that goes quite deep :/
>
>Can you please elaborate it a bit?
>

Right, this is one major reason for putting #DB/NMI in a separate stack level.
  

On Tue, Dec 20, 2022 at 09:44:33PM -0800, H. Peter Anvin wrote:

> Right, this is one major reason for putting #DB/NMI in a separate stack level.

But with the IST they each have their own stack, with FRED they'll share
this small stack.
  

On December 22, 2022 5:09:28 AM PST, Peter Zijlstra <peterz@infradead.org> wrote:
>On Tue, Dec 20, 2022 at 09:44:33PM -0800, H. Peter Anvin wrote:
>
>> Right, this is one major reason for putting #DB/NMI in a separate stack level.
>
>But with the IST they each have their own stack, with FRED they'll share
>this small stack.

So make the stack larger if you think we need it. You still end up using less total stack space with FRED.
  

On December 20, 2022 1:55:31 AM PST, Andrew Cooper <Andrew.Cooper3@citrix.com> wrote:
>On 20/12/2022 9:45 am, Peter Zijlstra wrote:
>> On Mon, Dec 19, 2022 at 10:36:48PM -0800, Xin Li wrote:
>>
>>> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
>>> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
>>> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
>>> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
>>> +	       FRED_STKLVL(X86_TRAP_DF,  3));
>>> +
>>> +	/* The FRED equivalents to IST stacks... */
>>> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
>>> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
>>> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
>> Not quite.. IIRC fred only switches to another stack when the level of
>> the exception is higher. Specifically, if we trigger #DB while inside
>> #NMI we will not switch to the #DB stack (since 1 < 2).
>
>There needs to be a new stack for #DF, and just possibly one for #MC. 
>NMI and #DB do not need separate stacks under FRED.
>
>> Now, as mentioned elsewhere, it all nests a lot saner, but stack
>> exhaustion is still a thing, given the above, what happens when a #DB
>> hits an #NMI which tickles a #VE or something?
>>
>> I don't think we've increased the exception stack size, but perhaps we
>> should for FRED?
>
>Not sure if it matters too much - it doesn't seem usefully different to
>IDT delivery.  #DB shouldn't get too deep, and NMI gets properly
>inhibited now.
>
>~Andrew
>

I still don't think you want to take #DB or – especially – NMI on the task stack while in the kernel. In fact, the plan is to get rid of the software irqstack handling, too, but at tglx's request that will be a later changeset (correctness first, then optimization.)