[v2,1/2] x86/speculation: Allow enabling STIBP with legacy IBRS

  Setting the IBRS bit implicitly enables STIBP to protect against
cross-thread branch target injection. With enhanced IBRS, the bit it set
once and is not cleared again. However, on CPUs with just legacy IBRS,
IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
STIBP, thus requiring some form of cross-thread protection in userspace.

Enable STIBP, either opt-in via prctl or seccomp, or always on depending
on the choice of mitigation selected via spectre_v2_user.

Reported-by: José Oliveira <joseloliveira11@gmail.com>
Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
Reviewed-by: Alexandra Sandulescu <aesa@google.com>
Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
Cc: stable@vger.kernel.org
Signed-off-by: KP Singh <kpsingh@kernel.org>
---
 arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)
  

On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> Setting the IBRS bit implicitly enables STIBP to protect against
> cross-thread branch target injection. With enhanced IBRS, the bit it set
> once and is not cleared again. However, on CPUs with just legacy IBRS,
> IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> STIBP, thus requiring some form of cross-thread protection in userspace.
> 
> Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> on the choice of mitigation selected via spectre_v2_user.
> 
> Reported-by: José Oliveira <joseloliveira11@gmail.com>
> Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Cc: stable@vger.kernel.org
> Signed-off-by: KP Singh <kpsingh@kernel.org>
> ---
>  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
>  1 file changed, 22 insertions(+), 11 deletions(-)
> 

Hi,

This is the friendly patch-bot of Greg Kroah-Hartman.  You have sent him
a patch that has triggered this response.  He used to manually respond
to these common problems, but in order to save his sanity (he kept
writing the same thing over and over, yet to different people), I was
created.  Hopefully you will not take offence and will fix the problem
in your patch and resubmit it so that it can be accepted into the Linux
kernel tree.

You are receiving this message because of the following common error(s)
as indicated below:

- This looks like a new version of a previously submitted patch, but you
  did not list below the --- line any changes from the previous version.
  Please read the section entitled "The canonical patch format" in the
  kernel file, Documentation/process/submitting-patches.rst for what
  needs to be done here to properly describe this.

If you wish to discuss this problem further, or you have questions about
how to resolve this issue, please feel free to respond to this email and
Greg will reply once he has dug out from the pending patches received
from other developers.

thanks,

greg k-h's patch email bot
  

On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> Setting the IBRS bit implicitly enables STIBP to protect against
> cross-thread branch target injection. With enhanced IBRS, the bit it set
> once and is not cleared again. However, on CPUs with just legacy IBRS,
> IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> STIBP, thus requiring some form of cross-thread protection in userspace.
> 
> Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> on the choice of mitigation selected via spectre_v2_user.
> 
> Reported-by: José Oliveira <joseloliveira11@gmail.com>
> Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Cc: stable@vger.kernel.org
> Signed-off-by: KP Singh <kpsingh@kernel.org>
> ---
>  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
>  1 file changed, 22 insertions(+), 11 deletions(-)

Why isn't patch 2/2 for stable as well?

thanks,

greg k-h
  

On Tue, Feb 21, 2023 at 11:29 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > Setting the IBRS bit implicitly enables STIBP to protect against
> > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > STIBP, thus requiring some form of cross-thread protection in userspace.
> >
> > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > on the choice of mitigation selected via spectre_v2_user.
> >
> > Reported-by: José Oliveira <joseloliveira11@gmail.com>
> > Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> > Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: KP Singh <kpsingh@kernel.org>
> > ---
> >  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> >  1 file changed, 22 insertions(+), 11 deletions(-)
>
> Why isn't patch 2/2 for stable as well?

It should be. I actually forgot to remove stable from the first one as
there are still ongoing discussions and people kept having to "drop
stable".  I can send a v3 with stable Cc'ed. Should it have a fixes
tag too?

>
> thanks,
>
> greg k-h
  

On Tue, Feb 21, 2023 at 11:35:29AM -0800, KP Singh wrote:
> On Tue, Feb 21, 2023 at 11:29 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > > Setting the IBRS bit implicitly enables STIBP to protect against
> > > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > > STIBP, thus requiring some form of cross-thread protection in userspace.
> > >
> > > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > > on the choice of mitigation selected via spectre_v2_user.
> > >
> > > Reported-by: José Oliveira <joseloliveira11@gmail.com>
> > > Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> > > Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> > > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > > Cc: stable@vger.kernel.org
> > > Signed-off-by: KP Singh <kpsingh@kernel.org>
> > > ---
> > >  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> > >  1 file changed, 22 insertions(+), 11 deletions(-)
> >
> > Why isn't patch 2/2 for stable as well?
> 
> It should be. I actually forgot to remove stable from the first one as
> there are still ongoing discussions and people kept having to "drop
> stable".  I can send a v3 with stable Cc'ed. Should it have a fixes
> tag too?

Why does anyone need to "drop stable" from a patch discussion?  That's
not a problem, we _WANT_ to see the patch review and discussion also
copied there to be aware of what is coming down the pipeline.  So
whomever said that is not correct, sorry.

And yes, a fixes: tag would be nice.

thanks,

greg k-h
  

On Tue, Feb 21, 2023 at 08:47:38PM +0100, Greg KH wrote:
> Why does anyone need to "drop stable" from a patch discussion?  That's
> not a problem, we _WANT_ to see the patch review and discussion also
> copied there to be aware of what is coming down the pipeline.  So
> whomever said that is not correct, sorry.

Someone dropped stable because you used to send automated formletter
mails that this is not how one should submit a patch to stable. I guess
that is not needed anymore so I'll stop dropping stable.
  

On Tue, Feb 21, 2023 at 08:57:26PM +0100, Borislav Petkov wrote:
> On Tue, Feb 21, 2023 at 08:47:38PM +0100, Greg KH wrote:
> > Why does anyone need to "drop stable" from a patch discussion?  That's
> > not a problem, we _WANT_ to see the patch review and discussion also
> > copied there to be aware of what is coming down the pipeline.  So
> > whomever said that is not correct, sorry.
> 
> Someone dropped stable because you used to send automated formletter
> mails that this is not how one should submit a patch to stable. I guess
> that is not needed anymore so I'll stop dropping stable.

I still send them, and so does 0-day, IF you send the emails
incorrectly.  So far, that's not been the case for this series at all
(hint, there needs to be a cc: stable in the signed-off-by area of the
patch, that's all.)

thanks,

greg k-h
  

On Tue, Feb 21, 2023 at 09:09:32PM +0100, Greg KH wrote:
> I still send them, and so does 0-day, IF you send the emails
> incorrectly.  So far, that's not been the case for this series at all
> (hint, there needs to be a cc: stable in the signed-off-by area of the
> patch, that's all.)

Aha, you send it when there's no Cc: stable in the SOB area but stable
is still CCed.

Bah, forget what I said.
  

On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> Setting the IBRS bit implicitly enables STIBP to protect against
> cross-thread branch target injection. With enhanced IBRS, the bit it set
> once and is not cleared again. However, on CPUs with just legacy IBRS,
> IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> STIBP, thus requiring some form of cross-thread protection in userspace.
> 
> Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> on the choice of mitigation selected via spectre_v2_user.
> 
> Reported-by: José Oliveira <joseloliveira11@gmail.com>
> Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Cc: stable@vger.kernel.org
> Signed-off-by: KP Singh <kpsingh@kernel.org>
> ---
>  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
>  1 file changed, 22 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index 85168740f76a..5be6075d8e36 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)
>  	return SPECTRE_V2_USER_CMD_AUTO;
>  }
>  
> -static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
>  {
> -	return mode == SPECTRE_V2_IBRS ||
> -	       mode == SPECTRE_V2_EIBRS ||
> +	return mode == SPECTRE_V2_EIBRS ||
>  	       mode == SPECTRE_V2_EIBRS_RETPOLINE ||
>  	       mode == SPECTRE_V2_EIBRS_LFENCE;
>  }
>  
> +static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +{
> +	return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
> +}
> +
> +static inline bool spectre_v2_user_needs_stibp(enum spectre_v2_mitigation mode)
> +{
> +	/*
> +	 * enhanced IBRS also protects against user-mode attacks as the IBRS bit

Maybe:
	 * Enhanced IBRS mode also protects against cross-thread user-to-user
	 * attacks as the IBRS bit

> +	 * remains always set which implicitly enables cross-thread protections.
> +	 * However, In legacy IBRS mode, the IBRS bit is set only in kernel
> +	 * and cleared on return to userspace. This disables the implicit
> +	 * cross-thread protections and STIBP is needed.
> +	 */
> +	return !spectre_v2_in_eibrs_mode(mode);
> +}
> +
>  static void __init
>  spectre_v2_user_select_mitigation(void)
>  {
> @@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void)
>  			"always-on" : "conditional");
>  	}
>  
> -	/*
> -	 * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> -	 * STIBP is not required.
> -	 */
> -	if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> -	    !smt_possible ||
> -	    spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> +	if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible ||
> +	    !spectre_v2_user_needs_stibp(spectre_v2_enabled))

As pointed out in other discussions, it will be great if can get rid of
eIBRS check, and do what the user asked for; or atleast print a warning
about not setting STIBP bit explicitly.

>  		return;
>  
>  	/*
> @@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
>  
>  static char *stibp_state(void)
>  {
> -	if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> +	if (!spectre_v2_user_needs_stibp(spectre_v2_enabled))

Decoupling STIBP and eIBRS will also get rid of this check.

>  		return "";
  

On Tue, Feb 21, 2023 at 7:07 PM Pawan Gupta
<pawan.kumar.gupta@linux.intel.com> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > Setting the IBRS bit implicitly enables STIBP to protect against
> > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > STIBP, thus requiring some form of cross-thread protection in userspace.
> >
> > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > on the choice of mitigation selected via spectre_v2_user.
> >
> > Reported-by: José Oliveira <joseloliveira11@gmail.com>
> > Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> > Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: KP Singh <kpsingh@kernel.org>
> > ---
> >  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> >  1 file changed, 22 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> > index 85168740f76a..5be6075d8e36 100644
> > --- a/arch/x86/kernel/cpu/bugs.c
> > +++ b/arch/x86/kernel/cpu/bugs.c
> > @@ -1124,14 +1124,30 @@ spectre_v2_parse_user_cmdline(void)
> >       return SPECTRE_V2_USER_CMD_AUTO;
> >  }
> >
> > -static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> > +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
> >  {
> > -     return mode == SPECTRE_V2_IBRS ||
> > -            mode == SPECTRE_V2_EIBRS ||
> > +     return mode == SPECTRE_V2_EIBRS ||
> >              mode == SPECTRE_V2_EIBRS_RETPOLINE ||
> >              mode == SPECTRE_V2_EIBRS_LFENCE;
> >  }
> >
> > +static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> > +{
> > +     return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
> > +}
> > +
> > +static inline bool spectre_v2_user_needs_stibp(enum spectre_v2_mitigation mode)
> > +{
> > +     /*
> > +      * enhanced IBRS also protects against user-mode attacks as the IBRS bit
>
> Maybe:
>          * Enhanced IBRS mode also protects against cross-thread user-to-user
>          * attacks as the IBRS bit

updated, thanks!

>
> > +      * remains always set which implicitly enables cross-thread protections.
> > +      * However, In legacy IBRS mode, the IBRS bit is set only in kernel
> > +      * and cleared on return to userspace. This disables the implicit
> > +      * cross-thread protections and STIBP is needed.
> > +      */
> > +     return !spectre_v2_in_eibrs_mode(mode);
> > +}
> > +
> >  static void __init
> >  spectre_v2_user_select_mitigation(void)
> >  {
> > @@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void)
> >                       "always-on" : "conditional");
> >       }
> >
> > -     /*
> > -      * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> > -      * STIBP is not required.
> > -      */
> > -     if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> > -         !smt_possible ||
> > -         spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> > +     if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible ||
> > +         !spectre_v2_user_needs_stibp(spectre_v2_enabled))
>
> As pointed out in other discussions, it will be great if can get rid of
> eIBRS check, and do what the user asked for; or atleast print a warning

I think I will keep it as pr_info as, with eIBRS, the user does not
really need STIBP and the mitigation is still effective.

> about not setting STIBP bit explicitly.

That is a bit more complicated as, for now, the user is not really
exposed to STIBP explicitly yet.

{ "auto", SPECTRE_V2_USER_CMD_AUTO, false },
{ "off", SPECTRE_V2_USER_CMD_NONE, false },
{ "on", SPECTRE_V2_USER_CMD_FORCE, true },
{ "prctl", SPECTRE_V2_USER_CMD_PRCTL, false },
{ "prctl,ibpb", SPECTRE_V2_USER_CMD_PRCTL_IBPB, false },
{ "seccomp", SPECTRE_V2_USER_CMD_SECCOMP, false },
{ "seccomp,ibpb", SPECTRE_V2_USER_CMD_SECCOMP_IBPB, false },

I would prefer to do it as a follow up and fix this bug first.

It's a bit gnarly and I think we really need to think about the
options that are exposed to the user [especially in light of Intel /
AMD subtelties].

With the current patch the userspace is still getting working V2
mitigations on both dimensions time (Process A followed by Process B
where A does BTI on the subsequent B that are flushed via an IBPB) and
space (i.e. cross-thread branch target injection) whenever necessary.




>
> >               return;
> >
> >       /*
> > @@ -2327,7 +2338,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
> >
> >  static char *stibp_state(void)
> >  {
> > -     if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> > +     if (!spectre_v2_user_needs_stibp(spectre_v2_enabled))
>
> Decoupling STIBP and eIBRS will also get rid of this check.
>
> >               return "";
  

On Tue, Feb 21, 2023 at 09:49:57PM -0800, KP Singh wrote:
> > > @@ -1193,13 +1209,8 @@ spectre_v2_user_select_mitigation(void)
> > >                       "always-on" : "conditional");
> > >       }
> > >
> > > -     /*
> > > -      * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> > > -      * STIBP is not required.
> > > -      */
> > > -     if (!boot_cpu_has(X86_FEATURE_STIBP) ||
> > > -         !smt_possible ||
> > > -         spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> > > +     if (!boot_cpu_has(X86_FEATURE_STIBP) || !smt_possible ||
> > > +         !spectre_v2_user_needs_stibp(spectre_v2_enabled))
> >
> > As pointed out in other discussions, it will be great if can get rid of
> > eIBRS check, and do what the user asked for; or atleast print a warning
> 
> I think I will keep it as pr_info as, with eIBRS, the user does not
> really need STIBP and the mitigation is still effective.

Thats fair.

> > about not setting STIBP bit explicitly.
> 
> That is a bit more complicated as, for now, the user is not really
> exposed to STIBP explicitly yet.

> I would prefer to do it as a follow up and fix this bug first.

On a second thought, STIBP bit being explicitly set or not shouldn't
matter as long as user is getting the STIBP protection that it asked
for.

A print may just help catch some bugs sooner than later.

> It's a bit gnarly and I think we really need to think about the
> options that are exposed to the user [especially in light of Intel /
> AMD subtelties].

With AMD's AutoIBRS support landing in mainline, and both (AutoIBRS and
eIBRS) sharing the same =eibrs mitigation mode, those subtelties becomes
more important.

Following up on Andrew's comment in the other thread, I hope AutoIBRS
does not require setting STIBP explicitly?:

  /sigh so we're still talking about 3 different things then.

  1) Intel's legacy IBRS
  2) AMD's regular IBRS
  3) AMD's AutoIBRS

  which all have different relevant behaviours for userspace.  Just so
  it's written out coherently in at least one place...
  [...]
  For any AMD configuration setting STIBP, there must be an IBPB after
  having set STIBP.   Setting STIBP alone does not evict previously
  created shared predictions.  This one can go subtly wrong for anyone
  who assumes that Intel STIBP and AMD STIBP have the same behaviour.
  

On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> Setting the IBRS bit implicitly enables STIBP to protect against
> cross-thread branch target injection. With enhanced IBRS, the bit it set
> once and is not cleared again. However, on CPUs with just legacy IBRS,
> IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> STIBP, thus requiring some form of cross-thread protection in userspace.
> 
> Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> on the choice of mitigation selected via spectre_v2_user.
> 
> Reported-by: José Oliveira <joseloliveira11@gmail.com>
> Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Cc: stable@vger.kernel.org
> Signed-off-by: KP Singh <kpsingh@kernel.org>
> ---
>  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
>  1 file changed, 22 insertions(+), 11 deletions(-)

Below is what I'd like to see. Holler if something's wrong.

It is totally untested ofc.

---
From: KP Singh <kpsingh@kernel.org>
Date: Tue, 21 Feb 2023 19:49:07 +0100
Subject: [PATCH] x86/speculation: Allow enabling STIBP with legacy IBRS
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

When plain IBRS is enabled (not enhanced IBRS), the logic in
spectre_v2_user_select_mitigation() determines that STIBP is not needed.

However, on return to userspace, the IBRS bit is cleared for performance
reasons. That leaves userspace threads vulnerable to cross-thread
predictions influence against which STIBP protects.

Exclude IBRS from the spectre_v2_in_ibrs_mode() check to allow for
enabling STIBP through seccomp/prctl().

  [ bp: Rewrite commit message and massage. ]

Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
Reported-by: José Oliveira <joseloliveira11@gmail.com>
Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
Signed-off-by: KP Singh <kpsingh@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20230221184908.2349578-1-kpsingh@kernel.org
---
 arch/x86/kernel/cpu/bugs.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index cf81848b72f4..9a969ab0e62a 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1133,14 +1133,18 @@ spectre_v2_parse_user_cmdline(void)
 	return SPECTRE_V2_USER_CMD_AUTO;
 }
 
-static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
+static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
 {
-	return mode == SPECTRE_V2_IBRS ||
-	       mode == SPECTRE_V2_EIBRS ||
+	return mode == SPECTRE_V2_EIBRS ||
 	       mode == SPECTRE_V2_EIBRS_RETPOLINE ||
 	       mode == SPECTRE_V2_EIBRS_LFENCE;
 }
 
+static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
+{
+	return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
+}
+
 static void __init
 spectre_v2_user_select_mitigation(void)
 {
@@ -1203,12 +1207,19 @@ spectre_v2_user_select_mitigation(void)
 	}
 
 	/*
-	 * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
-	 * STIBP is not required.
+	 * If no STIBP, enhanced IBRS is enabled, or SMT impossible, STIBP
+	 * is not required.
+	 *
+	 * Enhanced IBRS protects also against user-mode attacks as the IBRS bit
+	 * remains always set which implicitly enables cross-thread protections.
+	 * However, in legacy IBRS mode, the IBRS bit is set only on kernel
+	 * entry and cleared on return to userspace. This disables the implicit
+	 * cross-thread protections so allow for STIBP to be selected in that
+	 * case.
 	 */
 	if (!boot_cpu_has(X86_FEATURE_STIBP) ||
 	    !smt_possible ||
-	    spectre_v2_in_ibrs_mode(spectre_v2_enabled))
+	    spectre_v2_in_eibrs_mode(spectre_v2_enabled))
 		return;
 
 	/*
@@ -2340,7 +2351,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
 
 static char *stibp_state(void)
 {
-	if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
+	if (spectre_v2_in_eibrs_mode(spectre_v2_enabled))
 		return "";
 
 	switch (spectre_v2_user_stibp) {
  

On Tue, Feb 21, 2023 at 09:49:57PM -0800, KP Singh wrote:
> That is a bit more complicated as, for now, the user is not really
> exposed to STIBP explicitly yet.

Remember that we're exposing the normal user to a gazillion switches
already. And not every user has done a PhD in hw vulns like we were
forced to in the last, at least 5, years.

So whatever you do, you should try to make it work automatic - if
possible - and DTRT - i.e., sane defaults.

Every new functionality added to that madness needs a proper
justification.

Thx.
  

From: Borislav Petkov
> Sent: 22 February 2023 12:33
> 
> On Tue, Feb 21, 2023 at 09:49:57PM -0800, KP Singh wrote:
> > That is a bit more complicated as, for now, the user is not really
> > exposed to STIBP explicitly yet.
> 
> Remember that we're exposing the normal user to a gazillion switches
> already. And not every user has done a PhD in hw vulns like we were
> forced to in the last, at least 5, years.

It is also worth explaining the acronyms and what the mitigations
actually do.
There are also the big family where disabling hyperthreading
is probably a better way to avoid the vulnerability.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
  

On Wed, Feb 22, 2023 at 4:24 AM Borislav Petkov <bp@alien8.de> wrote:
>
> On Tue, Feb 21, 2023 at 07:49:07PM +0100, KP Singh wrote:
> > Setting the IBRS bit implicitly enables STIBP to protect against
> > cross-thread branch target injection. With enhanced IBRS, the bit it set
> > once and is not cleared again. However, on CPUs with just legacy IBRS,
> > IBRS bit set on user -> kernel and cleared on kernel -> user (a.k.a
> > KERNEL_IBRS). Clearing this bit also disables the implicitly enabled
> > STIBP, thus requiring some form of cross-thread protection in userspace.
> >
> > Enable STIBP, either opt-in via prctl or seccomp, or always on depending
> > on the choice of mitigation selected via spectre_v2_user.
> >
> > Reported-by: José Oliveira <joseloliveira11@gmail.com>
> > Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> > Reviewed-by: Alexandra Sandulescu <aesa@google.com>
> > Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: KP Singh <kpsingh@kernel.org>
> > ---
> >  arch/x86/kernel/cpu/bugs.c | 33 ++++++++++++++++++++++-----------
> >  1 file changed, 22 insertions(+), 11 deletions(-)
>
> Below is what I'd like to see. Holler if something's wrong.

Thanks for iterating. I think your commit description and rewrite
omits a few key subtleties which I have tried to reinforce in both the
commit log and the comments.

Q: What does STIBP have to do with IBRS?
A: Setting the IBRS bit implicitly enables STIBP / some form of cross
thread protection.

Q: Why does it work with eIBRS?
A: Because we set the IBRS bit once and leave it set when using eIBRS

I think this subtlety should be reinforced in the commit description
and code comments so that we don't get it wrong again. Your commit
does answer this one (thanks!)

Q: Why does it not work with the way the kernel currently implements
legacy IBRS?
A: Because the kernel clears the bit on returning to user space.

>
> It is totally untested ofc.
>
> ---
> From: KP Singh <kpsingh@kernel.org>
> Date: Tue, 21 Feb 2023 19:49:07 +0100
> Subject: [PATCH] x86/speculation: Allow enabling STIBP with legacy IBRS
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> When plain IBRS is enabled (not enhanced IBRS), the logic in
> spectre_v2_user_select_mitigation() determines that STIBP is not needed.
>
> However, on return to userspace, the IBRS bit is cleared for performance
> reasons. That leaves userspace threads vulnerable to cross-thread
> predictions influence against which STIBP protects.
>
> Exclude IBRS from the spectre_v2_in_ibrs_mode() check to allow for
> enabling STIBP through seccomp/prctl().
>
>   [ bp: Rewrite commit message and massage. ]
>
> Fixes: 7c693f54c873 ("x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS")
> Reported-by: José Oliveira <joseloliveira11@gmail.com>
> Reported-by: Rodrigo Branco <rodrigo@kernelhacking.com>
> Signed-off-by: KP Singh <kpsingh@kernel.org>
> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
> Cc: stable@vger.kernel.org
> Link: https://lore.kernel.org/r/20230221184908.2349578-1-kpsingh@kernel.org
> ---
>  arch/x86/kernel/cpu/bugs.c | 25 ++++++++++++++++++-------
>  1 file changed, 18 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index cf81848b72f4..9a969ab0e62a 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -1133,14 +1133,18 @@ spectre_v2_parse_user_cmdline(void)
>         return SPECTRE_V2_USER_CMD_AUTO;
>  }
>
> -static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +static inline bool spectre_v2_in_eibrs_mode(enum spectre_v2_mitigation mode)
>  {
> -       return mode == SPECTRE_V2_IBRS ||
> -              mode == SPECTRE_V2_EIBRS ||
> +       return mode == SPECTRE_V2_EIBRS ||
>                mode == SPECTRE_V2_EIBRS_RETPOLINE ||
>                mode == SPECTRE_V2_EIBRS_LFENCE;
>  }
>
> +static inline bool spectre_v2_in_ibrs_mode(enum spectre_v2_mitigation mode)
> +{
> +       return spectre_v2_in_eibrs_mode(mode) || mode == SPECTRE_V2_IBRS;
> +}
> +
>  static void __init
>  spectre_v2_user_select_mitigation(void)
>  {
> @@ -1203,12 +1207,19 @@ spectre_v2_user_select_mitigation(void)
>         }
>
>         /*
> -        * If no STIBP, IBRS or enhanced IBRS is enabled, or SMT impossible,
> -        * STIBP is not required.
> +        * If no STIBP, enhanced IBRS is enabled, or SMT impossible, STIBP
> +        * is not required.
> +        *
> +        * Enhanced IBRS protects also against user-mode attacks as the IBRS bit
> +        * remains always set which implicitly enables cross-thread protections.
> +        * However, in legacy IBRS mode, the IBRS bit is set only on kernel
> +        * entry and cleared on return to userspace. This disables the implicit
> +        * cross-thread protections so allow for STIBP to be selected in that
> +        * case.
>          */
>         if (!boot_cpu_has(X86_FEATURE_STIBP) ||
>             !smt_possible ||
> -           spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> +           spectre_v2_in_eibrs_mode(spectre_v2_enabled))
>                 return;
>
>         /*
> @@ -2340,7 +2351,7 @@ static ssize_t mmio_stale_data_show_state(char *buf)
>
>  static char *stibp_state(void)
>  {
> -       if (spectre_v2_in_ibrs_mode(spectre_v2_enabled))
> +       if (spectre_v2_in_eibrs_mode(spectre_v2_enabled))

The reason why I refactored this into a separate helper was to
document the subtleties I mentioned above and anchor them to one place
as the function is used in 2 places. But this is a maintainer's
choice, so it's your call :)

I do agree with Pawan that it's worth adding a pr_info about what the
kernel is doing about STIBP.

- KP

>                 return "";
>
>         switch (spectre_v2_user_stibp) {
> --
> 2.35.1
>
> --
> Regards/Gruss,
>     Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette
  

On Wed, Feb 22, 2023 at 09:16:21AM -0800, KP Singh wrote:
> Thanks for iterating. I think your commit description and rewrite
> omits a few key subtleties which I have tried to reinforce in both the
> commit log and the comments.
> 
> Q: What does STIBP have to do with IBRS?
> A: Setting the IBRS bit implicitly enables STIBP / some form of cross
> thread protection.

That belongs in the docs, if you want to explain this properly.

> Q: Why does it work with eIBRS?
> A: Because we set the IBRS bit once and leave it set when using eIBRS

Also docs.

> I think this subtlety should be reinforced in the commit description
> and code comments so that we don't get it wrong again. Your commit
> does answer this one (thanks!)

Commit messages are fine when explaining *why* a change is being done.
What is even finer is when you put a lenghtier explanation in our
documentation so that people can actually find it. Finding text in
commit messages is harder...

> Q: Why does it not work with the way the kernel currently implements
> legacy IBRS?
> A: Because the kernel clears the bit on returning to user space.

From the commit message:

    However, on return to userspace, the IBRS bit is cleared for performance
    reasons. That leaves userspace threads vulnerable to cross-thread
    predictions influence against which STIBP protects.

> The reason why I refactored this into a separate helper was to
> document the subtleties I mentioned above and anchor them to one place
> as the function is used in 2 places. But this is a maintainer's
> choice, so it's your call :)

The less code gets added in that thing, the better. Not yet another
helper pls.
 
> I do agree with Pawan that it's worth adding a pr_info about what the
> kernel is doing about STIBP.

STIBP status gets dumped through stibp_state().
  

On Wed, Feb 22, 2023 at 9:48 AM Borislav Petkov <bp@alien8.de> wrote:
>
> On Wed, Feb 22, 2023 at 09:16:21AM -0800, KP Singh wrote:
> > Thanks for iterating. I think your commit description and rewrite
> > omits a few key subtleties which I have tried to reinforce in both the
> > commit log and the comments.
> >
> > Q: What does STIBP have to do with IBRS?
> > A: Setting the IBRS bit implicitly enables STIBP / some form of cross
> > thread protection.
>
> That belongs in the docs, if you want to explain this properly.
>
> > Q: Why does it work with eIBRS?
> > A: Because we set the IBRS bit once and leave it set when using eIBRS
>
> Also docs.
>
> > I think this subtlety should be reinforced in the commit description
> > and code comments so that we don't get it wrong again. Your commit
> > does answer this one (thanks!)
>
> Commit messages are fine when explaining *why* a change is being done.
> What is even finer is when you put a lenghtier explanation in our
> documentation so that people can actually find it. Finding text in
> commit messages is harder...

Sure, I think the docs do already cover it, but I sort of disagree
with your statement around the commit message. I feel the more context
you can add in the commit message, the better it is. When I am looking
at the change log, it would be helpful to have the information that I
mentioned in the Q&A. Small things like, "eIBRS needs the IBRS bit set
which also enables cross-thread protections" is a very important
context for this patch IMHO. Without this one is just left head
scratching and scrambling to read lengthy docs and processor manuals.

But again, totally your call. Others, feel free to chime in.

>
> > Q: Why does it not work with the way the kernel currently implements
> > legacy IBRS?
> > A: Because the kernel clears the bit on returning to user space.
>
> From the commit message:
>
>     However, on return to userspace, the IBRS bit is cleared for performance
>     reasons. That leaves userspace threads vulnerable to cross-thread
>     predictions influence against which STIBP protects.

This sort of loosely implies that the IBRS bit also enables
cross-thread protections. Can you atleast add this one explicitly?

"Setting the IBRS bit also enables cross thread protections"

>
> > The reason why I refactored this into a separate helper was to
> > document the subtleties I mentioned above and anchor them to one place
> > as the function is used in 2 places. But this is a maintainer's
> > choice, so it's your call :)
>
> The less code gets added in that thing, the better. Not yet another
> helper pls.

Sure, your call.

>
> > I do agree with Pawan that it's worth adding a pr_info about what the
> > kernel is doing about STIBP.
>
> STIBP status gets dumped through stibp_state().

Not at the stage when the kernel decides to drop the STIBP protection
when eIBRS is enabled. If we had this information when we had a
positive POC, it would have been much easier to figure out what's
going on here.

- KP

>
> --
> Regards/Gruss,
>     Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette
  

On Wed, Feb 22, 2023 at 11:41:59AM -0800, KP Singh wrote:
> Sure, I think the docs do already cover it,

I mean *our docs*. The stuff you're adding in your patch 2.

> but I sort of disagree with your statement around the commit message.
> I feel the more context you can add in the commit message, the better
> it is.

That's ofc wrong. And you'll find that out when you do git archeology
and you come across a huuuge wall of text explaining the world and some
more.

No, commit messages should be to the point with a structure similar to
something like this:

1. Prepare the context for the explanation briefly.

2. Explain the problem at hand.

3. "It happens because of <...>"

4. "Fix it by doing X"

5. "(Potentially do Y)."

concentrating on *why* the fix is being done.

> When I am looking at the change log, it would be helpful to have the
> information that I mentioned in the Q&A. Small things like, "eIBRS
> needs the IBRS bit set which also enables cross-thread protections" is
> a very important context for this patch IMHO. Without this one is just
> left head scratching and scrambling to read lengthy docs and processor
> manuals.

Yes, that's why you say in the commit message: "For more details, see
Documentation/admin-guide/hw-vuln/spectre.rst." where:

1. you can explain in a lot more detail

2. put it in place where people can find it *easily*

> This sort of loosely implies that the IBRS bit also enables
> cross-thread protections. Can you atleast add this one explicitly?
> 
> "Setting the IBRS bit also enables cross thread protections"

Ok.

> Not at the stage when the kernel decides to drop the STIBP protection
> when eIBRS is enabled.

We can't dump every possible interaction between the mitigations. It is
a huge mess already. But I'm open to looking at improvements of the
situation *and* documenting stuff as we go.

Thx.
  

On Thu, Feb 23, 2023 at 7:45 AM Borislav Petkov <bp@alien8.de> wrote:
>
> On Wed, Feb 22, 2023 at 11:41:59AM -0800, KP Singh wrote:
> > Sure, I think the docs do already cover it,
>
> I mean *our docs*. The stuff you're adding in your patch 2.
>
> > but I sort of disagree with your statement around the commit message.
> > I feel the more context you can add in the commit message, the better
> > it is.
>
> That's ofc wrong. And you'll find that out when you do git archeology
> and you come across a huuuge wall of text explaining the world and some
> more.
>
> No, commit messages should be to the point with a structure similar to
> something like this:
>
> 1. Prepare the context for the explanation briefly.
>
> 2. Explain the problem at hand.
>
> 3. "It happens because of <...>"
>
> 4. "Fix it by doing X"
>
> 5. "(Potentially do Y)."
>
> concentrating on *why* the fix is being done.
>
> > When I am looking at the change log, it would be helpful to have the
> > information that I mentioned in the Q&A. Small things like, "eIBRS
> > needs the IBRS bit set which also enables cross-thread protections" is
> > a very important context for this patch IMHO. Without this one is just
> > left head scratching and scrambling to read lengthy docs and processor
> > manuals.
>
> Yes, that's why you say in the commit message: "For more details, see
> Documentation/admin-guide/hw-vuln/spectre.rst." where:
>
> 1. you can explain in a lot more detail
>
> 2. put it in place where people can find it *easily*
>
> > This sort of loosely implies that the IBRS bit also enables
> > cross-thread protections. Can you atleast add this one explicitly?
> >
> > "Setting the IBRS bit also enables cross thread protections"
>
> Ok.
>
> > Not at the stage when the kernel decides to drop the STIBP protection
> > when eIBRS is enabled.
>
> We can't dump every possible interaction between the mitigations. It is
> a huge mess already. But I'm open to looking at improvements of the
> situation *and* documenting stuff as we go.
>

Well, we can try our best to print a message when we take a decision
on behalf of the user. As I mentioned, had I got any message that the
kernel was doing this it would have been much easier for me to figure
out why the POC was working. It was tricky without this to figure out
and I am capturing my broad chain of thought here:

1. cross thread training seems to work, this seems to be a bug
2. let me enable cross thread protections with a prctl
3. it still seems to work, did I get a printnk that my prctl did not
succeed or the kernel did something special? - no - why?

I will still go ahead with the patch that was "re-written" for me and
we can add debug information subsequently / as a follow up.

> Thx.
>
> --
> Regards/Gruss,
>     Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette