[v2] MIPS: add speculation_barrier support

  speculation_barrier for MIPS needs sync+jr.hb (r2+),
so we implement __speculation_barrier in libgcc, like arm32 does.

gcc/ChangeLog:
	* config/mips/mips-protos.h (mips_emit_speculation_barrier): New
        prototype.
	* config/mips/mips.cc (speculation_barrier_libfunc): New static
        variable.
	(mips_init_libfuncs): Initialize it.
	(mips_emit_speculation_barrier): New function.
	* config/mips/mips.md (speculation_barrier): Call
        mips_emit_speculation_barrier.

libgcc/ChangeLog:
	* config/mips/lib1funcs.S: New file.
	define __speculation_barrier and include mips16.S.
	* config/mips/t-mips: define LIB1ASMSRC as mips/lib1funcs.S.
	define LIB1ASMFUNCS as _speculation_barrier.
	set version info for __speculation_barrier.
	* config/mips/libgcc-mips.ver: New file.
	* config/mips/t-mips16: don't define LIB1ASMSRC as mips16.S is
	included in lib1funcs.S now.
---
 gcc/config/mips/mips-protos.h      |  2 +
 gcc/config/mips/mips.cc            | 13 +++++++
 gcc/config/mips/mips.md            | 12 ++++++
 libgcc/config/mips/lib1funcs.S     | 60 ++++++++++++++++++++++++++++++
 libgcc/config/mips/libgcc-mips.ver | 21 +++++++++++
 libgcc/config/mips/t-mips          |  7 ++++
 libgcc/config/mips/t-mips16        |  3 +-
 7 files changed, 116 insertions(+), 2 deletions(-)
 create mode 100644 libgcc/config/mips/lib1funcs.S
 create mode 100644 libgcc/config/mips/libgcc-mips.ver
  

YunQiang Su <yunqiang.su@cipunited.com> writes:
> speculation_barrier for MIPS needs sync+jr.hb (r2+),
> so we implement __speculation_barrier in libgcc, like arm32 does.

Looks reasonable, but do you have a source for the fallback
pre-r2 handling?  (Thanks for adding that btw, since I realise
it's not your focus here.)

Nit: the copyright for the new files should start with this year,
unless you're copying something significant from an existing file.

Thanks,
Richard

>
> gcc/ChangeLog:
> 	* config/mips/mips-protos.h (mips_emit_speculation_barrier): New
>         prototype.
> 	* config/mips/mips.cc (speculation_barrier_libfunc): New static
>         variable.
> 	(mips_init_libfuncs): Initialize it.
> 	(mips_emit_speculation_barrier): New function.
> 	* config/mips/mips.md (speculation_barrier): Call
>         mips_emit_speculation_barrier.
>
> libgcc/ChangeLog:
> 	* config/mips/lib1funcs.S: New file.
> 	define __speculation_barrier and include mips16.S.
> 	* config/mips/t-mips: define LIB1ASMSRC as mips/lib1funcs.S.
> 	define LIB1ASMFUNCS as _speculation_barrier.
> 	set version info for __speculation_barrier.
> 	* config/mips/libgcc-mips.ver: New file.
> 	* config/mips/t-mips16: don't define LIB1ASMSRC as mips16.S is
> 	included in lib1funcs.S now.
> ---
>  gcc/config/mips/mips-protos.h      |  2 +
>  gcc/config/mips/mips.cc            | 13 +++++++
>  gcc/config/mips/mips.md            | 12 ++++++
>  libgcc/config/mips/lib1funcs.S     | 60 ++++++++++++++++++++++++++++++
>  libgcc/config/mips/libgcc-mips.ver | 21 +++++++++++
>  libgcc/config/mips/t-mips          |  7 ++++
>  libgcc/config/mips/t-mips16        |  3 +-
>  7 files changed, 116 insertions(+), 2 deletions(-)
>  create mode 100644 libgcc/config/mips/lib1funcs.S
>  create mode 100644 libgcc/config/mips/libgcc-mips.ver
>
> diff --git a/gcc/config/mips/mips-protos.h b/gcc/config/mips/mips-protos.h
> index 20483469105..da7902c235b 100644
> --- a/gcc/config/mips/mips-protos.h
> +++ b/gcc/config/mips/mips-protos.h
> @@ -388,4 +388,6 @@ extern void mips_register_frame_header_opt (void);
>  extern void mips_expand_vec_cond_expr (machine_mode, machine_mode, rtx *);
>  extern void mips_expand_vec_cmp_expr (rtx *);
>  
> +extern void mips_emit_speculation_barrier_function (void);
> +
>  #endif /* ! GCC_MIPS_PROTOS_H */
> diff --git a/gcc/config/mips/mips.cc b/gcc/config/mips/mips.cc
> index ca822758b41..139707fda34 100644
> --- a/gcc/config/mips/mips.cc
> +++ b/gcc/config/mips/mips.cc
> @@ -13611,6 +13611,9 @@ mips_autovectorize_vector_modes (vector_modes *modes, bool)
>    return 0;
>  }
>  
> +
> +static GTY(()) rtx speculation_barrier_libfunc;
> +
>  /* Implement TARGET_INIT_LIBFUNCS.  */
>  
>  static void
> @@ -13680,6 +13683,7 @@ mips_init_libfuncs (void)
>        synchronize_libfunc = init_one_libfunc ("__sync_synchronize");
>        init_sync_libfuncs (UNITS_PER_WORD);
>      }
> +  speculation_barrier_libfunc = init_one_libfunc ("__speculation_barrier");
>  }
>  
>  /* Build up a multi-insn sequence that loads label TARGET into $AT.  */
> @@ -19092,6 +19096,15 @@ mips_avoid_hazard (rtx_insn *after, rtx_insn *insn, int *hilo_delay,
>        }
>  }
>  
> +/* Emit a speculation barrier.
> +   JR.HB is needed, so we need to put
> +   speculation_barrier_libfunc in libgcc */
> +void
> +mips_emit_speculation_barrier_function ()
> +{
> +  emit_library_call (speculation_barrier_libfunc, LCT_NORMAL, VOIDmode);
> +}
> +
>  /* A SEQUENCE is breakable iff the branch inside it has a compact form
>     and the target has compact branches.  */
>  
> diff --git a/gcc/config/mips/mips.md b/gcc/config/mips/mips.md
> index ac1d77afc7d..5d04ac566dd 100644
> --- a/gcc/config/mips/mips.md
> +++ b/gcc/config/mips/mips.md
> @@ -160,6 +160,8 @@
>    ;; The `.insn' pseudo-op.
>    UNSPEC_INSN_PSEUDO
>    UNSPEC_JRHB
> +
> +  VUNSPEC_SPECULATION_BARRIER
>  ])
>  
>  (define_constants
> @@ -7455,6 +7457,16 @@
>    mips_expand_conditional_move (operands);
>    DONE;
>  })
> +
> +(define_expand "speculation_barrier"
> +  [(unspec_volatile [(const_int 0)] VUNSPEC_SPECULATION_BARRIER)]
> +  ""
> +  "
> +  mips_emit_speculation_barrier_function ();
> +  DONE;
> +  "
> +)
> +
>  
>  ;;
>  ;;  ....................
> diff --git a/libgcc/config/mips/lib1funcs.S b/libgcc/config/mips/lib1funcs.S
> new file mode 100644
> index 00000000000..45d74e2e762
> --- /dev/null
> +++ b/libgcc/config/mips/lib1funcs.S
> @@ -0,0 +1,60 @@
> +/* Copyright (C) 1995-2023 Free Software Foundation, Inc.
> +
> +This file is free software; you can redistribute it and/or modify it
> +under the terms of the GNU General Public License as published by the
> +Free Software Foundation; either version 3, or (at your option) any
> +later version.
> +
> +This file is distributed in the hope that it will be useful, but
> +WITHOUT ANY WARRANTY; without even the implied warranty of
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +General Public License for more details.
> +
> +Under Section 7 of GPL version 3, you are granted additional
> +permissions described in the GCC Runtime Library Exception, version
> +3.1, as published by the Free Software Foundation.
> +
> +You should have received a copy of the GNU General Public License and
> +a copy of the GCC Runtime Library Exception along with this program;
> +see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
> +<http://www.gnu.org/licenses/>.  */
> +
> +#include "mips16.S"
> +
> +#ifdef L_speculation_barrier
> +
> +/* MIPS16e1 has no sync/jr.hb instructions, and MIPS16e2 lacks of jr.hb.
> +   So, we use normal MIPS code here, just like what we do for __sync_* */
> +	.set nomips16
> +
> +	.set noreorder
> +	.globl	__speculation_barrier
> +	.ent	__speculation_barrier
> +
> +__speculation_barrier:
> +	.set	push
> +/* MIPS1 has no sync, and in fact it doesn't need it at all.
> +   We wish that all newer CPUs should run software with MIPS2+ */
> +#if __mips >= 2
> +	sync /* complementation barrier for memory */
> +#endif
> +#if __mips_isa_rev >= 2
> +/* MIPSr2+: sync+jr.hb is enough */
> +	jr.hb	$ra /* Jump with instruction hazard barrier */
> +#else
> +/* Make ssnop available, ssnop only recognized by GAS since mips32,
> +   however it's actually available since R5500,
> +   and it will be decoded as nop on earlier processors */
> +	.set mips32
> +/* MIPS1 to MIPSr1: R10000 have 7 stage pipeline,
> +   so 8 ssnop is sufficient to block all speculation on all CPUs */
> +	.rept 8
> +	ssnop
> +	.endr
> +	jr	$ra
> +#endif
> +	.set	pop
> +	.end	__speculation_barrier
> +
> +	.set reorder
> +#endif
> diff --git a/libgcc/config/mips/libgcc-mips.ver b/libgcc/config/mips/libgcc-mips.ver
> new file mode 100644
> index 00000000000..f0e9fc54965
> --- /dev/null
> +++ b/libgcc/config/mips/libgcc-mips.ver
> @@ -0,0 +1,21 @@
> +# Copyright (C) 2008-2023 Free Software Foundation, Inc.
> +#
> +# This file is part of GCC.
> +#
> +# GCC is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 3, or (at your option)
> +# any later version.
> +#
> +# GCC is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with GCC; see the file COPYING3.  If not see
> +# <http://www.gnu.org/licenses/>.
> +
> +GCC_14.0 {
> +  __speculation_barrier
> +}
> diff --git a/libgcc/config/mips/t-mips b/libgcc/config/mips/t-mips
> index 4fb8e136217..d05ef7cbf74 100644
> --- a/libgcc/config/mips/t-mips
> +++ b/libgcc/config/mips/t-mips
> @@ -7,3 +7,10 @@ softfp_truncations :=
>  softfp_exclude_libgcc2 := n
>  
>  LIB2ADD_ST += $(srcdir)/config/mips/lib2funcs.c
> +
> +
> +LIB1ASMSRC = mips/lib1funcs.S
> +LIB1ASMFUNCS = _speculation_barrier
> +
> +# Version these symbols if building libgcc.so.
> +SHLIB_MAPFILES += $(srcdir)/config/mips/libgcc-mips.ver
> diff --git a/libgcc/config/mips/t-mips16 b/libgcc/config/mips/t-mips16
> index 2bad5119d51..5fd9d60d7a3 100644
> --- a/libgcc/config/mips/t-mips16
> +++ b/libgcc/config/mips/t-mips16
> @@ -16,8 +16,7 @@
>  # along with GCC; see the file COPYING3.  If not see
>  # <http://www.gnu.org/licenses/>.
>  
> -LIB1ASMSRC = mips/mips16.S
> -LIB1ASMFUNCS = _m16addsf3 _m16subsf3 _m16mulsf3 _m16divsf3 \
> +LIB1ASMFUNCS += _m16addsf3 _m16subsf3 _m16mulsf3 _m16divsf3 \
>  	_m16eqsf2 _m16nesf2 _m16gtsf2 _m16gesf2 _m16lesf2 _m16ltsf2 \
>  	_m16unordsf2 \
>  	_m16fltsisf _m16fix_truncsfsi _m16fltunsisf \
  

On Wed, 3 May 2023, Richard Sandiford wrote:

> > speculation_barrier for MIPS needs sync+jr.hb (r2+),
> > so we implement __speculation_barrier in libgcc, like arm32 does.
> 
> Looks reasonable, but do you have a source for the fallback
> pre-r2 handling?  (Thanks for adding that btw, since I realise
> it's not your focus here.)

 Seconded WRT legacy MIPS support, really appreciated.

 I think there may be no authoritative source of information here, this is 
a grey area.  The longest SSNOP sequences I have seen were for the various 
Broadcom implementations and counted 7 instructions.  Both the Linux 
kernel and the CFE firmware has them.

 Also we may not be able to fully enforce ordering for the oldest devices 
that do not implement SYNC, as this is system-specific, e.g. involving 
branching on the CP0 condition with the BC0F instruction, and inventing an 
OS interface for that seems unreasonable at this point of history.

 OTOH Linux emulates any traps on SYNC and even though the handler does 
nothing beyond decoding the instruction the exception handling overhead 
ought to make all the effects from before the exception invocation fully 
visible after its completion.  So for Linux targets I think we ought to 
just emit SYNC unconditionally; glibc has already relied on it for years 
now.

 Overall, oh well, I guess we'll have to live with the limitations.

> > diff --git a/libgcc/config/mips/lib1funcs.S b/libgcc/config/mips/lib1funcs.S
> > new file mode 100644
> > index 00000000000..45d74e2e762
> > --- /dev/null
> > +++ b/libgcc/config/mips/lib1funcs.S
[...]
> > +/* Make ssnop available, ssnop only recognized by GAS since mips32,
> > +   however it's actually available since R5500,
> > +   and it will be decoded as nop on earlier processors */
> > +	.set mips32

 I'm fairly sure `.set mips32' will break with NewABI compilations and may 
cause issues if a higher ISA level with some ASEs enabled has been chosen 
by the user building GCC.  But it's not needed either, as contrary to the 
comment SSNOP is universally accepted by GAS, as it should:

{"ssnop",		"",		0x00000040, 0xffffffff, 0,              	INSN2_ALIAS,	I1,		0,	0 }, /* sll */

Very old versions of GAS indeed required MIPS32+ for SSNOP, but the bug 
was fixed back in 2010, landing in binutils 2.21.  I wouldn't bother 
striving to support such old versions of binutils, i.e. either upgrade or 
patch locally (it might then be worth mentioning in `gcc/doc/install.texi' 
though).

 As to producing SYNC for MIPS I, glibc has this:

#if _MIPS_SIM == _ABIO32 && __mips < 2
#define MIPS_PUSH_MIPS2 ".set	mips2\n\t"
#else
#define MIPS_PUSH_MIPS2
#endif

for the ISA override, so I guess we could take a similar approach.  It 
should be safe, as it doesn't change the ISA unnecessarily.

  Maciej
  

> 2023年5月3日 22:04，Maciej W. Rozycki <macro@orcam.me.uk> 写道：
> 
> On Wed, 3 May 2023, Richard Sandiford wrote:
> 
>>> speculation_barrier for MIPS needs sync+jr.hb (r2+),
>>> so we implement __speculation_barrier in libgcc, like arm32 does.
>> 
>> Looks reasonable, but do you have a source for the fallback
>> pre-r2 handling?  (Thanks for adding that btw, since I realise
>> it's not your focus here.)
> 
> Seconded WRT legacy MIPS support, really appreciated.

Hi all,

Just my two cents on legacy MIPS support.

Since it’s possible to run R2- binary on R2+ processor, we’d better find a
semantic that do eliminate speculation on all processors. While SSNOPs
on R2+ processors is pretty much undefined, there is no guarantee that
SSNOP sequence can eliminate speculation.

My proposal is for R2- CPUs we can do a dummy syscall to act as instruction
hazard barrier, since exception must clear the pipeline this should be true
for all known implementations.

The most lightweight syscall I know is to do a MIPS_ATOMIC_SET with
sysmips. A dummy variable on stack should do the track. Do let me know if there
is a better option.

I have a vague memory about a discussion finding that exception does not indicate
a memory barrier, so perhaps we still need a sync preceding to that syscall.


> 
> I think there may be no authoritative source of information here, this is 
> a grey area.  The longest SSNOP sequences I have seen were for the various 
> Broadcom implementations and counted 7 instructions.  Both the Linux 
> kernel and the CFE firmware has them.

Was it for SiByte or BMIPS?

> 
> Also we may not be able to fully enforce ordering for the oldest devices 
> that do not implement SYNC, as this is system-specific, e.g. involving 
> branching on the CP0 condition with the BC0F instruction, and inventing an 
> OS interface for that seems unreasonable at this point of history.

I guess this is not a valid concern for user space applications?
As per R4000 manual BC0F will issue “Coprocessor unusable exception”
exception and it’s certain that we have Staus.CU0 = 0 in user space.

BC0F is not mentioned in MIPS-IV and later MIPS specs.

Thanks
Jiaxun

> 
> OTOH Linux emulates any traps on SYNC and even though the handler does 
> nothing beyond decoding the instruction the exception handling overhead 
> ought to make all the effects from before the exception invocation fully 
> visible after its completion.  So for Linux targets I think we ought to 
> just emit SYNC unconditionally; glibc has already relied on it for years 
> now.
> 
> Overall, oh well, I guess we'll have to live with the limitations.
> 
>  Maciej
  

On Wed, 3 May 2023, Jiaxun Yang wrote:

> Since it’s possible to run R2- binary on R2+ processor, we’d better find a
> semantic that do eliminate speculation on all processors. While SSNOPs
> on R2+ processors is pretty much undefined, there is no guarantee that
> SSNOP sequence can eliminate speculation.

 Not exactly undefined on R2+, SSNOP is still required to single-issue, so 
it does act as an execution barrier.  Good point otherwise.

 Both EHB and J[AL]R.HB are backwards compatible however (except for an 
obscure 4Kc J[AL]R.HB erratum I came across once and which may be no 
longer relevant), so I think the legacy sequence ought to just return via 
JR.HB as well, therefore providing the required semantics with newer 
hardware.  If it does trap for 4Kc, then the OS can emulate it (and we can 
ignore it for bare metal, deferring to whoever might be interested for a 
workaround).

> My proposal is for R2- CPUs we can do a dummy syscall to act as instruction
> hazard barrier, since exception must clear the pipeline this should be true
> for all known implementations.

 I think the SSNOP approach should be sufficient.

> The most lightweight syscall I know is to do a MIPS_ATOMIC_SET with
> sysmips. A dummy variable on stack should do the track. Do let me know if there
> is a better option.

 That would have to be gettimeofday(2) then, the most performance-critical 
one, and also one that does not have side effects.  The real syscall and 
not VSDO emulation of course (there's a reason it's emulated via VSDO, 
which is exactly our reason too).

> I have a vague memory about a discussion finding that exception does not indicate
> a memory barrier, so perhaps we still need a sync preceding to that syscall.

 There is no claim that I could find in the architecture specification 
saying that taking an exception implies a memory barrier and therefore we 
must conclude it does not.  Likewise executing ERET.

 As I say I think the SSNOP approach should be sufficient, along with 
relying on SYNC emulation.

> > I think there may be no authoritative source of information here, this is 
> > a grey area.  The longest SSNOP sequences I have seen were for the various 
> > Broadcom implementations and counted 7 instructions.  Both the Linux 
> > kernel and the CFE firmware has them.
> 
> Was it for SiByte or BMIPS?

 Both AFAICT.

> > Also we may not be able to fully enforce ordering for the oldest devices 
> > that do not implement SYNC, as this is system-specific, e.g. involving 
> > branching on the CP0 condition with the BC0F instruction, and inventing an 
> > OS interface for that seems unreasonable at this point of history.
> 
> I guess this is not a valid concern for user space applications?
> As per R4000 manual BC0F will issue “Coprocessor unusable exception”
> exception and it’s certain that we have Staus.CU0 = 0 in user space.

 Exactly, which is why an OS service would have to provide the required 
semantics to the userland, and none might be available.  And we probably 
do not care anyway, because I gather this is a security feature to prevent 
certain types of data leaks via a side channel.  I wouldn't expect anyone 
doing any serious security-sensitive processing with legacy MIPS hardware.  
And then there's no speculative execution with all these pieces of legacy 
hardware (R3000, eh?) that have no suitable barriers provided, mostly 
because they are not relevant for them anyway.

 For bare metal we probably do not care about such legacy hardware either 
way.

 Overall I'd say let's do the best we can without bending backwards and 
then rely on people's common sense.

  Maciej
  

> 2023年5月7日 18:34，Maciej W. Rozycki <macro@orcam.me.uk> 写道：
> 
> On Wed, 3 May 2023, Jiaxun Yang wrote:
> 
>> Since it’s possible to run R2- binary on R2+ processor, we’d better find a
>> semantic that do eliminate speculation on all processors. While SSNOPs
>> on R2+ processors is pretty much undefined, there is no guarantee that
>> SSNOP sequence can eliminate speculation.
> 
> Not exactly undefined on R2+, SSNOP is still required to single-issue, so 
> it does act as an execution barrier.  Good point otherwise.
> 
> Both EHB and J[AL]R.HB are backwards compatible however (except for an 
> obscure 4Kc J[AL]R.HB erratum I came across once and which may be no 
> longer relevant), so I think the legacy sequence ought to just return via 
> JR.HB as well, therefore providing the required semantics with newer 
> hardware.  If it does trap for 4Kc, then the OS can emulate it (and we can 
> ignore it for bare metal, deferring to whoever might be interested for a 
> workaround).

Hmm, I just checked MIPS-IV manual, it seems like HB bit (bit 10) is defined as
zero for both JR and JALR.

Is it actually omitted in implementation?

Thanks
Jiaxun
  

On Sun, 7 May 2023, Jiaxun Yang wrote:

> > Both EHB and J[AL]R.HB are backwards compatible however (except for an 
> > obscure 4Kc J[AL]R.HB erratum I came across once and which may be no 
> > longer relevant), so I think the legacy sequence ought to just return via 
> > JR.HB as well, therefore providing the required semantics with newer 
> > hardware.  If it does trap for 4Kc, then the OS can emulate it (and we can 
> > ignore it for bare metal, deferring to whoever might be interested for a 
> > workaround).
> 
> Hmm, I just checked MIPS-IV manual, it seems like HB bit (bit 10) is defined as
> zero for both JR and JALR.
> 
> Is it actually omitted in implementation?

 What has become the hint field was supposed not to be decoded in earlier 
silicon, which is why such encodings were chosen for JR.HB and JALR.HB.  
I was told it was actually verified across existing silicon at the time 
the instructions were being defined with MIPSr1 (i.e. when the hint field 
was added to the ISA).  The issue with the 4Kc was an unfortunate mistake.

 Many instructions are not fully decoded causing encoding aliases, MFHI, 
etc. comes to mind.

  Maciej