x86: tighten .insn SAE and broadcast checking

Message ID 73d3228b-1cf8-2bc9-bf50-d14055241b3b@suse.com
State Unresolved
Headers
Series x86: tighten .insn SAE and broadcast checking |

Checks

Context Check Description
snail/binutils-gdb-check warning Git am fail log

Commit Message

Jan Beulich Sept. 25, 2023, 12:37 p.m. UTC
  SAE / embedded rounding are invalid when there's the memory operand, as
the bit encoding this specifies broadcast in that case.

Broadcast needs to be specified on the memory operand.
---
Arguably, until the AVX10 spec spells out how embedded rounding is to be
encoded for 256-bit operations, we may need to further insist on EVEX512
or EVEXLIG to be specified. In practice that would end up overly
limiting though, as can be observed when using .insn to encode APX insns
(where EVEX.brs is overloaded by EVEX.nd).
  

Patch

--- a/gas/config/tc-i386.c
+++ b/gas/config/tc-i386.c
@@ -11538,8 +11538,9 @@  s_insn (int dummy ATTRIBUTE_UNUSED)
 	  ? i.broadcast.type || i.broadcast.bytes
 	    || i.rounding.type != rc_none
 	    || i.mask.reg
-	  : (i.broadcast.type || i.broadcast.bytes)
-	    && i.rounding.type != rc_none))
+	  : (i.mem_operands && i.rounding.type != rc_none)
+	    || ((i.broadcast.type || i.broadcast.bytes)
+		&& !(i.flags[i.broadcast.operand] & Operand_Mem))))
     {
       as_bad (_("conflicting .insn operands"));
       goto done;