[00/18] x86: new .insn directive

  Especially when instructions which are not known to gas yet also take
register or, yet worse, memory operands, encoding such in code actually
wanting to make use of them is often difficult. Typically people resort
to hard-coding the involved registers, thus being able to express
things via .byte. To overcome this limitation (to a sufficient degree
at least), introduce .insn. This allows users to specify operands in
their "normal" shape (possibly in slightly altered order). Peculiarities
require two small syntax extensions; see the implementation or
documentation for details.

In order to re-use sufficiently much of the functionality md_assemble()
already uses, some adjustments to existing code were necessary. The one
item to call out here is the partial re-write of build_modrm_byte()
(patch 7), which actually turned out to simplify things. Subsequently
possible further tidying is carried out right away (patches 8 and 9),
even if not strictly related to the .insn work.

I'm pretty sure there are still corner cases which aren't taken care of
correctly. It's also quite possible that I've overlooked further places
in pre-existing code which need tweaking for .insn. People taking a
close look and/or playing with the new functionality would be much
appreciated.

The last patch in the series is strictly RFC, as I'm uncertain whether
we actually want this kind of a testcase.

01: introduce .insn directive
02: parse VEX and alike specifiers for .insn
03: parse special opcode modifiers for .insn
04: use set_rex_vrex() also for short-form handling
05: move more disp processing out of md_assemble()
06: adjust REX-prefix part of SSE2AVX test
07: re-work build_modrm_byte()'s register assignment
08: VexVVVV is now merely a boolean
09: drop "shimm" special case template expansions
10: AT&T: restrict recognition of the "absolute branch" prefix character
11: process instruction operands for .insn
12: decouple broadcast type and bytes fields
13: handle EVEX Disp8 for .insn
14: allow for multiple immediates in output_disp()
15: handle immediate operands for .insn
16: document .insn
17: convert testcases to use .insn
18: .insn example - VEX-encoded instructions of original Xeon Phi

Jan
  

On Fri, Mar 3, 2023 at 4:55 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> Especially when instructions which are not known to gas yet also take
> register or, yet worse, memory operands, encoding such in code actually
> wanting to make use of them is often difficult. Typically people resort
> to hard-coding the involved registers, thus being able to express
> things via .byte. To overcome this limitation (to a sufficient degree
> at least), introduce .insn. This allows users to specify operands in
> their "normal" shape (possibly in slightly altered order). Peculiarities
> require two small syntax extensions; see the implementation or
> documentation for details.
>
> In order to re-use sufficiently much of the functionality md_assemble()
> already uses, some adjustments to existing code were necessary. The one
> item to call out here is the partial re-write of build_modrm_byte()
> (patch 7), which actually turned out to simplify things. Subsequently
> possible further tidying is carried out right away (patches 8 and 9),
> even if not strictly related to the .insn work.
>
> I'm pretty sure there are still corner cases which aren't taken care of
> correctly. It's also quite possible that I've overlooked further places
> in pre-existing code which need tweaking for .insn. People taking a
> close look and/or playing with the new functionality would be much
> appreciated.
>
> The last patch in the series is strictly RFC, as I'm uncertain whether
> we actually want this kind of a testcase.
>
> 01: introduce .insn directive
> 02: parse VEX and alike specifiers for .insn
> 03: parse special opcode modifiers for .insn
> 04: use set_rex_vrex() also for short-form handling
> 05: move more disp processing out of md_assemble()
> 06: adjust REX-prefix part of SSE2AVX test
> 07: re-work build_modrm_byte()'s register assignment
> 08: VexVVVV is now merely a boolean
> 09: drop "shimm" special case template expansions
> 10: AT&T: restrict recognition of the "absolute branch" prefix character
> 11: process instruction operands for .insn
> 12: decouple broadcast type and bytes fields
> 13: handle EVEX Disp8 for .insn
> 14: allow for multiple immediates in output_disp()
> 15: handle immediate operands for .insn
> 16: document .insn
> 17: convert testcases to use .insn
> 18: .insn example - VEX-encoded instructions of original Xeon Phi
>
> Jan

X86 encoding scheme is quite complex.  It may get even more complex in
the future.  I suggest we wait for a while so that we can get clear pictures
what the future encoding scheme looks like.
  

> 01: introduce .insn directive
> 02: parse VEX and alike specifiers for .insn
> 03: parse special opcode modifiers for .insn
> 04: use set_rex_vrex() also for short-form handling
> 05: move more disp processing out of md_assemble()
> 06: adjust REX-prefix part of SSE2AVX test
> 07: re-work build_modrm_byte()'s register assignment
> 08: VexVVVV is now merely a boolean
> 09: drop "shimm" special case template expansions
> 10: AT&T: restrict recognition of the "absolute branch" prefix character
> 11: process instruction operands for .insn
> 12: decouple broadcast type and bytes fields
> 13: handle EVEX Disp8 for .insn
> 14: allow for multiple immediates in output_disp()
> 15: handle immediate operands for .insn
> 16: document .insn
> 17: convert testcases to use .insn
> 18: .insn example - VEX-encoded instructions of original Xeon Phi

Hi Jan,

Since this is a complex feature, do you have a branch for those interested
to test so that we could all give back some comments and see if there is
something missing?

I could also do that from scratch but I suppose it is much more easier since
you have a branch.

Thx,
Haochen
  

On 05.03.2023 11:07, Jiang, Haochen wrote:
>> 01: introduce .insn directive
>> 02: parse VEX and alike specifiers for .insn
>> 03: parse special opcode modifiers for .insn
>> 04: use set_rex_vrex() also for short-form handling
>> 05: move more disp processing out of md_assemble()
>> 06: adjust REX-prefix part of SSE2AVX test
>> 07: re-work build_modrm_byte()'s register assignment
>> 08: VexVVVV is now merely a boolean
>> 09: drop "shimm" special case template expansions
>> 10: AT&T: restrict recognition of the "absolute branch" prefix character
>> 11: process instruction operands for .insn
>> 12: decouple broadcast type and bytes fields
>> 13: handle EVEX Disp8 for .insn
>> 14: allow for multiple immediates in output_disp()
>> 15: handle immediate operands for .insn
>> 16: document .insn
>> 17: convert testcases to use .insn
>> 18: .insn example - VEX-encoded instructions of original Xeon Phi
> 
> Since this is a complex feature, do you have a branch for those interested
> to test so that we could all give back some comments and see if there is
> something missing?
> 
> I could also do that from scratch but I suppose it is much more easier since
> you have a branch.

No, sorry, my normal work env continues to be quilt based, and hence I'm not
really maintaining private git branches.

Jan
  

On 03.03.2023 17:50, H.J. Lu wrote:
> X86 encoding scheme is quite complex.  It may get even more complex in
> the future.

Indeed. Also some complexities did exist only transiently, like AMD's
DREX. MVEX is mentioned in the series, and I'm not sure whether to
call this "historic" as well. However, ...

>  I suggest we wait for a while so that we can get clear pictures
> what the future encoding scheme looks like.

... I have to admit that I'm puzzled by this suggestion. The rate at
which new encoding schemes appear is pretty low. So unless you know
of something to see the (public) light of day soon, I wonder what
meaning you assign to "a while". The plan certainly is for this work
to land well in time for 2.40, unless there are clear technical
issues speaking against it. In no event do I plan to wait very long
with committing not directly .insn-related changes in the series,
like patches 06 ... 09 or even 10, and maybe 04 as well as 12.

I shall perhaps also add that I view entirely new encoding schemes
as less of a problem - support for them can be added to .insn
handling incrementally. What could be more problematic are changes
to one of the existing schemes.

Jan
  

On Mon, Mar 6, 2023 at 1:26 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 03.03.2023 17:50, H.J. Lu wrote:
> > X86 encoding scheme is quite complex.  It may get even more complex in
> > the future.
>
> Indeed. Also some complexities did exist only transiently, like AMD's
> DREX. MVEX is mentioned in the series, and I'm not sure whether to
> call this "historic" as well. However, ...
>
> >  I suggest we wait for a while so that we can get clear pictures
> > what the future encoding scheme looks like.
>
> ... I have to admit that I'm puzzled by this suggestion. The rate at
> which new encoding schemes appear is pretty low. So unless you know
> of something to see the (public) light of day soon, I wonder what
> meaning you assign to "a while". The plan certainly is for this work
> to land well in time for 2.40, unless there are clear technical
> issues speaking against it. In no event do I plan to wait very long
> with committing not directly .insn-related changes in the series,
> like patches 06 ... 09 or even 10, and maybe 04 as well as 12.
>
> I shall perhaps also add that I view entirely new encoding schemes
> as less of a problem - support for them can be added to .insn
> handling incrementally. What could be more problematic are changes
> to one of the existing schemes.
>

We will evaluate if these changes will cause any potential issues for
the future.
  

On 07.03.2023 21:33, H.J. Lu wrote:
> On Mon, Mar 6, 2023 at 1:26 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 03.03.2023 17:50, H.J. Lu wrote:
>>> X86 encoding scheme is quite complex.  It may get even more complex in
>>> the future.
>>
>> Indeed. Also some complexities did exist only transiently, like AMD's
>> DREX. MVEX is mentioned in the series, and I'm not sure whether to
>> call this "historic" as well. However, ...
>>
>>>  I suggest we wait for a while so that we can get clear pictures
>>> what the future encoding scheme looks like.
>>
>> ... I have to admit that I'm puzzled by this suggestion. The rate at
>> which new encoding schemes appear is pretty low. So unless you know
>> of something to see the (public) light of day soon, I wonder what
>> meaning you assign to "a while". The plan certainly is for this work
>> to land well in time for 2.40, unless there are clear technical
>> issues speaking against it. In no event do I plan to wait very long
>> with committing not directly .insn-related changes in the series,
>> like patches 06 ... 09 or even 10, and maybe 04 as well as 12.
>>
>> I shall perhaps also add that I view entirely new encoding schemes
>> as less of a problem - support for them can be added to .insn
>> handling incrementally. What could be more problematic are changes
>> to one of the existing schemes.
> 
> We will evaluate if these changes will cause any potential issues for
> the future.

Okay, but please supply me with a time frame for this activity. It's
not reasonable to have me sit on this kind of work (and potentially
have to rebase it repeatedly over other work) for an extended period
of time. Furthermore, should potential issues be found, there'll
need to be enough detail for me to actually address these (rather
than wait until such features would become publicly known).

In the absence of a timeline I'll get v2 out soonish (I've found
that I failed to remember to re-run the testsuite on a broader set
of sub-targets after adding the immediate operands handling and the
conversion of some of the existing testcases; hence some adjustments
were needed there), and then wait like two weeks for comments of any
kind. Should none surface, I think I can rightfully commit the whole
lot then. (As indicated, I may commit a subset ahead of submitting
v2, not the least to limit the volume of the re-submission. But of
course again only unless comments surface which would need
addressing.)

Jan
  

> > We will evaluate if these changes will cause any potential issues for
> > the future.
> 
> Okay, but please supply me with a time frame for this activity. It's
> not reasonable to have me sit on this kind of work (and potentially
> have to rebase it repeatedly over other work) for an extended period
> of time. Furthermore, should potential issues be found, there'll
> need to be enough detail for me to actually address these (rather
> than wait until such features would become publicly known).
> 

We have created a branch from scratch and currently under evaluating.

Lingling will give her further opinion on this. I suppose two weeks might
be enough.

Really appreciate the work and looking forward to see it happen.

Thx,
Haochen

> In the absence of a timeline I'll get v2 out soonish (I've found
> that I failed to remember to re-run the testsuite on a broader set
> of sub-targets after adding the immediate operands handling and the
> conversion of some of the existing testcases; hence some adjustments
> were needed there), and then wait like two weeks for comments of any
> kind. Should none surface, I think I can rightfully commit the whole
> lot then. (As indicated, I may commit a subset ahead of submitting
> v2, not the least to limit the volume of the re-submission. But of
> course again only unless comments surface which would need
> addressing.)
> 
> Jan
  

It is a really good work. After checking and evaluating, I find these changes do not cause any  issues for my side.

Thanks,
Lingling
> -----Original Message-----
> From: Jiang, Haochen <haochen.jiang@intel.com>
> Sent: Wednesday, March 8, 2023 4:10 PM
> To: Beulich, Jan <JBeulich@suse.com>; H.J. Lu <hjl.tools@gmail.com>; Kong,
> Lingling <lingling.kong@intel.com>
> Cc: Binutils <binutils@sourceware.org>
> Subject: RE: [PATCH 00/18] x86: new .insn directive
> 
> > > We will evaluate if these changes will cause any potential issues
> > > for the future.
> >
> > Okay, but please supply me with a time frame for this activity. It's
> > not reasonable to have me sit on this kind of work (and potentially
> > have to rebase it repeatedly over other work) for an extended period
> > of time. Furthermore, should potential issues be found, there'll need
> > to be enough detail for me to actually address these (rather than wait
> > until such features would become publicly known).
> >
> 
> We have created a branch from scratch and currently under evaluating.
> 
> Lingling will give her further opinion on this. I suppose two weeks might be
> enough.
> 
> Really appreciate the work and looking forward to see it happen.
> 
> Thx,
> Haochen
> 
> > In the absence of a timeline I'll get v2 out soonish (I've found that
> > I failed to remember to re-run the testsuite on a broader set of
> > sub-targets after adding the immediate operands handling and the
> > conversion of some of the existing testcases; hence some adjustments
> > were needed there), and then wait like two weeks for comments of any
> > kind. Should none surface, I think I can rightfully commit the whole
> > lot then. (As indicated, I may commit a subset ahead of submitting v2,
> > not the least to limit the volume of the re-submission. But of course
> > again only unless comments surface which would need
> > addressing.)
> >
> > Jan