[v12,08/22] x86/virt/tdx: Get information about TDX module and TDX-capable memory

  Start to transit out the "multi-steps" to initialize the TDX module.

TDX provides increased levels of memory confidentiality and integrity.
This requires special hardware support for features like memory
encryption and storage of memory integrity checksums.  Not all memory
satisfies these requirements.

As a result, TDX introduced the concept of a "Convertible Memory Region"
(CMR).  During boot, the firmware builds a list of all of the memory
ranges which can provide the TDX security guarantees.

CMRs tell the kernel which memory is TDX compatible.  The kernel takes
CMRs (plus a little more metadata) and constructs "TD Memory Regions"
(TDMRs).  TDMRs let the kernel grant TDX protections to some or all of
the CMR areas.

The TDX module also reports necessary information to let the kernel
build TDMRs and run TDX guests in structure 'tdsysinfo_struct'.  The
list of CMRs, along with the TDX module information, is available to
the kernel by querying the TDX module.

As a preparation to construct TDMRs, get the TDX module information and
the list of CMRs.  Print out CMRs to help user to decode which memory
regions are TDX convertible.

The 'tdsysinfo_struct' is fairly large (1024 bytes) and contains a lot
of info about the TDX module.  Fully define the entire structure, but
only use the fields necessary to build the TDMRs and pr_info() some
basics about the module.  The rest of the fields will get used by KVM.

Signed-off-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
---

v11 -> v12:
 - Changed to use dynamic allocation for TDSYSINFO_STRUCT and CMR array
   (Kirill).
 - Keep SEAMCALL leaf macro definitions in order (Kirill)
 - Removed is_cmr_empty() but open code directly (David)
 - 'atribute' -> 'attribute' (David)

v10 -> v11:
 - No change.

v9 -> v10:
 - Added back "start to transit out..." as now per-cpu init has been
   moved out from tdx_enable().

v8 -> v9:
 - Removed "start to trransit out ..." part in changelog since this patch
   is no longer the first step anymore.
 - Changed to declare 'tdsysinfo' and 'cmr_array' as local static, and
   changed changelog accordingly (Dave).
 - Improved changelog to explain why to declare  'tdsysinfo_struct' in
   full but only use a few members of them (Dave).

v7 -> v8: (Dave)
 - Improved changelog to tell this is the first patch to transit out the
   "multi-steps" init_tdx_module().
 - Removed all CMR check/trim code but to depend on later SEAMCALL.
 - Variable 'vertical alignment' in print TDX module information.
 - Added DECLARE_PADDED_STRUCT() for padded structure.
 - Made tdx_sysinfo and tdx_cmr_array[] to be function local variable
   (and rename them accordingly), and added -Wframe-larger-than=4096 flag
   to silence the build warning.

v6 -> v7:
 - Simplified the check of CMRs due to the fact that TDX actually
   verifies CMRs (that are passed by the BIOS) before enabling TDX.
 - Changed the function name from check_cmrs() -> trim_empty_cmrs().
 - Added CMR page aligned check so that later patch can just get the PFN
   using ">> PAGE_SHIFT".

v5 -> v6:
 - Added to also print TDX module's attribute (Isaku).
 - Removed all arguments in tdx_gete_sysinfo() to use static variables
   of 'tdx_sysinfo' and 'tdx_cmr_array' directly as they are all used
   directly in other functions in later patches.
 - Added Isaku's Reviewed-by.

- v3 -> v5 (no feedback on v4):
 - Renamed sanitize_cmrs() to check_cmrs().
 - Removed unnecessary sanity check against tdx_sysinfo and tdx_cmr_array
   actual size returned by TDH.SYS.INFO.
 - Changed -EFAULT to -EINVAL in couple places.
 - Added comments around tdx_sysinfo and tdx_cmr_array saying they are
   used by TDH.SYS.INFO ABI.
 - Changed to pass 'tdx_sysinfo' and 'tdx_cmr_array' as function
   arguments in tdx_get_sysinfo().
 - Changed to only print BIOS-CMR when check_cmrs() fails.


---
 arch/x86/virt/vmx/tdx/tdx.c | 79 ++++++++++++++++++++++++++++++++++++-
 arch/x86/virt/vmx/tdx/tdx.h | 60 ++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+), 2 deletions(-)
  

On Tue, Jun 27, 2023 at 02:12:38AM +1200, Kai Huang wrote:
>  static int init_tdx_module(void)
>  {
> +	struct tdsysinfo_struct *sysinfo;
> +	struct cmr_info *cmr_array;
> +	int ret;
> +
> +	/*
> +	 * Get the TDSYSINFO_STRUCT and CMRs from the TDX module.
> +	 *
> +	 * The buffers of the TDSYSINFO_STRUCT and the CMR array passed
> +	 * to the TDX module must be 1024-bytes and 512-bytes aligned
> +	 * respectively.  Allocate one page to accommodate them both and
> +	 * also meet those alignment requirements.
> +	 */
> +	sysinfo = (struct tdsysinfo_struct *)__get_free_page(GFP_KERNEL);
> +	if (!sysinfo)
> +		return -ENOMEM;
> +	cmr_array = (struct cmr_info *)((unsigned long)sysinfo + PAGE_SIZE / 2);
> +
> +	BUILD_BUG_ON(PAGE_SIZE / 2 < TDSYSINFO_STRUCT_SIZE);
> +	BUILD_BUG_ON(PAGE_SIZE / 2 < sizeof(struct cmr_info) * MAX_CMRS);

This works, but why not just use slab for this? kmalloc has 512 and 1024
pools already and you won't waste memory for rounding up.

Something like this:

        sysinfo = kmalloc(TDSYSINFO_STRUCT_SIZE, GFP_KERNEL);
        if (!sysinfo)
                return -ENOMEM;

        cmr_array_size = sizeof(struct cmr_info) * MAX_CMRS;

        /* CMR array has to be 512-aligned */
        cmr_array_size = round_up(cmr_array_size, 512);

        cmr_array = kmalloc(cmr_array_size, GFP_KERNEL);
        if (!cmr_array) {
                kfree(sysinfo);
                return -ENOMEM;
        }

?
  

On Tue, 2023-06-27 at 12:51 +0300, kirill.shutemov@linux.intel.com wrote:
> On Tue, Jun 27, 2023 at 02:12:38AM +1200, Kai Huang wrote:
> >  static int init_tdx_module(void)
> >  {
> > +	struct tdsysinfo_struct *sysinfo;
> > +	struct cmr_info *cmr_array;
> > +	int ret;
> > +
> > +	/*
> > +	 * Get the TDSYSINFO_STRUCT and CMRs from the TDX module.
> > +	 *
> > +	 * The buffers of the TDSYSINFO_STRUCT and the CMR array passed
> > +	 * to the TDX module must be 1024-bytes and 512-bytes aligned
> > +	 * respectively.  Allocate one page to accommodate them both and
> > +	 * also meet those alignment requirements.
> > +	 */
> > +	sysinfo = (struct tdsysinfo_struct *)__get_free_page(GFP_KERNEL);
> > +	if (!sysinfo)
> > +		return -ENOMEM;
> > +	cmr_array = (struct cmr_info *)((unsigned long)sysinfo + PAGE_SIZE / 2);
> > +
> > +	BUILD_BUG_ON(PAGE_SIZE / 2 < TDSYSINFO_STRUCT_SIZE);
> > +	BUILD_BUG_ON(PAGE_SIZE / 2 < sizeof(struct cmr_info) * MAX_CMRS);
> 
> This works, but why not just use slab for this? kmalloc has 512 and 1024
> pools already and you won't waste memory for rounding up.
> 
> Something like this:
> 
>         sysinfo = kmalloc(TDSYSINFO_STRUCT_SIZE, GFP_KERNEL);
>         if (!sysinfo)
>                 return -ENOMEM;
> 
>         cmr_array_size = sizeof(struct cmr_info) * MAX_CMRS;
> 
>         /* CMR array has to be 512-aligned */
>         cmr_array_size = round_up(cmr_array_size, 512);

Should we define a macro for 512

	+#define CMR_INFO_ARRAY_ALIGNMENT	512

And get rid of this comment?  AFAICT Dave didn't like such comment mentioning
512-bytes aligned if we have a macro for that.

> 
>         cmr_array = kmalloc(cmr_array_size, GFP_KERNEL);
>         if (!cmr_array) {
>                 kfree(sysinfo);
>                 return -ENOMEM;
>         }
> 
> ?
> 

I confess the reason I used __get_free_page() was to avoid having to allocate
twice, and in case of failure, I need to handle additional memory free.  But I
can do if you think it's clearer?

I wouldn't worry about wasting memory.  The buffer is freed anyway for now. 
Long-termly it's just 4K.
  

On Tue, Jun 27, 2023 at 10:45:33AM +0000, Huang, Kai wrote:
> On Tue, 2023-06-27 at 12:51 +0300, kirill.shutemov@linux.intel.com wrote:
> > On Tue, Jun 27, 2023 at 02:12:38AM +1200, Kai Huang wrote:
> > >  static int init_tdx_module(void)
> > >  {
> > > +	struct tdsysinfo_struct *sysinfo;
> > > +	struct cmr_info *cmr_array;
> > > +	int ret;
> > > +
> > > +	/*
> > > +	 * Get the TDSYSINFO_STRUCT and CMRs from the TDX module.
> > > +	 *
> > > +	 * The buffers of the TDSYSINFO_STRUCT and the CMR array passed
> > > +	 * to the TDX module must be 1024-bytes and 512-bytes aligned
> > > +	 * respectively.  Allocate one page to accommodate them both and
> > > +	 * also meet those alignment requirements.
> > > +	 */
> > > +	sysinfo = (struct tdsysinfo_struct *)__get_free_page(GFP_KERNEL);
> > > +	if (!sysinfo)
> > > +		return -ENOMEM;
> > > +	cmr_array = (struct cmr_info *)((unsigned long)sysinfo + PAGE_SIZE / 2);
> > > +
> > > +	BUILD_BUG_ON(PAGE_SIZE / 2 < TDSYSINFO_STRUCT_SIZE);
> > > +	BUILD_BUG_ON(PAGE_SIZE / 2 < sizeof(struct cmr_info) * MAX_CMRS);
> > 
> > This works, but why not just use slab for this? kmalloc has 512 and 1024
> > pools already and you won't waste memory for rounding up.
> > 
> > Something like this:
> > 
> >         sysinfo = kmalloc(TDSYSINFO_STRUCT_SIZE, GFP_KERNEL);
> >         if (!sysinfo)
> >                 return -ENOMEM;
> > 
> >         cmr_array_size = sizeof(struct cmr_info) * MAX_CMRS;
> > 
> >         /* CMR array has to be 512-aligned */
> >         cmr_array_size = round_up(cmr_array_size, 512);
> 
> Should we define a macro for 512
> 
> 	+#define CMR_INFO_ARRAY_ALIGNMENT	512
> 
> And get rid of this comment?  AFAICT Dave didn't like such comment mentioning
> 512-bytes aligned if we have a macro for that.

Good idea.

> >         cmr_array = kmalloc(cmr_array_size, GFP_KERNEL);
> >         if (!cmr_array) {
> >                 kfree(sysinfo);
> >                 return -ENOMEM;
> >         }
> > 
> > ?
> > 
> 
> I confess the reason I used __get_free_page() was to avoid having to allocate
> twice, and in case of failure, I need to handle additional memory free.  But I
> can do if you think it's clearer?

Less trickery is always cleaner. Especially if the trick is not justified.

> I wouldn't worry about wasting memory.  The buffer is freed anyway for now. 
> Long-termly it's just 4K.
  

On Tue, 2023-06-27 at 14:37 +0300, kirill.shutemov@linux.intel.com wrote:
> On Tue, Jun 27, 2023 at 10:45:33AM +0000, Huang, Kai wrote:
> > On Tue, 2023-06-27 at 12:51 +0300, kirill.shutemov@linux.intel.com wrote:
> > > On Tue, Jun 27, 2023 at 02:12:38AM +1200, Kai Huang wrote:
> > > >  static int init_tdx_module(void)
> > > >  {
> > > > +	struct tdsysinfo_struct *sysinfo;
> > > > +	struct cmr_info *cmr_array;
> > > > +	int ret;
> > > > +
> > > > +	/*
> > > > +	 * Get the TDSYSINFO_STRUCT and CMRs from the TDX module.
> > > > +	 *
> > > > +	 * The buffers of the TDSYSINFO_STRUCT and the CMR array passed
> > > > +	 * to the TDX module must be 1024-bytes and 512-bytes aligned
> > > > +	 * respectively.  Allocate one page to accommodate them both and
> > > > +	 * also meet those alignment requirements.
> > > > +	 */
> > > > +	sysinfo = (struct tdsysinfo_struct *)__get_free_page(GFP_KERNEL);
> > > > +	if (!sysinfo)
> > > > +		return -ENOMEM;
> > > > +	cmr_array = (struct cmr_info *)((unsigned long)sysinfo + PAGE_SIZE / 2);
> > > > +
> > > > +	BUILD_BUG_ON(PAGE_SIZE / 2 < TDSYSINFO_STRUCT_SIZE);
> > > > +	BUILD_BUG_ON(PAGE_SIZE / 2 < sizeof(struct cmr_info) * MAX_CMRS);
> > > 
> > > This works, but why not just use slab for this? kmalloc has 512 and 1024
> > > pools already and you won't waste memory for rounding up.
> > > 
> > > Something like this:
> > > 
> > >         sysinfo = kmalloc(TDSYSINFO_STRUCT_SIZE, GFP_KERNEL);
> > >         if (!sysinfo)
> > >                 return -ENOMEM;
> > > 
> > >         cmr_array_size = sizeof(struct cmr_info) * MAX_CMRS;
> > > 
> > >         /* CMR array has to be 512-aligned */
> > >         cmr_array_size = round_up(cmr_array_size, 512);
> > 
> > Should we define a macro for 512
> > 
> > 	+#define CMR_INFO_ARRAY_ALIGNMENT	512
> > 
> > And get rid of this comment?  AFAICT Dave didn't like such comment mentioning
> > 512-bytes aligned if we have a macro for that.
> 
> Good idea.
> 
> > >         cmr_array = kmalloc(cmr_array_size, GFP_KERNEL);
> > >         if (!cmr_array) {
> > >                 kfree(sysinfo);
> > >                 return -ENOMEM;
> > >         }
> > > 
> > > ?
> > > 
> > 
> > I confess the reason I used __get_free_page() was to avoid having to allocate
> > twice, and in case of failure, I need to handle additional memory free.  But I
> > can do if you think it's clearer?
> 
> Less trickery is always cleaner. Especially if the trick is not justified.
> 
> 

Alright.  I'll change to allocating them separately if no opinion from others.
  

On Tue, Jun 27, 2023 at 02:12:38AM +1200, Kai Huang wrote:
> +static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
> +			   struct cmr_info *cmr_array)
> +{
> +	struct tdx_module_output out;
> +	u64 sysinfo_pa, cmr_array_pa;
> +	int ret;
> +
> +	sysinfo_pa = __pa(sysinfo);
> +	cmr_array_pa = __pa(cmr_array);
> +	ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
> +			cmr_array_pa, MAX_CMRS, NULL, &out);
> +	if (ret)
> +		return ret;
> +
> +	pr_info("TDX module: attributes 0x%x, vendor_id 0x%x, major_version %u, minor_version %u, build_date %u, build_num %u",
> +		sysinfo->attributes,	sysinfo->vendor_id,
> +		sysinfo->major_version, sysinfo->minor_version,
> +		sysinfo->build_date,	sysinfo->build_num);
> +
> +	/* R9 contains the actual entries written to the CMR array. */

So I'm vexed by this comment; it's either not enough or too much.

I mean, as given you assume we all know about the magic parameters to
TDH_SYS_INFO but then somehow need an explanation for how %r9 is changed
from the array size to the number of used entries.

Either describe the whole thing or none of it.

Me, I would prefer all of it, because I've no idea where to begin
looking for any of this, SDM doesn't seem to be the place. That doesn't
even list TDCALL/SEAMCALL in Volume 2 :-( Let alone describe the magic
values.

> +	print_cmrs(cmr_array, out.r9);
> +
> +	return 0;
> +}
  

On Wed, 2023-06-28 at 16:10 +0200, Peter Zijlstra wrote:
> On Tue, Jun 27, 2023 at 02:12:38AM +1200, Kai Huang wrote:
> > +static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
> > +			   struct cmr_info *cmr_array)
> > +{
> > +	struct tdx_module_output out;
> > +	u64 sysinfo_pa, cmr_array_pa;
> > +	int ret;
> > +
> > +	sysinfo_pa = __pa(sysinfo);
> > +	cmr_array_pa = __pa(cmr_array);
> > +	ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
> > +			cmr_array_pa, MAX_CMRS, NULL, &out);
> > +	if (ret)
> > +		return ret;
> > +
> > +	pr_info("TDX module: attributes 0x%x, vendor_id 0x%x, major_version %u, minor_version %u, build_date %u, build_num %u",
> > +		sysinfo->attributes,	sysinfo->vendor_id,
> > +		sysinfo->major_version, sysinfo->minor_version,
> > +		sysinfo->build_date,	sysinfo->build_num);
> > +
> > +	/* R9 contains the actual entries written to the CMR array. */
> 
> So I'm vexed by this comment; it's either not enough or too much.
> 
> I mean, as given you assume we all know about the magic parameters to
> TDH_SYS_INFO but then somehow need an explanation for how %r9 is changed
> from the array size to the number of used entries.
> 
> Either describe the whole thing or none of it.
> 
> Me, I would prefer all of it, because I've no idea where to begin
> looking for any of this, 
> 

Sure.  How about below?

+       /*
+        * TDH.SYS.INFO writes the TDSYSINFO_STRUCT and the CMR array
+        * to the buffers provided by the kernel (via RCX and R8
+        * respectively).  The buffer size of the TDSYSINFO_STRUCT
+        * (via RDX) and the maximum entries of the CMR array (via R9)
+        * passed to this SEAMCALL must be at least the size of
+        * TDSYSINFO_STRUCT and MAX_CMRS respectively.
+        *
+        * Upon a successful return, R9 contains the actual entries
+        * written to the CMR array.
+        */
        sysinfo_pa = __pa(sysinfo);
        cmr_array_pa = __pa(cmr_array);
        ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
@@ -228,7 +239,6 @@ static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
                sysinfo->major_version, sysinfo->minor_version,
                sysinfo->build_date,    sysinfo->build_num);
 
-       /* R9 contains the actual entries written to the CMR array. */
        print_cmrs(cmr_array, out.r9);

Or should I just repeat the spec like below?

+       /*
+        * TDH.SYS.INFO writes the TDSYSINFO_STRUCT and the CMR array
+        * to the buffers provided by the kernel:
+        *
+        * Input:
+        *  - RCX: The buffer of TDSYSINFO_STRUCT
+        *  - RDX: The size of the TDSYSINFO_STRUCT buffer, must be at
+        *         at least the size of TDSYSINFO_STRUCT
+        *  - R8: The buffer of the CMR array
+        *  - R9: The entry number of the array, must be at least
+        *        MAX_CMRS.
+        *
+        * Output (successful):
+        *  - RDX: The actual bytes written to the TDSYSINFO_STRUCT
+        *         buffer
+        *  - R9: The actual entries written to the CMR array.
+        */
        sysinfo_pa = __pa(sysinfo);
        cmr_array_pa = __pa(cmr_array);
        ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
@@ -228,7 +245,6 @@ static int tdx_get_sysinfo(struct tdsysinfo_struct *sysinfo,
                sysinfo->major_version, sysinfo->minor_version,
                sysinfo->build_date,    sysinfo->build_num);
 
-       /* R9 contains the actual entries written to the CMR array. */
        print_cmrs(cmr_array, out.r9);

> SDM doesn't seem to be the place. That doesn't
> even list TDCALL/SEAMCALL in Volume 2 :-( Let alone describe the magic
> values.
> 

TDX has it's own specs at here:

https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html

For this one you can find it in here:

https://cdrdv2.intel.com/v1/dl/getContent/733568
  

On Thu, Jun 29, 2023 at 09:15:39AM +0000, Huang, Kai wrote:

> Sure.  How about below?
> 
> +       /*
> +        * TDH.SYS.INFO writes the TDSYSINFO_STRUCT and the CMR array
> +        * to the buffers provided by the kernel (via RCX and R8
> +        * respectively).  The buffer size of the TDSYSINFO_STRUCT
> +        * (via RDX) and the maximum entries of the CMR array (via R9)
> +        * passed to this SEAMCALL must be at least the size of
> +        * TDSYSINFO_STRUCT and MAX_CMRS respectively.
> +        *
> +        * Upon a successful return, R9 contains the actual entries
> +        * written to the CMR array.
> +        */
>         sysinfo_pa = __pa(sysinfo);
>         cmr_array_pa = __pa(cmr_array);
>         ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,

> Or should I just repeat the spec like below?

> +       /*
> +        * TDH.SYS.INFO writes the TDSYSINFO_STRUCT and the CMR array
> +        * to the buffers provided by the kernel:
> +        *
> +        * Input:
> +        *  - RCX: The buffer of TDSYSINFO_STRUCT
> +        *  - RDX: The size of the TDSYSINFO_STRUCT buffer, must be at
> +        *         at least the size of TDSYSINFO_STRUCT
> +        *  - R8: The buffer of the CMR array
> +        *  - R9: The entry number of the array, must be at least
> +        *        MAX_CMRS.
> +        *
> +        * Output (successful):
> +        *  - RDX: The actual bytes written to the TDSYSINFO_STRUCT
> +        *         buffer
> +        *  - R9: The actual entries written to the CMR array.
> +        */
>         sysinfo_pa = __pa(sysinfo);
>         cmr_array_pa = __pa(cmr_array);
>         ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,

Either of them work for me, thanks!

> > SDM doesn't seem to be the place. That doesn't
> > even list TDCALL/SEAMCALL in Volume 2 :-( Let alone describe the magic
> > values.
> > 
> 
> TDX has it's own specs at here:
> 
> https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
> 
> For this one you can find it in here:
> 
> https://cdrdv2.intel.com/v1/dl/getContent/733568

Yeah, eventually found it. I still think both TDCALL and SEAMCALL should
be listed in SDM Vol.2 instruction listing -- every valid instruction
should be found there IMO.

I also feel strongly that a global ABI should be decided upon for them
and the SDM would be a good place to mention that.  leaving this to
individual calls like now is a giant pain in the rear.

As is, we have TDCALL leaf-0 with a giant regset but every other leaf
has (c,d,8,9) for input and +(10,11) for output. Lets fix that in stone.

Obviously I also very strongly feel any such ABI must not confict with
pre-existing calling conventions -- IOW, using BP is out, must not
happen.
  

On Fri, 2023-06-30 at 11:34 +0200, Peter Zijlstra wrote:
> On Thu, Jun 29, 2023 at 09:15:39AM +0000, Huang, Kai wrote:
> 
> > Sure.  How about below?
> > 
> > +       /*
> > +        * TDH.SYS.INFO writes the TDSYSINFO_STRUCT and the CMR array
> > +        * to the buffers provided by the kernel (via RCX and R8
> > +        * respectively).  The buffer size of the TDSYSINFO_STRUCT
> > +        * (via RDX) and the maximum entries of the CMR array (via R9)
> > +        * passed to this SEAMCALL must be at least the size of
> > +        * TDSYSINFO_STRUCT and MAX_CMRS respectively.
> > +        *
> > +        * Upon a successful return, R9 contains the actual entries
> > +        * written to the CMR array.
> > +        */
> >         sysinfo_pa = __pa(sysinfo);
> >         cmr_array_pa = __pa(cmr_array);
> >         ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
> 
> > Or should I just repeat the spec like below?
> 
> > +       /*
> > +        * TDH.SYS.INFO writes the TDSYSINFO_STRUCT and the CMR array
> > +        * to the buffers provided by the kernel:
> > +        *
> > +        * Input:
> > +        *  - RCX: The buffer of TDSYSINFO_STRUCT
> > +        *  - RDX: The size of the TDSYSINFO_STRUCT buffer, must be at
> > +        *         at least the size of TDSYSINFO_STRUCT
> > +        *  - R8: The buffer of the CMR array
> > +        *  - R9: The entry number of the array, must be at least
> > +        *        MAX_CMRS.
> > +        *
> > +        * Output (successful):
> > +        *  - RDX: The actual bytes written to the TDSYSINFO_STRUCT
> > +        *         buffer
> > +        *  - R9: The actual entries written to the CMR array.
> > +        */
> >         sysinfo_pa = __pa(sysinfo);
> >         cmr_array_pa = __pa(cmr_array);
> >         ret = seamcall(TDH_SYS_INFO, sysinfo_pa, TDSYSINFO_STRUCT_SIZE,
> 
> Either of them work for me, thanks!

I will choose the first one since it's shorter.  Thanks!

> 
> > > SDM doesn't seem to be the place. That doesn't
> > > even list TDCALL/SEAMCALL in Volume 2 :-( Let alone describe the magic
> > > values.
> > > 
> > 
> > TDX has it's own specs at here:
> > 
> > https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
> > 
> > For this one you can find it in here:
> > 
> > https://cdrdv2.intel.com/v1/dl/getContent/733568
> 
> Yeah, eventually found it. I still think both TDCALL and SEAMCALL should
> be listed in SDM Vol.2 instruction listing -- every valid instruction
> should be found there IMO.
> 
> I also feel strongly that a global ABI should be decided upon for them
> and the SDM would be a good place to mention that.  leaving this to
> individual calls like now is a giant pain in the rear.

Yeah I agree how the specs are organized is not ideal.  We have been having pain
during our development too.

> 
> As is, we have TDCALL leaf-0 with a giant regset but every other leaf
> has (c,d,8,9) for input and +(10,11) for output. Lets fix that in stone.
> 
> Obviously I also very strongly feel any such ABI must not confict with
> pre-existing calling conventions -- IOW, using BP is out, must not
> happen.

Fully agreed.