mbox series

[RFC,0/5] Add TDX intra host migration support

Message ID 20230407201921.2703758-1-sagis@google.com
Headers
Series Add TDX intra host migration support |

Message

Sagi Shahar April 7, 2023, 8:19 p.m. UTC 
  This patchset adds support for TDX intra host migration using the same
API which was added for SEV intra host migration here:
https://lore.kernel.org/all/20211021174303.385706-1-pgonda@google.com/

This patchset relies on the latest TDX patches from Intel:
- fd-based approach for supporing KVM v10 and
  https://lore.kernel.org/lkml/20221202061347.1070246-1-chao.p.peng@linux.intel.com/
- TDX host kernel support v10
  https://lore.kernel.org/lkml/cover.1678111292.git.kai.huang@intel.com/
- KVM TDX basic feature support v13
  https://lore.kernel.org/cover.1678643051.git.isaku.yamahata@intel.com

The tree can be found at https://github.com/googleprodkernel/linux-cc/tree/copyless
and is based on Intel's tdx tree at https://github.com/intel/tdx/tree/kvm-upstream

In the TDX case, we need to transfer the VM state from multiple sources:

 * HKID and encrypted VM state is transfered between the kvm_tdx
   objects.
 * Encrypted and runtime state is transfered between the vcpu_tdx
   objects.
 * The EPT table backing TD's private memory is transfered at the
   kvm-mmu level. This is needed since the secure EPT table managed by
   the TD module remains the same after the migration so moving the
   current private EPT table eliminates the need to rebuild the private
   EPT table to match the secure EPT table on the destination.
 * Information regarding the current shared/private memory is trasfered
   using the mem_attr_array stored at the kvm object.
 * Additional information derived from shared/private memory state is
   trasfered at the memslot level.

Tested with selftests locally. I will attach the self test in the next
version after we send the new TDX selftest framework patches based on
KVM TDX basic feature support v13.

Sagi Shahar (5):
  KVM: Split tdp_mmu_pages to private and shared lists
  KVM: SEV: Refactor common code out of sev_vm_move_enc_context_from
  KVM: TDX: Add base implementation for tdx_vm_move_enc_context_from
  KVM: TDX: Implement moving private pages between 2 TDs
  KVM: TDX: Add core logic for TDX intra-host migration

 arch/x86/include/asm/kvm_host.h |   5 +-
 arch/x86/kvm/mmu.h              |   2 +
 arch/x86/kvm/mmu/mmu.c          |  60 ++++++++
 arch/x86/kvm/mmu/tdp_mmu.c      |  88 +++++++++++-
 arch/x86/kvm/mmu/tdp_mmu.h      |   3 +
 arch/x86/kvm/svm/sev.c          | 175 +++--------------------
 arch/x86/kvm/vmx/main.c         |  10 ++
 arch/x86/kvm/vmx/tdx.c          | 245 ++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.h          |   2 +
 arch/x86/kvm/vmx/x86_ops.h      |   5 +
 arch/x86/kvm/x86.c              | 166 ++++++++++++++++++++++
 arch/x86/kvm/x86.h              |  16 +++
 12 files changed, 613 insertions(+), 164 deletions(-)

Comments

Zhi Wang April 14, 2023, 7:03 a.m. UTC | #1 
On Fri,  7 Apr 2023 20:19:16 +0000
Sagi Shahar <sagis@google.com> wrote:

Hi:

Is there any userspace using these APIs? I cant find them in AMD-QEMU repo
and upstream QEMU repo. It would nice to first take a look on how userspace
is going to use it.

> This patchset adds support for TDX intra host migration using the same
> API which was added for SEV intra host migration here:
> https://lore.kernel.org/all/20211021174303.385706-1-pgonda@google.com/
> 
> This patchset relies on the latest TDX patches from Intel:
> - fd-based approach for supporing KVM v10 and
>   https://lore.kernel.org/lkml/20221202061347.1070246-1-chao.p.peng@linux.intel.com/
> - TDX host kernel support v10
>   https://lore.kernel.org/lkml/cover.1678111292.git.kai.huang@intel.com/
> - KVM TDX basic feature support v13
>   https://lore.kernel.org/cover.1678643051.git.isaku.yamahata@intel.com
> 
> The tree can be found at https://github.com/googleprodkernel/linux-cc/tree/copyless
> and is based on Intel's tdx tree at https://github.com/intel/tdx/tree/kvm-upstream
> 
> In the TDX case, we need to transfer the VM state from multiple sources:
> 
>  * HKID and encrypted VM state is transfered between the kvm_tdx
>    objects.
>  * Encrypted and runtime state is transfered between the vcpu_tdx
>    objects.
>  * The EPT table backing TD's private memory is transfered at the
>    kvm-mmu level. This is needed since the secure EPT table managed by
>    the TD module remains the same after the migration so moving the
>    current private EPT table eliminates the need to rebuild the private
>    EPT table to match the secure EPT table on the destination.
>  * Information regarding the current shared/private memory is trasfered
>    using the mem_attr_array stored at the kvm object.
>  * Additional information derived from shared/private memory state is
>    trasfered at the memslot level.
> 
> Tested with selftests locally. I will attach the self test in the next
> version after we send the new TDX selftest framework patches based on
> KVM TDX basic feature support v13.
> 
> Sagi Shahar (5):
>   KVM: Split tdp_mmu_pages to private and shared lists
>   KVM: SEV: Refactor common code out of sev_vm_move_enc_context_from
>   KVM: TDX: Add base implementation for tdx_vm_move_enc_context_from
>   KVM: TDX: Implement moving private pages between 2 TDs
>   KVM: TDX: Add core logic for TDX intra-host migration
> 
>  arch/x86/include/asm/kvm_host.h |   5 +-
>  arch/x86/kvm/mmu.h              |   2 +
>  arch/x86/kvm/mmu/mmu.c          |  60 ++++++++
>  arch/x86/kvm/mmu/tdp_mmu.c      |  88 +++++++++++-
>  arch/x86/kvm/mmu/tdp_mmu.h      |   3 +
>  arch/x86/kvm/svm/sev.c          | 175 +++--------------------
>  arch/x86/kvm/vmx/main.c         |  10 ++
>  arch/x86/kvm/vmx/tdx.c          | 245 ++++++++++++++++++++++++++++++++
>  arch/x86/kvm/vmx/tdx.h          |   2 +
>  arch/x86/kvm/vmx/x86_ops.h      |   5 +
>  arch/x86/kvm/x86.c              | 166 ++++++++++++++++++++++
>  arch/x86/kvm/x86.h              |  16 +++
>  12 files changed, 613 insertions(+), 164 deletions(-)
>
Sagi Shahar April 14, 2023, 7:09 p.m. UTC | #2 
On Fri, Apr 14, 2023 at 12:04 AM Zhi Wang <zhi.wang.linux@gmail.com> wrote:
>
> On Fri,  7 Apr 2023 20:19:16 +0000
> Sagi Shahar <sagis@google.com> wrote:
>
> Hi:
>
> Is there any userspace using these APIs? I cant find them in AMD-QEMU repo
> and upstream QEMU repo. It would nice to first take a look on how userspace
> is going to use it.
>
We are using a different userspace VMM internally so we didn't make
changes to QEMU.
I've uploaded our selftests which exercise these APIs to our public
GitHub so you can take a look there:
https://github.com/googleprodkernel/linux-cc/commit/62c8dba4c3cf06e375018077a6d9f491c933dc6d

Note that these are a slightly older version based on TDX V10 API.
They also use the
KVM_CAP_VM_COPY_ENC_CONTEXT_FROM instead of KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM

> > This patchset adds support for TDX intra host migration using the same
> > API which was added for SEV intra host migration here:
> > https://lore.kernel.org/all/20211021174303.385706-1-pgonda@google.com/
> >
> > This patchset relies on the latest TDX patches from Intel:
> > - fd-based approach for supporing KVM v10 and
> >   https://lore.kernel.org/lkml/20221202061347.1070246-1-chao.p.peng@linux.intel.com/
> > - TDX host kernel support v10
> >   https://lore.kernel.org/lkml/cover.1678111292.git.kai.huang@intel.com/
> > - KVM TDX basic feature support v13
> >   https://lore.kernel.org/cover.1678643051.git.isaku.yamahata@intel.com
> >
> > The tree can be found at https://github.com/googleprodkernel/linux-cc/tree/copyless
> > and is based on Intel's tdx tree at https://github.com/intel/tdx/tree/kvm-upstream
> >
> > In the TDX case, we need to transfer the VM state from multiple sources:
> >
> >  * HKID and encrypted VM state is transfered between the kvm_tdx
> >    objects.
> >  * Encrypted and runtime state is transfered between the vcpu_tdx
> >    objects.
> >  * The EPT table backing TD's private memory is transfered at the
> >    kvm-mmu level. This is needed since the secure EPT table managed by
> >    the TD module remains the same after the migration so moving the
> >    current private EPT table eliminates the need to rebuild the private
> >    EPT table to match the secure EPT table on the destination.
> >  * Information regarding the current shared/private memory is trasfered
> >    using the mem_attr_array stored at the kvm object.
> >  * Additional information derived from shared/private memory state is
> >    trasfered at the memslot level.
> >
> > Tested with selftests locally. I will attach the self test in the next
> > version after we send the new TDX selftest framework patches based on
> > KVM TDX basic feature support v13.
> >
> > Sagi Shahar (5):
> >   KVM: Split tdp_mmu_pages to private and shared lists
> >   KVM: SEV: Refactor common code out of sev_vm_move_enc_context_from
> >   KVM: TDX: Add base implementation for tdx_vm_move_enc_context_from
> >   KVM: TDX: Implement moving private pages between 2 TDs
> >   KVM: TDX: Add core logic for TDX intra-host migration
> >
> >  arch/x86/include/asm/kvm_host.h |   5 +-
> >  arch/x86/kvm/mmu.h              |   2 +
> >  arch/x86/kvm/mmu/mmu.c          |  60 ++++++++
> >  arch/x86/kvm/mmu/tdp_mmu.c      |  88 +++++++++++-
> >  arch/x86/kvm/mmu/tdp_mmu.h      |   3 +
> >  arch/x86/kvm/svm/sev.c          | 175 +++--------------------
> >  arch/x86/kvm/vmx/main.c         |  10 ++
> >  arch/x86/kvm/vmx/tdx.c          | 245 ++++++++++++++++++++++++++++++++
> >  arch/x86/kvm/vmx/tdx.h          |   2 +
> >  arch/x86/kvm/vmx/x86_ops.h      |   5 +
> >  arch/x86/kvm/x86.c              | 166 ++++++++++++++++++++++
> >  arch/x86/kvm/x86.h              |  16 +++
> >  12 files changed, 613 insertions(+), 164 deletions(-)
> >
>