mbox series

[V4,0/9] x86/hyperv: Add AMD sev-snp enlightened guest support on hyperv

Message ID 20230804152254.686317-1-ltykernel@gmail.com
Headers
Series x86/hyperv: Add AMD sev-snp enlightened guest support on hyperv |

Message

Tianyu Lan Aug. 4, 2023, 3:22 p.m. UTC 
  From: Tianyu Lan <tiala@microsoft.com>

Hyper-V provides two modes for running SEV-SNP VMs:

1) In vTOM mode with a paravisor (see Section 15.36.8 of [1])
2) In "fully enlightened" mode with normal "C" bit control
   over page encryption, and no paravisor
 
For #1, the paravisor runs in VMPL 0, while Linux runs in VMPL 2
(see Section 15.36.7 of [1]). The paravisor is typically provided
by Hyper-V and handles most of the SNP-related functionality. As
such, most of the SNP functionality in the Linux guest is bypassed.
The guest operates in vTOM mode, where encryption is enabled by default.
The guest must still request page transitions between private and shared,
but there is relatively less SNP machinery required in the guest. Support
for this mode of operation first went upstream in the 5.15 kernel.

For #2, this patch set provides the initial support. The existing
SEV-SNP machinery in the kernel is fully used, but Hyper-V specific
updates are required to properly share Hyper-V communication pages
between the guest and host and to start APs at boot time.

In either mode, Hyper-V requires that the guest implement the SEV-SNP
Restricted Interrupt Injection feature (see Section 15.36.16 of [1],
and Section 5 of [2]). Without this feature, the guest is subject to
attack by a compromised hypervisor that can inject any exception at
any time, such as injecting an interrupt while the guest has interrupts
disabled. In vTOM mode, Restricted Interrupt Injection is implemented
by the paravisor, so no Linux guest changes are required. But in fully
enlightened mode, the Linux guest must provide the implementation.

This patch set is derived from an earlier patch set that includes both
the Hyper-V specific changes and Restricted Interrupt Injection support.[3]
But it is now limited to only the Hyper-V specific changes. The Restricted
Interrupt Injection support will come later in a separate patch set.


[1] https://www.amd.com/system/files/TechDocs/24593.pdf
[2] https://www.amd.com/system/files/TechDocs/56421-guest-hypervisor-communication-block-standardization.pdf
[3] https://lore.kernel.org/lkml/20230515165917.1306922-1-ltykernel@gmail.com/

Change since v3:
       * Fix fossil comment

Change since v2:
       * Update Change log.
       * Rework Hyper-V hypercall implementation.

Change since v1:
       * vTOM case uses paravisor_present flag and
       	 HV_ISOLATION_TYPE_SNP type.
       * Rework some patches' change log
       * Fix some comments in the patches

Tianyu Lan (9):
  x86/hyperv: Add sev-snp enlightened guest static key
  x86/hyperv: Set Virtual Trust Level in VMBus init message
  x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP
    enlightened guest
  drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP
    enlightened guest
  x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp
    enlightened guest
  clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp
    enlightened guest
  x86/hyperv: Add smp support for SEV-SNP guest
  x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES
  x86/hyperv: Initialize cpu and memory for SEV-SNP enlightened guest

 arch/x86/hyperv/hv_init.c          |  52 +++++++-
 arch/x86/hyperv/ivm.c              | 199 +++++++++++++++++++++++++++++
 arch/x86/include/asm/hyperv-tlfs.h |   7 +
 arch/x86/include/asm/mshyperv.h    |  56 ++++++--
 arch/x86/kernel/cpu/mshyperv.c     |  42 +++++-
 drivers/clocksource/hyperv_timer.c |   2 +-
 drivers/hv/connection.c            |   1 +
 drivers/hv/hv.c                    |  57 ++++++++-
 drivers/hv/hv_common.c             |  19 +++
 include/asm-generic/hyperv-tlfs.h  |   1 +
 include/asm-generic/mshyperv.h     |  13 +-
 include/linux/hyperv.h             |   4 +-
 12 files changed, 426 insertions(+), 27 deletions(-)

Comments

Wei Liu Aug. 4, 2023, 11:34 p.m. UTC | #1 
On Fri, Aug 04, 2023 at 11:22:44AM -0400, Tianyu Lan wrote:
> From: Tianyu Lan <tiala@microsoft.com>
[...]
> Tianyu Lan (9):
>   x86/hyperv: Add sev-snp enlightened guest static key
>   x86/hyperv: Set Virtual Trust Level in VMBus init message
>   x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP
>     enlightened guest
>   drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP
>     enlightened guest
>   x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp
>     enlightened guest
>   clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp
>     enlightened guest
>   x86/hyperv: Add smp support for SEV-SNP guest
>   x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES

I applied all but the last patch to hyperv-next. Thanks.

>   x86/hyperv: Initialize cpu and memory for SEV-SNP enlightened guest
Dexuan Cui Aug. 5, 2023, 11:37 p.m. UTC | #2 
> From: Wei Liu <wei.liu@kernel.org>
> Sent: Friday, August 4, 2023 4:35 PM
> To: Tianyu Lan <ltykernel@gmail.com>
>  [...]
> On Fri, Aug 04, 2023 at 11:22:44AM -0400, Tianyu Lan wrote:
> > From: Tianyu Lan <tiala@microsoft.com>
> [...]
> > Tianyu Lan (9):
[...]
> >   x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp ...

Unluckily this commit causes a crash on Intel CPUs (see the below call-trace).

I made a fix here:
https://github.com/dcui/linux/commit/c4db45f6256248435b2a303b264ecbb41320c41d
I guess Wei can squash the fix into Tianyu's commit in the hyperv-next branch?

[    0.861465] invalid opcode: 0000 [#1] PREEMPT SMP
[    0.865489] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.5.0-rc4-decui-hcl+ #1
[    0.869194] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 04/20/2023
[    0.875049] RIP: 0010:hv_query_ext_cap+0x7a/0x130
[    0.879626] Code: 83 72 57 48 c7 c6 00 00 00 80 48 2b 35 0f f2 9d 00 48 01 f0 48 8b 35 5d 3e 3d 01 48 85 f6 74 2a b9 01 80 00 00 31 d2 49 89 c0 <0f> 01 d9 c6 05 d4 f2 20 02 01 66 85 c0 74 aa 48 89 c6 48 c7 c7 20
[    0.897457] RSP: 0000:ffffffff82803e50 EFLAGS: 00010246
[    0.900194] RAX: 00000000074f72e0 RBX: 0000000000000000 RCX: 0000000000008001
[    0.903955] RDX: 0000000000000000 RSI: ffffc90000017000 RDI: 0000000000000000
[    0.908445] RBP: ffffffff82803e60 R08: 00000000074f72e0 R09: 0000000000000001
[    0.916354] R10: 0720072007200720 R11: 0720072007200720 R12: 0000000000000000
[    0.924285] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff8280edb8
[    0.929481] FS:  0000000000000000(0000) GS:ffff8880f6800000(0000) knlGS:0000000000000000
[    0.933589] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.936526] CR2: ffff888507fff000 CR3: 0000000006036001 CR4: 00000000000200f0
[    0.942279] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.950573] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.958417] Call Trace:
[    0.959827]  <TASK>
[    0.961044]  ? show_regs+0x64/0x70
[    0.962872]  ? __die_body+0x20/0x70
[    0.964741]  ? die+0x3e/0x60
[    0.966273]  ? do_trap+0xd7/0x100
[    0.967922]  ? do_error_trap+0x71/0x90
[    0.970027]  ? hv_query_ext_cap+0x7a/0x130
[    0.974573]  ? exc_invalid_op+0x53/0x70
[    0.978961]  ? hv_query_ext_cap+0x7a/0x130
[    0.983977]  ? asm_exc_invalid_op+0x1b/0x20
[    0.988932]  ? hv_query_ext_cap+0x7a/0x130
[    0.991413]  hyperv_init+0x2fe/0x5f0
[    0.993373]  apic_intr_mode_init+0x3e/0x100
[    0.995655]  x86_late_time_init+0x1b/0x30
[    0.997832]  start_kernel+0x617/0xa70
[    0.999546]  x86_64_start_reservations+0x18/0x30
[    1.003761]  x86_64_start_kernel+0xc0/0x110
[    1.008815]  secondary_startup_64_no_verify+0x17e/0x18b
[  Ubuntu
[ *Advanced options for Ubuntu
[    1.021465] ---[ end trace 0000000000000000 ]---
[    1.024093] RIP: 0010:hv_query_ext_cap+0x7a/0x130
[    1.026769] Code: 83 72 57 48 c7 c6 00 00 00 80 48 2b 35 0f f2 9d 00 48 01 f0 48 8b 35 5d 3e 3d 01 48 85 f6 74 2a b9 01 80 00 00 31 d2 49 89 c0 <0f> 01 d9 c6 05 d4 f2 20 02 01 66 85 c0 74 aa 48 89 c6 48 c7 c7 20
[    1.042358] RSP: 0000:ffffffff82803e50 EFLAGS: 00010246
[    1.049317] RAX: 00000000074f72e0 RBX: 0000000000000000 RCX: 0000000000008001
[    1.054740] RDX: 0000000000000000 RSI: ffffc90000017000 RDI: 0000000000000000
[    1.059058] RBP: ffffffff82803e60 R08: 00000000074f72e0 R09: 0000000000000001
[    1.062406] R10: 0720072007200720 R11: 0720072007200720 R12: 0000000000000000
[    1.070915] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff8280edb8
[    1.080300] FS:  0000000000000000(0000) GS:ffff8880f6800000(0000) knlGS:0000000000000000
[    1.086956] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.090242] CR2: ffff888507fff000 CR3: 0000000006036001 CR4: 00000000000200f0
[    1.093949] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    1.101352] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Dexuan Cui Aug. 6, 2023, 1:53 a.m. UTC | #3 
> From: Dexuan Cui
> Sent: Saturday, August 5, 2023 4:37 PM
> [...]
> > From: Wei Liu <wei.liu@kernel.org>
> > Sent: Friday, August 4, 2023 4:35 PM
> > To: Tianyu Lan <ltykernel@gmail.com>
> >  [...]
> > On Fri, Aug 04, 2023 at 11:22:44AM -0400, Tianyu Lan wrote:
> > > From: Tianyu Lan <tiala@microsoft.com>
> > [...]
> > > Tianyu Lan (9):
> [...]
> > >   x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp ...
> 
> Unluckily this commit causes a crash on Intel CPUs (see the below call-trace).
> 
> I made a fix here:
> https://github.com/dcui/linux/commit/c4db45f6256248435b2a303b264ecbb
> 41320c41d
> I guess Wei can squash the fix into Tianyu's commit in the hyperv-next
> branch?

I also made a patch to fix the ARM64 build:
https://github.com/dcui/linux/commit/a559709c612de2a212e288ef1a8b0abfabb168e5
Wei Liu Aug. 7, 2023, 4:32 a.m. UTC | #4 
On Fri, Aug 04, 2023 at 11:34:36PM +0000, Wei Liu wrote:
> On Fri, Aug 04, 2023 at 11:22:44AM -0400, Tianyu Lan wrote:
> > From: Tianyu Lan <tiala@microsoft.com>
> [...]
> > Tianyu Lan (9):
> >   x86/hyperv: Add sev-snp enlightened guest static key
> >   x86/hyperv: Set Virtual Trust Level in VMBus init message
> >   x86/hyperv: Mark Hyper-V vp assist page unencrypted in SEV-SNP
> >     enlightened guest
> >   drivers: hv: Mark percpu hvcall input arg page unencrypted in SEV-SNP
> >     enlightened guest
> >   x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp
> >     enlightened guest
> >   clocksource: hyper-v: Mark hyperv tsc page unencrypted in sev-snp
> >     enlightened guest
> >   x86/hyperv: Add smp support for SEV-SNP guest
> >   x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES
> 
> I applied all but the last patch to hyperv-next. Thanks.

This is causing build issues in linux-next. I've reverted this series
from hyperv-next.

> 
> >   x86/hyperv: Initialize cpu and memory for SEV-SNP enlightened guest
>