mbox series

[RFC,kernel,0/2] PreventHostIBS feature for SEV-ES and SNP guests

Message ID 20230206060545.628502-1-manali.shukla@amd.com
Headers
Series PreventHostIBS feature for SEV-ES and SNP guests |

Message

Manali Shukla Feb. 6, 2023, 6:05 a.m. UTC 
  Adds support for PreventHostIBS feature for SEV-ES and SNP guests.
Currently, the hypervisor is able to inspect instruction based samples
from the guest and gather execution information.  With enablement of
PreventHostIBS feature, SEV-ES and SNP guests may choose to disallow
use of instruction based sampling by the hypervisor in order to limit
the information gathered about their execution.  (More information in
Section 15.36.17 APM Volume 2)

While implementing this feature, unknown NMIs were being seen. On
further investigation, a race was found effecting the IBS FETCH/OP
MSR.

ENABLE bit and VALID bit for IBS_FETCH_CTL are contained in the same
MSR and same is the case with IBS_OP_CTL.

Consider the following scenario:
- The IBS MSR which has ENABLE bit set and VALID bit clear is read.
- During the process of clearing the ENABLE bit and writing the IBS
  MSR to disable IBS, an IBS event can occur that sets the VALID bit.
- The write operation on IBS MSR can clear the newly set VALID bit.
- Since this situation is occurring in the CLGI/STGI window
  (PreventHostIBS window), the actual NMI is not taken.
- Once VMRUN is issued, it will exit with VMEXIT_NMI and as soon as
  STGI is executed, the pending NMI will trigger.
- The IBS NMI handler checks for the VALID bit to determine if the NMI
  is generated because of IBS.
- Since VALID bit is now clear, it doesn't recognize that an IBS event
  is occurred which in turn generates the dazed and confused unknown
  NMI messages.

Per-cpu ibs_flags which indicates whether PreventHostIBS window is
active/inactive are added to avoid the above mentioned race.

An active PreventHostIBS window is set before calling VMRUN and
cleared after STGI. PreventHostIBS window check is added to
perf_ibs_handle_irq(), to avoid unknown NMIs and treat them as handled
when window is active.

There are 2 patches in this series.
1) Add amd_prevent_hostibs_window() function to set per-cpu ibs_flags
based on an active/inactive PreventHostIBS window.
2) Enable PreventHostIBS for SEV-ES and SNP guests.

Testing done:
- Executed program symbols in guest are not captured in host when
  PreventHostIBS feature is enabled.
- Generated 1000+ NMIs using cpuid command, no unknown NMIs are seen
  after enablement of PreventHostIBS feature.

Qemu commandline to enable PreventHostIBS on guest.

qemu-system-x86_64 -enable-kvm -cpu EPYC-v4,+nohostibs \ ..

Manali Shukla (2):
  perf/x86/amd: Add amd_prevent_hostibs_window() to set per-cpu
    ibs_flags
  KVM: SEV: PreventHostIBS enablement for SEV-ES and SNP guest

 arch/x86/events/amd/ibs.c          | 64 ++++++++++++++++++++++++++++++
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/include/asm/perf_event.h  | 20 ++++++++++
 arch/x86/kvm/svm/sev.c             | 10 +++++
 arch/x86/kvm/svm/svm.c             | 39 +++++++++++++++++-
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 133 insertions(+), 2 deletions(-)

Comments

Manali Shukla March 15, 2023, 5:03 a.m. UTC | #1 
On 2/6/2023 11:35 AM, Manali Shukla wrote:
> Adds support for PreventHostIBS feature for SEV-ES and SNP guests.
> Currently, the hypervisor is able to inspect instruction based samples
> from the guest and gather execution information.  With enablement of
> PreventHostIBS feature, SEV-ES and SNP guests may choose to disallow
> use of instruction based sampling by the hypervisor in order to limit
> the information gathered about their execution.  (More information in
> Section 15.36.17 APM Volume 2)
> 
> While implementing this feature, unknown NMIs were being seen. On
> further investigation, a race was found effecting the IBS FETCH/OP
> MSR.
> 
> ENABLE bit and VALID bit for IBS_FETCH_CTL are contained in the same
> MSR and same is the case with IBS_OP_CTL.
> 
> Consider the following scenario:
> - The IBS MSR which has ENABLE bit set and VALID bit clear is read.
> - During the process of clearing the ENABLE bit and writing the IBS
>   MSR to disable IBS, an IBS event can occur that sets the VALID bit.
> - The write operation on IBS MSR can clear the newly set VALID bit.
> - Since this situation is occurring in the CLGI/STGI window
>   (PreventHostIBS window), the actual NMI is not taken.
> - Once VMRUN is issued, it will exit with VMEXIT_NMI and as soon as
>   STGI is executed, the pending NMI will trigger.
> - The IBS NMI handler checks for the VALID bit to determine if the NMI
>   is generated because of IBS.
> - Since VALID bit is now clear, it doesn't recognize that an IBS event
>   is occurred which in turn generates the dazed and confused unknown
>   NMI messages.
> 
> Per-cpu ibs_flags which indicates whether PreventHostIBS window is
> active/inactive are added to avoid the above mentioned race.
> 
> An active PreventHostIBS window is set before calling VMRUN and
> cleared after STGI. PreventHostIBS window check is added to
> perf_ibs_handle_irq(), to avoid unknown NMIs and treat them as handled
> when window is active.
> 
> There are 2 patches in this series.
> 1) Add amd_prevent_hostibs_window() function to set per-cpu ibs_flags
> based on an active/inactive PreventHostIBS window.
> 2) Enable PreventHostIBS for SEV-ES and SNP guests.
> 
> Testing done:
> - Executed program symbols in guest are not captured in host when
>   PreventHostIBS feature is enabled.
> - Generated 1000+ NMIs using cpuid command, no unknown NMIs are seen
>   after enablement of PreventHostIBS feature.
> 
> Qemu commandline to enable PreventHostIBS on guest.
> 
> qemu-system-x86_64 -enable-kvm -cpu EPYC-v4,+nohostibs \ ..
> 
> Manali Shukla (2):
>   perf/x86/amd: Add amd_prevent_hostibs_window() to set per-cpu
>     ibs_flags
>   KVM: SEV: PreventHostIBS enablement for SEV-ES and SNP guest
> 
>  arch/x86/events/amd/ibs.c          | 64 ++++++++++++++++++++++++++++++
>  arch/x86/include/asm/cpufeatures.h |  1 +
>  arch/x86/include/asm/perf_event.h  | 20 ++++++++++
>  arch/x86/kvm/svm/sev.c             | 10 +++++
>  arch/x86/kvm/svm/svm.c             | 39 +++++++++++++++++-
>  arch/x86/kvm/svm/svm.h             |  1 +
>  6 files changed, 133 insertions(+), 2 deletions(-)
> 

A gentle reminder for the review.

-Manali
Manali Shukla March 23, 2023, 6:06 a.m. UTC | #2 
On 3/15/2023 10:33 AM, Manali Shukla wrote:
> On 2/6/2023 11:35 AM, Manali Shukla wrote:
>> Adds support for PreventHostIBS feature for SEV-ES and SNP guests.
>> Currently, the hypervisor is able to inspect instruction based samples
>> from the guest and gather execution information.  With enablement of
>> PreventHostIBS feature, SEV-ES and SNP guests may choose to disallow
>> use of instruction based sampling by the hypervisor in order to limit
>> the information gathered about their execution.  (More information in
>> Section 15.36.17 APM Volume 2)
>>
>> While implementing this feature, unknown NMIs were being seen. On
>> further investigation, a race was found effecting the IBS FETCH/OP
>> MSR.
>>
>> ENABLE bit and VALID bit for IBS_FETCH_CTL are contained in the same
>> MSR and same is the case with IBS_OP_CTL.
>>
>> Consider the following scenario:
>> - The IBS MSR which has ENABLE bit set and VALID bit clear is read.
>> - During the process of clearing the ENABLE bit and writing the IBS
>>   MSR to disable IBS, an IBS event can occur that sets the VALID bit.
>> - The write operation on IBS MSR can clear the newly set VALID bit.
>> - Since this situation is occurring in the CLGI/STGI window
>>   (PreventHostIBS window), the actual NMI is not taken.
>> - Once VMRUN is issued, it will exit with VMEXIT_NMI and as soon as
>>   STGI is executed, the pending NMI will trigger.
>> - The IBS NMI handler checks for the VALID bit to determine if the NMI
>>   is generated because of IBS.
>> - Since VALID bit is now clear, it doesn't recognize that an IBS event
>>   is occurred which in turn generates the dazed and confused unknown
>>   NMI messages.
>>
>> Per-cpu ibs_flags which indicates whether PreventHostIBS window is
>> active/inactive are added to avoid the above mentioned race.
>>
>> An active PreventHostIBS window is set before calling VMRUN and
>> cleared after STGI. PreventHostIBS window check is added to
>> perf_ibs_handle_irq(), to avoid unknown NMIs and treat them as handled
>> when window is active.
>>
>> There are 2 patches in this series.
>> 1) Add amd_prevent_hostibs_window() function to set per-cpu ibs_flags
>> based on an active/inactive PreventHostIBS window.
>> 2) Enable PreventHostIBS for SEV-ES and SNP guests.
>>
>> Testing done:
>> - Executed program symbols in guest are not captured in host when
>>   PreventHostIBS feature is enabled.
>> - Generated 1000+ NMIs using cpuid command, no unknown NMIs are seen
>>   after enablement of PreventHostIBS feature.
>>
>> Qemu commandline to enable PreventHostIBS on guest.
>>
>> qemu-system-x86_64 -enable-kvm -cpu EPYC-v4,+nohostibs \ ..
>>
>> Manali Shukla (2):
>>   perf/x86/amd: Add amd_prevent_hostibs_window() to set per-cpu
>>     ibs_flags
>>   KVM: SEV: PreventHostIBS enablement for SEV-ES and SNP guest
>>
>>  arch/x86/events/amd/ibs.c          | 64 ++++++++++++++++++++++++++++++
>>  arch/x86/include/asm/cpufeatures.h |  1 +
>>  arch/x86/include/asm/perf_event.h  | 20 ++++++++++
>>  arch/x86/kvm/svm/sev.c             | 10 +++++
>>  arch/x86/kvm/svm/svm.c             | 39 +++++++++++++++++-
>>  arch/x86/kvm/svm/svm.h             |  1 +
>>  6 files changed, 133 insertions(+), 2 deletions(-)
>>
> 
> A gentle reminder for the review.
> 
> -Manali

A gentle reminder for the review.

-Manali