diff mbox series

KVM: x86: Allow APICv APIC ID inhibit to be cleared on legacy kernels

Message ID 20221114202037.254176-1-gedwards@ddn.com
State New
Headers
Series KVM: x86: Allow APICv APIC ID inhibit to be cleared on legacy kernels |

Commit Message

Greg Edwards Nov. 14, 2022, 8:20 p.m. UTC 
  Legacy kernels prior to commit 4399c03c6780 ("x86/apic: Remove
verify_local_APIC()") write the xAPIC ID of the boot CPU twice to verify
a functioning local APIC.  This results in APIC acceleration inhibited
on these kernels for reason APICV_INHIBIT_REASON_APIC_ID_MODIFIED.

Allow the APICV_INHIBIT_REASON_APIC_ID_MODIFIED inhibit reason to be
cleared if/when the xAPIC ID is set back to the expected vcpu_id value.
This occurs on the second xAPIC ID write in verify_local_APIC().

Fixes: 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
Signed-off-by: Greg Edwards <gedwards@ddn.com>
---
 arch/x86/kvm/lapic.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

Comments

Sean Christopherson Nov. 14, 2022, 9:30 p.m. UTC | #1 
On Mon, Nov 14, 2022, Greg Edwards wrote:
> Legacy kernels prior to commit 4399c03c6780 ("x86/apic: Remove
> verify_local_APIC()") write the xAPIC ID of the boot CPU twice to verify
> a functioning local APIC.  This results in APIC acceleration inhibited
> on these kernels for reason APICV_INHIBIT_REASON_APIC_ID_MODIFIED.
> 
> Allow the APICV_INHIBIT_REASON_APIC_ID_MODIFIED inhibit reason to be
> cleared if/when the xAPIC ID is set back to the expected vcpu_id value.
> This occurs on the second xAPIC ID write in verify_local_APIC().
> 
> Fixes: 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
> Signed-off-by: Greg Edwards <gedwards@ddn.com>
> ---
>  arch/x86/kvm/lapic.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index d7639d126e6c..4064d0af094d 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -2075,8 +2075,13 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
>  	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
>  		return;
>  
> -	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
> +	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id) {
> +		/* Legacy kernels prior to 4399c03c6780 write APIC ID twice. */
> +		if (!kvm_apicv_activated(kvm))
> +			kvm_clear_apicv_inhibit(kvm,
> +					APICV_INHIBIT_REASON_APIC_ID_MODIFIED);

This sadly doesn't work because the inhibit is per-VM, i.e. will do the wrong
thing if there are still vCPU's with different APIC IDs.

Does skipping the check if the APIC is disabled help[*]?  At a glance, I can't
tell if the APIC is enabled/disabled at that point in time.  It's not a true fix,
but it's a lot easier to backport if it remedies the issue.

For a proper fix, this entire path should be moved to kvm_recalculate_apic_map()
so that can can safely toggle the inhibit, e.g. the recalc helper already deals
with multiple vCPUs changing their APIC state in parallel.  I don't think the fix
will be too difficult to craft such that it's backport friendly, but it would need
to be slotted into the series containing the aforementioned fix.

[*] https://lore.kernel.org/all/20221001005915.2041642-6-seanjc@google.com

---
 arch/x86/kvm/lapic.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 5de1c7aa1ce9..67260f7ce43a 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2072,6 +2072,9 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
 {
 	struct kvm *kvm = apic->vcpu->kvm;
 
+	if (!kvm_apic_hw_enabled(apic))
+		return;
+
 	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
 		return;
Greg Edwards Nov. 14, 2022, 11:53 p.m. UTC | #2 
On Mon, Nov 14, 2022 at 09:30:18PM +0000, Sean Christopherson wrote:
> On Mon, Nov 14, 2022, Greg Edwards wrote:
>> Legacy kernels prior to commit 4399c03c6780 ("x86/apic: Remove
>> verify_local_APIC()") write the xAPIC ID of the boot CPU twice to verify
>> a functioning local APIC.  This results in APIC acceleration inhibited
>> on these kernels for reason APICV_INHIBIT_REASON_APIC_ID_MODIFIED.
>>
>> Allow the APICV_INHIBIT_REASON_APIC_ID_MODIFIED inhibit reason to be
>> cleared if/when the xAPIC ID is set back to the expected vcpu_id value.
>> This occurs on the second xAPIC ID write in verify_local_APIC().
>>
>> Fixes: 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
>> Signed-off-by: Greg Edwards <gedwards@ddn.com>
>> ---
>>  arch/x86/kvm/lapic.c | 7 ++++++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
>> index d7639d126e6c..4064d0af094d 100644
>> --- a/arch/x86/kvm/lapic.c
>> +++ b/arch/x86/kvm/lapic.c
>> @@ -2075,8 +2075,13 @@ static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
>>  	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
>>  		return;
>>
>> -	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
>> +	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id) {
>> +		/* Legacy kernels prior to 4399c03c6780 write APIC ID twice. */
>> +		if (!kvm_apicv_activated(kvm))
>> +			kvm_clear_apicv_inhibit(kvm,
>> +					APICV_INHIBIT_REASON_APIC_ID_MODIFIED);
>
> This sadly doesn't work because the inhibit is per-VM, i.e. will do the wrong
> thing if there are still vCPU's with different APIC IDs.

That is true.  Thanks for pointing that out.

> Does skipping the check if the APIC is disabled help[*]?  At a glance, I can't
> tell if the APIC is enabled/disabled at that point in time.  It's not a true fix,
> but it's a lot easier to backport if it remedies the issue.

It does not.  I did find your patch series when I started investigating
this, but the behavior was the same on that series.

> For a proper fix, this entire path should be moved to kvm_recalculate_apic_map()
> so that can can safely toggle the inhibit, e.g. the recalc helper already deals
> with multiple vCPUs changing their APIC state in parallel.  I don't think the fix
> will be too difficult to craft such that it's backport friendly, but it would need
> to be slotted into the series containing the aforementioned fix.

Thank you for the suggestion.  I will take a look, and start from your
series.
diff mbox series

Patch

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index d7639d126e6c..4064d0af094d 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2075,8 +2075,13 @@  static void kvm_lapic_xapic_id_updated(struct kvm_lapic *apic)
 	if (KVM_BUG_ON(apic_x2apic_mode(apic), kvm))
 		return;
 
-	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id)
+	if (kvm_xapic_id(apic) == apic->vcpu->vcpu_id) {
+		/* Legacy kernels prior to 4399c03c6780 write APIC ID twice. */
+		if (!kvm_apicv_activated(kvm))
+			kvm_clear_apicv_inhibit(kvm,
+					APICV_INHIBIT_REASON_APIC_ID_MODIFIED);
 		return;
+	}
 
 	kvm_set_apicv_inhibit(apic->vcpu->kvm, APICV_INHIBIT_REASON_APIC_ID_MODIFIED);
 }