mbox series

[v4,00/12] KVM: x86/mmu: refine memtype related mmu zap

Message ID 20230714064656.20147-1-yan.y.zhao@intel.com
Headers
Series KVM: x86/mmu: refine memtype related mmu zap |

Message

Yan Zhao July 14, 2023, 6:46 a.m. UTC 
  This series refines mmu zap caused by EPT memory type update when guest
MTRRs are honored.

Patches 1-5 revolve around utilizing helper functions to check if
KVM TDP honors guest MTRRs, TDP zaps and page fault max_level reduction
are now only targeted to TDPs that honor guest MTRRs.

-The 5th patch will trigger zapping of TDP leaf entries if non-coherent
 DMA devices count goes from 0 to 1 or from 1 to 0.

Patches 6-7 are fixes and patches 9-12 are optimizations for mmu zaps
when guest MTRRs are honored.
Those mmu zaps are intended to remove stale memtypes of TDP entries
caused by changes of guest MTRRs and CR0.CD and are usually triggered from
all vCPUs in bursts.

- The 6th patch places TDP zap to when CR0.CD toggles and when guest MTRRs
  update under CR0.CD=0.

- The 7th-8th patches refine KVM_X86_QUIRK_CD_NW_CLEARED by removing the
  IPAT bit in EPT memtype when CR0.CD=1 and guest MTRRs are honored.

- The 9th-11th patches are optimizations of the mmu zap when guest MTRRs
  are honored by serializing vCPUs' gfn zap requests and calculating of
  precise fine-grained ranges to zap.
  They are put in mtrr.c because the optimizations are related to when
  guest MTRRs are honored and because it requires to read guest MTRRs
  for fine-grained ranges.
  Calls to kvm_unmap_gfn_range() are not included into the optimization,
  because they are not triggered from all vCPUs in bursts and not all of
  them are blockable. They usually happen at memslot removal and thus do
  not affect the mmu zaps when guest MTRRs are honored. Also, current
  performance data shows that there's no observable performance difference
  to mmu zaps by turning on/off auto numa balancing triggered
  kvm_unmap_gfn_range().

- The 12th patch further convert kvm_zap_gfn_range() to use shared
  mmu_lock in TDP MMU. It can visibly help to reduce cost in contentions
  along with vCPUs number increases.

A reference performance data for last 7 patches as below:

Base1: base code before patch 6
Base2: Base 1 + patches 6 + 7 + 8
       patch 6: move TDP zaps from guest MTRRs update to CR0.CD toggling
       patch 7: drop IPAT in memtype when CD=1 for
                KVM_X86_QUIRK_CD_NW_CLEARED
       patch 8: entralize code to get CD=1 memtype when guest MTRRs are
                honored 

patch 9:  serialize gfn zap
patch 10: fine-grained gfn zap 
patch 11: split and zap in-slot gfn ranges only **
patch 12: convert gfn zap to use shared mmu_lock

Comments

Sean Christopherson Oct. 5, 2023, 1:29 a.m. UTC | #1 
On Fri, 14 Jul 2023 14:46:56 +0800, Yan Zhao wrote:
> This series refines mmu zap caused by EPT memory type update when guest
> MTRRs are honored.
> 
> Patches 1-5 revolve around utilizing helper functions to check if
> KVM TDP honors guest MTRRs, TDP zaps and page fault max_level reduction
> are now only targeted to TDPs that honor guest MTRRs.
> 
> [...]

Applied 1-5 and 7 to kvm-x86 mmu.  I squashed 1 and 2 as introducing helpers to
consolidate existing code without converting the existing code is wierd and
makes it unnecessarily impossible to properly test the helpers when they're
added.

I skipped 6, "move TDP zaps from guest MTRRs update to CR0.CD toggling", for
now as your performance numbers showed that it slowed down the guest even
though the number of zaps went down.  I'm definitely not against the patch, I
just don't want to risk regressing guest performance, i.e. I don't wantt to
take it without the rest of the series that takes advantage of the change.

I massaged a few shortlogs and changelogs, but didn't touch any code.  Holler
if anything looks funky.

Thanks much!

[1/5] KVM: x86/mmu: Add helpers to return if KVM honors guest MTRRs
      https://github.com/kvm-x86/linux/commit/6590a37e7ec6
[2/5] KVM: x86/mmu: Zap SPTEs when CR0.CD is toggled iff guest MTRRs are honored
      https://github.com/kvm-x86/linux/commit/c0ad4a14c5af
[3/5] KVM: x86/mmu: Zap SPTEs on MTRR update iff guest MTRRs are honored
      https://github.com/kvm-x86/linux/commit/a1596812cce1
[4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops
      https://github.com/kvm-x86/linux/commit/3c4955c04b95
[5/5] KVM: VMX: drop IPAT in memtype when CD=1 for KVM_X86_QUIRK_CD_NW_CLEARED
      https://github.com/kvm-x86/linux/commit/f7b4bcd501ef

--
https://github.com/kvm-x86/linux/tree/next
Kai Huang Oct. 5, 2023, 2:19 a.m. UTC | #2 
On Wed, 2023-10-04 at 18:29 -0700, Sean Christopherson wrote:
> [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops
>       https://github.com/kvm-x86/linux/commit/3c4955c04b95

Xap -> Zap? :-)

Apologize if I missed something.
Sean Christopherson Oct. 5, 2023, 2:28 a.m. UTC | #3 
On Thu, Oct 05, 2023, Kai Huang wrote:
> On Wed, 2023-10-04 at 18:29 -0700, Sean Christopherson wrote:
> > [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops
> >       https://github.com/kvm-x86/linux/commit/3c4955c04b95
> 
> Xap -> Zap? :-)

Dagnabbit, I tried to capitalize z => Z and hit the wrong key.  I'll fixup.

Thanks Kai!
Sean Christopherson Oct. 6, 2023, 12:50 a.m. UTC | #4 
On Wed, Oct 04, 2023, Sean Christopherson wrote:
> On Thu, Oct 05, 2023, Kai Huang wrote:
> > On Wed, 2023-10-04 at 18:29 -0700, Sean Christopherson wrote:
> > > [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops
> > >       https://github.com/kvm-x86/linux/commit/3c4955c04b95
> > 
> > Xap -> Zap? :-)
> 
> Dagnabbit, I tried to capitalize z => Z and hit the wrong key.  I'll fixup.

LOL, the real irony is that this particular patch also has this in the changelog:

  [sean: fix misspelled words in comment and changelog]

Anyways, fixed.  New hashes are:

[4/5] KVM: x86/mmu: Zap KVM TDP when noncoherent DMA assignment starts/stops
      https://github.com/kvm-x86/linux/commit/539c103e2a13
[5/5] KVM: VMX: drop IPAT in memtype when CD=1 for KVM_X86_QUIRK_CD_NW_CLEARED
      https://github.com/kvm-x86/linux/commit/10ed442fefdd