[v2,00/11] arm64: ptdump: View the second stage page-tables

  Hi,

This can be used as a debugging tool for dumping the second stage
page-tables under debugfs.

From the previous feedback I re-worked the series and added support for
guest page-tables dumping under VHE & nVHE configuration. I extended the
list of reviewers as I missed the interested parties in the first round. 

When CONFIG_NVHE_EL2_PTDUMP_DEBUGFS is enabled under pKVM environment,
ptdump registers the 'host_stage2_kernel_page_tables' entry with debugfs.
Guests are registering a file named '%u_guest_stage2_page_tables' when
they are created. 

This allows us to dump the host stage-2 page-tables with the following command:
cat /sys/kernel/debug/host_stage2_kernel_page_tables.

The output is showing the entries in the following format:
<IPA range> <size> <descriptor type> <access permissions> <mem_attributes>

The tool interprets the pKVM ownership annotation stored in the invalid
entries and dumps to the console the ownership information. To be able
to access the host stage-2 page-tables from the kernel, a new hypervisor
call was introduced which allows us to snapshot the page-tables in a host
provided buffer. The hypervisor call is hidden behind CONFIG_NVHE_EL2_DEBUG
as this should be used under debugging environment.

Link to the first version:
https://lore.kernel.org/all/20230927112517.2631674-1-sebastianene@google.com/

Changelog:
  v1 -> v2:
  * use the stage-2 pagetable walker for dumping descriptors instead of
    the one provided by ptdump.

  * support for guests pagetables dumping under VHE/nVHE non-protected

Thanks,


Sebastian Ene (11):
  KVM: arm64: Add snap shooting the host stage-2 pagetables
  arm64: ptdump: Use the mask from the state structure
  arm64: ptdump: Add the walker function to the ptdump info structure
  KVM: arm64: Move pagetable definitions to common header
  arm64: ptdump: Introduce stage-2 pagetables format description
  arm64: ptdump: Add hooks on debugfs file operations
  arm64: ptdump: Register a debugfs entry for the host stage-2
    page-tables
  arm64: ptdump: Parse the host stage-2 page-tables from the snapshot
  arm64: ptdump: Interpret memory attributes based on runtime
    configuration
  arm64: ptdump: Interpret pKVM ownership annotations
  arm64: ptdump: Add support for guest stage-2 pagetables dumping

 arch/arm64/include/asm/kvm_asm.h              |   1 +
 arch/arm64/include/asm/kvm_pgtable.h          |  85 +++
 arch/arm64/include/asm/ptdump.h               |  27 +-
 arch/arm64/kvm/Kconfig                        |  12 +
 arch/arm64/kvm/hyp/include/nvhe/mem_protect.h |   8 +-
 arch/arm64/kvm/hyp/nvhe/hyp-main.c            |  18 +
 arch/arm64/kvm/hyp/nvhe/mem_protect.c         | 103 ++++
 arch/arm64/kvm/hyp/pgtable.c                  |  98 ++--
 arch/arm64/kvm/mmu.c                          |   3 +
 arch/arm64/mm/ptdump.c                        | 487 +++++++++++++++++-
 arch/arm64/mm/ptdump_debugfs.c                |  42 +-
 11 files changed, 822 insertions(+), 62 deletions(-)
  

On Thu, Oct 19, 2023 at 02:40:21PM +0000, Sebastian Ene wrote:
> Hi,
> 
> This can be used as a debugging tool for dumping the second stage
> page-tables under debugfs.
> 
> From the previous feedback I re-worked the series and added support for
> guest page-tables dumping under VHE & nVHE configuration. I extended the
> list of reviewers as I missed the interested parties in the first round. 
> 
> When CONFIG_NVHE_EL2_PTDUMP_DEBUGFS is enabled under pKVM environment,
> ptdump registers the 'host_stage2_kernel_page_tables' entry with debugfs.
> Guests are registering a file named '%u_guest_stage2_page_tables' when
> they are created. 

I believe guests entries should be also available for nVHE and VHE.

> 
> This allows us to dump the host stage-2 page-tables with the following command:
> cat /sys/kernel/debug/host_stage2_kernel_page_tables.

As it needs the debugfs anyway, this should probably live in the kvm/ debugfs
folder, while the VMs ptdump should be placed in their respective folder.

This is quite easy, you should get access to the global kvm_debugfs_dir and
struct kvm->debugfs_dentry.

> 
> The output is showing the entries in the following format:
> <IPA range> <size> <descriptor type> <access permissions> <mem_attributes>
> 
> The tool interprets the pKVM ownership annotation stored in the invalid
> entries and dumps to the console the ownership information. To be able
> to access the host stage-2 page-tables from the kernel, a new hypervisor
> call was introduced which allows us to snapshot the page-tables in a host
> provided buffer. The hypervisor call is hidden behind CONFIG_NVHE_EL2_DEBUG
> as this should be used under debugging environment.
> 
> Link to the first version:
> https://lore.kernel.org/all/20230927112517.2631674-1-sebastianene@google.com/
> 
> Changelog:
>   v1 -> v2:
>   * use the stage-2 pagetable walker for dumping descriptors instead of
>     the one provided by ptdump.
> 
>   * support for guests pagetables dumping under VHE/nVHE non-protected
> 
> Thanks,
> 
> 
> Sebastian Ene (11):
>   KVM: arm64: Add snap shooting the host stage-2 pagetables
>   arm64: ptdump: Use the mask from the state structure
>   arm64: ptdump: Add the walker function to the ptdump info structure
>   KVM: arm64: Move pagetable definitions to common header
>   arm64: ptdump: Introduce stage-2 pagetables format description
>   arm64: ptdump: Add hooks on debugfs file operations
>   arm64: ptdump: Register a debugfs entry for the host stage-2
>     page-tables
>   arm64: ptdump: Parse the host stage-2 page-tables from the snapshot
>   arm64: ptdump: Interpret memory attributes based on runtime
>     configuration
>   arm64: ptdump: Interpret pKVM ownership annotations
>   arm64: ptdump: Add support for guest stage-2 pagetables dumping
> 
>  arch/arm64/include/asm/kvm_asm.h              |   1 +
>  arch/arm64/include/asm/kvm_pgtable.h          |  85 +++
>  arch/arm64/include/asm/ptdump.h               |  27 +-
>  arch/arm64/kvm/Kconfig                        |  12 +
>  arch/arm64/kvm/hyp/include/nvhe/mem_protect.h |   8 +-
>  arch/arm64/kvm/hyp/nvhe/hyp-main.c            |  18 +
>  arch/arm64/kvm/hyp/nvhe/mem_protect.c         | 103 ++++
>  arch/arm64/kvm/hyp/pgtable.c                  |  98 ++--
>  arch/arm64/kvm/mmu.c                          |   3 +
>  arch/arm64/mm/ptdump.c                        | 487 +++++++++++++++++-
>  arch/arm64/mm/ptdump_debugfs.c                |  42 +-
>  11 files changed, 822 insertions(+), 62 deletions(-)
> 
> -- 
> 2.42.0.655.g421f12c284-goog
>
  

On Fri, Oct 20, 2023 at 09:19:33AM +0100, Vincent Donnefort wrote:
> On Thu, Oct 19, 2023 at 02:40:21PM +0000, Sebastian Ene wrote:
> > Hi,
> > 
> > This can be used as a debugging tool for dumping the second stage
> > page-tables under debugfs.
> > 
> > From the previous feedback I re-worked the series and added support for
> > guest page-tables dumping under VHE & nVHE configuration. I extended the
> > list of reviewers as I missed the interested parties in the first round. 
> > 
> > When CONFIG_NVHE_EL2_PTDUMP_DEBUGFS is enabled under pKVM environment,
> > ptdump registers the 'host_stage2_kernel_page_tables' entry with debugfs.
> > Guests are registering a file named '%u_guest_stage2_page_tables' when
> > they are created. 

Hi,

> 
> I believe guests entries should be also available for nVHE and VHE.
> 

Yes, we support dumping the guest stage-2 pagetables with this under
both modes. The host stage-2 is available only in
kvm.arm.mode="protected".

> > 
> > This allows us to dump the host stage-2 page-tables with the following command:
> > cat /sys/kernel/debug/host_stage2_kernel_page_tables.
> 
> As it needs the debugfs anyway, this should probably live in the kvm/ debugfs
> folder, while the VMs ptdump should be placed in their respective folder.
> 
> This is quite easy, you should get access to the global kvm_debugfs_dir and
> struct kvm->debugfs_dentry.
>

Right, I was thinking to place them under kvm/ debugfs entry but then I
noticed that ptdump files are not registered under this path.

> > 
> > The output is showing the entries in the following format:
> > <IPA range> <size> <descriptor type> <access permissions> <mem_attributes>
> > 
> > The tool interprets the pKVM ownership annotation stored in the invalid
> > entries and dumps to the console the ownership information. To be able
> > to access the host stage-2 page-tables from the kernel, a new hypervisor
> > call was introduced which allows us to snapshot the page-tables in a host
> > provided buffer. The hypervisor call is hidden behind CONFIG_NVHE_EL2_DEBUG
> > as this should be used under debugging environment.
> > 
> > Link to the first version:
> > https://lore.kernel.org/all/20230927112517.2631674-1-sebastianene@google.com/
> > 
> > Changelog:
> >   v1 -> v2:
> >   * use the stage-2 pagetable walker for dumping descriptors instead of
> >     the one provided by ptdump.
> > 
> >   * support for guests pagetables dumping under VHE/nVHE non-protected
> > 
> > Thanks,
> > 
> > 
> > Sebastian Ene (11):
> >   KVM: arm64: Add snap shooting the host stage-2 pagetables
> >   arm64: ptdump: Use the mask from the state structure
> >   arm64: ptdump: Add the walker function to the ptdump info structure
> >   KVM: arm64: Move pagetable definitions to common header
> >   arm64: ptdump: Introduce stage-2 pagetables format description
> >   arm64: ptdump: Add hooks on debugfs file operations
> >   arm64: ptdump: Register a debugfs entry for the host stage-2
> >     page-tables
> >   arm64: ptdump: Parse the host stage-2 page-tables from the snapshot
> >   arm64: ptdump: Interpret memory attributes based on runtime
> >     configuration
> >   arm64: ptdump: Interpret pKVM ownership annotations
> >   arm64: ptdump: Add support for guest stage-2 pagetables dumping
> > 
> >  arch/arm64/include/asm/kvm_asm.h              |   1 +
> >  arch/arm64/include/asm/kvm_pgtable.h          |  85 +++
> >  arch/arm64/include/asm/ptdump.h               |  27 +-
> >  arch/arm64/kvm/Kconfig                        |  12 +
> >  arch/arm64/kvm/hyp/include/nvhe/mem_protect.h |   8 +-
> >  arch/arm64/kvm/hyp/nvhe/hyp-main.c            |  18 +
> >  arch/arm64/kvm/hyp/nvhe/mem_protect.c         | 103 ++++
> >  arch/arm64/kvm/hyp/pgtable.c                  |  98 ++--
> >  arch/arm64/kvm/mmu.c                          |   3 +
> >  arch/arm64/mm/ptdump.c                        | 487 +++++++++++++++++-
> >  arch/arm64/mm/ptdump_debugfs.c                |  42 +-
> >  11 files changed, 822 insertions(+), 62 deletions(-)
> > 
> > -- 
> > 2.42.0.655.g421f12c284-goog
> >