diff mbox series

[v6,4/4] vfio: convey kvm that the vfio-pci device is wc safe

Message ID 20240207204652.22954-5-ankita@nvidia.com
State New
Headers
Series kvm: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory |

Commit Message

Ankit Agrawal Feb. 7, 2024, 8:46 p.m. UTC 
  From: Ankit Agrawal <ankita@nvidia.com>

The code to map the MMIO in S2 as NormalNC is enabled when conveyed
that the device is WC safe using a new flag VM_VFIO_ALLOW_WC.

Make vfio-pci set the VM_VFIO_ALLOW_WC flag.

This could be extended to other devices in the future once that
is deemed safe.

Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Catalin Marinas Feb. 8, 2024, 2:53 p.m. UTC | #1 
+ David H

On Thu, Feb 08, 2024 at 02:16:52AM +0530, ankita@nvidia.com wrote:
> From: Ankit Agrawal <ankita@nvidia.com>
> 
> The code to map the MMIO in S2 as NormalNC is enabled when conveyed
> that the device is WC safe using a new flag VM_VFIO_ALLOW_WC.
> 
> Make vfio-pci set the VM_VFIO_ALLOW_WC flag.
> 
> This could be extended to other devices in the future once that
> is deemed safe.
> 
> Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
> Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> Acked-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
>  drivers/vfio/pci/vfio_pci_core.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 1cbc990d42e0..c3f95ec7fc3a 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -1863,7 +1863,8 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
>  	 * See remap_pfn_range(), called from vfio_pci_fault() but we can't
>  	 * change vm_flags within the fault handler.  Set them now.
>  	 */
> -	vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
> +	vm_flags_set(vma, VM_VFIO_ALLOW_WC | VM_IO | VM_PFNMAP |
> +			VM_DONTEXPAND | VM_DONTDUMP);
>  	vma->vm_ops = &vfio_pci_mmap_ops;
>  
>  	return 0;

Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Alex Williamson Feb. 8, 2024, 5:30 p.m. UTC | #2 
On Thu, 8 Feb 2024 02:16:52 +0530
<ankita@nvidia.com> wrote:

> From: Ankit Agrawal <ankita@nvidia.com>
> 
> The code to map the MMIO in S2 as NormalNC is enabled when conveyed
> that the device is WC safe using a new flag VM_VFIO_ALLOW_WC.
> 
> Make vfio-pci set the VM_VFIO_ALLOW_WC flag.
> 
> This could be extended to other devices in the future once that
> is deemed safe.
> 
> Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
> Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> Acked-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
>  drivers/vfio/pci/vfio_pci_core.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 1cbc990d42e0..c3f95ec7fc3a 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -1863,7 +1863,8 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
>  	 * See remap_pfn_range(), called from vfio_pci_fault() but we can't
>  	 * change vm_flags within the fault handler.  Set them now.
>  	 */
> -	vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
> +	vm_flags_set(vma, VM_VFIO_ALLOW_WC | VM_IO | VM_PFNMAP |
> +			VM_DONTEXPAND | VM_DONTDUMP);
>  	vma->vm_ops = &vfio_pci_mmap_ops;
>  
>  	return 0;

The comment above this is justifying the flags as equivalent to those
set by the remap_pfn_range() path.  That's no longer the case and the
additional flag needs to be described there.

I'm honestly surprised that a vm_flags bit named so specifically for a
single driver has gotten this far.  It seems like the vfio use case for
this and associated FUD for other use cases could all be encompassed in
the comment where the bit is defined and we could use a name like
VM_ALLOW_ANY_UNCACHED or VM_IO_ANY.  Thanks,

Alex
Jason Gunthorpe Feb. 8, 2024, 5:54 p.m. UTC | #3 
On Thu, Feb 08, 2024 at 10:30:22AM -0700, Alex Williamson wrote:
> On Thu, 8 Feb 2024 02:16:52 +0530
> <ankita@nvidia.com> wrote:
> 
> > From: Ankit Agrawal <ankita@nvidia.com>
> > 
> > The code to map the MMIO in S2 as NormalNC is enabled when conveyed
> > that the device is WC safe using a new flag VM_VFIO_ALLOW_WC.
> > 
> > Make vfio-pci set the VM_VFIO_ALLOW_WC flag.
> > 
> > This could be extended to other devices in the future once that
> > is deemed safe.
> > 
> > Signed-off-by: Ankit Agrawal <ankita@nvidia.com>
> > Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> > Acked-by: Jason Gunthorpe <jgg@nvidia.com>
> > ---
> >  drivers/vfio/pci/vfio_pci_core.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> > index 1cbc990d42e0..c3f95ec7fc3a 100644
> > --- a/drivers/vfio/pci/vfio_pci_core.c
> > +++ b/drivers/vfio/pci/vfio_pci_core.c
> > @@ -1863,7 +1863,8 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
> >  	 * See remap_pfn_range(), called from vfio_pci_fault() but we can't
> >  	 * change vm_flags within the fault handler.  Set them now.
> >  	 */
> > -	vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
> > +	vm_flags_set(vma, VM_VFIO_ALLOW_WC | VM_IO | VM_PFNMAP |
> > +			VM_DONTEXPAND | VM_DONTDUMP);
> >  	vma->vm_ops = &vfio_pci_mmap_ops;
> >  
> >  	return 0;
> 
> The comment above this is justifying the flags as equivalent to those
> set by the remap_pfn_range() path.  That's no longer the case and the
> additional flag needs to be described there.
> 
> I'm honestly surprised that a vm_flags bit named so specifically for a
> single driver has gotten this far.  

IIRC there was a small bike shed and this is what we came up
with. Realistically it should not be used by anything but VFIO and KVM
together. Generic names do sometimes invite abuse :)

> It seems like the vfio use case for
> this and associated FUD for other use cases could all be encompassed
> in

I think Ankit is talking about vfio-platform drivers by "other
devices".

This is largely why it exists at all, there is a fear that the non-PCI
VFIO devices will not be implemented the same as the PCI devices. If
any platform devices have workloads that require WC and have HW that
is safe then they will set the flag somehow in the vfio platform
drivers.

> the comment where the bit is defined and we could use a name like
> VM_ALLOW_ANY_UNCACHED or VM_IO_ANY.  Thanks,

I'd pick VM_ALLOW_ANY_UNCACHED of those two

Thanks,
Jason
Ankit Agrawal Feb. 9, 2024, 2:02 p.m. UTC | #4 
> The comment above this is justifying the flags as equivalent to those
> set by the remap_pfn_range() path.  That's no longer the case and the
> additional flag needs to be described there.

Ack.

>> the comment where the bit is defined and we could use a name like
>> VM_ALLOW_ANY_UNCACHED or VM_IO_ANY.  Thanks,
>
> I'd pick VM_ALLOW_ANY_UNCACHED of those two

If there is consensus on this name, I'll make the change
s/VM_VFIO_ALLOW_WC/VM_ALLOW_ANY_UNCACHED.
diff mbox series

Patch

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 1cbc990d42e0..c3f95ec7fc3a 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1863,7 +1863,8 @@  int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
 	 * See remap_pfn_range(), called from vfio_pci_fault() but we can't
 	 * change vm_flags within the fault handler.  Set them now.
 	 */
-	vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
+	vm_flags_set(vma, VM_VFIO_ALLOW_WC | VM_IO | VM_PFNMAP |
+			VM_DONTEXPAND | VM_DONTDUMP);
 	vma->vm_ops = &vfio_pci_mmap_ops;
 
 	return 0;