[v4,02/17] iommu: Add nested domain support
Commit Message
From: Lu Baolu <baolu.lu@linux.intel.com>
Introduce a new domain type for a user I/O page table, which is nested on
top of another user space address represented by a UNMANAGED domain. The
mappings of a nested domain are managed by user space software, so it is
not necessary to have map/unmap callbacks. But the updates of the PTEs in
the nested page table will be propagated to the hardware caches on both
IOMMU (IOTLB) and devices (DevTLB/ATC).
A nested domain is allocated by the domain_alloc_user op, and attached to
a device through the existing iommu_attach_device/group() interfaces.
Add a new domain op cache_invalidate_user for the userspace to flush the
hardware caches for a nested domain through iommufd. No wrapper for it,
as it's only supposed to be used by iommufd.
Pass in invalidation requests to the cache_invalidate_user op, in form of
a user data array that conatins a number of invalidation entries. Add an
iommu_user_data_array struct and an iommu_copy_user_data_from_array helper
for iommu drivers to walk through the invalidation request array and fetch
the data entry inside.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Co-developed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Yi Liu <yi.l.liu@intel.com>
---
include/linux/iommu.h | 59 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 59 insertions(+)
Comments
On Thu, Sep 21, 2023 at 12:51:23AM -0700, Yi Liu wrote:
> From: Lu Baolu <baolu.lu@linux.intel.com>
>
> Introduce a new domain type for a user I/O page table, which is nested on
> top of another user space address represented by a UNMANAGED
> domain. The
Lets start using the world PAGING whenever you want to type
UNMANAGED. I'm trying to get rid of UNMANAGED.
> @@ -241,6 +245,21 @@ struct iommu_user_data {
> size_t len;
> };
>
> +/**
> + * struct iommu_user_data_array - iommu driver specific user space data array
> + * @uptr: Pointer to the user buffer array for copy_from_user()
> + * @entry_len: The fixed-width length of a entry in the array, in bytes
> + * @entry_num: The number of total entries in the array
> + *
> + * A array having a @entry_num number of @entry_len sized entries, each entry is
> + * user space data, i.e. an uAPI that is defined in include/uapi/linux/iommufd.h
> + */
> +struct iommu_user_data_array {
> + void __user *uptr;
> + size_t entry_len;
> + int entry_num;
> +};
Ditto about iommu-driver.h for most of this stuff
> +
> /**
> * iommu_copy_user_data - Copy iommu driver specific user space data
> * @dst_data: Pointer to an iommu driver specific user data that is defined in
> @@ -263,6 +282,34 @@ static inline int iommu_copy_user_data(void *dst_data,
> src_data->uptr, src_data->len);
> }
>
> +/**
> + * iommu_copy_user_data_from_array - Copy iommu driver specific user space data
> + * from an iommu_user_data_array input
> + * @dst_data: Pointer to an iommu driver specific user data that is defined in
> + * include/uapi/linux/iommufd.h
> + * @src_data: Pointer to a struct iommu_user_data_array for user space data array
> + * @index: Index to offset the location in the array to copy user data from
> + * @data_len: Length of current user data structure, i.e. sizeof(struct _dst)
> + * @min_len: Initial length of user data structure for backward compatibility.
> + * This should be offsetofend using the last member in the user data
> + * struct that was initially added to include/uapi/linux/iommufd.h
> + */
> +static inline int
> +iommu_copy_user_data_from_array(void *dst_data,
> + const struct iommu_user_data_array *src_array,
> + int index, size_t data_len, size_t min_len)
Index should be 'unsigned int'
Jason
On 2023/10/11 01:21, Jason Gunthorpe wrote:
> On Thu, Sep 21, 2023 at 12:51:23AM -0700, Yi Liu wrote:
>> From: Lu Baolu <baolu.lu@linux.intel.com>
>>
>> Introduce a new domain type for a user I/O page table, which is nested on
>> top of another user space address represented by a UNMANAGED
>> domain. The
>
> Lets start using the world PAGING whenever you want to type
> UNMANAGED. I'm trying to get rid of UNMANAGED.
sure.
>> @@ -241,6 +245,21 @@ struct iommu_user_data {
>> size_t len;
>> };
>>
>> +/**
>> + * struct iommu_user_data_array - iommu driver specific user space data array
>> + * @uptr: Pointer to the user buffer array for copy_from_user()
>> + * @entry_len: The fixed-width length of a entry in the array, in bytes
>> + * @entry_num: The number of total entries in the array
>> + *
>> + * A array having a @entry_num number of @entry_len sized entries, each entry is
>> + * user space data, i.e. an uAPI that is defined in include/uapi/linux/iommufd.h
>> + */
>> +struct iommu_user_data_array {
>> + void __user *uptr;
>> + size_t entry_len;
>> + int entry_num;
>> +};
>
> Ditto about iommu-driver.h for most of this stuff
ack.
>> +
>> /**
>> * iommu_copy_user_data - Copy iommu driver specific user space data
>> * @dst_data: Pointer to an iommu driver specific user data that is defined in
>> @@ -263,6 +282,34 @@ static inline int iommu_copy_user_data(void *dst_data,
>> src_data->uptr, src_data->len);
>> }
>>
>> +/**
>> + * iommu_copy_user_data_from_array - Copy iommu driver specific user space data
>> + * from an iommu_user_data_array input
>> + * @dst_data: Pointer to an iommu driver specific user data that is defined in
>> + * include/uapi/linux/iommufd.h
>> + * @src_data: Pointer to a struct iommu_user_data_array for user space data array
>> + * @index: Index to offset the location in the array to copy user data from
>> + * @data_len: Length of current user data structure, i.e. sizeof(struct _dst)
>> + * @min_len: Initial length of user data structure for backward compatibility.
>> + * This should be offsetofend using the last member in the user data
>> + * struct that was initially added to include/uapi/linux/iommufd.h
>> + */
>> +static inline int
>> +iommu_copy_user_data_from_array(void *dst_data,
>> + const struct iommu_user_data_array *src_array,
>> + int index, size_t data_len, size_t min_len)
>
> Index should be 'unsigned int'
yes.
On Thu, Oct 12, 2023 at 05:13:59PM +0800, Yi Liu wrote:
> > > @@ -241,6 +245,21 @@ struct iommu_user_data {
> > > size_t len;
> > > };
> > >
> > > +/**
> > > + * struct iommu_user_data_array - iommu driver specific user space data array
> > > + * @uptr: Pointer to the user buffer array for copy_from_user()
> > > + * @entry_len: The fixed-width length of a entry in the array, in bytes
> > > + * @entry_num: The number of total entries in the array
> > > + *
> > > + * A array having a @entry_num number of @entry_len sized entries, each entry is
> > > + * user space data, i.e. an uAPI that is defined in include/uapi/linux/iommufd.h
> > > + */
> > > +struct iommu_user_data_array {
> > > + void __user *uptr;
> > > + size_t entry_len;
> > > + int entry_num;
> > > +};
> >
> > Ditto about iommu-driver.h for most of this stuff
>
> ack.
FYI, I aligned with Jason in another conversation that we can
defer this iommu-driver.h thing since there's no more need to
include the previous uapi header in iommu.h
Thanks
Nic
@@ -66,6 +66,9 @@ struct iommu_domain_geometry {
#define __IOMMU_DOMAIN_SVA (1U << 4) /* Shared process address space */
+#define __IOMMU_DOMAIN_NESTED (1U << 5) /* User-managed address space nested
+ on a stage-2 translation */
+
#define IOMMU_DOMAIN_ALLOC_FLAGS ~__IOMMU_DOMAIN_DMA_FQ
/*
* This are the possible domain-types
@@ -92,6 +95,7 @@ struct iommu_domain_geometry {
__IOMMU_DOMAIN_DMA_API | \
__IOMMU_DOMAIN_DMA_FQ)
#define IOMMU_DOMAIN_SVA (__IOMMU_DOMAIN_SVA)
+#define IOMMU_DOMAIN_NESTED (__IOMMU_DOMAIN_NESTED)
struct iommu_domain {
unsigned type;
@@ -241,6 +245,21 @@ struct iommu_user_data {
size_t len;
};
+/**
+ * struct iommu_user_data_array - iommu driver specific user space data array
+ * @uptr: Pointer to the user buffer array for copy_from_user()
+ * @entry_len: The fixed-width length of a entry in the array, in bytes
+ * @entry_num: The number of total entries in the array
+ *
+ * A array having a @entry_num number of @entry_len sized entries, each entry is
+ * user space data, i.e. an uAPI that is defined in include/uapi/linux/iommufd.h
+ */
+struct iommu_user_data_array {
+ void __user *uptr;
+ size_t entry_len;
+ int entry_num;
+};
+
/**
* iommu_copy_user_data - Copy iommu driver specific user space data
* @dst_data: Pointer to an iommu driver specific user data that is defined in
@@ -263,6 +282,34 @@ static inline int iommu_copy_user_data(void *dst_data,
src_data->uptr, src_data->len);
}
+/**
+ * iommu_copy_user_data_from_array - Copy iommu driver specific user space data
+ * from an iommu_user_data_array input
+ * @dst_data: Pointer to an iommu driver specific user data that is defined in
+ * include/uapi/linux/iommufd.h
+ * @src_data: Pointer to a struct iommu_user_data_array for user space data array
+ * @index: Index to offset the location in the array to copy user data from
+ * @data_len: Length of current user data structure, i.e. sizeof(struct _dst)
+ * @min_len: Initial length of user data structure for backward compatibility.
+ * This should be offsetofend using the last member in the user data
+ * struct that was initially added to include/uapi/linux/iommufd.h
+ */
+static inline int
+iommu_copy_user_data_from_array(void *dst_data,
+ const struct iommu_user_data_array *src_array,
+ int index, size_t data_len, size_t min_len)
+{
+ struct iommu_user_data src_data;
+
+ if (WARN_ON(!src_array || index >= src_array->entry_num))
+ return -EINVAL;
+ if (!src_array->entry_num)
+ return -EINVAL;
+ src_data.uptr = src_array->uptr + src_array->entry_len * index;
+ src_data.len = src_array->entry_len;
+ return iommu_copy_user_data(dst_data, &src_data, data_len, min_len);
+}
+
/**
* struct iommu_ops - iommu ops and capabilities
* @capable: check capability
@@ -374,6 +421,15 @@ struct iommu_ops {
* @iotlb_sync_map: Sync mappings created recently using @map to the hardware
* @iotlb_sync: Flush all queued ranges from the hardware TLBs and empty flush
* queue
+ * @cache_invalidate_user: Flush hardware cache for user space IO page table.
+ * The @domain must be IOMMU_DOMAIN_NESTED. The @array
+ * passes in the cache invalidation requests, in form
+ * of a driver data structure. The driver must update
+ * array->entry_num to report the number of handled
+ * invalidation requests. The 32-bit @error_code can
+ * forward a driver specific error code to user space.
+ * Both the driver data structure and the error code
+ * must be defined in include/uapi/linux/iommufd.h
* @iova_to_phys: translate iova to physical address
* @enforce_cache_coherency: Prevent any kind of DMA from bypassing IOMMU_CACHE,
* including no-snoop TLPs on PCIe or other platform
@@ -403,6 +459,9 @@ struct iommu_domain_ops {
size_t size);
void (*iotlb_sync)(struct iommu_domain *domain,
struct iommu_iotlb_gather *iotlb_gather);
+ int (*cache_invalidate_user)(struct iommu_domain *domain,
+ struct iommu_user_data_array *array,
+ u32 *error_code);
phys_addr_t (*iova_to_phys)(struct iommu_domain *domain,
dma_addr_t iova);