[v3,2/7] iommu: Decouple iommu_present() from bus ops

  Much as I'd like to remove iommu_present(), the final remaining users
are proving stubbornly difficult to clean up, so kick that can down
the road and just rework it to preserve the current behaviour without
depending on bus ops.

Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>

---

v3: Tweak to use the ops-based check rather than group-based, to
    properly match the existing behaviour
---
 drivers/iommu/iommu.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
  

On Fri, Sep 15, 2023 at 05:58:06PM +0100, Robin Murphy wrote:
> Much as I'd like to remove iommu_present(), the final remaining users
> are proving stubbornly difficult to clean up, so kick that can down
> the road and just rework it to preserve the current behaviour without
> depending on bus ops.
> 
> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
> 
> ---
> 
> v3: Tweak to use the ops-based check rather than group-based, to
>     properly match the existing behaviour
> ---
>  drivers/iommu/iommu.c | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 4566d0001cd3..2f29ee9dea64 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -1907,9 +1907,24 @@ int bus_iommu_probe(const struct bus_type *bus)
>  	return 0;
>  }
>  
> +static int __iommu_present(struct device *dev, void *unused)
> +{
> +	return dev_has_iommu(dev);
> +}

This is not locked right..

Rather than perpetuate that, can we fix the two callers instead?

Maybe this for mtk:

diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
index 93552d76b6e778..e7fe0e6f27de85 100644
--- a/drivers/gpu/drm/mediatek/mtk_drm_drv.c
+++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
@@ -500,6 +500,8 @@ static int mtk_drm_kms_init(struct drm_device *drm)
                dev_err(drm->dev, "Need at least one OVL device\n");
                goto err_component_unbind;
        }
+       if (!device_iommu_mapped(dma_dev))
+               return -EPROBE_DEFER;
 
        for (i = 0; i < private->data->mmsys_dev_num; i++)
                private->all_drm_private[i]->dma_dev = dma_dev;
@@ -583,9 +585,6 @@ static int mtk_drm_bind(struct device *dev)
        struct drm_device *drm;
        int ret, i;
 
-       if (!iommu_present(&platform_bus_type))
-               return -EPROBE_DEFER;
-
        pdev = of_find_device_by_node(private->mutex_node);
        if (!pdev) {
                dev_err(dev, "Waiting for disp-mutex device %pOF\n",


? It doesn't seem to use the iommu API so I guess all it is doing is
trying to fix some kind of probe ordering issue? Maybe the probe
ordering issue is already gone and we can just delete the check?

And tegra:

	if (host1x_drm_wants_iommu(dev) && iommu_present(&platform_bus_type)) {
		tegra->domain = iommu_domain_alloc(&platform_bus_type);
		if (!tegra->domain) {

Lets do the same:

	if (host1x_drm_wants_iommu(dev) && device_iommu_mapped(dev->dev.parent)) {

?

Alternatively how about:

bool iommu_present(void)
{
	bool ret;

	spin_lock(&iommu_device_lock);
	ret = !list_empty(&iommu_device_list);
	spin_unlock(&iommu_device_lock);
	return ret;
}
EXPORT_SYMBOL_GPL(iommu_present);

Since neither of the two users is really needing anything more than that?

Jason
  

On 2023-09-18 18:12, Jason Gunthorpe wrote:
> On Fri, Sep 15, 2023 at 05:58:06PM +0100, Robin Murphy wrote:
>> Much as I'd like to remove iommu_present(), the final remaining users
>> are proving stubbornly difficult to clean up, so kick that can down
>> the road and just rework it to preserve the current behaviour without
>> depending on bus ops.
>>
>> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
>> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
>> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
>>
>> ---
>>
>> v3: Tweak to use the ops-based check rather than group-based, to
>>      properly match the existing behaviour
>> ---
>>   drivers/iommu/iommu.c | 17 ++++++++++++++++-
>>   1 file changed, 16 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 4566d0001cd3..2f29ee9dea64 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -1907,9 +1907,24 @@ int bus_iommu_probe(const struct bus_type *bus)
>>   	return 0;
>>   }
>>   
>> +static int __iommu_present(struct device *dev, void *unused)
>> +{
>> +	return dev_has_iommu(dev);
>> +}
> 
> This is not locked right..

Urgh, yes, I suppose technically this walk could run in parallel with 
the bus_iommu_probe() of another IOMMU instance that our caller here 
doesn't depend on. I agree that's suboptimal, even if it shouldn't 
happen in practice for the remaining in-tree callers.

> Rather than perpetuate that, can we fix the two callers instead?
> 
> Maybe this for mtk:
> 
> diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
> index 93552d76b6e778..e7fe0e6f27de85 100644
> --- a/drivers/gpu/drm/mediatek/mtk_drm_drv.c
> +++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c
> @@ -500,6 +500,8 @@ static int mtk_drm_kms_init(struct drm_device *drm)
>                  dev_err(drm->dev, "Need at least one OVL device\n");
>                  goto err_component_unbind;
>          }
> +       if (!device_iommu_mapped(dma_dev))
> +               return -EPROBE_DEFER;
>   
>          for (i = 0; i < private->data->mmsys_dev_num; i++)
>                  private->all_drm_private[i]->dma_dev = dma_dev;
> @@ -583,9 +585,6 @@ static int mtk_drm_bind(struct device *dev)
>          struct drm_device *drm;
>          int ret, i;
>   
> -       if (!iommu_present(&platform_bus_type))
> -               return -EPROBE_DEFER;
> -
>          pdev = of_find_device_by_node(private->mutex_node);
>          if (!pdev) {
>                  dev_err(dev, "Waiting for disp-mutex device %pOF\n",
> 
> 
> ? It doesn't seem to use the iommu API so I guess all it is doing is
> trying to fix some kind of probe ordering issue? Maybe the probe
> ordering issue is already gone and we can just delete the check?

As I've said before, the correct fix for this one is [1]. I've sent it 
twice now, it just gets ignored :(

> And tegra:
> 
> 	if (host1x_drm_wants_iommu(dev) && iommu_present(&platform_bus_type)) {
> 		tegra->domain = iommu_domain_alloc(&platform_bus_type);
> 		if (!tegra->domain) {
> 
> Lets do the same:
> 
> 	if (host1x_drm_wants_iommu(dev) && device_iommu_mapped(dev->dev.parent)) {
> 
> ?

IIRC the problem here is that the Host1x (or GPU?) wants to allocate a 
domain for the GPU (or Host1x) to use, even if the former isn't itself 
associated with the IOMMU, and at this point it doesn't actually have a 
suitable handle to the latter device.

> Alternatively how about:
> 
> bool iommu_present(void)
> {
> 	bool ret;
> 
> 	spin_lock(&iommu_device_lock);
> 	ret = !list_empty(&iommu_device_list);
> 	spin_unlock(&iommu_device_lock);
> 	return ret;
> }
> EXPORT_SYMBOL_GPL(iommu_present);
> 
> Since neither of the two users is really needing anything more than that?

Hmm, I guess maybe I did get a bit hung up on the bus notion... Indeed I 
think this wouldn't really be any more inaccurate than the current 
behaviour, and might be arguably truer to the intent of the function 
(whatever that is) since in the new design any instance is effectively 
present for all relevant buses anyway. I've respun along these lines 
(but retaining the argument with some token validation) and I don't hate 
it, so I'll send that as v4.

Thanks,
Robin.

[1] 
https://lore.kernel.org/dri-devel/49bafdabd2263cfc543bb22fb7f1bf32ea6bfd22.1683735862.git.robin.murphy@arm.com/
  

On Mon, Sep 18, 2023 at 08:21:45PM +0100, Robin Murphy wrote:

> > ? It doesn't seem to use the iommu API so I guess all it is doing is
> > trying to fix some kind of probe ordering issue? Maybe the probe
> > ordering issue is already gone and we can just delete the check?
> 
> As I've said before, the correct fix for this one is [1]. I've sent it twice
> now, it just gets ignored :(

IMHO at this point just put it in this series and have Joerg take it
:(

> Hmm, I guess maybe I did get a bit hung up on the bus notion... Indeed I
> think this wouldn't really be any more inaccurate than the current
> behaviour, and might be arguably truer to the intent of the function
> (whatever that is) since in the new design any instance is effectively
> present for all relevant buses anyway. I've respun along these lines (but
> retaining the argument with some token validation) and I don't hate it, so
> I'll send that as v4.

Eventually tegra is going to need to pass in a real struct device to
get the domain, so at that moment we can switch it to use the device
API on that real struct device. So this definately seems good enough
for now.

Jason