[v2,7/8] iommu: Retire bus ops
Commit Message
With the rest of the API internals converted, it's time to finally
tackle probe_device and how we bootstrap the per-device ops association
to begin with. This ends up being disappointingly straightforward, since
fwspec users are already doing it in order to find their of_xlate
callback, and it works out that we can easily do the equivalent for
other drivers too. Then shuffle the remaining awareness of iommu_ops
into the couple of core headers that still need it, and breathe a sigh
of relief.
Ding dong the bus ops are gone!
CC: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
CC: Christoph Hellwig <hch@lst.de>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---
v2: Clarify the iommu_ops_from_fwnode(NULL) assumption [Baolu]
drivers/iommu/iommu.c | 28 +++++++++++++++++-----------
include/acpi/acpi_bus.h | 2 ++
include/linux/device.h | 1 -
include/linux/device/bus.h | 5 -----
include/linux/dma-map-ops.h | 1 +
5 files changed, 20 insertions(+), 17 deletions(-)
Comments
On 2023/1/27 2:26, Robin Murphy wrote:
> With the rest of the API internals converted, it's time to finally
> tackle probe_device and how we bootstrap the per-device ops association
> to begin with. This ends up being disappointingly straightforward, since
> fwspec users are already doing it in order to find their of_xlate
> callback, and it works out that we can easily do the equivalent for
> other drivers too. Then shuffle the remaining awareness of iommu_ops
> into the couple of core headers that still need it, and breathe a sigh
> of relief.
>
> Ding dong the bus ops are gone!
>
> CC: Rafael J. Wysocki<rafael.j.wysocki@intel.com>
> CC: Christoph Hellwig<hch@lst.de>
> Acked-by: Greg Kroah-Hartman<gregkh@linuxfoundation.org>
> Signed-off-by: Robin Murphy<robin.murphy@arm.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Best regards,
baolu
Hi Robin,
I love your patch! Yet something to improve:
[auto build test ERROR on rafael-pm/linux-next]
[also build test ERROR on linus/master v6.2-rc5]
[cannot apply to joro-iommu/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Robin-Murphy/iommu-Decouple-iommu_present-from-bus-ops/20230128-141510
base: https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git linux-next
patch link: https://lore.kernel.org/r/198e82a6b1a28605409c395da4ec1a67b0e1587b.1674753627.git.robin.murphy%40arm.com
patch subject: [PATCH v2 7/8] iommu: Retire bus ops
config: s390-randconfig-r012-20230123 (https://download.01.org/0day-ci/archive/20230128/202301282015.hjj2YFYy-lkp@intel.com/config)
compiler: clang version 16.0.0 (https://github.com/llvm/llvm-project 4196ca3278f78c6e19246e54ab0ecb364e37d66a)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install s390 cross compiling tool for clang build
# apt-get install binutils-s390x-linux-gnu
# https://github.com/intel-lab-lkp/linux/commit/399f4b37d8065bffafeec12de1344a7ff6098e64
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Robin-Murphy/iommu-Decouple-iommu_present-from-bus-ops/20230128-141510
git checkout 399f4b37d8065bffafeec12de1344a7ff6098e64
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=s390 olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=s390 SHELL=/bin/bash drivers/iommu/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from drivers/iommu/iommufd/selftest.c:7:
In file included from include/linux/iommu.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:75:
include/asm-generic/io.h:547:31: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __raw_readb(PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:560:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __le16_to_cpu((__le16 __force)__raw_readw(PCI_IOBASE + addr));
~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:37:59: note: expanded from macro '__le16_to_cpu'
#define __le16_to_cpu(x) __swab16((__force __u16)(__le16)(x))
^
include/uapi/linux/swab.h:102:54: note: expanded from macro '__swab16'
#define __swab16(x) (__u16)__builtin_bswap16((__u16)(x))
^
In file included from drivers/iommu/iommufd/selftest.c:7:
In file included from include/linux/iommu.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:75:
include/asm-generic/io.h:573:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr));
~~~~~~~~~~ ^
include/uapi/linux/byteorder/big_endian.h:35:59: note: expanded from macro '__le32_to_cpu'
#define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x))
^
include/uapi/linux/swab.h:115:54: note: expanded from macro '__swab32'
#define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
^
In file included from drivers/iommu/iommufd/selftest.c:7:
In file included from include/linux/iommu.h:10:
In file included from include/linux/scatterlist.h:9:
In file included from arch/s390/include/asm/io.h:75:
include/asm-generic/io.h:584:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
__raw_writeb(value, PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:594:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
__raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:604:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
__raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr);
~~~~~~~~~~ ^
include/asm-generic/io.h:692:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
readsb(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:700:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
readsw(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:708:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
readsl(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:717:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
writesb(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:726:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
writesw(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
include/asm-generic/io.h:735:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
writesl(PCI_IOBASE + addr, buffer, count);
~~~~~~~~~~ ^
>> drivers/iommu/iommufd/selftest.c:276:39: error: field designator 'iommu_ops' does not refer to any field in type 'struct bus_type'
static struct bus_type mock_bus = { .iommu_ops = &mock_ops };
^
12 warnings and 1 error generated.
vim +276 drivers/iommu/iommufd/selftest.c
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 271
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 272 /* Create an hw_pagetable with the mock domain so we can test the domain ops */
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 273 static int iommufd_test_mock_domain(struct iommufd_ucmd *ucmd,
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 274 struct iommu_test_cmd *cmd)
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 275 {
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 @276 static struct bus_type mock_bus = { .iommu_ops = &mock_ops };
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 277 struct iommufd_hw_pagetable *hwpt;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 278 struct selftest_obj *sobj;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 279 struct iommufd_ioas *ioas;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 280 int rc;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 281
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 282 ioas = iommufd_get_ioas(ucmd, cmd->id);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 283 if (IS_ERR(ioas))
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 284 return PTR_ERR(ioas);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 285
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 286 sobj = iommufd_object_alloc(ucmd->ictx, sobj, IOMMUFD_OBJ_SELFTEST);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 287 if (IS_ERR(sobj)) {
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 288 rc = PTR_ERR(sobj);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 289 goto out_ioas;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 290 }
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 291 sobj->idev.ictx = ucmd->ictx;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 292 sobj->type = TYPE_IDEV;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 293 sobj->idev.mock_dev.bus = &mock_bus;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 294
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 295 hwpt = iommufd_device_selftest_attach(ucmd->ictx, ioas,
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 296 &sobj->idev.mock_dev);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 297 if (IS_ERR(hwpt)) {
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 298 rc = PTR_ERR(hwpt);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 299 goto out_sobj;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 300 }
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 301 sobj->idev.hwpt = hwpt;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 302
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 303 /* Userspace must destroy both of these IDs to destroy the object */
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 304 cmd->mock_domain.out_hwpt_id = hwpt->obj.id;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 305 cmd->mock_domain.out_device_id = sobj->obj.id;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 306 iommufd_object_finalize(ucmd->ictx, &sobj->obj);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 307 iommufd_put_object(&ioas->obj);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 308 return iommufd_ucmd_respond(ucmd, sizeof(*cmd));
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 309
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 310 out_sobj:
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 311 iommufd_object_abort(ucmd->ictx, &sobj->obj);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 312 out_ioas:
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 313 iommufd_put_object(&ioas->obj);
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 314 return rc;
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 315 }
f4b20bb34c83dc Jason Gunthorpe 2022-11-29 316
On Sat, Jan 28, 2023 at 08:55:09PM +0800, kernel test robot wrote:
> >> drivers/iommu/iommufd/selftest.c:276:39: error: field designator 'iommu_ops' does not refer to any field in type 'struct bus_type'
> static struct bus_type mock_bus = { .iommu_ops = &mock_ops };
So this shortcut isn't going to work..
Probably something like this is the simplest thing:
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index cfb5fe9a5e0ee8..27bab42b1ff841 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -274,6 +274,8 @@ static int iommufd_test_mock_domain(struct iommufd_ucmd *ucmd,
struct iommu_test_cmd *cmd)
{
static struct bus_type mock_bus = { .iommu_ops = &mock_ops };
+ static struct iommu_device mock_iommu = { .ops = &mock_ops };
+ static struct dev_iommu mock_dev_iommu = { .iommu_dev = &mock_iommu };
struct iommufd_hw_pagetable *hwpt;
struct selftest_obj *sobj;
struct iommufd_ioas *ioas;
@@ -291,6 +293,7 @@ static int iommufd_test_mock_domain(struct iommufd_ucmd *ucmd,
sobj->idev.ictx = ucmd->ictx;
sobj->type = TYPE_IDEV;
sobj->idev.mock_dev.bus = &mock_bus;
+ sobj->idev.mock_dev.iommu = &mock_dev_iommu;
hwpt = iommufd_device_selftest_attach(ucmd->ictx, ioas,
&sobj->idev.mock_dev);
And then delete mock_bus at this patch
Jason
On Thu, Jan 26, 2023 at 06:26:22PM +0000, Robin Murphy wrote:
> With the rest of the API internals converted, it's time to finally
> tackle probe_device and how we bootstrap the per-device ops association
> to begin with. This ends up being disappointingly straightforward, since
> fwspec users are already doing it in order to find their of_xlate
> callback, and it works out that we can easily do the equivalent for
> other drivers too. Then shuffle the remaining awareness of iommu_ops
> into the couple of core headers that still need it, and breathe a sigh
> of relief.
>
> Ding dong the bus ops are gone!
>
> CC: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> CC: Christoph Hellwig <hch@lst.de>
> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
> ---
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Jason
@@ -219,13 +219,6 @@ int iommu_device_register(struct iommu_device *iommu,
/* We need to be able to take module references appropriately */
if (WARN_ON(is_module_address((unsigned long)ops) && !ops->owner))
return -EINVAL;
- /*
- * Temporarily enforce global restriction to a single driver. This was
- * already the de-facto behaviour, since any possible combination of
- * existing drivers would compete for at least the PCI or platform bus.
- */
- if (iommu_buses[0]->iommu_ops && iommu_buses[0]->iommu_ops != ops)
- return -EBUSY;
iommu->ops = ops;
if (hwdev)
@@ -235,10 +228,8 @@ int iommu_device_register(struct iommu_device *iommu,
list_add_tail(&iommu->list, &iommu_device_list);
spin_unlock(&iommu_device_lock);
- for (int i = 0; i < ARRAY_SIZE(iommu_buses) && !err; i++) {
- iommu_buses[i]->iommu_ops = ops;
+ for (int i = 0; i < ARRAY_SIZE(iommu_buses) && !err; i++)
err = bus_iommu_probe(iommu_buses[i]);
- }
if (err)
iommu_device_unregister(iommu);
return err;
@@ -310,12 +301,27 @@ static u32 dev_iommu_get_max_pasids(struct device *dev)
static int __iommu_probe_device(struct device *dev, struct list_head *group_list)
{
- const struct iommu_ops *ops = dev->bus->iommu_ops;
+ const struct iommu_ops *ops;
struct iommu_device *iommu_dev;
+ struct iommu_fwspec *fwspec;
struct iommu_group *group;
static DEFINE_MUTEX(iommu_probe_device_lock);
int ret;
+ /*
+ * For FDT-based systems and ACPI IORT/VIOT, drivers register IOMMU
+ * instances with non-NULL fwnodes, and client devices should have been
+ * identified with a fwspec by this point. Otherwise, we can currently
+ * assume that only one of Intel, AMD, s390, PAMU or legacy SMMUv2 can
+ * be present, and that any of their registered instances has suitable
+ * ops for probing, and thus cheekily co-opt the same mechanism.
+ */
+ fwspec = dev_iommu_fwspec_get(dev);
+ if (fwspec && fwspec->ops)
+ ops = fwspec->ops;
+ else
+ ops = iommu_ops_from_fwnode(NULL);
+
if (!ops)
return -ENODEV;
/*
@@ -614,6 +614,8 @@ struct acpi_pci_root {
/* helper */
+struct iommu_ops;
+
bool acpi_dma_supported(const struct acpi_device *adev);
enum dev_dma_attr acpi_get_dma_attr(struct acpi_device *adev);
int acpi_iommu_fwspec_init(struct device *dev, u32 id,
@@ -41,7 +41,6 @@ struct class;
struct subsys_private;
struct device_node;
struct fwnode_handle;
-struct iommu_ops;
struct iommu_group;
struct dev_pin_info;
struct dev_iommu;
@@ -63,9 +63,6 @@ struct fwnode_handle;
* this bus.
* @pm: Power management operations of this bus, callback the specific
* device driver's pm-ops.
- * @iommu_ops: IOMMU specific operations for this bus, used to attach IOMMU
- * driver implementations to a bus and allow the driver to do
- * bus-specific setup
* @p: The private data of the driver core, only the driver core can
* touch this.
* @lock_key: Lock class key for use by the lock validator
@@ -109,8 +106,6 @@ struct bus_type {
const struct dev_pm_ops *pm;
- const struct iommu_ops *iommu_ops;
-
struct subsys_private *p;
struct lock_class_key lock_key;
@@ -10,6 +10,7 @@
#include <linux/pgtable.h>
struct cma;
+struct iommu_ops;
/*
* Values for struct dma_map_ops.flags: