From patchwork Tue May 30 05:37:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 9996 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1951922vqr; Mon, 29 May 2023 22:41:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4rNwJRIjAV9KWUr7w+q2dO3dZT0UhSldvjDcXun22/kF66E7lJXEKQ1ktExl8B41RsVM/G X-Received: by 2002:a05:6a20:d397:b0:104:a096:6ac7 with SMTP id iq23-20020a056a20d39700b00104a0966ac7mr1151351pzb.34.1685425275106; Mon, 29 May 2023 22:41:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685425275; cv=none; d=google.com; s=arc-20160816; b=rALVMOkxvaOdFmC77VXa0EvimgOYiGG3ZSxbz74Vun0SRGf3kRvIMnLtYGsFZ7PIWF StvTngVBuFM57dAqL52jJdC5aZBo00UBhgoHxOasY74ulbc2YZvpUzB9XUlWl0zxxKN1 VicoeRpIfe9WU60QQmqOaB09roN4VzjqpmbhqGn0IZFLSiPdqh0oUdUCsJ5YOEhLYKG3 zAa5ZKcPEohQ2wjAn0rkMU0YqDzZAPlxK5S1fXABcbJ2XpqdCgZleGjPERiXerwaYfUc YB+VRwjAmC67j1EaMD18IYe/MlrD2OKxgqwpct9k9IGjleGHFtrt3EJycj0uj4v+8Ueu 6gZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=FGhuHEZW1sSJcEQIntmFfkrU63ACgqoV1tEbJV7bVZQ=; b=epNmsg2ubJPEBNNJQX1wanmiBoiVoj2mF9Yd+RtCJpV9jjL4pvwvcCIA0SJkeEraBh 4/6EM5I+kO26EQLAX1zMpBSLmG9IIiLyoONdgcQ007Qfs/0UR94m1YwDDnhgxqH1PsHp xTrA8u4P5cxGaJD22OM/F+y7Zg6BAcwzdyISS9Mq+bTVXzQWnm6K+VWTkMnKGVa8DaQV /Dk4kjWeZBXouAfCVSd+WLhBG1fRU3s7PV4nyrCsnfKlyeda7lFoHo9eNjG2Blxy8K5C 3O5nSnn9zy6tXjRIiC/sHlb5ursxVxSZIoJ7OEbS7E0PPL316f6XCivswcs8ZwKEk1gM nZIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="eCe/hlL2"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i128-20020a639d86000000b005346c49e06esi6006805pgd.843.2023.05.29.22.41.01; Mon, 29 May 2023 22:41:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="eCe/hlL2"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230204AbjE3Fic (ORCPT + 99 others); Tue, 30 May 2023 01:38:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229843AbjE3Fi3 (ORCPT ); Tue, 30 May 2023 01:38:29 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D852EA; Mon, 29 May 2023 22:38:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1685425107; x=1716961107; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=SlmVU0gCOJtKzGhhoLJtPHVYcv/MzAh06Xb0nC+R2xw=; b=eCe/hlL2QJFzeMkcb5OxWRZk4o47Lb1qQ7XYBp1Kh41KfRW/E5gJBZU0 cgvHvpxuOgu1DNTR9XdXTU02OK26xa+/FXtcDIxvFSTFRVMlqOVDoSH2w ubt8ZLVQVsfU4VuGWDn5uXQaJLE8MSApYNf/S5qFrD48WKujGjmN8MDx8 6MmINFuTepaxFPegwhnZyPzqQzKST/6+6a1NjCxScBa6Sl6SvYOs4DsH3 2VdjzoiN2qI0nNDezcOb6HZBeBhf4QlARNlwX0GLErN5F3jhwBHJAWgHJ 2IgpRI0xx3AX+zepaDtjhd76Ypnb7D34xK8iVY9PwEkisvaXs73eT9Bat g==; X-IronPort-AV: E=McAfee;i="6600,9927,10725"; a="420579570" X-IronPort-AV: E=Sophos;i="6.00,203,1681196400"; d="scan'208";a="420579570" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2023 22:38:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10725"; a="739369265" X-IronPort-AV: E=Sophos;i="6.00,203,1681196400"; d="scan'208";a="739369265" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga001.jf.intel.com with ESMTP; 29 May 2023 22:38:20 -0700 From: Lu Baolu To: Jason Gunthorpe , Kevin Tian , Joerg Roedel , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Nicolin Chen , Yi Liu , Jacob Pan Cc: iommu@lists.linux.dev, linux-kselftest@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [RFC PATCHES 00/17] IOMMUFD: Deliver IO page faults to user space Date: Tue, 30 May 2023 13:37:07 +0800 Message-Id: <20230530053724.232765-1-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767296493316153460?= X-GMAIL-MSGID: =?utf-8?q?1767296493316153460?= Hi folks, This series implements the functionality of delivering IO page faults to user space through the IOMMUFD framework. The use case is nested translation, where modern IOMMU hardware supports two-stage translation tables. The second-stage translation table is managed by the host VMM while the first-stage translation table is owned by the user space. Hence, any IO page fault that occurs on the first-stage page table should be delivered to the user space and handled there. The user space should respond the page fault handling result to the device top-down through the IOMMUFD response uAPI. User space indicates its capablity of handling IO page faults by setting a user HWPT allocation flag IOMMU_HWPT_ALLOC_FLAGS_IOPF_CAPABLE. IOMMUFD will then setup its infrastructure for page fault delivery. Together with the iopf-capable flag, user space should also provide an eventfd where it will listen on any down-top page fault messages. On a successful return of the allocation of iopf-capable HWPT, a fault fd will be returned. User space can open and read fault messages from it once the eventfd is signaled. Besides the overall design, I'd like to hear comments about below designs: - The IOMMUFD fault message format. It is very similar to that in uapi/linux/iommu which has been discussed before and partially used by the IOMMU SVA implementation. I'd like to get more comments on the format when it comes to IOMMUFD. - The timeout value for the pending page fault messages. Ideally we should determine the timeout value from the device configuration, but I failed to find any statement in the PCI specification (version 6.x). A default 100 milliseconds is selected in the implementation, but it leave the room for grow the code for per-device setting. This series is only for review comment purpose. I used IOMMUFD selftest to verify the hwpt allocation, attach/detach and replace. But I didn't get a chance to run it with real hardware yet. I will do more test in the subsequent versions when I am confident that I am heading on the right way. This series is based on the latest implementation of the nested translation under discussion. The whole series and related patches are available on gitbub: https://github.com/LuBaolu/intel-iommu/commits/iommufd-io-pgfault-delivery-v1 Best regards, baolu Lu Baolu (17): iommu: Move iommu fault data to linux/iommu.h iommu: Support asynchronous I/O page fault response iommu: Add helper to set iopf handler for domain iommu: Pass device parameter to iopf handler iommu: Split IO page fault handling from SVA iommu: Add iommu page fault cookie helpers iommufd: Add iommu page fault data iommufd: IO page fault delivery initialization and release iommufd: Add iommufd hwpt iopf handler iommufd: Add IOMMU_HWPT_ALLOC_FLAGS_USER_PASID_TABLE for hwpt_alloc iommufd: Deliver fault messages to user space iommufd: Add io page fault response support iommufd: Add a timer for each iommufd fault data iommufd: Drain all pending faults when destroying hwpt iommufd: Allow new hwpt_alloc flags iommufd/selftest: Add IOPF feature for mock devices iommufd/selftest: Cover iopf-capable nested hwpt include/linux/iommu.h | 175 +++++++++- drivers/iommu/{iommu-sva.h => io-pgfault.h} | 25 +- drivers/iommu/iommu-priv.h | 3 + drivers/iommu/iommufd/iommufd_private.h | 32 ++ include/uapi/linux/iommu.h | 161 --------- include/uapi/linux/iommufd.h | 73 +++- tools/testing/selftests/iommu/iommufd_utils.h | 20 +- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/intel/svm.c | 2 +- drivers/iommu/io-pgfault.c | 7 +- drivers/iommu/iommu-sva.c | 4 +- drivers/iommu/iommu.c | 50 ++- drivers/iommu/iommufd/device.c | 64 +++- drivers/iommu/iommufd/hw_pagetable.c | 318 +++++++++++++++++- drivers/iommu/iommufd/main.c | 3 + drivers/iommu/iommufd/selftest.c | 71 ++++ tools/testing/selftests/iommu/iommufd.c | 17 +- MAINTAINERS | 1 - drivers/iommu/Kconfig | 4 + drivers/iommu/Makefile | 3 +- drivers/iommu/intel/Kconfig | 1 + 23 files changed, 837 insertions(+), 203 deletions(-) rename drivers/iommu/{iommu-sva.h => io-pgfault.h} (71%) delete mode 100644 include/uapi/linux/iommu.h