From patchwork Mon Oct 9 08:51:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 15083 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:403:3b70:6f57 with SMTP id x8csp1730452vqo; Mon, 9 Oct 2023 01:51:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHGp6e2k4WkiayS4l4I/WsyZ79GToW7cVBTlM8mwi+uXnRF4/B+GjzRpPtUKyww9zZwVbhA X-Received: by 2002:a9d:7d85:0:b0:6c4:897a:31d2 with SMTP id j5-20020a9d7d85000000b006c4897a31d2mr17120667otn.30.1696841507062; Mon, 09 Oct 2023 01:51:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696841507; cv=none; d=google.com; s=arc-20160816; b=UC2u/YmtGd+SrS2HyMm4oVeUm4JbYAX7kPR3xAb2us9u8WL7z6koQ6TrJnIhaPQeof tf15WSDEqix+/gz8Tt9fCNVhYQJY3YzF8PA8DKw7HaLbuvwuwb2PTo2dzRP5dcJWO60d bpCM2+gLg0Ydin02CSqW+ygqs4tVRCnOAF7jcUFn3G40UDOnGzHQIu6qdTg0q4NdFvly NjBre8VXV4vvC9qE3Caneh6rnLo8eyxHRLrE6wzF8/HZFpoypa2rUOlV1+3GfwhwjWlN KXwGfs+k/PtdMb1eTNwWXcpSK+ovNIZqrD21OfnDFUKG/6JAGzz3VBApUb6Ob+Nh/lFi nO8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=MqcPP1NylGYNDUis0BAFZxsAPZ0hjjheAvSH8Paqw3I=; fh=ncJBVmsnOSqrX1O37yfYEzicwaA2e7ARxnsU7aiysyE=; b=jDL+nhsY2HkNzOMhbteXKL6hxDCOkCV2GI2dRYIEKS82dBZfNvMem+ZsyC73Qn8FH+ U86aBDAwwprgOuoAWKVIBOQEWkt3Nz9bEeug/22VQGFzbRxqEvPDcAD0IFWoSgg9bysr saYGT7jjPDVigEq3KSMh89HtX6XMewrv976tIa2R7gu6kGBQ0AkmA8Nw8XpHGDj0f+Nm +0jtORyfOwqG6b6YkcDvYnhz2kJedCWTB57fNv79RgEtR95o3X2ayEArr82FFUK0aVd4 yYfd2clWOjeD9apAGf1rinENfQEEb7wi13Wn9S8giYjH3VU5t7ZY8yPUJEPiMURQ4QEz WVOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=C7Pgc6iv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id 33-20020a631861000000b00573fdbc93c0si9324807pgy.892.2023.10.09.01.51.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 01:51:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=C7Pgc6iv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 370C2802C7DC; Mon, 9 Oct 2023 01:51:46 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345635AbjJIIvj (ORCPT + 18 others); Mon, 9 Oct 2023 04:51:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345572AbjJIIv2 (ORCPT ); Mon, 9 Oct 2023 04:51:28 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.151]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBEFFB9; Mon, 9 Oct 2023 01:51:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1696841487; x=1728377487; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=QQiCA0/iZXkbwpDgf6IT/PWFnBNd1b0GHC0pta7p/Rk=; b=C7Pgc6ivLscECJZrN7j7SpK7k+GvObdDp1dJbcOviXfru0zrzQJLv30e XK7HwVJHTH/MYnV+dCA53ukAJZc3qhAuHypvdYYKYTrpnhARjXBRattlt dyPwA7zWxj8QbNn/f0kbRvBJeWFVVq3XoMlhNZINAXy/T6Kb7pShlIISr X46igJIIKJQkwtSS7hXzi/b8tg4hC8BR3co7ZchhrhwyfHTQnN36+3a2+ bSkvq8LcEAhiQpBa0PJbVLL7QHRvOHGIeSyjijHwOkBG9U+wylsfl3zCb RVr615HnfRjABssiC2+oMayVGR6Kh8dbq9/hHZbSzTvOkfJ8NJwl4Sg9J g==; X-IronPort-AV: E=McAfee;i="6600,9927,10857"; a="364390743" X-IronPort-AV: E=Sophos;i="6.03,209,1694761200"; d="scan'208";a="364390743" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Oct 2023 01:51:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10857"; a="818781640" X-IronPort-AV: E=Sophos;i="6.03,209,1694761200"; d="scan'208";a="818781640" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga008.fm.intel.com with ESMTP; 09 Oct 2023 01:51:25 -0700 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com Subject: [RFC 0/7] Add SIOV virtual device support Date: Mon, 9 Oct 2023 01:51:16 -0700 Message-Id: <20231009085123.463179-1-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 09 Oct 2023 01:51:46 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779267280142804811 X-GMAIL-MSGID: 1779267280142804811 Intel SIOV allows creating virtual devices of which the vRID is represented by a pasid of a physical device. It is called as SIOV virtual device in this series. Such devices can be bound to an iommufd as physical device does and then later be attached to an IOAS/hwpt using that pasid. Such PASIDs are called as default pasid. iommufd has already supported pasid attach[1]. So a simple way to support SIOV virtual device attachment is to let device driver call the iommufd_device_pasid_attach() and pass in the default pasid for the virtual device. This should work for now, but it may have problem if iommufd core wants to differentiate the default pasids with other kind of pasids (e.g. pasid given by userspace). In the later forwarding page request to userspace, the default pasids are not supposed to send to userspace as default pasids are mainly used by the SIOV device driver. With above reason, this series chooses to have a new API to bind the default pasid to iommufd, and extends the iommufd_device_attach() to convert the attachment to be pasid attach with the default pasid. Device drivers (e.g. VFIO) that support SIOV shall call the below APIs to interact with iommufd: - iommufd_device_bind_pasid(): Bind virtual device (a pasid of a device) to iommufd; - iommufd_device_attach(): Attach a SIOV virtual device to IOAS/HWPT; - iommufd_device_replace(): Replace IOAS/HWPT of a SIOV virtual device; - iommufd_device_detach(): Detach IOAS/HWPT of a SIOV virtual device; - iommufd_device_unbind(): Unbind virtual device from iommufd; For vfio devices, the device drivers that support SIOV should: - use below API to register vdev for SIOV virtual device vfio_register_pasid_iommu_dev() - use below API to bind vdev to iommufd in .bind_iommufd() callback iommufd_device_bind_pasid() - allocate pasid by itself before calling iommufd_device_bind_pasid() Complete code can be found at[2] [1] https://lore.kernel.org/linux-iommu/20230926092651.17041-1-yi.l.liu@intel.com/ [2] https://github.com/yiliu1765/iommufd/tree/iommufd_pasid_siov Regards, Yi Liu Kevin Tian (5): iommufd: Handle unsafe interrupts in a separate function iommufd: Introduce iommufd_alloc_device() iommufd: Add iommufd_device_bind_pasid() iommufd: Support attach/replace for SIOV virtual device {dev, pasid} vfio: Add vfio_register_pasid_iommu_dev() Yi Liu (2): iommufd/selftest: Extend IOMMU_TEST_OP_MOCK_DOMAIN to pass in pasid iommufd/selftest: Add test coverage for SIOV virtual device drivers/iommu/iommufd/device.c | 163 ++++++++++++++---- drivers/iommu/iommufd/iommufd_private.h | 7 + drivers/iommu/iommufd/iommufd_test.h | 2 + drivers/iommu/iommufd/selftest.c | 10 +- drivers/vfio/group.c | 18 ++ drivers/vfio/vfio.h | 8 + drivers/vfio/vfio_main.c | 10 ++ include/linux/iommufd.h | 3 + include/linux/vfio.h | 1 + tools/testing/selftests/iommu/iommufd.c | 75 ++++++-- .../selftests/iommu/iommufd_fail_nth.c | 42 ++++- tools/testing/selftests/iommu/iommufd_utils.h | 21 ++- 12 files changed, 296 insertions(+), 64 deletions(-)