From patchwork Mon Aug 14 01:17:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 135108 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp2487539vqi; Sun, 13 Aug 2023 20:08:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEpYiwwTsGXuxJtQD9bDP6AqfCBTqSdwgY3MAHsS5R4Tr0RocV42k39PPVi3cM3FSUB8L04 X-Received: by 2002:a05:6870:65a0:b0:1b0:60ff:b73f with SMTP id fp32-20020a05687065a000b001b060ffb73fmr9532036oab.8.1691982492425; Sun, 13 Aug 2023 20:08:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691982492; cv=none; d=google.com; s=arc-20160816; b=gpfN3lZLHetROQtppqGqbRkc6GadH1aSAXI4Ba94bJrgFyZ5OWAxXY1QLiwOwGXfPR m9pNtdPWPugj9/ofsQ30tvVdWCZHz9vS0d8tv+gb+eYP6gmeotzlt/HVI2H+31Aug+y7 os+zthjHWr9bhjHykKGnNB1TgypITnnP60wsFLKuGUrRedSPCRyuhAqw6IMuZLhRSRb6 7d5xA8ftwnLqVRL7QOUF5nQmMNYbSAnm8ycp+h2k6wsYCbBzyI3T71siS7sxjY8493RR siZnUDEI5s3HuFc47nlI59DN19sVwLxr/vzI/rA4B9TCepjZ6AHC+YnDr3WWtwO4GPOb 7TSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lAtE4BR30Yl9zgPzX74U11gG4N8P4nzvzBkfwyOlwCM=; fh=5v0QJkujOnxKSzhmeWiMX/0q9X35NwMz+SRMkKa4moc=; b=cZjKZnB5q2agHKW2txTbUaRYSdMXAuzp/DoSgMW922zPsK1kGSRbjJJO3KRyStqPa3 tYfNBIlOBPD6EVqf9u69zc4/+jANAume6CSHByJKpYQYd5ot7kMpXvDEX73uiVrIqan0 AuVHI9DrKd6fsI1/L/SUdMDziladVBS0Yu7c59VUw2HoC5szpEADaBX8lW3EwiyUtTTO ufib0mQBfM1xqJDWYo/9KeiP7lLzP2U49fXQqhZhKQXxOdJmLprQ97B4OtLR5UCU9hkm HMwvRs1IW04nOVePAbN2KiPLnc21cGkq/Aw2GJlW+AWsDInQQe41sLuTHaNMYE63tKEa h3zg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Qrv0sR5e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nv11-20020a17090b1b4b00b00269584b6a10si7676266pjb.15.2023.08.13.20.07.59; Sun, 13 Aug 2023 20:08:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Qrv0sR5e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231768AbjHNBVD (ORCPT + 99 others); Sun, 13 Aug 2023 21:21:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231716AbjHNBUd (ORCPT ); Sun, 13 Aug 2023 21:20:33 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DAF2E6E; Sun, 13 Aug 2023 18:20:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691976032; x=1723512032; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1VKEBWyIoCdoQh4ifxDFIHU7H3LnyvQp0GU6ahA/QQE=; b=Qrv0sR5eRASz2+cnddIJc2aipVni2jlP8zyLSrTa2yiEXlIPkSy2f3kq 5hbFOFk4l84ZpIq7ncNPOGwAH6DOSayKba1Dx6PGj2LYfmGmQAPUdsVIm 75khmaGbGL3fkKqb8grqrLWCxYI1zVhcFcRGhm5OaOiJlDxMoX/DkyCFD foyKxGLpTPwOvpmmDFU8pCyal3+EAwvk1AsMi7BDyPzHJvC7RBJi1RNdW lzicgRlKgmLhUI76WDmzm1BWFxHeHIwMaqcStcbaXm2bEoeed+0tdkzBy D8De356fcbE5gime2X5E6yP7GqKvSgSe3EcHShadsDCKiaDf4xrdXclQ5 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="375645305" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="375645305" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2023 18:20:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="726842292" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="726842292" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga007.jf.intel.com with ESMTP; 13 Aug 2023 18:20:28 -0700 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 1/3] iommu: Make single-device group for PASID explicit Date: Mon, 14 Aug 2023 09:17:57 +0800 Message-Id: <20230814011759.102089-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814011759.102089-1-baolu.lu@linux.intel.com> References: <20230814011759.102089-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774172233961004971 X-GMAIL-MSGID: 1774172233961004971 The PASID interfaces have always supported only single-device groups. This was first introduced in commit 26b25a2b98e45 ("iommu: Bind process address spaces to devices"), and has been kept consistent in subsequent commits. However, the core code doesn't explicitly check for this requirement after commit 201007ef707a8 ("PCI: Enable PASID only when ACS RR & UF enabled on upstream path"), which made this requirement implicit. Restore the check to make it explicit that the PASID interfaces only support devices belonging to single-device groups. Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 71b9c41f2a9e..f1eba60e573f 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3408,6 +3408,11 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, return -ENODEV; mutex_lock(&group->mutex); + if (list_count_nodes(&group->devices) != 1) { + ret = -EINVAL; + goto out_unlock; + } + curr = xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); if (curr) { ret = xa_err(curr) ? : -EBUSY; From patchwork Mon Aug 14 01:17:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 135096 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp2466149vqi; Sun, 13 Aug 2023 19:00:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IETwDdQbMNFn11++85fSBbenlOht21ZijM41rDkqyf3WBWkirJkiFkUGXLrOwJGAkRk4lFr X-Received: by 2002:a05:6512:3a8b:b0:4f9:5396:ed1b with SMTP id q11-20020a0565123a8b00b004f95396ed1bmr6115989lfu.28.1691978449927; Sun, 13 Aug 2023 19:00:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691978449; cv=none; d=google.com; s=arc-20160816; b=QzHoyKLVg24P+U0jbwgTXiwoptAkOBYG0ZpDDzE6cZcuHgis3OU5cuXhBu8HY5CpII sMFa7e1zZNxP4hBskaGQQtEMxhilmQPZxwZITSLqiavamROSgBAWypobTmdGM+qmPlrl M1SWMbSYS1/fPt+zf3khK+ETsw3cwvCGNC30foUnb0JKQhkwOyoXkshF35N4uShEStyI oXdFWhPDR+hD05d5JqczhKzh5ggy4kVoPj6HM1D0TRtStQYjmlfPhM3+/A9GGtXUxR4c PjFAh2tpzSGSr+BJenX74S+v2OPWc1x9AbOcbjvrH1QANGXf/4O1O9msTnTyw34KKLvc hQYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=caGdfsl8qrD1hqsG+ui4CV7T4ymj7b+qIJviCi6ZRbI=; fh=5v0QJkujOnxKSzhmeWiMX/0q9X35NwMz+SRMkKa4moc=; b=VdOpebJhvu51QzVWgrUMksV6bm46mDf7WIL9eSOieQmvFsbh0fX9nF3tbQvfBPgupa IZGcGRMWsrmU4KkLei4EZhnRpMIUHcK0V0eNGJyi8LAtTREmEDDE11Ea944ndBfzVUn8 BiBJeNsIsVozbqt744Gxg/R6ZsuWW747vvd19fQ0vOwlXBx78Bx5Vxrl/UGv6X2C+bSV DIXoW7SZDH1qB37wQB1FbwQkE+sAqKFLOt6sgE2CViN2c1kGDQ2vwvcdx9ZEMRXT8Kem eLXDCXKnfrzFyb16pqKAohTyQRtbUscthapbQpnoF0DX8oEKUGh7a1w+q3U7BgZYuSf2 SwDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MrspJpIn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n4-20020aa7db44000000b00523384d6840si6942652edt.453.2023.08.13.19.00.26; Sun, 13 Aug 2023 19:00:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MrspJpIn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231834AbjHNBVG (ORCPT + 99 others); Sun, 13 Aug 2023 21:21:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231724AbjHNBUg (ORCPT ); Sun, 13 Aug 2023 21:20:36 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85B72E75; Sun, 13 Aug 2023 18:20:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691976035; x=1723512035; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=277c02PzldzlnRnH2jM0ESjUVEL/whZkd2HGaLLh2vY=; b=MrspJpInasTenaA2hCo/inKO4d/avRNFMUOSiPLb82QNmcj33UBG6udA ol5MzsfKHPt7gkChUZwECEzxDDeabyAsIQGPK5hfWMhyIje2JSrqQd3YS snPTKpMxIDR4coxd2KXwE+Ycj/4cpFTd6wsI6ChRQRzT1aHu4h0dThJCz fhR5bvzoQ8HSonD3HYnO/uIcqp28KsHRV0q6ATV/eZLFZ3pewyV9aOv85 zt0gOdJLC8AGdsNDWTWRHoW6aZLIc3NzmR6OIquDqwpuPrLc9nFBhwa69 UNDkssJAMg1gLfxXWuhipbsAOmxepnEGneh933wwH540mU9AJ/j4ttb7k w==; X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="375645319" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="375645319" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2023 18:20:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="726842296" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="726842296" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga007.jf.intel.com with ESMTP; 13 Aug 2023 18:20:31 -0700 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 2/3] iommu: Consolidate pasid dma ownership check Date: Mon, 14 Aug 2023 09:17:58 +0800 Message-Id: <20230814011759.102089-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814011759.102089-1-baolu.lu@linux.intel.com> References: <20230814011759.102089-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774167994607554829 X-GMAIL-MSGID: 1774167994607554829 When switching device DMA ownership, it is required that all the device's pasid DMA be disabled. This is done by checking if the pasid array of the group is empty. Consolidate all the open code into a single helper. No intentional functionality change. Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f1eba60e573f..d4a06a37ce39 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3127,6 +3127,19 @@ static bool iommu_is_default_domain(struct iommu_group *group) return false; } +/* + * Assert no PASID DMA when claiming or releasing group's DMA ownership. + * The device pasid interfaces are only for device drivers that have + * claimed the DMA ownership. Return true if no pasid DMA setup, otherwise + * return false with a WARN(). + */ +static bool assert_pasid_dma_ownership(struct iommu_group *group) +{ + lockdep_assert_held(&group->mutex); + + return !WARN_ON(!xa_empty(&group->pasid_array)); +} + /** * iommu_device_use_default_domain() - Device driver wants to handle device * DMA through the kernel DMA API. @@ -3147,7 +3160,7 @@ int iommu_device_use_default_domain(struct device *dev) mutex_lock(&group->mutex); if (group->owner_cnt) { if (group->owner || !iommu_is_default_domain(group) || - !xa_empty(&group->pasid_array)) { + !assert_pasid_dma_ownership(group)) { ret = -EBUSY; goto unlock_out; } @@ -3177,7 +3190,7 @@ void iommu_device_unuse_default_domain(struct device *dev) return; mutex_lock(&group->mutex); - if (!WARN_ON(!group->owner_cnt || !xa_empty(&group->pasid_array))) + if (!WARN_ON(!group->owner_cnt) && assert_pasid_dma_ownership(group)) group->owner_cnt--; mutex_unlock(&group->mutex); @@ -3211,7 +3224,7 @@ static int __iommu_take_dma_ownership(struct iommu_group *group, void *owner) int ret; if ((group->domain && group->domain != group->default_domain) || - !xa_empty(&group->pasid_array)) + !assert_pasid_dma_ownership(group)) return -EBUSY; ret = __iommu_group_alloc_blocking_domain(group); @@ -3296,8 +3309,8 @@ EXPORT_SYMBOL_GPL(iommu_device_claim_dma_owner); static void __iommu_release_dma_ownership(struct iommu_group *group) { - if (WARN_ON(!group->owner_cnt || !group->owner || - !xa_empty(&group->pasid_array))) + if (WARN_ON(!group->owner_cnt || !group->owner) || + !assert_pasid_dma_ownership(group)) return; group->owner_cnt = 0; From patchwork Mon Aug 14 01:17:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 135134 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp2538398vqi; Sun, 13 Aug 2023 22:44:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFSlS8b0Dir9YLgKOPBy86bHkiJ2Azz4LeOmSqF+cCKsm6m+8/fJgGBHROLWr8NGEDwzPi3 X-Received: by 2002:a17:907:a054:b0:982:25d1:bfd3 with SMTP id gz20-20020a170907a05400b0098225d1bfd3mr6940715ejc.15.1691991887817; Sun, 13 Aug 2023 22:44:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691991887; cv=none; d=google.com; s=arc-20160816; b=hlw8v+7QCPdd6DYeRtvgejnRQDpacxdhr+GCgqw9mY22Cg6QDdDeDNEO76zRudlqWo mkfO1ZjUkST+wmhlATY66O1J8lINDfwXDv9SoICRWHXsWj9eBajCZy2Ed6D0jjKeiS7X LJueMRKo/2Ql+6CLsHa5rzXOPW618aPMxmjXZ3oHJ0y5VtoTpEmrHRDnHHyoKUpKrLS5 XOvrP+krh6hiHISKyrhVRdiTaUoWpEJmEs51ke4PdYDCJCp4i/d4xNfinioqZUK1k/SM aTFaaFvFesfw+ryIu7yV6UEt4MQoAdmGVPuuLzSF1GfgeKWtiv/GpQVIAmbjyGi7N29r pViw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=IeylqfW4Jh6Ar5373wsRFpFQYecwD8+Nf3Enz6PPnxQ=; fh=5v0QJkujOnxKSzhmeWiMX/0q9X35NwMz+SRMkKa4moc=; b=BIg6CwaGxnpSxwJD0qENAALhIN291BWCyU8mbZf3uOUGw8jgdOn5fZNdEwaZ1j8875 /NVjj8GB+LGDJW+ADGMSjhlH8afdPxuUh9FFIgY+pE1VaIRQA1W+PG7AL0TWoEQDuUCs ZV5mPp7GKHXwDhtug2LTC8+UKLbVr6lEABSknrOeqPfxk4AX3UNm0OiwZOtS1tm2X7fP fKjivEC0aVJ4kiSQ5cg3o02BteMAMqDVbr5iXcpQxElKt3/8piXbLhyz8O0/vTRXaOMH /hNedotuvG6rfKp2MhUCqY3DJ8BIF7w64vSuSPFzhtenpwW4FiKOwzwzoAVQy3tAoZFQ xePg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LoDGsb7P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kf23-20020a17090776d700b0099bd68129dcsi7319458ejc.760.2023.08.13.22.44.24; Sun, 13 Aug 2023 22:44:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=LoDGsb7P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231901AbjHNBVH (ORCPT + 99 others); Sun, 13 Aug 2023 21:21:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231753AbjHNBUm (ORCPT ); Sun, 13 Aug 2023 21:20:42 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 046B110E2; Sun, 13 Aug 2023 18:20:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691976039; x=1723512039; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xz9IkjBL49Aqu2OLcjsslSruJe0mn1CqaV/hLG6938M=; b=LoDGsb7PBZKgVGyaKwoW3K1U+On+2MI4JF4sOcAxpSRFwGdkJ27oJS/r XzdjnfSloXRtMHdz1IM32uAcyUcizCIN7xZzsJ2saXRj6ceG92KzXhFRq j95uWRD1Y2pnosc1p6cW6hllLMLVLmCo7m56NM2qdwVA1TJeTv6ZoRN3/ 0bGd6bDvRWpE6Wg1Xh1Yqo/bCFrrHULezgypId2gBjtTXpykg5UWkqipM Nf0XZgbWUAtXTITiwGssew6xa22ekyXryB2BqFVThhhtGRev83iwwozC/ bjvU8/UdInKviqnZd4P9hg1+hWBUsJzw6kmDiBrI99JUoz4hDWvmH4DJM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="375645336" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="375645336" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2023 18:20:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10801"; a="726842300" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="726842300" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orsmga007.jf.intel.com with ESMTP; 13 Aug 2023 18:20:35 -0700 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 3/3] iommu: Move pasid array from group to device Date: Mon, 14 Aug 2023 09:17:59 +0800 Message-Id: <20230814011759.102089-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814011759.102089-1-baolu.lu@linux.intel.com> References: <20230814011759.102089-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774182085479042800 X-GMAIL-MSGID: 1774182085479042800 The PASID (Process Address Space ID) feature is a device feature that allows a device driver to manage the PASID value and attach or detach its domain to the pasid. The pasid array, which is used to store the domain of each pasid, is currently stored in iommu group struct, but it would be more natural to move it to the device so that the device drivers don't need to understand the internal iommu group concept. Signed-off-by: Lu Baolu --- include/linux/iommu.h | 2 ++ drivers/iommu/iommu.c | 80 +++++++++++++++---------------------------- 2 files changed, 29 insertions(+), 53 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 411ce9b998dc..76c960741449 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -412,6 +412,7 @@ struct iommu_fault_param { * @iopf_param: I/O Page Fault queue and data * @fwspec: IOMMU fwspec data * @iommu_dev: IOMMU device this device is linked to + * @pasid_array: pasid-indexed array of domains attached to pasid * @priv: IOMMU Driver private data * @max_pasids: number of PASIDs this device can consume * @attach_deferred: the dma domain attachment is deferred @@ -427,6 +428,7 @@ struct dev_iommu { struct iopf_device_param *iopf_param; struct iommu_fwspec *fwspec; struct iommu_device *iommu_dev; + struct xarray pasid_array; void *priv; u32 max_pasids; u32 attach_deferred:1; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index d4a06a37ce39..35cccfb5c24a 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -51,7 +51,6 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; - struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -310,6 +309,7 @@ static struct dev_iommu *dev_iommu_get(struct device *dev) return NULL; mutex_init(¶m->lock); + xa_init(¶m->pasid_array); dev->iommu = param; return param; } @@ -919,7 +919,6 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); - xa_init(&group->pasid_array); ret = ida_alloc(&iommu_group_ida, GFP_KERNEL); if (ret < 0) { @@ -3135,9 +3134,15 @@ static bool iommu_is_default_domain(struct iommu_group *group) */ static bool assert_pasid_dma_ownership(struct iommu_group *group) { + struct group_device *device; + lockdep_assert_held(&group->mutex); + for_each_group_device(group, device) { + if (WARN_ON(!xa_empty(&device->dev->iommu->pasid_array))) + return false; + } - return !WARN_ON(!xa_empty(&group->pasid_array)); + return true; } /** @@ -3371,33 +3376,6 @@ bool iommu_group_dma_owner_claimed(struct iommu_group *group) } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); -static int __iommu_set_group_pasid(struct iommu_domain *domain, - struct iommu_group *group, ioasid_t pasid) -{ - struct group_device *device; - int ret = 0; - - for_each_group_device(group, device) { - ret = domain->ops->set_dev_pasid(domain, device->dev, pasid); - if (ret) - break; - } - - return ret; -} - -static void __iommu_remove_group_pasid(struct iommu_group *group, - ioasid_t pasid) -{ - struct group_device *device; - const struct iommu_ops *ops; - - for_each_group_device(group, device) { - ops = dev_iommu_ops(device->dev); - ops->remove_dev_pasid(device->dev, pasid); - } -} - /* * iommu_attach_device_pasid() - Attach a domain to pasid of device * @domain: the iommu domain. @@ -3411,6 +3389,7 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, { /* Caller must be a probed driver on dev */ struct iommu_group *group = dev->iommu_group; + struct dev_iommu *param = dev->iommu; void *curr; int ret; @@ -3422,23 +3401,23 @@ int iommu_attach_device_pasid(struct iommu_domain *domain, mutex_lock(&group->mutex); if (list_count_nodes(&group->devices) != 1) { - ret = -EINVAL; - goto out_unlock; + mutex_unlock(&group->mutex); + return -EINVAL; } + mutex_unlock(&group->mutex); - curr = xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + mutex_lock(¶m->lock); + curr = xa_cmpxchg(¶m->pasid_array, pasid, NULL, domain, GFP_KERNEL); if (curr) { ret = xa_err(curr) ? : -EBUSY; goto out_unlock; } - ret = __iommu_set_group_pasid(domain, group, pasid); - if (ret) { - __iommu_remove_group_pasid(group, pasid); - xa_erase(&group->pasid_array, pasid); - } + ret = domain->ops->set_dev_pasid(domain, dev, pasid); + if (ret) + xa_erase(¶m->pasid_array, pasid); out_unlock: - mutex_unlock(&group->mutex); + mutex_unlock(¶m->lock); return ret; } EXPORT_SYMBOL_GPL(iommu_attach_device_pasid); @@ -3455,13 +3434,13 @@ EXPORT_SYMBOL_GPL(iommu_attach_device_pasid); void iommu_detach_device_pasid(struct iommu_domain *domain, struct device *dev, ioasid_t pasid) { - /* Caller must be a probed driver on dev */ - struct iommu_group *group = dev->iommu_group; + const struct iommu_ops *ops = dev_iommu_ops(dev); + struct dev_iommu *param = dev->iommu; - mutex_lock(&group->mutex); - __iommu_remove_group_pasid(group, pasid); - WARN_ON(xa_erase(&group->pasid_array, pasid) != domain); - mutex_unlock(&group->mutex); + mutex_lock(¶m->lock); + ops->remove_dev_pasid(dev, pasid); + WARN_ON(xa_erase(¶m->pasid_array, pasid) != domain); + mutex_unlock(¶m->lock); } EXPORT_SYMBOL_GPL(iommu_detach_device_pasid); @@ -3483,18 +3462,13 @@ struct iommu_domain *iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid, unsigned int type) { - /* Caller must be a probed driver on dev */ - struct iommu_group *group = dev->iommu_group; struct iommu_domain *domain; - if (!group) - return NULL; - - xa_lock(&group->pasid_array); - domain = xa_load(&group->pasid_array, pasid); + xa_lock(&dev->iommu->pasid_array); + domain = xa_load(&dev->iommu->pasid_array, pasid); if (type && domain && domain->type != type) domain = ERR_PTR(-EBUSY); - xa_unlock(&group->pasid_array); + xa_unlock(&dev->iommu->pasid_array); return domain; }