From patchwork Tue Dec 5 01:21:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 173637 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3150381vqy; Mon, 4 Dec 2023 17:26:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IEzit/DmgK5a/CFdZE0s5KwRexlcnttIs9YaJP9d+YsRDnNtsj3Uq8tU4ga44eFSq3YvaTS X-Received: by 2002:a17:902:8f8c:b0:1d0:6ffd:6e81 with SMTP id z12-20020a1709028f8c00b001d06ffd6e81mr2730837plo.121.1701739615468; Mon, 04 Dec 2023 17:26:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701739615; cv=none; d=google.com; s=arc-20160816; b=tASmUP1Cb53iS3gU0ls8CnXoChKMW11fLsr5m4wKpj03oSp+WRpNPDU1OXl/vtcd+T PRE/JVv6pQclcdrvkexXXiRRAlbRq2fz54KXvpJktDzNK8KeizDzO8NTfTTuZ0tKZtYm ID/Ygg2L83RYZiQBtnKoz0AVpo5ZhDeWWdDSxhndDk8Z1vG+0dG9FwJWJAVqElHcQLV3 FkiAks8km3bKRrCN0xsEXrF7giRvx6oD6OgG5NHg8koN60NHQhpCv3b+4mnrnl42/2Uh SyuA0wHnTKEil3qRM8OSMcza9hChyJwMQBhgOJbAIbYkB/IZ6RVjJEgNBZxh8U6akWNL 6zMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=h++Neoncd1/PzwitZoVkDEc546fX7t9BWg7GiqXvjMo=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=X7Ss6sii+z2DjjgWsLgUOriWBC9+38B4ldhgPVGALx1BehJVYMZfEdN+hgZEUlG0jm H1scDx9+yKEb7ThWrEf/iZHeMBNtiOAnZ328+Wn3c0Hx61trHLqN+by0xgITaE23nb/x F4DHp+dqLtFfqbmB8qvalc/SAfNEjsFhDxPww/MI16YUtp5XGsGJwOP8VvHsoVfkkPWA 07zmFhbQ08xiSY13TIYBRZp5q26CfO55z9EoUH08frrT1fiaa9M1uHgLOo8HIKb6tbaY KSHSoNW/fRDZM+3yuk0Vu7Xn7ZjFNzIFTFM5ULlSwUA0nUhYHw7PaxPOPutSihhdXVXP L78w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MNAO7IeA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id q10-20020a170902daca00b001d0bcb62f98si1118432plx.368.2023.12.04.17.26.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:26:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MNAO7IeA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6B5CA8047565; Mon, 4 Dec 2023 17:26:54 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229575AbjLEB0m (ORCPT + 99 others); Mon, 4 Dec 2023 20:26:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230128AbjLEB0i (ORCPT ); Mon, 4 Dec 2023 20:26:38 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AB83A4 for ; Mon, 4 Dec 2023 17:26:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701739604; x=1733275604; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8FCirIQaoMt87kwAb+y8rC0FuOBeH82W4TnVNvwESEg=; b=MNAO7IeAyN2YgsnfUYtT/kUm8q00bFnUQPavtk4+WVAk6OAa6xCZF44q DeQhnKJtcN5rVgquxYYx68MOj2W9ySghJqz/hegkScM6pQTXYTwwK2ZWI vINxYTkWMVn2PS5I7W/S5DSBpSsu5ttWsKZEp8ihuEhHZjGSWVCKkfgLg oxrKdjf2Rt0OnTW8n5q5OaXx1ObmrZwp0iqoiPY6bp9jgrfsNcNcgCPeA HTfXKL8tD9vjMrMjakdtqbi8R2UaVX8Yr5ETsLtNretLydmiwyAeSOQlh 5nx17XwPK9dyVGE/h2krFjE0GIemsiCc+qaIBw2bpl9A2+DEi47PEeOy3 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="460313330" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="460313330" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 17:26:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="1102276189" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="1102276189" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmsmga005.fm.intel.com with ESMTP; 04 Dec 2023 17:26:41 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 1/6] iommu/vt-d: Setup scalable mode context entry in probe path Date: Tue, 5 Dec 2023 09:21:58 +0800 Message-Id: <20231205012203.244584-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231205012203.244584-1-baolu.lu@linux.intel.com> References: <20231205012203.244584-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 04 Dec 2023 17:26:54 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784403318878497621 X-GMAIL-MSGID: 1784403318878497621 In contrast to legacy mode, the DMA translation table is configured in the PASID table entry instead of the context entry for scalable mode. For this reason, it is more appropriate to set up the scalable mode context entry in the device_probe callback and direct it to the appropriate PASID table. The iommu domain attach/detach operations only affect the PASID table entry. Therefore, there is no need to modify the context entry when configuring the translation type and page table. The only exception is the kdump case, where context entry setup is postponed until the device driver invokes the first DMA interface. Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.h | 1 + drivers/iommu/intel/iommu.c | 17 +++- drivers/iommu/intel/pasid.c | 180 ++++++++++++++++++++++++++++++++++++ 3 files changed, 195 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/intel/pasid.h b/drivers/iommu/intel/pasid.h index 8d40d4c66e31..58d7049081b9 100644 --- a/drivers/iommu/intel/pasid.h +++ b/drivers/iommu/intel/pasid.h @@ -319,4 +319,5 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu, bool fault_ignore); void intel_pasid_setup_page_snoop_control(struct intel_iommu *iommu, struct device *dev, u32 pasid); +int intel_pasid_setup_sm_context(struct device *dev, bool deferred); #endif /* __INTEL_PASID_H */ diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 84b78e42a470..9dc005031dd2 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4270,15 +4270,26 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) ret = intel_pasid_alloc_table(dev); if (ret) { dev_err(dev, "PASID table allocation failed\n"); - dev_iommu_priv_set(dev, NULL); - kfree(info); - return ERR_PTR(ret); + goto err_clear_priv; + } + + ret = intel_pasid_setup_sm_context(dev, false); + if (ret) { + dev_err(dev, "Scalable context entry setup failed\n"); + goto err_free_table; } } intel_iommu_debugfs_create_dev(info); return &iommu->iommu; +err_free_table: + intel_pasid_free_table(dev); +err_clear_priv: + dev_iommu_priv_set(dev, NULL); + kfree(info); + + return ERR_PTR(ret); } static void intel_iommu_release_device(struct device *dev) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 3239cefa4c33..9e505060617a 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -304,6 +304,11 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu, return -EINVAL; } + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err(dev, "Context entry is not configured\n"); + return -ENODEV; + } + spin_lock(&iommu->lock); pte = intel_pasid_get_entry(dev, pasid); if (!pte) { @@ -384,6 +389,11 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu, return -EINVAL; } + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err(dev, "Context entry is not configured\n"); + return -ENODEV; + } + pgd = domain->pgd; agaw = iommu_skip_agaw(domain, iommu, &pgd); if (agaw < 0) { @@ -505,6 +515,11 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu, u16 did = FLPT_DEFAULT_DID; struct pasid_entry *pte; + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err(dev, "Context entry is not configured\n"); + return -ENODEV; + } + spin_lock(&iommu->lock); pte = intel_pasid_get_entry(dev, pasid); if (!pte) { @@ -623,6 +638,11 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu, struct device *dev, return -EINVAL; } + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err_ratelimited(dev, "Context entry is not configured\n"); + return -ENODEV; + } + spin_lock(&iommu->lock); pte = intel_pasid_get_entry(dev, pasid); if (!pte) { @@ -666,3 +686,163 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu, struct device *dev, return 0; } + +/* + * Interface to set a pasid table to the scalable mode context table entry: + */ + +/* + * Get the PASID directory size for scalable mode context entry. + * Value of X in the PDTS field of a scalable mode context entry + * indicates PASID directory with 2^(X + 7) entries. + */ +static unsigned long context_get_sm_pds(struct pasid_table *table) +{ + unsigned long pds, max_pde; + + max_pde = table->max_pasid >> PASID_PDE_SHIFT; + pds = find_first_bit(&max_pde, MAX_NR_PASID_BITS); + if (pds < 7) + return 0; + + return pds - 7; +} + +static int context_entry_set_pasid_table(struct context_entry *context, + struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct pasid_table *table = info->pasid_table; + struct intel_iommu *iommu = info->iommu; + unsigned long pds; + + context_clear_entry(context); + + pds = context_get_sm_pds(table); + context->lo = (u64)virt_to_phys(table->table) | context_pdts(pds); + context_set_sm_rid2pasid(context, IOMMU_NO_PASID); + + if (info->ats_supported) + context_set_sm_dte(context); + if (info->pri_supported) + context_set_sm_pre(context); + if (info->pasid_supported) + context_set_pasid(context); + + context_set_fault_enable(context); + context_set_present(context); + if (!ecap_coherent(iommu->ecap)) + clflush_cache_range(context, sizeof(*context)); + + return 0; +} + +static int device_pasid_table_setup(struct device *dev, u8 bus, u8 devfn) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + struct context_entry *context; + int ret = 0; + + spin_lock(&iommu->lock); + context = iommu_context_addr(iommu, bus, devfn, true); + if (!context) { + ret = -ENOMEM; + goto out_unlock; + } + + if (context_present(context) && !context_copied(iommu, bus, devfn)) + goto out_unlock; + + /* + * Cache invalidation for changes to a scalable-mode context table + * entry. + * + * Section 6.5.3.3 of the VT-d spec: + * - Device-selective context-cache invalidation; + * - Domain-selective PASID-cache invalidation to affected domains + * (can be skipped if all PASID entries were not-present); + * - Domain-selective IOTLB invalidation to affected domains; + * - Global Device-TLB invalidation to affected functions. + * + * For kdump cases, old valid entries may be cached due to the + * in-flight DMA and copied pgtable, but there is no unmapping + * behaviour for them, thus we need explicit cache flushes for all + * affected domain IDs and PASIDs used in the copied PASID table. + * Given that we have no idea about which domain IDs and PASIDs were + * used in the copied tables, upgrade them to global PASID and IOTLB + * cache invalidation. + * + * For kdump case, at this point, the device is supposed to finish + * reset at its driver probe stage, so no in-flight DMA will exist, + * and we don't need to worry anymore hereafter. + */ + if (context_copied(iommu, bus, devfn)) { + context_clear_entry(context); + clear_context_copied(iommu, bus, devfn); + iommu->flush.flush_context(iommu, 0, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + qi_flush_pasid_cache(iommu, 0, QI_PC_GLOBAL, 0); + iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH); + devtlb_invalidation_with_pasid(iommu, dev, IOMMU_NO_PASID); + } + + context_entry_set_pasid_table(context, dev); + + /* + * It's a non-present to present mapping. If hardware doesn't cache + * non-present entry we only need to flush the write-buffer. If the + * _does_ cache non-present entries, then it does so in the special + * domain ID #0, which we have to flush: + */ + if (cap_caching_mode(iommu->cap)) { + iommu->flush.flush_context(iommu, 0, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH); + } else { + iommu_flush_write_buffer(iommu); + } + +out_unlock: + spin_unlock(&iommu->lock); + return ret; +} + +static int pci_pasid_table_setup(struct pci_dev *pdev, u16 alias, void *data) +{ + struct device *dev = data; + + if (dev != &pdev->dev) + return 0; + + return device_pasid_table_setup(dev, PCI_BUS_NUM(alias), alias & 0xff); +} + +/* + * Set the device's PASID table to its context table entry. + * + * The PASID table is set to the context entries of both device itself + * and its alias requester ID for DMA. If it is called in domain attach + * paths, set @deferred to true, false in other cases. + */ +int intel_pasid_setup_sm_context(struct device *dev, bool deferred) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + + /* + * Skip pasid table setting up if context entry is copied and + * function is not called in deferred attachment context. + */ + if (deferred ^ context_copied(iommu, info->bus, info->devfn)) + return 0; + + if (!dev_is_pci(dev)) + return device_pasid_table_setup(dev, info->bus, info->devfn); + + return pci_for_each_dma_alias(to_pci_dev(dev), pci_pasid_table_setup, dev); +} From patchwork Tue Dec 5 01:21:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 173639 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3150417vqy; Mon, 4 Dec 2023 17:27:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IH9CDPtpNxzcI2Qxto1wp8KD0rN6cIA9PDJcUSlpistInIgdgLyvEjROq9KfHTSHDltg67Y X-Received: by 2002:a05:6358:896:b0:170:911:8b80 with SMTP id m22-20020a056358089600b0017009118b80mr4587313rwj.28.1701739625182; Mon, 04 Dec 2023 17:27:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701739625; cv=none; d=google.com; s=arc-20160816; b=XSOYI5LSlimES9U4OpGAyHSIit1rtQBtz9j+PCGsWbz9WqfVwNXp/CUtb2uS8r6j/+ n4FhaLUKzU95BKGiQnZDWy9erw+QcxS3aO4teQn0AEaKgXuCw7SKGTvv6GfAyZ0MFtwE QBJdmDkwcx6L7wVhaQ8mxr9bIWOoTmqwQ1Oj5wBSiuxTwEnb8sQuPBiQOGECjZVfuAYq OnMgf5YvcEc9MZB5s5+tt6GiSWR5lwVjUuZRRhdeACX1plzj1ltLqAw5TyGZtbGaArF/ hGQYCJHH5wkcIDvTtlOS/D8oBs43EsR6vZ55l4s7AQ9xn0WAXNLwzaa3I1b714Otv1U6 p7XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=D9783EcctUH9T4YtZ6ODfJcco64c8ZJo4us428xrASo=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=0+fJtyA3dIJESK8iAVRRD/MdJ6tf09f4vv/mHm3xxa8vByIZdt2dy+C9V6KIAuMRqi v+86hD6WO2jLmcI/krN9sIVrnqbLaC+TS7z/g7bY6TopCyuRyBFYaS9AusjAHwH77+T7 T3J0bMZmuQBbkuSHbiOgtu9b4mByvwG8Kvkxnw9PzJ4j2XPzsPi/LzorqnRWI7Ke6qD0 noLrMYb+WF4lzreYK/0TPsN6bqIBVHPzkiVAVmwZ1tCcuB7EDgEL78knIXcFWYpnA1dW ss8nEUZgMewAcMllBneTlRyCTcW1jmrTl/ZidwZ62Tv2xZ5nO0RpM4Ij7SvXmc30ULob w7og== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Ju2X6AqR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id w6-20020a636206000000b005c666ab748bsi4502162pgb.335.2023.12.04.17.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:27:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Ju2X6AqR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 0970280C6302; Mon, 4 Dec 2023 17:26:57 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343874AbjLEB0p (ORCPT + 99 others); Mon, 4 Dec 2023 20:26:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231593AbjLEB0k (ORCPT ); Mon, 4 Dec 2023 20:26:40 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D756CCD for ; Mon, 4 Dec 2023 17:26:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701739606; x=1733275606; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nBO6r/Mialf5vBFGAoopzywwKjVALm8cMNFioZBcghU=; b=Ju2X6AqR8Zi+GFlMSvFwKe7OIMmVrecAd89TaM2lWf+ZqlYS8KpxoVeD 0lGfjVxs/LuTy6mPCit3gi+DLSi9SJOBg7C5x4fkpqGCfQg2BqVe6ZpWi r9ICNy4QmbynhROdxAUXWRf+lcS44Wb+yPaSuDMZ7X1epYlOMz9L+zB6A FAe1VRxdXC6ajgU4nBX9cuLINIyVEv058/FpIhlkFj2XW80a2IPFOv1Ae KYDx0Lql5ZXslcHtlglSvwzeQOUGmHzm/u6I7nW8aSvU7I+rE7KfOForg 9sq9hJgvg14RyQt9/1oum7qeL4JGycwyx7U+ea3aTerHUlxmp/I3nWBGE w==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="460313342" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="460313342" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 17:26:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="1102276212" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="1102276212" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmsmga005.fm.intel.com with ESMTP; 04 Dec 2023 17:26:44 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 2/6] iommu/vt-d: Remove scalable mode context entry setup from attach_dev Date: Tue, 5 Dec 2023 09:21:59 +0800 Message-Id: <20231205012203.244584-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231205012203.244584-1-baolu.lu@linux.intel.com> References: <20231205012203.244584-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 04 Dec 2023 17:26:57 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784403329145957542 X-GMAIL-MSGID: 1784403329145957542 The scalable mode context entry is now setup in the probe_device path, eliminating the need to configure it in the attach_dev path. Removes the redundant code from the attach_dev path to avoid dead code. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 156 ++++++++++-------------------------- 1 file changed, 44 insertions(+), 112 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 9dc005031dd2..a324b3a3a005 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1727,34 +1727,17 @@ static void domain_exit(struct dmar_domain *domain) kfree(domain); } -/* - * Get the PASID directory size for scalable mode context entry. - * Value of X in the PDTS field of a scalable mode context entry - * indicates PASID directory with 2^(X + 7) entries. - */ -static unsigned long context_get_sm_pds(struct pasid_table *table) -{ - unsigned long pds, max_pde; - - max_pde = table->max_pasid >> PASID_PDE_SHIFT; - pds = find_first_bit(&max_pde, MAX_NR_PASID_BITS); - if (pds < 7) - return 0; - - return pds - 7; -} - static int domain_context_mapping_one(struct dmar_domain *domain, struct intel_iommu *iommu, - struct pasid_table *table, u8 bus, u8 devfn) { struct device_domain_info *info = domain_lookup_dev_info(domain, iommu, bus, devfn); u16 did = domain_id_iommu(domain, iommu); int translation = CONTEXT_TT_MULTI_LEVEL; + struct dma_pte *pgd = domain->pgd; struct context_entry *context; - int ret; + int agaw, ret; if (hw_pass_through && domain_type_is_si(domain)) translation = CONTEXT_TT_PASS_THROUGH; @@ -1797,65 +1780,37 @@ static int domain_context_mapping_one(struct dmar_domain *domain, } context_clear_entry(context); + context_set_domain_id(context, did); - if (sm_supported(iommu)) { - unsigned long pds; - - /* Setup the PASID DIR pointer: */ - pds = context_get_sm_pds(table); - context->lo = (u64)virt_to_phys(table->table) | - context_pdts(pds); - - /* Setup the RID_PASID field: */ - context_set_sm_rid2pasid(context, IOMMU_NO_PASID); - + if (translation != CONTEXT_TT_PASS_THROUGH) { /* - * Setup the Device-TLB enable bit and Page request - * Enable bit: + * Skip top levels of page tables for iommu which has + * less agaw than default. Unnecessary for PT mode. */ + for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { + ret = -ENOMEM; + pgd = phys_to_virt(dma_pte_addr(pgd)); + if (!dma_pte_present(pgd)) + goto out_unlock; + } + if (info && info->ats_supported) - context_set_sm_dte(context); - if (info && info->pri_supported) - context_set_sm_pre(context); - if (info && info->pasid_supported) - context_set_pasid(context); + translation = CONTEXT_TT_DEV_IOTLB; + else + translation = CONTEXT_TT_MULTI_LEVEL; + + context_set_address_root(context, virt_to_phys(pgd)); + context_set_address_width(context, agaw); } else { - struct dma_pte *pgd = domain->pgd; - int agaw; - - context_set_domain_id(context, did); - - if (translation != CONTEXT_TT_PASS_THROUGH) { - /* - * Skip top levels of page tables for iommu which has - * less agaw than default. Unnecessary for PT mode. - */ - for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { - ret = -ENOMEM; - pgd = phys_to_virt(dma_pte_addr(pgd)); - if (!dma_pte_present(pgd)) - goto out_unlock; - } - - if (info && info->ats_supported) - translation = CONTEXT_TT_DEV_IOTLB; - else - translation = CONTEXT_TT_MULTI_LEVEL; - - context_set_address_root(context, virt_to_phys(pgd)); - context_set_address_width(context, agaw); - } else { - /* - * In pass through mode, AW must be programmed to - * indicate the largest AGAW value supported by - * hardware. And ASR is ignored by hardware. - */ - context_set_address_width(context, iommu->msagaw); - } - - context_set_translation_type(context, translation); + /* + * In pass through mode, AW must be programmed to + * indicate the largest AGAW value supported by + * hardware. And ASR is ignored by hardware. + */ + context_set_address_width(context, iommu->msagaw); } + context_set_translation_type(context, translation); context_set_fault_enable(context); context_set_present(context); if (!ecap_coherent(iommu->ecap)) @@ -1885,43 +1840,29 @@ static int domain_context_mapping_one(struct dmar_domain *domain, return ret; } -struct domain_context_mapping_data { - struct dmar_domain *domain; - struct intel_iommu *iommu; - struct pasid_table *table; -}; - static int domain_context_mapping_cb(struct pci_dev *pdev, u16 alias, void *opaque) { - struct domain_context_mapping_data *data = opaque; + struct device_domain_info *info = dev_iommu_priv_get(&pdev->dev); + struct intel_iommu *iommu = info->iommu; + struct dmar_domain *domain = opaque; - return domain_context_mapping_one(data->domain, data->iommu, - data->table, PCI_BUS_NUM(alias), - alias & 0xff); + return domain_context_mapping_one(domain, iommu, + PCI_BUS_NUM(alias), alias & 0xff); } static int domain_context_mapping(struct dmar_domain *domain, struct device *dev) { struct device_domain_info *info = dev_iommu_priv_get(dev); - struct domain_context_mapping_data data; struct intel_iommu *iommu = info->iommu; u8 bus = info->bus, devfn = info->devfn; - struct pasid_table *table; - - table = intel_pasid_get_table(dev); if (!dev_is_pci(dev)) - return domain_context_mapping_one(domain, iommu, table, - bus, devfn); - - data.domain = domain; - data.iommu = iommu; - data.table = table; + return domain_context_mapping_one(domain, iommu, bus, devfn); return pci_for_each_dma_alias(to_pci_dev(dev), - &domain_context_mapping_cb, &data); + domain_context_mapping_cb, domain); } /* Returns a number of VTD pages, but aligned to MM page size */ @@ -2278,28 +2219,19 @@ static int dmar_domain_attach_device(struct dmar_domain *domain, list_add(&info->link, &domain->devices); spin_unlock_irqrestore(&domain->lock, flags); - /* PASID table is mandatory for a PCI device in scalable mode. */ - if (sm_supported(iommu) && !dev_is_real_dma_subdevice(dev)) { - /* Setup the PASID entry for requests without PASID: */ - if (hw_pass_through && domain_type_is_si(domain)) - ret = intel_pasid_setup_pass_through(iommu, - dev, IOMMU_NO_PASID); - else if (domain->use_first_level) - ret = domain_setup_first_level(iommu, domain, dev, - IOMMU_NO_PASID); - else - ret = intel_pasid_setup_second_level(iommu, domain, - dev, IOMMU_NO_PASID); - if (ret) { - dev_err(dev, "Setup RID2PASID failed\n"); - device_block_translation(dev); - return ret; - } - } + if (dev_is_real_dma_subdevice(dev)) + return 0; + + if (!sm_supported(iommu)) + ret = domain_context_mapping(domain, dev); + else if (hw_pass_through && domain_type_is_si(domain)) + ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID); + else if (domain->use_first_level) + ret = domain_setup_first_level(iommu, domain, dev, IOMMU_NO_PASID); + else + ret = intel_pasid_setup_second_level(iommu, domain, dev, IOMMU_NO_PASID); - ret = domain_context_mapping(domain, dev); if (ret) { - dev_err(dev, "Domain context map failed\n"); device_block_translation(dev); return ret; } From patchwork Tue Dec 5 01:22:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 173638 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3150408vqy; Mon, 4 Dec 2023 17:27:01 -0800 (PST) X-Google-Smtp-Source: AGHT+IELO+Tr2r22+vjB5b1QLaaoYMHZOj5UsyJS9grNu+c61eQ8fswo1WSqQiHxDsyo00pIng/9 X-Received: by 2002:a05:6a00:e0a:b0:6ce:38a2:c5d9 with SMTP id bq10-20020a056a000e0a00b006ce38a2c5d9mr652380pfb.26.1701739621544; Mon, 04 Dec 2023 17:27:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701739621; cv=none; d=google.com; s=arc-20160816; b=ER3D0mKv6vLaxW4YzU2ni5HQXpCa3n1eOTEgeEU7DCOpmfKatos/rdhqixpG/Lh6li KUDFhH4A9/imlGNiOWOQZtgpQmT09TVymxsYOtuIxIEOztp0cdGeCagoF/5mRqYufI4L J+Jp7V4QOUy7aDTNrnZDWt5M0FviX/BMM0JW3WKgU/raW44m938HNDW0BRy0z+vTcPh8 1rlJGFL0itfkba7Wi0NI4cYTHdvWE19zLpBZ/qwIxj5ZAxLrcotQNDPtWCuj3Zq87yql nxoGvaSvN/6l/jyawnLfMXPLrqWMo6tHg3MB4VSfn6amno+To1opUDp7x5Jeylfjr+tS OUCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tpDdb98tk8zw1cj5dzIynH6qv+WHcH2U079WXO5IXls=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=HDECbhqWYC2Nu5MwDArXi+Qx4V/fTvQVlcrRYqHJdmKROHNCYIknXvZqSpZjDuY1dK 2MSnuLSlDNXEd21b02ou7pnUt9LEhcqgrrlsnQVozBp0ZwgZHxgsmBLd4WPbk5gFWUCY dPoFimxwJnrzB0ciKa9SeeQTT+/Wc9F2zCMWNGAemfFYfBGsnIgmVPTzKVObt2L3KCZd vlJeW7A2+XjsrtEzj5gLzs4qg5WF5lfXg/RPeOnI87YFPugtthFLcPxZYe26rLK+kW3R d6l7KwjC4tvW8dMs6DLaunpye9hT5aiiDpC5+QwFgxWEreNKJeRFnO9vTKzYrBpVAp7G 7R4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kGbXTOfw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id ka25-20020a056a00939900b006cbb132d46dsi4229446pfb.244.2023.12.04.17.27.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:27:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kGbXTOfw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6D8B38047479; Mon, 4 Dec 2023 17:27:00 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234803AbjLEB0r (ORCPT + 99 others); Mon, 4 Dec 2023 20:26:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343956AbjLEB0n (ORCPT ); Mon, 4 Dec 2023 20:26:43 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F339135 for ; Mon, 4 Dec 2023 17:26:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701739609; x=1733275609; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Hu+gkVWZE83d4KZHeWPMt+YqjiEu98XZ0PIi+Ei4+Wc=; b=kGbXTOfw7rmc2yHYFRp2bYzEnow3iLpmg9BXkjkBADc6nVk/PPgsdyaz zfHOaFcYUGu830o36kiLSUmDgcu1coITXIPmXjAMJsy3oTON3aLFtDA2N TnOLai1IXQNaxpkNSdUeRK/BccY4ircQ4zxiYE1mmhmMCBKYkIw+0jfju dfbskFqk2tqcL0wMJ9AvVFkeECBNW4t4LKRK9OoUAz6Sn6JHZvvSG9GjS /zhnWP5meGjgv9YOTCRS6EaZNaRYtXagNKt/KwcOk1e5sLGxBvkEanu1t 0ppzIiPhC0nsU+68avY0BloCf9qqvqXiDAC8NsL2lcM14uXV1iQCmtK7n Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="460313349" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="460313349" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 17:26:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="1102276239" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="1102276239" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmsmga005.fm.intel.com with ESMTP; 04 Dec 2023 17:26:46 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 3/6] iommu/vt-d: Refactor domain_context_mapping_one() to be reusable Date: Tue, 5 Dec 2023 09:22:00 +0800 Message-Id: <20231205012203.244584-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231205012203.244584-1-baolu.lu@linux.intel.com> References: <20231205012203.244584-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 04 Dec 2023 17:27:00 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784403325309140527 X-GMAIL-MSGID: 1784403325309140527 Extract common code from domain_context_mapping_one() into new functions, making it reusable by other functions such as the upcoming identity domain implementation. No intentional functional changes. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 99 ++++++++++++++++++++++--------------- 1 file changed, 58 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index a324b3a3a005..605cd1c52e95 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1727,6 +1727,61 @@ static void domain_exit(struct dmar_domain *domain) kfree(domain); } +/* + * For kdump cases, old valid entries may be cached due to the + * in-flight DMA and copied pgtable, but there is no unmapping + * behaviour for them, thus we need an explicit cache flush for + * the newly-mapped device. For kdump, at this point, the device + * is supposed to finish reset at its driver probe stage, so no + * in-flight DMA will exist, and we don't need to worry anymore + * hereafter. + */ +static void copied_context_tear_down(struct intel_iommu *iommu, + struct context_entry *context, + u8 bus, u8 devfn) +{ + u16 did_old; + + if (!context_copied(iommu, bus, devfn)) + return; + + assert_spin_locked(&iommu->lock); + + did_old = context_domain_id(context); + context_clear_entry(context); + + if (did_old < cap_ndoms(iommu->cap)) { + iommu->flush.flush_context(iommu, did_old, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + iommu->flush.flush_iotlb(iommu, did_old, 0, 0, + DMA_TLB_DSI_FLUSH); + } + + clear_context_copied(iommu, bus, devfn); +} + +/* + * It's a non-present to present mapping. If hardware doesn't cache + * non-present entry we only need to flush the write-buffer. If the + * _does_ cache non-present entries, then it does so in the special + * domain #0, which we have to flush: + */ +static void context_present_cache_flush(struct intel_iommu *iommu, u16 did, + u8 bus, u8 devfn) +{ + if (cap_caching_mode(iommu->cap)) { + iommu->flush.flush_context(iommu, 0, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); + } else { + iommu_flush_write_buffer(iommu); + } +} + static int domain_context_mapping_one(struct dmar_domain *domain, struct intel_iommu *iommu, u8 bus, u8 devfn) @@ -1755,31 +1810,9 @@ static int domain_context_mapping_one(struct dmar_domain *domain, if (context_present(context) && !context_copied(iommu, bus, devfn)) goto out_unlock; - /* - * For kdump cases, old valid entries may be cached due to the - * in-flight DMA and copied pgtable, but there is no unmapping - * behaviour for them, thus we need an explicit cache flush for - * the newly-mapped device. For kdump, at this point, the device - * is supposed to finish reset at its driver probe stage, so no - * in-flight DMA will exist, and we don't need to worry anymore - * hereafter. - */ - if (context_copied(iommu, bus, devfn)) { - u16 did_old = context_domain_id(context); - - if (did_old < cap_ndoms(iommu->cap)) { - iommu->flush.flush_context(iommu, did_old, - (((u16)bus) << 8) | devfn, - DMA_CCMD_MASK_NOBIT, - DMA_CCMD_DEVICE_INVL); - iommu->flush.flush_iotlb(iommu, did_old, 0, 0, - DMA_TLB_DSI_FLUSH); - } - - clear_context_copied(iommu, bus, devfn); - } - + copied_context_tear_down(iommu, context, bus, devfn); context_clear_entry(context); + context_set_domain_id(context, did); if (translation != CONTEXT_TT_PASS_THROUGH) { @@ -1815,23 +1848,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain, context_set_present(context); if (!ecap_coherent(iommu->ecap)) clflush_cache_range(context, sizeof(*context)); - - /* - * It's a non-present to present mapping. If hardware doesn't cache - * non-present entry we only need to flush the write-buffer. If the - * _does_ cache non-present entries, then it does so in the special - * domain #0, which we have to flush: - */ - if (cap_caching_mode(iommu->cap)) { - iommu->flush.flush_context(iommu, 0, - (((u16)bus) << 8) | devfn, - DMA_CCMD_MASK_NOBIT, - DMA_CCMD_DEVICE_INVL); - iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); - } else { - iommu_flush_write_buffer(iommu); - } - + context_present_cache_flush(iommu, did, bus, devfn); ret = 0; out_unlock: From patchwork Tue Dec 5 01:22:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 173640 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3150536vqy; Mon, 4 Dec 2023 17:27:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IEnPqQYtHFHEXY81wuwuYrdVDs+XsRVCf8mbrjQS4/D3PyefL7cqTJd4NfNzzGyURJnNFUa X-Received: by 2002:a05:6870:5587:b0:1fb:75a:de77 with SMTP id qj7-20020a056870558700b001fb075ade77mr5942975oac.101.1701739642262; Mon, 04 Dec 2023 17:27:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701739642; cv=none; d=google.com; s=arc-20160816; b=MWFLfc6p4dOAKxJevfZjoQp4/UXK/MsdSIeEKJlxM5SDnfejdH2kbeuECbpJPj8unZ CSGvQwrnV/uMtyfwR5AB/uN+aT2X/CwZmICmbhQJT0Xpww44ok6MCTb1uYCKqCuVBGc7 TX/BY9r+v/LPfTMZcbiUTb1jzNZvIj1awDMtrwDPQ9Swl9a0Fbj5ALR7ox66SoGoZG6p ztJ6GrBw379cwLG7JtrZyY7bUzsSaZqQK/9DHv7NK4N2enrsRZDDOyGQQIveY23XzNhV YdKR5ascMCxt1hTQF9X/bWo87+o6y1RGk/Jh4Q6vqa9e9TPniy1Qc0GFE5gkGinrgshM k0QQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9AL6iUv3Y0ax5BwVbY+fewfcg6z+TqTgjX69TVUAruU=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=K6eS0KevrnzvRygh5VyQAnur3LSU3812EYbFZi2teiDyA0xdEgufqI5mjfTAmzGVmO vilz9JsVr3YlsZVHflDeXSbsdXV9H4S5nPgxr0U6VS2O4KtEYX3WGJkqLwMQyNfTbqXt /LkKZbsUmO8t1gDO8eskub/GHzMvs+opiylFYaumW/J1mo4UFZwHxUSN6GQzUrh+A/GE AJsj2GfSye/zumjAUbsQfL80QYZ4dnr9INh74HqtkxnUfGUNSKMYqQJERd9ECz/Yh8SC d+mWcOoLctSfyZHpOaSb86kiw2mWZlnFhYEc10vmBA4hZwMOXGi6jVGXeQ17el/8gVr7 lXUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lqYLp0A4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id h10-20020a63f90a000000b005c668a5a906si4191999pgi.282.2023.12.04.17.27.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:27:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lqYLp0A4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id AB93280220B2; Mon, 4 Dec 2023 17:27:18 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346223AbjLEB0u (ORCPT + 99 others); Mon, 4 Dec 2023 20:26:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343885AbjLEB0p (ORCPT ); Mon, 4 Dec 2023 20:26:45 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90B5010F for ; Mon, 4 Dec 2023 17:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701739611; x=1733275611; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UMkeLFUpYAwPaBYnzxx/I6VFYpGlFn70X93po/+cGE0=; b=lqYLp0A4kNT760HrbdSo1KwKyMJ3G9KFKqYsAxfkMH5JUZ9XZ1slPe6w JTeAgHGnIGEyo0YZl1aaCB8gWxKiYyQFJ+La1l8rF3oy98kQChkehvaC5 8j6ByK7c+2r7Ls8ysl+BzFohww1iZnHM5nNh0T+XqQ2iL4gZExvRIU1V+ 2Ho/49FnS8X21wrzKZ963UDxSPGUwZrPjO4q/wZINQ9gVgNJd+9GZBi9q AHzOzsfAq8jg/9RCvl8wO3mWDTlP8SOlpNG2z3Bnq3uxypVzLE0gPteRr q7AHOfBRoSI/1X4BcDN35fJlgzMHgAMRDnujxfaJ77U4yIpXv+FX7806l Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="460313355" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="460313355" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 17:26:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="1102276279" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="1102276279" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmsmga005.fm.intel.com with ESMTP; 04 Dec 2023 17:26:49 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 4/6] iommu/vt-d: Remove 1:1 mappings from identity domain Date: Tue, 5 Dec 2023 09:22:01 +0800 Message-Id: <20231205012203.244584-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231205012203.244584-1-baolu.lu@linux.intel.com> References: <20231205012203.244584-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Mon, 04 Dec 2023 17:27:18 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784403346870766218 X-GMAIL-MSGID: 1784403346870766218 Older VT-d hardware implementations did not support pass-through translation mode. The iommu driver relied on a DMA domain with all physical memory addresses identically mapped to the same IOVA to simulate pass-through translation. This workaround is no longer necessary due to the evolution of iommu core. The core has introduced def_domain_type op, allowing the iommu driver to specify its capabilities. Additionally, the identity domain has become a static system domain with "never fail" attach semantics. Eliminate support for the 1:1 mapping domain on older hardware and removes the unused code that created the 1:1 page table. This paves a way for the implementation of a global static identity domain. Signed-off-by: Lu Baolu Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 118 +++--------------------------------- 1 file changed, 10 insertions(+), 108 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 605cd1c52e95..7022cc183120 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -2146,29 +2146,10 @@ static bool dev_is_real_dma_subdevice(struct device *dev) pci_real_dma_dev(to_pci_dev(dev)) != to_pci_dev(dev); } -static int iommu_domain_identity_map(struct dmar_domain *domain, - unsigned long first_vpfn, - unsigned long last_vpfn) -{ - /* - * RMRR range might have overlap with physical memory range, - * clear it first - */ - dma_pte_clear_range(domain, first_vpfn, last_vpfn); - - return __domain_mapping(domain, first_vpfn, - first_vpfn, last_vpfn - first_vpfn + 1, - DMA_PTE_READ|DMA_PTE_WRITE, GFP_KERNEL); -} - static int md_domain_init(struct dmar_domain *domain, int guest_width); static int __init si_domain_init(int hw) { - struct dmar_rmrr_unit *rmrr; - struct device *dev; - int i, nid, ret; - si_domain = alloc_domain(IOMMU_DOMAIN_IDENTITY); if (!si_domain) return -EFAULT; @@ -2179,44 +2160,6 @@ static int __init si_domain_init(int hw) return -EFAULT; } - if (hw) - return 0; - - for_each_online_node(nid) { - unsigned long start_pfn, end_pfn; - int i; - - for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { - ret = iommu_domain_identity_map(si_domain, - mm_to_dma_pfn_start(start_pfn), - mm_to_dma_pfn_end(end_pfn)); - if (ret) - return ret; - } - } - - /* - * Identity map the RMRRs so that devices with RMRRs could also use - * the si_domain. - */ - for_each_rmrr_units(rmrr) { - for_each_active_dev_scope(rmrr->devices, rmrr->devices_cnt, - i, dev) { - unsigned long long start = rmrr->base_address; - unsigned long long end = rmrr->end_address; - - if (WARN_ON(end < start || - end >> agaw_to_width(si_domain->agaw))) - continue; - - ret = iommu_domain_identity_map(si_domain, - mm_to_dma_pfn_start(start >> PAGE_SHIFT), - mm_to_dma_pfn_end(end >> PAGE_SHIFT)); - if (ret) - return ret; - } - } - return 0; } @@ -2301,6 +2244,9 @@ static bool device_rmrr_is_relaxable(struct device *dev) */ static int device_def_domain_type(struct device *dev) { + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + if (dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(dev); @@ -2311,6 +2257,13 @@ static int device_def_domain_type(struct device *dev) return IOMMU_DOMAIN_IDENTITY; } + /* + * Hardware does not support the passthrough translation mode. + * Always use a dynamaic mapping domain. + */ + if (!ecap_pass_through(iommu->ecap)) + return IOMMU_DOMAIN_DMA; + return 0; } @@ -3301,52 +3254,6 @@ int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info) return 0; } -static int intel_iommu_memory_notifier(struct notifier_block *nb, - unsigned long val, void *v) -{ - struct memory_notify *mhp = v; - unsigned long start_vpfn = mm_to_dma_pfn_start(mhp->start_pfn); - unsigned long last_vpfn = mm_to_dma_pfn_end(mhp->start_pfn + - mhp->nr_pages - 1); - - switch (val) { - case MEM_GOING_ONLINE: - if (iommu_domain_identity_map(si_domain, - start_vpfn, last_vpfn)) { - pr_warn("Failed to build identity map for [%lx-%lx]\n", - start_vpfn, last_vpfn); - return NOTIFY_BAD; - } - break; - - case MEM_OFFLINE: - case MEM_CANCEL_ONLINE: - { - struct dmar_drhd_unit *drhd; - struct intel_iommu *iommu; - LIST_HEAD(freelist); - - domain_unmap(si_domain, start_vpfn, last_vpfn, &freelist); - - rcu_read_lock(); - for_each_active_iommu(iommu, drhd) - iommu_flush_iotlb_psi(iommu, si_domain, - start_vpfn, mhp->nr_pages, - list_empty(&freelist), 0); - rcu_read_unlock(); - put_pages_list(&freelist); - } - break; - } - - return NOTIFY_OK; -} - -static struct notifier_block intel_iommu_memory_nb = { - .notifier_call = intel_iommu_memory_notifier, - .priority = 0 -}; - static void intel_disable_iommus(void) { struct intel_iommu *iommu = NULL; @@ -3643,12 +3550,7 @@ int __init intel_iommu_init(void) iommu_pmu_register(iommu); } - up_read(&dmar_global_lock); - if (si_domain && !hw_pass_through) - register_memory_notifier(&intel_iommu_memory_nb); - - down_read(&dmar_global_lock); if (probe_acpi_namespace_devices()) pr_warn("ACPI name space devices didn't probe correctly\n"); From patchwork Tue Dec 5 01:22:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 173641 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3150602vqy; Mon, 4 Dec 2023 17:27:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IHfJ9L8/IwfzeONcfo2O24cu51iJw+AbniFHko/I/JyATMjJ/RwskAsomhbEQs4fgOVoeye X-Received: by 2002:a05:6a00:4108:b0:6cd:d6eb:2f19 with SMTP id bu8-20020a056a00410800b006cdd6eb2f19mr726614pfb.9.1701739654128; Mon, 04 Dec 2023 17:27:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701739654; cv=none; d=google.com; s=arc-20160816; b=ZIyKEptfQBUS9GeBtXYaLSDGfN17HJiA/nR1JesEOFPmnacxxE37qDNAVlBhOC9otX xMPRS+ViVhfnWtOy1AaV1KsteFV0LZRvEqPycm9WCfGS3nm/mq5GMdWNayclbEvU5tJp 2Lm4w7U9kgKoEn4ZY0WE7edvMRMylDY8oAeZdSyhd39FytghM2IWJQlCmO0jkzmgjWkF PaG/4AjJlQIZQITCTxkPPShE72Vox/svhbLnb4MLhRcAsq9DB+lav1j0s7SlDEYx3g1u oW1AVMGTUdJZZOelkWdPoelORzzSvbxw5sYp2Mp9ja4voK5eaRWUar/Sd9r7RINBUqz4 u8GA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=A19Bje9X0ZZrcLkP01GMRJsAHCUKEcN22PGAQ2JdBQs=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=dq2ycn+YyNoSoT1+LnW3AoBN3C9lxHEMJ/IwyqRh2raoSoTczWZSdXw9RkZm+Qt3nD lZyTf6aWpYzpkwgjrlfxj0p1J9TOO/ENWRoUaCAfFyx/F/wqPrXTRwlJdxI/+lD/LZXo dH7Tjb7lOugFKsTcfPXRaaC0hw1zWfRTYDLq9z4RHea5r2whNtJThr9JWgDt3UtMcr6s MIoXs/P8FVUtyZefJFuubdKbc8JSNlvdZMZk2jYt9gu7VrqbpJdethl1BlALmofA5I3M OTPZkciAJnC63364xtcdqpQmiJRahpKdQ6B6CeO3v+gPoTQrobgE3/t/rwFXgq0Q8EIP Pt6g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="FiXM6H/6"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id k16-20020a056a00135000b006ce45456a1dsi3388283pfu.184.2023.12.04.17.27.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:27:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="FiXM6H/6"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id E9BBA80A055B; Mon, 4 Dec 2023 17:27:30 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346191AbjLEB0z (ORCPT + 99 others); Mon, 4 Dec 2023 20:26:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234805AbjLEB0t (ORCPT ); Mon, 4 Dec 2023 20:26:49 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D546127 for ; Mon, 4 Dec 2023 17:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701739614; x=1733275614; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GFKeHdIylAj+4b4uIVEJnbm29OkKM9wsrhT6h9/WUk4=; b=FiXM6H/6pC823aIk9+mjocYa0KQX9YJezgwZgUFoxPg6ArtAgNTX9t82 9flPusC2nA2/SdwtPaNKhnMMvORAvY4l40eoHUuIVwvu11O+m4KbGmA1x yYa8NUOkIGkjaiCPO0uA5Ysy+CVuslkOhzRkurDrBCJ5t4VEbYg+E7L9f uj1o+W2xmx9oNINmOIVRcmX1I2UHBK2aUMFH3ihfzd27CxICxKtEVe/zz /kfGJSbt1GuCaYV/dtLJeeRm4dyFQyRtJ3Ov4DqNCd5b3dAbxwDAMqqyE PN/HUwtUFpV1un5+uhZLKbFEvheR986Db92x3uu+yKu1YqrYb01KulkiU g==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="460313362" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="460313362" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 17:26:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="1102276309" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="1102276309" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmsmga005.fm.intel.com with ESMTP; 04 Dec 2023 17:26:51 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 5/6] iommu/vt-d: Add support for static identity domain Date: Tue, 5 Dec 2023 09:22:02 +0800 Message-Id: <20231205012203.244584-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231205012203.244584-1-baolu.lu@linux.intel.com> References: <20231205012203.244584-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 04 Dec 2023 17:27:31 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784403359460064140 X-GMAIL-MSGID: 1784403359460064140 Add a global static identity domain with guaranteed attach semantics. Software determines VT-d hardware support for pass-through translation by inspecting the capability register. If pass-through translation is not supported, the device is instructed to use DMA domain for its default domain. While most recent VT-d hardware implementations support pass- through translation, this capability is only lacking in some early VT-d implementations. With the static identity domain in place, the domain field of per-device iommu driver data can be either a pointer to a DMA translation domain, or NULL, indicating that the static identity domain is attached. Refine some code to accommodate this change. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 122 +++++++++++++++++++++++++++++++++--- drivers/iommu/intel/svm.c | 2 +- 2 files changed, 116 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 7022cc183120..3c747d19495e 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1282,7 +1282,8 @@ static void iommu_enable_pci_caps(struct device_domain_info *info) if (info->ats_supported && pci_ats_page_aligned(pdev) && !pci_enable_ats(pdev, VTD_PAGE_SHIFT)) { info->ats_enabled = 1; - domain_update_iotlb(info->domain); + if (info->domain) + domain_update_iotlb(info->domain); } } @@ -1298,7 +1299,8 @@ static void iommu_disable_pci_caps(struct device_domain_info *info) if (info->ats_enabled) { pci_disable_ats(pdev); info->ats_enabled = 0; - domain_update_iotlb(info->domain); + if (info->domain) + domain_update_iotlb(info->domain); } if (info->pasid_enabled) { @@ -1549,8 +1551,7 @@ static int iommu_init_domains(struct intel_iommu *iommu) * second-level or nested translation. We reserve a domain id for * this purpose. */ - if (sm_supported(iommu)) - set_bit(FLPT_DEFAULT_DID, iommu->domain_ids); + set_bit(FLPT_DEFAULT_DID, iommu->domain_ids); return 0; } @@ -3614,6 +3615,9 @@ static void dmar_remove_one_dev_info(struct device *dev) domain_context_clear(info); } + if (!domain) + return; + spin_lock_irqsave(&domain->lock, flags); list_del(&info->link); spin_unlock_irqrestore(&domain->lock, flags); @@ -3822,11 +3826,9 @@ int prepare_domain_attach_device(struct iommu_domain *domain, static int intel_iommu_attach_device(struct iommu_domain *domain, struct device *dev) { - struct device_domain_info *info = dev_iommu_priv_get(dev); int ret; - if (info->domain) - device_block_translation(dev); + device_block_translation(dev); ret = prepare_domain_attach_device(domain, dev); if (ret) @@ -4431,6 +4433,9 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) goto out_tear_down; } + if (domain->type == IOMMU_DOMAIN_IDENTITY) + goto out_tear_down; + dmar_domain = to_dmar_domain(domain); spin_lock_irqsave(&dmar_domain->lock, flags); list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain) { @@ -4605,8 +4610,111 @@ static const struct iommu_dirty_ops intel_dirty_ops = { .read_and_clear_dirty = intel_iommu_read_and_clear_dirty, }; +static int context_setup_pass_through(struct device *dev, u8 bus, u8 devfn) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + struct context_entry *context; + + spin_lock(&iommu->lock); + context = iommu_context_addr(iommu, bus, devfn, 1); + if (!context) { + spin_unlock(&iommu->lock); + return -ENOMEM; + } + + if (context_present(context) && !context_copied(iommu, bus, devfn)) { + spin_unlock(&iommu->lock); + return 0; + } + + copied_context_tear_down(iommu, context, bus, devfn); + context_clear_entry(context); + context_set_domain_id(context, FLPT_DEFAULT_DID); + + /* + * In pass through mode, AW must be programmed to indicate the largest + * AGAW value supported by hardware. And ASR is ignored by hardware. + */ + context_set_address_width(context, iommu->msagaw); + context_set_translation_type(context, CONTEXT_TT_PASS_THROUGH); + context_set_fault_enable(context); + context_set_present(context); + if (!ecap_coherent(iommu->ecap)) + clflush_cache_range(context, sizeof(*context)); + context_present_cache_flush(iommu, FLPT_DEFAULT_DID, bus, devfn); + spin_unlock(&iommu->lock); + + return 0; +} + +static int context_setup_pass_through_cb(struct pci_dev *pdev, u16 alias, void *data) +{ + struct device *dev = data; + + if (dev != &pdev->dev) + return 0; + + return context_setup_pass_through(dev, PCI_BUS_NUM(alias), alias & 0xff); +} + +static int device_setup_pass_through(struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + + if (!dev_is_pci(dev)) + return context_setup_pass_through(dev, info->bus, info->devfn); + + return pci_for_each_dma_alias(to_pci_dev(dev), + context_setup_pass_through_cb, dev); +} + +static int identity_domain_attach_dev(struct iommu_domain *domain, + struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + int ret; + + device_block_translation(dev); + + if (dev_is_real_dma_subdevice(dev)) + return 0; + + if (sm_supported(iommu)) { + ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID); + if (!ret) + iommu_enable_pci_caps(info); + } else { + ret = device_setup_pass_through(dev); + } + + return ret; +} + +static int identity_domain_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + + if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev)) + return -EOPNOTSUPP; + + return intel_pasid_setup_pass_through(iommu, dev, pasid); +} + +static struct iommu_domain identity_domain = { + .type = IOMMU_DOMAIN_IDENTITY, + .ops = &(const struct iommu_domain_ops) { + .attach_dev = identity_domain_attach_dev, + .set_dev_pasid = identity_domain_set_dev_pasid, + }, +}; + const struct iommu_ops intel_iommu_ops = { .blocked_domain = &blocking_domain, + .identity_domain = &identity_domain, .capable = intel_iommu_capable, .hw_info = intel_iommu_hw_info, .domain_alloc = intel_iommu_domain_alloc, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 442ff9905ca9..d78beb132f5b 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -493,7 +493,7 @@ void intel_drain_pasid_prq(struct device *dev, u32 pasid) domain = info->domain; pdev = to_pci_dev(dev); sid = PCI_DEVID(info->bus, info->devfn); - did = domain_id_iommu(domain, iommu); + did = domain ? domain_id_iommu(domain, iommu) : FLPT_DEFAULT_DID; qdep = pci_ats_queue_depth(pdev); /* From patchwork Tue Dec 5 01:22:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 173642 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3150635vqy; Mon, 4 Dec 2023 17:27:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IHzf7tTQLCZkxqvmg82y6JbZGBs1Y7zznmRoaLBKVpKRKJa714jHXqtWasV8HKgDsCSQMep X-Received: by 2002:a17:902:dacb:b0:1d0:68a:4a46 with SMTP id q11-20020a170902dacb00b001d0068a4a46mr6271787plx.3.1701739661719; Mon, 04 Dec 2023 17:27:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701739661; cv=none; d=google.com; s=arc-20160816; b=xeEgfVRn9aAiGoIIEzTYCpGmdo9SNQrFy67ZhaVL9+zwgFUVMxBQXV5ZYWet8JxUwG TMmGYQCkI5x5HRrKF8iQw/J9Bhfb998/MnWNFel9xj75SLrbjADVi50jHULwoiW55fu9 8NtcmCXL8EMqJSjFOEY5AxflgEcaw6VxlDUD7SVA7cGlfY5eUA94VEQRB/RSTT8FjDz2 jdjsGpUQS/mDKomgtpizqXNjG0P14vbTGjNzI0tgKjxuXxhMm3+adwxMbWNLyLhji1rt khioYghh8s13MbKWjORonYv+e1CLLp0Oiwc+gI4mXxYpuiQ8n6FFf4iVvVhLkvouzHxd dfWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Zk4vMfEPB/oHaz6nFp2C1Z+mlyvylkA2Qf3YyvpOxsg=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=TSZ2lqpwK4NCopmWALsS6EP5Kf9F+qlSfdTqSz3mfAzEXCbMC/UgTEMKGCvmO30N// l1RxFgjmWuxa9xlCiCQmTvapVatU0oCP/ImhjXRjCKvokJkbzkMbrkdNP2y7gU7WL3Xl KYy6t65T9J7TeiHC0GVAnCNMrVGPXtRNsYuEgk3sc4sNBNgrNTVceaytRthQzmPb9Q4M Y+ME+LjV3Sg4D5FCaWxsxhavATUNhmO6Rn91pN20ZDfDpjfpI2Wk+smI183GRP8aDr0X zxrp4tiWk02Q5RyZ+amgPF3WS/uh14CkUjG40kA6xSAxr6SQ+k+Ck3IMQWK89qo5oztN IwRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WwosC1tF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id n9-20020a170903110900b001cfce7988edsi5375563plh.417.2023.12.04.17.27.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:27:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WwosC1tF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 6EE908032E6C; Mon, 4 Dec 2023 17:27:32 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346321AbjLEB1G (ORCPT + 99 others); Mon, 4 Dec 2023 20:27:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346337AbjLEB05 (ORCPT ); Mon, 4 Dec 2023 20:26:57 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54F1418A for ; Mon, 4 Dec 2023 17:26:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701739617; x=1733275617; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u1gb+xXuSLrs67n4B6W8q40HRhcbFjefDWfMBwrFYbE=; b=WwosC1tF7difF/42duQS89tBujS6p6/yB5253cwVNu2HyBSOTBFuR7k7 JYij1OibK//jlsxs3NkXGi43TGMr94PjhjJLE2vpg4Pcprh7zqBoXvWIY u3rHMGye3cNkQWj/q6yRxpe2F/b75p5XziV4E0ZpOcqxbxMqXnwGF967r VAyRGm68sE/CVoysVWSZM0P7Hwz3cboURO/Evdw4AxNiww8TLplZ3P1Y+ KwHH69epjJuA5FSmBIh/SipqQNpY6mBoTMCZ5MGavpRQrZRrnLdalkc8Z w4F8bb+Z+y1c4mXwgSgMNlbfqibPcBjAVEIJNniJgeEFtvmJqTOugnVpX Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="460313368" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="460313368" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 17:26:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="1102276337" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="1102276337" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmsmga005.fm.intel.com with ESMTP; 04 Dec 2023 17:26:53 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH v2 6/6] iommu/vt-d: Cleanup si_domain Date: Tue, 5 Dec 2023 09:22:03 +0800 Message-Id: <20231205012203.244584-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231205012203.244584-1-baolu.lu@linux.intel.com> References: <20231205012203.244584-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 04 Dec 2023 17:27:32 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784403367511248945 X-GMAIL-MSGID: 1784403367511248945 The static identity domain has been introduced, rendering the si_domain obsolete. Remove si_domain and cleanup the code accordingly. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 118 +++++++----------------------------- 1 file changed, 23 insertions(+), 95 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 3c747d19495e..91443b34111b 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -97,15 +97,6 @@ static phys_addr_t root_entry_uctp(struct root_entry *re) return re->hi & VTD_PAGE_MASK; } -/* - * This domain is a statically identity mapping domain. - * 1. This domain creats a static 1:1 mapping to all usable memory. - * 2. It maps to each iommu if successful. - * 3. Each iommu mapps to this domain if successful. - */ -static struct dmar_domain *si_domain; -static int hw_pass_through = 1; - struct dmar_rmrr_unit { struct list_head list; /* list of rmrr units */ struct acpi_dmar_header *hdr; /* ACPI header */ @@ -240,11 +231,6 @@ void free_pgtable_page(void *vaddr) free_page((unsigned long)vaddr); } -static int domain_type_is_si(struct dmar_domain *domain) -{ - return domain->domain.type == IOMMU_DOMAIN_IDENTITY; -} - static int domain_pfn_supported(struct dmar_domain *domain, unsigned long pfn) { int addr_width = agaw_to_width(domain->agaw) - VTD_PAGE_SHIFT; @@ -1795,9 +1781,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain, struct context_entry *context; int agaw, ret; - if (hw_pass_through && domain_type_is_si(domain)) - translation = CONTEXT_TT_PASS_THROUGH; - pr_debug("Set context mapping for %02x:%02x.%d\n", bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); @@ -1816,34 +1799,24 @@ static int domain_context_mapping_one(struct dmar_domain *domain, context_set_domain_id(context, did); - if (translation != CONTEXT_TT_PASS_THROUGH) { - /* - * Skip top levels of page tables for iommu which has - * less agaw than default. Unnecessary for PT mode. - */ - for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { - ret = -ENOMEM; - pgd = phys_to_virt(dma_pte_addr(pgd)); - if (!dma_pte_present(pgd)) - goto out_unlock; - } - - if (info && info->ats_supported) - translation = CONTEXT_TT_DEV_IOTLB; - else - translation = CONTEXT_TT_MULTI_LEVEL; - - context_set_address_root(context, virt_to_phys(pgd)); - context_set_address_width(context, agaw); - } else { - /* - * In pass through mode, AW must be programmed to - * indicate the largest AGAW value supported by - * hardware. And ASR is ignored by hardware. - */ - context_set_address_width(context, iommu->msagaw); + /* + * Skip top levels of page tables for iommu which has + * less agaw than default. Unnecessary for PT mode. + */ + for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { + ret = -ENOMEM; + pgd = phys_to_virt(dma_pte_addr(pgd)); + if (!dma_pte_present(pgd)) + goto out_unlock; } + if (info && info->ats_supported) + translation = CONTEXT_TT_DEV_IOTLB; + else + translation = CONTEXT_TT_MULTI_LEVEL; + + context_set_address_root(context, virt_to_phys(pgd)); + context_set_address_width(context, agaw); context_set_translation_type(context, translation); context_set_fault_enable(context); context_set_present(context); @@ -2077,14 +2050,10 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8 return; } - if (sm_supported(iommu)) { - if (hw_pass_through && domain_type_is_si(info->domain)) - did_old = FLPT_DEFAULT_DID; - else - did_old = domain_id_iommu(info->domain, iommu); - } else { - did_old = context_domain_id(context); - } + if (info->domain) + did_old = domain_id_iommu(info->domain, iommu); + else + did_old = FLPT_DEFAULT_DID; context_clear_entry(context); __iommu_flush_cache(iommu, context, sizeof(*context)); @@ -2147,23 +2116,6 @@ static bool dev_is_real_dma_subdevice(struct device *dev) pci_real_dma_dev(to_pci_dev(dev)) != to_pci_dev(dev); } -static int md_domain_init(struct dmar_domain *domain, int guest_width); - -static int __init si_domain_init(int hw) -{ - si_domain = alloc_domain(IOMMU_DOMAIN_IDENTITY); - if (!si_domain) - return -EFAULT; - - if (md_domain_init(si_domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) { - domain_exit(si_domain); - si_domain = NULL; - return -EFAULT; - } - - return 0; -} - static int dmar_domain_attach_device(struct dmar_domain *domain, struct device *dev) { @@ -2185,8 +2137,6 @@ static int dmar_domain_attach_device(struct dmar_domain *domain, if (!sm_supported(iommu)) ret = domain_context_mapping(domain, dev); - else if (hw_pass_through && domain_type_is_si(domain)) - ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID); else if (domain->use_first_level) ret = domain_setup_first_level(iommu, domain, dev, IOMMU_NO_PASID); else @@ -2197,8 +2147,7 @@ static int dmar_domain_attach_device(struct dmar_domain *domain, return ret; } - if (sm_supported(info->iommu) || !domain_type_is_si(info->domain)) - iommu_enable_pci_caps(info); + iommu_enable_pci_caps(info); return 0; } @@ -2548,8 +2497,6 @@ static int __init init_dmars(void) } } - if (!ecap_pass_through(iommu->ecap)) - hw_pass_through = 0; intel_svm_check(iommu); } @@ -2572,10 +2519,6 @@ static int __init init_dmars(void) check_tylersburg_isoch(); - ret = si_domain_init(hw_pass_through); - if (ret) - goto free_iommu; - /* * for each drhd * enable fault log @@ -2621,10 +2564,6 @@ static int __init init_dmars(void) disable_dmar_iommu(iommu); free_dmar_iommu(iommu); } - if (si_domain) { - domain_exit(si_domain); - si_domain = NULL; - } return ret; } @@ -2999,12 +2938,6 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru) if (ret) goto out; - if (hw_pass_through && !ecap_pass_through(iommu->ecap)) { - pr_warn("%s: Doesn't support hardware pass through.\n", - iommu->name); - return -ENXIO; - } - sp = domain_update_iommu_superpage(NULL, iommu) - 1; if (sp >= 0 && !(cap_super_page_val(iommu->cap) & (1 << sp))) { pr_warn("%s: Doesn't support large page.\n", @@ -3718,8 +3651,6 @@ static struct iommu_domain *intel_iommu_domain_alloc(unsigned type) domain->geometry.force_aperture = true; return domain; - case IOMMU_DOMAIN_IDENTITY: - return &si_domain->domain; case IOMMU_DOMAIN_SVA: return intel_svm_domain_alloc(); default: @@ -3779,8 +3710,7 @@ intel_iommu_domain_alloc_user(struct device *dev, u32 flags, static void intel_iommu_domain_free(struct iommu_domain *domain) { - if (domain != &si_domain->domain) - domain_exit(to_dmar_domain(domain)); + domain_exit(to_dmar_domain(domain)); } int prepare_domain_attach_device(struct iommu_domain *domain, @@ -4487,9 +4417,7 @@ static int intel_iommu_set_dev_pasid(struct iommu_domain *domain, if (ret) goto out_free; - if (domain_type_is_si(dmar_domain)) - ret = intel_pasid_setup_pass_through(iommu, dev, pasid); - else if (dmar_domain->use_first_level) + if (dmar_domain->use_first_level) ret = domain_setup_first_level(iommu, dmar_domain, dev, pasid); else