From patchwork Mon Nov 20 11:29:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 167085 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp2134780vqn; Mon, 20 Nov 2023 03:36:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IHr7Yo+NLJCfvV2UXdl79XqVV5iFdx64eQRWLZr5KvqKSPkIyF9sOOppgTqKMGX9ZLj39AZ X-Received: by 2002:a17:902:eb53:b0:1cc:5c8f:4056 with SMTP id i19-20020a170902eb5300b001cc5c8f4056mr7251216pli.42.1700480216722; Mon, 20 Nov 2023 03:36:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700480216; cv=none; d=google.com; s=arc-20160816; b=mnLhePmxIZz4iBkkOd6YI/ZrZwwMJCzjsv2y2J8cOO0IYDY8sklDYuKP4DDqP5+pDN UhDPh/gDQZh0SBi3PlBN2lB/H/2dXlP8L6KMIN2IbGlQoy3zwmTApIgDqdvQRvVkbRMa pjcGiQA5gjKh3+LYOo+thysn+PnjBFbHY/J2Eq5Qp0E9RdycJ+sspAEp0fs/MlXpZRf+ wq9lviTuTWkngur/RmBTCOBOL34yRriO7MtGmJrCbcr70bTaFrs8GtlJc9HHb9OEL0/U 3586XKVaHLoSTTwuJvJkkMY9Hv9zdWCzJZjVYVS6ek0PkVOwdJUFuMLyAB+MfE5PYYcr 1OHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3Hp91Ycl4UOEyAswdWSiMlnHsoB7XGF7XzsAWp82G7w=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=U4yl0WgUDeKpQjGCk7nU831sLYB1Pl0dmWxmlj3lgYVRHJePXFvg6wvvF1LNjRcAnD nNCdc8Ejg0zzVXyRImgsooqIeLJ1h92cBL8HmUCtR2aWlcWxWspLIDBshpyuk71Uysp6 8fR9NSaq9k5XmGP70ydvv9m1y6QvfoMKBbF1seoutXJScBveQTUjozDeBJIfTuYcer7c Ouwj+Y/gghqsU+9A9jf8h7KIqrot7frrmmk8l8zij0BqT0T3c6rSW5oyQxvBBiETQ25D GrjrCol60jJPmTYa9UuNqGFAh/zk3oRJKLTiYA+IFz1UyP6ruhEWtp6TjEpOhLZ4eTBd zAvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SeSHNJTr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id t24-20020a1709028c9800b001c9c83947d1si7587552plo.645.2023.11.20.03.36.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 03:36:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SeSHNJTr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id EDA2C8089E62; Mon, 20 Nov 2023 03:35:56 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233331AbjKTLea (ORCPT + 27 others); Mon, 20 Nov 2023 06:34:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233237AbjKTLeX (ORCPT ); Mon, 20 Nov 2023 06:34:23 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5AB7A7 for ; Mon, 20 Nov 2023 03:34:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700480058; x=1732016058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6xRpfIwAcngpnwa9c8V3XoSGyTL3xX5Z/sBDIadwOT4=; b=SeSHNJTrLVipKGMX2bxEP7qfvcxyLquazr2Q15YDfCs9cJuNH9O1lrbn 4PBx6lA2WAnGD08JZUqHVWsaz+0IEvxB1MOiyPWWRFk8Xf0vA51KHo7qo QvPpMFsGWSH6W5PzhttJrY/87GCRt4Es4Tk5Q1LCmpEXCiaK1ReVsPkAu aWHPzFKXY/I02QD6S4uqFX2LP4Nj+foBH5rXaVgESRfyOKmHslp+1/QFh J7mlngqrh75LkV2zGi/8VKhyIm7D1juHZWxdrAglsSilCFAQwM68wYI6E TbqtjsUtk/3XtOgOAFG266I2uPjATID/5TP4MqFeiKyGNY8bEWm9yTP1Z w==; X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="376634979" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="376634979" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2023 03:34:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="856963612" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="856963612" Received: from allen-box.sh.intel.com ([10.239.159.127]) by FMSMGA003.fm.intel.com with ESMTP; 20 Nov 2023 03:34:05 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH 1/5] iommu/vt-d: Setup scalable mode context entry in probe path Date: Mon, 20 Nov 2023 19:29:40 +0800 Message-Id: <20231120112944.142741-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120112944.142741-1-baolu.lu@linux.intel.com> References: <20231120112944.142741-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 20 Nov 2023 03:35:57 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783082743830659184 X-GMAIL-MSGID: 1783082743830659184 In contrast to legacy mode, the DMA translation table is configured in the PASID table entry instead of the context entry for scalable mode. For this reason, it is more appropriate to set up the scalable mode context entry in the device_probe callback and direct it to the appropriate PASID table. The iommu domain attach/detach operations only affect the PASID table entry. Therefore, there is no need to modify the context entry when configuring the translation type and page table. The only exception is the kdump case, where context entry setup is postponed until the device driver invokes the first DMA interface. Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.h | 1 + drivers/iommu/intel/iommu.c | 17 +++- drivers/iommu/intel/pasid.c | 180 ++++++++++++++++++++++++++++++++++++ 3 files changed, 195 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/intel/pasid.h b/drivers/iommu/intel/pasid.h index 8d40d4c66e31..58d7049081b9 100644 --- a/drivers/iommu/intel/pasid.h +++ b/drivers/iommu/intel/pasid.h @@ -319,4 +319,5 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu, bool fault_ignore); void intel_pasid_setup_page_snoop_control(struct intel_iommu *iommu, struct device *dev, u32 pasid); +int intel_pasid_setup_sm_context(struct device *dev, bool deferred); #endif /* __INTEL_PASID_H */ diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 5092330f6a10..e34a32cd2b9a 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4270,15 +4270,26 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) ret = intel_pasid_alloc_table(dev); if (ret) { dev_err(dev, "PASID table allocation failed\n"); - dev_iommu_priv_set(dev, NULL); - kfree(info); - return ERR_PTR(ret); + goto err_clear_priv; + } + + ret = intel_pasid_setup_sm_context(dev, false); + if (ret) { + dev_err(dev, "Scalable context entry setup failed\n"); + goto err_free_table; } } intel_iommu_debugfs_create_dev(info); return &iommu->iommu; +err_free_table: + intel_pasid_free_table(dev); +err_clear_priv: + dev_iommu_priv_set(dev, NULL); + kfree(info); + +return ERR_PTR(ret); } static void intel_iommu_release_device(struct device *dev) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 3239cefa4c33..9e505060617a 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -304,6 +304,11 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu, return -EINVAL; } + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err(dev, "Context entry is not configured\n"); + return -ENODEV; + } + spin_lock(&iommu->lock); pte = intel_pasid_get_entry(dev, pasid); if (!pte) { @@ -384,6 +389,11 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu, return -EINVAL; } + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err(dev, "Context entry is not configured\n"); + return -ENODEV; + } + pgd = domain->pgd; agaw = iommu_skip_agaw(domain, iommu, &pgd); if (agaw < 0) { @@ -505,6 +515,11 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu, u16 did = FLPT_DEFAULT_DID; struct pasid_entry *pte; + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err(dev, "Context entry is not configured\n"); + return -ENODEV; + } + spin_lock(&iommu->lock); pte = intel_pasid_get_entry(dev, pasid); if (!pte) { @@ -623,6 +638,11 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu, struct device *dev, return -EINVAL; } + if (intel_pasid_setup_sm_context(dev, true)) { + dev_err_ratelimited(dev, "Context entry is not configured\n"); + return -ENODEV; + } + spin_lock(&iommu->lock); pte = intel_pasid_get_entry(dev, pasid); if (!pte) { @@ -666,3 +686,163 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu, struct device *dev, return 0; } + +/* + * Interface to set a pasid table to the scalable mode context table entry: + */ + +/* + * Get the PASID directory size for scalable mode context entry. + * Value of X in the PDTS field of a scalable mode context entry + * indicates PASID directory with 2^(X + 7) entries. + */ +static unsigned long context_get_sm_pds(struct pasid_table *table) +{ + unsigned long pds, max_pde; + + max_pde = table->max_pasid >> PASID_PDE_SHIFT; + pds = find_first_bit(&max_pde, MAX_NR_PASID_BITS); + if (pds < 7) + return 0; + + return pds - 7; +} + +static int context_entry_set_pasid_table(struct context_entry *context, + struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct pasid_table *table = info->pasid_table; + struct intel_iommu *iommu = info->iommu; + unsigned long pds; + + context_clear_entry(context); + + pds = context_get_sm_pds(table); + context->lo = (u64)virt_to_phys(table->table) | context_pdts(pds); + context_set_sm_rid2pasid(context, IOMMU_NO_PASID); + + if (info->ats_supported) + context_set_sm_dte(context); + if (info->pri_supported) + context_set_sm_pre(context); + if (info->pasid_supported) + context_set_pasid(context); + + context_set_fault_enable(context); + context_set_present(context); + if (!ecap_coherent(iommu->ecap)) + clflush_cache_range(context, sizeof(*context)); + + return 0; +} + +static int device_pasid_table_setup(struct device *dev, u8 bus, u8 devfn) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + struct context_entry *context; + int ret = 0; + + spin_lock(&iommu->lock); + context = iommu_context_addr(iommu, bus, devfn, true); + if (!context) { + ret = -ENOMEM; + goto out_unlock; + } + + if (context_present(context) && !context_copied(iommu, bus, devfn)) + goto out_unlock; + + /* + * Cache invalidation for changes to a scalable-mode context table + * entry. + * + * Section 6.5.3.3 of the VT-d spec: + * - Device-selective context-cache invalidation; + * - Domain-selective PASID-cache invalidation to affected domains + * (can be skipped if all PASID entries were not-present); + * - Domain-selective IOTLB invalidation to affected domains; + * - Global Device-TLB invalidation to affected functions. + * + * For kdump cases, old valid entries may be cached due to the + * in-flight DMA and copied pgtable, but there is no unmapping + * behaviour for them, thus we need explicit cache flushes for all + * affected domain IDs and PASIDs used in the copied PASID table. + * Given that we have no idea about which domain IDs and PASIDs were + * used in the copied tables, upgrade them to global PASID and IOTLB + * cache invalidation. + * + * For kdump case, at this point, the device is supposed to finish + * reset at its driver probe stage, so no in-flight DMA will exist, + * and we don't need to worry anymore hereafter. + */ + if (context_copied(iommu, bus, devfn)) { + context_clear_entry(context); + clear_context_copied(iommu, bus, devfn); + iommu->flush.flush_context(iommu, 0, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + qi_flush_pasid_cache(iommu, 0, QI_PC_GLOBAL, 0); + iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH); + devtlb_invalidation_with_pasid(iommu, dev, IOMMU_NO_PASID); + } + + context_entry_set_pasid_table(context, dev); + + /* + * It's a non-present to present mapping. If hardware doesn't cache + * non-present entry we only need to flush the write-buffer. If the + * _does_ cache non-present entries, then it does so in the special + * domain ID #0, which we have to flush: + */ + if (cap_caching_mode(iommu->cap)) { + iommu->flush.flush_context(iommu, 0, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH); + } else { + iommu_flush_write_buffer(iommu); + } + +out_unlock: + spin_unlock(&iommu->lock); + return ret; +} + +static int pci_pasid_table_setup(struct pci_dev *pdev, u16 alias, void *data) +{ + struct device *dev = data; + + if (dev != &pdev->dev) + return 0; + + return device_pasid_table_setup(dev, PCI_BUS_NUM(alias), alias & 0xff); +} + +/* + * Set the device's PASID table to its context table entry. + * + * The PASID table is set to the context entries of both device itself + * and its alias requester ID for DMA. If it is called in domain attach + * paths, set @deferred to true, false in other cases. + */ +int intel_pasid_setup_sm_context(struct device *dev, bool deferred) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + + /* + * Skip pasid table setting up if context entry is copied and + * function is not called in deferred attachment context. + */ + if (deferred ^ context_copied(iommu, info->bus, info->devfn)) + return 0; + + if (!dev_is_pci(dev)) + return device_pasid_table_setup(dev, info->bus, info->devfn); + + return pci_for_each_dma_alias(to_pci_dev(dev), pci_pasid_table_setup, dev); +} From patchwork Mon Nov 20 11:29:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 167088 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp2135573vqn; Mon, 20 Nov 2023 03:38:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IFsr+K5FTR6u8azMGiA3OzzGXTWIcIUTzQWTz7l9YBFJJhh/Niuyt/6gYg4BcjRrKStEj63 X-Received: by 2002:a17:902:d304:b0:1cc:787f:fb7 with SMTP id b4-20020a170902d30400b001cc787f0fb7mr8903356plc.19.1700480323906; Mon, 20 Nov 2023 03:38:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700480323; cv=none; d=google.com; s=arc-20160816; b=q6GJ4kuZack8YOfAxqkZOLWkYNIL0xx/+StQOMxuMfWVJDW3hvMzvED8CJYgTxZQUH myLy4dv3kwGNaVPZ7GmV2yAC0SCP0I5T5lDsaXvkcc2LnYU+ktkgz24wsQlGD2ok6EIQ 3JW8m6Z/ywd0EKEiRket8CthKGQsqz0Mj7L3XTn8U+7YhFT1lKJ3w8wmQZJzd//4Ctar ooU5V7Tm0V2Azq6o/6oYuZmbYMTmC9qNK0W2CoEVFBI1ku1gpTnFXpoF/a4ScxkO4MNW Fz8zMLqlKgE4YD9NDSccKpZtFK11qzM8YGBJdi1NP3JsBnT4J67+bA9skq3DKHolbCY0 KYgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Xml5wOO63rHPVsCjvnBdVd80wr5390B/eXStsnKJUHg=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=xxnnvmxn6zDa7QgGG5Yvv8NMLDUFcLULwzg+XGj9FCaPZshdMXPjHF4kxoU/g5Z08Q MDp2UdfRpRS/WnCmC3UTpVRzYadWwErJXykcOAgWSxaV1bOLaM49N8PQjGvLHbazCpKC wMLgLQ1oFj0p+KisvjHuK+OIDNvZgOXwWATpY9DluqsEoUxMJ2ZwAvbeRfVe4a89I6Z5 V7+fnK1x0vTt5NrN1z/Sozbn1QFsS+6wrDBSUJY6xRD58deAlZOa9rqDlTruq3JJzafa Pw/CSyId8jOwZwBmXF398fO754y3eZYLzOVQd3ZWvGUmnenaxpcpQkcpzV7yrZFB8gUC AC/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QUWuPnFF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id lk5-20020a17090308c500b001caac2f7aa0si8754390plb.27.2023.11.20.03.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 03:38:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QUWuPnFF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id B164C809220E; Mon, 20 Nov 2023 03:34:37 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233238AbjKTLe0 (ORCPT + 27 others); Mon, 20 Nov 2023 06:34:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233206AbjKTLeX (ORCPT ); Mon, 20 Nov 2023 06:34:23 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC9239C for ; Mon, 20 Nov 2023 03:34:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700480058; x=1732016058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d4PZaUm84w7H83XcsuQyvk1lkaQccPFbLEhvPQCk0Xc=; b=QUWuPnFFVsnjz6uFzfTHeAFauT1KWSziLTjKTQlRaefv3ENHRk65wyiF h6us9XI5lY5jdsHvXVySINOvnO9c87fIrEMiM9ygCfWy2hPcVRw64nIOg MKrkwb2+Opkas+CDxt+MGkzVsH7eLWzr3Jgd5vY4WFjUWyJfM/9nWnH8u 9GOGlrnpzSFGp14T23xL/u9DWk5ANuQhOy55alIyOxoSSnlvs/2JKQVmn 7ZYI6wR4hbQ7VDVh+GJxb+T6LKn5XVmLrMs6k2xuyZUfBnGqzpo/IhjbY EGeE7ViI3+M+hPGBDoTMAWeEtdPt4RD56lhl9D9opDGHiABjCg4jxPtTx A==; X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="376634991" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="376634991" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2023 03:34:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="856963621" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="856963621" Received: from allen-box.sh.intel.com ([10.239.159.127]) by FMSMGA003.fm.intel.com with ESMTP; 20 Nov 2023 03:34:10 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH 2/5] iommu/vt-d: Remove scalable mode context entry setup from attach_dev Date: Mon, 20 Nov 2023 19:29:41 +0800 Message-Id: <20231120112944.142741-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120112944.142741-1-baolu.lu@linux.intel.com> References: <20231120112944.142741-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 03:34:37 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783082855846561890 X-GMAIL-MSGID: 1783082855846561890 The scalable mode context entry is now setup in the probe_device path, eliminating the need to configure it in the attach_dev path. Removes the redundant code from the attach_dev path to avoid dead code. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 156 ++++++++++-------------------------- 1 file changed, 44 insertions(+), 112 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index e34a32cd2b9a..77769ad4706e 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1727,34 +1727,17 @@ static void domain_exit(struct dmar_domain *domain) kfree(domain); } -/* - * Get the PASID directory size for scalable mode context entry. - * Value of X in the PDTS field of a scalable mode context entry - * indicates PASID directory with 2^(X + 7) entries. - */ -static unsigned long context_get_sm_pds(struct pasid_table *table) -{ - unsigned long pds, max_pde; - - max_pde = table->max_pasid >> PASID_PDE_SHIFT; - pds = find_first_bit(&max_pde, MAX_NR_PASID_BITS); - if (pds < 7) - return 0; - - return pds - 7; -} - static int domain_context_mapping_one(struct dmar_domain *domain, struct intel_iommu *iommu, - struct pasid_table *table, u8 bus, u8 devfn) { struct device_domain_info *info = domain_lookup_dev_info(domain, iommu, bus, devfn); u16 did = domain_id_iommu(domain, iommu); int translation = CONTEXT_TT_MULTI_LEVEL; + struct dma_pte *pgd = domain->pgd; struct context_entry *context; - int ret; + int agaw, ret; if (hw_pass_through && domain_type_is_si(domain)) translation = CONTEXT_TT_PASS_THROUGH; @@ -1797,65 +1780,37 @@ static int domain_context_mapping_one(struct dmar_domain *domain, } context_clear_entry(context); + context_set_domain_id(context, did); - if (sm_supported(iommu)) { - unsigned long pds; - - /* Setup the PASID DIR pointer: */ - pds = context_get_sm_pds(table); - context->lo = (u64)virt_to_phys(table->table) | - context_pdts(pds); - - /* Setup the RID_PASID field: */ - context_set_sm_rid2pasid(context, IOMMU_NO_PASID); - + if (translation != CONTEXT_TT_PASS_THROUGH) { /* - * Setup the Device-TLB enable bit and Page request - * Enable bit: + * Skip top levels of page tables for iommu which has + * less agaw than default. Unnecessary for PT mode. */ + for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { + ret = -ENOMEM; + pgd = phys_to_virt(dma_pte_addr(pgd)); + if (!dma_pte_present(pgd)) + goto out_unlock; + } + if (info && info->ats_supported) - context_set_sm_dte(context); - if (info && info->pri_supported) - context_set_sm_pre(context); - if (info && info->pasid_supported) - context_set_pasid(context); + translation = CONTEXT_TT_DEV_IOTLB; + else + translation = CONTEXT_TT_MULTI_LEVEL; + + context_set_address_root(context, virt_to_phys(pgd)); + context_set_address_width(context, agaw); } else { - struct dma_pte *pgd = domain->pgd; - int agaw; - - context_set_domain_id(context, did); - - if (translation != CONTEXT_TT_PASS_THROUGH) { - /* - * Skip top levels of page tables for iommu which has - * less agaw than default. Unnecessary for PT mode. - */ - for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { - ret = -ENOMEM; - pgd = phys_to_virt(dma_pte_addr(pgd)); - if (!dma_pte_present(pgd)) - goto out_unlock; - } - - if (info && info->ats_supported) - translation = CONTEXT_TT_DEV_IOTLB; - else - translation = CONTEXT_TT_MULTI_LEVEL; - - context_set_address_root(context, virt_to_phys(pgd)); - context_set_address_width(context, agaw); - } else { - /* - * In pass through mode, AW must be programmed to - * indicate the largest AGAW value supported by - * hardware. And ASR is ignored by hardware. - */ - context_set_address_width(context, iommu->msagaw); - } - - context_set_translation_type(context, translation); + /* + * In pass through mode, AW must be programmed to + * indicate the largest AGAW value supported by + * hardware. And ASR is ignored by hardware. + */ + context_set_address_width(context, iommu->msagaw); } + context_set_translation_type(context, translation); context_set_fault_enable(context); context_set_present(context); if (!ecap_coherent(iommu->ecap)) @@ -1885,43 +1840,29 @@ static int domain_context_mapping_one(struct dmar_domain *domain, return ret; } -struct domain_context_mapping_data { - struct dmar_domain *domain; - struct intel_iommu *iommu; - struct pasid_table *table; -}; - static int domain_context_mapping_cb(struct pci_dev *pdev, u16 alias, void *opaque) { - struct domain_context_mapping_data *data = opaque; + struct device_domain_info *info = dev_iommu_priv_get(&pdev->dev); + struct intel_iommu *iommu = info->iommu; + struct dmar_domain *domain = opaque; - return domain_context_mapping_one(data->domain, data->iommu, - data->table, PCI_BUS_NUM(alias), - alias & 0xff); + return domain_context_mapping_one(domain, iommu, + PCI_BUS_NUM(alias), alias & 0xff); } static int domain_context_mapping(struct dmar_domain *domain, struct device *dev) { struct device_domain_info *info = dev_iommu_priv_get(dev); - struct domain_context_mapping_data data; struct intel_iommu *iommu = info->iommu; u8 bus = info->bus, devfn = info->devfn; - struct pasid_table *table; - - table = intel_pasid_get_table(dev); if (!dev_is_pci(dev)) - return domain_context_mapping_one(domain, iommu, table, - bus, devfn); - - data.domain = domain; - data.iommu = iommu; - data.table = table; + return domain_context_mapping_one(domain, iommu, bus, devfn); return pci_for_each_dma_alias(to_pci_dev(dev), - &domain_context_mapping_cb, &data); + domain_context_mapping_cb, domain); } /* Returns a number of VTD pages, but aligned to MM page size */ @@ -2278,28 +2219,19 @@ static int dmar_domain_attach_device(struct dmar_domain *domain, list_add(&info->link, &domain->devices); spin_unlock_irqrestore(&domain->lock, flags); - /* PASID table is mandatory for a PCI device in scalable mode. */ - if (sm_supported(iommu) && !dev_is_real_dma_subdevice(dev)) { - /* Setup the PASID entry for requests without PASID: */ - if (hw_pass_through && domain_type_is_si(domain)) - ret = intel_pasid_setup_pass_through(iommu, - dev, IOMMU_NO_PASID); - else if (domain->use_first_level) - ret = domain_setup_first_level(iommu, domain, dev, - IOMMU_NO_PASID); - else - ret = intel_pasid_setup_second_level(iommu, domain, - dev, IOMMU_NO_PASID); - if (ret) { - dev_err(dev, "Setup RID2PASID failed\n"); - device_block_translation(dev); - return ret; - } - } + if (dev_is_real_dma_subdevice(dev)) + return 0; + + if (!sm_supported(iommu)) + ret = domain_context_mapping(domain, dev); + else if (hw_pass_through && domain_type_is_si(domain)) + ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID); + else if (domain->use_first_level) + ret = domain_setup_first_level(iommu, domain, dev, IOMMU_NO_PASID); + else + ret = intel_pasid_setup_second_level(iommu, domain, dev, IOMMU_NO_PASID); - ret = domain_context_mapping(domain, dev); if (ret) { - dev_err(dev, "Domain context map failed\n"); device_block_translation(dev); return ret; } From patchwork Mon Nov 20 11:29:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 167089 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp2135597vqn; Mon, 20 Nov 2023 03:38:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IHM+tgQYCSk2lIlafSu1BHm7j9391e9eM6py3DzwK5WY/r6OlpDv2yWB8mwZaikxkvCvAjN X-Received: by 2002:a05:6a00:1301:b0:6cb:3b17:e2dd with SMTP id j1-20020a056a00130100b006cb3b17e2ddmr9519495pfu.6.1700480327423; Mon, 20 Nov 2023 03:38:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700480327; cv=none; d=google.com; s=arc-20160816; b=M+Ed93nE58P/oXem/2TDcZ8S3IBYU+N+sSvs9Og3927RHHjfAWpk9Qc53mUpSzp/Wd ON2FaE77NGccoMwI3Z2ia5M485Ubni2PsUb6zN1ufrW2ZN0Mj0aLhse6t3i95jfj6eje ElXXJPQAtqgUNy2LV4V4zNA8j4FYJsFq4rq36Bk4Q88t6L0q2WD6/ooGnVL+oOrbTQBL davnyQPn5W/rifAQGAtJNhCeYepsfA47RpsNs7WGVTP/aa5vFk9bbO9m7IqvXm5GzQ5D lUywMGbxSIlqly5DsfOZ9P8y4M95m9z22fqcZ/wfLjKczGHzzLDbkkKpCPL1algsxPIY mKgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fflUaMc3n7+B8BJO3ecASO/1BT7fsYGx1CFTIxo1Sq8=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=DeU6HtZz+MeiMOJTJFUH2lLfF8fjC6wd9+JhZjTcPm3qUHbHJvCM6Y5UU8SbdAchfJ Ql7mLq0Ag+rJ6T5NFGkz2eJSNgW9Dq0k/Kro5mK8ieKYXj7SjRrQpEhtbXMjTg930aR2 Ap44AJcytbxneSdA90Tk4ScL9xSaAXsGuxX1S3PlSCsUwLBJwVCro1Yy+D9tlxw3eMus ybQENYZFiEiC+XaGjrbUhwTVCE+3hwQcOnKA4ODrJ0NwKFPDkk2lhpyRLXOn3ESGFg7T wbkojrXO33cfRrsQG9k/QxGMBTJIrxDUpQ9n/OEOTRyskdLnKVHvLkuKIdoH+yI3iPp2 MWMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WUTeIb4q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id e14-20020a63ee0e000000b005bd0432d9b0si5182678pgi.100.2023.11.20.03.38.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 03:38:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WUTeIb4q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id C63B380657C8; Mon, 20 Nov 2023 03:34:45 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233255AbjKTLee (ORCPT + 27 others); Mon, 20 Nov 2023 06:34:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233220AbjKTLeX (ORCPT ); Mon, 20 Nov 2023 06:34:23 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54E38EE for ; Mon, 20 Nov 2023 03:34:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700480060; x=1732016060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2nGnhJsGWMGRpp0Vox7JpzqGcC9rTc12DoKGBk9V+/8=; b=WUTeIb4qzflTymiRObQ2bOBFCp6ZwW7EHLUel2fjFtF6XS6TorgQf5fF iV/cawayJZlvck0PUAkYpO5FUgs/MyjE9yx29ohWp2QJ5S8st5C+g6Loc r16QDcTDDT63hs6jzjAOuIU++P+RTCM4N3iV+eqvRZnrgEMDKAvsHXqvL fJMp5kMaqzREvEKFQJ4R1t9A4lvfsZn5bpJQjulWeKAYx70r9xwnAMsL3 mBIwM7O211DhDMfgOXPNdWmXc4EU90epZ6rHuxhdKQqrPz0//ejMvA7t3 qg93soyYkXe4aABcFW9/46u47to0j5cATm5v9kBxEo8JwLo2ehX2N5fFd Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="376635000" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="376635000" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2023 03:34:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="856963632" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="856963632" Received: from allen-box.sh.intel.com ([10.239.159.127]) by FMSMGA003.fm.intel.com with ESMTP; 20 Nov 2023 03:34:12 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH 3/5] iommu/vt-d: Refactor domain_context_mapping_one() to be reusable Date: Mon, 20 Nov 2023 19:29:42 +0800 Message-Id: <20231120112944.142741-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120112944.142741-1-baolu.lu@linux.intel.com> References: <20231120112944.142741-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 03:34:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783082859548846455 X-GMAIL-MSGID: 1783082859548846455 Extract common code from domain_context_mapping_one() into new functions, making it reusable by other functions such as the upcoming identity domain implementation. No intentional functional changes. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 99 ++++++++++++++++++++++--------------- 1 file changed, 58 insertions(+), 41 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 77769ad4706e..86339dc30243 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1727,6 +1727,61 @@ static void domain_exit(struct dmar_domain *domain) kfree(domain); } +/* + * For kdump cases, old valid entries may be cached due to the + * in-flight DMA and copied pgtable, but there is no unmapping + * behaviour for them, thus we need an explicit cache flush for + * the newly-mapped device. For kdump, at this point, the device + * is supposed to finish reset at its driver probe stage, so no + * in-flight DMA will exist, and we don't need to worry anymore + * hereafter. + */ +static void copied_context_tear_down(struct intel_iommu *iommu, + struct context_entry *context, + u8 bus, u8 devfn) +{ + u16 did_old; + + if (!context_copied(iommu, bus, devfn)) + return; + + assert_spin_locked(&iommu->lock); + + did_old = context_domain_id(context); + context_clear_entry(context); + + if (did_old < cap_ndoms(iommu->cap)) { + iommu->flush.flush_context(iommu, did_old, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + iommu->flush.flush_iotlb(iommu, did_old, 0, 0, + DMA_TLB_DSI_FLUSH); + } + + clear_context_copied(iommu, bus, devfn); +} + +/* + * It's a non-present to present mapping. If hardware doesn't cache + * non-present entry we only need to flush the write-buffer. If the + * _does_ cache non-present entries, then it does so in the special + * domain #0, which we have to flush: + */ +static void context_present_cache_flush(struct intel_iommu *iommu, u16 did, + u8 bus, u8 devfn) +{ + if (cap_caching_mode(iommu->cap)) { + iommu->flush.flush_context(iommu, 0, + (((u16)bus) << 8) | devfn, + DMA_CCMD_MASK_NOBIT, + DMA_CCMD_DEVICE_INVL); + iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); + } else { + iommu_flush_write_buffer(iommu); + } +} + static int domain_context_mapping_one(struct dmar_domain *domain, struct intel_iommu *iommu, u8 bus, u8 devfn) @@ -1755,31 +1810,9 @@ static int domain_context_mapping_one(struct dmar_domain *domain, if (context_present(context) && !context_copied(iommu, bus, devfn)) goto out_unlock; - /* - * For kdump cases, old valid entries may be cached due to the - * in-flight DMA and copied pgtable, but there is no unmapping - * behaviour for them, thus we need an explicit cache flush for - * the newly-mapped device. For kdump, at this point, the device - * is supposed to finish reset at its driver probe stage, so no - * in-flight DMA will exist, and we don't need to worry anymore - * hereafter. - */ - if (context_copied(iommu, bus, devfn)) { - u16 did_old = context_domain_id(context); - - if (did_old < cap_ndoms(iommu->cap)) { - iommu->flush.flush_context(iommu, did_old, - (((u16)bus) << 8) | devfn, - DMA_CCMD_MASK_NOBIT, - DMA_CCMD_DEVICE_INVL); - iommu->flush.flush_iotlb(iommu, did_old, 0, 0, - DMA_TLB_DSI_FLUSH); - } - - clear_context_copied(iommu, bus, devfn); - } - + copied_context_tear_down(iommu, context, bus, devfn); context_clear_entry(context); + context_set_domain_id(context, did); if (translation != CONTEXT_TT_PASS_THROUGH) { @@ -1815,23 +1848,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain, context_set_present(context); if (!ecap_coherent(iommu->ecap)) clflush_cache_range(context, sizeof(*context)); - - /* - * It's a non-present to present mapping. If hardware doesn't cache - * non-present entry we only need to flush the write-buffer. If the - * _does_ cache non-present entries, then it does so in the special - * domain #0, which we have to flush: - */ - if (cap_caching_mode(iommu->cap)) { - iommu->flush.flush_context(iommu, 0, - (((u16)bus) << 8) | devfn, - DMA_CCMD_MASK_NOBIT, - DMA_CCMD_DEVICE_INVL); - iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); - } else { - iommu_flush_write_buffer(iommu); - } - + context_present_cache_flush(iommu, did, bus, devfn); ret = 0; out_unlock: From patchwork Mon Nov 20 11:29:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 167086 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp2135486vqn; Mon, 20 Nov 2023 03:38:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IHW7os0JaScoNI8y5xXPuDcHmhU/74Y5U17bUH0Xw9zNmd+io/H+FL8NMFc4VkJqzakbiAy X-Received: by 2002:a17:902:ab93:b0:1c1:e7b2:27ad with SMTP id f19-20020a170902ab9300b001c1e7b227admr4498561plr.60.1700480307932; Mon, 20 Nov 2023 03:38:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700480307; cv=none; d=google.com; s=arc-20160816; b=G7DRY3ctAJtnehKXCCry2W18qmzczY3lrkYCnVjO9Vq+xBWdhFOhU0XFtvZXoUs04f c8EiHaJh9YFV99A8qSn25WpQ6mrckRXiQ4yB9x3mCnHi+6a/yp/aij9Ys9jp7p19ka1r dKWGmZJDovAZkBgyjeHgRIMLVjflnHGYA9JVo/iVViv81bM6godqXlsp42uxlo0D3zyX 9lkxDtJO63rgmNRL7IFLs3JrJf4kzvaFK9Sx/kts8LbXnAiy7qRFfhAqyfbIhaz8cMXo ybNrxKzxFIysjQa4S0VNZGt1BlHaNS+0VdIGpLiH1WK5KFnRiUJnAbtaV8d+EqfufNpM fgvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=x/CRD7wr22zWC0/9o7Yji8JAzmWkfVcDr+S9vPdkcPM=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=q2C/pTEDuOILKbMkH2qwvLuRk8LY3/9Hq/sg7Jr1mRm6PBxDdoMYXx4jz7zkYomSa2 PI4yfYQsHREKaD5vFaTzdFejJmRMmiaCSbGSaWqaHJtWRDI25aTVVYgteN2ll/fbhujF QSgIFAzS4Qm2u/LYtyqKuRxjoM02P3EGQShvYJgx4kTAqbpKYcwM3NjgsnfYIR2hY42o X3yrJ3RrIQAnHt4kWnAO/nRUPA6Q8V/nnJRskAFgHWoEB3toWdnr+ukcn4SPRX2jfxDM MIHjnfxC00g1Yd4OKNgDmOjkY/qxDTMFt4fagjnd2V3x88Irj4L2t1Kghi3cy4UBqkyC ZiiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YIfDtsx1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id n12-20020a170902f60c00b001cc55bca838si8470569plg.576.2023.11.20.03.38.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 03:38:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YIfDtsx1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 9328A8082B87; Mon, 20 Nov 2023 03:35:50 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233330AbjKTLei (ORCPT + 27 others); Mon, 20 Nov 2023 06:34:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233242AbjKTLeY (ORCPT ); Mon, 20 Nov 2023 06:34:24 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E73CA2 for ; Mon, 20 Nov 2023 03:34:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700480060; x=1732016060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hkQJzkzCcaerOUDvfwwdjPlnrDsDO5uZT2M+8YTSiu0=; b=YIfDtsx12BoeRjHUSKSDkTV6B+iyQnRUX2L4iQSgk7lzp5ZEvqE7/q4P 3qc0xYWdV0+m3AVIg6BJ1Nnn92guh2LOcWyGz6/cJgmCUSV5yUeTID/O7 qBWb+WgtpRtJcy9+WlSnrBnF3Fi7Tp8ZoaxOVtJOEW963adHvMHEGv7tq 5BaG+j2EYVP93eAK6qEmA9Wd8TvRG2RIMh/2ZmkLeSEIuAyErzgygLl62 zbT/Vw0rOnexhNhQDmqkJ7IKX3YuyVMHaoFyOXu/V6IHbSyumbD/lFLpj 7tNat1IKZRtzaXbRNHgzbvIBt/Wrt1DJw6LnGvusHEt9W1dfJGCIRs5Qb A==; X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="376635018" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="376635018" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2023 03:34:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="856963643" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="856963643" Received: from allen-box.sh.intel.com ([10.239.159.127]) by FMSMGA003.fm.intel.com with ESMTP; 20 Nov 2023 03:34:14 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH 4/5] iommu/vt-d: Add support for static identity domain Date: Mon, 20 Nov 2023 19:29:43 +0800 Message-Id: <20231120112944.142741-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120112944.142741-1-baolu.lu@linux.intel.com> References: <20231120112944.142741-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 20 Nov 2023 03:35:50 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783082839600486218 X-GMAIL-MSGID: 1783082839600486218 Add a global static identity domain with guaranteed attach semantics. Software determines VT-d hardware support for pass-through translation by inspecting the capability register. If pass-through translation is not supported, the device is instructed to use DMA domain for its default domain. While most recent VT-d hardware implementations support pass- through translation, this capability is only lacking in some early VT-d implementations. With the static identity domain in place, the domain field of per-device iommu driver data can be either a pointer to a DMA translation domain, or NULL, indicating that the static identity domain is attached. Refine some code to accommodate this change. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 129 ++++++++++++++++++++++++++++++++++-- drivers/iommu/intel/svm.c | 2 +- 2 files changed, 125 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 86339dc30243..ad2e53821a05 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1282,7 +1282,8 @@ static void iommu_enable_pci_caps(struct device_domain_info *info) if (info->ats_supported && pci_ats_page_aligned(pdev) && !pci_enable_ats(pdev, VTD_PAGE_SHIFT)) { info->ats_enabled = 1; - domain_update_iotlb(info->domain); + if (info->domain) + domain_update_iotlb(info->domain); } } @@ -1298,7 +1299,8 @@ static void iommu_disable_pci_caps(struct device_domain_info *info) if (info->ats_enabled) { pci_disable_ats(pdev); info->ats_enabled = 0; - domain_update_iotlb(info->domain); + if (info->domain) + domain_update_iotlb(info->domain); } if (info->pasid_enabled) { @@ -2301,6 +2303,9 @@ static bool device_rmrr_is_relaxable(struct device *dev) */ static int device_def_domain_type(struct device *dev) { + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + if (dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(dev); @@ -2311,6 +2316,13 @@ static int device_def_domain_type(struct device *dev) return IOMMU_DOMAIN_IDENTITY; } + /* + * Hardware does not support the passthrough translation mode. + * Always use a dynamaic mapping domain. + */ + if (!ecap_pass_through(iommu->ecap)) + return IOMMU_DOMAIN_DMA; + return 0; } @@ -3712,6 +3724,9 @@ static void dmar_remove_one_dev_info(struct device *dev) domain_context_clear(info); } + if (!domain) + return; + spin_lock_irqsave(&domain->lock, flags); list_del(&info->link); spin_unlock_irqrestore(&domain->lock, flags); @@ -3920,11 +3935,9 @@ int prepare_domain_attach_device(struct iommu_domain *domain, static int intel_iommu_attach_device(struct iommu_domain *domain, struct device *dev) { - struct device_domain_info *info = dev_iommu_priv_get(dev); int ret; - if (info->domain) - device_block_translation(dev); + device_block_translation(dev); ret = prepare_domain_attach_device(domain, dev); if (ret) @@ -4529,6 +4542,9 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) goto out_tear_down; } + if (domain->type == IOMMU_DOMAIN_IDENTITY) + goto out_tear_down; + dmar_domain = to_dmar_domain(domain); spin_lock_irqsave(&dmar_domain->lock, flags); list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain) { @@ -4703,8 +4719,111 @@ const struct iommu_dirty_ops intel_dirty_ops = { .read_and_clear_dirty = intel_iommu_read_and_clear_dirty, }; +static int context_setup_pass_through(struct device *dev, u8 bus, u8 devfn) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + struct context_entry *context; + + spin_lock(&iommu->lock); + context = iommu_context_addr(iommu, bus, devfn, 1); + if (!context) { + spin_unlock(&iommu->lock); + return -ENOMEM; + } + + if (context_present(context) && !context_copied(iommu, bus, devfn)) { + spin_unlock(&iommu->lock); + return 0; + } + + copied_context_tear_down(iommu, context, bus, devfn); + context_clear_entry(context); + context_set_domain_id(context, FLPT_DEFAULT_DID); + + /* + * In pass through mode, AW must be programmed to indicate the largest + * AGAW value supported by hardware. And ASR is ignored by hardware. + */ + context_set_address_width(context, iommu->msagaw); + context_set_translation_type(context, CONTEXT_TT_PASS_THROUGH); + context_set_fault_enable(context); + context_set_present(context); + if (!ecap_coherent(iommu->ecap)) + clflush_cache_range(context, sizeof(*context)); + context_present_cache_flush(iommu, FLPT_DEFAULT_DID, bus, devfn); + spin_unlock(&iommu->lock); + + return 0; +} + +static int context_setup_pass_through_cb(struct pci_dev *pdev, u16 alias, void *data) +{ + struct device *dev = data; + + if (dev != &pdev->dev) + return 0; + + return context_setup_pass_through(dev, PCI_BUS_NUM(alias), alias & 0xff); +} + +static int device_setup_pass_through(struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + + if (!dev_is_pci(dev)) + return context_setup_pass_through(dev, info->bus, info->devfn); + + return pci_for_each_dma_alias(to_pci_dev(dev), + context_setup_pass_through_cb, dev); +} + +static int identity_domain_attach_dev(struct iommu_domain *domain, + struct device *dev) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + int ret; + + device_block_translation(dev); + + if (dev_is_real_dma_subdevice(dev)) + return 0; + + if (sm_supported(iommu)) { + ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID); + if (!ret) + iommu_enable_pci_caps(info); + } else { + ret = device_setup_pass_through(dev); + } + + return ret; +} + +static int identity_domain_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct device_domain_info *info = dev_iommu_priv_get(dev); + struct intel_iommu *iommu = info->iommu; + + if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev)) + return -EOPNOTSUPP; + + return intel_pasid_setup_pass_through(iommu, dev, pasid); +} + +static struct iommu_domain identity_domain = { + .type = IOMMU_DOMAIN_IDENTITY, + .ops = &(const struct iommu_domain_ops) { + .attach_dev = identity_domain_attach_dev, + .set_dev_pasid = identity_domain_set_dev_pasid, + }, +}; + const struct iommu_ops intel_iommu_ops = { .blocked_domain = &blocking_domain, + .identity_domain = &identity_domain, .capable = intel_iommu_capable, .hw_info = intel_iommu_hw_info, .domain_alloc = intel_iommu_domain_alloc, diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 64c02e6ab1b3..e744f7439e97 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -493,7 +493,7 @@ void intel_drain_pasid_prq(struct device *dev, u32 pasid) domain = info->domain; pdev = to_pci_dev(dev); sid = PCI_DEVID(info->bus, info->devfn); - did = domain_id_iommu(domain, iommu); + did = domain ? domain_id_iommu(domain, iommu) : FLPT_DEFAULT_DID; qdep = pci_ats_queue_depth(pdev); /* From patchwork Mon Nov 20 11:29:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 167087 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp2135523vqn; Mon, 20 Nov 2023 03:38:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IHqFntvko8VgOGqSKI85FwkaJFlrLLUHJF6IByQS2dIjl0Y31UC01DgvFAPPN5kwCvp1tuo X-Received: by 2002:a05:6830:1d61:b0:6b7:52ce:ff38 with SMTP id l1-20020a0568301d6100b006b752ceff38mr8773414oti.16.1700480316342; Mon, 20 Nov 2023 03:38:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700480316; cv=none; d=google.com; s=arc-20160816; b=Up3qLS4HqRZsefG9kmTuOnyOmuLBs2Zjn7EM4JURFAYhro+1n6Pbe1ZhzVsEx07OVZ mnVSb1bJJjJravQSvmyJiJxqsLEBtutANsODXtUM8UW3z60OVeM+84DlaiKQ79vUgFWv p1Fjrx7CBsgLfK/tqJ51sLp4tjs23BK561IeXzkS7O08RsTF3bREiI6/VjG8+VKW75Vd rxyTP4a28ZEi3FIrXhfEBMdutIarVGLlDJRArVfo0FxNKiZ20hBBy67ct5hJ/4YT7+jl /l2FgVxz82fy6VCkUCsHsi9M5vJG9hQvkkm3HadyAt4/zHftTACTwQufT+C8ewuES+m4 LKpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BuKW+zcQXL9Fwql6MW8OXlOIlYqAE1Ub3kIdls4qymY=; fh=5K4v/0ku97kpa0vTvhWLyUEWwXZiIFVMmzwLLirrjJ4=; b=No9H79/ErOfpu+yCsqQoyQn/dh1LkztEr8JL6A0qaAE+j3CZOmeWMWPFVaxSgQba7Z x+yKEmkjGZUmkDCgyr9RS7HzmKj9TDrQK0WCb7DRBOSU33G+aoGLZSKtIvcC3rQcJHdF zZQ3viqHXlFzNrTnenISh25h9BxbDKOcLdWxN2tSecy7ewSZ/5q84sgoorXpQmXMJtQs 87Etru1Y5FNxJoOcGrfwmPudz8v6lGJwyqrTUi7ELoTdZeC3/PlVBppMYnntctiXysw1 wAIOub7McT6yxBgF0djGkznjSHrooI7u9cZM8jCpZCnRnBlhd87xupF5f4Pmit1NcOLP d9Kg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Ix8yGg6i; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id x16-20020a63fe50000000b005be10ed80d8si8193128pgj.87.2023.11.20.03.38.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 03:38:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Ix8yGg6i; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 30E488051B6A; Mon, 20 Nov 2023 03:36:00 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233356AbjKTLel (ORCPT + 27 others); Mon, 20 Nov 2023 06:34:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233247AbjKTLeY (ORCPT ); Mon, 20 Nov 2023 06:34:24 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A37C3D2 for ; Mon, 20 Nov 2023 03:34:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1700480060; x=1732016060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ALg7aRm8HDUzcoLeQTe5aQTvAuPKnbdlNSYa+yIhmEU=; b=Ix8yGg6iouMkWg1Qj/t04174m/bnEPqinCnP/OctNXAWB4K1i4txXBNt VeVxDHXVb3hwQ5eSAnJOwwe7G7VWswlU8yZn/KKfMJ3o5ATMrxbzN1eb4 kIR2S8v9tIFBasPhHrkDWKxVWZ9HewRUgHkFN0t9DcHQ25INxG6X9LKn3 Regzp3LYFRid2KIlsQYYwfh6XhZVPzIFbnv4wgvHxG1967EWci0Zz1HYe 1sgnL+/y5CuFCApzX+c+u7cefLatX1xbXUAktZcbfNJiXdxBZXewsJNGh YGfhbasR/hwLi8DI8Q+cGn6bDxoeqvOn2zrnFg71tcOcDteEfVsaW3VDC g==; X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="376635032" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="376635032" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Nov 2023 03:34:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10899"; a="856963657" X-IronPort-AV: E=Sophos;i="6.04,213,1695711600"; d="scan'208";a="856963657" Received: from allen-box.sh.intel.com ([10.239.159.127]) by FMSMGA003.fm.intel.com with ESMTP; 20 Nov 2023 03:34:17 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Lu Baolu Subject: [PATCH 5/5] iommu/vt-d: Remove si_domain Date: Mon, 20 Nov 2023 19:29:44 +0800 Message-Id: <20231120112944.142741-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231120112944.142741-1-baolu.lu@linux.intel.com> References: <20231120112944.142741-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 20 Nov 2023 03:36:00 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783082848129061009 X-GMAIL-MSGID: 1783082848129061009 The static identity domain has been introduced, rendering the si_domain obsolete. Remove si_domain and cleanup the code accordingly. Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 226 ++++-------------------------------- 1 file changed, 23 insertions(+), 203 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index ad2e53821a05..90dd7f24beda 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -97,15 +97,6 @@ static phys_addr_t root_entry_uctp(struct root_entry *re) return re->hi & VTD_PAGE_MASK; } -/* - * This domain is a statically identity mapping domain. - * 1. This domain creats a static 1:1 mapping to all usable memory. - * 2. It maps to each iommu if successful. - * 3. Each iommu mapps to this domain if successful. - */ -static struct dmar_domain *si_domain; -static int hw_pass_through = 1; - struct dmar_rmrr_unit { struct list_head list; /* list of rmrr units */ struct acpi_dmar_header *hdr; /* ACPI header */ @@ -240,11 +231,6 @@ void free_pgtable_page(void *vaddr) free_page((unsigned long)vaddr); } -static int domain_type_is_si(struct dmar_domain *domain) -{ - return domain->domain.type == IOMMU_DOMAIN_IDENTITY; -} - static int domain_pfn_supported(struct dmar_domain *domain, unsigned long pfn) { int addr_width = agaw_to_width(domain->agaw) - VTD_PAGE_SHIFT; @@ -1796,9 +1782,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain, struct context_entry *context; int agaw, ret; - if (hw_pass_through && domain_type_is_si(domain)) - translation = CONTEXT_TT_PASS_THROUGH; - pr_debug("Set context mapping for %02x:%02x.%d\n", bus, PCI_SLOT(devfn), PCI_FUNC(devfn)); @@ -1817,34 +1800,24 @@ static int domain_context_mapping_one(struct dmar_domain *domain, context_set_domain_id(context, did); - if (translation != CONTEXT_TT_PASS_THROUGH) { - /* - * Skip top levels of page tables for iommu which has - * less agaw than default. Unnecessary for PT mode. - */ - for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { - ret = -ENOMEM; - pgd = phys_to_virt(dma_pte_addr(pgd)); - if (!dma_pte_present(pgd)) - goto out_unlock; - } - - if (info && info->ats_supported) - translation = CONTEXT_TT_DEV_IOTLB; - else - translation = CONTEXT_TT_MULTI_LEVEL; - - context_set_address_root(context, virt_to_phys(pgd)); - context_set_address_width(context, agaw); - } else { - /* - * In pass through mode, AW must be programmed to - * indicate the largest AGAW value supported by - * hardware. And ASR is ignored by hardware. - */ - context_set_address_width(context, iommu->msagaw); + /* + * Skip top levels of page tables for iommu which has + * less agaw than default. Unnecessary for PT mode. + */ + for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { + ret = -ENOMEM; + pgd = phys_to_virt(dma_pte_addr(pgd)); + if (!dma_pte_present(pgd)) + goto out_unlock; } + if (info && info->ats_supported) + translation = CONTEXT_TT_DEV_IOTLB; + else + translation = CONTEXT_TT_MULTI_LEVEL; + + context_set_address_root(context, virt_to_phys(pgd)); + context_set_address_width(context, agaw); context_set_translation_type(context, translation); context_set_fault_enable(context); context_set_present(context); @@ -2078,14 +2051,10 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8 return; } - if (sm_supported(iommu)) { - if (hw_pass_through && domain_type_is_si(info->domain)) - did_old = FLPT_DEFAULT_DID; - else - did_old = domain_id_iommu(info->domain, iommu); - } else { - did_old = context_domain_id(context); - } + if (info->domain) + did_old = domain_id_iommu(info->domain, iommu); + else + did_old = FLPT_DEFAULT_DID; context_clear_entry(context); __iommu_flush_cache(iommu, context, sizeof(*context)); @@ -2148,80 +2117,6 @@ static bool dev_is_real_dma_subdevice(struct device *dev) pci_real_dma_dev(to_pci_dev(dev)) != to_pci_dev(dev); } -static int iommu_domain_identity_map(struct dmar_domain *domain, - unsigned long first_vpfn, - unsigned long last_vpfn) -{ - /* - * RMRR range might have overlap with physical memory range, - * clear it first - */ - dma_pte_clear_range(domain, first_vpfn, last_vpfn); - - return __domain_mapping(domain, first_vpfn, - first_vpfn, last_vpfn - first_vpfn + 1, - DMA_PTE_READ|DMA_PTE_WRITE, GFP_KERNEL); -} - -static int md_domain_init(struct dmar_domain *domain, int guest_width); - -static int __init si_domain_init(int hw) -{ - struct dmar_rmrr_unit *rmrr; - struct device *dev; - int i, nid, ret; - - si_domain = alloc_domain(IOMMU_DOMAIN_IDENTITY); - if (!si_domain) - return -EFAULT; - - if (md_domain_init(si_domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) { - domain_exit(si_domain); - si_domain = NULL; - return -EFAULT; - } - - if (hw) - return 0; - - for_each_online_node(nid) { - unsigned long start_pfn, end_pfn; - int i; - - for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { - ret = iommu_domain_identity_map(si_domain, - mm_to_dma_pfn_start(start_pfn), - mm_to_dma_pfn_end(end_pfn)); - if (ret) - return ret; - } - } - - /* - * Identity map the RMRRs so that devices with RMRRs could also use - * the si_domain. - */ - for_each_rmrr_units(rmrr) { - for_each_active_dev_scope(rmrr->devices, rmrr->devices_cnt, - i, dev) { - unsigned long long start = rmrr->base_address; - unsigned long long end = rmrr->end_address; - - if (WARN_ON(end < start || - end >> agaw_to_width(si_domain->agaw))) - continue; - - ret = iommu_domain_identity_map(si_domain, - mm_to_dma_pfn_start(start >> PAGE_SHIFT), - mm_to_dma_pfn_end(end >> PAGE_SHIFT)); - if (ret) - return ret; - } - } - - return 0; -} - static int dmar_domain_attach_device(struct dmar_domain *domain, struct device *dev) { @@ -2243,8 +2138,6 @@ static int dmar_domain_attach_device(struct dmar_domain *domain, if (!sm_supported(iommu)) ret = domain_context_mapping(domain, dev); - else if (hw_pass_through && domain_type_is_si(domain)) - ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID); else if (domain->use_first_level) ret = domain_setup_first_level(iommu, domain, dev, IOMMU_NO_PASID); else @@ -2255,8 +2148,7 @@ static int dmar_domain_attach_device(struct dmar_domain *domain, return ret; } - if (sm_supported(info->iommu) || !domain_type_is_si(info->domain)) - iommu_enable_pci_caps(info); + iommu_enable_pci_caps(info); return 0; } @@ -2606,8 +2498,6 @@ static int __init init_dmars(void) } } - if (!ecap_pass_through(iommu->ecap)) - hw_pass_through = 0; intel_svm_check(iommu); } @@ -2630,10 +2520,6 @@ static int __init init_dmars(void) check_tylersburg_isoch(); - ret = si_domain_init(hw_pass_through); - if (ret) - goto free_iommu; - /* * for each drhd * enable fault log @@ -2679,10 +2565,6 @@ static int __init init_dmars(void) disable_dmar_iommu(iommu); free_dmar_iommu(iommu); } - if (si_domain) { - domain_exit(si_domain); - si_domain = NULL; - } return ret; } @@ -3057,12 +2939,6 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru) if (ret) goto out; - if (hw_pass_through && !ecap_pass_through(iommu->ecap)) { - pr_warn("%s: Doesn't support hardware pass through.\n", - iommu->name); - return -ENXIO; - } - sp = domain_update_iommu_superpage(NULL, iommu) - 1; if (sp >= 0 && !(cap_super_page_val(iommu->cap) & (1 << sp))) { pr_warn("%s: Doesn't support large page.\n", @@ -3313,52 +3189,6 @@ int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info) return 0; } -static int intel_iommu_memory_notifier(struct notifier_block *nb, - unsigned long val, void *v) -{ - struct memory_notify *mhp = v; - unsigned long start_vpfn = mm_to_dma_pfn_start(mhp->start_pfn); - unsigned long last_vpfn = mm_to_dma_pfn_end(mhp->start_pfn + - mhp->nr_pages - 1); - - switch (val) { - case MEM_GOING_ONLINE: - if (iommu_domain_identity_map(si_domain, - start_vpfn, last_vpfn)) { - pr_warn("Failed to build identity map for [%lx-%lx]\n", - start_vpfn, last_vpfn); - return NOTIFY_BAD; - } - break; - - case MEM_OFFLINE: - case MEM_CANCEL_ONLINE: - { - struct dmar_drhd_unit *drhd; - struct intel_iommu *iommu; - LIST_HEAD(freelist); - - domain_unmap(si_domain, start_vpfn, last_vpfn, &freelist); - - rcu_read_lock(); - for_each_active_iommu(iommu, drhd) - iommu_flush_iotlb_psi(iommu, si_domain, - start_vpfn, mhp->nr_pages, - list_empty(&freelist), 0); - rcu_read_unlock(); - put_pages_list(&freelist); - } - break; - } - - return NOTIFY_OK; -} - -static struct notifier_block intel_iommu_memory_nb = { - .notifier_call = intel_iommu_memory_notifier, - .priority = 0 -}; - static void intel_disable_iommus(void) { struct intel_iommu *iommu = NULL; @@ -3655,12 +3485,7 @@ int __init intel_iommu_init(void) iommu_pmu_register(iommu); } - up_read(&dmar_global_lock); - if (si_domain && !hw_pass_through) - register_memory_notifier(&intel_iommu_memory_nb); - - down_read(&dmar_global_lock); if (probe_acpi_namespace_devices()) pr_warn("ACPI name space devices didn't probe correctly\n"); @@ -3827,8 +3652,6 @@ static struct iommu_domain *intel_iommu_domain_alloc(unsigned type) domain->geometry.force_aperture = true; return domain; - case IOMMU_DOMAIN_IDENTITY: - return &si_domain->domain; case IOMMU_DOMAIN_SVA: return intel_svm_domain_alloc(); default: @@ -3888,8 +3711,7 @@ intel_iommu_domain_alloc_user(struct device *dev, u32 flags, static void intel_iommu_domain_free(struct iommu_domain *domain) { - if (domain != &si_domain->domain) - domain_exit(to_dmar_domain(domain)); + domain_exit(to_dmar_domain(domain)); } int prepare_domain_attach_device(struct iommu_domain *domain, @@ -4596,9 +4418,7 @@ static int intel_iommu_set_dev_pasid(struct iommu_domain *domain, if (ret) goto out_free; - if (domain_type_is_si(dmar_domain)) - ret = intel_pasid_setup_pass_through(iommu, dev, pasid); - else if (dmar_domain->use_first_level) + if (dmar_domain->use_first_level) ret = domain_setup_first_level(iommu, dmar_domain, dev, pasid); else