From patchwork Mon Feb 19 11:15:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 203051 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp1265771dyc; Mon, 19 Feb 2024 05:01:23 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXeLG1m+9JV+dDPDNCPbIZ9JUdckOtshFp6+l+fgj/vr6IRvTUiXK528EpyOESfjyPxgggSZ4H3ivLTqsDLCqkGtAMaEQ== X-Google-Smtp-Source: AGHT+IHcd3gXBEgUvqsRjt6E5MBh4Y4YEriI64VUa/NDk7ZjqTc2j7Fzk3zK/y4WX3DpTvmrg5/B X-Received: by 2002:a25:ab06:0:b0:dc2:2e01:4ff0 with SMTP id u6-20020a25ab06000000b00dc22e014ff0mr10381954ybi.45.1708347682873; Mon, 19 Feb 2024 05:01:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708347682; cv=pass; d=google.com; s=arc-20160816; b=iVIUnJCQipUgtobmQzczEx8tqpXXA5FexRlCOKh35tyVhklOdc9VdTj1HQxwMeQL8u TqhmgXYpLZ+0DU+4uqne/eE3lk/M4awiDbOjssMx3OFp17PZ9fiWzf65uZs5kY+BtAv2 od7700PHPyNhIlUkWCZFJSFSvBhd+dzc6bzc+4AGwImOVrb82g3aOSWm/zmMLaZhtExT wQvVftk92QEi1KlOfw+jc5VXl7dQ9N97m/diCA4PftHbHdSSJ77wpjbzMlIHUPEyfJ5R xGF7UbZVzaZEpwBxaowWMNvSnT+6SM7UmirYV65sLCZax5/cJ4mgIJr3KZ1Bwa2If91l gZgg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=pE5BXOCWCdps/MhTXNSchEKQtId/Nkr1fwNGGFapMBU=; fh=xIXrYyuim0+qQ0OPX7PXG3ceqpJb0Om6wEk3bRGARVs=; b=IfMcTS4ADXq7mN0CVLBPOybtcXkxlnUT3mpX2pDe2oYGDixV7yBiNC/9fZFxsYKyQ9 FNX/Ca0GXBs0MIDxT7A/+9CH8Pa2MUvTFM4xGWBQ0ZUGgESVA4e6TP/Yi9nb8thKzpNc uMAUr0OekwM3mL67SFFmv0SVIutt0dfEbPQNNdS3xLFPLd/3eQvJnfefdzjPFblIfM9t glRRSG2w8eOn76HoURj0fEQDJ3nvlFm3WE2F5cS4p5c9I5UBvpVYdpN9/Zlk4O9ref+G hXzzbVCOV+q2fOX2OY1OkGMsPi76sxug5HRV98EOfR8L3sBNjEasTfTwq4sF/fPVpaz2 tjxQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WdZKyfKu; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-71186-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-71186-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id t6-20020a25f606000000b00dce3f1bc8c5si1471609ybd.671.2024.02.19.05.01.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Feb 2024 05:01:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-71186-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WdZKyfKu; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-71186-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-71186-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 1D1251C22357 for ; Mon, 19 Feb 2024 11:22:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D25072C848; Mon, 19 Feb 2024 11:21:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WdZKyfKu" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D07EA2C693 for ; Mon, 19 Feb 2024 11:21:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708341716; cv=none; b=MfD2HEBJIdYwkU7fpFk1O8KWsfA2PLR6TXMkqkiCVXHUel2y2Ah2b+ibWeD+klleOsoRwsTbtQys1LjBNBexGt51oOE16V38WfLyYBPgrOOiaOVraiXxo8Rbxd72X7iuzB7mNCYh0fMXmyu5mRPw6RDOTkqQ6T1MYHnV3mcC1iw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708341716; c=relaxed/simple; bh=FU4fhoGWvuGe6qI9myzpUIIpInnM2vTTQxC1ESzz82w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Tu47pkt+j/8aCXsxkgq3I5cUn6ZItjXZfBXBBdbFZ5hs6eX3ClYVFeCiDxn9TsgdkPGesgDs21PAHO9y0V2y8U6pd1kNq0sI5owP+LimxvUM52/Lro+qAcURAB5fFUFCa6nPLtK/cYzBfplf8Aagztl4aXPEnKpT66hL800hMGo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WdZKyfKu; arc=none smtp.client-ip=198.175.65.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708341715; x=1739877715; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FU4fhoGWvuGe6qI9myzpUIIpInnM2vTTQxC1ESzz82w=; b=WdZKyfKuTF7tRNL0utkI/BiM6S9ojPgiKh1aGLJ0Y0ksG7DWVYlHYnTl NXZXrmsKeNbAH5Pt5qEf6LymeOgE86Gj+ozOl8d9bHPdOwS2eF5S9kQ3m Ju/lhEyMvRTPzu1/p9bP/1FF4XB5kquZcmDyqnAKlU4yQLnpASHbt02A8 rdwLZSliJKEZOH/6gqgyIpJfdutlDLQK1JbxaZv3UVkLIfdgsw1jekPoD WXiyUn4Mm2Vd132VFEOzXDts26VtjDRjWIkQKukEEq2TwnDBglG02CWq1 jzNULSCucvg+/HVgjhKuXtz+67rzGV+q/pelA7FaKo1v3q1kPh4kjWGSq w==; X-IronPort-AV: E=McAfee;i="6600,9927,10988"; a="13823334" X-IronPort-AV: E=Sophos;i="6.06,170,1705392000"; d="scan'208";a="13823334" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2024 03:21:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,170,1705392000"; d="scan'208";a="4826784" Received: from allen-box.sh.intel.com ([10.239.159.127]) by orviesa006.jf.intel.com with ESMTP; 19 Feb 2024 03:21:52 -0800 From: Lu Baolu To: Joerg Roedel Cc: Yi Liu , Arnd Bergmann , Kevin Tian , Joao Martins , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 01/10] iommu/vt-d: Track nested domains in parent Date: Mon, 19 Feb 2024 19:15:52 +0800 Message-Id: <20240219111601.96405-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240219111601.96405-1-baolu.lu@linux.intel.com> References: <20240219111601.96405-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791332380067088634 X-GMAIL-MSGID: 1791332380067088634 From: Yi Liu Today the parent domain (s2_domain) is unaware of which DID's are used by and which devices are attached to nested domains (s1_domain) nested on it. This leads to a problem that some operations (flush iotlb/devtlb and enable dirty tracking) on parent domain only apply to DID's and devices directly tracked in the parent domain hence are incomplete. This tracks the nested domains in list in parent domain. With this, operations on parent domain can loop the nested domains and refer to the devices and iommu_array to ensure the operations on parent domain take effect on all the affected devices and iommus. Signed-off-by: Yi Liu Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20240208082307.15759-2-yi.l.liu@intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.h | 6 ++++++ drivers/iommu/intel/iommu.c | 18 ++++++++++++++---- drivers/iommu/intel/nested.c | 12 +++++++++++- 3 files changed, 31 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index d02f916d8e59..9b27edb73aa9 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -627,6 +627,10 @@ struct dmar_domain { int agaw; /* maximum mapped address */ u64 max_addr; + /* Protect the s1_domains list */ + spinlock_t s1_lock; + /* Track s1_domains nested on this domain */ + struct list_head s1_domains; }; /* Nested user domain */ @@ -637,6 +641,8 @@ struct dmar_domain { unsigned long s1_pgtbl; /* page table attributes */ struct iommu_hwpt_vtd_s1 s1_cfg; + /* link to parent domain siblings */ + struct list_head s2_link; }; }; diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 6fb5f6fceea1..e393c62776f3 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -3883,6 +3883,7 @@ intel_iommu_domain_alloc_user(struct device *dev, u32 flags, bool dirty_tracking = flags & IOMMU_HWPT_ALLOC_DIRTY_TRACKING; bool nested_parent = flags & IOMMU_HWPT_ALLOC_NEST_PARENT; struct intel_iommu *iommu = info->iommu; + struct dmar_domain *dmar_domain; struct iommu_domain *domain; /* Must be NESTING domain */ @@ -3908,11 +3909,16 @@ intel_iommu_domain_alloc_user(struct device *dev, u32 flags, if (!domain) return ERR_PTR(-ENOMEM); - if (nested_parent) - to_dmar_domain(domain)->nested_parent = true; + dmar_domain = to_dmar_domain(domain); + + if (nested_parent) { + dmar_domain->nested_parent = true; + INIT_LIST_HEAD(&dmar_domain->s1_domains); + spin_lock_init(&dmar_domain->s1_lock); + } if (dirty_tracking) { - if (to_dmar_domain(domain)->use_first_level) { + if (dmar_domain->use_first_level) { iommu_domain_free(domain); return ERR_PTR(-EOPNOTSUPP); } @@ -3924,8 +3930,12 @@ intel_iommu_domain_alloc_user(struct device *dev, u32 flags, static void intel_iommu_domain_free(struct iommu_domain *domain) { + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + + WARN_ON(dmar_domain->nested_parent && + !list_empty(&dmar_domain->s1_domains)); if (domain != &si_domain->domain) - domain_exit(to_dmar_domain(domain)); + domain_exit(dmar_domain); } int prepare_domain_attach_device(struct iommu_domain *domain, diff --git a/drivers/iommu/intel/nested.c b/drivers/iommu/intel/nested.c index f26c7f1c46cc..6a75f6eb18f1 100644 --- a/drivers/iommu/intel/nested.c +++ b/drivers/iommu/intel/nested.c @@ -70,7 +70,13 @@ static int intel_nested_attach_dev(struct iommu_domain *domain, static void intel_nested_domain_free(struct iommu_domain *domain) { - kfree(to_dmar_domain(domain)); + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + struct dmar_domain *s2_domain = dmar_domain->s2_domain; + + spin_lock(&s2_domain->s1_lock); + list_del(&dmar_domain->s2_link); + spin_unlock(&s2_domain->s1_lock); + kfree(dmar_domain); } static void nested_flush_dev_iotlb(struct dmar_domain *domain, u64 addr, @@ -201,5 +207,9 @@ struct iommu_domain *intel_nested_domain_alloc(struct iommu_domain *parent, spin_lock_init(&domain->lock); xa_init(&domain->iommu_array); + spin_lock(&s2_domain->s1_lock); + list_add(&domain->s2_link, &s2_domain->s1_domains); + spin_unlock(&s2_domain->s1_lock); + return &domain->domain; }