From patchwork Tue Jan 30 08:08:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 194218 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1314637dyb; Tue, 30 Jan 2024 07:52:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IFczYYyu9fx9DbGYpNwV6FMUv4x572q9o0OT66F8FUTuZy6gX8YC4Yqs2fzsDWpzFhk0Ed7 X-Received: by 2002:a05:6402:430b:b0:55d:71c5:5186 with SMTP id m11-20020a056402430b00b0055d71c55186mr7820935edc.1.1706629946003; Tue, 30 Jan 2024 07:52:26 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706629945; cv=pass; d=google.com; s=arc-20160816; b=PVoI9UOfjXbllaMHNcnp4wnKhfF3itiMMJr7bZWHvy7kL0BltnVINP0TbKyq9BmytL kehaiLNoF89k7G+7OotGtqd8RkqYaNIOapouwByIo7VfS2RJ4WBf1BhvMPpwfpp/fKQU OGnrWTJYBfYkYEw3q8PULXUjdr1g8+LkVDiKJkdvjdPNDUkvQo5rRGMCd++IEtt08mAK bAdetlHxhTSDDDXecR/EM/c0+2Hqz0NHKJ07K79wFm2UPi8/R9WCsHEI26rcm97xt0q9 4Llbjt5FWDpFKROGay+hkDRlE5UHod2aKYT0/utB3upzxi9ZfAAo5/RkYgm2G93fWtXg VnfA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=rYZfneW1Cpi0gDNA3bCUB2zkZSDptLP7ppZvreco2V4=; fh=oaHNS1xUSnQss3sOwgE3xgF1EHmYE8WzykuHrV1+rzg=; b=t8s1lFqkJJuAeI8qxnbbnVpj2q6BJUNdBrWEPGx7Q0RjiXxjgC2LTuH6/4aHPVeEyo rBQbv5YeQm3j+KmQ1NaIGtAvlG43RyxglN0lIj89ZWiN7oncEvs4twVr181XfvD2Sa7L hc9sGDKj13nS4YmAjMmczx4tRXPCYjqZWr8nhdTxFbA1JF/3nGIq3b1tbu3QOE7epvZR Z0QbPrTteldqJGhkekxURFgCSjtwpr9niNyzWrjYqGBY7dwDUTZF23GMKIGaoyXlqsyM nUSGHJDPkyq6Z1mirHLsgn7fKDIK6ymZYcQyJcCFl6qSKCdz3uJaV9ukbGC/09hjRLCC i7eQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cIRn+Eui; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-44114-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-44114-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id z100-20020a509e6d000000b0055ef30c2450si2398365ede.618.2024.01.30.07.52.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jan 2024 07:52:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-44114-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cIRn+Eui; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-44114-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-44114-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id C0E961F2A485 for ; Tue, 30 Jan 2024 08:19:25 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DA481679E4; Tue, 30 Jan 2024 08:15:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="cIRn+Eui" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDFE467720; Tue, 30 Jan 2024 08:14:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706602497; cv=none; b=WTcHOJnHwNFJTWYORXhqqOgAqLUj2Bey+dI0hDGgEy7dx3ApvUQ0G7VtYOBsy0kY2TylKpu9XOooOiequg+uJ+pSylVAom8otUb8Td0p8ypIAY/8PDQjCZGSTHUFVeTPgSk7gMy+iEBsGLQZOT6QCDK1+qkkIaRd+CCaMXXvNVg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706602497; c=relaxed/simple; bh=xJjYsaefgZGHvEwRQ3CV5IkekvSpCZPTHHD42unBWt0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=c/AvxVoPEDTXHqyaBRkD3iVfUBHhPD8HmWLFBd/dbUDeb3i+bgNysfX1wNcSBMhCMvQXD7ZZp1xQIjduwDGZ37c42ljP/hm7wz8m/qSZ46aItHqlATqCfC2xTtfPK9r8tMZ47Q+T9m3gRzWscaoCkeQI4g/+mitTMMZu1E+kziI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=cIRn+Eui; arc=none smtp.client-ip=198.175.65.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706602496; x=1738138496; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xJjYsaefgZGHvEwRQ3CV5IkekvSpCZPTHHD42unBWt0=; b=cIRn+EuiflYZJi4dETlXkSiGJo8LesnkGGxtZ+nbC0Waw/c7GrvKduFu 5MM65WppIsowwYqSmxSiDC9u/bzA5vpgmRHNGA8UGxZoQdnIlcVK3tnn0 sWtNyVHXYOKqD90EpGf0mvPCMowMeQm64mnvA3ro7itrkb/vyc0u0VhCu cVJmA5fBHwBekcuwOjTeIY8JbLexTNGegidIAY1d/BRLRNt9g5Qqbesik 3sbz4Wr41fv670Se20dygmqSDFxzMbH+XC6FWFOeCurXIborTRgQJD9yB uxUEDtewMC5Q+5S4zkYoM/nyA7m5XgiZD3KpdMNqErlVncaqp+Ym/HLSu w==; X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="10588452" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="10588452" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jan 2024 00:14:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="3633840" Received: from allen-box.sh.intel.com ([10.239.159.127]) by fmviesa003.fm.intel.com with ESMTP; 30 Jan 2024 00:14:51 -0800 From: Lu Baolu To: Joerg Roedel , Will Deacon , Robin Murphy , Jason Gunthorpe , Kevin Tian , Jean-Philippe Brucker , Nicolin Chen Cc: Yi Liu , Jacob Pan , Longfang Liu , Yan Zhao , Joel Granados , iommu@lists.linux.dev, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jason Gunthorpe Subject: [PATCH v11 11/16] iommu: Refine locking for per-device fault data management Date: Tue, 30 Jan 2024 16:08:30 +0800 Message-Id: <20240130080835.58921-12-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240130080835.58921-1-baolu.lu@linux.intel.com> References: <20240130080835.58921-1-baolu.lu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789531202248111084 X-GMAIL-MSGID: 1789531202248111084 The per-device fault data is a data structure that is used to store information about faults that occur on a device. This data is allocated when IOPF is enabled on the device and freed when IOPF is disabled. The data is used in the paths of iopf reporting, handling, responding, and draining. The fault data is protected by two locks: - dev->iommu->lock: This lock is used to protect the allocation and freeing of the fault data. - dev->iommu->fault_parameter->lock: This lock is used to protect the fault data itself. Apply the locking mechanism to the fault reporting and responding paths. The fault_parameter->lock is also added in iopf_queue_discard_partial(). It does not fix any real issue, as iopf_queue_discard_partial() is only used in the VT-d driver's prq_event_thread(), which is a single-threaded path that reports the IOPFs. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Reviewed-by: Jason Gunthorpe Tested-by: Yan Zhao Tested-by: Longfang Liu --- drivers/iommu/io-pgfault.c | 61 +++++++++++++++++++------------------- 1 file changed, 30 insertions(+), 31 deletions(-) diff --git a/drivers/iommu/io-pgfault.c b/drivers/iommu/io-pgfault.c index c1e88da973ce..5aea8402be47 100644 --- a/drivers/iommu/io-pgfault.c +++ b/drivers/iommu/io-pgfault.c @@ -53,7 +53,7 @@ static struct iommu_domain *get_domain_for_iopf(struct device *dev, /** * iommu_handle_iopf - IO Page Fault handler * @fault: fault event - * @dev: struct device. + * @iopf_param: the fault parameter of the device. * * Add a fault to the device workqueue, to be handled by mm. * @@ -90,29 +90,21 @@ static struct iommu_domain *get_domain_for_iopf(struct device *dev, * * Return: 0 on success and <0 on error. */ -static int iommu_handle_iopf(struct iommu_fault *fault, struct device *dev) +static int iommu_handle_iopf(struct iommu_fault *fault, + struct iommu_fault_param *iopf_param) { int ret; struct iopf_group *group; struct iommu_domain *domain; struct iopf_fault *iopf, *next; - struct iommu_fault_param *iopf_param; - struct dev_iommu *param = dev->iommu; + struct device *dev = iopf_param->dev; - lockdep_assert_held(¶m->lock); + lockdep_assert_held(&iopf_param->lock); if (fault->type != IOMMU_FAULT_PAGE_REQ) /* Not a recoverable page fault */ return -EOPNOTSUPP; - /* - * As long as we're holding param->lock, the queue can't be unlinked - * from the device and therefore cannot disappear. - */ - iopf_param = param->fault_param; - if (!iopf_param) - return -ENODEV; - if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) { iopf = kzalloc(sizeof(*iopf), GFP_KERNEL); if (!iopf) @@ -186,18 +178,19 @@ static int iommu_handle_iopf(struct iommu_fault *fault, struct device *dev) */ int iommu_report_device_fault(struct device *dev, struct iopf_fault *evt) { - struct dev_iommu *param = dev->iommu; + struct iommu_fault_param *fault_param; struct iopf_fault *evt_pending = NULL; - struct iommu_fault_param *fparam; + struct dev_iommu *param = dev->iommu; int ret = 0; - if (!param || !evt) - return -EINVAL; - - /* we only report device fault if there is a handler registered */ mutex_lock(¶m->lock); - fparam = param->fault_param; + fault_param = param->fault_param; + if (!fault_param) { + mutex_unlock(¶m->lock); + return -EINVAL; + } + mutex_lock(&fault_param->lock); if (evt->fault.type == IOMMU_FAULT_PAGE_REQ && (evt->fault.prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) { evt_pending = kmemdup(evt, sizeof(struct iopf_fault), @@ -206,20 +199,18 @@ int iommu_report_device_fault(struct device *dev, struct iopf_fault *evt) ret = -ENOMEM; goto done_unlock; } - mutex_lock(&fparam->lock); - list_add_tail(&evt_pending->list, &fparam->faults); - mutex_unlock(&fparam->lock); + list_add_tail(&evt_pending->list, &fault_param->faults); } - ret = iommu_handle_iopf(&evt->fault, dev); + ret = iommu_handle_iopf(&evt->fault, fault_param); if (ret && evt_pending) { - mutex_lock(&fparam->lock); list_del(&evt_pending->list); - mutex_unlock(&fparam->lock); kfree(evt_pending); } done_unlock: + mutex_unlock(&fault_param->lock); mutex_unlock(¶m->lock); + return ret; } EXPORT_SYMBOL_GPL(iommu_report_device_fault); @@ -232,18 +223,23 @@ int iommu_page_response(struct device *dev, struct iopf_fault *evt; struct iommu_fault_page_request *prm; struct dev_iommu *param = dev->iommu; + struct iommu_fault_param *fault_param; const struct iommu_ops *ops = dev_iommu_ops(dev); bool has_pasid = msg->flags & IOMMU_PAGE_RESP_PASID_VALID; if (!ops->page_response) return -ENODEV; - if (!param || !param->fault_param) + mutex_lock(¶m->lock); + fault_param = param->fault_param; + if (!fault_param) { + mutex_unlock(¶m->lock); return -EINVAL; + } /* Only send response if there is a fault report pending */ - mutex_lock(¶m->fault_param->lock); - if (list_empty(¶m->fault_param->faults)) { + mutex_lock(&fault_param->lock); + if (list_empty(&fault_param->faults)) { dev_warn_ratelimited(dev, "no pending PRQ, drop response\n"); goto done_unlock; } @@ -251,7 +247,7 @@ int iommu_page_response(struct device *dev, * Check if we have a matching page request pending to respond, * otherwise return -EINVAL */ - list_for_each_entry(evt, ¶m->fault_param->faults, list) { + list_for_each_entry(evt, &fault_param->faults, list) { prm = &evt->fault.prm; if (prm->grpid != msg->grpid) continue; @@ -279,7 +275,8 @@ int iommu_page_response(struct device *dev, } done_unlock: - mutex_unlock(¶m->fault_param->lock); + mutex_unlock(&fault_param->lock); + mutex_unlock(¶m->lock); return ret; } EXPORT_SYMBOL_GPL(iommu_page_response); @@ -362,11 +359,13 @@ int iopf_queue_discard_partial(struct iopf_queue *queue) mutex_lock(&queue->lock); list_for_each_entry(iopf_param, &queue->devices, queue_list) { + mutex_lock(&iopf_param->lock); list_for_each_entry_safe(iopf, next, &iopf_param->partial, list) { list_del(&iopf->list); kfree(iopf); } + mutex_unlock(&iopf_param->lock); } mutex_unlock(&queue->lock); return 0;