From patchwork Fri Mar 1 08:07:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 208755 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp987046dyb; Fri, 1 Mar 2024 02:36:04 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWHf1fWgqS2VGkSyo/YTv+bAXqxtrDpoHTsqmUwgdFR4Ts4wbiNgrugYXHAOihCnbAG1I4IspV5J3DbIMmwnBBhM6ddZw== X-Google-Smtp-Source: AGHT+IF4TJiTgnYfXDG4aspO7eozu9u+fKMJLOugw9wfwxMA3i2XFs+wAI22MVVHocwv0tTyBAEA X-Received: by 2002:a05:6a20:d48f:b0:1a0:e4da:e100 with SMTP id im15-20020a056a20d48f00b001a0e4dae100mr1088531pzb.33.1709289363918; Fri, 01 Mar 2024 02:36:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709289363; cv=pass; d=google.com; s=arc-20160816; b=q07LTlM7yOk2SxDeH2b157LC4yQKvNs0B+PQbFiOtkdiVeUa0zGYfZA0LfeUcFa+3e UR0zsunPNGIcqvjljIQuOkbWa1j3H7S9E3YfRl2ne5FtdSzUbJ8/LmM6R25Z7B+67wC4 XFOBKk9/1ijVVOVmISi6pVY1+offSbv2QIo7bB48tfjNDKrm03MTmN2zDVpSuKnFWoEf OFp5bzoXHULW01ir6H+EhDUDsq+06eh05UNpVuRa1Ev7qle3OwMdOyvzcQB4KmAMPPfI 0RSxDIbXdQKBb1/NX/OAJuAAs9nXQHJI/Hrs6aRT+jnHtvfHESW23ECSMmMNPq93uMCM /35A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=omm/lp+RPrfGo4jfzoZALvQUooEIgG5MpB7/PKWymYk=; fh=CcvPup2qFKOU+RSlxpmYRdCRaLkLlngXWCtLCjXBjlA=; b=bk2XHEBz5iU7vSyNvntTjJWl1wQeX1q2I98e38V9ulL0+j3+ZDHFQXpJhQuAKRxbVp tPo8+GtEgZVxeapcb2Qz9PJ1uRubh4fXJOBJYk7Q6nIku8zQzvrTJBr8hPFoyOORpJ5U sbacAnp9mD3LKBQf3zLa7eisIRZw5sN4LvRG9d6WccJCDQidJkZgAGkO3JMJ/RmZ4lbO OOj14vxcb+gkRAyK83nHVMk38johgcqElwF1s7UukxvrsQ3vvk2v//KF/HgFc7Y3JOp5 3e90muhkH/Q+xSMejZrAbEcFRMhXc7jUkTGB1HFlS/lUpZWvLW9HKoVa6m85h8uzy8gu U5Ig==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=eauSVewi; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-88051-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-88051-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id pi10-20020a17090b1e4a00b002997d3984e7si3508007pjb.100.2024.03.01.02.36.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 02:36:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-88051-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=eauSVewi; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-88051-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-88051-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 39378289EEA for ; Fri, 1 Mar 2024 08:08:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3082A69E17; Fri, 1 Mar 2024 08:08:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="eauSVewi" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EC3F69E03 for ; Fri, 1 Mar 2024 08:07:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709280479; cv=none; b=ilofXo76dDOrf7Q7cReSJE81UWoY0AqA6KTREedP9390DRu6XJCUC67/7WsHDQuEvB3vCnAigAoLDUUhKCTQtlKSl97WkwH2hePC73rS2ID57QTY9fgThxlo3XVun9yL3QbQphLWgk/d+h6ZTDE2IxAy4rcvLVR8MNhTGMgMy7I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709280479; c=relaxed/simple; bh=/cVxTPK8YMoBH+k7owpPAvLO8OBKEisXNbpy2Gi5tXs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KoF1CLz9yG9TkX63uvWlG8awDNXXACKPpP8wSnbC62btpXFzyLdeKZCkL0VBKwzd7XZbRJ9ytvnVgqNUaEAbJHk738jUzAHvokFPVGkfXy4KGdgCaxyUMB36D/498Wqv5Vo+iudEgtFfefhd+T1/SXKrjM/Bj3iU2e4RRxA9940= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=eauSVewi; arc=none smtp.client-ip=198.175.65.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709280478; x=1740816478; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/cVxTPK8YMoBH+k7owpPAvLO8OBKEisXNbpy2Gi5tXs=; b=eauSVewi23opmyVnh7gJL1Jq27r6WB7PpKnu+2xGJduAfTYjM82P7jgo IGfF1/Uy8lOTUb46dMNiq5ELOT5d3dWMcqLrMBtWbpPuuhEnAg4Udnas3 nGvRdhNiD7xfj9phHRgFdQLweDHPCiNAn0LW78AQtR6ezrjKLQAeeuH/z 3a3mZxsfuRc+C+9feWKELdJR97MUgNt3BuHTHilu8MT/HY0tFxmwrMmRq G30mcgkExt8NI3E5q8YjhjfrphfbDlbtEW1F5xCC9meP9JEZbcQi+C69e am8y9zRB0lUmTg73SdMvQ63ktydxdq2E31DyPtjQnZfugVsBZ1fyWrP3V Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10999"; a="15240430" X-IronPort-AV: E=Sophos;i="6.06,195,1705392000"; d="scan'208";a="15240430" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Mar 2024 00:07:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,195,1705392000"; d="scan'208";a="8187527" Received: from unknown (HELO ply01-vm-store.amr.corp.intel.com) ([10.238.153.201]) by orviesa009.jf.intel.com with ESMTP; 01 Mar 2024 00:07:52 -0800 From: Ethan Zhao To: baolu.lu@linux.intel.com, bhelgaas@google.com, robin.murphy@arm.com, jgg@ziepe.ca Cc: kevin.tian@intel.com, dwmw2@infradead.org, will@kernel.org, lukas@wunner.de, yi.l.liu@intel.com, dan.carpenter@linaro.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, Ethan Zhao Subject: [PATCH v14 3/3] iommu/vt-d: improve ITE fault handling if target device isn't present Date: Fri, 1 Mar 2024 03:07:27 -0500 Message-Id: <20240301080727.3529832-4-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240301080727.3529832-1-haifeng.zhao@linux.intel.com> References: <20240301080727.3529832-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792319803798052162 X-GMAIL-MSGID: 1792319803798052162 Because surprise removal could happen anytime, e.g. user could request safe removal to EP(endpoint device) via sysfs and brings its link down to do surprise removal cocurrently. such aggressive cases would cause ATS invalidation request issued to non-existence target device, then deadly loop to retry that request after ITE fault triggered in interrupt context. this patch aims to optimize the ITE handling by checking the target device presence state to avoid retrying the timeout request blindly, thus avoid hard lockup or system hang. Devices TLB should only be invalidated when devices are in the iommu->device_rbtree (probed, not released) and present. Fixes: 6ba6c3a4cacf ("VT-d: add device IOTLB invalidation support") Reviewed-by: Dan Carpenter Signed-off-by: Ethan Zhao --- drivers/iommu/intel/dmar.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index d14797aabb7a..fa1ac111a6e2 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1273,6 +1273,8 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) { u32 fault; int head, tail; + u64 iqe_err, ite_sid; + struct device *dev = NULL; struct q_inval *qi = iommu->qi; int shift = qi_shift(iommu); @@ -1317,6 +1319,13 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) tail = readl(iommu->reg + DMAR_IQT_REG); tail = ((tail >> shift) - 1 + QI_LENGTH) % QI_LENGTH; + /* + * SID field is valid only when the ITE field is Set in FSTS_REG + * see Intel VT-d spec r4.1, section 11.4.9.9 + */ + iqe_err = dmar_readq(iommu->reg + DMAR_IQER_REG); + ite_sid = DMAR_IQER_REG_ITESID(iqe_err); + writel(DMA_FSTS_ITE, iommu->reg + DMAR_FSTS_REG); pr_info("Invalidation Time-out Error (ITE) cleared\n"); @@ -1326,6 +1335,19 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) head = (head - 2 + QI_LENGTH) % QI_LENGTH; } while (head != tail); + /* + * If device was released or isn't present, no need to retry + * the ATS invalidate request anymore. + * + * 0 value of ite_sid means old VT-d device, no ite_sid value. + * see Intel VT-d spec r4.1, section 11.4.9.9 + */ + if (ite_sid) { + dev = device_rbtree_find(iommu, ite_sid); + if (!dev || !dev_is_pci(dev) || + !pci_device_is_present(to_pci_dev(dev))) + return -ETIMEDOUT; + } if (qi->desc_status[wait_index] == QI_ABORT) return -EAGAIN; }