From patchwork Mon Dec 25 08:25:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 183155 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp290008dyb; Mon, 25 Dec 2023 00:26:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IELO9aS1w+yXyqS+wdnJt2j3sWByESjf/nFhM4966el4O6BPyZFKrVOcmM9TM/A+QtPeYk7 X-Received: by 2002:a05:620a:27c5:b0:781:2d60:ad22 with SMTP id i5-20020a05620a27c500b007812d60ad22mr6663380qkp.118.1703492815646; Mon, 25 Dec 2023 00:26:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703492815; cv=none; d=google.com; s=arc-20160816; b=YsyYntaQ2ZVD/kJcso/Xr0IiusXYJ8GK8+BstIchcG79FlgpVIRWN/SrlXS72v8ZaB Sz4VipqENKLxRX9hrlZg1Nzp2Vwm5pDqTW17IKQnw1cUF6xv/OOC8rzOux+hvBDLD8W/ hqWTqvSRz+m07cVjAk9+RureuU41fbkBHyUvkF9MUuaQRbHNihB4M99a3vN2zl/xMyaV SwRpPHW8Hey55fK3NV+5REnr5gppyOPn2Wklezz5WqZYMnb8LyyiXOK9c3C8KCJ9wX5z MQ69tYg3Ch2PW30K3X6KkRWwam42ei2c3lrO/pkazjQ6AN6UDdMp8llMbm/gB/BPlUWn Yc9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=3OHohJ+QkgygS3e7Fw/RLf/mz8+aFwix0BS+Z/mm3g0=; fh=cQTxynQ7rGExgmbyXq8tNmd/VmE1Fzgt9QW51hABj2c=; b=0Oq4jyTbp+LSCDKwh6MNkut3csGIBR0U1ZCG21XP255A+jd10u9w0Tp7OpzhLeeNnT bCj7idqeY2DiV5Gw4EZ61MdL/Pleut485Pt9sMtdUGkbIy4PciQKGTfnCgL+H0p0CuZ2 U294nn73dZCTAdCcjR5JrxvY1dPFYXu03A24By9QTZwfmU679Lxywzq0l+ae+4vm2Flp fdp6FMGqnJTPOJq0wyCQtZLawEjA/4IJ0/GGFJ7s5H73KomRpZuBSv/wIy9RG65G12Gf Cfx/SaEjyDBfdMJ1XCUwmgQo8iDOjTYthC5DMhunuutyq5WvLSq2u+XmkrOP/PU2G3Yr i1iA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=X6Qdk983; spf=pass (google.com: domain of linux-kernel+bounces-10953-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-10953-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id i2-20020a05620a404200b0077da5e2ff28si10392455qko.289.2023.12.25.00.26.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Dec 2023 00:26:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-10953-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=X6Qdk983; spf=pass (google.com: domain of linux-kernel+bounces-10953-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-10953-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 71C9F1C215E7 for ; Mon, 25 Dec 2023 08:26:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 42CDB10A12; Mon, 25 Dec 2023 08:25:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="X6Qdk983" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A944107A6; Mon, 25 Dec 2023 08:25:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703492747; x=1735028747; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZwYvHcOe8vw7UWjBi/7Fe5r5f9iUZlNZFiDgjbuH7C8=; b=X6Qdk983rwKO0bmHTSNtmu3EYH1bfNjO5CDEDR/MsbObVP23gNznM5Ty 7gkkyy8tXer661W7G0OVvmhdXVK6RyMe5rV+Em0poWVlduVZxg8i/voms 5rkurAMl4tLqZus/HSm3Mt0pO8PfLSET/bNxD9/u5Mb42bYPmDN+Yx2Mr VZVjbkMUC6eTZt6hYpCue044d13G3gdmz/Y4TET5H8DLzfzgHw9z9IiEV jog2C/myWptOSKPwngTZ7rli/u5r6uxNyZL1SLFDaIwT92f5594/l84h4 JjhGst1RqR9fNBylL56VWBCghWPP2sWUPETVTMyTucr2kIjhzpPTQLg6V g==; X-IronPort-AV: E=McAfee;i="6600,9927,10934"; a="3108189" X-IronPort-AV: E=Sophos;i="6.04,302,1695711600"; d="scan'208";a="3108189" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Dec 2023 00:25:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10934"; a="1024856393" X-IronPort-AV: E=Sophos;i="6.04,302,1695711600"; d="scan'208";a="1024856393" Received: from ply01-vm-store.bj.intel.com ([10.238.153.201]) by fmsmga006.fm.intel.com with ESMTP; 25 Dec 2023 00:25:43 -0800 From: Ethan Zhao To: bhelgaas@google.com, baolu.lu@linux.intel.com, dwmw2@infradead.org, will@kernel.org, robin.murphy@arm.com, lukas@wunner.de Cc: linux-pci@vger.kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC PATCH v7 4/4] iommu/vt-d: don't issue device-TLB invalidate request when device is disconnected Date: Mon, 25 Dec 2023 03:25:32 -0500 Message-Id: <20231225082532.427362-5-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20231225082532.427362-1-haifeng.zhao@linux.intel.com> References: <20231225082532.427362-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786241682513160561 X-GMAIL-MSGID: 1786241682513160561 Except those aggressive hotplug cases - surpise remove a hotplug device while its safe removal is requested and handled in process by: 1. pull it out directly. 2. turn off its power. 3. bring the link down. 4. just died there that moment. etc, in a word, 'gone' or 'disconnected'. Mostly are regular normal safe removal and surprise removal unplug. these hot unplug handling process could be optimized for fix the ATS invalidation hang issue by calling pci_dev_is_disconnected() in function devtlb_invalidation_with_pasid() to check target device state to avoid sending meaningless ATS invalidation request to iommu when device is gone. (see IMPLEMENTATION NOTE in PCIe spec r6.1 section 10.3.1) For safe removal, device wouldn't be removed until the whole software handling process is done, it wouldn't trigger the hard lock up issue caused by too long ATS invalidation timeout wait. in safe removal path, device state isn't set to pci_channel_io_perm_failure in pciehp_unconfigure_device() by checking 'presence' parameter, calling pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return false there, wouldn't break the function. For surprise removal, device state is set to pci_channel_io_perm_failure in pciehp_unconfigure_device(), means device is already gone (disconnected) call pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return true to break the function not to send ATS invalidation request to the disconnected device blindly, thus avoid the further long time waiting triggers the hard lockup. safe removal & surprise removal pciehp_ist() pciehp_handle_presence_or_link_change() pciehp_disable_slot() remove_board() pciehp_unconfigure_device(presence) Tested-by: Haorong Ye Signed-off-by: Ethan Zhao --- drivers/iommu/intel/pasid.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 1c87fb1b1039..a08bdbec90eb 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -481,6 +481,9 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, if (!info || !info->ats_enabled) return; + if (pci_dev_is_disconnected(to_pci_dev(dev))) + return; + sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; pfsid = info->pfsid;