From patchwork Wed Dec 27 02:59:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ethan Zhao X-Patchwork-Id: 183397 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp1215207dyb; Tue, 26 Dec 2023 19:01:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IEv+qsiwqnL4dyom/Q94OYT4PHWZ5KfeZZaT97J+Okt3PpRTbTZbE5ttB9HW4Cl68GX86h9 X-Received: by 2002:a17:90a:4961:b0:28b:e9e9:e0f with SMTP id c88-20020a17090a496100b0028be9e90e0fmr3678116pjh.17.1703646105136; Tue, 26 Dec 2023 19:01:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703646105; cv=none; d=google.com; s=arc-20160816; b=jWFtRXjQlNEVFYlP/JIshtJ9Cu0cRW0jDJqFOPItGy6YXzSGfsKMZETdBNr5LCDGYl 0F47FPB1g9wznDVys23DAnSJlG4Kfug+6U4MvqKWWwp7eqHX6+NYs+zI+kPv0DWekdtX JtkTGiaaRKI/LPmEqto7qSwhICnjaB9TB10F8iY5c3q9rxL7iKQLkD1aSWvrI/YnECsa wmoHQvb8Ujur3FVUwBx+57l9sm7rdBOuzfaE5byxu6QjckdoVx9fhZRWQaU24nen4Twy V8fkT83qak4lSUwp+twNNiZZjbybZ4iktk0pphtLFV2Tn9D0HNMiH9XXKgST6ml6i30H HIJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=XW0OX8z5K1UjkVyKaF1Sy1WLJYpebt+CLrJdKW4ZbsM=; fh=cQTxynQ7rGExgmbyXq8tNmd/VmE1Fzgt9QW51hABj2c=; b=kkhENfa5Bl1Mk7D4WUo3jtZo+cpWALHb4YE+lJENSGgufvTubSaYoKVHyuMJIVIG2p wi0fUEns1t3JrOAe9WztI6pn32fH2fij2OExuY7xBkvlf/DTbKTb1hb0kkR7ml0DcX0H bb+B7L/6IXPNlhm6Kw9/BOVffkRhQtzqbTYctfZJ+Ue05lDUm/wrt0pKZ035VY9dfFK5 RVRx6hqLcOWZZvwsXFzI6sFETmlyW4s/dTGianUXOOIRKJLKcqvIWNLclBfX9d7JUYAz /X62iR624D3OFAttt+5aO6khpRpRPbUdVKgU1kaaNMIVlj0txSOyp93NKThdsfoWL60/ xGSQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KQzOWUTu; spf=pass (google.com: domain of linux-kernel+bounces-11789-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-11789-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id il4-20020a17090b164400b0028c40644383si5840899pjb.68.2023.12.26.19.01.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 19:01:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-11789-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KQzOWUTu; spf=pass (google.com: domain of linux-kernel+bounces-11789-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-11789-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E6978282364 for ; Wed, 27 Dec 2023 03:01:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8433B6FC7; Wed, 27 Dec 2023 02:59:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KQzOWUTu" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D8E13C0C; Wed, 27 Dec 2023 02:59:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703645989; x=1735181989; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WG21CSSwsE/85hwPEnEufidCuVnAVIKF6sagH3cxk1U=; b=KQzOWUTuEGy9HQkXR0br008Pv+RjkzW+WXL4I/YPam1QR1Ru7PmDABCa dIZ8cStmlfZQef0Vhm14irrh47/oqXd7eGE6/mKrFVI9DNHnF8zpnFyOx 2LymV/XXhTwEmKxNtbtDkFjC/Fp90fgTvYjq/Rhkhqx/qUIbNZKHMgxH1 cN8EXtnkkQVrHyswuXQzOdIaNyEncP9L+wU6QAjTuIgad2UvW8rgH5vYQ lrL9LYD9vTxNJh+ppVQ5s1qh/XsdWJ4OlXIS/TYyJ18Io9xhTTmOd5RB8 yYYxvHFbu5rAbMQTOfJbmQ/ZUf00d2S4nNzD/NCNnl6HfD8AxOFLpgk8k A==; X-IronPort-AV: E=McAfee;i="6600,9927,10935"; a="3705805" X-IronPort-AV: E=Sophos;i="6.04,307,1695711600"; d="scan'208";a="3705805" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Dec 2023 18:59:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10935"; a="951386099" X-IronPort-AV: E=Sophos;i="6.04,307,1695711600"; d="scan'208";a="951386099" Received: from ply01-vm-store.bj.intel.com ([10.238.153.201]) by orsmga005.jf.intel.com with ESMTP; 26 Dec 2023 18:59:37 -0800 From: Ethan Zhao To: bhelgaas@google.com, baolu.lu@linux.intel.com, dwmw2@infradead.org, will@kernel.org, robin.murphy@arm.com, lukas@wunner.de Cc: linux-pci@vger.kernel.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [RFC PATCH v8 4/5] iommu/vt-d: don't issue device-TLB invalidate request when device is disconnected Date: Tue, 26 Dec 2023 21:59:22 -0500 Message-Id: <20231227025923.536148-5-haifeng.zhao@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20231227025923.536148-1-haifeng.zhao@linux.intel.com> References: <20231227025923.536148-1-haifeng.zhao@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786402418530179622 X-GMAIL-MSGID: 1786402418530179622 Except those aggressive hotplug cases - surprise remove a hotplug device while its safe removal is requested and handled in process by: 1. pull it out directly. 2. turn off its power. 3. bring the link down. 4. just died there that moment. etc, in a word, 'gone' or 'disconnected'. Mostly are regular normal safe removal and surprise removal unplug. these hot unplug handling process could be optimized for fix the ATS invalidation hang issue by calling pci_dev_is_disconnected() in function devtlb_invalidation_with_pasid() to check target device state to avoid sending meaningless ATS invalidation request to iommu when device is gone. (see IMPLEMENTATION NOTE in PCIe spec r6.1 section 10.3.1) For safe removal, device wouldn't be removed untill the whole software handling process is done, it wouldn't trigger the hard lock up issue caused by too long ATS invalidation timeout wait. in safe removal path, device state isn't set to pci_channel_io_perm_failure in pciehp_unconfigure_device() by checking 'presence' parameter, calling pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return false there, wouldn't break the function. For surprise removal, device state is set to pci_channel_io_perm_failure in pciehp_unconfigure_device(), means device is already gone (disconnected) call pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return true to break the function not to send ATS invalidation request to the disconnected device blindly, thus avoid the further long time waiting triggers the hard lockup. safe removal & surprise removal pciehp_ist() pciehp_handle_presence_or_link_change() pciehp_disable_slot() remove_board() pciehp_unconfigure_device(presence) Tested-by: Haorong Ye Signed-off-by: Ethan Zhao --- drivers/iommu/intel/pasid.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 1c87fb1b1039..a08bdbec90eb 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -481,6 +481,9 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, if (!info || !info->ats_enabled) return; + if (pci_dev_is_disconnected(to_pci_dev(dev))) + return; + sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; pfsid = info->pfsid;