From patchwork Thu Feb 16 13:08:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 58068 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp296632wrn; Thu, 16 Feb 2023 05:21:38 -0800 (PST) X-Google-Smtp-Source: AK7set88gNgHYb7URUPrsA1RQigAic3jpRpwj/9uvj9+yTKdiReXqYurQjTpmBmjhmBMzQ4cnVi0 X-Received: by 2002:a17:906:1906:b0:885:6a2e:f941 with SMTP id a6-20020a170906190600b008856a2ef941mr5979364eje.54.1676553698805; Thu, 16 Feb 2023 05:21:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676553698; cv=none; d=google.com; s=arc-20160816; b=A06vhY3MbGer9RRKqO1uBl5b0QonszOdKVXyknsCr5pJY1wLcWgBcjvGNXuRUCrMEe S/PEubyRIJbrftbdf42X9AJdQYqBc0EonJAQ03q43vX0/FUwGdGtDzQyfih5R3sChPZg 6G/CE8imuRCfhm9lHbDs28qUcPYwvR2v28/sPwqxvT8wye6HtTN+mrIORHJvUUbkRcgm fMtz3lnl3Vk1KKwrVj8wuNpgcQlyApKwfv+NSlNPyTqn3aOrCBIdL/liMWbyAMWZnVYo SSSKY9r4VihhYVDxLsf/lqekhJQfY/x9k+3g0Tou3f10jo4yWdDKry330j3l9WFq/MeF JArg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=DLwLts8HqAX9yR8RrJo0BbUnXCEPFvO6oHZ+FVknetQ=; b=Hwe9exYujd/pmcLE60m2wydCuK6beji91UCTf6KwKvYFkp+cfqGxLT7eHDmbSdDjGA cO0MeBO9k31sDxluM8XQFQ73QuZJzVLqtj5VQkJBEK1GnMnKWLg7rEED06Ir0yvycCis cGfrWpJe8eKIrc/3hFDNsbq8eQ5+Hlyzj7XJU1E2D/7IQKf7wQzFM91w6wTR9L1JTAUa cjjpHK0W7Ys2r9/qTclv2wyHrD3AyFQfl0EvOyNUHQu69ZuZj7p6FwkZRNXFUTtU3jj5 AtenNnTCv4VDp0K7idFfhuEo9yM2+iYpdAdQWVactN6srHn6sGEZk39mu1xzQAudslNQ PuEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=T1XUAA3M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fd5-20020a1709072a0500b008b14c74c385si1783557ejc.364.2023.02.16.05.20.53; Thu, 16 Feb 2023 05:21:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=T1XUAA3M; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229935AbjBPNQq (ORCPT + 99 others); Thu, 16 Feb 2023 08:16:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229771AbjBPNQp (ORCPT ); Thu, 16 Feb 2023 08:16:45 -0500 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72BACE3AF for ; Thu, 16 Feb 2023 05:16:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676553404; x=1708089404; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sEeJE3lb+SBSL58VtACxqpXosFpusquhp6qgy5s/iGw=; b=T1XUAA3MWPFDgdtiknMKjYTjsN67QMu1fm75Ss7Fo18F1unez2T5iiYf 0Dwfu8Ql44DkonqeguWvUqWnsqh64swH4Lbqoj+6EK9oGdn4i/jSO7c0q qLlW4bW1Th5zV1dVi8IWmE4PtCur+a6E9QYbYBVj9WAXcGqJeb1u3XO9M TlFLt0o7weXbZ6Zz3GazEVRNySsxhnda+pXmkJ+LQwE4uKD2BUq0CeiXW JGrvwesUcJOPsrdY6fcFSsYBXZa2WjF2fMjP/bqwGV90HiyEdxMtM3XKV v9h96zAuXhT4AhhmHqxwFKkz5uZ5MWV+4ECLcrY2RZ3H5XnjxNb+tV/gG A==; X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="333883295" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="333883295" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2023 05:16:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="647674924" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="647674924" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga006.jf.intel.com with ESMTP; 16 Feb 2023 05:16:42 -0800 From: Lu Baolu To: Joerg Roedel Cc: Jacob Pan , Tina Zhang , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 1/4] iommu/vt-d: Fix error handling in sva enable/disable paths Date: Thu, 16 Feb 2023 21:08:13 +0800 Message-Id: <20230216130816.151824-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230216130816.151824-1-baolu.lu@linux.intel.com> References: <20230216130816.151824-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757993971116046000?= X-GMAIL-MSGID: =?utf-8?q?1757993971116046000?= Roll back all previous actions in error paths of intel_iommu_enable_sva() and intel_iommu_disable_sva(). Fixes: d5b9e4bfe0d8 ("iommu/vt-d: Report prq to io-pgfault framework") Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Link: https://lore.kernel.org/r/20230208051559.700109-1-baolu.lu@linux.intel.com --- drivers/iommu/intel/iommu.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 59df7e42fd53..994dffa1db57 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4642,8 +4642,12 @@ static int intel_iommu_enable_sva(struct device *dev) return -EINVAL; ret = iopf_queue_add_device(iommu->iopf_queue, dev); - if (!ret) - ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); + if (ret) + return ret; + + ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); + if (ret) + iopf_queue_remove_device(iommu->iopf_queue, dev); return ret; } @@ -4655,8 +4659,12 @@ static int intel_iommu_disable_sva(struct device *dev) int ret; ret = iommu_unregister_device_fault_handler(dev); - if (!ret) - ret = iopf_queue_remove_device(iommu->iopf_queue, dev); + if (ret) + return ret; + + ret = iopf_queue_remove_device(iommu->iopf_queue, dev); + if (ret) + iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev); return ret; } From patchwork Thu Feb 16 13:08:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 58065 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp296402wrn; Thu, 16 Feb 2023 05:21:06 -0800 (PST) X-Google-Smtp-Source: AK7set8H3nAL7VigZiFdhBQQ2zq3um7DqJCXSgVqJLyt8UEPGlYeFkWBIqeGpxD4cwsS3Q9iRxVz X-Received: by 2002:a17:906:a009:b0:8a4:e2aa:6cd9 with SMTP id p9-20020a170906a00900b008a4e2aa6cd9mr6313816ejy.19.1676553666323; Thu, 16 Feb 2023 05:21:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676553666; cv=none; d=google.com; s=arc-20160816; b=HjxiOAROIj+WVSTDtCDb6RJGoF7dKfZGZcLU6ZwcyrAo5idzLzk5Ot2kGMachTBBhQ cclC00Ly6+kXdlvVq4gNb+LugJWdGuVkciybkPYrdS2XnBx55mgXld1YOdbLVWcVMivM WP56z9c22Jlf0i01942YtS72FH3q1pyzInd6Nxb3UsIRbhL0Ef32LCEtgkM5CM7Mu4/u 1zuYZiTU7NSX6VIPenmh6+BPxibojoiSmpyyf8IdCmZrkciS+0sDWZZkDcnNT875aBuz SPoVJbGIGMj9MAb468OX0np/GVOmyfYyBE5oxfmIqOWXVHJvSj4UzIo5raQZblxsTL52 BLpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wTHxpqwWDAZek4yPr+E/9doSeZJWG5SrQ2XBexN6hv4=; b=ufTIpNXtXvOM1MnEiB2lpljFDrl20B1vdjqi5aIPoqFzIjiEbAHaHRrVlvcyW6JQ85 zw9HlKk4LylBVcghse0PgD1tRxPFifTeOTDQg1sWXL69AQry7ohmBeoGcDbDl7bCpoZS ektRkPq+EVRNRXvvnjaHlBSZgi2YPWXDo0wXrQH9a80wpAUulRHZrQvn0+WlSKXow7vS XmZixBeAZgisLZt5p/G6B1ACBf/n497y+9zbk5XoFy5goYvsX4v+LTxjryq5Y4Rn8ZLS uuJ9ABM5Gmtuhug3JMShfOWD48CrVKSN6hzbgpe4R5qw7u5nrBR+SYQlMf2Bz8qOfZ5a j+wQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Uxt5voOe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fp35-20020a1709069e2300b0088850d567d3si1993621ejc.115.2023.02.16.05.20.43; Thu, 16 Feb 2023 05:21:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Uxt5voOe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229999AbjBPNQw (ORCPT + 99 others); Thu, 16 Feb 2023 08:16:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230010AbjBPNQr (ORCPT ); Thu, 16 Feb 2023 08:16:47 -0500 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C8A95354B for ; Thu, 16 Feb 2023 05:16:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676553406; x=1708089406; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q4t6DQ3FZM/W5yMNCkd2+Emh3Kq12Ff5P8dXSp0yfmQ=; b=Uxt5voOevKGx+j1MWKBGB+Vzk76dMJK2ZC3cYiZhfuSGt9hxnHEGWs0h fdhd80CKsb8n/gsRqBg+S0agLCfELy5QETMKCh2woGMQbbBz+SIOe+kv3 oNCBBIClo17qFSVmL43KlqZSK1vu9lluETe7GYGE0r51YSIq52oOSspCP YR1oOGeTWyOF6mZq/n+wRP6Gv/+rCNeaqNHZ1jaj5NR2kCkp4MkvyDRDT WlsJEHTvgdxNS1bwzcJi1DacIx0+yeZ/qHH2efBoL/84KftHFgycDV16U 9Miaj6PefBNnyxL/BOBQqb2pUjfjWLVV07uzIyPfC+z1cbk4iQ9lY88Ps Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="333883307" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="333883307" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2023 05:16:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="647674936" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="647674936" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga006.jf.intel.com with ESMTP; 16 Feb 2023 05:16:44 -0800 From: Lu Baolu To: Joerg Roedel Cc: Jacob Pan , Tina Zhang , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode Date: Thu, 16 Feb 2023 21:08:14 +0800 Message-Id: <20230216130816.151824-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230216130816.151824-1-baolu.lu@linux.intel.com> References: <20230216130816.151824-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757993937299087129?= X-GMAIL-MSGID: =?utf-8?q?1757993937299087129?= From: Jacob Pan Intel IOMMU driver implements IOTLB flush queue with domain selective or PASID selective invalidations. In this case there's no need to track IOVA page range and sync IOTLBs, which may cause significant performance hit. This patch adds a check to avoid IOVA gather page and IOTLB sync for the lazy path. The performance difference on Sapphire Rapids 100Gb NIC is improved by the following (as measured by iperf send): w/o this fix~48 Gbits/s. with this fix ~54 Gbits/s Cc: Fixes: 2a2b8eaa5b25 ("iommu: Handle freelists when using deferred flushing in iommu drivers") Reviewed-by: Robin Murphy Reviewed-by: Kevin Tian Tested-by: Sanjay Kumar Signed-off-by: Sanjay Kumar Signed-off-by: Jacob Pan Link: https://lore.kernel.org/r/20230209175330.1783556-1-jacob.jun.pan@linux.intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 994dffa1db57..ce36a16efc97 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4346,7 +4346,12 @@ static size_t intel_iommu_unmap(struct iommu_domain *domain, if (dmar_domain->max_addr == iova + size) dmar_domain->max_addr = iova; - iommu_iotlb_gather_add_page(domain, gather, iova, size); + /* + * We do not use page-selective IOTLB invalidation in flush queue, + * so there is no need to track page and sync iotlb. + */ + if (!iommu_iotlb_gather_queued(gather)) + iommu_iotlb_gather_add_page(domain, gather, iova, size); return size; } From patchwork Thu Feb 16 13:08:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 58066 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp296541wrn; Thu, 16 Feb 2023 05:21:24 -0800 (PST) X-Google-Smtp-Source: AK7set8T9E4+CP0bYWyhmNu+8qnGTAYuv1qMLwrp5wll5pqAL97ynJ8PwyU2YOPJOcWU8TvEk/BS X-Received: by 2002:a05:6402:1051:b0:4ac:c690:d637 with SMTP id e17-20020a056402105100b004acc690d637mr5737256edu.31.1676553684198; Thu, 16 Feb 2023 05:21:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676553684; cv=none; d=google.com; s=arc-20160816; b=JLBh8V8aK/lprWEXq1RgAGf+3c0r+kkQVY0fhATZuosVHISBt30MmB0FkjiiLaARFc MJ2E+uPdyOe82Pe9QRyFPGt/MD6ub0Zb+/4u4azTSgdxwtbnmM3vUQKIpVheWa8Tw+KB 4A2jGDQytge3j+xWlKMCVIGb9clAKmiRHnsNEZHv+FbvrnkxF3+i6HNH+3mLQ7L+O/cu bw4W6G+kg/wsXabItPhWKpNvf7kKe2Q/k7qVv8clDNwSio9RIbRcE63Z69XWziFUgoKr 4xkzAqScOGhF/7QQnUg8+3aa0vxXrZixOrZubChlzSw6kWnT9ZSmuKZulZm92LUirJUI pj4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RxBp6w/rW1r173LKk3QX6jZNehSM8GEeSaR1wtYgX5Q=; b=Uq1TrtDIUcZBDk6BBd4XSiRBWQbvkDv9ljdZe1ULN+TgnJwnhjr92OQ5u7u96rNzUJ sx4us/ypkbRO3FVwBTOZ91/uwJ1lKUFr090B1uJH1AIEM6G/8Wt3+1F/loj2dm3a7YhH wRb0RWZeN2d9W/lk+9vrJkHl55vH7Gy4IeN5MoBf/EG5IYJj74L0jUOmvMfnvxO337OD VDXMuA6yl3yJWZeGMkWMhflOAjySHLy5n/PSYmp+RwydoudSCeESs3pird48yy1Y1I92 ce0FjkNCyqdWwwy3XvcsdZmapnWJyLeFpKrtUm6V6rRlrf+q0g/1wXS1X23ZiCgbe4uf RTcg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Fw9FVcsl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c20-20020aa7df14000000b004ad72247823si446270edy.176.2023.02.16.05.20.59; Thu, 16 Feb 2023 05:21:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Fw9FVcsl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230072AbjBPNQy (ORCPT + 99 others); Thu, 16 Feb 2023 08:16:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230116AbjBPNQv (ORCPT ); Thu, 16 Feb 2023 08:16:51 -0500 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E65D54575 for ; Thu, 16 Feb 2023 05:16:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676553408; x=1708089408; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JjSVq1WVeCrfaYihL0lX05Q/SUamFWX/MkV36WR1wlU=; b=Fw9FVcslDK49cdJS1G4dKSpMVOIHYoYykDAYWWopuDvqQzXJsDlZFCxt aVKIgiy2lZDTp+/SWUxqorwj19B+h0EG9lxTwKKCshNXKN0aPuPNlrE18 In+LmYLgb/OiaAOYWBHrgqXWz+IrjhIIqsXd6qtaSmwS+DZhPmjk+hMJ0 6cWIbhikVsR3fmeZOV2KplH1gZHBHFqCIItpelMwzLhuRiMhW3ppkSjOk pt2OT6XmxDbG2vyL8qGb8vy7/djjy98Wi3Dlrs+O5kcSruE6MlIGPbm0N R7tbKeTNGyhi5R18NSYbB2CwdL7leKinvOiMN80bUKKlGVmEKxcvfD/2K A==; X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="333883313" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="333883313" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2023 05:16:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="647674949" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="647674949" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga006.jf.intel.com with ESMTP; 16 Feb 2023 05:16:46 -0800 From: Lu Baolu To: Joerg Roedel Cc: Jacob Pan , Tina Zhang , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] iommu/vt-d: Fix PASID directory pointer coherency Date: Thu, 16 Feb 2023 21:08:15 +0800 Message-Id: <20230216130816.151824-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230216130816.151824-1-baolu.lu@linux.intel.com> References: <20230216130816.151824-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757993955972607592?= X-GMAIL-MSGID: =?utf-8?q?1757993955972607592?= From: Jacob Pan On platforms that do not support IOMMU Extended capability bit 0 Page-walk Coherency, CPU caches are not snooped when IOMMU is accessing any translation structures. IOMMU access goes only directly to memory. Intel IOMMU code was missing a flush for the PASID table directory that resulted in the unrecoverable fault as shown below. This patch adds clflush calls whenever allocating and updating a PASID table directory to ensure cache coherency. On the reverse direction, there's no need to clflush the PASID directory pointer when we deactivate a context entry in that IOMMU hardware will not see the old PASID directory pointer after we clear the context entry. PASID directory entries are also never freed once allocated. DMAR: DRHD: handling fault status reg 3 DMAR: [DMA Read NO_PASID] Request device [00:0d.2] fault addr 0x1026a4000 [fault reason 0x51] SM: Present bit in Directory Entry is clear DMAR: Dump dmar1 table entries for IOVA 0x1026a4000 DMAR: scalable mode root entry: hi 0x0000000102448001, low 0x0000000101b3e001 DMAR: context entry: hi 0x0000000000000000, low 0x0000000101b4d401 DMAR: pasid dir entry: 0x0000000101b4e001 DMAR: pasid table entry[0]: 0x0000000000000109 DMAR: pasid table entry[1]: 0x0000000000000001 DMAR: pasid table entry[2]: 0x0000000000000000 DMAR: pasid table entry[3]: 0x0000000000000000 DMAR: pasid table entry[4]: 0x0000000000000000 DMAR: pasid table entry[5]: 0x0000000000000000 DMAR: pasid table entry[6]: 0x0000000000000000 DMAR: pasid table entry[7]: 0x0000000000000000 DMAR: PTE not present at level 4 Cc: Fixes: 0bbeb01a4faf ("iommu/vt-d: Manage scalalble mode PASID tables") Reviewed-by: Kevin Tian Reported-by: Sukumar Ghorai Signed-off-by: Ashok Raj Signed-off-by: Jacob Pan Link: https://lore.kernel.org/r/20230209212843.1788125-1-jacob.jun.pan@linux.intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/pasid.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index fb3c7020028d..979f796175b1 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -128,6 +128,9 @@ int intel_pasid_alloc_table(struct device *dev) pasid_table->max_pasid = 1 << (order + PAGE_SHIFT + 3); info->pasid_table = pasid_table; + if (!ecap_coherent(info->iommu->ecap)) + clflush_cache_range(pasid_table->table, size); + return 0; } @@ -215,6 +218,10 @@ static struct pasid_entry *intel_pasid_get_entry(struct device *dev, u32 pasid) free_pgtable_page(entries); goto retry; } + if (!ecap_coherent(info->iommu->ecap)) { + clflush_cache_range(entries, VTD_PAGE_SIZE); + clflush_cache_range(&dir[dir_index].val, sizeof(*dir)); + } } return &entries[index]; From patchwork Thu Feb 16 13:08:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 58067 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp296540wrn; Thu, 16 Feb 2023 05:21:24 -0800 (PST) X-Google-Smtp-Source: AK7set8K9oNZCMmxDlfV4lbLg+C8RVHzrrm/DsaReEfJiQ5mr6iX4JV2VhiBhJ9krVqREHHqhJv8 X-Received: by 2002:a17:907:98e6:b0:88c:3cc7:37d6 with SMTP id ke6-20020a17090798e600b0088c3cc737d6mr6036123ejc.53.1676553684060; Thu, 16 Feb 2023 05:21:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676553684; cv=none; d=google.com; s=arc-20160816; b=yjUHw0Gt94rVTp1M3ko1VnTgp1a1P6ifDtXDNILwBlwMUmD6vMSpSz193bBkprN6h5 gNca7jW17/Kzbr8jHTIUx4GD1D5GfkJ1+y3XXxRBpXVHeTEe1tc5Rk71kX51DE4W0FJn 7zxzuDXCDSNIbC8gXH88VqdNVfJp/2H9hXTerEbD4a/viv103w7vr8yO5ud8l1ln46bV WpOy1WO/uSfmC2jDHx9u0AUFgBVez450A24o78XQhkdGpN174EecChC99qm5N9zaJwog 6LMFGghh15L9eJ6wWQDZNIcNUA+3rg5YiXQBCvu9hYPlKshJ5zjERfx2CYUejjNzItM/ EADQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RQgZRyW5cK0P66DjZrE7xoMb8soHPJh59VYOd0lxoZY=; b=PcW/Ak2YFUrl/sZVn2sIw9FE6+VWKoAIv4gMafr6rOKvHiun7LQ8frtT6mnrq/rHCy /wFG5GGX1dqUJH34JpkY11HmkXxdgC/NMYf/it9bKLPAvok112gqMgkMuCIBXhyhsLnG uEjm87BdTwMjScBfURx9Lr58yHOyHqsvtObIuDZg1QCaZCgFibxa4ofPsJ/+ostHAqdB +wyklBcjwwiC459ToMol53Ql0Q4mRCBKib30w1o9+d9dxSBZdPwnl5Vph/zpB7irnjC2 VqiUposZeP0P27EuLtU+rt8iGSdGJp428bcAGaiN6W5LLMVys+y03e8TSYhRHas4w2Ik dQ1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YulFT56S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fx24-20020a1709069e9800b00886de65e96csi2178166ejc.965.2023.02.16.05.20.58; Thu, 16 Feb 2023 05:21:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YulFT56S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbjBPNQ7 (ORCPT + 99 others); Thu, 16 Feb 2023 08:16:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230170AbjBPNQz (ORCPT ); Thu, 16 Feb 2023 08:16:55 -0500 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87DA9552A5 for ; Thu, 16 Feb 2023 05:16:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676553410; x=1708089410; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NCCqLc87j+n2TX4S0I5r808qPN4XxFBlCpxujEbhTfo=; b=YulFT56SjFb5xcJNjrsTMZ61tm+hAGXJ9iPgi0ZhYio+YWJwEUbuuVGA 9Uo+c907ZmoKXoyiPi5IVCEy/MSjZHcmgWLAHJJTmJtQsp8EK6q7MtUe2 2Y2sm8fQ2H2k1Qb1M3x2ysEisQWaorJ1/yPLfmtoxg9HDb8BdhUg/S6Vd GLh6Y9hlIy2urMQgdF+gUtYT4feIurXh+SKQPxmPWp/uwaOjm+BsVozPP 1LLKHOg5vQ3JdNfvKvw1uPo4OuOspiAjqwyN7d4zrvxwcl88ftTcsxu3W mAxVH1eHcGxjL734ennk1/9UIUwfGHfP/K7DW8+v8XZUSbBEIzb+0TgLI Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="333883331" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="333883331" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2023 05:16:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="647674974" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="647674974" Received: from allen-box.sh.intel.com ([10.239.159.48]) by orsmga006.jf.intel.com with ESMTP; 16 Feb 2023 05:16:48 -0800 From: Lu Baolu To: Joerg Roedel Cc: Jacob Pan , Tina Zhang , iommu@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] iommu/vt-d: Allow to use flush-queue when first level is default Date: Thu, 16 Feb 2023 21:08:16 +0800 Message-Id: <20230216130816.151824-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230216130816.151824-1-baolu.lu@linux.intel.com> References: <20230216130816.151824-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757993955911043566?= X-GMAIL-MSGID: =?utf-8?q?1757993955911043566?= From: Tina Zhang Commit 29b32839725f ("iommu/vt-d: Do not use flush-queue when caching-mode is on") forced default domains to be strict mode as long as IOMMU caching-mode is flagged. The reason for doing this is that when vIOMMU uses VT-d caching mode to synchronize shadowing page tables, the strict mode shows better performance. However, this optimization is orthogonal to the first-level page table because the Intel VT-d architecture does not define the caching mode of the first-level page table. Refer to VT-d spec, section 6.1, "When the CM field is reported as Set, any software updates to remapping structures other than first-stage mapping (including updates to not- present entries or present entries whose programming resulted in translation faults) requires explicit invalidation of the caches." Exclude the first-level page table from this optimization. Generally using first-stage translation in vIOMMU implies nested translation enabled in the physical IOMMU. In this case the first-stage page table is wholly captured by the guest. The vIOMMU only needs to transfer the cache invalidations on vIOMMU to the physical IOMMU. Forcing the default domain to strict mode will cause more frequent cache invalidations, resulting in performance degradation. In a real performance benchmark test measured by iperf receive, the performance result on Sapphire Rapids 100Gb NIC shows: w/ this fix ~51 Gbits/s, w/o this fix ~39.3 Gbits/s. Theoretically a first-stage IOMMU page table can still be shadowed in absence of the caching mode, e.g. with host write-protecting guest IOMMU page table to synchronize changed PTEs with the physical IOMMU page table. In this case the shadowing overhead is decoupled from emulating IOTLB invalidation then the overhead of the latter part is solely decided by the frequency of IOTLB invalidations. Hence allowing guest default dma domain to be lazy can also benefit the overall performance by reducing the total VM-exit numbers. Fixes: 29b32839725f ("iommu/vt-d: Do not use flush-queue when caching-mode is on") Reported-by: Sanjay Kumar Suggested-by: Sanjay Kumar Signed-off-by: Tina Zhang Reviewed-by: Kevin Tian Link: https://lore.kernel.org/r/20230214025618.2292889-1-tina.zhang@intel.com Signed-off-by: Lu Baolu --- drivers/iommu/intel/iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index ce36a16efc97..52afcdaf7c7f 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4005,7 +4005,8 @@ int __init intel_iommu_init(void) * is likely to be much lower than the overhead of synchronizing * the virtual and physical IOMMU page-tables. */ - if (cap_caching_mode(iommu->cap)) { + if (cap_caching_mode(iommu->cap) && + !first_level_by_default(IOMMU_DOMAIN_DMA)) { pr_info_once("IOMMU batching disallowed due to virtualization\n"); iommu_set_dma_strict(); }