From patchwork Thu Dec 28 15:06:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183755 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2048197dyb; Thu, 28 Dec 2023 07:07:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IG4d8S343EngHCSw9lb8FxioJa2gvjcw5F7erE+ulpbFqEjgXXOmIKtavGDOar07gZYv191 X-Received: by 2002:ad4:522a:0:b0:67f:fc7b:64b0 with SMTP id r10-20020ad4522a000000b0067ffc7b64b0mr8167326qvq.88.1703776043233; Thu, 28 Dec 2023 07:07:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776043; cv=none; d=google.com; s=arc-20160816; b=PJVAIARd7mG53N4U70aOlsmL3sRg/aTaCJHAq1hAc9OopEKhO1yD//IIlbcpWPa4Mj H+fR9RNtDHpbPuAzWMu2yjhIyq6rxnw34eacxCSlyAfW0vB3riyvwpp9Da7LlQQPGrzA i3T7Yo7W3OQwWKfhYFQ9u/R9YL/yshmqjpSQgD5Pz0OYQ++hR9p3wHuub4Fw+EWD4qlC jn9WeQ8dAGs8Q+atCQ6MkWfAj1PujKbDiOUqCsqH1NU9lo/ZSbt7Hq1ZWgbBNShWarhB oJIrIJUDCIun7MF6Fn/lWT+qGFqL6wTh6jJ/+mmc2eRaa+B2O34FlVwyYSbS/j4tdbZy nYag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cmmE61uWQDmtK6kaEvaJmx5NWQkq2czzSV7UX/pgsTU=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=uBPjhBarAcJx8NhKg0UZ8rRBv5zWQMlan2Zd/RKl2zr/+Br05V5O1rD2b+jBRzuoHC spHKTZHUz/SE6Y7Uq858ycd/E5XdO4WZin1pcKWOmOxdhXtfd5rUhKQtlRDIz953gRay uN0bsJKP5HC/kLbXvwoQ+DLEIjt7WSQVgZUCXVP15WMuRRC5SDftwAHjuG72bE1WoRMh 0lgxSevZ5YUmNERNGTzPiMy6BedrqVZ0TvIE8mHBJqFz21Kw5/CJpuS8jpiXaQ/P4Vy+ atEZyPR99YbrAXPdyToczMgBdg6IxSdevYP0t8ONqARNwydDCTf6hYqrVZEJUvH9wu5t XAcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=iVVLOZcD; spf=pass (google.com: domain of linux-kernel+bounces-12727-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12727-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id i8-20020a0cf108000000b0067fa0a08b63si13201882qvl.374.2023.12.28.07.07.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:07:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12727-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=iVVLOZcD; spf=pass (google.com: domain of linux-kernel+bounces-12727-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12727-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0C4951C222B6 for ; Thu, 28 Dec 2023 15:07:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 377E0101EB; Thu, 28 Dec 2023 15:06:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iVVLOZcD" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1F7EF4E4; Thu, 28 Dec 2023 15:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703775994; x=1735311994; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y9zEpidkk+p5pkry7bIuq4ZUGve0T4w0aUbFgMriFaU=; b=iVVLOZcDa5jc0TGngTkEHztBACciRzM8yuZmEwU72U12LUwNFH/1yhcM 8lWz85CnvBB6yqMniIWHz7E1JaLjYByhktSroe8Qa3H0OprNmw0yjENLC NqcNloo3SkdwW3kXH0kiWtfDKbCCucS98bxjgNf89YDMpgB+Y7L5Zi2Rm UZ5IItDVdhCOp1hbtw8BKztWdjYGaIAS4++SPYKk9DFjcOcErawwiTfTJ flkRAj30eQOF5JT4fAbzA12wkXBLe+Af0iYG32jPF/7aYodb9KG/kX/8j 0Qn69ZS1+iBPs0ZK54pttTdAwtnBwEg5PAgQ1DpEgqa86g+X9efWMUq4+ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701900" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701900" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155158" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155158" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:32 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 01/10] iommu: Add cache_invalidate_user op Date: Thu, 28 Dec 2023 07:06:20 -0800 Message-Id: <20231228150629.13149-2-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538668371742811 X-GMAIL-MSGID: 1786538668371742811 From: Lu Baolu The updates of the PTEs in the nested page table will be propagated to the hardware caches. Add a new domain op cache_invalidate_user for the userspace to flush the hardware caches for a nested domain through iommufd. No wrapper for it, as it's only supposed to be used by iommufd. Then, pass in invalidation requests in form of a user data array conatining a number of invalidation data entries. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- include/linux/iommu.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 6291aa7b079b..93c0d12dd047 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -284,6 +284,23 @@ struct iommu_user_data { size_t len; }; +/** + * struct iommu_user_data_array - iommu driver specific user space data array + * @type: The data type of all the entries in the user buffer array + * @uptr: Pointer to the user buffer array + * @entry_len: The fixed-width length of an entry in the array, in bytes + * @entry_num: The number of total entries in the array + * + * The user buffer includes an array of requests with format defined in + * include/uapi/linux/iommufd.h + */ +struct iommu_user_data_array { + unsigned int type; + void __user *uptr; + size_t entry_len; + u32 entry_num; +}; + /** * __iommu_copy_struct_from_user - Copy iommu driver specific user space data * @dst_data: Pointer to an iommu driver specific user data that is defined in @@ -440,6 +457,13 @@ struct iommu_ops { * @iotlb_sync_map: Sync mappings created recently using @map to the hardware * @iotlb_sync: Flush all queued ranges from the hardware TLBs and empty flush * queue + * @cache_invalidate_user: Flush hardware cache for user space IO page table. + * The @domain must be IOMMU_DOMAIN_NESTED. The @array + * passes in the cache invalidation requests, in form + * of a driver data structure. The driver must update + * array->entry_num to report the number of handled + * invalidation requests. The driver data structure + * must be defined in include/uapi/linux/iommufd.h * @iova_to_phys: translate iova to physical address * @enforce_cache_coherency: Prevent any kind of DMA from bypassing IOMMU_CACHE, * including no-snoop TLPs on PCIe or other platform @@ -465,6 +489,8 @@ struct iommu_domain_ops { size_t size); void (*iotlb_sync)(struct iommu_domain *domain, struct iommu_iotlb_gather *iotlb_gather); + int (*cache_invalidate_user)(struct iommu_domain *domain, + struct iommu_user_data_array *array); phys_addr_t (*iova_to_phys)(struct iommu_domain *domain, dma_addr_t iova); From patchwork Thu Dec 28 15:06:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183756 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2048549dyb; Thu, 28 Dec 2023 07:07:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IH1E88pMjVlnpwADiFox8WZKvILzXBC4aKxODPIn1a8KFc/JOrevUO/OJtBEE4Pcd/X6MmO X-Received: by 2002:ac8:5894:0:b0:427:7bb7:ed42 with SMTP id t20-20020ac85894000000b004277bb7ed42mr12398877qta.105.1703776067664; Thu, 28 Dec 2023 07:07:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776067; cv=none; d=google.com; s=arc-20160816; b=f+8UEcmPMBp8vaCZ074emCpgpU/NlCHA5ViX2Df+RmWjc9UAZw5+5o12W0lvLXHBB7 7zyVcZq2fxSpx5th/JAHg0Ig4uz6CuMyuwM2m1Pjkb/C80pJvAIjgHTBMrUumBMXKnZe 3J8HTpzAuMxl1QU1H/mR0m/UHQPesya2POJSNkhc829z+pPSMobUiCNqclXA2wPLP6yb xOaLtbsO9QyErko8gj8nD9iHnJ8pkDrAul30MGdaEkzgC3GXxlL/5KY/6tLfG3rlazdA bA+vAbm5R/4Yn4cqvPVv6QUpSe8HD86HvHQftX3IXMjYinaSphMBk5GT1kGjKlV+gIMM htHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=wAwlr7sQ/I48JWhbjP2tWzBDi/kfFU0Ofrr6FCUbrp4=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=ty5JpHuhhB+/iVh7XVdIdZo04TUVxKeKbPO196UNeARJR5E0egjDah2qfqZCPr2ZNG LKSXviifQsmWuxQxCR+YJ8UClAJ2tPyhoUm8sAnYblKEk4H7e+bv0Kk62ar4FBshi718 u26ybzxkvJQgXu0oLH2xVDRUxHNXE0SotC2bfCCSEyx9Dc179Yc1p4+7/bDAWMdw/mul /rEHxwSzk6hwwOx9Reyy/YV7ojTVvv1cY6EYrbBe7A936pTMCdhVznBZe7TS1nF4b7vl b1PCkeIBuvexu8nnkHXP0jHw8xBbAPFeRckKzIlaZ2S/+teyhTCjUj/8RWYGRTIyfVa7 oAnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="P/pTf+/m"; spf=pass (google.com: domain of linux-kernel+bounces-12728-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12728-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id t20-20020a0cde14000000b0067f0802b3c1si16297702qvk.603.2023.12.28.07.07.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:07:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12728-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="P/pTf+/m"; spf=pass (google.com: domain of linux-kernel+bounces-12728-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12728-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 57D081C21D2F for ; Thu, 28 Dec 2023 15:07:47 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BA1DA10960; Thu, 28 Dec 2023 15:06:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="P/pTf+/m" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50C85D517; Thu, 28 Dec 2023 15:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703775996; x=1735311996; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cNouUsps1r+dmbqyfgc6rMPqSKe8Zay6ScVLkagmhB4=; b=P/pTf+/mlc0jfcwJepOnpX7W71ZBgUT78ewwpJyEatiyXOysLmfczC08 Uf+NlAB5yKizvDoAmKfHNuEph0I86tvMlLowFNNzbKGvPrGQYcbcmyHEB 5tg/AEsOnqbs1af2XaHNyj+ZWWL9FcLfJtQWX6gTLg+9BC41H7QHB8xS5 rMrdDFXUdXCuVM1m2tKIGxra2VoSCYioeq0EIOsn+4HZFfII9NSwb/aa2 M+luaXjcKk9/VUtGtgkEzYjGMgs/7Wv42J+8AFO4FLy2lA9WBX4K8I7Ka 7d6krVe5joZ7lp4l3ZRITiQICPh1FlJz5d39cn4Y2Yi2hyczhawhED5PB w==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701912" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701912" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155163" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155163" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:34 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 02/10] iommufd: Add IOMMU_HWPT_INVALIDATE Date: Thu, 28 Dec 2023 07:06:21 -0800 Message-Id: <20231228150629.13149-3-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538693975531735 X-GMAIL-MSGID: 1786538693975531735 In nested translation, the stage-1 page table is user-managed but cached by the IOMMU hardware, so an update on present page table entries in the stage-1 page table should be followed with a cache invalidation. Add an IOMMU_HWPT_INVALIDATE ioctl to support such a cache invalidation. It takes hwpt_id to specify the iommu_domain, and a multi-entry array to support multiple invalidation data in one ioctl. enum iommu_hwpt_invalidate_data_type is defined to tag the data type of the entries in the multi-entry array. Reviewed-by: Kevin Tian Co-developed-by: Nicolin Chen Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- drivers/iommu/iommufd/hw_pagetable.c | 41 +++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 10 ++++++ drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 43 +++++++++++++++++++++++++ 4 files changed, 97 insertions(+) diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index cbb5df0a6c32..4e8711f19f72 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -371,3 +371,44 @@ int iommufd_hwpt_get_dirty_bitmap(struct iommufd_ucmd *ucmd) iommufd_put_object(ucmd->ictx, &hwpt_paging->common.obj); return rc; } + +int iommufd_hwpt_invalidate(struct iommufd_ucmd *ucmd) +{ + struct iommu_hwpt_invalidate *cmd = ucmd->cmd; + struct iommu_user_data_array data_array = { + .type = cmd->data_type, + .uptr = u64_to_user_ptr(cmd->data_uptr), + .entry_len = cmd->entry_len, + .entry_num = cmd->entry_num, + }; + struct iommufd_hw_pagetable *hwpt; + u32 done_num = 0; + int rc; + + if (cmd->__reserved) { + rc = -EOPNOTSUPP; + goto out; + } + + if (cmd->entry_num && (!cmd->data_uptr || !cmd->entry_len)) { + rc = -EINVAL; + goto out; + } + + hwpt = iommufd_get_hwpt_nested(ucmd, cmd->hwpt_id); + if (IS_ERR(hwpt)) { + rc = PTR_ERR(hwpt); + goto out; + } + + rc = hwpt->domain->ops->cache_invalidate_user(hwpt->domain, + &data_array); + done_num = data_array.entry_num; + + iommufd_put_object(ucmd->ictx, &hwpt->obj); +out: + cmd->entry_num = done_num; + if (iommufd_ucmd_respond(ucmd, sizeof(*cmd))) + return -EFAULT; + return rc; +} diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index abae041e256f..991f864d1f9b 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -328,6 +328,15 @@ iommufd_get_hwpt_paging(struct iommufd_ucmd *ucmd, u32 id) IOMMUFD_OBJ_HWPT_PAGING), struct iommufd_hwpt_paging, common.obj); } + +static inline struct iommufd_hw_pagetable * +iommufd_get_hwpt_nested(struct iommufd_ucmd *ucmd, u32 id) +{ + return container_of(iommufd_get_object(ucmd->ictx, id, + IOMMUFD_OBJ_HWPT_NESTED), + struct iommufd_hw_pagetable, obj); +} + int iommufd_hwpt_set_dirty_tracking(struct iommufd_ucmd *ucmd); int iommufd_hwpt_get_dirty_bitmap(struct iommufd_ucmd *ucmd); @@ -345,6 +354,7 @@ void iommufd_hwpt_paging_abort(struct iommufd_object *obj); void iommufd_hwpt_nested_destroy(struct iommufd_object *obj); void iommufd_hwpt_nested_abort(struct iommufd_object *obj); int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd); +int iommufd_hwpt_invalidate(struct iommufd_ucmd *ucmd); static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx, struct iommufd_hw_pagetable *hwpt) diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index c9091e46d208..39b32932c61e 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -322,6 +322,7 @@ union ucmd_buffer { struct iommu_hw_info info; struct iommu_hwpt_alloc hwpt; struct iommu_hwpt_get_dirty_bitmap get_dirty_bitmap; + struct iommu_hwpt_invalidate cache; struct iommu_hwpt_set_dirty_tracking set_dirty_tracking; struct iommu_ioas_alloc alloc; struct iommu_ioas_allow_iovas allow_iovas; @@ -360,6 +361,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { __reserved), IOCTL_OP(IOMMU_HWPT_GET_DIRTY_BITMAP, iommufd_hwpt_get_dirty_bitmap, struct iommu_hwpt_get_dirty_bitmap, data), + IOCTL_OP(IOMMU_HWPT_INVALIDATE, iommufd_hwpt_invalidate, + struct iommu_hwpt_invalidate, __reserved), IOCTL_OP(IOMMU_HWPT_SET_DIRTY_TRACKING, iommufd_hwpt_set_dirty_tracking, struct iommu_hwpt_set_dirty_tracking, __reserved), IOCTL_OP(IOMMU_IOAS_ALLOC, iommufd_ioas_alloc_ioctl, diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 0b2bc6252e2c..824560c50ec6 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -49,6 +49,7 @@ enum { IOMMUFD_CMD_GET_HW_INFO, IOMMUFD_CMD_HWPT_SET_DIRTY_TRACKING, IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP, + IOMMUFD_CMD_HWPT_INVALIDATE, }; /** @@ -613,4 +614,46 @@ struct iommu_hwpt_get_dirty_bitmap { #define IOMMU_HWPT_GET_DIRTY_BITMAP _IO(IOMMUFD_TYPE, \ IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP) +/** + * enum iommu_hwpt_invalidate_data_type - IOMMU HWPT Cache Invalidation + * Data Type + * @IOMMU_HWPT_INVALIDATE_DATA_VTD_S1: Invalidation data for VTD_S1 + */ +enum iommu_hwpt_invalidate_data_type { + IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, +}; + +/** + * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) + * @size: sizeof(struct iommu_hwpt_invalidate) + * @hwpt_id: ID of a nested HWPT for cache invalidation + * @data_uptr: User pointer to an array of driver-specific cache invalidation + * data. + * @data_type: One of enum iommu_hwpt_invalidate_data_type, defining the data + * type of all the entries in the invalidation request array. It + * should be a type supported by the hwpt pointed by @hwpt_id. + * @entry_len: Length (in bytes) of a request entry in the request array + * @entry_num: Input the number of cache invalidation requests in the array. + * Output the number of requests successfully handled by kernel. + * @__reserved: Must be 0. + * + * Invalidate the iommu cache for user-managed page table. Modifications on a + * user-managed page table should be followed by this operation to sync cache. + * Each ioctl can support one or more cache invalidation requests in the array + * that has a total size of @entry_len * @entry_num. + * + * An empty invalidation request array by setting @entry_num==0 is allowed, and + * @entry_len and @data_uptr would be ignored in this case. This can be used to + * check if the given @data_type is supported or not by kernel. + */ +struct iommu_hwpt_invalidate { + __u32 size; + __u32 hwpt_id; + __aligned_u64 data_uptr; + __u32 data_type; + __u32 entry_len; + __u32 entry_num; + __u32 __reserved; +}; +#define IOMMU_HWPT_INVALIDATE _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_INVALIDATE) #endif From patchwork Thu Dec 28 15:06:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183757 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2048794dyb; Thu, 28 Dec 2023 07:08:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IFp/hS4W3Jlu/oVWR07cYbV/Mg4An1I64sl5steKurvHKnRkeV8Qbk24lzjmJhrR6Gcfs9U X-Received: by 2002:a05:651c:1684:b0:2cc:811f:f9ae with SMTP id bd4-20020a05651c168400b002cc811ff9aemr3812031ljb.47.1703776085795; Thu, 28 Dec 2023 07:08:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776085; cv=none; d=google.com; s=arc-20160816; b=aoGlcnhoJ+SCwhwaxRXdeh8QhsNA0zYUXbAX3uYPNqzJ2BgVPP0nZxwkcKQYn2X4/E Pi5HWRaZhAGZnh2qWTOMRAwoORimpYRyC5SEcanP3yTxyPW1Sg4RmXa+L1f9y6yFJyme rFQxmdtIjY1aFTWL/LbiFqB4qdUh5PM+SA97Cl6TDQVjvSIOtBIMxwMbTdkMerNkrs8S vhYJDFOzAKWYXolHnxW8MKP2i+saDu4dDT2K4VY9D0HhZjnd1RjOHT+/lB8sSWqh58xt Ryxpi0176grgdWkJM8gSHvCluOgOhRYD1pjYMxXTLJFI/AakLzllYy28W2S7guptWX70 edpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=euG6NwFyvGpQyx4bY1dT20hCi7fPaxrH9e/fkW+iluU=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=Gz8bVlEWssc5I33ATzdXkys8ys9lI4ZcG+W3uLpcFwbeU8x6t+pudPCSjLTT/wfYZ8 TkHBwOUvA5zHsHCKpAWzk/E/W0ysbs4yqt+lvvVw0WOcPgFSfGtpc3Vedoi+DIuvS9P/ 2coxbcef09Fu7r16fWj7sIBNr1C2vxn0z/2tjrRz5n2ZxG/IlWcefCRvyXMDN6tMknFi eLaLWzq7YW+8qnISRy941L4nhS3/UfNJ9JOnEQp01HZcnrOfNmilxYQrvmWj7cFLtD6X x87lN55sYuGypRfk0RAWX1cS+YH5Qq9NBpwmd4UT6d9wdGK37kc1cFC6cxx9Y6xjzaK8 ItBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Iuo4r4gd; spf=pass (google.com: domain of linux-kernel+bounces-12729-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12729-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id j23-20020a508a97000000b00553a472e61esi7333062edj.404.2023.12.28.07.08.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:08:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12729-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Iuo4r4gd; spf=pass (google.com: domain of linux-kernel+bounces-12729-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12729-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 6483F1F21DDB for ; Thu, 28 Dec 2023 15:08:05 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C6E3810A05; Thu, 28 Dec 2023 15:06:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Iuo4r4gd" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98885D2FC; Thu, 28 Dec 2023 15:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703775997; x=1735311997; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a0MtJfDdMrZSQXPjcVw/ypmtgFbBgW6CUZyjsKPTKtk=; b=Iuo4r4gd9snYlQG0bx4a9zV793nlDdGKWo2AAUq4Z0yP8BVQ0jBct2+r yTqDbXe68URphdnsR0S1cFGBScMpxwdajAjvFtZS/CmhIN7UtJkshflaJ lsYvVU5Qt9987EeLxYKlm6o4gVl2c014bAKUVyL2n8fQKMpw8+AtlrLvh rnjCPcRvDL9iG7gk/iZ5IczQKiRR0eEmu0UeXcn42/ButjR7RNTBV/dpb dOLoVeqQIHdV+kUhHTK6QS/0OiDpllx1AOuebwc+84GWjvbkGFWtVuWb6 GQuLt681Cg2TCjgnfuXicGn9yUVFv5pdyIE08A8bndl9+smzgpnFdQ1Eh A==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701924" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701924" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155167" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155167" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:35 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 03/10] iommu: Add iommu_copy_struct_from_user_array helper Date: Thu, 28 Dec 2023 07:06:22 -0800 Message-Id: <20231228150629.13149-4-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538713027665764 X-GMAIL-MSGID: 1786538713027665764 From: Nicolin Chen Wrap up the data pointer/num sanity and __iommu_copy_struct_from_user call for iommu drivers to copy driver specific data at a specific location in the struct iommu_user_data_array, and iommu_respond_struct_to_user_array() to copy response to a specific location in the struct iommu_user_data_array. And expect it to be used in cache_invalidate_user ops for example. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- include/linux/iommu.h | 74 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 93c0d12dd047..c3434c9eaa6d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -341,6 +341,80 @@ static inline int __iommu_copy_struct_from_user( sizeof(*kdst), \ offsetofend(typeof(*kdst), min_last)) +/** + * __iommu_copy_struct_from_user_array - Copy iommu driver specific user space + * data from an iommu_user_data_array + * @dst_data: Pointer to an iommu driver specific user data that is defined in + * include/uapi/linux/iommufd.h + * @src_array: Pointer to a struct iommu_user_data_array for a user space array + * @data_type: The data type of the @dst_data. Must match with @src_array.type + * @index: Index to the location in the array to copy user data from + * @data_len: Length of current user data structure, i.e. sizeof(struct _dst) + * @min_len: Initial length of user data structure for backward compatibility. + * This should be offsetofend using the last member in the user data + * struct that was initially added to include/uapi/linux/iommufd.h + */ +static inline int +__iommu_copy_struct_from_user_array(void *dst_data, + const struct iommu_user_data_array *src_array, + unsigned int data_type, unsigned int index, + size_t data_len, size_t min_len) +{ + struct iommu_user_data src_data; + + if (WARN_ON(!src_array || index >= src_array->entry_num)) + return -EINVAL; + if (!src_array->entry_num) + return -EINVAL; + src_data.uptr = src_array->uptr + src_array->entry_len * index; + src_data.len = src_array->entry_len; + src_data.type = src_array->type; + + return __iommu_copy_struct_from_user(dst_data, &src_data, data_type, + data_len, min_len); +} + +/** + * iommu_copy_struct_from_user_array - Copy iommu driver specific user space + * data from an iommu_user_data_array + * @kdst: Pointer to an iommu driver specific user data that is defined in + * include/uapi/linux/iommufd.h + * @user_array: Pointer to a struct iommu_user_data_array for a user space + * array + * @data_type: The data type of the @kdst. Must match with @user_array->type + * @index: Index to the location in the array to copy user data from + * @min_last: The last memember of the data structure @kdst points in the + * initial version. + * Return 0 for success, otherwise -error. + */ +#define iommu_copy_struct_from_user_array(kdst, user_array, data_type, \ + index, min_last) \ + __iommu_copy_struct_from_user_array(kdst, user_array, data_type, \ + index, sizeof(*kdst), \ + offsetofend(typeof(*kdst), \ + min_last)) + +/** + * iommu_respond_struct_to_user_array - Copy the response in @ksrc back to + * a specific entry of user array + * @user_array: Pointer to a struct iommu_user_data_array for a user space + * array + * @index: Index to the location in the array to copy response + * @ksrc: Pointer to kernel structure + * @klen: Length of @ksrc struct + * + * This only copies response of one entry (@index) in @user_array. + */ +static inline int +iommu_respond_struct_to_user_array(const struct iommu_user_data_array *array, + unsigned int index, void *ksrc, size_t klen) +{ + if (copy_to_user(array->uptr + array->entry_len * index, + ksrc, min_t(size_t, array->entry_len, klen))) + return -EFAULT; + return 0; +} + /** * struct iommu_ops - iommu ops and capabilities * @capable: check capability From patchwork Thu Dec 28 15:06:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183758 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2048934dyb; Thu, 28 Dec 2023 07:08:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IGQqUSKxqFoUMK909GgCUGtY6gkqvjgC93Sx6SbI3CsSo6XyDxxiyQQBSEmzsvjIqXoTeG0 X-Received: by 2002:a17:907:36c4:b0:a27:6615:1654 with SMTP id bj4-20020a17090736c400b00a2766151654mr974535ejc.118.1703776096750; Thu, 28 Dec 2023 07:08:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776096; cv=none; d=google.com; s=arc-20160816; b=wTAUfyzSuyB9i51aE1QICmDdhui0JCVc6My3R2j8mHzRE97uhYWYQE789YYMPOdkAN AcsysfrkXzn3dYIJPKgQ+N300fuLaPTFPHRweQufXoj9MSNAMiR/fwZm/NAvpOCpKu2N fIn6ypU38HkxR9w4TsNRb9BYHk20jG2dhrwhK6azcD8OYdgpb4YeWlK6/rXz7iKUvp8J jphWSSaKpkTguuDWHPwie+8HrzWzI579i1P2BGoKnVLYZwNiBOmY6MREtAhn/PHaekGo ANXdvXlGz2lJ7+kP1eRV4pW4qHsTuXfMdrlrEDSyB1fhYSKY7JiCXtm85bF5F8OpMKv4 s8aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=yNPeBnelu5/Jdu09XB17bQVfbxsyBYVzS5LQqInjiLI=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=qu9tVGI/tygFlvh/030KhH3dSWGY0LbyENg39OIv9oJ1LlvRcn1Y+JZQajGVG+sccO 333Pi5F/oRux8G+v8lSY5LB1D1+lskPhOOCpxZRJhwozDj4k2YDIehTM2JwoOUWy72Et GfTf1ow51aQ+YjNQfKt4YOkDIMTlX0vXEDX5L8poUEUpwmxCx0R7d1WGTctsHUmK3Hvm adVTgNcGz300mVaLwWvNMNC/DasxdpUbX7gjz+Xdh0JIHBbnLke+uOOY6lQqDnISsWnJ S1073hkFjLB1fZlyAmCD4akkf8pxoOs/PdGSyMBoyDKHgtpUnMCSEw89baAD7+dzhvY0 b7GQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="A/nwh0MY"; spf=pass (google.com: domain of linux-kernel+bounces-12730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12730-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id bk3-20020a170906b0c300b00a277bdd5af9si100098ejb.672.2023.12.28.07.08.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:08:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="A/nwh0MY"; spf=pass (google.com: domain of linux-kernel+bounces-12730-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12730-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 5FF351F242E3 for ; Thu, 28 Dec 2023 15:08:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4D3A810A25; Thu, 28 Dec 2023 15:06:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="A/nwh0MY" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2725101DD; Thu, 28 Dec 2023 15:06:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703775999; x=1735311999; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YaKsYJmpoP1J2+af2wMppPzhvG3pSWEOhFLhzXoh4BU=; b=A/nwh0MYqkfjoz4FsREjIMRaBSxBLIGf/N0IQ/9ISszL2PnJI9n3J2rS TG9LfMhL3NfM+60QumyKhwy7u5zX+oZO/0D4TbhPk306IhaNvh2ZzcH3Z lrYdjZ8noeULSNudi+FEPiMeUMerlFFe+VJcqp1BrsCe1Ub1Qic4IwUCZ t6t45yEr6karyGtl6o8KvrbQHbphQ8BK58DDYmTTr1PwKPrr0RwxRuFXu Sg3RSU26OX2j1HMOZlewYsTHxjnG/j/3MDepD0AiZKLb4Tgh6SvatiPXK R679Wh6mTdREMnW0Q2NEP+PygLOii9lcOsbj6d9R9dk+OijjnsM20yFJ/ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701938" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701938" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155170" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155170" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:37 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 04/10] iommufd/selftest: Add mock_domain_cache_invalidate_user support Date: Thu, 28 Dec 2023 07:06:23 -0800 Message-Id: <20231228150629.13149-5-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538724367788547 X-GMAIL-MSGID: 1786538724367788547 From: Nicolin Chen Add mock_domain_cache_invalidate_user() data structure to support user space selftest program to cover user cache invalidation pathway. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- drivers/iommu/iommufd/iommufd_test.h | 34 ++++++++++++++++ drivers/iommu/iommufd/selftest.c | 60 ++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index 7910fbe1962d..2eef5afde711 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -148,4 +148,38 @@ struct iommu_hwpt_selftest { __u32 iotlb; }; +/* Should not be equal to any defined value in enum iommu_hwpt_invalidate_data_type */ +#define IOMMU_HWPT_INVALIDATE_DATA_SELFTEST 0xdeadbeef +#define IOMMU_HWPT_INVALIDATE_DATA_SELFTEST_INVALID 0xdadbeef + +/** + * enum iommu_hwpt_invalidate_selftest_error - Hardware error of invalidation + * @IOMMU_TEST_INVALIDATE_FAKE_ERROR: Fake hw error per test program's request + */ +enum iommu_hwpt_invalidate_selftest_error { + IOMMU_TEST_INVALIDATE_FAKE_ERROR = (1 << 0) +}; + +/** + * struct iommu_hwpt_invalidate_selftest - Invalidation data for Mock driver + * (IOMMU_HWPT_INVALIDATE_DATA_SELFTEST) + * @flags: Invalidate flags + * @iotlb_id: Invalidate iotlb entry index + * @hw_error: One of enum iommu_hwpt_invalidate_selftest_error + * @__reserved: Must be 0 + * + * If IOMMU_TEST_INVALIDATE_ALL is set in @flags, @iotlb_id will be ignored + * @hw_error meaningful only if the request is processed successfully. + * If IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR is set in @flags, report a + * hw error back, cache would not be invalidated in this case. + */ +struct iommu_hwpt_invalidate_selftest { +#define IOMMU_TEST_INVALIDATE_FLAG_ALL (1 << 0) +#define IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR (1 << 1) + __u32 flags; + __u32 iotlb_id; + __u32 hw_error; + __u32 __reserved; +}; + #endif diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index 022ef8f55088..ebc6c15abf67 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -473,9 +473,69 @@ static void mock_domain_free_nested(struct iommu_domain *domain) kfree(mock_nested); } +static int +mock_domain_cache_invalidate_user(struct iommu_domain *domain, + struct iommu_user_data_array *array) +{ + struct mock_iommu_domain_nested *mock_nested = + container_of(domain, struct mock_iommu_domain_nested, domain); + u32 hw_error = 0, processed = 0; + struct iommu_hwpt_invalidate_selftest inv; + int i = 0, j; + int rc = 0; + + if (array->type != IOMMU_HWPT_INVALIDATE_DATA_SELFTEST) { + rc = -EINVAL; + goto out; + } + + for ( ; i < array->entry_num; i++) { + rc = iommu_copy_struct_from_user_array(&inv, array, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + i, __reserved); + if (rc) + break; + + if ((inv.flags & ~(IOMMU_TEST_INVALIDATE_FLAG_ALL | + IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR)) || + inv.__reserved) { + rc = -EOPNOTSUPP; + break; + } + + if (inv.iotlb_id > MOCK_NESTED_DOMAIN_IOTLB_ID_MAX) { + rc = -EINVAL; + break; + } + + if (inv.flags & IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR) { + hw_error = IOMMU_TEST_INVALIDATE_FAKE_ERROR; + } else if (inv.flags & IOMMU_TEST_INVALIDATE_FLAG_ALL) { + /* Invalidate all mock iotlb entries and ignore iotlb_id */ + for (j = 0; j < MOCK_NESTED_DOMAIN_IOTLB_NUM; j++) + mock_nested->iotlb[j] = 0; + } else { + mock_nested->iotlb[inv.iotlb_id] = 0; + } + + inv.hw_error = hw_error; + rc = iommu_respond_struct_to_user_array(array, i, (void *)&inv, + sizeof(inv)); + if (rc) + break; + + processed++; + } + +out: + array->entry_num = processed; + return rc; +} + static struct iommu_domain_ops domain_nested_ops = { .free = mock_domain_free_nested, .attach_dev = mock_domain_nop_attach, + .cache_invalidate_user = mock_domain_cache_invalidate_user, }; static inline struct iommufd_hw_pagetable * From patchwork Thu Dec 28 15:06:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183761 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2049947dyb; Thu, 28 Dec 2023 07:09:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IF5e6MP4+gM0nWVaVd0SCBrnmf0sQd1P3r0vt2V2p3Ogk4P4/oJvPBqBK7mCXXpfQkDUjLO X-Received: by 2002:a05:6a20:5487:b0:196:3829:5d95 with SMTP id i7-20020a056a20548700b0019638295d95mr1491241pzk.125.1703776183258; Thu, 28 Dec 2023 07:09:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776183; cv=none; d=google.com; s=arc-20160816; b=UkCgIc5d2lPwaTaQCBL3jULxIjbjkPBlDxdVJTHeGTl6MufICNpmlZHusIwHXhWdti 8BU2XMaicjIJWbFynIn9pf/IWfrgCwaLml0+C4QN5BZipXxnUPQNUQq7i6SzWnUtNDYg HePOtZuZJm4VN82PLcXxZCNxfzJw2wsmsi/25CHpLzeiCe45Fe+sO/nqJ9ALcZS+O5aH AsSGez4WqRCGh02rBtWM9BuV5z1sRCJmk6QL1ozD76kvcc+5Rwx6E/AhtI7m9FtEHW5u GnwW/C91ERWxnGjaMEB3UvEa4rQF6l116+xTFbobHwwnIetkiv1MUQ82IEDpF18GklON NBDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=R1Pq9t0df0nCyf89XYLwYiW7B8f68Ipv1K1w/6MQoXY=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=QsrZWXKjhZaj89gIBU8WKMymvQPxusqhjcZPpYi16iOcbXn9BSEqu1b1ZpclP/3JoH oNglw2o96y7gY5WP3GyFgR5BhNdu+h/fwgvwbpAGLuPKL2b1sWmnaltess+WRicCmYl/ JJ24m/kLNFhio0sKdtPro1ocGka5ia0U7e00iJ78AZkInSJ+CUhNHGmvkrWgZzBWewnO Uibd2TqgXyxrXNYHGMk7auky/QLoncRp4QYrkhF4jcCPqjOoNEcb1yBO/xFRTFLEnxeq Cd/vw+4pztn8y67PaXCtDWCw790O0C1+XzmVLNm0LxcLaPL0B9FjWgBv+G/44B8365CR fSzw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="dg/lshXj"; spf=pass (google.com: domain of linux-kernel+bounces-12732-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12732-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id il5-20020a17090b164500b0028bdb493d1esi14437060pjb.115.2023.12.28.07.09.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:09:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12732-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="dg/lshXj"; spf=pass (google.com: domain of linux-kernel+bounces-12732-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12732-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 8D52AB23EFC for ; Thu, 28 Dec 2023 15:09:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C67B31173E; Thu, 28 Dec 2023 15:06:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dg/lshXj" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A29210951; Thu, 28 Dec 2023 15:06:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703776000; x=1735312000; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=w9DQy+hnxhoJ51h3H6W2Kl69R5yaKXv0XDk0dB+JKAk=; b=dg/lshXjZU9HppyWov0GcVGHou7ppD1HLytcWRnwacFZ7uYY0KYY++zE 4HcQE9VpsnualRr+81qxQ/S57Mwhm6jfshW4h8Pe6LQbdER5t7rnL1Uyk zdckasn1VeWIJtt88aa1FSCXur5zict2VnLt2iIlZcDeQKbBMnA7YAQDI xBWCHKDa+3AvAMebmlb4KywObIiN3gyXGwJ0ZfOR9pNIerPplrAYIucJC Cl4eSFIHRH/VQaYdlm+H70UkF7+RB54tjj9NEWgQrmmr2I79fGtCUkZ+9 AaA6909POICxAQkUVj3E1fhGUfUMLF9rI+OstLtrTiYbn+Ve3gqztQDvp w==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701955" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701955" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155173" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155173" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:38 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 05/10] iommufd/selftest: Add IOMMU_TEST_OP_MD_CHECK_IOTLB test op Date: Thu, 28 Dec 2023 07:06:24 -0800 Message-Id: <20231228150629.13149-6-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538814799542578 X-GMAIL-MSGID: 1786538814799542578 From: Nicolin Chen Allow to test whether IOTLB has been invalidated or not. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- drivers/iommu/iommufd/iommufd_test.h | 5 ++++ drivers/iommu/iommufd/selftest.c | 26 +++++++++++++++++++ tools/testing/selftests/iommu/iommufd.c | 4 +++ tools/testing/selftests/iommu/iommufd_utils.h | 24 +++++++++++++++++ 4 files changed, 59 insertions(+) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index 2eef5afde711..1cedd6b5ba2b 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -21,6 +21,7 @@ enum { IOMMU_TEST_OP_ACCESS_REPLACE_IOAS, IOMMU_TEST_OP_MOCK_DOMAIN_FLAGS, IOMMU_TEST_OP_DIRTY, + IOMMU_TEST_OP_MD_CHECK_IOTLB, }; enum { @@ -121,6 +122,10 @@ struct iommu_test_cmd { __aligned_u64 uptr; __aligned_u64 out_nr_dirty; } dirty; + struct { + __u32 id; + __u32 iotlb; + } check_iotlb; }; __u32 last; }; diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index ebc6c15abf67..9528528cab27 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -853,6 +853,28 @@ static int iommufd_test_md_check_refs(struct iommufd_ucmd *ucmd, return 0; } +static int iommufd_test_md_check_iotlb(struct iommufd_ucmd *ucmd, + u32 mockpt_id, unsigned int iotlb_id, + u32 iotlb) +{ + struct mock_iommu_domain_nested *mock_nested; + struct iommufd_hw_pagetable *hwpt; + int rc = 0; + + hwpt = get_md_pagetable_nested(ucmd, mockpt_id, &mock_nested); + if (IS_ERR(hwpt)) + return PTR_ERR(hwpt); + + mock_nested = container_of(hwpt->domain, + struct mock_iommu_domain_nested, domain); + + if (iotlb_id > MOCK_NESTED_DOMAIN_IOTLB_ID_MAX || + mock_nested->iotlb[iotlb_id] != iotlb) + rc = -EINVAL; + iommufd_put_object(ucmd->ictx, &hwpt->obj); + return rc; +} + struct selftest_access { struct iommufd_access *access; struct file *file; @@ -1334,6 +1356,10 @@ int iommufd_test(struct iommufd_ucmd *ucmd) return iommufd_test_md_check_refs( ucmd, u64_to_user_ptr(cmd->check_refs.uptr), cmd->check_refs.length, cmd->check_refs.refs); + case IOMMU_TEST_OP_MD_CHECK_IOTLB: + return iommufd_test_md_check_iotlb(ucmd, cmd->id, + cmd->check_iotlb.id, + cmd->check_iotlb.iotlb); case IOMMU_TEST_OP_CREATE_ACCESS: return iommufd_test_create_access(ucmd, cmd->id, cmd->create_access.flags); diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index 6ed328c863c4..c8763b880a16 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -330,6 +330,10 @@ TEST_F(iommufd_ioas, alloc_hwpt_nested) &nested_hwpt_id[1], IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[0], + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[1], + IOMMU_TEST_IOTLB_DEFAULT); /* Negative test: a nested hwpt on top of a nested hwpt */ test_err_hwpt_alloc_nested(EINVAL, self->device_id, diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index ad9202335656..fe0a0f566b67 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -195,6 +195,30 @@ static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, _test_cmd_hwpt_alloc(self->fd, device_id, pt_id, flags, \ hwpt_id, data_type, data, data_len)) +#define test_cmd_hwpt_check_iotlb(hwpt_id, iotlb_id, expected) \ + ({ \ + struct iommu_test_cmd test_cmd = { \ + .size = sizeof(test_cmd), \ + .op = IOMMU_TEST_OP_MD_CHECK_IOTLB, \ + .id = hwpt_id, \ + .check_iotlb = { \ + .id = iotlb_id, \ + .iotlb = expected, \ + }, \ + }; \ + ASSERT_EQ(0, \ + ioctl(self->fd, \ + _IOMMU_TEST_CMD(IOMMU_TEST_OP_MD_CHECK_IOTLB), \ + &test_cmd)); \ + }) + +#define test_cmd_hwpt_check_iotlb_all(hwpt_id, expected) \ + ({ \ + int i; \ + for (i = 0; i < MOCK_NESTED_DOMAIN_IOTLB_NUM; i++) \ + test_cmd_hwpt_check_iotlb(hwpt_id, i, expected); \ + }) + static int _test_cmd_access_replace_ioas(int fd, __u32 access_id, unsigned int ioas_id) { From patchwork Thu Dec 28 15:06:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183759 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2049476dyb; Thu, 28 Dec 2023 07:09:04 -0800 (PST) X-Google-Smtp-Source: AGHT+IFmvcDuZ2JGpwD1expTN5m59GTAh/qIutHE6x4w5Ru6MmjzxxZ7kqjho/XKS0xO1YJOsxOT X-Received: by 2002:a2e:154e:0:b0:2cc:67ad:4153 with SMTP id 14-20020a2e154e000000b002cc67ad4153mr3939448ljv.25.1703776144599; Thu, 28 Dec 2023 07:09:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776144; cv=none; d=google.com; s=arc-20160816; b=wdomdtlclYdutlbEd2A7fTHwkNGvbQCeikFTCW5tCxsOZPHNl9eRdabVaXRQIpU7go RY1sbSB3pEg4ORsoBEPpRuaC7HbJudBSwrqCsfhSS5MC5f/CnSUTQy+CetVUBX5/76Mj EnqogM8/+UhN1B+RHbWJTGw6yePVteI1HfEWHp2EIfvgXLUe4FU7IFi9INQTJPYdi6C2 T1qc5BNP0yr0PXYLhX47vi5iPvWMGqlZpbdP0X3buAYsu66//uB+oV0BjemoTnXn7l6z L/g1lVxM6FxetB+YJdXqLk6SmDIcEFxqGzy8o0AjtjP+tZQ0ApCPF4yySnZ2J9C8lx9Y Dvwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=OyEOktip7wvyaDgnq8oGd4eiLr6iHThnb83RM4icWuY=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=jMpc9x/VCIDulGnw0OF8DpbASUKTJpq7khZZlZjVjrjPl+eHDd+4RnW5S/M3tWcs5X q32nV85avZmXAmK9mNo5ZPOOw77xqIgdHCLhptLJol3MaMBLiYh0zsRzdddwwccmA1oi 14PnHoTqg3zwnfrohIf40QiFwVOlljwHigNYdudT6wNHQQgqK4+oNd4V6XsS2ZI1MK6E IzE6YLMA/Sd7eLaD/3UUK4cSBofSk/QSz6OYkQi5WKTnSwJ4r/QfQeP/NzR2KkLomzSm ToCLJ8Paus9dKf/ArJXn+K1Lokw9EQbZJs1nk0H3sQiNbAmSXACXRcPw4ePUZSnEKQWH Wy+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="iKTYWq/4"; spf=pass (google.com: domain of linux-kernel+bounces-12731-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12731-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id a6-20020a50c306000000b005553483448csi2229950edb.517.2023.12.28.07.09.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:09:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12731-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="iKTYWq/4"; spf=pass (google.com: domain of linux-kernel+bounces-12731-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12731-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 0EB871F24172 for ; Thu, 28 Dec 2023 15:09:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 814E211728; Thu, 28 Dec 2023 15:06:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iKTYWq/4" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AF4B10A13; Thu, 28 Dec 2023 15:06:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703776002; x=1735312002; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xD/D1mwaQz0aVDoqEolYgPt9rT0WuEPycyZdvUJpam8=; b=iKTYWq/4aQ7iHAvQUg/Bhe4dHfipbVyGBWOBtc1xwYR1NrFsHy6sqZwq l+/zmrMLkenL8m9VfbG5sA28AYDYUWCarFgyhUe18Wk8+eIMD7+rSdISp ThA6z7fKvHfllT+ki+WZR7jV0jXdYuVt5EBb8ok1VzEWnwFaltT6H831i MKsxC8IVgDz126No+zHdYx1z8ox1Cjt5/H96y6D2EySynVVXsXy05c6xE PvaPzlJ2gS47IvI2JTG3XD9cnZqhASQQoTcSYOyv7kUvK40sN+hr9hAxE Qmlqu/p5O3D/NkhONdaYEH1eaMWgqWXnWhkAl9Al051gd//4Ak3/WtiJz g==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701971" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701971" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155177" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155177" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:40 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 06/10] iommufd/selftest: Add coverage for IOMMU_HWPT_INVALIDATE ioctl Date: Thu, 28 Dec 2023 07:06:25 -0800 Message-Id: <20231228150629.13149-7-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538774856211923 X-GMAIL-MSGID: 1786538774856211923 From: Nicolin Chen Add test cases for the IOMMU_HWPT_INVALIDATE ioctl and verify it by using the new IOMMU_TEST_OP_MD_CHECK_IOTLB. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- tools/testing/selftests/iommu/iommufd.c | 186 ++++++++++++++++++ tools/testing/selftests/iommu/iommufd_utils.h | 33 ++++ 2 files changed, 219 insertions(+) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index c8763b880a16..ff0aa77911f3 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -116,6 +116,7 @@ TEST_F(iommufd, cmd_length) TEST_LENGTH(iommu_destroy, IOMMU_DESTROY, id); TEST_LENGTH(iommu_hw_info, IOMMU_GET_HW_INFO, __reserved); TEST_LENGTH(iommu_hwpt_alloc, IOMMU_HWPT_ALLOC, __reserved); + TEST_LENGTH(iommu_hwpt_invalidate, IOMMU_HWPT_INVALIDATE, __reserved); TEST_LENGTH(iommu_ioas_alloc, IOMMU_IOAS_ALLOC, out_ioas_id); TEST_LENGTH(iommu_ioas_iova_ranges, IOMMU_IOAS_IOVA_RANGES, out_iova_alignment); @@ -271,7 +272,9 @@ TEST_F(iommufd_ioas, alloc_hwpt_nested) struct iommu_hwpt_selftest data = { .iotlb = IOMMU_TEST_IOTLB_DEFAULT, }; + struct iommu_hwpt_invalidate_selftest inv_reqs[2] = {}; uint32_t nested_hwpt_id[2] = {}; + uint32_t num_inv; uint32_t parent_hwpt_id = 0; uint32_t parent_hwpt_id_not_work = 0; uint32_t test_hwpt_id = 0; @@ -344,6 +347,189 @@ TEST_F(iommufd_ioas, alloc_hwpt_nested) EXPECT_ERRNO(EBUSY, _test_ioctl_destroy(self->fd, parent_hwpt_id)); + /* hwpt_invalidate only supports a user-managed hwpt (nested) */ + num_inv = 1; + test_err_hwpt_invalidate(ENOENT, parent_hwpt_id, inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Check data_type by passing zero-length array */ + num_inv = 0; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: Invalid data_type */ + num_inv = 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST_INVALID, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: structure size sanity */ + num_inv = 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs) + 1, &num_inv); + assert(!num_inv); + + num_inv = 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + 1, &num_inv); + assert(!num_inv); + + /* Negative test: invalid flag is passed */ + num_inv = 1; + inv_reqs[0].flags = 0xffffffff; + test_err_hwpt_invalidate(EOPNOTSUPP, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: non-zero __reserved is passed */ + num_inv = 1; + inv_reqs[0].flags = 0; + inv_reqs[0].__reserved = 0x1234; + test_err_hwpt_invalidate(EOPNOTSUPP, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid data_uptr when array is not empty */ + num_inv = 1; + inv_reqs[0].flags = 0; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], NULL, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid entry_len when array is not empty */ + num_inv = 1; + inv_reqs[0].flags = 0; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + 0, &num_inv); + assert(!num_inv); + + /* Negative test: invalid iotlb_id */ + num_inv = 1; + inv_reqs[0].flags = 0; + inv_reqs[0].__reserved = 0; + inv_reqs[0].iotlb_id = MOCK_NESTED_DOMAIN_IOTLB_ID_MAX + 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: trigger error */ + num_inv = 1; + inv_reqs[0].flags = IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR; + inv_reqs[0].iotlb_id = 0; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(inv_reqs[0].hw_error == IOMMU_TEST_INVALIDATE_FAKE_ERROR); + + /* Negative test: trigger error with ALL flag set */ + num_inv = 1; + inv_reqs[0].flags = IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR | + IOMMU_TEST_INVALIDATE_FLAG_ALL; + inv_reqs[0].iotlb_id = 0; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(inv_reqs[0].hw_error == IOMMU_TEST_INVALIDATE_FAKE_ERROR); + + /* + * Invalidate the 1st iotlb entry but fail the 2nd request + * - mock driver error, the hw_error field is meaningful, + * the ioctl returns 0. + */ + num_inv = 2; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 0; + inv_reqs[1].flags = IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR; + inv_reqs[1].iotlb_id = 1; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 2); + assert(!inv_reqs[0].hw_error); + assert(inv_reqs[1].hw_error == IOMMU_TEST_INVALIDATE_FAKE_ERROR); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 0, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 1, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 2, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 3, + IOMMU_TEST_IOTLB_DEFAULT); + + /* + * Invalidate the 1st iotlb entry but fail the 2nd request + * - ioctl error, the hw_error field is meaningless + */ + num_inv = 2; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 0; + inv_reqs[1].flags = 0; + inv_reqs[1].iotlb_id = MOCK_NESTED_DOMAIN_IOTLB_ID_MAX + 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(!inv_reqs[0].hw_error); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 0, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 1, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 2, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 3, + IOMMU_TEST_IOTLB_DEFAULT); + + /* Invalidate the 2nd iotlb entry and verify */ + num_inv = 1; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 1; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!inv_reqs[0].hw_error); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 0, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 1, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 2, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 3, + IOMMU_TEST_IOTLB_DEFAULT); + + /* Invalidate the 3rd and 4th iotlb entries and verify */ + num_inv = 2; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 2; + inv_reqs[1].flags = 0; + inv_reqs[1].iotlb_id = 3; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 2); + assert(!inv_reqs[0].hw_error); + assert(!inv_reqs[1].hw_error); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[0], 0); + + /* Invalidate all iotlb entries for nested_hwpt_id[1] and verify */ + num_inv = 1; + inv_reqs[0].flags = IOMMU_TEST_INVALIDATE_FLAG_ALL; + test_cmd_hwpt_invalidate(nested_hwpt_id[1], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(!inv_reqs[0].hw_error); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[1], 0); + /* Attach device to nested_hwpt_id[0] that then will be busy */ test_cmd_mock_domain_replace(self->stdev_id, nested_hwpt_id[0]); EXPECT_ERRNO(EBUSY, diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index fe0a0f566b67..7f41fb796a8a 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -219,6 +219,39 @@ static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, test_cmd_hwpt_check_iotlb(hwpt_id, i, expected); \ }) +static int _test_cmd_hwpt_invalidate(int fd, __u32 hwpt_id, void *reqs, + uint32_t data_type, uint32_t lreq, + uint32_t *nreqs) +{ + struct iommu_hwpt_invalidate cmd = { + .size = sizeof(cmd), + .hwpt_id = hwpt_id, + .data_type = data_type, + .data_uptr = (uint64_t)reqs, + .entry_len = lreq, + .entry_num = *nreqs, + }; + int rc = ioctl(fd, IOMMU_HWPT_INVALIDATE, &cmd); + *nreqs = cmd.entry_num; + return rc; +} + +#define test_cmd_hwpt_invalidate(hwpt_id, reqs, data_type, lreq, nreqs) \ + ({ \ + ASSERT_EQ(0, \ + _test_cmd_hwpt_invalidate(self->fd, hwpt_id, reqs, \ + data_type, \ + lreq, nreqs)); \ + }) +#define test_err_hwpt_invalidate(_errno, hwpt_id, reqs, data_type, lreq, \ + nreqs) \ + ({ \ + EXPECT_ERRNO(_errno, \ + _test_cmd_hwpt_invalidate(self->fd, hwpt_id, \ + reqs, data_type, \ + lreq, nreqs)); \ + }) + static int _test_cmd_access_replace_ioas(int fd, __u32 access_id, unsigned int ioas_id) { From patchwork Thu Dec 28 15:06:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183760 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2049834dyb; Thu, 28 Dec 2023 07:09:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IEzv8KNvAGIuNcTDQUyeifcDJKzWvicwtzOdstUt3oNHpDRpY5Gvy7X8PoAPn5q6kwFBQcc X-Received: by 2002:a05:6a21:1507:b0:196:2fb1:8014 with SMTP id nq7-20020a056a21150700b001962fb18014mr3182029pzb.37.1703776172773; Thu, 28 Dec 2023 07:09:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776172; cv=none; d=google.com; s=arc-20160816; b=JLcbfeMXzKzFTgjiGL3Qy3jtKzx0whUor90mwOaVs3+9JzwmedwYL21A2wgnNeVVjm 2Y1qTmmLVMvk/ile0nH5OM7nx6bD1kt+3rU1jVR3YV0g5Due/l1QaI1PyR5JCuN0xpdC lWsqmoR6DE8D4pCnm7u6neM3BrV0B6y5U4/ww5alZYkgjvu3XiqA3An2Vfs2LkkjpXoK zQ3ioif9chv99qeNvYWxTZzsb5nt+iwXjnNYRmIEz2hBk0rRfjJzGwbCB7lmmXbyhXkb 7HUZW4L1e4Ed2TX4W3duW7v9pCCtSXsQ0EBK0c9KsAv4n5MbW9m8HEuBGojEqN/3BELJ covg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cZ1v3ds8Dj12GurOttsb8+ULGsCYD7mApzNcD01S5AE=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=t2AI8iEiywCGz5d0XAfyiEtoJFf0t3g22Fu8LX99YhEj4prGVNK6d4GOxTBUP4CsbH VsVAS3za1xNJfKted+manQhECP90gXUi5DRszkzMXeh6Xav2OBw9EUcRaB4tSG6ZqyFl /OznxOLmQUA8WtTccVeGKl2qlJ/P1mXlCkV4HH1kqqzPg3j47llR94iIn3faN3ueWy96 pvV3MeyhI67FUXhwUnmJ7lVKK5p1cD1VAHQEukpRGmtYSRPc6gLh5OWrMMPRQrh0gg0M 0/HwEUUCwUNdqICfrRTRJ6cQlhHBUz1K85npmjw2LwHoTaPrTI6tOZkXbL/PbOWrG8mJ nGrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Bhg1rR4U; spf=pass (google.com: domain of linux-kernel+bounces-12733-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12733-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id o6-20020a635d46000000b005bdc49d22a4si13062729pgm.103.2023.12.28.07.09.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:09:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12733-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Bhg1rR4U; spf=pass (google.com: domain of linux-kernel+bounces-12733-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12733-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 86D23285C34 for ; Thu, 28 Dec 2023 15:09:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4465A11CBF; Thu, 28 Dec 2023 15:06:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Bhg1rR4U" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0793210A2B; Thu, 28 Dec 2023 15:06:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703776004; x=1735312004; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SC8qX0NKkdCBYaLv5ZTATD7zYtjDxB5zvo1VbERLiRY=; b=Bhg1rR4UL67BijbCsYgbA0do9VIGXy2VVnRucqK72GVrVCHSl8KpwzGm DjfGW/d8jh1vGiPEzch+wotXbgmQ94sPu46AM7cR88B8VLwkYXvajyPqm XIt1zrErt8lfuVg+xKr6+nOIzC6bACGQJPHyN6gdxkqI3z/ooN0N7Mgxt Yl/0KycuhvqMMNUkzXj9lXuKwmMGN4BvpUmhDUE+ul4Zch8KefVDujRbd Dxo8oi1h3obVpV+rxS6XqZVBlS7SAZfTDjpAAIkrHPqWWWkR+J+saNcxD CbILETKTMplchTLDkei2I58m2kiF5JktcjTDfpbNFm7pLutmDa/hWqjIR g==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701985" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701985" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155180" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155180" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:42 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 07/10] iommu/vt-d: Allow qi_submit_sync() to return the QI faults Date: Thu, 28 Dec 2023 07:06:26 -0800 Message-Id: <20231228150629.13149-8-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538804061708550 X-GMAIL-MSGID: 1786538804061708550 From: Lu Baolu This allows qi_submit_sync() to return back faults to callers. Signed-off-by: Lu Baolu Signed-off-by: Yi Liu Reviewed-by: Kevin Tian --- drivers/iommu/intel/dmar.c | 42 ++++++++++++++++++++--------- drivers/iommu/intel/iommu.h | 2 +- drivers/iommu/intel/irq_remapping.c | 2 +- drivers/iommu/intel/pasid.c | 2 +- drivers/iommu/intel/svm.c | 6 ++--- 5 files changed, 36 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 23cb80d62a9a..cba5de54e94b 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1267,7 +1267,8 @@ static void qi_dump_fault(struct intel_iommu *iommu, u32 fault) (unsigned long long)desc->qw1); } -static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) +static int qi_check_fault(struct intel_iommu *iommu, int index, + int wait_index, u32 *fsts) { u32 fault; int head, tail; @@ -1278,8 +1279,12 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) return -EAGAIN; fault = readl(iommu->reg + DMAR_FSTS_REG); - if (fault & (DMA_FSTS_IQE | DMA_FSTS_ITE | DMA_FSTS_ICE)) + fault &= DMA_FSTS_IQE | DMA_FSTS_ITE | DMA_FSTS_ICE; + if (fault) { + if (fsts) + *fsts |= fault; qi_dump_fault(iommu, fault); + } /* * If IQE happens, the head points to the descriptor associated @@ -1324,7 +1329,15 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) head = (head - 2 + QI_LENGTH) % QI_LENGTH; } while (head != tail); - if (qi->desc_status[wait_index] == QI_ABORT) + if (qi->desc_status[wait_index] == QI_ABORT) { + /* + * If the caller is interested in the error, no need + * to retry, just return the time out error to the + * caller. + */ + if (fsts) + return -ETIMEDOUT; + } return -EAGAIN; } @@ -1342,9 +1355,11 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) * time, a wait descriptor will be appended to each submission to ensure * hardware has completed the invalidation before return. Wait descriptors * can be part of the submission but it will not be polled for completion. + * If callers are interested in the QI faults that occur during the handling + * of requests, the QI faults are saved in @fault. */ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, - unsigned int count, unsigned long options) + unsigned int count, unsigned long options, u32 *fault) { struct q_inval *qi = iommu->qi; s64 devtlb_start_ktime = 0; @@ -1422,6 +1437,9 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, */ writel(qi->free_head << shift, iommu->reg + DMAR_IQT_REG); + if (fault) + *fault = 0; + while (qi->desc_status[wait_index] != QI_DONE) { /* * We will leave the interrupts disabled, to prevent interrupt @@ -1430,7 +1448,7 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, * a deadlock where the interrupt context can wait indefinitely * for free slots in the queue. */ - rc = qi_check_fault(iommu, index, wait_index); + rc = qi_check_fault(iommu, index, wait_index, fault); if (rc) break; @@ -1476,7 +1494,7 @@ void qi_global_iec(struct intel_iommu *iommu) desc.qw3 = 0; /* should never fail */ - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, @@ -1490,7 +1508,7 @@ void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, @@ -1514,7 +1532,7 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, @@ -1545,7 +1563,7 @@ void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* PASID-based IOTLB invalidation */ @@ -1586,7 +1604,7 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, QI_EIOTLB_AM(mask); } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* PASID-based device IOTLB Invalidate */ @@ -1639,7 +1657,7 @@ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw1 |= QI_DEV_EIOTLB_SIZE; } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, @@ -1649,7 +1667,7 @@ void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) | QI_PC_GRAN(granu) | QI_PC_TYPE; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index ce030c5b5772..c6de958e4f54 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -881,7 +881,7 @@ void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, u32 pasid); int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, - unsigned int count, unsigned long options); + unsigned int count, unsigned long options, u32 *fault); /* * Options used in qi_submit_sync: * QI_OPT_WAIT_DRAIN - Wait for PRQ drain completion, spec 6.5.2.8. diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index 29b9e55dcf26..f834afa3672d 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -153,7 +153,7 @@ static int qi_flush_iec(struct intel_iommu *iommu, int index, int mask) desc.qw2 = 0; desc.qw3 = 0; - return qi_submit_sync(iommu, &desc, 1, 0); + return qi_submit_sync(iommu, &desc, 1, 0, NULL); } static int modify_irte(struct irq_2_iommu *irq_iommu, diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 74e8e4c17e81..67f924760ba8 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -467,7 +467,7 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } static void diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ac12f76c1212..660d049ad5b6 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -543,7 +543,7 @@ void intel_drain_pasid_prq(struct device *dev, u32 pasid) QI_DEV_IOTLB_PFSID(info->pfsid); qi_retry: reinit_completion(&iommu->prq_complete); - qi_submit_sync(iommu, desc, 3, QI_OPT_WAIT_DRAIN); + qi_submit_sync(iommu, desc, 3, QI_OPT_WAIT_DRAIN, NULL); if (readl(iommu->reg + DMAR_PRS_REG) & DMA_PRS_PRO) { wait_for_completion(&iommu->prq_complete); goto qi_retry; @@ -646,7 +646,7 @@ static void handle_bad_prq_event(struct intel_iommu *iommu, desc.qw3 = 0; } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } static irqreturn_t prq_event_thread(int irq, void *d) @@ -811,7 +811,7 @@ int intel_svm_page_response(struct device *dev, ktime_to_ns(ktime_get()) - prm->private_data[0]); } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } out: return ret; From patchwork Thu Dec 28 15:06:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183762 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2050312dyb; Thu, 28 Dec 2023 07:10:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IGip/u++5d9+uPAaX7+j4q6ryJ+Q4VSAf2tqwrJTaMYGd0q+8EI73EnHjTiylITCQ9X5kCU X-Received: by 2002:a05:6808:215:b0:3bb:bd2b:4ee9 with SMTP id l21-20020a056808021500b003bbbd2b4ee9mr2523430oie.19.1703776212779; Thu, 28 Dec 2023 07:10:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776212; cv=none; d=google.com; s=arc-20160816; b=iP5oITjmW/67DLLVkmsAqLvkMhYh6xKJblKxUjbtKdC1aKT0i/L2Z75NoJILcYbAY8 xDLpoSaaOF8BDvIWxuwXcZKkQEH1BwxuULtByoDOb52pS0jeHXmRKCUYAR/yegYie4I0 PzpLu05DrWDAJ0/bU5EWH/zs5+Ne+AVw2BbRX57x+oltuSuLt6v4KWbI4O5nZbGw0XLm JXD1u7DL0CLV2ZbozlYq35kONT/zEiPneXNpnHWQlOwyiEj4JHV7rXUQlziALW9GGmyT q/6MdCa6U4BAtYaYAGo9GrZomWm4+lRJ38RrS5KGXrOh1f1oCHFEwRoh1S+CEerLti08 0+Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=L6FJeFx1qcrUZ9ABovfHCN9YAoHx1iC/opgbUQDGHzM=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=uEyR93jSv0g0FwzWjfgfmuscB3GywdHq+1BgDaF1EgLqONSc5OY4a+cFiWdhW2Cmk5 7qb/8heAUYSAvKoLOHGl/dMR2jM839HIkBJotnyvVtxEfLiTM6Ofv1n5KvRBEHcU7F+c 7ETAC9sssesPK2y59OF/161A6gu0gh3ddqke3hRXb6a5RKL9K27Vz+UVPgiY/1YC2VmQ wdcVPvvbmfP0X7QKF+Fk3lLNCb0iKjZhJdMo+EGrWWZ1dHNxk1pHCFGGLGUbXAJVuBBg AewbkpxmV2BSiz2uC3sAwIMH+5OokmOqGgORGGvzj04wyZBjzz+tt4mP29CAYVRBs1MS iR4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BzuSwbAL; spf=pass (google.com: domain of linux-kernel+bounces-12734-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12734-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id jn26-20020a05610219da00b004670a7e78d3si878958vsb.553.2023.12.28.07.10.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:10:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12734-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BzuSwbAL; spf=pass (google.com: domain of linux-kernel+bounces-12734-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12734-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 8D65D1C233AB for ; Thu, 28 Dec 2023 15:09:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D8B9312B6E; Thu, 28 Dec 2023 15:06:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BzuSwbAL" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE73C111BD; Thu, 28 Dec 2023 15:06:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703776005; x=1735312005; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cBvbUxPN+N6HrpgbF5Ik40UwOxug9Ccl8rEhJWBhYbU=; b=BzuSwbALY0pDmkh/7Rp62mo6fqn2qRok44OKA8o256eFejLfMXlnwtIB oXWOHL2JEWCKLcnowoJqS56ySKfKT5L4ulhiiOGlxoHQ6uw5j1Etdg8zH V+bo8jljhsnlVw/VTuz6AR21ZOfpjEz35p9BDFHFPaz+6YhLiSEjUOv34 xcRJ3DOgTktRic3ht0BmC2gLv0IbYeukukalyuJNKO9ZQy+5cxM4pSNRD JYReeK9IwD4qgZSRfvJJRWxO+6hkTwmrI2ZhtmSHB57TDFMIJbvm16Aia b2ejRNjyIgBlOI8XVawW4+Y4CuH7lQ2S1nNV1iVPGiuSt0u0STKUcvcR8 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393701999" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393701999" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155183" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155183" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:43 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 08/10] iommu/vt-d: Convert stage-1 cache invalidation to return QI fault Date: Thu, 28 Dec 2023 07:06:27 -0800 Message-Id: <20231228150629.13149-9-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538846313073801 X-GMAIL-MSGID: 1786538846313073801 From: Lu Baolu This makes the pasid based cache invalidation and device TLB invalidation to return QI faults to callers. This is needed when usersapce invalidates cache after modifying the stage-1 page table used in nested translation. Hardware errors during invalidation should be reported to user. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Signed-off-by: Yi Liu --- drivers/iommu/intel/dmar.c | 13 +++++++------ drivers/iommu/intel/iommu.c | 12 ++++++------ drivers/iommu/intel/iommu.h | 6 +++--- drivers/iommu/intel/pasid.c | 12 +++++++----- drivers/iommu/intel/svm.c | 8 ++++---- 5 files changed, 27 insertions(+), 24 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index cba5de54e94b..ecb6ba1828aa 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1536,7 +1536,7 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, } void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u16 qdep, u64 addr, unsigned mask) + u16 qdep, u64 addr, unsigned mask, u32 *fault) { struct qi_desc desc; @@ -1563,12 +1563,12 @@ void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0, NULL); + qi_submit_sync(iommu, &desc, 1, 0, fault); } /* PASID-based IOTLB invalidation */ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, - unsigned long npages, bool ih) + unsigned long npages, bool ih, u32 *fault) { struct qi_desc desc = {.qw2 = 0, .qw3 = 0}; @@ -1604,12 +1604,13 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, QI_EIOTLB_AM(mask); } - qi_submit_sync(iommu, &desc, 1, 0, NULL); + qi_submit_sync(iommu, &desc, 1, 0, fault); } /* PASID-based device IOTLB Invalidate */ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u32 pasid, u16 qdep, u64 addr, unsigned int size_order) + u32 pasid, u16 qdep, u64 addr, + unsigned int size_order, u32 *fault) { unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1); struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0}; @@ -1657,7 +1658,7 @@ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw1 |= QI_DEV_EIOTLB_SIZE; } - qi_submit_sync(iommu, &desc, 1, 0, NULL); + qi_submit_sync(iommu, &desc, 1, 0, fault); } void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 897159dba47d..68e494f1d03a 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1462,7 +1462,7 @@ static void __iommu_flush_dev_iotlb(struct device_domain_info *info, sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, addr, mask); + qdep, addr, mask, NULL); quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep); } @@ -1490,7 +1490,7 @@ static void iommu_flush_dev_iotlb(struct dmar_domain *domain, PCI_DEVID(info->bus, info->devfn), info->pfsid, dev_pasid->pasid, info->ats_qdep, addr, - mask); + mask, NULL); } spin_unlock_irqrestore(&domain->lock, flags); } @@ -1505,10 +1505,10 @@ static void domain_flush_pasid_iotlb(struct intel_iommu *iommu, spin_lock_irqsave(&domain->lock, flags); list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) - qi_flush_piotlb(iommu, did, dev_pasid->pasid, addr, npages, ih); + qi_flush_piotlb(iommu, did, dev_pasid->pasid, addr, npages, ih, NULL); if (!list_empty(&domain->devices)) - qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih); + qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih, NULL); spin_unlock_irqrestore(&domain->lock, flags); } @@ -5195,10 +5195,10 @@ void quirk_extra_dev_tlb_flush(struct device_domain_info *info, sid = PCI_DEVID(info->bus, info->devfn); if (pasid == IOMMU_NO_PASID) { qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, address, mask); + qdep, address, mask, NULL); } else { qi_flush_dev_iotlb_pasid(info->iommu, sid, info->pfsid, - pasid, qdep, address, mask); + pasid, qdep, address, mask, NULL); } } diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index c6de958e4f54..ce9bd08dcd05 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -866,14 +866,14 @@ void qi_flush_context(struct intel_iommu *iommu, u16 did, void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, unsigned int size_order, u64 type); void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u16 qdep, u64 addr, unsigned mask); + u16 qdep, u64 addr, unsigned mask, u32 *fault); void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, - unsigned long npages, bool ih); + unsigned long npages, bool ih, u32 *fault); void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, u32 pasid, u16 qdep, u64 addr, - unsigned int size_order); + unsigned int size_order, u32 *fault); void quirk_extra_dev_tlb_flush(struct device_domain_info *info, unsigned long address, unsigned long pages, u32 pasid, u16 qdep); diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 67f924760ba8..4a7fe551d8a6 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -492,9 +492,11 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, * efficient to flush devTLB specific to the PASID. */ if (pasid == IOMMU_NO_PASID) - qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, 64 - VTD_PAGE_SHIFT); + qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, + 64 - VTD_PAGE_SHIFT, NULL); else - qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, 64 - VTD_PAGE_SHIFT); + qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, + 64 - VTD_PAGE_SHIFT, NULL); } void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev, @@ -521,7 +523,7 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev, pasid_cache_invalidation_with_pasid(iommu, did, pasid); if (pgtt == PASID_ENTRY_PGTT_PT || pgtt == PASID_ENTRY_PGTT_FL_ONLY) - qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0, NULL); else iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); @@ -543,7 +545,7 @@ static void pasid_flush_caches(struct intel_iommu *iommu, if (cap_caching_mode(iommu->cap)) { pasid_cache_invalidation_with_pasid(iommu, did, pasid); - qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0, NULL); } else { iommu_flush_write_buffer(iommu); } @@ -834,7 +836,7 @@ void intel_pasid_setup_page_snoop_control(struct intel_iommu *iommu, * Addr[63:12]=0x7FFFFFFF_FFFFF) to affected functions */ pasid_cache_invalidation_with_pasid(iommu, did, pasid); - qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0, NULL); /* Device IOTLB doesn't need to be flushed in caching mode. */ if (!cap_caching_mode(iommu->cap)) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 660d049ad5b6..bf7b4c5c21f4 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -179,11 +179,11 @@ static void __flush_svm_range_dev(struct intel_svm *svm, if (WARN_ON(!pages)) return; - qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages, ih); + qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages, ih, NULL); if (info->ats_enabled) { qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info->pfsid, svm->pasid, sdev->qdep, address, - order_base_2(pages)); + order_base_2(pages), NULL); quirk_extra_dev_tlb_flush(info, address, order_base_2(pages), svm->pasid, sdev->qdep); } @@ -225,11 +225,11 @@ static void intel_flush_svm_all(struct intel_svm *svm) list_for_each_entry_rcu(sdev, &svm->devs, list) { info = dev_iommu_priv_get(sdev->dev); - qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, 0, -1UL, 0); + qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, 0, -1UL, 0, NULL); if (info->ats_enabled) { qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info->pfsid, svm->pasid, sdev->qdep, - 0, 64 - VTD_PAGE_SHIFT); + 0, 64 - VTD_PAGE_SHIFT, NULL); quirk_extra_dev_tlb_flush(info, 0, 64 - VTD_PAGE_SHIFT, svm->pasid, sdev->qdep); } From patchwork Thu Dec 28 15:06:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183764 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2051038dyb; Thu, 28 Dec 2023 07:11:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IGfJ9pbmO8AakZp0lYw5yc3JrwvEpGBNcnCdHH3Gx1bXhbuuqDOpi3StGxz8MfuF/Xga1/+ X-Received: by 2002:a05:6a21:9981:b0:195:be2c:12e2 with SMTP id ve1-20020a056a21998100b00195be2c12e2mr7018126pzb.27.1703776283074; Thu, 28 Dec 2023 07:11:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776283; cv=none; d=google.com; s=arc-20160816; b=APupZYgiR6ZSo/v+ArAWi/r5yP5wy6b8evIZ+mI4Ji9jsRUn+qow5sMkrIgB3aqCmz 0dG/VQ53t2O7v2n7aG8PIEeg1F925zskdJNyLz8W+3jNbxdhn1PdVUpHrNw8boia66J+ o+xCYsl557A9KhUjaOIsvr6ljBrfskh5uMW3OqqbI9CZeLjFyZYtueoHixe57pYnldU0 NiFxu41ObPKu+wgZpM3FN2YewiEFvrPBEUdX/UH1TO8ztjszhKKjCCNsTGcDOANsTeFd 7zmUDn1+JDJqCysSQln0ZNE5bwNN4kKafhLL2m9lsgFqZDj/E6AB6xElGNd2WB+WKPt8 kzWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=KmghMqZ76DJtyDhTv2WoB5KuW/yZWkE85ExZ1zPyKIE=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=tURX/cMI93G5aMRapeIes7yV/agjQZgE7XpRF+x4lImfUhIFY5iuT9sk6+LL80d2YV 6SrBmwFYIpm23UtSQ3tM67xNZV5ZBWBzL3RlidfNvssEikw1uYcRYsFE73kyFlfUIkI7 Fkco0C2DHls3SCLcar0GdUqcgtZF3yBMqJW91sCiCVHo4OK5ZozQV3SDIf7c5ukifY/U XsJyEKluDIfhJFQHMx+mbEOovwfMC0FC7tpdChkhRqEocqZUNAoZUW+EarB7F3xnyg3A ED3m0Kstjug6gyXjbdqaQlCcWhGsy3rn0ybUh6ybn2+G5h/6aRWnf3SlX1Oe1RtfSUTS beCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=TKFDMsST; spf=pass (google.com: domain of linux-kernel+bounces-12735-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12735-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id e3-20020a656483000000b005ce02d8ef08si9271996pgv.884.2023.12.28.07.11.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:11:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12735-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=TKFDMsST; spf=pass (google.com: domain of linux-kernel+bounces-12735-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12735-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 2D15FB23F44 for ; Thu, 28 Dec 2023 15:10:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8DC0112B85; Thu, 28 Dec 2023 15:06:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TKFDMsST" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BDAD11726; Thu, 28 Dec 2023 15:06:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703776006; x=1735312006; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pIwuSBuivaLiuysTG8SR9U5i6WTksuS89VvUb25xUrk=; b=TKFDMsSTWT9chyIj0YQxG2cSMjxL9WR+LmdXSXBN/eGqtLd977gmvyxx dPgjXTDHlUFRBkbe+LuzZBKTt07G5qX+dfHARUU3MFLnB/H87/D5pxfkR fO6gjMH1H3JCwRI9KPUKPnvN7rRKsBrmXO/X5KS6QhlvkzOfW7cuDH3HK Q7VeLJvabvsj7uWT/SWkUmdbOIcHN3df4E4nhSYYP9BqEicr8Y6BzXZe6 0na00xnSO45TrWhcCZysH2TAoi+lq8AzYDEiG77y400PklJiLSxtGs1kY FQvoMX+eVjukDzs2BS97H5dRMlhnYG50ChJ6UfmySzg2rhIIX7SYLN7Jl A==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393702019" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393702019" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155186" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155186" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:45 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 09/10] iommufd: Add data structure for Intel VT-d stage-1 cache invalidation Date: Thu, 28 Dec 2023 07:06:28 -0800 Message-Id: <20231228150629.13149-10-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538919565453342 X-GMAIL-MSGID: 1786538919565453342 This adds the data structure invalidating caches for the nested domain allocated with IOMMU_HWPT_DATA_VTD_S1 type. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Signed-off-by: Yi Liu --- include/uapi/linux/iommufd.h | 58 ++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 824560c50ec6..37c778055201 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -623,6 +623,64 @@ enum iommu_hwpt_invalidate_data_type { IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, }; +/** + * enum iommu_hwpt_vtd_s1_invalidate_flags - Flags for Intel VT-d + * stage-1 cache invalidation + * @IOMMU_VTD_INV_FLAGS_LEAF: Indicates whether the invalidation applies + * to all-levels page structure cache or just + * the leaf PTE cache. + */ +enum iommu_hwpt_vtd_s1_invalidate_flags { + IOMMU_VTD_INV_FLAGS_LEAF = 1 << 0, +}; + +/** + * enum iommu_hwpt_vtd_s1_invalidate_error - Hardware error of invalidation + * @IOMMU_HWPT_INVALIDATE_VTD_S1_ICE: Invalidation Completion Error, details + * refer to 11.4.7.1 Fault Status Register + * of VT-d specification. + * @IOMMU_HWPT_INVALIDATE_VTD_S1_ITE: Invalidation Time-out Error, details + * refer to 11.4.7.1 Fault Status Register + * of VT-d specification. + */ +enum iommu_hwpt_vtd_s1_invalidate_error { + IOMMU_HWPT_INVALIDATE_VTD_S1_ICE = 1 << 0, + IOMMU_HWPT_INVALIDATE_VTD_S1_ITE = 1 << 1, +}; + +/** + * struct iommu_hwpt_vtd_s1_invalidate - Intel VT-d cache invalidation + * (IOMMU_HWPT_INVALIDATE_DATA_VTD_S1) + * @addr: The start address of the range to be invalidated. It needs to + * be 4KB aligned. + * @npages: Number of contiguous 4K pages to be invalidated. + * @flags: Combination of enum iommu_hwpt_vtd_s1_invalidate_flags + * @hw_error: One of enum iommu_hwpt_vtd_s1_invalidate_error + * + * The Intel VT-d specific invalidation data for user-managed stage-1 cache + * invalidation in nested translation. Userspace uses this structure to + * tell the impacted cache scope after modifying the stage-1 page table. + * + * Invalidating all the caches related to the page table by setting @addr + * to be 0 and @npages to be U64_MAX. + * + * The device TLB will be invalidated automatically if ATS is enabled. + * + * An entry is considered 'handled' after it passes the audit and submitted + * to the IOMMU by the underlying driver. Check the @entry_num output of + * struct iommu_hwpt_invalidate for the number of handled entries. A 'handled' + * request may still fail in hardware for various reasons, e.g. due to timeout + * on waiting for device response upon a device TLB invalidation request. In + * such case the hardware error info is reported in the @hw_error field of the + * handled entry. + */ +struct iommu_hwpt_vtd_s1_invalidate { + __aligned_u64 addr; + __aligned_u64 npages; + __u32 flags; + __u32 hw_error; +}; + /** * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) * @size: sizeof(struct iommu_hwpt_invalidate) From patchwork Thu Dec 28 15:06:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 183766 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp2051276dyb; Thu, 28 Dec 2023 07:11:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IEL8YMNYfohmDYW/0LJGeDFBVtnISsRfTQmjIysyIgxoyzBipQn7lyfkmXq7WlzPcBwjgJc X-Received: by 2002:a05:6e02:1521:b0:360:164b:cd8d with SMTP id i1-20020a056e02152100b00360164bcd8dmr4668331ilu.58.1703776303483; Thu, 28 Dec 2023 07:11:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703776303; cv=none; d=google.com; s=arc-20160816; b=AMgW84MM4d+0QHP+QUDSJc8bUYQyy/tZxHN0K64iEXXukgB1xtDRdFcSYjvrLJMH89 AC5poApIJNM4zSGxbPJYBPPAD0vMeyvZwhiuxw5tx1Mfv7g/SuBrfyEG5zcbIebcxB7s 1nwkZk5aktKo5uGdrkxE4rJx/Kv4wEPCuhnXO9WWLeU7SFcXY6+IROBjeNkhN49mnPaE r8LCQshPToJpdP6Oz3dFwRwtEE+Hi4dEMlLFF0aWFvatIZFd9TUPCsII2jNNrM25dZic ens5QkUvZX2qT253ac9LmAGNoAQzSuU2RdsF2yhIUEbl+iz6ZViwYMvIe4lrrYxB2y/D fscg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=zQNZjPeD8fsqgwz/LoarCf7ojcKSQIn18JmgfykPaqI=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=WDsHI2gSWDkjjvHoWhwcl5xSmQLqSjgxfhjfvBiH2vkvL71/DiUZCV1KNWr+ePjAWD 3VM938/nBKFAQVOFoa3aBkxF3IMJ8uQ3GSGMzfciSx5ldETt1SSVYvpxCe7GhvWaSUic E3nH69KS3TAKhoA2j3hFMIA81gdAAXpuwoIP2v2018to/pJT/GUEUMwZcoVlWyoq13co J1GtOcNJbHOeZLKVbDNdjfUwQDUv3gLWg39yDeMgRPZOYkoosREpAjY5No8szfFkljYn /JoPfkyxPKoS9AL+TXHrmX5ES4gP1MhrDxmTmAbETLNvmquT9V5R5h+KfG7b30UsTu2Y gjZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BOLZvKdk; spf=pass (google.com: domain of linux-kernel+bounces-12736-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12736-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id m25-20020a635819000000b005cdfaea88adsi10081337pgb.774.2023.12.28.07.11.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Dec 2023 07:11:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-12736-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=BOLZvKdk; spf=pass (google.com: domain of linux-kernel+bounces-12736-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-12736-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 49D75B21240 for ; Thu, 28 Dec 2023 15:10:25 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8548112E40; Thu, 28 Dec 2023 15:06:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BOLZvKdk" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8338F125B4; Thu, 28 Dec 2023 15:06:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1703776008; x=1735312008; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k8MfoxWF1GWpKLv7i4xBh341gPxN5h7L+16xXDYToqk=; b=BOLZvKdkMAnHJj5Nh2WZHs4QgMGD32PaDf2WLI7ozSAd10Glb5szWbGP 1R7JZhUoXodM4kGkYS6ry0ZH6X3tLjSforQDBA0HIOse8AuvTRADjTcex b/XS5ldfFdyTlZYPTl/LHF5HMaCEngF8x8p0lYWB2jV/2t8h7V8JcycQY nzNpvzTmNyD8NaueYisrHViGE16+PRxxjmzSt1M84LPwR/Sf8eCw0MF2F o6fYmFdLEgj1evZ1ufiW7+hohjMpkjNTnmCZ3ibDQMxOrXp7pMOBCU15r DOQhnH+jtVZHXT/eL7QGAkloFPsmrws/ETSna+MXs1O7tR1NCWTkSqfWb Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="393702038" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="393702038" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Dec 2023 07:06:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10936"; a="869155189" X-IronPort-AV: E=Sophos;i="6.04,312,1695711600"; d="scan'208";a="869155189" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by FMSMGA003.fm.intel.com with ESMTP; 28 Dec 2023 07:06:46 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v9 10/10] iommu/vt-d: Add iotlb flush for nested domain Date: Thu, 28 Dec 2023 07:06:29 -0800 Message-Id: <20231228150629.13149-11-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231228150629.13149-1-yi.l.liu@intel.com> References: <20231228150629.13149-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786538940928239823 X-GMAIL-MSGID: 1786538940928239823 From: Lu Baolu This implements the .cache_invalidate_user() callback to support iotlb flush for nested domain. Signed-off-by: Lu Baolu Co-developed-by: Yi Liu Signed-off-by: Yi Liu Reviewed-by: Kevin Tian --- drivers/iommu/intel/nested.c | 107 +++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/drivers/iommu/intel/nested.c b/drivers/iommu/intel/nested.c index b5a5563ab32c..f1f86437939c 100644 --- a/drivers/iommu/intel/nested.c +++ b/drivers/iommu/intel/nested.c @@ -73,9 +73,116 @@ static void intel_nested_domain_free(struct iommu_domain *domain) kfree(to_dmar_domain(domain)); } +static void nested_flush_dev_iotlb(struct dmar_domain *domain, u64 addr, + unsigned mask, u32 *fault) +{ + struct device_domain_info *info; + unsigned long flags; + u16 sid, qdep; + + spin_lock_irqsave(&domain->lock, flags); + list_for_each_entry(info, &domain->devices, link) { + if (!info->ats_enabled) + continue; + sid = info->bus << 8 | info->devfn; + qdep = info->ats_qdep; + qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, + qdep, addr, mask, fault); + quirk_extra_dev_tlb_flush(info, addr, mask, + IOMMU_NO_PASID, qdep); + } + spin_unlock_irqrestore(&domain->lock, flags); +} + +static void intel_nested_flush_cache(struct dmar_domain *domain, u64 addr, + unsigned long npages, bool ih, u32 *error) +{ + struct iommu_domain_info *info; + unsigned long i; + unsigned mask; + u32 fault; + + xa_for_each(&domain->iommu_array, i, info) + qi_flush_piotlb(info->iommu, + domain_id_iommu(domain, info->iommu), + IOMMU_NO_PASID, addr, npages, ih, NULL); + + if (!domain->has_iotlb_device) + return; + + if (npages == U64_MAX) + mask = 64 - VTD_PAGE_SHIFT; + else + mask = ilog2(__roundup_pow_of_two(npages)); + + nested_flush_dev_iotlb(domain, addr, mask, &fault); + + *error = 0; + /* + * Invalidation queue error (i.e. IQE) will not be reported to user + * as it's caused only by driver internal bug. + */ + if (fault & DMA_FSTS_ICE) + *error |= IOMMU_HWPT_INVALIDATE_VTD_S1_ICE; + if (fault & DMA_FSTS_ITE) + *error |= IOMMU_HWPT_INVALIDATE_VTD_S1_ITE; +} + +static int intel_nested_cache_invalidate_user(struct iommu_domain *domain, + struct iommu_user_data_array *array) +{ + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + struct iommu_hwpt_vtd_s1_invalidate inv_entry; + u32 processed = 0; + int ret = 0; + u32 index; + + if (array->type != IOMMU_HWPT_INVALIDATE_DATA_VTD_S1) { + ret = -EINVAL; + goto out; + } + + for (index = 0; index < array->entry_num; index++) { + ret = iommu_copy_struct_from_user_array(&inv_entry, array, + IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, + index, hw_error); + if (ret) + break; + + if (inv_entry.flags & ~IOMMU_VTD_INV_FLAGS_LEAF) { + ret = -EOPNOTSUPP; + break; + } + + if (!IS_ALIGNED(inv_entry.addr, VTD_PAGE_SIZE) || + ((inv_entry.npages == U64_MAX) && inv_entry.addr)) { + ret = -EINVAL; + break; + } + + intel_nested_flush_cache(dmar_domain, inv_entry.addr, + inv_entry.npages, + inv_entry.flags & IOMMU_VTD_INV_FLAGS_LEAF, + &inv_entry.hw_error); + + ret = iommu_respond_struct_to_user_array(array, index, + (void *)&inv_entry, + sizeof(inv_entry)); + if (ret) + break; + + processed++; + } + +out: + array->entry_num = processed; + return ret; +} + static const struct iommu_domain_ops intel_nested_domain_ops = { .attach_dev = intel_nested_attach_dev, .free = intel_nested_domain_free, + .cache_invalidate_user = intel_nested_cache_invalidate_user, }; struct iommu_domain *intel_nested_domain_alloc(struct iommu_domain *parent,