From patchwork Tue Jan 2 14:38:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184367 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4483167dyb; Tue, 2 Jan 2024 06:40:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IG8jqSjJZvOSCVNwAkzkW+1GtPqe7+EBJj4Krlhu7j0B14cuFDixvUxK27s+RQrsXyCaDzy X-Received: by 2002:ac2:44c6:0:b0:50e:546b:53bb with SMTP id d6-20020ac244c6000000b0050e546b53bbmr6705726lfm.136.1704206411643; Tue, 02 Jan 2024 06:40:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206411; cv=none; d=google.com; s=arc-20160816; b=hpxljo4kP7eAVsoH0Ic+evhqr0l5EPdekZkfeGvq8DEPCXW/YApK9fk0emRjj2oa7z uYDZ4loFYb9O8XGZxGq6o+cojmCRQ+qfwrfFWWFIRUVrp3MsbFBU1hiyJE/Y+dH5xcqg oIPEircUjxoAqF3Z3S49dYWL8Mz01AnImMU7I6pon0waRASBY9CPud66q/lN2LOXF6FZ Bb+TVZhY2AuyJGWZSf2J0QpBszYThfZ2rkDgZJXA7sMPkdgpYo+Qv/9DEkxfvrLL20tq pjvsOf6XljHYNQbFDMHggwQu3BXZGUBgf33jtJn+4EAnv/bL399xhmL37QGbPGhpJveM KDlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cmmE61uWQDmtK6kaEvaJmx5NWQkq2czzSV7UX/pgsTU=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=OIEc/gFcPWarI23p45TsYQpmpnHVLKeb5BFoWzZuPg/lhGcKpL88F7n7DREVP+yi41 aQCUaO/YKbkzDk+gSu0oLCTfxSl4Y+1w6YrILBdrL3vAyP/xmTk1Iw40m2PThf71IGYw dbketwfVkJRTPPcvFHn4yVO0GKSBrTuqhVr/xcGE3VJ3xGl0kICp/bi3GZal7NLk8J3r xuPh7HJ6VC+Q8pgQwtAUDhaQjhsu2C8mHPc10HZ9MjWKxBM+fsz8GtipqiXfRbVSNyiT mIBOSfDertmWZTUER4Kj3tABLbANs3hIWEzjYLmJ18t8XJinGDhjYejRoezBraxsW+Hu Njag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ijH6wurV; spf=pass (google.com: domain of linux-kernel+bounces-14490-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14490-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id i25-20020a508719000000b0055337154dfbsi8027303edb.437.2024.01.02.06.40.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:40:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14490-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ijH6wurV; spf=pass (google.com: domain of linux-kernel+bounces-14490-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14490-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3C9121F22DB2 for ; Tue, 2 Jan 2024 14:40:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CBDCE1548F; Tue, 2 Jan 2024 14:38:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ijH6wurV" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BFEF12E70; Tue, 2 Jan 2024 14:38:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206319; x=1735742319; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y9zEpidkk+p5pkry7bIuq4ZUGve0T4w0aUbFgMriFaU=; b=ijH6wurV3hdAHIIaT9POzAP1MCuGujZ52C48D7K68O+36lAJpqZ1do6J z4D2VYRCJwbfNmJAF7N24EK3yP9+MFf2x6FIyo6gwzOyNAT4it4TBERZ5 NU2s0AOi+GyITKaogWTkLpu5MKTe37AGCU7nlScVX6fCEbCzlqeOxuMtZ KyjrXrntd8lsJLJtbr9memzX2B+qZFTkA1tdcvjVg/BietnfG+m+UBnhd Z7CxxDMvSfXq4Pb1l+MUjOG9yju67GFYPwxN4637t5illMUrxFiGWK677 Rxcs1Xo2cgWmA6UmUuo8EZ4JwcIaMSZTkqAY0uakj3/YRAy3kDucjbL4i g==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270411" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270411" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234251" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234251" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:37 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 01/10] iommu: Add cache_invalidate_user op Date: Tue, 2 Jan 2024 06:38:25 -0800 Message-Id: <20240102143834.146165-2-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786989942326869306 X-GMAIL-MSGID: 1786989942326869306 From: Lu Baolu The updates of the PTEs in the nested page table will be propagated to the hardware caches. Add a new domain op cache_invalidate_user for the userspace to flush the hardware caches for a nested domain through iommufd. No wrapper for it, as it's only supposed to be used by iommufd. Then, pass in invalidation requests in form of a user data array conatining a number of invalidation data entries. Signed-off-by: Lu Baolu Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- include/linux/iommu.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 6291aa7b079b..93c0d12dd047 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -284,6 +284,23 @@ struct iommu_user_data { size_t len; }; +/** + * struct iommu_user_data_array - iommu driver specific user space data array + * @type: The data type of all the entries in the user buffer array + * @uptr: Pointer to the user buffer array + * @entry_len: The fixed-width length of an entry in the array, in bytes + * @entry_num: The number of total entries in the array + * + * The user buffer includes an array of requests with format defined in + * include/uapi/linux/iommufd.h + */ +struct iommu_user_data_array { + unsigned int type; + void __user *uptr; + size_t entry_len; + u32 entry_num; +}; + /** * __iommu_copy_struct_from_user - Copy iommu driver specific user space data * @dst_data: Pointer to an iommu driver specific user data that is defined in @@ -440,6 +457,13 @@ struct iommu_ops { * @iotlb_sync_map: Sync mappings created recently using @map to the hardware * @iotlb_sync: Flush all queued ranges from the hardware TLBs and empty flush * queue + * @cache_invalidate_user: Flush hardware cache for user space IO page table. + * The @domain must be IOMMU_DOMAIN_NESTED. The @array + * passes in the cache invalidation requests, in form + * of a driver data structure. The driver must update + * array->entry_num to report the number of handled + * invalidation requests. The driver data structure + * must be defined in include/uapi/linux/iommufd.h * @iova_to_phys: translate iova to physical address * @enforce_cache_coherency: Prevent any kind of DMA from bypassing IOMMU_CACHE, * including no-snoop TLPs on PCIe or other platform @@ -465,6 +489,8 @@ struct iommu_domain_ops { size_t size); void (*iotlb_sync)(struct iommu_domain *domain, struct iommu_iotlb_gather *iotlb_gather); + int (*cache_invalidate_user)(struct iommu_domain *domain, + struct iommu_user_data_array *array); phys_addr_t (*iova_to_phys)(struct iommu_domain *domain, dma_addr_t iova); From patchwork Tue Jan 2 14:38:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184368 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4483213dyb; Tue, 2 Jan 2024 06:40:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IGEx+RhbS3rnpj19soHFYUvkzUqpMxmfjfnkgn5F3hracRwUQ5ypvbpc8LuGsQkmZCQLpOO X-Received: by 2002:a05:6214:250e:b0:67f:2ed5:437b with SMTP id gf14-20020a056214250e00b0067f2ed5437bmr22486226qvb.20.1704206416432; Tue, 02 Jan 2024 06:40:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206416; cv=none; d=google.com; s=arc-20160816; b=gytJ7S8NiDCfZbaO5OgotXX6kgINcneCPnfuMlDRxyWV9TDKaSSe/XsTbiQyXhHScP 5j62iqmBMwcQrHq74C6SH/RHrolTZ3oA2OdGm846TrCP/7lJxPiNulMcezJ59asSQp8j 93XDtGrGLedNVDUKcbw3kjiA6B0KThEi+5HcAqF61Tiv+EYelmzicSi3wikpiQ6GzH0b p14+wdf9BxbwTVUD3xnbIeyS4pcKzoeP8KqtNn4y+v5wCOuiMiNkqoTVv/+/5RRVswhY JXQBlINxRcadkcE/7saOg3Amc/suyubM6dZPeQjQlFvkrRxaQkiGvZQHHnWiFsVOj9G/ pZug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=wAwlr7sQ/I48JWhbjP2tWzBDi/kfFU0Ofrr6FCUbrp4=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=YPzhOQsFgzc6blieNf1wctmXBt+w34Zf/ZUVpz38AS+8pXcm4cHwaUBC9aE+KQb/wG 82psEZeNYCkp+uHbWBizX9cBqmzLdSYBLMufiq3n0MzT/m+KpumpqrDhvCcJFcWW/Bp1 r6c07oB8S3Y/DoQmMFjqdW1FNp5ODu972FZVi71FsV0Rc8VtZPBRg66icwHDhOskQtnO GdsTfOYD8nH27g1CnUpsT9+qiJOIvAk9iuGa9Ydu2EsK69MaC9nUVsHR54nlpG/xGkwi LzYwkYDUx7599Mngw7O8O1tqSSQBTnBvYa48ffoqaY1kXkJFkjM9y/tPyJEjVhlhsHvk 5DYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mLfGvQ1o; spf=pass (google.com: domain of linux-kernel+bounces-14491-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14491-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id r17-20020a0c9e91000000b0067f79531c05si27158514qvd.593.2024.01.02.06.40.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:40:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14491-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mLfGvQ1o; spf=pass (google.com: domain of linux-kernel+bounces-14491-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14491-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0F3B41C20257 for ; Tue, 2 Jan 2024 14:40:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 203F61549C; Tue, 2 Jan 2024 14:38:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mLfGvQ1o" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE26C13AC5; Tue, 2 Jan 2024 14:38:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206321; x=1735742321; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cNouUsps1r+dmbqyfgc6rMPqSKe8Zay6ScVLkagmhB4=; b=mLfGvQ1oGLyRsScjnIlEtOapSxNOYog+kRTnWCF7D5O1vjMH9wVwjMlo NSSEfRoz8HrguRLDUe8q5wC5Togkbx9B6lRhVS6bky1OSVFmwwUD5tLIB 1pUszp+/zigjCOLMM8fsxrc93Rkp5GLO4L1c/V0WgaJSRCrhV/lMBUdL6 pgG9HuyfUFga1w5Cs13tDa/rX8yNnx8kZhCZf0HE4YirkODYA1qoNTRGE vFgKbmUwITJkhqagdznIPBUWkTIyQjeeA3EfpskkBFGY/VnqMk5kgHHSR 6oCURWDcYAJ9ed2v5xcuZyyEbDsGQ4dr0xIUszQnwxFMm9Kis5o6SKNk3 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270425" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270425" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234264" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234264" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:39 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 02/10] iommufd: Add IOMMU_HWPT_INVALIDATE Date: Tue, 2 Jan 2024 06:38:26 -0800 Message-Id: <20240102143834.146165-3-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786989947323501831 X-GMAIL-MSGID: 1786989947323501831 In nested translation, the stage-1 page table is user-managed but cached by the IOMMU hardware, so an update on present page table entries in the stage-1 page table should be followed with a cache invalidation. Add an IOMMU_HWPT_INVALIDATE ioctl to support such a cache invalidation. It takes hwpt_id to specify the iommu_domain, and a multi-entry array to support multiple invalidation data in one ioctl. enum iommu_hwpt_invalidate_data_type is defined to tag the data type of the entries in the multi-entry array. Reviewed-by: Kevin Tian Co-developed-by: Nicolin Chen Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- drivers/iommu/iommufd/hw_pagetable.c | 41 +++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 10 ++++++ drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 43 +++++++++++++++++++++++++ 4 files changed, 97 insertions(+) diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index cbb5df0a6c32..4e8711f19f72 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -371,3 +371,44 @@ int iommufd_hwpt_get_dirty_bitmap(struct iommufd_ucmd *ucmd) iommufd_put_object(ucmd->ictx, &hwpt_paging->common.obj); return rc; } + +int iommufd_hwpt_invalidate(struct iommufd_ucmd *ucmd) +{ + struct iommu_hwpt_invalidate *cmd = ucmd->cmd; + struct iommu_user_data_array data_array = { + .type = cmd->data_type, + .uptr = u64_to_user_ptr(cmd->data_uptr), + .entry_len = cmd->entry_len, + .entry_num = cmd->entry_num, + }; + struct iommufd_hw_pagetable *hwpt; + u32 done_num = 0; + int rc; + + if (cmd->__reserved) { + rc = -EOPNOTSUPP; + goto out; + } + + if (cmd->entry_num && (!cmd->data_uptr || !cmd->entry_len)) { + rc = -EINVAL; + goto out; + } + + hwpt = iommufd_get_hwpt_nested(ucmd, cmd->hwpt_id); + if (IS_ERR(hwpt)) { + rc = PTR_ERR(hwpt); + goto out; + } + + rc = hwpt->domain->ops->cache_invalidate_user(hwpt->domain, + &data_array); + done_num = data_array.entry_num; + + iommufd_put_object(ucmd->ictx, &hwpt->obj); +out: + cmd->entry_num = done_num; + if (iommufd_ucmd_respond(ucmd, sizeof(*cmd))) + return -EFAULT; + return rc; +} diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index abae041e256f..991f864d1f9b 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -328,6 +328,15 @@ iommufd_get_hwpt_paging(struct iommufd_ucmd *ucmd, u32 id) IOMMUFD_OBJ_HWPT_PAGING), struct iommufd_hwpt_paging, common.obj); } + +static inline struct iommufd_hw_pagetable * +iommufd_get_hwpt_nested(struct iommufd_ucmd *ucmd, u32 id) +{ + return container_of(iommufd_get_object(ucmd->ictx, id, + IOMMUFD_OBJ_HWPT_NESTED), + struct iommufd_hw_pagetable, obj); +} + int iommufd_hwpt_set_dirty_tracking(struct iommufd_ucmd *ucmd); int iommufd_hwpt_get_dirty_bitmap(struct iommufd_ucmd *ucmd); @@ -345,6 +354,7 @@ void iommufd_hwpt_paging_abort(struct iommufd_object *obj); void iommufd_hwpt_nested_destroy(struct iommufd_object *obj); void iommufd_hwpt_nested_abort(struct iommufd_object *obj); int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd); +int iommufd_hwpt_invalidate(struct iommufd_ucmd *ucmd); static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx, struct iommufd_hw_pagetable *hwpt) diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index c9091e46d208..39b32932c61e 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -322,6 +322,7 @@ union ucmd_buffer { struct iommu_hw_info info; struct iommu_hwpt_alloc hwpt; struct iommu_hwpt_get_dirty_bitmap get_dirty_bitmap; + struct iommu_hwpt_invalidate cache; struct iommu_hwpt_set_dirty_tracking set_dirty_tracking; struct iommu_ioas_alloc alloc; struct iommu_ioas_allow_iovas allow_iovas; @@ -360,6 +361,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { __reserved), IOCTL_OP(IOMMU_HWPT_GET_DIRTY_BITMAP, iommufd_hwpt_get_dirty_bitmap, struct iommu_hwpt_get_dirty_bitmap, data), + IOCTL_OP(IOMMU_HWPT_INVALIDATE, iommufd_hwpt_invalidate, + struct iommu_hwpt_invalidate, __reserved), IOCTL_OP(IOMMU_HWPT_SET_DIRTY_TRACKING, iommufd_hwpt_set_dirty_tracking, struct iommu_hwpt_set_dirty_tracking, __reserved), IOCTL_OP(IOMMU_IOAS_ALLOC, iommufd_ioas_alloc_ioctl, diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 0b2bc6252e2c..824560c50ec6 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -49,6 +49,7 @@ enum { IOMMUFD_CMD_GET_HW_INFO, IOMMUFD_CMD_HWPT_SET_DIRTY_TRACKING, IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP, + IOMMUFD_CMD_HWPT_INVALIDATE, }; /** @@ -613,4 +614,46 @@ struct iommu_hwpt_get_dirty_bitmap { #define IOMMU_HWPT_GET_DIRTY_BITMAP _IO(IOMMUFD_TYPE, \ IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP) +/** + * enum iommu_hwpt_invalidate_data_type - IOMMU HWPT Cache Invalidation + * Data Type + * @IOMMU_HWPT_INVALIDATE_DATA_VTD_S1: Invalidation data for VTD_S1 + */ +enum iommu_hwpt_invalidate_data_type { + IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, +}; + +/** + * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) + * @size: sizeof(struct iommu_hwpt_invalidate) + * @hwpt_id: ID of a nested HWPT for cache invalidation + * @data_uptr: User pointer to an array of driver-specific cache invalidation + * data. + * @data_type: One of enum iommu_hwpt_invalidate_data_type, defining the data + * type of all the entries in the invalidation request array. It + * should be a type supported by the hwpt pointed by @hwpt_id. + * @entry_len: Length (in bytes) of a request entry in the request array + * @entry_num: Input the number of cache invalidation requests in the array. + * Output the number of requests successfully handled by kernel. + * @__reserved: Must be 0. + * + * Invalidate the iommu cache for user-managed page table. Modifications on a + * user-managed page table should be followed by this operation to sync cache. + * Each ioctl can support one or more cache invalidation requests in the array + * that has a total size of @entry_len * @entry_num. + * + * An empty invalidation request array by setting @entry_num==0 is allowed, and + * @entry_len and @data_uptr would be ignored in this case. This can be used to + * check if the given @data_type is supported or not by kernel. + */ +struct iommu_hwpt_invalidate { + __u32 size; + __u32 hwpt_id; + __aligned_u64 data_uptr; + __u32 data_type; + __u32 entry_len; + __u32 entry_num; + __u32 __reserved; +}; +#define IOMMU_HWPT_INVALIDATE _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_INVALIDATE) #endif From patchwork Tue Jan 2 14:38:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184369 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4483495dyb; Tue, 2 Jan 2024 06:40:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IFYPz0bayS6wnP1cANzQxieRGJAbj/PY00hY5pg1owJ/+cPE6I3ey0/ka/AYq/3TzHdRCz1 X-Received: by 2002:a0d:dfd1:0:b0:5e7:ae43:e90f with SMTP id i200-20020a0ddfd1000000b005e7ae43e90fmr9890648ywe.3.1704206442485; Tue, 02 Jan 2024 06:40:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206442; cv=none; d=google.com; s=arc-20160816; b=Uw1fCQoldvODHuau2J+jb2QWP5DWFZ645W+yJsV9SK0JdCbtLC59UP88h3PyN5M3eK PYR0Lda2GtZT/XqnQ0YL/mBN/Vjs1p2tQP+pcRYGvf1kBj2Axk9XMTbicDzqwFOZoL9a 1XTO6Qs2/f+2N/3TtbFGTr36ZMXcxmCTrKVSktUXKPn34somVbhXi1MsJ6WA2BQY89Y7 7yw/WcmweES+j8OxajQyXAd2SCvWd5bbfW77JwqprS26XTL0v1woEZrWt4lXWcT/2vA5 DbaIScTrFKaZWqAnJ40BsJaixDwLlKySNW5wcW80ajbLW6trxfqdluLyXG0g0Wus+XTn pcxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=euG6NwFyvGpQyx4bY1dT20hCi7fPaxrH9e/fkW+iluU=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=p5f8FB4lAGFxXdfp3ar627DInrEeS6GExi0fqLQ2qh8/6D9XxUir3rP2ix2gSKE1eT dwVSjJ8BMuV+yl3pS500g7s4A2nM6uHqd0b+VnJe7pM+0Az4mUwtwAvrBIP2j3cf8AUx xr41hik9lSlNPhUfhSDfaecuQiTGKHsv/ZYCp2+iMebWaHwn7QalQoxW73uKqknqeTsf VWYlx3butTil+CvykONQBzIfaxl5QVPpkMUOxxMbKxZt2QJRXj0cTINMLVPwQmnkkcp7 AIIX37MStdD9Eh6FJcPUkHgFbvDp/3iz0AYJon84xj3jmo8sIG5TrOzcSkbamyuAxuW5 TSfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=a0jfwGRT; spf=pass (google.com: domain of linux-kernel+bounces-14492-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14492-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id b8-20020a0cf048000000b0067f93558a30si25339802qvl.362.2024.01.02.06.40.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:40:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14492-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=a0jfwGRT; spf=pass (google.com: domain of linux-kernel+bounces-14492-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14492-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3AEB61C22064 for ; Tue, 2 Jan 2024 14:40:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A9020156DA; Tue, 2 Jan 2024 14:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="a0jfwGRT" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A72F1428F; Tue, 2 Jan 2024 14:38:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206322; x=1735742322; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a0MtJfDdMrZSQXPjcVw/ypmtgFbBgW6CUZyjsKPTKtk=; b=a0jfwGRTbIaqtdf6y0CU2WneY+Nw6M952ZPe2cpVzBuJBX8pO6Y+bDAG ofCMOAG16BJ/OtgCXZndEPX0Le37OJYCsMFTSXKxT5aquVIiiJbDR25og U9jzUa/wlS9E6j//vGOgjVcmsH612vNyWiwJNLeQXgccOh4qre0z76A8t YHUm5+VDQ+ay03v83NJNUjCYp3UQ5GhGLeWsD+9zt6eUMGWseo1rITre/ EufdOG1AjKRa2kfXbe+w/NQX2PbR+UqHcH149j9EST8iy2rGeafdN6Y2E W3pI2u59NBdaa1QO4J8Oa4uDDDTWbuc0zFT1lU8jmPP2Iqnh8DgGooi5+ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270440" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270440" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234273" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234273" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:40 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 03/10] iommu: Add iommu_copy_struct_from_user_array helper Date: Tue, 2 Jan 2024 06:38:27 -0800 Message-Id: <20240102143834.146165-4-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786989974170757770 X-GMAIL-MSGID: 1786989974170757770 From: Nicolin Chen Wrap up the data pointer/num sanity and __iommu_copy_struct_from_user call for iommu drivers to copy driver specific data at a specific location in the struct iommu_user_data_array, and iommu_respond_struct_to_user_array() to copy response to a specific location in the struct iommu_user_data_array. And expect it to be used in cache_invalidate_user ops for example. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- include/linux/iommu.h | 74 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 93c0d12dd047..c3434c9eaa6d 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -341,6 +341,80 @@ static inline int __iommu_copy_struct_from_user( sizeof(*kdst), \ offsetofend(typeof(*kdst), min_last)) +/** + * __iommu_copy_struct_from_user_array - Copy iommu driver specific user space + * data from an iommu_user_data_array + * @dst_data: Pointer to an iommu driver specific user data that is defined in + * include/uapi/linux/iommufd.h + * @src_array: Pointer to a struct iommu_user_data_array for a user space array + * @data_type: The data type of the @dst_data. Must match with @src_array.type + * @index: Index to the location in the array to copy user data from + * @data_len: Length of current user data structure, i.e. sizeof(struct _dst) + * @min_len: Initial length of user data structure for backward compatibility. + * This should be offsetofend using the last member in the user data + * struct that was initially added to include/uapi/linux/iommufd.h + */ +static inline int +__iommu_copy_struct_from_user_array(void *dst_data, + const struct iommu_user_data_array *src_array, + unsigned int data_type, unsigned int index, + size_t data_len, size_t min_len) +{ + struct iommu_user_data src_data; + + if (WARN_ON(!src_array || index >= src_array->entry_num)) + return -EINVAL; + if (!src_array->entry_num) + return -EINVAL; + src_data.uptr = src_array->uptr + src_array->entry_len * index; + src_data.len = src_array->entry_len; + src_data.type = src_array->type; + + return __iommu_copy_struct_from_user(dst_data, &src_data, data_type, + data_len, min_len); +} + +/** + * iommu_copy_struct_from_user_array - Copy iommu driver specific user space + * data from an iommu_user_data_array + * @kdst: Pointer to an iommu driver specific user data that is defined in + * include/uapi/linux/iommufd.h + * @user_array: Pointer to a struct iommu_user_data_array for a user space + * array + * @data_type: The data type of the @kdst. Must match with @user_array->type + * @index: Index to the location in the array to copy user data from + * @min_last: The last memember of the data structure @kdst points in the + * initial version. + * Return 0 for success, otherwise -error. + */ +#define iommu_copy_struct_from_user_array(kdst, user_array, data_type, \ + index, min_last) \ + __iommu_copy_struct_from_user_array(kdst, user_array, data_type, \ + index, sizeof(*kdst), \ + offsetofend(typeof(*kdst), \ + min_last)) + +/** + * iommu_respond_struct_to_user_array - Copy the response in @ksrc back to + * a specific entry of user array + * @user_array: Pointer to a struct iommu_user_data_array for a user space + * array + * @index: Index to the location in the array to copy response + * @ksrc: Pointer to kernel structure + * @klen: Length of @ksrc struct + * + * This only copies response of one entry (@index) in @user_array. + */ +static inline int +iommu_respond_struct_to_user_array(const struct iommu_user_data_array *array, + unsigned int index, void *ksrc, size_t klen) +{ + if (copy_to_user(array->uptr + array->entry_len * index, + ksrc, min_t(size_t, array->entry_len, klen))) + return -EFAULT; + return 0; +} + /** * struct iommu_ops - iommu ops and capabilities * @capable: check capability From patchwork Tue Jan 2 14:38:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184370 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4483724dyb; Tue, 2 Jan 2024 06:41:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IHc+cX/spAzB6LHRqChH/CagSFRm5tG03pOQBv0klAllmWRBNY2Ao2w7YPvW/iXINlYt+a5 X-Received: by 2002:a50:9b03:0:b0:554:526d:894e with SMTP id o3-20020a509b03000000b00554526d894emr10828889edi.1.1704206469379; Tue, 02 Jan 2024 06:41:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206469; cv=none; d=google.com; s=arc-20160816; b=UuMd3acGtFMDEq81FgNd4Y0y2CuQOpMci2DnNX3XJZ1l1gkgcrZ6us2MOCQbplkXX6 czVPevRc4815ts8WwMJQ3kY4rDQ3CWaTrudT2ozoJHrOPrWmWxhPOjttMWDNDJx5J6+G a8uXA9mh7451K6SsGfLGP4oZWd/pOv2CHRPW1sZenB+sRccBapC8S4bluQW5cRnSuWtZ 4HR5v6YNIUc9bKoUTeEC5vG4YjOEHbl8Gx6jV4Cal43lAbaMJGwotdnuxvX6zVs6rAGt hxt3LvhCmM4OEyJyAHwVfwety0xFs46xO3O3e9Xcn5FbypuMHVpmA6OGwnhMC3btvJ5w q96w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=yNPeBnelu5/Jdu09XB17bQVfbxsyBYVzS5LQqInjiLI=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=LTp4XPax3qAdh3q08m0awvtUTszVi6JVd8qp6hcc2uWHbX2gKZmO68Zdzv7Kav7W99 4blMtHzHnubfJ3MArqkz7tBXL1JPHP42YzI7rNndX/oLkD2q/+4Ck5QYvnixIj7RNCQe dA8BzjLcTJRF2TT/SkSQ6DHjg+EXQVODSe/xDz5q5ZryvMugSujYcNqGH42wDpOr94/V yFU7YRh9J+/GCMpAg/2tOny9s8AUF8eYPK8j4e/K7UaCOGyI5L+b6X7NaJsZ0DSYVfBm E2+cqqZbwBF4oEv3WQaY2cfnGKKdvmkGpwgOkkmkFk0FM1AJuFvbLSrbfO5GB3YnpK4b ZpTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=NKjrrFqA; spf=pass (google.com: domain of linux-kernel+bounces-14493-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14493-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id 27-20020a508e5b000000b005529d0207eesi11324696edx.377.2024.01.02.06.41.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:41:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14493-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=NKjrrFqA; spf=pass (google.com: domain of linux-kernel+bounces-14493-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14493-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 016641F2126A for ; Tue, 2 Jan 2024 14:41:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 18CAC15ACC; Tue, 2 Jan 2024 14:38:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="NKjrrFqA" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF74114F86; Tue, 2 Jan 2024 14:38:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206324; x=1735742324; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YaKsYJmpoP1J2+af2wMppPzhvG3pSWEOhFLhzXoh4BU=; b=NKjrrFqAmZsnkQYIWVO0nWK3VOoM16tuU+pYHpC417tnb3pquNUHLJvF kqmRAIxsIPjuE2TExJvnI+J4WNl9kwnJRM2hFXzpK92isQ2birSpOakp5 jWnpnXjg6G8U/RqRfqOGoJZJhVPBvrFp5qgUVlfkfkZ4OjXeul4TA3qAg 96mSssRgdmtAsNZFMDHIe4whl4DfTeTMMGwqnOYqtsIuAu8uFLIXwbI6j zFwnAsoxGh9TETCw0b/Y+t/lqyWL1Qu68U+wh9K3XAv8eVKwlN8ukrniv tsaWLE9iF5G7qzUL1EKJYn9STGqid3dFC+BkIUdNK92Qnmz/H9KMPytbu w==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270453" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270453" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234282" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234282" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:42 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 04/10] iommufd/selftest: Add mock_domain_cache_invalidate_user support Date: Tue, 2 Jan 2024 06:38:28 -0800 Message-Id: <20240102143834.146165-5-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990003011885769 X-GMAIL-MSGID: 1786990003011885769 From: Nicolin Chen Add mock_domain_cache_invalidate_user() data structure to support user space selftest program to cover user cache invalidation pathway. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- drivers/iommu/iommufd/iommufd_test.h | 34 ++++++++++++++++ drivers/iommu/iommufd/selftest.c | 60 ++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index 7910fbe1962d..2eef5afde711 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -148,4 +148,38 @@ struct iommu_hwpt_selftest { __u32 iotlb; }; +/* Should not be equal to any defined value in enum iommu_hwpt_invalidate_data_type */ +#define IOMMU_HWPT_INVALIDATE_DATA_SELFTEST 0xdeadbeef +#define IOMMU_HWPT_INVALIDATE_DATA_SELFTEST_INVALID 0xdadbeef + +/** + * enum iommu_hwpt_invalidate_selftest_error - Hardware error of invalidation + * @IOMMU_TEST_INVALIDATE_FAKE_ERROR: Fake hw error per test program's request + */ +enum iommu_hwpt_invalidate_selftest_error { + IOMMU_TEST_INVALIDATE_FAKE_ERROR = (1 << 0) +}; + +/** + * struct iommu_hwpt_invalidate_selftest - Invalidation data for Mock driver + * (IOMMU_HWPT_INVALIDATE_DATA_SELFTEST) + * @flags: Invalidate flags + * @iotlb_id: Invalidate iotlb entry index + * @hw_error: One of enum iommu_hwpt_invalidate_selftest_error + * @__reserved: Must be 0 + * + * If IOMMU_TEST_INVALIDATE_ALL is set in @flags, @iotlb_id will be ignored + * @hw_error meaningful only if the request is processed successfully. + * If IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR is set in @flags, report a + * hw error back, cache would not be invalidated in this case. + */ +struct iommu_hwpt_invalidate_selftest { +#define IOMMU_TEST_INVALIDATE_FLAG_ALL (1 << 0) +#define IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR (1 << 1) + __u32 flags; + __u32 iotlb_id; + __u32 hw_error; + __u32 __reserved; +}; + #endif diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index 022ef8f55088..ebc6c15abf67 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -473,9 +473,69 @@ static void mock_domain_free_nested(struct iommu_domain *domain) kfree(mock_nested); } +static int +mock_domain_cache_invalidate_user(struct iommu_domain *domain, + struct iommu_user_data_array *array) +{ + struct mock_iommu_domain_nested *mock_nested = + container_of(domain, struct mock_iommu_domain_nested, domain); + u32 hw_error = 0, processed = 0; + struct iommu_hwpt_invalidate_selftest inv; + int i = 0, j; + int rc = 0; + + if (array->type != IOMMU_HWPT_INVALIDATE_DATA_SELFTEST) { + rc = -EINVAL; + goto out; + } + + for ( ; i < array->entry_num; i++) { + rc = iommu_copy_struct_from_user_array(&inv, array, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + i, __reserved); + if (rc) + break; + + if ((inv.flags & ~(IOMMU_TEST_INVALIDATE_FLAG_ALL | + IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR)) || + inv.__reserved) { + rc = -EOPNOTSUPP; + break; + } + + if (inv.iotlb_id > MOCK_NESTED_DOMAIN_IOTLB_ID_MAX) { + rc = -EINVAL; + break; + } + + if (inv.flags & IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR) { + hw_error = IOMMU_TEST_INVALIDATE_FAKE_ERROR; + } else if (inv.flags & IOMMU_TEST_INVALIDATE_FLAG_ALL) { + /* Invalidate all mock iotlb entries and ignore iotlb_id */ + for (j = 0; j < MOCK_NESTED_DOMAIN_IOTLB_NUM; j++) + mock_nested->iotlb[j] = 0; + } else { + mock_nested->iotlb[inv.iotlb_id] = 0; + } + + inv.hw_error = hw_error; + rc = iommu_respond_struct_to_user_array(array, i, (void *)&inv, + sizeof(inv)); + if (rc) + break; + + processed++; + } + +out: + array->entry_num = processed; + return rc; +} + static struct iommu_domain_ops domain_nested_ops = { .free = mock_domain_free_nested, .attach_dev = mock_domain_nop_attach, + .cache_invalidate_user = mock_domain_cache_invalidate_user, }; static inline struct iommufd_hw_pagetable * From patchwork Tue Jan 2 14:38:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184371 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4483996dyb; Tue, 2 Jan 2024 06:41:37 -0800 (PST) X-Google-Smtp-Source: AGHT+IHKwiEYOg1TFSP9il2q03wv8mxpbXESn1ywoLtHy+bQ7JvvpzT/pDaQeEyeX+Vf982n2cCI X-Received: by 2002:a17:906:5301:b0:a27:4218:792a with SMTP id h1-20020a170906530100b00a274218792amr2076429ejo.20.1704206497273; Tue, 02 Jan 2024 06:41:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206497; cv=none; d=google.com; s=arc-20160816; b=rC4WipThf0sCpHOiBJgO1Yto9NY3jslmeXpb1hZG7eXQnxyHK/+DZCe/USDDlWpYkD 9alTR0QqflGT0lcc3uYcW8VwkNp3g17YIJKem/bLOmgN8ARfh7f+mmW4zEHvKqUanFfv aLCJur8zePWxqvlvyTiMETyWJRZHo7bIFEKAevnOyahNdUcC+OZ2wl7P5QWJwZofNDB2 +jnkMvxgp+qRtsivr1VDbbRGfJHHvY8/vAfdh3saqbETRVhknBansBIrv/i1bodDNYPk Yy7TopTup0wi+u4LZ0I4k+jXoCRSj0Zs9Zw6Q6aOPLa8JYaZss4BJHecmIZMBrZOXKSm bxdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=R1Pq9t0df0nCyf89XYLwYiW7B8f68Ipv1K1w/6MQoXY=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=rsLEGhVd+/4NN9p1oUdN22L0GfxQRl80TObMKAcUMrf720KfYF+e1JFKSCQGw2KD4j EvkTpTKY6zygq/+EzRk0YK+VBa/dJ0qEqXLrMh5vRNbxaxy4FKiJt2+vhDq5M9kQmpdX TlvRb/XhWGoGLDtdIpgV9AI+rE8Radf8CsnGKas9W6ZyZbEeHdxjcgRQ8lfZYupNl3OL AEzjw4/z8qi4cyjqtuBTDbakSwocsobRlAxo4yx6hOrpdsXOUem8NaRe2+KhYcnQvNov bIjLXlCEp0J3LDYierdtNCR74b821MLoPqj580o/gw2cSDMztIGLxcnOUTbkM5xqL/9W HDrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AOOsaDKn; spf=pass (google.com: domain of linux-kernel+bounces-14494-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14494-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id fw10-20020a170907500a00b00a27a3bcdb9bsi3011635ejc.867.2024.01.02.06.41.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:41:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14494-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AOOsaDKn; spf=pass (google.com: domain of linux-kernel+bounces-14494-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14494-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id AE3051F22A3B for ; Tue, 2 Jan 2024 14:41:36 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 789A915E97; Tue, 2 Jan 2024 14:38:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AOOsaDKn" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 146241549B; Tue, 2 Jan 2024 14:38:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206325; x=1735742325; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=w9DQy+hnxhoJ51h3H6W2Kl69R5yaKXv0XDk0dB+JKAk=; b=AOOsaDKn1xJ4f2Zoafgn6VuDoxSuL2bG9G738ueL26RbUqdLrn3sEbMy /ZvR0XkX5GbXLDKkO6lpiB/DdBd4C2/nM0a0V0ed7XDA3oMYOrp5Dy/+L BMOZeO8QkktVhXM52Sf5fDhkaqnEYOAJdW/ywjuG0aC2q3TTBxn+RY1Qp YEx9atFg27aodetQm0CPJUkMSQip3XtSMOeU5KVT7ikD9t2sZz8VvJmbH huhxX/laM6QkC59GLlQxcQ8gMQLaohlpjo7R74jdZiCa3LI7jy3KJZpwz b9JjOl+S93uo2df21o9mpOC1Qg8wbiZ+ClynaU69A2cgh3oJ1xLY7sEKI Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270466" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270466" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234292" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234292" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:43 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 05/10] iommufd/selftest: Add IOMMU_TEST_OP_MD_CHECK_IOTLB test op Date: Tue, 2 Jan 2024 06:38:29 -0800 Message-Id: <20240102143834.146165-6-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990031565904220 X-GMAIL-MSGID: 1786990031565904220 From: Nicolin Chen Allow to test whether IOTLB has been invalidated or not. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- drivers/iommu/iommufd/iommufd_test.h | 5 ++++ drivers/iommu/iommufd/selftest.c | 26 +++++++++++++++++++ tools/testing/selftests/iommu/iommufd.c | 4 +++ tools/testing/selftests/iommu/iommufd_utils.h | 24 +++++++++++++++++ 4 files changed, 59 insertions(+) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index 2eef5afde711..1cedd6b5ba2b 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -21,6 +21,7 @@ enum { IOMMU_TEST_OP_ACCESS_REPLACE_IOAS, IOMMU_TEST_OP_MOCK_DOMAIN_FLAGS, IOMMU_TEST_OP_DIRTY, + IOMMU_TEST_OP_MD_CHECK_IOTLB, }; enum { @@ -121,6 +122,10 @@ struct iommu_test_cmd { __aligned_u64 uptr; __aligned_u64 out_nr_dirty; } dirty; + struct { + __u32 id; + __u32 iotlb; + } check_iotlb; }; __u32 last; }; diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index ebc6c15abf67..9528528cab27 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -853,6 +853,28 @@ static int iommufd_test_md_check_refs(struct iommufd_ucmd *ucmd, return 0; } +static int iommufd_test_md_check_iotlb(struct iommufd_ucmd *ucmd, + u32 mockpt_id, unsigned int iotlb_id, + u32 iotlb) +{ + struct mock_iommu_domain_nested *mock_nested; + struct iommufd_hw_pagetable *hwpt; + int rc = 0; + + hwpt = get_md_pagetable_nested(ucmd, mockpt_id, &mock_nested); + if (IS_ERR(hwpt)) + return PTR_ERR(hwpt); + + mock_nested = container_of(hwpt->domain, + struct mock_iommu_domain_nested, domain); + + if (iotlb_id > MOCK_NESTED_DOMAIN_IOTLB_ID_MAX || + mock_nested->iotlb[iotlb_id] != iotlb) + rc = -EINVAL; + iommufd_put_object(ucmd->ictx, &hwpt->obj); + return rc; +} + struct selftest_access { struct iommufd_access *access; struct file *file; @@ -1334,6 +1356,10 @@ int iommufd_test(struct iommufd_ucmd *ucmd) return iommufd_test_md_check_refs( ucmd, u64_to_user_ptr(cmd->check_refs.uptr), cmd->check_refs.length, cmd->check_refs.refs); + case IOMMU_TEST_OP_MD_CHECK_IOTLB: + return iommufd_test_md_check_iotlb(ucmd, cmd->id, + cmd->check_iotlb.id, + cmd->check_iotlb.iotlb); case IOMMU_TEST_OP_CREATE_ACCESS: return iommufd_test_create_access(ucmd, cmd->id, cmd->create_access.flags); diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index 6ed328c863c4..c8763b880a16 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -330,6 +330,10 @@ TEST_F(iommufd_ioas, alloc_hwpt_nested) &nested_hwpt_id[1], IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data)); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[0], + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[1], + IOMMU_TEST_IOTLB_DEFAULT); /* Negative test: a nested hwpt on top of a nested hwpt */ test_err_hwpt_alloc_nested(EINVAL, self->device_id, diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index ad9202335656..fe0a0f566b67 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -195,6 +195,30 @@ static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, _test_cmd_hwpt_alloc(self->fd, device_id, pt_id, flags, \ hwpt_id, data_type, data, data_len)) +#define test_cmd_hwpt_check_iotlb(hwpt_id, iotlb_id, expected) \ + ({ \ + struct iommu_test_cmd test_cmd = { \ + .size = sizeof(test_cmd), \ + .op = IOMMU_TEST_OP_MD_CHECK_IOTLB, \ + .id = hwpt_id, \ + .check_iotlb = { \ + .id = iotlb_id, \ + .iotlb = expected, \ + }, \ + }; \ + ASSERT_EQ(0, \ + ioctl(self->fd, \ + _IOMMU_TEST_CMD(IOMMU_TEST_OP_MD_CHECK_IOTLB), \ + &test_cmd)); \ + }) + +#define test_cmd_hwpt_check_iotlb_all(hwpt_id, expected) \ + ({ \ + int i; \ + for (i = 0; i < MOCK_NESTED_DOMAIN_IOTLB_NUM; i++) \ + test_cmd_hwpt_check_iotlb(hwpt_id, i, expected); \ + }) + static int _test_cmd_access_replace_ioas(int fd, __u32 access_id, unsigned int ioas_id) { From patchwork Tue Jan 2 14:38:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184372 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4484255dyb; Tue, 2 Jan 2024 06:42:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IGCs8m8ou+N3rdoVZF0sh+dsq8xvUHQ2o2Ij2hpO9dyAMdX51FTT/EfYWy7N0IBfAa40R2A X-Received: by 2002:a05:6359:1b93:b0:173:50b:26ed with SMTP id ur19-20020a0563591b9300b00173050b26edmr13029017rwb.36.1704206528197; Tue, 02 Jan 2024 06:42:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206528; cv=none; d=google.com; s=arc-20160816; b=ItnRdn4tAeUnzZHnmNvIDDlrKNS/wZusHTD8bo88/oOPhlkkqGFqdpfbYxni2kZyXk Ba5XFSl53gltiGIDCmJTuFbSI/CBOs/qgNtHFyqyc4k47H4AlV1Je/Gd1vG2VjyyIpzL qev+T6ECyV2rXhPr/Suv1HxWte/LtnfGzb1ETs5sCi7XW2YZtCuZDFY/15ATZYwWt+P4 6h6WeqQ8c9tSj/dv9QGpzhQdlpp7mGMn79MpQkujvZuRrfSkRZY3kzfhL/7mC4BaTqco PnB+s0KdnR6lfsixZ99Z/kMR4aCzsJByQWmrnLRAqR2hsEBKwd5IvOUDCd3evgvIhBaJ bUYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=OyEOktip7wvyaDgnq8oGd4eiLr6iHThnb83RM4icWuY=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=qtAzk93ESvtRwKJUMjEgfEUnh+E5x9noZI4hCLPZpjHxw5kviVrUwnsyEW9f5szhzi 9gdnYIbt7+YCtrAHjT4GuLBLQQcSgTQSXRlPKeUV09bCOv5MPml/VgI9hSJrqnrl+Lgg cDGaCLuYHZfuzGaXVtjZchUs9XQ0iDKK5EUBsjG//G1kFzGsyV6eUob1qea6tLXomeu0 1ub2ZJzFWKAvS8u4uHL+05uGrJ2+O+Nzo7UCprsfeg+vkegK+99huvv6F0o1ywW4vnfX 8oSc1lr2i+NG40zoOUGldNHH6UPY4RFSbCtc8T5cuQEDBulQRtTypDKKzeNrQdoLPhYP GN4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=F2H6YYhF; spf=pass (google.com: domain of linux-kernel+bounces-14495-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14495-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id p16-20020a17090a429000b0028c7d7e0d2asi11762834pjg.43.2024.01.02.06.42.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:42:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14495-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=F2H6YYhF; spf=pass (google.com: domain of linux-kernel+bounces-14495-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14495-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E882128391A for ; Tue, 2 Jan 2024 14:42:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8E60216427; Tue, 2 Jan 2024 14:38:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F2H6YYhF" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A6A4156C7; Tue, 2 Jan 2024 14:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206326; x=1735742326; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xD/D1mwaQz0aVDoqEolYgPt9rT0WuEPycyZdvUJpam8=; b=F2H6YYhF1sJZnqgZi3qNh5fBpIxaUhUi3RqbqyEIfrFXTYZIdRLQkDw0 YFJnNr8yc/XC7Kg5Z6o+0tL6YQhDe9Fqttg0EWxkW/ZRlifKcaTaVVkwU fMvBavPGmKaVZLNIJsKlw1oiKx9QVU94bsZLC1KZP1ku0GGVcnzn3UhQ5 MS9FnsZzSoIdCcpsCB31BF2I1McidVjwuqgp+n+/dPt0M5Ne2w66JfkO9 4WWLTC2mKRCjDcyOSN4xx15TQU0xdphmYz7HEloT5bb1HU9/OFh1ekUAP 9Qar1JDvQ13aCe9b2M9TY/CHjWn7JT65Ze5GrftDldoNeFoLYaugJcQ/l g==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270478" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270478" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234300" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234300" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:44 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 06/10] iommufd/selftest: Add coverage for IOMMU_HWPT_INVALIDATE ioctl Date: Tue, 2 Jan 2024 06:38:30 -0800 Message-Id: <20240102143834.146165-7-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990064469523550 X-GMAIL-MSGID: 1786990064469523550 From: Nicolin Chen Add test cases for the IOMMU_HWPT_INVALIDATE ioctl and verify it by using the new IOMMU_TEST_OP_MD_CHECK_IOTLB. Reviewed-by: Kevin Tian Signed-off-by: Nicolin Chen Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- tools/testing/selftests/iommu/iommufd.c | 186 ++++++++++++++++++ tools/testing/selftests/iommu/iommufd_utils.h | 33 ++++ 2 files changed, 219 insertions(+) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index c8763b880a16..ff0aa77911f3 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -116,6 +116,7 @@ TEST_F(iommufd, cmd_length) TEST_LENGTH(iommu_destroy, IOMMU_DESTROY, id); TEST_LENGTH(iommu_hw_info, IOMMU_GET_HW_INFO, __reserved); TEST_LENGTH(iommu_hwpt_alloc, IOMMU_HWPT_ALLOC, __reserved); + TEST_LENGTH(iommu_hwpt_invalidate, IOMMU_HWPT_INVALIDATE, __reserved); TEST_LENGTH(iommu_ioas_alloc, IOMMU_IOAS_ALLOC, out_ioas_id); TEST_LENGTH(iommu_ioas_iova_ranges, IOMMU_IOAS_IOVA_RANGES, out_iova_alignment); @@ -271,7 +272,9 @@ TEST_F(iommufd_ioas, alloc_hwpt_nested) struct iommu_hwpt_selftest data = { .iotlb = IOMMU_TEST_IOTLB_DEFAULT, }; + struct iommu_hwpt_invalidate_selftest inv_reqs[2] = {}; uint32_t nested_hwpt_id[2] = {}; + uint32_t num_inv; uint32_t parent_hwpt_id = 0; uint32_t parent_hwpt_id_not_work = 0; uint32_t test_hwpt_id = 0; @@ -344,6 +347,189 @@ TEST_F(iommufd_ioas, alloc_hwpt_nested) EXPECT_ERRNO(EBUSY, _test_ioctl_destroy(self->fd, parent_hwpt_id)); + /* hwpt_invalidate only supports a user-managed hwpt (nested) */ + num_inv = 1; + test_err_hwpt_invalidate(ENOENT, parent_hwpt_id, inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Check data_type by passing zero-length array */ + num_inv = 0; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: Invalid data_type */ + num_inv = 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST_INVALID, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: structure size sanity */ + num_inv = 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs) + 1, &num_inv); + assert(!num_inv); + + num_inv = 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + 1, &num_inv); + assert(!num_inv); + + /* Negative test: invalid flag is passed */ + num_inv = 1; + inv_reqs[0].flags = 0xffffffff; + test_err_hwpt_invalidate(EOPNOTSUPP, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: non-zero __reserved is passed */ + num_inv = 1; + inv_reqs[0].flags = 0; + inv_reqs[0].__reserved = 0x1234; + test_err_hwpt_invalidate(EOPNOTSUPP, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid data_uptr when array is not empty */ + num_inv = 1; + inv_reqs[0].flags = 0; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], NULL, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: invalid entry_len when array is not empty */ + num_inv = 1; + inv_reqs[0].flags = 0; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + 0, &num_inv); + assert(!num_inv); + + /* Negative test: invalid iotlb_id */ + num_inv = 1; + inv_reqs[0].flags = 0; + inv_reqs[0].__reserved = 0; + inv_reqs[0].iotlb_id = MOCK_NESTED_DOMAIN_IOTLB_ID_MAX + 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!num_inv); + + /* Negative test: trigger error */ + num_inv = 1; + inv_reqs[0].flags = IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR; + inv_reqs[0].iotlb_id = 0; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(inv_reqs[0].hw_error == IOMMU_TEST_INVALIDATE_FAKE_ERROR); + + /* Negative test: trigger error with ALL flag set */ + num_inv = 1; + inv_reqs[0].flags = IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR | + IOMMU_TEST_INVALIDATE_FLAG_ALL; + inv_reqs[0].iotlb_id = 0; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(inv_reqs[0].hw_error == IOMMU_TEST_INVALIDATE_FAKE_ERROR); + + /* + * Invalidate the 1st iotlb entry but fail the 2nd request + * - mock driver error, the hw_error field is meaningful, + * the ioctl returns 0. + */ + num_inv = 2; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 0; + inv_reqs[1].flags = IOMMU_TEST_INVALIDATE_FLAG_TRIGGER_ERROR; + inv_reqs[1].iotlb_id = 1; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 2); + assert(!inv_reqs[0].hw_error); + assert(inv_reqs[1].hw_error == IOMMU_TEST_INVALIDATE_FAKE_ERROR); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 0, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 1, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 2, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 3, + IOMMU_TEST_IOTLB_DEFAULT); + + /* + * Invalidate the 1st iotlb entry but fail the 2nd request + * - ioctl error, the hw_error field is meaningless + */ + num_inv = 2; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 0; + inv_reqs[1].flags = 0; + inv_reqs[1].iotlb_id = MOCK_NESTED_DOMAIN_IOTLB_ID_MAX + 1; + test_err_hwpt_invalidate(EINVAL, nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(!inv_reqs[0].hw_error); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 0, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 1, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 2, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 3, + IOMMU_TEST_IOTLB_DEFAULT); + + /* Invalidate the 2nd iotlb entry and verify */ + num_inv = 1; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 1; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(!inv_reqs[0].hw_error); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 0, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 1, 0); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 2, + IOMMU_TEST_IOTLB_DEFAULT); + test_cmd_hwpt_check_iotlb(nested_hwpt_id[0], 3, + IOMMU_TEST_IOTLB_DEFAULT); + + /* Invalidate the 3rd and 4th iotlb entries and verify */ + num_inv = 2; + inv_reqs[0].flags = 0; + inv_reqs[0].iotlb_id = 2; + inv_reqs[1].flags = 0; + inv_reqs[1].iotlb_id = 3; + test_cmd_hwpt_invalidate(nested_hwpt_id[0], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 2); + assert(!inv_reqs[0].hw_error); + assert(!inv_reqs[1].hw_error); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[0], 0); + + /* Invalidate all iotlb entries for nested_hwpt_id[1] and verify */ + num_inv = 1; + inv_reqs[0].flags = IOMMU_TEST_INVALIDATE_FLAG_ALL; + test_cmd_hwpt_invalidate(nested_hwpt_id[1], inv_reqs, + IOMMU_HWPT_INVALIDATE_DATA_SELFTEST, + sizeof(*inv_reqs), &num_inv); + assert(num_inv == 1); + assert(!inv_reqs[0].hw_error); + test_cmd_hwpt_check_iotlb_all(nested_hwpt_id[1], 0); + /* Attach device to nested_hwpt_id[0] that then will be busy */ test_cmd_mock_domain_replace(self->stdev_id, nested_hwpt_id[0]); EXPECT_ERRNO(EBUSY, diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index fe0a0f566b67..7f41fb796a8a 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -219,6 +219,39 @@ static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, test_cmd_hwpt_check_iotlb(hwpt_id, i, expected); \ }) +static int _test_cmd_hwpt_invalidate(int fd, __u32 hwpt_id, void *reqs, + uint32_t data_type, uint32_t lreq, + uint32_t *nreqs) +{ + struct iommu_hwpt_invalidate cmd = { + .size = sizeof(cmd), + .hwpt_id = hwpt_id, + .data_type = data_type, + .data_uptr = (uint64_t)reqs, + .entry_len = lreq, + .entry_num = *nreqs, + }; + int rc = ioctl(fd, IOMMU_HWPT_INVALIDATE, &cmd); + *nreqs = cmd.entry_num; + return rc; +} + +#define test_cmd_hwpt_invalidate(hwpt_id, reqs, data_type, lreq, nreqs) \ + ({ \ + ASSERT_EQ(0, \ + _test_cmd_hwpt_invalidate(self->fd, hwpt_id, reqs, \ + data_type, \ + lreq, nreqs)); \ + }) +#define test_err_hwpt_invalidate(_errno, hwpt_id, reqs, data_type, lreq, \ + nreqs) \ + ({ \ + EXPECT_ERRNO(_errno, \ + _test_cmd_hwpt_invalidate(self->fd, hwpt_id, \ + reqs, data_type, \ + lreq, nreqs)); \ + }) + static int _test_cmd_access_replace_ioas(int fd, __u32 access_id, unsigned int ioas_id) { From patchwork Tue Jan 2 14:38:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184373 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4484556dyb; Tue, 2 Jan 2024 06:42:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IH0JhOIS/Zl9wugpac5h1Oek8AckmYvlpZ7RCsea4D8QMAYpXzArF29CQCHc8/pTJwGL2AX X-Received: by 2002:a19:e011:0:b0:50e:2bc2:33a0 with SMTP id x17-20020a19e011000000b0050e2bc233a0mr2928749lfg.186.1704206561585; Tue, 02 Jan 2024 06:42:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206561; cv=none; d=google.com; s=arc-20160816; b=cB9CiwC0K0RnbjS2UKaNoYgF16I3mFq1ZB0f4XB+SdzMaIiYxauaBg0RpJqT9OlDOV 7ebvU6w7kRjLL9cjGBBk2tAka4SxtzHZ/LPRwTd35LFCmHeGQoEqbTAxkE9HbgboeYKy W9O1eNmtGF5uguBZSMoVEmhpzGzttRDC9fsmidbEqSm7+urpeFkNDc+EHU2cPSlULGGC 5bfzCI1z0D5JwOSN4xOkc6xXUlPDz1W8viOgKeDek3K7OJfa++S+VGZY4mQ5x0yAG28s k1Ou8sab+v3chthdDLfzTyeSEOQYEj9xKV4xnj+Cadf5jvxr317PSIeglyBn1insBZlH 3DMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=UdYp/uZsmRnnOoPdAiwHolTtssNK4HIJJ+KuAFI1uSg=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=UkrTkTBq8EZ+bd2zrkHVVLL/ryzHXUhtSdvs7VgLeqfOMhmM/WrSjDfXUlFO/GX2EW iol7QBeWsthEz7lFFoGjBpuslN1MjejSUiur4eZF+NcsDIazb10BviydW+tpH2cj4ENM cLRelPyv7ouwSLnCISvMZN7AkMFU8rhT5+vTuNdksNMpQU1707ZgJaDs992RbtNATfAn RBMOwg/AzC6zmOsxjuPw3bHMK3l7+b1t3ydvZ6XzIscHm5dm1fkSD4Vd9rwe2gs5HPpT ClHI7QYtJassBIcg2MQsAZNFKq4Yu84AJSSV1FJkeLrS/DGd92tKKOsSaLVji8GPD7nP 5L3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SdLgbPm5; spf=pass (google.com: domain of linux-kernel+bounces-14496-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14496-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id wn2-20020a170907068200b00a26f3d5af61si6973131ejb.846.2024.01.02.06.42.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:42:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14496-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SdLgbPm5; spf=pass (google.com: domain of linux-kernel+bounces-14496-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14496-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 091901F22DCE for ; Tue, 2 Jan 2024 14:42:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9B369168CA; Tue, 2 Jan 2024 14:38:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SdLgbPm5" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3DDB0156F0; Tue, 2 Jan 2024 14:38:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206327; x=1735742327; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pEYBNa0M9lZzJ/nWD3V7BKxpMiETKpIwIqr86MnTO2s=; b=SdLgbPm5bgFc+GQLBVR9dGVphI93+U2OixdDF6SBG5F2hhb6K/mxNk+5 Hh2bjLjwMwMqOjoy8YtgUrHpIBtKyqXQJzlT1o7X33tYd23Ed7Kl4XLcW LANhIuIp+tS0HdocvlcQwEZzlZR5hCz2QiCCmHvZgNoqbZteJFG1G6nJW OmG3IQC9BUTkq7EkJD4GpI7aO8jJic+r7x+4psqy7L2LMsKG5g3eTTAuc xpm7A2iFlT4vau+JbyYOGnvZSDk1cQg+CCe+9Y09a1sKagpdYVv+l2gS+ ONd2YgMzcfEXvIevH2pgpU/J5NwFqqzPZcb7ccRbIHCVbEg9VkmsO5sbw w==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270491" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270491" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234307" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234307" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:45 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 07/10] iommu/vt-d: Allow qi_submit_sync() to return the QI faults Date: Tue, 2 Jan 2024 06:38:31 -0800 Message-Id: <20240102143834.146165-8-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990099381206893 X-GMAIL-MSGID: 1786990099381206893 From: Lu Baolu This allows qi_submit_sync() to return back faults to callers. qi_submit_sync() has a retry when timeout error happens, thus it cannot return timeout error back to the caller. This was discussed in a separate thread [1]. Here we keep it intact and just make sure no retry for the newly added user domain cache invalidation by checking if the caller is interested in the fault or not. [1] https://lore.kernel.org/all/20231228001646.587653-6-haifeng.zhao@linux.intel.com/ Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Signed-off-by: Yi Liu --- drivers/iommu/intel/dmar.c | 35 +++++++++++++++++++---------- drivers/iommu/intel/iommu.h | 2 +- drivers/iommu/intel/irq_remapping.c | 2 +- drivers/iommu/intel/pasid.c | 2 +- drivers/iommu/intel/svm.c | 6 ++--- 5 files changed, 29 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 23cb80d62a9a..c3251c36c567 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1267,7 +1267,8 @@ static void qi_dump_fault(struct intel_iommu *iommu, u32 fault) (unsigned long long)desc->qw1); } -static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) +static int qi_check_fault(struct intel_iommu *iommu, int index, + int wait_index, u32 *fsts) { u32 fault; int head, tail; @@ -1278,8 +1279,12 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) return -EAGAIN; fault = readl(iommu->reg + DMAR_FSTS_REG); - if (fault & (DMA_FSTS_IQE | DMA_FSTS_ITE | DMA_FSTS_ICE)) + fault &= DMA_FSTS_IQE | DMA_FSTS_ITE | DMA_FSTS_ICE; + if (fault) { + if (fsts) + *fsts |= fault; qi_dump_fault(iommu, fault); + } /* * If IQE happens, the head points to the descriptor associated @@ -1324,8 +1329,9 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) head = (head - 2 + QI_LENGTH) % QI_LENGTH; } while (head != tail); + /* No need to retry if the caller is interested in the timeout error */ if (qi->desc_status[wait_index] == QI_ABORT) - return -EAGAIN; + return fsts ? -ETIMEDOUT : -EAGAIN; } if (fault & DMA_FSTS_ICE) { @@ -1342,9 +1348,11 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index) * time, a wait descriptor will be appended to each submission to ensure * hardware has completed the invalidation before return. Wait descriptors * can be part of the submission but it will not be polled for completion. + * If callers are interested in the QI faults that occur during the handling + * of requests, the QI faults are saved in @fault. */ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, - unsigned int count, unsigned long options) + unsigned int count, unsigned long options, u32 *fault) { struct q_inval *qi = iommu->qi; s64 devtlb_start_ktime = 0; @@ -1422,6 +1430,9 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, */ writel(qi->free_head << shift, iommu->reg + DMAR_IQT_REG); + if (fault) + *fault = 0; + while (qi->desc_status[wait_index] != QI_DONE) { /* * We will leave the interrupts disabled, to prevent interrupt @@ -1430,7 +1441,7 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, * a deadlock where the interrupt context can wait indefinitely * for free slots in the queue. */ - rc = qi_check_fault(iommu, index, wait_index); + rc = qi_check_fault(iommu, index, wait_index, fault); if (rc) break; @@ -1476,7 +1487,7 @@ void qi_global_iec(struct intel_iommu *iommu) desc.qw3 = 0; /* should never fail */ - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, @@ -1490,7 +1501,7 @@ void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid, u8 fm, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, @@ -1514,7 +1525,7 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, @@ -1545,7 +1556,7 @@ void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* PASID-based IOTLB invalidation */ @@ -1586,7 +1597,7 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, QI_EIOTLB_AM(mask); } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* PASID-based device IOTLB Invalidate */ @@ -1639,7 +1650,7 @@ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw1 |= QI_DEV_EIOTLB_SIZE; } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, @@ -1649,7 +1660,7 @@ void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) | QI_PC_GRAN(granu) | QI_PC_TYPE; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } /* diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index ce030c5b5772..c6de958e4f54 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -881,7 +881,7 @@ void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, u32 pasid); int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, - unsigned int count, unsigned long options); + unsigned int count, unsigned long options, u32 *fault); /* * Options used in qi_submit_sync: * QI_OPT_WAIT_DRAIN - Wait for PRQ drain completion, spec 6.5.2.8. diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c index 29b9e55dcf26..f834afa3672d 100644 --- a/drivers/iommu/intel/irq_remapping.c +++ b/drivers/iommu/intel/irq_remapping.c @@ -153,7 +153,7 @@ static int qi_flush_iec(struct intel_iommu *iommu, int index, int mask) desc.qw2 = 0; desc.qw3 = 0; - return qi_submit_sync(iommu, &desc, 1, 0); + return qi_submit_sync(iommu, &desc, 1, 0, NULL); } static int modify_irte(struct irq_2_iommu *irq_iommu, diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 74e8e4c17e81..67f924760ba8 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -467,7 +467,7 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } static void diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index ac12f76c1212..660d049ad5b6 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -543,7 +543,7 @@ void intel_drain_pasid_prq(struct device *dev, u32 pasid) QI_DEV_IOTLB_PFSID(info->pfsid); qi_retry: reinit_completion(&iommu->prq_complete); - qi_submit_sync(iommu, desc, 3, QI_OPT_WAIT_DRAIN); + qi_submit_sync(iommu, desc, 3, QI_OPT_WAIT_DRAIN, NULL); if (readl(iommu->reg + DMAR_PRS_REG) & DMA_PRS_PRO) { wait_for_completion(&iommu->prq_complete); goto qi_retry; @@ -646,7 +646,7 @@ static void handle_bad_prq_event(struct intel_iommu *iommu, desc.qw3 = 0; } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } static irqreturn_t prq_event_thread(int irq, void *d) @@ -811,7 +811,7 @@ int intel_svm_page_response(struct device *dev, ktime_to_ns(ktime_get()) - prm->private_data[0]); } - qi_submit_sync(iommu, &desc, 1, 0); + qi_submit_sync(iommu, &desc, 1, 0, NULL); } out: return ret; From patchwork Tue Jan 2 14:38:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184374 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4484925dyb; Tue, 2 Jan 2024 06:43:25 -0800 (PST) X-Google-Smtp-Source: AGHT+IFPy/rR9+DxFtPXOWxUa6dKWx5oT8TPbGwOVanZmOAqoCNwxSUuWOGfirshrCXwl7FED+VB X-Received: by 2002:a05:622a:718f:b0:427:e850:6f1a with SMTP id jd15-20020a05622a718f00b00427e8506f1amr17766838qtb.62.1704206605085; Tue, 02 Jan 2024 06:43:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206605; cv=none; d=google.com; s=arc-20160816; b=yJPv7EI0Kl/nkeI6WsJ/zSnXzbhSapM8Lp9WIzwJl7hfnBsytFIazoyCwP9Wkf5hv+ bGyh0sPyvz5uq5vHcR5TEA9ge+sAxpPTeZ/Re/qXPTRvTgE6bCMp8lL5h2eglrTlRdXW +fsBr4B3aCYdzW3rCaKRQyY4iF0rMYiAvS0HAx9xlBvNnY7s168XBgBsipA2nHENoPbf KjMWmQUZ0ZVbRsLtVBjnuLfLiuWjhWBwEUHXc2uNcIscmRKzl93HvR7o2s5wonyXD+Ux yPohaXZdgUUaZqZ/meIwatL1Nj/f32tpx0iBcS555gANMjZW1zZvXmmX65o53pC+sius y6lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=142etHkkEMqrbu6VMjLBZ3yNi09W4w8wUN05Bb27YJo=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=s6N7A+lbeM1JLb4y0MVWpXyw1aU7aMpKkPgS8QLRj/+NrNAlCFzXo3s4akbFHfGSvJ Vi7Pgpxa1mGT8JqiJPNZ29HQuUBrxYzFQ3NN95NNjAX0tUGFcGD4i1izRk8XUXz8AzD4 UopMorYBajL5Vrp3Fe56F8gsQIg62ZUfksreLZ+H9c2LQFANLkdQ0FsTDy38uT4GzL/x dNg8lFdKY92J+RxEufQye8icIZ8ms7xNCw5vc0mINFKq3OyDQbjDeRwKDHVJpOkra+ee Dncrb4Bsxi7YA0NFbEXH3ZLs72P6SOo8sDmX/Wb8EJ1TSKBY4Mt5hdJwRpEiJUu1kWWV nrag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kTwbhFmM; spf=pass (google.com: domain of linux-kernel+bounces-14497-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14497-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id c21-20020ac87dd5000000b004281c2dab17si5345981qte.123.2024.01.02.06.43.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:43:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14497-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kTwbhFmM; spf=pass (google.com: domain of linux-kernel+bounces-14497-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14497-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id CF35A1C2229F for ; Tue, 2 Jan 2024 14:43:24 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D1E6E171CC; Tue, 2 Jan 2024 14:38:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kTwbhFmM" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BF7D15AD7; Tue, 2 Jan 2024 14:38:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206328; x=1735742328; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RhiXnLLG5dHDW1It9pkSt3+L8su0CvX3udFjW+SbWh4=; b=kTwbhFmM9F8X53iRnHB3BEdJV5HqvT2Gh5ccWsXL1FXMpPHybQqMVOk0 H13NbF8shK/LNL75XdrvJmBxCD2uAHhbOghtIwWOeYaN2cNd5Yar1P2wk RH8JrSbhV+c9EdlkZJ30+UGL+zdkNvnpQCRHjJ89mmRtwYb+u7rjbEnYK xy9Ek+nhnLnbMV8EEyTEis5W5NXfdHDf2yrfOddzzbhp8TNaDdEuzYh6C KnBa3tch3Tvt64zjd5hkXxhwvqlsSTunysncfdKnxVZvXnAew1O+rMD7R Wag6uOK78Ox0rc4M922lfzFhg45axEwk85YLtsSKr3u0uTvMmLVY4VlS7 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270502" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270502" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234316" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234316" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:46 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 08/10] iommu/vt-d: Convert stage-1 cache invalidation to return QI fault Date: Tue, 2 Jan 2024 06:38:32 -0800 Message-Id: <20240102143834.146165-9-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990145366213245 X-GMAIL-MSGID: 1786990145366213245 From: Lu Baolu This makes the pasid based cache invalidation and device TLB invalidation to return QI faults to callers. This is needed when usersapce invalidates cache after modifying the stage-1 page table used in nested translation. Hardware errors during invalidation should be reported to user. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Signed-off-by: Yi Liu --- drivers/iommu/intel/dmar.c | 13 +++++++------ drivers/iommu/intel/iommu.c | 12 ++++++------ drivers/iommu/intel/iommu.h | 6 +++--- drivers/iommu/intel/pasid.c | 12 +++++++----- drivers/iommu/intel/svm.c | 8 ++++---- 5 files changed, 27 insertions(+), 24 deletions(-) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index c3251c36c567..d9e38ff34437 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1529,7 +1529,7 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, } void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u16 qdep, u64 addr, unsigned mask) + u16 qdep, u64 addr, unsigned mask, u32 *fault) { struct qi_desc desc; @@ -1556,12 +1556,12 @@ void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw2 = 0; desc.qw3 = 0; - qi_submit_sync(iommu, &desc, 1, 0, NULL); + qi_submit_sync(iommu, &desc, 1, 0, fault); } /* PASID-based IOTLB invalidation */ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, - unsigned long npages, bool ih) + unsigned long npages, bool ih, u32 *fault) { struct qi_desc desc = {.qw2 = 0, .qw3 = 0}; @@ -1597,12 +1597,13 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, QI_EIOTLB_AM(mask); } - qi_submit_sync(iommu, &desc, 1, 0, NULL); + qi_submit_sync(iommu, &desc, 1, 0, fault); } /* PASID-based device IOTLB Invalidate */ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u32 pasid, u16 qdep, u64 addr, unsigned int size_order) + u32 pasid, u16 qdep, u64 addr, + unsigned int size_order, u32 *fault) { unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1); struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0}; @@ -1650,7 +1651,7 @@ void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, desc.qw1 |= QI_DEV_EIOTLB_SIZE; } - qi_submit_sync(iommu, &desc, 1, 0, NULL); + qi_submit_sync(iommu, &desc, 1, 0, fault); } void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 897159dba47d..68e494f1d03a 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1462,7 +1462,7 @@ static void __iommu_flush_dev_iotlb(struct device_domain_info *info, sid = info->bus << 8 | info->devfn; qdep = info->ats_qdep; qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, addr, mask); + qdep, addr, mask, NULL); quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep); } @@ -1490,7 +1490,7 @@ static void iommu_flush_dev_iotlb(struct dmar_domain *domain, PCI_DEVID(info->bus, info->devfn), info->pfsid, dev_pasid->pasid, info->ats_qdep, addr, - mask); + mask, NULL); } spin_unlock_irqrestore(&domain->lock, flags); } @@ -1505,10 +1505,10 @@ static void domain_flush_pasid_iotlb(struct intel_iommu *iommu, spin_lock_irqsave(&domain->lock, flags); list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) - qi_flush_piotlb(iommu, did, dev_pasid->pasid, addr, npages, ih); + qi_flush_piotlb(iommu, did, dev_pasid->pasid, addr, npages, ih, NULL); if (!list_empty(&domain->devices)) - qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih); + qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih, NULL); spin_unlock_irqrestore(&domain->lock, flags); } @@ -5195,10 +5195,10 @@ void quirk_extra_dev_tlb_flush(struct device_domain_info *info, sid = PCI_DEVID(info->bus, info->devfn); if (pasid == IOMMU_NO_PASID) { qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, - qdep, address, mask); + qdep, address, mask, NULL); } else { qi_flush_dev_iotlb_pasid(info->iommu, sid, info->pfsid, - pasid, qdep, address, mask); + pasid, qdep, address, mask, NULL); } } diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index c6de958e4f54..ce9bd08dcd05 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -866,14 +866,14 @@ void qi_flush_context(struct intel_iommu *iommu, u16 did, void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, unsigned int size_order, u64 type); void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid, - u16 qdep, u64 addr, unsigned mask); + u16 qdep, u64 addr, unsigned mask, u32 *fault); void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr, - unsigned long npages, bool ih); + unsigned long npages, bool ih, u32 *fault); void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid, u32 pasid, u16 qdep, u64 addr, - unsigned int size_order); + unsigned int size_order, u32 *fault); void quirk_extra_dev_tlb_flush(struct device_domain_info *info, unsigned long address, unsigned long pages, u32 pasid, u16 qdep); diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index 67f924760ba8..4a7fe551d8a6 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -492,9 +492,11 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu, * efficient to flush devTLB specific to the PASID. */ if (pasid == IOMMU_NO_PASID) - qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, 64 - VTD_PAGE_SHIFT); + qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, + 64 - VTD_PAGE_SHIFT, NULL); else - qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, 64 - VTD_PAGE_SHIFT); + qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, + 64 - VTD_PAGE_SHIFT, NULL); } void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev, @@ -521,7 +523,7 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev, pasid_cache_invalidation_with_pasid(iommu, did, pasid); if (pgtt == PASID_ENTRY_PGTT_PT || pgtt == PASID_ENTRY_PGTT_FL_ONLY) - qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0, NULL); else iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); @@ -543,7 +545,7 @@ static void pasid_flush_caches(struct intel_iommu *iommu, if (cap_caching_mode(iommu->cap)) { pasid_cache_invalidation_with_pasid(iommu, did, pasid); - qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0, NULL); } else { iommu_flush_write_buffer(iommu); } @@ -834,7 +836,7 @@ void intel_pasid_setup_page_snoop_control(struct intel_iommu *iommu, * Addr[63:12]=0x7FFFFFFF_FFFFF) to affected functions */ pasid_cache_invalidation_with_pasid(iommu, did, pasid); - qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0, NULL); /* Device IOTLB doesn't need to be flushed in caching mode. */ if (!cap_caching_mode(iommu->cap)) diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c index 660d049ad5b6..bf7b4c5c21f4 100644 --- a/drivers/iommu/intel/svm.c +++ b/drivers/iommu/intel/svm.c @@ -179,11 +179,11 @@ static void __flush_svm_range_dev(struct intel_svm *svm, if (WARN_ON(!pages)) return; - qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages, ih); + qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages, ih, NULL); if (info->ats_enabled) { qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info->pfsid, svm->pasid, sdev->qdep, address, - order_base_2(pages)); + order_base_2(pages), NULL); quirk_extra_dev_tlb_flush(info, address, order_base_2(pages), svm->pasid, sdev->qdep); } @@ -225,11 +225,11 @@ static void intel_flush_svm_all(struct intel_svm *svm) list_for_each_entry_rcu(sdev, &svm->devs, list) { info = dev_iommu_priv_get(sdev->dev); - qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, 0, -1UL, 0); + qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, 0, -1UL, 0, NULL); if (info->ats_enabled) { qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info->pfsid, svm->pasid, sdev->qdep, - 0, 64 - VTD_PAGE_SHIFT); + 0, 64 - VTD_PAGE_SHIFT, NULL); quirk_extra_dev_tlb_flush(info, 0, 64 - VTD_PAGE_SHIFT, svm->pasid, sdev->qdep); } From patchwork Tue Jan 2 14:38:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184376 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4485949dyb; Tue, 2 Jan 2024 06:45:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IEitKsb4pnsllPxSc/dmLo2K/Fb3aCLI7o/OpJdOaSlr4aL5DzSS4+XmCJd4wdrSdn8Px06 X-Received: by 2002:a05:6e02:1805:b0:35f:d1aa:f41c with SMTP id a5-20020a056e02180500b0035fd1aaf41cmr30950135ilv.10.1704206722227; Tue, 02 Jan 2024 06:45:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206722; cv=none; d=google.com; s=arc-20160816; b=aHO0J3Cz2RiYJdrk/VpPeE9K3jYZum/niIY6M5vTbZWO+e052cng8HKYD6AYYM3HSL jaEiAQGACl7VBFQuAJrlSvv1OQFXBN26Cjnr6Yktacc1nDKdnzpaBf1vinAgX7vcdP1I iL6e3dsRM6eIlFOvE/PzcZ2336RazjVErL/J8oeY5dIBEJ316dcV84npKtHoMpyIo17q BFhShc1fC9L4XLxc2opqXlUG6KoNeBOoXlwvPW6D6FcRYTecAVwzELpP8jvGaXxVpbOM avol8Sq6sYQe/of7hAgKyE0r7BrVAAvE6okzle9HMza0auR8Gl1otZg5435udp3klpbT sTQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=KmghMqZ76DJtyDhTv2WoB5KuW/yZWkE85ExZ1zPyKIE=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=uCj+R61B/gr4VkBHB/bUsWr1eve1kUYXWmDM85ZZ55c2RTry53WGuPCgs+Yff1Zans PU+tHDenYW3hk7UNsOzdiC5ERPqWgFCwVu/Ys/jMe5ZaLJ7+Xo7tIoHQCOxNAx3p90As KQUHzyPo1HLFA49G/fOpY1nKRMKnV/Ij1CDJWUFRiqN3K+jtMOT3PshjN6Mgt3MpYRtE 0PwTwp7m15GSc24nMU4vciR+wrdMbR+hXp84m9McqU1B6v7tulmUB4KkIXouCipuClcE FRmGWXBBzXxogQ1l51VyNQTjv5b+6K2tcB6o6h9XttEoBuS+P3tVSWjkL2EOgFIaw8En 7ReA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=GBVAPcaJ; spf=pass (google.com: domain of linux-kernel+bounces-14499-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14499-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id h33-20020a63f921000000b005cdc5c9d6a2si20291566pgi.576.2024.01.02.06.45.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:45:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14499-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=GBVAPcaJ; spf=pass (google.com: domain of linux-kernel+bounces-14499-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14499-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 5DE2FB22726 for ; Tue, 2 Jan 2024 14:43:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4630E17725; Tue, 2 Jan 2024 14:38:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="GBVAPcaJ" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A06815AF7; Tue, 2 Jan 2024 14:38:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206329; x=1735742329; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pIwuSBuivaLiuysTG8SR9U5i6WTksuS89VvUb25xUrk=; b=GBVAPcaJSTmBzpBxXX0MCBb3qVmexi6bWxw7/x3M4/DwdxNMLaRmyF/1 G4dMX7sTVd4PxAefMsCoRxpL55RSewMYxZe0dgcw8oAyoQP+Wh2h1OuiH yMNurTB92kTmkntD+PaWDGrLyMrRnODjkzCw47cR6KrPEeCQxKB+fOKwy KSrsqD0BVZ0l2XWorm7RpOjoFVVrlUdVcLIpG/Hue3/W5j12T0SPlRHWr zCBQn7rAgG2bPUqbzzetkp6WurVPCGzsN1m0GiBGG6/T1+riRLmSrzOWt doHXYchtffZ3GnPi1o5T6xYv17RIO0K/guetR2kdnxXu79Y99nZ+YF+2E A==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270514" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270514" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234326" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234326" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:47 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 09/10] iommufd: Add data structure for Intel VT-d stage-1 cache invalidation Date: Tue, 2 Jan 2024 06:38:33 -0800 Message-Id: <20240102143834.146165-10-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990268282475916 X-GMAIL-MSGID: 1786990268282475916 This adds the data structure invalidating caches for the nested domain allocated with IOMMU_HWPT_DATA_VTD_S1 type. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Signed-off-by: Yi Liu --- include/uapi/linux/iommufd.h | 58 ++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 824560c50ec6..37c778055201 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -623,6 +623,64 @@ enum iommu_hwpt_invalidate_data_type { IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, }; +/** + * enum iommu_hwpt_vtd_s1_invalidate_flags - Flags for Intel VT-d + * stage-1 cache invalidation + * @IOMMU_VTD_INV_FLAGS_LEAF: Indicates whether the invalidation applies + * to all-levels page structure cache or just + * the leaf PTE cache. + */ +enum iommu_hwpt_vtd_s1_invalidate_flags { + IOMMU_VTD_INV_FLAGS_LEAF = 1 << 0, +}; + +/** + * enum iommu_hwpt_vtd_s1_invalidate_error - Hardware error of invalidation + * @IOMMU_HWPT_INVALIDATE_VTD_S1_ICE: Invalidation Completion Error, details + * refer to 11.4.7.1 Fault Status Register + * of VT-d specification. + * @IOMMU_HWPT_INVALIDATE_VTD_S1_ITE: Invalidation Time-out Error, details + * refer to 11.4.7.1 Fault Status Register + * of VT-d specification. + */ +enum iommu_hwpt_vtd_s1_invalidate_error { + IOMMU_HWPT_INVALIDATE_VTD_S1_ICE = 1 << 0, + IOMMU_HWPT_INVALIDATE_VTD_S1_ITE = 1 << 1, +}; + +/** + * struct iommu_hwpt_vtd_s1_invalidate - Intel VT-d cache invalidation + * (IOMMU_HWPT_INVALIDATE_DATA_VTD_S1) + * @addr: The start address of the range to be invalidated. It needs to + * be 4KB aligned. + * @npages: Number of contiguous 4K pages to be invalidated. + * @flags: Combination of enum iommu_hwpt_vtd_s1_invalidate_flags + * @hw_error: One of enum iommu_hwpt_vtd_s1_invalidate_error + * + * The Intel VT-d specific invalidation data for user-managed stage-1 cache + * invalidation in nested translation. Userspace uses this structure to + * tell the impacted cache scope after modifying the stage-1 page table. + * + * Invalidating all the caches related to the page table by setting @addr + * to be 0 and @npages to be U64_MAX. + * + * The device TLB will be invalidated automatically if ATS is enabled. + * + * An entry is considered 'handled' after it passes the audit and submitted + * to the IOMMU by the underlying driver. Check the @entry_num output of + * struct iommu_hwpt_invalidate for the number of handled entries. A 'handled' + * request may still fail in hardware for various reasons, e.g. due to timeout + * on waiting for device response upon a device TLB invalidation request. In + * such case the hardware error info is reported in the @hw_error field of the + * handled entry. + */ +struct iommu_hwpt_vtd_s1_invalidate { + __aligned_u64 addr; + __aligned_u64 npages; + __u32 flags; + __u32 hw_error; +}; + /** * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) * @size: sizeof(struct iommu_hwpt_invalidate) From patchwork Tue Jan 2 14:38:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 184377 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp4486066dyb; Tue, 2 Jan 2024 06:45:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IGVoEvo6Q6ZWt/llfK6gn5LDWYQWwNtUxD/jT3rwFBLc7uTPz9v8rzEJ+oN8vAvO8ZemCF9 X-Received: by 2002:a05:6a00:4b48:b0:6da:2f8e:bdf4 with SMTP id kr8-20020a056a004b4800b006da2f8ebdf4mr3326740pfb.60.1704206735389; Tue, 02 Jan 2024 06:45:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704206735; cv=none; d=google.com; s=arc-20160816; b=qPSresC52/nDMph9FihiiSMGhU2mNbyFolzop9Kh+fqvx06AEwNxlc3RBFhpWvZfUu oCyDieQMwOh4OtQ8HM6H+m1Esxo95UlNq78YCk04noMpwrZ7mskMRrDMnx6SonL+h4Zr xKg7W/2Q6pfiJ1SerXCg2TZFN0SBGVM5t9sW/bp9Vfb+gih8N+uj73qs7TsDM3fXBIlD vyGLCK6NnBU/ZjobK8xLG0OfB0g+ZfD+J2FsYxeJD3gm6UhvG5aB9S67OUV36bVBVOKs 7c4tDrY+VPhBBoyQdpcFeL4u50WrmkG9OWaTmZ0Km9YIn49J7eMKt7ZAEStQDRpzOMiD 4UUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=zFRLIM6g1/LyMeA3dvMIRJhxzPMyYMd444td+z/oYM4=; fh=JbgkfaGVOLJQIgFMcRSNKOzjVQFwxW1pSzjP6daOUwc=; b=I2UpFRR0jdwylmKS+7nBMEz8e7xw3DWR9txjol+DtQeCSRjyTOIuPGFTWcfC/EW7f0 pfbKm0g5UkkOaDpIpJTX4XI+Yssc0qtRmkUoNU1W2rEmBpKWcxmowxht18ez4rQO9MGG PIOSsEFRwY175qK27Erf5X45B+tn88s8I7+6vEyktdvN5itwG9AGsCwPi7HQKj/c/Z2u XqgIGCLw3OWT7xgaPA6R2tQgEPkLd8K3M5V1QUevXZcw5ZbdC7uNlQStX6EVHXDJoIAw qR4/5Q2UhAN2PxCyzwIQP89ugcnKPiRBGsPOWGtBT0g59cpCY0cYgE0ISY7W9Ryn4KXo HB0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=REQ2etLw; spf=pass (google.com: domain of linux-kernel+bounces-14500-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14500-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id 79-20020a630152000000b005ce034f8422si16318743pgb.43.2024.01.02.06.45.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 06:45:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-14500-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=REQ2etLw; spf=pass (google.com: domain of linux-kernel+bounces-14500-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-14500-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id CE5FAB22843 for ; Tue, 2 Jan 2024 14:43:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 67E1917741; Tue, 2 Jan 2024 14:38:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="REQ2etLw" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BAF916414; Tue, 2 Jan 2024 14:38:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704206330; x=1735742330; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NVOmCLUpjiadLIpufkixMz24Btn82kCB3C2V3+0Ea9k=; b=REQ2etLwb2gi5kW0E5jPTrkdWI9qunSdqYYFXJbrPgp04pmhod8/U+Tk NVftdg57DePg1G5Uvtwezn9zah47skegpBG0y9wdm0SsJNkZGZOhKPXQv snKHNbqPHM++pzmFmCotUX0k9d08is5g5Ws+IzhacC14xzC+KTBNPU8L2 R6pQ+d3AgxtSmai4zCBVNUuSOT37Ao2twYMp9QynF9BmLasbUgHZH3r5H tJXN+23O0vm91hBDhK6vWd4IG+OVsMbx0VosbMR8O8O6hh1l9KG04jvN2 TWRQbgS6s6WdOTItorrxxlrkTkxh6d99IRP/biIAPeDHSkzIOwq01YC20 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="10270524" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="10270524" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 06:38:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="923234334" X-IronPort-AV: E=Sophos;i="6.04,325,1695711600"; d="scan'208";a="923234334" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by fmsmga001.fm.intel.com with ESMTP; 02 Jan 2024 06:38:48 -0800 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com, yan.y.zhao@intel.com, j.granados@samsung.com Subject: [PATCH v10 10/10] iommu/vt-d: Add iotlb flush for nested domain Date: Tue, 2 Jan 2024 06:38:34 -0800 Message-Id: <20240102143834.146165-11-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240102143834.146165-1-yi.l.liu@intel.com> References: <20240102143834.146165-1-yi.l.liu@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1786990281791601661 X-GMAIL-MSGID: 1786990281791601661 From: Lu Baolu This implements the .cache_invalidate_user() callback to support iotlb flush for nested domain. Reviewed-by: Kevin Tian Signed-off-by: Lu Baolu Co-developed-by: Yi Liu Signed-off-by: Yi Liu --- drivers/iommu/intel/nested.c | 107 +++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/drivers/iommu/intel/nested.c b/drivers/iommu/intel/nested.c index b5a5563ab32c..f1f86437939c 100644 --- a/drivers/iommu/intel/nested.c +++ b/drivers/iommu/intel/nested.c @@ -73,9 +73,116 @@ static void intel_nested_domain_free(struct iommu_domain *domain) kfree(to_dmar_domain(domain)); } +static void nested_flush_dev_iotlb(struct dmar_domain *domain, u64 addr, + unsigned mask, u32 *fault) +{ + struct device_domain_info *info; + unsigned long flags; + u16 sid, qdep; + + spin_lock_irqsave(&domain->lock, flags); + list_for_each_entry(info, &domain->devices, link) { + if (!info->ats_enabled) + continue; + sid = info->bus << 8 | info->devfn; + qdep = info->ats_qdep; + qi_flush_dev_iotlb(info->iommu, sid, info->pfsid, + qdep, addr, mask, fault); + quirk_extra_dev_tlb_flush(info, addr, mask, + IOMMU_NO_PASID, qdep); + } + spin_unlock_irqrestore(&domain->lock, flags); +} + +static void intel_nested_flush_cache(struct dmar_domain *domain, u64 addr, + unsigned long npages, bool ih, u32 *error) +{ + struct iommu_domain_info *info; + unsigned long i; + unsigned mask; + u32 fault; + + xa_for_each(&domain->iommu_array, i, info) + qi_flush_piotlb(info->iommu, + domain_id_iommu(domain, info->iommu), + IOMMU_NO_PASID, addr, npages, ih, NULL); + + if (!domain->has_iotlb_device) + return; + + if (npages == U64_MAX) + mask = 64 - VTD_PAGE_SHIFT; + else + mask = ilog2(__roundup_pow_of_two(npages)); + + nested_flush_dev_iotlb(domain, addr, mask, &fault); + + *error = 0; + /* + * Invalidation queue error (i.e. IQE) will not be reported to user + * as it's caused only by driver internal bug. + */ + if (fault & DMA_FSTS_ICE) + *error |= IOMMU_HWPT_INVALIDATE_VTD_S1_ICE; + if (fault & DMA_FSTS_ITE) + *error |= IOMMU_HWPT_INVALIDATE_VTD_S1_ITE; +} + +static int intel_nested_cache_invalidate_user(struct iommu_domain *domain, + struct iommu_user_data_array *array) +{ + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + struct iommu_hwpt_vtd_s1_invalidate inv_entry; + u32 processed = 0; + int ret = 0; + u32 index; + + if (array->type != IOMMU_HWPT_INVALIDATE_DATA_VTD_S1) { + ret = -EINVAL; + goto out; + } + + for (index = 0; index < array->entry_num; index++) { + ret = iommu_copy_struct_from_user_array(&inv_entry, array, + IOMMU_HWPT_INVALIDATE_DATA_VTD_S1, + index, hw_error); + if (ret) + break; + + if (inv_entry.flags & ~IOMMU_VTD_INV_FLAGS_LEAF) { + ret = -EOPNOTSUPP; + break; + } + + if (!IS_ALIGNED(inv_entry.addr, VTD_PAGE_SIZE) || + ((inv_entry.npages == U64_MAX) && inv_entry.addr)) { + ret = -EINVAL; + break; + } + + intel_nested_flush_cache(dmar_domain, inv_entry.addr, + inv_entry.npages, + inv_entry.flags & IOMMU_VTD_INV_FLAGS_LEAF, + &inv_entry.hw_error); + + ret = iommu_respond_struct_to_user_array(array, index, + (void *)&inv_entry, + sizeof(inv_entry)); + if (ret) + break; + + processed++; + } + +out: + array->entry_num = processed; + return ret; +} + static const struct iommu_domain_ops intel_nested_domain_ops = { .attach_dev = intel_nested_attach_dev, .free = intel_nested_domain_free, + .cache_invalidate_user = intel_nested_cache_invalidate_user, }; struct iommu_domain *intel_nested_domain_alloc(struct iommu_domain *parent,