From patchwork Fri Oct 20 09:24:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 155916 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp928694vqb; Fri, 20 Oct 2023 02:24:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHzGWVhI1qre7eTOZ3mkSVP8PCaWrUXkQJ+RcewB5kO+F/bbpaSjP152DLasb4++H8YCtLA X-Received: by 2002:a05:6a20:12d4:b0:174:52b7:f63 with SMTP id v20-20020a056a2012d400b0017452b70f63mr1429788pzg.26.1697793892803; Fri, 20 Oct 2023 02:24:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697793892; cv=none; d=google.com; s=arc-20160816; b=IQp+ki6ss1BawlOsPOnU9+m/p6wU6Gj4VIbBtEuXoaVpuVvfeQLZjPWIGyW9lbWv9v 18OC0byDxShVx9X/Q+PEzHbYGFp8LLc8LFXPwmvb3CqHae7yCwQ7Vqo/PSSiKb8byj2F uSowP+k6hUxzxGYc7UtT+5G9RDwYoUZjoWBG1HATL1+d4JgxYbobS9Wj4Yn/ENNF8KWN bD7APypkds/chV/Ko5EL1iM+kzxH+A7mxDallF76hhlXzkmyQMPU5ok+FH64AFIKGlMn rAv4tu4foXWobIdxblbxE4h2sTeiSAXxsFTMoME9Sr/7QTaPat3bDIEPS//qZmE3oBCN QV5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nmA4PZmrE+Onffm2NXAGLyMbuQAixbMOvvKSunKfLJM=; fh=AYfEg8UxxCZNlRHxnJWvHuqsheVQzM8ymhb/+J0VVdk=; b=H9qD5QAakhsPAJ5jvvPG9B7ZvG6Z/biaFkTr6mKWYfYl5DEMXJ0C2hRCHY1P38Hzm4 77IOQoeIFTDWii506jKwjPnTSbmlilAX4fGoGs5aAcaTqtqzble0augRpHJg0JBLBMfp Jw+B1WqHkUUpc6sqyu0er5GvYjO1h4rfbSB1m6w/hB5WB170M+1oN0h4c4jAriB3kOuG q4dj+9nWTnZql/embe9pUglt+iU9Z6MyBeneB9P+KOLFhdGz5Z/mBZr7ylc5BW/Fnwh5 kSUNxxJchvR2NrXEliLwlFQPjpiL6Fka/tmhevvnpZ7H2Z6xIDcQ798oXnvxB44wiqxi GNkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="fq/1VI6X"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id b137-20020a63348f000000b005859c29ce86si1513395pga.84.2023.10.20.02.24.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Oct 2023 02:24:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="fq/1VI6X"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 0FFC582CA0D3; Fri, 20 Oct 2023 02:24:52 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376655AbjJTJYr (ORCPT + 25 others); Fri, 20 Oct 2023 05:24:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376656AbjJTJYe (ORCPT ); Fri, 20 Oct 2023 05:24:34 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07FD0106; Fri, 20 Oct 2023 02:24:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1697793872; x=1729329872; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2PM7o5xYDJmlHz5TKehP6+0yc9LVqHG0PRNTYo4xOBg=; b=fq/1VI6XpsOhLLNmLf5+YnjRcUCYmRoFWDAlr5xfaHprjM3de/ybrSHp 2uWsqEWXHrFptgZj4xejTFq69hC7WJDrtQdl6kODYC33lonxRtg4QRCIl WxjbpDavMinwmzUo1Vz2f1wzmcr1LRnhrqIuOX+EZabwqdfB3ZrHz0PnD EDAzy3wdsEF8/fNMQhTguB6K8Ur1cgsAwo7BrjWhHos7ezJH486d4INrx xIKDPNMFJ4voOoot3AqjW3zUFbK6OLGJlQQKjeZm7bpg5IaVmSXn7ieQp lIv5BwefLL4KdaT9V3+H/BaW8UZaWzGRQWvF/gFELEfohp6T3hjaDCXPx A==; X-IronPort-AV: E=McAfee;i="6600,9927,10868"; a="472685474" X-IronPort-AV: E=Sophos;i="6.03,238,1694761200"; d="scan'208";a="472685474" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2023 02:24:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10868"; a="750859673" X-IronPort-AV: E=Sophos;i="6.03,238,1694761200"; d="scan'208";a="750859673" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by orsmga007.jf.intel.com with ESMTP; 20 Oct 2023 02:24:29 -0700 From: Yi Liu To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com, kevin.tian@intel.com, robin.murphy@arm.com, baolu.lu@linux.intel.com Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com, kvm@vger.kernel.org, mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com, yi.l.liu@intel.com, yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com, shameerali.kolothum.thodi@huawei.com, lulu@redhat.com, suravee.suthikulpanit@amd.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, zhenzhong.duan@intel.com, joao.m.martins@oracle.com, xin.zeng@intel.com Subject: [PATCH v5 2/6] iommufd: Add IOMMU_HWPT_INVALIDATE Date: Fri, 20 Oct 2023 02:24:22 -0700 Message-Id: <20231020092426.13907-3-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231020092426.13907-1-yi.l.liu@intel.com> References: <20231020092426.13907-1-yi.l.liu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 20 Oct 2023 02:24:52 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780265928890892420 X-GMAIL-MSGID: 1780265928890892420 In nested translation, the stage-1 page table is user-managed but cached by the IOMMU hardware, so an update on present page table entries in the stage-1 page table should be followed with a cache invalidation. Add an IOMMU_HWPT_INVALIDATE ioctl to support such a cache invalidation. It takes hwpt_id to specify the iommu_domain, and a multi-entry array to support multiple invalidation requests in one ioctl. Check cache_invalidate_user op in the iommufd_hw_pagetable_alloc_nested, since all nested domains need that. Co-developed-by: Nicolin Chen Signed-off-by: Nicolin Chen Signed-off-by: Yi Liu --- drivers/iommu/iommufd/hw_pagetable.c | 35 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 9 +++++++ drivers/iommu/iommufd/main.c | 3 +++ include/uapi/linux/iommufd.h | 36 +++++++++++++++++++++++++ 4 files changed, 83 insertions(+) diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index 362c694aac9b..6090d59c4982 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -238,6 +238,11 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, rc = -EINVAL; goto out_abort; } + /* Driver is buggy by missing cache_invalidate_user in domain_ops */ + if (WARN_ON_ONCE(!hwpt->domain->ops->cache_invalidate_user)) { + rc = -EINVAL; + goto out_abort; + } return hwpt_nested; out_abort: @@ -323,3 +328,33 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) iommufd_put_object(&idev->obj); return rc; } + +int iommufd_hwpt_invalidate(struct iommufd_ucmd *ucmd) +{ + struct iommu_hwpt_invalidate *cmd = ucmd->cmd; + struct iommu_user_data_array data_array = { + .type = cmd->req_type, + .uptr = u64_to_user_ptr(cmd->reqs_uptr), + .entry_len = cmd->req_len, + .entry_num = cmd->req_num, + }; + struct iommufd_hw_pagetable *hwpt; + int rc = 0; + + if (cmd->req_type == IOMMU_HWPT_DATA_NONE) + return -EINVAL; + if (!cmd->reqs_uptr || !cmd->req_len || !cmd->req_num) + return -EINVAL; + + hwpt = iommufd_hw_pagetable_get_nested(ucmd, cmd->hwpt_id); + if (IS_ERR(hwpt)) + return PTR_ERR(hwpt); + + rc = hwpt->domain->ops->cache_invalidate_user(hwpt->domain, &data_array, + &cmd->out_driver_error_code); + cmd->req_num = data_array.entry_num; + if (iommufd_ucmd_respond(ucmd, sizeof(*cmd))) + return -EFAULT; + iommufd_put_object(&hwpt->obj); + return rc; +} diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index a829458fe762..7ec629599844 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -279,6 +279,7 @@ void iommufd_hwpt_paging_abort(struct iommufd_object *obj); void iommufd_hwpt_nested_destroy(struct iommufd_object *obj); void iommufd_hwpt_nested_abort(struct iommufd_object *obj); int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd); +int iommufd_hwpt_invalidate(struct iommufd_ucmd *ucmd); static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx, struct iommufd_hw_pagetable *hwpt) @@ -300,6 +301,14 @@ static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx, refcount_dec(&hwpt->obj.users); } +static inline struct iommufd_hw_pagetable * +iommufd_hw_pagetable_get_nested(struct iommufd_ucmd *ucmd, u32 id) +{ + return container_of(iommufd_get_object(ucmd->ictx, id, + IOMMUFD_OBJ_HWPT_NESTED), + struct iommufd_hw_pagetable, obj); +} + struct iommufd_group { struct kref ref; struct mutex lock; diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index f94c14f20918..1f2d2d262e12 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -307,6 +307,7 @@ union ucmd_buffer { struct iommu_destroy destroy; struct iommu_hw_info info; struct iommu_hwpt_alloc hwpt; + struct iommu_hwpt_invalidate cache; struct iommu_ioas_alloc alloc; struct iommu_ioas_allow_iovas allow_iovas; struct iommu_ioas_copy ioas_copy; @@ -342,6 +343,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { __reserved), IOCTL_OP(IOMMU_HWPT_ALLOC, iommufd_hwpt_alloc, struct iommu_hwpt_alloc, __reserved), + IOCTL_OP(IOMMU_HWPT_INVALIDATE, iommufd_hwpt_invalidate, + struct iommu_hwpt_invalidate, out_driver_error_code), IOCTL_OP(IOMMU_IOAS_ALLOC, iommufd_ioas_alloc_ioctl, struct iommu_ioas_alloc, out_ioas_id), IOCTL_OP(IOMMU_IOAS_ALLOW_IOVAS, iommufd_ioas_allow_iovas, diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index d82bc663c5cd..fc305a48ab81 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -47,6 +47,7 @@ enum { IOMMUFD_CMD_VFIO_IOAS, IOMMUFD_CMD_HWPT_ALLOC, IOMMUFD_CMD_GET_HW_INFO, + IOMMUFD_CMD_HWPT_INVALIDATE, }; /** @@ -477,4 +478,39 @@ struct iommu_hw_info { __u32 __reserved; }; #define IOMMU_GET_HW_INFO _IO(IOMMUFD_TYPE, IOMMUFD_CMD_GET_HW_INFO) + +/** + * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE) + * @size: sizeof(struct iommu_hwpt_invalidate) + * @hwpt_id: HWPT ID of a nested HWPT for cache invalidation + * @reqs_uptr: User pointer to an array having @req_num of cache invalidation + * requests. The request entries in the array are of fixed width + * @req_len, and contain a user data structure for invalidation + * request specific to the given hardware page table. + * @req_type: One of enum iommu_hwpt_data_type, defining the data type of all + * the entries in the invalidation request array. It should suit + * with the data_type passed per the allocation of the hwpt pointed + * by @hwpt_id. + * @req_len: Length (in bytes) of a request entry in the request array + * @req_num: Input the number of cache invalidation requests in the array. + * Output the number of requests successfully handled by kernel. + * @out_driver_error_code: Report a driver speicifc error code upon failure. + * It's optional, driver has a choice to fill it or + * not. + * + * Invalidate the iommu cache for user-managed page table. Modifications on a + * user-managed page table should be followed by this operation to sync cache. + * Each ioctl can support one or more cache invalidation requests in the array + * that has a total size of @req_len * @req_num. + */ +struct iommu_hwpt_invalidate { + __u32 size; + __u32 hwpt_id; + __aligned_u64 reqs_uptr; + __u32 req_type; + __u32 req_len; + __u32 req_num; + __u32 out_driver_error_code; +}; +#define IOMMU_HWPT_INVALIDATE _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_INVALIDATE) #endif