From patchwork Tue Oct 10 09:02:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Si-Wei Liu X-Patchwork-Id: 150593 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2908:b0:403:3b70:6f57 with SMTP id ib8csp57197vqb; Tue, 10 Oct 2023 02:06:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEwEyOiqnFia2iC4CguUczpUBkeVMInm7+yTvVbl33oxIGj9nPecH0gVSvjHOuNMknxSXvR X-Received: by 2002:a05:6808:2d6:b0:3a7:3988:87ee with SMTP id a22-20020a05680802d600b003a7398887eemr17722281oid.58.1696928775590; Tue, 10 Oct 2023 02:06:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696928775; cv=none; d=google.com; s=arc-20160816; b=dhKjBHkWXihIYW2kWM2jecJL4g6mxmGPm7NVreGE/pkPUGTptqR12LNkuUOPDN2Ac5 R1r3sZsVqeJJklCaV0fstjfbNVS6JA3rD8tiVV5IcjpxUnZX90+UukZxu3gAf99HN0fC gW/umD84aHbWxHFjr+FIvobcthqKhWTgSoZBfrLdEqT+jr1ylI4DzefRqjprDS8g3Zli rgJVgwdL9Hzy5cT9Bh6tfmKpwIrhaxUWbCV6M6MbYcLrBk7bAgKOhYHq2WYxFH6X5Cel ZKJ/2k/jCWgbJTXutnZwjKQ5mVeO3GUYT4QqFEWhYqpFvm897rgmsMKQjIdfl5+iVWT7 sFCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=OMuWF+ib0rZpZceqxHvEWuT+8aI6hH6IoBKKRKDUxqQ=; fh=FYvCndxPQizgygd8hAQJp2i0SXZazO8uVJMWtxk8XU4=; b=SbaV6T93myDoJQ+bN3a6TZocVJa4jq77GsTA3NWaOxArdYHMXD4VXTzqdNd/hF+C/k AuWO19w8vSKzlfwIJX2czLscxQemjJ7R84fhgaLus4na7YcWZGq1feJl1+WSxNnqc6CN AsXWoppJTsKoGQpzWcBnJZGeAdII3f/3gfNNP8sIG1FHqGdfSvMeX4PZ9PfsDObMBs05 G4SK/3dm8fpDu6JoCI3wpTYbalH1i2AwXqc+ziFJU6IYTXgjMdrh2XWWSVNBo5pIIX74 64I6DqyKP+P7kHrJvfOWW7zvYNdpUviVRKsxrZevKlNELqFyDcUF/2zHXl9/1HmdASn7 SKwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-03-30 header.b=cbcrn05D; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id f20-20020a63f114000000b00566016fc08csi11614336pgi.83.2023.10.10.02.06.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Oct 2023 02:06:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-03-30 header.b=cbcrn05D; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 119F58023749; Tue, 10 Oct 2023 02:06:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229516AbjJJJFw (ORCPT + 20 others); Tue, 10 Oct 2023 05:05:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229721AbjJJJFq (ORCPT ); Tue, 10 Oct 2023 05:05:46 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28FBDCA for ; Tue, 10 Oct 2023 02:05:43 -0700 (PDT) Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 39A804bs017830; Tue, 10 Oct 2023 09:05:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2023-03-30; bh=OMuWF+ib0rZpZceqxHvEWuT+8aI6hH6IoBKKRKDUxqQ=; b=cbcrn05DehNm2FC9NaVjr7SBJLN7aIKRHY/OVM+w8R6Xj9siw6c+h5UFe5234+bR8lf6 zrskHKjNc5xMqoz7gsyzsxrdDTOrvkvqsNRkg5X8i7cJElPAdhSW84R7SmBT6STDbt1U sf02MXbqjLj3yss2lnMvEPlbRAkjV2r7ILw9a9vcFu6Ihvyhl+H61h3UMJXBFoJFQu+1 dQGcgIwTMhUoks4iNCVk9h1N4wxvJlrccaCrWnoEw3QRxXX1pcM3XYqirxLS2B5g9gqn g50FUf1ptXULdekmcjooqqm8jRPIBssvXsyG5IepZ8km/qLuNV/2CKewl2JiMetLSpgc WQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3tjx8cck9b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 10 Oct 2023 09:05:37 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 39A6sxrF021395; Tue, 10 Oct 2023 09:05:35 GMT Received: from ban25x6uut24.us.oracle.com (ban25x6uut24.us.oracle.com [10.153.73.24]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3tjwsc5cy2-4; Tue, 10 Oct 2023 09:05:35 +0000 From: Si-Wei Liu To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com, xuanzhuo@linux.alibaba.com, dtatulea@nvidia.com Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit Date: Tue, 10 Oct 2023 02:02:59 -0700 Message-Id: <1696928580-7520-4-git-send-email-si-wei.liu@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1696928580-7520-1-git-send-email-si-wei.liu@oracle.com> References: <1696928580-7520-1-git-send-email-si-wei.liu@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-10_04,2023-10-09_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 suspectscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxlogscore=999 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2309180000 definitions=main-2310100067 X-Proofpoint-ORIG-GUID: tiCYM9t-hsTeWDq1eosaeRopCiPEuEdj X-Proofpoint-GUID: tiCYM9t-hsTeWDq1eosaeRopCiPEuEdj X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 10 Oct 2023 02:06:10 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779358787629955945 X-GMAIL-MSGID: 1779358787629955945 Userspace needs this feature flag to distinguish if vhost-vdpa iotlb in the kernel supports persistent IOTLB mapping across device reset. Without it, userspace has no way to tell apart if it's running on an older kernel, which could silently drop all iotlb mapping across vDPA reset. There are 3 cases that backend may claim this feature bit on: - parent device that has to work with platform IOMMU - parent device with on-chip IOMMU that has the expected .reset_map support in driver - parent device with vendor specific IOMMU implementation that explicitly declares the specific backend feature The reason why .reset_map is being one of the pre-condition for persistent iotlb is because without it, vhost-vdpa can't switch back iotlb to the initial state later on, especially for the on-chip IOMMU case which starts with identity mapping at device creation. virtio-vdpa requires on-chip IOMMU to perform 1:1 passthrough translation from PA to IOVA as-is to begin with, and .reset_map is the only means to turn back iotlb to the identity mapping mode after vhost-vdpa is gone. Signed-off-by: Si-Wei Liu --- drivers/vhost/vdpa.c | 15 +++++++++++++++ include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+) diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index a3f8160..c92794f 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -413,6 +413,15 @@ static bool vhost_vdpa_has_desc_group(const struct vhost_vdpa *v) return ops->get_vq_desc_group; } +static bool vhost_vdpa_has_persistent_map(const struct vhost_vdpa *v) +{ + struct vdpa_device *vdpa = v->vdpa; + const struct vdpa_config_ops *ops = vdpa->config; + + return (!ops->set_map && !ops->dma_map) || ops->reset_map || + vhost_vdpa_get_backend_features(v) & BIT_ULL(VHOST_BACKEND_F_IOTLB_PERSIST); +} + static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep) { struct vdpa_device *vdpa = v->vdpa; @@ -725,6 +734,7 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, return -EFAULT; if (features & ~(VHOST_VDPA_BACKEND_FEATURES | BIT_ULL(VHOST_BACKEND_F_DESC_ASID) | + BIT_ULL(VHOST_BACKEND_F_IOTLB_PERSIST) | BIT_ULL(VHOST_BACKEND_F_SUSPEND) | BIT_ULL(VHOST_BACKEND_F_RESUME) | BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK))) @@ -741,6 +751,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, if ((features & BIT_ULL(VHOST_BACKEND_F_DESC_ASID)) && !vhost_vdpa_has_desc_group(v)) return -EOPNOTSUPP; + if ((features & BIT_ULL(VHOST_BACKEND_F_IOTLB_PERSIST)) && + !vhost_vdpa_has_persistent_map(v)) + return -EOPNOTSUPP; vhost_set_backend_features(&v->vdev, features); return 0; } @@ -796,6 +809,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, features |= BIT_ULL(VHOST_BACKEND_F_RESUME); if (vhost_vdpa_has_desc_group(v)) features |= BIT_ULL(VHOST_BACKEND_F_DESC_ASID); + if (vhost_vdpa_has_persistent_map(v)) + features |= BIT_ULL(VHOST_BACKEND_F_IOTLB_PERSIST); features |= vhost_vdpa_get_backend_features(v); if (copy_to_user(featurep, &features, sizeof(features))) r = -EFAULT; diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h index 18ad6ae..d765690 100644 --- a/include/uapi/linux/vhost_types.h +++ b/include/uapi/linux/vhost_types.h @@ -190,5 +190,7 @@ struct vhost_vdpa_iova_range { * buffers may reside. Requires VHOST_BACKEND_F_IOTLB_ASID. */ #define VHOST_BACKEND_F_DESC_ASID 0x7 +/* IOTLB don't flush memory mapping across device reset */ +#define VHOST_BACKEND_F_IOTLB_PERSIST 0x8 #endif