From patchwork Wed Jan 10 20:40:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Sistare X-Patchwork-Id: 187071 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp1044549dyi; Wed, 10 Jan 2024 12:44:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IEFkuuF6SsfnzJgJp4qhWYixMlyHm36a1eyXMGSC73dPbfikDU1Yw5tRfK3+y4jbgmUZr7S X-Received: by 2002:ac2:5e6c:0:b0:50e:d186:ca59 with SMTP id a12-20020ac25e6c000000b0050ed186ca59mr11109lfr.19.1704919487364; Wed, 10 Jan 2024 12:44:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704919487; cv=none; d=google.com; s=arc-20160816; b=p7XXh70AwetEISgay+TbbfmgdEBeXc51rA6X9YAcVndAVXOLWMdVqnrVcCab3KueCd ADjQkaMLXWVjbLia6YorDUvw9w2l53hMARYR1eMStvINnr97JDquRQ2CHO3R/ilTz/Lf y8RjV8FZ3XUaRAKNFZfCkwEOGM1eYnzufz3d+TbybiCQk1h00ALrEAIlXX/DT4BLwt05 9GKqHonKLDhPVdAzBaS+UT8KKu8g+uPSwUWjziu7eUH/okoiwEPfoFp2fED50R55jrkS CnOMHgk6k2S9HjyBdZfg9jIu9VZPRyg2FgLsx8NBxBM8wBbdFGrnqkuhYU4XH2d5L1uO rLTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=h71dZ5JIuyeY4E0yCnqZDDaeoo7HDhGRC726pRtjHQU=; fh=NLOUc4UpZ/0A9JrieYT9bxhjFpxay5qeZXON7PgxqoM=; b=vNyZ+lOKJPB1djo6EXGNjlGm+yyuY/IwtkNC6pr105L9qxfEQc8+C2GIMMWWUt/fpP XPZehZnA0bPU195HaHNxcgJorQrQVvoqO6dOy4tQJQuT0TDTPiUfJKiT82ABIcG5UjzM Km5wvABzRgS6wtxZVodjI37nUGWvRK/BILaMwoZNiA009NAeHwyF4FYPeJNawRmmD2+C owWGRYH08SH7O6TUjVvDoxW/Eu5pSYKGycDO6VVSkbcNezRDExB3aJ9KJZ8GDh5GhMVe soRuP5bvOg3+ydAFDgWbg5voae0buAfmcPeLEEDPAMq+MR5vKiAHCjSreMGLR4Ctf4xS G2tA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=YwjuYn2S; spf=pass (google.com: domain of linux-kernel+bounces-22759-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-22759-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id mb12-20020a170906eb0c00b00a29e983e09asi2006589ejb.398.2024.01.10.12.44.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jan 2024 12:44:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-22759-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2023-11-20 header.b=YwjuYn2S; spf=pass (google.com: domain of linux-kernel+bounces-22759-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-22759-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EDC611F23153 for ; Wed, 10 Jan 2024 20:44:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 41CE9537E6; Wed, 10 Jan 2024 20:40:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="YwjuYn2S" Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C9274F60C for ; Wed, 10 Jan 2024 20:40:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 40AJfmVP028156; Wed, 10 Jan 2024 20:40:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2023-11-20; bh=h71dZ5JIuyeY4E0yCnqZDDaeoo7HDhGRC726pRtjHQU=; b=YwjuYn2SLz0xe5VkayBFoZr9EXJ73Hw0soGv/rlO9PHqsfT3ZC+/skZTteKsregCnTxc pXDKZbWiYARJJqsupF57xGDZfzizVwMKpbGN4HK/yWYcaFqa9wwJQbH5wJKrVn+4qDiU 60uAAh6JkJIhixK8bByD7rNJvApXoYSsU0QRMVO92GbNnERG9tHNT4SWEl6gLrEuyMxQ 7X+i01WvTBCU0vsEZNVBYLrxO51fZFk2R2REBs6A+xbvKgJk+XwPVOpi6Bhs00YEaTe8 e9Pqsw04MDdpHCBpfIGuadZGUtXarz9CNigcIyN0A4DjsrsAuFqB/zyi/NwM7BWjg249 ow== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3vj1m404a9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 10 Jan 2024 20:40:24 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 40AKdD5K030150; Wed, 10 Jan 2024 20:40:23 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3vfutp5xay-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 10 Jan 2024 20:40:23 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 40AKeGrf005067; Wed, 10 Jan 2024 20:40:23 GMT Received: from ca-dev63.us.oracle.com (ca-dev63.us.oracle.com [10.211.8.221]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3vfutp5x5e-9; Wed, 10 Jan 2024 20:40:23 +0000 From: Steve Sistare To: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org Cc: "Michael S. Tsirkin" , Jason Wang , Si-Wei Liu , Eugenio Perez Martin , Xuan Zhuo , Dragos Tatulea , Eli Cohen , Xie Yongji , Steve Sistare Subject: [RFC V1 08/13] vduse: flush workers on suspend Date: Wed, 10 Jan 2024 12:40:10 -0800 Message-Id: <1704919215-91319-9-git-send-email-steven.sistare@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1704919215-91319-1-git-send-email-steven.sistare@oracle.com> References: <1704919215-91319-1-git-send-email-steven.sistare@oracle.com> X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-01-10_10,2024-01-10_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 adultscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2401100163 X-Proofpoint-ORIG-GUID: 8NpbJVwE_0qX_hcElGpqR-L_JVyvHOtf X-Proofpoint-GUID: 8NpbJVwE_0qX_hcElGpqR-L_JVyvHOtf Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787737656511013135 X-GMAIL-MSGID: 1787737656511013135 To pass ownership of a live vdpa device to a new process, the user suspends the device, calls VHOST_NEW_OWNER to change the mm, and calls VHOST_IOTLB_REMAP to change the user virtual addresses to match the new mm. Flush workers in suspend to guarantee that no worker sees the new mm and old VA in between. Signed-off-by: Steve Sistare --- drivers/vdpa/vdpa_user/vduse_dev.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index 0ddd4b8abecb..6b25457a037d 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -472,6 +472,18 @@ static void vduse_dev_reset(struct vduse_dev *dev) up_write(&dev->rwsem); } +static void vduse_flush_work(struct vduse_dev *dev) +{ + flush_work(&dev->inject); + + for (int i = 0; i < dev->vq_num; i++) { + struct vduse_virtqueue *vq = dev->vqs[i]; + + flush_work(&vq->inject); + flush_work(&vq->kick); + } +} + static int vduse_vdpa_set_vq_address(struct vdpa_device *vdpa, u16 idx, u64 desc_area, u64 driver_area, u64 device_area) @@ -713,6 +725,17 @@ static int vduse_vdpa_reset(struct vdpa_device *vdpa) return ret; } +static int vduse_vdpa_suspend(struct vdpa_device *vdpa) +{ + struct vduse_dev *dev = vdpa_to_vduse(vdpa); + + down_write(&dev->rwsem); + vduse_flush_work(dev); + up_write(&dev->rwsem); + + return 0; +} + static u32 vduse_vdpa_get_generation(struct vdpa_device *vdpa) { struct vduse_dev *dev = vdpa_to_vduse(vdpa); @@ -794,6 +817,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .set_vq_affinity = vduse_vdpa_set_vq_affinity, .get_vq_affinity = vduse_vdpa_get_vq_affinity, .reset = vduse_vdpa_reset, + .suspend = vduse_vdpa_suspend, .set_map = vduse_vdpa_set_map, .free = vduse_vdpa_free, };