From patchwork Tue Nov 21 07:30:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 167522 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp453819vqb; Mon, 20 Nov 2023 23:32:18 -0800 (PST) X-Google-Smtp-Source: AGHT+IHQc4E+LOaZnwjwlNbdtbZSboSDOfY0QIeshE3FxLCmCzjB8YwAemP9NTwDpBVE2i4XLR0Q X-Received: by 2002:a05:6870:ce8e:b0:1e9:ce72:35e0 with SMTP id xe14-20020a056870ce8e00b001e9ce7235e0mr11257315oab.0.1700551938358; Mon, 20 Nov 2023 23:32:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700551938; cv=none; d=google.com; s=arc-20160816; b=buZKvMsWXVfTUgMOd68JsSNM7GN+PjWz6aC3nK3fZc9MGCip+DiE4zvm9rjRziZZVN 5NV5yiGj1CaOkk4bSFGsoPqEJ2Gon7QAy2j0cSfAMy0aBy2fwO1YhswO5v8ECN7ULrCc 7/GhFhk2ng+ir0EL6QXB+DJmJ2lcHVmywdKi8E3larnqQzlhO2EswoIV62pJpfYd3GqK qTQDsh6qGVzi6jgp57DxZ+OuZhry1TwKXfEy/9wngnwCFbPW9jYbWEaDljjVf+pHQ2F6 BZ3zopJ/cYREfEDpRI4CLBeKjf/3ZqY3z1F+qiw5S4qECp6rJ4v85+qKMYgmFBHBL4dr zAHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=3tApwkEnppPf1Qrj7Li8zNPuhfxV/BinwmEu4nnIOfk=; fh=ARFoO5Pk7SniYIzyAg9Z8eUcw3AUCiaEHOgWusT66QQ=; b=bmvRt8wUK7yCXcfGl/hL96n+7ijI0WIPuPliTwdyj3W62Bw0akGpnyx5xHQUe4Monv Z3/QVMb/wTIspVa0bTYyiZmBzU/2fs4fZuB69GAg0elf017OSWzIdLSJLV/OhAHbOup/ h8nmHBK2l1iwizQ1vZeQFi0ohlpMByvjd6oUCtClrbv2DfyK26CQA+Lf2I1flZzpAXZ0 bqmHMYgUQDzTkyqwLzRHy80ZOZPgu6VGdGIDLmKZryDgd3f+DRt/V/Dm3D89woBHueoz nyto8lIf9cVh3YJXzC8n2JAMfH+z/67lTgA+tyayHl329XZvqSK6bW4s7AYW5rQuShKg EJbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=fMudZnVc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id q36-20020a631f64000000b0059f0cebd04csi9466860pgm.722.2023.11.20.23.32.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 23:32:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=fMudZnVc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 2D9BD8069342; Mon, 20 Nov 2023 23:31:24 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229824AbjKUHbL (ORCPT + 99 others); Tue, 21 Nov 2023 02:31:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229469AbjKUHbJ (ORCPT ); Tue, 21 Nov 2023 02:31:09 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B798DF5 for ; Mon, 20 Nov 2023 23:31:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700551865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3tApwkEnppPf1Qrj7Li8zNPuhfxV/BinwmEu4nnIOfk=; b=fMudZnVcy5CeCrEl5iInr12m0K9RP667vn0SZEdAvkbQD56XKwXDJuLs/QK/aubWR1K/g9 VCF7DF/FlP9A3aUnpYH6E6dc7NEMNPW96R4IfC/BKGpJGHcomSkRTOvF8PRl7r0Uf3i9ku xLLnSNR34ikJ55opHGSY1qc5giqcejs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-621-feDxu8XYPN2sBP8rRuve5g-1; Tue, 21 Nov 2023 02:31:00 -0500 X-MC-Unique: feDxu8XYPN2sBP8rRuve5g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E1CA385AFC0; Tue, 21 Nov 2023 07:30:59 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.165]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3238920268CF; Tue, 21 Nov 2023 07:30:56 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v2 1/5] vduse: Add function to get/free the pages for reconnection Date: Tue, 21 Nov 2023 15:30:46 +0800 Message-Id: <20231121073050.287080-2-lulu@redhat.com> In-Reply-To: <20231121073050.287080-1-lulu@redhat.com> References: <20231121073050.287080-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 20 Nov 2023 23:31:24 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783157949068967530 X-GMAIL-MSGID: 1783157949068967530 Add the function vduse_alloc_reconnnect_info_mem and vduse_alloc_reconnnect_info_mem In this 2 function, vduse will get/free (vq_num + 1)*page   Page 0 will be used to save the reconnection information, The Userspace App will maintain this. Page 1 ~ vq_num + 1 will save the reconnection information for vqs. Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 80 ++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index 26b7e29cb900..6209e2f00278 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -41,6 +41,16 @@ #define VDUSE_IOVA_SIZE (128 * 1024 * 1024) #define VDUSE_MSG_DEFAULT_TIMEOUT 30 +/* struct vdpa_reconnect_info saved the alloc pages info + * these pages will mmaped to userspace for reconnection + */ +struct vdpa_reconnect_info { + /* Offset (within vm_file) in PAGE_SIZE */ + u32 index; + /* virtual address for this page*/ + unsigned long vaddr; +}; + struct vduse_virtqueue { u16 index; u16 num_max; @@ -57,6 +67,7 @@ struct vduse_virtqueue { struct vdpa_callback cb; struct work_struct inject; struct work_struct kick; + struct vdpa_reconnect_info reconnect_info; }; struct vduse_dev; @@ -106,6 +117,7 @@ struct vduse_dev { u32 vq_align; struct vduse_umem *umem; struct mutex mem_lock; + struct vdpa_reconnect_info reconnect_info; }; struct vduse_dev_msg { @@ -1030,6 +1042,64 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev, return ret; } +static int vduse_alloc_reconnnect_info_mem(struct vduse_dev *dev) +{ + struct vdpa_reconnect_info *info; + struct vduse_virtqueue *vq; + + for (int i = 0; i < dev->vq_num + 1; i++) { + if (i == 0) { + /*page 0 is use to save status,Userland APP will use this to + *save the information needed in reconnection, + *kernel don't need to maintain this + */ + info = &dev->reconnect_info; + info->vaddr = get_zeroed_page(GFP_KERNEL); + if (info->vaddr == 0) + return -ENOMEM; + /* index is vm Offset in PAGE_SIZE */ + info->index = 0; + } + + /*page 1~ vq_num + 1 save the reconnect info for vq*/ + vq = &dev->vqs[i]; + info = &vq->reconnect_info; + info->vaddr = get_zeroed_page(GFP_KERNEL); + if (info->vaddr == 0) + return -ENOMEM; + + info->index = i + 1; + } + + return 0; +} + +static int vduse_free_reconnnect_info_mem(struct vduse_dev *dev) +{ + struct vdpa_reconnect_info *info; + struct vduse_virtqueue *vq; + + for (int i = 0; i < dev->vq_num + 1; i++) { + if (i == 0) { + info = &dev->reconnect_info; + if (info->vaddr) + free_page(info->vaddr); + info->index = 0; + info->vaddr = 0; + } + + vq = &dev->vqs[i]; + info = &vq->reconnect_info; + + if (info->vaddr) + free_page(info->vaddr); + info->vaddr = 0; + info->index = 0; + } + + return 0; +} + static long vduse_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -1390,6 +1460,8 @@ static int vduse_destroy_dev(char *name) mutex_unlock(&dev->lock); return -EBUSY; } + vduse_free_reconnnect_info_mem(dev); + dev->connected = true; mutex_unlock(&dev->lock); @@ -1542,9 +1614,17 @@ static int vduse_create_dev(struct vduse_dev_config *config, ret = PTR_ERR(dev->dev); goto err_dev; } + + ret = vduse_alloc_reconnnect_info_mem(dev); + if (ret < 0) + goto err_mem; + __module_get(THIS_MODULE); return 0; + +err_mem: + vduse_free_reconnnect_info_mem(dev); err_dev: idr_remove(&vduse_idr, dev->minor); err_idr: From patchwork Tue Nov 21 07:30:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 167524 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp454451vqb; Mon, 20 Nov 2023 23:34:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IEvIGanAwd/eWLMJoMte1Ceo1POA3iGhFtoK0wDGiBu/XlkTfH0UjmpUYoBRe2Kib+4DNau X-Received: by 2002:a05:6808:2390:b0:3b2:e520:dfce with SMTP id bp16-20020a056808239000b003b2e520dfcemr5035282oib.58.1700552047894; Mon, 20 Nov 2023 23:34:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700552047; cv=none; d=google.com; s=arc-20160816; b=rijYrzA6HI+6jqc+OVxB2fQUVQ1iQcyIcDTwSD17YKdg2NvS23gMfSoTU6PZADQpvP 9LzswQ/LT8ZC8OZTVvB8wCp3R5AxiQFfsuTg+9/YcUaiYg1DX2TCrOih0OPcUOvFnLvo +j2qxzOa0FaAyvCC+P/+yQi0bZQYpWFQlg5PhQep8yRVkj7qOe92TM1EK8MQek9BEz3V 6Ay7p0Yld8i4LYeSYpqjqNjH8CMzphPx68uTISA2u+8M396L9o8X3A5s6v0+brPY1w5o lEsIQNdhqQfX8cey2oMcH3g0f6aM3Nu32wFJ6DWuRokwLTxYhQyPSZgthqCR4jL5oRAs MUNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=zVxsZpHEP/sfaMQQhdclVFIY4BoolokQBaiLl5ZMHTY=; fh=ARFoO5Pk7SniYIzyAg9Z8eUcw3AUCiaEHOgWusT66QQ=; b=CUk6lpFIcvwXQ41pxpb4tuNk1V4Yx/cmGf2+l0BJ3UUd/wu5ZPvQXnUwotJSr/kB2T nzYJ8dHE8cIz7NpbyEM8LZO31Huaj962uxJp/zJ9dk6BTUzPUPmVwZHT5W+PdoK1iylp zWSmaK7ymwUfcOIS5fh1hm181zI2q+FeLmBr8XZW7hSSsSwGzfZ3Y9L61e7N1Ug2MFUk UBLpxHifCAzD8AqHKB8A1ThwrVzPO3mbaELutSLwXXZbJ98qcRc6H63VpEo0BjKAOED9 0bn0M0l1KmjAP5ucyQS21s6I85if/VcTMOs1d0as7v/KmocoFO4cFDSithZ3/t8fRQ5I vZdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=J0JNd9s3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id s22-20020a056a00195600b006c0fe2dfec1si10079653pfk.72.2023.11.20.23.34.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 23:34:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=J0JNd9s3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 0BDEF80A22ED; Mon, 20 Nov 2023 23:31:25 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229815AbjKUHbP (ORCPT + 99 others); Tue, 21 Nov 2023 02:31:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229747AbjKUHbL (ORCPT ); Tue, 21 Nov 2023 02:31:11 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C5A4F5 for ; Mon, 20 Nov 2023 23:31:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700551865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zVxsZpHEP/sfaMQQhdclVFIY4BoolokQBaiLl5ZMHTY=; b=J0JNd9s3dkuzwcsgucZm3mF+bmX+1nvXczrA8yREFZdnUqlZmZSHNm51yGn4A2V9+9lShL WRuy7/fKngCi4+kk4mKi05+FVJTd5DLxZHO+sHaUTzB4Gv4fejyWSONa2x1Ssofhi+Jk6R 1WqepIeeGuCJ1lWZTvYa1BocMEQHKfM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-191-6p5Rl5WSMAOaFohM4XU9kQ-1; Tue, 21 Nov 2023 02:31:03 -0500 X-MC-Unique: 6p5Rl5WSMAOaFohM4XU9kQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4AC788007B3; Tue, 21 Nov 2023 07:31:03 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.165]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E6AA20268CF; Tue, 21 Nov 2023 07:31:00 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v2 2/5] vduse: Add file operation for mmap Date: Tue, 21 Nov 2023 15:30:47 +0800 Message-Id: <20231121073050.287080-3-lulu@redhat.com> In-Reply-To: <20231121073050.287080-1-lulu@redhat.com> References: <20231121073050.287080-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 23:31:25 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783158064035710707 X-GMAIL-MSGID: 1783158064035710707 Add the operation for mmap, The user space APP will use this function to map the pages to userspace Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 79 ++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index 6209e2f00278..ccb30e98bab5 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1376,6 +1376,83 @@ static struct vduse_dev *vduse_dev_get_from_minor(int minor) return dev; } +static vm_fault_t vduse_vm_fault(struct vm_fault *vmf) +{ + struct vduse_dev *dev = vmf->vma->vm_file->private_data; + struct vm_area_struct *vma = vmf->vma; + u16 index = vma->vm_pgoff; + struct vduse_virtqueue *vq; + struct vdpa_reconnect_info *info; + + /* index 0 page reserved for vduse status*/ + if (index == 0) { + info = &dev->reconnect_info; + } else { + /* index 1+vq_number page reserved for vduse vqs*/ + vq = &dev->vqs[index - 1]; + info = &vq->reconnect_info; + } + if (remap_pfn_range(vma, vmf->address & PAGE_MASK, + PFN_DOWN(virt_to_phys((void *)info->vaddr)), + PAGE_SIZE, vma->vm_page_prot)) + return VM_FAULT_SIGBUS; + return VM_FAULT_NOPAGE; +} + +static const struct vm_operations_struct vduse_vm_ops = { + .fault = vduse_vm_fault, +}; + +static int vduse_dev_mmap(struct file *file, struct vm_area_struct *vma) +{ + struct vduse_dev *dev = file->private_data; + struct vdpa_reconnect_info *info; + unsigned long index = vma->vm_pgoff; + struct vduse_virtqueue *vq; + + if (vma->vm_end - vma->vm_start != PAGE_SIZE) + return -EINVAL; + if ((vma->vm_flags & VM_SHARED) == 0) + return -EINVAL; + + /*check if Userspace App map the page number larger than kernel allocated*/ + if (index > dev->vq_num + 1) + return -EINVAL; + + /* index 0 page reserved for vduse status*/ + if (index == 0) { + info = &dev->reconnect_info; + } else { + /* index 1+vq_number page reserved for vduse vqs*/ + vq = &dev->vqs[index - 1]; + info = &vq->reconnect_info; + } + /*check if map pages was allocated and inited by kernel */ + if (info->vaddr == 0) + return -EOPNOTSUPP; + + /* check if the address is page aligned, if not, + * this address maybe damaged + */ + if (virt_to_phys((void *)info->vaddr) & (PAGE_SIZE - 1)) + return -EINVAL; + + /* check if Userspace App mapped the correct size + * the userspace App should map one page each time + */ + if (vma->vm_end - vma->vm_start != PAGE_SIZE) + return -EOPNOTSUPP; + /* VM_IO: set as a memory-mapped I/O region,This will for vq information + * VM_PFNMAP: only need the pure PFN + * VM_DONTEXPAND: do not need to use mremap() in this function + * VM_DONTDUMP:Do not include in the core dump + */ + vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP); + vma->vm_ops = &vduse_vm_ops; + + return 0; +} + static int vduse_dev_open(struct inode *inode, struct file *file) { int ret; @@ -1408,6 +1485,8 @@ static const struct file_operations vduse_dev_fops = { .unlocked_ioctl = vduse_dev_ioctl, .compat_ioctl = compat_ptr_ioctl, .llseek = noop_llseek, + .mmap = vduse_dev_mmap, + }; static struct vduse_dev *vduse_dev_create(void) From patchwork Tue Nov 21 07:30:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 167523 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp453824vqb; Mon, 20 Nov 2023 23:32:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IEo9CTPK74hGtEdfaPuua1rHZguA2wTafiXzPYZbm8Ika7GLbvl4imfQRe1pDT5KyQ/VSxl X-Received: by 2002:a05:6808:150d:b0:3a8:1c07:17d3 with SMTP id u13-20020a056808150d00b003a81c0717d3mr15289757oiw.19.1700551939046; Mon, 20 Nov 2023 23:32:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700551939; cv=none; d=google.com; s=arc-20160816; b=eR66+VVfPTxcbN+CCyOL+/7JQ3SfBlP0BRFe1BeMzrSsUzQUM7Z8F0NEJetH7VAXyy s6sz+AXuhxejEerKJLLMRtP0GlfGM22OU0VNfEljhxc4ZBDeZDS6cIPElPY0Yt2/g+CW YmTXojRdEmsIcIHQw/RJ/3/jjE4/pxyWcUzHDUi9WV86o1GVogXsB+mVy6VOLY6BaLW0 sPMTOpQ3cewkDgSI+U4crq00yHUoD14Ectvl6jXVW4hTgN814dfvDgbkuKxX5IHSloip VdKJplZsEPCD7WyJAAHe1nm7YyEivL5lSfNyZ1WhxDQalzfIGArltaVQfkLlGvaCFBJ0 MD/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0iZa9dT80BCE+foVlf/TZ2RRVWFHn8gNFqGFDP3tw+0=; fh=ARFoO5Pk7SniYIzyAg9Z8eUcw3AUCiaEHOgWusT66QQ=; b=FnJkeIVZdkHnLZF4qcSNMa8sU+ET7lWtsCKo7o7bDqYUU0pWHCA7ey72RfbVaIqzsy YjLR5RJepYBWemq17HZ/44oFroR8ztpbqh74wa2B3xXO3u4boQanWf15n5ilGPCqvVtp dfVhTr40t8uKI/KPWSa2zxsNsA+pt2OYoTDIApATgo354wpzxIMOZhYnlpzzje2V77jW Tv0d4zoN821pFkmZBUvvPjZtjsa4WCSnyQZGTatshSpXp7aCr1V7pNGocw36NNJksITa El7pS88t5aSJJNhfN2NNo1yZsswYjhg/1wiIAE8obJkYHDmBvGMv9pFEmuO1LbwTSr3l m1wg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=gt3drWoR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id n12-20020a634d4c000000b0058556a636dbsi8678703pgl.434.2023.11.20.23.32.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 23:32:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=gt3drWoR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id D970780BC53E; Mon, 20 Nov 2023 23:31:52 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230089AbjKUHbT (ORCPT + 99 others); Tue, 21 Nov 2023 02:31:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229771AbjKUHbO (ORCPT ); Tue, 21 Nov 2023 02:31:14 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA085C8 for ; Mon, 20 Nov 2023 23:31:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700551869; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0iZa9dT80BCE+foVlf/TZ2RRVWFHn8gNFqGFDP3tw+0=; b=gt3drWoRcXu8WPxeKfUFdVs+nD4H3ojB0l0mA5vva9/fDeL6RoN72+9nFVuCY6J3v39zkg M6k0Wc1EtjtIWZeEfh0mC9NzChWz3WCLwC8a7Pzi3GxoDtJlHWLod0I+nkH2SDBQ5/iER9 uHbEYAkZYUXmzXfQ8O6CqGsOw3oMYDM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-kPUB6VRzNNmmbs2KrBeXsg-1; Tue, 21 Nov 2023 02:31:06 -0500 X-MC-Unique: kPUB6VRzNNmmbs2KrBeXsg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A74B11C0896D; Tue, 21 Nov 2023 07:31:06 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.165]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB04E20268CF; Tue, 21 Nov 2023 07:31:03 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v2 3/5] vduse: Add new ioctl VDUSE_GET_RECONNECT_INFO Date: Tue, 21 Nov 2023 15:30:48 +0800 Message-Id: <20231121073050.287080-4-lulu@redhat.com> In-Reply-To: <20231121073050.287080-1-lulu@redhat.com> References: <20231121073050.287080-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Mon, 20 Nov 2023 23:31:53 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783157949871973901 X-GMAIL-MSGID: 1783157949871973901 In VDUSE_GET_RECONNECT_INFO, the Userspace App can get the map size and The number of mapping memory pages from the kernel. The userspace App can use this information to map the pages. Add struct vhost_reconnect_data/vhost_reconnect_vring for sync the information in reconnection Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 15 +++++++++ include/uapi/linux/vduse.h | 50 ++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index ccb30e98bab5..d0fe9a7e86ab 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1343,6 +1343,21 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, ret = 0; break; } + case VDUSE_GET_RECONNECT_INFO: { + struct vduse_reconnect_mmap_info info; + + ret = -EFAULT; + if (copy_from_user(&info, argp, sizeof(info))) + break; + + info.size = PAGE_SIZE; + info.max_index = dev->vq_num + 1; + + if (copy_to_user(argp, &info, sizeof(info))) + break; + ret = 0; + break; + } default: ret = -ENOIOCTLCMD; break; diff --git a/include/uapi/linux/vduse.h b/include/uapi/linux/vduse.h index 11bd48c72c6c..c0b7133aebfd 100644 --- a/include/uapi/linux/vduse.h +++ b/include/uapi/linux/vduse.h @@ -3,6 +3,7 @@ #define _UAPI_VDUSE_H_ #include +#include #define VDUSE_BASE 0x81 @@ -350,4 +351,53 @@ struct vduse_dev_response { }; }; +/** + * struct vhost_reconnect_data - saved the reconnect info for device + * @version; version for userspace APP + * @reconnected: indetify if this is reconnected. userspace APP needs set this bit to 1 + * while reconnecting.kernel will use this bit to indetify if this is + * reconnect + * @features; Device features negotiated in the last connect. + * @status; Device status in last reconnect + * @nr_vrings; number of active vqs in last connect + * @struct virtio_net_config config; the config in last connect + */ + +struct vhost_reconnect_data { + __u32 version; + __u32 reconnected; + __u64 features; + __u8 status; + __u32 nr_vrings; + struct virtio_net_config config; +}; + +/** + * struct vhost_reconnect_vring -saved the reconnect info for vqs + * when use split mode will only use last_avail_idx + * when use packed mode will use both last_avail_idx and avail_wrap_counter + * userspace App + * @last_avail_idx: device last available index + * @avail_wrap_counter: Driver ring wrap counter + */ +struct vhost_reconnect_vring { + __u16 last_avail_idx; + __u16 avail_wrap_counter; +}; + +/** + * struct vduse_reconnect_mmap_info + * @size: mapping memory size, here we use page_size + * @max_index: the number of pages allocated in kernel,just + * use for sanity check + */ + +struct vduse_reconnect_mmap_info { + __u32 size; + __u32 max_index; +}; + +#define VDUSE_GET_RECONNECT_INFO \ + _IOWR(VDUSE_BASE, 0x1b, struct vduse_reconnect_mmap_info) + #endif /* _UAPI_VDUSE_H_ */ From patchwork Tue Nov 21 07:30:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 167521 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp453779vqb; Mon, 20 Nov 2023 23:32:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IEyLZVOguM1aRsXECR/MxCwSJWOu7TpRZu2g86feHEcKyxmVAbiNCh+SCkLd0LBwe7oS/wd X-Received: by 2002:a05:6870:a706:b0:1ef:d2b2:fe16 with SMTP id g6-20020a056870a70600b001efd2b2fe16mr9047583oam.43.1700551932834; Mon, 20 Nov 2023 23:32:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700551932; cv=none; d=google.com; s=arc-20160816; b=CPXsOWJZfAhuo7REakRdX1aFDMcmZSAGCq95a3fSDOPGmBnTl5G2BbZZOsHr8yALTP vVrwxob3ScTNRegMuZBiQOevGhog8dzyoGeavOXLxjj5qm+ajYZKoK6nHqdyD2frl6+9 Zick3C6lxl0NbpX5pxeldgypePlOvLtT/z4GHPSc+BXosuJ8c9y6ZnghUIVShl1Ijojt 6uv1OaHriuufMazB3S02bm61NVPAMZVByhpOnWZNJPcL9h1nJhSABZDHNaDM5nDwE3VV 3xbcfcmTBqul8adE/86vvYv2Jz9crWeWb6A99xnXLCKrLrzBnwryda63OQTfYFfg0424 FyaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=OAkVxc3aGCIVy+mum1Tmv8tqRWzijzVSekSFIcVpdTI=; fh=ARFoO5Pk7SniYIzyAg9Z8eUcw3AUCiaEHOgWusT66QQ=; b=SOOwFvwsNNcHYI1bUegYivbcpyMeDXltTwspuUqGEvaFID5BYc+JLN+KI99haQydap 4ZoMVFiCD0OCXfonR8II5+r8JApApdTgA7ENa+O7OsUYTB+YQLxtRml9H/rR5seixjyL FGDgEbw/DYcBzEAQwlitVS4SPa2uEN9qE2HlobD5eN7Q0ybwdRqfA+rDSKYHJiwIcuEB kFxW5a8TI7QE1MyiW6yh7UryQEc7Az5RtsQpwrXXrHevkVFI0bz8td2GkcwL8bMo8fAr VvBtCoz3m4Bqf463f/77MeHYXIILiOEFjSy7MvgzQVscM/7Ic8SbbE6f2xg1SqQJj9y9 yVvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=A0kMw3TI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id u17-20020a056a00125100b0068e4c3f3ac2si9894761pfi.245.2023.11.20.23.32.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 23:32:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=A0kMw3TI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 8631D80A9AAE; Mon, 20 Nov 2023 23:32:07 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229891AbjKUHbp (ORCPT + 99 others); Tue, 21 Nov 2023 02:31:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231390AbjKUHbh (ORCPT ); Tue, 21 Nov 2023 02:31:37 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96D9919A for ; Mon, 20 Nov 2023 23:31:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700551893; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OAkVxc3aGCIVy+mum1Tmv8tqRWzijzVSekSFIcVpdTI=; b=A0kMw3TIhs2EbU962e+26T7JKuBRfCzifPBOFlsOEzmydSStpHeyuICM3WnPwWI5mjeGC+ /NSCpogwkqVX10M3U52GfNVpOqshuiQ4vokJJsXDen60k3Q8H+2AiP88jmH1NhDQbpAGvj nc48Mtw/Go18XrupVKr4QXfqCbiFXxU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-34-CVgMNopqMVOpQb5uC-0KgA-1; Tue, 21 Nov 2023 02:31:32 -0500 X-MC-Unique: CVgMNopqMVOpQb5uC-0KgA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B5126857F7A; Tue, 21 Nov 2023 07:31:31 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.165]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0087D1C060B0; Tue, 21 Nov 2023 07:31:28 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v2 4/5] vduse: update the vq_info in ioctl Date: Tue, 21 Nov 2023 15:30:49 +0800 Message-Id: <20231121073050.287080-5-lulu@redhat.com> In-Reply-To: <20231121073050.287080-1-lulu@redhat.com> References: <20231121073050.287080-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 20 Nov 2023 23:32:07 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783157943513329330 X-GMAIL-MSGID: 1783157943513329330 In VDUSE_VQ_GET_INFO, the driver will sync the last_avail_idx with reconnect info, After mapping the reconnect pages to userspace The userspace App will update the reconnect_time in struct vhost_reconnect_vring, If this is not 0 then it means this vq is reconnected and will update the last_avail_idx Signed-off-by: Cindy Lu --- drivers/vdpa/vdpa_user/vduse_dev.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vduse_dev.c index d0fe9a7e86ab..6bc5fc2b88cc 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -1209,6 +1209,9 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, struct vduse_vq_info vq_info; struct vduse_virtqueue *vq; u32 index; + struct vdpa_reconnect_info *area; + struct vhost_reconnect_vring *vq_reconnect; + struct vhost_reconnect_data *dev_reconnect; ret = -EFAULT; if (copy_from_user(&vq_info, argp, sizeof(vq_info))) @@ -1225,6 +1228,12 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, vq_info.device_addr = vq->device_addr; vq_info.num = vq->num; + area = &dev->reconnect_info; + dev_reconnect = (struct vhost_reconnect_data *)area->vaddr; + + area = &vq->reconnect_info; + vq_reconnect = (struct vhost_reconnect_vring *)area->vaddr; + if (dev->driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { vq_info.packed.last_avail_counter = vq->state.packed.last_avail_counter; @@ -1234,9 +1243,22 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, vq->state.packed.last_used_counter; vq_info.packed.last_used_idx = vq->state.packed.last_used_idx; - } else + /*check if the vq is reconnect, if yes then update the last_avail_idx*/ + if (dev_reconnect->reconnected != 0) { + vq_info.packed.last_avail_idx = + vq_reconnect->last_avail_idx; + vq_info.packed.last_avail_counter = + vq_reconnect->avail_wrap_counter; + } + } else { vq_info.split.avail_index = vq->state.split.avail_index; + /*check if the vq is reconnect, if yes then update the last_avail_idx*/ + if (dev_reconnect->reconnected != 0) { + vq_info.split.avail_index = + vq_reconnect->last_avail_idx; + } + } vq_info.ready = vq->ready; From patchwork Tue Nov 21 07:30:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cindy Lu X-Patchwork-Id: 167520 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2b07:b0:403:3b70:6f57 with SMTP id io7csp453692vqb; Mon, 20 Nov 2023 23:31:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IGTOQhdGKsNVy7nm8O5BboRIfsTeN04cBANGb0rrPbjFa42/QHE3+8GxFE67DaL73Blexdn X-Received: by 2002:a9d:4d88:0:b0:6c4:f25f:612f with SMTP id u8-20020a9d4d88000000b006c4f25f612fmr9681189otk.28.1700551918746; Mon, 20 Nov 2023 23:31:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700551918; cv=none; d=google.com; s=arc-20160816; b=gEf6DE67lUpRTnpkZKBBEERkbw4czQ4ihJKRe4PWilD5fFQEPZwyCd5hTAydAaMmCL E9CrDJUoBNyZbl6UbBeUWu67ELyKc4pDWaVzcIZwffrwcNWvQN4jI8pP39wIX5CX6P/S r7DMn0rhEBdMwcABrazbAah6sLny8/mgfxJpXQ2FK2Cya3VjCkOTINSHAgs2rLzzekSD ocIz8hPbS5KZ4H0PYSWwDgf7kfrC5Y85Sdq1VZSXNGIcQ6oymtzkn6mgdyprC6uuFYOO S0EwfVFv4dmAkdPSumXC2kREci2GOWejMXnb91FwIq0Fgc/k1XPrwRLJStHo/8+NOPFI 6Uiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=77Bzl/+Kq0+MzNg2uBfJvKrlDQ1jm6M8H9m1x9Ji14Q=; fh=ARFoO5Pk7SniYIzyAg9Z8eUcw3AUCiaEHOgWusT66QQ=; b=LUHlpwBTToXWsBsm5FPqi0wUIT1U5+jlaOFqYHbCUGNO6hfH6xW9KmXX3BOHjiiuPj 46vWgbzeClIsw6ZJCnWr2tSaUML2iZOUcnk7nQuTRg9mIwYItKkXJOfK/my+u+8EH8G/ L98gsSb8Az2QQVtq22mzmm8zttgk7XGXsHoOx6Hcgb+6Bv9SDN57fF/h4xfTS9+7aOL1 G/TRaf1QvU/3A7xdvgwHoOc0u609d4EAeDNfxfWrgVtDC8JrqdDuqwxKQ3cooSq61vC1 Khx4svSn4JHRvkow2ufDtKU8YR4AuxfkLvQOTcm1r19XvTdTXWDcgDW15gei2pY38OVT w5sw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=G9bNE1EP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id 6-20020a630806000000b005b7dd20f8c1si9741164pgi.20.2023.11.20.23.31.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Nov 2023 23:31:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=G9bNE1EP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 01455807780D; Mon, 20 Nov 2023 23:31:48 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229771AbjKUHbt (ORCPT + 99 others); Tue, 21 Nov 2023 02:31:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229747AbjKUHbn (ORCPT ); Tue, 21 Nov 2023 02:31:43 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FDBF10E for ; Mon, 20 Nov 2023 23:31:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700551899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=77Bzl/+Kq0+MzNg2uBfJvKrlDQ1jm6M8H9m1x9Ji14Q=; b=G9bNE1EPKLxDZi+B3kkNegL2Jeft7dD1SS7TQiSudQ9jlOXgv29gDnJg2zAb/WFJk4xrhI gSuoq5Uv6Bi4GdiByrtPeHSaPoxwVbN05nFrCSQm7zaXXmhVBrzYCzsLYtAb2iKVnCJi2R egN9LBEvpVy/tfYMNsBUyEY0GfVaYLA= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-602-ETEATdwXPl-IT_04BPdwNQ-1; Tue, 21 Nov 2023 02:31:35 -0500 X-MC-Unique: ETEATdwXPl-IT_04BPdwNQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1F6543804A4D; Tue, 21 Nov 2023 07:31:35 +0000 (UTC) Received: from server.redhat.com (unknown [10.72.112.165]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60E281C060B0; Tue, 21 Nov 2023 07:31:32 +0000 (UTC) From: Cindy Lu To: lulu@redhat.com, jasowang@redhat.com, mst@redhat.com, xieyongji@bytedance.com, linux-kernel@vger.kernel.org, maxime.coquelin@redhat.com Subject: [PATCH v2 5/5] Documentation: Add reconnect process for VDUSE Date: Tue, 21 Nov 2023 15:30:50 +0800 Message-Id: <20231121073050.287080-6-lulu@redhat.com> In-Reply-To: <20231121073050.287080-1-lulu@redhat.com> References: <20231121073050.287080-1-lulu@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Mon, 20 Nov 2023 23:31:49 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783157928631731606 X-GMAIL-MSGID: 1783157928631731606 Add the document to explained how the reconnect process include how the Userspace App need to do and how it works with kernel Signed-off-by: Cindy Lu --- Documentation/userspace-api/vduse.rst | 29 +++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/Documentation/userspace-api/vduse.rst b/Documentation/userspace-api/vduse.rst index bdb880e01132..6e01c21d94df 100644 --- a/Documentation/userspace-api/vduse.rst +++ b/Documentation/userspace-api/vduse.rst @@ -231,3 +231,32 @@ able to start the dataplane processing as follows: after the used ring is filled. For more details on the uAPI, please see include/uapi/linux/vduse.h. + +HOW VDUSE devices reconnectoin works +---------------- +0. Userspace APP checks if the device /dev/vduse/vduse_name exists, if not create + the device. If yes means this is reconnect, goto 3 + +1. Create a new VDUSE instance with ioctl(VDUSE_CREATE_DEV) on + /dev/vduse/control. + +2. In ioctl(VDUSE_CREATE_DEV), the kernel alloc the memory to sync the reconnect + information. + +3. Userspace App will mmap the pages to userspace + + If this first time to connect, userspace App need save the reconnect + information (struct vhost_reconnect_data) in mapped pages + + If this is reconnect, userspace App need to check if the saved information + OK to reconnect, Also userspace App MUST set the bit reconnected in + struct vhost_reconnect_data to 1. (kernel will use this bit to identify if the + userAPP is reconnected ) + +4. Successfully start the userspace App. + userspace APP need to call ioctl VDUSE_VQ_GET_INFO to sync the vq information. also + APP need to save the reconnect information (struct vhost_reconnect_vring) + while running + +5. When the Userspace App exit, Userspace App need to unmap all the reconnect + pages