Message ID | 20230703142514.363256-1-eperezma@redhat.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp574084vqx; Mon, 3 Jul 2023 07:43:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6U/PMMyhhXyC/D7f9rxy8zgXYNZoTgwcMW7bW7a3tuyFCNQoz7IaVWRjfAm6mxKaPSkVVy X-Received: by 2002:a05:6a20:12cf:b0:112:cf5:d5cc with SMTP id v15-20020a056a2012cf00b001120cf5d5ccmr16089138pzg.1.1688395399253; Mon, 03 Jul 2023 07:43:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688395399; cv=none; d=google.com; s=arc-20160816; b=PNRVWF0YyYl/f2cqj5mkY8NKdu4qRw07y5LW4a6g8z3fTpdbxKkAN+e/Rvu2prTmku 0X3uBl07FbuqMsN9riSXnAju7DdKP+VtrrWdL0bGrLbNIVh1FYTd0cd48PJ77WAArOGL iTfYzkwY/a6hYwRbImniqDVfsU9o9AlQPDk10p67OvTnoOfIlmBAgUQ+ZGFkej//IgmB IiNcRhkY/uyVWKsUoqzXW9Q0npcwbjFTtxMbzVvqASAeFfJTEZoKd4OMzrNg2PN4KIzj VTqS+NyVDSk6QAttEpRnTj5gZ8/jzRJt1qk7b/8WhMhHZqcdZ55uHf5CFDOgT6BfJgxR g8hQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=ZHqn83Z4nuY+pmIfzFaoPwoFjguWWUwYIDMm/eINnuY=; fh=A0DgUgVdUmtzGXJBFwB6c7s3C1g8dVY08OSGpwg54Pw=; b=X+6liHmzLbAd5dr7Lc0DnAtbRblA1bqrrqIShmgMaBJZrUYLLsHrMR/Xd6BI8evq1K Y1ON5yqotScbtih+Tt0ceh88wLGWX9HSwXQK8Kzq3ursHDUA65I7MJF13eojeO1RogO2 NBjV7ZetAB7M2WisAmMhsy1COOFPRieNCuxc9/L6sG6AJZAgT4lhMRckP+f7jJw7jyZF XY3TeBZS50wGnKhHm2FPnupDyO2Joww07z+x3xvpODyme7qdpv9CNrw8bm2czFXP9ASp G7c2btsXCjzDuzhJ/yXiFKF/YfUrqts4kbU45C7Mi71dAGA4aM/SC6TrdmWtHs3eh/pt OFWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="Mv/FYPcz"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r22-20020a632b16000000b005574c3ac68esi2170156pgr.489.2023.07.03.07.43.00; Mon, 03 Jul 2023 07:43:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="Mv/FYPcz"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230149AbjGCO0d (ORCPT <rfc822;ivan.orlov0322@gmail.com> + 99 others); Mon, 3 Jul 2023 10:26:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229792AbjGCO0G (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 3 Jul 2023 10:26:06 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30176B3 for <linux-kernel@vger.kernel.org>; Mon, 3 Jul 2023 07:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1688394323; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZHqn83Z4nuY+pmIfzFaoPwoFjguWWUwYIDMm/eINnuY=; b=Mv/FYPczBDOKFYEX90k9g6S6XSFSOgNw+bg9VXeaj1/1d8WdwMJVQi1GbUVGGe4WDW1byo ims2MGJgSiqQvTyzlLtbapMtJ7gXHFjCJ8dhLxbkjgZgeGfEeCBFf5bY3s3tAwbXKjggeV 7o2yZeY+8JzYJAe1liTNus37BuLYxYU= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-204-lyG-mLyyMWCT9OVj9mlS3Q-1; Mon, 03 Jul 2023 10:25:18 -0400 X-MC-Unique: lyG-mLyyMWCT9OVj9mlS3Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D43BB3813F2C; Mon, 3 Jul 2023 14:25:17 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.105]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6F7F32014E17; Mon, 3 Jul 2023 14:25:16 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= <eperezma@redhat.com> To: mst@redhat.com Cc: linux-kernel@vger.kernel.org, Dragos Tatulea <dtatulea@nvidia.com>, virtualization@lists.linux-foundation.org, leiyang@redhat.com, Xuan Zhuo <xuanzhuo@linux.alibaba.com>, Si-Wei Liu <si-wei.liu@oracle.com>, Jason Wang <jasowang@redhat.com> Subject: [PATCH] mlx5_vdpa: offer VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK Date: Mon, 3 Jul 2023 16:25:14 +0200 Message-Id: <20230703142514.363256-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770410894366956113?= X-GMAIL-MSGID: =?utf-8?q?1770410894366956113?= |
Series |
mlx5_vdpa: offer VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK
|
|
Commit Message
Eugenio Perez Martin
July 3, 2023, 2:25 p.m. UTC
Offer this backend feature as mlx5 is compatible with it. It allows it
to do live migration with CVQ, dynamically switching between passthrough
and shadow virtqueue.
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++
1 file changed, 7 insertions(+)
Comments
On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > Offer this backend feature as mlx5 is compatible with it. It allows it > to do live migration with CVQ, dynamically switching between passthrough > and shadow virtqueue. > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Same comment. > --- > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > index 9138ef2fb2c8..5f309a16b9dc 100644 > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > @@ -7,6 +7,7 @@ > #include <uapi/linux/virtio_net.h> > #include <uapi/linux/virtio_ids.h> > #include <uapi/linux/vdpa.h> > +#include <uapi/linux/vhost_types.h> > #include <linux/virtio_config.h> > #include <linux/auxiliary_bus.h> > #include <linux/mlx5/cq.h> > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > flush_workqueue(ndev->mvdev.wq); > } > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > +{ > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > +} > + > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > { > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > .get_vq_align = mlx5_vdpa_get_vq_align, > .get_vq_group = mlx5_vdpa_get_vq_group, > .get_device_features = mlx5_vdpa_get_device_features, > + .get_backend_features = mlx5_vdpa_get_backend_features, > .set_driver_features = mlx5_vdpa_set_driver_features, > .get_driver_features = mlx5_vdpa_get_driver_features, > .set_config_cb = mlx5_vdpa_set_config_cb, > -- > 2.39.3
On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: >> Offer this backend feature as mlx5 is compatible with it. It allows it >> to do live migration with CVQ, dynamically switching between passthrough >> and shadow virtqueue. >> >> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > Same comment. to which? -Siwei > >> --- >> drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ >> 1 file changed, 7 insertions(+) >> >> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c >> index 9138ef2fb2c8..5f309a16b9dc 100644 >> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c >> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c >> @@ -7,6 +7,7 @@ >> #include <uapi/linux/virtio_net.h> >> #include <uapi/linux/virtio_ids.h> >> #include <uapi/linux/vdpa.h> >> +#include <uapi/linux/vhost_types.h> >> #include <linux/virtio_config.h> >> #include <linux/auxiliary_bus.h> >> #include <linux/mlx5/cq.h> >> @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) >> flush_workqueue(ndev->mvdev.wq); >> } >> >> +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) >> +{ >> + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); >> +} >> + >> static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) >> { >> struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); >> @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { >> .get_vq_align = mlx5_vdpa_get_vq_align, >> .get_vq_group = mlx5_vdpa_get_vq_group, >> .get_device_features = mlx5_vdpa_get_device_features, >> + .get_backend_features = mlx5_vdpa_get_backend_features, >> .set_driver_features = mlx5_vdpa_set_driver_features, >> .get_driver_features = mlx5_vdpa_get_driver_features, >> .set_config_cb = mlx5_vdpa_set_config_cb, >> -- >> 2.39.3
On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > to do live migration with CVQ, dynamically switching between passthrough > > > and shadow virtqueue. > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > Same comment. > to which? > > -Siwei VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it as a kernel/userspace ABI: what if one wants to start rings in some other specific order? As was discussed on list, a better promise is not to access ring until the 1st kick. vdpa can then do a kick when it wants the device to start accessing rings. > > > > > --- > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > 1 file changed, 7 insertions(+) > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > @@ -7,6 +7,7 @@ > > > #include <uapi/linux/virtio_net.h> > > > #include <uapi/linux/virtio_ids.h> > > > #include <uapi/linux/vdpa.h> > > > +#include <uapi/linux/vhost_types.h> > > > #include <linux/virtio_config.h> > > > #include <linux/auxiliary_bus.h> > > > #include <linux/mlx5/cq.h> > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > flush_workqueue(ndev->mvdev.wq); > > > } > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > +{ > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > +} > > > + > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > { > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > .get_device_features = mlx5_vdpa_get_device_features, > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > -- > > > 2.39.3
On Tue, Jul 4, 2023 at 6:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > and shadow virtqueue. > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > Same comment. > > to which? > > > > -Siwei > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > as a kernel/userspace ABI: what if one wants to start rings in some > other specific order? Just enable a queue by writing e.g 1 to queue_enable in a specific order? > As was discussed on list, a better promise is not to access ring > until the 1st kick. vdpa can then do a kick when it wants > the device to start accessing rings. Rethink about the ACCESS_AFTER_KICK, it sounds functional equivalent to allow queue_enable after DRIVER_OK, but it seems to have distanvages: A busy polling software device may disable notifications and never respond to register to any kick notifiers. ACCESS_AFTER_KICK will introduce overhead to those implementations. Thanks > > > > > > > > --- > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > 1 file changed, 7 insertions(+) > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > @@ -7,6 +7,7 @@ > > > > #include <uapi/linux/virtio_net.h> > > > > #include <uapi/linux/virtio_ids.h> > > > > #include <uapi/linux/vdpa.h> > > > > +#include <uapi/linux/vhost_types.h> > > > > #include <linux/virtio_config.h> > > > > #include <linux/auxiliary_bus.h> > > > > #include <linux/mlx5/cq.h> > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > flush_workqueue(ndev->mvdev.wq); > > > > } > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > +{ > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > +} > > > > + > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > { > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > -- > > > > 2.39.3 >
On Wed, Jul 05, 2023 at 01:11:37PM +0800, Jason Wang wrote: > On Tue, Jul 4, 2023 at 6:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > > and shadow virtqueue. > > > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > > Same comment. > > > to which? > > > > > > -Siwei > > > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > > as a kernel/userspace ABI: what if one wants to start rings in some > > other specific order? > > Just enable a queue by writing e.g 1 to queue_enable in a specific order? But then at driver ok time we don't know how many queues are there. > > As was discussed on list, a better promise is not to access ring > > until the 1st kick. vdpa can then do a kick when it wants > > the device to start accessing rings. > > Rethink about the ACCESS_AFTER_KICK, it sounds functional equivalent > to allow queue_enable after DRIVER_OK, but it seems to have > distanvages: > > A busy polling software device may disable notifications and never > respond to register to any kick notifiers. ACCESS_AFTER_KICK will > introduce overhead to those implementations. > > Thanks It's just the 1st kick, then you can disable. No? > > > > > > > > > > > --- > > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > @@ -7,6 +7,7 @@ > > > > > #include <uapi/linux/virtio_net.h> > > > > > #include <uapi/linux/virtio_ids.h> > > > > > #include <uapi/linux/vdpa.h> > > > > > +#include <uapi/linux/vhost_types.h> > > > > > #include <linux/virtio_config.h> > > > > > #include <linux/auxiliary_bus.h> > > > > > #include <linux/mlx5/cq.h> > > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > > flush_workqueue(ndev->mvdev.wq); > > > > > } > > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > > +{ > > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > > +} > > > > > + > > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > > { > > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > > -- > > > > > 2.39.3 > >
On Wed, Jul 5, 2023 at 1:31 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Jul 05, 2023 at 01:11:37PM +0800, Jason Wang wrote: > > On Tue, Jul 4, 2023 at 6:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > > > and shadow virtqueue. > > > > > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > > > Same comment. > > > > to which? > > > > > > > > -Siwei > > > > > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > > > as a kernel/userspace ABI: what if one wants to start rings in some > > > other specific order? > > > > Just enable a queue by writing e.g 1 to queue_enable in a specific order? > > > But then at driver ok time we don't know how many queues are there. There should be a device specific interface for this, for example, num_queue_paris. So the device knows at most how many queues there are. Or anything I miss? > > > > As was discussed on list, a better promise is not to access ring > > > until the 1st kick. vdpa can then do a kick when it wants > > > the device to start accessing rings. > > > > Rethink about the ACCESS_AFTER_KICK, it sounds functional equivalent > > to allow queue_enable after DRIVER_OK, but it seems to have > > distanvages: > > > > A busy polling software device may disable notifications and never > > respond to register to any kick notifiers. ACCESS_AFTER_KICK will > > introduce overhead to those implementations. > > > > Thanks > > It's just the 1st kick, then you can disable. No? Yes, but: 1) adding hooks for queue_enable 2) adding new codes to register event notifier and toggle the notifier 1) seems much easier? Or for most devices, it already behaves like this. Thanks > > > > > > > > > > > > > > > --- > > > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > @@ -7,6 +7,7 @@ > > > > > > #include <uapi/linux/virtio_net.h> > > > > > > #include <uapi/linux/virtio_ids.h> > > > > > > #include <uapi/linux/vdpa.h> > > > > > > +#include <uapi/linux/vhost_types.h> > > > > > > #include <linux/virtio_config.h> > > > > > > #include <linux/auxiliary_bus.h> > > > > > > #include <linux/mlx5/cq.h> > > > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > > > flush_workqueue(ndev->mvdev.wq); > > > > > > } > > > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > > > +{ > > > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > > > +} > > > > > > + > > > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > > > { > > > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > > > -- > > > > > > 2.39.3 > > > >
On Wed, Jul 05, 2023 at 01:47:44PM +0800, Jason Wang wrote: > On Wed, Jul 5, 2023 at 1:31 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Wed, Jul 05, 2023 at 01:11:37PM +0800, Jason Wang wrote: > > > On Tue, Jul 4, 2023 at 6:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > > > > > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > > > > and shadow virtqueue. > > > > > > > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > > > > Same comment. > > > > > to which? > > > > > > > > > > -Siwei > > > > > > > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > > > > as a kernel/userspace ABI: what if one wants to start rings in some > > > > other specific order? > > > > > > Just enable a queue by writing e.g 1 to queue_enable in a specific order? > > > > > > But then at driver ok time we don't know how many queues are there. > > There should be a device specific interface for this, for example, > num_queue_paris. So the device knows at most how many queues there > are. Or anything I miss? That's a device limitations. Does not tell the device how much is used. > > > > > > As was discussed on list, a better promise is not to access ring > > > > until the 1st kick. vdpa can then do a kick when it wants > > > > the device to start accessing rings. > > > > > > Rethink about the ACCESS_AFTER_KICK, it sounds functional equivalent > > > to allow queue_enable after DRIVER_OK, but it seems to have > > > distanvages: > > > > > > A busy polling software device may disable notifications and never > > > respond to register to any kick notifiers. ACCESS_AFTER_KICK will > > > introduce overhead to those implementations. > > > > > > Thanks > > > > It's just the 1st kick, then you can disable. No? > > Yes, but: > > 1) adding hooks for queue_enable > 2) adding new codes to register event notifier and toggle the notifier > > 1) seems much easier? Or for most devices, it already behaves like this. > > Thanks Well libvhostuser checks enabled queues at DRIVER_OK, does it not? > > > > > > > > > > > > > > > > > > > --- > > > > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > > @@ -7,6 +7,7 @@ > > > > > > > #include <uapi/linux/virtio_net.h> > > > > > > > #include <uapi/linux/virtio_ids.h> > > > > > > > #include <uapi/linux/vdpa.h> > > > > > > > +#include <uapi/linux/vhost_types.h> > > > > > > > #include <linux/virtio_config.h> > > > > > > > #include <linux/auxiliary_bus.h> > > > > > > > #include <linux/mlx5/cq.h> > > > > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > > > > flush_workqueue(ndev->mvdev.wq); > > > > > > > } > > > > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > > > > +{ > > > > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > > > > +} > > > > > > > + > > > > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > > > > { > > > > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > > > > -- > > > > > > > 2.39.3 > > > > > >
On Wed, Jul 5, 2023 at 2:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Jul 05, 2023 at 01:47:44PM +0800, Jason Wang wrote: > > On Wed, Jul 5, 2023 at 1:31 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Wed, Jul 05, 2023 at 01:11:37PM +0800, Jason Wang wrote: > > > > On Tue, Jul 4, 2023 at 6:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > > > > > > > > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > > > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > > > > > and shadow virtqueue. > > > > > > > > > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > > > > > Same comment. > > > > > > to which? > > > > > > > > > > > > -Siwei > > > > > > > > > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > > > > > as a kernel/userspace ABI: what if one wants to start rings in some > > > > > other specific order? > > > > > > > > Just enable a queue by writing e.g 1 to queue_enable in a specific order? > > > > > > > > > But then at driver ok time we don't know how many queues are there. > > > > There should be a device specific interface for this, for example, > > num_queue_paris. So the device knows at most how many queues there > > are. Or anything I miss? > > That's a device limitations. Does not tell the device how much is used. I think I miss something, how kick differs from queue_enable in this way? > > > > > > > > > As was discussed on list, a better promise is not to access ring > > > > > until the 1st kick. vdpa can then do a kick when it wants > > > > > the device to start accessing rings. > > > > > > > > Rethink about the ACCESS_AFTER_KICK, it sounds functional equivalent > > > > to allow queue_enable after DRIVER_OK, but it seems to have > > > > distanvages: > > > > > > > > A busy polling software device may disable notifications and never > > > > respond to register to any kick notifiers. ACCESS_AFTER_KICK will > > > > introduce overhead to those implementations. > > > > > > > > Thanks > > > > > > It's just the 1st kick, then you can disable. No? > > > > Yes, but: > > > > 1) adding hooks for queue_enable > > 2) adding new codes to register event notifier and toggle the notifier > > > > 1) seems much easier? Or for most devices, it already behaves like this. > > > > Thanks > > Well libvhostuser checks enabled queues at DRIVER_OK, does it not? Probably, but I meant: 1) This behaviour has been supported by some device (e.g MLX) 2) This is the current behaviour of Qemu for vhost-net devices: static void virtio_net_queue_enable(VirtIODevice *vdev, uint32_t queue_index) { VirtIONet *n = VIRTIO_NET(vdev); NetClientState *nc; int r; .... if (get_vhost_net(nc->peer) && nc->peer->info->type == NET_CLIENT_DRIVER_TAP) { r = vhost_net_virtqueue_restart(vdev, nc, queue_index); if (r < 0) { error_report("unable to restart vhost net virtqueue: %d, " "when resetting the queue", queue_index); } } } Thanks > > > > > > > > > > > > > > > > > > > > > > > > --- > > > > > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > > > @@ -7,6 +7,7 @@ > > > > > > > > #include <uapi/linux/virtio_net.h> > > > > > > > > #include <uapi/linux/virtio_ids.h> > > > > > > > > #include <uapi/linux/vdpa.h> > > > > > > > > +#include <uapi/linux/vhost_types.h> > > > > > > > > #include <linux/virtio_config.h> > > > > > > > > #include <linux/auxiliary_bus.h> > > > > > > > > #include <linux/mlx5/cq.h> > > > > > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > > > > > flush_workqueue(ndev->mvdev.wq); > > > > > > > > } > > > > > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > > > > > +{ > > > > > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > > > > > +} > > > > > > > > + > > > > > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > > > > > { > > > > > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > > > > > -- > > > > > > > > 2.39.3 > > > > > > > > >
On Tue, Jul 4, 2023 at 12:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > and shadow virtqueue. > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > Same comment. > > to which? > > > > -Siwei > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > as a kernel/userspace ABI: what if one wants to start rings in some > other specific order? > As was discussed on list, a better promise is not to access ring > until the 1st kick. vdpa can then do a kick when it wants > the device to start accessing rings. > Friendly ping about this series, Now that VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK has been merged for vdpa_sim, does it make sense for mlx too? Thanks! > > > > > > > --- > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > 1 file changed, 7 insertions(+) > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > @@ -7,6 +7,7 @@ > > > > #include <uapi/linux/virtio_net.h> > > > > #include <uapi/linux/virtio_ids.h> > > > > #include <uapi/linux/vdpa.h> > > > > +#include <uapi/linux/vhost_types.h> > > > > #include <linux/virtio_config.h> > > > > #include <linux/auxiliary_bus.h> > > > > #include <linux/mlx5/cq.h> > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > flush_workqueue(ndev->mvdev.wq); > > > > } > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > +{ > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > +} > > > > + > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > { > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > -- > > > > 2.39.3 >
On Wed, Oct 04, 2023 at 02:56:53PM +0200, Eugenio Perez Martin wrote: > On Tue, Jul 4, 2023 at 12:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > > and shadow virtqueue. > > > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > > Same comment. > > > to which? > > > > > > -Siwei > > > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > > as a kernel/userspace ABI: what if one wants to start rings in some > > other specific order? > > As was discussed on list, a better promise is not to access ring > > until the 1st kick. vdpa can then do a kick when it wants > > the device to start accessing rings. > > > > Friendly ping about this series, > > Now that VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK has been merged for > vdpa_sim, does it make sense for mlx too? > > Thanks! For sure. I was just busy with a qemu pull, will handle this next. > > > > > > > > > --- > > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > @@ -7,6 +7,7 @@ > > > > > #include <uapi/linux/virtio_net.h> > > > > > #include <uapi/linux/virtio_ids.h> > > > > > #include <uapi/linux/vdpa.h> > > > > > +#include <uapi/linux/vhost_types.h> > > > > > #include <linux/virtio_config.h> > > > > > #include <linux/auxiliary_bus.h> > > > > > #include <linux/mlx5/cq.h> > > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > > flush_workqueue(ndev->mvdev.wq); > > > > > } > > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > > +{ > > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > > +} > > > > > + > > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > > { > > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > > -- > > > > > 2.39.3 > >
On Wed, Oct 4, 2023 at 6:27 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Wed, Oct 04, 2023 at 02:56:53PM +0200, Eugenio Perez Martin wrote: > > On Tue, Jul 4, 2023 at 12:16 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > On Mon, Jul 03, 2023 at 05:26:02PM -0700, Si-Wei Liu wrote: > > > > > > > > > > > > On 7/3/2023 8:46 AM, Michael S. Tsirkin wrote: > > > > > On Mon, Jul 03, 2023 at 04:25:14PM +0200, Eugenio Pérez wrote: > > > > > > Offer this backend feature as mlx5 is compatible with it. It allows it > > > > > > to do live migration with CVQ, dynamically switching between passthrough > > > > > > and shadow virtqueue. > > > > > > > > > > > > Signed-off-by: Eugenio Pérez <eperezma@redhat.com> > > > > > Same comment. > > > > to which? > > > > > > > > -Siwei > > > > > > VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK is too narrow a use-case to commit to it > > > as a kernel/userspace ABI: what if one wants to start rings in some > > > other specific order? > > > As was discussed on list, a better promise is not to access ring > > > until the 1st kick. vdpa can then do a kick when it wants > > > the device to start accessing rings. > > > > > > > Friendly ping about this series, > > > > Now that VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK has been merged for > > vdpa_sim, does it make sense for mlx too? > > > > Thanks! > > For sure. I was just busy with a qemu pull, will handle this next. > Sounds great, thanks! > > > > > > > > > > > --- > > > > > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 7 +++++++ > > > > > > 1 file changed, 7 insertions(+) > > > > > > > > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > index 9138ef2fb2c8..5f309a16b9dc 100644 > > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c > > > > > > @@ -7,6 +7,7 @@ > > > > > > #include <uapi/linux/virtio_net.h> > > > > > > #include <uapi/linux/virtio_ids.h> > > > > > > #include <uapi/linux/vdpa.h> > > > > > > +#include <uapi/linux/vhost_types.h> > > > > > > #include <linux/virtio_config.h> > > > > > > #include <linux/auxiliary_bus.h> > > > > > > #include <linux/mlx5/cq.h> > > > > > > @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) > > > > > > flush_workqueue(ndev->mvdev.wq); > > > > > > } > > > > > > +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) > > > > > > +{ > > > > > > + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); > > > > > > +} > > > > > > + > > > > > > static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) > > > > > > { > > > > > > struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); > > > > > > @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { > > > > > > .get_vq_align = mlx5_vdpa_get_vq_align, > > > > > > .get_vq_group = mlx5_vdpa_get_vq_group, > > > > > > .get_device_features = mlx5_vdpa_get_device_features, > > > > > > + .get_backend_features = mlx5_vdpa_get_backend_features, > > > > > > .set_driver_features = mlx5_vdpa_set_driver_features, > > > > > > .get_driver_features = mlx5_vdpa_get_driver_features, > > > > > > .set_config_cb = mlx5_vdpa_set_config_cb, > > > > > > -- > > > > > > 2.39.3 > > > >
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index 9138ef2fb2c8..5f309a16b9dc 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -7,6 +7,7 @@ #include <uapi/linux/virtio_net.h> #include <uapi/linux/virtio_ids.h> #include <uapi/linux/vdpa.h> +#include <uapi/linux/vhost_types.h> #include <linux/virtio_config.h> #include <linux/auxiliary_bus.h> #include <linux/mlx5/cq.h> @@ -2499,6 +2500,11 @@ static void unregister_link_notifier(struct mlx5_vdpa_net *ndev) flush_workqueue(ndev->mvdev.wq); } +static u64 mlx5_vdpa_get_backend_features(const struct vdpa_device *vdpa) +{ + return BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK); +} + static int mlx5_vdpa_set_driver_features(struct vdpa_device *vdev, u64 features) { struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev); @@ -3140,6 +3146,7 @@ static const struct vdpa_config_ops mlx5_vdpa_ops = { .get_vq_align = mlx5_vdpa_get_vq_align, .get_vq_group = mlx5_vdpa_get_vq_group, .get_device_features = mlx5_vdpa_get_device_features, + .get_backend_features = mlx5_vdpa_get_backend_features, .set_driver_features = mlx5_vdpa_set_driver_features, .get_driver_features = mlx5_vdpa_get_driver_features, .set_config_cb = mlx5_vdpa_set_config_cb,