Message ID | 1665685754-13971-1-git-send-email-ssengar@linux.microsoft.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp428389wrs; Thu, 13 Oct 2022 11:51:49 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7zu5g35BRT2DFc1Uxe5l2jaIpyyp6AnXi3D9guyRQdbRF7VaVTUGu0J/56KHg7Np7H4+2x X-Received: by 2002:a17:903:200a:b0:181:75fc:e0c6 with SMTP id s10-20020a170903200a00b0018175fce0c6mr1118752pla.93.1665687109163; Thu, 13 Oct 2022 11:51:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665687109; cv=none; d=google.com; s=arc-20160816; b=DCE4FnyoEb4BIssl/fWUx5J3q0PrwSS6MVGCXRmafjK9zJ6yK6uCZZyxqucWFNkssP wV9mIVLa1UU4TtQrth1Ca8Bm3IUQNI1LXlO9uhvabqThkxFO/sx//cn7fw07Lpo9vhO4 bOgje5doboXNuNx47fYTNcjNgKSNEjsV4xptXN1Lp1JjVUdkGnOYDSSQoKbkp2PboBeE YVkEqcQ7N1HUWFULFEVfMF2jSGIlptpENVCA22b2knTase2W5D5t2cz5yeqdMBxaPYj0 3/2jsTfBaSyyK/DGuKC1+n3Um5B8hoG1eIiNt0BDDLPTrddQ6C1ZUiEH/LZRdc0zlu/q nK0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:subject:to:from:dkim-signature :dkim-filter; bh=LEICLpqkAHA55SgqSRkWd7Q239A6x6Ah2FjEDgwjgI4=; b=y/DtFQtZxnw8WyyYpIvO4LKqGXcIan++bipPdDDS6P7lgz16p70WZel+SXh8D9xbVd qpYyLZ+rzXAb6GmIf0EbqER8xQy8yCOuQCjHudNfb66WbyNVjv0MK0c39SFsjlHOGQPD MS1CyUX9G9+wykkAVFE3RlTKjRK/HTq6kPUkQhAvqt1O4tkU2/3z5y3YIew/ihC/8+Uo s/PJ23+PD8lYtwMRGG6r/Z4HGNgmZiKMMUZA2COi8DZbnKpepvh/HZEPFY6lnlPg5mro j9R+TSsHJ34rFrKLMEvu4i4QZ4l1HovFBbiChyOnF1z7+TLAskdraFGP3U1lmPQDNLWB 2bbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=mTgLYdKd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j192-20020a638bc9000000b00462c469b730si123563pge.648.2022.10.13.11.51.37; Thu, 13 Oct 2022 11:51:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=mTgLYdKd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232105AbiJMSeb (ORCPT <rfc822;ouuuleilei@gmail.com> + 99 others); Thu, 13 Oct 2022 14:34:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232033AbiJMSeL (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 13 Oct 2022 14:34:11 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 805A418D444; Thu, 13 Oct 2022 11:30:39 -0700 (PDT) Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id C70CC20F9E34; Thu, 13 Oct 2022 11:29:18 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com C70CC20F9E34 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1665685758; bh=LEICLpqkAHA55SgqSRkWd7Q239A6x6Ah2FjEDgwjgI4=; h=From:To:Subject:Date:From; b=mTgLYdKdGykhbNSoo5PIyT6nrzCY02AjSwwWN1D+AJWEqOVyWGaSNkSIdS/O1q2QC gcuBfWEL0sd93Q2+qKoJ7Zar+bWhnV1lLfQ7VQn8QvclUX/QrWEj2w6WbQCtVCD+9F guGi4CgEcqPMHiJ17RHKT2JW415rIHwbiYmis5g4= From: Saurabh Sengar <ssengar@linux.microsoft.com> To: ssengar@microsoft.com, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, longli@microsoft.com, gregkh@linuxfoundation.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, mikelley@microsoft.com Subject: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus devices Date: Thu, 13 Oct 2022 11:29:14 -0700 Message-Id: <1665685754-13971-1-git-send-email-ssengar@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 X-Spam-Status: No, score=-19.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746599526594483099?= X-GMAIL-MSGID: =?utf-8?q?1746599526594483099?= |
Series |
uio_hv_generic: Enable interrupt for low speed VMBus devices
|
|
Commit Message
Saurabh Singh Sengar
Oct. 13, 2022, 6:29 p.m. UTC
Hyper-V is adding some "specialty" synthetic devices. Instead of writing
new kernel-level VMBus drivers for these devices, the devices will be
presented to user space via this existing Hyper-V generic UIO driver, so
that a user space driver can handle the device. Since these new synthetic
devices are low speed devices, they don't support monitor bits and we must
use vmbus_setevent() to enable interrupts from the host.
Signed-off-by: Saurabh Sengar <ssengar@linux.microsoft.com>
---
drivers/uio/uio_hv_generic.c | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
Comments
On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > Hyper-V is adding some "specialty" synthetic devices. What devices are those specifically? > Instead of writing new kernel-level VMBus drivers for these devices, > the devices will be presented to user space via this existing Hyper-V > generic UIO driver, so that a user space driver can handle the device. > Since these new synthetic devices are low speed devices, they don't > support monitor bits and we must use vmbus_setevent() to enable > interrupts from the host. That is not what the UIO interface is for. Please write real drivers so that they tie into the specific user/kernel apis for those device types. Without a specific list of what these devices are, I can not recommend that anyone use the UIO api for them as that's probably not a good idea. Also, if you do do this, you need to list where the source for that userspace code is so that users can get it and have their distros package it up for them. I do not see that here at all. > > Signed-off-by: Saurabh Sengar <ssengar@linux.microsoft.com> > --- > drivers/uio/uio_hv_generic.c | 9 +++------ > 1 file changed, 3 insertions(+), 6 deletions(-) > > diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c > index c08a6cfd119f..8e5aa4a1247f 100644 > --- a/drivers/uio/uio_hv_generic.c > +++ b/drivers/uio/uio_hv_generic.c > @@ -84,6 +84,9 @@ hv_uio_irqcontrol(struct uio_info *info, s32 irq_state) > dev->channel->inbound.ring_buffer->interrupt_mask = !irq_state; > virt_mb(); > > + if (!dev->channel->offermsg.monitor_allocated && irq_state) > + vmbus_setevent(dev->channel); > + > return 0; > } > > @@ -239,12 +242,6 @@ hv_uio_probe(struct hv_device *dev, > void *ring_buffer; > int ret; > > - /* Communicating with host has to be via shared memory not hypercall */ > - if (!channel->offermsg.monitor_allocated) { > - dev_err(&dev->device, "vmbus channel requires hypercall\n"); I do not understand, why is this check not made anymore here? Why constantly make the call above in the irq handler instead? Isn't that going to be massively slow? thanks, greg k-h
> Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus > devices > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > > Hyper-V is adding some "specialty" synthetic devices. > > What devices are those specifically? > > > Instead of writing new kernel-level VMBus drivers for these devices, > > the devices will be presented to user space via this existing Hyper-V > > generic UIO driver, so that a user space driver can handle the device. > > Since these new synthetic devices are low speed devices, they don't > > support monitor bits and we must use vmbus_setevent() to enable > > interrupts from the host. > > That is not what the UIO interface is for. Please write real drivers so that > they tie into the specific user/kernel apis for those device types. > > Without a specific list of what these devices are, I can not recommend that > anyone use the UIO api for them as that's probably not a good idea. There are some VMBUS drivers currently not implemented in Linux. Out of all VMBUS drivers, two use "monitored bits": they are network and storage drivers. All the rest VMBUS drivers use hypercall for host notification and signal for next interrupt. One example of such driver is to collect process level crash information for diagnostic purposes. Also, we want to move some existing kernel mode VMBUS drivers to user-space, such as hv_kvp and hv_filecopy. They don't really fit into an existing kernel API, and they create their own devices under /dev and communicates with a user-mode daemon to do most of the work. It's a better model that we can move those drivers entirely into user-mode. > > Also, if you do do this, you need to list where the source for that userspace > code is so that users can get it and have their distros package it up for them. I > do not see that here at all. > > > > > > Signed-off-by: Saurabh Sengar <ssengar@linux.microsoft.com> > > --- > > drivers/uio/uio_hv_generic.c | 9 +++------ > > 1 file changed, 3 insertions(+), 6 deletions(-) > > > > diff --git a/drivers/uio/uio_hv_generic.c > > b/drivers/uio/uio_hv_generic.c index c08a6cfd119f..8e5aa4a1247f 100644 > > --- a/drivers/uio/uio_hv_generic.c > > +++ b/drivers/uio/uio_hv_generic.c > > @@ -84,6 +84,9 @@ hv_uio_irqcontrol(struct uio_info *info, s32 irq_state) > > dev->channel->inbound.ring_buffer->interrupt_mask = !irq_state; > > virt_mb(); > > > > + if (!dev->channel->offermsg.monitor_allocated && irq_state) > > + vmbus_setevent(dev->channel); > > + > > return 0; > > } > > > > @@ -239,12 +242,6 @@ hv_uio_probe(struct hv_device *dev, > > void *ring_buffer; > > int ret; > > > > - /* Communicating with host has to be via shared memory not > hypercall */ > > - if (!channel->offermsg.monitor_allocated) { > > - dev_err(&dev->device, "vmbus channel requires > hypercall\n"); > > I do not understand, why is this check not made anymore here? Why > constantly make the call above in the irq handler instead? Isn't that going to > be massively slow? Some VMBUS devices exposed by the Hyper-V are not modeled as high speed, they use hypercall, not monitored bits. Because they don't fit into other kernel API (as explained above), can we use UIO for those devices? Thanks, Long
On Tue, Oct 18, 2022 at 06:31:16AM +0000, Long Li wrote: > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus > > devices > > > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > > > Hyper-V is adding some "specialty" synthetic devices. > > > > What devices are those specifically? > > > > > Instead of writing new kernel-level VMBus drivers for these devices, > > > the devices will be presented to user space via this existing Hyper-V > > > generic UIO driver, so that a user space driver can handle the device. > > > Since these new synthetic devices are low speed devices, they don't > > > support monitor bits and we must use vmbus_setevent() to enable > > > interrupts from the host. > > > > That is not what the UIO interface is for. Please write real drivers so that > > they tie into the specific user/kernel apis for those device types. > > > > Without a specific list of what these devices are, I can not recommend that > > anyone use the UIO api for them as that's probably not a good idea. > > There are some VMBUS drivers currently not implemented in Linux. Out of all > VMBUS drivers, two use "monitored bits": they are network and storage drivers. > All the rest VMBUS drivers use hypercall for host notification and signal for next > interrupt. One example of such driver is to collect process level crash information > for diagnostic purposes. > > Also, we want to move some existing kernel mode VMBUS drivers to user-space, > such as hv_kvp and hv_filecopy. They don't really fit into an existing kernel API, and > they create their own devices under /dev and communicates with a user-mode > daemon to do most of the work. It's a better model that we can move those drivers > entirely into user-mode. How are you going to be able to remove drivers that export an existing user/kernel api and not break current systems? > > Also, if you do do this, you need to list where the source for that userspace > > code is so that users can get it and have their distros package it up for them. I > > do not see that here at all. > > > > > > > > > > Signed-off-by: Saurabh Sengar <ssengar@linux.microsoft.com> > > > --- > > > drivers/uio/uio_hv_generic.c | 9 +++------ > > > 1 file changed, 3 insertions(+), 6 deletions(-) > > > > > > diff --git a/drivers/uio/uio_hv_generic.c > > > b/drivers/uio/uio_hv_generic.c index c08a6cfd119f..8e5aa4a1247f 100644 > > > --- a/drivers/uio/uio_hv_generic.c > > > +++ b/drivers/uio/uio_hv_generic.c > > > @@ -84,6 +84,9 @@ hv_uio_irqcontrol(struct uio_info *info, s32 irq_state) > > > dev->channel->inbound.ring_buffer->interrupt_mask = !irq_state; > > > virt_mb(); > > > > > > + if (!dev->channel->offermsg.monitor_allocated && irq_state) > > > + vmbus_setevent(dev->channel); > > > + > > > return 0; > > > } > > > > > > @@ -239,12 +242,6 @@ hv_uio_probe(struct hv_device *dev, > > > void *ring_buffer; > > > int ret; > > > > > > - /* Communicating with host has to be via shared memory not > > hypercall */ > > > - if (!channel->offermsg.monitor_allocated) { > > > - dev_err(&dev->device, "vmbus channel requires > > hypercall\n"); > > > > I do not understand, why is this check not made anymore here? Why > > constantly make the call above in the irq handler instead? Isn't that going to > > be massively slow? > > Some VMBUS devices exposed by the Hyper-V are not modeled as high speed, > they use hypercall, not monitored bits. Because they don't fit into other kernel > API (as explained above), can we use UIO for those devices? UIO is for mmaped memory regions, like PCI devices, how is this a valid Hyper-V api at all? confused, greg k-h
> Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus > devices > > On Tue, Oct 18, 2022 at 06:31:16AM +0000, Long Li wrote: > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed > > > VMBus devices > > > > > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > > > > Hyper-V is adding some "specialty" synthetic devices. > > > > > > What devices are those specifically? > > > > > > > Instead of writing new kernel-level VMBus drivers for these > > > > devices, the devices will be presented to user space via this > > > > existing Hyper-V generic UIO driver, so that a user space driver can > handle the device. > > > > Since these new synthetic devices are low speed devices, they > > > > don't support monitor bits and we must use vmbus_setevent() to > > > > enable interrupts from the host. > > > > > > That is not what the UIO interface is for. Please write real > > > drivers so that they tie into the specific user/kernel apis for those device > types. > > > > > > Without a specific list of what these devices are, I can not > > > recommend that anyone use the UIO api for them as that's probably not > a good idea. > > > > There are some VMBUS drivers currently not implemented in Linux. Out > > of all VMBUS drivers, two use "monitored bits": they are network and > storage drivers. > > All the rest VMBUS drivers use hypercall for host notification and > > signal for next interrupt. One example of such driver is to collect > > process level crash information for diagnostic purposes. > > > > Also, we want to move some existing kernel mode VMBUS drivers to > > user-space, such as hv_kvp and hv_filecopy. They don't really fit into > > an existing kernel API, and they create their own devices under /dev > > and communicates with a user-mode daemon to do most of the work. It's > > a better model that we can move those drivers entirely into user-mode. > > How are you going to be able to remove drivers that export an existing > user/kernel api and not break current systems? It will be some configuration challenges, but it's doable. Newer Linux VMs can be configured in a way to use the UIO interfaces. This doesn't break any existing VMs because the old model will continue to work when UIO is not used. Also, this doesn't require any Hyper-V changes. > > > > Also, if you do do this, you need to list where the source for that > > > userspace code is so that users can get it and have their distros > > > package it up for them. I do not see that here at all. > > > > > > > > > > > > > > Signed-off-by: Saurabh Sengar <ssengar@linux.microsoft.com> > > > > --- > > > > drivers/uio/uio_hv_generic.c | 9 +++------ > > > > 1 file changed, 3 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/drivers/uio/uio_hv_generic.c > > > > b/drivers/uio/uio_hv_generic.c index c08a6cfd119f..8e5aa4a1247f > > > > 100644 > > > > --- a/drivers/uio/uio_hv_generic.c > > > > +++ b/drivers/uio/uio_hv_generic.c > > > > @@ -84,6 +84,9 @@ hv_uio_irqcontrol(struct uio_info *info, s32 > irq_state) > > > > dev->channel->inbound.ring_buffer->interrupt_mask = !irq_state; > > > > virt_mb(); > > > > > > > > + if (!dev->channel->offermsg.monitor_allocated && irq_state) > > > > + vmbus_setevent(dev->channel); > > > > + > > > > return 0; > > > > } > > > > > > > > @@ -239,12 +242,6 @@ hv_uio_probe(struct hv_device *dev, > > > > void *ring_buffer; > > > > int ret; > > > > > > > > - /* Communicating with host has to be via shared memory not > > > hypercall */ > > > > - if (!channel->offermsg.monitor_allocated) { > > > > - dev_err(&dev->device, "vmbus channel requires > > > hypercall\n"); > > > > > > I do not understand, why is this check not made anymore here? Why > > > constantly make the call above in the irq handler instead? Isn't > > > that going to be massively slow? > > > > Some VMBUS devices exposed by the Hyper-V are not modeled as high > > speed, they use hypercall, not monitored bits. Because they don't fit > > into other kernel API (as explained above), can we use UIO for those > devices? > > UIO is for mmaped memory regions, like PCI devices, how is this a valid > Hyper-V api at all? Currently uio_hv_generic is used to mmap VMBUS ring buffers to user-mode. The primary use case is for DPDK. However, the same mmap model can be used for all other VMBUS devices, as they all use the same ring buffers model. Long
On Tue, Oct 18, 2022 at 06:57:33AM +0000, Long Li wrote: > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus > > devices > > > > On Tue, Oct 18, 2022 at 06:31:16AM +0000, Long Li wrote: > > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed > > > > VMBus devices > > > > > > > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > > > > > Hyper-V is adding some "specialty" synthetic devices. > > > > > > > > What devices are those specifically? > > > > > > > > > Instead of writing new kernel-level VMBus drivers for these > > > > > devices, the devices will be presented to user space via this > > > > > existing Hyper-V generic UIO driver, so that a user space driver can > > handle the device. > > > > > Since these new synthetic devices are low speed devices, they > > > > > don't support monitor bits and we must use vmbus_setevent() to > > > > > enable interrupts from the host. > > > > > > > > That is not what the UIO interface is for. Please write real > > > > drivers so that they tie into the specific user/kernel apis for those device > > types. > > > > > > > > Without a specific list of what these devices are, I can not > > > > recommend that anyone use the UIO api for them as that's probably not > > a good idea. > > > > > > There are some VMBUS drivers currently not implemented in Linux. Out > > > of all VMBUS drivers, two use "monitored bits": they are network and > > storage drivers. > > > All the rest VMBUS drivers use hypercall for host notification and > > > signal for next interrupt. One example of such driver is to collect > > > process level crash information for diagnostic purposes. > > > > > > Also, we want to move some existing kernel mode VMBUS drivers to > > > user-space, such as hv_kvp and hv_filecopy. They don't really fit into > > > an existing kernel API, and they create their own devices under /dev > > > and communicates with a user-mode daemon to do most of the work. It's > > > a better model that we can move those drivers entirely into user-mode. > > > > How are you going to be able to remove drivers that export an existing > > user/kernel api and not break current systems? > > It will be some configuration challenges, but it's doable. How exactly? We can not break userspace when you upgrade a kernel version. > Newer Linux > VMs can be configured in a way to use the UIO interfaces. This doesn't break > any existing VMs because the old model will continue to work when UIO is not > used. Also, this doesn't require any Hyper-V changes. Hyper-v changes are not the issue here :) Again, you have to support the situation where a system upgrades to a new kernel and all the same functionality must be there. How will you do that if you remove drivers from a newer kernel? > > UIO is for mmaped memory regions, like PCI devices, how is this a valid > > Hyper-V api at all? > > Currently uio_hv_generic is used to mmap VMBUS ring buffers to user-mode. > The primary use case is for DPDK. However, the same mmap model can be > used for all other VMBUS devices, as they all use the same ring buffers model. Ok, that feels like overkill... You need to also post the source for these new userspace drivers somewhere for us to review. I don't see a link to them in the changelog text :( thanks, greg k-h
> Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus > devices > > On Tue, Oct 18, 2022 at 06:57:33AM +0000, Long Li wrote: > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed > > > VMBus devices > > > > > > On Tue, Oct 18, 2022 at 06:31:16AM +0000, Long Li wrote: > > > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low > > > > > speed VMBus devices > > > > > > > > > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > > > > > > Hyper-V is adding some "specialty" synthetic devices. > > > > > > > > > > What devices are those specifically? > > > > > > > > > > > Instead of writing new kernel-level VMBus drivers for these > > > > > > devices, the devices will be presented to user space via this > > > > > > existing Hyper-V generic UIO driver, so that a user space > > > > > > driver can > > > handle the device. > > > > > > Since these new synthetic devices are low speed devices, they > > > > > > don't support monitor bits and we must use vmbus_setevent() to > > > > > > enable interrupts from the host. > > > > > > > > > > That is not what the UIO interface is for. Please write real > > > > > drivers so that they tie into the specific user/kernel apis for > > > > > those device > > > types. > > > > > > > > > > Without a specific list of what these devices are, I can not > > > > > recommend that anyone use the UIO api for them as that's > > > > > probably not > > > a good idea. > > > > > > > > There are some VMBUS drivers currently not implemented in Linux. > > > > Out of all VMBUS drivers, two use "monitored bits": they are > > > > network and > > > storage drivers. > > > > All the rest VMBUS drivers use hypercall for host notification and > > > > signal for next interrupt. One example of such driver is to > > > > collect process level crash information for diagnostic purposes. > > > > > > > > Also, we want to move some existing kernel mode VMBUS drivers to > > > > user-space, such as hv_kvp and hv_filecopy. They don't really fit > > > > into an existing kernel API, and they create their own devices > > > > under /dev and communicates with a user-mode daemon to do most of > > > > the work. It's a better model that we can move those drivers entirely > into user-mode. > > > > > > How are you going to be able to remove drivers that export an > > > existing user/kernel api and not break current systems? > > > > It will be some configuration challenges, but it's doable. > > How exactly? > > We can not break userspace when you upgrade a kernel version. > > > Newer Linux > > VMs can be configured in a way to use the UIO interfaces. This doesn't > > break any existing VMs because the old model will continue to work > > when UIO is not used. Also, this doesn't require any Hyper-V changes. > > Hyper-v changes are not the issue here :) > > Again, you have to support the situation where a system upgrades to a new > kernel and all the same functionality must be there. How will you do that if > you remove drivers from a newer kernel? Currently there is a hv_kvp_daemon that interacts with the /dev/device created by the hv_kvp kernel driver, this is the only program interacts with this device. This program is acting like a user-space driver, in a sense. With the new kernel, if the KVP kernel mode is not present, this kvp daemon will not start. All the KVP functionality is handled by the new UIO driver. > > > > UIO is for mmaped memory regions, like PCI devices, how is this a > > > valid Hyper-V api at all? > > > > Currently uio_hv_generic is used to mmap VMBUS ring buffers to user- > mode. > > The primary use case is for DPDK. However, the same mmap model can be > > used for all other VMBUS devices, as they all use the same ring buffers > model. > > Ok, that feels like overkill... > > You need to also post the source for these new userspace drivers > somewhere for us to review. I don't see a link to them in the changelog > text :( Yes, we will share the user space VMBUS drivers. What's the concern of creating VMBus driver in user-mode? There are many examples of kernel drivers moving to user-mode. For example, DPDK wants a network driver in user-mode, SPDK wants a storage driver in user-mode, although they already have corresponding kernel drivers. Long
On Tue, Oct 18, 2022 at 07:36:54AM +0000, Long Li wrote: > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed VMBus > > devices > > > > On Tue, Oct 18, 2022 at 06:57:33AM +0000, Long Li wrote: > > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low speed > > > > VMBus devices > > > > > > > > On Tue, Oct 18, 2022 at 06:31:16AM +0000, Long Li wrote: > > > > > > Subject: Re: [PATCH] uio_hv_generic: Enable interrupt for low > > > > > > speed VMBus devices > > > > > > > > > > > > On Thu, Oct 13, 2022 at 11:29:14AM -0700, Saurabh Sengar wrote: > > > > > > > Hyper-V is adding some "specialty" synthetic devices. > > > > > > > > > > > > What devices are those specifically? > > > > > > > > > > > > > Instead of writing new kernel-level VMBus drivers for these > > > > > > > devices, the devices will be presented to user space via this > > > > > > > existing Hyper-V generic UIO driver, so that a user space > > > > > > > driver can > > > > handle the device. > > > > > > > Since these new synthetic devices are low speed devices, they > > > > > > > don't support monitor bits and we must use vmbus_setevent() to > > > > > > > enable interrupts from the host. > > > > > > > > > > > > That is not what the UIO interface is for. Please write real > > > > > > drivers so that they tie into the specific user/kernel apis for > > > > > > those device > > > > types. > > > > > > > > > > > > Without a specific list of what these devices are, I can not > > > > > > recommend that anyone use the UIO api for them as that's > > > > > > probably not > > > > a good idea. > > > > > > > > > > There are some VMBUS drivers currently not implemented in Linux. > > > > > Out of all VMBUS drivers, two use "monitored bits": they are > > > > > network and > > > > storage drivers. > > > > > All the rest VMBUS drivers use hypercall for host notification and > > > > > signal for next interrupt. One example of such driver is to > > > > > collect process level crash information for diagnostic purposes. > > > > > > > > > > Also, we want to move some existing kernel mode VMBUS drivers to > > > > > user-space, such as hv_kvp and hv_filecopy. They don't really fit > > > > > into an existing kernel API, and they create their own devices > > > > > under /dev and communicates with a user-mode daemon to do most of > > > > > the work. It's a better model that we can move those drivers entirely > > into user-mode. > > > > > > > > How are you going to be able to remove drivers that export an > > > > existing user/kernel api and not break current systems? > > > > > > It will be some configuration challenges, but it's doable. > > > > How exactly? > > > > We can not break userspace when you upgrade a kernel version. > > > > > Newer Linux > > > VMs can be configured in a way to use the UIO interfaces. This doesn't > > > break any existing VMs because the old model will continue to work > > > when UIO is not used. Also, this doesn't require any Hyper-V changes. > > > > Hyper-v changes are not the issue here :) > > > > Again, you have to support the situation where a system upgrades to a new > > kernel and all the same functionality must be there. How will you do that if > > you remove drivers from a newer kernel? > > Currently there is a hv_kvp_daemon that interacts with the /dev/device > created by the hv_kvp kernel driver, this is the only program interacts with > this device. This program is acting like a user-space driver, in a sense. > > With the new kernel, if the KVP kernel mode is not present, this kvp daemon > will not start. All the KVP functionality is handled by the new UIO driver. And those changes have already been implemented and pushed out somewhere? > > > > UIO is for mmaped memory regions, like PCI devices, how is this a > > > > valid Hyper-V api at all? > > > > > > Currently uio_hv_generic is used to mmap VMBUS ring buffers to user- > > mode. > > > The primary use case is for DPDK. However, the same mmap model can be > > > used for all other VMBUS devices, as they all use the same ring buffers > > model. > > > > Ok, that feels like overkill... > > > > You need to also post the source for these new userspace drivers > > somewhere for us to review. I don't see a link to them in the changelog > > text :( > > Yes, we will share the user space VMBUS drivers. What's the concern of creating > VMBus driver in user-mode? There are many examples of kernel drivers moving to > user-mode. For example, DPDK wants a network driver in user-mode, > SPDK wants a storage driver in user-mode, although they already have corresponding kernel > drivers. When moving out of the kernel to userspace, you loose the common and neutral user/kernel api and are now forced to rely on a userspace program for access to that device. no longer can you do something like normal socket calls, but instead, you have to rely on a library for the same functionality. Same for block devices. Do you really want to give up that common interface to your block device and rely on a library instead of the kernel? If so, great, but don't take away the functionality from all users, as you will break their systems. thanks greg k-h
diff --git a/drivers/uio/uio_hv_generic.c b/drivers/uio/uio_hv_generic.c index c08a6cfd119f..8e5aa4a1247f 100644 --- a/drivers/uio/uio_hv_generic.c +++ b/drivers/uio/uio_hv_generic.c @@ -84,6 +84,9 @@ hv_uio_irqcontrol(struct uio_info *info, s32 irq_state) dev->channel->inbound.ring_buffer->interrupt_mask = !irq_state; virt_mb(); + if (!dev->channel->offermsg.monitor_allocated && irq_state) + vmbus_setevent(dev->channel); + return 0; } @@ -239,12 +242,6 @@ hv_uio_probe(struct hv_device *dev, void *ring_buffer; int ret; - /* Communicating with host has to be via shared memory not hypercall */ - if (!channel->offermsg.monitor_allocated) { - dev_err(&dev->device, "vmbus channel requires hypercall\n"); - return -ENOTSUPP; - } - pdata = devm_kzalloc(&dev->device, sizeof(*pdata), GFP_KERNEL); if (!pdata) return -ENOMEM;