[v2,0/6] Device tree support for Hyper-V VMBus driver

  This set of patches expands the VMBus driver to include device tree
support.

The first two patches enable compilation of Hyper-V APIs in a non-ACPI
build.

The third patch converts the VMBus driver from acpi to more generic
platform driver.

Further to add device tree documentation for VMBus, it needs to club with
other virtualization driver's documentation. For this rename the virtio
folder to more generic hypervisor, so that all the hypervisor based
devices can co-exist in a single place in device tree documentation. The
fourth patch does this renaming.

The fifth patch introduces the device tree documentation for VMBus.

The sixth patch adds device tree support to the VMBus driver. Currently
this is tested only for x86 and it may not work for other archs.

[v2]
- Convert VMBus acpi device to platform device, and added device tree support
  in separate patch. This enables using same driver structure for both the flows.
- In Device tree documentation, changed virtio folder to hypervisor and moved
  VMBus documentation there.
- Moved bindings before Device tree patch.
- Removed stale ".data" and ".name" field from of_device match table.
- Removed debug print.
- Fix "make DT_CHECKER_FLAGS=-m dt_binding_check" warnings.

Saurabh Sengar (6):
  drivers/clocksource/hyper-v: non ACPI support in hyperv clock
  Drivers: hv: allow non ACPI compilation for
    hv_is_hibernation_supported
  Drivers: hv: vmbus: Convert acpi_device to platform_device
  dt-bindings: hypervisor: Rename virtio to hypervisor
  dt-bindings: hypervisor: Add dt-bindings for VMBus
  Driver: VMBus: Add device tree support

 .../bindings/{virtio => hypervisor}/mmio.yaml |   2 +-
 .../bindings/hypervisor/msft,vmbus.yaml       |  50 ++++++
 .../{virtio => hypervisor}/pci-iommu.yaml     |   2 +-
 .../{virtio => hypervisor}/virtio-device.yaml |   2 +-
 .../devicetree/bindings/vendor-prefixes.yaml  |   2 +
 MAINTAINERS                                   |   3 +-
 drivers/clocksource/hyperv_timer.c            |  15 +-
 drivers/hv/hv_common.c                        |   4 +
 drivers/hv/vmbus_drv.c                        | 153 ++++++++++++++----
 9 files changed, 193 insertions(+), 40 deletions(-)
 rename Documentation/devicetree/bindings/{virtio => hypervisor}/mmio.yaml (95%)
 create mode 100644 Documentation/devicetree/bindings/hypervisor/msft,vmbus.yaml
 rename Documentation/devicetree/bindings/{virtio => hypervisor}/pci-iommu.yaml (98%)
 rename Documentation/devicetree/bindings/{virtio => hypervisor}/virtio-device.yaml (93%)
  

On Tue, Jan 31, 2023 at 12:10 PM Saurabh Sengar
<ssengar@linux.microsoft.com> wrote:
>
> This set of patches expands the VMBus driver to include device tree
> support.
>
> The first two patches enable compilation of Hyper-V APIs in a non-ACPI
> build.
>
> The third patch converts the VMBus driver from acpi to more generic
> platform driver.
>
> Further to add device tree documentation for VMBus, it needs to club with
> other virtualization driver's documentation. For this rename the virtio
> folder to more generic hypervisor, so that all the hypervisor based
> devices can co-exist in a single place in device tree documentation. The
> fourth patch does this renaming.
>
> The fifth patch introduces the device tree documentation for VMBus.
>
> The sixth patch adds device tree support to the VMBus driver. Currently
> this is tested only for x86 and it may not work for other archs.

I can read all the patches and see *what* they do. You don't really
need to list that here. I'm still wondering *why*. That is what the
cover letter and commit messages should answer. Why do you need DT
support? How does this even work on x86? FDT is only enabled for
CE4100 platform.

Rob
  

On Tue, Jan 31, 2023 at 02:27:51PM -0600, Rob Herring wrote:
> On Tue, Jan 31, 2023 at 12:10 PM Saurabh Sengar
> <ssengar@linux.microsoft.com> wrote:
> >
> > This set of patches expands the VMBus driver to include device tree
> > support.
> >
> > The first two patches enable compilation of Hyper-V APIs in a non-ACPI
> > build.
> >
> > The third patch converts the VMBus driver from acpi to more generic
> > platform driver.
> >
> > Further to add device tree documentation for VMBus, it needs to club with
> > other virtualization driver's documentation. For this rename the virtio
> > folder to more generic hypervisor, so that all the hypervisor based
> > devices can co-exist in a single place in device tree documentation. The
> > fourth patch does this renaming.
> >
> > The fifth patch introduces the device tree documentation for VMBus.
> >
> > The sixth patch adds device tree support to the VMBus driver. Currently
> > this is tested only for x86 and it may not work for other archs.
> 
> I can read all the patches and see *what* they do. You don't really
> need to list that here. I'm still wondering *why*. That is what the
> cover letter and commit messages should answer. Why do you need DT
> support? How does this even work on x86? FDT is only enabled for
> CE4100 platform.

HI Rob,

Thanks for your comments.
We are working on a solution where kernel is booted without ACPI tables to keep
the overall system's memory footprints slim and possibly faster boot time.
We have tested this by enabling CONFIG_OF for x86.

I can add this info in cover letter in next version.

Regards,
Saurabh

> 
> Rob
  

On Tue, Jan 31, 2023 at 06:04:49PM -0800, Saurabh Singh Sengar wrote:
> On Tue, Jan 31, 2023 at 02:27:51PM -0600, Rob Herring wrote:
> > On Tue, Jan 31, 2023 at 12:10 PM Saurabh Sengar
> > <ssengar@linux.microsoft.com> wrote:
> > >
> > > This set of patches expands the VMBus driver to include device tree
> > > support.
> > >
> > > The first two patches enable compilation of Hyper-V APIs in a non-ACPI
> > > build.
> > >
> > > The third patch converts the VMBus driver from acpi to more generic
> > > platform driver.
> > >
> > > Further to add device tree documentation for VMBus, it needs to club with
> > > other virtualization driver's documentation. For this rename the virtio
> > > folder to more generic hypervisor, so that all the hypervisor based
> > > devices can co-exist in a single place in device tree documentation. The
> > > fourth patch does this renaming.
> > >
> > > The fifth patch introduces the device tree documentation for VMBus.
> > >
> > > The sixth patch adds device tree support to the VMBus driver. Currently
> > > this is tested only for x86 and it may not work for other archs.
> > 
> > I can read all the patches and see *what* they do. You don't really
> > need to list that here. I'm still wondering *why*. That is what the
> > cover letter and commit messages should answer. Why do you need DT
> > support? How does this even work on x86? FDT is only enabled for
> > CE4100 platform.
> 
> HI Rob,
> 
> Thanks for your comments.
> We are working on a solution where kernel is booted without ACPI tables to keep
> the overall system's memory footprints slim and possibly faster boot time.
> We have tested this by enabling CONFIG_OF for x86.

It's CONFIG_OF_EARLY_FLATTREE which you would need and that's not user 
selectable. At a minimum, you need some kconfig changes. Where are 
those?

Also see my comment on v1 about running DT validation on your dtb. I'm 
sure running it would point out other issues. Such as the root level 
comaptible string(s) need to be documented. You need cpu nodes, 
interrupt controller, timers, etc. Those all have to be documented.

Rob
  

On Wed, Feb 01, 2023 at 08:51:46AM -0600, Rob Herring wrote:
> On Tue, Jan 31, 2023 at 06:04:49PM -0800, Saurabh Singh Sengar wrote:
> > On Tue, Jan 31, 2023 at 02:27:51PM -0600, Rob Herring wrote:
> > > On Tue, Jan 31, 2023 at 12:10 PM Saurabh Sengar
> > > <ssengar@linux.microsoft.com> wrote:
> > > >
> > > > This set of patches expands the VMBus driver to include device tree
> > > > support.
> > > >
> > > > The first two patches enable compilation of Hyper-V APIs in a non-ACPI
> > > > build.
> > > >
> > > > The third patch converts the VMBus driver from acpi to more generic
> > > > platform driver.
> > > >
> > > > Further to add device tree documentation for VMBus, it needs to club with
> > > > other virtualization driver's documentation. For this rename the virtio
> > > > folder to more generic hypervisor, so that all the hypervisor based
> > > > devices can co-exist in a single place in device tree documentation. The
> > > > fourth patch does this renaming.
> > > >
> > > > The fifth patch introduces the device tree documentation for VMBus.
> > > >
> > > > The sixth patch adds device tree support to the VMBus driver. Currently
> > > > this is tested only for x86 and it may not work for other archs.
> > > 
> > > I can read all the patches and see *what* they do. You don't really
> > > need to list that here. I'm still wondering *why*. That is what the
> > > cover letter and commit messages should answer. Why do you need DT
> > > support? How does this even work on x86? FDT is only enabled for
> > > CE4100 platform.
> > 
> > HI Rob,
> > 
> > Thanks for your comments.
> > We are working on a solution where kernel is booted without ACPI tables to keep
> > the overall system's memory footprints slim and possibly faster boot time.
> > We have tested this by enabling CONFIG_OF for x86.
> 
> It's CONFIG_OF_EARLY_FLATTREE which you would need and that's not user 
> selectable. At a minimum, you need some kconfig changes. Where are 
> those?

You are right we have define a new config flag in Kconfig, and selected CONFIG_OF
and CONFIG_OF_EARLY_FLATTREE. We are working on upstreaming that patch as well
however that will be a separate patch series.

> 
> Also see my comment on v1 about running DT validation on your dtb. I'm 
> sure running it would point out other issues. Such as the root level 
> comaptible string(s) need to be documented. You need cpu nodes, 
> interrupt controller, timers, etc. Those all have to be documented.

I will be changing the parent node to soc node as suggested by Krzysztof
in other thread.

soc {
        #address-cells = <2>;
        #size-cells = <2>;

	vmbus@ff0000000 {
            #address-cells = <2>;
            #size-cells = <1>;
            compatible = "Microsoft,vmbus";
            ranges = <0x00 0x00 0x0f 0xf0000000 0x10000000>;
        };
};

This will be sufficient.

Regards,
Saurabh

> 
> Rob
  

On 01/02/2023 17:34, Saurabh Singh Sengar wrote:
>> Also see my comment on v1 about running DT validation on your dtb. I'm 
>> sure running it would point out other issues. Such as the root level 
>> comaptible string(s) need to be documented. You need cpu nodes, 
>> interrupt controller, timers, etc. Those all have to be documented.
> 
> I will be changing the parent node to soc node as suggested by Krzysztof
> in other thread.
> 
> soc {
>         #address-cells = <2>;
>         #size-cells = <2>;
> 
> 	vmbus@ff0000000 {
>             #address-cells = <2>;
>             #size-cells = <1>;
>             compatible = "Microsoft,vmbus";
>             ranges = <0x00 0x00 0x0f 0xf0000000 0x10000000>;
>         };
> };
> 
> This will be sufficient.

It will be ok for the example, but will not be ok for supporting your
use case. Please solve all the points from Rob's comment above. Where is
their documentation?

Best regards,
Krzysztof
  

On Wed, Feb 01, 2023 at 06:15:23PM +0100, Krzysztof Kozlowski wrote:
> On 01/02/2023 17:34, Saurabh Singh Sengar wrote:
> >> Also see my comment on v1 about running DT validation on your dtb. I'm 
> >> sure running it would point out other issues. Such as the root level 
> >> comaptible string(s) need to be documented. You need cpu nodes, 
> >> interrupt controller, timers, etc. Those all have to be documented.
> > 
> > I will be changing the parent node to soc node as suggested by Krzysztof
> > in other thread.
> > 
> > soc {
> >         #address-cells = <2>;
> >         #size-cells = <2>;
> > 
> > 	vmbus@ff0000000 {
> >             #address-cells = <2>;
> >             #size-cells = <1>;
> >             compatible = "Microsoft,vmbus";
> >             ranges = <0x00 0x00 0x0f 0xf0000000 0x10000000>;
> >         };
> > };
> > 
> > This will be sufficient.
> 
> It will be ok for the example, but will not be ok for supporting your
> use case. Please solve all the points from Rob's comment above. Where is
> their documentation?
> 
> Best regards,
> Krzysztof

Hi Rob/ Krzysztof,

I am happy to update the documentation as requested. Please note
that, apart from CPUs, there is no other device node in the tree.

Here are some of the info related to our system:

Timers:
VMBus code uses a Hyper-V Synthetic timer and there is no device tree
node or ACPI method required for this. This is implemented as
drivers/clocksource/hyperv_timer.c

Interrupt controller:
The hypervisor virtualizes interrupt delivery to virtual processors.
This is done through the use of a synthetic interrupt controller
(SynIC) which is an extension of a virtualized local APIC. In the cpu
DT nodes we have APIC ids.

Below are the cpu nodes we use, please suggest if I need to update any
document for it.
	cpus {
		#address-cells = <1>;
		#size-cells = <0>;

		cpu@0 {
			device_type = "cpu";
			reg = <0>;
			status = "okay";
		};

		cpu@1 {
			device_type = "cpu";
			reg = <1>;
			status = "okay";
		};
	};

Regards,
Saurabh
  

On Wed, Feb 1, 2023 at 10:34 AM Saurabh Singh Sengar
<ssengar@linux.microsoft.com> wrote:
>
> On Wed, Feb 01, 2023 at 08:51:46AM -0600, Rob Herring wrote:
> > On Tue, Jan 31, 2023 at 06:04:49PM -0800, Saurabh Singh Sengar wrote:
> > > On Tue, Jan 31, 2023 at 02:27:51PM -0600, Rob Herring wrote:
> > > > On Tue, Jan 31, 2023 at 12:10 PM Saurabh Sengar
> > > > <ssengar@linux.microsoft.com> wrote:
> > > > >
> > > > > This set of patches expands the VMBus driver to include device tree
> > > > > support.
> > > > >
> > > > > The first two patches enable compilation of Hyper-V APIs in a non-ACPI
> > > > > build.
> > > > >
> > > > > The third patch converts the VMBus driver from acpi to more generic
> > > > > platform driver.
> > > > >
> > > > > Further to add device tree documentation for VMBus, it needs to club with
> > > > > other virtualization driver's documentation. For this rename the virtio
> > > > > folder to more generic hypervisor, so that all the hypervisor based
> > > > > devices can co-exist in a single place in device tree documentation. The
> > > > > fourth patch does this renaming.
> > > > >
> > > > > The fifth patch introduces the device tree documentation for VMBus.
> > > > >
> > > > > The sixth patch adds device tree support to the VMBus driver. Currently
> > > > > this is tested only for x86 and it may not work for other archs.
> > > >
> > > > I can read all the patches and see *what* they do. You don't really
> > > > need to list that here. I'm still wondering *why*. That is what the
> > > > cover letter and commit messages should answer. Why do you need DT
> > > > support? How does this even work on x86? FDT is only enabled for
> > > > CE4100 platform.
> > >
> > > HI Rob,
> > >
> > > Thanks for your comments.
> > > We are working on a solution where kernel is booted without ACPI tables to keep
> > > the overall system's memory footprints slim and possibly faster boot time.
> > > We have tested this by enabling CONFIG_OF for x86.
> >
> > It's CONFIG_OF_EARLY_FLATTREE which you would need and that's not user
> > selectable. At a minimum, you need some kconfig changes. Where are
> > those?
>
> You are right we have define a new config flag in Kconfig, and selected CONFIG_OF
> and CONFIG_OF_EARLY_FLATTREE. We are working on upstreaming that patch as well
> however that will be a separate patch series.

Fair enough, but that should come first IMO. Really I just want to see
a complete picture. That can be a reference to a git branch(es) or
other patch series. But again, what I want to see in particular is the
actual DT and validation run on it.

> > Also see my comment on v1 about running DT validation on your dtb. I'm
> > sure running it would point out other issues. Such as the root level
> > comaptible string(s) need to be documented. You need cpu nodes,
> > interrupt controller, timers, etc. Those all have to be documented.
>
> I will be changing the parent node to soc node as suggested by Krzysztof
> in other thread.

Another issue yes, but orthogonal to my comments.

>
> soc {
>         #address-cells = <2>;
>         #size-cells = <2>;

You are missing 'ranges' here. Without it, addresses aren't translatable.

You are also missing 'compatible = "simple-bus";'. This happens to
work on x86 because of legacy reasons, but we don't want new cases
added.

>
>         vmbus@ff0000000 {
>             #address-cells = <2>;
>             #size-cells = <1>;
>             compatible = "Microsoft,vmbus";

'Microsoft' is not a vendor prefix.

>             ranges = <0x00 0x00 0x0f 0xf0000000 0x10000000>;
>         };
> };
>
> This will be sufficient.

All these comments are unnecessary because the tools will now check
these things and we shouldn't have to.

Rob
  

On Tue, Jan 31, 2023 at 06:04:49PM -0800, Saurabh Singh Sengar wrote:
>
> We are working on a solution where kernel is booted without ACPI tables to keep
> the overall system's memory footprints slim and possibly faster boot time.
> We have tested this by enabling CONFIG_OF for x86.
>

Very interesting. Do you comparison on how slow/fast DT version w.r.t
ACPI and similarly for the memory footprint ? It would be good to add
those information as well. I know some systems just use ACPI static tables
as they don't run ACPICA runtime interpretter, was that experimented as
well or it wasn't used due to some specific(details please) limitations
without it.

> I can add this info in cover letter in next version.

Yes please.
  

On Tue, Feb 07, 2023 at 11:53:54AM -0600, Rob Herring wrote:
> On Wed, Feb 1, 2023 at 10:34 AM Saurabh Singh Sengar
> <ssengar@linux.microsoft.com> wrote:
> >
> > On Wed, Feb 01, 2023 at 08:51:46AM -0600, Rob Herring wrote:
> > > On Tue, Jan 31, 2023 at 06:04:49PM -0800, Saurabh Singh Sengar wrote:
> > > > On Tue, Jan 31, 2023 at 02:27:51PM -0600, Rob Herring wrote:
> > > > > On Tue, Jan 31, 2023 at 12:10 PM Saurabh Sengar
> > > > > <ssengar@linux.microsoft.com> wrote:
> > > > > >
> > > > > > This set of patches expands the VMBus driver to include device tree
> > > > > > support.
(...)
> > You are right we have define a new config flag in Kconfig, and selected CONFIG_OF
> > and CONFIG_OF_EARLY_FLATTREE. We are working on upstreaming that patch as well
> > however that will be a separate patch series.
> 
> Fair enough, but that should come first IMO. Really I just want to see
> a complete picture. That can be a reference to a git branch(es) or
> other patch series. But again, what I want to see in particular is the
> actual DT and validation run on it.

Thank you for explaining the concern. I now understand it fully. I have come to the
realization that enabling the vmbus device tree should not be impacted by any changes.
To address this, I will add the following lines to the HYPERV Kconfig definition I
used for testing:

	select OF if !ACPI
	select OF_EARLY_FLATTREE if !ACPI"

> 
> > > Also see my comment on v1 about running DT validation on your dtb. I'm
> > > sure running it would point out other issues. Such as the root level
> > > comaptible string(s) need to be documented. You need cpu nodes,
> > > interrupt controller, timers, etc. Those all have to be documented.
> >
> > I will be changing the parent node to soc node as suggested by Krzysztof
> > in other thread.
> 
> Another issue yes, but orthogonal to my comments.
> 
> >
> > soc {
> >         #address-cells = <2>;
> >         #size-cells = <2>;
> 
> You are missing 'ranges' here. Without it, addresses aren't translatable.
> 
> You are also missing 'compatible = "simple-bus";'. This happens to
> work on x86 because of legacy reasons, but we don't want new cases
> added.

I am a bit unclear on the reason for adding the ranges property in the root node.
To provide more context, I have included my full device tree below. I believe that
having the ranges property in the VMBus device node is sufficient. Please let me
know if this can be improved.


/dts-v1/;

/ {
	#address-cells = <0>;
	#size-cells = <0>;
	model = "microsoft,test";

	cpus {
		#address-cells = <0x01>;
		#size-cells = <0x00>;

		cpu@0 {
			device_type = "cpu";
			reg = <0x00>;
			status = "okay";
		};

		cpu@1 {
			device_type = "cpu";
			reg = <0x01>;
			status = "okay";
		};
	};

        vmbus@ff0000000 {
            #address-cells = <2>;
            #size-cells = <2>;                            
            compatible = "microsoft,vmbus";             
            ranges = <0x0f 0xf0000000 0x0f 0xf0000000 0x0 0x10000000>;        
       };
};

> 
> >
> >         vmbus@ff0000000 {
> >             #address-cells = <2>;
> >             #size-cells = <1>;
> >             compatible = "Microsoft,vmbus";
> 
> 'Microsoft' is not a vendor prefix.
> 
> >             ranges = <0x00 0x00 0x0f 0xf0000000 0x10000000>;
> >         };
> > };
> >
> > This will be sufficient.
> 
> All these comments are unnecessary because the tools will now check
> these things and we shouldn't have to.

Agree, and its fixed in latest version.

Regards,
Saurabh
> 
> Rob