[v1,net-next,3/7] dt-bindings: net: dsa: qca8k: utilize shared dsa.yaml

  The dsa.yaml binding contains duplicated bindings for address and size
cells, as well as the reference to dsa-port.yaml. Instead of duplicating
this information, remove the reference to dsa-port.yaml and include the
full reference to dsa.yaml.

Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
Suggested-by: Vladimir Oltean <olteanv@gmail.com>
---
 .../devicetree/bindings/net/dsa/qca8k.yaml         | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)
  

On Mon, 24 Oct 2022 22:03:51 -0700, Colin Foster wrote:
> The dsa.yaml binding contains duplicated bindings for address and size
> cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> this information, remove the reference to dsa-port.yaml and include the
> full reference to dsa.yaml.
> 
> Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
> Suggested-by: Vladimir Oltean <olteanv@gmail.com>
> ---
>  .../devicetree/bindings/net/dsa/qca8k.yaml         | 14 +++-----------
>  1 file changed, 3 insertions(+), 11 deletions(-)
> 

My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check'
on your patch (DT_CHECKER_FLAGS is new in v5.13):

yamllint warnings/errors:

dtschema/dtc warnings/errors:
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/net/dsa/qca8k.example.dtb: switch@10: Unevaluated properties are not allowed ('#address-cells', '#size-cells' were unexpected)
	From schema: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/net/dsa/qca8k.example.dtb: switch@10: ports:port@6: Unevaluated properties are not allowed ('qca,sgmii-rxclk-falling-edge' was unexpected)
	From schema: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
/builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/net/dsa/qca8k.example.dtb: switch@10: Unevaluated properties are not allowed ('#address-cells', '#size-cells' were unexpected)
	From schema: /builds/robherring/dt-review-ci/linux/Documentation/devicetree/bindings/net/dsa/qca8k.yaml

doc reference errors (make refcheckdocs):

See https://patchwork.ozlabs.org/patch/

This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.

If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:

pip3 install dtschema --upgrade

Please check and re-submit.
  

On Mon, Oct 24, 2022 at 10:03:51PM -0700, Colin Foster wrote:
> The dsa.yaml binding contains duplicated bindings for address and size
> cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> this information, remove the reference to dsa-port.yaml and include the
> full reference to dsa.yaml.

I don't think this works without further restructuring. Essentially, 
'unevaluatedProperties' on works on a single level. So every level has 
to define all properties at that level either directly in 
properties/patternProperties or within a $ref.

See how graph.yaml is structured and referenced for an example how this 
has to work.

> 
> Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
> Suggested-by: Vladimir Oltean <olteanv@gmail.com>
> ---
>  .../devicetree/bindings/net/dsa/qca8k.yaml         | 14 +++-----------
>  1 file changed, 3 insertions(+), 11 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/net/dsa/qca8k.yaml b/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
> index 978162df51f7..7884f68cab73 100644
> --- a/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
> +++ b/Documentation/devicetree/bindings/net/dsa/qca8k.yaml
> @@ -66,22 +66,16 @@ properties:
>                   With the legacy mapping the reg corresponding to the internal
>                   mdio is the switch reg with an offset of -1.
>  
> +$ref: "dsa.yaml#"
> +
>  patternProperties:
>    "^(ethernet-)?ports$":
>      type: object
> -    properties:
> -      '#address-cells':
> -        const: 1
> -      '#size-cells':
> -        const: 0
> -
>      patternProperties:
>        "^(ethernet-)?port@[0-6]$":
>          type: object
>          description: Ethernet switch ports
>  
> -        $ref: dsa-port.yaml#
> -
>          properties:
>            qca,sgmii-rxclk-falling-edge:
>              $ref: /schemas/types.yaml#/definitions/flag
> @@ -104,8 +98,6 @@ patternProperties:
>                SGMII on the QCA8337, it is advised to set this unless a communication
>                issue is observed.
>  
> -        unevaluatedProperties: false
> -

Dropping this means any undefined properties in port nodes won't be an 
error. Once I fix all the issues related to these missing, there will be 
a meta-schema checking for this (this could be one I fixed already).

>  oneOf:
>    - required:
>        - ports
> @@ -116,7 +108,7 @@ required:
>    - compatible
>    - reg
>  
> -additionalProperties: true

This should certainly be changed though. We should only have 'true' for 
incomplete collections of properties. IOW, for common bindings.

> +unevaluatedProperties: false
>  
>  examples:
>    - |
> -- 
> 2.25.1
> 
>
  

Hi Rob,

On Tue, Oct 25, 2022 at 04:21:14PM -0500, Rob Herring wrote:
> On Mon, Oct 24, 2022 at 10:03:51PM -0700, Colin Foster wrote:
> > The dsa.yaml binding contains duplicated bindings for address and size
> > cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> > this information, remove the reference to dsa-port.yaml and include the
> > full reference to dsa.yaml.
> 
> I don't think this works without further restructuring. Essentially, 
> 'unevaluatedProperties' on works on a single level. So every level has 
> to define all properties at that level either directly in 
> properties/patternProperties or within a $ref.
> 
> See how graph.yaml is structured and referenced for an example how this 
> has to work.
> 
> > @@ -104,8 +98,6 @@ patternProperties:
> >                SGMII on the QCA8337, it is advised to set this unless a communication
> >                issue is observed.
> >  
> > -        unevaluatedProperties: false
> > -
> 
> Dropping this means any undefined properties in port nodes won't be an 
> error. Once I fix all the issues related to these missing, there will be 
> a meta-schema checking for this (this could be one I fixed already).

I may be misreading, but here, "unevaluatedProperties: false" from dsa.yaml
(under patternProperties: "^(ethernet-)?port@[0-9]+$":) is on the same
level as the "unevaluatedProperties: false" that Colin is deleting.

In fact, I believe that it is precisely due to the "unevaluatedProperties: false"
from dsa.yaml that this is causing a failure now:

net/dsa/qca8k.example.dtb: switch@10: ports:port@6: Unevaluated properties are not allowed ('qca,sgmii-rxclk-falling-edge' was unexpected)

Could you please explain why is the 'qca,sgmii-rxclk-falling-edge'
property not evaluated from the perspective of dsa.yaml in the example?
It's a head scratcher to me.

May it have something to do with the fact that Colin's addition:

$ref: "dsa.yaml#"

is not expressed as:

allOf:
  - $ref: "dsa.yaml#"

?

If yes, can you explain exactly what is the difference with respect to
unevaluatedProperties?

> >  oneOf:
> >    - required:
> >        - ports
> > @@ -116,7 +108,7 @@ required:
> >    - compatible
> >    - reg
> >  
> > -additionalProperties: true
> 
> This should certainly be changed though. We should only have 'true' for 
> incomplete collections of properties. IOW, for common bindings.
> 
> > +unevaluatedProperties: false
  

Hi Rob,

On Tue, Oct 25, 2022 at 04:21:14PM -0500, Rob Herring wrote:
> On Mon, Oct 24, 2022 at 10:03:51PM -0700, Colin Foster wrote:
> > The dsa.yaml binding contains duplicated bindings for address and size
> > cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> > this information, remove the reference to dsa-port.yaml and include the
> > full reference to dsa.yaml.
> 
> I don't think this works without further restructuring. Essentially, 
> 'unevaluatedProperties' on works on a single level. So every level has 
> to define all properties at that level either directly in 
> properties/patternProperties or within a $ref.
> 
> See how graph.yaml is structured and referenced for an example how this 
> has to work.

Thanks for pointing me to this. I didn't know about
https://github.com/devicetree-org/dt-schema until now, so I'll take a
look. I was primarily reading the schemas in net/dsa/* to try to get a
full understanding of the DT schema nuances, so these types of nudges
really help me.

And I see that Vladimir Oltean has responded to other parts of the
email, so I'll leave this as a simple "thanks" and keep that context
going forward.
  

Hi Rob and Vladimir,

On Thu, Oct 27, 2022 at 04:25:53AM +0300, Vladimir Oltean wrote:
> Hi Rob,
> 
> On Tue, Oct 25, 2022 at 04:21:14PM -0500, Rob Herring wrote:
> > On Mon, Oct 24, 2022 at 10:03:51PM -0700, Colin Foster wrote:
> > > The dsa.yaml binding contains duplicated bindings for address and size
> > > cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> > > this information, remove the reference to dsa-port.yaml and include the
> > > full reference to dsa.yaml.
> > 
> > I don't think this works without further restructuring. Essentially, 
> > 'unevaluatedProperties' on works on a single level. So every level has 
> > to define all properties at that level either directly in 
> > properties/patternProperties or within a $ref.
> > 
> > See how graph.yaml is structured and referenced for an example how this 
> > has to work.
> > 
> > > @@ -104,8 +98,6 @@ patternProperties:
> > >                SGMII on the QCA8337, it is advised to set this unless a communication
> > >                issue is observed.
> > >  
> > > -        unevaluatedProperties: false
> > > -
> > 
> > Dropping this means any undefined properties in port nodes won't be an 
> > error. Once I fix all the issues related to these missing, there will be 
> > a meta-schema checking for this (this could be one I fixed already).
> 
> I may be misreading, but here, "unevaluatedProperties: false" from dsa.yaml
> (under patternProperties: "^(ethernet-)?port@[0-9]+$":) is on the same
> level as the "unevaluatedProperties: false" that Colin is deleting.
> 
> In fact, I believe that it is precisely due to the "unevaluatedProperties: false"
> from dsa.yaml that this is causing a failure now:
> 
> net/dsa/qca8k.example.dtb: switch@10: ports:port@6: Unevaluated properties are not allowed ('qca,sgmii-rxclk-falling-edge' was unexpected)
> 
> Could you please explain why is the 'qca,sgmii-rxclk-falling-edge'
> property not evaluated from the perspective of dsa.yaml in the example?
> It's a head scratcher to me.
> 
> May it have something to do with the fact that Colin's addition:
> 
> $ref: "dsa.yaml#"
> 
> is not expressed as:
> 
> allOf:
>   - $ref: "dsa.yaml#"
> 
> ?

Looking into documentation (I promise I did some reading / research to
try to get a stronger understanding of the documentation yaml) I came
across the history of ethernet-controller.yaml which suggests to me that
the pattern:

allOf:
  - $ref: 

is frowned upon
commit 3d21a4609335: ("dt-bindings: Remove cases of 'allOf' containing a
'$ref'")

I do have a knack for misinterpreting data, but I read that as:
allOf:
  - $ref:
shouldn't be used unless there's more than one list entry.


All that aside, I did upgrade from 2022.5 to 2022.9 just now and do see
these dtschema errors now. I'll be sure to use this before resubmitting.


> 
> If yes, can you explain exactly what is the difference with respect to
> unevaluatedProperties?
> 
> > >  oneOf:
> > >    - required:
> > >        - ports
> > > @@ -116,7 +108,7 @@ required:
> > >    - compatible
> > >    - reg
> > >  
> > > -additionalProperties: true
> > 
> > This should certainly be changed though. We should only have 'true' for 
> > incomplete collections of properties. IOW, for common bindings.

That makes a lot of sense - and helps me understand why I had so much
trouble understanding why it originally was "additionalProperties: true"


I'll obviously take another look at this. The nxp,sja1105.yaml seemed to
be most akin to what the qca8k.yaml needed to be - that is "take
dsa.yaml and add a couple extra properties to the ports nodes". But
there's always subleties.


> > 
> > > +unevaluatedProperties: false
  

On Thu, Oct 27, 2022 at 04:25:53AM +0300, Vladimir Oltean wrote:
> Hi Rob,
> 
> On Tue, Oct 25, 2022 at 04:21:14PM -0500, Rob Herring wrote:
> > On Mon, Oct 24, 2022 at 10:03:51PM -0700, Colin Foster wrote:
> > > The dsa.yaml binding contains duplicated bindings for address and size
> > > cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> > > this information, remove the reference to dsa-port.yaml and include the
> > > full reference to dsa.yaml.
> > 
> > I don't think this works without further restructuring. Essentially, 
> > 'unevaluatedProperties' on works on a single level. So every level has 
> > to define all properties at that level either directly in 
> > properties/patternProperties or within a $ref.
> > 
> > See how graph.yaml is structured and referenced for an example how this 
> > has to work.
> > 
> > > @@ -104,8 +98,6 @@ patternProperties:
> > >                SGMII on the QCA8337, it is advised to set this unless a communication
> > >                issue is observed.
> > >  
> > > -        unevaluatedProperties: false
> > > -
> > 
> > Dropping this means any undefined properties in port nodes won't be an 
> > error. Once I fix all the issues related to these missing, there will be 
> > a meta-schema checking for this (this could be one I fixed already).
> 
> I may be misreading, but here, "unevaluatedProperties: false" from dsa.yaml
> (under patternProperties: "^(ethernet-)?port@[0-9]+$":) is on the same
> level as the "unevaluatedProperties: false" that Colin is deleting.
> 
> In fact, I believe that it is precisely due to the "unevaluatedProperties: false"
> from dsa.yaml that this is causing a failure now:
> 
> net/dsa/qca8k.example.dtb: switch@10: ports:port@6: Unevaluated properties are not allowed ('qca,sgmii-rxclk-falling-edge' was unexpected)
> 
> Could you please explain why is the 'qca,sgmii-rxclk-falling-edge'
> property not evaluated from the perspective of dsa.yaml in the example?
> It's a head scratcher to me.

A schema with unevaluatedProperties can "see" into a $ref, but the 
ref'ed schema having unevaluatedProperties can't see back to the 
referring schema for properties defined there.

So if a schema is referenced by other schemas which can define their own 
additional properties, that schema cannot have 'unevaluatedProperties: 
false'. If both schemas have 'unevaluatedProperties: false', then it's 
just redundant. We may end up doing that just because it's not obvious 
when we have both or not, and no unevaluatedProperties/ 
additionalProperties at all is a bigger issue. I'm working on a 
meta-schema to check this.


> May it have something to do with the fact that Colin's addition:
> 
> $ref: "dsa.yaml#"
> 
> is not expressed as:
> 
> allOf:
>   - $ref: "dsa.yaml#"
> 
> ?

No. Either way behaves the same. We generally only use 'allOf' when 
there might be more than 1 entry. That is mostly just at the top-level.

Rob
  

On Mon, Oct 31, 2022 at 10:44:09AM -0500, Rob Herring wrote:
> On Thu, Oct 27, 2022 at 04:25:53AM +0300, Vladimir Oltean wrote:
> > Hi Rob,
> > 
> > On Tue, Oct 25, 2022 at 04:21:14PM -0500, Rob Herring wrote:
> > > On Mon, Oct 24, 2022 at 10:03:51PM -0700, Colin Foster wrote:
> > > > The dsa.yaml binding contains duplicated bindings for address and size
> > > > cells, as well as the reference to dsa-port.yaml. Instead of duplicating
> > > > this information, remove the reference to dsa-port.yaml and include the
> > > > full reference to dsa.yaml.
> > > 
> > > I don't think this works without further restructuring. Essentially, 
> > > 'unevaluatedProperties' on works on a single level. So every level has 
> > > to define all properties at that level either directly in 
> > > properties/patternProperties or within a $ref.
> > > 
> > > See how graph.yaml is structured and referenced for an example how this 
> > > has to work.
> > > 
> > > > @@ -104,8 +98,6 @@ patternProperties:
> > > >                SGMII on the QCA8337, it is advised to set this unless a communication
> > > >                issue is observed.
> > > >  
> > > > -        unevaluatedProperties: false
> > > > -
> > > 
> > > Dropping this means any undefined properties in port nodes won't be an 
> > > error. Once I fix all the issues related to these missing, there will be 
> > > a meta-schema checking for this (this could be one I fixed already).
> > 
> > I may be misreading, but here, "unevaluatedProperties: false" from dsa.yaml
> > (under patternProperties: "^(ethernet-)?port@[0-9]+$":) is on the same
> > level as the "unevaluatedProperties: false" that Colin is deleting.
> > 
> > In fact, I believe that it is precisely due to the "unevaluatedProperties: false"
> > from dsa.yaml that this is causing a failure now:
> > 
> > net/dsa/qca8k.example.dtb: switch@10: ports:port@6: Unevaluated properties are not allowed ('qca,sgmii-rxclk-falling-edge' was unexpected)
> > 
> > Could you please explain why is the 'qca,sgmii-rxclk-falling-edge'
> > property not evaluated from the perspective of dsa.yaml in the example?
> > It's a head scratcher to me.
> 
> A schema with unevaluatedProperties can "see" into a $ref, but the 
> ref'ed schema having unevaluatedProperties can't see back to the 
> referring schema for properties defined there.
> 
> So if a schema is referenced by other schemas which can define their own 
> additional properties, that schema cannot have 'unevaluatedProperties: 
> false'. If both schemas have 'unevaluatedProperties: false', then it's 
> just redundant. We may end up doing that just because it's not obvious 
> when we have both or not, and no unevaluatedProperties/ 
> additionalProperties at all is a bigger issue. I'm working on a 
> meta-schema to check this.

Thanks for this information. So if I'm understanding correctly:

 - All DSA chips I'm modifying should reference dsa.yaml, as they
   currently are.
 - As such, these all should have unevaluatedProperties: true, so they
   can see into dsa.yaml.
 - dsa.yaml, and any schema that gets $ref:'d, can not have
   unevaluatedProperties: false, unless the desire is to forbid any
   other properties to be added.

I'll get another patch set out this week with all these changes, and
tested against the latest dt_bindings_check.

> 
> 
> > May it have something to do with the fact that Colin's addition:
> > 
> > $ref: "dsa.yaml#"
> > 
> > is not expressed as:
> > 
> > allOf:
> >   - $ref: "dsa.yaml#"
> > 
> > ?
> 
> No. Either way behaves the same. We generally only use 'allOf' when 
> there might be more than 1 entry. That is mostly just at the top-level.
> 
> Rob