[00/14] renesas: rzg3s: Add support for Ethernet

  From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

Hi,

Series adds Ethernet support for Renesas RZ/G3S Ethernet.
Along with it preparatory cleanups and fixes were included.

Patches 1-4 are clock specific.
Patches 5-8 are pinctrl specific.
Patches 9-13 are device tree specific.
Patch 14 updates multi_v7_defconfig with RAVB flag.

Thank you,
Claudiu Beznea

Claudiu Beznea (14):
  clk: renesas: rzg2l-cpg: Reuse code in rzg2l_cpg_reset()
  clk: renesas: rzg2l-cpg: Check reset monitor registers
  clk: renesas: rzg2l-cpg: Add support for MSTOP
  clk: renesas: r9a08g045-cpg: Add clock and reset support for ETH0 and
    ETH1
  pinctrl: renesas: rzg2l: Move arg in the main function block
  pinctrl: renesas: rzg2l: Add pin configuration support for pinmux
    groups
  pinctrl: renesas: rzg2l: Add support to select power source for
    Ethernet pins
  pinctrl: renesas: rzg2l: add output enable support
  dt-bindings: net: renesas,etheravb: Document RZ/G3S support
  arm64: renesas: r9a08g045: Add Ethernet nodes
  arm64: renesas: rzg3s-smarc-som: Invert the logic for SW_SD2_EN macro
  arm64: dts: renesas: Improve documentation for SW_SD0_DEV_SEL
  arm64: dts: renesas: rzg3s-smarc-som: Enable Ethernet interfaces
  arm: multi_v7_defconfig: Enable CONFIG_RAVB

 .../bindings/net/renesas,etheravb.yaml        |   1 +
 arch/arm/configs/multi_v7_defconfig           |   1 +
 arch/arm64/boot/dts/renesas/r9a08g045.dtsi    |  32 ++++
 .../boot/dts/renesas/rzg3s-smarc-som.dtsi     | 153 +++++++++++++++-
 drivers/clk/renesas/r9a07g043-cpg.c           | 116 ++++++------
 drivers/clk/renesas/r9a07g044-cpg.c           | 158 ++++++++---------
 drivers/clk/renesas/r9a08g045-cpg.c           |  64 +++++--
 drivers/clk/renesas/r9a09g011-cpg.c           | 116 ++++++------
 drivers/clk/renesas/rzg2l-cpg.c               | 166 +++++++++++++++---
 drivers/clk/renesas/rzg2l-cpg.h               |  21 ++-
 drivers/pinctrl/renesas/pinctrl-rzg2l.c       | 166 ++++++++++++++++--
 11 files changed, 736 insertions(+), 258 deletions(-)
  

On Mon, Nov 20, 2023 at 8:00 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:

> Patches 5-8 are pinctrl specific.

I expect Geert to pick these once he's happy with them and merge them
into his tree for pull request to my pinctrl tree.

If you want some other merging approach then inform us!

Yours,
Linus Walleij
  

On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>
> Code in rzg2l_cpg_reset() is equivalent with the combined code of
> rzg2l_cpg_assert() and rzg2l_cpg_deassert(). There is no need to have
> different versions thus re-use rzg2l_cpg_assert() and rzg2l_cpg_deassert().
>
> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
i.e. will queue in renesas-clk-for-v6.8.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
  

Hi Claudiu,

On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>
> Hardware manual of both RZ/G2L and RZ/G3S specifies that reset monitor
> registers need to be interrogated when the reset signals are toggled
> (chapters "Procedures for Supplying and Stopping Reset Signals" and
> "Procedure for Activating Modules"). Without this there is a chance that
> different modules (e.g. Ethernet) to not be ready after reset signal is
> toggled leading to failures (on probe or resume from deep sleep states).
>
> Fixes: ef3c613ccd68 ("clk: renesas: Add CPG core wrapper for RZ/G2L SoC")
> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

Thanks for your patch!

> In case you apply this patch and patch 1/13 as is, please add a Depend-on
> tag on this patch to point to patch 1/13 for proper backporting.

There is no such Depend-on tag? Anyway, this patch won't apply if 1/13
is not backported...

> --- a/drivers/clk/renesas/rzg2l-cpg.c
> +++ b/drivers/clk/renesas/rzg2l-cpg.c
> @@ -1416,12 +1416,23 @@ static int rzg2l_cpg_assert(struct reset_controller_dev *rcdev,
>         struct rzg2l_cpg_priv *priv = rcdev_to_priv(rcdev);
>         const struct rzg2l_cpg_info *info = priv->info;
>         unsigned int reg = info->resets[id].off;
> -       u32 value = BIT(info->resets[id].bit) << 16;
> +       u32 dis = BIT(info->resets[id].bit);
> +       u32 value = dis << 16;
> +       int ret = 0;
>
>         dev_dbg(rcdev->dev, "assert id:%ld offset:0x%x\n", id, CLK_RST_R(reg));
>
>         writel(value, priv->base + CLK_RST_R(reg));
> -       return 0;
> +
> +       if (info->has_clk_mon_regs) {
> +               ret = readl_poll_timeout_atomic(priv->base + CLK_MRST_R(reg), value,
> +                                               value & dis, 10, 200);
> +       } else {
> +               /* Wait for at least one cycle of the RCLK clock (@ ca. 32 kHz) */
> +               udelay(35);
> +       }

I think this should also take into account CPG_RST_MON on RZ/V2M,
cfr. rzg2l_cpg_status().

> +
> +       return ret;
>  }
>
>  static int rzg2l_cpg_deassert(struct reset_controller_dev *rcdev,
> @@ -1432,12 +1443,22 @@ static int rzg2l_cpg_deassert(struct reset_controller_dev *rcdev,
>         unsigned int reg = info->resets[id].off;
>         u32 dis = BIT(info->resets[id].bit);
>         u32 value = (dis << 16) | dis;
> +       int ret = 0;
>
>         dev_dbg(rcdev->dev, "deassert id:%ld offset:0x%x\n", id,
>                 CLK_RST_R(reg));
>
>         writel(value, priv->base + CLK_RST_R(reg));
> -       return 0;
> +
> +       if (info->has_clk_mon_regs) {
> +               ret = readl_poll_timeout_atomic(priv->base + CLK_MRST_R(reg), value,
> +                                               !(value & dis), 10, 200);
> +       } else {
> +               /* Wait for at least one cycle of the RCLK clock (@ ca. 32 kHz) */
> +               udelay(35);
> +       }

Likewise.

> +
> +       return ret;
>  }
>
>  static int rzg2l_cpg_reset(struct reset_controller_dev *rcdev,

The rest LGTM.

Gr{oetje,eeting}s,

                        Geert
  

Hi Claudiu,

On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>
> RZ/{G2L, V2L, G3S} based CPG versions have support for saving extra
> power when clocks are disabled by activating module standby. This is done
> though MSTOP specific registers that are part of CPG. Each individual
> module have one or more bits associated in one MSTOP register (see table
> "Registers for Module Standby Mode" from HW manuals). Hardware manual
> associates modules' clocks to one or more MSTOP bits. There are 3 mappings
> available (identified by researching RZ/G2L, RZ/G3S, RZ/V2L HW manuals):
>
> case 1: N clocks mapped to N MSTOP bits (with N={0, ..., X})
> case 2: N clocks mapped to 1 MSTOP bit  (with N={0, ..., X})
> case 3: N clocks mapped to M MSTOP bits (with N={0, ..., X}, M={0, ..., Y})
>
> Case 3 has been currently identified on RZ/V2L for VCPL4 module.
>
> To cover all 3 cases the individual platform drivers will provide to
> clock driver MSTOP register offset and associated bits in this register
> as a bitmask and the clock driver will apply this bitmask to proper
> MSTOP register.
>
> As most of the modules have more than one clock and these clocks are
> mapped to 1 MSTOP bitmap that need to be applied to MSTOP registers,
> to avoid switching the module to/out of standby when the module has
> enabled/disabled clocks a counter has been associated to each module
> (though struct mstop::count) which is incremented/decremented every
> time a module's clock is enabled/disabled and the settings to MSTOP
> register are applied only when the counter reaches zero (counter zero
> means either 1st clock of the module is going to be enabled or all clocks
> of the module are going to be disabled).

Thanks for your patch!

> The MSTOP functionality has been instantiated at the moment for RZ/G3S.

Do you plan to add support for the other SoCs, too?

> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

> --- a/drivers/clk/renesas/r9a08g045-cpg.c
> +++ b/drivers/clk/renesas/r9a08g045-cpg.c
> @@ -187,23 +187,39 @@ static const struct cpg_core_clk r9a08g045_core_clks[] __initconst = {
>  };
>
>  static const struct rzg2l_mod_clk r9a08g045_mod_clks[] = {
> -       DEF_MOD("gic_gicclk",           R9A08G045_GIC600_GICCLK, R9A08G045_CLK_P1, 0x514, 0),
> -       DEF_MOD("ia55_clk",             R9A08G045_IA55_CLK, R9A08G045_CLK_P1, 0x518, 1),
> -       DEF_MOD("dmac_aclk",            R9A08G045_DMAC_ACLK, R9A08G045_CLK_P3, 0x52c, 0),
> -       DEF_MOD("sdhi0_imclk",          R9A08G045_SDHI0_IMCLK, CLK_SD0_DIV4, 0x554, 0),
> -       DEF_MOD("sdhi0_imclk2",         R9A08G045_SDHI0_IMCLK2, CLK_SD0_DIV4, 0x554, 1),
> -       DEF_MOD("sdhi0_clk_hs",         R9A08G045_SDHI0_CLK_HS, R9A08G045_CLK_SD0, 0x554, 2),
> -       DEF_MOD("sdhi0_aclk",           R9A08G045_SDHI0_ACLK, R9A08G045_CLK_P1, 0x554, 3),
> -       DEF_MOD("sdhi1_imclk",          R9A08G045_SDHI1_IMCLK, CLK_SD1_DIV4, 0x554, 4),
> -       DEF_MOD("sdhi1_imclk2",         R9A08G045_SDHI1_IMCLK2, CLK_SD1_DIV4, 0x554, 5),
> -       DEF_MOD("sdhi1_clk_hs",         R9A08G045_SDHI1_CLK_HS, R9A08G045_CLK_SD1, 0x554, 6),
> -       DEF_MOD("sdhi1_aclk",           R9A08G045_SDHI1_ACLK, R9A08G045_CLK_P1, 0x554, 7),
> -       DEF_MOD("sdhi2_imclk",          R9A08G045_SDHI2_IMCLK, CLK_SD2_DIV4, 0x554, 8),
> -       DEF_MOD("sdhi2_imclk2",         R9A08G045_SDHI2_IMCLK2, CLK_SD2_DIV4, 0x554, 9),
> -       DEF_MOD("sdhi2_clk_hs",         R9A08G045_SDHI2_CLK_HS, R9A08G045_CLK_SD2, 0x554, 10),
> -       DEF_MOD("sdhi2_aclk",           R9A08G045_SDHI2_ACLK, R9A08G045_CLK_P1, 0x554, 11),
> -       DEF_MOD("scif0_clk_pck",        R9A08G045_SCIF0_CLK_PCK, R9A08G045_CLK_P0, 0x584, 0),
> -       DEF_MOD("gpio_hclk",            R9A08G045_GPIO_HCLK, R9A08G045_OSCCLK, 0x598, 0),
> +       DEF_MOD("gic_gicclk",           R9A08G045_GIC600_GICCLK, R9A08G045_CLK_P1, 0x514, 0,
> +                                       MSTOP(ACPU, BIT(3))),

According to Rev. 1.00 of the Hardware User's Manual, bit 3 of the
CPG_BUS_ACPU_MSTOP register is reserved?

Also, gic_gicclk is a critical module clock, so I guess this module
must never be put into standby?

> --- a/drivers/clk/renesas/rzg2l-cpg.c
> +++ b/drivers/clk/renesas/rzg2l-cpg.c
> @@ -1177,6 +1177,17 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
>                 core->name, PTR_ERR(clk));
>  }
>
> +/**
> + * struct mstop - MSTOP specific data structure
> + * @count: reference counter for MSTOP settings (when zero the settings
> + *        are applied to register)
> + * @conf: MSTOP configuration (register offset, setup bits)
> + */
> +struct mstop {
> +       u32 count;
> +       u32 conf;
> +};
> +
>  /**
>   * struct mstp_clock - MSTP gating clock
>   *
> @@ -1186,6 +1197,7 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
>   * @enabled: soft state of the clock, if it is coupled with another clock
>   * @priv: CPG/MSTP private data
>   * @sibling: pointer to the other coupled clock
> + * @mstop: MSTOP configuration
>   */
>  struct mstp_clock {
>         struct clk_hw hw;
> @@ -1194,10 +1206,46 @@ struct mstp_clock {
>         bool enabled;
>         struct rzg2l_cpg_priv *priv;
>         struct mstp_clock *sibling;
> +       struct mstop *mstop;
>  };
>
>  #define to_mod_clock(_hw) container_of(_hw, struct mstp_clock, hw)
>
> +/* Need to be called with a lock held to avoid concurent access to mstop->count. */

concurrent

> +static void rzg2l_mod_clock_module_set_standby(struct mstp_clock *clock,
> +                                              bool standby)
> +{
> +       struct rzg2l_cpg_priv *priv = clock->priv;
> +       struct mstop *mstop = clock->mstop;
> +       bool update = false;
> +       u32 value;
> +
> +       if (!mstop)
> +               return;
> +
> +       value = MSTOP_MASK(mstop->conf) << 16;
> +
> +       if (standby) {
> +               value |= MSTOP_MASK(mstop->conf);
> +               /* Avoid overflow. */
> +               if (mstop->count > 0)
> +                       mstop->count--;

Should we add a WARN() here, or is it sufficient to rely on the WARN()
in drivers/clk/clk.c:clk_core_disable()?

> +
> +               if (!mstop->count)
> +                       update = true;
> +       } else {
> +               if (!mstop->count)
> +                       update = true;
> +
> +               /* Avoid overflow. */
> +               if (mstop->count + 1 != 0)
> +                       mstop->count++;

Trying to avoid an overflow won't help much here.  The counter
will be wrong afterwards anyway, and when decrementing again later, the
module will be put in standby too soon...

> +       }
> +
> +       if (update)
> +               writel(value, priv->base + MSTOP_OFF(mstop->conf));
> +}
> +
>  static int rzg2l_mod_clock_endisable(struct clk_hw *hw, bool enable)
>  {
>         struct mstp_clock *clock = to_mod_clock(hw);

> @@ -1401,6 +1474,37 @@ rzg2l_cpg_register_mod_clk(const struct rzg2l_mod_clk *mod,
>                 }
>         }
>
> +       if (mod->mstop_conf) {
> +               struct mstop *mstop = rzg2l_mod_clock_get_mstop(priv, mod->mstop_conf);
> +
> +               if (mstop) {
> +                       clock->mstop = mstop;

Please move the common assignment after the if/else block...

> +               } else {

... so this can just become "if (!mstop) {".

> +                       mstop = devm_kzalloc(dev, sizeof(*mstop), GFP_KERNEL);
> +                       if (!mstop) {
> +                               clk_unregister(clk);
> +                               goto fail;

Please use "goto unregister", and call clk_unregister() after the new
unregister label.

> +                       }
> +
> +                       mstop->conf = mod->mstop_conf;
> +                       clock->mstop = mstop;
> +               }
> +
> +               if (rzg2l_mod_clock_is_enabled(&clock->hw)) {
> +                       if (clock->sibling)
> +                               clock->mstop->count = 1;
> +                       else
> +                               clock->mstop->count++;
> +               }
> +
> +               /*
> +                * Out of reset all modules are enabled. Set module to standby
> +                * in case associated clocks are disabled at probe.

Is that always true?
What about kexec and crashdump kernels?

> +                */
> +               if (!clock->mstop->count)
> +                       rzg2l_mod_clock_module_set_standby(clock, true);
> +       }
> +
>         return;
>
>  fail:
> diff --git a/drivers/clk/renesas/rzg2l-cpg.h b/drivers/clk/renesas/rzg2l-cpg.h
> index 6e38c8fc888c..10ee8aa4a5da 100644
> --- a/drivers/clk/renesas/rzg2l-cpg.h
> +++ b/drivers/clk/renesas/rzg2l-cpg.h

> @@ -68,6 +73,10 @@
>  #define SEL_PLL6_2     SEL_PLL_PACK(CPG_PL6_ETH_SSEL, 0, 1)
>  #define SEL_GPU2       SEL_PLL_PACK(CPG_PL6_SSEL, 12, 1)
>
> +#define MSTOP(name, bitmask)   ((CPG_##name##_MSTOP) << 16 | (bitmask))

I believe the bitmask is always a single bit.
So perhaps let MSTOP() take the bit number instead of the bitmaskl?
You can still store BIT(bit) inside the macro.

> +#define MSTOP_OFF(conf)                ((conf) >> 16)
> +#define MSTOP_MASK(conf)       ((conf) & GENMASK(15, 0))
> +
>  #define EXTAL_FREQ_IN_MEGA_HZ  (24)
>
>  /**

Gr{oetje,eeting}s,

                        Geert
  

On 23.11.2023 17:53, Geert Uytterhoeven wrote:
> Hi Claudiu,
> 
> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>>
>> Hardware manual of both RZ/G2L and RZ/G3S specifies that reset monitor
>> registers need to be interrogated when the reset signals are toggled
>> (chapters "Procedures for Supplying and Stopping Reset Signals" and
>> "Procedure for Activating Modules"). Without this there is a chance that
>> different modules (e.g. Ethernet) to not be ready after reset signal is
>> toggled leading to failures (on probe or resume from deep sleep states).
>>
>> Fixes: ef3c613ccd68 ("clk: renesas: Add CPG core wrapper for RZ/G2L SoC")
>> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> 
> Thanks for your patch!
> 
>> In case you apply this patch and patch 1/13 as is, please add a Depend-on
>> tag on this patch to point to patch 1/13 for proper backporting.
> 
> There is no such Depend-on tag? Anyway, this patch won't apply if 1/13

typo again... it should have been "Depends-on" which is true, it is not
documented anywhere, but I saw it is used in some commits. Maybe I should
stop using it...

> is not backported...
> 
>> --- a/drivers/clk/renesas/rzg2l-cpg.c
>> +++ b/drivers/clk/renesas/rzg2l-cpg.c
>> @@ -1416,12 +1416,23 @@ static int rzg2l_cpg_assert(struct reset_controller_dev *rcdev,
>>         struct rzg2l_cpg_priv *priv = rcdev_to_priv(rcdev);
>>         const struct rzg2l_cpg_info *info = priv->info;
>>         unsigned int reg = info->resets[id].off;
>> -       u32 value = BIT(info->resets[id].bit) << 16;
>> +       u32 dis = BIT(info->resets[id].bit);
>> +       u32 value = dis << 16;
>> +       int ret = 0;
>>
>>         dev_dbg(rcdev->dev, "assert id:%ld offset:0x%x\n", id, CLK_RST_R(reg));
>>
>>         writel(value, priv->base + CLK_RST_R(reg));
>> -       return 0;
>> +
>> +       if (info->has_clk_mon_regs) {
>> +               ret = readl_poll_timeout_atomic(priv->base + CLK_MRST_R(reg), value,
>> +                                               value & dis, 10, 200);
>> +       } else {
>> +               /* Wait for at least one cycle of the RCLK clock (@ ca. 32 kHz) */
>> +               udelay(35);
>> +       }
> 
> I think this should also take into account CPG_RST_MON on RZ/V2M,
> cfr. rzg2l_cpg_status().

Hm... ok, I'll have a look though it will be a bit difficult to test it ATM.

> 
>> +
>> +       return ret;
>>  }
>>
>>  static int rzg2l_cpg_deassert(struct reset_controller_dev *rcdev,
>> @@ -1432,12 +1443,22 @@ static int rzg2l_cpg_deassert(struct reset_controller_dev *rcdev,
>>         unsigned int reg = info->resets[id].off;
>>         u32 dis = BIT(info->resets[id].bit);
>>         u32 value = (dis << 16) | dis;
>> +       int ret = 0;
>>
>>         dev_dbg(rcdev->dev, "deassert id:%ld offset:0x%x\n", id,
>>                 CLK_RST_R(reg));
>>
>>         writel(value, priv->base + CLK_RST_R(reg));
>> -       return 0;
>> +
>> +       if (info->has_clk_mon_regs) {
>> +               ret = readl_poll_timeout_atomic(priv->base + CLK_MRST_R(reg), value,
>> +                                               !(value & dis), 10, 200);
>> +       } else {
>> +               /* Wait for at least one cycle of the RCLK clock (@ ca. 32 kHz) */
>> +               udelay(35);
>> +       }
> 
> Likewise.
> 
>> +
>> +       return ret;
>>  }
>>
>>  static int rzg2l_cpg_reset(struct reset_controller_dev *rcdev,
> 
> The rest LGTM.
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
>
  

Hi Claudiu,

On Thu, Nov 23, 2023 at 5:35 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> > From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> >
> > RZ/{G2L, V2L, G3S} based CPG versions have support for saving extra
> > power when clocks are disabled by activating module standby. This is done
> > though MSTOP specific registers that are part of CPG. Each individual
> > module have one or more bits associated in one MSTOP register (see table
> > "Registers for Module Standby Mode" from HW manuals). Hardware manual
> > associates modules' clocks to one or more MSTOP bits. There are 3 mappings
> > available (identified by researching RZ/G2L, RZ/G3S, RZ/V2L HW manuals):
> >
> > case 1: N clocks mapped to N MSTOP bits (with N={0, ..., X})
> > case 2: N clocks mapped to 1 MSTOP bit  (with N={0, ..., X})
> > case 3: N clocks mapped to M MSTOP bits (with N={0, ..., X}, M={0, ..., Y})
> >
> > Case 3 has been currently identified on RZ/V2L for VCPL4 module.
> >
> > To cover all 3 cases the individual platform drivers will provide to
> > clock driver MSTOP register offset and associated bits in this register
> > as a bitmask and the clock driver will apply this bitmask to proper
> > MSTOP register.
> >
> > As most of the modules have more than one clock and these clocks are
> > mapped to 1 MSTOP bitmap that need to be applied to MSTOP registers,
> > to avoid switching the module to/out of standby when the module has
> > enabled/disabled clocks a counter has been associated to each module
> > (though struct mstop::count) which is incremented/decremented every
> > time a module's clock is enabled/disabled and the settings to MSTOP
> > register are applied only when the counter reaches zero (counter zero
> > means either 1st clock of the module is going to be enabled or all clocks
> > of the module are going to be disabled).
>
> Thanks for your patch!
>
> > The MSTOP functionality has been instantiated at the moment for RZ/G3S.
> >
> > Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

> > --- a/drivers/clk/renesas/rzg2l-cpg.c
> > +++ b/drivers/clk/renesas/rzg2l-cpg.c
> > @@ -1177,6 +1177,17 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
> >                 core->name, PTR_ERR(clk));
> >  }
> >
> > +/**
> > + * struct mstop - MSTOP specific data structure
> > + * @count: reference counter for MSTOP settings (when zero the settings
> > + *        are applied to register)
> > + * @conf: MSTOP configuration (register offset, setup bits)
> > + */
> > +struct mstop {
> > +       u32 count;
> > +       u32 conf;
> > +};
> > +
> >  /**
> >   * struct mstp_clock - MSTP gating clock
> >   *
> > @@ -1186,6 +1197,7 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
> >   * @enabled: soft state of the clock, if it is coupled with another clock
> >   * @priv: CPG/MSTP private data
> >   * @sibling: pointer to the other coupled clock
> > + * @mstop: MSTOP configuration
> >   */
> >  struct mstp_clock {
> >         struct clk_hw hw;
> > @@ -1194,10 +1206,46 @@ struct mstp_clock {
> >         bool enabled;
> >         struct rzg2l_cpg_priv *priv;
> >         struct mstp_clock *sibling;
> > +       struct mstop *mstop;
> >  };
> >
> >  #define to_mod_clock(_hw) container_of(_hw, struct mstp_clock, hw)
> >
> > +/* Need to be called with a lock held to avoid concurent access to mstop->count. */
>
> concurrent
>
> > +static void rzg2l_mod_clock_module_set_standby(struct mstp_clock *clock,
> > +                                              bool standby)
> > +{
> > +       struct rzg2l_cpg_priv *priv = clock->priv;
> > +       struct mstop *mstop = clock->mstop;
> > +       bool update = false;
> > +       u32 value;
> > +
> > +       if (!mstop)
> > +               return;
> > +
> > +       value = MSTOP_MASK(mstop->conf) << 16;
> > +
> > +       if (standby) {
> > +               value |= MSTOP_MASK(mstop->conf);
> > +               /* Avoid overflow. */
> > +               if (mstop->count > 0)
> > +                       mstop->count--;
>
> Should we add a WARN() here, or is it sufficient to rely on the WARN()
> in drivers/clk/clk.c:clk_core_disable()?
>
> > +
> > +               if (!mstop->count)
> > +                       update = true;
> > +       } else {
> > +               if (!mstop->count)
> > +                       update = true;
> > +
> > +               /* Avoid overflow. */
> > +               if (mstop->count + 1 != 0)
> > +                       mstop->count++;
>
> Trying to avoid an overflow won't help much here.  The counter
> will be wrong afterwards anyway, and when decrementing again later, the
> module will be put in standby too soon...
>
> > +       }
> > +
> > +       if (update)
> > +               writel(value, priv->base + MSTOP_OFF(mstop->conf));
> > +}

After giving this some more thought, it feels odd to derive the standby
state of a module from the state of its module clocks, while the latter
are already controlled through Runtime PM and a Clock Domain.

A first alternative solution could be to drop the GENPD_FLAG_PM_CLK
flag from the RZ/G2L CPG clock domain, and provide your own
gpd_dev_ops.start() and .stop() callbacks that take care of both
module standby and clocks (through pm_clk_{resume,suspend}().
(See https://elixir.bootlin.com/linux/v6.7-rc2/source/drivers/base/power/domain.c#L2093
for the GENPD_FLAG_PM_CLK case).
That still leaves you with a need to associate an MSTOP register and
bitmask with a device through its module clocks.

A second alternative solution could be to increase #power-domain-cells
from zero to one, and register individual PM Domains for each module,
and control module standby from the generic_pm_domain.power_{on,off}()
callbacks.  Devices would specify the module using the power-domains =
<&cpg <id> > property in DT, with <id> one of the to-be-added list of
modules in include/dt-bindings/clock/r9a08g045-cpg.h.  The RZ/G2L CPG
driver can handle the mapping from <id> to MSTOP register and bitmask.
This solution requires updates to DT, but you can keep compatibility
with old DTBs by only registering the new PM Domains when
#power-domain-cells is one.
The extra power saving would only be applicable with new DTBs, though.

Thoughts?

> > --- a/drivers/clk/renesas/rzg2l-cpg.h
> > +++ b/drivers/clk/renesas/rzg2l-cpg.h
>
> > @@ -68,6 +73,10 @@
> >  #define SEL_PLL6_2     SEL_PLL_PACK(CPG_PL6_ETH_SSEL, 0, 1)
> >  #define SEL_GPU2       SEL_PLL_PACK(CPG_PL6_SSEL, 12, 1)
> >
> > +#define MSTOP(name, bitmask)   ((CPG_##name##_MSTOP) << 16 | (bitmask))
>
> I believe the bitmask is always a single bit.
> So perhaps let MSTOP() take the bit number instead of the bitmaskl?
> You can still store BIT(bit) inside the macro.

I was wrong, the N->N or N->M cases need a bitmask.

> > +#define MSTOP_OFF(conf)                ((conf) >> 16)
> > +#define MSTOP_MASK(conf)       ((conf) & GENMASK(15, 0))
> > +
> >  #define EXTAL_FREQ_IN_MEGA_HZ  (24)

Gr{oetje,eeting}s,

                        Geert
  

Hi, Geert,

On 23.11.2023 18:35, Geert Uytterhoeven wrote:
> Hi Claudiu,
> 
> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>>
>> RZ/{G2L, V2L, G3S} based CPG versions have support for saving extra
>> power when clocks are disabled by activating module standby. This is done
>> though MSTOP specific registers that are part of CPG. Each individual
>> module have one or more bits associated in one MSTOP register (see table
>> "Registers for Module Standby Mode" from HW manuals). Hardware manual
>> associates modules' clocks to one or more MSTOP bits. There are 3 mappings
>> available (identified by researching RZ/G2L, RZ/G3S, RZ/V2L HW manuals):
>>
>> case 1: N clocks mapped to N MSTOP bits (with N={0, ..., X})
>> case 2: N clocks mapped to 1 MSTOP bit  (with N={0, ..., X})
>> case 3: N clocks mapped to M MSTOP bits (with N={0, ..., X}, M={0, ..., Y})
>>
>> Case 3 has been currently identified on RZ/V2L for VCPL4 module.
>>
>> To cover all 3 cases the individual platform drivers will provide to
>> clock driver MSTOP register offset and associated bits in this register
>> as a bitmask and the clock driver will apply this bitmask to proper
>> MSTOP register.
>>
>> As most of the modules have more than one clock and these clocks are
>> mapped to 1 MSTOP bitmap that need to be applied to MSTOP registers,
>> to avoid switching the module to/out of standby when the module has
>> enabled/disabled clocks a counter has been associated to each module
>> (though struct mstop::count) which is incremented/decremented every
>> time a module's clock is enabled/disabled and the settings to MSTOP
>> register are applied only when the counter reaches zero (counter zero
>> means either 1st clock of the module is going to be enabled or all clocks
>> of the module are going to be disabled).
> 
> Thanks for your patch!
> 
>> The MSTOP functionality has been instantiated at the moment for RZ/G3S.
> 
> Do you plan to add support for the other SoCs, too?

Yes.

> 
>> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> 
>> --- a/drivers/clk/renesas/r9a08g045-cpg.c
>> +++ b/drivers/clk/renesas/r9a08g045-cpg.c
>> @@ -187,23 +187,39 @@ static const struct cpg_core_clk r9a08g045_core_clks[] __initconst = {
>>  };
>>
>>  static const struct rzg2l_mod_clk r9a08g045_mod_clks[] = {
>> -       DEF_MOD("gic_gicclk",           R9A08G045_GIC600_GICCLK, R9A08G045_CLK_P1, 0x514, 0),
>> -       DEF_MOD("ia55_clk",             R9A08G045_IA55_CLK, R9A08G045_CLK_P1, 0x518, 1),
>> -       DEF_MOD("dmac_aclk",            R9A08G045_DMAC_ACLK, R9A08G045_CLK_P3, 0x52c, 0),
>> -       DEF_MOD("sdhi0_imclk",          R9A08G045_SDHI0_IMCLK, CLK_SD0_DIV4, 0x554, 0),
>> -       DEF_MOD("sdhi0_imclk2",         R9A08G045_SDHI0_IMCLK2, CLK_SD0_DIV4, 0x554, 1),
>> -       DEF_MOD("sdhi0_clk_hs",         R9A08G045_SDHI0_CLK_HS, R9A08G045_CLK_SD0, 0x554, 2),
>> -       DEF_MOD("sdhi0_aclk",           R9A08G045_SDHI0_ACLK, R9A08G045_CLK_P1, 0x554, 3),
>> -       DEF_MOD("sdhi1_imclk",          R9A08G045_SDHI1_IMCLK, CLK_SD1_DIV4, 0x554, 4),
>> -       DEF_MOD("sdhi1_imclk2",         R9A08G045_SDHI1_IMCLK2, CLK_SD1_DIV4, 0x554, 5),
>> -       DEF_MOD("sdhi1_clk_hs",         R9A08G045_SDHI1_CLK_HS, R9A08G045_CLK_SD1, 0x554, 6),
>> -       DEF_MOD("sdhi1_aclk",           R9A08G045_SDHI1_ACLK, R9A08G045_CLK_P1, 0x554, 7),
>> -       DEF_MOD("sdhi2_imclk",          R9A08G045_SDHI2_IMCLK, CLK_SD2_DIV4, 0x554, 8),
>> -       DEF_MOD("sdhi2_imclk2",         R9A08G045_SDHI2_IMCLK2, CLK_SD2_DIV4, 0x554, 9),
>> -       DEF_MOD("sdhi2_clk_hs",         R9A08G045_SDHI2_CLK_HS, R9A08G045_CLK_SD2, 0x554, 10),
>> -       DEF_MOD("sdhi2_aclk",           R9A08G045_SDHI2_ACLK, R9A08G045_CLK_P1, 0x554, 11),
>> -       DEF_MOD("scif0_clk_pck",        R9A08G045_SCIF0_CLK_PCK, R9A08G045_CLK_P0, 0x584, 0),
>> -       DEF_MOD("gpio_hclk",            R9A08G045_GPIO_HCLK, R9A08G045_OSCCLK, 0x598, 0),
>> +       DEF_MOD("gic_gicclk",           R9A08G045_GIC600_GICCLK, R9A08G045_CLK_P1, 0x514, 0,
>> +                                       MSTOP(ACPU, BIT(3))),
> 
> According to Rev. 1.00 of the Hardware User's Manual, bit 3 of the
> CPG_BUS_ACPU_MSTOP register is reserved?

Hm... you're right. I've followed table 44.4 Registers for Module Standby
Mode to populate MSTOPs in r9a08g045_mod_clks[]. That table indicates bit 3
for GIC.

> 
> Also, gic_gicclk is a critical module clock, so I guess this module
> must never be put into standby?

Good point. I'll remove the MSTOPs for critical clocks.

> 
>> --- a/drivers/clk/renesas/rzg2l-cpg.c
>> +++ b/drivers/clk/renesas/rzg2l-cpg.c
>> @@ -1177,6 +1177,17 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
>>                 core->name, PTR_ERR(clk));
>>  }
>>
>> +/**
>> + * struct mstop - MSTOP specific data structure
>> + * @count: reference counter for MSTOP settings (when zero the settings
>> + *        are applied to register)
>> + * @conf: MSTOP configuration (register offset, setup bits)
>> + */
>> +struct mstop {
>> +       u32 count;
>> +       u32 conf;
>> +};
>> +
>>  /**
>>   * struct mstp_clock - MSTP gating clock
>>   *
>> @@ -1186,6 +1197,7 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
>>   * @enabled: soft state of the clock, if it is coupled with another clock
>>   * @priv: CPG/MSTP private data
>>   * @sibling: pointer to the other coupled clock
>> + * @mstop: MSTOP configuration
>>   */
>>  struct mstp_clock {
>>         struct clk_hw hw;
>> @@ -1194,10 +1206,46 @@ struct mstp_clock {
>>         bool enabled;
>>         struct rzg2l_cpg_priv *priv;
>>         struct mstp_clock *sibling;
>> +       struct mstop *mstop;
>>  };
>>
>>  #define to_mod_clock(_hw) container_of(_hw, struct mstp_clock, hw)
>>
>> +/* Need to be called with a lock held to avoid concurent access to mstop->count. */
> 
> concurrent
> 
>> +static void rzg2l_mod_clock_module_set_standby(struct mstp_clock *clock,
>> +                                              bool standby)
>> +{
>> +       struct rzg2l_cpg_priv *priv = clock->priv;
>> +       struct mstop *mstop = clock->mstop;
>> +       bool update = false;
>> +       u32 value;
>> +
>> +       if (!mstop)
>> +               return;
>> +
>> +       value = MSTOP_MASK(mstop->conf) << 16;
>> +
>> +       if (standby) {
>> +               value |= MSTOP_MASK(mstop->conf);
>> +               /* Avoid overflow. */
>> +               if (mstop->count > 0)
>> +                       mstop->count--;
> 
> Should we add a WARN() here, or is it sufficient to rely on the WARN()
> in drivers/clk/clk.c:clk_core_disable()?

I think it would be good to have it as mstop->count could be
incremented/decremented by more than one clock and could overflow faster
than struct clk_core::enable_count

> 
>> +
>> +               if (!mstop->count)
>> +                       update = true;
>> +       } else {
>> +               if (!mstop->count)
>> +                       update = true;
>> +
>> +               /* Avoid overflow. */
>> +               if (mstop->count + 1 != 0)
>> +                       mstop->count++;
> 
> Trying to avoid an overflow won't help much here.  The counter
> will be wrong afterwards anyway, and when decrementing again later, the
> module will be put in standby too soon...

That's true. Would you prefer to have a WARN() for this too?

> 
>> +       }
>> +
>> +       if (update)
>> +               writel(value, priv->base + MSTOP_OFF(mstop->conf));
>> +}
>> +
>>  static int rzg2l_mod_clock_endisable(struct clk_hw *hw, bool enable)
>>  {
>>         struct mstp_clock *clock = to_mod_clock(hw);
> 
>> @@ -1401,6 +1474,37 @@ rzg2l_cpg_register_mod_clk(const struct rzg2l_mod_clk *mod,
>>                 }
>>         }
>>
>> +       if (mod->mstop_conf) {
>> +               struct mstop *mstop = rzg2l_mod_clock_get_mstop(priv, mod->mstop_conf);
>> +
>> +               if (mstop) {
>> +                       clock->mstop = mstop;
> 
> Please move the common assignment after the if/else block...
> 
>> +               } else {
> 
> ... so this can just become "if (!mstop) {".

Ok, I'll review it.

> 
>> +                       mstop = devm_kzalloc(dev, sizeof(*mstop), GFP_KERNEL);
>> +                       if (!mstop) {
>> +                               clk_unregister(clk);
>> +                               goto fail;
> 
> Please use "goto unregister", and call clk_unregister() after the new
> unregister label.

I kept it like this as I considered otherwise the error path might become
unnecessary complicated.

> 
>> +                       }
>> +
>> +                       mstop->conf = mod->mstop_conf;
>> +                       clock->mstop = mstop;
>> +               }
>> +
>> +               if (rzg2l_mod_clock_is_enabled(&clock->hw)) {
>> +                       if (clock->sibling)
>> +                               clock->mstop->count = 1;
>> +                       else
>> +                               clock->mstop->count++;
>> +               }
>> +
>> +               /*
>> +                * Out of reset all modules are enabled. Set module to standby
>> +                * in case associated clocks are disabled at probe.
> 
> Is that always true?
> What about kexec and crashdump kernels?

I was referring to the hardware reset. In case we reach this point with
clocks already enabled by a previous kernel the state of the clocks in
hardware should be enabled and the mstop->count should be updated
accordingly by the above if block. Let me know if I'm missing something.

> 
>> +                */
>> +               if (!clock->mstop->count)
>> +                       rzg2l_mod_clock_module_set_standby(clock, true);
>> +       }
>> +
>>         return;
>>
>>  fail:
>> diff --git a/drivers/clk/renesas/rzg2l-cpg.h b/drivers/clk/renesas/rzg2l-cpg.h
>> index 6e38c8fc888c..10ee8aa4a5da 100644
>> --- a/drivers/clk/renesas/rzg2l-cpg.h
>> +++ b/drivers/clk/renesas/rzg2l-cpg.h
> 
>> @@ -68,6 +73,10 @@
>>  #define SEL_PLL6_2     SEL_PLL_PACK(CPG_PL6_ETH_SSEL, 0, 1)
>>  #define SEL_GPU2       SEL_PLL_PACK(CPG_PL6_SSEL, 12, 1)
>>
>> +#define MSTOP(name, bitmask)   ((CPG_##name##_MSTOP) << 16 | (bitmask))
> 
> I believe the bitmask is always a single bit.
> So perhaps let MSTOP() take the bit number instead of the bitmaskl?
> You can still store BIT(bit) inside the macro.

It is not always the case. That is why I've added the bitmask. The
identified scenarios are highlighted in commit description:

case 1: N clocks mapped to N MSTOP bits (with N={0, ..., X})
case 2: N clocks mapped to 1 MSTOP bit  (with N={0, ..., X})
case 3: N clocks mapped to M MSTOP bits (with N={0, ..., X}, M={0, ..., Y})

Thank you for your review,
Claudiu Beznea

> 
>> +#define MSTOP_OFF(conf)                ((conf) >> 16)
>> +#define MSTOP_MASK(conf)       ((conf) & GENMASK(15, 0))
>> +
>>  #define EXTAL_FREQ_IN_MEGA_HZ  (24)
>>
>>  /**
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
>
  

Hi, Geert,

On 24.11.2023 11:08, Geert Uytterhoeven wrote:
> Hi Claudiu,
> 
> On Thu, Nov 23, 2023 at 5:35 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
>>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>>>
>>> RZ/{G2L, V2L, G3S} based CPG versions have support for saving extra
>>> power when clocks are disabled by activating module standby. This is done
>>> though MSTOP specific registers that are part of CPG. Each individual
>>> module have one or more bits associated in one MSTOP register (see table
>>> "Registers for Module Standby Mode" from HW manuals). Hardware manual
>>> associates modules' clocks to one or more MSTOP bits. There are 3 mappings
>>> available (identified by researching RZ/G2L, RZ/G3S, RZ/V2L HW manuals):
>>>
>>> case 1: N clocks mapped to N MSTOP bits (with N={0, ..., X})
>>> case 2: N clocks mapped to 1 MSTOP bit  (with N={0, ..., X})
>>> case 3: N clocks mapped to M MSTOP bits (with N={0, ..., X}, M={0, ..., Y})
>>>
>>> Case 3 has been currently identified on RZ/V2L for VCPL4 module.
>>>
>>> To cover all 3 cases the individual platform drivers will provide to
>>> clock driver MSTOP register offset and associated bits in this register
>>> as a bitmask and the clock driver will apply this bitmask to proper
>>> MSTOP register.
>>>
>>> As most of the modules have more than one clock and these clocks are
>>> mapped to 1 MSTOP bitmap that need to be applied to MSTOP registers,
>>> to avoid switching the module to/out of standby when the module has
>>> enabled/disabled clocks a counter has been associated to each module
>>> (though struct mstop::count) which is incremented/decremented every
>>> time a module's clock is enabled/disabled and the settings to MSTOP
>>> register are applied only when the counter reaches zero (counter zero
>>> means either 1st clock of the module is going to be enabled or all clocks
>>> of the module are going to be disabled).
>>
>> Thanks for your patch!
>>
>>> The MSTOP functionality has been instantiated at the moment for RZ/G3S.
>>>
>>> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> 
>>> --- a/drivers/clk/renesas/rzg2l-cpg.c
>>> +++ b/drivers/clk/renesas/rzg2l-cpg.c
>>> @@ -1177,6 +1177,17 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
>>>                 core->name, PTR_ERR(clk));
>>>  }
>>>
>>> +/**
>>> + * struct mstop - MSTOP specific data structure
>>> + * @count: reference counter for MSTOP settings (when zero the settings
>>> + *        are applied to register)
>>> + * @conf: MSTOP configuration (register offset, setup bits)
>>> + */
>>> +struct mstop {
>>> +       u32 count;
>>> +       u32 conf;
>>> +};
>>> +
>>>  /**
>>>   * struct mstp_clock - MSTP gating clock
>>>   *
>>> @@ -1186,6 +1197,7 @@ rzg2l_cpg_register_core_clk(const struct cpg_core_clk *core,
>>>   * @enabled: soft state of the clock, if it is coupled with another clock
>>>   * @priv: CPG/MSTP private data
>>>   * @sibling: pointer to the other coupled clock
>>> + * @mstop: MSTOP configuration
>>>   */
>>>  struct mstp_clock {
>>>         struct clk_hw hw;
>>> @@ -1194,10 +1206,46 @@ struct mstp_clock {
>>>         bool enabled;
>>>         struct rzg2l_cpg_priv *priv;
>>>         struct mstp_clock *sibling;
>>> +       struct mstop *mstop;
>>>  };
>>>
>>>  #define to_mod_clock(_hw) container_of(_hw, struct mstp_clock, hw)
>>>
>>> +/* Need to be called with a lock held to avoid concurent access to mstop->count. */
>>
>> concurrent
>>
>>> +static void rzg2l_mod_clock_module_set_standby(struct mstp_clock *clock,
>>> +                                              bool standby)
>>> +{
>>> +       struct rzg2l_cpg_priv *priv = clock->priv;
>>> +       struct mstop *mstop = clock->mstop;
>>> +       bool update = false;
>>> +       u32 value;
>>> +
>>> +       if (!mstop)
>>> +               return;
>>> +
>>> +       value = MSTOP_MASK(mstop->conf) << 16;
>>> +
>>> +       if (standby) {
>>> +               value |= MSTOP_MASK(mstop->conf);
>>> +               /* Avoid overflow. */
>>> +               if (mstop->count > 0)
>>> +                       mstop->count--;
>>
>> Should we add a WARN() here, or is it sufficient to rely on the WARN()
>> in drivers/clk/clk.c:clk_core_disable()?
>>
>>> +
>>> +               if (!mstop->count)
>>> +                       update = true;
>>> +       } else {
>>> +               if (!mstop->count)
>>> +                       update = true;
>>> +
>>> +               /* Avoid overflow. */
>>> +               if (mstop->count + 1 != 0)
>>> +                       mstop->count++;
>>
>> Trying to avoid an overflow won't help much here.  The counter
>> will be wrong afterwards anyway, and when decrementing again later, the
>> module will be put in standby too soon...
>>
>>> +       }
>>> +
>>> +       if (update)
>>> +               writel(value, priv->base + MSTOP_OFF(mstop->conf));
>>> +}
> 
> After giving this some more thought, it feels odd to derive the standby
> state of a module from the state of its module clocks, while the latter
> are already controlled through Runtime PM and a Clock Domain.

Thanks for sharing this.

> 
> A first alternative solution could be to drop the GENPD_FLAG_PM_CLK
> flag from the RZ/G2L CPG clock domain, and provide your own
> gpd_dev_ops.start() and .stop() callbacks that take care of both
> module standby and clocks (through pm_clk_{resume,suspend}().
> (See https://elixir.bootlin.com/linux/v6.7-rc2/source/drivers/base/power/domain.c#L2093
> for the GENPD_FLAG_PM_CLK case).
> That still leaves you with a need to associate an MSTOP register and
> bitmask with a device through its module clocks.
> 
> A second alternative solution could be to increase #power-domain-cells
> from zero to one, and register individual PM Domains for each module,
> and control module standby from the generic_pm_domain.power_{on,off}()
> callbacks.  Devices would specify the module using the power-domains =
> <&cpg <id> > property in DT, with <id> one of the to-be-added list of
> modules in include/dt-bindings/clock/r9a08g045-cpg.h.  The RZ/G2L CPG
> driver can handle the mapping from <id> to MSTOP register and bitmask.
> This solution requires updates to DT, but you can keep compatibility
> with old DTBs by only registering the new PM Domains when
> #power-domain-cells is one.
> The extra power saving would only be applicable with new DTBs, though.

I prefer this alternative even though it cannot be applied for old DTBs, it
looks to me that is more modular. What do you think?

The only thing is that MSTOP is not really a power off/on switch (if it
would be implemented with generic_pm_domain.power_{on, off}) but is more
like a clock disable/enable functionality (it should not be an issue
though, just saying)... According to manual (I'm referring to Figure 41.4
Block Connection Overview for Module Standby Mode of HW manula of RZ/G3S),
it disables/enables the module's bus clock.

Thank you,
Claudiu Beznea

> 
> Thoughts?
> 
>>> --- a/drivers/clk/renesas/rzg2l-cpg.h
>>> +++ b/drivers/clk/renesas/rzg2l-cpg.h
>>
>>> @@ -68,6 +73,10 @@
>>>  #define SEL_PLL6_2     SEL_PLL_PACK(CPG_PL6_ETH_SSEL, 0, 1)
>>>  #define SEL_GPU2       SEL_PLL_PACK(CPG_PL6_SSEL, 12, 1)
>>>
>>> +#define MSTOP(name, bitmask)   ((CPG_##name##_MSTOP) << 16 | (bitmask))
>>
>> I believe the bitmask is always a single bit.
>> So perhaps let MSTOP() take the bit number instead of the bitmaskl?
>> You can still store BIT(bit) inside the macro.
> 
> I was wrong, the N->N or N->M cases need a bitmask.
> 
>>> +#define MSTOP_OFF(conf)                ((conf) >> 16)
>>> +#define MSTOP_MASK(conf)       ((conf) & GENMASK(15, 0))
>>> +
>>>  #define EXTAL_FREQ_IN_MEGA_HZ  (24)
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
>
  

Hi Claudiu,

On Mon, Nov 27, 2023 at 8:37 AM claudiu beznea <claudiu.beznea@tuxon.dev> wrote:
> On 24.11.2023 11:08, Geert Uytterhoeven wrote:
> > On Thu, Nov 23, 2023 at 5:35 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> >>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> >>>
> >>> RZ/{G2L, V2L, G3S} based CPG versions have support for saving extra
> >>> power when clocks are disabled by activating module standby. This is done
> >>> though MSTOP specific registers that are part of CPG. Each individual
> >>> module have one or more bits associated in one MSTOP register (see table
> >>> "Registers for Module Standby Mode" from HW manuals). Hardware manual
> >>> associates modules' clocks to one or more MSTOP bits. There are 3 mappings
> >>> available (identified by researching RZ/G2L, RZ/G3S, RZ/V2L HW manuals):
> >>>
> >>> case 1: N clocks mapped to N MSTOP bits (with N={0, ..., X})
> >>> case 2: N clocks mapped to 1 MSTOP bit  (with N={0, ..., X})
> >>> case 3: N clocks mapped to M MSTOP bits (with N={0, ..., X}, M={0, ..., Y})
> >>>
> >>> Case 3 has been currently identified on RZ/V2L for VCPL4 module.
> >>>
> >>> To cover all 3 cases the individual platform drivers will provide to
> >>> clock driver MSTOP register offset and associated bits in this register
> >>> as a bitmask and the clock driver will apply this bitmask to proper
> >>> MSTOP register.
> >>>
> >>> As most of the modules have more than one clock and these clocks are
> >>> mapped to 1 MSTOP bitmap that need to be applied to MSTOP registers,
> >>> to avoid switching the module to/out of standby when the module has
> >>> enabled/disabled clocks a counter has been associated to each module
> >>> (though struct mstop::count) which is incremented/decremented every
> >>> time a module's clock is enabled/disabled and the settings to MSTOP
> >>> register are applied only when the counter reaches zero (counter zero
> >>> means either 1st clock of the module is going to be enabled or all clocks
> >>> of the module are going to be disabled).

> > After giving this some more thought, it feels odd to derive the standby
> > state of a module from the state of its module clocks, while the latter
> > are already controlled through Runtime PM and a Clock Domain.
> >
> > A first alternative solution could be to drop the GENPD_FLAG_PM_CLK
> > flag from the RZ/G2L CPG clock domain, and provide your own
> > gpd_dev_ops.start() and .stop() callbacks that take care of both
> > module standby and clocks (through pm_clk_{resume,suspend}().
> > (See https://elixir.bootlin.com/linux/v6.7-rc2/source/drivers/base/power/domain.c#L2093
> > for the GENPD_FLAG_PM_CLK case).
> > That still leaves you with a need to associate an MSTOP register and
> > bitmask with a device through its module clocks.
> >
> > A second alternative solution could be to increase #power-domain-cells
> > from zero to one, and register individual PM Domains for each module,
> > and control module standby from the generic_pm_domain.power_{on,off}()
> > callbacks.  Devices would specify the module using the power-domains =
> > <&cpg <id> > property in DT, with <id> one of the to-be-added list of
> > modules in include/dt-bindings/clock/r9a08g045-cpg.h.  The RZ/G2L CPG
> > driver can handle the mapping from <id> to MSTOP register and bitmask.
> > This solution requires updates to DT, but you can keep compatibility
> > with old DTBs by only registering the new PM Domains when
> > #power-domain-cells is one.
> > The extra power saving would only be applicable with new DTBs, though.
>
> I prefer this alternative even though it cannot be applied for old DTBs, it
> looks to me that is more modular. What do you think?

I prefer the second alternative, too.

> The only thing is that MSTOP is not really a power off/on switch (if it
> would be implemented with generic_pm_domain.power_{on, off}) but is more

That's fine: Linux' PM Domains are fairly generic and abstract, and
not limited to pure power domains/areas.

> like a clock disable/enable functionality (it should not be an issue
> though, just saying)... According to manual (I'm referring to Figure 41.4
> Block Connection Overview for Module Standby Mode of HW manula of RZ/G3S),
> it disables/enables the module's bus clock.

Thanks for the pointer! That picture nicely shows the internal behavior.
For comparison, on SH/R-Mobile and R-Car SoCs there is a similar
internal structure, but it is less visible to the programmer:
there are no individual controls for each clock or reset that is
fed into a module.  These are all hidden behind a single Module
Stop resp. Reset control bit.  In Linux, we modeled the module stop
bit as a gate clock, controlled by Runtime PM through the Clock Domain's
.start()/.stop() callbacks.

Note that you also have to take into account Figure 41.2 ("Modules in
Power Domain").  When adding support for power transitions later, you
can register a PM Domain representing PD_ISOVCC, and use that as the
parent PM Domain for the individual PM Domains for modules belonging
to PD_ISOVCC.  All of that can be handled in the driver, and would
not need any changes to DT.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
  

Hi Claudiu,

On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>
> On RZ/G3S different Ethernet pins needs to be configured with different
> settings (e.g. power-source need to be set, RGMII TXC, TX_CTL pins need
> output-enable). Commit adjust driver to allow specifying pin configuration
> for pinmux groups. With this DT settings like the following are taken
> into account by driver:
>
> eth0_pins: eth0 {
>         tx_ctl {
>                 pinmux = <RZG2L_PORT_PINMUX(1, 1, 1)>;  /* ET0_TX_CTL */
>                 power-source = <1800>;
>                 output-enable;
>                 drive-strength-microamp = <5200>;
>         };
> };
>
> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

Thanks for your patch!

> --- a/drivers/pinctrl/renesas/pinctrl-rzg2l.c
> +++ b/drivers/pinctrl/renesas/pinctrl-rzg2l.c
> @@ -376,8 +376,11 @@ static int rzg2l_dt_subnode_to_map(struct pinctrl_dev *pctldev,
>                 goto done;
>         }
>
> -       if (num_pinmux)
> +       if (num_pinmux) {
>                 nmaps += 1;
> +               if (num_configs)
> +                       nmaps += 1;

I think this would be more readable, and better follow the style of
the surrounding statements, if this new check would not be nested
under the num_pinmux check.

> +       }
>
>         if (num_pins)
>                 nmaps += num_pins;

Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>

Gr{oetje,eeting}s,

                        Geert
  

Hi Claudiu,

On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>
> Add Ethernet nodes available on RZ/G3S (R9A08G045).
>
> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>

Thanks for your patch!

> --- a/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
> +++ b/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
> @@ -149,6 +149,38 @@ sdhi2: mmc@11c20000 {
>                         status = "disabled";
>                 };
>
> +               eth0: ethernet@11c30000 {
> +                       compatible = "renesas,r9a08g045-gbeth", "renesas,rzg2l-gbeth";
> +                       reg = <0 0x11c30000 0 0x10000>;
> +                       interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>,
> +                                    <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
> +                                    <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
> +                       interrupt-names = "mux", "fil", "arp_ns";
> +                       clocks = <&cpg CPG_MOD R9A08G045_ETH0_CLK_AXI>,
> +                                <&cpg CPG_MOD R9A08G045_ETH0_CLK_CHI>,
> +                                <&cpg CPG_MOD R9A08G045_ETH0_REFCLK>;
> +                       clock-names = "axi", "chi", "refclk";
> +                       resets = <&cpg R9A08G045_ETH0_RST_HW_N>;
> +                       power-domains = <&cpg>;

Perhaps add a default phy mode, like on other SoCs?

    phy-mode = "rgmii"';

Also missing:

    #address-cells = <1>;
    #size-cells = <0>;

> +                       status = "disabled";
> +               };

Same comments for eth1.

Gr{oetje,eeting}s,

                        Geert
  

Hi, Geert,

On 01.12.2023 19:35, Geert Uytterhoeven wrote:
> Hi Claudiu,
> 
> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>>
>> Add Ethernet nodes available on RZ/G3S (R9A08G045).
>>
>> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> 
> Thanks for your patch!
> 
>> --- a/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
>> +++ b/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
>> @@ -149,6 +149,38 @@ sdhi2: mmc@11c20000 {
>>                         status = "disabled";
>>                 };
>>
>> +               eth0: ethernet@11c30000 {
>> +                       compatible = "renesas,r9a08g045-gbeth", "renesas,rzg2l-gbeth";
>> +                       reg = <0 0x11c30000 0 0x10000>;
>> +                       interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>,
>> +                                    <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
>> +                                    <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
>> +                       interrupt-names = "mux", "fil", "arp_ns";
>> +                       clocks = <&cpg CPG_MOD R9A08G045_ETH0_CLK_AXI>,
>> +                                <&cpg CPG_MOD R9A08G045_ETH0_CLK_CHI>,
>> +                                <&cpg CPG_MOD R9A08G045_ETH0_REFCLK>;
>> +                       clock-names = "axi", "chi", "refclk";
>> +                       resets = <&cpg R9A08G045_ETH0_RST_HW_N>;
>> +                       power-domains = <&cpg>;
> 
> Perhaps add a default phy mode, like on other SoCs?
> 
>     phy-mode = "rgmii"';

I skipped this (even it was available on the other SoCs) as I consider the
phy-mode is board specific.

> 
> Also missing:
> 
>     #address-cells = <1>;
>     #size-cells = <0>;

Same for these.

> 
>> +                       status = "disabled";
>> +               };
> 
> Same comments for eth1.
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
>
  

Hi Claudiu,

On Mon, Dec 4, 2023 at 8:41 AM claudiu beznea <claudiu.beznea@tuxon.dev> wrote:
> On 01.12.2023 19:35, Geert Uytterhoeven wrote:
> > On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> >> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> >>
> >> Add Ethernet nodes available on RZ/G3S (R9A08G045).
> >>
> >> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> >
> > Thanks for your patch!
> >
> >> --- a/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
> >> +++ b/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
> >> @@ -149,6 +149,38 @@ sdhi2: mmc@11c20000 {
> >>                         status = "disabled";
> >>                 };
> >>
> >> +               eth0: ethernet@11c30000 {
> >> +                       compatible = "renesas,r9a08g045-gbeth", "renesas,rzg2l-gbeth";
> >> +                       reg = <0 0x11c30000 0 0x10000>;
> >> +                       interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>,
> >> +                                    <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
> >> +                                    <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
> >> +                       interrupt-names = "mux", "fil", "arp_ns";
> >> +                       clocks = <&cpg CPG_MOD R9A08G045_ETH0_CLK_AXI>,
> >> +                                <&cpg CPG_MOD R9A08G045_ETH0_CLK_CHI>,
> >> +                                <&cpg CPG_MOD R9A08G045_ETH0_REFCLK>;
> >> +                       clock-names = "axi", "chi", "refclk";
> >> +                       resets = <&cpg R9A08G045_ETH0_RST_HW_N>;
> >> +                       power-domains = <&cpg>;
> >
> > Perhaps add a default phy mode, like on other SoCs?
> >
> >     phy-mode = "rgmii"';
>
> I skipped this (even it was available on the other SoCs) as I consider the
> phy-mode is board specific.

IC.  Still, it's good to have some consistency across boards.

> > Also missing:
> >
> >     #address-cells = <1>;
> >     #size-cells = <0>;
>
> Same for these.

These are required, and always have the same values, so it makes more
sense to have them in the SoC .dtsi file, once.

Gr{oetje,eeting}s,

                        Geert
  

On 04.12.2023 10:02, Geert Uytterhoeven wrote:
> Hi Claudiu,
> 
> On Mon, Dec 4, 2023 at 8:41 AM claudiu beznea <claudiu.beznea@tuxon.dev> wrote:
>> On 01.12.2023 19:35, Geert Uytterhoeven wrote:
>>> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
>>>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>>>>
>>>> Add Ethernet nodes available on RZ/G3S (R9A08G045).
>>>>
>>>> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
>>>
>>> Thanks for your patch!
>>>
>>>> --- a/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
>>>> +++ b/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
>>>> @@ -149,6 +149,38 @@ sdhi2: mmc@11c20000 {
>>>>                         status = "disabled";
>>>>                 };
>>>>
>>>> +               eth0: ethernet@11c30000 {
>>>> +                       compatible = "renesas,r9a08g045-gbeth", "renesas,rzg2l-gbeth";
>>>> +                       reg = <0 0x11c30000 0 0x10000>;
>>>> +                       interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>,
>>>> +                                    <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
>>>> +                                    <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
>>>> +                       interrupt-names = "mux", "fil", "arp_ns";
>>>> +                       clocks = <&cpg CPG_MOD R9A08G045_ETH0_CLK_AXI>,
>>>> +                                <&cpg CPG_MOD R9A08G045_ETH0_CLK_CHI>,
>>>> +                                <&cpg CPG_MOD R9A08G045_ETH0_REFCLK>;
>>>> +                       clock-names = "axi", "chi", "refclk";
>>>> +                       resets = <&cpg R9A08G045_ETH0_RST_HW_N>;
>>>> +                       power-domains = <&cpg>;
>>>
>>> Perhaps add a default phy mode, like on other SoCs?
>>>
>>>     phy-mode = "rgmii"';
>>
>> I skipped this (even it was available on the other SoCs) as I consider the
>> phy-mode is board specific.
> 
> IC.  Still, it's good to have some consistency across boards.
> 
>>> Also missing:
>>>
>>>     #address-cells = <1>;
>>>     #size-cells = <0>;
>>
>> Same for these.
> 
> These are required, and always have the same values, so it makes more
> sense to have them in the SoC .dtsi file, once.

I remember I had a compilation warning with an Ethernet controller
configured with fixed-link having #address-cells, #size-cells. With
fixed-link these were not needed.

Anyway... I'll keep all in dtsi if you prefer it this way.

Thank you,
Claudiu Beznea

> 
> Gr{oetje,eeting}s,
> 
>                         Geert
>
  

Hi Claudiu,

On Mon, Dec 4, 2023 at 9:38 AM claudiu beznea <claudiu.beznea@tuxon.dev> wrote:
> On 04.12.2023 10:02, Geert Uytterhoeven wrote:
> > On Mon, Dec 4, 2023 at 8:41 AM claudiu beznea <claudiu.beznea@tuxon.dev> wrote:
> >> On 01.12.2023 19:35, Geert Uytterhoeven wrote:
> >>> On Mon, Nov 20, 2023 at 8:01 AM Claudiu <claudiu.beznea@tuxon.dev> wrote:
> >>>> From: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> >>>>
> >>>> Add Ethernet nodes available on RZ/G3S (R9A08G045).
> >>>>
> >>>> Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
> >>>
> >>> Thanks for your patch!
> >>>
> >>>> --- a/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
> >>>> +++ b/arch/arm64/boot/dts/renesas/r9a08g045.dtsi
> >>>> @@ -149,6 +149,38 @@ sdhi2: mmc@11c20000 {
> >>>>                         status = "disabled";
> >>>>                 };
> >>>>
> >>>> +               eth0: ethernet@11c30000 {
> >>>> +                       compatible = "renesas,r9a08g045-gbeth", "renesas,rzg2l-gbeth";
> >>>> +                       reg = <0 0x11c30000 0 0x10000>;
> >>>> +                       interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>,
> >>>> +                                    <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
> >>>> +                                    <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
> >>>> +                       interrupt-names = "mux", "fil", "arp_ns";
> >>>> +                       clocks = <&cpg CPG_MOD R9A08G045_ETH0_CLK_AXI>,
> >>>> +                                <&cpg CPG_MOD R9A08G045_ETH0_CLK_CHI>,
> >>>> +                                <&cpg CPG_MOD R9A08G045_ETH0_REFCLK>;
> >>>> +                       clock-names = "axi", "chi", "refclk";
> >>>> +                       resets = <&cpg R9A08G045_ETH0_RST_HW_N>;
> >>>> +                       power-domains = <&cpg>;
> >>>
> >>> Perhaps add a default phy mode, like on other SoCs?
> >>>
> >>>     phy-mode = "rgmii"';
> >>
> >> I skipped this (even it was available on the other SoCs) as I consider the
> >> phy-mode is board specific.
> >
> > IC.  Still, it's good to have some consistency across boards.
> >
> >>> Also missing:
> >>>
> >>>     #address-cells = <1>;
> >>>     #size-cells = <0>;
> >>
> >> Same for these.
> >
> > These are required, and always have the same values, so it makes more
> > sense to have them in the SoC .dtsi file, once.
>
> I remember I had a compilation warning with an Ethernet controller
> configured with fixed-link having #address-cells, #size-cells. With
> fixed-link these were not needed.

I think EtherAVB always use MDIO for management, so fixed-link is
not applicable.

> Anyway... I'll keep all in dtsi if you prefer it this way.

Yes please, thanks!

Gr{oetje,eeting}s,

                        Geert