[v5,3/3] cpufreq: qcom-hw: Add CPU clock provider support

  Qcom CPUFreq hardware (EPSS/OSM) controls clock and voltage to the CPU
cores. But this relationship is not represented with the clk framework
so far.

So, let's make the qcom-cpufreq-hw driver a clock provider. This makes the
clock producer/consumer relationship cleaner and is also useful for CPU
related frameworks like OPP to know the frequency at which the CPUs are
running.

The clock frequency provided by the driver is for each frequency domain.
We cannot get the frequency of each CPU core because, not all platforms
support per-core DCVS feature.

Also the frequency supplied by the driver is the actual frequency that
comes out of the EPSS/OSM block after the DCVS operation. This frequency is
not same as what the CPUFreq framework has set but it is the one that gets
supplied to the CPUs after throttling by LMh.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 drivers/cpufreq/qcom-cpufreq-hw.c | 43 +++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
  

On Tue, Nov 08, 2022 at 09:10:37PM +0530, Manivannan Sadhasivam wrote:
> Qcom CPUFreq hardware (EPSS/OSM) controls clock and voltage to the CPU
> cores. But this relationship is not represented with the clk framework
> so far.
> 
> So, let's make the qcom-cpufreq-hw driver a clock provider. This makes the
> clock producer/consumer relationship cleaner and is also useful for CPU
> related frameworks like OPP to know the frequency at which the CPUs are
> running.
> 
> The clock frequency provided by the driver is for each frequency domain.
> We cannot get the frequency of each CPU core because, not all platforms
> support per-core DCVS feature.
> 
> Also the frequency supplied by the driver is the actual frequency that
> comes out of the EPSS/OSM block after the DCVS operation. This frequency is
> not same as what the CPUFreq framework has set but it is the one that gets
> supplied to the CPUs after throttling by LMh.
>

OK now I see more info here. How different is this value from the one
returned by qcom_cpufreq_hw_get() ?
  

Hi,

On Tue, Nov 08, 2022 at 09:10:37PM +0530, Manivannan Sadhasivam wrote:
> Qcom CPUFreq hardware (EPSS/OSM) controls clock and voltage to the CPU
> cores. But this relationship is not represented with the clk framework
> so far.
> 
> So, let's make the qcom-cpufreq-hw driver a clock provider. This makes the
> clock producer/consumer relationship cleaner and is also useful for CPU
> related frameworks like OPP to know the frequency at which the CPUs are
> running.
> 
> The clock frequency provided by the driver is for each frequency domain.
> We cannot get the frequency of each CPU core because, not all platforms
> support per-core DCVS feature.
> 
> Also the frequency supplied by the driver is the actual frequency that
> comes out of the EPSS/OSM block after the DCVS operation. This frequency is
> not same as what the CPUFreq framework has set but it is the one that gets
> supplied to the CPUs after throttling by LMh.
> 
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> ---
>  drivers/cpufreq/qcom-cpufreq-hw.c | 43 +++++++++++++++++++++++++++++++
>  1 file changed, 43 insertions(+)
> 
> diff --git a/drivers/cpufreq/qcom-cpufreq-hw.c b/drivers/cpufreq/qcom-cpufreq-hw.c
> index 5e0598730a04..86bb11de347f 100644
> --- a/drivers/cpufreq/qcom-cpufreq-hw.c
> +++ b/drivers/cpufreq/qcom-cpufreq-hw.c
> @@ -4,6 +4,7 @@
>   */
>  
>  #include <linux/bitfield.h>
> +#include <linux/clk-provider.h>
>  #include <linux/cpufreq.h>
>  #include <linux/init.h>
>  #include <linux/interconnect.h>
> @@ -54,6 +55,7 @@ struct qcom_cpufreq_data {
>  	bool cancel_throttle;
>  	struct delayed_work throttle_work;
>  	struct cpufreq_policy *policy;
> +	struct clk_hw cpu_clk;
>  
>  	bool per_core_dcvs;
>  
> @@ -615,8 +617,20 @@ static struct cpufreq_driver cpufreq_qcom_hw_driver = {
>  	.ready		= qcom_cpufreq_ready,
>  };
>  
> +static unsigned long qcom_cpufreq_hw_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> +{
> +	struct qcom_cpufreq_data *data = container_of(hw, struct qcom_cpufreq_data, cpu_clk);
> +
> +	return qcom_lmh_get_throttle_freq(data);
> +}
> +
> +static const struct clk_ops qcom_cpufreq_hw_clk_ops = {
> +	.recalc_rate = qcom_cpufreq_hw_recalc_rate,
> +};
> +
>  static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
>  {
> +	struct clk_hw_onecell_data *clk_data;
>  	struct device *dev = &pdev->dev;
>  	struct device *cpu_dev;
>  	struct clk *clk;
> @@ -659,8 +673,16 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
>  
>  	qcom_cpufreq.soc_data = of_device_get_match_data(dev);
>  
> +	clk_data = devm_kzalloc(dev, struct_size(clk_data, hws, num_domains), GFP_KERNEL);
> +	if (!clk_data)
> +		return -ENOMEM;
> +
> +	clk_data->num = num_domains;
> +
>  	for (i = 0; i < num_domains; i++) {
>  		struct qcom_cpufreq_data *data = &qcom_cpufreq.data[i];
> +		struct clk_init_data init = {};
> +		const char *clk_name;
>  		struct resource *res;
>  		void __iomem *base;
>  
> @@ -672,6 +694,27 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
>  
>  		data->base = base;
>  		data->res = res;
> +
> +		/* Register CPU clock for each frequency domain */
> +		clk_name = devm_kasprintf(dev, GFP_KERNEL, "qcom_cpufreq%d", i);
> +		init.name = clk_name;

nit: 'clk_name' isn't really needed, the result of devm_kasprintf() could be
assigned directly to 'init.name'. 'init' could be renamed to 'clk_init' if
the purpose of using 'clk_name' is to make clear that this is the name of a
clock.

> +		init.flags = CLK_GET_RATE_NOCACHE;
> +		init.ops = &qcom_cpufreq_hw_clk_ops;
> +		data->cpu_clk.init = &init;
> +
> +		ret = devm_clk_hw_register(dev, &data->cpu_clk);
> +		if (ret < 0) {
> +			dev_err(dev, "Failed to register Qcom CPUFreq clock\n");
> +			return ret;
> +		}
> +
> +		clk_data->hws[i] = &data->cpu_clk;
> +	}
> +
> +	ret = devm_of_clk_add_hw_provider(dev, of_clk_hw_onecell_get, clk_data);
> +	if (ret < 0) {
> +		dev_err(dev, "Failed to add Qcom CPUFreq clock provider\n");
> +		return ret;
>  	}
>  
>  	ret = cpufreq_register_driver(&cpufreq_qcom_hw_driver);
> -- 
> 2.25.1
>
  

On Tue, Nov 08, 2022 at 03:57:17PM +0000, Sudeep Holla wrote:
> On Tue, Nov 08, 2022 at 09:10:37PM +0530, Manivannan Sadhasivam wrote:
> > Qcom CPUFreq hardware (EPSS/OSM) controls clock and voltage to the CPU
> > cores. But this relationship is not represented with the clk framework
> > so far.
> > 
> > So, let's make the qcom-cpufreq-hw driver a clock provider. This makes the
> > clock producer/consumer relationship cleaner and is also useful for CPU
> > related frameworks like OPP to know the frequency at which the CPUs are
> > running.
> > 
> > The clock frequency provided by the driver is for each frequency domain.
> > We cannot get the frequency of each CPU core because, not all platforms
> > support per-core DCVS feature.
> > 
> > Also the frequency supplied by the driver is the actual frequency that
> > comes out of the EPSS/OSM block after the DCVS operation. This frequency is
> > not same as what the CPUFreq framework has set but it is the one that gets
> > supplied to the CPUs after throttling by LMh.
> >
> 
> OK now I see more info here. How different is this value from the one
> returned by qcom_cpufreq_hw_get() ?
> 

qcom_cpufreq_hw_get() returns the frequency that got programmed by the cpufreq
core. But that frequency is not necessarily the one that gets delivered to the
CPU cores because the EPSS/OSM hardware block may vary the frequency after the
DCVS operation.

So this frequency is the final one that gets delivered to the CPU cores.

Thanks,
Mani

> -- 
> Regards,
> Sudeep
  

On Tue, Nov 08, 2022 at 06:27:36PM +0000, Matthias Kaehlcke wrote:
> Hi,
> 
> On Tue, Nov 08, 2022 at 09:10:37PM +0530, Manivannan Sadhasivam wrote:
> > Qcom CPUFreq hardware (EPSS/OSM) controls clock and voltage to the CPU
> > cores. But this relationship is not represented with the clk framework
> > so far.
> > 
> > So, let's make the qcom-cpufreq-hw driver a clock provider. This makes the
> > clock producer/consumer relationship cleaner and is also useful for CPU
> > related frameworks like OPP to know the frequency at which the CPUs are
> > running.
> > 
> > The clock frequency provided by the driver is for each frequency domain.
> > We cannot get the frequency of each CPU core because, not all platforms
> > support per-core DCVS feature.
> > 
> > Also the frequency supplied by the driver is the actual frequency that
> > comes out of the EPSS/OSM block after the DCVS operation. This frequency is
> > not same as what the CPUFreq framework has set but it is the one that gets
> > supplied to the CPUs after throttling by LMh.
> > 
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > ---
> >  drivers/cpufreq/qcom-cpufreq-hw.c | 43 +++++++++++++++++++++++++++++++
> >  1 file changed, 43 insertions(+)
> > 
> > diff --git a/drivers/cpufreq/qcom-cpufreq-hw.c b/drivers/cpufreq/qcom-cpufreq-hw.c
> > index 5e0598730a04..86bb11de347f 100644
> > --- a/drivers/cpufreq/qcom-cpufreq-hw.c
> > +++ b/drivers/cpufreq/qcom-cpufreq-hw.c
> > @@ -4,6 +4,7 @@
> >   */
> >  
> >  #include <linux/bitfield.h>
> > +#include <linux/clk-provider.h>
> >  #include <linux/cpufreq.h>
> >  #include <linux/init.h>
> >  #include <linux/interconnect.h>
> > @@ -54,6 +55,7 @@ struct qcom_cpufreq_data {
> >  	bool cancel_throttle;
> >  	struct delayed_work throttle_work;
> >  	struct cpufreq_policy *policy;
> > +	struct clk_hw cpu_clk;
> >  
> >  	bool per_core_dcvs;
> >  
> > @@ -615,8 +617,20 @@ static struct cpufreq_driver cpufreq_qcom_hw_driver = {
> >  	.ready		= qcom_cpufreq_ready,
> >  };
> >  
> > +static unsigned long qcom_cpufreq_hw_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
> > +{
> > +	struct qcom_cpufreq_data *data = container_of(hw, struct qcom_cpufreq_data, cpu_clk);
> > +
> > +	return qcom_lmh_get_throttle_freq(data);
> > +}
> > +
> > +static const struct clk_ops qcom_cpufreq_hw_clk_ops = {
> > +	.recalc_rate = qcom_cpufreq_hw_recalc_rate,
> > +};
> > +
> >  static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
> >  {
> > +	struct clk_hw_onecell_data *clk_data;
> >  	struct device *dev = &pdev->dev;
> >  	struct device *cpu_dev;
> >  	struct clk *clk;
> > @@ -659,8 +673,16 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
> >  
> >  	qcom_cpufreq.soc_data = of_device_get_match_data(dev);
> >  
> > +	clk_data = devm_kzalloc(dev, struct_size(clk_data, hws, num_domains), GFP_KERNEL);
> > +	if (!clk_data)
> > +		return -ENOMEM;
> > +
> > +	clk_data->num = num_domains;
> > +
> >  	for (i = 0; i < num_domains; i++) {
> >  		struct qcom_cpufreq_data *data = &qcom_cpufreq.data[i];
> > +		struct clk_init_data init = {};
> > +		const char *clk_name;
> >  		struct resource *res;
> >  		void __iomem *base;
> >  
> > @@ -672,6 +694,27 @@ static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev)
> >  
> >  		data->base = base;
> >  		data->res = res;
> > +
> > +		/* Register CPU clock for each frequency domain */
> > +		clk_name = devm_kasprintf(dev, GFP_KERNEL, "qcom_cpufreq%d", i);
> > +		init.name = clk_name;
> 
> nit: 'clk_name' isn't really needed, the result of devm_kasprintf() could be
> assigned directly to 'init.name'. 'init' could be renamed to 'clk_init' if
> the purpose of using 'clk_name' is to make clear that this is the name of a
> clock.
> 

Ack.

Thanks,
Mani

> > +		init.flags = CLK_GET_RATE_NOCACHE;
> > +		init.ops = &qcom_cpufreq_hw_clk_ops;
> > +		data->cpu_clk.init = &init;
> > +
> > +		ret = devm_clk_hw_register(dev, &data->cpu_clk);
> > +		if (ret < 0) {
> > +			dev_err(dev, "Failed to register Qcom CPUFreq clock\n");
> > +			return ret;
> > +		}
> > +
> > +		clk_data->hws[i] = &data->cpu_clk;
> > +	}
> > +
> > +	ret = devm_of_clk_add_hw_provider(dev, of_clk_hw_onecell_get, clk_data);
> > +	if (ret < 0) {
> > +		dev_err(dev, "Failed to add Qcom CPUFreq clock provider\n");
> > +		return ret;
> >  	}
> >  
> >  	ret = cpufreq_register_driver(&cpufreq_qcom_hw_driver);
> > -- 
> > 2.25.1
> >
  

On Wed, Nov 09, 2022 at 01:19:08PM +0530, Manivannan Sadhasivam wrote:

[...]

> qcom_cpufreq_hw_get() returns the frequency that got programmed by the cpufreq
> core. But that frequency is not necessarily the one that gets delivered to the
> CPU cores because the EPSS/OSM hardware block may vary the frequency after the
> DCVS operation.
> 
> So this frequency is the final one that gets delivered to the CPU cores.
> 

OK, thanks for the info. Just wondering if there is any issue making
qcom_cpufreq_hw_get() return this value instead of all these complexity.
I think the DT binding is too confusing as cpufreq-dt uses that to manage
DVFS which this one uses it.

If possible we should just make cpufreq_get(cpu) return the value you need
and use the same where ever you need. Sorry if I am missing something obvious
but I am struggling to see that.
  

On Wed, Nov 09, 2022 at 11:08:31AM +0000, Sudeep Holla wrote:
> On Wed, Nov 09, 2022 at 01:19:08PM +0530, Manivannan Sadhasivam wrote:
> 
> [...]
> 
> > qcom_cpufreq_hw_get() returns the frequency that got programmed by the cpufreq
> > core. But that frequency is not necessarily the one that gets delivered to the
> > CPU cores because the EPSS/OSM hardware block may vary the frequency after the
> > DCVS operation.
> > 
> > So this frequency is the final one that gets delivered to the CPU cores.
> > 
> 
> OK, thanks for the info. Just wondering if there is any issue making
> qcom_cpufreq_hw_get() return this value instead of all these complexity.
> I think the DT binding is too confusing as cpufreq-dt uses that to manage
> DVFS which this one uses it.
> 
> If possible we should just make cpufreq_get(cpu) return the value you need
> and use the same where ever you need. Sorry if I am missing something obvious
> but I am struggling to see that.
> 

I don't think using the final DCVS frequency would be applicable for cpufreq
core.

cpufreq core sets the desired frequency in the form of index using the
target_index() callback and the qcom-cpufreq-hw driver uses that index directly
to select the specific entry in the hardware LUT (Look Up Table).

Then with get() callback, the frequency will be returned based on the LUT index
read from the hardware. In this case, the frequency is going to be static
(i.e, what gets set by the cpufreq core will be the same). I believe this is
what the API also expects.

In the case of qcom_lmh_get_throttle_freq(), the frequency is going to be
dynamic (i.e changes with every internal DCVS operation). But this is exactly
what the OPP core expects with clk_get_rate() of CPU clock, so using
qcom_lmh_get_throttle_freq() makes sense there.

Thanks,
Mani

> -- 
> Regards,
> Sudeep
  

On Wed, Nov 09, 2022 at 06:05:26PM +0530, Manivannan Sadhasivam wrote:
> I don't think using the final DCVS frequency would be applicable for cpufreq
> core.
> 
> cpufreq core sets the desired frequency in the form of index using the
> target_index() callback and the qcom-cpufreq-hw driver uses that index directly
> to select the specific entry in the hardware LUT (Look Up Table).
> 
> Then with get() callback, the frequency will be returned based on the LUT index
> read from the hardware. In this case, the frequency is going to be static
> (i.e, what gets set by the cpufreq core will be the same). I believe this is
> what the API also expects.
>

I guessed so and hence thought of asking. Is the cpufreq_get() API expected
to return something close to what was set or is it expected to return the
real set h/w value if and when possible. I wanted to confirm if that is the
expectation from the cpufreq core or is it just the way qcom cpufreq-hw
driver(probably many others too) work today.

> In the case of qcom_lmh_get_throttle_freq(), the frequency is going to be
> dynamic (i.e changes with every internal DCVS operation). But this is exactly
> what the OPP core expects with clk_get_rate() of CPU clock, so using
> qcom_lmh_get_throttle_freq() makes sense there.
>

OK, the reason I ask is that IIRC the ACPI CPPC driver might get the exact
delivered frequency rather than something based on the set value, so it
shouldn't be a requirement but I may be wrong.

Viresh, thoughts ?

--
Regards,
Sudeep
  

On 09-11-22, 16:47, Sudeep Holla wrote:
> Is the cpufreq_get() API expected
> to return something close to what was set or is it expected to return the
> real set h/w value if and when possible.

The real frequency the hardware is running at.
  

On Mon, Nov 14, 2022 at 08:46:58AM +0530, Viresh Kumar wrote:
> On 09-11-22, 16:47, Sudeep Holla wrote:
> > Is the cpufreq_get() API expected
> > to return something close to what was set or is it expected to return the
> > real set h/w value if and when possible.
> 
> The real frequency the hardware is running at.
> 

Oh... In that case, qcom-cpufreq-hw driver is not returning the real frequency
but instead whatever set by the cpufreq core previously using target_index().

Should I fix it too in the next version of this series?

Thanks,
Mani

> -- 
> viresh
  

On 14-11-22, 11:55, Manivannan Sadhasivam wrote:
> On Mon, Nov 14, 2022 at 08:46:58AM +0530, Viresh Kumar wrote:
> > On 09-11-22, 16:47, Sudeep Holla wrote:
> > > Is the cpufreq_get() API expected
> > > to return something close to what was set or is it expected to return the
> > > real set h/w value if and when possible.
> > 
> > The real frequency the hardware is running at.
> > 
> 
> Oh... In that case, qcom-cpufreq-hw driver is not returning the real frequency
> but instead whatever set by the cpufreq core previously using target_index().
> 
> Should I fix it too in the next version of this series?

Yes.