[v2,04/22] clk: qcom: smd-rpm: Export clock scaling availability

  Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
clock rate requests will not be commited in hardware. This poses a
race threat since we're accessing the bus clocks directly from within
the interconnect framework.

Add a marker to indicate that we're good to go with sending new requests
and export it so that it can be referenced from icc.

Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
---
 drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
 include/linux/soc/qcom/smd-rpm.h | 2 ++
 2 files changed, 11 insertions(+)
  

On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
> clock rate requests will not be commited in hardware. This poses a
> race threat since we're accessing the bus clocks directly from within
> the interconnect framework.
> 
> Add a marker to indicate that we're good to go with sending new requests
> and export it so that it can be referenced from icc.
> 
> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
> ---
>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>  2 files changed, 11 insertions(+)
> 
> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
> index 937cb1515968..482fe30ee6f0 100644
> --- a/drivers/clk/qcom/clk-smd-rpm.c
> +++ b/drivers/clk/qcom/clk-smd-rpm.c
> @@ -151,6 +151,7 @@
>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>  
>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
> +static bool smd_rpm_clk_scaling;
>  
>  struct clk_smd_rpm {
>  	const int rpm_res_type;
> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>  	return r->rate;
>  }
>  
> +bool qcom_smd_rpm_scaling_available(void)
> +{
> +	return smd_rpm_clk_scaling;
> +}
> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
> +
>  static int clk_smd_rpm_enable_scaling(void)
>  {
>  	int ret;
> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>  		return ret;
>  	}
>  
> +	smd_rpm_clk_scaling = true;
> +

If you move the platform_device_register_data(&rpdev->dev,
"icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
avoid the race completely and drop this API. I think that would be
cleaner. And it will likely probe much faster because probe deferral
is slow. :)

Thanks,
Stephan
  

On 10.06.2023 13:35, Stephan Gerhold wrote:
> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
>> clock rate requests will not be commited in hardware. This poses a
>> race threat since we're accessing the bus clocks directly from within
>> the interconnect framework.
>>
>> Add a marker to indicate that we're good to go with sending new requests
>> and export it so that it can be referenced from icc.
>>
>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
>> ---
>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>>  2 files changed, 11 insertions(+)
>>
>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>> index 937cb1515968..482fe30ee6f0 100644
>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>> @@ -151,6 +151,7 @@
>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>>  
>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
>> +static bool smd_rpm_clk_scaling;
>>  
>>  struct clk_smd_rpm {
>>  	const int rpm_res_type;
>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>>  	return r->rate;
>>  }
>>  
>> +bool qcom_smd_rpm_scaling_available(void)
>> +{
>> +	return smd_rpm_clk_scaling;
>> +}
>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
>> +
>>  static int clk_smd_rpm_enable_scaling(void)
>>  {
>>  	int ret;
>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>>  		return ret;
>>  	}
>>  
>> +	smd_rpm_clk_scaling = true;
>> +
> 
> If you move the platform_device_register_data(&rpdev->dev,
> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
> avoid the race completely and drop this API. I think that would be
> cleaner. And it will likely probe much faster because probe deferral
> is slow. :)
Sounds like an idea.. especially since it's pretty much the only
dependency other than SMDRPM itself!

Konrad
> 
> Thanks,
> Stephan
  

On 10.06.2023 14:15, Konrad Dybcio wrote:
> 
> 
> On 10.06.2023 13:35, Stephan Gerhold wrote:
>> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
>>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
>>> clock rate requests will not be commited in hardware. This poses a
>>> race threat since we're accessing the bus clocks directly from within
>>> the interconnect framework.
>>>
>>> Add a marker to indicate that we're good to go with sending new requests
>>> and export it so that it can be referenced from icc.
>>>
>>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
>>> ---
>>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>>>  2 files changed, 11 insertions(+)
>>>
>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>> index 937cb1515968..482fe30ee6f0 100644
>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>> @@ -151,6 +151,7 @@
>>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>>>  
>>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
>>> +static bool smd_rpm_clk_scaling;
>>>  
>>>  struct clk_smd_rpm {
>>>  	const int rpm_res_type;
>>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>>>  	return r->rate;
>>>  }
>>>  
>>> +bool qcom_smd_rpm_scaling_available(void)
>>> +{
>>> +	return smd_rpm_clk_scaling;
>>> +}
>>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
>>> +
>>>  static int clk_smd_rpm_enable_scaling(void)
>>>  {
>>>  	int ret;
>>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>>>  		return ret;
>>>  	}
>>>  
>>> +	smd_rpm_clk_scaling = true;
>>> +
>>
>> If you move the platform_device_register_data(&rpdev->dev,
>> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
>> avoid the race completely and drop this API. I think that would be
>> cleaner. And it will likely probe much faster because probe deferral
>> is slow. :)
> Sounds like an idea.. especially since it's pretty much the only
> dependency other than SMDRPM itself!
It sounds great, but to not break bisecting one has to:

1. change the registration in soc/smd-rpm to store rpm ptr in driver
   data, in addition to parent driver data

2. change icc/smd-rpm to use the device and not parent data

3. add a platform_device_register_data call in clk-smd-rpm that will
   always fail because the device is always registered

4. remove the registration from soc/smd-rpm


I know you'd love to see me break my [PATCH xx/42] record, but I'd say
that deserves its own series and *this* patch could be a good transition
middleground :P

Hopefully Stephen, Bjorn and Georgi won't send a hitman after me for
abusing atomic cross-subsystem merges..

Konrad

> 
> Konrad
>>
>> Thanks,
>> Stephan
  

On Sat, Jun 10, 2023 at 08:53:05PM +0200, Konrad Dybcio wrote:
> On 10.06.2023 14:15, Konrad Dybcio wrote:
> > On 10.06.2023 13:35, Stephan Gerhold wrote:
> >> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
> >>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
> >>> clock rate requests will not be commited in hardware. This poses a
> >>> race threat since we're accessing the bus clocks directly from within
> >>> the interconnect framework.
> >>>
> >>> Add a marker to indicate that we're good to go with sending new requests
> >>> and export it so that it can be referenced from icc.
> >>>
> >>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
> >>> ---
> >>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
> >>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
> >>>  2 files changed, 11 insertions(+)
> >>>
> >>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
> >>> index 937cb1515968..482fe30ee6f0 100644
> >>> --- a/drivers/clk/qcom/clk-smd-rpm.c
> >>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
> >>> @@ -151,6 +151,7 @@
> >>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
> >>>  
> >>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
> >>> +static bool smd_rpm_clk_scaling;
> >>>  
> >>>  struct clk_smd_rpm {
> >>>  	const int rpm_res_type;
> >>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
> >>>  	return r->rate;
> >>>  }
> >>>  
> >>> +bool qcom_smd_rpm_scaling_available(void)
> >>> +{
> >>> +	return smd_rpm_clk_scaling;
> >>> +}
> >>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
> >>> +
> >>>  static int clk_smd_rpm_enable_scaling(void)
> >>>  {
> >>>  	int ret;
> >>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
> >>>  		return ret;
> >>>  	}
> >>>  
> >>> +	smd_rpm_clk_scaling = true;
> >>> +
> >>
> >> If you move the platform_device_register_data(&rpdev->dev,
> >> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
> >> avoid the race completely and drop this API. I think that would be
> >> cleaner. And it will likely probe much faster because probe deferral
> >> is slow. :)
> > Sounds like an idea.. especially since it's pretty much the only
> > dependency other than SMDRPM itself!
> It sounds great, but to not break bisecting one has to:
> 
> 1. change the registration in soc/smd-rpm to store rpm ptr in driver
>    data, in addition to parent driver data
> 
> 2. change icc/smd-rpm to use the device and not parent data
> 
> 3. add a platform_device_register_data call in clk-smd-rpm that will
>    always fail because the device is always registered
> 
> 4. remove the registration from soc/smd-rpm
> 

Logically the icc_smd_rpm device still fits better as child of
smd-rpm and not clk-smd-rpm. So I would probably just continue
registering it on the parent device from clk-smd-rpm.
Then there are no changes necessary in icc_smd_rpm.

You could use this. Both touched files are Bjorn-maintained so should be
manageable to have it in one commit. (note: compile-tested only)

Thanks,
Stephan

From a2610adb2551b01e76b9de8e4cbcc89853814a8f Mon Sep 17 00:00:00 2001
From: Stephan Gerhold <stephan@gerhold.net>
Date: Sat, 10 Jun 2023 21:19:48 +0200
Subject: [PATCH] soc: qcom: smd-rpm: Move icc_smd_rpm registration to
 clk-smd-rpm

icc_smd_rpm will do bus clock votes itself rather than taking the
unnecessary detour through the clock subsystem. However, it can only
do that after the clocks have been handed off and scaling has been
enabled in the RPM in clk-smd-rpm.

Move the icc_smd_rpm registration from smd-rpm.c to clk-smd-rpm.c
to avoid any possible races. icc_smd_rpm gets the driver data from
the smd-rpm device, so still register the platform device on the
smd-rpm parent device.

Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
---
 drivers/clk/qcom/clk-smd-rpm.c | 21 +++++++++++++++++++++
 drivers/soc/qcom/smd-rpm.c     | 23 +----------------------
 2 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
index e4de74b68797..91adb16889b3 100644
--- a/drivers/clk/qcom/clk-smd-rpm.c
+++ b/drivers/clk/qcom/clk-smd-rpm.c
@@ -1302,12 +1302,20 @@ static struct clk_hw *qcom_smdrpm_clk_hw_get(struct of_phandle_args *clkspec,
 	return desc->clks[idx] ? &desc->clks[idx]->hw : ERR_PTR(-ENOENT);
 }
 
+static void rpm_smd_unregister_icc(void *data)
+{
+	struct platform_device *icc_pdev = data;
+
+	platform_device_unregister(icc_pdev);
+}
+
 static int rpm_smd_clk_probe(struct platform_device *pdev)
 {
 	int ret;
 	size_t num_clks, i;
 	struct clk_smd_rpm **rpm_smd_clks;
 	const struct rpm_smd_clk_desc *desc;
+	struct platform_device *icc_pdev;
 
 	rpmcc_smd_rpm = dev_get_drvdata(pdev->dev.parent);
 	if (!rpmcc_smd_rpm) {
@@ -1357,6 +1365,19 @@ static int rpm_smd_clk_probe(struct platform_device *pdev)
 	if (ret)
 		goto err;
 
+	icc_pdev = platform_device_register_data(pdev->dev.parent,
+						 "icc_smd_rpm", -1, NULL, 0);
+	if (IS_ERR(icc_pdev)) {
+		dev_err(&pdev->dev, "Failed to register icc_smd_rpm device: %pE\n",
+			icc_pdev);
+		/* No need to unregister clocks because of this */
+	} else {
+		ret = devm_add_action_or_reset(&pdev->dev, rpm_smd_unregister_icc,
+					       icc_pdev);
+		if (ret)
+			goto err;
+	}
+
 	return 0;
 err:
 	dev_err(&pdev->dev, "Error registering SMD clock driver (%d)\n", ret);
diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
index 0c1aa809cc4e..427dd5392b82 100644
--- a/drivers/soc/qcom/smd-rpm.c
+++ b/drivers/soc/qcom/smd-rpm.c
@@ -19,7 +19,6 @@
 /**
  * struct qcom_smd_rpm - state of the rpm device driver
  * @rpm_channel:	reference to the smd channel
- * @icc:		interconnect proxy device
  * @dev:		rpm device
  * @ack:		completion for acks
  * @lock:		mutual exclusion around the send/complete pair
@@ -27,7 +26,6 @@
  */
 struct qcom_smd_rpm {
 	struct rpmsg_endpoint *rpm_channel;
-	struct platform_device *icc;
 	struct device *dev;
 
 	struct completion ack;
@@ -197,7 +195,6 @@ static int qcom_smd_rpm_callback(struct rpmsg_device *rpdev,
 static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
 {
 	struct qcom_smd_rpm *rpm;
-	int ret;
 
 	rpm = devm_kzalloc(&rpdev->dev, sizeof(*rpm), GFP_KERNEL);
 	if (!rpm)
@@ -210,24 +207,7 @@ static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
 	rpm->rpm_channel = rpdev->ept;
 	dev_set_drvdata(&rpdev->dev, rpm);
 
-	rpm->icc = platform_device_register_data(&rpdev->dev, "icc_smd_rpm", -1,
-						 NULL, 0);
-	if (IS_ERR(rpm->icc))
-		return PTR_ERR(rpm->icc);
-
-	ret = of_platform_populate(rpdev->dev.of_node, NULL, NULL, &rpdev->dev);
-	if (ret)
-		platform_device_unregister(rpm->icc);
-
-	return ret;
-}
-
-static void qcom_smd_rpm_remove(struct rpmsg_device *rpdev)
-{
-	struct qcom_smd_rpm *rpm = dev_get_drvdata(&rpdev->dev);
-
-	platform_device_unregister(rpm->icc);
-	of_platform_depopulate(&rpdev->dev);
+	return devm_of_platform_populate(&rpdev->dev);
 }
 
 static const struct of_device_id qcom_smd_rpm_of_match[] = {
@@ -256,7 +236,6 @@ MODULE_DEVICE_TABLE(of, qcom_smd_rpm_of_match);
 
 static struct rpmsg_driver qcom_smd_rpm_driver = {
 	.probe = qcom_smd_rpm_probe,
-	.remove = qcom_smd_rpm_remove,
 	.callback = qcom_smd_rpm_callback,
 	.drv  = {
 		.name  = "qcom_smd_rpm",
  

On 10.06.2023 21:25, Stephan Gerhold wrote:
> On Sat, Jun 10, 2023 at 08:53:05PM +0200, Konrad Dybcio wrote:
>> On 10.06.2023 14:15, Konrad Dybcio wrote:
>>> On 10.06.2023 13:35, Stephan Gerhold wrote:
>>>> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
>>>>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
>>>>> clock rate requests will not be commited in hardware. This poses a
>>>>> race threat since we're accessing the bus clocks directly from within
>>>>> the interconnect framework.
>>>>>
>>>>> Add a marker to indicate that we're good to go with sending new requests
>>>>> and export it so that it can be referenced from icc.
>>>>>
>>>>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
>>>>> ---
>>>>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>>>>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>>>>>  2 files changed, 11 insertions(+)
>>>>>
>>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>>>> index 937cb1515968..482fe30ee6f0 100644
>>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>>>> @@ -151,6 +151,7 @@
>>>>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>>>>>  
>>>>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
>>>>> +static bool smd_rpm_clk_scaling;
>>>>>  
>>>>>  struct clk_smd_rpm {
>>>>>  	const int rpm_res_type;
>>>>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>>>>>  	return r->rate;
>>>>>  }
>>>>>  
>>>>> +bool qcom_smd_rpm_scaling_available(void)
>>>>> +{
>>>>> +	return smd_rpm_clk_scaling;
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
>>>>> +
>>>>>  static int clk_smd_rpm_enable_scaling(void)
>>>>>  {
>>>>>  	int ret;
>>>>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>>>>>  		return ret;
>>>>>  	}
>>>>>  
>>>>> +	smd_rpm_clk_scaling = true;
>>>>> +
>>>>
>>>> If you move the platform_device_register_data(&rpdev->dev,
>>>> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
>>>> avoid the race completely and drop this API. I think that would be
>>>> cleaner. And it will likely probe much faster because probe deferral
>>>> is slow. :)
>>> Sounds like an idea.. especially since it's pretty much the only
>>> dependency other than SMDRPM itself!
>> It sounds great, but to not break bisecting one has to:
>>
>> 1. change the registration in soc/smd-rpm to store rpm ptr in driver
>>    data, in addition to parent driver data
>>
>> 2. change icc/smd-rpm to use the device and not parent data
>>
>> 3. add a platform_device_register_data call in clk-smd-rpm that will
>>    always fail because the device is always registered
>>
>> 4. remove the registration from soc/smd-rpm
>>
> 
> Logically the icc_smd_rpm device still fits better as child of
> smd-rpm and not clk-smd-rpm. So I would probably just continue
> registering it on the parent device from clk-smd-rpm.
> Then there are no changes necessary in icc_smd_rpm.
> 
> You could use this. Both touched files are Bjorn-maintained so should be
> manageable to have it in one commit. (note: compile-tested only)
> 
> Thanks,
> Stephan
> 
> From a2610adb2551b01e76b9de8e4cbcc89853814a8f Mon Sep 17 00:00:00 2001
> From: Stephan Gerhold <stephan@gerhold.net>
> Date: Sat, 10 Jun 2023 21:19:48 +0200
> Subject: [PATCH] soc: qcom: smd-rpm: Move icc_smd_rpm registration to
>  clk-smd-rpm
> 
> icc_smd_rpm will do bus clock votes itself rather than taking the
> unnecessary detour through the clock subsystem. However, it can only
> do that after the clocks have been handed off and scaling has been
> enabled in the RPM in clk-smd-rpm.
> 
> Move the icc_smd_rpm registration from smd-rpm.c to clk-smd-rpm.c
> to avoid any possible races. icc_smd_rpm gets the driver data from
> the smd-rpm device, so still register the platform device on the
> smd-rpm parent device.
> 
> Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> ---
Generally it looks good.. I'll give it a spin next week. One
thing below.

>  drivers/clk/qcom/clk-smd-rpm.c | 21 +++++++++++++++++++++
>  drivers/soc/qcom/smd-rpm.c     | 23 +----------------------
>  2 files changed, 22 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
> index e4de74b68797..91adb16889b3 100644
> --- a/drivers/clk/qcom/clk-smd-rpm.c
> +++ b/drivers/clk/qcom/clk-smd-rpm.c
> @@ -1302,12 +1302,20 @@ static struct clk_hw *qcom_smdrpm_clk_hw_get(struct of_phandle_args *clkspec,
>  	return desc->clks[idx] ? &desc->clks[idx]->hw : ERR_PTR(-ENOENT);
>  }
>  
> +static void rpm_smd_unregister_icc(void *data)
> +{
> +	struct platform_device *icc_pdev = data;
> +
> +	platform_device_unregister(icc_pdev);
> +}
> +
>  static int rpm_smd_clk_probe(struct platform_device *pdev)
>  {
>  	int ret;
>  	size_t num_clks, i;
>  	struct clk_smd_rpm **rpm_smd_clks;
>  	const struct rpm_smd_clk_desc *desc;
> +	struct platform_device *icc_pdev;
>  
>  	rpmcc_smd_rpm = dev_get_drvdata(pdev->dev.parent);
>  	if (!rpmcc_smd_rpm) {
> @@ -1357,6 +1365,19 @@ static int rpm_smd_clk_probe(struct platform_device *pdev)
>  	if (ret)
>  		goto err;
>  
> +	icc_pdev = platform_device_register_data(pdev->dev.parent,
> +						 "icc_smd_rpm", -1, NULL, 0);
> +	if (IS_ERR(icc_pdev)) {
> +		dev_err(&pdev->dev, "Failed to register icc_smd_rpm device: %pE\n",
> +			icc_pdev);
> +		/* No need to unregister clocks because of this */
> +	} else {
> +		ret = devm_add_action_or_reset(&pdev->dev, rpm_smd_unregister_icc,
> +					       icc_pdev);
> +		if (ret)
> +			goto err;
> +	}
> +
>  	return 0;
>  err:
>  	dev_err(&pdev->dev, "Error registering SMD clock driver (%d)\n", ret);
> diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
> index 0c1aa809cc4e..427dd5392b82 100644
> --- a/drivers/soc/qcom/smd-rpm.c
> +++ b/drivers/soc/qcom/smd-rpm.c
> @@ -19,7 +19,6 @@
>  /**
>   * struct qcom_smd_rpm - state of the rpm device driver
>   * @rpm_channel:	reference to the smd channel
> - * @icc:		interconnect proxy device
>   * @dev:		rpm device
>   * @ack:		completion for acks
>   * @lock:		mutual exclusion around the send/complete pair
> @@ -27,7 +26,6 @@
>   */
>  struct qcom_smd_rpm {
>  	struct rpmsg_endpoint *rpm_channel;
> -	struct platform_device *icc;
>  	struct device *dev;
>  
>  	struct completion ack;
> @@ -197,7 +195,6 @@ static int qcom_smd_rpm_callback(struct rpmsg_device *rpdev,
>  static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>  {
>  	struct qcom_smd_rpm *rpm;
> -	int ret;
>  
>  	rpm = devm_kzalloc(&rpdev->dev, sizeof(*rpm), GFP_KERNEL);
>  	if (!rpm)
> @@ -210,24 +207,7 @@ static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>  	rpm->rpm_channel = rpdev->ept;
>  	dev_set_drvdata(&rpdev->dev, rpm);
>  
> -	rpm->icc = platform_device_register_data(&rpdev->dev, "icc_smd_rpm", -1,
> -						 NULL, 0);
> -	if (IS_ERR(rpm->icc))
> -		return PTR_ERR(rpm->icc);
> -
> -	ret = of_platform_populate(rpdev->dev.of_node, NULL, NULL, &rpdev->dev);
> -	if (ret)
> -		platform_device_unregister(rpm->icc);
> -
> -	return ret;
> -}
> -
> -static void qcom_smd_rpm_remove(struct rpmsg_device *rpdev)
> -{
> -	struct qcom_smd_rpm *rpm = dev_get_drvdata(&rpdev->dev);
> -
> -	platform_device_unregister(rpm->icc);
> -	of_platform_depopulate(&rpdev->dev);
> +	return devm_of_platform_populate(&rpdev->dev);
>  }
>  
>  static const struct of_device_id qcom_smd_rpm_of_match[] = {
> @@ -256,7 +236,6 @@ MODULE_DEVICE_TABLE(of, qcom_smd_rpm_of_match);
>  
>  static struct rpmsg_driver qcom_smd_rpm_driver = {
>  	.probe = qcom_smd_rpm_probe,
> -	.remove = qcom_smd_rpm_remove,
This reaches over the removal of the icc registration, the depopulate
call should stay.

Konrad
>  	.callback = qcom_smd_rpm_callback,
>  	.drv  = {
>  		.name  = "qcom_smd_rpm",
  

On Sat, Jun 10, 2023 at 09:39:00PM +0200, Konrad Dybcio wrote:
> On 10.06.2023 21:25, Stephan Gerhold wrote:
> > On Sat, Jun 10, 2023 at 08:53:05PM +0200, Konrad Dybcio wrote:
> >> On 10.06.2023 14:15, Konrad Dybcio wrote:
> >>> On 10.06.2023 13:35, Stephan Gerhold wrote:
> >>>> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
> >>>>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
> >>>>> clock rate requests will not be commited in hardware. This poses a
> >>>>> race threat since we're accessing the bus clocks directly from within
> >>>>> the interconnect framework.
> >>>>>
> >>>>> Add a marker to indicate that we're good to go with sending new requests
> >>>>> and export it so that it can be referenced from icc.
> >>>>>
> >>>>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
> >>>>> ---
> >>>>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
> >>>>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
> >>>>>  2 files changed, 11 insertions(+)
> >>>>>
> >>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
> >>>>> index 937cb1515968..482fe30ee6f0 100644
> >>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
> >>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
> >>>>> @@ -151,6 +151,7 @@
> >>>>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
> >>>>>  
> >>>>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
> >>>>> +static bool smd_rpm_clk_scaling;
> >>>>>  
> >>>>>  struct clk_smd_rpm {
> >>>>>  	const int rpm_res_type;
> >>>>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
> >>>>>  	return r->rate;
> >>>>>  }
> >>>>>  
> >>>>> +bool qcom_smd_rpm_scaling_available(void)
> >>>>> +{
> >>>>> +	return smd_rpm_clk_scaling;
> >>>>> +}
> >>>>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
> >>>>> +
> >>>>>  static int clk_smd_rpm_enable_scaling(void)
> >>>>>  {
> >>>>>  	int ret;
> >>>>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
> >>>>>  		return ret;
> >>>>>  	}
> >>>>>  
> >>>>> +	smd_rpm_clk_scaling = true;
> >>>>> +
> >>>>
> >>>> If you move the platform_device_register_data(&rpdev->dev,
> >>>> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
> >>>> avoid the race completely and drop this API. I think that would be
> >>>> cleaner. And it will likely probe much faster because probe deferral
> >>>> is slow. :)
> >>> Sounds like an idea.. especially since it's pretty much the only
> >>> dependency other than SMDRPM itself!
> >> It sounds great, but to not break bisecting one has to:
> >>
> >> 1. change the registration in soc/smd-rpm to store rpm ptr in driver
> >>    data, in addition to parent driver data
> >>
> >> 2. change icc/smd-rpm to use the device and not parent data
> >>
> >> 3. add a platform_device_register_data call in clk-smd-rpm that will
> >>    always fail because the device is always registered
> >>
> >> 4. remove the registration from soc/smd-rpm
> >>
> > 
> > Logically the icc_smd_rpm device still fits better as child of
> > smd-rpm and not clk-smd-rpm. So I would probably just continue
> > registering it on the parent device from clk-smd-rpm.
> > Then there are no changes necessary in icc_smd_rpm.
> > 
> > You could use this. Both touched files are Bjorn-maintained so should be
> > manageable to have it in one commit. (note: compile-tested only)
> > 
> > Thanks,
> > Stephan
> > 
> > From a2610adb2551b01e76b9de8e4cbcc89853814a8f Mon Sep 17 00:00:00 2001
> > From: Stephan Gerhold <stephan@gerhold.net>
> > Date: Sat, 10 Jun 2023 21:19:48 +0200
> > Subject: [PATCH] soc: qcom: smd-rpm: Move icc_smd_rpm registration to
> >  clk-smd-rpm
> > 
> > icc_smd_rpm will do bus clock votes itself rather than taking the
> > unnecessary detour through the clock subsystem. However, it can only
> > do that after the clocks have been handed off and scaling has been
> > enabled in the RPM in clk-smd-rpm.
> > 
> > Move the icc_smd_rpm registration from smd-rpm.c to clk-smd-rpm.c
> > to avoid any possible races. icc_smd_rpm gets the driver data from
> > the smd-rpm device, so still register the platform device on the
> > smd-rpm parent device.
> > 
> > Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> > ---
> Generally it looks good.. I'll give it a spin next week. One
> thing below.
> 
> >  drivers/clk/qcom/clk-smd-rpm.c | 21 +++++++++++++++++++++
> >  drivers/soc/qcom/smd-rpm.c     | 23 +----------------------
> >  2 files changed, 22 insertions(+), 22 deletions(-)
> > 
> > diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
> > index e4de74b68797..91adb16889b3 100644
> > --- a/drivers/clk/qcom/clk-smd-rpm.c
> > +++ b/drivers/clk/qcom/clk-smd-rpm.c
> > @@ -1302,12 +1302,20 @@ static struct clk_hw *qcom_smdrpm_clk_hw_get(struct of_phandle_args *clkspec,
> >  	return desc->clks[idx] ? &desc->clks[idx]->hw : ERR_PTR(-ENOENT);
> >  }
> >  
> > +static void rpm_smd_unregister_icc(void *data)
> > +{
> > +	struct platform_device *icc_pdev = data;
> > +
> > +	platform_device_unregister(icc_pdev);
> > +}
> > +
> >  static int rpm_smd_clk_probe(struct platform_device *pdev)
> >  {
> >  	int ret;
> >  	size_t num_clks, i;
> >  	struct clk_smd_rpm **rpm_smd_clks;
> >  	const struct rpm_smd_clk_desc *desc;
> > +	struct platform_device *icc_pdev;
> >  
> >  	rpmcc_smd_rpm = dev_get_drvdata(pdev->dev.parent);
> >  	if (!rpmcc_smd_rpm) {
> > @@ -1357,6 +1365,19 @@ static int rpm_smd_clk_probe(struct platform_device *pdev)
> >  	if (ret)
> >  		goto err;
> >  
> > +	icc_pdev = platform_device_register_data(pdev->dev.parent,
> > +						 "icc_smd_rpm", -1, NULL, 0);
> > +	if (IS_ERR(icc_pdev)) {
> > +		dev_err(&pdev->dev, "Failed to register icc_smd_rpm device: %pE\n",
> > +			icc_pdev);
> > +		/* No need to unregister clocks because of this */
> > +	} else {
> > +		ret = devm_add_action_or_reset(&pdev->dev, rpm_smd_unregister_icc,
> > +					       icc_pdev);
> > +		if (ret)
> > +			goto err;
> > +	}
> > +
> >  	return 0;
> >  err:
> >  	dev_err(&pdev->dev, "Error registering SMD clock driver (%d)\n", ret);
> > diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
> > index 0c1aa809cc4e..427dd5392b82 100644
> > --- a/drivers/soc/qcom/smd-rpm.c
> > +++ b/drivers/soc/qcom/smd-rpm.c
> > @@ -19,7 +19,6 @@
> >  /**
> >   * struct qcom_smd_rpm - state of the rpm device driver
> >   * @rpm_channel:	reference to the smd channel
> > - * @icc:		interconnect proxy device
> >   * @dev:		rpm device
> >   * @ack:		completion for acks
> >   * @lock:		mutual exclusion around the send/complete pair
> > @@ -27,7 +26,6 @@
> >   */
> >  struct qcom_smd_rpm {
> >  	struct rpmsg_endpoint *rpm_channel;
> > -	struct platform_device *icc;
> >  	struct device *dev;
> >  
> >  	struct completion ack;
> > @@ -197,7 +195,6 @@ static int qcom_smd_rpm_callback(struct rpmsg_device *rpdev,
> >  static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
> >  {
> >  	struct qcom_smd_rpm *rpm;
> > -	int ret;
> >  
> >  	rpm = devm_kzalloc(&rpdev->dev, sizeof(*rpm), GFP_KERNEL);
> >  	if (!rpm)
> > @@ -210,24 +207,7 @@ static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
> >  	rpm->rpm_channel = rpdev->ept;
> >  	dev_set_drvdata(&rpdev->dev, rpm);
> >  
> > -	rpm->icc = platform_device_register_data(&rpdev->dev, "icc_smd_rpm", -1,
> > -						 NULL, 0);
> > -	if (IS_ERR(rpm->icc))
> > -		return PTR_ERR(rpm->icc);
> > -
> > -	ret = of_platform_populate(rpdev->dev.of_node, NULL, NULL, &rpdev->dev);
> > -	if (ret)
> > -		platform_device_unregister(rpm->icc);
> > -
> > -	return ret;
> > -}
> > -
> > -static void qcom_smd_rpm_remove(struct rpmsg_device *rpdev)
> > -{
> > -	struct qcom_smd_rpm *rpm = dev_get_drvdata(&rpdev->dev);
> > -
> > -	platform_device_unregister(rpm->icc);
> > -	of_platform_depopulate(&rpdev->dev);
> > +	return devm_of_platform_populate(&rpdev->dev);
> >  }
> >  
> >  static const struct of_device_id qcom_smd_rpm_of_match[] = {
> > @@ -256,7 +236,6 @@ MODULE_DEVICE_TABLE(of, qcom_smd_rpm_of_match);
> >  
> >  static struct rpmsg_driver qcom_smd_rpm_driver = {
> >  	.probe = qcom_smd_rpm_probe,
> > -	.remove = qcom_smd_rpm_remove,
> This reaches over the removal of the icc registration, the depopulate
> call should stay.
> 

I switched the of_platform_populate() to devm_of_platform_populate(),
that's why the remove callback is no longer necessary. It's a bit
hidden, perhaps it would be enough to add to the commit message:

"While at it, switch the remaining of_platform_populate() call to the
 devm variant and remove the remove callback."

Or maybe it should be split into two patches.

Thanks,
Stephan
  

On 11.06.2023 11:20, Stephan Gerhold wrote:
> On Sat, Jun 10, 2023 at 09:39:00PM +0200, Konrad Dybcio wrote:
>> On 10.06.2023 21:25, Stephan Gerhold wrote:
>>> On Sat, Jun 10, 2023 at 08:53:05PM +0200, Konrad Dybcio wrote:
>>>> On 10.06.2023 14:15, Konrad Dybcio wrote:
>>>>> On 10.06.2023 13:35, Stephan Gerhold wrote:
>>>>>> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
>>>>>>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
>>>>>>> clock rate requests will not be commited in hardware. This poses a
>>>>>>> race threat since we're accessing the bus clocks directly from within
>>>>>>> the interconnect framework.
>>>>>>>
>>>>>>> Add a marker to indicate that we're good to go with sending new requests
>>>>>>> and export it so that it can be referenced from icc.
>>>>>>>
>>>>>>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
>>>>>>> ---
>>>>>>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>>>>>>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>>>>>>>  2 files changed, 11 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>> index 937cb1515968..482fe30ee6f0 100644
>>>>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>> @@ -151,6 +151,7 @@
>>>>>>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>>>>>>>  
>>>>>>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
>>>>>>> +static bool smd_rpm_clk_scaling;
>>>>>>>  
>>>>>>>  struct clk_smd_rpm {
>>>>>>>  	const int rpm_res_type;
>>>>>>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>>>>>>>  	return r->rate;
>>>>>>>  }
>>>>>>>  
>>>>>>> +bool qcom_smd_rpm_scaling_available(void)
>>>>>>> +{
>>>>>>> +	return smd_rpm_clk_scaling;
>>>>>>> +}
>>>>>>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
>>>>>>> +
>>>>>>>  static int clk_smd_rpm_enable_scaling(void)
>>>>>>>  {
>>>>>>>  	int ret;
>>>>>>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>>>>>>>  		return ret;
>>>>>>>  	}
>>>>>>>  
>>>>>>> +	smd_rpm_clk_scaling = true;
>>>>>>> +
>>>>>>
>>>>>> If you move the platform_device_register_data(&rpdev->dev,
>>>>>> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
>>>>>> avoid the race completely and drop this API. I think that would be
>>>>>> cleaner. And it will likely probe much faster because probe deferral
>>>>>> is slow. :)
>>>>> Sounds like an idea.. especially since it's pretty much the only
>>>>> dependency other than SMDRPM itself!
>>>> It sounds great, but to not break bisecting one has to:
>>>>
>>>> 1. change the registration in soc/smd-rpm to store rpm ptr in driver
>>>>    data, in addition to parent driver data
>>>>
>>>> 2. change icc/smd-rpm to use the device and not parent data
>>>>
>>>> 3. add a platform_device_register_data call in clk-smd-rpm that will
>>>>    always fail because the device is always registered
>>>>
>>>> 4. remove the registration from soc/smd-rpm
>>>>
>>>
>>> Logically the icc_smd_rpm device still fits better as child of
>>> smd-rpm and not clk-smd-rpm. So I would probably just continue
>>> registering it on the parent device from clk-smd-rpm.
>>> Then there are no changes necessary in icc_smd_rpm.
>>>
>>> You could use this. Both touched files are Bjorn-maintained so should be
>>> manageable to have it in one commit. (note: compile-tested only)
>>>
>>> Thanks,
>>> Stephan
>>>
>>> From a2610adb2551b01e76b9de8e4cbcc89853814a8f Mon Sep 17 00:00:00 2001
>>> From: Stephan Gerhold <stephan@gerhold.net>
>>> Date: Sat, 10 Jun 2023 21:19:48 +0200
>>> Subject: [PATCH] soc: qcom: smd-rpm: Move icc_smd_rpm registration to
>>>  clk-smd-rpm
>>>
>>> icc_smd_rpm will do bus clock votes itself rather than taking the
>>> unnecessary detour through the clock subsystem. However, it can only
>>> do that after the clocks have been handed off and scaling has been
>>> enabled in the RPM in clk-smd-rpm.
>>>
>>> Move the icc_smd_rpm registration from smd-rpm.c to clk-smd-rpm.c
>>> to avoid any possible races. icc_smd_rpm gets the driver data from
>>> the smd-rpm device, so still register the platform device on the
>>> smd-rpm parent device.
>>>
>>> Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
>>> ---
>> Generally it looks good.. I'll give it a spin next week. One
>> thing below.
>>
>>>  drivers/clk/qcom/clk-smd-rpm.c | 21 +++++++++++++++++++++
>>>  drivers/soc/qcom/smd-rpm.c     | 23 +----------------------
>>>  2 files changed, 22 insertions(+), 22 deletions(-)
>>>
>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>> index e4de74b68797..91adb16889b3 100644
>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>> @@ -1302,12 +1302,20 @@ static struct clk_hw *qcom_smdrpm_clk_hw_get(struct of_phandle_args *clkspec,
>>>  	return desc->clks[idx] ? &desc->clks[idx]->hw : ERR_PTR(-ENOENT);
>>>  }
>>>  
>>> +static void rpm_smd_unregister_icc(void *data)
>>> +{
>>> +	struct platform_device *icc_pdev = data;
>>> +
>>> +	platform_device_unregister(icc_pdev);
>>> +}
>>> +
>>>  static int rpm_smd_clk_probe(struct platform_device *pdev)
>>>  {
>>>  	int ret;
>>>  	size_t num_clks, i;
>>>  	struct clk_smd_rpm **rpm_smd_clks;
>>>  	const struct rpm_smd_clk_desc *desc;
>>> +	struct platform_device *icc_pdev;
>>>  
>>>  	rpmcc_smd_rpm = dev_get_drvdata(pdev->dev.parent);
>>>  	if (!rpmcc_smd_rpm) {
>>> @@ -1357,6 +1365,19 @@ static int rpm_smd_clk_probe(struct platform_device *pdev)
>>>  	if (ret)
>>>  		goto err;
>>>  
>>> +	icc_pdev = platform_device_register_data(pdev->dev.parent,
>>> +						 "icc_smd_rpm", -1, NULL, 0);
>>> +	if (IS_ERR(icc_pdev)) {
>>> +		dev_err(&pdev->dev, "Failed to register icc_smd_rpm device: %pE\n",
>>> +			icc_pdev);
>>> +		/* No need to unregister clocks because of this */
>>> +	} else {
>>> +		ret = devm_add_action_or_reset(&pdev->dev, rpm_smd_unregister_icc,
>>> +					       icc_pdev);
>>> +		if (ret)
>>> +			goto err;
>>> +	}
>>> +
>>>  	return 0;
>>>  err:
>>>  	dev_err(&pdev->dev, "Error registering SMD clock driver (%d)\n", ret);
>>> diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
>>> index 0c1aa809cc4e..427dd5392b82 100644
>>> --- a/drivers/soc/qcom/smd-rpm.c
>>> +++ b/drivers/soc/qcom/smd-rpm.c
>>> @@ -19,7 +19,6 @@
>>>  /**
>>>   * struct qcom_smd_rpm - state of the rpm device driver
>>>   * @rpm_channel:	reference to the smd channel
>>> - * @icc:		interconnect proxy device
>>>   * @dev:		rpm device
>>>   * @ack:		completion for acks
>>>   * @lock:		mutual exclusion around the send/complete pair
>>> @@ -27,7 +26,6 @@
>>>   */
>>>  struct qcom_smd_rpm {
>>>  	struct rpmsg_endpoint *rpm_channel;
>>> -	struct platform_device *icc;
>>>  	struct device *dev;
>>>  
>>>  	struct completion ack;
>>> @@ -197,7 +195,6 @@ static int qcom_smd_rpm_callback(struct rpmsg_device *rpdev,
>>>  static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>>>  {
>>>  	struct qcom_smd_rpm *rpm;
>>> -	int ret;
>>>  
>>>  	rpm = devm_kzalloc(&rpdev->dev, sizeof(*rpm), GFP_KERNEL);
>>>  	if (!rpm)
>>> @@ -210,24 +207,7 @@ static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>>>  	rpm->rpm_channel = rpdev->ept;
>>>  	dev_set_drvdata(&rpdev->dev, rpm);
>>>  
>>> -	rpm->icc = platform_device_register_data(&rpdev->dev, "icc_smd_rpm", -1,
>>> -						 NULL, 0);
>>> -	if (IS_ERR(rpm->icc))
>>> -		return PTR_ERR(rpm->icc);
>>> -
>>> -	ret = of_platform_populate(rpdev->dev.of_node, NULL, NULL, &rpdev->dev);
>>> -	if (ret)
>>> -		platform_device_unregister(rpm->icc);
>>> -
>>> -	return ret;
>>> -}
>>> -
>>> -static void qcom_smd_rpm_remove(struct rpmsg_device *rpdev)
>>> -{
>>> -	struct qcom_smd_rpm *rpm = dev_get_drvdata(&rpdev->dev);
>>> -
>>> -	platform_device_unregister(rpm->icc);
>>> -	of_platform_depopulate(&rpdev->dev);
>>> +	return devm_of_platform_populate(&rpdev->dev);
>>>  }
>>>  
>>>  static const struct of_device_id qcom_smd_rpm_of_match[] = {
>>> @@ -256,7 +236,6 @@ MODULE_DEVICE_TABLE(of, qcom_smd_rpm_of_match);
>>>  
>>>  static struct rpmsg_driver qcom_smd_rpm_driver = {
>>>  	.probe = qcom_smd_rpm_probe,
>>> -	.remove = qcom_smd_rpm_remove,
>> This reaches over the removal of the icc registration, the depopulate
>> call should stay.
>>
> 
> I switched the of_platform_populate() to devm_of_platform_populate(),
> that's why the remove callback is no longer necessary. It's a bit
> hidden, perhaps it would be enough to add to the commit message:
> 
> "While at it, switch the remaining of_platform_populate() call to the
>  devm variant and remove the remove callback."
> 
> Or maybe it should be split into two patches.
Gave it a spin, I think it ends up being worse if an IPA rpm clock is
consumed by one of the icc providers, and that's sadly the case
for almost all platforms (or supposed to be).. :/ Only qcm2290 doesn't
seem to care if we poke the network interface units with half the soc
off :P

Konrad
> 
> Thanks,
> Stephan
  

On 12.06.2023 14:51, Konrad Dybcio wrote:
> On 11.06.2023 11:20, Stephan Gerhold wrote:
>> On Sat, Jun 10, 2023 at 09:39:00PM +0200, Konrad Dybcio wrote:
>>> On 10.06.2023 21:25, Stephan Gerhold wrote:
>>>> On Sat, Jun 10, 2023 at 08:53:05PM +0200, Konrad Dybcio wrote:
>>>>> On 10.06.2023 14:15, Konrad Dybcio wrote:
>>>>>> On 10.06.2023 13:35, Stephan Gerhold wrote:
>>>>>>> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
>>>>>>>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
>>>>>>>> clock rate requests will not be commited in hardware. This poses a
>>>>>>>> race threat since we're accessing the bus clocks directly from within
>>>>>>>> the interconnect framework.
>>>>>>>>
>>>>>>>> Add a marker to indicate that we're good to go with sending new requests
>>>>>>>> and export it so that it can be referenced from icc.
>>>>>>>>
>>>>>>>> Signed-off-by: Konrad Dybcio <konrad.dybcio@linaro.org>
>>>>>>>> ---
>>>>>>>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>>>>>>>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>>>>>>>>  2 files changed, 11 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>>> index 937cb1515968..482fe30ee6f0 100644
>>>>>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>>> @@ -151,6 +151,7 @@
>>>>>>>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>>>>>>>>  
>>>>>>>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
>>>>>>>> +static bool smd_rpm_clk_scaling;
>>>>>>>>  
>>>>>>>>  struct clk_smd_rpm {
>>>>>>>>  	const int rpm_res_type;
>>>>>>>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>>>>>>>>  	return r->rate;
>>>>>>>>  }
>>>>>>>>  
>>>>>>>> +bool qcom_smd_rpm_scaling_available(void)
>>>>>>>> +{
>>>>>>>> +	return smd_rpm_clk_scaling;
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
>>>>>>>> +
>>>>>>>>  static int clk_smd_rpm_enable_scaling(void)
>>>>>>>>  {
>>>>>>>>  	int ret;
>>>>>>>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>>>>>>>>  		return ret;
>>>>>>>>  	}
>>>>>>>>  
>>>>>>>> +	smd_rpm_clk_scaling = true;
>>>>>>>> +
>>>>>>>
>>>>>>> If you move the platform_device_register_data(&rpdev->dev,
>>>>>>> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
>>>>>>> avoid the race completely and drop this API. I think that would be
>>>>>>> cleaner. And it will likely probe much faster because probe deferral
>>>>>>> is slow. :)
>>>>>> Sounds like an idea.. especially since it's pretty much the only
>>>>>> dependency other than SMDRPM itself!
>>>>> It sounds great, but to not break bisecting one has to:
>>>>>
>>>>> 1. change the registration in soc/smd-rpm to store rpm ptr in driver
>>>>>    data, in addition to parent driver data
>>>>>
>>>>> 2. change icc/smd-rpm to use the device and not parent data
>>>>>
>>>>> 3. add a platform_device_register_data call in clk-smd-rpm that will
>>>>>    always fail because the device is always registered
>>>>>
>>>>> 4. remove the registration from soc/smd-rpm
>>>>>
>>>>
>>>> Logically the icc_smd_rpm device still fits better as child of
>>>> smd-rpm and not clk-smd-rpm. So I would probably just continue
>>>> registering it on the parent device from clk-smd-rpm.
>>>> Then there are no changes necessary in icc_smd_rpm.
>>>>
>>>> You could use this. Both touched files are Bjorn-maintained so should be
>>>> manageable to have it in one commit. (note: compile-tested only)
>>>>
>>>> Thanks,
>>>> Stephan
>>>>
>>>> From a2610adb2551b01e76b9de8e4cbcc89853814a8f Mon Sep 17 00:00:00 2001
>>>> From: Stephan Gerhold <stephan@gerhold.net>
>>>> Date: Sat, 10 Jun 2023 21:19:48 +0200
>>>> Subject: [PATCH] soc: qcom: smd-rpm: Move icc_smd_rpm registration to
>>>>  clk-smd-rpm
>>>>
>>>> icc_smd_rpm will do bus clock votes itself rather than taking the
>>>> unnecessary detour through the clock subsystem. However, it can only
>>>> do that after the clocks have been handed off and scaling has been
>>>> enabled in the RPM in clk-smd-rpm.
>>>>
>>>> Move the icc_smd_rpm registration from smd-rpm.c to clk-smd-rpm.c
>>>> to avoid any possible races. icc_smd_rpm gets the driver data from
>>>> the smd-rpm device, so still register the platform device on the
>>>> smd-rpm parent device.
>>>>
>>>> Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
>>>> ---
>>> Generally it looks good.. I'll give it a spin next week. One
>>> thing below.
>>>
>>>>  drivers/clk/qcom/clk-smd-rpm.c | 21 +++++++++++++++++++++
>>>>  drivers/soc/qcom/smd-rpm.c     | 23 +----------------------
>>>>  2 files changed, 22 insertions(+), 22 deletions(-)
>>>>
>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>>> index e4de74b68797..91adb16889b3 100644
>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>>> @@ -1302,12 +1302,20 @@ static struct clk_hw *qcom_smdrpm_clk_hw_get(struct of_phandle_args *clkspec,
>>>>  	return desc->clks[idx] ? &desc->clks[idx]->hw : ERR_PTR(-ENOENT);
>>>>  }
>>>>  
>>>> +static void rpm_smd_unregister_icc(void *data)
>>>> +{
>>>> +	struct platform_device *icc_pdev = data;
>>>> +
>>>> +	platform_device_unregister(icc_pdev);
>>>> +}
>>>> +
>>>>  static int rpm_smd_clk_probe(struct platform_device *pdev)
>>>>  {
>>>>  	int ret;
>>>>  	size_t num_clks, i;
>>>>  	struct clk_smd_rpm **rpm_smd_clks;
>>>>  	const struct rpm_smd_clk_desc *desc;
>>>> +	struct platform_device *icc_pdev;
>>>>  
>>>>  	rpmcc_smd_rpm = dev_get_drvdata(pdev->dev.parent);
>>>>  	if (!rpmcc_smd_rpm) {
>>>> @@ -1357,6 +1365,19 @@ static int rpm_smd_clk_probe(struct platform_device *pdev)
>>>>  	if (ret)
>>>>  		goto err;
>>>>  
>>>> +	icc_pdev = platform_device_register_data(pdev->dev.parent,
>>>> +						 "icc_smd_rpm", -1, NULL, 0);
>>>> +	if (IS_ERR(icc_pdev)) {
>>>> +		dev_err(&pdev->dev, "Failed to register icc_smd_rpm device: %pE\n",
>>>> +			icc_pdev);
>>>> +		/* No need to unregister clocks because of this */
>>>> +	} else {
>>>> +		ret = devm_add_action_or_reset(&pdev->dev, rpm_smd_unregister_icc,
>>>> +					       icc_pdev);
>>>> +		if (ret)
>>>> +			goto err;
>>>> +	}
>>>> +
>>>>  	return 0;
>>>>  err:
>>>>  	dev_err(&pdev->dev, "Error registering SMD clock driver (%d)\n", ret);
>>>> diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
>>>> index 0c1aa809cc4e..427dd5392b82 100644
>>>> --- a/drivers/soc/qcom/smd-rpm.c
>>>> +++ b/drivers/soc/qcom/smd-rpm.c
>>>> @@ -19,7 +19,6 @@
>>>>  /**
>>>>   * struct qcom_smd_rpm - state of the rpm device driver
>>>>   * @rpm_channel:	reference to the smd channel
>>>> - * @icc:		interconnect proxy device
>>>>   * @dev:		rpm device
>>>>   * @ack:		completion for acks
>>>>   * @lock:		mutual exclusion around the send/complete pair
>>>> @@ -27,7 +26,6 @@
>>>>   */
>>>>  struct qcom_smd_rpm {
>>>>  	struct rpmsg_endpoint *rpm_channel;
>>>> -	struct platform_device *icc;
>>>>  	struct device *dev;
>>>>  
>>>>  	struct completion ack;
>>>> @@ -197,7 +195,6 @@ static int qcom_smd_rpm_callback(struct rpmsg_device *rpdev,
>>>>  static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>>>>  {
>>>>  	struct qcom_smd_rpm *rpm;
>>>> -	int ret;
>>>>  
>>>>  	rpm = devm_kzalloc(&rpdev->dev, sizeof(*rpm), GFP_KERNEL);
>>>>  	if (!rpm)
>>>> @@ -210,24 +207,7 @@ static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>>>>  	rpm->rpm_channel = rpdev->ept;
>>>>  	dev_set_drvdata(&rpdev->dev, rpm);
>>>>  
>>>> -	rpm->icc = platform_device_register_data(&rpdev->dev, "icc_smd_rpm", -1,
>>>> -						 NULL, 0);
>>>> -	if (IS_ERR(rpm->icc))
>>>> -		return PTR_ERR(rpm->icc);
>>>> -
>>>> -	ret = of_platform_populate(rpdev->dev.of_node, NULL, NULL, &rpdev->dev);
>>>> -	if (ret)
>>>> -		platform_device_unregister(rpm->icc);
>>>> -
>>>> -	return ret;
>>>> -}
>>>> -
>>>> -static void qcom_smd_rpm_remove(struct rpmsg_device *rpdev)
>>>> -{
>>>> -	struct qcom_smd_rpm *rpm = dev_get_drvdata(&rpdev->dev);
>>>> -
>>>> -	platform_device_unregister(rpm->icc);
>>>> -	of_platform_depopulate(&rpdev->dev);
>>>> +	return devm_of_platform_populate(&rpdev->dev);
>>>>  }
>>>>  
>>>>  static const struct of_device_id qcom_smd_rpm_of_match[] = {
>>>> @@ -256,7 +236,6 @@ MODULE_DEVICE_TABLE(of, qcom_smd_rpm_of_match);
>>>>  
>>>>  static struct rpmsg_driver qcom_smd_rpm_driver = {
>>>>  	.probe = qcom_smd_rpm_probe,
>>>> -	.remove = qcom_smd_rpm_remove,
>>> This reaches over the removal of the icc registration, the depopulate
>>> call should stay.
>>>
>>
>> I switched the of_platform_populate() to devm_of_platform_populate(),
>> that's why the remove callback is no longer necessary. It's a bit
>> hidden, perhaps it would be enough to add to the commit message:
>>
>> "While at it, switch the remaining of_platform_populate() call to the
>>  devm variant and remove the remove callback."
>>
>> Or maybe it should be split into two patches.
> Gave it a spin, I think it ends up being worse if an IPA rpm clock is
> consumed by one of the icc providers, and that's sadly the case
> for almost all platforms (or supposed to be).. :/ Only qcm2290 doesn't
> seem to care if we poke the network interface units with half the soc
> off :P
Actually it's more complex and that rings the same tons-of-probe-deferrals
bell that we have to get rid of.. The icc_smd_rpm driver actually does
probe faster with your patch, but that's not the case with the SoC icc
provider driver which does probe later :/

Don't think either solution is perfect, but I'm willing to take your
solution, as the one I proposed is a bit of a dead end for future
improvements.

Konrad
> 
> Konrad
>>
>> Thanks,
>> Stephan