Revert "regulator: qcom-rpmh: Revert "regulator: qcom-rpmh: Use PROBE_FORCE_SYNCHRONOUS""

  This reverts commit ad44ac082fdff7ee57fe125432f7d9d7cb610a23.

This patch restores the synchronous probing for
qcom-rpmh-regulator because asynchronous probing broke
Dragonboard 845c (SDM845) running AOSP. UFSHC fail to
initialize properly and DB845c fails to boot regardless
of "rootwait" bootarg being present or not
https://bugs.linaro.org/show_bug.cgi?id=5975.

Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
---
 drivers/regulator/qcom-rpmh-regulator.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
  

Hi,

On Mon, May 15, 2023 at 7:53 AM Amit Pundir <amit.pundir@linaro.org> wrote:
>
> This reverts commit ad44ac082fdff7ee57fe125432f7d9d7cb610a23.
>
> This patch restores the synchronous probing for
> qcom-rpmh-regulator because asynchronous probing broke
> Dragonboard 845c (SDM845) running AOSP. UFSHC fail to
> initialize properly and DB845c fails to boot regardless
> of "rootwait" bootarg being present or not
> https://bugs.linaro.org/show_bug.cgi?id=5975.
>
> Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
> ---
>  drivers/regulator/qcom-rpmh-regulator.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

As mentioned in the other thread [1], I'd rather at least do a little
debugging before jumping to a revert, just so we don't end up having
to do a revert of a revert for the same driver again.

[1] https://lore.kernel.org/r/CAD=FV=VSFDe445WEVTHXxU1WS_HGUV5jR5E8_Vgd4eyhn3rHyA@mail.gmail.com
  

On Mon, May 15, 2023 at 08:23:23PM +0530, Amit Pundir wrote:
> This reverts commit ad44ac082fdff7ee57fe125432f7d9d7cb610a23.
> 
> This patch restores the synchronous probing for
> qcom-rpmh-regulator because asynchronous probing broke
> Dragonboard 845c (SDM845) running AOSP. UFSHC fail to
> initialize properly and DB845c fails to boot regardless
> of "rootwait" bootarg being present or not
> https://bugs.linaro.org/show_bug.cgi?id=5975.
> 

Looking at the first attachment, I would suggest that UFS fails because
it's not able to get hold of its regulators, just as any other device
with supplies would.

The typical cause for rpmh timeouts is that you're no longer able to
talk to the rpmh. Could you please attempt to trace the system and see
what's happening in parallel that would cause such issue.

Also note that such issues often also results in UFS timeouts, which
results in the familiar UFS debug dumps.


In the second log, the system crashes 51 seconds after rpmh probes,
around the time where other sync_state is happening. This too would seem
related to missing resource votes, but I would expect being a separate
issue.

PS. this is a patch in the regulator code, but I don't see Mark in the
recipients list...

Regards,
Bjorn

> Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
> ---
>  drivers/regulator/qcom-rpmh-regulator.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/regulator/qcom-rpmh-regulator.c b/drivers/regulator/qcom-rpmh-regulator.c
> index b0a58c62b1e2..30659922b0aa 100644
> --- a/drivers/regulator/qcom-rpmh-regulator.c
> +++ b/drivers/regulator/qcom-rpmh-regulator.c
> @@ -1517,7 +1517,7 @@ MODULE_DEVICE_TABLE(of, rpmh_regulator_match_table);
>  static struct platform_driver rpmh_regulator_driver = {
>  	.driver = {
>  		.name = "qcom-rpmh-regulator",
> -		.probe_type = PROBE_PREFER_ASYNCHRONOUS,
> +		.probe_type = PROBE_FORCE_SYNCHRONOUS,
>  		.of_match_table	= of_match_ptr(rpmh_regulator_match_table),
>  	},
>  	.probe = rpmh_regulator_probe,
> -- 
> 2.25.1
>
  

+ Mark.

On Mon, 15 May 2023 at 21:26, Bjorn Andersson <andersson@kernel.org> wrote:
>
> On Mon, May 15, 2023 at 08:23:23PM +0530, Amit Pundir wrote:
> > This reverts commit ad44ac082fdff7ee57fe125432f7d9d7cb610a23.
> >
> > This patch restores the synchronous probing for
> > qcom-rpmh-regulator because asynchronous probing broke
> > Dragonboard 845c (SDM845) running AOSP. UFSHC fail to
> > initialize properly and DB845c fails to boot regardless
> > of "rootwait" bootarg being present or not
> > https://bugs.linaro.org/show_bug.cgi?id=5975.
> >
>
> Looking at the first attachment, I would suggest that UFS fails because
> it's not able to get hold of its regulators, just as any other device
> with supplies would.
>
> The typical cause for rpmh timeouts is that you're no longer able to
> talk to the rpmh. Could you please attempt to trace the system and see
> what's happening in parallel that would cause such issue.

This is happening very early in the boot process (first stag init),
wherein init is supposed to load the kernel modules from ramdisk and
mount the partitions.

System does boot successfully 2/5 times, so this is definitely a
timing issue as Douglas also pointed out in the other thread. I'll try
to follow his suggestions and narrow down the issue further.

>
> Also note that such issues often also results in UFS timeouts, which
> results in the familiar UFS debug dumps.
>
>
> In the second log, the system crashes 51 seconds after rpmh probes,
> around the time where other sync_state is happening. This too would seem
> related to missing resource votes, but I would expect being a separate
> issue.
>
> PS. this is a patch in the regulator code, but I don't see Mark in the
> recipients list...

Sorry about that. I thought I had Mark copied over from the other
thread, after-all he was the one who suggested this patch, but I
didn't :facepalm:

Regards,
Amit Pundir


>
> Regards,
> Bjorn
>
> > Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
> > ---
> >  drivers/regulator/qcom-rpmh-regulator.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/regulator/qcom-rpmh-regulator.c b/drivers/regulator/qcom-rpmh-regulator.c
> > index b0a58c62b1e2..30659922b0aa 100644
> > --- a/drivers/regulator/qcom-rpmh-regulator.c
> > +++ b/drivers/regulator/qcom-rpmh-regulator.c
> > @@ -1517,7 +1517,7 @@ MODULE_DEVICE_TABLE(of, rpmh_regulator_match_table);
> >  static struct platform_driver rpmh_regulator_driver = {
> >       .driver = {
> >               .name = "qcom-rpmh-regulator",
> > -             .probe_type = PROBE_PREFER_ASYNCHRONOUS,
> > +             .probe_type = PROBE_FORCE_SYNCHRONOUS,
> >               .of_match_table = of_match_ptr(rpmh_regulator_match_table),
> >       },
> >       .probe = rpmh_regulator_probe,
> > --
> > 2.25.1
> >
  

On Mon, May 15, 2023 at 11:01:31PM +0530, Amit Pundir wrote:
> On Mon, 15 May 2023 at 21:26, Bjorn Andersson <andersson@kernel.org> wrote:

> > PS. this is a patch in the regulator code, but I don't see Mark in the
> > recipients list...

> Sorry about that. I thought I had Mark copied over from the other
> thread, after-all he was the one who suggested this patch, but I
> didn't :facepalm:

Please resend a copy of the patch to me and...

Please submit patches using subject lines reflecting the style for the
subsystem, this makes it easier for people to identify relevant patches.
Look at what existing commits in the area you're changing are doing and
make sure your subject lines visually resemble what they're doing.
There's no need to resubmit to fix this alone.
  

[CCing the regression list, as it should be in the loop for regressions:
https://docs.kernel.org/admin-guide/reporting-regressions.html]

On 15.05.23 16:53, Amit Pundir wrote:
> This reverts commit ad44ac082fdff7ee57fe125432f7d9d7cb610a23.
> 
> This patch restores the synchronous probing for
> qcom-rpmh-regulator because asynchronous probing broke
> Dragonboard 845c (SDM845) running AOSP. UFSHC fail to
> initialize properly and DB845c fails to boot regardless
> of "rootwait" bootarg being present or not
> https://bugs.linaro.org/show_bug.cgi?id=5975.
> 
> Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
> ---
>  drivers/regulator/qcom-rpmh-regulator.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/regulator/qcom-rpmh-regulator.c b/drivers/regulator/qcom-rpmh-regulator.c
> index b0a58c62b1e2..30659922b0aa 100644
> --- a/drivers/regulator/qcom-rpmh-regulator.c
> +++ b/drivers/regulator/qcom-rpmh-regulator.c
> @@ -1517,7 +1517,7 @@ MODULE_DEVICE_TABLE(of, rpmh_regulator_match_table);
>  static struct platform_driver rpmh_regulator_driver = {
>  	.driver = {
>  		.name = "qcom-rpmh-regulator",
> -		.probe_type = PROBE_PREFER_ASYNCHRONOUS,
> +		.probe_type = PROBE_FORCE_SYNCHRONOUS,
>  		.of_match_table	= of_match_ptr(rpmh_regulator_match_table),
>  	},
>  	.probe = rpmh_regulator_probe,

Amit, just wondering: what happened to this? It seems there was some
agreement to go down this route to fix your regression, but then nothing
happened anymore since about a week. Or am I missing something?

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.

#regzbot ^backmonitor:
https://lore.kernel.org/lkml/CAMi1Hd1avQDcDQf137m2auz2znov4XL8YGrLZsw5edb-NtRJRw@mail.gmail.com/
  

On Wed, 24 May 2023 at 19:21, Linux regression tracking (Thorsten
Leemhuis) <regressions@leemhuis.info> wrote:
>
> [CCing the regression list, as it should be in the loop for regressions:
> https://docs.kernel.org/admin-guide/reporting-regressions.html]
>
> On 15.05.23 16:53, Amit Pundir wrote:
> > This reverts commit ad44ac082fdff7ee57fe125432f7d9d7cb610a23.
> >
> > This patch restores the synchronous probing for
> > qcom-rpmh-regulator because asynchronous probing broke
> > Dragonboard 845c (SDM845) running AOSP. UFSHC fail to
> > initialize properly and DB845c fails to boot regardless
> > of "rootwait" bootarg being present or not
> > https://bugs.linaro.org/show_bug.cgi?id=5975.
> >
> > Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
> > ---
> >  drivers/regulator/qcom-rpmh-regulator.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/regulator/qcom-rpmh-regulator.c b/drivers/regulator/qcom-rpmh-regulator.c
> > index b0a58c62b1e2..30659922b0aa 100644
> > --- a/drivers/regulator/qcom-rpmh-regulator.c
> > +++ b/drivers/regulator/qcom-rpmh-regulator.c
> > @@ -1517,7 +1517,7 @@ MODULE_DEVICE_TABLE(of, rpmh_regulator_match_table);
> >  static struct platform_driver rpmh_regulator_driver = {
> >       .driver = {
> >               .name = "qcom-rpmh-regulator",
> > -             .probe_type = PROBE_PREFER_ASYNCHRONOUS,
> > +             .probe_type = PROBE_FORCE_SYNCHRONOUS,
> >               .of_match_table = of_match_ptr(rpmh_regulator_match_table),
> >       },
> >       .probe = rpmh_regulator_probe,
>
> Amit, just wondering: what happened to this? It seems there was some
> agreement to go down this route to fix your regression, but then nothing
> happened anymore since about a week. Or am I missing something?

Hi, I didn't get around to try out new things last week as suggested
on the original thread [1]. I'll get back to debugging it later in the
week hopefully.

Regards,
Amit Pundir
[1] https://lore.kernel.org/all/CAD=FV=VSFDe445WEVTHXxU1WS_HGUV5jR5E8_Vgd4eyhn3rHyA@mail.gmail.com/

>
> Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
> --
> Everything you wanna know about Linux kernel regression tracking:
> https://linux-regtracking.leemhuis.info/about/#tldr
> If I did something stupid, please tell me, as explained on that page.
>
> #regzbot ^backmonitor:
> https://lore.kernel.org/lkml/CAMi1Hd1avQDcDQf137m2auz2znov4XL8YGrLZsw5edb-NtRJRw@mail.gmail.com/