Message ID | 20230511073428.10264-1-u.kleine-koenig@pengutronix.de |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp4189171vqo; Thu, 11 May 2023 00:53:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ56gX8/PRLxSlGyLwqG8QEVVBslH3pTFbyc4gdJOJzJIeTTChBCEfT1MfEScmWaTYWqIoSQ X-Received: by 2002:a05:6a21:7891:b0:101:209e:bc57 with SMTP id bf17-20020a056a21789100b00101209ebc57mr12373333pzc.50.1683791637930; Thu, 11 May 2023 00:53:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683791637; cv=none; d=google.com; s=arc-20160816; b=tUcbIsNTz9nixh4S/G6x361zNWQesYXRKumsThGCVmkdj2rvikiewh4NPEa3GZAXug bN7qoRUrmPkK+8BaDSQIZXchbDHU7whdBzuSMI4ufx1eooiZnL8oZedln/oFFr4CXYZh 5+GZQuZXJo4Zo5WvwvclxdodxYwuGVVqiDoluAANNiXr6Gb20K2vSYNmt4G4n/g5fn4i 1EKBLM0dMfI3GKyvH5hUX71wAbTdpLXSrZzuI5FT+1qAAVDqpOlD9VKbEqFZzh0h9/6E /b28b66Tc/+twDofIMTcWM1clU9fn4o9Eqi7E3VzuCax91zDFRTaiW0FlqyOVXEwvBj3 T58Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=5XR6+55yi4tPNhQ/iD4NXvad20nbO2OjrT9N59wMiGE=; b=1CJrrBL/6etSK6Cpy+2JkpdZ/QULNJVdYtwK0TnpukAdVCK+CzM7oBStZec22c8lfP wFINYsSnKeebI4b2STea9DYl6Hc0NBRzYiZdzvTGOp0D2Sa+LGhZJtLHn7KHIoT9K+ue 3s7ov3YRML1l7n0qdXcI1V67KFTvY34YNAjCNGIBRfyCNdJAkBvQkLIlmvQB8a9DZSWO VXsOcDNVq9V4vYLeUAigmW8j7VfJRJ7GyqwsrHyJdtgZlLSFGEGHmzTMmR/iOWYJ9CL9 WRIHyxQLt5oPhVONKGn3WkuqJX/fDnvpod/iZ7DuB1x5LLUZon+VK5YlxHkJWBPFIx1L FOJw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a189-20020a624dc6000000b0064583a79521si7357107pfb.283.2023.05.11.00.53.45; Thu, 11 May 2023 00:53:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231802AbjEKHfV (ORCPT <rfc822;peekingduck44@gmail.com> + 99 others); Thu, 11 May 2023 03:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237717AbjEKHfH (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 11 May 2023 03:35:07 -0400 Received: from metis.ext.pengutronix.de (metis.ext.pengutronix.de [IPv6:2001:67c:670:201:290:27ff:fe1d:cc33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5533F9EE5 for <linux-kernel@vger.kernel.org>; Thu, 11 May 2023 00:34:37 -0700 (PDT) Received: from drehscheibe.grey.stw.pengutronix.de ([2a0a:edc0:0:c01:1d::a2]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <ukl@pengutronix.de>) id 1px0os-0007Zn-TE; Thu, 11 May 2023 09:34:34 +0200 Received: from [2a0a:edc0:0:900:1d::77] (helo=ptz.office.stw.pengutronix.de) by drehscheibe.grey.stw.pengutronix.de with esmtp (Exim 4.94.2) (envelope-from <ukl@pengutronix.de>) id 1px0os-002fr1-2v; Thu, 11 May 2023 09:34:34 +0200 Received: from ukl by ptz.office.stw.pengutronix.de with local (Exim 4.94.2) (envelope-from <ukl@pengutronix.de>) id 1px0or-003LJM-58; Thu, 11 May 2023 09:34:33 +0200 From: =?utf-8?q?Uwe_Kleine-K=C3=B6nig?= <u.kleine-koenig@pengutronix.de> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Rafael J. Wysocki" <rafael@kernel.org> Cc: linux-kernel@vger.kernel.org, kernel@pengutronix.de Subject: [PATCH] driver core: Call pm_runtime_put_sync() only after device_remove() Date: Thu, 11 May 2023 09:34:28 +0200 Message-Id: <20230511073428.10264-1-u.kleine-koenig@pengutronix.de> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Developer-Signature: v=1; a=openpgp-sha256; l=1613; i=u.kleine-koenig@pengutronix.de; h=from:subject; bh=inxu00lxAvQhhTdGwdmmOXbixsATQzMqe2CB3zBDu5I=; b=owEBbQGS/pANAwAKAY+A+1h9Ev5OAcsmYgBkXJqDyBlNnJ+V0dwc6hrHJ43dopD/xjtOQqTyu VX7vDqJ/uKJATMEAAEKAB0WIQQ/gaxpOnoeWYmt/tOPgPtYfRL+TgUCZFyagwAKCRCPgPtYfRL+ TmGdB/9Mu2bv39+Vfrpo+4ssj59lgz65Oc97qu74DfG3s7/lLI0AsUWxoEOuE++iqeatemZlwvH 4QURYl7UbHzj+pUAfDx6gxTzCmUqPd/40uC35U61Ah+h1YN3hMLRKmmtDw67FjeqsX1i7G+rqI9 8jTXKOk427HvZPHsX3nuyWBkUqhIRwdEEh3DPBbngoRQYkQkJZV2dxBQ9warhDnRbYqV7Nv19o+ Ielf3E3CW/ZhfnJIeTHGcviYojHygYaa88y0NgKThdu9e1h0meTIiCQR7oEr8OBhifGj6HIDB/M R+QHgkZwc8FwpmGXfN80j4gqIkQZVq59jdg6TpWwqhSMwncO X-Developer-Key: i=u.kleine-koenig@pengutronix.de; a=openpgp; fpr=0D2511F322BFAB1C1580266BE2DCDD9132669BD6 Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 2a0a:edc0:0:c01:1d::a2 X-SA-Exim-Mail-From: ukl@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-kernel@vger.kernel.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765583500160633111?= X-GMAIL-MSGID: =?utf-8?q?1765583500160633111?= |
Series |
driver core: Call pm_runtime_put_sync() only after device_remove()
|
|
Commit Message
Uwe Kleine-König
May 11, 2023, 7:34 a.m. UTC
Many drivers that use runtime PM call pm_runtime_get_sync() or one of
its variants in their remove callback. So calling pm_runtime_put_sync()
directly before calling the remove callback results (under some
conditions) in the driver's suspend routine being called just to resume
it again afterwards.
So delay the pm_runtime_put_sync() call until after device_remove().
Confirmed on a stm32mp157a that doing
echo 4400e000.can > /sys/bus/platform/drivers/m_can_platform/unbind
(starting with a runtime-pm suspended 4400e000.can) results in one call
less of m_can_runtime_resume() and m_can_runtime_suspend() each after
this change was applied.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
---
Hello,
side note: To test I added a dev_info() to m_can_runtime_resume() and
m_can_runtime_suspend(). I was surprised that directly after boot I had:
# dmesg | grep -E '4400e000.can: m_can_runtime_(resume|suspend)' | wc -l
15
I didn't go down that rabbit hole to debug this.
Best regards
Uwe
drivers/base/dd.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
Comments
On Thu, May 11, 2023 at 9:34 AM Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote: > > Many drivers that use runtime PM call pm_runtime_get_sync() or one of > its variants in their remove callback. So calling pm_runtime_put_sync() > directly before calling the remove callback results (under some > conditions) in the driver's suspend routine being called just to resume > it again afterwards. > > So delay the pm_runtime_put_sync() call until after device_remove(). > > Confirmed on a stm32mp157a that doing > > echo 4400e000.can > /sys/bus/platform/drivers/m_can_platform/unbind > > (starting with a runtime-pm suspended 4400e000.can) results in one call > less of m_can_runtime_resume() and m_can_runtime_suspend() each after > this change was applied. > > Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> I'm not against this change, although I kind of expect it to trigger some fallout that will need to be addressed. So caveat emtor. Anyway Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> > --- > Hello, > > side note: To test I added a dev_info() to m_can_runtime_resume() and > m_can_runtime_suspend(). I was surprised that directly after boot I had: > > # dmesg | grep -E '4400e000.can: m_can_runtime_(resume|suspend)' | wc -l > 15 > > I didn't go down that rabbit hole to debug this. > > Best regards > Uwe > > drivers/base/dd.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/base/dd.c b/drivers/base/dd.c > index 9c09ca5c4ab6..d97f6b1486d1 100644 > --- a/drivers/base/dd.c > +++ b/drivers/base/dd.c > @@ -1267,10 +1267,10 @@ static void __device_release_driver(struct device *dev, struct device *parent) > > bus_notify(dev, BUS_NOTIFY_UNBIND_DRIVER); > > - pm_runtime_put_sync(dev); > - > device_remove(dev); > > + pm_runtime_put_sync(dev); > + > if (dev->bus && dev->bus->dma_cleanup) > dev->bus->dma_cleanup(dev); > > -- > 2.39.2 >
On Thu, May 11, 2023 at 12:18:09PM +0200, Rafael J. Wysocki wrote: > On Thu, May 11, 2023 at 9:34 AM Uwe Kleine-König > <u.kleine-koenig@pengutronix.de> wrote: > > > > Many drivers that use runtime PM call pm_runtime_get_sync() or one of > > its variants in their remove callback. So calling pm_runtime_put_sync() > > directly before calling the remove callback results (under some > > conditions) in the driver's suspend routine being called just to resume > > it again afterwards. > > > > So delay the pm_runtime_put_sync() call until after device_remove(). > > > > Confirmed on a stm32mp157a that doing > > > > echo 4400e000.can > /sys/bus/platform/drivers/m_can_platform/unbind > > > > (starting with a runtime-pm suspended 4400e000.can) results in one call > > less of m_can_runtime_resume() and m_can_runtime_suspend() each after > > this change was applied. > > > > Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> > > I'm not against this change, although I kind of expect it to trigger > some fallout that will need to be addressed. So caveat emtor. > > Anyway > > Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> Thanks for your review tag. I wondered if there will be some fallout, and don't know what to expect yet. Sounds like getting it into next soon is a good idea?! Best regards Uwe
On Thu, May 11, 2023 at 12:39:23PM +0200, Uwe Kleine-König wrote: > On Thu, May 11, 2023 at 12:18:09PM +0200, Rafael J. Wysocki wrote: > > On Thu, May 11, 2023 at 9:34 AM Uwe Kleine-König > > <u.kleine-koenig@pengutronix.de> wrote: > > > > > > Many drivers that use runtime PM call pm_runtime_get_sync() or one of > > > its variants in their remove callback. So calling pm_runtime_put_sync() > > > directly before calling the remove callback results (under some > > > conditions) in the driver's suspend routine being called just to resume > > > it again afterwards. > > > > > > So delay the pm_runtime_put_sync() call until after device_remove(). > > > > > > Confirmed on a stm32mp157a that doing > > > > > > echo 4400e000.can > /sys/bus/platform/drivers/m_can_platform/unbind > > > > > > (starting with a runtime-pm suspended 4400e000.can) results in one call > > > less of m_can_runtime_resume() and m_can_runtime_suspend() each after > > > this change was applied. > > > > > > Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> > > > > I'm not against this change, although I kind of expect it to trigger > > some fallout that will need to be addressed. So caveat emtor. > > > > Anyway > > > > Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> > > Thanks for your review tag. I wondered if there will be some fallout, > and don't know what to expect yet. Sounds like getting it into next soon > is a good idea?! No, this seems like very bad idea and even violates the documentation which clearly states that the usage counter is balanced before calling remove() so that drivers can use pm_runtime_suspend() to put devices into suspended state. There's is really no good reason to even try to change as this is in no way a fast path. NAK. Johan
On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > On Thu, May 11, 2023 at 12:39:23PM +0200, Uwe Kleine-König wrote: > > On Thu, May 11, 2023 at 12:18:09PM +0200, Rafael J. Wysocki wrote: > > > On Thu, May 11, 2023 at 9:34 AM Uwe Kleine-König > > > <u.kleine-koenig@pengutronix.de> wrote: > > > > > > > > Many drivers that use runtime PM call pm_runtime_get_sync() or one of > > > > its variants in their remove callback. So calling pm_runtime_put_sync() > > > > directly before calling the remove callback results (under some > > > > conditions) in the driver's suspend routine being called just to resume > > > > it again afterwards. > > > > > > > > So delay the pm_runtime_put_sync() call until after device_remove(). > > > > > > > > Confirmed on a stm32mp157a that doing > > > > > > > > echo 4400e000.can > /sys/bus/platform/drivers/m_can_platform/unbind > > > > > > > > (starting with a runtime-pm suspended 4400e000.can) results in one call > > > > less of m_can_runtime_resume() and m_can_runtime_suspend() each after > > > > this change was applied. > > > > > > > > Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> > > > > > > I'm not against this change, although I kind of expect it to trigger > > > some fallout that will need to be addressed. So caveat emtor. > > > > > > Anyway > > > > > > Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> > > > > Thanks for your review tag. I wondered if there will be some fallout, > > and don't know what to expect yet. Sounds like getting it into next soon > > is a good idea?! > > No, this seems like very bad idea and even violates the documentation > which clearly states that the usage counter is balanced before calling > remove() so that drivers can use pm_runtime_suspend() to put devices > into suspended state. I missed that, sorry. > There's is really no good reason to even try to change as this is in no > way a fast path. Still, I think that while the "put" part needs to be done before device_remove(), the actual state change can be carried out later. So something like pm_runtime_put_noidle(dev); device_remove(dev); pm_runtime_suspend(dev); would generally work, wouldn't it?
Hello Johan, On Thu, May 11, 2023 at 01:48:25PM +0200, Johan Hovold wrote: > On Thu, May 11, 2023 at 12:39:23PM +0200, Uwe Kleine-König wrote: > > On Thu, May 11, 2023 at 12:18:09PM +0200, Rafael J. Wysocki wrote: > > > On Thu, May 11, 2023 at 9:34 AM Uwe Kleine-König > > > <u.kleine-koenig@pengutronix.de> wrote: > > > > > > > > Many drivers that use runtime PM call pm_runtime_get_sync() or one of > > > > its variants in their remove callback. So calling pm_runtime_put_sync() > > > > directly before calling the remove callback results (under some > > > > conditions) in the driver's suspend routine being called just to resume > > > > it again afterwards. > > > > > > > > So delay the pm_runtime_put_sync() call until after device_remove(). > > > > > > > > Confirmed on a stm32mp157a that doing > > > > > > > > echo 4400e000.can > /sys/bus/platform/drivers/m_can_platform/unbind > > > > > > > > (starting with a runtime-pm suspended 4400e000.can) results in one call > > > > less of m_can_runtime_resume() and m_can_runtime_suspend() each after > > > > this change was applied. > > > > > > > > Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> > > > > > > I'm not against this change, although I kind of expect it to trigger > > > some fallout that will need to be addressed. So caveat emtor. > > > > > > Anyway > > > > > > Reviewed-by: Rafael J. Wysocki <rafael@kernel.org> > > > > Thanks for your review tag. I wondered if there will be some fallout, > > and don't know what to expect yet. Sounds like getting it into next soon > > is a good idea?! > > No, this seems like very bad idea and even violates the documentation > which clearly states that the usage counter is balanced before calling > remove() so that drivers can use pm_runtime_suspend() to put devices > into suspended state. I grepped around a bit and found: To allow bus types and drivers to put devices into the suspended state by calling pm_runtime_suspend() from their ->remove() routines, the driver core executes pm_runtime_put_sync() after running the BUS_NOTIFY_UNBIND_DRIVER notifications in __device_release_driver(). This requires bus types and drivers to make their ->remove() callbacks avoid races with runtime PM directly, but it also allows more flexibility in the handling of devices during the removal of their drivers. Hmm, while I see your point, it's still ugly. I'll think about it. Best regards Uwe
On Thu, May 11, 2023 at 04:44:25PM +0200, Rafael J. Wysocki wrote: > On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > No, this seems like very bad idea and even violates the documentation > > which clearly states that the usage counter is balanced before calling > > remove() so that drivers can use pm_runtime_suspend() to put devices > > into suspended state. > > I missed that, sorry. > > > There's is really no good reason to even try to change as this is in no > > way a fast path. > > Still, I think that while the "put" part needs to be done before > device_remove(), the actual state change can be carried out later. > > So something like > > pm_runtime_put_noidle(dev); > > device_remove(dev); > > pm_runtime_suspend(dev); > > would generally work, wouldn't it? No, as drivers typically disable runtime pm in their remove callbacks, that pm_runtime_suspend() would amount to a no-op (and calling the driver pm ops post unbind and the driver having freed its data would not work either). Johan
On Fri, May 12, 2023 at 9:39 AM Johan Hovold <johan@kernel.org> wrote: > > On Thu, May 11, 2023 at 04:44:25PM +0200, Rafael J. Wysocki wrote: > > On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > > > No, this seems like very bad idea and even violates the documentation > > > which clearly states that the usage counter is balanced before calling > > > remove() so that drivers can use pm_runtime_suspend() to put devices > > > into suspended state. > > > > I missed that, sorry. > > > > > There's is really no good reason to even try to change as this is in no > > > way a fast path. > > > > Still, I think that while the "put" part needs to be done before > > device_remove(), the actual state change can be carried out later. > > > > So something like > > > > pm_runtime_put_noidle(dev); > > > > device_remove(dev); > > > > pm_runtime_suspend(dev); > > > > would generally work, wouldn't it? > > No, as drivers typically disable runtime pm in their remove callbacks, What exactly do you mean by "typically"? None of the PCI drivers should do that, for instance. > that pm_runtime_suspend() would amount to a no-op (and calling the > driver pm ops post unbind and the driver having freed its data would > not work either). Well, not really. There are drivers and there are bus types/PM domains. Drivers need not disable PM-runtime in their "remove" callbacks if they know that the bus type/PM domain will take care of handling PM-runtime properly after the driver's remove callback has run and the bus type/PM domain may very well want its PM-runtime suspend callback to run then (for example, to remove power from the unused device). Arguably it can invoke runtime_suspend() from its "remove" callback, so it's not like this is a big deal, but IMO it helps if the most general case is considered. Anyway, the question here really is: Does it make sense to carry out a runtime suspend immediately before device_remove()? Honestly, I'm not sure about that.
On Fri, May 12, 2023 at 04:04:59PM +0200, Rafael J. Wysocki wrote: > On Fri, May 12, 2023 at 9:39 AM Johan Hovold <johan@kernel.org> wrote: > > > > On Thu, May 11, 2023 at 04:44:25PM +0200, Rafael J. Wysocki wrote: > > > On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > > > > > No, this seems like very bad idea and even violates the documentation > > > > which clearly states that the usage counter is balanced before calling > > > > remove() so that drivers can use pm_runtime_suspend() to put devices > > > > into suspended state. > > > > > > I missed that, sorry. > > > > > > > There's is really no good reason to even try to change as this is in no > > > > way a fast path. > > > > > > Still, I think that while the "put" part needs to be done before > > > device_remove(), the actual state change can be carried out later. > > > > > > So something like > > > > > > pm_runtime_put_noidle(dev); > > > > > > device_remove(dev); > > > > > > pm_runtime_suspend(dev); > > > > > > would generally work, wouldn't it? > > > > No, as drivers typically disable runtime pm in their remove callbacks, > > What exactly do you mean by "typically"? None of the PCI drivers > should do that, for instance. I had platform drivers in mind, but so do i2c drivers for example. > > that pm_runtime_suspend() would amount to a no-op (and calling the > > driver pm ops post unbind and the driver having freed its data would > > not work either). > > Well, not really. > > There are drivers and there are bus types/PM domains. Drivers need > not disable PM-runtime in their "remove" callbacks if they know that > the bus type/PM domain will take care of handling PM-runtime properly > after the driver's remove callback has run and the bus type/PM domain > may very well want its PM-runtime suspend callback to run then (for > example, to remove power from the unused device). Arguably it can > invoke runtime_suspend() from its "remove" callback, so it's not like > this is a big deal, but IMO it helps if the most general case is > considered. My point was that hundreds of drivers do and for these this call becomes a no-op. Same for buses that disable runtime pm at remove. > Anyway, the question here really is: Does it make sense to carry out a > runtime suspend immediately before device_remove()? Honestly, I'm not > sure about that. I'd say it doesn't really matter as driver unbind is not a common operation and drivers using autosuspend would generally not be affected either. You can try to rework this, but clearly it needs more thought than simply moving the put sync and some drivers may also be relying on the current behaviour. A quick grep reveals a few which would be left active if you change pm_runtime_put_sync() to pm_runtime_put_noidle(), even if that could be fixed driver by driver of course. Johan
On Fri, May 12, 2023 at 5:00 PM Johan Hovold <johan@kernel.org> wrote: > > On Fri, May 12, 2023 at 04:04:59PM +0200, Rafael J. Wysocki wrote: > > On Fri, May 12, 2023 at 9:39 AM Johan Hovold <johan@kernel.org> wrote: > > > > > > On Thu, May 11, 2023 at 04:44:25PM +0200, Rafael J. Wysocki wrote: > > > > On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > > > > > > > No, this seems like very bad idea and even violates the documentation > > > > > which clearly states that the usage counter is balanced before calling > > > > > remove() so that drivers can use pm_runtime_suspend() to put devices > > > > > into suspended state. > > > > > > > > I missed that, sorry. > > > > > > > > > There's is really no good reason to even try to change as this is in no > > > > > way a fast path. > > > > > > > > Still, I think that while the "put" part needs to be done before > > > > device_remove(), the actual state change can be carried out later. > > > > > > > > So something like > > > > > > > > pm_runtime_put_noidle(dev); > > > > > > > > device_remove(dev); > > > > > > > > pm_runtime_suspend(dev); > > > > > > > > would generally work, wouldn't it? > > > > > > No, as drivers typically disable runtime pm in their remove callbacks, > > > > What exactly do you mean by "typically"? None of the PCI drivers > > should do that, for instance. > > I had platform drivers in mind, but so do i2c drivers for example. > > > > that pm_runtime_suspend() would amount to a no-op (and calling the > > > driver pm ops post unbind and the driver having freed its data would > > > not work either). > > > > Well, not really. > > > > There are drivers and there are bus types/PM domains. Drivers need > > not disable PM-runtime in their "remove" callbacks if they know that > > the bus type/PM domain will take care of handling PM-runtime properly > > after the driver's remove callback has run and the bus type/PM domain > > may very well want its PM-runtime suspend callback to run then (for > > example, to remove power from the unused device). Arguably it can > > invoke runtime_suspend() from its "remove" callback, so it's not like > > this is a big deal, but IMO it helps if the most general case is > > considered. > > My point was that hundreds of drivers do and for these this call becomes > a no-op. Same for buses that disable runtime pm at remove. > > > Anyway, the question here really is: Does it make sense to carry out a > > runtime suspend immediately before device_remove()? Honestly, I'm not > > sure about that. > > I'd say it doesn't really matter as driver unbind is not a common > operation and drivers using autosuspend would generally not be affected > either. > > You can try to rework this, but clearly it needs more thought than > simply moving the put sync and some drivers may also be relying on the > current behaviour. > > A quick grep reveals a few which would be left active if you change > pm_runtime_put_sync() to pm_runtime_put_noidle(), even if that could be > fixed driver by driver of course. OK, fair enough.
On Fri, May 12, 2023 at 09:40:01AM +0200, Johan Hovold wrote: > On Thu, May 11, 2023 at 04:44:25PM +0200, Rafael J. Wysocki wrote: > > On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > > > No, this seems like very bad idea and even violates the documentation > > > which clearly states that the usage counter is balanced before calling > > > remove() so that drivers can use pm_runtime_suspend() to put devices > > > into suspended state. > > > > I missed that, sorry. > > > > > There's is really no good reason to even try to change as this is in no > > > way a fast path. > > > > Still, I think that while the "put" part needs to be done before > > device_remove(), the actual state change can be carried out later. > > > > So something like > > > > pm_runtime_put_noidle(dev); > > > > device_remove(dev); > > > > pm_runtime_suspend(dev); > > > > would generally work, wouldn't it? > > No, as drivers typically disable runtime pm in their remove callbacks, > that pm_runtime_suspend() would amount to a no-op (and calling the > driver pm ops post unbind and the driver having freed its data would > not work either). However if a driver author also cares for the CONFIG_PM=n case, calling pm_runtime_suspend() doesn't have the intended effect and so it's unfortunately complicated to rely on runtime-pm to power down your device and you have to do it by hand anyhow (unless you let your driver depend on CONFIG_PM). So I'm not convinced that "A driver can call pm_runtime_suspend() to power down" is a useful thing to have. In the end something like 72362dcdf654 ("can: mcp251xfd: mcp251xfd_unregister(): simplify runtime PM handling") might be an approach. But IMHO it's more complicated than it should be and honestly I'm not sure if it's safe and correct this way. Best regards Uwe
On Fri, May 12, 2023 at 08:49:25PM +0200, Uwe Kleine-König wrote: > On Fri, May 12, 2023 at 09:40:01AM +0200, Johan Hovold wrote: > > On Thu, May 11, 2023 at 04:44:25PM +0200, Rafael J. Wysocki wrote: > > > On Thu, May 11, 2023 at 1:48 PM Johan Hovold <johan@kernel.org> wrote: > > > > > > No, this seems like very bad idea and even violates the documentation > > > > which clearly states that the usage counter is balanced before calling > > > > remove() so that drivers can use pm_runtime_suspend() to put devices > > > > into suspended state. > > > > > > I missed that, sorry. > > > > > > > There's is really no good reason to even try to change as this is in no > > > > way a fast path. > > > > > > Still, I think that while the "put" part needs to be done before > > > device_remove(), the actual state change can be carried out later. > > > > > > So something like > > > > > > pm_runtime_put_noidle(dev); > > > > > > device_remove(dev); > > > > > > pm_runtime_suspend(dev); > > > > > > would generally work, wouldn't it? > > > > No, as drivers typically disable runtime pm in their remove callbacks, > > that pm_runtime_suspend() would amount to a no-op (and calling the > > driver pm ops post unbind and the driver having freed its data would > > not work either). > > However if a driver author also cares for the CONFIG_PM=n case, calling > pm_runtime_suspend() doesn't have the intended effect and so it's > unfortunately complicated to rely on runtime-pm to power down your > device and you have to do it by hand anyhow (unless you let your driver > depend on CONFIG_PM). So I'm not convinced that "A driver can call > pm_runtime_suspend() to power down" is a useful thing to have. Right, but we do have drivers that have CONFIG_PM as an explicit dependency. > In the end something like 72362dcdf654 ("can: mcp251xfd: > mcp251xfd_unregister(): simplify runtime PM handling") might be an > approach. But IMHO it's more complicated than it should be and honestly > I'm not sure if it's safe and correct this way. Yeah, unfortunately runtime PM is fairly underspecified so we end up with this multitude of implementations, many of which are broken in various ways. A smaller API with documented best-practices may have helped, but that's not where we are right now. Looks like 72362dcdf654 ("can: mcp251xfd: mcp251xfd_unregister(): simplify runtime PM handling") introduces yet another way to do things, and which will break if anyone enables (or tries to use this pattern in another driver with) autosuspend... Johan
On 17.05.2023 10:28:01, Johan Hovold wrote: > Right, but we do have drivers that have CONFIG_PM as an explicit > dependency. > > > In the end something like 72362dcdf654 ("can: mcp251xfd: > > mcp251xfd_unregister(): simplify runtime PM handling") might be an > > approach. But IMHO it's more complicated than it should be and honestly > > I'm not sure if it's safe and correct this way. > > Yeah, unfortunately runtime PM is fairly underspecified so we end up > with this multitude of implementations, many of which are broken in > various ways. A smaller API with documented best-practices may have > helped, but that's not where we are right now. > > Looks like 72362dcdf654 ("can: mcp251xfd: mcp251xfd_unregister(): > simplify runtime PM handling") introduces yet another way to do things, > and which will break if anyone enables (or tries to use this pattern in > another driver with) autosuspend... ACK - I think in that driver it works, as the runtime PM is resumed during interface up and suspended in interface down. IMHO autosuspend would not bring any benefits here... regards, Marc
diff --git a/drivers/base/dd.c b/drivers/base/dd.c index 9c09ca5c4ab6..d97f6b1486d1 100644 --- a/drivers/base/dd.c +++ b/drivers/base/dd.c @@ -1267,10 +1267,10 @@ static void __device_release_driver(struct device *dev, struct device *parent) bus_notify(dev, BUS_NOTIFY_UNBIND_DRIVER); - pm_runtime_put_sync(dev); - device_remove(dev); + pm_runtime_put_sync(dev); + if (dev->bus && dev->bus->dma_cleanup) dev->bus->dma_cleanup(dev);