Message ID | fe235a1f65bb6c86d2afcdf52d85f80ae728dcc5.1683722688.git.geert+renesas@glider.be |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3627528vqo; Wed, 10 May 2023 06:34:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ45g04VHxdH4vBTr8kM99WPmUICWNWLW5munT5b4ThdG+xwQtvFbC/QmR85ww7J3XGtmhnE X-Received: by 2002:a54:4518:0:b0:38d:e85e:e7ea with SMTP id l24-20020a544518000000b0038de85ee7eamr2907838oil.7.1683725642510; Wed, 10 May 2023 06:34:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683725642; cv=none; d=google.com; s=arc-20160816; b=z8Gtb+R5aNFwKY5JYwozFYucufAFrSYV3XShGmskBND5L4pxNFP+bIK/2VQRvjZbBP J9FNuWZ3RcwdCKoc2unRqL0LF9YBOayXqFr8Gcd3G2OPlpRouMds0qYDWeS4eitxRdI4 hfM53TzzOJ9v2HfTmn3J/rvmq7pkjU2S+RUJ/EyDTe08OMQXig7RYKgRQbdH1Lf2Kcet 9wCtXIyNb9sHJwNoYkeoguLpGVY+vwMk0agYPP3TIbG/kWjpWPjTuw5+FnyqJwlrJ03Y T+nXKAeiHc51cix9Ep5A2nE/lxDcxa4UkjKa4DbD7MHIn+5tq0Lh3MUHqyvA/sTPEZUe 36dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Tp0MDIluaMWXnQRKuWvD5mz6XjgVberXqy75EQcHSFU=; b=F8D4USmvqf87S4jElxBYXXVrcULZzVpW7fxyQRvNMub6zaJIaKEMpf95itDbPtuS3C 61ZLHVkojXRwwV9hRPLQkZ6tWSsVzLQWIyqQdTTnnR/mTniVeNld8JAsGefC0gBxF8Oq 85VrmETn8/BOpHAF09LXrrV4PpWjOLlkB78SOFJSpbhQ9m0h65Ir2vWTF0TJvqEqb0Dm 8f0PqbB13KA19acKZkp3OYqwA4c70V1sG9mj+hdNysKzrlokLPyuMtsvYDBwF90fsC+E AbZeEdeChLTb23i6EVT2L4SDG0rKgDpXFhMxV2bBkyuw4kGDk6px66tIRltiS3rsylo6 Ifow== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n12-20020a4a848c000000b005473fd64fd8si7697816oog.54.2023.05.10.06.33.49; Wed, 10 May 2023 06:34:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236978AbjEJNXm (ORCPT <rfc822;jeantsuru.cumc.mandola@gmail.com> + 99 others); Wed, 10 May 2023 09:23:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236644AbjEJNXk (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Wed, 10 May 2023 09:23:40 -0400 Received: from albert.telenet-ops.be (albert.telenet-ops.be [IPv6:2a02:1800:110:4::f00:1a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FFED5BAC for <linux-kernel@vger.kernel.org>; Wed, 10 May 2023 06:23:35 -0700 (PDT) Received: from ramsan.of.borg ([IPv6:2a02:1810:ac12:ed30:cc75:d8ef:4074:8af9]) by albert.telenet-ops.be with bizsmtp id v1PG2900C3l7qvk061PGDS; Wed, 10 May 2023 15:23:32 +0200 Received: from rox.of.borg ([192.168.97.57]) by ramsan.of.borg with esmtp (Exim 4.95) (envelope-from <geert@linux-m68k.org>) id 1pwjme-001nDk-DC; Wed, 10 May 2023 15:23:16 +0200 Received: from geert by rox.of.borg with local (Exim 4.95) (envelope-from <geert@linux-m68k.org>) id 1pwjmm-00F6qK-40; Wed, 10 May 2023 15:23:16 +0200 From: Geert Uytterhoeven <geert+renesas@glider.be> To: Stephen Boyd <sboyd@kernel.org>, Tomasz Figa <tomasz.figa@gmail.com>, Sylwester Nawrocki <s.nawrocki@samsung.com>, Will Deacon <will@kernel.org>, Arnd Bergmann <arnd@arndb.de>, Wolfram Sang <wsa+renesas@sang-engineering.com>, Dejin Zheng <zhengdejin5@gmail.com>, Kai-Heng Feng <kai.heng.feng@canonical.com>, Nicholas Piggin <npiggin@gmail.com>, Heiko Carstens <hca@linux.ibm.com>, Peter Zijlstra <peterz@infradead.org>, Russell King <linux@armlinux.org.uk>, John Stultz <jstultz@google.com>, Thomas Gleixner <tglx@linutronix.de>, Tony Lindgren <tony@atomide.com>, Krzysztof Kozlowski <krzk@kernel.org>, Tero Kristo <tero.kristo@linux.intel.com>, Ulf Hansson <ulf.hansson@linaro.org>, "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>, Vincent Guittot <vincent.guittot@linaro.org> Cc: linux-arm-kernel@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Geert Uytterhoeven <geert+renesas@glider.be> Subject: [PATCH v2 1/2] iopoll: Call cpu_relax() in busy loops Date: Wed, 10 May 2023 15:23:13 +0200 Message-Id: <fe235a1f65bb6c86d2afcdf52d85f80ae728dcc5.1683722688.git.geert+renesas@glider.be> X-Mailer: git-send-email 2.34.1 In-Reply-To: <cover.1683722688.git.geert+renesas@glider.be> References: <cover.1683722688.git.geert+renesas@glider.be> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765514299115099362?= X-GMAIL-MSGID: =?utf-8?q?1765514299115099362?= |
Series |
iopoll: Busy loop and timeout improvements
|
|
Commit Message
Geert Uytterhoeven
May 10, 2023, 1:23 p.m. UTC
It is considered good practice to call cpu_relax() in busy loops, see Documentation/process/volatile-considered-harmful.rst. This can not only lower CPU power consumption or yield to a hyperthreaded twin processor, but also allows an architecture to mitigate hardware issues (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the architecture-specific cpu_relax() implementation. In addition, cpu_relax() is also a compiler barrier. It is not immediately obvious that the @op argument "function" will result in an actual function call (e.g. in case of inlining). Where a function call is a C sequence point, this is lost on inlining. Therefore, with agressive enough optimization it might be possible for the compiler to hoist the: (val) = op(args); "load" out of the loop because it doesn't see the value changing. The addition of cpu_relax() would inhibit this. As the iopoll helpers lack calls to cpu_relax(), people are sometimes reluctant to use them, and may fall back to open-coded polling loops (including cpu_relax() calls) instead. Fix this by adding calls to cpu_relax() to the iopoll helpers: - For the non-atomic case, it is sufficient to call cpu_relax() in case of a zero sleep-between-reads value, as a call to usleep_range() is a safe barrier otherwise. However, it doesn't hurt to add the call regardless, for simplicity, and for similarity with the atomic case below. - For the atomic case, cpu_relax() must be called regardless of the sleep-between-reads value, as there is no guarantee all architecture-specific implementations of udelay() handle this. Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Arnd Bergmann <arnd@arndb.de> --- v2: - Add Acked-by, - Add compiler barrier and inlining explanation (thanks, Peter!). --- include/linux/iopoll.h | 2 ++ 1 file changed, 2 insertions(+)
Comments
* Geert Uytterhoeven <geert+renesas@glider.be> [230510 13:23]: > It is considered good practice to call cpu_relax() in busy loops, see > Documentation/process/volatile-considered-harmful.rst. This can not > only lower CPU power consumption or yield to a hyperthreaded twin > processor, but also allows an architecture to mitigate hardware issues > (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the > architecture-specific cpu_relax() implementation. > > In addition, cpu_relax() is also a compiler barrier. It is not > immediately obvious that the @op argument "function" will result in an > actual function call (e.g. in case of inlining). > > Where a function call is a C sequence point, this is lost on inlining. > Therefore, with agressive enough optimization it might be possible for > the compiler to hoist the: > > (val) = op(args); > > "load" out of the loop because it doesn't see the value changing. The > addition of cpu_relax() would inhibit this. > > As the iopoll helpers lack calls to cpu_relax(), people are sometimes > reluctant to use them, and may fall back to open-coded polling loops > (including cpu_relax() calls) instead. > > Fix this by adding calls to cpu_relax() to the iopoll helpers: > - For the non-atomic case, it is sufficient to call cpu_relax() in > case of a zero sleep-between-reads value, as a call to > usleep_range() is a safe barrier otherwise. However, it doesn't > hurt to add the call regardless, for simplicity, and for similarity > with the atomic case below. > - For the atomic case, cpu_relax() must be called regardless of the > sleep-between-reads value, as there is no guarantee all > architecture-specific implementations of udelay() handle this. Reviewed-by: Tony Lindgren <tony@atomide.com>
On Wed, 10 May 2023 at 15:23, Geert Uytterhoeven <geert+renesas@glider.be> wrote: > > It is considered good practice to call cpu_relax() in busy loops, see > Documentation/process/volatile-considered-harmful.rst. This can not > only lower CPU power consumption or yield to a hyperthreaded twin > processor, but also allows an architecture to mitigate hardware issues > (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the > architecture-specific cpu_relax() implementation. > > In addition, cpu_relax() is also a compiler barrier. It is not > immediately obvious that the @op argument "function" will result in an > actual function call (e.g. in case of inlining). > > Where a function call is a C sequence point, this is lost on inlining. > Therefore, with agressive enough optimization it might be possible for > the compiler to hoist the: > > (val) = op(args); > > "load" out of the loop because it doesn't see the value changing. The > addition of cpu_relax() would inhibit this. > > As the iopoll helpers lack calls to cpu_relax(), people are sometimes > reluctant to use them, and may fall back to open-coded polling loops > (including cpu_relax() calls) instead. > > Fix this by adding calls to cpu_relax() to the iopoll helpers: > - For the non-atomic case, it is sufficient to call cpu_relax() in > case of a zero sleep-between-reads value, as a call to > usleep_range() is a safe barrier otherwise. However, it doesn't > hurt to add the call regardless, for simplicity, and for similarity > with the atomic case below. > - For the atomic case, cpu_relax() must be called regardless of the > sleep-between-reads value, as there is no guarantee all > architecture-specific implementations of udelay() handle this. > > Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> > Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> > Acked-by: Arnd Bergmann <arnd@arndb.de> Makes sense to me! Feel free to add: Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org> Kind regards Uffe > --- > v2: > - Add Acked-by, > - Add compiler barrier and inlining explanation (thanks, Peter!). > --- > include/linux/iopoll.h | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h > index 2c8860e406bd8cae..0417360a6db9b0d6 100644 > --- a/include/linux/iopoll.h > +++ b/include/linux/iopoll.h > @@ -53,6 +53,7 @@ > } \ > if (__sleep_us) \ > usleep_range((__sleep_us >> 2) + 1, __sleep_us); \ > + cpu_relax(); \ > } \ > (cond) ? 0 : -ETIMEDOUT; \ > }) > @@ -95,6 +96,7 @@ > } \ > if (__delay_us) \ > udelay(__delay_us); \ > + cpu_relax(); \ > } \ > (cond) ? 0 : -ETIMEDOUT; \ > }) > -- > 2.34.1 >
From: Tony Lindgren > Sent: 11 May 2023 07:49 > > * Geert Uytterhoeven <geert+renesas@glider.be> [230510 13:23]: > > It is considered good practice to call cpu_relax() in busy loops, see > > Documentation/process/volatile-considered-harmful.rst. This can not > > only lower CPU power consumption or yield to a hyperthreaded twin > > processor, but also allows an architecture to mitigate hardware issues > > (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the > > architecture-specific cpu_relax() implementation. Don't you also need to call cond_resched() (at least some times). Otherwise the process can't be pre-empted and a RT process that last ran on that cpu will never be scheduled. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Hi David, On Thu, May 11, 2023 at 12:49 PM David Laight <David.Laight@aculab.com> wrote: > > * Geert Uytterhoeven <geert+renesas@glider.be> [230510 13:23]: > > > It is considered good practice to call cpu_relax() in busy loops, see > > > Documentation/process/volatile-considered-harmful.rst. This can not > > > only lower CPU power consumption or yield to a hyperthreaded twin > > > processor, but also allows an architecture to mitigate hardware issues > > > (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the > > > architecture-specific cpu_relax() implementation. > > Don't you also need to call cond_resched() (at least some times). > Otherwise the process can't be pre-empted and a RT process > that last ran on that cpu will never be scheduled. According to [1], cond_resched() must be called at least once per few tens of milliseconds. read_poll_timeout() uses usleep_range(), which calls schedule_hrtimeout_range(). read_poll_timeout_atomic() should not be used with multi-ms timeouts anyway. So I guess we're OK? [1] https://elixir.bootlin.com/linux/latest/source/Documentation/RCU/Design/Requirements/Requirements.rst#L2348 Gr{oetje,eeting}s, Geert
From: Geert Uytterhoeven > Sent: 23 May 2023 08:30 > > Hi David, > > On Thu, May 11, 2023 at 12:49 PM David Laight <David.Laight@aculab.com> wrote: > > > * Geert Uytterhoeven <geert+renesas@glider.be> [230510 13:23]: > > > > It is considered good practice to call cpu_relax() in busy loops, see > > > > Documentation/process/volatile-considered-harmful.rst. This can not > > > > only lower CPU power consumption or yield to a hyperthreaded twin > > > > processor, but also allows an architecture to mitigate hardware issues > > > > (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the > > > > architecture-specific cpu_relax() implementation. > > > > Don't you also need to call cond_resched() (at least some times). > > Otherwise the process can't be pre-empted and a RT process > > that last ran on that cpu will never be scheduled. > > According to [1], cond_resched() must be called at least once per few > tens of milliseconds. Hmmm.... tens of milliseconds is really much too long for RT threads. A limit nearer 1ms would be barely acceptable. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h index 2c8860e406bd8cae..0417360a6db9b0d6 100644 --- a/include/linux/iopoll.h +++ b/include/linux/iopoll.h @@ -53,6 +53,7 @@ } \ if (__sleep_us) \ usleep_range((__sleep_us >> 2) + 1, __sleep_us); \ + cpu_relax(); \ } \ (cond) ? 0 : -ETIMEDOUT; \ }) @@ -95,6 +96,7 @@ } \ if (__delay_us) \ udelay(__delay_us); \ + cpu_relax(); \ } \ (cond) ? 0 : -ETIMEDOUT; \ })