From patchwork Wed May 10 13:23:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Geert Uytterhoeven X-Patchwork-Id: 92086 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3627528vqo; Wed, 10 May 2023 06:34:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ45g04VHxdH4vBTr8kM99WPmUICWNWLW5munT5b4ThdG+xwQtvFbC/QmR85ww7J3XGtmhnE X-Received: by 2002:a54:4518:0:b0:38d:e85e:e7ea with SMTP id l24-20020a544518000000b0038de85ee7eamr2907838oil.7.1683725642510; Wed, 10 May 2023 06:34:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683725642; cv=none; d=google.com; s=arc-20160816; b=z8Gtb+R5aNFwKY5JYwozFYucufAFrSYV3XShGmskBND5L4pxNFP+bIK/2VQRvjZbBP J9FNuWZ3RcwdCKoc2unRqL0LF9YBOayXqFr8Gcd3G2OPlpRouMds0qYDWeS4eitxRdI4 hfM53TzzOJ9v2HfTmn3J/rvmq7pkjU2S+RUJ/EyDTe08OMQXig7RYKgRQbdH1Lf2Kcet 9wCtXIyNb9sHJwNoYkeoguLpGVY+vwMk0agYPP3TIbG/kWjpWPjTuw5+FnyqJwlrJ03Y T+nXKAeiHc51cix9Ep5A2nE/lxDcxa4UkjKa4DbD7MHIn+5tq0Lh3MUHqyvA/sTPEZUe 36dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Tp0MDIluaMWXnQRKuWvD5mz6XjgVberXqy75EQcHSFU=; b=F8D4USmvqf87S4jElxBYXXVrcULZzVpW7fxyQRvNMub6zaJIaKEMpf95itDbPtuS3C 61ZLHVkojXRwwV9hRPLQkZ6tWSsVzLQWIyqQdTTnnR/mTniVeNld8JAsGefC0gBxF8Oq 85VrmETn8/BOpHAF09LXrrV4PpWjOLlkB78SOFJSpbhQ9m0h65Ir2vWTF0TJvqEqb0Dm 8f0PqbB13KA19acKZkp3OYqwA4c70V1sG9mj+hdNysKzrlokLPyuMtsvYDBwF90fsC+E AbZeEdeChLTb23i6EVT2L4SDG0rKgDpXFhMxV2bBkyuw4kGDk6px66tIRltiS3rsylo6 Ifow== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n12-20020a4a848c000000b005473fd64fd8si7697816oog.54.2023.05.10.06.33.49; Wed, 10 May 2023 06:34:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236978AbjEJNXm (ORCPT + 99 others); Wed, 10 May 2023 09:23:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236644AbjEJNXk (ORCPT ); Wed, 10 May 2023 09:23:40 -0400 Received: from albert.telenet-ops.be (albert.telenet-ops.be [IPv6:2a02:1800:110:4::f00:1a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FFED5BAC for ; Wed, 10 May 2023 06:23:35 -0700 (PDT) Received: from ramsan.of.borg ([IPv6:2a02:1810:ac12:ed30:cc75:d8ef:4074:8af9]) by albert.telenet-ops.be with bizsmtp id v1PG2900C3l7qvk061PGDS; Wed, 10 May 2023 15:23:32 +0200 Received: from rox.of.borg ([192.168.97.57]) by ramsan.of.borg with esmtp (Exim 4.95) (envelope-from ) id 1pwjme-001nDk-DC; Wed, 10 May 2023 15:23:16 +0200 Received: from geert by rox.of.borg with local (Exim 4.95) (envelope-from ) id 1pwjmm-00F6qK-40; Wed, 10 May 2023 15:23:16 +0200 From: Geert Uytterhoeven To: Stephen Boyd , Tomasz Figa , Sylwester Nawrocki , Will Deacon , Arnd Bergmann , Wolfram Sang , Dejin Zheng , Kai-Heng Feng , Nicholas Piggin , Heiko Carstens , Peter Zijlstra , Russell King , John Stultz , Thomas Gleixner , Tony Lindgren , Krzysztof Kozlowski , Tero Kristo , Ulf Hansson , "Rafael J . Wysocki" , Vincent Guittot Cc: linux-arm-kernel@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Geert Uytterhoeven Subject: [PATCH v2 1/2] iopoll: Call cpu_relax() in busy loops Date: Wed, 10 May 2023 15:23:13 +0200 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765514299115099362?= X-GMAIL-MSGID: =?utf-8?q?1765514299115099362?= It is considered good practice to call cpu_relax() in busy loops, see Documentation/process/volatile-considered-harmful.rst. This can not only lower CPU power consumption or yield to a hyperthreaded twin processor, but also allows an architecture to mitigate hardware issues (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the architecture-specific cpu_relax() implementation. In addition, cpu_relax() is also a compiler barrier. It is not immediately obvious that the @op argument "function" will result in an actual function call (e.g. in case of inlining). Where a function call is a C sequence point, this is lost on inlining. Therefore, with agressive enough optimization it might be possible for the compiler to hoist the: (val) = op(args); "load" out of the loop because it doesn't see the value changing. The addition of cpu_relax() would inhibit this. As the iopoll helpers lack calls to cpu_relax(), people are sometimes reluctant to use them, and may fall back to open-coded polling loops (including cpu_relax() calls) instead. Fix this by adding calls to cpu_relax() to the iopoll helpers: - For the non-atomic case, it is sufficient to call cpu_relax() in case of a zero sleep-between-reads value, as a call to usleep_range() is a safe barrier otherwise. However, it doesn't hurt to add the call regardless, for simplicity, and for similarity with the atomic case below. - For the atomic case, cpu_relax() must be called regardless of the sleep-between-reads value, as there is no guarantee all architecture-specific implementations of udelay() handle this. Signed-off-by: Geert Uytterhoeven Acked-by: Peter Zijlstra (Intel) Acked-by: Arnd Bergmann Reviewed-by: Tony Lindgren Reviewed-by: Ulf Hansson --- v2: - Add Acked-by, - Add compiler barrier and inlining explanation (thanks, Peter!). --- include/linux/iopoll.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h index 2c8860e406bd8cae..0417360a6db9b0d6 100644 --- a/include/linux/iopoll.h +++ b/include/linux/iopoll.h @@ -53,6 +53,7 @@ } \ if (__sleep_us) \ usleep_range((__sleep_us >> 2) + 1, __sleep_us); \ + cpu_relax(); \ } \ (cond) ? 0 : -ETIMEDOUT; \ }) @@ -95,6 +96,7 @@ } \ if (__delay_us) \ udelay(__delay_us); \ + cpu_relax(); \ } \ (cond) ? 0 : -ETIMEDOUT; \ }) From patchwork Wed May 10 13:23:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Geert Uytterhoeven X-Patchwork-Id: 92083 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3621751vqo; Wed, 10 May 2023 06:26:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ71qcRY6R0Gg9s5RB7Y+Q93BVP7P4BbHNdOjXV/Gjxlrn/03LoNrRkH4RHk1MMBXOaLVwlU X-Received: by 2002:a17:90b:1d87:b0:24f:13e7:e42a with SMTP id pf7-20020a17090b1d8700b0024f13e7e42amr17686190pjb.28.1683725191295; Wed, 10 May 2023 06:26:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683725191; cv=none; d=google.com; s=arc-20160816; b=FkmbKJk1pCQzwdyX+5mURxUsw+PcW5ZZ6N5+/9RFCGz+cIhdyJjrFCppSND5cIVpG4 lWfRrK2HNjjo/bTCcGEAERuFBqaSK917KHmvhJuULLleiVEzLBNr9W6I+FimWoL95SAE SRlJBRgRFEqtThQZqu44u9G9vQPSMmMX89tGa0+yP633VAq7Xq5ElIAh49m7BqP2o6U4 DoB0et0Nfghr3BMmWs1+vg40dU0Q+qAgVaM2QJVYh8nk6anLaZukLyMlYCdvJnYiBCIK n8BAj3rbce/5m76aO6q6Ln1ati2IJ2lRm6rViouNx9tB6nssqgcV2mtjcMTQqlT9HasA Ku6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=KzSEkuOT4fK76AGbxXvu2za4uYJtXtxmttiJMrbYDTY=; b=vz6g3GTIvVQwOWhd6pCpjcS1HCHohqHRYk2nOdpGxqJVOnCopfcQcUDGIrEEP8QCFV KAefocqYaxv31Ev/Wu1cOkIy7+brmK3k2fUz0ql4ltHzkadnnuYVeoAm8B4yw/Va44hM cJ8NN6YTrBHR33DgLrZmyHwnuqOlffo+7JAwfPMVhkFnnhvYDHEIsK7hLhnajMvHdp21 ry130u2uIOMoDfFqMjRzGVitz2/avH3Ty4ff+bgHSCkG6vd+SLa3UE7IgykkYtIZkTby r3NosNwG+7ChkB3evj7n2JBdDlx8k7TNOGaB9nUwfnW9eMQ4bN8jUZEjZEBaNkwZ5znn qyig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o14-20020a17090a4e8e00b0025027e0ad3dsi13115238pjh.81.2023.05.10.06.26.16; Wed, 10 May 2023 06:26:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237038AbjEJNXs (ORCPT + 99 others); Wed, 10 May 2023 09:23:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236019AbjEJNXl (ORCPT ); Wed, 10 May 2023 09:23:41 -0400 Received: from andre.telenet-ops.be (andre.telenet-ops.be [IPv6:2a02:1800:120:4::f00:15]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A257961B7 for ; Wed, 10 May 2023 06:23:35 -0700 (PDT) Received: from ramsan.of.borg ([IPv6:2a02:1810:ac12:ed30:cc75:d8ef:4074:8af9]) by andre.telenet-ops.be with bizsmtp id v1PG2900T3l7qvk011PGsd; Wed, 10 May 2023 15:23:32 +0200 Received: from rox.of.borg ([192.168.97.57]) by ramsan.of.borg with esmtp (Exim 4.95) (envelope-from ) id 1pwjme-001nDl-DW; Wed, 10 May 2023 15:23:16 +0200 Received: from geert by rox.of.borg with local (Exim 4.95) (envelope-from ) id 1pwjmm-00F6qN-4j; Wed, 10 May 2023 15:23:16 +0200 From: Geert Uytterhoeven To: Stephen Boyd , Tomasz Figa , Sylwester Nawrocki , Will Deacon , Arnd Bergmann , Wolfram Sang , Dejin Zheng , Kai-Heng Feng , Nicholas Piggin , Heiko Carstens , Peter Zijlstra , Russell King , John Stultz , Thomas Gleixner , Tony Lindgren , Krzysztof Kozlowski , Tero Kristo , Ulf Hansson , "Rafael J . Wysocki" , Vincent Guittot Cc: linux-arm-kernel@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, Geert Uytterhoeven Subject: [PATCH v2 2/2] iopoll: Do not use timekeeping in read_poll_timeout_atomic() Date: Wed, 10 May 2023 15:23:14 +0200 Message-Id: <8db63020d18fc22e137e4a8f0aa15e6b9949a6f6.1683722688.git.geert+renesas@glider.be> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765513826447849718?= X-GMAIL-MSGID: =?utf-8?q?1765513826447849718?= read_poll_timeout_atomic() uses ktime_get() to implement the timeout feature, just like its non-atomic counterpart. However, there are several issues with this, due to its use in atomic contexts: 1. When called in the s2ram path (as typically done by clock or PM domain drivers), timekeeping may be suspended, triggering the WARN_ON(timekeeping_suspended) in ktime_get(): WARNING: CPU: 0 PID: 654 at kernel/time/timekeeping.c:843 ktime_get+0x28/0x78 Calling ktime_get_mono_fast_ns() instead of ktime_get() would get rid of that warning. However, that would break timeout handling, as (at least on systems with an ARM architectured timer), the time returned by ktime_get_mono_fast_ns() does not advance while timekeeping is suspended. Interestingly, (on the same ARM systems) the time returned by ktime_get() does advance while timekeeping is suspended, despite the warning. 2. Depending on the actual clock source, and especially before a high-resolution clocksource (e.g. the ARM architectured timer) becomes available, time may not advance in atomic contexts, thus breaking timeout handling. Fix this by abandoning the idea that one can rely on timekeeping to implement timeout handling in all atomic contexts, and switch from a global time-based to a locally-estimated timeout handling. In most (all?) cases the timeout condition is exceptional and an error condition, hence any additional delays due to underestimating wall clock time are irrelevant. Signed-off-by: Geert Uytterhoeven Acked-by: Arnd Bergmann Reviewed-by: Tony Lindgren Reviewed-by: Ulf Hansson --- Alternatively, one could use a mixed approach (use both ktime_get_mono_fast_ns() and a local (under)estimate, and timeout on the earliest occasion), but I think that would complicate things without much gain. v2: - New. --- include/linux/iopoll.h | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h index 0417360a6db9b0d6..bb2e1d9117e96679 100644 --- a/include/linux/iopoll.h +++ b/include/linux/iopoll.h @@ -81,22 +81,30 @@ delay_before_read, args...) \ ({ \ u64 __timeout_us = (timeout_us); \ + s64 __left_ns = __timeout_us * NSEC_PER_USEC; \ unsigned long __delay_us = (delay_us); \ - ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \ - if (delay_before_read && __delay_us) \ + u64 __delay_ns = __delay_us * NSEC_PER_USEC; \ + if (delay_before_read && __delay_us) { \ udelay(__delay_us); \ + if (__timeout_us) \ + __left_ns -= __delay_ns; \ + } \ for (;;) { \ (val) = op(args); \ if (cond) \ break; \ - if (__timeout_us && \ - ktime_compare(ktime_get(), __timeout) > 0) { \ + if (__timeout_us && __left_ns < 0) { \ (val) = op(args); \ break; \ } \ - if (__delay_us) \ + if (__delay_us) { \ udelay(__delay_us); \ + if (__timeout_us) \ + __left_ns -= __delay_ns; \ + } \ cpu_relax(); \ + if (__timeout_us) \ + __left_ns--; \ } \ (cond) ? 0 : -ETIMEDOUT; \ })