From patchwork Fri Nov 24 22:32:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 169599 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce62:0:b0:403:3b70:6f57 with SMTP id o2csp1580819vqx; Fri, 24 Nov 2023 14:33:48 -0800 (PST) X-Google-Smtp-Source: AGHT+IHTV9WHnCXczCEr41XoHgF7mRMSVuZto0DgELNPmWGVGwHTSUHCEt+Q+Cj+yQBjX8i5oWYx X-Received: by 2002:a17:902:dacc:b0:1cf:a70b:39b7 with SMTP id q12-20020a170902dacc00b001cfa70b39b7mr3606398plx.43.1700865228307; Fri, 24 Nov 2023 14:33:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700865228; cv=none; d=google.com; s=arc-20160816; b=JgZuvNz24Bc4gChJWnyoLKPem7FVYGrfMHnkUE31cKVC+czi/LnuPBbGTkJjZPjKf0 myH2PfYLkli5d4pP1kENKPLvXwbAPTr7NZ+DlBUQQFtyuEKF6p+EQm+okD9P0DwtQDbF belBQZYQ4QYe3c/yX+/C/XdzxRtusFQZE2DXgpBDnAt3+22fO6smE1Uu077kl1dkWNBl zUUGEAPviX6r20HKvbzN6lqu6NZkmLwzrUnvzpxXIivKiJVXJb8u3zC6nAXmfG8N3qHC Bpq1op7OA3ik0LYHavszXpDb6pC5eXh0inZIJ+6UeMnVzBdFfYBm1Vgp7ty8XLVV43C/ 70lw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4yOFnHb474rWjPxsHqMvh84l3hTm8YWaV2xwbY226lI=; fh=FP7vh/uLSL8T47BAINXnJeKRz+lJwSPdPP4Si4L9mlE=; b=sk7F31jzKwOuJO0zAm3z4fTLGsQ3/wzmmMZ/3n2I4opk66x8QwM9kIRKIkLSkoYwP1 1014iDJkFkCw1qYQvrY8yyxbXwq/yB76OrLCgIP62udk4tKz2hwfOS/OzkdG/YBLU5kj jSYM+4CEs02vRrK8DQ4/Gsl4F0/KTXFO27WOaQfqd22iJJaUGHxTEYHdPO+dzkw11zQe 00VN1Rx9jbffROMCakwcqASkvsOi6DapoCZaBQ87c79+XQqGotxoWT0urvIIVRIJ9M6G blVVxmkFyeJ3BZgWLnJ67kyGENBqTiL6zhgMJTSeqCryUvXzVtIb/A6gpTwwWO4lCk4y 15qg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LG5Q5EEg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id l10-20020a170903120a00b001cc31c32e0fsi4473579plh.91.2023.11.24.14.33.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Nov 2023 14:33:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=LG5Q5EEg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 67E1980C711C; Fri, 24 Nov 2023 14:33:45 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235330AbjKXWdW (ORCPT + 99 others); Fri, 24 Nov 2023 17:33:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231927AbjKXWdE (ORCPT ); Fri, 24 Nov 2023 17:33:04 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A45421BE2 for ; Fri, 24 Nov 2023 14:33:00 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32778C433CB; Fri, 24 Nov 2023 22:32:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700865180; bh=qkwnYSoqlBcfb84gH/3LY3O2XgUdAI1yXrt+QgVwpng=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LG5Q5EEgKX6cIW/WbSemc+jpnC4AVH6XFhH2IavHYRsaAdG4Yf+u7scHj7+B1U+6v xAy6ws8g/2QZl9G9qSIivZfAAFEPDvLaEuigQutSypLnKxUcY7qegY5zBggdPdK5mM z0/EUYh2gJ00egk/QgMjyJQAAnXHEv6Ubxa7m8wa9lXW2EnpwOYlhxQmPMusty05N7 tBwsmU262FNzth+F8s3JnhxTU7QRrb25NzWeROZxbquU5igA3thQ+tVAKccsE2imGA 4zsxcSoRgJUEWm/QC/O8cjb2+EireCnBmjStyIuN4gdVp1mYDxLynBoxICn14o/EXz JCheLVxxPDzAA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , "Rafael J . Wysocki" , Daniel Lezcano , linux-pm@vger.kernel.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra Subject: [PATCH 7/7] cpuidle: Handle TIF_NR_POLLING on behalf of software polling idle states Date: Fri, 24 Nov 2023 23:32:26 +0100 Message-ID: <20231124223226.24249-8-frederic@kernel.org> X-Mailer: git-send-email 2.42.1 In-Reply-To: <20231124223226.24249-1-frederic@kernel.org> References: <20231124223226.24249-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Fri, 24 Nov 2023 14:33:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783486457842109891 X-GMAIL-MSGID: 1783486457842109891 Software polling idle states set again TIF_NR_POLLING and clear it upon exit. This involves error prone duplicated code and wasted cycles performing atomic operations, sometimes RmW fully ordered. To avoid this, benefit instead from the same generic TIF_NR_POLLING handling that is currently in use for hardware polling states. Signed-off-by: Frederic Weisbecker --- drivers/cpuidle/cpuidle-powernv.c | 10 ---------- drivers/cpuidle/cpuidle-pseries.c | 11 ----------- drivers/cpuidle/cpuidle.c | 4 ++-- drivers/cpuidle/poll_state.c | 30 ++++++++++++------------------ 4 files changed, 14 insertions(+), 41 deletions(-) diff --git a/drivers/cpuidle/cpuidle-powernv.c b/drivers/cpuidle/cpuidle-powernv.c index 675b8eb81ebd..b88bbf7ead41 100644 --- a/drivers/cpuidle/cpuidle-powernv.c +++ b/drivers/cpuidle/cpuidle-powernv.c @@ -71,8 +71,6 @@ static int snooze_loop(struct cpuidle_device *dev, { u64 snooze_exit_time; - set_thread_flag(TIF_POLLING_NRFLAG); - local_irq_enable(); snooze_exit_time = get_tb() + get_snooze_timeout(dev, drv, index); @@ -81,21 +79,13 @@ static int snooze_loop(struct cpuidle_device *dev, HMT_very_low(); while (!need_resched()) { if (likely(snooze_timeout_en) && get_tb() > snooze_exit_time) { - /* - * Task has not woken up but we are exiting the polling - * loop anyway. Require a barrier after polling is - * cleared to order subsequent test of need_resched(). - */ - clear_thread_flag(TIF_POLLING_NRFLAG); dev->poll_time_limit = true; - smp_mb(); break; } } HMT_medium(); ppc64_runlatch_on(); - clear_thread_flag(TIF_POLLING_NRFLAG); local_irq_disable(); diff --git a/drivers/cpuidle/cpuidle-pseries.c b/drivers/cpuidle/cpuidle-pseries.c index 4e08c9a39172..0ae76512b740 100644 --- a/drivers/cpuidle/cpuidle-pseries.c +++ b/drivers/cpuidle/cpuidle-pseries.c @@ -39,8 +39,6 @@ int snooze_loop(struct cpuidle_device *dev, struct cpuidle_driver *drv, { u64 snooze_exit_time; - set_thread_flag(TIF_POLLING_NRFLAG); - pseries_idle_prolog(); raw_local_irq_enable(); snooze_exit_time = get_tb() + snooze_timeout; @@ -50,21 +48,12 @@ int snooze_loop(struct cpuidle_device *dev, struct cpuidle_driver *drv, HMT_low(); HMT_very_low(); if (likely(snooze_timeout_en) && get_tb() > snooze_exit_time) { - /* - * Task has not woken up but we are exiting the polling - * loop anyway. Require a barrier after polling is - * cleared to order subsequent test of need_resched(). - */ dev->poll_time_limit = true; - clear_thread_flag(TIF_POLLING_NRFLAG); - smp_mb(); break; } } HMT_medium(); - clear_thread_flag(TIF_POLLING_NRFLAG); - raw_local_irq_disable(); pseries_idle_epilog(); diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c index 49078cc83f4a..9eb811b5d8b6 100644 --- a/drivers/cpuidle/cpuidle.c +++ b/drivers/cpuidle/cpuidle.c @@ -236,8 +236,8 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev, broadcast = false; } - polling = target_state->flags & CPUIDLE_FLAG_POLLING_HARD; - + polling = (target_state->flags & (CPUIDLE_FLAG_POLLING_SOFT | + CPUIDLE_FLAG_POLLING_HARD)); /* * If the target state doesn't poll on need_resched(), this is * the last check after which further TIF_NEED_RESCHED remote setting diff --git a/drivers/cpuidle/poll_state.c b/drivers/cpuidle/poll_state.c index a2fe173de117..3bfa251b344a 100644 --- a/drivers/cpuidle/poll_state.c +++ b/drivers/cpuidle/poll_state.c @@ -13,35 +13,29 @@ static int __cpuidle poll_idle(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { - u64 time_start; - - time_start = local_clock_noinstr(); + u64 time_start = local_clock_noinstr(); + unsigned int loop_count = 0; + u64 limit; dev->poll_time_limit = false; raw_local_irq_enable(); - if (!current_set_polling_and_test()) { - unsigned int loop_count = 0; - u64 limit; - limit = cpuidle_poll_time(drv, dev); + limit = cpuidle_poll_time(drv, dev); - while (!need_resched()) { - cpu_relax(); - if (loop_count++ < POLL_IDLE_RELAX_COUNT) - continue; + while (!need_resched()) { + cpu_relax(); + if (loop_count++ < POLL_IDLE_RELAX_COUNT) + continue; - loop_count = 0; - if (local_clock_noinstr() - time_start > limit) { - dev->poll_time_limit = true; - break; - } + loop_count = 0; + if (local_clock_noinstr() - time_start > limit) { + dev->poll_time_limit = true; + break; } } raw_local_irq_disable(); - current_clr_polling(); - return index; }