From patchwork Fri Mar 3 09:23:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63836 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp315075wrd; Fri, 3 Mar 2023 01:33:25 -0800 (PST) X-Google-Smtp-Source: AK7set+1GWWoKK+TXZKGHqMnMO2BEaZrYOjlSMA2v0hW+s7I/sTxp2Xp/TaFYv8L9SgT1egLh0IT X-Received: by 2002:a17:907:7d8d:b0:8b1:387b:c0c8 with SMTP id oz13-20020a1709077d8d00b008b1387bc0c8mr1281347ejc.15.1677836005700; Fri, 03 Mar 2023 01:33:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836005; cv=none; d=google.com; s=arc-20160816; b=pLOdNE6JBqrZySoBUlOHrRvhGdarlzCyOWA9SAPLx4KxrYoJoTygW60vOtZKrm5AyP GdyWjrvmndpzbsV8TsMT+608g/sH6AhagGF49EBd74qnXe+L9+FiOdu/RFCMEvZfujrm F0Supaz+eCsfERfPv9V7ylCQ2tb2dJftnshS3ACbZBxM22+UERRPL4xl0ymlXN2Es7Sn N3VJgpBSGsztl0gHTTGLJ1f5Q2Vi1wSl1/frWn3HqzrhRnSx6XFGIEGv8F9VjvveLIpa hGShaKnpFCmDgtW9d8pWOOHf21DTc2AmHslM5uUUQHLu7FAzv5vhTlvXoEjti5WApQuS CMJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=euxPKuYLhPTVh9ngivHIKL/kZ3tV5PNQ27t0ey9mzGs=; b=I9wv8V0Gm4X7hHeWcbsnweabfyyLmWjsR7k4+Drx8p+ab6SDGiu+GJlvc0ImzULJkw yxaXaYhUUjl/ubsPym3VJHhw5OqBgQ9dnGN3hQa/ldVE2wFcWIll+I1ILMm5dpYxtExh QxSXZkartK7pTuhxQcfq0qUTjamV9h4m7ArXEofxieH3Ek8/tW6JXMpqmvcq3BXyFdu9 rtVhK7cTOFD/OVdFNqM4p/oe/E6HQym74JZaOqKV7IuzuDsNvemWK8vc154F6SvlW4cc ZgLgkBHnuU/xBu66JE7dmMJbaaPJ+OAXk2eJ45oRq1Sz+4SYY6V2cOmre28ieSVdIckE fknw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=ifOapB4Y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nd32-20020a17090762a000b008da033ef977si2325337ejc.731.2023.03.03.01.33.03; Fri, 03 Mar 2023 01:33:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=ifOapB4Y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230513AbjCCJ0F (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229673AbjCCJZN (ORCPT ); Fri, 3 Mar 2023 04:25:13 -0500 Received: from mailgw01.mediatek.com (unknown [60.244.123.138]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEB4B2D17D; Fri, 3 Mar 2023 01:24:55 -0800 (PST) X-UUID: 3e51e5beb9a511eda06fc9ecc4dadd91-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=euxPKuYLhPTVh9ngivHIKL/kZ3tV5PNQ27t0ey9mzGs=; b=ifOapB4YJ68gvsnZvKOXsgNfmdo11uRntI84LZS4RxtCNwwJ7HhLFGHSitFO/rKzTkdjRWoSxFgXXD7qQAfChMdaU1DNKEaupbTZpJUyF+lGGm1TvyXoilXARIJK6MUsbW9nf054S72zTzGGGvh8esVexr8WOGGcBl5lFDgaz34=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:61787ae0-d8c9-428b-aa6a-f15d629a3074,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:456babf4-ddba-41c3-91d9-10eeade8eac7,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-UUID: 3e51e5beb9a511eda06fc9ecc4dadd91-20230303 Received: from mtkmbs11n1.mediatek.inc [(172.21.101.185)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1092691571; Fri, 03 Mar 2023 17:24:48 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs13n2.mediatek.inc (172.21.101.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:24:47 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:24:47 +0800 From: Cheng-Jui Wang To: , "Rafael J. Wysocki" , Daniel Lezcano , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , Subject: [PATCH 01/10] cpuidle/poll: Ensure IRQs stay disabled after cpuidle_state::enter() calls Date: Fri, 3 Mar 2023 17:23:23 +0800 Message-ID: <20230303092347.4825-2-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338567707889485?= X-GMAIL-MSGID: =?utf-8?q?1759338567707889485?= From: Peter Zijlstra commit 5e26aa93391195a64871db5d96d7163f0062ca4f upstream. Make cpuidle_state::enter() methods IRQ state invariant on exit. Additionally make sure to use raw_local_irq_*() methods since this cpuidle callback will be called with RCU already disabled. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Rafael J. Wysocki Reviewed-by: Frederic Weisbecker Link: https://lore.kernel.org/r/20230112195539.515253662@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- drivers/cpuidle/poll_state.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/cpuidle/poll_state.c b/drivers/cpuidle/poll_state.c index f7e83613ae94..1f578ed09c73 100644 --- a/drivers/cpuidle/poll_state.c +++ b/drivers/cpuidle/poll_state.c @@ -17,7 +17,7 @@ static int __cpuidle poll_idle(struct cpuidle_device *dev, dev->poll_time_limit = false; - local_irq_enable(); + raw_local_irq_enable(); if (!current_set_polling_and_test()) { unsigned int loop_count = 0; u64 limit; @@ -36,6 +36,8 @@ static int __cpuidle poll_idle(struct cpuidle_device *dev, } } } + raw_local_irq_disable(); + current_clr_polling(); return index; From patchwork Fri Mar 3 09:23:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63835 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp314868wrd; Fri, 3 Mar 2023 01:33:03 -0800 (PST) X-Google-Smtp-Source: AK7set9jXJabgG2UMJbtrE7dY1JVM3EHLFME9Yw25XxnLCF0uqKAAzqg92HdndzjoaELOrU4hHPb X-Received: by 2002:a17:906:6bc1:b0:8b1:304e:58a3 with SMTP id t1-20020a1709066bc100b008b1304e58a3mr867650ejs.52.1677835983075; Fri, 03 Mar 2023 01:33:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677835983; cv=none; d=google.com; s=arc-20160816; b=LosQaWKPXisoauagVi08ba2am4b+ojt6rsRVignbjoiEsZSAY8PDdQ+XrkfIvJsyYx 7b6p5uWe+7q0X25+cA1ax3GUuZuTngiPOJet+9f1zAtBUSQ7X+ksR3kZDLTqvd8gcqga ln0WLgOdEaQOqsENO9Njd4shnDFNn/slTBBsRWT6JJs0V3J+qbr9KZljLPr+SD/jViUy 9Q82D+xrjygkqfn8ibwbzlZqpdIxWeV6+wHBjHKOMpaEUL8ifbPg7ApjmIYRBNn0rOxu WeZlxjyR9/snm0ypqKWpMCTb/F6tQsLLJprVolFJedv1b5Ryai6k+YfzzEyKzWY1CT+j xCSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ksYXpcLRpyhSF2BCe6z2U0gcUedKGDwO8T2OIU6P68o=; b=lGFStd3wSYvDdNe+o/BYjWOxpuIoT87nr3wMgqdVHkXflGsPGz9iE/ZbNEAPH3hFAB 5+mgCIJJvA/BtGm9iWVuWrK8R22jcSS7ezYKQFlT/ymRaQmeQq/1kyn1REBEXby3HFr1 o5EkJ7k0PVO2qKJicpvAcwc5y+CklIOae1cTyYRFkDGQvmW0bjJJ83Wz+wmy5JELkLfQ arv2yDv8wA0AvY92j2f1+4GKI5a4WzevGJcJMDe+kv4VKCWRAg1sUlQ6jlndx2Eoi+Z/ YknXou20GZatRZPJCr9DjXql4P9eQQ5vaD9gffAVMFSy92ghWqvQtjfb4WbACmpZiGXs fAMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=nZkwvpzB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w8-20020a170906184800b008b17b235680si1875861eje.131.2023.03.03.01.32.40; Fri, 03 Mar 2023 01:33:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=nZkwvpzB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230407AbjCCJ0A (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231201AbjCCJZj (ORCPT ); Fri, 3 Mar 2023 04:25:39 -0500 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02FE657D24; Fri, 3 Mar 2023 01:25:17 -0800 (PST) X-UUID: 4be222acb9a511ed945fc101203acc17-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=ksYXpcLRpyhSF2BCe6z2U0gcUedKGDwO8T2OIU6P68o=; b=nZkwvpzBLeBHTWsRQGIM2m0jnQeaCbo/drr3iLgfi6ylcl5lvp6/HCCB/Z9+pLkH2JeQtZL0D0jDqIB/9RJ0XNp5pX2QrCSzSsw4tOz1Wd1mrKLIAzRucVV6i9Sw8lj2/fAE1xALggFOOJzGs6V4CTr8cx1UFslxo9sBmprh2E8=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:729e9142-281b-4380-bde8-04e35560c591,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:53,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:28 X-CID-INFO: VERSION:1.1.20,REQID:729e9142-281b-4380-bde8-04e35560c591,IP:0,URL :0,TC:0,Content:-25,EDM:0,RT:0,SF:53,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:28 X-CID-META: VersionHash:25b5999,CLOUDID:ef273d27-564d-42d9-9875-7c868ee415ec,B ulkID:2303031725130DEM1UT0,BulkQuantity:0,Recheck:0,SF:38|29|28|100|17|19| 48|101|102,TC:nil,Content:0,EDM:-3,IP:nil,URL:1,File:nil,Bulk:nil,QS:nil,B EC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 4be222acb9a511ed945fc101203acc17-20230303 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 860502259; Fri, 03 Mar 2023 17:25:11 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs13n1.mediatek.inc (172.21.101.193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:10 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:09 +0800 From: Cheng-Jui Wang To: , Anup Patel , "Rafael J. Wysocki" , Daniel Lezcano , Paul Walmsley , Palmer Dabbelt , Albert Ou , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , , Subject: [PATCH 02/10] cpuidle, riscv: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:24 +0800 Message-ID: <20230303092347.4825-3-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338543589206384?= X-GMAIL-MSGID: =?utf-8?q?1759338543589206384?= From: Peter Zijlstra commit 8e9ab9e8da1eae61fdff35690d998eaf8cd527dc upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again, at least twice, before going idle is suboptimal. That is, once implicitly through the cpu_pm_*() calls and once explicitly doing ct_irq_*_irqon(). Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Anup Patel Reviewed-by: Frederic Weisbecker Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195539.637185846@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- drivers/cpuidle/cpuidle-riscv-sbi.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/drivers/cpuidle/cpuidle-riscv-sbi.c b/drivers/cpuidle/cpuidle-riscv-sbi.c index 05fe2902df9a..cbdbb11b972b 100644 --- a/drivers/cpuidle/cpuidle-riscv-sbi.c +++ b/drivers/cpuidle/cpuidle-riscv-sbi.c @@ -121,12 +121,12 @@ static int __sbi_enter_domain_idle_state(struct cpuidle_device *dev, return -1; /* Do runtime PM to manage a hierarchical CPU toplogy. */ - ct_irq_enter_irqson(); if (s2idle) dev_pm_genpd_suspend(pd_dev); else pm_runtime_put_sync_suspend(pd_dev); - ct_irq_exit_irqson(); + + ct_idle_enter(); if (sbi_is_domain_state_available()) state = sbi_get_domain_state(); @@ -135,12 +135,12 @@ static int __sbi_enter_domain_idle_state(struct cpuidle_device *dev, ret = sbi_suspend(state) ? -1 : idx; - ct_irq_enter_irqson(); + ct_idle_exit(); + if (s2idle) dev_pm_genpd_resume(pd_dev); else pm_runtime_get_sync(pd_dev); - ct_irq_exit_irqson(); cpu_pm_exit(); @@ -251,6 +251,7 @@ static int sbi_dt_cpu_init_topology(struct cpuidle_driver *drv, * of a shared state for the domain, assumes the domain states are all * deeper states. */ + drv->states[state_count - 1].flags |= CPUIDLE_FLAG_RCU_IDLE; drv->states[state_count - 1].enter = sbi_enter_domain_idle_state; drv->states[state_count - 1].enter_s2idle = sbi_enter_s2idle_domain_idle_state; From patchwork Fri Mar 3 09:23:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63837 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp315991wrd; Fri, 3 Mar 2023 01:35:27 -0800 (PST) X-Google-Smtp-Source: AK7set9ix8UnPJgv3F183d2Zh3vLM8Gx/EwZZf3fiY5u+VVsV9pE980evaFDlNqJODnPyL2yB81h X-Received: by 2002:a17:906:6d85:b0:8c3:3439:24d9 with SMTP id h5-20020a1709066d8500b008c3343924d9mr834149ejt.24.1677836127205; Fri, 03 Mar 2023 01:35:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836127; cv=none; d=google.com; s=arc-20160816; b=Igl4GJsKfHYZAw8IUOHT2coHkVoVD+NPN2lv24pASUYImxfrP8LjqOVuKWdNX7vVz0 ajWk8FQAB71MpAviZyJ/QeWTEle+FuYH5N+fgVHoKprMRpi2E2l4Kda0ARLbYQS5GeZu 9KSrrHStEk4bOvA2KGT82qOZy0llG/7XMYxPILXlkbov4XOBDWz1iQXeza4+/I7zQbRe 63q5XxJG/F8XbGzMiFjqix3V/0lGn3qXA5djaILXzjjN+1t7/EPjyrIBA38BLvQ4N7Kr tQ5sQhnFT8hw7mJ9HGv0atclmMPllEDfh1nv0DwUPPue5c7S94D1ZBK43tdGRfyJRAcD 695A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=bSqog9Arn5WTEwlshb3YMHHP4Vbkyic4vpImRhlxN2s=; b=WfYT/oryjyvSLKkGHi5ZZZN8rycTPjOeYea9zU7VQ7/HswZRTPv8yy01u8tUoPjDpT DYZKDqKHAnzz/itNowla8YlcU8WnCbi8aPGcDpR95gCl7QwoEDc7etwP7BryvBxGqzt/ qUA2Ta4hunUI9/0TGEnSp2j3bt3zliNefgsjrmCR90TqZOwt4tuozXistPk+I7uc/N3L gh5UmAItLk19d58e1KvIKJ9VbFCd4srDzjamQJIcIozaxYKT+GkLfwLTqVXhgrd/hUoV 90jcnnY1Mu6TbnheJRbaZjBpaONZZwHx5Ok9ZIzGnWyENhXl+2ujM3Y2iyX71Ltki0JM lyfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=QoDQZZsG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hs32-20020a1709073ea000b008daada6238dsi610737ejc.64.2023.03.03.01.35.04; Fri, 03 Mar 2023 01:35:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=QoDQZZsG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229925AbjCCJ0E (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231205AbjCCJZk (ORCPT ); Fri, 3 Mar 2023 04:25:40 -0500 Received: from mailgw01.mediatek.com (unknown [60.244.123.138]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8D6E47410; Fri, 3 Mar 2023 01:25:18 -0800 (PST) X-UUID: 4d9ce2b2b9a511eda06fc9ecc4dadd91-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=bSqog9Arn5WTEwlshb3YMHHP4Vbkyic4vpImRhlxN2s=; b=QoDQZZsGQTgmBo/JH8zHNLrEWXnnB/v1Fw8/8F/Yi3KOuO8sjOZtM6dfeqXLerob30RKTF2QGUmPnpxt7g7Ub7PVxwBksizZeOMyHjjGJ+NYudFv5qYv1rKfu397hBSQaxp2ctTMBV28TPYJ1GLqAGfkG6aeTWxZUcfZcAN3D08=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:959b73a5-cab3-4986-8e8b-e46402f9969d,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:0c70abf4-ddba-41c3-91d9-10eeade8eac7,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 4d9ce2b2b9a511eda06fc9ecc4dadd91-20230303 Received: from mtkmbs11n1.mediatek.inc [(172.21.101.185)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1005376028; Fri, 03 Mar 2023 17:25:14 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:13 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:12 +0800 From: Cheng-Jui Wang To: , "Rafael J. Wysocki" , "Daniel Lezcano" , Thierry Reding , Jonathan Hunter , "Matthias Brugger" CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , , Subject: [PATCH 03/10] cpuidle, tegra: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:25 +0800 Message-ID: <20230303092347.4825-4-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338694792906621?= X-GMAIL-MSGID: =?utf-8?q?1759338694792906621?= From: Peter Zijlstra commit 5fca0d9f5d76664786ca6c09076341def165a677 upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again, at least twice, before going idle is suboptimal. Notably once implicitly through the cpu_pm_*() calls and once explicitly doing RCU_NONIDLE(). Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Frederic Weisbecker Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195539.699546331@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- drivers/cpuidle/cpuidle-tegra.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/cpuidle/cpuidle-tegra.c b/drivers/cpuidle/cpuidle-tegra.c index 9845629aeb6d..3ca5cfb9d322 100644 --- a/drivers/cpuidle/cpuidle-tegra.c +++ b/drivers/cpuidle/cpuidle-tegra.c @@ -180,9 +180,11 @@ static int tegra_cpuidle_state_enter(struct cpuidle_device *dev, } local_fiq_disable(); - RCU_NONIDLE(tegra_pm_set_cpu_in_lp2()); + tegra_pm_set_cpu_in_lp2(); cpu_pm_enter(); + ct_idle_enter(); + switch (index) { case TEGRA_C7: err = tegra_cpuidle_c7_enter(); @@ -197,8 +199,10 @@ static int tegra_cpuidle_state_enter(struct cpuidle_device *dev, break; } + ct_idle_exit(); + cpu_pm_exit(); - RCU_NONIDLE(tegra_pm_clear_cpu_in_lp2()); + tegra_pm_clear_cpu_in_lp2(); local_fiq_enable(); return err ?: index; @@ -226,6 +230,7 @@ static int tegra_cpuidle_enter(struct cpuidle_device *dev, struct cpuidle_driver *drv, int index) { + bool do_rcu = drv->states[index].flags & CPUIDLE_FLAG_RCU_IDLE; unsigned int cpu = cpu_logical_map(dev->cpu); int ret; @@ -233,9 +238,13 @@ static int tegra_cpuidle_enter(struct cpuidle_device *dev, if (dev->states_usage[index].disable) return -1; - if (index == TEGRA_C1) + if (index == TEGRA_C1) { + if (do_rcu) + ct_idle_enter(); ret = arm_cpuidle_simple_enter(dev, drv, index); - else + if (do_rcu) + ct_idle_exit(); + } else ret = tegra_cpuidle_state_enter(dev, index, cpu); if (ret < 0) { @@ -285,7 +294,8 @@ static struct cpuidle_driver tegra_idle_driver = { .exit_latency = 2000, .target_residency = 2200, .power_usage = 100, - .flags = CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE, .name = "C7", .desc = "CPU core powered off", }, @@ -295,6 +305,7 @@ static struct cpuidle_driver tegra_idle_driver = { .target_residency = 10000, .power_usage = 0, .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE | CPUIDLE_FLAG_COUPLED, .name = "CC6", .desc = "CPU cluster powered off", From patchwork Fri Mar 3 09:23:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63841 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp316768wrd; Fri, 3 Mar 2023 01:37:20 -0800 (PST) X-Google-Smtp-Source: AK7set8mU04nFL4t3Eg1yLuoz2Sm3tZcfGwzo6kqtKbaJjj6j8UEsxpq7CKl6GbIGJ1RbTy52nU4 X-Received: by 2002:a17:906:1c13:b0:896:427b:148 with SMTP id k19-20020a1709061c1300b00896427b0148mr933019ejg.1.1677836240644; Fri, 03 Mar 2023 01:37:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836240; cv=none; d=google.com; s=arc-20160816; b=lYsDjmadujcccqs3GiHVqE1D5cC47b5KuVY6GLQCAvv/9Oj66vMQ48BOWg/aWFOs+o C+rLFCm5TOFS2gyZ5JwyVMu3WnYyrdebYm3GeoVXvimLUmMKfJGsVBvfAYkll6AREwFP iW/DAHOUMc+9W/jTQ1AyaWJF9wr4pB5IUO0GePcSPDDwRDIbH2Es04yQUZzQWii8QtEx 7FWc3lGW33YZ/t3aFzPZbTBBPEw6klivSdTagTEIsweKquIuDEjvVBbsP+wbeNocBwBS 8viWWoZmChQBXzP78kWe0EWAnJwaXVjpISHx3iXoOQBVFuP2AaBIG3XKIwAK9h1UAJp9 VmJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=bjh2a7FP8kwe5353qnWJfqpDLdsQ8XYJqlQFAAUXlZs=; b=piSPAOk7bxuwiE5KxRAR3AhzH3jLLW1pzbPXIcP8ghENrulIq6ZD1g7cur1D7ul4J+ PCOw/Z3VZSFceeZVuT1TosbpsI86MLG4CKfG9zR3YpDV46XboYEaEiO6took7Jz8YBQs 9noFEjP/dKV/z+zCr4bkFGzoIMZm/VLk72CdDHcm8N9ntu1pRjCI1NtNn+Pd9Hx2xMHp sC6Efu22zqkeXOb1WLH4xFcmytEov7GBzovwiBKUOaENRO77Cnq4DBBuLLT5oFpZC60T q8/8AC3R1JCww1c59/EJtUd70eWLcbZKCJSCAD6qpA5X1UJA59Zu2II3UaUuQ/U8tmvj 5Hgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=DS3yt4Mq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m11-20020a50ef0b000000b004aac683bad9si70632eds.379.2023.03.03.01.36.58; Fri, 03 Mar 2023 01:37:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=DS3yt4Mq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbjCCJ0d (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231238AbjCCJZp (ORCPT ); Fri, 3 Mar 2023 04:25:45 -0500 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D0305C116; Fri, 3 Mar 2023 01:25:24 -0800 (PST) X-UUID: 504404d2b9a511ed945fc101203acc17-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=bjh2a7FP8kwe5353qnWJfqpDLdsQ8XYJqlQFAAUXlZs=; b=DS3yt4MqRBr1Nu853NpZGTU+eZrENVxT7XlziQ9WKScgNUCLyMx/9YvcxeY6MoWzP3+8id+ALp96cfknQItgeCJXqCJrUQRxJmXCwk2FWMjoKYR//Zp/0ooFA1sbMR0MZpLVX41+4ero/QvziE++1V7fawvYyxs6KagJ2M6c+74=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:b37d915a-7f8f-4d3c-a56a-f2d7a436cb16,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:18be1db2-beed-4dfc-bd9c-e1b22fa6ccc4,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 504404d2b9a511ed945fc101203acc17-20230303 Received: from mtkmbs13n2.mediatek.inc [(172.21.101.108)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1507035352; Fri, 03 Mar 2023 17:25:18 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:17 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:17 +0800 From: Cheng-Jui Wang To: , Lorenzo Pieralisi , Sudeep Holla , "Rafael J. Wysocki" , Daniel Lezcano , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , Subject: [PATCH 04/10] cpuidle, psci: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:26 +0800 Message-ID: <20230303092347.4825-5-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338813841645268?= X-GMAIL-MSGID: =?utf-8?q?1759338813841645268?= From: Peter Zijlstra commit e038f7b8028a1d1bc8ac82351c71ea538f19a879 upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again, at least twice, before going idle is suboptimal. Notably once implicitly through the cpu_pm_*() calls and once explicitly doing ct_irq_*_irqon(). Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Kajetan Puchalski Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Frederic Weisbecker Reviewed-by: Guo Ren Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195539.760296658@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- drivers/cpuidle/cpuidle-psci.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 57bc3e3ae391..6a259c94b8b2 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -69,12 +69,12 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, return -1; /* Do runtime PM to manage a hierarchical CPU toplogy. */ - ct_irq_enter_irqson(); if (s2idle) dev_pm_genpd_suspend(pd_dev); else pm_runtime_put_sync_suspend(pd_dev); - ct_irq_exit_irqson(); + + ct_idle_enter(); state = psci_get_domain_state(); if (!state) @@ -82,13 +82,12 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, ret = psci_cpu_suspend_enter(state) ? -1 : idx; - ct_irq_enter_irqson(); + ct_idle_exit(); + if (s2idle) dev_pm_genpd_resume(pd_dev); else pm_runtime_get_sync(pd_dev); - ct_irq_exit_irqson(); - cpu_pm_exit(); /* Clear the domain state to start fresh when back from idle. */ @@ -240,6 +239,7 @@ static int psci_dt_cpu_init_topology(struct cpuidle_driver *drv, * of a shared state for the domain, assumes the domain states are all * deeper states. */ + drv->states[state_count - 1].flags |= CPUIDLE_FLAG_RCU_IDLE; drv->states[state_count - 1].enter = psci_enter_domain_idle_state; drv->states[state_count - 1].enter_s2idle = psci_enter_s2idle_domain_idle_state; psci_cpuidle_use_cpuhp = true; From patchwork Fri Mar 3 09:23:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63842 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp316773wrd; Fri, 3 Mar 2023 01:37:21 -0800 (PST) X-Google-Smtp-Source: AK7set/F6RLHMQ8izaYR7A0kiZp+JdsfyY7AuR0b5eqqPfLbrrcRo8UgStknED82/t+ZgwQMyIBv X-Received: by 2002:a17:907:6eac:b0:8b1:2d7f:ed57 with SMTP id sh44-20020a1709076eac00b008b12d7fed57mr1161351ejc.42.1677836241412; Fri, 03 Mar 2023 01:37:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836241; cv=none; d=google.com; s=arc-20160816; b=q6TAw6BcTHgRrPUS+e2xb61J4vFKB152/m4idXNGaTdjBg1A9vUiSGZrN6n52wjEGZ OiyctugkuQBpydfXVEbjA/eAs+rRGDAYNZSugY74ceD9A8g6ur2ZjhlRTT4zRFOz4osp 0Zz/PBpJwaALXFyXNNwWdJDsvRSu3VagEj2o48i1aN0L0pn9hOBpTuCkI1+zz3z3lnNg SPkb3gDKhx3POA7C98gG7H31Ab8WA19h5gKxqO8FQnF05NqZwKP3JW3xztTAQr46nq2n 9GjGaa1mQwNVGmU1a56veV4o3/+3RGQZw0N10pIwGtVZuN24S0gsFxIpxguPcugxNQrb zLtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Sh4Pn9fpBEAZmsI3UxrbkJdlT6TxlulP3ExKBV+rbeM=; b=N+SRqHFdE0F1p4LIGgP8pdk5xngWW2lbAI2YNvNlLzZdVWmAbcVPLMILjvmegGRw3o H3ykirEIjYzOfJYNERqTUGDM21NaqDmgXaa0xixfe1a+nrDbOQvkhJzAF3BLUMFhYx+p MxC+Uqi6T39PZ/IHxctY3d7ftXFHcScPO5QaDyYpqKSCyAPUGF77yaDp1pVeU2CADAOW TReLn7BTLpJ1pFif56uvUj4Krqf6VlvB1+NV2X5VBiaxpMLAtKUsM3oR0Ho1uch0x5Pe mg6YosRT5YkqQ34CS0q1e+tiFas+yYxH5prguOQ3I5Pr5pIpiy/4/FYBgb5BnmD9IS3p vxlQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=DtqdSYiV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gz27-20020a170907a05b00b008eb6f52bf2bsi1854364ejc.936.2023.03.03.01.36.59; Fri, 03 Mar 2023 01:37:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=DtqdSYiV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230497AbjCCJ0f (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231240AbjCCJZq (ORCPT ); Fri, 3 Mar 2023 04:25:46 -0500 Received: from mailgw01.mediatek.com (unknown [60.244.123.138]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAE805B5F5; Fri, 3 Mar 2023 01:25:27 -0800 (PST) X-UUID: 5200c634b9a511eda06fc9ecc4dadd91-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=Sh4Pn9fpBEAZmsI3UxrbkJdlT6TxlulP3ExKBV+rbeM=; b=DtqdSYiVQFX3WU/Da4SoReNG6WVPTWDFXttgPCpyCGG5z5Qh/hJQ0NmFmSTxu+00ZEC+f2OqQz7mxhMi7VT5fZ3D4KdWV/0zIShfylGqaItL96emrzAJrel2LYwY6EyEq1jX8bYy2ezv7uCQncQz0fkhgG1iuIzKw8waw8X+clc=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:58504ac7-4e0f-47dd-aaa5-6cbe8455e7fb,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:eabe1db2-beed-4dfc-bd9c-e1b22fa6ccc4,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 5200c634b9a511eda06fc9ecc4dadd91-20230303 Received: from mtkmbs11n2.mediatek.inc [(172.21.101.187)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 243878210; Fri, 03 Mar 2023 17:25:21 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:20 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:20 +0800 From: Cheng-Jui Wang To: , Russell King , Shawn Guo , Sascha Hauer , "Pengutronix Kernel Team" , Fabio Estevam , "NXP Linux Team" , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , Subject: [PATCH 05/10] cpuidle, ARM/imx6: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:27 +0800 Message-ID: <20230303092347.4825-6-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338814366589638?= X-GMAIL-MSGID: =?utf-8?q?1759338814366589638?= From: Peter Zijlstra commit b3f46658ce40a3467cda82f920dd9d5325ab0eaf upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again, at least twice, before going idle is suboptimal. Notably both cpu_pm_enter() and cpu_cluster_pm_enter() implicity re-enable RCU. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Frederic Weisbecker Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195539.821714572@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- arch/arm/mach-imx/cpuidle-imx6sx.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/arm/mach-imx/cpuidle-imx6sx.c b/arch/arm/mach-imx/cpuidle-imx6sx.c index 74ea1720e3d8..1dc01f6b0f36 100644 --- a/arch/arm/mach-imx/cpuidle-imx6sx.c +++ b/arch/arm/mach-imx/cpuidle-imx6sx.c @@ -47,7 +47,9 @@ static int imx6sx_enter_wait(struct cpuidle_device *dev, cpu_pm_enter(); cpu_cluster_pm_enter(); + ct_idle_enter(); cpu_suspend(0, imx6sx_idle_finish); + ct_idle_exit(); cpu_cluster_pm_exit(); cpu_pm_exit(); @@ -87,7 +89,8 @@ static struct cpuidle_driver imx6sx_cpuidle_driver = { */ .exit_latency = 300, .target_residency = 500, - .flags = CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE, .enter = imx6sx_enter_wait, .name = "LOW-POWER-IDLE", .desc = "ARM power off", From patchwork Fri Mar 3 09:23:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63844 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp316903wrd; Fri, 3 Mar 2023 01:37:47 -0800 (PST) X-Google-Smtp-Source: AK7set9p9QBEX4MGAkbGRODb1V+VUIN1fwkiuy3/F4ZsZaArazYGWw/UEVMYcjpJBQPvczqKvcgv X-Received: by 2002:a17:906:2da2:b0:8ae:f1cd:9551 with SMTP id g2-20020a1709062da200b008aef1cd9551mr813273eji.76.1677836267435; Fri, 03 Mar 2023 01:37:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836267; cv=none; d=google.com; s=arc-20160816; b=ZnYQ89X1K5rm3b84LCH5Znkm9Y2RpnKy24FNqSKEHzgzVPWUWP18DikX0dWFZ96ZDx lWWLfMsAeyw96sVwxDB3wKjxUBiaj/4EP1wy55FwD+t7Ye/8wNaJj3i9LlHykcEA1aYR 2t1msIbhZjqRqwkBmvKSWFqWbhMIENZaKZOXZUNJxdhN3zGyVsSLTe6dOCRFFTMFb4St fMTVjmOhW1eBBIXqjlGYRwEFIH2KCL+M72im3mzvZY+pwXKW/RHV1j12qe0kmJgLlbu2 QGIjiQDb8pcZbCdICD/xdzOh/ZAzDo3llqYUloltpXNo2MmPZqt0rT7Kzf/lwMgtZiPl RzZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=33Xnh9Ic4FVkYsy5QR78PaQynE0bkdH1vhnuUL215cI=; b=GFLtMSTkgSfOFZu+2zdNxcwnALLJ2b8RHg2F4Cd/DTta/15gMzPzWPF1d+tL0xu28J CjlmMaGCgh9RMfsIyrgGfi9lQEnXiGN5SIFjxKe6ZI7PuHbu3Xqs5AKdNMyl4+34fLxt vCE83/ONrq0efJljP0yRCl9suWB3Q+/AGWqyG1qivUZADlz2am9PHixrFQTpqHnDrx3z FSpT9PNAIhec1MiyoK9H3UpeDgSDkRPSj7G9YkkBbXDQPv0qrjfzXaNIIxLSxg9UchEP BkDgaRyKWzUlkgC2+bWlQh7P1xYmzk75Rre+B8Cuy3+cWCpKWHeknlLxp2B5PqbxnJUd dEAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=tlJWr6sO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id le4-20020a170907170400b009087dc51377si1678248ejc.903.2023.03.03.01.37.25; Fri, 03 Mar 2023 01:37:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=tlJWr6sO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231184AbjCCJ0k (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229750AbjCCJZx (ORCPT ); Fri, 3 Mar 2023 04:25:53 -0500 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 028F65C128; Fri, 3 Mar 2023 01:25:33 -0800 (PST) X-UUID: 54b7d962b9a511ed945fc101203acc17-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=33Xnh9Ic4FVkYsy5QR78PaQynE0bkdH1vhnuUL215cI=; b=tlJWr6sOcO1yIvNnh7cl6RaFciYm0YjZJM4c55SvCgzaNv+8KKOXlrxZnUm4II/nRCT4BdVj+oQoAVxJ2EP9chpAnBMAU6YqWlvQBVzWJOEZngclBTq/KzJQeWyf0NTbi4BfW2Z5kpAfa89DGjFebx6HsgG3mHjWw37z95P27xo=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:78831060-8048-48cc-bfee-01ceaa29cd85,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:d672abf4-ddba-41c3-91d9-10eeade8eac7,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 54b7d962b9a511ed945fc101203acc17-20230303 Received: from mtkmbs13n2.mediatek.inc [(172.21.101.108)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1637940446; Fri, 03 Mar 2023 17:25:26 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:24 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:24 +0800 From: Cheng-Jui Wang To: , Tony Lindgren , Russell King , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , Subject: [PATCH 06/10] cpuidle, OMAP3: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:28 +0800 Message-ID: <20230303092347.4825-7-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338842045805988?= X-GMAIL-MSGID: =?utf-8?q?1759338842045805988?= From: Peter Zijlstra commit 4d1be9e745382b41492b0cb9000829863db7133a upstream. Doing RCU-idle outside the driver, only to then teporarily enable it again before going idle is suboptimal. Notably the cpu_pm_*() calls implicitly re-enable RCU for a bit. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Frederic Weisbecker Reviewed-by: Tony Lindgren Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195539.883561913@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- arch/arm/mach-omap2/cpuidle34xx.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/arm/mach-omap2/cpuidle34xx.c b/arch/arm/mach-omap2/cpuidle34xx.c index 090a8aafb25e..cedf5cbe451f 100644 --- a/arch/arm/mach-omap2/cpuidle34xx.c +++ b/arch/arm/mach-omap2/cpuidle34xx.c @@ -133,7 +133,9 @@ static int omap3_enter_idle(struct cpuidle_device *dev, } /* Execute ARM wfi */ + ct_idle_enter(); omap_sram_idle(); + ct_idle_exit(); /* * Call idle CPU PM enter notifier chain to restore @@ -265,6 +267,7 @@ static struct cpuidle_driver omap3_idle_driver = { .owner = THIS_MODULE, .states = { { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 2 + 2, .target_residency = 5, @@ -272,6 +275,7 @@ static struct cpuidle_driver omap3_idle_driver = { .desc = "MPU ON + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 10 + 10, .target_residency = 30, @@ -279,6 +283,7 @@ static struct cpuidle_driver omap3_idle_driver = { .desc = "MPU ON + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 50 + 50, .target_residency = 300, @@ -286,6 +291,7 @@ static struct cpuidle_driver omap3_idle_driver = { .desc = "MPU RET + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 1500 + 1800, .target_residency = 4000, @@ -293,6 +299,7 @@ static struct cpuidle_driver omap3_idle_driver = { .desc = "MPU OFF + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 2500 + 7500, .target_residency = 12000, @@ -300,6 +307,7 @@ static struct cpuidle_driver omap3_idle_driver = { .desc = "MPU RET + CORE RET", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 3000 + 8500, .target_residency = 15000, @@ -307,6 +315,7 @@ static struct cpuidle_driver omap3_idle_driver = { .desc = "MPU OFF + CORE RET", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 10000 + 30000, .target_residency = 30000, @@ -328,6 +337,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .owner = THIS_MODULE, .states = { { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 110 + 162, .target_residency = 5, @@ -335,6 +345,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .desc = "MPU ON + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 106 + 180, .target_residency = 309, @@ -342,6 +353,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .desc = "MPU ON + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 107 + 410, .target_residency = 46057, @@ -349,6 +361,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .desc = "MPU RET + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 121 + 3374, .target_residency = 46057, @@ -356,6 +369,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .desc = "MPU OFF + CORE ON", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 855 + 1146, .target_residency = 46057, @@ -363,6 +377,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .desc = "MPU RET + CORE RET", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 7580 + 4134, .target_residency = 484329, @@ -370,6 +385,7 @@ static struct cpuidle_driver omap3430_idle_driver = { .desc = "MPU OFF + CORE RET", }, { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = omap3_enter_idle_bm, .exit_latency = 7505 + 15274, .target_residency = 484329, From patchwork Fri Mar 3 09:23:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63843 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp316900wrd; Fri, 3 Mar 2023 01:37:46 -0800 (PST) X-Google-Smtp-Source: AK7set8ufu5k6mNh3eauv6siIkynvgeM/JIH4XL4EdeJhVx6hsyidFXb7I3XJSaqBbBNSUN0UaKf X-Received: by 2002:a17:906:1405:b0:870:baa6:6762 with SMTP id p5-20020a170906140500b00870baa66762mr974650ejc.14.1677836266806; Fri, 03 Mar 2023 01:37:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836266; cv=none; d=google.com; s=arc-20160816; b=z8XNp2+pd/FZbh3OlX7H1jJkyI8gF7gHXoYE9DepIRwZSBiPfnsIsPEpqOi/ywAo7T mvbFHa+iqt342t8ZEIm5myP5GhcYmhIOn0kYiXvaQRnmCJIHCCiwSrtVapWWXo8848I5 xFJUMRja0NH2avNW97Xw90+zDttuQ+K2rCLAVrHFD/AimCYhXnarvfnUGiTQOjRjrUtT 9v1CMzMnfj3Wy1vxHmKKSXiAjq+cNx7vfCKE8hS1pw/lnE7z+Pnv3oYLUwDud7hQ1Cac nH//1WUM+GOnXqVuYicuVM2tSLi0Zcg40PtoLN4eutlf7UEfKHGGGe0DcqWbufhmyydg j/SA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=vdekVTpo3BnkDXcjiGV3jzfc/4pX+DxY56QvRtM7y5o=; b=yvOmEguVrJFmV+8t3RruE7KrR3tKEGGrWDaQ6qi4M/QULy755oZBHKhEij11UabjDf XC5HCaqImDQkI9SdrYT00ZcG/4PuDvGyh03iiQGWcLtmMO7oNBjTgWL3WhrEe5k+3v5w um1kXLGXf3qDeutUYZZBEyJivq95j1f0DHh1hJ9c54PLAr5XLbuVY3mlhJTKsiUKxhMU XJ1o11j+Zot0BlaS16Su0IuFV9qHM2P15TIsl70snYkDE+5l6/2St/+tgY+GHjIEflJk NYVcDTfUU6ZQrcK3m327TNnt+fPV4dkE7oT7CLYBQqwT2yIfoT3eKbF5Fzo4Zk7V4pWs 073Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=bmuRX0ib; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k20-20020a1709062a5400b008e828688d69si1851857eje.798.2023.03.03.01.37.24; Fri, 03 Mar 2023 01:37:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=bmuRX0ib; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231175AbjCCJ0h (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230117AbjCCJZz (ORCPT ); Fri, 3 Mar 2023 04:25:55 -0500 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9D49457C3; Fri, 3 Mar 2023 01:25:36 -0800 (PST) X-UUID: 57841836b9a511ed945fc101203acc17-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=vdekVTpo3BnkDXcjiGV3jzfc/4pX+DxY56QvRtM7y5o=; b=bmuRX0ibtm57RW6QySzUKSuB546AW9ChSoM8iJ3xM5WIERPau/+qKJIjU6FJNQB8Rl/6u5yuqEpv1CuRHuHT0kEuf1ZwpIyDzWw8npNdNd7yQyOKOYfBQ4RmcQhMEfkInWmY+BJPD4zFr4VORNosPrG7Eo9FGPU1/Z6ktn4Tapo=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:b757c10e-74e6-41d6-97cf-242deb3f6d5a,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:5b2b3d27-564d-42d9-9875-7c868ee415ec,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 57841836b9a511ed945fc101203acc17-20230303 Received: from mtkmbs10n2.mediatek.inc [(172.21.101.183)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1425578257; Fri, 03 Mar 2023 17:25:30 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:29 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:29 +0800 From: Cheng-Jui Wang To: , "Rafael J. Wysocki" , "Daniel Lezcano" , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , Subject: [PATCH 07/10] cpuidle, armada: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:29 +0800 Message-ID: <20230303092347.4825-8-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338840996672487?= X-GMAIL-MSGID: =?utf-8?q?1759338840996672487?= From: Peter Zijlstra commit 4ce40e9dbe83153f60d7e4ccd24a1eb4f8264f6a upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again before going idle is suboptimal. Notably the cpu_pm_*() calls implicitly re-enable RCU for a bit. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Frederic Weisbecker Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195539.946630819@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- drivers/cpuidle/cpuidle-mvebu-v7.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/cpuidle/cpuidle-mvebu-v7.c b/drivers/cpuidle/cpuidle-mvebu-v7.c index 01a856971f05..c9568aa9410c 100644 --- a/drivers/cpuidle/cpuidle-mvebu-v7.c +++ b/drivers/cpuidle/cpuidle-mvebu-v7.c @@ -36,7 +36,10 @@ static int mvebu_v7_enter_idle(struct cpuidle_device *dev, if (drv->states[index].flags & MVEBU_V7_FLAG_DEEP_IDLE) deepidle = true; + ct_idle_enter(); ret = mvebu_v7_cpu_suspend(deepidle); + ct_idle_exit(); + cpu_pm_exit(); if (ret) @@ -49,6 +52,7 @@ static struct cpuidle_driver armadaxp_idle_driver = { .name = "armada_xp_idle", .states[0] = ARM_CPUIDLE_WFI_STATE, .states[1] = { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = mvebu_v7_enter_idle, .exit_latency = 100, .power_usage = 50, @@ -57,6 +61,7 @@ static struct cpuidle_driver armadaxp_idle_driver = { .desc = "CPU power down", }, .states[2] = { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = mvebu_v7_enter_idle, .exit_latency = 1000, .power_usage = 5, @@ -72,6 +77,7 @@ static struct cpuidle_driver armada370_idle_driver = { .name = "armada_370_idle", .states[0] = ARM_CPUIDLE_WFI_STATE, .states[1] = { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = mvebu_v7_enter_idle, .exit_latency = 100, .power_usage = 5, @@ -87,6 +93,7 @@ static struct cpuidle_driver armada38x_idle_driver = { .name = "armada_38x_idle", .states[0] = ARM_CPUIDLE_WFI_STATE, .states[1] = { + .flags = CPUIDLE_FLAG_RCU_IDLE, .enter = mvebu_v7_enter_idle, .exit_latency = 10, .power_usage = 5, From patchwork Fri Mar 3 09:23:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63840 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp316633wrd; Fri, 3 Mar 2023 01:36:57 -0800 (PST) X-Google-Smtp-Source: AK7set/CS17KomrTmapSXXJlD80sFHLSk+Rmgh7MbRlR6SQrKxUOc8vPVIW0GjJhhjin1pr5U/B0 X-Received: by 2002:a17:907:3f82:b0:8b1:7e88:c20f with SMTP id hr2-20020a1709073f8200b008b17e88c20fmr1571689ejc.15.1677836216873; Fri, 03 Mar 2023 01:36:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836216; cv=none; d=google.com; s=arc-20160816; b=k0rNX8G+V75v9iVqraamfSB0NnWtD6yAOWy4vnF8PsfLQ80pQW4yFEIIYmsOQpyaBq MqEAOxSPXBnjK4RNR0r3SQDcduMCW888kusUtwixc3ar4cx6x1OPCriDPv05n35CQbFw 6vDtEPNfmlBTiQQEO32jlbYpXyEFafpXdftWBdDWhyeDdthWCPbIFVGvRtR3ZvyX1To+ qdzSnuu53cEdRpI9y6iJx5MFOBo9h2vJYSmSD8a1xdG+Q2cfeqaw59FSC0J6UYdyyrYr N7lSF3o/lT0KE5EOqdOHkstA7UsVSCTwaFJh4bdq0IF6PdCQYjCgCUKTk3XiP1U9TuAn fe+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=wxgc+VSxR4aq0wFsffWx8Hxh0codNXg2znNzU8yURpE=; b=M0rfSGU8tOGhQbS16djb5sM32EZf354GcEmFmUAQ1/5F8Q3hc6aEbT5cgErqZkBUZO 2nl1xIvH6cfJ5vKvGvnSC7vwg37fRmGFq7in8jDWU8980en5/tyFxZNV9y9VqQOB8txn z3PMxs0flvUJwTJDypnhQ8USCgJd3+Ep9srBG2aebq/pPs4cuuYkkyWGZCD9kjWiW9wG aSTECL3Uzsm8jGDpmf9kjUfyKQUl2xGaBUlIqujzB9GKLETZ9jq7Ftv19EYSXO3AaP0s D5ddsZe7HDtoR3rqdN/RO5ZEpT5H0MS5rYooH1X5/AulN38FJHLGkMixzlF2FIIapCeP 8bLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=mETEQjF9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y6-20020a056402134600b004c10381668csi2208256edw.286.2023.03.03.01.36.33; Fri, 03 Mar 2023 01:36:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=mETEQjF9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229962AbjCCJ03 (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230396AbjCCJZ7 (ORCPT ); Fri, 3 Mar 2023 04:25:59 -0500 Received: from mailgw01.mediatek.com (unknown [60.244.123.138]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14BF527487; Fri, 3 Mar 2023 01:25:39 -0800 (PST) X-UUID: 59a87418b9a511eda06fc9ecc4dadd91-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=wxgc+VSxR4aq0wFsffWx8Hxh0codNXg2znNzU8yURpE=; b=mETEQjF99KNfrua0g3vIjP+IWe1YXuHkkqGcCOkfTMfA3rEcDecS0ELWv5XRNKOEvj8zodQP582ieN5gd6ys6IzsIR/NTzcduXFaEpGIOqz2AD1S8vNzlatx35AM5x18vSCIZ9pgE+t2gQ0AixvRM3POjv23gxmaEdB5PMQnUgw=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:dd0e0493-d2de-435a-9c33-845ca5b8f8c2,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:3d74abf4-ddba-41c3-91d9-10eeade8eac7,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-UUID: 59a87418b9a511eda06fc9ecc4dadd91-20230303 Received: from mtkmbs10n2.mediatek.inc [(172.21.101.183)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 427268035; Fri, 03 Mar 2023 17:25:34 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.185) by mtkmbs11n1.mediatek.inc (172.21.101.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:33 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:33 +0800 From: Cheng-Jui Wang To: , Tony Lindgren , Russell King , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , Subject: [PATCH 08/10] cpuidle, OMAP4: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:30 +0800 Message-ID: <20230303092347.4825-9-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338789365320999?= X-GMAIL-MSGID: =?utf-8?q?1759338789365320999?= From: Peter Zijlstra commit c3d42418dca53d6c498a48c408f7a45289593650 upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again, some *four* times, before going idle is suboptimal. Notably three times explicitly using RCU_NONIDLE() and once implicitly through cpu_pm_*(). Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Reviewed-by: Frederic Weisbecker Reviewed-by: Tony Lindgren Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230112195540.007918454@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- arch/arm/mach-omap2/cpuidle44xx.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c index de37027ad758..953ad888737c 100644 --- a/arch/arm/mach-omap2/cpuidle44xx.c +++ b/arch/arm/mach-omap2/cpuidle44xx.c @@ -105,7 +105,9 @@ static int omap_enter_idle_smp(struct cpuidle_device *dev, } raw_spin_unlock_irqrestore(&mpu_lock, flag); + ct_idle_enter(); omap4_enter_lowpower(dev->cpu, cx->cpu_state); + ct_idle_exit(); raw_spin_lock_irqsave(&mpu_lock, flag); if (cx->mpu_state_vote == num_online_cpus()) @@ -151,10 +153,10 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, (cx->mpu_logic_state == PWRDM_POWER_OFF); /* Enter broadcast mode for periodic timers */ - RCU_NONIDLE(tick_broadcast_enable()); + tick_broadcast_enable(); /* Enter broadcast mode for one-shot timers */ - RCU_NONIDLE(tick_broadcast_enter()); + tick_broadcast_enter(); /* * Call idle CPU PM enter notifier chain so that @@ -166,7 +168,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, if (dev->cpu == 0) { pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state); - RCU_NONIDLE(omap_set_pwrdm_state(mpu_pd, cx->mpu_state)); + omap_set_pwrdm_state(mpu_pd, cx->mpu_state); /* * Call idle CPU cluster PM enter notifier chain @@ -178,14 +180,16 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, index = 0; cx = state_ptr + index; pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state); - RCU_NONIDLE(omap_set_pwrdm_state(mpu_pd, cx->mpu_state)); + omap_set_pwrdm_state(mpu_pd, cx->mpu_state); mpuss_can_lose_context = 0; } } } + ct_idle_enter(); omap4_enter_lowpower(dev->cpu, cx->cpu_state); cpu_done[dev->cpu] = true; + ct_idle_exit(); /* Wakeup CPU1 only if it is not offlined */ if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) { @@ -194,9 +198,9 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, mpuss_can_lose_context) gic_dist_disable(); - RCU_NONIDLE(clkdm_deny_idle(cpu_clkdm[1])); - RCU_NONIDLE(omap_set_pwrdm_state(cpu_pd[1], PWRDM_POWER_ON)); - RCU_NONIDLE(clkdm_allow_idle(cpu_clkdm[1])); + clkdm_deny_idle(cpu_clkdm[1]); + omap_set_pwrdm_state(cpu_pd[1], PWRDM_POWER_ON); + clkdm_allow_idle(cpu_clkdm[1]); if (IS_PM44XX_ERRATUM(PM_OMAP4_ROM_SMP_BOOT_ERRATUM_GICD) && mpuss_can_lose_context) { @@ -222,7 +226,7 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, cpu_pm_exit(); cpu_pm_out: - RCU_NONIDLE(tick_broadcast_exit()); + tick_broadcast_exit(); fail: cpuidle_coupled_parallel_barrier(dev, &abort_barrier); @@ -247,7 +251,8 @@ static struct cpuidle_driver omap4_idle_driver = { /* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */ .exit_latency = 328 + 440, .target_residency = 960, - .flags = CPUIDLE_FLAG_COUPLED, + .flags = CPUIDLE_FLAG_COUPLED | + CPUIDLE_FLAG_RCU_IDLE, .enter = omap_enter_idle_coupled, .name = "C2", .desc = "CPUx OFF, MPUSS CSWR", @@ -256,7 +261,8 @@ static struct cpuidle_driver omap4_idle_driver = { /* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */ .exit_latency = 460 + 518, .target_residency = 1100, - .flags = CPUIDLE_FLAG_COUPLED, + .flags = CPUIDLE_FLAG_COUPLED | + CPUIDLE_FLAG_RCU_IDLE, .enter = omap_enter_idle_coupled, .name = "C3", .desc = "CPUx OFF, MPUSS OSWR", @@ -282,7 +288,8 @@ static struct cpuidle_driver omap5_idle_driver = { /* C2 - CPU0 RET + CPU1 RET + MPU CSWR */ .exit_latency = 48 + 60, .target_residency = 100, - .flags = CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE, .enter = omap_enter_idle_smp, .name = "C2", .desc = "CPUx CSWR, MPUSS CSWR", From patchwork Fri Mar 3 09:23:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63838 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp316126wrd; Fri, 3 Mar 2023 01:35:47 -0800 (PST) X-Google-Smtp-Source: AK7set864Rs6OPeA9nLJt5BQJBwbcmO8o8XkkHHJcU0xHYSzzrushuvYGWMZe8HyXZYYEFeM6zyc X-Received: by 2002:a05:6402:1655:b0:4af:8436:2f59 with SMTP id s21-20020a056402165500b004af84362f59mr1192620edx.31.1677836147048; Fri, 03 Mar 2023 01:35:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836147; cv=none; d=google.com; s=arc-20160816; b=kZJp43c5r1QRtbd6/mKCowrTTeEJAlmo7SJvVdxz+zhNS8ZVU0hbtQ6NUxQGklaF64 Dk2vEalDVxKQ0YPZKeaFivJbVdJKHPjUsZq2gomyaZc3B6s0+0b6Z8AVl/U8Ge2G54nb Sn4Y8NmCdNl/te8WyGXAxapnY9B8nX9D6awgwCoyWiFiRl78SMeUkP8E4ldarV161YlB UioAT8cS0I0Vq0BHGVQp2gFyD9gWqoXDB2wyqeIcD//54Gyeu2dPnd0Tsh6ktsR4ZDG/ Lwgr5wKbsB1SOKoezvzpIywDQCp6+lVxaj19cPEetlvqiRm4VLczDCa9APSerwz1aXMK 58nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=sAKxJxp2pTMt5d5uXFCDQYXYlXBjhMQeL4MNIQdq0wI=; b=qajQ5sMIk8ArS0ZW5hc+5ezo2UdhMg7kfMmRBSRNgH5/ysNf/ltOJtUInS3dxkH25g N3tLO5E+qeWh8iwxByBWBWECncEsW9nT0GIeYePZBy6KijgMs4t7D/iwp+NglqdaryCc UrQ8WcBatS01LjylDjWd2P0tFJpSG63lFX5DL7U+3IgjhlDjRFZ2WGR7H/g0gXs6DNj7 6LTsTdG78uqWhRVFiOffkIa/q2xyAgs6v38/RJv46w2AondLhvRFMg2sWWxKCg57dZf0 lh4/jd9vHBBrhBjGnRsEznKiY8yhTR0YEl3DcOkOaMTnqL9Pf/H7bBu6AD0tC7clPZ5M nemw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=mChWkFMq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l2-20020a170906938200b008b90d160acbsi1888090ejx.579.2023.03.03.01.35.24; Fri, 03 Mar 2023 01:35:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=mChWkFMq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbjCCJ00 (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230430AbjCCJ0A (ORCPT ); Fri, 3 Mar 2023 04:26:00 -0500 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E34915B8B; Fri, 3 Mar 2023 01:25:42 -0800 (PST) X-UUID: 5c25f094b9a511ed945fc101203acc17-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=sAKxJxp2pTMt5d5uXFCDQYXYlXBjhMQeL4MNIQdq0wI=; b=mChWkFMqJXymhQ2oCR/6LLHWqMGYyjukjhdaipGNLYzCqQQjs0a/tOUZIKXM9KSit6JqC0SYyS6crUmsIP97Rg+Jk8iseiFJWGrJ0Rpnv8/mqu8TC2W0qE0YhgoAFUUe1ywQDbBJFwRtXs2bEsvz8AL7kL0CioclMxWRvyvH+sM=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:3efae0e9-6759-4e88-90fa-e9f7b99563eb,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:8d75abf4-ddba-41c3-91d9-10eeade8eac7,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-UUID: 5c25f094b9a511ed945fc101203acc17-20230303 Received: from mtkmbs13n2.mediatek.inc [(172.21.101.108)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1564383775; Fri, 03 Mar 2023 17:25:38 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs13n1.mediatek.inc (172.21.101.193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:37 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:37 +0800 From: Cheng-Jui Wang To: , "Rafael J. Wysocki" , Len Brown , Lorenzo Pieralisi , Daniel Lezcano , Matthias Brugger CC: Peter Zijlstra , Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , , Subject: [PATCH 09/10] cpuidle, dt: Push RCU-idle into driver Date: Fri, 3 Mar 2023 17:23:31 +0800 Message-ID: <20230303092347.4825-10-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338715465456401?= X-GMAIL-MSGID: =?utf-8?q?1759338715465456401?= From: Peter Zijlstra commit 0c5ffc3d7b15978c6b184938cd6b8af06e436424 upstream. Doing RCU-idle outside the driver, only to then temporarily enable it again before going idle is suboptimal. Notably: this converts all dt_init_idle_driver() and __CPU_PM_CPU_IDLE_ENTER() users for they are inextrably intertwined. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Acked-by: Rafael J. Wysocki Acked-by: Frederic Weisbecker Link: https://lore.kernel.org/r/20230112195540.068981667@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- drivers/acpi/processor_idle.c | 2 ++ drivers/cpuidle/cpuidle-big_little.c | 8 ++++++-- drivers/cpuidle/dt_idle_states.c | 2 +- include/linux/cpuidle.h | 2 ++ 4 files changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c index fc5b5b2c9e81..ae42d530d080 100644 --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -1221,6 +1221,8 @@ static int acpi_processor_setup_lpi_states(struct acpi_processor *pr) state->target_residency = lpi->min_residency; if (lpi->arch_flags) state->flags |= CPUIDLE_FLAG_TIMER_STOP; + if (i != 0 && lpi->entry_method == ACPI_CSTATE_FFH) + state->flags |= CPUIDLE_FLAG_RCU_IDLE; state->enter = acpi_idle_lpi_enter; drv->safe_state_index = i; } diff --git a/drivers/cpuidle/cpuidle-big_little.c b/drivers/cpuidle/cpuidle-big_little.c index abe51185f243..fffd4ed0c4d1 100644 --- a/drivers/cpuidle/cpuidle-big_little.c +++ b/drivers/cpuidle/cpuidle-big_little.c @@ -64,7 +64,8 @@ static struct cpuidle_driver bl_idle_little_driver = { .enter = bl_enter_powerdown, .exit_latency = 700, .target_residency = 2500, - .flags = CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE, .name = "C1", .desc = "ARM little-cluster power down", }, @@ -85,7 +86,8 @@ static struct cpuidle_driver bl_idle_big_driver = { .enter = bl_enter_powerdown, .exit_latency = 500, .target_residency = 2000, - .flags = CPUIDLE_FLAG_TIMER_STOP, + .flags = CPUIDLE_FLAG_TIMER_STOP | + CPUIDLE_FLAG_RCU_IDLE, .name = "C1", .desc = "ARM big-cluster power down", }, @@ -124,11 +126,13 @@ static int bl_enter_powerdown(struct cpuidle_device *dev, struct cpuidle_driver *drv, int idx) { cpu_pm_enter(); + ct_idle_enter(); cpu_suspend(0, bl_powerdown_finisher); /* signals the MCPM core that CPU is out of low power state */ mcpm_cpu_powered_up(); + ct_idle_exit(); cpu_pm_exit(); diff --git a/drivers/cpuidle/dt_idle_states.c b/drivers/cpuidle/dt_idle_states.c index 448bc796b0b4..fcfa7490eecf 100644 --- a/drivers/cpuidle/dt_idle_states.c +++ b/drivers/cpuidle/dt_idle_states.c @@ -77,7 +77,7 @@ static int init_state_node(struct cpuidle_state *idle_state, if (err) desc = state_node->name; - idle_state->flags = 0; + idle_state->flags = CPUIDLE_FLAG_RCU_IDLE; if (of_property_read_bool(state_node, "local-timer-stop")) idle_state->flags |= CPUIDLE_FLAG_TIMER_STOP; /* diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h index fce476275e16..0ddc11e44302 100644 --- a/include/linux/cpuidle.h +++ b/include/linux/cpuidle.h @@ -289,7 +289,9 @@ extern s64 cpuidle_governor_latency_req(unsigned int cpu); if (!is_retention) \ __ret = cpu_pm_enter(); \ if (!__ret) { \ + ct_idle_enter(); \ __ret = low_level_idle_enter(state); \ + ct_idle_exit(); \ if (!is_retention) \ cpu_pm_exit(); \ } \ From patchwork Fri Mar 3 09:23:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng-Jui Wang X-Patchwork-Id: 63845 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp317663wrd; Fri, 3 Mar 2023 01:40:04 -0800 (PST) X-Google-Smtp-Source: AK7set8TGsnLrT5j/x2E5MSx3npTFMX29T35HLXwrr0y598dwIkFgWk60uX47xVU7G2cmC7nOWvI X-Received: by 2002:a05:6402:691:b0:4ac:d42c:8bd8 with SMTP id f17-20020a056402069100b004acd42c8bd8mr1430255edy.9.1677836404530; Fri, 03 Mar 2023 01:40:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677836404; cv=none; d=google.com; s=arc-20160816; b=cWjKW9+LC8viAgHmQQWADbJa/Pj1j+CbRvMoyYjfr3CESxwX6PCZ64mANNoAQK0hPW +Ho1TUf8QrMcjy6y6u49lKFGCX4pf6yeGeVp6b9QHx1BfGK6IHte5HmQe5d8n9Rfq0K3 D1BvDIk5hQbzdkOLH7Crqsw3w+k05llQF761GaFLTbMkf8Qk4Mu2zQPS/zv+RI3jwnvs M5Sso837b2aoGS6YDjl1ZQ66W+PTk67EMszEOavhv4dMWBYpaIhJksb+XGR46/FGrhR1 bwchKt1s9yl44kY2F0U1ltEhmzo3T4/VTqlVm+yahSam9xejHhFiOW9XhY84+NCjH2EQ xU3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=lfQPLS0OtxVQGP/jr3dKG0EW+ShMqmnk1TgI/tMjKmo=; b=VyJlK8XVXWCJIWYusmF54x+xefrhAqaE3d5CGy0JDs5ddxiec5mJ5dNdyeCxo2DMJq 2R4bkkk/7JdNv9OLrOvt/xv5rCxcsGYHLsKl5lAs00Dbj4VUtouJm54XGvCxoKNPrIRz JRMAeY3kZc5/5R/hc2pl84GWB1Ztr5DdXVhbAigHoSnDvLDmStlUfSejGmDPQ/c8F1vf T7C0NTwKqKv4ur973xbjmmVlErvpBWWAU7Aeo+EuTdmkdTG59goPzm+eeFe2PylIw+5p SElLtKkhedq+cnTIIg+HvohgArcCiYdcahmDKLIS3bevapNgpspOgX1iu54VPvyGApTu jbQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=C376Sfcd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l21-20020aa7d955000000b004ace5d2ced8si2128890eds.599.2023.03.03.01.39.42; Fri, 03 Mar 2023 01:40:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@mediatek.com header.s=dk header.b=C376Sfcd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=mediatek.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231218AbjCCJ0n (ORCPT + 99 others); Fri, 3 Mar 2023 04:26:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230514AbjCCJ0F (ORCPT ); Fri, 3 Mar 2023 04:26:05 -0500 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7742198; Fri, 3 Mar 2023 01:25:48 -0800 (PST) X-UUID: 5fa04616b9a511ed945fc101203acc17-20230303 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=lfQPLS0OtxVQGP/jr3dKG0EW+ShMqmnk1TgI/tMjKmo=; b=C376SfcdE+BsQfA5urjdMrGQj4Ux25YNbd4cBaSeUSy+4T7smy9V7Tjcj1RSuLuQwRsepEFeo2EDq6Ve+Ab/+CNGqjGjfd5lk33t9qsl1X1JlprUTXiMCm7UUCuj0YyQQOBJmmmZRJ8HLY/wrPnlu+AHg192kNmScI7u56oaLVQ=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.20,REQID:0b1028b5-1ef1-41a6-9117-fa22408ef090,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:-25 X-CID-META: VersionHash:25b5999,CLOUDID:2fc41db2-beed-4dfc-bd9c-e1b22fa6ccc4,B ulkID:nil,BulkQuantity:0,Recheck:0,SF:102,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0 X-CID-BVR: 0 X-UUID: 5fa04616b9a511ed945fc101203acc17-20230303 Received: from mtkmbs11n1.mediatek.inc [(172.21.101.185)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 2141692127; Fri, 03 Mar 2023 17:25:44 +0800 Received: from mtkmbs11n1.mediatek.inc (172.21.101.186) by mtkmbs13n2.mediatek.inc (172.21.101.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Fri, 3 Mar 2023 17:25:43 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs11n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Fri, 3 Mar 2023 17:25:43 +0800 From: Cheng-Jui Wang To: , Russell King , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Tony Lindgren , "Rafael J. Wysocki" , Len Brown , Lorenzo Pieralisi , Daniel Lezcano , Sudeep Holla , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thierry Reding , Jonathan Hunter , Thomas Gleixner , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , Frederic Weisbecker , Matthias Brugger CC: Ingo Molnar , Suren Baghdasaryan , Cheng-Jui Wang , , , , , , , , Subject: [PATCH 10/10] cpuidle: Fix ct_idle_*() usage Date: Fri, 3 Mar 2023 17:23:32 +0800 Message-ID: <20230303092347.4825-11-cheng-jui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> References: <20230303092347.4825-1-cheng-jui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759338985584397537?= X-GMAIL-MSGID: =?utf-8?q?1759338985584397537?= From: Peter Zijlstra commit a01353cf1896ea5b8a7bbc5e2b2d38feed8b7aaa upstream. The whole disable-RCU, enable-IRQS dance is very intricate since changing IRQ state is traced, which depends on RCU. Add two helpers for the cpuidle case that mirror the entry code: ct_cpuidle_enter() ct_cpuidle_exit() And fix all the cases where the enter/exit dance was buggy. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Acked-by: Rafael J. Wysocki Acked-by: Frederic Weisbecker Link: https://lore.kernel.org/r/20230112195540.130014793@infradead.org Signed-off-by: Suren Baghdasaryan Signed-off-by: Cheng-Jui Wang --- arch/arm/mach-imx/cpuidle-imx6q.c | 4 +-- arch/arm/mach-imx/cpuidle-imx6sx.c | 4 +-- arch/arm/mach-omap2/cpuidle34xx.c | 4 +-- arch/arm/mach-omap2/cpuidle44xx.c | 8 ++--- drivers/acpi/processor_idle.c | 8 +++-- drivers/cpuidle/cpuidle-big_little.c | 4 +-- drivers/cpuidle/cpuidle-mvebu-v7.c | 4 +-- drivers/cpuidle/cpuidle-psci.c | 4 +-- drivers/cpuidle/cpuidle-riscv-sbi.c | 4 +-- drivers/cpuidle/cpuidle-tegra.c | 8 ++--- drivers/cpuidle/cpuidle.c | 11 +++---- include/linux/clockchips.h | 4 +-- include/linux/cpuidle.h | 34 +++++++++++++++++++-- kernel/sched/idle.c | 45 ++++++++-------------------- kernel/time/tick-broadcast.c | 6 +++- 15 files changed, 86 insertions(+), 66 deletions(-) diff --git a/arch/arm/mach-imx/cpuidle-imx6q.c b/arch/arm/mach-imx/cpuidle-imx6q.c index d086cbae09c3..c24c78a67cc1 100644 --- a/arch/arm/mach-imx/cpuidle-imx6q.c +++ b/arch/arm/mach-imx/cpuidle-imx6q.c @@ -25,9 +25,9 @@ static int imx6q_enter_wait(struct cpuidle_device *dev, imx6_set_lpm(WAIT_UNCLOCKED); raw_spin_unlock(&cpuidle_lock); - ct_idle_enter(); + ct_cpuidle_enter(); cpu_do_idle(); - ct_idle_exit(); + ct_cpuidle_exit(); raw_spin_lock(&cpuidle_lock); if (num_idle_cpus-- == num_online_cpus()) diff --git a/arch/arm/mach-imx/cpuidle-imx6sx.c b/arch/arm/mach-imx/cpuidle-imx6sx.c index 1dc01f6b0f36..479f06286b50 100644 --- a/arch/arm/mach-imx/cpuidle-imx6sx.c +++ b/arch/arm/mach-imx/cpuidle-imx6sx.c @@ -47,9 +47,9 @@ static int imx6sx_enter_wait(struct cpuidle_device *dev, cpu_pm_enter(); cpu_cluster_pm_enter(); - ct_idle_enter(); + ct_cpuidle_enter(); cpu_suspend(0, imx6sx_idle_finish); - ct_idle_exit(); + ct_cpuidle_exit(); cpu_cluster_pm_exit(); cpu_pm_exit(); diff --git a/arch/arm/mach-omap2/cpuidle34xx.c b/arch/arm/mach-omap2/cpuidle34xx.c index cedf5cbe451f..6d63769cef0f 100644 --- a/arch/arm/mach-omap2/cpuidle34xx.c +++ b/arch/arm/mach-omap2/cpuidle34xx.c @@ -133,9 +133,9 @@ static int omap3_enter_idle(struct cpuidle_device *dev, } /* Execute ARM wfi */ - ct_idle_enter(); + ct_cpuidle_enter(); omap_sram_idle(); - ct_idle_exit(); + ct_cpuidle_exit(); /* * Call idle CPU PM enter notifier chain to restore diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c index 953ad888737c..3c97d5676e81 100644 --- a/arch/arm/mach-omap2/cpuidle44xx.c +++ b/arch/arm/mach-omap2/cpuidle44xx.c @@ -105,9 +105,9 @@ static int omap_enter_idle_smp(struct cpuidle_device *dev, } raw_spin_unlock_irqrestore(&mpu_lock, flag); - ct_idle_enter(); + ct_cpuidle_enter(); omap4_enter_lowpower(dev->cpu, cx->cpu_state); - ct_idle_exit(); + ct_cpuidle_exit(); raw_spin_lock_irqsave(&mpu_lock, flag); if (cx->mpu_state_vote == num_online_cpus()) @@ -186,10 +186,10 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev, } } - ct_idle_enter(); + ct_cpuidle_enter(); omap4_enter_lowpower(dev->cpu, cx->cpu_state); cpu_done[dev->cpu] = true; - ct_idle_exit(); + ct_cpuidle_exit(); /* Wakeup CPU1 only if it is not offlined */ if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) { diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c index ae42d530d080..ff40b001e0fa 100644 --- a/drivers/acpi/processor_idle.c +++ b/drivers/acpi/processor_idle.c @@ -644,6 +644,8 @@ static int __cpuidle acpi_idle_enter_bm(struct cpuidle_driver *drv, */ bool dis_bm = pr->flags.bm_control; + instrumentation_begin(); + /* If we can skip BM, demote to a safe state. */ if (!cx->bm_sts_skip && acpi_idle_bm_check()) { dis_bm = false; @@ -665,11 +667,11 @@ static int __cpuidle acpi_idle_enter_bm(struct cpuidle_driver *drv, raw_spin_unlock(&c3_lock); } - ct_idle_enter(); + ct_cpuidle_enter(); acpi_idle_do_entry(cx); - ct_idle_exit(); + ct_cpuidle_exit(); /* Re-enable bus master arbitration */ if (dis_bm) { @@ -679,6 +681,8 @@ static int __cpuidle acpi_idle_enter_bm(struct cpuidle_driver *drv, raw_spin_unlock(&c3_lock); } + instrumentation_end(); + return index; } diff --git a/drivers/cpuidle/cpuidle-big_little.c b/drivers/cpuidle/cpuidle-big_little.c index fffd4ed0c4d1..5858db21e08c 100644 --- a/drivers/cpuidle/cpuidle-big_little.c +++ b/drivers/cpuidle/cpuidle-big_little.c @@ -126,13 +126,13 @@ static int bl_enter_powerdown(struct cpuidle_device *dev, struct cpuidle_driver *drv, int idx) { cpu_pm_enter(); - ct_idle_enter(); + ct_cpuidle_enter(); cpu_suspend(0, bl_powerdown_finisher); /* signals the MCPM core that CPU is out of low power state */ mcpm_cpu_powered_up(); - ct_idle_exit(); + ct_cpuidle_exit(); cpu_pm_exit(); diff --git a/drivers/cpuidle/cpuidle-mvebu-v7.c b/drivers/cpuidle/cpuidle-mvebu-v7.c index c9568aa9410c..20bfb26d5a88 100644 --- a/drivers/cpuidle/cpuidle-mvebu-v7.c +++ b/drivers/cpuidle/cpuidle-mvebu-v7.c @@ -36,9 +36,9 @@ static int mvebu_v7_enter_idle(struct cpuidle_device *dev, if (drv->states[index].flags & MVEBU_V7_FLAG_DEEP_IDLE) deepidle = true; - ct_idle_enter(); + ct_cpuidle_enter(); ret = mvebu_v7_cpu_suspend(deepidle); - ct_idle_exit(); + ct_cpuidle_exit(); cpu_pm_exit(); diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 6a259c94b8b2..32ad942d3e5a 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -74,7 +74,7 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, else pm_runtime_put_sync_suspend(pd_dev); - ct_idle_enter(); + ct_cpuidle_enter(); state = psci_get_domain_state(); if (!state) @@ -82,7 +82,7 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, ret = psci_cpu_suspend_enter(state) ? -1 : idx; - ct_idle_exit(); + ct_cpuidle_exit(); if (s2idle) dev_pm_genpd_resume(pd_dev); diff --git a/drivers/cpuidle/cpuidle-riscv-sbi.c b/drivers/cpuidle/cpuidle-riscv-sbi.c index cbdbb11b972b..0a480f5799a7 100644 --- a/drivers/cpuidle/cpuidle-riscv-sbi.c +++ b/drivers/cpuidle/cpuidle-riscv-sbi.c @@ -126,7 +126,7 @@ static int __sbi_enter_domain_idle_state(struct cpuidle_device *dev, else pm_runtime_put_sync_suspend(pd_dev); - ct_idle_enter(); + ct_cpuidle_enter(); if (sbi_is_domain_state_available()) state = sbi_get_domain_state(); @@ -135,7 +135,7 @@ static int __sbi_enter_domain_idle_state(struct cpuidle_device *dev, ret = sbi_suspend(state) ? -1 : idx; - ct_idle_exit(); + ct_cpuidle_exit(); if (s2idle) dev_pm_genpd_resume(pd_dev); diff --git a/drivers/cpuidle/cpuidle-tegra.c b/drivers/cpuidle/cpuidle-tegra.c index 3ca5cfb9d322..9c2903c1b1c0 100644 --- a/drivers/cpuidle/cpuidle-tegra.c +++ b/drivers/cpuidle/cpuidle-tegra.c @@ -183,7 +183,7 @@ static int tegra_cpuidle_state_enter(struct cpuidle_device *dev, tegra_pm_set_cpu_in_lp2(); cpu_pm_enter(); - ct_idle_enter(); + ct_cpuidle_enter(); switch (index) { case TEGRA_C7: @@ -199,7 +199,7 @@ static int tegra_cpuidle_state_enter(struct cpuidle_device *dev, break; } - ct_idle_exit(); + ct_cpuidle_exit(); cpu_pm_exit(); tegra_pm_clear_cpu_in_lp2(); @@ -240,10 +240,10 @@ static int tegra_cpuidle_enter(struct cpuidle_device *dev, if (index == TEGRA_C1) { if (do_rcu) - ct_idle_enter(); + ct_cpuidle_enter(); ret = arm_cpuidle_simple_enter(dev, drv, index); if (do_rcu) - ct_idle_exit(); + ct_cpuidle_exit(); } else ret = tegra_cpuidle_state_enter(dev, index, cpu); diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c index 6eceb1988243..caeb543c7ea0 100644 --- a/drivers/cpuidle/cpuidle.c +++ b/drivers/cpuidle/cpuidle.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -152,12 +153,12 @@ static void enter_s2idle_proper(struct cpuidle_driver *drv, */ stop_critical_timings(); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) - ct_idle_enter(); + ct_cpuidle_enter(); target_state->enter_s2idle(dev, drv, index); if (WARN_ON_ONCE(!irqs_disabled())) - local_irq_disable(); + raw_local_irq_disable(); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) - ct_idle_exit(); + ct_cpuidle_exit(); tick_unfreeze(); start_critical_timings(); @@ -235,10 +236,10 @@ int cpuidle_enter_state(struct cpuidle_device *dev, struct cpuidle_driver *drv, stop_critical_timings(); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) - ct_idle_enter(); + ct_cpuidle_enter(); entered_state = target_state->enter(dev, drv, index); if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE)) - ct_idle_exit(); + ct_cpuidle_exit(); start_critical_timings(); sched_clock_idle_wakeup_event(); diff --git a/include/linux/clockchips.h b/include/linux/clockchips.h index 8ae9a95ebf5b..9aac31d856f3 100644 --- a/include/linux/clockchips.h +++ b/include/linux/clockchips.h @@ -211,7 +211,7 @@ extern int tick_receive_broadcast(void); extern void tick_setup_hrtimer_broadcast(void); extern int tick_check_broadcast_expired(void); # else -static inline int tick_check_broadcast_expired(void) { return 0; } +static __always_inline int tick_check_broadcast_expired(void) { return 0; } static inline void tick_setup_hrtimer_broadcast(void) { } # endif @@ -219,7 +219,7 @@ static inline void tick_setup_hrtimer_broadcast(void) { } static inline void clockevents_suspend(void) { } static inline void clockevents_resume(void) { } -static inline int tick_check_broadcast_expired(void) { return 0; } +static __always_inline int tick_check_broadcast_expired(void) { return 0; } static inline void tick_setup_hrtimer_broadcast(void) { } #endif /* !CONFIG_GENERIC_CLOCKEVENTS */ diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h index 0ddc11e44302..630c879143c7 100644 --- a/include/linux/cpuidle.h +++ b/include/linux/cpuidle.h @@ -14,6 +14,7 @@ #include #include #include +#include #define CPUIDLE_STATE_MAX 10 #define CPUIDLE_NAME_LEN 16 @@ -115,6 +116,35 @@ struct cpuidle_device { DECLARE_PER_CPU(struct cpuidle_device *, cpuidle_devices); DECLARE_PER_CPU(struct cpuidle_device, cpuidle_dev); +static __always_inline void ct_cpuidle_enter(void) +{ + lockdep_assert_irqs_disabled(); + /* + * Idle is allowed to (temporary) enable IRQs. It + * will return with IRQs disabled. + * + * Trace IRQs enable here, then switch off RCU, and have + * arch_cpu_idle() use raw_local_irq_enable(). Note that + * ct_idle_enter() relies on lockdep IRQ state, so switch that + * last -- this is very similar to the entry code. + */ + trace_hardirqs_on_prepare(); + lockdep_hardirqs_on_prepare(); + instrumentation_end(); + ct_idle_enter(); + lockdep_hardirqs_on(_RET_IP_); +} + +static __always_inline void ct_cpuidle_exit(void) +{ + /* + * Carefully undo the above. + */ + lockdep_hardirqs_off(_RET_IP_); + ct_idle_exit(); + instrumentation_begin(); +} + /**************************** * CPUIDLE DRIVER INTERFACE * ****************************/ @@ -289,9 +319,9 @@ extern s64 cpuidle_governor_latency_req(unsigned int cpu); if (!is_retention) \ __ret = cpu_pm_enter(); \ if (!__ret) { \ - ct_idle_enter(); \ + ct_cpuidle_enter(); \ __ret = low_level_idle_enter(state); \ - ct_idle_exit(); \ + ct_cpuidle_exit(); \ if (!is_retention) \ cpu_pm_exit(); \ } \ diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index f26ab2675f7d..e924602ec43b 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -51,18 +51,22 @@ __setup("hlt", cpu_idle_nopoll_setup); static noinline int __cpuidle cpu_idle_poll(void) { + instrumentation_begin(); trace_cpu_idle(0, smp_processor_id()); stop_critical_timings(); - ct_idle_enter(); - local_irq_enable(); + ct_cpuidle_enter(); + raw_local_irq_enable(); while (!tif_need_resched() && (cpu_idle_force_poll || tick_check_broadcast_expired())) cpu_relax(); + raw_local_irq_disable(); - ct_idle_exit(); + ct_cpuidle_exit(); start_critical_timings(); trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id()); + local_irq_enable(); + instrumentation_end(); return 1; } @@ -85,44 +89,21 @@ void __weak arch_cpu_idle(void) */ void __cpuidle default_idle_call(void) { - if (current_clr_polling_and_test()) { - local_irq_enable(); - } else { - + instrumentation_begin(); + if (!current_clr_polling_and_test()) { trace_cpu_idle(1, smp_processor_id()); stop_critical_timings(); - /* - * arch_cpu_idle() is supposed to enable IRQs, however - * we can't do that because of RCU and tracing. - * - * Trace IRQs enable here, then switch off RCU, and have - * arch_cpu_idle() use raw_local_irq_enable(). Note that - * ct_idle_enter() relies on lockdep IRQ state, so switch that - * last -- this is very similar to the entry code. - */ - trace_hardirqs_on_prepare(); - lockdep_hardirqs_on_prepare(); - ct_idle_enter(); - lockdep_hardirqs_on(_THIS_IP_); - + ct_cpuidle_enter(); arch_cpu_idle(); - - /* - * OK, so IRQs are enabled here, but RCU needs them disabled to - * turn itself back on.. funny thing is that disabling IRQs - * will cause tracing, which needs RCU. Jump through hoops to - * make it 'work'. - */ raw_local_irq_disable(); - lockdep_hardirqs_off(_THIS_IP_); - ct_idle_exit(); - lockdep_hardirqs_on(_THIS_IP_); - raw_local_irq_enable(); + ct_cpuidle_exit(); start_critical_timings(); trace_cpu_idle(PWR_EVENT_EXIT, smp_processor_id()); } + local_irq_enable(); + instrumentation_end(); } static int call_cpuidle_s2idle(struct cpuidle_driver *drv, diff --git a/kernel/time/tick-broadcast.c b/kernel/time/tick-broadcast.c index f7fe6fe36173..93bf2b4e47e5 100644 --- a/kernel/time/tick-broadcast.c +++ b/kernel/time/tick-broadcast.c @@ -622,9 +622,13 @@ struct cpumask *tick_get_broadcast_oneshot_mask(void) * to avoid a deep idle transition as we are about to get the * broadcast IPI right away. */ -int tick_check_broadcast_expired(void) +noinstr int tick_check_broadcast_expired(void) { +#ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H + return arch_test_bit(smp_processor_id(), cpumask_bits(tick_broadcast_force_mask)); +#else return cpumask_test_cpu(smp_processor_id(), tick_broadcast_force_mask); +#endif } /*