From patchwork Wed Aug 2 13:24:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 129863 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp490565vqx; Wed, 2 Aug 2023 07:17:31 -0700 (PDT) X-Google-Smtp-Source: APBJJlFF3nFYcpHpGKLbzShBNspf6vyQ8dEDR3ZghuqA1h4OUBlHB0VIR3n3M0HV/YqVJkAd98RC X-Received: by 2002:a17:90a:cb17:b0:268:a61:ba86 with SMTP id z23-20020a17090acb1700b002680a61ba86mr13711417pjt.16.1690985851080; Wed, 02 Aug 2023 07:17:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690985851; cv=none; d=google.com; s=arc-20160816; b=Dj/VL8MgTQup1UrxRECBrfdfEQOxmqI7IvxcNZHGTXjoPoTmXsYTzF3UoBjxsjslAR td8I/FgJud0EtUavbB55y8UaYviZmSENl0ReCi68KauRYnAiPrmQlVw4ptmi4xTohGKu kVdtGFFzbccSkcOvWeozt/lUazcKDJx/5t+wiQBWFMvhDPJzhsRCKGx+AXyNIyZWx0ux 3CtOhyDvkqjb+Q89bllj7W1jKUaK2et4X7DLXH28UD8zzvoaTqGlSll2gAPEI5kNwFsn A94+xcZWbP6r+K7zyvDWhyCuWPK1ttdrN0DrUSh0i9PQcD4IANuBikLdHgZsZ1nn0Egh fQuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=JAEdkNYRa3AXqsFdc832avh6PD2FjtrGNGx2xJmd/wI=; fh=lspZAbmoQCRoxA9wrY498vIgy5FZA7NXITJfi9W+/v8=; b=JCVgNrU/LBBhEB+NqlpBcppsj02Ttvv0slB23n6WFFhFdKnqns0NrD4Ap7kbeemrGn RusEMeYmZV1ZUb4PILA6awZbFYxPw1fNEjUqy78UgkalLL0BQ+ptGfMLQDj3pwh7a9rv JW8ELGNAw5OFw/DgOAc2+Al7AaZSOIWW4XAkOzUTdKCEu06ny11Pl06x+NDxrsBAhP82 xUztANKWcs7TmJCIHG+DNTq0dn46LOGnF82wsxI5TbfgDYoqLL6gypEN24Szz8TtVmbK HsWY3OucGLKOmvNyrvyCKQA5+423JaQEP94W33muGIBTKO41fMIJzWPzzg6s7vNaDB4I TOGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=fU6G3YXw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o9-20020a17090ac08900b0026815fe629esi1299983pjs.70.2023.08.02.07.17.04; Wed, 02 Aug 2023 07:17:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=fU6G3YXw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234212AbjHBNbo (ORCPT + 99 others); Wed, 2 Aug 2023 09:31:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233444AbjHBNbc (ORCPT ); Wed, 2 Aug 2023 09:31:32 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B25D1BFD; Wed, 2 Aug 2023 06:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=JAEdkNYRa3AXqsFdc832avh6PD2FjtrGNGx2xJmd/wI=; b=fU6G3YXwQ34Cb1/wQwfYvcumtt LdxhBQ2iLVc8CMCboG1kI/Le/16V4MPrdeVKmPfbuURe5qbMWj5RjiOyNU+Nq4b8NQetB6dPC/SVx 0X/DZFYsbtW4wMTqAdyzw/kYkZpsK5CgMufWNDUJJgcUSYP7tdpOLysCSaq6mBykBA+mtFdznhE1U eYmf+IZ5Tjg5P2CQdVYTBGWwf9qDFsVZtjZ9rHW4cDBosVOdTLhEebk5G0ceo9nIDkSUXOSr9ixQ1 tJhwIjp4FFEyr8Y0BxmXi07xsYgMy/mVdfLiYz8L2Htb4dQxwMFUTu6VUQ4Zrq2K7ChqThiWnJ0gE ZhsV3++Q==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qRBwZ-00FxfC-21; Wed, 02 Aug 2023 13:31:17 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 12B0A300301; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id ECE022107D7E6; Wed, 2 Aug 2023 15:31:14 +0200 (CEST) Message-ID: <20230802132925.549643116@infradead.org> User-Agent: quilt/0.66 Date: Wed, 02 Aug 2023 15:24:32 +0200 From: Peter Zijlstra To: anna-maria@linutronix.de, rafael@kernel.org, tglx@linutronix.de, frederic@kernel.org, gautham.shenoy@amd.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, daniel.lezcano@linaro.org, linux-pm@vger.kernel.org, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, kajetan.puchalski@arm.com Subject: [RFC][PATCH v2 1/5] tick/nohz: Introduce tick_get_sleep_length() References: <20230802132431.254614419@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773127179851284529 X-GMAIL-MSGID: 1773127179851284529 Add a variant of tick_nohz_get_sleep_length() that conditionally does the NOHZ part. tick_get_sleep_length(false) returns the delta_next return value of tick_nohz_get_sleep_length(), while tick_get_sleep_length(true) returns the regular return of tick_nohz_get_sleep_length(). This allows eliding tick_nohz_next_event() -- which is going to be expensive with timer-pull. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/tick.h | 5 +++++ kernel/time/tick-sched.c | 35 +++++++++++++++++++++++------------ 2 files changed, 28 insertions(+), 12 deletions(-) --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -136,6 +136,7 @@ extern void tick_nohz_irq_exit(void); extern bool tick_nohz_idle_got_tick(void); extern ktime_t tick_nohz_get_next_hrtimer(void); extern ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next); +extern ktime_t tick_get_sleep_length(bool nohz); extern unsigned long tick_nohz_get_idle_calls(void); extern unsigned long tick_nohz_get_idle_calls_cpu(int cpu); extern u64 get_cpu_idle_time_us(int cpu, u64 *last_update_time); @@ -168,6 +169,10 @@ static inline ktime_t tick_nohz_get_slee *delta_next = TICK_NSEC; return *delta_next; } +static inline ktime_t tick_get_sleep_length(bool nohz) +{ + return TICK_NSEC; +} static inline u64 get_cpu_idle_time_us(int cpu, u64 *unused) { return -1; } static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; } --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1218,17 +1218,7 @@ ktime_t tick_nohz_get_next_hrtimer(void) return __this_cpu_read(tick_cpu_device.evtdev)->next_event; } -/** - * tick_nohz_get_sleep_length - return the expected length of the current sleep - * @delta_next: duration until the next event if the tick cannot be stopped - * - * Called from power state control code with interrupts disabled. - * - * The return value of this function and/or the value returned by it through the - * @delta_next pointer can be negative which must be taken into account by its - * callers. - */ -ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next) +static ktime_t __tick_nohz_get_sleep_length(ktime_t *delta_next, bool nohz) { struct clock_event_device *dev = __this_cpu_read(tick_cpu_device.evtdev); struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); @@ -1244,7 +1234,7 @@ ktime_t tick_nohz_get_sleep_length(ktime *delta_next = ktime_sub(dev->next_event, now); - if (!can_stop_idle_tick(cpu, ts)) + if (!nohz || !can_stop_idle_tick(cpu, ts)) return *delta_next; next_event = tick_nohz_next_event(ts, cpu); @@ -1262,6 +1252,27 @@ ktime_t tick_nohz_get_sleep_length(ktime } /** + * tick_nohz_get_sleep_length - return the expected length of the current sleep + * @delta_next: duration until the next event if the tick cannot be stopped + * + * Called from power state control code with interrupts disabled. + * + * The return value of this function and/or the value returned by it through the + * @delta_next pointer can be negative which must be taken into account by its + * callers. + */ +ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next) +{ + return __tick_nohz_get_sleep_length(delta_next, true); +} + +ktime_t tick_get_sleep_length(bool nohz) +{ + ktime_t delta; + return __tick_nohz_get_sleep_length(&delta, nohz); +} + +/** * tick_nohz_get_idle_calls_cpu - return the current idle calls counter value * for a particular CPU. * From patchwork Wed Aug 2 13:24:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 129860 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp487632vqx; Wed, 2 Aug 2023 07:13:41 -0700 (PDT) X-Google-Smtp-Source: APBJJlF4K4qZK2RxAH5gpqNi4dZFW8SRfPiLCDFaQwawJfQL5t+jP5cFOJjRqZmU/6PBfgv7PUEi X-Received: by 2002:a05:6a00:2e1c:b0:687:3bc2:58de with SMTP id fc28-20020a056a002e1c00b006873bc258demr9650092pfb.6.1690985621202; Wed, 02 Aug 2023 07:13:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690985621; cv=none; d=google.com; s=arc-20160816; b=MHQT5ybJWOa9wWToRzEgdN1mwMst2cdtypj2csCbaaNqycTxg9pmVQ0bcT2DIsLSLT 2BqQn+hzx66jBhC6DcxwOBTlsZT4s+6NQ0EQGDVPy5UNOH69/l2mX4a9RLjk7lrj8VmM nshD1Qc+3aAt9FCwhOtpf7TIGCu0kYtJ8t2lOnzdf4PzgN1JLFCx1LhTVa5A+WqAQAAc RrrqWHEb0fIC/RplEQw0k/5wplxSniaZwi3V7XVGGmecdZYrfQ900Sqp4yeDPbMxTy1Y jgqpvLK/23O5JchzuSxS4zK7sBinKJRvrgzEWewkfR/fnv6zNqAux4VUEyuOQ0l5yOjn ztwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=/jk8cjAzNp5umM9o/8ZP9pvl8gboqgF2w9reMo9nXqk=; fh=lspZAbmoQCRoxA9wrY498vIgy5FZA7NXITJfi9W+/v8=; b=aoV8Bb5kYhBVNJCYDrUDjThGwDk7nyBeH4qBsZ6IO3exypCcrqzXXvJP9kz42lzt/F TS9rg+Bes5KCRs5vKSRuRfeADtjPrGIgxsUkSpyyt0hHTeiSMcGbL98qbXQnaKlxdmx0 UPFXXGMDcvmq/T70DM6wNIsdwMfqUzLVhUSpESwqborffmifIEKFrwDrnRgtvl+6WAnw 0+aPrQkdhPnzdCrQhTYDqMIstPNA9uj2QhQH/t8nhHK5BMhDazKDLJNoEzKLHEoIfk0W R/OW63vYHACthtWGAH84I5mZTDx1mDfuDgv/ZcHhwWka7fmje9GdpAKo7qlx2OOJvqwX wI6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=rP7UMCWA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ea22-20020a056a004c1600b00686db9a11bbsi6552202pfb.61.2023.08.02.07.13.27; Wed, 02 Aug 2023 07:13:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=rP7UMCWA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232279AbjHBNbb (ORCPT + 99 others); Wed, 2 Aug 2023 09:31:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231612AbjHBNb3 (ORCPT ); Wed, 2 Aug 2023 09:31:29 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70F8A1BF9; Wed, 2 Aug 2023 06:31:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=/jk8cjAzNp5umM9o/8ZP9pvl8gboqgF2w9reMo9nXqk=; b=rP7UMCWAI6sRygyJ9fEfu2wtlg +1g4wOf8fa5r/s/Y6W9qw52cgpojG2KcQPpmfY/7P78768ajegVsRX48aFbqVG20QhnxuvYG9SJ0S uZf/6cSOgfQCWCQ3N0CZloMXdrKLxmAYwO2Al3+k0mRJtPGI5NTnUYBW0RttPTn663iDq5P1yoKUU 0fogOAMmncYeQNU6LEeoJTCWMIMFp215RTWiG2ItW1PFQxTMOwVLppVf7dFkw2qEnMh2PX5iW+OQS D3qujBk2eh30GOHf0epVr1bMReR4azwigRCde6XkGsqHHf5bz0lfxd+grG8x3iIuJi7O2/PuE6Q8c Ory2eoGA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qRBwZ-00FBnW-JB; Wed, 02 Aug 2023 13:31:15 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 173A03006E2; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id EF6922066B201; Wed, 2 Aug 2023 15:31:14 +0200 (CEST) Message-ID: <20230802132925.618923837@infradead.org> User-Agent: quilt/0.66 Date: Wed, 02 Aug 2023 15:24:33 +0200 From: Peter Zijlstra To: anna-maria@linutronix.de, rafael@kernel.org, tglx@linutronix.de, frederic@kernel.org, gautham.shenoy@amd.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, daniel.lezcano@linaro.org, linux-pm@vger.kernel.org, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, kajetan.puchalski@arm.com Subject: [RFC][PATCH v2 2/5] cpuidle: Inject tick boundary state References: <20230802132431.254614419@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773126938701842327 X-GMAIL-MSGID: 1773126938701842327 In order to facilitate governors that track history in idle-state buckets (TEO) making a useful decision about NOHZ, make sure we have a bucket that counts tick-and-longer. In order to be inclusive of the tick itself -- after all, if we do not disable NOHZ we'll sleep for a full tick, the actual boundary should be just short of a full tick. IOW, when registering the idle-states, add one that is always disabled, just to have a bucket. Signed-off-by: Peter Zijlstra (Intel) --- drivers/cpuidle/cpuidle.h | 2 + drivers/cpuidle/driver.c | 48 +++++++++++++++++++++++++++++++++++++++++++++- include/linux/cpuidle.h | 2 - 3 files changed, 50 insertions(+), 2 deletions(-) --- a/drivers/cpuidle/cpuidle.h +++ b/drivers/cpuidle/cpuidle.h @@ -72,4 +72,6 @@ static inline void cpuidle_coupled_unreg } #endif +#define SHORT_TICK_NSEC (TICK_NSEC - TICK_NSEC/32) + #endif /* __DRIVER_CPUIDLE_H */ --- a/drivers/cpuidle/driver.c +++ b/drivers/cpuidle/driver.c @@ -147,13 +147,37 @@ static void cpuidle_setup_broadcast_time tick_broadcast_disable(); } +static int tick_enter(struct cpuidle_device *dev, + struct cpuidle_driver *drv, + int index) +{ + return -ENODEV; +} + +static void __cpuidle_state_init_tick(struct cpuidle_state *s) +{ + strcpy(s->name, "TICK"); + strcpy(s->desc, "(no-op)"); + + s->target_residency_ns = SHORT_TICK_NSEC; + s->target_residency = div_u64(SHORT_TICK_NSEC, NSEC_PER_USEC); + + s->exit_latency_ns = 0; + s->exit_latency = 0; + + s->flags |= CPUIDLE_FLAG_UNUSABLE; + + s->enter = tick_enter; + s->enter_s2idle = tick_enter; +} + /** * __cpuidle_driver_init - initialize the driver's internal data * @drv: a valid pointer to a struct cpuidle_driver */ static void __cpuidle_driver_init(struct cpuidle_driver *drv) { - int i; + int tick = 0, i; /* * Use all possible CPUs as the default, because if the kernel boots @@ -163,6 +187,9 @@ static void __cpuidle_driver_init(struct if (!drv->cpumask) drv->cpumask = (struct cpumask *)cpu_possible_mask; + if (WARN_ON_ONCE(drv->state_count >= CPUIDLE_STATE_MAX-2)) + tick = 1; + for (i = 0; i < drv->state_count; i++) { struct cpuidle_state *s = &drv->states[i]; @@ -192,6 +219,25 @@ static void __cpuidle_driver_init(struct s->exit_latency_ns = 0; else s->exit_latency = div_u64(s->exit_latency_ns, NSEC_PER_USEC); + + if (!tick && s->target_residency_ns >= SHORT_TICK_NSEC) { + tick = 1; + + if (s->target_residency_ns == SHORT_TICK_NSEC) + continue; + + memmove(&drv->states[i+1], &drv->states[i], + sizeof(struct cpuidle_state) * (CPUIDLE_STATE_MAX - i - 1)); + __cpuidle_state_init_tick(s); + drv->state_count++; + i++; + } + } + + if (!tick) { + struct cpuidle_state *s = &drv->states[i]; + __cpuidle_state_init_tick(s); + drv->state_count++; } } --- a/include/linux/cpuidle.h +++ b/include/linux/cpuidle.h @@ -16,7 +16,7 @@ #include #include -#define CPUIDLE_STATE_MAX 10 +#define CPUIDLE_STATE_MAX 16 #define CPUIDLE_NAME_LEN 16 #define CPUIDLE_DESC_LEN 32 From patchwork Wed Aug 2 13:24:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 129857 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp485455vqx; Wed, 2 Aug 2023 07:10:39 -0700 (PDT) X-Google-Smtp-Source: APBJJlEbAFECe58iP3XkHLRmpH9XtYTn6CBblVmKCx2SWArCNPk5GC/j8rDUP6NYC9aiyHb8q1o1 X-Received: by 2002:a05:6a20:dda6:b0:10f:52e2:49ec with SMTP id kw38-20020a056a20dda600b0010f52e249ecmr13317045pzb.53.1690985439426; Wed, 02 Aug 2023 07:10:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690985439; cv=none; d=google.com; s=arc-20160816; b=q50vp293EfQjCyIcd94oNp0yxcXwEl61k+lu5QMWQyz0Bu31+Z9F/4hDH8j5/FaZFr OaprQ4w3WYOSQyoHQ78e4JQWridQuSk4KevEb8jZFzsK80qexHCdqP42W54g9oSQCHJo kIjr4Y528Y5JBadHk8S4ozvTGuDn2AGmj040sGgIUdOjGCLMyX/WxhmMilfAqy7W06i9 VA3GpkWvoFygwP/TOrpJyGvJCEb0QRcP0yidVL1/rAgwh4wCkbrx5KvynE5ep17JPKr4 AFhcZdPXEVvH6iB31OA03F2SRcSWbaMg1gVQMAg+QnqxbtSsuinMgGwBrKs4sDmhSWif psrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=eQsShqCZQhSRZELiV+wvARE7XUysNmI7Pn/FkVFS3UY=; fh=lspZAbmoQCRoxA9wrY498vIgy5FZA7NXITJfi9W+/v8=; b=ceQuYGM47DDwgPOJxc3KheDzU92t066cdLwjuKKjLrfUgwq9RVrEDkw1nZ2DTraoPc s7D/Oy/nH8MH8FAiKDjg30wmRccxISmLd1ipfQdWKwsXhVQhhMffoTKKHoo1vI6Y9CqG /PfzZWTLKysGL3DJbnYIfXS2b851n6wryN8jxi6CmJ9pyY59+Yb0HegFbjEXbIk1xX75 PubRnylrz1H1Eg6w9ltCBOajOfqZMvQbxkqtKoxAlGmpdskPlK7a89v6m9XbRyfOMoy1 m+HL4mEU/I8VISQSvmY/9tIFtrjhZtQi7YpZqp9CzX2dJb7tNWde8GMkdDmeSspCRAQK jcsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=CoKaaQvw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id be3-20020a656e43000000b00563fbbe378asi10938581pgb.59.2023.08.02.07.10.22; Wed, 02 Aug 2023 07:10:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=CoKaaQvw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234317AbjHBNbr (ORCPT + 99 others); Wed, 2 Aug 2023 09:31:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233772AbjHBNbe (ORCPT ); Wed, 2 Aug 2023 09:31:34 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97402269A; Wed, 2 Aug 2023 06:31:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=eQsShqCZQhSRZELiV+wvARE7XUysNmI7Pn/FkVFS3UY=; b=CoKaaQvwg2eF7NA5/aiaXSbUSO NKjRN2+A9m+yrIzlnelbVshYusq0wWHEJjIfqOmc2tEHNYIyC3pFm+zlhPbFKFi5aDDI6j5bNAYHl Xo6lGXsqowLKdsPT9y0eWuX9IwL3ppBs+zRNi5rpFJDe9+ZmJftsDpEHfLyaL8tUvu84bDkoWshP2 cFxLMrucsBCZWPIBecpJM7evZxCMfgK2ivTEQo5eFLWUAnviBmdijWpZZzw3iTKIlJ6CM/eSQ1GiF 4InJSRVopZBFp8pMJPyCZDxdhm9KupCLJIcqPpAz3K+V8uzGQBXu1J5kCVMS135vXTCrH0Hwim2tC OALx8RAA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qRBwZ-00Fxf9-1s; Wed, 02 Aug 2023 13:31:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 1942E300768; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 00BDA2107D7E7; Wed, 2 Aug 2023 15:31:14 +0200 (CEST) Message-ID: <20230802132925.686745535@infradead.org> User-Agent: quilt/0.66 Date: Wed, 02 Aug 2023 15:24:34 +0200 From: Peter Zijlstra To: anna-maria@linutronix.de, rafael@kernel.org, tglx@linutronix.de, frederic@kernel.org, gautham.shenoy@amd.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, daniel.lezcano@linaro.org, linux-pm@vger.kernel.org, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, kajetan.puchalski@arm.com Subject: [RFC][PATCH v2 3/5] cpuidle/teo: Simplify a little References: <20230802132431.254614419@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773126748167860322 X-GMAIL-MSGID: 1773126748167860322 Remove some of the early exit cases that rely on state_count, since we have the additional tick state. Declutters some of the next patches, can possibly be re-instated later if desired. Signed-off-by: Peter Zijlstra (Intel) --- drivers/cpuidle/governors/teo.c | 31 +------------------------------ 1 file changed, 1 insertion(+), 30 deletions(-) --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -187,7 +187,6 @@ struct teo_bin { * @next_recent_idx: Index of the next @recent_idx entry to update. * @recent_idx: Indices of bins corresponding to recent "intercepts". * @util_threshold: Threshold above which the CPU is considered utilized - * @utilized: Whether the last sleep on the CPU happened while utilized */ struct teo_cpu { s64 time_span_ns; @@ -197,7 +196,6 @@ struct teo_cpu { int next_recent_idx; int recent_idx[NR_RECENT]; unsigned long util_threshold; - bool utilized; }; static DEFINE_PER_CPU(struct teo_cpu, teo_cpus); @@ -379,33 +377,6 @@ static int teo_select(struct cpuidle_dri duration_ns = tick_nohz_get_sleep_length(&delta_tick); cpu_data->sleep_length_ns = duration_ns; - /* Check if there is any choice in the first place. */ - if (drv->state_count < 2) { - idx = 0; - goto end; - } - if (!dev->states_usage[0].disable) { - idx = 0; - if (drv->states[1].target_residency_ns > duration_ns) - goto end; - } - - cpu_data->utilized = teo_cpu_is_utilized(dev->cpu, cpu_data); - /* - * If the CPU is being utilized over the threshold and there are only 2 - * states to choose from, the metrics need not be considered, so choose - * the shallowest non-polling state and exit. - */ - if (drv->state_count < 3 && cpu_data->utilized) { - for (i = 0; i < drv->state_count; ++i) { - if (!dev->states_usage[i].disable && - !(drv->states[i].flags & CPUIDLE_FLAG_POLLING)) { - idx = i; - goto end; - } - } - } - /* * Find the deepest idle state whose target residency does not exceed * the current sleep length and the deepest idle state not deeper than @@ -541,7 +512,7 @@ static int teo_select(struct cpuidle_dri * If the CPU is being utilized over the threshold, choose a shallower * non-polling state to improve latency */ - if (cpu_data->utilized) + if (teo_cpu_is_utilized(dev->cpu, cpu_data)) idx = teo_find_shallower_state(drv, dev, idx, duration_ns, true); end: From patchwork Wed Aug 2 13:24:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 129864 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp491152vqx; Wed, 2 Aug 2023 07:18:22 -0700 (PDT) X-Google-Smtp-Source: APBJJlFIWH0r42GAML9CKHW5dz9WTUrZ28AZqC9gOQh+diMapviIlbEC9FgfEdaQ/vIJoorzQe94 X-Received: by 2002:a17:902:7486:b0:1bc:5d0:e8db with SMTP id h6-20020a170902748600b001bc05d0e8dbmr11342760pll.62.1690985902238; Wed, 02 Aug 2023 07:18:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690985902; cv=none; d=google.com; s=arc-20160816; b=hnJE+3ho8A3xTfqenbGMWlZ4pkBE1UWPje7CMzq4z/2UX8GcMtP2SSPJ3jKlP2GS7V aiwF/rhxWc7NZN/hzaA5VfIJqxBwor3aYEEfhFgywd/e04S58Zu9hhks8VFdsEkRp89q GGqAGvVstk087VQzO7ntL888qTliUclsSB5GCw+qf1vS+BGmLNwZMjs98yNUkw8Zbw6t z6BQ4HQVkeMbf5zlUjWDhGWQyHUIbDH/OmblU4I728pzXq4MIV4O8jTsHcx6DnES2IJY YiGEeqMcAcupM8CpJ5iAhjXQNPZrfgbvXi0MDeMxDd+K5KahiEPN2wqnZkIPHcFO51Ti wmaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=6U0TKuKXrlOoIUDQ5shVBrCtBNLJq3+WFZcswjHGIas=; fh=lspZAbmoQCRoxA9wrY498vIgy5FZA7NXITJfi9W+/v8=; b=HqSnNFk+Q4IU23UCmURF4JSPkjs0ToG9HPaVh3KDkrTlph+9AMxQ1cRt4I9EgiyT+f eiV8IVTtcfiIhUNRSygexeZyXsFS0u6RBlhNrb1C6vcAUyDh8QWCTK89V2E1uXQv06x1 b2DX16HUkauimydKe8tYBOelnknWLUMWi4Z0d/Yetcw5gr2zhvJv7HMnahFipyO4cWZI 30Tam1TGJu0eL19GpAVZS6vlYv1BapcDc6pcBY3zdgQXC7ZptmMMO0DxbjDj++sQzD5a /ldQG/7YOFqiMFnMt46KoH8LXRfHbRsDPSJFOEVGHpGTEZesaudNbLUE9uvsjBFUCpij ZXUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=jF3QA3R+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kk15-20020a170903070f00b001b9d800b487si1912171plb.87.2023.08.02.07.17.46; Wed, 02 Aug 2023 07:18:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=jF3QA3R+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234119AbjHBNbf (ORCPT + 99 others); Wed, 2 Aug 2023 09:31:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233127AbjHBNbc (ORCPT ); Wed, 2 Aug 2023 09:31:32 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B2B02690; Wed, 2 Aug 2023 06:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=6U0TKuKXrlOoIUDQ5shVBrCtBNLJq3+WFZcswjHGIas=; b=jF3QA3R+FU2BVbcTtrdX7BmbsT kQjgCqDuI7Ao572IhkkUlGo37pjSWJJhXhZQYhD0b0U0HjS+O8JirD0sPl9cyOHe8DJwiYFELAHQM 8Nv67xq9fV9gEhVxs5RutYYfcqujldqUK0jJn2KPHlO8cx44u/OOSP+PIMNL6p/5IszceRN5PSqXb gzZlKs7dpWYG7faIBj/w7W7Y1CeF+mkRAYaUNxpo6dOVKi+cyu6ea/HslbuD8Oq9R2IiylaC0i54D cyLvPWpCEk7VIL9rqH3tXCJRvgV8x5hvHBxtwHPGTF7edGAfxH45cEbj8vB25BeHNPe06W1DWU5zs B5iW5y3w==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qRBwZ-00FxfA-1v; Wed, 02 Aug 2023 13:31:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 1C0AB3008C6; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 0527D2107D7E5; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Message-ID: <20230802132925.755164086@infradead.org> User-Agent: quilt/0.66 Date: Wed, 02 Aug 2023 15:24:35 +0200 From: Peter Zijlstra To: anna-maria@linutronix.de, rafael@kernel.org, tglx@linutronix.de, frederic@kernel.org, gautham.shenoy@amd.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, daniel.lezcano@linaro.org, linux-pm@vger.kernel.org, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, kajetan.puchalski@arm.com Subject: [RFC][PATCH v2 4/5] cpuidle/teo: Avoid tick_nohz_next_event() References: <20230802132431.254614419@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773127233650208053 X-GMAIL-MSGID: 1773127233650208053 Use the new tick_get_sleep_length() call in conjunction with the new TICK state to elide tick_nohz_next_event() when possible. Specifically, start the state selection using the existing next timer (tick or earlier). And only when state selection lands on the TICK state ask for the NOHZ next timer. Signed-off-by: Peter Zijlstra (Intel) --- drivers/cpuidle/governors/teo.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -139,6 +139,7 @@ #include #include #include +#include "../cpuidle.h" /* * The number of bits to shift the CPU's capacity by in order to determine @@ -363,8 +364,7 @@ static int teo_select(struct cpuidle_dri int constraint_idx = 0; int idx0 = 0, idx = -1; bool alt_intercepts, alt_recent; - ktime_t delta_tick; - s64 duration_ns; + s64 duration_ns, tick_ns; int i; if (dev->last_state_idx >= 0) { @@ -374,8 +374,7 @@ static int teo_select(struct cpuidle_dri cpu_data->time_span_ns = local_clock(); - duration_ns = tick_nohz_get_sleep_length(&delta_tick); - cpu_data->sleep_length_ns = duration_ns; + duration_ns = tick_ns = tick_get_sleep_length(false); /* * Find the deepest idle state whose target residency does not exceed @@ -407,6 +406,14 @@ static int teo_select(struct cpuidle_dri if (s->target_residency_ns > duration_ns) break; + if (s->target_residency_ns == SHORT_TICK_NSEC) { + /* + * We hit the tick state, see if it makes sense to + * disable the tick and go deeper still. + */ + duration_ns = tick_get_sleep_length(true); + } + idx = i; if (s->exit_latency_ns <= latency_req) @@ -417,6 +424,8 @@ static int teo_select(struct cpuidle_dri idx_recent_sum = recent_sum; } + cpu_data->sleep_length_ns = duration_ns; + /* Avoid unnecessary overhead. */ if (idx < 0) { idx = 0; /* No states enabled, must use 0. */ @@ -531,8 +540,8 @@ static int teo_select(struct cpuidle_dri * that. */ if (idx > idx0 && - drv->states[idx].target_residency_ns > delta_tick) - idx = teo_find_shallower_state(drv, dev, idx, delta_tick, false); + drv->states[idx].target_residency_ns > tick_ns) + idx = teo_find_shallower_state(drv, dev, idx, tick_ns, false); } return idx; From patchwork Wed Aug 2 13:24:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Zijlstra X-Patchwork-Id: 129886 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f41:0:b0:3e4:2afc:c1 with SMTP id v1csp525138vqx; Wed, 2 Aug 2023 08:09:39 -0700 (PDT) X-Google-Smtp-Source: APBJJlGoDemXfpApJfZ0BBliZSWAIpjDSuQtC6IKBtiddVGfaOaS+VZv7speLQQze0r+aV67UEkQ X-Received: by 2002:a05:6a21:47c7:b0:13b:a261:5cfe with SMTP id as7-20020a056a2147c700b0013ba2615cfemr19473062pzc.32.1690988979055; Wed, 02 Aug 2023 08:09:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690988979; cv=none; d=google.com; s=arc-20160816; b=UPo3ZcsHGdDEO2FnhJGQKhAIEMa+Ey0TZXJZmagJJhG4J4wGC3+2IRWouKw9VEgSqy NS/Lhs9ccFR6ML8b2ZbobHHqgeMT+1+JgZgEKKitnBvPNUHogsnVrL9hRK8PyBldsODY rHXThrlQ3RVvUnynfzuic1FDrubkvmLe4xN1iSUCIC1BjfCYP6uAqTAyM9lotn4XKBCa q9hTlXJ2+GRB0PtmSEaVkzCOv88EGg++WHllGm33/CWx6YykFeMDZ2XYRSpG6aDeHJYW jPQZcQap7etyMsM87vTbyiOiWVs+ZeI5ADyNNecfha+2ks7iGUyB0YDhxPO9uHJkBqp5 3VHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id:dkim-signature; bh=FEv2iT5iG0jeFDcJt9mW5bYGJa6fvQIcftXsRiIKsDM=; fh=lspZAbmoQCRoxA9wrY498vIgy5FZA7NXITJfi9W+/v8=; b=0dp2Fku97Tu5KiQfEcVmLZNv49T1mv9Fn+KBi8RKYdn6nwH4Srmax3sktApMkC6xOn dXADlygQaoQdkvznadYp5Jym4SmWK+EWCd9pLsL0aY7Q9kPqjMnO1OqAX0X9dBaxqTzG B+M8/9Tmw4oEsjYjeJQz018u5AOe6UjXjoiJUprpbvvSruhsikSqxuroaNXTKj2vEGbl +GJ+OLOoKh8Ndn3594V/SwySjVjoTwmNknhuvRnWQFqiQTpEcsbu2mHEVd2xQGAHaqP0 aVDoxegYzTO5Gov+41442JKKDCk0eK4UuZRlkI/nnGuhWzbvSWZnjwGtwrzq0QeZb9yL Eziw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=OJeHWtCn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h191-20020a636cc8000000b005644829012fsi673808pgc.701.2023.08.02.08.09.23; Wed, 02 Aug 2023 08:09:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=casper.20170209 header.b=OJeHWtCn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234357AbjHBNbu (ORCPT + 99 others); Wed, 2 Aug 2023 09:31:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234161AbjHBNbf (ORCPT ); Wed, 2 Aug 2023 09:31:35 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D86302698; Wed, 2 Aug 2023 06:31:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=FEv2iT5iG0jeFDcJt9mW5bYGJa6fvQIcftXsRiIKsDM=; b=OJeHWtCn/qCA8UDeTUprU4JcoE 77B5rES8YJQG+Q34u94zXR/7WTH2Or1bRj2FBQDgfqyoVbbhzgSlM58+STeDCDdBKRDhygd1kseue CZmWUYiCbUxqCXDdVvTAD+m4In1W14yavTbJ/LOI8YuWgbV6XlFA0xDIfjfKpZ3Dxdtv19mss6loJ rRfiEacR0MiXzF5PPxbZcZ6dd8IHuh8Wz8s6n8CA2TXE3+BorgkSgKhIEEyOEO3leo4VBS6/hKCJ+ 9PGvGM7SgNwVSXAGA3lcMmmvP64s3JDMdBMzXPLpXRggMhrWL8KyTOyuHAUGThBTh0GonGYJgAuwO DjZ5gJrw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1qRBwa-00FBnY-5G; Wed, 02 Aug 2023 13:31:16 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 56827300DCF; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 0788F2107D7EA; Wed, 2 Aug 2023 15:31:15 +0200 (CEST) Message-ID: <20230802132925.824393002@infradead.org> User-Agent: quilt/0.66 Date: Wed, 02 Aug 2023 15:24:36 +0200 From: Peter Zijlstra To: anna-maria@linutronix.de, rafael@kernel.org, tglx@linutronix.de, frederic@kernel.org, gautham.shenoy@amd.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, daniel.lezcano@linaro.org, linux-pm@vger.kernel.org, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, kajetan.puchalski@arm.com Subject: [RFC][PATCH v2 5/5] cpuidle,teo: Improve state selection References: <20230802132431.254614419@infradead.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773130459441361387 X-GMAIL-MSGID: 1773130459441361387 When selecting a state, stop when history tells us 66% of recent idles were at or below our current state. Signed-off-by: Peter Zijlstra (Intel) --- drivers/cpuidle/governors/teo.c | 6 ++++++ 1 file changed, 6 insertions(+) --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -361,6 +361,7 @@ static int teo_select(struct cpuidle_dri unsigned int recent_sum = 0; unsigned int idx_hit_sum = 0; unsigned int hit_sum = 0; + unsigned int thresh_sum = 0; int constraint_idx = 0; int idx0 = 0, idx = -1; bool alt_intercepts, alt_recent; @@ -376,6 +377,8 @@ static int teo_select(struct cpuidle_dri duration_ns = tick_ns = tick_get_sleep_length(false); + thresh_sum = 2 * cpu_data->total / 3; /* 66% */ + /* * Find the deepest idle state whose target residency does not exceed * the current sleep length and the deepest idle state not deeper than @@ -406,6 +409,9 @@ static int teo_select(struct cpuidle_dri if (s->target_residency_ns > duration_ns) break; + if (intercept_sum + hit_sum > thresh_sum) + break; + if (s->target_residency_ns == SHORT_TICK_NSEC) { /* * We hit the tick state, see if it makes sense to