From patchwork Wed Feb 8 15:19:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 54459 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp3519223wrn; Wed, 8 Feb 2023 07:29:05 -0800 (PST) X-Google-Smtp-Source: AK7set+uU1P8VVmgKJVQTr26gBzKTpVmUTuc7fzzzuO+YCZRYstk/vwgOytdeNtMLwxwX1REcjq+ X-Received: by 2002:a05:6a21:3398:b0:be:dd6f:3ffd with SMTP id yy24-20020a056a21339800b000bedd6f3ffdmr9895664pzb.6.1675870144904; Wed, 08 Feb 2023 07:29:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675870144; cv=none; d=google.com; s=arc-20160816; b=qkh/YhOg+icmzsYWaH/ZPCtUKAohKLkwGHwzHBOQVzFu33+PNptwKIA7B5pf1vjgEw IjIvFRFzDi767Lv0/tS8o0IBV9JMyYom2ijl9FwAlxtX2gJ/FY+Xr8NMwBY8OvamPVd7 Uoik10qeVKqk+8bKBuTlI/QzuVJg2xo6a7735W1/QZ8mzY29caTEY+ylgGKb7/uPg5RT O0cTQoxHCGKiUvrpCb5P7t9Md0/deue4oA0hHWxgQXbLl8/bHslBjPB9WHMGlPq10y/w MtgJrqxuZmBlYlXtT8YBzTiKVsBsV/ImCvwvgJbWNQvjF13tAN6RIubckmxUhggPsDs2 HSQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=GtNhQRz+NIk9AMOl6/pv9rKJeWdTcmvv3LlO73sQQbQ=; b=SeBmWdU6RmDbbpeUB3/Ux3PiLnes25KVcvlGebGEaaofU2X6MaOhW2ZPyFrsbFClOk sxpcFzHr2Qj8Vs8QRbZ9KmUltYNlutQhG4XN3qdLBOEsfgMWG1RlNVWZS8TOSq7HE81I Rik/WaA5Ij6NvP3a5rk6vmoGSO7+vBNsl//6YMCwFMvJnzVSZedRREqhG/+D6gVPgKTl 2jCaiopDyWuLPdOYeOrGOpevwFHGvTFnlBvttjKxMEMXHs4NekK/4G0LFyKP6aP4qZPY UEH+CeNwpShwfT5iSMQo+EUKzteM3ylww+NoydF+B65rGOpar91XVDyBW2G1iXuOUqTr SwhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PPe7WJJm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t141-20020a635f93000000b0047711269d75si19266669pgb.516.2023.02.08.07.28.51; Wed, 08 Feb 2023 07:29:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PPe7WJJm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230486AbjBHPTh (ORCPT + 99 others); Wed, 8 Feb 2023 10:19:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231433AbjBHPTg (ORCPT ); Wed, 8 Feb 2023 10:19:36 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF0A41E9D0 for ; Wed, 8 Feb 2023 07:19:34 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5BDC0616EE for ; Wed, 8 Feb 2023 15:19:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 380F7C433D2; Wed, 8 Feb 2023 15:19:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1675869573; bh=MtJnJJ6b33n0/jOJNoXl97RZ3B1Mim0+uCOBp8fF7YU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=PPe7WJJmiROwbI47CQqsSOtI8tfmPNTiV7XC5X2qqbWLXadO/GyUCOWmtJigQsqa0 WF7BA+irZM8HLNSRqoO+buG5+biYdYZGBCFQVUAldgAoijPD61CZIrMPGOX11O/Umj n06T5Q/oKpefknR3LlCGjpqg/0VZ4EeSDQ8pR8x9s+flxnHHAICQvF0uvI/089mTes Nop81c15SjffXyQ8gvpcZGkPmv37QHU6yRdqrXiklNXjineRdHiEFm0Wye4lykOxun q+kgez/NNAjoHnuDsXA8Ul842LKqG/qqaN05G75SE/iAe9ucPpu7Veoboe1w1M3HEp MvxAtddYEhcmg== Date: Wed, 8 Feb 2023 16:19:30 +0100 From: Frederic Weisbecker To: Thomas Gleixner Cc: Yu Liao , fweisbec@gmail.com, mingo@kernel.org, liwei391@huawei.com, adobriyan@gmail.com, mirsad.todorovac@alu.unizg.hr, linux-kernel@vger.kernel.org, Peter Zijlstra Subject: [PATCH] timers/nohz: Restructure and reshuffle struct tick_sched Message-ID: References: <20230128020051.2328465-1-liaoyu15@huawei.com> <87357q228f.ffs@tglx> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87357q228f.ffs@tglx> X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757277213133732279?= X-GMAIL-MSGID: =?utf-8?q?1757277213133732279?= On Tue, Jan 31, 2023 at 03:44:00PM +0100, Thomas Gleixner wrote: > @Frederic: This structure is patently unstructured and if we fix this > issue here then we really want to look at the cache line layout for > real. I'm not really good at that, but here is my try: --- From: Frederic Weisbecker Date: Wed, 8 Feb 2023 16:08:11 +0100 Subject: [PATCH] timers/nohz: Restructure and reshuffle struct tick_sched Restructure and group fields by access in order to optimize cache layout. While at it, also add missing kernel doc for two fields: @last_jiffies and @idle_expires. Reported-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.h | 66 +++++++++++++++++++++++++--------------- 1 file changed, 41 insertions(+), 25 deletions(-) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 504649513399..c6663254d17d 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -22,65 +22,81 @@ enum tick_nohz_mode { /** * struct tick_sched - sched tick emulation and no idle tick control/stats - * @sched_timer: hrtimer to schedule the periodic tick in high - * resolution mode - * @check_clocks: Notification mechanism about clocksource changes - * @nohz_mode: Mode - one state of tick_nohz_mode + * * @inidle: Indicator that the CPU is in the tick idle mode * @tick_stopped: Indicator that the idle tick has been stopped * @idle_active: Indicator that the CPU is actively in the tick idle mode; * it is reset during irq handling phases. - * @do_timer_lst: CPU was the last one doing do_timer before going idle + * @do_timer_last: CPU was the last one doing do_timer before going idle * @got_idle_tick: Tick timer function has run with @inidle set + * @stalled_jiffies: Number of stalled jiffies detected across ticks + * @last_tick_jiffies: Value of jiffies seen on last tick + * @sched_timer: hrtimer to schedule the periodic tick in high + * resolution mode * @last_tick: Store the last tick expiry time when the tick * timer is modified for nohz sleeps. This is necessary * to resume the tick timer operation in the timeline * when the CPU returns from nohz sleep. * @next_tick: Next tick to be fired when in dynticks mode. * @idle_jiffies: jiffies at the entry to idle for idle time accounting + * @idle_waketime: Time when the idle was interrupted + * @idle_entrytime: Time when the idle call was entered + * @nohz_mode: Mode - one state of tick_nohz_mode + * @last_jiffies: Base jiffies snapshot when next event was last computed + * @timer_expires_base: Base time clock monotonic for @timer_expires + * @timer_expires: Anticipated timer expiration time (in case sched tick is stopped) + * @next_timer: Expiry time of next expiring timer for debugging purpose only + * @idle_expires: Next tick in idle, for debugging purpose only * @idle_calls: Total number of idle calls * @idle_sleeps: Number of idle calls, where the sched tick was stopped - * @idle_entrytime: Time when the idle call was entered - * @idle_waketime: Time when the idle was interrupted * @idle_exittime: Time when the idle state was left * @idle_sleeptime: Sum of the time slept in idle with sched tick stopped * @iowait_sleeptime: Sum of the time slept in idle with sched tick stopped, with IO outstanding - * @timer_expires: Anticipated timer expiration time (in case sched tick is stopped) - * @timer_expires_base: Base time clock monotonic for @timer_expires - * @next_timer: Expiry time of next expiring timer for debugging purpose only * @tick_dep_mask: Tick dependency mask - is set, if someone needs the tick - * @last_tick_jiffies: Value of jiffies seen on last tick - * @stalled_jiffies: Number of stalled jiffies detected across ticks + * @check_clocks: Notification mechanism about clocksource changes */ struct tick_sched { - struct hrtimer sched_timer; - unsigned long check_clocks; - enum tick_nohz_mode nohz_mode; - + /* Common flags */ unsigned int inidle : 1; unsigned int tick_stopped : 1; unsigned int idle_active : 1; unsigned int do_timer_last : 1; unsigned int got_idle_tick : 1; + /* Tick handling: jiffies stall check */ + unsigned int stalled_jiffies; + unsigned long last_tick_jiffies; + + /* Tick handling */ + struct hrtimer sched_timer; ktime_t last_tick; ktime_t next_tick; unsigned long idle_jiffies; - unsigned long idle_calls; - unsigned long idle_sleeps; - ktime_t idle_entrytime; ktime_t idle_waketime; - ktime_t idle_exittime; - ktime_t idle_sleeptime; - ktime_t iowait_sleeptime; + + /* Idle entry */ + ktime_t idle_entrytime; + + /* Tick stop */ + enum tick_nohz_mode nohz_mode; unsigned long last_jiffies; - u64 timer_expires; u64 timer_expires_base; + u64 timer_expires; u64 next_timer; ktime_t idle_expires; + unsigned long idle_calls; + unsigned long idle_sleeps; + + /* Idle exit */ + ktime_t idle_exittime; + ktime_t idle_sleeptime; + ktime_t iowait_sleeptime; + + /* Full dynticks handling */ atomic_t tick_dep_mask; - unsigned long last_tick_jiffies; - unsigned int stalled_jiffies; + + /* Clocksource changes */ + unsigned long check_clocks; }; extern struct tick_sched *tick_get_tick_sched(int cpu);