From patchwork Wed Jan 31 23:11:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195054 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp90848dyc; Wed, 31 Jan 2024 15:32:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IFD9DmEoe4S15w/oK1Fx65xq1OuYGvva4g6WOQ8FVWzspYCyWfusNO1YO4HcmnI0ljjmICH X-Received: by 2002:a05:6512:488c:b0:510:f47:8d82 with SMTP id eq12-20020a056512488c00b005100f478d82mr493368lfb.22.1706743922878; Wed, 31 Jan 2024 15:32:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743922; cv=pass; d=google.com; s=arc-20160816; b=W602scij7qAphYwKmu6V4P1w6o+Ic5lSUXmkmJF421j4UUXivPMhoWOacy6jclBxAH KCYlHe99IrU6RdMmPoFdmRXrg4aoHYKbAma6b051d5OlZmGSOZbEM/KopqBkzf+ZASwo r/08jlCcThbg0YvEuJiBh7nrbxcZVJOXhiTdvXfUiKMdclJPbYlPNIIL0/oA7Bgz0Ico hpZ5ydjL3F0Ii4TZzae5evahgdCALpxMj+YuEa2GJWn7rlThInixhIexVt1znmnPTYqI RPqbG+dEK71UiriZsDq/VuviuZuZOFQA/jymU8A2qLC8sec4E1vBYTyq9LqoSb5Ssi53 dRbw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=4FNnF5rE5cvlPQ4c78h6C4TNDC4iD5rBBiuRXy/ICCI=; fh=epOPia7mcRsVHOric4UylZLvbQ3DwB+bJbmZua6DgOU=; b=MOr0mYmVGekORwAiuq+BHLs90l6OU9HZeR2j/TZoTUkSxpPmG0MQnGDJd1mux8LQBk YJS3TGRaq26ZJJONGiojRf9Pe23UEnwV5cRr6vesy7z5lAF+IJfAgxWn+pmVkCkIlMIr a5FTNRx7sYIzsH49PWQRK4HxpLgdhe/k81m6Ma73wEjATVi9RtaHREEMc3uxGfsPyc3z 1Anz08DgBwYT1ENJag60g95cjSneOKpcuHXtNy52r7ngAyDdUiT6KJzr0mQO0yLesTJm 4FvcOmXSBIX8Rj2u54hzJ4xKNZq3yAZnmit+QRVdoEvkJbOpSiIZrGzClIC2sI611U25 tZpQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TigkfeWi; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCWBRZA/VOuVh+zglZe6cPdxKjx+v5NMPf3NzKvBTaUlwshivGFVV6qeQ+3cxkOcYe1hu9nQZtaOxauzuSyRaLRDfPwSow== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id c12-20020a17090654cc00b00a353ca3d908si5084898ejp.556.2024.01.31.15.32.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:32:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TigkfeWi; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 794011F22CAB for ; Wed, 31 Jan 2024 23:31:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 40D971386DF; Wed, 31 Jan 2024 23:12:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TigkfeWi" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C884487A0 for ; Wed, 31 Jan 2024 23:11:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742716; cv=none; b=U9YKb50H0UyqvUtTTnGHIB8Cb4ZgsbA0kqF7+tPhxyDkrJQzGTeORCVT34SeLjY9wzEw7gSfu+IdBQGtSCIQIVVXJwlxUpjv5EQo54+z1jvtwah6WYraurTmQjF2cEsuRM4tYu35fcAsyTs+F9pP4fO2O9/+HWq/VCZyqKUYKYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742716; c=relaxed/simple; bh=PFXG2cNgodV6VNtmvaOYQJZ4ukS7LvHmLJbcHg8But0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kC6Nw7yq9KZechh4H65Tux2vDcwLS58kpfKbx4WNpuanTuoM7/yCpg3NBECDXTbhgj81R5gGAT3Jg0VKmmbv4lLll+Fbtx5OwTVMMuabt+zAG2fsJ2nDx+2ON5QRjWmbxaPWZzB4LnvvRV1S6/wf8qumu4Ctdvwuk6XPhrRPqwM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TigkfeWi; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27B48C43390; Wed, 31 Jan 2024 23:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742716; bh=PFXG2cNgodV6VNtmvaOYQJZ4ukS7LvHmLJbcHg8But0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TigkfeWi5OciTxd6vouIcuP3VbvngwUHh9OxOQE1NDfq+rhevMECMtUo4UKMwBCJo GabMt7pLZ6CRDFCrtnm8oCa/FNRmzMKn6uf+cf8l+4+fkpKbGK2Jan6HQJSBmr1/y3 /1Wq5ERemJpEmds2wUCAHPmnkNegrKO02UEhPG0Ax4UhcCPgVdDIL81Iz9LeRYez4f 694uCydc7xTLuQ+4GeNRmRc/vl9VvPKzx/5kybi5m3ANqzmm69WNFp9cKhyEzNMWmx gG4G3jgpCjDHi6GNZxlfy6DzLMQ/HgiOyiXLk9oKYzFGZNkXHKTzVjlrqBB8nnYU5+ 65PeHnOjdwoxQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 13/15] tick: Split nohz and highres features from nohz_mode Date: Thu, 1 Feb 2024 00:11:18 +0100 Message-ID: <20240131231120.12006-14-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650715705764942 X-GMAIL-MSGID: 1789650715705764942 The nohz mode field tells about low resolution nohz mode or high resolution nohz mode but it doesn't tell about high resolution non-nohz mode. In order to retrieve the latter state, tick_cancel_sched_timer() must fiddle with struct hrtimer's internals to guess if the tick has been initialized in high resolution. Move instead the nohz mode field information into the tick flags and provide two new bits: one to know if the tick is in nohz mode and another one to know if the tick is in high resolution. The combination of those two flags provides all the needed informations to determine which of the three tick modes is running. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/hrtimer.c | 2 +- kernel/time/tick-sched.c | 32 +++++++++++++++++--------------- kernel/time/tick-sched.h | 13 +++++-------- kernel/time/timer_list.c | 5 +++-- 4 files changed, 26 insertions(+), 26 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 3b456ec7d4fb..d8ba7985fe0d 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -746,7 +746,7 @@ static void hrtimer_switch_to_hres(void) base->hres_active = 1; hrtimer_resolution = HIGH_RES_NSEC; - tick_setup_sched_timer(NOHZ_MODE_HIGHRES); + tick_setup_sched_timer(true); /* "Retrigger" the interrupt to get things going */ retrigger_next_event(NULL); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 3e0a53c1b8bd..67759e7e025a 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -831,7 +831,7 @@ static void tick_nohz_restart(struct tick_sched *ts, ktime_t now) /* Forward the time to expire in the future */ hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) { + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) { hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); } else { @@ -997,14 +997,14 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * the tick timer. */ if (unlikely(expires == KTIME_MAX)) { - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) hrtimer_cancel(&ts->sched_timer); else tick_program_event(KTIME_MAX, 1); return; } - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) { + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) { hrtimer_start(&ts->sched_timer, expires, HRTIMER_MODE_ABS_PINNED_HARD); } else { @@ -1065,7 +1065,7 @@ static void tick_nohz_full_update_tick(struct tick_sched *ts) if (!tick_nohz_full_cpu(smp_processor_id())) return; - if (ts->nohz_mode == NOHZ_MODE_INACTIVE) + if (!tick_sched_flag_test(ts, TS_FLAG_NOHZ)) return; __tick_nohz_full_update_tick(ts, ktime_get()); @@ -1125,7 +1125,7 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) return false; } - if (unlikely(ts->nohz_mode == NOHZ_MODE_INACTIVE)) + if (unlikely(!tick_sched_flag_test(ts, TS_FLAG_NOHZ))) return false; if (need_resched()) @@ -1449,11 +1449,11 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev) tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); } -static inline void tick_nohz_activate(struct tick_sched *ts, int mode) +static inline void tick_nohz_activate(struct tick_sched *ts) { if (!tick_nohz_enabled) return; - ts->nohz_mode = mode; + tick_sched_flag_set(ts, TS_FLAG_NOHZ); /* One update is enough */ if (!test_and_set_bit(0, &tick_nohz_active)) timers_update_nohz(); @@ -1474,7 +1474,7 @@ static void tick_nohz_switch_to_nohz(void) * Recycle the hrtimer in 'ts', so we can share the * highres code. */ - tick_setup_sched_timer(NOHZ_MODE_LOWRES); + tick_setup_sched_timer(false); } static inline void tick_nohz_irq_enter(void) @@ -1502,7 +1502,7 @@ static inline void tick_nohz_irq_enter(void) static inline void tick_nohz_switch_to_nohz(void) { } static inline void tick_nohz_irq_enter(void) { } -static inline void tick_nohz_activate(struct tick_sched *ts, int mode) { } +static inline void tick_nohz_activate(struct tick_sched *ts) { } #endif /* CONFIG_NO_HZ_COMMON */ @@ -1529,15 +1529,17 @@ early_param("skew_tick", skew_tick); * tick_setup_sched_timer - setup the tick emulation timer * @mode: tick_nohz_mode to setup for */ -void tick_setup_sched_timer(int mode) +void tick_setup_sched_timer(bool hrtimer) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); /* Emulate tick processing via per-CPU hrtimers: */ hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && hrtimer) { + tick_sched_flag_set(ts, TS_FLAG_HIGHRES); ts->sched_timer.function = tick_nohz_handler; + } /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); @@ -1551,11 +1553,11 @@ void tick_setup_sched_timer(int mode) } hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && hrtimer) hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); else tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - tick_nohz_activate(ts, mode); + tick_nohz_activate(ts); } void tick_cancel_sched_timer(int cpu) @@ -1564,7 +1566,7 @@ void tick_cancel_sched_timer(int cpu) ktime_t idle_sleeptime, iowait_sleeptime; unsigned long idle_calls, idle_sleeps; - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && ts->sched_timer.base) + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) hrtimer_cancel(&ts->sched_timer); idle_sleeptime = ts->idle_sleeptime; @@ -1614,7 +1616,7 @@ int tick_check_oneshot_change(int allow_nohz) if (!test_and_clear_bit(0, &ts->check_clocks)) return 0; - if (ts->nohz_mode != NOHZ_MODE_INACTIVE) + if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) return 0; if (!timekeeping_valid_for_hres() || !tick_is_oneshot_available()) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 07a4c0144c47..bbe72a078985 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -14,12 +14,6 @@ struct tick_device { enum tick_device_mode mode; }; -enum tick_nohz_mode { - NOHZ_MODE_INACTIVE, - NOHZ_MODE_LOWRES, - NOHZ_MODE_HIGHRES, -}; - /* The CPU is in the tick idle mode */ #define TS_FLAG_INIDLE BIT(0) /* The idle tick has been stopped */ @@ -31,6 +25,10 @@ enum tick_nohz_mode { #define TS_FLAG_IDLE_ACTIVE BIT(2) /* CPU was the last one doing do_timer before going idle */ #define TS_FLAG_DO_TIMER_LAST BIT(3) +/* NO_HZ is enabled */ +#define TS_FLAG_NOHZ BIT(4) +/* High resolution tick mode */ +#define TS_FLAG_HIGHRES BIT(5) /** * struct tick_sched - sched tick emulation and no idle tick control/stats @@ -84,7 +82,6 @@ struct tick_sched { ktime_t idle_entrytime; /* Tick stop */ - enum tick_nohz_mode nohz_mode; unsigned long last_jiffies; u64 timer_expires_base; u64 timer_expires; @@ -107,7 +104,7 @@ struct tick_sched { extern struct tick_sched *tick_get_tick_sched(int cpu); -extern void tick_setup_sched_timer(int mode); +extern void tick_setup_sched_timer(bool hrtimer); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS extern void tick_cancel_sched_timer(int cpu); #else diff --git a/kernel/time/timer_list.c b/kernel/time/timer_list.c index 38f81d836fc5..1c311c46da50 100644 --- a/kernel/time/timer_list.c +++ b/kernel/time/timer_list.c @@ -152,7 +152,8 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now) { struct tick_sched *ts = tick_get_tick_sched(cpu); - P(nohz_mode); + P_flag(nohz, TS_FLAG_NOHZ); + P_flag(highres, TS_FLAG_HIGHRES); P_ns(last_tick); P_flag(tick_stopped, TS_FLAG_STOPPED); P(idle_jiffies); @@ -259,7 +260,7 @@ static void timer_list_show_tickdevices_header(struct seq_file *m) static inline void timer_list_header(struct seq_file *m, u64 now) { - SEQ_printf(m, "Timer List Version: v0.9\n"); + SEQ_printf(m, "Timer List Version: v0.10\n"); SEQ_printf(m, "HRTIMER_MAX_CLOCK_BASES: %d\n", HRTIMER_MAX_CLOCK_BASES); SEQ_printf(m, "now at %Ld nsecs\n", (unsigned long long)now); SEQ_printf(m, "\n");