From patchwork Wed Jan 31 23:11:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195058 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp92422dyc; Wed, 31 Jan 2024 15:35:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IGZGZtobg7pMjKUmuHg1m4veLMOzpuVjaEbczNITwUKjhVEv2fYMzKItiKeKwvvaFZRLkXG X-Received: by 2002:a05:6a20:8813:b0:19e:30a6:fe29 with SMTP id c19-20020a056a20881300b0019e30a6fe29mr4057323pzf.31.1706744146033; Wed, 31 Jan 2024 15:35:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744146; cv=pass; d=google.com; s=arc-20160816; b=neCS3qmo86pZitk/K9k0tCGEAQu5CmiZggVdKKovY7gj/Cl8NdmaxjfzWj6vuIKBep shk0P3xtRJo4Nbe7bMeJau3fxTiD4YA22VOaAaqq3A76DktfMr+iREWVpSMe6fszggLj HsWPolvr5Q2VSjS89C/VBPLwBV+85Hm4vIlnB/wvX47w21XB8/DD8NGyJEmfhZhLCBlg NHSQjwhb7BBf+EGdvm4J/+annic1BJUwxdCC2dxD5j4aFCIVIK3mou1lCU+QYz7cMxJJ it8ltjj6Lv2MoTqEBERpXWU9armbVCdjHt0GpQCpCLNkVNrFIiKkfcCXSl2As6YbH7eD 8QJA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=bqdgz4a2Vosy44Kq9W9c7kQ07JCUTKLlnzlAUHAChGw=; fh=ScL1KgNlsCSMOMls8FdVes2RtNBfPF5B94OUciFWjRI=; b=WQGzIzyCVdgM3fR1C9d/r/2UQGVz5li2kydP0RWMbvV+Bb2MDvyDfziGCHuGuHssAU uGe71z6VMfot9kpjd4O4G1Bvrby+oABDWrjbw/2OA5xm4z76+lX/+ZGIOX3wKX2aQAvl qltD/C1NYLW629KGqYzVf8buuKsfOMaLYm37ksZ8m2fk//YuTOB+O3SezgAC3pbIfjUv ioNVDMNQyv18ljCklYbAA279idLSMEYkt1xPQT6cLaBp5rdZueIzBg1Fo9KfHclIDjnR oojjqO9tq7iBrLyM3c5uKiRCFxSP6O/DG6hw5KkVOf64YT/V2fQvSHpJePZ8k5C0/GEQ cPzw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Jcz0RxS5; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47345-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47345-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCWATJ+E9c0h3pykZIioN0h+ONBG2tGO9SD/XVt7SGOaY8n2Xmco7ZGTvTZylL2KFd6FQqvhCp2oev7nKv7dBPt27nQvNA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id h14-20020a65638e000000b005d8b5aace83si8666474pgv.289.2024.01.31.15.35.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:35:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47345-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Jcz0RxS5; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47345-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47345-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id ACB3128DFC9 for ; Wed, 31 Jan 2024 23:28:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B62B9482C9; Wed, 31 Jan 2024 23:11:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Jcz0RxS5" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CC0454779F for ; Wed, 31 Jan 2024 23:11:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742688; cv=none; b=upUUaGIgKKRra+kYquCw5MPlk6jN/Ew/F1uZ+dtmCy6I5dvuo24nWeNTTBtWlLor23UYuAQSPwQdCRfew98jXcf0NnYhtm7SeqT5+DrrLfLf7qkGTnfFCapARyQo9G+1VGPnAgHJfvQEGLHvm6jSapj+jwokKAuU8FxlFKSztls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742688; c=relaxed/simple; bh=UXIzCahLdyiQvMBsYWJwhIWl4FuY8yQGzDXMDNNxt3s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fUufyCZq01goZkf/QQdx5aTNSLnkjxiOraqojxUcmxZ/eQOaIuWTl/UgnOjsVVBG9dSMKTaJT8uoufVj5fq+aSmsD8F8nr3SSk6aqip7x8b8Nwt25Swkd+CqzKmzvxfgJhVsiy0x5rWsA2RC7RwmRVXXZqjAv2fkZpquekefCkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Jcz0RxS5; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D0D1C43390; Wed, 31 Jan 2024 23:11:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742688; bh=UXIzCahLdyiQvMBsYWJwhIWl4FuY8yQGzDXMDNNxt3s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Jcz0RxS5Nj6zTmOchivsDrJJ120f5+DYPsEVze44edHqmnt2RypXDM6mik0J2bz4q 15EN8gIwd1ajpbqwk0QKKI93HDFSurVxuTtLKM8MirV82qcl3ml7IurOVx3VEdriqz +2Frq3VQEI+8PBy94aNsjjO7/GT5jm5gvAJtaH7bWMfH9deLqMFMpdNRu7f7drL58J LDr9ibG6obCZz2BASGdVB7sdwymAZYBJ9+FhFoQlqvQ0e7ShPGSAu45zHC7yQOe0Qd 30FcJwzziCZ3aFxJP5W2iCAoE+SAKvqcq6D3JTzpgTuO0rfbGyWLfxabgbaneL42QQ vySgfhS8/Azhg== From: Frederic Weisbecker To: LKML Cc: Peng Liu , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Joel Fernandes , Frederic Weisbecker Subject: [PATCH 01/15] tick/nohz: Remove duplicate between tick_nohz_switch_to_nohz() and tick_setup_sched_timer() Date: Thu, 1 Feb 2024 00:11:06 +0100 Message-ID: <20240131231120.12006-2-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650949976260734 X-GMAIL-MSGID: 1789650949976260734 From: Peng Liu The ts->sched_timer initialization work of tick_nohz_switch_to_nohz() is almost the same as that of tick_setup_sched_timer(), so adjust the latter to get it reused by tick_nohz_switch_to_nohz(). This also makes low-res mode sched_timer benefit from the tick skew boot option. Signed-off-by: Peng Liu Signed-off-by: Frederic Weisbecker --- kernel/time/hrtimer.c | 2 +- kernel/time/tick-sched.c | 39 ++++++++++++++++++--------------------- kernel/time/tick-sched.h | 2 +- 3 files changed, 20 insertions(+), 23 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 760793998cdd..355b5a957f7f 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -746,7 +746,7 @@ static void hrtimer_switch_to_hres(void) base->hres_active = 1; hrtimer_resolution = HIGH_RES_NSEC; - tick_setup_sched_timer(); + tick_setup_sched_timer(NOHZ_MODE_HIGHRES); /* "Retrigger" the interrupt to get things going */ retrigger_next_event(NULL); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 01fb50c1b17e..639ee689e7cc 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1433,9 +1433,6 @@ static inline void tick_nohz_activate(struct tick_sched *ts, int mode) */ static void tick_nohz_switch_to_nohz(void) { - struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - ktime_t next; - if (!tick_nohz_enabled) return; @@ -1444,16 +1441,9 @@ static void tick_nohz_switch_to_nohz(void) /* * Recycle the hrtimer in 'ts', so we can share the - * hrtimer_forward_now() function with the highres code. + * highres code. */ - hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - /* Get the next period */ - next = tick_init_jiffy_update(); - - hrtimer_set_expires(&ts->sched_timer, next); - hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); - tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - tick_nohz_activate(ts, NOHZ_MODE_LOWRES); + tick_setup_sched_timer(NOHZ_MODE_LOWRES); } static inline void tick_nohz_irq_enter(void) @@ -1532,7 +1522,11 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) return HRTIMER_RESTART; } +#else +#define tick_nohz_highres_handler NULL +#endif /* CONFIG_HIGH_RES_TIMERS */ +#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS static int sched_skew_tick; static int __init skew_tick(char *str) @@ -1545,15 +1539,17 @@ early_param("skew_tick", skew_tick); /** * tick_setup_sched_timer - setup the tick emulation timer + * @mode: tick_nohz_mode to setup for */ -void tick_setup_sched_timer(void) +void tick_setup_sched_timer(int mode) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - ktime_t now = ktime_get(); /* Emulate tick processing via per-CPU hrtimers: */ hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - ts->sched_timer.function = tick_nohz_highres_handler; + + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) + ts->sched_timer.function = tick_nohz_highres_handler; /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); @@ -1566,13 +1562,14 @@ void tick_setup_sched_timer(void) hrtimer_add_expires_ns(&ts->sched_timer, offset); } - hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); - hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); - tick_nohz_activate(ts, NOHZ_MODE_HIGHRES); + hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) + hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); + else + tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); + tick_nohz_activate(ts, mode); } -#endif /* HIGH_RES_TIMERS */ -#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS void tick_cancel_sched_timer(int cpu) { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); @@ -1594,7 +1591,7 @@ void tick_cancel_sched_timer(int cpu) ts->idle_calls = idle_calls; ts->idle_sleeps = idle_sleeps; } -#endif +#endif /* CONFIG_NO_HZ_COMMON || CONFIG_HIGH_RES_TIMERS */ /* * Async notification about clocksource changes diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 5ed5a9d41d5a..35808bbb8a47 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -102,7 +102,7 @@ struct tick_sched { extern struct tick_sched *tick_get_tick_sched(int cpu); -extern void tick_setup_sched_timer(void); +extern void tick_setup_sched_timer(int mode); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS extern void tick_cancel_sched_timer(int cpu); #else From patchwork Wed Jan 31 23:11:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195046 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp89184dyc; Wed, 31 Jan 2024 15:28:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IFr1H8wtsyM/NJRaGfllEwLv7ORdMwtus+/n/cu8zrcBp6N9vW2GdLYtxVJOaxRezJulazP X-Received: by 2002:a05:622a:1006:b0:42a:9cfc:bdc9 with SMTP id d6-20020a05622a100600b0042a9cfcbdc9mr6231582qte.46.1706743713525; Wed, 31 Jan 2024 15:28:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743713; cv=pass; d=google.com; s=arc-20160816; b=0kEQgV6jCBKgXGkxye1V4l0DD4z29E0pwGL0yoH0lpo8dxIKEXNkvB4uf9Er+qWaUR NXtaTSATBTzsNrFujJc7ANXgGc1lyHrhRQ/p1z4p5V1LL8Ki0mvb6GeCp02eJuFcUsYe HP8ea9JyFA8CuhGzpAtXPK6pCJEjPvbwGscv3f5kKKrthA3pidwhg3TG7q6O/KevJxTm M4Snf4v/JsU9xKTEJonl4CsjEkm0r0NGMbMuOB0FUWnhhbRYwb34dZmA4pkGMw+QVZTI kMcLIeWlAdjnILUP3bnKBG2lFAshM3DWkvTuiffUw0jrTtuo8aWiYtHdhk+dZOGB5OnA 1I3Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=+L+HSC8rOLuZ0WMiTgHGVL4FnKICnnw5RkLFVgIGjbg=; fh=5HlbKeZTyfNHU+ImhrpQsymbj5H7PQdULDvEu6vYMoc=; b=vHxVBF8hELqgXUa46Lh5ofrTKbhOWO3/zZJxzQ6+umRHodwA5LYR6zo4F6IxCORMmf Dx0lKcityWZSGo48UPAUUSH4GOoGLhMo2ZvmYszCTakW8DE2g4ZJ7cVuHnMSKzHg0Jh8 BBbtZcmP2qxy7hYW+2/s8iBSc6850H1A1WQHE46h05aQTUKA0Sn5PTiOGaly0nDCcdvm p4BMr5BrvP217n3i8HIpIUyvEwgLLDok8mkTUVd5gtOQQm6yz5qkA9s4HYepk8hUSt7Z NFsIKxXoDWlFH1D2Mgh/ehS2/9chmd9D1x0+R6uRQIlPavrwy7K+xVjd/gXpttqcBryM qkVA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Rii2qKbA; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47346-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47346-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCW7XbcLFh8YK6F0DR9sBWjRX4w4MPhu5AKZoEEKldfQJQuD9CAz81Tq0NZlPGKtqQchm2tVREesbEeZzT/TK+SUpaNrdg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id d20-20020ac85ad4000000b0042bf2dcf641si256749qtd.60.2024.01.31.15.28.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:28:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47346-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Rii2qKbA; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47346-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47346-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 548AD1C23C2C for ; Wed, 31 Jan 2024 23:28:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B223548793; Wed, 31 Jan 2024 23:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Rii2qKbA" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2AC9482E2 for ; Wed, 31 Jan 2024 23:11:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742691; cv=none; b=VIjJIdyZt4uiU1kljg9FBeAkgeuSlHP3LNvXba12K8rYxQn8BOuWP44ClBm58erspK7bB6FhfUMsdPASDc8niy6tHIFuYoBTLeZg96wtKQL8vndu9hhryKLku3RZiIeEcbE/Gf47ypxKpG0SQNCUmGJUg6AKy53iXQK6G4iPp14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742691; c=relaxed/simple; bh=wo077DcopptE4PXrUnE2wvN1oXs9MI9u90VSK05yiuA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uTCIa7Zkpqlu84+PYmDjJMih0mAxFMcfw7E6EQe7zNZHnqFD9LWbxbzwipqmT9LXA9Q7D6YCvw1I1JYYy4HyrVtNfGirrKeip8M0Z0+V6IMej0sL3jf1tVd6gmb8t/arTbWYkEMrYpeaDIUa2UY8rZZ5iA5JZs0Kyyu3mAmxe40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Rii2qKbA; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8E15C433F1; Wed, 31 Jan 2024 23:11:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742690; bh=wo077DcopptE4PXrUnE2wvN1oXs9MI9u90VSK05yiuA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Rii2qKbAxLpzyE1Gk6YhTQxo7MFmtFY9qpdUNiBxws/Bha2Gx933vVZzdTZbuMOlo yOLZnxrcuEC2aaniYWxvl2zhg+F7C9AGnMKYb6sOGU1/Rql2m9NSscRbjVi/JIoOhq peGT9zaXKuDUu5yuqBilBrL/nQVfMZPvVlCqytA2h7G8moZdVuKmfdn3xrfeOrTBW3 J3PtmJgtDTPwJnp1DI7vY/3L828wjKncB8Sn/BrqYQCaMJ/9/j0qB+7Tyifmsb8lhL GZvV9llaO9FTEs8id/l2ce0XVMc9g+znXetHgEJmnfVQSz7h5gniZho+wuVUeRSrXp hGMEqhSeybjLQ== From: Frederic Weisbecker To: LKML Cc: Peng Liu , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Joel Fernandes , Frederic Weisbecker Subject: [PATCH 02/15] tick/nohz: Remove duplicate between lowres and highres handlers Date: Thu, 1 Feb 2024 00:11:07 +0100 Message-ID: <20240131231120.12006-3-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650495904240956 X-GMAIL-MSGID: 1789650495904240956 From: Peng Liu tick_nohz_lowres_handler() does the same work as tick_nohz_highres_handler() plus the clockevent device reprogramming, so make the former reuse the latter and rename it accordingly. Signed-off-by: Peng Liu Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 96 +++++++++++++++------------------------- 1 file changed, 36 insertions(+), 60 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 639ee689e7cc..cc19c4ff5a25 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -255,6 +255,40 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) update_process_times(user_mode(regs)); profile_tick(CPU_PROFILING); } + +/* + * We rearm the timer until we get disabled by the idle code. + * Called with interrupts disabled. + */ +static enum hrtimer_restart tick_nohz_handler(struct hrtimer *timer) +{ + struct tick_sched *ts = container_of(timer, struct tick_sched, sched_timer); + struct pt_regs *regs = get_irq_regs(); + ktime_t now = ktime_get(); + + tick_sched_do_timer(ts, now); + + /* + * Do not call when we are not in IRQ context and have + * no valid 'regs' pointer + */ + if (regs) + tick_sched_handle(ts, regs); + else + ts->next_tick = 0; + + /* + * In dynticks mode, tick reprogram is deferred: + * - to the idle task if in dynticks-idle + * - to IRQ exit if in full-dynticks. + */ + if (unlikely(ts->tick_stopped)) + return HRTIMER_NORESTART; + + hrtimer_forward(timer, now, TICK_NSEC); + + return HRTIMER_RESTART; +} #endif #ifdef CONFIG_NO_HZ_FULL @@ -1391,31 +1425,15 @@ void tick_nohz_idle_exit(void) * at the clockevent level. hrtimer can't be used instead, because its * infrastructure actually relies on the tick itself as a backend in * low-resolution mode (see hrtimer_run_queues()). - * - * This low-resolution handler still makes use of some hrtimer APIs meanwhile - * for convenience with expiration calculation and forwarding. */ static void tick_nohz_lowres_handler(struct clock_event_device *dev) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - struct pt_regs *regs = get_irq_regs(); - ktime_t now = ktime_get(); dev->next_event = KTIME_MAX; - tick_sched_do_timer(ts, now); - tick_sched_handle(ts, regs); - - /* - * In dynticks mode, tick reprogram is deferred: - * - to the idle task if in dynticks-idle - * - to IRQ exit if in full-dynticks. - */ - if (likely(!ts->tick_stopped)) { - hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); + if (likely(tick_nohz_handler(&ts->sched_timer) == HRTIMER_RESTART)) tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - } - } static inline void tick_nohz_activate(struct tick_sched *ts, int mode) @@ -1484,48 +1502,6 @@ void tick_irq_enter(void) tick_nohz_irq_enter(); } -/* - * High resolution timer specific code - */ -#ifdef CONFIG_HIGH_RES_TIMERS -/* - * We rearm the timer until we get disabled by the idle code. - * Called with interrupts disabled. - */ -static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) -{ - struct tick_sched *ts = - container_of(timer, struct tick_sched, sched_timer); - struct pt_regs *regs = get_irq_regs(); - ktime_t now = ktime_get(); - - tick_sched_do_timer(ts, now); - - /* - * Do not call when we are not in IRQ context and have - * no valid 'regs' pointer - */ - if (regs) - tick_sched_handle(ts, regs); - else - ts->next_tick = 0; - - /* - * In dynticks mode, tick reprogram is deferred: - * - to the idle task if in dynticks-idle - * - to IRQ exit if in full-dynticks. - */ - if (unlikely(ts->tick_stopped)) - return HRTIMER_NORESTART; - - hrtimer_forward(timer, now, TICK_NSEC); - - return HRTIMER_RESTART; -} -#else -#define tick_nohz_highres_handler NULL -#endif /* CONFIG_HIGH_RES_TIMERS */ - #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS static int sched_skew_tick; @@ -1549,7 +1525,7 @@ void tick_setup_sched_timer(int mode) hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) - ts->sched_timer.function = tick_nohz_highres_handler; + ts->sched_timer.function = tick_nohz_handler; /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); From patchwork Wed Jan 31 23:11:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195059 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp92591dyc; Wed, 31 Jan 2024 15:36:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IFvM4zKs2UeGuqj0YXJYqwYU9AiCxuPKSQjQ+afE4MjdcmrrvrsozGBaTfa3gms7Ar1CyPR X-Received: by 2002:a92:d48b:0:b0:363:7d59:91aa with SMTP id p11-20020a92d48b000000b003637d5991aamr2925051ilg.30.1706744169280; Wed, 31 Jan 2024 15:36:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744169; cv=pass; d=google.com; s=arc-20160816; b=0hFQ6SFfQco/prfK2DYF550PCbIKo4LsExIVHgJ445S3xIStKW+Td/yY5sc6RJmGpi m1mLikF0SCof+RQUCnYcfvCPF5idNl00EIVRTspl2OgVlEktKD7iOLc19hh2/GR1C2U+ W5RPUK6eYubBthILk5GWG/gwx++bZqBP7GK2W0GIXeG+ftrxYK3DTphxiCc4ZIfW3WSF C42hsibWeOwG75IcdpuLuebA3nAt2XXbXYFSkJjXBP4qlctxtfGOQDbYn9skQIhh1jSK 6I1apOx64hen8HdthFHjwOnyKwlRs87G6mgtdnydAe972WNQQCifGJuUwrgDpYVWKw70 thAw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=gO3Ogl1t6huVsb74EsiZTIL3NsAE2kJ1/uYEbMiCQZ0=; fh=XoK3syNE/+AVz8QZ1b+pUel97zby6hjZE3c3n6JmCXg=; b=hxDj/tM8JOMyzZvs+zIBUHUhXHap0xbFHztEzohU+d4Wk1SdlVPFYhRUfjk7f9Et1c QhSbEYCifLgFX9wTafyaio1217Efpyah/dOTNC8VRAjkOFydNDZ8I1tNXIehVIjbUP3a g3Fw/rD8tsX7EoLjyNMKYVaSGXYw7pxg7PkWsa+hINFps6eVDPwtgnAKPaM5Ysoc7Yv6 SOLpRDvbeoMD8W1Yd3aYlZlFFkjuJuuteCVWMSNXPALhe/5pkaGVrnire/VquJRUgKry WC1TQtKk0IZXXqkhVdem9vDeqVYEELUhjI7zgllWkYXdEALs5d4GHGmZGRTFX7Ycx0lj 6j5A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Pu2BMKfx; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47347-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47347-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCV4iFJppefblJXNaWR36SspioYZdz8smV5pGcJ4qSqpT9jVi+kSCqWR1PTqIdWkCJnqZrxaoSz3GZmp5WlxYnpvsveMHA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id y192-20020a638ac9000000b005cdff9cb384si10433258pgd.372.2024.01.31.15.36.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:36:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47347-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Pu2BMKfx; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47347-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47347-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id CE293290F76 for ; Wed, 31 Jan 2024 23:28:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2F5F348CF0; Wed, 31 Jan 2024 23:11:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Pu2BMKfx" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D80F4878D for ; Wed, 31 Jan 2024 23:11:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742693; cv=none; b=sdK11/339JHagYebTadWOENMVet56zmPBJE6+4GEm5BiwiLfgd7+3QMkEwc0uQkMeYvlVnho86uXdAZc+1sA+q5Z/q/XXZIJBA2LUJ/vzIcpxSIxXGAcBFcY48C7jQq0qFPjatENIe1OdZdB/ZCYY6e67D8eZ4iUatK+A72X2Ak= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742693; c=relaxed/simple; bh=bdfS8bPbTsmzbdiim69fTRLC6qHuhmCc45TsOfVHL2w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vet5CWtG7LGZadHLiSuC7jdQx3VpphMH+kSFGa/dfyrLBl2Px3oqWPWHCkfkBBNvvwV0QFbAN09HYKsu+amU7YNLFIUQSpP38axkWqhLE2zXM/mFdWtGjDrx7q8BXsCu1hlHDroX/UmIJkchuduvmPYC/+YuK6JGENMcq9hnYeA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Pu2BMKfx; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37A22C433C7; Wed, 31 Jan 2024 23:11:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742693; bh=bdfS8bPbTsmzbdiim69fTRLC6qHuhmCc45TsOfVHL2w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Pu2BMKfxpDonJh96TUh4I4OS27tDicdRWAeUYtR89pMgcIYaqvcf33MmyCmIuECpE z5F7ZOFxbLEqTQD9oibni2BhUXXUGcyVRw7GxBc/nq15gVXeHpcTCShV3VHDCmuRtl xog5UJ54vaweu4GcjxieY4FB0WAttRG/rZQ8mIlZu0NLGqvdnII2KWgBSWrU68h20v tUq3UNMeul79lwRvFtl2cC9v1gYQjjsL7Tfee/Zu5FJT+iV4Z2IqfHty/UJ76oKjI+ JilVAiHp2sMGWEFPwrFIwsgapuxVwOTAsoYrmUbDtPTCHrCN6tAkgvdbJGYg1hS7wV 26sr/sDAvRe3Q== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 03/15] tick: Remove useless oneshot ifdeffery Date: Thu, 1 Feb 2024 00:11:08 +0100 Message-ID: <20240131231120.12006-4-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650973960823917 X-GMAIL-MSGID: 1789650973960823917 tick-sched.c is only built when CONFIG_TICK_ONESHOT=y, which is selected only if CONFIG_NO_HZ_COMMON=y or CONFIG_HIGH_RES_TIMERS=y. Therefore the related ifdeferry in this file is needless and can be removed. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index cc19c4ff5a25..e674269692ab 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -43,7 +43,6 @@ struct tick_sched *tick_get_tick_sched(int cpu) return &per_cpu(tick_cpu_sched, cpu); } -#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS) /* * The time when the last jiffy update happened. Write access must hold * jiffies_lock and jiffies_seq. tick_nohz_next_event() needs to get a @@ -289,7 +288,6 @@ static enum hrtimer_restart tick_nohz_handler(struct hrtimer *timer) return HRTIMER_RESTART; } -#endif #ifdef CONFIG_NO_HZ_FULL cpumask_var_t tick_nohz_full_mask; @@ -635,7 +633,7 @@ void __init tick_nohz_init(void) pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n", cpumask_pr_args(tick_nohz_full_mask)); } -#endif +#endif /* #ifdef CONFIG_NO_HZ_FULL */ /* * NOHZ - aka dynamic tick functionality @@ -1502,7 +1500,6 @@ void tick_irq_enter(void) tick_nohz_irq_enter(); } -#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS static int sched_skew_tick; static int __init skew_tick(char *str) @@ -1567,7 +1564,6 @@ void tick_cancel_sched_timer(int cpu) ts->idle_calls = idle_calls; ts->idle_sleeps = idle_sleeps; } -#endif /* CONFIG_NO_HZ_COMMON || CONFIG_HIGH_RES_TIMERS */ /* * Async notification about clocksource changes From patchwork Wed Jan 31 23:11:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195047 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp89361dyc; Wed, 31 Jan 2024 15:29:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IGjrUilinAA+Mbfu0p/icw5uk7vIUXudDMAeINSXwBhHJ/VJM78pKgtBv270hlzXJtxACOy X-Received: by 2002:a05:6214:518b:b0:685:d4bc:38d0 with SMTP id kl11-20020a056214518b00b00685d4bc38d0mr3101782qvb.21.1706743742744; Wed, 31 Jan 2024 15:29:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743742; cv=pass; d=google.com; s=arc-20160816; b=OP/pb8PEubcUyGsIZrPWjKwfOlGpbd2hXjbUaGj17RpwyKdozezQ7TKVNqZYNvKyut bBUloT+36MlYA1WjHWUYUdF0OzLE6gkKj6o5/fg+vFFd3Qs0xW3c8R1sp1GAZzQLfJLH G7fXuBryY7mqRvEi7I42BKSDWQEv/E4n16QFQT/jHP98V+e61+P/ljPy8lI6HDqKMhU7 Rc5M+Shr4oBeegqgWMUYzNhQWZK3Z/Tt5tvwLcrOLB3JAU4l7lLsvEZYkNDFm6UO4UDP 5BT4TYXruTQzzHF/EhjX8RK30gY+POOH+zO2RT+T5jFPz5SGTke2CMTBstLhcZqLO3fW NOJQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=IJMtx77EFUiJta24w2yvRKT7G4jwiQEGOXfBx2EnHl8=; fh=ry2/XNGJljawTk0j4CFTYZbi0ubVnAfW08LCKrtHXXQ=; b=bC1/fHDXsRJ/HRxhaKAKud/ant9dXq22KtMrWKQ9JLQ/ScWbXnK42skgA1bhVukCCW 1K+a65SQ3tNIsZ1wvafh4GxXQLqqZqBS0hyOfZwwEEyvIRriwckIPQxZiBExS4uPQedE /acSiHGZt3XOfWRbCv6Yj35VXQQu6LHrYWiNUmVzPBbEnbKk84Rfjkt5znUJHmVj2wpw MeAA79KtnDmB1pewL/s8SKay30DOmrhU2Slf3AkOO2RA7cqqHeZmnQZIkfU/05EPLGXi XCwRqUOhGXFqw9MxAbGcrFqh22TPm5qlC/x7ebK58Nr2qtFTyY5GgImZeLIP67wx3xzT yu4Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=emkGsz4c; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47348-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47348-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUl04b/iU6V3AwcMf7KyWQx1FBlFZc7biOQFss2fty2cWvq5/EFX4C3kNgh9pHu/iWPBj3GSRoHxh4nwkqeMZvpmf2IZQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id r8-20020a056214212800b0068c51beaeeasi7049719qvc.127.2024.01.31.15.29.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:29:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47348-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=emkGsz4c; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47348-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47348-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 887E71C244F7 for ; Wed, 31 Jan 2024 23:29:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id F12FC4B5A7; Wed, 31 Jan 2024 23:11:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="emkGsz4c" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56B4F487A9 for ; Wed, 31 Jan 2024 23:11:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742695; cv=none; b=ffi+HrVdbmCMru86eQd8f276za6yDx0BD2lypv+nLg1YIClk69EWjcBiHtwd0JixDOm5ykE6yZXwUFxaE+yPr+2eglFMOXLt1F+v2gHdfKLRoVfLB5at+ZIwnDmA+OBZX6JyfvUIIMvkRUEp7D/M+EYCOaFVtksc0aej4bH7nz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742695; c=relaxed/simple; bh=0QAgU9z275M0DklHxMY3gX1axsMyYua5aTjHQfiW1kI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iRlyYsVHsghiRMUWslDEaSbceb6+wuVOXtlrP2oAiRAVIAvDIqIT1zbeM7tcdgAlTj2q7f3wiK9m5aCQWrfCibf4VHcpQG7x5I6qFKpYvObj9ZLWWPunnxP6iZhC5UWQWO4LLO60yRun33/7+yTYeN4znzaEyifbsfCmkZrhk64= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=emkGsz4c; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7F5B4C43394; Wed, 31 Jan 2024 23:11:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742695; bh=0QAgU9z275M0DklHxMY3gX1axsMyYua5aTjHQfiW1kI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=emkGsz4cAKUCPe1skkiVLn6N0ywXEcObelKHbU1OOLULFuhWypqSivcxbNE71Txjy EMiFBzhgpRbyZSaoIJqYeeqNsQoOhcDTl89g73U/8mOp4kZLwjcN4LSkl4uNusKobz VZ1hgpX48OInX72j0QopSWdKEDvXs3jBLtymp1HY4PW2CDd3v7cCgi1Nm6SS5qZGky Fb72rCdRaN7XMIaFg5eJjPwL8VSh6e30eGVXyYDtXO9Yt7oc2+GmXBzosdpnW7ysMK jtAcWHj4VyHd9WTRlofzB+MxxY0yQVNn9TKhnwTh4mOo7u6qdlJT5C7INv2HB76R7C fKmE1+4arkEJw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 04/15] tick: Use IS_ENABLED() whenever possible Date: Thu, 1 Feb 2024 00:11:09 +0100 Message-ID: <20240131231120.12006-5-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650526973417404 X-GMAIL-MSGID: 1789650526973417404 Avoid ifdeferry if it can be converted to IS_ENABLED() whenever possible Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-common.c | 4 +--- kernel/time/tick-sched.c | 14 +++++--------- 2 files changed, 6 insertions(+), 12 deletions(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index e9138cd7a0f5..0084e1ae2583 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -111,15 +111,13 @@ void tick_handle_periodic(struct clock_event_device *dev) tick_periodic(cpu); -#if defined(CONFIG_HIGH_RES_TIMERS) || defined(CONFIG_NO_HZ_COMMON) /* * The cpu might have transitioned to HIGHRES or NOHZ mode via * update_process_times() -> run_local_timers() -> * hrtimer_run_queues(). */ - if (dev->event_handler != tick_handle_periodic) + if (IS_ENABLED(CONFIG_TICK_ONESHOT) && dev->event_handler != tick_handle_periodic) return; -#endif if (!clockevent_state_oneshot(dev)) return; diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index e674269692ab..c4bf796a3285 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -186,7 +186,6 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) { int cpu = smp_processor_id(); -#ifdef CONFIG_NO_HZ_COMMON /* * Check if the do_timer duty was dropped. We don't care about * concurrency: This happens only when the CPU in charge went @@ -197,13 +196,13 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) * If nohz_full is enabled, this should not happen because the * 'tick_do_timer_cpu' CPU never relinquishes. */ - if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) { + if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && + unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) { #ifdef CONFIG_NO_HZ_FULL WARN_ON_ONCE(tick_nohz_full_running); #endif tick_do_timer_cpu = cpu; } -#endif /* Check if jiffies need an update */ if (tick_do_timer_cpu == cpu) @@ -230,7 +229,6 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) { -#ifdef CONFIG_NO_HZ_COMMON /* * When we are idle and the tick is stopped, we have to touch * the watchdog as we might not schedule for a really long @@ -239,7 +237,7 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) * idle" jiffy stamp so the idle accounting adjustment we do * when we go busy again does not account too many ticks. */ - if (ts->tick_stopped) { + if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && ts->tick_stopped) { touch_softlockup_watchdog_sched(); if (is_idle_task(current)) ts->idle_jiffies++; @@ -250,7 +248,7 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) */ ts->next_tick = 0; } -#endif + update_process_times(user_mode(regs)); profile_tick(CPU_PROFILING); } @@ -1549,10 +1547,8 @@ void tick_cancel_sched_timer(int cpu) ktime_t idle_sleeptime, iowait_sleeptime; unsigned long idle_calls, idle_sleeps; -# ifdef CONFIG_HIGH_RES_TIMERS - if (ts->sched_timer.base) + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && ts->sched_timer.base) hrtimer_cancel(&ts->sched_timer); -# endif idle_sleeptime = ts->idle_sleeptime; iowait_sleeptime = ts->iowait_sleeptime; From patchwork Wed Jan 31 23:11:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195048 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp89436dyc; Wed, 31 Jan 2024 15:29:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IF5/r7rFOscwQiQg3XbKejZZjo3GPhCxlUK94Eks22oI/UBXE8yeNLFSzSmMdHK4sX+HEWA X-Received: by 2002:ad4:5aa2:0:b0:68c:583d:5425 with SMTP id u2-20020ad45aa2000000b0068c583d5425mr4219437qvg.11.1706743754022; Wed, 31 Jan 2024 15:29:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743754; cv=pass; d=google.com; s=arc-20160816; b=b7H9xpD9cAqbRDXJy2k+ckGrxljfcWOTE8I1sw9Z/mY3sw4MkhwYnUhENjt+k/3J4M mLvaCDMrJNkx+Jq0zLBO6mezdNBVZ70jFZXnGf39a8W9wGk7ZTTG2GTF6FQhLC/c2GgS Rb08mfRMrNJ8MkIYeIzLDXJz+i6R4hvo4EH/91E/8za6Dfx3raPkj4lsneibZFEOxvVw aID0RuX1OooqQN/FA8TTsPlvqkrG4LCtmVs4ykmInbE6AnAA/upF9SFdNYMVuqoO6VQN Y1WTTRznbp2TSdBiuq9gxdgZiDwvdsuC7rrNAm7KVYrrWIc1/DiMJxz0mrmYqNOLuWvY hbEw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=pNj47kIyXW1SYNzubx06sDf4tiyhEd43Byh31HssXnI=; fh=UggXvyNdRJA03wu1sx4ieSqm140cS10Rh2PTeHMcAJw=; b=HzV5hpnuY2Znt+fSwzMopqp+awSfeURY1bWbHvQ3ECai5NtITUr9OSHUN4mnBS5unk gZ0Mr7mPR50M8EpFYq0J/ko7ZtFfrXH9paJREtxB+ER3zBZFAmRQFY0meqAgPyXCi5DK t4CSMNisffp8qJ/9ku1VO+T4KnkVb1g+qBrPEi8SOj4oc7w0VW7SHU5RHnN9xCF9dREh b1TVUEYa9HCbtFglKUgizl+Dk0THSr0iy1RANwx9nwz/i5OdZJg69QKcZLVIqyOUVHWC H7aUv0fdwcHtZq9XUu/96CqUIE6aPuahfoYupKEnf3eP4xOwWuhMCm/sTrKY6Hn8djwd o3zA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=gbsKGu5K; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47349-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47349-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCX6ejOtWu5Kp8DegR+w2E90YjSohZARybVKyK3+bysjgBMAvhXk8mWS5tk/cLXHqXR0rN3+VhRK/QUJPaA+QTZbuRfHwA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id fn13-20020ad45d6d000000b00680b0f40fb4si13642462qvb.206.2024.01.31.15.29.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:29:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47349-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=gbsKGu5K; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47349-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47349-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id CBF111C247F0 for ; Wed, 31 Jan 2024 23:29:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1EC964BAB6; Wed, 31 Jan 2024 23:11:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gbsKGu5K" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75E9F4A992 for ; Wed, 31 Jan 2024 23:11:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742697; cv=none; b=KG8bxaSCM1FA0fhX2WyN2CphNwwIq42l7YMnTEmuJ+BKYtYkg9MSktqEJ6tcf7U4UM3vs7MyjN1rar0Q6BmBJicVO/bg7bSkSiN78mkIbvH0oPMytnjsIhKIYLewCjr/ZR4No+yG+bDNfr047+Cp5TWIz4iwe2ccl4Je8oKEYNE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742697; c=relaxed/simple; bh=DNVESk4JIRTsWnUA2aWEEPPDA9xQmhtXj6gZKFnVhtk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vq8BMrSLFQkx1fZ3cu/qsSPI1ZZxW5cw2pG1zW/Mv0SPgjbgcGTP3ac69oyEjT4FcIAq9W+SUFA19NEfZW4Rz4meO4UTuzVQy2Scl1h9HJxiz2/lqxtx868/l24VXbNsDxxbhVxaxI/IxX/jO7CCIl4XrZEVe3QraideXHA8xj4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gbsKGu5K; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9DDDAC433F1; Wed, 31 Jan 2024 23:11:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742697; bh=DNVESk4JIRTsWnUA2aWEEPPDA9xQmhtXj6gZKFnVhtk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gbsKGu5K9ARWebN1/wgqhx3Hax4JJH7izJQs//Hwto3c1/HHCv0aLnZ8162+bZQ3+ RF0yyAuOeYzTu3E18vifIj2SnzSk1CDYnljj3qYvoPEfA2kAwAdzkg/mIV3OdTsW8V FAZ9R+6cMeFJrJNGkeczff7FEn32rkVytbk60Z1c58buVh5zvo2pTR7FRfJ9BDMeQo 1fbifoNBrHmnyKC9Os57r1uaZJ8aKcrEfG9NHxKeG0tcVhZCEMSm1Dy8NhGO1JI2wc RLnbP7SsWTjG4HX3WOalwaDMQ0bW9RRtv4E7BKl1RQS4wbiJsT51lo0xDkuVrHnURL PfFdHveSF5s+A== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 05/15] tick: s/tick_nohz_stop_sched_tick/tick_nohz_full_stop_tick Date: Thu, 1 Feb 2024 00:11:10 +0100 Message-ID: <20240131231120.12006-6-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650538767969888 X-GMAIL-MSGID: 1789650538767969888 tick_nohz_stop_sched_tick() is only about nohz_full and not about dynticks-idle. Reflect that in the function name to avoid confusion. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index c4bf796a3285..1fd7319b58b3 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -997,7 +997,7 @@ static void tick_nohz_retain_tick(struct tick_sched *ts) } #ifdef CONFIG_NO_HZ_FULL -static void tick_nohz_stop_sched_tick(struct tick_sched *ts, int cpu) +static void tick_nohz_full_stop_tick(struct tick_sched *ts, int cpu) { if (tick_nohz_next_event(ts, cpu)) tick_nohz_stop_tick(ts, cpu); @@ -1032,7 +1032,7 @@ static void __tick_nohz_full_update_tick(struct tick_sched *ts, int cpu = smp_processor_id(); if (can_stop_full_tick(cpu, ts)) - tick_nohz_stop_sched_tick(ts, cpu); + tick_nohz_full_stop_tick(ts, cpu); else if (ts->tick_stopped) tick_nohz_restart_sched_tick(ts, now); #endif From patchwork Wed Jan 31 23:11:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195060 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp92792dyc; Wed, 31 Jan 2024 15:36:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IGlSUauDmPTUO/AbcKqT1xI+8ozzM6FGrv5susB6SCZI3+g/cZWpnk1QhRvzFOoNQIGkA5B X-Received: by 2002:a9d:75c1:0:b0:6e1:29f3:6fb9 with SMTP id c1-20020a9d75c1000000b006e129f36fb9mr3572982otl.21.1706744201129; Wed, 31 Jan 2024 15:36:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744201; cv=pass; d=google.com; s=arc-20160816; b=mrN5gV8dpVXGOv7Sdr3h1IHQIFK9A8ySVNCvtpSMmHniiaehb2rxbNEl+1j9IfkeCP JKhOBEjTSABDEoPdim9gCPY31fM3d8tfrYbYyEKU5/ewBoK5fpSTY8D+CEKyVenP3j2H wQjGG/mJIcFUh2297KWCoY2+tEvzJ0E5ijAcPMBHWd1ABHktOSHy8Kc2QQlRg1eB5qRg f+QW80eU7S+MGc5gjcNC84cKEXloqcggplAEGwsWpkbuWOTAVELy82hxqdTIqc9SQGkw dPlZf0Hzf+SVuOTzAup8Uh0/zcvSpYuXkQM5co5rFs5bNJaJp5f0YRv/5SbOcXXfqP44 ZL8Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ziL4SAvQ/sgjxMhMMgc8ycsxfgWK0xY4iu+npqlkMSo=; fh=EW8bR3+X/VH/xW2CIvqUa4i5vOibANobvqFmWYTXOb4=; b=UQqGpIFAaS7GSQ49XH1lcXgMOBwndQs6ZI4orCpZpyd12ypFW6GsTSY9JI0iSBz56T CBVHQGuSquMZOc12Kdecb80fFSqCkaC7syvp1Lh7UE4n1WUSQl6AGqO7jRLeJbl/2Pm0 0DbaVgz3zLtGJmqFw2aSaJr+uM9N6v1Ij5/3cxCGx/tEXO4Ecg7Y1SgNfKwwnvgIb2dB zPNPE/L0aBPTVxfEmhA9huKhGKUa6dod+TMAJI1ShNkmwK/QwiJeW0C5B8mLhJxUjVkE +LAi5zp6vsRDPD9k+a1GYEUM658zQGEveXkASaZzSqL8bRbltcq7FPXyfQb3v16aqKYt VwHw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ptQ5tTmA; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47350-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47350-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCU551PrXHMyE9zTWfrCTBwmLbS+9P4xWxSQYgvuIVHVK0MPGw6Sv47oH6LNzBRE6pajk/ow33u0HAkYbFL95/0uiQeS2A== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id h14-20020a65638e000000b005d8b5aace83si8666474pgv.289.2024.01.31.15.36.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:36:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47350-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=ptQ5tTmA; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47350-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47350-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 50376294844 for ; Wed, 31 Jan 2024 23:29:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9FB824D10A; Wed, 31 Jan 2024 23:11:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ptQ5tTmA" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0549D4BAB5 for ; Wed, 31 Jan 2024 23:11:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742700; cv=none; b=lKnvnCbWc0o0wCqWBbMuCyyXokpkxQ7WK7ENvDMM7ysCFU21YWrugMqVHwADJzAuYg6B8xBnr75/CEff6ImpEDh/zvJllf3tAo3XBbTJ++sO2R4xHssR591MrbkvUy+JWnR9dTogT+eppSKjUlcXUhfgYek//rvilmrlsPCGXpE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742700; c=relaxed/simple; bh=TdBitky7dKeAUk4dshu31PUBwwuT3BbUa7Yr8mFUDQs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TwWnW7NP4xFmAGIOGHI5qYW6w30VLdtsfTriVEKHC3oT8aIW6vJAaD0Q1s+zroso2jq2wjBziWNY9OPxnNvj975g4nzZzp6JdCZuBaQ0mdlFzyJfUKI96GsUkQO3TUSj2iPpm5X1LHbwwXJ9eqezc7ToMZ1LDJAVT9MAyr80gr8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ptQ5tTmA; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id D1864C433F1; Wed, 31 Jan 2024 23:11:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742699; bh=TdBitky7dKeAUk4dshu31PUBwwuT3BbUa7Yr8mFUDQs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ptQ5tTmA0NWl7GDKMTOl4pZGyfrqVtXGeGeZa1F2Qwp7vxZFNqSQJRi144tZoa59B xPeSmj50BnbAyAFc0S4P3FzQw6JQqV+l/bH10lqVNSh6XBfeho36sk8Kj4vgT0lZey n4joxNoePbGmdvnB8oP7NCysCkX523xulBEg6doDg+ZTHGxWKhui70ovZ45pXc+ndN AeTqcYkHx0A9t+627Bdplw2QBTWuWSs+OWRhvLPEXuoN/ibv/vYatAJZiMdZlRCWp1 TSxfH3V+a79FWBsawBUBqhNt4K5yQmltxyam5ra0TTAkI8cmRbOaVGH+nIbT9W+VPM ecKXuEynHMQzw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 06/15] tick: No need to clear ts->next_tick again Date: Thu, 1 Feb 2024 00:11:11 +0100 Message-ID: <20240131231120.12006-7-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789651007018011095 X-GMAIL-MSGID: 1789651007018011095 The tick sched structure is already cleared from tick_cancel_sched_timer(), so there is no need to clear that field again. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 1fd7319b58b3..a07a04bb9d27 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1100,11 +1100,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) if (unlikely(!cpu_online(cpu))) { if (cpu == tick_do_timer_cpu) tick_do_timer_cpu = TICK_DO_TIMER_NONE; - /* - * Make sure the CPU doesn't get fooled by obsolete tick - * deadline if it comes back online later. - */ - ts->next_tick = 0; return false; } From patchwork Wed Jan 31 23:11:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195050 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp89634dyc; Wed, 31 Jan 2024 15:29:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IE+qNwZbTPWieb7afrmKP8PxknfX4iSkBfUQBwLhj2tP7qY2fyYq8xql/eIYZmd2DbjwIKe X-Received: by 2002:a17:906:190a:b0:a36:7608:8f91 with SMTP id a10-20020a170906190a00b00a3676088f91mr2407035eje.6.1706743784461; Wed, 31 Jan 2024 15:29:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743784; cv=pass; d=google.com; s=arc-20160816; b=ifaotssDwhYaZSqt4+0A7tiLOpo4nZ/r2VjVpcAUpP7oS4sBn3jDj3bUzSQOKWNS36 Klaf4KEN4O+EBxkug7wfRboTKbCiDNfoDig/BTfS6yruUOvPfUs402NU5xgL/GABDvJg 9udfpUp6aOcff8/Uj0b+Rj3/EDRgggYrub7mdZYH/mOMxfXRyyvzoOKm8MFYt+C9QZqE qv+RGjuRZr4yq9lsJAZ6KdpPeR4EotIAfpY9POdP2LuaaxPZQnmXUkkeEH6zmvVrlHWC p3uf9xJp2EZpWGaGmmGehPrJbPL5Sq1U9PEB85YPmLRvsw2sg97Ch2yMI7oietJ5cBE1 3WsA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=rikaqKwz5QTTyQ6tTp4fmEVke9sYVhWGQulgxnOCRww=; fh=iQ/AwNXhKvHr/Zkyhc18x2XfjzSP48t+bD8cBfuYRuA=; b=IgdJ5icsZExDOuBDC5aUGAtisxZmkEUT0OeKThlUFxZpCQv9Oh4S7cj/L5EkWLMKQR uHEISfLz9Y2b2EkvYnkXOVV7TCCRWCWGoy0MPmZ9Q+ws93DnkIasNKoGmqj/0oN5IHcL ry19aj/KYUzkhVxbuTkxXK17AC4og7qVBplbIWwpOgUF9mpfs7Mz9YQHeWWjxrYru8nC Cq1aBkNfmM8bHf2VSa54fUlwpkGOJbTyQObvcS3fImmf/Ley3ZTQ48n0jRHV0XpRp5s3 jhcMo+MS7l2O65s1hAlwkdLD9KGtgFSubpIRlzkCWZXr0eYt3O04hj8sbGUGNAv17tdL Gi1Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=uUjLqIMD; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUrq9SbSLrMBZLRj8cGUTx72EV9Xb13tFLthv9VJF7WvjUQnUKjiLpxcOZf/uUpVyvegd91WbajPkUI8Ng8lGoa/M556A== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id y9-20020a170906558900b00a348dd696a1si6133230ejp.971.2024.01.31.15.29.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:29:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=uUjLqIMD; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47351-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E17371F224D5 for ; Wed, 31 Jan 2024 23:29:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 54BDE54F87; Wed, 31 Jan 2024 23:11:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uUjLqIMD" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E9394CE17 for ; Wed, 31 Jan 2024 23:11:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742702; cv=none; b=qk10eZy8rYMhxYoYKtt2rHREgWbxSBs3T+GJfFT/fLZ4QWol2Ld6WusAZXl2dplffBtKGJRIymwjqp/eQZXfR1yGfRdGR71IpgsiAYiucuaPROr5aBb9AV9ND1dyKbSJJvMqRVAzuJ//NoG25HlcahtMsHGSjeUHnXuKSTLIePs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742702; c=relaxed/simple; bh=VeKB2CcnfxaMa74jOXPT020xZoApnn0QOwGRqmxYQ/I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HJMCbkL9FcCTBmvLA58S2LSatitbMh1dOiB5iZADHPRtlPritsv3du3BU+/6uy9hecvpPsoooZ413ZJ0k6uCN0Bo0OO8MVUU+6p0jSoMTZRAW68gzuqeh7AWk/0hqAnzrtAVoa+MUGa3sXvjMjd+/sKECS6tBwUaXpkZ1akuE0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uUjLqIMD; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F997C433C7; Wed, 31 Jan 2024 23:11:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742701; bh=VeKB2CcnfxaMa74jOXPT020xZoApnn0QOwGRqmxYQ/I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uUjLqIMDv27lgka4aqV7u90w6slLnq1qvEGzmwp/78x00vv5pGUv+wkUCg/ovObJL jNnRDFadbB1AxT9Cq3QQwX8DbQdu9PPQZckckHOrBwW5kyjnsuf9WYFSRRDGlGxnQJ aU+QgjNU0Cn6ocBCfXCWPDwZOjHa3oilSHSfc27Vdh1iCSb58AgtowMefFEzHIGBJD sdsapMO/D0CXETIVtg6dYzdUJcbRJV7l2qgbrM8OqRzrSopPyudezBxlSBPT83jSEm CXCZfjEi/NxSm4lKi1aSUHAHbzTnYUuNkL9JtyEsMAvSxMz3DWwh5bUEQYUBM7/1Ek sP/6P31JpxDaA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 07/15] tick: Start centralizing tick related CPU hotplug operations Date: Thu, 1 Feb 2024 00:11:12 +0100 Message-ID: <20240131231120.12006-8-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650570197718187 X-GMAIL-MSGID: 1789650570197718187 During the CPU offlining process, the various timer tick features are shut down from scattered places, sometimes from teardown callbacks on stop machine, sometimes through explicit calls, sometimes from the control CPU after the CPU died. The reason why these shutdown operations are spread around is not always clear and it makes the tick lifecycle hard to follow. The tick should be shut down in order from highest to lowest level: On stop machine from the dying CPU (high-level): 1) Hand-over the timekeeping duty (tick_handover_do_timer()) 2) Cancel the tick implementation called by the clockevent callback (tick_cancel_sched_timer()) 3) Shutdown broadcasting (tick_offline_cpu() / tick_broadcast_offline()) On stop machine from the dying CPU (low-level): 4) Shutdown clockevents drivers (CPUHP_AP_*_TIMER_STARTING states) From the control CPU after the CPU died (low-level): 5) Shutdown/unregister/cleanup clockevents for the dead CPU (tick_cleanup_dead_cpu()) Instead the current order is 2, 4 (both from CPU hotplug states), then 1 and 3 through direct calls. This layout and order don't make much sense. The operations 1, 2, 3 should be gathered together and in order. Sort this situation with creating a new TICK shut-down CPU hotplug state and start with introducing the timekeeping duty hand-over there. The state must precede hrtimers migration because the tick hrtimer will be stopped from it in a further patch. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- include/linux/cpuhotplug.h | 1 + include/linux/tick.h | 8 ++++++-- kernel/cpu.c | 8 +++++--- kernel/time/tick-common.c | 17 +++++++++++------ 4 files changed, 23 insertions(+), 11 deletions(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 172d0a743e5d..74fcdd2b82c8 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -184,6 +184,7 @@ enum cpuhp_state { CPUHP_AP_ARM64_ISNDEP_STARTING, CPUHP_AP_SMPCFD_DYING, CPUHP_AP_HRTIMERS_DYING, + CPUHP_AP_TICK_DYING, CPUHP_AP_X86_TBOOT_DYING, CPUHP_AP_ARM_CACHE_B15_RAC_DYING, CPUHP_AP_ONLINE, diff --git a/include/linux/tick.h b/include/linux/tick.h index 716d17f31c45..afff4c207bd8 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -19,16 +19,20 @@ extern void __init tick_init(void); extern void tick_suspend_local(void); /* Should be core only, but XEN resume magic and ARM BL switcher require it */ extern void tick_resume_local(void); -extern void tick_handover_do_timer(void); extern void tick_cleanup_dead_cpu(int cpu); #else /* CONFIG_GENERIC_CLOCKEVENTS */ static inline void tick_init(void) { } static inline void tick_suspend_local(void) { } static inline void tick_resume_local(void) { } -static inline void tick_handover_do_timer(void) { } static inline void tick_cleanup_dead_cpu(int cpu) { } #endif /* !CONFIG_GENERIC_CLOCKEVENTS */ +#if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_HOTPLUG_CPU) +extern int tick_cpu_dying(unsigned int cpu); +#else +#define tick_cpu_dying NULL +#endif + #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_SUSPEND) extern void tick_freeze(void); extern void tick_unfreeze(void); diff --git a/kernel/cpu.c b/kernel/cpu.c index e6ec3ba4950b..263508073da8 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1324,8 +1324,6 @@ static int take_cpu_down(void *_param) */ cpuhp_invoke_callback_range_nofail(false, cpu, st, target); - /* Give up timekeeping duties */ - tick_handover_do_timer(); /* Remove CPU from timer broadcasting */ tick_offline_cpu(cpu); /* Park the stopper thread */ @@ -2205,7 +2203,11 @@ static struct cpuhp_step cpuhp_hp_states[] = { .startup.single = NULL, .teardown.single = hrtimers_cpu_dying, }, - + [CPUHP_AP_TICK_DYING] = { + .name = "tick:dying", + .startup.single = NULL, + .teardown.single = tick_cpu_dying, + }, /* Entry state on starting. Interrupts enabled from here on. Transient * state for synchronsization */ [CPUHP_AP_ONLINE] = { diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 0084e1ae2583..a89ef450fda7 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -397,15 +397,20 @@ EXPORT_SYMBOL_GPL(tick_broadcast_oneshot_control); #ifdef CONFIG_HOTPLUG_CPU /* - * Transfer the do_timer job away from a dying cpu. - * - * Called with interrupts disabled. No locking required. If - * tick_do_timer_cpu is owned by this cpu, nothing can change it. + * Stop the tick and transfer the timekeeping job away from a dying cpu. */ -void tick_handover_do_timer(void) +int tick_cpu_dying(unsigned int dying_cpu) { - if (tick_do_timer_cpu == smp_processor_id()) + /* + * If the current CPU is the timekeeper, it's the only one that + * can safely hand over its duty. Also all online CPUs are in + * stop machine, guaranteed not to be idle, therefore it's safe + * to pick any online successor. + */ + if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); + + return 0; } /* From patchwork Wed Jan 31 23:11:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195084 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp100040dyc; Wed, 31 Jan 2024 15:56:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IF3PZrsLHbVyjHnqvqXgkKQ60RcFpBM906MkaX+qbIReY7WJNPx5QtcKwD4BYw7dakpl/R0 X-Received: by 2002:a92:cd08:0:b0:363:7d3c:7d3a with SMTP id z8-20020a92cd08000000b003637d3c7d3amr3712269iln.16.1706745369371; Wed, 31 Jan 2024 15:56:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706745369; cv=pass; d=google.com; s=arc-20160816; b=l6qNi2THidC+Re54ddME+kgIQN2UenrPZgCxDzW457ogSp6fXJNr4Vku9eSkUmEITE /ziLvtkoDwd1VoO4Wc+ccD9bs0U0w4ZDx2O9eBeir/ZOOecmAifKNBh1v7lgyENdviwP 0hlcdCAixlvxGQMyKDDLaAA+k+YaYXGP68v3iUxkR6kWBb1jh3IBbU1KWaXjFx3+uted f/t5NGb3a1pw6xZWvEclZ9T7AoLuQhiH5vWgd9hAyMIFIddWUNvkdpTrj+lUlnzynf07 0H+BolSJG83SA8VfKA22N5GHVF9JUvsbhxaToY53q3gjxzOcs+9T+sAwOsNPQiymu11y kdrA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=x2F/PHiemsR+7stqaAaLQFC6GLg7F8xORBV+jQXd7kM=; fh=HdrexM/am6OkvUOpbdYD9YhqIaVkv5Klo16cXfpmHf0=; b=ngFlYKlwt8LLAxwu3uoX9MPMPvD8giv/FrlUYkA3kGitH4DSLQXeWQPkPR94H/3wYG UkQeoIpZTmvZW41kH1v78ZRmOh81uAnTMZJTv4wNiAQtTT1WL+eQW/x9E1j0T4BICJil 1JBcqYJc4ynS+5WlVTf76fZHRojrAs09DK+0/xk+iiREFQyyZigXB82K7ga81JQFvPPz pCn0D+D75XPs+H2M10ujhyZm77FdZIvsetUjWC58Ht6+EMC7joqqJhyvur0eK1MD243D ut9OehqZ+RAYZUMPgHGYxrWrLRtIWBYiFwpQgIj3Gm3Zmp4AbDr2HHAYUNUwDJCigYmV LQqg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lWjScDyy; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47352-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47352-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCXLmSLMqitRTfdiCw4tyYXk53B5nb+zkY+9cOEDC57q6NdNsBmCe6WepQhjm3hUJYJmdPXatsiIa4/VioxlVICkHdMeXA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id c13-20020a65420d000000b005dbe2220330si707895pgq.171.2024.01.31.15.56.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:56:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47352-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lWjScDyy; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47352-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47352-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id AE2A3B36E6C for ; Wed, 31 Jan 2024 23:30:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 79B7B78B6C; Wed, 31 Jan 2024 23:11:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lWjScDyy" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C70334F1E0 for ; Wed, 31 Jan 2024 23:11:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742704; cv=none; b=fJ0QkHEcoUJZ6x1gIjTeKOkliDhqDbi5twqiTYO3O6UuLlCaBDcMy7o4/bTbyLiMwvsWDhm8vPtRyHJUcwHAl8P4GmmJam2kpM8bGuTt6Sk0phxM1CDotVFv+nRHhvg+X0z8uKV4hCy9F0TI6nPaUY0tw+Hpn3mBjQt7ke2x7ig= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742704; c=relaxed/simple; bh=gpKFxD2je2tWWE8lEBMTW3z9BiZKvLJ/r4INUAW0y0Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Fg6wflQ9XOcdEjMxeyqfPm1mc0MhIovhnVEkKFj6brDfqiZpLi/ryd1sOh8NdBrnKm0CMEz3mjuFNq1ViEU3DM4qgZr4/czETQWfaE1JgN6n/9FFiN+HUmTrClWklibCzf+yzJYfwrbNraaiw3a9nYdfw2mFXI5VRCe5NL2yie0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lWjScDyy; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73B57C433F1; Wed, 31 Jan 2024 23:11:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742704; bh=gpKFxD2je2tWWE8lEBMTW3z9BiZKvLJ/r4INUAW0y0Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lWjScDyyCyJYL8k/VWsDJeqGTPrU6Q0ILhUZ+4te5yiJ/JzBrk0rDkMmdvyiOs10F OMMPgOtmXWKz0CevqJZgruEl/nveIdYeiaDJ+xj1jOoBMA5W8i6eUpbrEb2XnoTQNd nsEilF9i79TxkT4Kk9okYcjaKXHkiSh/r3aSBpm2/DmzVVwY9ZiQq4DsantDtFGvFE XVppD2DLIStIt0tGqbTFkOyhoAYlQ3ck1tMxjhXCc7//QDwjrqFEhweWbNr77UvbOV xXzG53fNWZ5npmxYnuwC7QNJzN2sU7uXxjmwjKvtrormdv2HiOLu5gRtY1fqF9C/ka VwuyUtAJFlMww== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 08/15] tick: Move tick cancellation up to CPUHP_AP_TICK_DYING Date: Thu, 1 Feb 2024 00:11:13 +0100 Message-ID: <20240131231120.12006-9-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789652232175037950 X-GMAIL-MSGID: 1789652232175037950 The tick hrtimer is cancelled right before hrtimers are migrated. This is done from the hrtimer subsystem even though it shouldn't know about its actual users. Move instead the tick hrtimer cancellation to the relevant CPU hotplug state that aims at centralizing high level tick shutdown operations so that the related flow is easy to follow. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/hrtimer.c | 2 -- kernel/time/tick-common.c | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 355b5a957f7f..3b456ec7d4fb 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -2224,8 +2224,6 @@ int hrtimers_cpu_dying(unsigned int dying_cpu) struct hrtimer_cpu_base *old_base, *new_base; int i, ncpu = cpumask_first(cpu_active_mask); - tick_cancel_sched_timer(dying_cpu); - old_base = this_cpu_ptr(&hrtimer_bases); new_base = &per_cpu(hrtimer_bases, ncpu); diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index a89ef450fda7..b4af8c743b73 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -410,6 +410,8 @@ int tick_cpu_dying(unsigned int dying_cpu) if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); + tick_cancel_sched_timer(dying_cpu); + return 0; } From patchwork Wed Jan 31 23:11:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195061 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp93072dyc; Wed, 31 Jan 2024 15:37:20 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdS+fcOYSMVvCsB9Pd5IC7kOQAK5WiI7SIilfaPTC7JjsCSood9LLNN49jS1MV0Dqb4auv X-Received: by 2002:a92:d34e:0:b0:363:7f6e:e868 with SMTP id a14-20020a92d34e000000b003637f6ee868mr715491ilh.19.1706744240454; Wed, 31 Jan 2024 15:37:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744240; cv=pass; d=google.com; s=arc-20160816; b=KCdfXcIZByIc78ns995yFTf5qbDDsRGyiGUgPHa7uWsEI/8ojewZSw48WzYI1zNpAU 6bHlE3cXdqyL4+nUzIQoHzxnjGqRyc4HkJ+aUvv6loLIvOHpVVAiXnVXrjTsfeisDojT 5usBqA2SMRFfHAcaTkOO2cfkzPrtb2c2Ey/xQSSQyOxsEb2MdUFV8fq2+RpTAdQgv72Y IUdq9an+l3FRDjRQGn7oZZVdtlBtuG6kB+W89I1SJ7mgvT6ZTHgE+cq7UuTVdBpV9U6E DiEHF/G7btatAs2eewz+NSHefHYjrEeHTAZFIvNQ1hf8mhDKLzRpv4oeuw3NiDOrlS+L R3Sg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=g4noiabl/09fRDiQUrw88BLuen4Vzt8p3dhk8Ys+lHE=; fh=4SdP0HqR7u5QzeC5ZYB3F7EhDzmU0mwvZ9m/7mcx8R4=; b=vqanIrxT8suumLybdDyhzHxy6R2ct1V7Q6RB+oBrO9ldGwS8oegP3XPIig67L23dIY 2kZQG1+zMJ1HoGCAEWWJKMtlKwhD7GlmEmsB2HdgRcgVDRCx+7oCZa6OiPQcmAmnZR2r YAY82fil4ItW7JJPyMRBPO92WOTdf/DpCpwcg+ZN/yBnbcUtpnQPzSgu8eW0piEL21C7 HgEFzg4HW8Yfx8GI/cxoaKMe+iHEtxeOoP5uryuki8lz4MCdJE8SGyU8Ns+mYtaZoPlq gP+7k4F9JtrTWB5c4dfKcBMfCMUsrwWxJjg0mmwBmvWvs0VCtkm72Xc0r/xQPgGMAGOX K5tA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=W1iv1NUe; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47353-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47353-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUvfvyy5cXZttdLjz6Ax0C2jt9QzkrHO7sDoTUQdvwwa6ucetkoRhFeggFBf9DdiHuoWCRys23IH6ShIEo4kryDcg8wXQ== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id f12-20020a65628c000000b005d64d951c89si7863212pgv.143.2024.01.31.15.37.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:37:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47353-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=W1iv1NUe; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47353-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47353-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 8E6962965D6 for ; Wed, 31 Jan 2024 23:30:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C4C0012BF19; Wed, 31 Jan 2024 23:11:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="W1iv1NUe" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BE6D69D01 for ; Wed, 31 Jan 2024 23:11:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742707; cv=none; b=mdH+SzE6wkUq/L79Q1/TuyG2PqlAwatf1vCUGxR+YTITm0mQ8b8qRUEWmxd4wZUJ+1wmD7kewphitx3YuEqYpcJmlhdBbjDVsajmYOmLFFfyLkZCfNFcsa84/MNABsPyuU1sTcWR59sOHUQBVUbhL5FCmUo/Nlw52Abr5kwImTA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742707; c=relaxed/simple; bh=MlUbkGKGjJi82z5TxfBnlqTsq0RVRvhDZpea1eJZW/w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cN8lrmYSlJm5QtHqwUDJTB+49io/aFH0cq3HmcEaNahUPLMaJJGXKPMI0bylSy4Xu/YEDopIMBzPEib6CRX8okYLAt/LnyNWJcTfLKkFhH1QwVHE3h94uFeV5xxVs93EIcV3/UqVwhU2jXu3lnexNI8WTnYcB3YIRTrowu/ptYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=W1iv1NUe; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id B24EDC43390; Wed, 31 Jan 2024 23:11:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742706; bh=MlUbkGKGjJi82z5TxfBnlqTsq0RVRvhDZpea1eJZW/w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W1iv1NUe6YFmz80ocBTAV/rgd85OVpBLfRdAaRWhQko51iiRTuZSjsESN4+4epWte MVqy+xp9ee0/+vODHuypmRFuzISZkELUlpAu787iYFNS42CnqGtirMrLbzkR5VpZG7 799amkNTKfWi/jPIUPdVuyX3w9ZzLFroBT4gXMpK1qRoadY6eD8kstdsNBTvMG69SR /CpvlyVZSV9He4HLIv7UKhtr6uzkHAMpIuEk/i7juVIS356hIHUZwtv0xmk+KbZWss PwEBeaHHcfW2Z9bGlNdWGLx0J9eWJOB+ICo20YHNgg6EgJ4LoZSn7O9CQ21lbm/N5g FCZIT1JPDKKWg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 09/15] tick: Move broadcast cancellation up to CPUHP_AP_TICK_DYING Date: Thu, 1 Feb 2024 00:11:14 +0100 Message-ID: <20240131231120.12006-10-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789651048780202896 X-GMAIL-MSGID: 1789651048780202896 The broadcast shutdown code is executed through a random explicit call within stop machine from the outgoing CPU. However the tick broadcast is a midware between the tick callback and the clocksource, therefore it makes more sense to shut it down after the tick callback and before the clocksource drivers. Move it instead to the common tick shutdown CPU hotplug state where related operations can be ordered from highest to lowest level. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- include/linux/tick.h | 6 ------ kernel/cpu.c | 2 -- kernel/time/tick-common.c | 3 +++ kernel/time/tick-internal.h | 2 ++ 4 files changed, 5 insertions(+), 8 deletions(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index afff4c207bd8..c7840ae8ebaf 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -73,12 +73,6 @@ extern void tick_broadcast_control(enum tick_broadcast_mode mode); static inline void tick_broadcast_control(enum tick_broadcast_mode mode) { } #endif /* BROADCAST */ -#if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_HOTPLUG_CPU) -extern void tick_offline_cpu(unsigned int cpu); -#else -static inline void tick_offline_cpu(unsigned int cpu) { } -#endif - #ifdef CONFIG_GENERIC_CLOCKEVENTS extern int tick_broadcast_oneshot_control(enum tick_broadcast_state state); #else diff --git a/kernel/cpu.c b/kernel/cpu.c index 263508073da8..5a8ad4f5ccf3 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1324,8 +1324,6 @@ static int take_cpu_down(void *_param) */ cpuhp_invoke_callback_range_nofail(false, cpu, st, target); - /* Remove CPU from timer broadcasting */ - tick_offline_cpu(cpu); /* Park the stopper thread */ stop_machine_park(cpu); return 0; diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index b4af8c743b73..522414089c0d 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -412,6 +412,9 @@ int tick_cpu_dying(unsigned int dying_cpu) tick_cancel_sched_timer(dying_cpu); + /* Remove CPU from timer broadcasting */ + tick_offline_cpu(dying_cpu); + return 0; } diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h index 481b7ab65e2c..a939ff6de97c 100644 --- a/kernel/time/tick-internal.h +++ b/kernel/time/tick-internal.h @@ -137,8 +137,10 @@ static inline bool tick_broadcast_oneshot_available(void) { return tick_oneshot_ #endif /* !(BROADCAST && ONESHOT) */ #if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_HOTPLUG_CPU) +extern void tick_offline_cpu(unsigned int cpu); extern void tick_broadcast_offline(unsigned int cpu); #else +static inline void tick_offline_cpu(unsigned int cpu) { } static inline void tick_broadcast_offline(unsigned int cpu) { } #endif From patchwork Wed Jan 31 23:11:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195062 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp93168dyc; Wed, 31 Jan 2024 15:37:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFRCPBPFSTCKnutom1M3kPV7B4E3EuYE7l5lMhDCRwaMVbo6/8aMnlQCaU7nb8uUIcRF5vN X-Received: by 2002:a05:6a20:8813:b0:19e:30a6:fe29 with SMTP id c19-20020a056a20881300b0019e30a6fe29mr4061510pzf.31.1706744253892; Wed, 31 Jan 2024 15:37:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744253; cv=pass; d=google.com; s=arc-20160816; b=spUhRlP6RFgPti+8w7xfGpMz2GsenLy6VJ8SRowCI+BOaR5RN1ZBRNK8SwGl6zFnLL xtBEYgQ4nigJrUFGpvpV4Ll2aua8b7w4msYn+AvdhrxnJw5r8UmxWJXW93PXcaNrYNMA zS5HAV5rau9RM/iCt2JO0SQ9q4LhqbdEMqvtxryWeuCZo8IP6XKom66DZ9Ad0tFJzfb4 tthhcfXUQaz5n8qIn1oF57fNXzbPS3b9bp59BTGR79Pa5tl5Dw/V6zmtC/ZCq+MLtqpm mTMmgej/ayE7LGaYqYEXzI4ENat0oZas7mk9+pw+vVHSC2livlVXPsfYXsnvdEjRAJkL 9C7w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=cFSuVjv2FnCTnAbyXHWEfFPMC5xjEe1q4efox4MIC94=; fh=fioWudWJonTuqdWXgxgc7aKCh5Gu2ar35DezYnuu9b8=; b=EP4if2Hrr8lKnsRblXzhMkQtb+OSaM2D5gpVSbnrRONooozyr2Z56/76/HJOy0Np5S +ZFv6eqMJ9cPYUVqxVhQ7If8dfUN4XiY+XFhtS7gp20sTZLxfLimIyeVvdEnfHB9Z+b/ oTR/LVbl+0L41LDoxL+4M4ayAYNsYgj175qF2eSCH3DtWRhCG9IrILd9ng/RuQYxK8C2 dR+Ju/thrjjXUD0MSJCoM1LTsNbZZ/aXfsYlyDQak/tZXOohZ4YYVRKazah+ul+Gs5lc toScanjcYOA9gOeRUkT/t6gDdBxlWGwRFHNqqnRLAWMMzpi/R+nrI9mLrN/EF646PzWu TIhg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=XXxcfO8+; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47354-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47354-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCVQ2vfbWnOx2/UefcyBtZgn2XEvFAs30oDHuhSV4LcixvPXU9rjGAY4LCmJq+griFOyjeyz+/VPjJPayD+VoxCatAHdEw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id h14-20020a65638e000000b005d8b5aace83si8666474pgv.289.2024.01.31.15.37.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:37:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47354-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=XXxcfO8+; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47354-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47354-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EA29F296A58 for ; Wed, 31 Jan 2024 23:30:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3658C13343E; Wed, 31 Jan 2024 23:11:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XXxcfO8+" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F03B12BEBE for ; Wed, 31 Jan 2024 23:11:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742709; cv=none; b=M1KHCVTvN3VOtpUWV2hUHHG9Uy/InO00JQxGd/0QSx6BWsDQYNvfA5o11DcMSek8zynFe6LLmswsChmXsdRFEbLbDBdoVGf7t+GH8GVkKoEPnSPxWhQmg0LQza291iXZgJOsZO6d9TZbV5cZKrlWoXUZcFaTRRze9s+vkwbw23A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742709; c=relaxed/simple; bh=yj/CebTCEK8vWtZdTKGdNz/nxnJEqwYwgzARrGP9DTU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FHun7Ee/QzOCmO49/XcfWnnlORdaVNL4qGXc1H9nb/E4PivXjeGgNQ0NscUzEBEKOuy1b+HqqsMNzcTZmyVZmhqshc9Yd5hIxFtHr5dpKEYpCDM3fPVYFGk0O+VL84L9KfVqBWg0kz/xhxr1UAhi3bieYWc6sfCaxh2dIQfNOyE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XXxcfO8+; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2CD82C433F1; Wed, 31 Jan 2024 23:11:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742708; bh=yj/CebTCEK8vWtZdTKGdNz/nxnJEqwYwgzARrGP9DTU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XXxcfO8+OSxfeXXn/jfn6uK+epAT2LG5oixSOXkrlhT8llyCXH2uiHdGNQc/C6bTt d2c7eeqMOtd6yLGhjHg79Ysw5W2yMFWGtDSxcJLKyG+oJ/MsOIay0+LKCtjml31oG5 +t217wHeMRRDJmwR6Y1BTZT1QH5WPUuku8L2o8gVKPyW4QRJxjvaHJjBMOob0C2bee yZraAGTpac9N4aVl7Bf5n4fRHw7w8gijebTxxO+Bcdi9qSfOrbMjLx4wvqaur5OpCR CE1TNVugxbyoq+mozFcOFeMIOSqjwh3/mB2Vjf3HVrPTvOfvCA+7slVFeO1B/I3AH5 TlQpFnp7qRO4w== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 10/15] tick: Assume the tick can't be stopped in NOHZ_MODE_INACTIVE mode Date: Thu, 1 Feb 2024 00:11:15 +0100 Message-ID: <20240131231120.12006-11-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789651062763701685 X-GMAIL-MSGID: 1789651062763701685 The full-nohz update function checks if the nohz mode is active before proceeding. It considers one exception though: if the tick is already stopped even though the nohz mode is inactive, it still moves on in order to update/restart the tick if needed. However in order for the tick to be stopped, the nohz_mode has to be either NOHZ_MODE_LOWRES or NOHZ_MODE_HIGHRES. Therefore it doesn't make sense to test if the tick is stopped before verifying NOHZ_MODE_INACTIVE mode. Remove the needless related condition. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index a07a04bb9d27..0d0370d8111e 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1043,7 +1043,7 @@ static void tick_nohz_full_update_tick(struct tick_sched *ts) if (!tick_nohz_full_cpu(smp_processor_id())) return; - if (!ts->tick_stopped && ts->nohz_mode == NOHZ_MODE_INACTIVE) + if (ts->nohz_mode == NOHZ_MODE_INACTIVE) return; __tick_nohz_full_update_tick(ts, ktime_get()); From patchwork Wed Jan 31 23:11:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195052 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp90503dyc; Wed, 31 Jan 2024 15:31:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IEgY5I6suYlwULAvls5Ew44W3tYBahmVYbypRPPJmzouDeBcFtR5aBZRDO7UqgHUWCitnvO X-Received: by 2002:a37:e30d:0:b0:784:93d:c905 with SMTP id y13-20020a37e30d000000b00784093dc905mr963532qki.25.1706743882083; Wed, 31 Jan 2024 15:31:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743882; cv=pass; d=google.com; s=arc-20160816; b=vFqFK9gbPuce8wLRW1UR2NEsl1go1TRxpTVSjUgIUPUd7d/SOTShc5GyQ5pA5tpQpp GqwUjJ1OX1Z/zfqD+dk6aInceCIz8A3jHEmJ8W95WTlLWxFKYdqpCpdsnUgZJfIDtRvy FKxQg0DrWdyNhOTwJmjy4ObCXPxBg7ukoWP4OPEO6Y6dzQ8tAvHa90PTrOLR6aV1yTAd 5+itZfFYNTmWBEzH6G9xWynmpPEp2panACnHu9e6pF2ekBqMpAXqVSpkjIvZusvCnMgo qBkCbVxtY+mEafiX6bcmImWKsVb5AFP/jDxiAJgjTb0xiHWkrZHyuLer5NLV5dg9Wnhc 5qnw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=32PBNjbxtnXDVrDZlxu6pNH0KE8j3mQZCqSzuvfBD2Q=; fh=e0jm8O4mpaMhoJPc+134gmfVNgtq7zfpKWki8r+aFi8=; b=iKVOEUR0eOtbsp1D1v8/TmVxY334p3dXBQznTucUtHNzCmSjstVggLET5DrpNJgKvw UrqAE47GeW+cGQmiM2nkZgvKz6Alad+MvBli7Nu4BSGSVO/qAlib8OxW1xQj2Iil0Qg0 0CTTzScsblURRAzd3GsqyxcsqOVCBRUvpzRNNB8VmBGZ5oCiO3Mur4BFQBgEc2/6P94X TpHxfCp1j5N6a/h6iGw46ItQpyNmNeUDmioIoMfLvJSdFcgTVy6m5WdxKc53fNDaFDTS PP+P0SVrlqiKXqfMOYWwIE3NYaEOnIeF0eaRrxNPUTRAuaLsKbO+HFBspGpHuA0jH3Ru 6BsA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="FoLMS/qV"; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47355-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47355-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUFSt9BnZKaQ8UbZVdcPvMhauLHgJZUW1hUwNb6DJ/RS3JufRay9acJzzJj1HMQ6lXI2laIlJbu6tuvlN3VDAOsHb8ddA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id q2-20020a05620a038200b0078321c60d62si6427008qkm.762.2024.01.31.15.31.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:31:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47355-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="FoLMS/qV"; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47355-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47355-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9B7F11C27A56 for ; Wed, 31 Jan 2024 23:30:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 330CC3B7A9; Wed, 31 Jan 2024 23:11:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FoLMS/qV" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B76D12FF6E for ; Wed, 31 Jan 2024 23:11:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742711; cv=none; b=chNKGfPzykxl4KEsRGV0PfZHFy5SAGibRecDZodojej7GFdN9tNsfJ8tpM0VdMfRQ01Z1chDJAma9k6BiG4wyy6anNPnOFT8giuonP13EFWpJxh5QmDsCWZZXTOJnursbRVopHEWHky/ne4g9BwyCdno/AT226kIATyzwd/6cKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742711; c=relaxed/simple; bh=ENJnGfyd2+EEzXokoqGcyz6LV1lezsDunVEKa8lSN3M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eDVhSEUSV7jlK8EhgbeIvMxJqc4wG0/qxy5dTHD84UkN8NxKVfVHW5Bb4VvErWLvPTxtRjiFkANezAMPW3gfa6XlNapKBNLDBPA3Apsq+MH6YqXFQ9y6sWWNvP87bDe080RKn8dK4OEItqa7tnqY0Hw2CCENqKbF5Riik7JfLQ0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FoLMS/qV; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6ABABC43390; Wed, 31 Jan 2024 23:11:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742711; bh=ENJnGfyd2+EEzXokoqGcyz6LV1lezsDunVEKa8lSN3M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FoLMS/qVXCgH12a1Qre5TUQYW2zMqW7aif3QRBnwmKnh/CwClddGFrVOQ3/wHBehZ TpOsRQfoWIKxlSNiYGI7h0xrDjIIm/aMCRmehB12wpuZWLFxlU8AAP/+oEqCya/Eck q5TvYxnTcFLJTVE47WljMBjsX8Fj1t2gjHmGIioovO9JI7rN7iPCy2h9VTQvwk81jP NqefnWqWuoMD56dIHL6vDQa+qUT+ckIMaRM6Ilo/TJQsdN9nkWAPCbggi6v2zRTuu2 peEAgUekX9mf6T7XW5fduGkpFukgYKmfkn49+VIj2KBdP74xgTa5idCqhdx+/ZdJvn QzTpk1RI/c65g== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 11/15] tick: Move got_idle_tick away from common flags Date: Thu, 1 Feb 2024 00:11:16 +0100 Message-ID: <20240131231120.12006-12-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650672949759873 X-GMAIL-MSGID: 1789650672949759873 tick_nohz_idle_got_tick() is called by cpuidle_reflect() within the idle loop with interrupts enabled. This function modifies the struct tick_sched's bitfield "got_idle_tick". However this bitfield is stored within the same mask as other bitfields that can be modified from interrupts. Fortunately so far it looks like the only race that can happen is while writing ->got_idle_tick to 0, an interrupt fires and writes the ->idle_active field to 0. It's then possible that the interrupted write to ->got_idle_tick writes back the old value of ->idle_active back to 1. However if that happens, the worst possible outcome is that the time spent between that interrupt and the upcoming call to tick_nohz_idle_exit() is accounted as idle, which is negligible quantity. Still all the bitfield writes within this struct tick_sched's shadow mask should be IRQ-safe. Therefore move this bitfield out to its own storage to avoid further suprises. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 35808bbb8a47..3b555e0fa937 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -61,7 +61,6 @@ struct tick_sched { unsigned int tick_stopped : 1; unsigned int idle_active : 1; unsigned int do_timer_last : 1; - unsigned int got_idle_tick : 1; /* Tick handling: jiffies stall check */ unsigned int stalled_jiffies; @@ -73,6 +72,7 @@ struct tick_sched { ktime_t next_tick; unsigned long idle_jiffies; ktime_t idle_waketime; + unsigned int got_idle_tick; /* Idle entry */ seqcount_t idle_sleeptime_seq; From patchwork Wed Jan 31 23:11:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195063 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp93326dyc; Wed, 31 Jan 2024 15:37:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IFv2ckZcjv7ROKzP2u3XoRzPKhG7abXsxNYp5Wx7Clo3FgeCuSnzyYZGOue1A534YFdVV3u X-Received: by 2002:a05:6a20:e09:b0:19c:9f1b:1d8b with SMTP id ej9-20020a056a200e0900b0019c9f1b1d8bmr2725009pzb.16.1706744278961; Wed, 31 Jan 2024 15:37:58 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744278; cv=pass; d=google.com; s=arc-20160816; b=QYONmoz6gu6fRxa6Gq/XuSTbWXPbzSxI3LfWRr5ImWVBBZN3sD7eQC1nUerxL8vQXz nvwggESd+ESXXqZu0QEUNNgiOilp0cFnPGICvPaCH89oliCDyFajOSZZzibrw6+BPLxa bLfcXjsn9wMNmiDEU6lIpACO+QVlg658D1ZsIKoBTVGXG1lEmcd+f9gjDTvqcd7I34yq qFnGnn1JOhBx0sF4Nc1ihBt02JnCMeS+exn57g5SspRj1BuhjbaxlhwSOUfxhiG5QzOs sIhEF3aHsoI7pvgk8yvE+a0WguV9ndGFjPnH/Y/OWv1LwTYJavrHikIlUJGJFva5AyjP WPVQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=8ObH/Y9xzHBDUl1D95xWrj0Lzt+hYVEsyO4NWsOAmOQ=; fh=h+p7UyqHrZ9etc37GTE+p33pY/t59X0i2vHjC7ORXTo=; b=FPltf0v1+jjqdiFNJCh/oARlvuyGYrAoP7ZYC3v0y8b6pEy9jNXZ7Jzz+E+K2d724S rQlk0Ur+kZbTYbYYi/zm74Q/fbxpP5nQfQkcdYJdISNIMWdVaQV/JLsHzmUTAlpJ4ux/ KKO8vSRX11qS3qFayqObIZi26RjRSBEl8kfdjfk5dnbxsi39bWGDQgife7D53PXuy66W BblmRdRvc11xcWpvCCmB+UxsZahZriMK8vmyXdNEOZ1sK7dU4Q8hH7kOrTHWey7K3l0j vfKL/EJkC110kUF2kReCV4s0we5t3zh2y1InLZiEIYmTgv34bMmIy/YMX+o9WAxFUN24 hybA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=S5b4cLei; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47356-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47356-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUqRIycysZBoMXupNATp0AWCpIeL/E7aynej7PC/CaI6qicOWtRiSS3kCNSmtUcqNadMawffQdPBR/0J9xHf6e7uSTxnQ== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id ld10-20020a056a004f8a00b006ddd309cd5asi10900710pfb.168.2024.01.31.15.37.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:37:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47356-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=S5b4cLei; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47356-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47356-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 41EB1297982 for ; Wed, 31 Jan 2024 23:31:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8502912CDBA; Wed, 31 Jan 2024 23:11:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="S5b4cLei" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 372443B7AC for ; Wed, 31 Jan 2024 23:11:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742714; cv=none; b=a8oPYPaxr7IxsUR8EG3vSBnXFjxqODdKZZRJ0UJTBw4u26gPQpZnOMaqbOVmvhklluqvzHReuR2PVJ/cWQx54NJdA2G25wKw3ZUOJopjOaKa9MNdqp9LqtKmAfmcJ6ZHZR2ohZMtVm9lzFTzjf5AUB4VixpwU6ID7Oky6Ah0zNI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742714; c=relaxed/simple; bh=XkWuJ5fw7w08+HyxxUB2hvTzRk6iMKe6n5kSUDnXdCU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Iywfo+rVXRLlWO0+EHDx1BURDAV0raMPKJPTuDtveh3VQxGdHtR+BSpwtl66UQkPe1JYw02eJuZGTsrjCycfy9L8Nj/2LBY4Sj5hqZ/ATReBvsKXec/fwDmfAQzQCRHNBJCAqQU0+LP+Sa25WHaV9A0t9ojuUCBOLnOL0HfKNvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=S5b4cLei; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C432EC433C7; Wed, 31 Jan 2024 23:11:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742713; bh=XkWuJ5fw7w08+HyxxUB2hvTzRk6iMKe6n5kSUDnXdCU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S5b4cLei5IfmrCS+8j9QFaJebBny3WJtbGc/f9EVulMrGVzS/54wGZTJF0peJa/Cl bK1ZQJ4EO3EE9hh/dPjBpGSmZr7uUxkrWAUFF/ACgug0HDwUugd+Tcf1H6eEaG4GIk JI/JdkxiTZRvA2WHUjKvQTcoKWpxLKiYVyecmX1+IxtG2WWEBkOXec+q2qRUVLT0C2 obg+9jE7sHwhCaHo3FaRwbuXCOTFgNhC/BVLc7qO9R0MQQ48o0eCOw4HFJ40395MVN TieVdH0w7fDKXkNHbQ//ZOn0lXBbir/65yix2pi06A96NaiceYP+em1lSUBcKHTuqs NIkQ96W2GbrXw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 12/15] tick: Move individual bit features to debuggable mask accesses Date: Thu, 1 Feb 2024 00:11:17 +0100 Message-ID: <20240131231120.12006-13-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789651088839660459 X-GMAIL-MSGID: 1789651088839660459 The individual bitfields of struct tick_sched must be modified from IRQs disabled places, otherwise local modifications can race due to them sharing the same memory storage. The recent move of the "got_idle_tick" bitfield to its own storage shows that the use of these bitfields, as pretty as they look, can be as much error prone. In order to avoid future issues of the like and make sure that those bitfields are safely accessed, move those flags to an explicit mask along with a mutator function performing the basic IRQs disabled sanity check. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 86 +++++++++++++++++++++++++--------------- kernel/time/tick-sched.h | 23 ++++++----- kernel/time/timer_list.c | 5 ++- 3 files changed, 72 insertions(+), 42 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 0d0370d8111e..3e0a53c1b8bd 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -180,6 +180,26 @@ static ktime_t tick_init_jiffy_update(void) return period; } +static inline int tick_sched_flag_test(struct tick_sched *ts, + unsigned long flag) +{ + return !!(ts->flags & flag); +} + +static inline void tick_sched_flag_set(struct tick_sched *ts, + unsigned long flag) +{ + lockdep_assert_irqs_disabled(); + ts->flags |= flag; +} + +static inline void tick_sched_flag_clear(struct tick_sched *ts, + unsigned long flag) +{ + lockdep_assert_irqs_disabled(); + ts->flags &= ~flag; +} + #define MAX_STALLED_JIFFIES 5 static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) @@ -223,7 +243,7 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) } } - if (ts->inidle) + if (tick_sched_flag_test(ts, TS_FLAG_INIDLE)) ts->got_idle_tick = 1; } @@ -237,7 +257,8 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) * idle" jiffy stamp so the idle accounting adjustment we do * when we go busy again does not account too many ticks. */ - if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && ts->tick_stopped) { + if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && + tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { touch_softlockup_watchdog_sched(); if (is_idle_task(current)) ts->idle_jiffies++; @@ -279,7 +300,7 @@ static enum hrtimer_restart tick_nohz_handler(struct hrtimer *timer) * - to the idle task if in dynticks-idle * - to IRQ exit if in full-dynticks. */ - if (unlikely(ts->tick_stopped)) + if (unlikely(tick_sched_flag_test(ts, TS_FLAG_STOPPED))) return HRTIMER_NORESTART; hrtimer_forward(timer, now, TICK_NSEC); @@ -559,7 +580,7 @@ void __tick_nohz_task_switch(void) ts = this_cpu_ptr(&tick_cpu_sched); - if (ts->tick_stopped) { + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { if (atomic_read(¤t->tick_dep_mask) || atomic_read(¤t->signal->tick_dep_mask)) tick_nohz_full_kick(); @@ -656,14 +677,14 @@ bool tick_nohz_tick_stopped(void) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - return ts->tick_stopped; + return tick_sched_flag_test(ts, TS_FLAG_STOPPED); } bool tick_nohz_tick_stopped_cpu(int cpu) { struct tick_sched *ts = per_cpu_ptr(&tick_cpu_sched, cpu); - return ts->tick_stopped; + return tick_sched_flag_test(ts, TS_FLAG_STOPPED); } /** @@ -693,7 +714,7 @@ static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) { ktime_t delta; - if (WARN_ON_ONCE(!ts->idle_active)) + if (WARN_ON_ONCE(!tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE))) return; delta = ktime_sub(now, ts->idle_entrytime); @@ -705,7 +726,7 @@ static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); ts->idle_entrytime = now; - ts->idle_active = 0; + tick_sched_flag_clear(ts, TS_FLAG_IDLE_ACTIVE); write_seqcount_end(&ts->idle_sleeptime_seq); sched_clock_idle_wakeup_event(); @@ -715,7 +736,7 @@ static void tick_nohz_start_idle(struct tick_sched *ts) { write_seqcount_begin(&ts->idle_sleeptime_seq); ts->idle_entrytime = ktime_get(); - ts->idle_active = 1; + tick_sched_flag_set(ts, TS_FLAG_IDLE_ACTIVE); write_seqcount_end(&ts->idle_sleeptime_seq); sched_clock_idle_sleep_event(); @@ -737,7 +758,7 @@ static u64 get_cpu_sleep_time_us(struct tick_sched *ts, ktime_t *sleeptime, do { seq = read_seqcount_begin(&ts->idle_sleeptime_seq); - if (ts->idle_active && compute_delta) { + if (tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE) && compute_delta) { ktime_t delta = ktime_sub(now, ts->idle_entrytime); idle = ktime_add(*sleeptime, delta); @@ -888,7 +909,7 @@ static ktime_t tick_nohz_next_event(struct tick_sched *ts, int cpu) * We've not stopped the tick yet, and there's a timer in the * next period, so no point in stopping it either, bail. */ - if (!ts->tick_stopped) { + if (!tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { ts->timer_expires = 0; goto out; } @@ -901,7 +922,8 @@ static ktime_t tick_nohz_next_event(struct tick_sched *ts, int cpu) */ delta = timekeeping_max_deferment(); if (cpu != tick_do_timer_cpu && - (tick_do_timer_cpu != TICK_DO_TIMER_NONE || !ts->do_timer_last)) + (tick_do_timer_cpu != TICK_DO_TIMER_NONE || + !tick_sched_flag_test(ts, TS_FLAG_DO_TIMER_LAST))) delta = KTIME_MAX; /* Calculate the next expiry time */ @@ -935,13 +957,13 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) */ if (cpu == tick_do_timer_cpu) { tick_do_timer_cpu = TICK_DO_TIMER_NONE; - ts->do_timer_last = 1; + tick_sched_flag_set(ts, TS_FLAG_DO_TIMER_LAST); } else if (tick_do_timer_cpu != TICK_DO_TIMER_NONE) { - ts->do_timer_last = 0; + tick_sched_flag_clear(ts, TS_FLAG_DO_TIMER_LAST); } /* Skip reprogram of event if it's not changed */ - if (ts->tick_stopped && (expires == ts->next_tick)) { + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED) && (expires == ts->next_tick)) { /* Sanity check: make sure clockevent is actually programmed */ if (expires == KTIME_MAX || ts->next_tick == hrtimer_get_expires(&ts->sched_timer)) return; @@ -959,12 +981,12 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * call we save the current tick time, so we can restart the * scheduler tick in tick_nohz_restart_sched_tick(). */ - if (!ts->tick_stopped) { + if (!tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { calc_load_nohz_start(); quiet_vmstat(); ts->last_tick = hrtimer_get_expires(&ts->sched_timer); - ts->tick_stopped = 1; + tick_sched_flag_set(ts, TS_FLAG_STOPPED); trace_tick_stop(1, TICK_DEP_MASK_NONE); } @@ -1021,7 +1043,7 @@ static void tick_nohz_restart_sched_tick(struct tick_sched *ts, ktime_t now) touch_softlockup_watchdog_sched(); /* Cancel the scheduled timer and restore the tick: */ - ts->tick_stopped = 0; + tick_sched_flag_clear(ts, TS_FLAG_STOPPED); tick_nohz_restart(ts, now); } @@ -1033,7 +1055,7 @@ static void __tick_nohz_full_update_tick(struct tick_sched *ts, if (can_stop_full_tick(cpu, ts)) tick_nohz_full_stop_tick(ts, cpu); - else if (ts->tick_stopped) + else if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) tick_nohz_restart_sched_tick(ts, now); #endif } @@ -1153,14 +1175,14 @@ void tick_nohz_idle_stop_tick(void) ts->idle_calls++; if (expires > 0LL) { - int was_stopped = ts->tick_stopped; + int was_stopped = tick_sched_flag_test(ts, TS_FLAG_STOPPED); tick_nohz_stop_tick(ts, cpu); ts->idle_sleeps++; ts->idle_expires = expires; - if (!was_stopped && ts->tick_stopped) { + if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { ts->idle_jiffies = ts->last_jiffies; nohz_balance_enter_idle(cpu); } @@ -1196,7 +1218,7 @@ void tick_nohz_idle_enter(void) WARN_ON_ONCE(ts->timer_expires_base); - ts->inidle = 1; + tick_sched_flag_set(ts, TS_FLAG_INIDLE); tick_nohz_start_idle(ts); local_irq_enable(); @@ -1225,7 +1247,7 @@ void tick_nohz_irq_exit(void) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - if (ts->inidle) + if (tick_sched_flag_test(ts, TS_FLAG_INIDLE)) tick_nohz_start_idle(ts); else tick_nohz_full_update_tick(ts); @@ -1279,7 +1301,7 @@ ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next) ktime_t now = ts->idle_entrytime; ktime_t next_event; - WARN_ON_ONCE(!ts->inidle); + WARN_ON_ONCE(!tick_sched_flag_test(ts, TS_FLAG_INIDLE)); *delta_next = ktime_sub(dev->next_event, now); @@ -1351,7 +1373,7 @@ void tick_nohz_idle_restart_tick(void) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - if (ts->tick_stopped) { + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { ktime_t now = ktime_get(); tick_nohz_restart_sched_tick(ts, now); tick_nohz_account_idle_time(ts, now); @@ -1392,12 +1414,12 @@ void tick_nohz_idle_exit(void) local_irq_disable(); - WARN_ON_ONCE(!ts->inidle); + WARN_ON_ONCE(!tick_sched_flag_test(ts, TS_FLAG_INIDLE)); WARN_ON_ONCE(ts->timer_expires_base); - ts->inidle = 0; - idle_active = ts->idle_active; - tick_stopped = ts->tick_stopped; + tick_sched_flag_clear(ts, TS_FLAG_INIDLE); + idle_active = tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE); + tick_stopped = tick_sched_flag_test(ts, TS_FLAG_STOPPED); if (idle_active || tick_stopped) now = ktime_get(); @@ -1460,10 +1482,10 @@ static inline void tick_nohz_irq_enter(void) struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); ktime_t now; - if (!ts->idle_active && !ts->tick_stopped) + if (!tick_sched_flag_test(ts, TS_FLAG_STOPPED | TS_FLAG_IDLE_ACTIVE)) return; now = ktime_get(); - if (ts->idle_active) + if (tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE)) tick_nohz_stop_idle(ts, now); /* * If all CPUs are idle we may need to update a stale jiffies value. @@ -1472,7 +1494,7 @@ static inline void tick_nohz_irq_enter(void) * rare case (typically stop machine). So we must make sure we have a * last resort. */ - if (ts->tick_stopped) + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) tick_nohz_update_jiffies(now); } diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 3b555e0fa937..07a4c0144c47 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -20,14 +20,22 @@ enum tick_nohz_mode { NOHZ_MODE_HIGHRES, }; +/* The CPU is in the tick idle mode */ +#define TS_FLAG_INIDLE BIT(0) +/* The idle tick has been stopped */ +#define TS_FLAG_STOPPED BIT(1) +/* + * Indicator that the CPU is actively in the tick idle mode; + * it is reset during irq handling phases. + */ +#define TS_FLAG_IDLE_ACTIVE BIT(2) +/* CPU was the last one doing do_timer before going idle */ +#define TS_FLAG_DO_TIMER_LAST BIT(3) + /** * struct tick_sched - sched tick emulation and no idle tick control/stats * - * @inidle: Indicator that the CPU is in the tick idle mode - * @tick_stopped: Indicator that the idle tick has been stopped - * @idle_active: Indicator that the CPU is actively in the tick idle mode; - * it is reset during irq handling phases. - * @do_timer_last: CPU was the last one doing do_timer before going idle + * @flags: State flags gathering the TS_FLAG_* features * @got_idle_tick: Tick timer function has run with @inidle set * @stalled_jiffies: Number of stalled jiffies detected across ticks * @last_tick_jiffies: Value of jiffies seen on last tick @@ -57,10 +65,7 @@ enum tick_nohz_mode { */ struct tick_sched { /* Common flags */ - unsigned int inidle : 1; - unsigned int tick_stopped : 1; - unsigned int idle_active : 1; - unsigned int do_timer_last : 1; + unsigned long flags; /* Tick handling: jiffies stall check */ unsigned int stalled_jiffies; diff --git a/kernel/time/timer_list.c b/kernel/time/timer_list.c index ed7d6ad694fb..38f81d836fc5 100644 --- a/kernel/time/timer_list.c +++ b/kernel/time/timer_list.c @@ -147,11 +147,14 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now) # define P_ns(x) \ SEQ_printf(m, " .%-15s: %Lu nsecs\n", #x, \ (unsigned long long)(ktime_to_ns(ts->x))) +# define P_flag(x, f) \ + SEQ_printf(m, " .%-15s: %d\n", #x, !!(ts->flags & (f))) + { struct tick_sched *ts = tick_get_tick_sched(cpu); P(nohz_mode); P_ns(last_tick); - P(tick_stopped); + P_flag(tick_stopped, TS_FLAG_STOPPED); P(idle_jiffies); P(idle_calls); P(idle_sleeps); From patchwork Wed Jan 31 23:11:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195054 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp90848dyc; Wed, 31 Jan 2024 15:32:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IFD9DmEoe4S15w/oK1Fx65xq1OuYGvva4g6WOQ8FVWzspYCyWfusNO1YO4HcmnI0ljjmICH X-Received: by 2002:a05:6512:488c:b0:510:f47:8d82 with SMTP id eq12-20020a056512488c00b005100f478d82mr493368lfb.22.1706743922878; Wed, 31 Jan 2024 15:32:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743922; cv=pass; d=google.com; s=arc-20160816; b=W602scij7qAphYwKmu6V4P1w6o+Ic5lSUXmkmJF421j4UUXivPMhoWOacy6jclBxAH KCYlHe99IrU6RdMmPoFdmRXrg4aoHYKbAma6b051d5OlZmGSOZbEM/KopqBkzf+ZASwo r/08jlCcThbg0YvEuJiBh7nrbxcZVJOXhiTdvXfUiKMdclJPbYlPNIIL0/oA7Bgz0Ico hpZ5ydjL3F0Ii4TZzae5evahgdCALpxMj+YuEa2GJWn7rlThInixhIexVt1znmnPTYqI RPqbG+dEK71UiriZsDq/VuviuZuZOFQA/jymU8A2qLC8sec4E1vBYTyq9LqoSb5Ssi53 dRbw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=4FNnF5rE5cvlPQ4c78h6C4TNDC4iD5rBBiuRXy/ICCI=; fh=epOPia7mcRsVHOric4UylZLvbQ3DwB+bJbmZua6DgOU=; b=MOr0mYmVGekORwAiuq+BHLs90l6OU9HZeR2j/TZoTUkSxpPmG0MQnGDJd1mux8LQBk YJS3TGRaq26ZJJONGiojRf9Pe23UEnwV5cRr6vesy7z5lAF+IJfAgxWn+pmVkCkIlMIr a5FTNRx7sYIzsH49PWQRK4HxpLgdhe/k81m6Ma73wEjATVi9RtaHREEMc3uxGfsPyc3z 1Anz08DgBwYT1ENJag60g95cjSneOKpcuHXtNy52r7ngAyDdUiT6KJzr0mQO0yLesTJm 4FvcOmXSBIX8Rj2u54hzJ4xKNZq3yAZnmit+QRVdoEvkJbOpSiIZrGzClIC2sI611U25 tZpQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TigkfeWi; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCWBRZA/VOuVh+zglZe6cPdxKjx+v5NMPf3NzKvBTaUlwshivGFVV6qeQ+3cxkOcYe1hu9nQZtaOxauzuSyRaLRDfPwSow== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id c12-20020a17090654cc00b00a353ca3d908si5084898ejp.556.2024.01.31.15.32.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:32:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TigkfeWi; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47357-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 794011F22CAB for ; Wed, 31 Jan 2024 23:31:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 40D971386DF; Wed, 31 Jan 2024 23:12:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TigkfeWi" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C884487A0 for ; Wed, 31 Jan 2024 23:11:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742716; cv=none; b=U9YKb50H0UyqvUtTTnGHIB8Cb4ZgsbA0kqF7+tPhxyDkrJQzGTeORCVT34SeLjY9wzEw7gSfu+IdBQGtSCIQIVVXJwlxUpjv5EQo54+z1jvtwah6WYraurTmQjF2cEsuRM4tYu35fcAsyTs+F9pP4fO2O9/+HWq/VCZyqKUYKYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742716; c=relaxed/simple; bh=PFXG2cNgodV6VNtmvaOYQJZ4ukS7LvHmLJbcHg8But0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kC6Nw7yq9KZechh4H65Tux2vDcwLS58kpfKbx4WNpuanTuoM7/yCpg3NBECDXTbhgj81R5gGAT3Jg0VKmmbv4lLll+Fbtx5OwTVMMuabt+zAG2fsJ2nDx+2ON5QRjWmbxaPWZzB4LnvvRV1S6/wf8qumu4Ctdvwuk6XPhrRPqwM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TigkfeWi; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27B48C43390; Wed, 31 Jan 2024 23:11:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742716; bh=PFXG2cNgodV6VNtmvaOYQJZ4ukS7LvHmLJbcHg8But0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TigkfeWi5OciTxd6vouIcuP3VbvngwUHh9OxOQE1NDfq+rhevMECMtUo4UKMwBCJo GabMt7pLZ6CRDFCrtnm8oCa/FNRmzMKn6uf+cf8l+4+fkpKbGK2Jan6HQJSBmr1/y3 /1Wq5ERemJpEmds2wUCAHPmnkNegrKO02UEhPG0Ax4UhcCPgVdDIL81Iz9LeRYez4f 694uCydc7xTLuQ+4GeNRmRc/vl9VvPKzx/5kybi5m3ANqzmm69WNFp9cKhyEzNMWmx gG4G3jgpCjDHi6GNZxlfy6DzLMQ/HgiOyiXLk9oKYzFGZNkXHKTzVjlrqBB8nnYU5+ 65PeHnOjdwoxQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 13/15] tick: Split nohz and highres features from nohz_mode Date: Thu, 1 Feb 2024 00:11:18 +0100 Message-ID: <20240131231120.12006-14-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650715705764942 X-GMAIL-MSGID: 1789650715705764942 The nohz mode field tells about low resolution nohz mode or high resolution nohz mode but it doesn't tell about high resolution non-nohz mode. In order to retrieve the latter state, tick_cancel_sched_timer() must fiddle with struct hrtimer's internals to guess if the tick has been initialized in high resolution. Move instead the nohz mode field information into the tick flags and provide two new bits: one to know if the tick is in nohz mode and another one to know if the tick is in high resolution. The combination of those two flags provides all the needed informations to determine which of the three tick modes is running. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/hrtimer.c | 2 +- kernel/time/tick-sched.c | 32 +++++++++++++++++--------------- kernel/time/tick-sched.h | 13 +++++-------- kernel/time/timer_list.c | 5 +++-- 4 files changed, 26 insertions(+), 26 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 3b456ec7d4fb..d8ba7985fe0d 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -746,7 +746,7 @@ static void hrtimer_switch_to_hres(void) base->hres_active = 1; hrtimer_resolution = HIGH_RES_NSEC; - tick_setup_sched_timer(NOHZ_MODE_HIGHRES); + tick_setup_sched_timer(true); /* "Retrigger" the interrupt to get things going */ retrigger_next_event(NULL); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 3e0a53c1b8bd..67759e7e025a 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -831,7 +831,7 @@ static void tick_nohz_restart(struct tick_sched *ts, ktime_t now) /* Forward the time to expire in the future */ hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) { + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) { hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); } else { @@ -997,14 +997,14 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * the tick timer. */ if (unlikely(expires == KTIME_MAX)) { - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) hrtimer_cancel(&ts->sched_timer); else tick_program_event(KTIME_MAX, 1); return; } - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) { + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) { hrtimer_start(&ts->sched_timer, expires, HRTIMER_MODE_ABS_PINNED_HARD); } else { @@ -1065,7 +1065,7 @@ static void tick_nohz_full_update_tick(struct tick_sched *ts) if (!tick_nohz_full_cpu(smp_processor_id())) return; - if (ts->nohz_mode == NOHZ_MODE_INACTIVE) + if (!tick_sched_flag_test(ts, TS_FLAG_NOHZ)) return; __tick_nohz_full_update_tick(ts, ktime_get()); @@ -1125,7 +1125,7 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) return false; } - if (unlikely(ts->nohz_mode == NOHZ_MODE_INACTIVE)) + if (unlikely(!tick_sched_flag_test(ts, TS_FLAG_NOHZ))) return false; if (need_resched()) @@ -1449,11 +1449,11 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev) tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); } -static inline void tick_nohz_activate(struct tick_sched *ts, int mode) +static inline void tick_nohz_activate(struct tick_sched *ts) { if (!tick_nohz_enabled) return; - ts->nohz_mode = mode; + tick_sched_flag_set(ts, TS_FLAG_NOHZ); /* One update is enough */ if (!test_and_set_bit(0, &tick_nohz_active)) timers_update_nohz(); @@ -1474,7 +1474,7 @@ static void tick_nohz_switch_to_nohz(void) * Recycle the hrtimer in 'ts', so we can share the * highres code. */ - tick_setup_sched_timer(NOHZ_MODE_LOWRES); + tick_setup_sched_timer(false); } static inline void tick_nohz_irq_enter(void) @@ -1502,7 +1502,7 @@ static inline void tick_nohz_irq_enter(void) static inline void tick_nohz_switch_to_nohz(void) { } static inline void tick_nohz_irq_enter(void) { } -static inline void tick_nohz_activate(struct tick_sched *ts, int mode) { } +static inline void tick_nohz_activate(struct tick_sched *ts) { } #endif /* CONFIG_NO_HZ_COMMON */ @@ -1529,15 +1529,17 @@ early_param("skew_tick", skew_tick); * tick_setup_sched_timer - setup the tick emulation timer * @mode: tick_nohz_mode to setup for */ -void tick_setup_sched_timer(int mode) +void tick_setup_sched_timer(bool hrtimer) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); /* Emulate tick processing via per-CPU hrtimers: */ hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && hrtimer) { + tick_sched_flag_set(ts, TS_FLAG_HIGHRES); ts->sched_timer.function = tick_nohz_handler; + } /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); @@ -1551,11 +1553,11 @@ void tick_setup_sched_timer(int mode) } hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && mode == NOHZ_MODE_HIGHRES) + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && hrtimer) hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); else tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - tick_nohz_activate(ts, mode); + tick_nohz_activate(ts); } void tick_cancel_sched_timer(int cpu) @@ -1564,7 +1566,7 @@ void tick_cancel_sched_timer(int cpu) ktime_t idle_sleeptime, iowait_sleeptime; unsigned long idle_calls, idle_sleeps; - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && ts->sched_timer.base) + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) hrtimer_cancel(&ts->sched_timer); idle_sleeptime = ts->idle_sleeptime; @@ -1614,7 +1616,7 @@ int tick_check_oneshot_change(int allow_nohz) if (!test_and_clear_bit(0, &ts->check_clocks)) return 0; - if (ts->nohz_mode != NOHZ_MODE_INACTIVE) + if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) return 0; if (!timekeeping_valid_for_hres() || !tick_is_oneshot_available()) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 07a4c0144c47..bbe72a078985 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -14,12 +14,6 @@ struct tick_device { enum tick_device_mode mode; }; -enum tick_nohz_mode { - NOHZ_MODE_INACTIVE, - NOHZ_MODE_LOWRES, - NOHZ_MODE_HIGHRES, -}; - /* The CPU is in the tick idle mode */ #define TS_FLAG_INIDLE BIT(0) /* The idle tick has been stopped */ @@ -31,6 +25,10 @@ enum tick_nohz_mode { #define TS_FLAG_IDLE_ACTIVE BIT(2) /* CPU was the last one doing do_timer before going idle */ #define TS_FLAG_DO_TIMER_LAST BIT(3) +/* NO_HZ is enabled */ +#define TS_FLAG_NOHZ BIT(4) +/* High resolution tick mode */ +#define TS_FLAG_HIGHRES BIT(5) /** * struct tick_sched - sched tick emulation and no idle tick control/stats @@ -84,7 +82,6 @@ struct tick_sched { ktime_t idle_entrytime; /* Tick stop */ - enum tick_nohz_mode nohz_mode; unsigned long last_jiffies; u64 timer_expires_base; u64 timer_expires; @@ -107,7 +104,7 @@ struct tick_sched { extern struct tick_sched *tick_get_tick_sched(int cpu); -extern void tick_setup_sched_timer(int mode); +extern void tick_setup_sched_timer(bool hrtimer); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS extern void tick_cancel_sched_timer(int cpu); #else diff --git a/kernel/time/timer_list.c b/kernel/time/timer_list.c index 38f81d836fc5..1c311c46da50 100644 --- a/kernel/time/timer_list.c +++ b/kernel/time/timer_list.c @@ -152,7 +152,8 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now) { struct tick_sched *ts = tick_get_tick_sched(cpu); - P(nohz_mode); + P_flag(nohz, TS_FLAG_NOHZ); + P_flag(highres, TS_FLAG_HIGHRES); P_ns(last_tick); P_flag(tick_stopped, TS_FLAG_STOPPED); P(idle_jiffies); @@ -259,7 +260,7 @@ static void timer_list_show_tickdevices_header(struct seq_file *m) static inline void timer_list_header(struct seq_file *m, u64 now) { - SEQ_printf(m, "Timer List Version: v0.9\n"); + SEQ_printf(m, "Timer List Version: v0.10\n"); SEQ_printf(m, "HRTIMER_MAX_CLOCK_BASES: %d\n", HRTIMER_MAX_CLOCK_BASES); SEQ_printf(m, "now at %Ld nsecs\n", (unsigned long long)now); SEQ_printf(m, "\n"); From patchwork Wed Jan 31 23:11:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195053 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp90698dyc; Wed, 31 Jan 2024 15:31:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IEGTWT+iv4NJy+Yhh4Nl/prN6oWygXHo/9Da8d6C50LN56rpPDzJRI2KMLQHJOGzvryDMuC X-Received: by 2002:ac8:59c5:0:b0:42a:7f80:92fd with SMTP id f5-20020ac859c5000000b0042a7f8092fdmr3508102qtf.29.1706743902434; Wed, 31 Jan 2024 15:31:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706743902; cv=pass; d=google.com; s=arc-20160816; b=y+BK0IUz7T1jf6NLKUR1maR45p+kxBbww4oOBwvt8TxfS3vsKAO6mMbsUcXmTQ7VBf Ps3SX7ZwiY5wlEfugZrc70kGEQcAndz+pzsCWMT5zniBCWDebiU4yeUAn3+/NckUAPPG SDbt1z0dPFaKtVgBC4OBqFSOYP0Ol2D4I/ft2w98PI302mM8ZmkPtgIMhK52wp9BAfjq IB0qggkJsTb/zrtVeU14rXg+g8qbwgnDWdE1JFBi0ecit+DkJVwS6pOYZ0tdDFgvEUG4 xqbNwDPLhby8GYuYMLVwWurUAWGW3sTv8y5nTA3BtAenrH4X3TmJGkK6mt/hWUhaA+bU /rjQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=S5L/TQzvt7PsP+GgbH/DQZ+Z2T1bsG7taL7oQ7CgSZs=; fh=O3ZUtKJ6KziDRV4ZUNe1vwtTSomVa9u/dw4yioNA6uM=; b=udy15M7E6stRL+4/YNAC1FcpLRZnlYbgNrg2qLkv2uwZ2dJJR4l7Pd8D8GyHik0ubW 5oL4k7alm1n2HssRyWjHCRbxIfkmYS91nooGamX8ThcKoDoQbZFzHN33RlFRLDYnI6yZ daVjOWbmKAzsZAn1YFaPvww49EsyCi05iBtdZi5KPuTtPe2rFL7b37pRCIPy80O+wRUr TSoDj/0fA88xN73ATNigJm6XKo7ydrQsT15QdEsrLeHdspOSN0vWEmnwC3cYhjHmra33 F26lzJObCrfWCG5d6eZbmlxXfItneIfG6vlozW3QMk0Isz90I+As/NyfyijkAJeFbe3G ye2Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=hjdHutIP; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47358-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47358-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCUrVnLYG8vkBUeRL5dKxMfUGyLUqD5i1xk3RuDS5yabH1bzNbwIjc2KxrTna8lYwfMNXoUOKYKROPpEYcPGSNZbblxYBg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id z9-20020a05622a060900b004299b7236b8si12963687qta.632.2024.01.31.15.31.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:31:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47358-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=hjdHutIP; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47358-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47358-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3B5FE1C265E4 for ; Wed, 31 Jan 2024 23:31:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 816F613AA4E; Wed, 31 Jan 2024 23:12:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hjdHutIP" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8090E12BF13 for ; Wed, 31 Jan 2024 23:11:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742718; cv=none; b=BZgXryhWIkkWSkNJ9S8jjRwGdx5VJHTyQYptlslnBtXqQhrOxO16M6GH83oWBQdHgTCVFjbmpqaQYxk7l0nTFqx1I2hl02Dv5uWZYxfPKEQPH6bmxhQkvywEUTBdSuYRE5FZt4fAS1HlPSD3LcU/aZYO0+cUYmyfQoYxB9gVHsM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742718; c=relaxed/simple; bh=FIezuNO5LWygz4x2wl/CWc53JKa0Be31X2mAyqu+B2I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HfoRPrrvIrJXHBy3deBYBUcic167TcOMibyQLWMabX/AcAhRtr6mO8h4+8YY9Fp6idIgAxmnfn+2TymOJMpYINyugYmHT5ws8gcZm8SSE3O3ATrKDtd1aYLVYJCVBhECcgjGGfZbFVT+ny6yMFbBCf01arneUcQzJm6RZI+UpOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hjdHutIP; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93D9CC433F1; Wed, 31 Jan 2024 23:11:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742718; bh=FIezuNO5LWygz4x2wl/CWc53JKa0Be31X2mAyqu+B2I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hjdHutIPw6SyPone9MGAdgDQBfK/rFP6rFlD0pFPZLH5dIDU9hsxGP4/HuuepWr1j k9avffTxMdMvlMasC2NF+hmKKjbkp8Ahjm3zLClIzzafqM0hp0kUSTz+4CN4RF7o2X 9hEjIHM08c2XHKe4euz2uC0NsuDl3oDnGOXJH/esUiEMzeMBcqyrKRzWNbl+RBT6ej zSIL19C2Kb0nYkC4FVHlOEiJ9t0bLTK8lLWJViUDMskM69fRWYwOhUTNR4/qb8lvrq bcXp2TAzUB2hniEs2xw/TDm/z+H+hpVmNH6faW0WY0r3/VdZU6wfA6MP8VqpJcahkV lTZYF/l4bVv+Q== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 14/15] tick: Shut down low-res tick from dying CPU Date: Thu, 1 Feb 2024 00:11:19 +0100 Message-ID: <20240131231120.12006-15-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789650694273157887 X-GMAIL-MSGID: 1789650694273157887 The timekeeping duty is handed over from the outgoing CPU within stop machine. This works well if CONFIG_NO_HZ_COMMON=n or the tick is in high-res mode. However in low-res dynticks mode, the tick isn't cancelled until the clockevent is shut down, which can happen later. The tick may therefore fire again once IRQs are re-enabled on stop machine and until IRQs are disabled for good upon the last call to idle. That's so many opportunities for a timekeeper to go idle and the outgoing CPU to take over that duty. This is why tick_nohz_idle_stop_tick() is called one last time on idle if the CPU is seen offline: so that the timekeeping duty is handed over again in case the CPU has re-taken the duty. This means there are two timekeeping handovers on CPU down hotplug with different undocumented constraints and purposes: 1) A handover on stop machine for !dynticks || highres. All online CPUs are guaranteed to be non-idle and the timekeeping duty can be safely handed-over. The hrtimer tick is cancelled so it is guaranteed that in dynticks mode the outgoing CPU won't take again the duty. 2) A handover on last idle call for dynticks && lowres. Setting the duty to TICK_DO_TIMER_NONE makes sure that a CPU will take over the timekeeping. Prepare for consolidating the handover to a single place (the first one) with shutting down the low-res tick as well from tick_cancel_sched_timer() as well. This will simplify the handover and unify the tick cancellation between high-res and low-res. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- kernel/time/tick-common.c | 3 ++- kernel/time/tick-sched.c | 32 +++++++++++++++++++++++++------- kernel/time/tick-sched.h | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 522414089c0d..9cd09eea06d6 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -410,7 +410,8 @@ int tick_cpu_dying(unsigned int dying_cpu) if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); - tick_cancel_sched_timer(dying_cpu); + /* Make sure the CPU won't try to retake the timekeeping duty */ + tick_sched_timer_dying(dying_cpu); /* Remove CPU from timer broadcasting */ tick_offline_cpu(dying_cpu); diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 67759e7e025a..cb8e4a171288 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -308,6 +308,14 @@ static enum hrtimer_restart tick_nohz_handler(struct hrtimer *timer) return HRTIMER_RESTART; } +static void tick_sched_timer_cancel(struct tick_sched *ts) +{ + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) + hrtimer_cancel(&ts->sched_timer); + else if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) + tick_program_event(KTIME_MAX, 1); +} + #ifdef CONFIG_NO_HZ_FULL cpumask_var_t tick_nohz_full_mask; EXPORT_SYMBOL_GPL(tick_nohz_full_mask); @@ -997,10 +1005,7 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * the tick timer. */ if (unlikely(expires == KTIME_MAX)) { - if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) - hrtimer_cancel(&ts->sched_timer); - else - tick_program_event(KTIME_MAX, 1); + tick_sched_timer_cancel(ts); return; } @@ -1560,14 +1565,27 @@ void tick_setup_sched_timer(bool hrtimer) tick_nohz_activate(ts); } -void tick_cancel_sched_timer(int cpu) +/* + * Shut down the tick and make sure the CPU won't try to retake the timekeeping + * duty before disabling IRQs in idle for the last time. + */ +void tick_sched_timer_dying(int cpu) { + struct tick_device *td = &per_cpu(tick_cpu_device, cpu); struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); + struct clock_event_device *dev = td->evtdev; ktime_t idle_sleeptime, iowait_sleeptime; unsigned long idle_calls, idle_sleeps; - if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) - hrtimer_cancel(&ts->sched_timer); + /* This must happen before hrtimers are migrated! */ + tick_sched_timer_cancel(ts); + + /* + * If the clockevents doesn't support CLOCK_EVT_STATE_ONESHOT_STOPPED, + * make sure not to call low-res tick handler. + */ + if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) + dev->event_handler = clockevents_handle_noop; idle_sleeptime = ts->idle_sleeptime; iowait_sleeptime = ts->iowait_sleeptime; diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index bbe72a078985..58d8d1c49dd3 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -106,9 +106,9 @@ extern struct tick_sched *tick_get_tick_sched(int cpu); extern void tick_setup_sched_timer(bool hrtimer); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS -extern void tick_cancel_sched_timer(int cpu); +extern void tick_sched_timer_dying(int cpu); #else -static inline void tick_cancel_sched_timer(int cpu) { } +static inline void tick_sched_timer_dying(int cpu) { } #endif #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST From patchwork Wed Jan 31 23:11:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 195064 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:106:209c:c626 with SMTP id mn5csp93446dyc; Wed, 31 Jan 2024 15:38:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IGZUJgwAL3CU5ixL7yZbVs/Giaysg7rKR56J058RFzlHSbO7R109JLF38EtDeimiOthrfnU X-Received: by 2002:a05:6a00:4b85:b0:6dd:8522:5005 with SMTP id ks5-20020a056a004b8500b006dd85225005mr3083717pfb.2.1706744297349; Wed, 31 Jan 2024 15:38:17 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706744297; cv=pass; d=google.com; s=arc-20160816; b=bTjg14pCJbKtwJiLfcHlB2cmbMe+UjZn1NEr3Q9QeTIV3JKOomJktF5AvM7vig06qh Swh47T5GZ/boYf3gAIb05UCvzBzBwjbTEgc5o3eRzglz+EEe5gqEcyFr+YGBFAuDswss XQpyvH+33ZX8tuOe7pMJx7M2V8c1PKNt/5ZNWn70TM0M4+GJcmyJKqgu1+WNhpjdzIOu a965MJB0ReCEd3FUHRv64D/xKsu40Ed8MOvfMFes+hduAO4xtbwifx2He754O9LgGWeR ZcV6Bhb0fWYASWyoSbQPPVWfc+J7jX+tPP5nVtqibwM/OiM1wfUf56WyUyP7OIUvmT4G UM7w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=nxSR2BczxCCa3uv0qTI3Je1VkKszE9x/yANuRI8FHgw=; fh=UmI+/0OfOtbFDBBi50ZQgo5qM3CiQ3UIzeKIp9YHdJY=; b=gL75sWalJ/ZYswX4HWCCKjCTxjNspKw7qBdSzSP+4pVZ8BTPGZVFcbQ+b2PLShzUnx wqlfIPR3L7Z9YXgYIr52iSOOOmYu5CHXW5pV+OD6RRdbZUPtO0/t+wAmRn+BLi3GzeVX xxpcrviItlbQA3qcvihs2ONe8YNPgu1eaDef6ejhhyGKzvbNJ5UWxD6l4NmH6kiGC+28 gFQCdAXeeRNvdvC1jlnfYmbrgEcXYlfgWbvM87Jos3j8VJBbLWrD1zqylvYFksIxTYK+ C7tKgG00DLLwVklyv5ppS5quEwjGsuKyzoBN8dysw2pHiHqOfBzRNzxjd7qLvnSQEPPo KFyA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fBTfiSUM; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47359-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47359-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Forwarded-Encrypted: i=1; AJvYcCVpPy/8VSU1e6ODaQNXXnhwJki7MsYcVvGG8MqIwdWMNGu0k+QohIjd1uP3K+Q//phwDRIYngRxrCDN7VmyMmujjdduQw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id h21-20020a056a00219500b006dd82ed2c1csi10704999pfi.84.2024.01.31.15.38.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 15:38:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47359-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fBTfiSUM; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-47359-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47359-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id A55F929248F for ; Wed, 31 Jan 2024 23:31:47 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6D67F13B7A5; Wed, 31 Jan 2024 23:12:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fBTfiSUM" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B2C7813A25F for ; Wed, 31 Jan 2024 23:12:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742720; cv=none; b=dOyAFKmDHTTQP4d7idGOZHesCfPbC6WHWS9Whle1KOJki9Y7DNCz8DAFuNl+TuosqdXn+aqiA+IjOnVXPvL7/FVTR9NpGCh8WvkcU+af4hvVxARzeKQo8Qf9bEktJFSctlC1Gt6GuQErzJxj6ud45BfNRwCBCG+cfula19cDe/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706742720; c=relaxed/simple; bh=u2P1r5FEYqsMg24uO7pS2JZU0ctu1FvUTqkt+nPckno=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h10O9MJ5Wzr7T+yIIapIIviC5zJRxo32EZ1hx5y1Ih1Zmm1sGWAzuJY2UMeHFRWpR4FAnN7lLgW9EbQiqbez6jVVZ8CG0mzaBFx+FK+JsWN7ASfuXhpeU2F4e+PLG3yTez/qT+zUtdKbRiEdtrl/V8uPrkLW40iDcO+rF7Tqii4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fBTfiSUM; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C88DAC433A6; Wed, 31 Jan 2024 23:11:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706742720; bh=u2P1r5FEYqsMg24uO7pS2JZU0ctu1FvUTqkt+nPckno=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fBTfiSUMVYOYZ/98kxN4SU48/pwvj2neLluIuuDRY7aLiJ2KIiovzLWi6+aJ5bhJc 95KL2r+0LZw8e914CEY7qxy9oXioveIxzbRlVGMX5Npq0yaQJ3DjKUJrnAxaIGq2M1 bmUkzXdxcRG1aFHUWvTwl1BJlBN160jRIJfqyPCyVVMAsOLmJsYKqbIAW98K7vLxnO zoj8micYhI9ER/BeI4IqxO3mPADK0yP8XYXsoCp7ncMmtvXLMfisdi23KLewQpi85g Lzs/+JkfrJBoKqhr6yg5GniHWyupM5aI0Fb5zdzGoNBV7+eztPu8GVuY8WfGd9CLR6 MSbEsjA1ahb0w== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 15/15] tick: Assume timekeeping is correctly handed over upon last offline idle call Date: Thu, 1 Feb 2024 00:11:20 +0100 Message-ID: <20240131231120.12006-16-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240131231120.12006-1-frederic@kernel.org> References: <20240131231120.12006-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789651108215428662 X-GMAIL-MSGID: 1789651108215428662 The timekeeping duty is handed over from the outgoing CPU on stop machine, then the oneshot tick is stopped right after. Therefore it's guaranteed that the current CPU isn't the timekeeper upon its last call to idle. Besides, calling tick_nohz_idle_stop_tick() while the dying CPU goes into idle suggests that the tick is going to be stopped while it is actually stopped already from the appropriate CPU hotplug state. Remove the confusing call and the obsolete case handling and convert it to a sanity check that verifies the above assumption. Reviewed-by: Thomas Gleixner Signed-off-by: Frederic Weisbecker --- include/linux/tick.h | 2 ++ kernel/cpu.c | 1 + kernel/sched/idle.c | 1 - kernel/time/tick-common.c | 4 ++++ kernel/time/tick-sched.c | 13 +------------ 5 files changed, 8 insertions(+), 13 deletions(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index c7840ae8ebaf..44fddfa93e18 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -29,8 +29,10 @@ static inline void tick_cleanup_dead_cpu(int cpu) { } #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_HOTPLUG_CPU) extern int tick_cpu_dying(unsigned int cpu); +extern void tick_assert_timekeeping_handover(void); #else #define tick_cpu_dying NULL +static inline void tick_assert_timekeeping_handover(void) { } #endif #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_SUSPEND) diff --git a/kernel/cpu.c b/kernel/cpu.c index 5a8ad4f5ccf3..7e84a7b0675e 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1399,6 +1399,7 @@ void cpuhp_report_idle_dead(void) struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); BUG_ON(st->state != CPUHP_AP_OFFLINE); + tick_assert_timekeeping_handover(); rcutree_report_cpu_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 31231925f1ec..b15d40cad7ea 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -291,7 +291,6 @@ static void do_idle(void) local_irq_disable(); if (cpu_is_offline(cpu)) { - tick_nohz_idle_stop_tick(); cpuhp_report_idle_dead(); arch_cpu_idle_dead(); } diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 9cd09eea06d6..fb0fdec8719a 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -396,6 +396,10 @@ int tick_broadcast_oneshot_control(enum tick_broadcast_state state) EXPORT_SYMBOL_GPL(tick_broadcast_oneshot_control); #ifdef CONFIG_HOTPLUG_CPU +void tick_assert_timekeeping_handover(void) +{ + WARN_ON_ONCE(tick_do_timer_cpu == smp_processor_id()); +} /* * Stop the tick and transfer the timekeeping job away from a dying cpu. */ diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index cb8e4a171288..0dcd1c0e0a4e 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1117,18 +1117,7 @@ static bool report_idle_softirq(void) static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) { - /* - * If this CPU is offline and it is the one which updates - * jiffies, then give up the assignment and let it be taken by - * the CPU which runs the tick timer next. If we don't drop - * this here, the jiffies might be stale and do_timer() never - * gets invoked. - */ - if (unlikely(!cpu_online(cpu))) { - if (cpu == tick_do_timer_cpu) - tick_do_timer_cpu = TICK_DO_TIMER_NONE; - return false; - } + WARN_ON_ONCE(cpu_is_offline(cpu)); if (unlikely(!tick_sched_flag_test(ts, TS_FLAG_NOHZ))) return false;