From patchwork Wed Jan 24 17:04:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191660 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1125735dyi; Wed, 24 Jan 2024 09:05:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IGaIpWQHSfdVLG8zndXo8A/IZ6JW9SikspcmmbvzEjjYi/ye7xMTT86085BrWy9qULFwmh1 X-Received: by 2002:a17:907:c287:b0:a30:6498:3e2 with SMTP id tk7-20020a170907c28700b00a30649803e2mr1196350ejc.125.1706115952461; Wed, 24 Jan 2024 09:05:52 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706115952; cv=pass; d=google.com; s=arc-20160816; b=Fbv/AjUUSKB+CXeXjbXvYZFfWNUpf879UmIw62vvNx4+QG67WOyvKSdmTmb7yUHSfT IV2SRh0ETk/sHf0ejc/BEy4bXsgk8SBx/cSoxXAe7TG4QG3By+lY0kD62tQURmdzZbtP aPLVA7tyd1gjh71lluG+RSfZQgrCaL/eAiiM+IKNypz58WI3B0Kq+pDsAmzoEcfwbzM1 7a2vhfjCE2cLFvurLCFf5shNqe+T1xPoJOKCz9TChDIbQDloCeWwVsEhyT+L+5Melk1R aP0BrwGKkrL8LPGWa5S6YvA1I8Bvfmd+n6ua+ilTDvm+FApxrNsgaLLew5hiOi4qRE/J VGqQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=JJ1SgK/gl45ZvPSddGsJDu7EvaMBUYysGPZGRNkaLCo=; fh=4P8gqRLFmANjcHHSJu0T2ajtncG7F3hkmyD1v1G3zYE=; b=CB/xQN1wMp+BLsXBqidMsSawMkIqcJHrtGrcGL3tAPZSptsglhCokOtvWenqN1ox6U YTx4uBgYq2se8uLt1npNTbfC6kuPxAdfS4a85nMrOHJFUA/GNGNXXcx/ZOyP8mcDUGrV 0mcxq5jq4eUWAoiBV/OnkpjEeLGJjqwWsPfS45BwbfhF/v14vwP2LtukK8deEIw7GI/t IifH/z7oBG3YVQSXxNoqrhWTFKvXhpKOpVeHbU5S1qkg3UfLkJAAAUbihBAeOAnEZo3x 1mTgEFnlOsIBPgmSmLPPKWkfr08utiAwsfcGTnuyOkwPtELk+nvje6dTh9/jI+w5S0St CxVQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TQfsInHm; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37380-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37380-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id u17-20020a1709064ad100b00a315b1f4168si7834ejt.1038.2024.01.24.09.05.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:05:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37380-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=TQfsInHm; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37380-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37380-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 11EA41F2273C for ; Wed, 24 Jan 2024 17:05:52 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4756C81AD6; Wed, 24 Jan 2024 17:05:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TQfsInHm" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7714F80058 for ; Wed, 24 Jan 2024 17:05:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115912; cv=none; b=sq6GvXmd6rj8zwJn142eYKOJr05cM3XxCdkoISQT2COC0kad5TS/jEylTsHFcDLvNzuKzNQIzmKQAJO5Z1ZRr8mF9oy3yQH2ycSmsqPDQ59IYl6ekAvtZ/YyfsUnBYxmlycek0j/4eqqcZ3l8l5wKzcws1neUloj1HGjr/NNThM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115912; c=relaxed/simple; bh=yHPR6j3OL36RltRUGFvaGlb/MMYM5ilnHWsVaQEKuT4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NcYvRfNQJirxiwr1aQKCCNn6zfHo78+kZ+BpuMORA6CYAKyemi3dxvrjJJ3JiQ7dtrk2mhT8PUlHPMKut3a/S22lxtFs/2KNZPwDafzKS9GDI0rAqmltLCWEpHsoxVkwES0xGXLaePujzwKjLutTrwFVVRw90Bz8ZzAfBaFLhcc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TQfsInHm; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C6EABC433F1; Wed, 24 Jan 2024 17:05:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115912; bh=yHPR6j3OL36RltRUGFvaGlb/MMYM5ilnHWsVaQEKuT4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TQfsInHmRq9DSrjfNMLhsPDqw+p77hD7gmMndejyr2j4ovFyXStTsUR4BY+7zGZNW 6TjJ3foqNPdU+BM3L4D3ZjbXEmn3i02koEdCeCtpGjj9cGDNPvlBMaKtnoeju5/eIp 8OtrIh2aLovc5ot5goCSePo/QKJWwRwepBPwoZNQV6NH1MqsPIo96hy7Hr+S2S3Uet G83GVehCyNDctDdTNBjm2XxQjRYwCLUtwThdxRjjgM4gldwz8t2JVNxW3IPnWS9gbk oQvjMkqxUetd/8h6SZ4U7jd3wT0DXCTeSag24Nxustpi7eStWk3HdWSCuiLS2dR4pZ znaL1S3Gth1hQ== From: Frederic Weisbecker To: LKML Cc: Peng Liu , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Joel Fernandes , Frederic Weisbecker Subject: [PATCH 01/15] tick/nohz: Remove duplicate between tick_nohz_switch_to_nohz() and tick_setup_sched_timer() Date: Wed, 24 Jan 2024 18:04:45 +0100 Message-ID: <20240124170459.24850-2-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992240646438917 X-GMAIL-MSGID: 1788992240646438917 From: Peng Liu The ts->sched_timer initialization work of tick_nohz_switch_to_nohz() is almost the same as that of tick_setup_sched_timer(), so adjust the latter to get it reused by tick_nohz_switch_to_nohz(). This also makes low-res mode sched_timer benefit from the tick skew boot option. Signed-off-by: Peng Liu Signed-off-by: Frederic Weisbecker --- kernel/time/hrtimer.c | 2 +- kernel/time/tick-sched.c | 38 +++++++++++++++++--------------------- kernel/time/tick-sched.h | 2 +- 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 760793998cdd..355b5a957f7f 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -746,7 +746,7 @@ static void hrtimer_switch_to_hres(void) base->hres_active = 1; hrtimer_resolution = HIGH_RES_NSEC; - tick_setup_sched_timer(); + tick_setup_sched_timer(NOHZ_MODE_HIGHRES); /* "Retrigger" the interrupt to get things going */ retrigger_next_event(NULL); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index d2501673028d..e90dbb7ae70a 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1433,9 +1433,6 @@ static inline void tick_nohz_activate(struct tick_sched *ts, int mode) */ static void tick_nohz_switch_to_nohz(void) { - struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - ktime_t next; - if (!tick_nohz_enabled) return; @@ -1444,16 +1441,9 @@ static void tick_nohz_switch_to_nohz(void) /* * Recycle the hrtimer in 'ts', so we can share the - * hrtimer_forward_now() function with the highres code. + * highres code. */ - hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - /* Get the next period */ - next = tick_init_jiffy_update(); - - hrtimer_set_expires(&ts->sched_timer, next); - hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); - tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - tick_nohz_activate(ts, NOHZ_MODE_LOWRES); + tick_setup_sched_timer(NOHZ_MODE_LOWRES); } static inline void tick_nohz_irq_enter(void) @@ -1532,7 +1522,9 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) return HRTIMER_RESTART; } +#endif /* HIGH_RES_TIMERS */ +#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS static int sched_skew_tick; static int __init skew_tick(char *str) @@ -1545,15 +1537,18 @@ early_param("skew_tick", skew_tick); /** * tick_setup_sched_timer - setup the tick emulation timer + * @mode: tick_nohz_mode to setup for */ -void tick_setup_sched_timer(void) +void tick_setup_sched_timer(int mode) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - ktime_t now = ktime_get(); /* Emulate tick processing via per-CPU hrtimers: */ hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - ts->sched_timer.function = tick_nohz_highres_handler; +#ifdef CONFIG_HIGH_RES_TIMERS + if (mode == NOHZ_MODE_HIGHRES) + ts->sched_timer.function = tick_nohz_highres_handler; +#endif /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); @@ -1566,13 +1561,14 @@ void tick_setup_sched_timer(void) hrtimer_add_expires_ns(&ts->sched_timer, offset); } - hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); - hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); - tick_nohz_activate(ts, NOHZ_MODE_HIGHRES); + hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); + if (mode == NOHZ_MODE_HIGHRES) + hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); + else + tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); + tick_nohz_activate(ts, mode); } -#endif /* HIGH_RES_TIMERS */ -#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS void tick_cancel_sched_timer(int cpu) { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); @@ -1589,7 +1585,7 @@ void tick_cancel_sched_timer(int cpu) ts->idle_sleeptime = idle_sleeptime; ts->iowait_sleeptime = iowait_sleeptime; } -#endif +#endif /* CONFIG_NO_HZ_COMMON || CONFIG_HIGH_RES_TIMERS */ /* * Async notification about clocksource changes diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 5ed5a9d41d5a..35808bbb8a47 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -102,7 +102,7 @@ struct tick_sched { extern struct tick_sched *tick_get_tick_sched(int cpu); -extern void tick_setup_sched_timer(void); +extern void tick_setup_sched_timer(int mode); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS extern void tick_cancel_sched_timer(int cpu); #else From patchwork Wed Jan 24 17:04:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191661 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1126373dyi; Wed, 24 Jan 2024 09:06:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IGNmXbMj2Ft4DMUCiCoeDn3TDDE42QZIsgxFcXxCRsn7fvPm6wZ/YK+G3fA4JZBnVpcdLQy X-Received: by 2002:a05:6830:448b:b0:6e0:c044:232c with SMTP id r11-20020a056830448b00b006e0c044232cmr2457192otv.6.1706116001455; Wed, 24 Jan 2024 09:06:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116001; cv=pass; d=google.com; s=arc-20160816; b=eoHl9nfoFGFrg/ErYq2FHEYBhZEdmbwYl1TIeL5213Mx0ItyneqUGcGzJLh5NUcX6n 0NYWHk4yp5NhiOTtIeNSKXywHrc/0i3JvZV19HEXCZyilsCN0FqYoAbkPElqArbEDv4e WisnBz4zW0sRZwvMKOycqYCwyYfoT1RYW5Jmf8a6vmSYf54s7CDUoMNpM8zg1Gok8B9A xuWoP52WPTCEaPaRMFIIGHqSFS1wvtnloREiMa7Y63/WIRC7g98yTQ853W+fzNqYyqDS UcYA33JXSU/Oqpq9oV1C7MDFq++HlrE+unqkqF6xV2HwuUIpP+Azs6d+dINbguOOHtkn XdCg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=T4oFFxrGJjvJMdipGRPgFDEw8qOVkPwwk5ZnkPIzNBU=; fh=4P8gqRLFmANjcHHSJu0T2ajtncG7F3hkmyD1v1G3zYE=; b=I8cHxaIYaM1SFX9oF8ZoHGfYScd+yV6rL3L4sLhRsxSDOLt4fxWDIDIZyupCZVQaFX bX1qGB6At3MwBAEkLOGTLV+IGjc3jWAEk7Y6AKysN1opk9MLskkDzo+4jqaj4RFXGXsz kpD3fhyUwmJ+oafG3YrGPRbi8RVz38KRO0AmZWnAJyvhvt6PCxwjg8fcguSUGZEMfUcD wohhfs4XZ32EZ2r6ln5rnfuVZaaUsvEOAvaJNoPUzZ4wwSinIx7lAmbG33R2imZhZ8uk wM+lNNiFOJxNew3H1qpVMqcgpHYa1hf5j7nDQrpBC4HM259hih3LHazbdFJypqbxZo1A yZAg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Xo3Ks91e; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37381-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37381-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id e24-20020a056102225800b0046b025c4ef7si377252vsb.258.2024.01.24.09.06.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:06:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37381-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Xo3Ks91e; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37381-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37381-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 713501C23A19 for ; Wed, 24 Jan 2024 17:06:05 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 20D8A82D76; Wed, 24 Jan 2024 17:05:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Xo3Ks91e" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5738D81AD7 for ; Wed, 24 Jan 2024 17:05:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115915; cv=none; b=nkrKTWWXI54OZOH5poe2CxavT6tW90QW6iMzIvrtJydmOK7kpSFbEgXkxQukD6/UazkJ2kVpj08Bbv7E9f8nQaG55rUUx2Qp0RaFjVCSVUbZnukiXotXTW2k1Msk814y521+bTw8+vm3o5uFKFG6srfg62e+7m6m7uWb33yJJcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115915; c=relaxed/simple; bh=JNABFf9l7peDqU/Keofwd5fi4DbQISwry21UShiw/q4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MXPJ0cUVsDu3rGhQ8gfm85XXe4VoWdNOlOc/0gYOQECguK9Q+IF+1ShT5h56hPXtECMjNzy1F4R/pDtGV51+5+kHGIreeezvIs5IepVhlk/cMGrcVszK8e6uOzPAlD3zyzapFBB6008goEfbUwN1g19Boj80DaCj+l/07m8qhBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Xo3Ks91e; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA85AC43394; Wed, 24 Jan 2024 17:05:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115914; bh=JNABFf9l7peDqU/Keofwd5fi4DbQISwry21UShiw/q4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xo3Ks91ecsh5kAWpzzWn/VCsbSUBy31Cy7IrMNMJ8qT4K4ItWhbuJHlVv9lp7S4ih SPxbLceATVw8ZhiElnQu9MvxclyW4/+BRhen36Z63F/IqaGKkG4zqZDEtKrvbZ/Y/i RkvCFOP5bF51av/xUGH7zAyTRS38bHy6pOE39CBoPSaSzwokrkcwGrfBo4LfjeMIj2 qlTcUeTwCRkUcHxdPURNW50i5cTlD1y58yHPQv/Q/BxlM0qG4ql9gnz2FD7J1NAUGL Oqmas3KSUEmuRhfewfJsuhluRYjR1adLecv8+QgJowfLfqVIJhvaq3WQCQvYKels1t T3YNgHklN1Dtg== From: Frederic Weisbecker To: LKML Cc: Peng Liu , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Joel Fernandes , Frederic Weisbecker Subject: [PATCH 02/15] tick/nohz: Remove duplicate between lowres and highres handlers Date: Wed, 24 Jan 2024 18:04:46 +0100 Message-ID: <20240124170459.24850-3-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992292480723835 X-GMAIL-MSGID: 1788992292480723835 From: Peng Liu tick_nohz_lowres_handler() does the same work as tick_nohz_highres_handler() plus the clockevent device reprogramming, so make the former reuse the latter. Signed-off-by: Peng Liu Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 96 ++++++++++++++++------------------------ 1 file changed, 38 insertions(+), 58 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index e90dbb7ae70a..6f69ae88e47e 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -255,6 +255,41 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) update_process_times(user_mode(regs)); profile_tick(CPU_PROFILING); } + +/* + * We rearm the timer until we get disabled by the idle code. + * Called with interrupts disabled. + */ +static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) +{ + struct tick_sched *ts = + container_of(timer, struct tick_sched, sched_timer); + struct pt_regs *regs = get_irq_regs(); + ktime_t now = ktime_get(); + + tick_sched_do_timer(ts, now); + + /* + * Do not call when we are not in IRQ context and have + * no valid 'regs' pointer + */ + if (regs) + tick_sched_handle(ts, regs); + else + ts->next_tick = 0; + + /* + * In dynticks mode, tick reprogram is deferred: + * - to the idle task if in dynticks-idle + * - to IRQ exit if in full-dynticks. + */ + if (unlikely(ts->tick_stopped)) + return HRTIMER_NORESTART; + + hrtimer_forward(timer, now, TICK_NSEC); + + return HRTIMER_RESTART; +} #endif #ifdef CONFIG_NO_HZ_FULL @@ -1392,30 +1427,17 @@ void tick_nohz_idle_exit(void) * infrastructure actually relies on the tick itself as a backend in * low-resolution mode (see hrtimer_run_queues()). * - * This low-resolution handler still makes use of some hrtimer APIs meanwhile - * for convenience with expiration calculation and forwarding. + * This low-resolution handler still reuse tick_nohz_highres_handler() since + * most of the work is independent from the clockevent level. */ static void tick_nohz_lowres_handler(struct clock_event_device *dev) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - struct pt_regs *regs = get_irq_regs(); - ktime_t now = ktime_get(); dev->next_event = KTIME_MAX; - tick_sched_do_timer(ts, now); - tick_sched_handle(ts, regs); - - /* - * In dynticks mode, tick reprogram is deferred: - * - to the idle task if in dynticks-idle - * - to IRQ exit if in full-dynticks. - */ - if (likely(!ts->tick_stopped)) { - hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); + if (likely(tick_nohz_highres_handler(&ts->sched_timer) == HRTIMER_RESTART)) tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - } - } static inline void tick_nohz_activate(struct tick_sched *ts, int mode) @@ -1484,46 +1506,6 @@ void tick_irq_enter(void) tick_nohz_irq_enter(); } -/* - * High resolution timer specific code - */ -#ifdef CONFIG_HIGH_RES_TIMERS -/* - * We rearm the timer until we get disabled by the idle code. - * Called with interrupts disabled. - */ -static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) -{ - struct tick_sched *ts = - container_of(timer, struct tick_sched, sched_timer); - struct pt_regs *regs = get_irq_regs(); - ktime_t now = ktime_get(); - - tick_sched_do_timer(ts, now); - - /* - * Do not call when we are not in IRQ context and have - * no valid 'regs' pointer - */ - if (regs) - tick_sched_handle(ts, regs); - else - ts->next_tick = 0; - - /* - * In dynticks mode, tick reprogram is deferred: - * - to the idle task if in dynticks-idle - * - to IRQ exit if in full-dynticks. - */ - if (unlikely(ts->tick_stopped)) - return HRTIMER_NORESTART; - - hrtimer_forward(timer, now, TICK_NSEC); - - return HRTIMER_RESTART; -} -#endif /* HIGH_RES_TIMERS */ - #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS static int sched_skew_tick; @@ -1545,10 +1527,8 @@ void tick_setup_sched_timer(int mode) /* Emulate tick processing via per-CPU hrtimers: */ hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); -#ifdef CONFIG_HIGH_RES_TIMERS if (mode == NOHZ_MODE_HIGHRES) ts->sched_timer.function = tick_nohz_highres_handler; -#endif /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); From patchwork Wed Jan 24 17:04:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191663 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1126522dyi; Wed, 24 Jan 2024 09:06:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IG9LLkSKhn6+gFhpcPudaU0BXu0koy6Onoijom1vrZzWSDP0xw79NBaqBh9LT8OdHPpZj0y X-Received: by 2002:a1f:eecf:0:b0:4bd:72ee:bb92 with SMTP id m198-20020a1feecf000000b004bd72eebb92mr628550vkh.22.1706116012432; Wed, 24 Jan 2024 09:06:52 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116012; cv=pass; d=google.com; s=arc-20160816; b=dk1gVHQ0UO8pLqis7OqbldCU/X5walQbVNJx1PBbxYhpELJdXS9MUKy0J/LsVxx7c8 MqqwHSwLPsOH8i/7Ddyy19t/xK+8q2lwzffmjK1MtAQZt5Z+IUxqBewjrzuO5OguSi9U 4jvwpBEtvAdf0acFjotBtOkvTBoAKa95Uk5tXdSborvppSdhgdXaITTtFNWXqV1lk9Qz 41UJ5sn1fOyoly7cg1iGwi/jtPD1BM1x34DvlDRJcXPQt12f5QqZKZqharzBQuN9GaJy m+JHZXQVOxujA/IeAjJ7AiVbtZ5g+ofkT0WfaBNyjbhCjloZeW8yuhvcM5w8IBP+JpEL RAQA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=B1bkJC9+oIekc3hNpu/4jNDQW/f//PjYZGSOY0GYEUU=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=RhtzaTJNw3fKk6z2HlHSIBww+0BPisN+LCysBAEvuaTnLK5CvTYZK2ac+Mobgj8ZRM kV7TQ3kY2K/zwxPlGM/ylZwUYnvRgKosha+5+A4eGxVfAciQZ8vdU9ElV33q+Ym0YefI VZWx7P45LAQTzHthAf6jDFZlhUBq2FQ+NW4Gu34Apq3BdZx2dXi10hlmiV6/Ga0iZAnQ z4FNhjBaqnfjTeDNi9DWUW2MZovPwdOgefTd1BIQ99nyDwoBSvIyj8EKl1Ixuwq7nr1T pBu0if/vQXybEUD3mIhp3uJ/L3B+KFBjlQ1fuEgswTezYx8C1UdYt8x06wVKDC0Bjyrj 62Ww== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=I+5+0qTf; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37382-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37382-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id l197-20020a1ffece000000b004b7545f10d7si28421vki.266.2024.01.24.09.06.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:06:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37382-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=I+5+0qTf; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37382-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37382-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 24F111C24B2D for ; Wed, 24 Jan 2024 17:06:18 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5629B85C62; Wed, 24 Jan 2024 17:05:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="I+5+0qTf" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA03B82D61 for ; Wed, 24 Jan 2024 17:05:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115917; cv=none; b=cqPmwllO7cmTuUgXFTfz8d9bN1QYr7wKutuMAzWOnlJKIFGcdRFyYaB/Hg8g76GTpEoF8hhdp90kt7AL+4VqFJieeqs2sU/CMUl7r6qaPKAtOt6m6ISwi8SbiN335v4Yre8MuhgRDyUFvDei6PxkMcWSHQ7JqrcrCUcsig64SYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115917; c=relaxed/simple; bh=uzYnPBI5fpXuU09Kdz9I43Q6jP1A/pGf8ZYQb0OnNe0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YZq5/08FZtkuvNkmkm8l02PLOgOXR3Y2briGns8YmaI7TbIiDud64sW8ZRsJ8B9SPhtp+YVJzJGaA/VA2mLpe65/BI+aGWS7PHiiQrMVug9/vLfYE9AID8A85UO+4HN+4mB36zWi60mQYaQ6TO9GkqFoQupTExQQ8CD1asf5G/A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=I+5+0qTf; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B0B0C43390; Wed, 24 Jan 2024 17:05:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115917; bh=uzYnPBI5fpXuU09Kdz9I43Q6jP1A/pGf8ZYQb0OnNe0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I+5+0qTfzlsiEbgesln38n9/trBqNWVhrZLe9aNxDqgiIzvWBj2yy8vGBUTTyhf20 6KxW7NkM/wYuzCk6p9cwoQrCjCe48GcAgTZmqZ08XHpzq3jB4jC5mrElGtEmjE5TtY h9EyX1jch0iWmK8rjFGnta7q7X7QwwHn9lMArleRfHYtzR3qJrVTUPRTHJz7sZAS0j xdTTqs/41lJh+j7HiZ6cAJOlbsLiUlEdW3coIAPrVo4RMr3YYraCfYrTDC/2D/Ynza duOOEZwpXvC0Ip6fpvzbqaYMJLbUpUuUus4b2v4akUmhfvlAI/JbGkQPVdvIyN6OCS VhoAE78NCB8vw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 03/15] tick: Remove useless oneshot ifdeffery Date: Wed, 24 Jan 2024 18:04:47 +0100 Message-ID: <20240124170459.24850-4-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992303793046321 X-GMAIL-MSGID: 1788992303793046321 tick-sched.c is only built when CONFIG_TICK_ONESHOT=y, which is selected only if CONFIG_NO_HZ_COMMON=y or CONFIG_HIGH_RES_TIMERS=y. Therefore the related ifdeferry in this file is needless and can be removed. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-sched.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 6f69ae88e47e..75385b9a233e 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -43,7 +43,6 @@ struct tick_sched *tick_get_tick_sched(int cpu) return &per_cpu(tick_cpu_sched, cpu); } -#if defined(CONFIG_NO_HZ_COMMON) || defined(CONFIG_HIGH_RES_TIMERS) /* * The time when the last jiffy update happened. Write access must hold * jiffies_lock and jiffies_seq. tick_nohz_next_event() needs to get a @@ -290,7 +289,6 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) return HRTIMER_RESTART; } -#endif #ifdef CONFIG_NO_HZ_FULL cpumask_var_t tick_nohz_full_mask; @@ -636,7 +634,7 @@ void __init tick_nohz_init(void) pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n", cpumask_pr_args(tick_nohz_full_mask)); } -#endif +#endif /* #ifdef CONFIG_NO_HZ_FULL */ /* * NOHZ - aka dynamic tick functionality @@ -1506,7 +1504,6 @@ void tick_irq_enter(void) tick_nohz_irq_enter(); } -#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS static int sched_skew_tick; static int __init skew_tick(char *str) @@ -1565,7 +1562,6 @@ void tick_cancel_sched_timer(int cpu) ts->idle_sleeptime = idle_sleeptime; ts->iowait_sleeptime = iowait_sleeptime; } -#endif /* CONFIG_NO_HZ_COMMON || CONFIG_HIGH_RES_TIMERS */ /* * Async notification about clocksource changes From patchwork Wed Jan 24 17:04:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191662 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1126414dyi; Wed, 24 Jan 2024 09:06:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFa8URgP0VZD8mLt+4yabHydL6UovVXhuN1oRf2jV6QxwXWmv1FkYB26inRz0Wt4ccgQ1n1 X-Received: by 2002:a17:906:a3da:b0:a2b:dcad:1f7 with SMTP id ca26-20020a170906a3da00b00a2bdcad01f7mr480429ejb.109.1706116003767; Wed, 24 Jan 2024 09:06:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116003; cv=pass; d=google.com; s=arc-20160816; b=YSP3aQ7V8IGRyyE80wtx4j7ivMqkVcLvZ+3tCoo4orYjLMXOCImxsxEMiW9ptARvDA NtNKDnIrNgiLx65ydR6exQ7STCH1Sizxfvljk19JFkesvnc8NlCmYZjW2rsjgzYcGF+R nOSQVQizYBmgqyp9+08aiH0AX0MP/kuuTrbrETlX9c1VZA/JBWA69ohQMYdCELo+AdzE FEqpoyL6hzyz9JUW/QiAdmjq+spc5Zv9Nf4jrxIrmxQ2Gls6NDbHvqhYODbxaOx2Tt+N iwxA8PD0iowtRtyjQd1FOFALfWbi7j93bc3ByiqfNIxEJCi24la7tR9CcWjzsFdbXQiQ VVVQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=zLncIQzvWeb817S22YOKy1fo3f/q7aikkNCM8xOUeNA=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=ivgDK+r7poV3vSmPg15qPIVgVG05ZtSAuk0dBtVeFqPqdAbJpJkZyXydoeigPGVkZS Z6lsgf22U7fpd3UGOrnQXp6c4WvdmMsd+T3L8vE6fiBm9FDz00j34BVNrUHI7X5pMt3N cw7G7HRstIqHzvX9cBNnq8D+3S4H0hLeEh3ioQztPG6Q1YYO3P0ZhS9dg4gD21+gA+DL XXqvLv8z9NGUPp+Y5hKTtaKf7H7QuffzBhT5aw4yCgzXfL2y7qSomRVRbI5iEzpiamh9 vEp1RhCeCRDcr+X5eYPRM1wU2ZYICHnHd91VPr+64EGnsnay7+MzwiCKEMviKCq269GT TbRw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=R2DXBDkk; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37383-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37383-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id b26-20020a170906491a00b00a2ca0482c51si72019ejq.695.2024.01.24.09.06.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:06:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37383-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=R2DXBDkk; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37383-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37383-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 874221F22558 for ; Wed, 24 Jan 2024 17:06:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9E5FF86138; Wed, 24 Jan 2024 17:05:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R2DXBDkk" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C31D85C59 for ; Wed, 24 Jan 2024 17:05:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115920; cv=none; b=OrZY4TSJLZiDsY+1DALORbwGqRorYtEnb3r3R6sKyMNXlYbyBCfGYupCe0foQklpW2gO9k1i1AKuNsR+PnJjdHojkEVycgCsaetIgnXIvsXEBYtytsJKEWwuilMiEw7VsK6rGuhvtHok5wDzA/4tabHqKft9D1b9jkifLzI1yS0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115920; c=relaxed/simple; bh=BgMq2YElGqLaGH2mfhaxMYUEX5WNvbQSroN6VFODaUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IalOgNjtasWD1DB43W5r+kt/QylKxVnilNFmKQgf3b1YCyHR7LxHWyw681gwvLWTXjG1HeLygAd7X1W6O3st7st5X+gcGZJGPjwcUPvQBJ9Ii+4N6seE7ZwtoKy0dJMdrpMKZxME3zc8s4Q4U6K6yUk4GVixjKNm+emXDgwdQKo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R2DXBDkk; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id B2A3BC43399; Wed, 24 Jan 2024 17:05:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115919; bh=BgMq2YElGqLaGH2mfhaxMYUEX5WNvbQSroN6VFODaUU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R2DXBDkkZHJUQjzAm0QNqlFd12H10f2Q2r/mGZsLT8iKi3Ub0XjWZuzC7FH0rrCnh 32adMHW4RZOCOAxHK6ViN++qd/NtrGkisLdVDwKtlQlthEgxB7PFQNYsCiLnkh1pyV pyObvs8UPCntxM2Pgqr7b+xiV3QoMjLFR7ad0EjUR9OQ3gR6yO9QFNNDEdNxJOL+SY PlaiqImx2VUYI5P0DcCYa2CHtdPFIj7PyYmx+Ikqrl3e/Z60UzGGY9DDOT8q6orWpl wyhIuFPlGwn/XhyFgJvRzZ2XfXGaYQFJmQK0+7q8H92U/Ws6fKw7+19gqNV8vUI3xE WzkeXnHu9LXSw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 04/15] tick: Use IS_ENABLED() whenever possible Date: Wed, 24 Jan 2024 18:04:48 +0100 Message-ID: <20240124170459.24850-5-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992294312259742 X-GMAIL-MSGID: 1788992294312259742 Avoid ifdeferry if it can be converted to IS_ENABLED() whenever possible Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-common.c | 4 +--- kernel/time/tick-sched.c | 14 +++++--------- 2 files changed, 6 insertions(+), 12 deletions(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index e9138cd7a0f5..0084e1ae2583 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -111,15 +111,13 @@ void tick_handle_periodic(struct clock_event_device *dev) tick_periodic(cpu); -#if defined(CONFIG_HIGH_RES_TIMERS) || defined(CONFIG_NO_HZ_COMMON) /* * The cpu might have transitioned to HIGHRES or NOHZ mode via * update_process_times() -> run_local_timers() -> * hrtimer_run_queues(). */ - if (dev->event_handler != tick_handle_periodic) + if (IS_ENABLED(CONFIG_TICK_ONESHOT) && dev->event_handler != tick_handle_periodic) return; -#endif if (!clockevent_state_oneshot(dev)) return; diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 75385b9a233e..c311b7388685 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -186,7 +186,6 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) { int cpu = smp_processor_id(); -#ifdef CONFIG_NO_HZ_COMMON /* * Check if the do_timer duty was dropped. We don't care about * concurrency: This happens only when the CPU in charge went @@ -197,13 +196,13 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) * If nohz_full is enabled, this should not happen because the * 'tick_do_timer_cpu' CPU never relinquishes. */ - if (unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) { + if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && + unlikely(tick_do_timer_cpu == TICK_DO_TIMER_NONE)) { #ifdef CONFIG_NO_HZ_FULL WARN_ON_ONCE(tick_nohz_full_running); #endif tick_do_timer_cpu = cpu; } -#endif /* Check if jiffies need an update */ if (tick_do_timer_cpu == cpu) @@ -230,7 +229,6 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) { -#ifdef CONFIG_NO_HZ_COMMON /* * When we are idle and the tick is stopped, we have to touch * the watchdog as we might not schedule for a really long @@ -239,7 +237,7 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) * idle" jiffy stamp so the idle accounting adjustment we do * when we go busy again does not account too many ticks. */ - if (ts->tick_stopped) { + if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && ts->tick_stopped) { touch_softlockup_watchdog_sched(); if (is_idle_task(current)) ts->idle_jiffies++; @@ -250,7 +248,7 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) */ ts->next_tick = 0; } -#endif + update_process_times(user_mode(regs)); profile_tick(CPU_PROFILING); } @@ -1551,10 +1549,8 @@ void tick_cancel_sched_timer(int cpu) struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); ktime_t idle_sleeptime, iowait_sleeptime; -# ifdef CONFIG_HIGH_RES_TIMERS - if (ts->sched_timer.base) + if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && ts->sched_timer.base) hrtimer_cancel(&ts->sched_timer); -# endif idle_sleeptime = ts->idle_sleeptime; iowait_sleeptime = ts->iowait_sleeptime; From patchwork Wed Jan 24 17:04:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191664 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1126583dyi; Wed, 24 Jan 2024 09:06:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IGv4BaOUDfHpnWYTEtrdTEWutn+7gUQ6ANRNO6hEf8hjicqWprjVa5UaKPEN3gvkrBgmXXC X-Received: by 2002:a05:6a00:2301:b0:6db:e463:96d0 with SMTP id h1-20020a056a00230100b006dbe46396d0mr5056409pfh.55.1706116015445; Wed, 24 Jan 2024 09:06:55 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116015; cv=pass; d=google.com; s=arc-20160816; b=VzO1rvWUsjhLKqrH8PoXtorlnMsA+NyP+lJK/xyi0/Ye3mEfR4mJJUyWTrTpt+q+J9 W7puKkwApqObxJsLU9k/M3Pv0M7WHCuxio3y+CdiSTxiHYu1k3YN+45i1csjsyS8ypqv HfykTxoyLbpQVRwbuvQDnVu5whwlymQKnG+0jqDHvOEkrCtOtAHNkU1PGa2SWiP3K+XX 0kuO5HZGtOs8b0fB95+jYbscz9QKt6Itb9Nk0lxdqMgmiHPtTFvXr83kooMAM+97fl95 7XqHWv/oVb6GB4uW5fJd2hc3xV/LzzXoM8CbhbK0UhoVJn36ubnRO1JPU3O9LxwYCVCf RZhg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=66hwNqqhUPtWR3HSUIYD1dXJyrJ5q2XEHwrvMKcC+E8=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=Wlakj5cMEQmyryxLG9IdUkI8vQpzNeSq/QrsxYz5/s6KV0d/0sPqNJA8UEU2sh0Z8B eAro5jAYhfW1yyk5RMDZG9Siqz+HM7vwa/5EEJaL6se2Tkyee2BnroHBmXNAoYbgGbjF VKfyWI4dc/lxRDmxWDs2cYXELU0i9kGQUwMuzg9X9VahuB4WzvGsEwXXCbE6gB+qr2yv 5DpujEpsEO7btsxxUOKWa1jPS3Ad2vXZK0HpT6myHSUvAnoZSjvHPzrexoDsyxYlkDKv HTz80b2dsxVjlloclPzCHhWbgOoyRpTepFG5UDo5IIgpWgn3tN+aslZdCg76wrOzguT5 UAcA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qIz2KY1g; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37384-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37384-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id jw1-20020a056a00928100b006ddc17332c2si408194pfb.321.2024.01.24.09.06.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:06:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37384-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=qIz2KY1g; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37384-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37384-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 630A9288012 for ; Wed, 24 Jan 2024 17:06:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D504886AC5; Wed, 24 Jan 2024 17:05:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qIz2KY1g" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A8B58612A for ; Wed, 24 Jan 2024 17:05:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115922; cv=none; b=bCIx9Cq2y2Kxzw7g98fnVevu355T6PhPACp7pkioX3YLNSYu6T74fHlp+YoZG5ssPWPrZ4JsQUDhdLTyXicQhe3PS4RPaWIyKTSaOOduKYjWkiBuT1+rN580EjPbc9hV58P6gDTvUHs/4H0XQF3DOw1x5nTLkKwWKqAQ0OiQHRc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115922; c=relaxed/simple; bh=0LrXhB2ixhZpFzsE9HHdFubiObIt2PX57sKxj0n5aZI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bFnqp3qjthDqeUS52Gj5EdwGQbVw+QvtE0qej3ENbN+G6iUixKKx0eLB1DPCpAFGVAdMkiTXNyxZbXQyGZaI6r5/Ze55X1GYSsASqEVoR0KMX9NS+yUtqow/VlnGqsioftuq4z4fD2Aye+DHKsCb4FBqJJka3gcuaijBfPKSJNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qIz2KY1g; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id F3A0CC433F1; Wed, 24 Jan 2024 17:05:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115921; bh=0LrXhB2ixhZpFzsE9HHdFubiObIt2PX57sKxj0n5aZI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qIz2KY1gZDEGMSGD7zEtFfKm94op/OWJg68D5tTMsPTovhjhswFAsd1eMM2gEqXUg vG+2GMOtOUU0ZCGH3cU2W84E3pvQfPZM+affknLuf8UYMlkLk+hyxDS+Obi+tZ4q9c V+xgwTPOS1Z0QUZMaO6qwUZoqG1BWdQSr9JAU7bdsk4Th1ya4sgBfB05UuCcW2GkOH YPDL/aNftJ3PPbE/jp/UjzKqUfupXVSspRj9Dg0neer946GgN9BltG1J2TbeRiIBe/ gKOQNl6ofysUJIoqB9YLW6y/NYFOccznSrfEw7ayKD343vfU6cwfsvVJ2jh+QdQTfK LT1AjCs2vg3hQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 05/15] tick: s/tick_nohz_stop_sched_tick/tick_nohz_full_stop_tick Date: Wed, 24 Jan 2024 18:04:49 +0100 Message-ID: <20240124170459.24850-6-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992306667530212 X-GMAIL-MSGID: 1788992306667530212 tick_nohz_stop_sched_tick() is only about nohz_full and not about dynticks-idle. Reflect that in the function name to avoid confusion. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-sched.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index c311b7388685..4089bd274d8e 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -998,7 +998,7 @@ static void tick_nohz_retain_tick(struct tick_sched *ts) } #ifdef CONFIG_NO_HZ_FULL -static void tick_nohz_stop_sched_tick(struct tick_sched *ts, int cpu) +static void tick_nohz_full_stop_tick(struct tick_sched *ts, int cpu) { if (tick_nohz_next_event(ts, cpu)) tick_nohz_stop_tick(ts, cpu); @@ -1033,7 +1033,7 @@ static void __tick_nohz_full_update_tick(struct tick_sched *ts, int cpu = smp_processor_id(); if (can_stop_full_tick(cpu, ts)) - tick_nohz_stop_sched_tick(ts, cpu); + tick_nohz_full_stop_tick(ts, cpu); else if (ts->tick_stopped) tick_nohz_restart_sched_tick(ts, now); #endif From patchwork Wed Jan 24 17:04:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191665 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1127159dyi; Wed, 24 Jan 2024 09:07:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFCNylPHv/pQTWsdbgM7k7NnIOFadXWoJR3GEhVRi0GKao5NiTQQqgGd3D67wLMjz/RoqC9 X-Received: by 2002:a05:6a20:1b1f:b0:19b:5c7e:ae41 with SMTP id ch31-20020a056a201b1f00b0019b5c7eae41mr1103899pzb.84.1706116053691; Wed, 24 Jan 2024 09:07:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116053; cv=pass; d=google.com; s=arc-20160816; b=b+ZyRGI3BdaGX1vtHYcKsgiNMKfW9TredMbE2GChVVydv3gFwP4ZjAMWI7svaYBP77 YjNOR8KfZMI3EMdNDBk27istVSj4xmL1kBcOecZsDyK8VjfWDVnW+pDjzcgrUTZ+027L wneVk9GEYS4KtPrYJYVoRxg8fZWX/YntFp6U/kzyU/pQ6g4oHP3Z3+rohJBHQV1mVpfS Uul9dyfDlNoero4VD7mRj2JOUGW/ZietRfGowaQX7LlMnGTMC1o/o9XHm1+kTzAMfWUR 5ZUiCtzmo09qrmleAH2skDAyM7XQy7IPJCQ8VkqQlSp0TaqHkNmSMoDDwy6raDSdqF06 EvZg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Qz0puHhEd+YAhMNLcw60pRt9IPO9P02QNt6U2mEfI2g=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=IEdWxue46Iu/QAzRAvzbp2l2pi/hVDWhRmh/uQjL31Ukkmo3njB+gwVDXpplIDYFUG vaa8uQMWFgHuHzGuHhvq1oa4qf+rOSeoeKce0fQJHOSEeo7onnylQ6WwgTjPWrj+5WML b6GxBxO4G+YnbGg32xpboDhUazAPK+1gPHvGJ9rKLd9qgWMem+DEWYu9p/Y5KnD0MR7Z EV1g7UhMw/WxrBkVQIR3UQk5WxKntjNihgMTsdppvQvsNJ8IHkIPulI60hnhxZGjXGE5 JPJ6a1jQzTW8EEf/UCV3iCMmDXfj5p30ee+hXq9iwfuKOvX5p+9j+23K5vDo72IX6gsA b2BQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=CVw4YTMJ; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37385-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37385-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id q62-20020a632a41000000b005ceef2ad0dbsi11890830pgq.384.2024.01.24.09.07.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:07:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37385-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=CVw4YTMJ; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37385-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37385-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 88BBA281881 for ; Wed, 24 Jan 2024 17:06:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CB74C7F7E6; Wed, 24 Jan 2024 17:05:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CVw4YTMJ" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 395B286152 for ; Wed, 24 Jan 2024 17:05:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115924; cv=none; b=GfuuP+5qxrfS+wwQy5xfeApEt5Z7WxmHtkp3ZaWQ0BSxIoyUeL8jdXZeBxxSY3biS/HtdjCz/8nDpHhbopeMWBO0DYjiBG7Al45dNPLLEUNyvewt58MABXmndmmen5Ff6gvKLTRF3gt5XKZromV/EPOJhCnBEibEJjiID07csFk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115924; c=relaxed/simple; bh=GdB05R0GsBmz9FGndTlKhr/I91b5e4z4ItUWDOBSGsM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jTlUINVpxTsGxpCi2DOAgeCj8CIvxIPWWFsjyEHz0ZC947X6rgA2mpNeuV2GMIUPhu+DfIdgeiP8hx0d3Rt+e1PkMa8rosvV5ijJ1C5roRjgcIOc7p9VecVy5qAsd1gVGokLeXC/uIWWR500J2pJbAmBmUv1b0l9CgP7/ALlwEY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CVw4YTMJ; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46808C43394; Wed, 24 Jan 2024 17:05:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115924; bh=GdB05R0GsBmz9FGndTlKhr/I91b5e4z4ItUWDOBSGsM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CVw4YTMJqW3je/Ce8E99eZlXXP+h6cjZcztkpnVmQmdffy4SHzim8AdAxwB2FRNia k9q1sFndfren1L9iuNRuj8j/8TUNDqj41uJgOkC+pwFPluy37N6iHxSH3X6wx9I1yx c15hgzyWIIJePPSV/89Lc5HvugTL1klwPWZ26ayhuVcSQ3RpeVCZYmbW4KZy0Qdxx0 uPA+0qH07tpRav03TSVifwofBjqCTglIKqNkj7mzo1iM/1slrXXcNqCx2S5jgU8MQ1 hrpe+hWJAMeAbH1Q+1rtCHIdZh3p9WB4BLRw9ikRxSaIKe7BkVCNzdAQ3+WgA2+Qdv Sw2NlLFaGXdRw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 06/15] tick: No need to clear ts->next_tick again Date: Wed, 24 Jan 2024 18:04:50 +0100 Message-ID: <20240124170459.24850-7-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992347243277728 X-GMAIL-MSGID: 1788992347243277728 The tick sched structure is already cleared from tick_cancel_sched_timer(), so there is no need to clear that field again. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-sched.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 4089bd274d8e..e8e3b5d447e9 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1101,11 +1101,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) if (unlikely(!cpu_online(cpu))) { if (cpu == tick_do_timer_cpu) tick_do_timer_cpu = TICK_DO_TIMER_NONE; - /* - * Make sure the CPU doesn't get fooled by obsolete tick - * deadline if it comes back online later. - */ - ts->next_tick = 0; return false; } From patchwork Wed Jan 24 17:04:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191666 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1127263dyi; Wed, 24 Jan 2024 09:07:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IF+zV8FP1j7myt+HBmCr7ghD+VHFsa6iPsyaDBVCGHVXZOfBeF/wGtJhnmeSqa6qR9ebjal X-Received: by 2002:a05:6122:318d:b0:4b7:6fa8:ded6 with SMTP id ch13-20020a056122318d00b004b76fa8ded6mr4157293vkb.6.1706116061581; Wed, 24 Jan 2024 09:07:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116061; cv=pass; d=google.com; s=arc-20160816; b=TFh6zFV/2Vk34zqb/kzYiFe+MaBemu93VWgslNuoHhmda6Gq5s49D2YFgCu7WqIHgN JZ9JTTmni8R8lB/+sTVTiXBFNo4QJek97tKjI8NARAyiHndtYdE4LHFGY1xuSKwPdsaX 3n7smauaL3SOLpmr92wOh+weteX1Dz6BsCuDUrdy3bMRq84QEW2lTtB0hB5sFV3r+Gbd UYLIxfx7/zGr6/vDaM0cDTnf88lcG+lYc1ci1f4TIuL+Yh6mqC2TYA1u3LjiG89VVGQJ PJaFW/npnrynZJRq9He9mPpp/SwRSS9a2g6ewO3btYXKMLpfmR8qL4uoVjwXxlTFfPeD 5MBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=EcnRobD5VbU0k32+n4DWrFYyDUmsqklwscFjzhqA3s4=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=k1Vo/PQWSFt8rxYd1MGsIzSMwhBpd7WYZHMLBotJzOGyFYW79kJqFfkt59orxTA/Xr ztejTzyvOcWQtsFmoUrmp9WsbYNdsyOpKf0P0JfMYRLNYBegXzUQEt69+O3fHfNJ5leR txIZj3pwDIVP9w01RaLlnN8IJjGpiUrluo05mYtPXY6EmCjQpBvuHpHM8KFGNF6AqZsI Vg8P5/YPLvPBzxP7QgImsZw8TjW6HLbujp0SLtioRpydjnFyBrhsTuxMiwq23XpkAemm Cv/LUWWwLtUnlI1t8rkY2ORtkCJE+gm8/mMPR9Fzw+BSi9VY5TJ/SFdECDTSs3nf0jdR I4Aw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PXzVaOqe; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37386-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37386-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id j21-20020ac5ce15000000b004b73722de2asi29006vki.188.2024.01.24.09.07.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:07:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37386-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=PXzVaOqe; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37386-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37386-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id A5E091C24CB4 for ; Wed, 24 Jan 2024 17:07:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A8FB286ADA; Wed, 24 Jan 2024 17:05:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PXzVaOqe" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9D7A7F7E5 for ; Wed, 24 Jan 2024 17:05:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115926; cv=none; b=WxHJzhZmbOHcjiFqeWPtyZ4s4iZlv10AM+R/I6Dj/VWvywZYm7arfql8GhVVAKstHu+NXKU1e2ZAmpA9lyBletSi4rh6g29dH3g7UbrqZIqyj1JNtlWF31OFMaAJt8tbionnTQcnaSDeY7qiJBII9koB4FSybNhedDaLIHlE1Ws= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115926; c=relaxed/simple; bh=QnM+UJz4XeNONc8p34P/cNRqaLoZXdwjMjeyYr3C37A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JUM60QDD6AQwWxIcYh+eRzAuZ4BCGdG0AMMWWzBmOPRaLfXIVhvkgCcCej7F7+iKxcz/AHZTUhdXdHbB0//LpAP+8btTkPPx3iy8SdPWSYWTmuoPTEWETs3XUwnMoGxjdiIB8SawMge9IMfDD0KzlbIos5qejTosPHROy2bE+0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PXzVaOqe; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88904C433C7; Wed, 24 Jan 2024 17:05:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115926; bh=QnM+UJz4XeNONc8p34P/cNRqaLoZXdwjMjeyYr3C37A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PXzVaOqeB2ae8XATVtYY0ATKykzEeznIoL8vtipt7Hh4+mPoJWCee6eppLkzEuMRW lEErjWM7fz7Rk1PxV0+WBfqkLD6ozrB9y3LApzb0e9xzeRRgXEF1bncjj7D6Eqq3QH qSuzGt9P6a2HxJD2vj/iRsRQwyKP3ZjnCggKfEhozjBvpUCHAlKMjyvpvp76Xu67Fe cSZLXw1RK2vb0EPb65Aymf7Xtv/qex3lK/NA96qq+MVpo7MZPfrLjR9DBuBkCAehc5 u8JxxYmQtpYKnaunlXSz2I8qHsq9nsdF8UoGzMomNZpF2EIsj8yBpYM/Q2p59QSFDv InVDcjj58+94A== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 07/15] tick: Start centralizing tick related CPU hotplug operations Date: Wed, 24 Jan 2024 18:04:51 +0100 Message-ID: <20240124170459.24850-8-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992355495387487 X-GMAIL-MSGID: 1788992355495387487 During the CPU offlining process, the various timer tick features are shut down from scattered places, sometimes from teardown callbacks on stop machine, sometimes through explicit calls, sometimes from the control CPU after the CPU died. The reason why these shutdown operations are spread around is not always clear and it makes the tick lifecycle hard to follow. The tick should be shut down in order from highest to lowest level: On stop machine from the dying CPU (high-level): 1) Hand-over the timekeeping duty (tick_handover_do_timer()) 2) Cancel the tick implementation called by the clockevent callback (tick_cancel_sched_timer()) 3) Shutdown broadcasting (tick_offline_cpu() / tick_broadcast_offline()) On stop machine from the dying CPU (low-level): 4) Shutdown clockevents drivers (CPUHP_AP_*_TIMER_STARTING states) From the control CPU after the CPU died (low-level): 5) Shutdown/unregister/cleanup clockevents for the dead CPU (tick_cleanup_dead_cpu()) Instead the current order is 2, 4 (both from CPU hotplug states), then 1 and 3 through direct calls. This layout and order don't make much sense. The operations 1, 2, 3 should be gathered together and in order. Sort this situation with creating a new TICK shut-down CPU hotplug state and start with introducing the timekeeping duty hand-over there. The state must precede hrtimers migration because the tick hrtimer will be stopped from it in a further patch. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- include/linux/cpuhotplug.h | 1 + include/linux/tick.h | 8 ++++++-- kernel/cpu.c | 8 +++++--- kernel/time/tick-common.c | 17 +++++++++++------ 4 files changed, 23 insertions(+), 11 deletions(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 172d0a743e5d..74fcdd2b82c8 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -184,6 +184,7 @@ enum cpuhp_state { CPUHP_AP_ARM64_ISNDEP_STARTING, CPUHP_AP_SMPCFD_DYING, CPUHP_AP_HRTIMERS_DYING, + CPUHP_AP_TICK_DYING, CPUHP_AP_X86_TBOOT_DYING, CPUHP_AP_ARM_CACHE_B15_RAC_DYING, CPUHP_AP_ONLINE, diff --git a/include/linux/tick.h b/include/linux/tick.h index 716d17f31c45..afff4c207bd8 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -19,16 +19,20 @@ extern void __init tick_init(void); extern void tick_suspend_local(void); /* Should be core only, but XEN resume magic and ARM BL switcher require it */ extern void tick_resume_local(void); -extern void tick_handover_do_timer(void); extern void tick_cleanup_dead_cpu(int cpu); #else /* CONFIG_GENERIC_CLOCKEVENTS */ static inline void tick_init(void) { } static inline void tick_suspend_local(void) { } static inline void tick_resume_local(void) { } -static inline void tick_handover_do_timer(void) { } static inline void tick_cleanup_dead_cpu(int cpu) { } #endif /* !CONFIG_GENERIC_CLOCKEVENTS */ +#if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_HOTPLUG_CPU) +extern int tick_cpu_dying(unsigned int cpu); +#else +#define tick_cpu_dying NULL +#endif + #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_SUSPEND) extern void tick_freeze(void); extern void tick_unfreeze(void); diff --git a/kernel/cpu.c b/kernel/cpu.c index e6ec3ba4950b..263508073da8 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1324,8 +1324,6 @@ static int take_cpu_down(void *_param) */ cpuhp_invoke_callback_range_nofail(false, cpu, st, target); - /* Give up timekeeping duties */ - tick_handover_do_timer(); /* Remove CPU from timer broadcasting */ tick_offline_cpu(cpu); /* Park the stopper thread */ @@ -2205,7 +2203,11 @@ static struct cpuhp_step cpuhp_hp_states[] = { .startup.single = NULL, .teardown.single = hrtimers_cpu_dying, }, - + [CPUHP_AP_TICK_DYING] = { + .name = "tick:dying", + .startup.single = NULL, + .teardown.single = tick_cpu_dying, + }, /* Entry state on starting. Interrupts enabled from here on. Transient * state for synchronsization */ [CPUHP_AP_ONLINE] = { diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 0084e1ae2583..a89ef450fda7 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -397,15 +397,20 @@ EXPORT_SYMBOL_GPL(tick_broadcast_oneshot_control); #ifdef CONFIG_HOTPLUG_CPU /* - * Transfer the do_timer job away from a dying cpu. - * - * Called with interrupts disabled. No locking required. If - * tick_do_timer_cpu is owned by this cpu, nothing can change it. + * Stop the tick and transfer the timekeeping job away from a dying cpu. */ -void tick_handover_do_timer(void) +int tick_cpu_dying(unsigned int dying_cpu) { - if (tick_do_timer_cpu == smp_processor_id()) + /* + * If the current CPU is the timekeeper, it's the only one that + * can safely hand over its duty. Also all online CPUs are in + * stop machine, guaranteed not to be idle, therefore it's safe + * to pick any online successor. + */ + if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); + + return 0; } /* From patchwork Wed Jan 24 17:04:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191667 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1127368dyi; Wed, 24 Jan 2024 09:07:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IG2XkhZSZTze7f8pIZONB9UqTJ2AaSoQW8WwHfLd8V+GAU1eqMgKy2w6FnTrGFSWn2zgdyz X-Received: by 2002:a9d:63ce:0:b0:6db:d8a9:4205 with SMTP id e14-20020a9d63ce000000b006dbd8a94205mr2006992otl.32.1706116069640; Wed, 24 Jan 2024 09:07:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116069; cv=pass; d=google.com; s=arc-20160816; b=uTjUdYEqGzXggVdhMZZtBdAMPUvwjL+uCICJa6J3SDCXwFSpzdNZx817x8w4OASNT/ ylpYTfBQxhHl5gswXi8DbCUE56rBDyayNjlGslkAoK4+lCFEKXAI4hd/8RO1h2xCdLv2 INB+hHUE9i2CveHEzGdHdoxEbL+qy/CW4KZ9lB4h6i+dzCqe8YSXm/So2uhDAjImStQd 72uibAiA8lG2FNMpu2YAfKBVVDrldDkVKMZfURD0r5YHNQ0blL5Ga9lORTAVE3h6DgO9 4bhRkg3Od2cpppWLOTwcN+SdsTyQmtfDM6IJBBsxadVuPJhvVcV3cUcZWgTzzTvauYPo ekwQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=47IDbJ/2Qd9096ADsC8pqCdxiKBGNBRLqrnYDq4xfIE=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=IU0atQj1Iv+IoicOmaISjGxdT6Rf6NQjF+QDQ9BGTzUCHPpeYJfDLKCPG9sWjK+wmd jlMBt2APt9ml+HXH4khBmKptJhDE+wGBQxj8u8B8cy+Imo/IGSCcTVqxi5myUR1X8gAc RE/EJFB4Mn1nyAB5Rtm4Tv2KsZvJDbpjh5SUu1ZimHXzA/uJzDUuhyuF/bOYe0dW/IKS N+yvVeq9MzC894ezVNeV53LOI2tmmscBBtO7JE1OygSKl/F0hSnJUzMa/Jug/cs5+AbJ BHCAxqsTUZXKiL+CzpYl6dv+oN2CtsWoYWIxRDtfkvem9nRyhvrodFjP++qIoQV0Dkvr 8kgA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=apydmotH; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37387-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37387-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id g6-20020a0561020cc600b0046b0cb8362csi234808vst.127.2024.01.24.09.07.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:07:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37387-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=apydmotH; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37387-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37387-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id BEE751C24480 for ; Wed, 24 Jan 2024 17:07:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 54DF186AF8; Wed, 24 Jan 2024 17:05:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="apydmotH" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDE9186AC6 for ; Wed, 24 Jan 2024 17:05:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115928; cv=none; b=LBjxQe5InB4ggCyCDnIhOng2V+9mV3DlT/pLLeXT4HM3Ojaq6U5Wa5h7Y2TQqeQTa5vlPTkfhJ6Z3JGW+KKAApcJfPVtCR5rus5L22/AsXKS/K3i2mSTPcjGdmfRbDQbXPPe2IYtd8gALjWTWE3dOaWDCZ9JiIbVM6J9u+T455k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115928; c=relaxed/simple; bh=j4R9xWxhSKcn6+gqDWFiRkuawBSZ7hDhyExkS5TrEj4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S/aPUQGaZlSsIZk/sf1oPPzGCoDpiq2cApEJW5fjxfXSbTCUo8P8qgIWVYhljOfPNW7Q4RORf6FVRWHR+BFq1fBKL26VgaieaTPDjm2bhO/ECA4v+8C0M77nKC8j0iXmiA0/0VRp9fy8ybBOLGiYgRLz041nxqMREeSEmSrYjj8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=apydmotH; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id CABACC43399; Wed, 24 Jan 2024 17:05:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115928; bh=j4R9xWxhSKcn6+gqDWFiRkuawBSZ7hDhyExkS5TrEj4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=apydmotH4lzn6QzExX3lySa/jAZFJcG0Q6HLcUJenVqEfGEBvTTaQEWeVuJhQXXtQ keVRazCDXwRioMOrhz6Tc469cP/ptxVQ/9EkxPD9LIz9AxKtCnISCzOpWg8tW1Qo54 q9bBS2UbpAQw2JGomGCHIAkujRiVXlzE7CH02WlvP93Sm5COZ2scJdGEm/SoxBBoGO us2HpGruBkN7RmAgBB0vKiNsT5aO5kfMjC2WKNE/frpmH9jUPoYs5AwVpZw/jY7IB7 JASAbXqxph4xOG9DSkVQ2Dskxksq40FWyqvZMWFnjzNkH8+TUQyP8eRXbgDifbAggL FAFsABhS7LqDg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 08/15] tick: Move tick cancellation up to CPUHP_AP_TICK_DYING Date: Wed, 24 Jan 2024 18:04:52 +0100 Message-ID: <20240124170459.24850-9-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992363865023243 X-GMAIL-MSGID: 1788992363865023243 The tick hrtimer is cancelled right before hrtimers are migrated. This is done from the hrtimer subsystem even though it shouldn't know about its actual users. Move instead the tick hrtimer cancellation to the relevant CPU hotplug state that aims at centralizing high level tick shutdown operations so that the related flow is easy to follow. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/hrtimer.c | 2 -- kernel/time/tick-common.c | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 355b5a957f7f..3b456ec7d4fb 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -2224,8 +2224,6 @@ int hrtimers_cpu_dying(unsigned int dying_cpu) struct hrtimer_cpu_base *old_base, *new_base; int i, ncpu = cpumask_first(cpu_active_mask); - tick_cancel_sched_timer(dying_cpu); - old_base = this_cpu_ptr(&hrtimer_bases); new_base = &per_cpu(hrtimer_bases, ncpu); diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index a89ef450fda7..b4af8c743b73 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -410,6 +410,8 @@ int tick_cpu_dying(unsigned int dying_cpu) if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); + tick_cancel_sched_timer(dying_cpu); + return 0; } From patchwork Wed Jan 24 17:04:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191668 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1127483dyi; Wed, 24 Jan 2024 09:07:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IEY4dtGbG6z6HNJVo9HLQ9i0iuHJlPNc6fimrBJdKs1nO8d96jBGD0ZQ88bv27LItCGxpg+ X-Received: by 2002:a05:6122:2008:b0:4bd:4a95:b8c5 with SMTP id l8-20020a056122200800b004bd4a95b8c5mr2037361vkd.28.1706116079330; Wed, 24 Jan 2024 09:07:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116079; cv=pass; d=google.com; s=arc-20160816; b=pU0iVVPHuzfVsNVpo+g+vLA66uGWSfej8x5+ju2pWOZL+rmx90Kn3HKGDhWju5fmcc GVs3uBaXwOe8kraFixj8alyTe5VXYTrZmgzpZz5cyUcDoKeygtsSm6osS1MoDoyqO2vB G6QfCwaCL3d0wSTK+rW3xp2FzISC/rvE8t99OjOn3GWEskbVFbgFLn7mfVkGs5EazadY +OOp6I3iv+7pIyQZvan+qBOTJzBYqgKiRH/enXbeWgJvr4dqHV/QWBxoP7TF6B1n3E7c Y+PFJECvYKjZafx5beMcYZCyi/Uemk4H3cncYdXX32GqfDJlkV4pJ06VMlBU6i2nX7KW Z4nw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=43CniSD3u5oq5MrsQmsJXBigbxpQ5ZjoUnsuOut4OiE=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=CBAqJfq7FFkZLzlJIoEH8L0ailtyFc83wvfru/N+GP3V3giOZudUce8oZFN6s3t2fI 3Pow5l8CQlwgug19UsnBtN1G4ID2l/lVNbVj9RJZG/AT5jF1SjlK5w1oltE+zkzjeE4x gC5zgaUNQ0P+uq9GjN+adIv0GqCp6YIGe9dd4x0p5/d68nlSBd96N77GUJJriOSbNIiQ vFRSFIlvZrXKhMRkJ3Cm6RbVEy2aN4PZAgKWEq1uH8c+B938KYNCwWRFl/HLIUd+A98k uPUWp2OgwetgZt35wa+XJykpSuZ407X2cmcYwUQPAinFnc7cAaSUziw8fSdUHNLUdxKG TZCQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=VER8K9h2; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37388-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37388-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id ay34-20020a056122182200b004b729fd1b5esi45170vkb.115.2024.01.24.09.07.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:07:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37388-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=VER8K9h2; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37388-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37388-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D4E9D1C2254C for ; Wed, 24 Jan 2024 17:07:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9367C1272BE; Wed, 24 Jan 2024 17:05:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VER8K9h2" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09F6D86AF3 for ; Wed, 24 Jan 2024 17:05:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115931; cv=none; b=KBiuaANn1slV+VDuDWjTm6rK2P0VNpH1BXU3JopLi3Y5M/zmwrapAoluzNKrR/LGIzQtdKfHNhHOx6j8bjQjoPUwogbC8NOclJzc1TKxPxiB4nbDhT3Kr8CJA+Cg4OrABH5g32NSh5jwxNTRz+h2G5KC6z4whgDE9yPfZ49Del0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115931; c=relaxed/simple; bh=M4HS4i9INNBmGr3d/GfNARoMdgXE99jEy5fvedm3YDE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GQxiWmZhi8GFwV9dZTVCUdSog90bfxBuD4u4WXqgwwIvxF/ep+RFMt9ahFEc3Uw5jjQJliwJXLKTH0IqnTvuYenM0WD2T7gb90R4EstTn1GJ9tTd3vbHGFXQUxs0r1CVfJOjy3a5cE8vFqGvFH6VqKybtGEqeRutWuT5Zz9NA1I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VER8K9h2; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 18E49C43394; Wed, 24 Jan 2024 17:05:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115930; bh=M4HS4i9INNBmGr3d/GfNARoMdgXE99jEy5fvedm3YDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VER8K9h2lzl9Xmakflxcyi4nVrh3leQVSrph5vFLdBqjPjsF9Anyt/c21xBTvxPu6 TERvPKs0A/R5dr1/2RhuSDG04alj6MUmkMj+t8zTJH6fl43dG2eQ0/St04tvTo4f16 UT+9WOxWMMBrrYhr4eGRwbXelsZ/14wkhhf4qIKDqQZqkymmQy5RU+yVC/45in3u1u mWRV+ZX8/vZ+uIM7WY5P4az1ZBWYpMkGcc5mSyNfv9h9N4enfCN9IxHHCA6+hUMujN 6p9mPdQ650qF5uFDzVfqlwBJeXcFgo6UJCbX7mrOgT5F5AGNMn3JQIgD/v0frLVkvz JShPiwyDA/zsA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 09/15] tick: Move broadcast cancellation up to CPUHP_AP_TICK_DYING Date: Wed, 24 Jan 2024 18:04:53 +0100 Message-ID: <20240124170459.24850-10-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992373793639699 X-GMAIL-MSGID: 1788992373793639699 The broadcast shutdown code is executed through a random explicit call within stop machine from the outgoing CPU. However The tick broadcast is a midware between the tick callback and the clocksource, therefore it makes more sense to shut it down after the tick callback and before the clocksource drivers. Move it instead to the common tick shutdown CPU hotplug state where related operations can be ordered from highest to lowest level. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- include/linux/tick.h | 6 ------ kernel/cpu.c | 2 -- kernel/time/tick-common.c | 3 +++ kernel/time/tick-internal.h | 2 ++ 4 files changed, 5 insertions(+), 8 deletions(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index afff4c207bd8..c7840ae8ebaf 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -73,12 +73,6 @@ extern void tick_broadcast_control(enum tick_broadcast_mode mode); static inline void tick_broadcast_control(enum tick_broadcast_mode mode) { } #endif /* BROADCAST */ -#if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_HOTPLUG_CPU) -extern void tick_offline_cpu(unsigned int cpu); -#else -static inline void tick_offline_cpu(unsigned int cpu) { } -#endif - #ifdef CONFIG_GENERIC_CLOCKEVENTS extern int tick_broadcast_oneshot_control(enum tick_broadcast_state state); #else diff --git a/kernel/cpu.c b/kernel/cpu.c index 263508073da8..5a8ad4f5ccf3 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1324,8 +1324,6 @@ static int take_cpu_down(void *_param) */ cpuhp_invoke_callback_range_nofail(false, cpu, st, target); - /* Remove CPU from timer broadcasting */ - tick_offline_cpu(cpu); /* Park the stopper thread */ stop_machine_park(cpu); return 0; diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index b4af8c743b73..522414089c0d 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -412,6 +412,9 @@ int tick_cpu_dying(unsigned int dying_cpu) tick_cancel_sched_timer(dying_cpu); + /* Remove CPU from timer broadcasting */ + tick_offline_cpu(dying_cpu); + return 0; } diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h index 481b7ab65e2c..a939ff6de97c 100644 --- a/kernel/time/tick-internal.h +++ b/kernel/time/tick-internal.h @@ -137,8 +137,10 @@ static inline bool tick_broadcast_oneshot_available(void) { return tick_oneshot_ #endif /* !(BROADCAST && ONESHOT) */ #if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_HOTPLUG_CPU) +extern void tick_offline_cpu(unsigned int cpu); extern void tick_broadcast_offline(unsigned int cpu); #else +static inline void tick_offline_cpu(unsigned int cpu) { } static inline void tick_broadcast_offline(unsigned int cpu) { } #endif From patchwork Wed Jan 24 17:04:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191669 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1127617dyi; Wed, 24 Jan 2024 09:08:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IGd8t7nD9xEwuc1a9d8IlN29/xQSKqlqifT8COeaZImdLr5lQ268LW+Ape8I2PATPQnK9na X-Received: by 2002:a05:6808:693:b0:3bc:3c91:e2f1 with SMTP id k19-20020a056808069300b003bc3c91e2f1mr1547135oig.55.1706116089359; Wed, 24 Jan 2024 09:08:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116089; cv=pass; d=google.com; s=arc-20160816; b=OkhF3Fq1innZRMfbVECEG5H0JGOVE2UuXNDrnD1vLQ2H7/KRqxpNtWlDZT7AQnHLFC 4lceiQ4l/QuR77Cs2Wmyo2peHcbkttja5WL8voBHXabSyFz3g469fBmrce6rrIWh30+W 3fmxLMzMPDZ4ScKXcbbiANKlMNtSSJjS0H0mndJyhnWVJRuvleV2CmCRSaL8jP1Jyyf3 PEjmXOrHZtL81zHqgDUrDQlGxBOjeWuC2648XzUqx4kVog7coDRQnKdcCEOTevWwVu9V 55h2uToaTg2xCXpmrXcqiTfSG0RkmZo+AP8c04d1Y7bMuf9nsNTAgGdvFcRuCOTKV11H KF+A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Z/3+VfU54wYuMD8im1IhMoaPzO2qnhXXIIcFb/vaHI0=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=D1hYoShU7ie3HN8+6f7wR8shzeKbNNp40fpqiQAoo2C+zhs4lNP7LLkgqwN27U33Bi rPSYNmRdpYVH9CZYJVKc+2j6ajY03JtnAXl5xa5hBdWf5gE8Kz4R8Pu2l8Y7ZvIJpFPL ex8CMceFhfCrubmOqh5hHeHzCNPaXn+/Q2huJQaQZqe3LxOCNgHeBv8os9tN8n3XGp2m vklZKFqaibBGZ8nrO99SsPcy7pS3lL2Bqp2ucrjJL3DOHl6el8f7FixuT6idHPa+KWue PVhb4j2FCg9vMHTld8kSPAtasDXrXTwQxMxlBECrkGrNd1OV9CBpgvVe7+Y8fsNxNwiz OEig== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Dyiwnhku; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37389-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37389-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id je1-20020a056102178100b004672c09d48bsi1219998vsb.730.2024.01.24.09.08.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:08:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37389-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Dyiwnhku; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37389-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37389-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4B1311C23393 for ; Wed, 24 Jan 2024 17:07:48 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2D0937E789; Wed, 24 Jan 2024 17:05:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Dyiwnhku" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9187C1272BD for ; Wed, 24 Jan 2024 17:05:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115933; cv=none; b=OOS6kInZWYh81VpAQhgxWCMN9dELvQPq+E+r4WghaiG6DhBZSxBXU2lXUe4oepEcBB3T2YmT3A+8nun97Mo7bV4o5nIB5OssHBh88+XikaywdTwATHH0RyUwNi71Mh1b5xwoHyjrnlrMBy3/eVeOpR6KfIrcvQJYdDrfL8l/3Ms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115933; c=relaxed/simple; bh=xd9lp8Dgdk3GeTXPSlcpQ+XIBvUrAdTXmBLHdrTRoNA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cTdM0eT2EbmWRNh1Fxv8OqQtOlI2cqY2fHqnya8sOSjRq6VzdweKLZmXNhciSDo8lcNfY6nssL3fQO+M64TqiuW3M0YRYy8YSy+qWZ+FQrBq8wtknjR9HLX2lHbgUaTUwlJb5fsa27JtNq4gkvGwePvFuadSY07a9v2VEGQo9aA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Dyiwnhku; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5C9F7C43390; Wed, 24 Jan 2024 17:05:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115933; bh=xd9lp8Dgdk3GeTXPSlcpQ+XIBvUrAdTXmBLHdrTRoNA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Dyiwnhku/izDEmU5i9UcrzJoBersWd9xawziwUnL9ig6NdCzpjo+HyFv4hbHyVClj EvSJLB5669yZJOa0zipBW5kH5Yop6DN8kEhUF+w02RlKTo9WXM7JpZFtr9//XMpqvs 8KKejZReOjht4rqgK536aSxhUz1OrPEqj1O3ZISMVP+9lbr99Pl7+Pfbu9AY2IMw87 Wb9AcrbamYC+qpqXFkWFlQckCSuZxK3139LdAr/S0ibu8/KQCmx29i8B5/2/rxRMfJ oiMEdGprze6Q6eV0FwydQmmw/ep+pErXxGF7PQmf8C4MMvE5GI8w5p7MHst9sLpDle PzPuHWXkYByoQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 10/15] tick: Assume the tick can't be stopped in NOHZ_MODE_INACTIVE mode Date: Wed, 24 Jan 2024 18:04:54 +0100 Message-ID: <20240124170459.24850-11-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992383978069664 X-GMAIL-MSGID: 1788992383978069664 The full-nohz update function checks if the nohz mode is active before proceeding. It considers one exception though: if the tick is already stopped even though the nohz mode is inactive, it still moves on in order to update/restart the tick if needed. However in order for the tick to be stopped, the nohz_mode has to be either NOHZ_MODE_LOWRES or NOHZ_MODE_HIGHRES. Therefore it doesn't make sense to test if the tick is stopped before verifying NOHZ_MODE_INACTIVE mode. Remove the needless related condition. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-sched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index e8e3b5d447e9..3104fff89007 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1044,7 +1044,7 @@ static void tick_nohz_full_update_tick(struct tick_sched *ts) if (!tick_nohz_full_cpu(smp_processor_id())) return; - if (!ts->tick_stopped && ts->nohz_mode == NOHZ_MODE_INACTIVE) + if (ts->nohz_mode == NOHZ_MODE_INACTIVE) return; __tick_nohz_full_update_tick(ts, ktime_get()); From patchwork Wed Jan 24 17:04:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191673 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1128998dyi; Wed, 24 Jan 2024 09:10:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IFETkXTR3wyWqsCwVa9PKXyHq0RbL2q2cVdzv1I0zmp73MWA8efTcQl9EAhQv8KIbdTNJfO X-Received: by 2002:aa7:888b:0:b0:6d9:3794:304c with SMTP id z11-20020aa7888b000000b006d93794304cmr4113705pfe.66.1706116202415; Wed, 24 Jan 2024 09:10:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116202; cv=pass; d=google.com; s=arc-20160816; b=YRH+Q8A8q+CPKRBrPNlMXdbssUuHZCwXqKtiWqPHo5+2vce7u9zSSDIK8lfAMAqgCU Tj92Zb2QoUh2Z9ombeEXqfTa8b9Uzhc6FfHLtxaBhUR7Pj8oKmQL0yOF3+EFhZHgkDNM F3xsPTqnTcLeIjlOG06grmjdQ2dmbZo50T6CUZWBeaZlGwLCksbbbeZKnhRaBDyb2GQv tqdsW211LRLX/fxMl89j6tignEVXuiZxfkbGze4m1eDbm4G9nghjnMp1/C+Bf+QLOTg6 X+sJ4ewRpSL7yxmAwQmZSJgg5hNVhdp773aA1Qn8t+dzDWuPDTgcEqH6C3rZBYNcNAeD I8sA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=7VZuerbsDX4IqmesX+lo7p08jYxPXCDasaEk71cYJdU=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=UsCRf+rlzA9q4EJ9kALRFRvmC2KmdaeuDb1zxxQ7/lt8UzwFPpVjp6cBYFckIiVtgs tcO7JfnK1jLxH9sbhW8hWHg5xooN2pCcKk3egeKYuQmxxrQ/FMFlj/m2ijwXAskU1Mrh 2uXNZHJVGP8deoKsTLFoMkLnBbNC0+32fsKMgfm3z+/qR2BF4R4mx/LVN0bgxNhhr+vA ah+SEyKFpok0YE9CyZ0f7FLMFAkLB8zqANLA8kRQ317wg1IXPigI5BFRSRJM9R4bfUee 3+JMdLIr3DZaZ+NOcMXLztIJsskG+FYX5wOJWRInWSZNaqhnLZgR2zO5hosVpbMyXfrf ZjUw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=iiSLcaLQ; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37390-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37390-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id bs68-20020a632847000000b005cee039f233si11592085pgb.425.2024.01.24.09.10.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:10:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37390-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=iiSLcaLQ; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37390-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37390-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id B96D3284BEE for ; Wed, 24 Jan 2024 17:08:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 946DE1272DD; Wed, 24 Jan 2024 17:05:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iiSLcaLQ" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 055C07E782 for ; Wed, 24 Jan 2024 17:05:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115936; cv=none; b=Jw9B+6o31noB5LOfnpRa091yqSmbGt1bFGP7rpF+cWFAHmyAKKY8pXaHeyNvAmdRacOmViNfeWd21D7TLRukeoiBCdg+dI7fEwaxgLnCV04+GFzhamOEggPMoRLhAdwARZstckFNMBAHw7cjRC51N5XPV6+CM7TEIP+Zrufrrqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115936; c=relaxed/simple; bh=q9uMk1kYzNMw7MFUczwSrP/48nreFiHWAfLI4g8u4gM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rp3h3RsGKcZHWJlkFGXq8Mrj/hKVDWEHrlg08ljOlO39A6mIVJydNLnABK/nF361++UUWaoMYhNNyTE+f/tAcxB/12i3OITzaMcDB1xsGhVCZzRLX0+LcVBi6FqyoT/LIlQ8Xku0dk0k2WzRshJpdtIzUOFFq1sgASPZDitAVR8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iiSLcaLQ; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9FE68C433F1; Wed, 24 Jan 2024 17:05:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115935; bh=q9uMk1kYzNMw7MFUczwSrP/48nreFiHWAfLI4g8u4gM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iiSLcaLQx4KzXnHfYnbkXLUfNdon+Rz77GiBfnJ/CRKY2fmwi7IOc3WWrkReuHGmO UB4Ti93CVUj4DdmIXvQqNOsH+E2SPdkOEVTg0lnjKJ0aLLF2CczUKyEosz6c7FokAO 5lLSmgGf8ITFUSvoqK/daOUEogfBm2En0OT1VsTyppvELjdUFqUZkX7sa+XcOcvDl5 uDzJGLuO+DyS2wXXXQiGvCtdYN8RfyhhIQJiJ7M/Zwtnkx4CSXNsuyk5n5CIMspIE+ J+KCUbz1WZrYnNLvlb5ZpxdwSYQzcSKIZtWstFz8SxFkpoiIRzjz68rHuDRrB88I0M ysSiDrjn+4cvw== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 11/15] tick: Move got_idle_tick away from common flags Date: Wed, 24 Jan 2024 18:04:55 +0100 Message-ID: <20240124170459.24850-12-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992502707331170 X-GMAIL-MSGID: 1788992502707331170 tick_nohz_idle_got_tick() is called by cpuidle_reflect() within the idle loop with interrupts enabled. This function modifies the struct tick_sched's bitfield "got_idle_tick". However this bitfield is stored within the same mask as other bitfields that can be modified from interrupts. Fortunately so far it looks like the only race that can happen is while writing ->got_idle_tick to 0, an interrupt fires and writes the ->idle_active field to 0. It's then possible that the interrupted write to ->got_idle_tick writes back the old value of ->idle_active back to 1. However if that happens, the worst possible outcome is that the time spent between that interrupt and the upcoming call to tick_nohz_idle_exit() is accounted as idle, which is negligible quantity. Still all the bitfield writes within this struct tick_sched's shadow mask should be IRQ-safe. Therefore move this bitfield out to its own storage to avoid further suprises. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-sched.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 35808bbb8a47..3b555e0fa937 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -61,7 +61,6 @@ struct tick_sched { unsigned int tick_stopped : 1; unsigned int idle_active : 1; unsigned int do_timer_last : 1; - unsigned int got_idle_tick : 1; /* Tick handling: jiffies stall check */ unsigned int stalled_jiffies; @@ -73,6 +72,7 @@ struct tick_sched { ktime_t next_tick; unsigned long idle_jiffies; ktime_t idle_waketime; + unsigned int got_idle_tick; /* Idle entry */ seqcount_t idle_sleeptime_seq; From patchwork Wed Jan 24 17:04:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191670 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1127946dyi; Wed, 24 Jan 2024 09:08:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IHqTXcFepX2/yTwO7S2TF6W4g1vXcGdJKc4oFsqj+2FOlFgOIHHB8BNo6q5wWaHwzZMTroh X-Received: by 2002:a05:6871:88c:b0:210:a19a:ae3d with SMTP id r12-20020a056871088c00b00210a19aae3dmr3958914oaq.80.1706116114538; Wed, 24 Jan 2024 09:08:34 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116114; cv=pass; d=google.com; s=arc-20160816; b=qS+3p+7+6OfnLiqDWcUQsuUREuAnuuPuwW99w0p7a/zj1DhF+zvIe2sRycOY5wZptK 828uigyt7AB4wiE60bm5S8/CC1Cv4S/0nPhRJlXwzKGnfMCQc/DbGTdjyminxkiMSZro UgUns+Ue+1y5vGFEf15mgbJOO3zbv6oYkG9bMvfGX1zK6KDRSi3zlI0/BtpvJidfYYmt VIPOYIoGVg1Jkekzjo9IDle1pFgDOGpa9RhkkgbIz/0+430/2Nw29yUOvE/fmCpQus6d BvISxOWnBwdoWeYN4Q9LopYw0ZNW13Ny6wZshN96liYKyugTNBItfE3iXG0yMlFHADCk Tn2w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=4ls+2HoIuRGyg5k9x7qAEH5HSLVHU5q1dUQqjoHaRq4=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=rC3A+mhkAG6l7Qv0sdLpntt96cRGj4GS3G9NSyVB5eJ35mfz66qk0c8gVsfENPA3Du KlZazHR2HrW/etqSzmEQLVIab6ofhLkD9xcj8DXxJKb3uLc0GOYtJCQOybadjXat5RE3 w2vsMGt3/Vu+PjGwb5Ob33Cs8G4X3pkXQ8H2pxM9MxH8c3d3fI0LFVXiI2JZ1ACc+Oo9 1GqaeKFNotzntOGSJ7QSlsfkXTd5jOE0quy4/zI18RE1QWwFQiOEjsdx6n0mCDcn2jGX A5jLbqVTe23+2T8M9xKp5CFDZmwLOtJFwe6nEvf0xjCJrz/uN2wCvPRJuSVsGfyaNUB3 WO3A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=afFu1XEj; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37391-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37391-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id l65-20020a1fed44000000b004b7545f10casi30769vkh.166.2024.01.24.09.08.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:08:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37391-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=afFu1XEj; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37391-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37391-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 85F0C1C23A9C for ; Wed, 24 Jan 2024 17:08:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A3E63128393; Wed, 24 Jan 2024 17:05:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="afFu1XEj" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B05E1272C8 for ; Wed, 24 Jan 2024 17:05:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115938; cv=none; b=R7hEmDPYXFPxAxqQTqM7+Z9sHuIDZd9VaR1pETTiUFOue3B0wy1vza1mctVpKdKcwpVS3gvp35Y8YbY/FbA7l8tL8HNLL68Snkuk1rU1bGkcvMlYZhBqdfoG0o3Ah2rlc1dsJMFZUC5IdUlawnaNd7UyYFUVn2SJ2Khs11jN7D0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115938; c=relaxed/simple; bh=vnSjPjyGuwHT0H1EryHVqfaY+UithxNYnUokHTzxfk4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n3Yq3P1T7oUqbcHuX7nPrGLgsFDPqL7VwyrGlJJKQbdIuKoYsr3abUQzlVobbgmVDuHn1EaxqpTo5zJdhAXoxehijliT3x58pVqfyfP8g7i5fPuHvQ96AdcQxjIs6EvYpQxvvo1+55atr/eVKKXXkLZowhfZiIbE/t93kSMbCwc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=afFu1XEj; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E26A2C433A6; Wed, 24 Jan 2024 17:05:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115937; bh=vnSjPjyGuwHT0H1EryHVqfaY+UithxNYnUokHTzxfk4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=afFu1XEjFeSwBajqRmosrrxBggwnp/nNyeClFN2urrOfHJQQptCOWy147a4i2+GBa cBcDrsslq8L5NHYE3xFkLChWED7FVkPR18wQtkrjJbfMhYPZ0QbfvU3dw/+aNBs8Rg pRpvq4baTBzf6j5ai49A10AvXDKddfB23OmY2e6V90QZThqt/96Oa2QAm6BJPyA8LM 416Y5S5DLzzLgocpNlCmajaj9itiKEHZ26Ds/Hudauo10wCcfgvTtUxIcxAeGjAzk8 xfPGJRrlE0xSniUwCAZnvWKz63DBjXWceAOvalNH11nv1yq+X9LgIQYbiwbBVKyY4p VkDrEC8X8M9/A== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 12/15] tick: Move individual bit features to debuggable mask accesses Date: Wed, 24 Jan 2024 18:04:56 +0100 Message-ID: <20240124170459.24850-13-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992411187326406 X-GMAIL-MSGID: 1788992411187326406 The individual bitfields of struct tick_sched must be modified from IRQs disabled places, otherwise local modifications can race due to them sharing the same memory storage. The recent move of the "got_idle_tick" bitfield to its own storage shows that the use of these bitfields, as pretty as they look, can be as much error prone. In order to avoid future issues of the like and make sure that those bitfields are safely accessed, move those flags to an explicit mask along with a mutator function performing the basic IRQs disabled sanity check. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-sched.c | 86 +++++++++++++++++++++++++--------------- kernel/time/tick-sched.h | 23 ++++++----- kernel/time/timer_list.c | 5 ++- 3 files changed, 72 insertions(+), 42 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 3104fff89007..8dc9c7557776 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -180,6 +180,26 @@ static ktime_t tick_init_jiffy_update(void) return period; } +static inline int tick_sched_flag_test(struct tick_sched *ts, + unsigned long flag) +{ + return !!(ts->flags & flag); +} + +static inline void tick_sched_flag_set(struct tick_sched *ts, + unsigned long flag) +{ + lockdep_assert_irqs_disabled(); + ts->flags |= flag; +} + +static inline void tick_sched_flag_clear(struct tick_sched *ts, + unsigned long flag) +{ + lockdep_assert_irqs_disabled(); + ts->flags &= ~flag; +} + #define MAX_STALLED_JIFFIES 5 static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) @@ -223,7 +243,7 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now) } } - if (ts->inidle) + if (tick_sched_flag_test(ts, TS_FLAG_INIDLE)) ts->got_idle_tick = 1; } @@ -237,7 +257,8 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs) * idle" jiffy stamp so the idle accounting adjustment we do * when we go busy again does not account too many ticks. */ - if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && ts->tick_stopped) { + if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && + tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { touch_softlockup_watchdog_sched(); if (is_idle_task(current)) ts->idle_jiffies++; @@ -280,7 +301,7 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) * - to the idle task if in dynticks-idle * - to IRQ exit if in full-dynticks. */ - if (unlikely(ts->tick_stopped)) + if (unlikely(tick_sched_flag_test(ts, TS_FLAG_STOPPED))) return HRTIMER_NORESTART; hrtimer_forward(timer, now, TICK_NSEC); @@ -560,7 +581,7 @@ void __tick_nohz_task_switch(void) ts = this_cpu_ptr(&tick_cpu_sched); - if (ts->tick_stopped) { + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { if (atomic_read(¤t->tick_dep_mask) || atomic_read(¤t->signal->tick_dep_mask)) tick_nohz_full_kick(); @@ -657,14 +678,14 @@ bool tick_nohz_tick_stopped(void) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - return ts->tick_stopped; + return tick_sched_flag_test(ts, TS_FLAG_STOPPED); } bool tick_nohz_tick_stopped_cpu(int cpu) { struct tick_sched *ts = per_cpu_ptr(&tick_cpu_sched, cpu); - return ts->tick_stopped; + return tick_sched_flag_test(ts, TS_FLAG_STOPPED); } /** @@ -694,7 +715,7 @@ static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) { ktime_t delta; - if (WARN_ON_ONCE(!ts->idle_active)) + if (WARN_ON_ONCE(!tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE))) return; delta = ktime_sub(now, ts->idle_entrytime); @@ -706,7 +727,7 @@ static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); ts->idle_entrytime = now; - ts->idle_active = 0; + tick_sched_flag_clear(ts, TS_FLAG_IDLE_ACTIVE); write_seqcount_end(&ts->idle_sleeptime_seq); sched_clock_idle_wakeup_event(); @@ -716,7 +737,7 @@ static void tick_nohz_start_idle(struct tick_sched *ts) { write_seqcount_begin(&ts->idle_sleeptime_seq); ts->idle_entrytime = ktime_get(); - ts->idle_active = 1; + tick_sched_flag_set(ts, TS_FLAG_IDLE_ACTIVE); write_seqcount_end(&ts->idle_sleeptime_seq); sched_clock_idle_sleep_event(); @@ -738,7 +759,7 @@ static u64 get_cpu_sleep_time_us(struct tick_sched *ts, ktime_t *sleeptime, do { seq = read_seqcount_begin(&ts->idle_sleeptime_seq); - if (ts->idle_active && compute_delta) { + if (tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE) && compute_delta) { ktime_t delta = ktime_sub(now, ts->idle_entrytime); idle = ktime_add(*sleeptime, delta); @@ -889,7 +910,7 @@ static ktime_t tick_nohz_next_event(struct tick_sched *ts, int cpu) * We've not stopped the tick yet, and there's a timer in the * next period, so no point in stopping it either, bail. */ - if (!ts->tick_stopped) { + if (!tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { ts->timer_expires = 0; goto out; } @@ -902,7 +923,8 @@ static ktime_t tick_nohz_next_event(struct tick_sched *ts, int cpu) */ delta = timekeeping_max_deferment(); if (cpu != tick_do_timer_cpu && - (tick_do_timer_cpu != TICK_DO_TIMER_NONE || !ts->do_timer_last)) + (tick_do_timer_cpu != TICK_DO_TIMER_NONE || + !tick_sched_flag_test(ts, TS_FLAG_DO_TIMER_LAST))) delta = KTIME_MAX; /* Calculate the next expiry time */ @@ -936,13 +958,13 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) */ if (cpu == tick_do_timer_cpu) { tick_do_timer_cpu = TICK_DO_TIMER_NONE; - ts->do_timer_last = 1; + tick_sched_flag_set(ts, TS_FLAG_DO_TIMER_LAST); } else if (tick_do_timer_cpu != TICK_DO_TIMER_NONE) { - ts->do_timer_last = 0; + tick_sched_flag_clear(ts, TS_FLAG_DO_TIMER_LAST); } /* Skip reprogram of event if it's not changed */ - if (ts->tick_stopped && (expires == ts->next_tick)) { + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED) && (expires == ts->next_tick)) { /* Sanity check: make sure clockevent is actually programmed */ if (expires == KTIME_MAX || ts->next_tick == hrtimer_get_expires(&ts->sched_timer)) return; @@ -960,12 +982,12 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * call we save the current tick time, so we can restart the * scheduler tick in tick_nohz_restart_sched_tick(). */ - if (!ts->tick_stopped) { + if (!tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { calc_load_nohz_start(); quiet_vmstat(); ts->last_tick = hrtimer_get_expires(&ts->sched_timer); - ts->tick_stopped = 1; + tick_sched_flag_set(ts, TS_FLAG_STOPPED); trace_tick_stop(1, TICK_DEP_MASK_NONE); } @@ -1022,7 +1044,7 @@ static void tick_nohz_restart_sched_tick(struct tick_sched *ts, ktime_t now) touch_softlockup_watchdog_sched(); /* Cancel the scheduled timer and restore the tick: */ - ts->tick_stopped = 0; + tick_sched_flag_clear(ts, TS_FLAG_STOPPED); tick_nohz_restart(ts, now); } @@ -1034,7 +1056,7 @@ static void __tick_nohz_full_update_tick(struct tick_sched *ts, if (can_stop_full_tick(cpu, ts)) tick_nohz_full_stop_tick(ts, cpu); - else if (ts->tick_stopped) + else if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) tick_nohz_restart_sched_tick(ts, now); #endif } @@ -1154,14 +1176,14 @@ void tick_nohz_idle_stop_tick(void) ts->idle_calls++; if (expires > 0LL) { - int was_stopped = ts->tick_stopped; + int was_stopped = tick_sched_flag_test(ts, TS_FLAG_STOPPED); tick_nohz_stop_tick(ts, cpu); ts->idle_sleeps++; ts->idle_expires = expires; - if (!was_stopped && ts->tick_stopped) { + if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { ts->idle_jiffies = ts->last_jiffies; nohz_balance_enter_idle(cpu); } @@ -1197,7 +1219,7 @@ void tick_nohz_idle_enter(void) WARN_ON_ONCE(ts->timer_expires_base); - ts->inidle = 1; + tick_sched_flag_set(ts, TS_FLAG_INIDLE); tick_nohz_start_idle(ts); local_irq_enable(); @@ -1226,7 +1248,7 @@ void tick_nohz_irq_exit(void) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - if (ts->inidle) + if (tick_sched_flag_test(ts, TS_FLAG_INIDLE)) tick_nohz_start_idle(ts); else tick_nohz_full_update_tick(ts); @@ -1280,7 +1302,7 @@ ktime_t tick_nohz_get_sleep_length(ktime_t *delta_next) ktime_t now = ts->idle_entrytime; ktime_t next_event; - WARN_ON_ONCE(!ts->inidle); + WARN_ON_ONCE(!tick_sched_flag_test(ts, TS_FLAG_INIDLE)); *delta_next = ktime_sub(dev->next_event, now); @@ -1352,7 +1374,7 @@ void tick_nohz_idle_restart_tick(void) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); - if (ts->tick_stopped) { + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { ktime_t now = ktime_get(); tick_nohz_restart_sched_tick(ts, now); tick_nohz_account_idle_time(ts, now); @@ -1393,12 +1415,12 @@ void tick_nohz_idle_exit(void) local_irq_disable(); - WARN_ON_ONCE(!ts->inidle); + WARN_ON_ONCE(!tick_sched_flag_test(ts, TS_FLAG_INIDLE)); WARN_ON_ONCE(ts->timer_expires_base); - ts->inidle = 0; - idle_active = ts->idle_active; - tick_stopped = ts->tick_stopped; + tick_sched_flag_clear(ts, TS_FLAG_INIDLE); + idle_active = tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE); + tick_stopped = tick_sched_flag_test(ts, TS_FLAG_STOPPED); if (idle_active || tick_stopped) now = ktime_get(); @@ -1464,10 +1486,10 @@ static inline void tick_nohz_irq_enter(void) struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); ktime_t now; - if (!ts->idle_active && !ts->tick_stopped) + if (!tick_sched_flag_test(ts, TS_FLAG_STOPPED | TS_FLAG_IDLE_ACTIVE)) return; now = ktime_get(); - if (ts->idle_active) + if (tick_sched_flag_test(ts, TS_FLAG_IDLE_ACTIVE)) tick_nohz_stop_idle(ts, now); /* * If all CPUs are idle we may need to update a stale jiffies value. @@ -1476,7 +1498,7 @@ static inline void tick_nohz_irq_enter(void) * rare case (typically stop machine). So we must make sure we have a * last resort. */ - if (ts->tick_stopped) + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) tick_nohz_update_jiffies(now); } diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 3b555e0fa937..07a4c0144c47 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -20,14 +20,22 @@ enum tick_nohz_mode { NOHZ_MODE_HIGHRES, }; +/* The CPU is in the tick idle mode */ +#define TS_FLAG_INIDLE BIT(0) +/* The idle tick has been stopped */ +#define TS_FLAG_STOPPED BIT(1) +/* + * Indicator that the CPU is actively in the tick idle mode; + * it is reset during irq handling phases. + */ +#define TS_FLAG_IDLE_ACTIVE BIT(2) +/* CPU was the last one doing do_timer before going idle */ +#define TS_FLAG_DO_TIMER_LAST BIT(3) + /** * struct tick_sched - sched tick emulation and no idle tick control/stats * - * @inidle: Indicator that the CPU is in the tick idle mode - * @tick_stopped: Indicator that the idle tick has been stopped - * @idle_active: Indicator that the CPU is actively in the tick idle mode; - * it is reset during irq handling phases. - * @do_timer_last: CPU was the last one doing do_timer before going idle + * @flags: State flags gathering the TS_FLAG_* features * @got_idle_tick: Tick timer function has run with @inidle set * @stalled_jiffies: Number of stalled jiffies detected across ticks * @last_tick_jiffies: Value of jiffies seen on last tick @@ -57,10 +65,7 @@ enum tick_nohz_mode { */ struct tick_sched { /* Common flags */ - unsigned int inidle : 1; - unsigned int tick_stopped : 1; - unsigned int idle_active : 1; - unsigned int do_timer_last : 1; + unsigned long flags; /* Tick handling: jiffies stall check */ unsigned int stalled_jiffies; diff --git a/kernel/time/timer_list.c b/kernel/time/timer_list.c index ed7d6ad694fb..38f81d836fc5 100644 --- a/kernel/time/timer_list.c +++ b/kernel/time/timer_list.c @@ -147,11 +147,14 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now) # define P_ns(x) \ SEQ_printf(m, " .%-15s: %Lu nsecs\n", #x, \ (unsigned long long)(ktime_to_ns(ts->x))) +# define P_flag(x, f) \ + SEQ_printf(m, " .%-15s: %d\n", #x, !!(ts->flags & (f))) + { struct tick_sched *ts = tick_get_tick_sched(cpu); P(nohz_mode); P_ns(last_tick); - P(tick_stopped); + P_flag(tick_stopped, TS_FLAG_STOPPED); P(idle_jiffies); P(idle_calls); P(idle_sleeps); From patchwork Wed Jan 24 17:04:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191671 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1128034dyi; Wed, 24 Jan 2024 09:08:42 -0800 (PST) X-Google-Smtp-Source: AGHT+IFbbZ8hLM9GrV52C7HMUscsHRSL7XYp2fLPFrowvp9Gm0hXK7eMvrAtRu56NnY/k/U7L40s X-Received: by 2002:a05:6871:5211:b0:210:90fa:1ff7 with SMTP id ht17-20020a056871521100b0021090fa1ff7mr3470550oac.74.1706116122344; Wed, 24 Jan 2024 09:08:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116122; cv=pass; d=google.com; s=arc-20160816; b=bkxkC4bVLXGfngD9nvq7KRJFVznHyrofOfDESvMwiCk2/ckEQCGTODlJf2aujYFEeT FDlHG83PXCX6YeMC/3QI2y5xRhe+Ws8T+0qmIf7zG8vRi8XKRxREe2S18Glr+6CyXsHb aGthlMuOUxqIM+csbmgeChX0WKtpuqzLvUMd23XArC5709B3LpgMeLEQgBiyEjDFKVLD GqoA10BrerHooUBYKmK16/DGjxNkJGdnQ6tjHi00IQiAnw3YhvDwdDpNWBaVGvqYiC1g LXMOW0GwvrtESr4QTIvi0WnLtDRHACFVud9juj525XVfkSYePxN9A8fZ5I4LjRdtTdNA Y9pQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=qiYQtdm7amIAvnePQ5Ey4tg1hfsue3Cpr2xaaCF0UW8=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=1J7VrsZ2lzdfhnMhv9l+SvTeUhF4LyZtSoMbW2ywEJ1vMrPCrLYYwr4fYLVWjyDuq+ wGnihV94uMPmcpa7lFeIvIcqQESBqdhVwf+ABDjtkK8awe2unmXcDVxA7WbQKXgsyrNI Qa/Hr3NOx+WidVDf74DLe4pYb3c+LJhStfGM6IqtKlfOokEhgOS9CWu134HH9Wa220gd vKR9J7ZUPLfsse8twTGXZj1tkspcHgK+JBFruarBos4ST/4dz8gqTqB/9RQ9nSeOUBXM SV+OgaKMgp0K+8jV7EBzvZKr971tCNlbGuu+irP5edupXD6RljOamlm+MAyVEyO4JYjl bvqA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=DuQkInVA; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37392-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37392-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id j21-20020ac5ce15000000b004b73722de2asi29006vki.188.2024.01.24.09.08.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:08:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37392-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=DuQkInVA; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37392-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37392-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 437781C2354C for ; Wed, 24 Jan 2024 17:08:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E84B91292CD; Wed, 24 Jan 2024 17:05:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DuQkInVA" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BD7D128372 for ; Wed, 24 Jan 2024 17:05:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115940; cv=none; b=XPVyY9aJR53i7hK6mGSe3IoERRgItRDTpiWFaPk067Td+c4xK0KkvkC0IHZSxlsads41VX8cqmHGu5puPDZTnteOUmO2Vukk0bwyHVE9o70Qj1weErTlLO3YY7ttyWAayxvjZC3fhMgfcnQDdJuSqlKKu394tVnh1+ksC15OPS4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115940; c=relaxed/simple; bh=lsGUXL5LMIFhpic7ChOHhkP9MANhD8t8ZIHOwtUbcmM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fG5tYot9QGAGK5axnEuIKdBWvRqSSYbXjH31hO0uFGnLkPeUT7LwWWx7cSTPmbIXhCn3jgzLOuvgxafU2M5asOu6SiiUdQfz7CtYjPDfKtbPk8w00/qKg/e5E6dVoDRE5+mpiwWMv3vIwLu2AiWfSLT99e5ZLpyeMaAhg2iXQSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DuQkInVA; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 31A44C43390; Wed, 24 Jan 2024 17:05:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115940; bh=lsGUXL5LMIFhpic7ChOHhkP9MANhD8t8ZIHOwtUbcmM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DuQkInVAUkFFe9x3NG73vufvMIvDmGKVKl1pu+wVnZqx+sYtQIRaR2APFcfngKhhG Wd7xhRYwmVwcGjeTqrpcsZxsb2q74P3nKJMMR+cRJX8hFF/K800l2Q7EweXLjcceaI X42ebGIrKiKadxwUtlGnsgNMYWTlGxvBqgoFdLYCdRokfuxAhfEE9ZTZx+9nmmyC/8 peFsNlziSq3cKqkqSGvNzWumzlsawhLrBqQWOiX2YkXDjpzdUcnSe/VMsvh3AXYR3d WXEqsBUu4rEqgYRHGp0L65gE4NU18RNs348ts8c/GdLE5zGMOnFoHoX22b3vMQmGiz KUSObyTXdCfEA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 13/15] tick: Split nohz and highres features from nohz_mode Date: Wed, 24 Jan 2024 18:04:57 +0100 Message-ID: <20240124170459.24850-14-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992419029614935 X-GMAIL-MSGID: 1788992419029614935 The nohz mode field tells about low resolution nohz mode or high resolution nohz mode but it doesn't tell about high resolution non-nohz mode. In order to retrieve the latter state, tick_cancel_sched_timer() must fiddle with struct hrtimer's internals to guess if the tick has been initialized in high resolution. Move instead the nohz mode field information into the tick flags and provide two new bits: one to know if the tick is in nohz mode and another one to know if the tick is in high resolution. The combination of those two flags provides all the needed informations to determine which of the three tick modes is running. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/hrtimer.c | 2 +- kernel/time/tick-sched.c | 32 +++++++++++++++++--------------- kernel/time/tick-sched.h | 13 +++++-------- kernel/time/timer_list.c | 5 +++-- 4 files changed, 26 insertions(+), 26 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 3b456ec7d4fb..d8ba7985fe0d 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -746,7 +746,7 @@ static void hrtimer_switch_to_hres(void) base->hres_active = 1; hrtimer_resolution = HIGH_RES_NSEC; - tick_setup_sched_timer(NOHZ_MODE_HIGHRES); + tick_setup_sched_timer(true); /* "Retrigger" the interrupt to get things going */ retrigger_next_event(NULL); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 8dc9c7557776..274ac5941b16 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -832,7 +832,7 @@ static void tick_nohz_restart(struct tick_sched *ts, ktime_t now) /* Forward the time to expire in the future */ hrtimer_forward(&ts->sched_timer, now, TICK_NSEC); - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) { + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) { hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); } else { @@ -998,14 +998,14 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * the tick timer. */ if (unlikely(expires == KTIME_MAX)) { - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) hrtimer_cancel(&ts->sched_timer); else tick_program_event(KTIME_MAX, 1); return; } - if (ts->nohz_mode == NOHZ_MODE_HIGHRES) { + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) { hrtimer_start(&ts->sched_timer, expires, HRTIMER_MODE_ABS_PINNED_HARD); } else { @@ -1066,7 +1066,7 @@ static void tick_nohz_full_update_tick(struct tick_sched *ts) if (!tick_nohz_full_cpu(smp_processor_id())) return; - if (ts->nohz_mode == NOHZ_MODE_INACTIVE) + if (!tick_sched_flag_test(ts, TS_FLAG_NOHZ)) return; __tick_nohz_full_update_tick(ts, ktime_get()); @@ -1126,7 +1126,7 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) return false; } - if (unlikely(ts->nohz_mode == NOHZ_MODE_INACTIVE)) + if (unlikely(!tick_sched_flag_test(ts, TS_FLAG_NOHZ))) return false; if (need_resched()) @@ -1453,11 +1453,11 @@ static void tick_nohz_lowres_handler(struct clock_event_device *dev) tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); } -static inline void tick_nohz_activate(struct tick_sched *ts, int mode) +static inline void tick_nohz_activate(struct tick_sched *ts) { if (!tick_nohz_enabled) return; - ts->nohz_mode = mode; + tick_sched_flag_set(ts, TS_FLAG_NOHZ); /* One update is enough */ if (!test_and_set_bit(0, &tick_nohz_active)) timers_update_nohz(); @@ -1478,7 +1478,7 @@ static void tick_nohz_switch_to_nohz(void) * Recycle the hrtimer in 'ts', so we can share the * highres code. */ - tick_setup_sched_timer(NOHZ_MODE_LOWRES); + tick_setup_sched_timer(false); } static inline void tick_nohz_irq_enter(void) @@ -1506,7 +1506,7 @@ static inline void tick_nohz_irq_enter(void) static inline void tick_nohz_switch_to_nohz(void) { } static inline void tick_nohz_irq_enter(void) { } -static inline void tick_nohz_activate(struct tick_sched *ts, int mode) { } +static inline void tick_nohz_activate(struct tick_sched *ts) { } #endif /* CONFIG_NO_HZ_COMMON */ @@ -1533,14 +1533,16 @@ early_param("skew_tick", skew_tick); * tick_setup_sched_timer - setup the tick emulation timer * @mode: tick_nohz_mode to setup for */ -void tick_setup_sched_timer(int mode) +void tick_setup_sched_timer(bool hrtimer) { struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); /* Emulate tick processing via per-CPU hrtimers: */ hrtimer_init(&ts->sched_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD); - if (mode == NOHZ_MODE_HIGHRES) + if (hrtimer) { + tick_sched_flag_set(ts, TS_FLAG_HIGHRES); ts->sched_timer.function = tick_nohz_highres_handler; + } /* Get the next period (per-CPU) */ hrtimer_set_expires(&ts->sched_timer, tick_init_jiffy_update()); @@ -1554,11 +1556,11 @@ void tick_setup_sched_timer(int mode) } hrtimer_forward_now(&ts->sched_timer, TICK_NSEC); - if (mode == NOHZ_MODE_HIGHRES) + if (hrtimer) hrtimer_start_expires(&ts->sched_timer, HRTIMER_MODE_ABS_PINNED_HARD); else tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1); - tick_nohz_activate(ts, mode); + tick_nohz_activate(ts); } void tick_cancel_sched_timer(int cpu) @@ -1566,7 +1568,7 @@ void tick_cancel_sched_timer(int cpu) struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); ktime_t idle_sleeptime, iowait_sleeptime; - if (IS_ENABLED(CONFIG_HIGH_RES_TIMERS) && ts->sched_timer.base) + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) hrtimer_cancel(&ts->sched_timer); idle_sleeptime = ts->idle_sleeptime; @@ -1612,7 +1614,7 @@ int tick_check_oneshot_change(int allow_nohz) if (!test_and_clear_bit(0, &ts->check_clocks)) return 0; - if (ts->nohz_mode != NOHZ_MODE_INACTIVE) + if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) return 0; if (!timekeeping_valid_for_hres() || !tick_is_oneshot_available()) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 07a4c0144c47..bbe72a078985 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -14,12 +14,6 @@ struct tick_device { enum tick_device_mode mode; }; -enum tick_nohz_mode { - NOHZ_MODE_INACTIVE, - NOHZ_MODE_LOWRES, - NOHZ_MODE_HIGHRES, -}; - /* The CPU is in the tick idle mode */ #define TS_FLAG_INIDLE BIT(0) /* The idle tick has been stopped */ @@ -31,6 +25,10 @@ enum tick_nohz_mode { #define TS_FLAG_IDLE_ACTIVE BIT(2) /* CPU was the last one doing do_timer before going idle */ #define TS_FLAG_DO_TIMER_LAST BIT(3) +/* NO_HZ is enabled */ +#define TS_FLAG_NOHZ BIT(4) +/* High resolution tick mode */ +#define TS_FLAG_HIGHRES BIT(5) /** * struct tick_sched - sched tick emulation and no idle tick control/stats @@ -84,7 +82,6 @@ struct tick_sched { ktime_t idle_entrytime; /* Tick stop */ - enum tick_nohz_mode nohz_mode; unsigned long last_jiffies; u64 timer_expires_base; u64 timer_expires; @@ -107,7 +104,7 @@ struct tick_sched { extern struct tick_sched *tick_get_tick_sched(int cpu); -extern void tick_setup_sched_timer(int mode); +extern void tick_setup_sched_timer(bool hrtimer); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS extern void tick_cancel_sched_timer(int cpu); #else diff --git a/kernel/time/timer_list.c b/kernel/time/timer_list.c index 38f81d836fc5..1c311c46da50 100644 --- a/kernel/time/timer_list.c +++ b/kernel/time/timer_list.c @@ -152,7 +152,8 @@ static void print_cpu(struct seq_file *m, int cpu, u64 now) { struct tick_sched *ts = tick_get_tick_sched(cpu); - P(nohz_mode); + P_flag(nohz, TS_FLAG_NOHZ); + P_flag(highres, TS_FLAG_HIGHRES); P_ns(last_tick); P_flag(tick_stopped, TS_FLAG_STOPPED); P(idle_jiffies); @@ -259,7 +260,7 @@ static void timer_list_show_tickdevices_header(struct seq_file *m) static inline void timer_list_header(struct seq_file *m, u64 now) { - SEQ_printf(m, "Timer List Version: v0.9\n"); + SEQ_printf(m, "Timer List Version: v0.10\n"); SEQ_printf(m, "HRTIMER_MAX_CLOCK_BASES: %d\n", HRTIMER_MAX_CLOCK_BASES); SEQ_printf(m, "now at %Ld nsecs\n", (unsigned long long)now); SEQ_printf(m, "\n"); From patchwork Wed Jan 24 17:04:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191675 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1129400dyi; Wed, 24 Jan 2024 09:10:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IHiInCPUw8/1PyC2xCS3bQMiX9LaCYAblOvsXyVTsQgJQwiLG1Na6hdKnQ5XOtWccu4YU8c X-Received: by 2002:a17:902:9a0c:b0:1d5:dab0:380c with SMTP id v12-20020a1709029a0c00b001d5dab0380cmr1403523plp.48.1706116233732; Wed, 24 Jan 2024 09:10:33 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116233; cv=pass; d=google.com; s=arc-20160816; b=G0tEk0j2a/5XUTEULQPFuzztEUbXeI7FNL/Hb53HQP6oEvkkZ78LSezCNPykOuqdzV oMwzok29ablefVHBtsVtOnioy8MxaWLVeU9Oj0PhVhHAmkPWpucoIqmEzzkxIhGdKOEB z3lJ99HcBT+S5FusMWMjojA7VH6Z6+iPYRVa4Tzms8r7UMf5fbf6oNKkLNjMJMrLhENF lO/sXTfkjWQ0xQ6oylFbB+Tli3cTBODIEVcfM+nfkfiURDUje/zOgamOWuKirSyRMtgO 2cPa8v5/N3ifwmY3mRUgKbEUi1ZoJ7uhnn9FQODdRuwcPPo6j0VJc6IUzTwW5pDc5vdM i1wg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=5okaiAd6pmvjlrSqLYz72yEVYH257iwhIGw2wTeO4mw=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=sY1ZYNewg70tBmO77Pyw9a1pgsnXALK2QYzsZcfcZnoZPzGdrZC2OVE9A10S4DfNJX 9QPg+/bvI0I/x786PFk2cqdxDDUsKHInuuZltpZu+9/3tVd4mlci/ZJXgqz4zDC/pPLr jDtBnH2yGcsenp0ii1xcUYGPy0EJtKACf83udK8lMZ+Thi+qx85AD5RHfSYAGDZ067iG utLgWnQefaZUD7dij1+Ra5oHV+ezr22JvL3pFN3kgYkvoQz2jc9UVboq4QcFk3Xg2qSa U/R1onNhcngcu2yDFxyzbkDk6bLuZVsspe4514wK8JOEfGOvYB5bgb7IWlD95juorpd0 4utA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=kojVIWxt; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37393-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37393-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id u8-20020a170902e5c800b001d7596dfc97si5207252plf.511.2024.01.24.09.10.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:10:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37393-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=kojVIWxt; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37393-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37393-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 73F4628B353 for ; Wed, 24 Jan 2024 17:08:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AE6E91292F2; Wed, 24 Jan 2024 17:05:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kojVIWxt" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFF22128399 for ; Wed, 24 Jan 2024 17:05:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115942; cv=none; b=U6jAA5H9kzoaCbaUt6TGGiqxmz/BnknmbnQUEu4mHFG0+7BL+gcaIwq8ITZsIBVgYR6FEqW5J4aK6taJlpSUO/NGbsHBtrnzd0JNvf8RcRM+VU7Ixq01KM3GMeV4WZV7QlfCh1fwHhMyijd177AU0PzD7Cz+Cxe+nY8VbzupYW4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115942; c=relaxed/simple; bh=eSXm80QnH1JhZdpRmNd1upMtxM0h6BzQ1w/ftFOWWPc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q/uyLFzjyV0ub05OamzbTyijWRtzFJzdy0o7MCrieyxfbHA4lRTm6eraBormJ6BhSc4UQ3Hz7YBge4xum8I9UiWG5DYM8ZL/Kh0nEBW1uGpdnbFXajmKW50yklB+alue8DVVHGJgXzIjFa2TfNX6AOCGb0LlLRnj+18KcjZL4ro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kojVIWxt; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 75272C433F1; Wed, 24 Jan 2024 17:05:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115942; bh=eSXm80QnH1JhZdpRmNd1upMtxM0h6BzQ1w/ftFOWWPc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kojVIWxtXJKhyTSyng1Lga8n+MLK0Pk421TnVKDDBf1CabhJTJB9wm+to4PFy2/DK OBBiPcozPTTB9A9z7GcjUmeOpJbZkxPFGKBNyZ8IRLKb/VuNwKEpve4V2QqyawwbVP vK6GrAf8ug64HuewPiNYHfQDkaa1SG6r0geZ6OL0ONVgzU0lYTF/64ETDSa3oTQp6u V3UFL+K/B7o29Rx5BQxRxfbar7iCQ4g0lVfaWWBuiMRsZ/CnuSbufT8WM1ITdDnTbq AXgUfk4z/MmAbD0j34ZaEkTEboJ9CDkdWqBLK7r/AyqE32Q5aeYU48+2OQeTjOWU/w 4BfZ60jSoqb6g== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 14/15] tick: Shut down low-res tick from dying CPU Date: Wed, 24 Jan 2024 18:04:58 +0100 Message-ID: <20240124170459.24850-15-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992535568910029 X-GMAIL-MSGID: 1788992535568910029 The timekeeping duty is handed over from the outgoing CPU within stop machine. This works well if CONFIG_NO_HZ_COMMON=n or the tick is in high-res mode. However in low-res dynticks mode, the tick isn't cancelled until the clockevent is shut down, which can happen later. The tick may therefore fire again once IRQs are re-enabled on stop machine and until IRQs are disabled for good upon the last call to idle. That's so many opportunities for a timekeeper to go idle and the outgoing CPU to take over that duty. This is why tick_nohz_idle_stop_tick() is called one last time on idle if the CPU is seen offline: so that the timekeeping duty is handed over again in case the CPU has re-taken the duty. This means there are two timekeeping handovers on CPU down hotplug with different undocumented constraints and purposes: 1) A handover on stop machine for !dynticks || highres. All online CPUs are guaranteed to be non-idle and the timekeeping duty can be safely handed-over. The hrtimer tick is cancelled so it is guaranteed that in dynticks mode the outgoing CPU won't take again the duty. 2) A handover on last idle call for dynticks && lowres. Setting the duty to TICK_DO_TIMER_NONE makes sure that a CPU will take over the timekeeping. Prepare for consolidating the handover to a single place (the first one) with shutting down the low-res tick as well from tick_cancel_sched_timer() as well. This will simplify the handover and unify the tick cancellation between high-res and low-res. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- kernel/time/tick-common.c | 3 ++- kernel/time/tick-sched.c | 32 +++++++++++++++++++++++++------- kernel/time/tick-sched.h | 4 ++-- 3 files changed, 29 insertions(+), 10 deletions(-) diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 522414089c0d..9cd09eea06d6 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -410,7 +410,8 @@ int tick_cpu_dying(unsigned int dying_cpu) if (tick_do_timer_cpu == dying_cpu) tick_do_timer_cpu = cpumask_first(cpu_online_mask); - tick_cancel_sched_timer(dying_cpu); + /* Make sure the CPU won't try to retake the timekeeping duty */ + tick_sched_timer_dying(dying_cpu); /* Remove CPU from timer broadcasting */ tick_offline_cpu(dying_cpu); diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 274ac5941b16..5e7fe19b9977 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -309,6 +309,14 @@ static enum hrtimer_restart tick_nohz_highres_handler(struct hrtimer *timer) return HRTIMER_RESTART; } +static void tick_sched_timer_cancel(struct tick_sched *ts) +{ + if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) + hrtimer_cancel(&ts->sched_timer); + else if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) + tick_program_event(KTIME_MAX, 1); +} + #ifdef CONFIG_NO_HZ_FULL cpumask_var_t tick_nohz_full_mask; EXPORT_SYMBOL_GPL(tick_nohz_full_mask); @@ -998,10 +1006,7 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) * the tick timer. */ if (unlikely(expires == KTIME_MAX)) { - if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) - hrtimer_cancel(&ts->sched_timer); - else - tick_program_event(KTIME_MAX, 1); + tick_sched_timer_cancel(ts); return; } @@ -1563,13 +1568,26 @@ void tick_setup_sched_timer(bool hrtimer) tick_nohz_activate(ts); } -void tick_cancel_sched_timer(int cpu) +/* + * Shut down the tick and make sure the CPU won't try to retake the timekeeping + * duty before disabling IRQs in idle for the last time. + */ +void tick_sched_timer_dying(int cpu) { + struct tick_device *td = &per_cpu(tick_cpu_device, cpu); struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); + struct clock_event_device *dev = td->evtdev; ktime_t idle_sleeptime, iowait_sleeptime; - if (tick_sched_flag_test(ts, TS_FLAG_HIGHRES)) - hrtimer_cancel(&ts->sched_timer); + /* This must happen before hrtimers are migrated! */ + tick_sched_timer_cancel(ts); + + /* + * If the clockevents doesn't support CLOCK_EVT_STATE_ONESHOT_STOPPED, + * make sure not to call low-res tick handler. + */ + if (tick_sched_flag_test(ts, TS_FLAG_NOHZ)) + dev->event_handler = clockevents_handle_noop; idle_sleeptime = ts->idle_sleeptime; iowait_sleeptime = ts->iowait_sleeptime; diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index bbe72a078985..58d8d1c49dd3 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -106,9 +106,9 @@ extern struct tick_sched *tick_get_tick_sched(int cpu); extern void tick_setup_sched_timer(bool hrtimer); #if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS -extern void tick_cancel_sched_timer(int cpu); +extern void tick_sched_timer_dying(int cpu); #else -static inline void tick_cancel_sched_timer(int cpu) { } +static inline void tick_sched_timer_dying(int cpu) { } #endif #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST From patchwork Wed Jan 24 17:04:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 191672 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2553:b0:103:945f:af90 with SMTP id p19csp1128237dyi; Wed, 24 Jan 2024 09:08:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IGLC1m8tqgIYpYjZYiBHFl4at7e+gyY2rCnDFEBfj8fJWKLlH0vLELLIhTYcmVJYcVcfHAx X-Received: by 2002:a05:6102:675:b0:46b:14b0:3ae7 with SMTP id z21-20020a056102067500b0046b14b03ae7mr160744vsf.11.1706116139027; Wed, 24 Jan 2024 09:08:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706116139; cv=pass; d=google.com; s=arc-20160816; b=X4YaPw9HxU9686yvOSRFiEMC/j7fXmsUgNjWC9+EUWqBdCeWzdnOP4i6nLSoS0jAvd N2TV6VHDG8lyqFtlEe0iWAzaQ74dSRZbJS66kr6Ut/MD1ZjjLDreVnbs7DwIfTxetGgj D1Gw0qRRLHo5BhuTdWk9wYwq7Drlk65lLV2l9HRwV0e8sVEBi7YptLnVRvSplpaps5aO NEvdGP8ZezLxB1saQDZuwNf66esj4CiRosW8RAMDUlkG9s7p6uc/nS2XH4i6BQo66ncS 1I8BCKJ4iYhaNBjQvXsFHZguAE3Ek33vtbgOeV08Z0UPTQ/eGkjokQ9e3ps4Sb7F9sTk cQKA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ATtxyFyWosgiuBCF0vbKhLOwvqcrCscHoEDHdQSFBYA=; fh=9AYDxpIF3ws21qQDvjghHSFUuXpDdpq2Zh/SbnasVRo=; b=qMLaK7DfIiKFxpsDKWXNy3jnwQuBCBFDfbRHDGrCM2E8Bi7g2ZaG7Xsqekl3KIhQYZ UgYzSjAaJUiDpWfwphl8ZL+BOQsf30G8bn2iXs2HYHrFDiCe1NAzBAWkDtywSnBwp4ub H9NtZNiMXvunS2QyIGhnjG2hLPT5747QZFWAcHzDkfHvfDHsOowYRf+PWVdTK2pmZeqy UJ6mBGcKfPfgr5VAthw8CjIqlRC188i/tNENP1zdi5BervE/cBOV/GBdx1frcpmFDCri nwdnyhlQBinN8v2iRKKoOQ70BhXu+ox+pPNwdXXiUwQQolJprJ/g2qqs19RbztdbiUur agyA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Vg4qqDuy; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37394-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id g4-20020a056102244400b0046b11941446si172592vss.602.2024.01.24.09.08.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 09:08:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-37394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=Vg4qqDuy; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-37394-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-37394-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 203341C20DC3 for ; Wed, 24 Jan 2024 17:08:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A7A458121C; Wed, 24 Jan 2024 17:05:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Vg4qqDuy" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BA841292E1 for ; Wed, 24 Jan 2024 17:05:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115945; cv=none; b=KOPk3zID6PKiNf6aMm3hbRdgnrx+ImTPFKIeUHPfDS1dzcQtTRJ8y1QtPGClVIkd078ek1RkMJKvU5FSPSzBaoSv6gqiqM8L+wbdnpeoU39wk9TEcEd1N4F4g5iXtbsS6PGkfNncHcP/d4AYCn1HsSrE5V2rGqSGX/YZ+uhEPpQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706115945; c=relaxed/simple; bh=3rLpbH9TxDLwqTYzfL+gAm8SDz7YKrWvuQPGpKHVxno=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hluESTlVXO7vLEj/ov6lIDC8qrK9x87weWYtTn38dOYejREm0CDtfFoDqmD774SzV3JpqroEc99JZkkHoM+oVlD5+av/GAQrlS2ehU+ympWngiJ5z81V55qbDOtWs0VVL/SeJyVYKqzG3yEzuVp55HSmXWITJuv07ceDQakr07k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Vg4qqDuy; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC9C5C433C7; Wed, 24 Jan 2024 17:05:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706115944; bh=3rLpbH9TxDLwqTYzfL+gAm8SDz7YKrWvuQPGpKHVxno=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vg4qqDuykNEPszLPrOj3WVHt2+IWNF4VYtJnXEuBGLuZ97X8YmliQh60bLzFAxuuQ bohOfSKHiqFzobB1EoJrFe1QzZ0Ch7FRDrljo3acl03u4KUM3HfdZCItw5LvIEQNGL +UlyYuYN3NMntl/X3ReQeK3JMxCNVIADU+/xUcFINYVfPrU5DFBLwaZFO7NUG/azBL eCqPaUZzKZ+SthVI78rsCXQNNXXyWk8xaAmSKo0AdUpbwHG+nOZmoUeuOGMXmiDMaC AgNFHECZDuxyYFihka2vblwHHATOyTTHioLAlJ/sQrj/+hj4GUIPFp6IDLal0iYY2R lF5jsKMKQzGUg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Thomas Gleixner , Ingo Molnar , Anna-Maria Behnsen , Peng Liu , Joel Fernandes Subject: [PATCH 15/15] tick: Assume timekeeping is correctly handed over upon last offline idle call Date: Wed, 24 Jan 2024 18:04:59 +0100 Message-ID: <20240124170459.24850-16-frederic@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240124170459.24850-1-frederic@kernel.org> References: <20240124170459.24850-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788992436196092808 X-GMAIL-MSGID: 1788992436196092808 The timekeeping duty is handed over from the outgoing CPU on stop machine, then the oneshot tick is stopped right after. Therefore it's guaranteed that the current CPU isn't the timekeeper upon its last call to idle. Besides, calling tick_nohz_idle_stop_tick() while the dying CPU goes into idle suggests that the tick is going to be stopped while it is actually stopped already from the appropriate CPU hotplug state. Remove the confusing call and the obsolete case handling and convert it to a sanity check that verifies the above assumption. Signed-off-by: Frederic Weisbecker Reviewed-by: Thomas Gleixner --- include/linux/tick.h | 2 ++ kernel/cpu.c | 1 + kernel/sched/idle.c | 1 - kernel/time/tick-common.c | 4 ++++ kernel/time/tick-sched.c | 13 +------------ 5 files changed, 8 insertions(+), 13 deletions(-) diff --git a/include/linux/tick.h b/include/linux/tick.h index c7840ae8ebaf..44fddfa93e18 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -29,8 +29,10 @@ static inline void tick_cleanup_dead_cpu(int cpu) { } #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_HOTPLUG_CPU) extern int tick_cpu_dying(unsigned int cpu); +extern void tick_assert_timekeeping_handover(void); #else #define tick_cpu_dying NULL +static inline void tick_assert_timekeeping_handover(void) { } #endif #if defined(CONFIG_GENERIC_CLOCKEVENTS) && defined(CONFIG_SUSPEND) diff --git a/kernel/cpu.c b/kernel/cpu.c index 5a8ad4f5ccf3..7e84a7b0675e 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -1399,6 +1399,7 @@ void cpuhp_report_idle_dead(void) struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state); BUG_ON(st->state != CPUHP_AP_OFFLINE); + tick_assert_timekeeping_handover(); rcutree_report_cpu_dead(); st->state = CPUHP_AP_IDLE_DEAD; /* diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 31231925f1ec..b15d40cad7ea 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -291,7 +291,6 @@ static void do_idle(void) local_irq_disable(); if (cpu_is_offline(cpu)) { - tick_nohz_idle_stop_tick(); cpuhp_report_idle_dead(); arch_cpu_idle_dead(); } diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c index 9cd09eea06d6..fb0fdec8719a 100644 --- a/kernel/time/tick-common.c +++ b/kernel/time/tick-common.c @@ -396,6 +396,10 @@ int tick_broadcast_oneshot_control(enum tick_broadcast_state state) EXPORT_SYMBOL_GPL(tick_broadcast_oneshot_control); #ifdef CONFIG_HOTPLUG_CPU +void tick_assert_timekeeping_handover(void) +{ + WARN_ON_ONCE(tick_do_timer_cpu == smp_processor_id()); +} /* * Stop the tick and transfer the timekeeping job away from a dying cpu. */ diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 5e7fe19b9977..a98e026244b0 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1118,18 +1118,7 @@ static bool report_idle_softirq(void) static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) { - /* - * If this CPU is offline and it is the one which updates - * jiffies, then give up the assignment and let it be taken by - * the CPU which runs the tick timer next. If we don't drop - * this here, the jiffies might be stale and do_timer() never - * gets invoked. - */ - if (unlikely(!cpu_online(cpu))) { - if (cpu == tick_do_timer_cpu) - tick_do_timer_cpu = TICK_DO_TIMER_NONE; - return false; - } + WARN_ON_ONCE(cpu_is_offline(cpu)); if (unlikely(!tick_sched_flag_test(ts, TS_FLAG_NOHZ))) return false;