From patchwork Wed May 10 07:28:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 91923 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp3431350vqo; Wed, 10 May 2023 00:34:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ49n6DvIf7SwyQuvdP28bxHhr4Sj711XrgKou3OTigRXOB8NhIXA0qFZ15fmH4vZFx3tfaI X-Received: by 2002:a05:6a00:2daa:b0:643:b054:534e with SMTP id fb42-20020a056a002daa00b00643b054534emr19384015pfb.31.1683704088945; Wed, 10 May 2023 00:34:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683704088; cv=none; d=google.com; s=arc-20160816; b=wEaK3PUTgFE8/H13+lSiSt4AYnfgyl8NF1uARWPn9m2CeYfJ+TwMzse3tmOzknIndI K2Jdfe8em71Q0FJ1Vsg50YA9pxqEDmzednmLpnTAd/hKI/e+vkZ198UNxdbGabccM7eX jN7o8saQBAhaWM0PZkZeSbpfJ+Zn9nkXJmeEl3KpPzLDbmo2sdBpWnxFVizqhajXyv6N 6nIYfsiZY/1xZpjWR64d1B8p9CQpMw6rsIgdZWuKcdb8E0I8Ia3Mx5fzHQesEOCj7bGc NKQ7wQgz77F95WEGGF6K2y4nFHNu8fa/hu1s9oavWwg9uKTz/xEERptXzOXmgP+aYZUc 31nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=qpYAvi1JZijNMB51RYPkA79oCnc7/oZP5SpTj4wG4c4=; b=R8SM36AosXEaCnzxzE5OwbFoaF6bgPLLYjPWNzWjsmPuLyLPr/nTh1Um0oF1y1tmq4 MfsVNH/mcpocT4Z8r7GxEL95Om6E9T13rlu9fncp/mswBhFfC9tMGVuQrI5+MmXUcn/o ogNVySaQhg0iOfl/ua6Gc9BwWI6mlACVfAj4hd4v1jHF5L1t1uyZHbI337RlODT2Oc19 Vx9dkbmysYbrPuZf8bSKtAGLMfXCpx6BLWw2jXNT/SVEfzfNdO0udyF9KvQ/mhUu9wh5 DGglmPgOiAf2n9ad/upUHdhshRjHsyCwgnqDIgdBXczrga4xpXHajfHAIewdQXg7vdQr 7YEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=EkQIpQ3L; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a19-20020aa795b3000000b0064309797534si4461894pfk.182.2023.05.10.00.34.34; Wed, 10 May 2023 00:34:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=EkQIpQ3L; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236520AbjEJHaf (ORCPT + 99 others); Wed, 10 May 2023 03:30:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236461AbjEJH3q (ORCPT ); Wed, 10 May 2023 03:29:46 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6AC57EED for ; Wed, 10 May 2023 00:29:16 -0700 (PDT) From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1683703724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qpYAvi1JZijNMB51RYPkA79oCnc7/oZP5SpTj4wG4c4=; b=EkQIpQ3L05Kbr0jdsrdbOsZPWFIO9yNQS8/GXqugE5nEBWMu9G2BW16Z9sqq/fIO64uTMG ToTD4ENSXRIq0O2oU2wua10g2zeEegz4bsXEBgvzgg5wHnyXjMDspGzDQiwSxGWOLlw+Bi yUq1Auu3+vatFgkVf/rL5/PUK8yAKY6nJpLP63Cj2NSbXtxvsVVCEpqUwsd0zldJaVgFH9 Or37+4RRCq917vM9cqb+KF0bDvdS7ZKjyrglnwmpG52HUzw/r4OALRnQYO7HTWl4tAxZfM xbEmjC6Z28+gfCx0lrOsHPHKg53T9THOugOopZ3MVhvnnx3df2wI99B4cS6CIw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1683703724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qpYAvi1JZijNMB51RYPkA79oCnc7/oZP5SpTj4wG4c4=; b=thWf/y9jmuIG9h3rzxdg00R55Wdxw7np8EFxSDE0QxlY8huJTj3Q5TzdHN8Fk2LiEcaVdn 3Ur+RM3afCT0+qCA== To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , John Stultz , Thomas Gleixner , Eric Dumazet , "Rafael J . Wysocki" , Arjan van de Ven , "Paul E . McKenney" , Frederic Weisbecker , Rik van Riel , Steven Rostedt , Sebastian Siewior , Giovanni Gherdovich , Lukasz Luba , "Gautham R . Shenoy" , Anna-Maria Behnsen Subject: [PATCH v6 15/21] timer: Add get next timer interrupt functionality for remote CPUs Date: Wed, 10 May 2023 09:28:11 +0200 Message-Id: <20230510072817.116056-16-anna-maria@linutronix.de> In-Reply-To: <20230510072817.116056-1-anna-maria@linutronix.de> References: <20230510072817.116056-1-anna-maria@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765491698840203970?= X-GMAIL-MSGID: =?utf-8?q?1765491698840203970?= To prepare for the conversion of the NOHZ timer placement to a pull at expiry time model it's required to have functionality available getting the next timer interrupt on a remote CPU. Locking of the timer bases and getting the information for the next timer interrupt functionality is split into separate functions. This is required to be compliant with lock ordering when the new model is in place. Signed-off-by: Anna-Maria Behnsen --- v6: - introduce timer_lock_remote_bases() to fix race --- kernel/time/tick-internal.h | 9 +++++ kernel/time/timer.c | 71 ++++++++++++++++++++++++++++++++++--- 2 files changed, 75 insertions(+), 5 deletions(-) diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h index 649f2b48e8f0..c7bba0bb19d9 100644 --- a/kernel/time/tick-internal.h +++ b/kernel/time/tick-internal.h @@ -8,6 +8,11 @@ #include "timekeeping.h" #include "tick-sched.h" +struct timer_events { + u64 local; + u64 global; +}; + #ifdef CONFIG_GENERIC_CLOCKEVENTS # define TICK_DO_TIMER_NONE -1 @@ -164,6 +169,10 @@ static inline void timers_update_nohz(void) { } DECLARE_PER_CPU(struct hrtimer_cpu_base, hrtimer_bases); extern u64 get_next_timer_interrupt(unsigned long basej, u64 basem); +extern void fetch_next_timer_interrupt_remote(unsigned long basej, u64 basem, + struct timer_events *tevt, + unsigned int cpu); +extern bool timer_lock_remote_bases(unsigned int cpu); void timer_clear_idle(void); #define CLOCK_SET_WALL \ diff --git a/kernel/time/timer.c b/kernel/time/timer.c index 4bb6c168d106..2370c9a15ba6 100644 --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -221,11 +221,6 @@ struct timer_base { static DEFINE_PER_CPU(struct timer_base, timer_bases[NR_BASES]); -struct timer_events { - u64 local; - u64 global; -}; - #ifdef CONFIG_NO_HZ_COMMON static DEFINE_STATIC_KEY_FALSE(timers_nohz_active); @@ -2013,6 +2008,72 @@ static unsigned long fetch_next_timer_interrupt(struct timer_base *base_local, return local_first ? nextevt_local : nextevt_global; } +/** + * fetch_next_timer_interrupt_remote + * @basej: base time jiffies + * @basem: base time clock monotonic + * @tevt: Pointer to the storage for the expiry values + * @cpu: Remote CPU + * + * Stores the next pending local and global timer expiry values in the + * struct pointed to by @tevt. If a queue is empty the corresponding + * field is set to KTIME_MAX. If local event expires before global + * event, global event is set to KTIME_MAX as well. + * + * Caller needs to make sure timer base locks are held (use + * timer_lock_remote_bases() for this purpose). Caller must make sure + * interrupts are reopened, if required. + */ +void fetch_next_timer_interrupt_remote(unsigned long basej, u64 basem, + struct timer_events *tevt, + unsigned int cpu) +{ + struct timer_base *base_local, *base_global; + + /* Preset local / global events */ + tevt->local = tevt->global = KTIME_MAX; + + base_local = per_cpu_ptr(&timer_bases[BASE_LOCAL], cpu); + base_global = per_cpu_ptr(&timer_bases[BASE_GLOBAL], cpu); + + lockdep_assert_held(&base_local->lock); + lockdep_assert_held(&base_global->lock); + + fetch_next_timer_interrupt(base_local, base_global, basej, basem, tevt); + + raw_spin_unlock(&base_global->lock); + raw_spin_unlock(&base_local->lock); +} + + +/** + * timer_lock_remote_bases - lock timer bases of cpu + * @cpu: Remote CPU + * + * Locks the remote timer bases. + * + * Returns false if cpu is offline, otherwise true is returned. + */ +bool timer_lock_remote_bases(unsigned int cpu) +{ + struct timer_base *base_local, *base_global; + + /* + * Pretend that there is no timer pending if the cpu is offline. + * Possible pending timers will be migrated later to an active cpu. + */ + if (cpu_is_offline(cpu)) + return false; + + base_local = per_cpu_ptr(&timer_bases[BASE_LOCAL], cpu); + base_global = per_cpu_ptr(&timer_bases[BASE_GLOBAL], cpu); + + raw_spin_lock_irq(&base_local->lock); + raw_spin_lock_nested(&base_global->lock, SINGLE_DEPTH_NESTING); + + return true; +} + /* * Forward base clock is done only when @basej is past base->clk, otherwise * base-clk might be rewind.