From patchwork Wed Oct 4 12:34:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 148274 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:254a:b0:403:3b70:6f57 with SMTP id hf10csp98839vqb; Wed, 4 Oct 2023 05:36:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IECqgDSFhXlSTRfc2ZR0Y/Cw2K1LawYNIZolujw27V2UMzUyytlLpcoCuqWu9pqn0MntcIb X-Received: by 2002:a17:902:8e87:b0:1b2:4852:9a5f with SMTP id bg7-20020a1709028e8700b001b248529a5fmr2028529plb.54.1696423016475; Wed, 04 Oct 2023 05:36:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696423016; cv=none; d=google.com; s=arc-20160816; b=r+fVwFZ/mQ1f1efHb1e9NoBf6nQy7Y5VFEWj/DUFbBoaY7z5ex+YadM0qowWiuNpJT 6SjZekZKcOsOB6QMAivRX/85NlbWPKasd0V1ULww8HY9ZlTdRvUG+8TmnsHgQn75b5mM FKfexpWa1o7AZRLqTZZXcuEkLzkGhjoe2ItpmjQjPLVeo8Dk1ktEnD2VBzV0pgleYdh4 s2MgO204MnoM3YxTFn2BiygTozpGwvbIIJwYdCZ3v3W/M7nqno7n55+qOAM2b004nktN eviNElqE+WrF7qvUki+uJ9znNFwS8o2s0194w5uXdPvxZ2j7rmsFoukHKfnMpbe6EwTO L1tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=QACr81bz714MeCGL3gllUl2Nlnw9ik8CMB4MxkBQEQc=; fh=PG6uS4TiiUSDyl8D/joYkWbwCgDm4ug0ir2h7tHBJXQ=; b=SXphRL4x12k6LofPSm47W1JXNdpELel1mmRyAQM1vyqO/rTHY1Hjh5IrvhRe7ud+ME P409QbdmjXa7GgR9GX0nrdvkbErZu+4x3bXe1S83jj+CoXEvbhZ4yBM2uE1Ro6/YO0SM pROeo9slzGWGxmKTP4BOoso9jaOKDdK3K0If1A78KLV7XV/h1PlNLqEbhFUyJevu/9qW X8JwGRVHPb7FqfZQXbG0FWziEb2r9wJboY9lL7mXa05KDmUtqlF5EpILIwll3j5i58hv dRDNtP3P7Ju9djLJDcy9yVt29V2DD2Yq1qEITaImsPWi9uLuNHsqsAwT1GNN/w8JNw6l sk2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=K8cDLMkh; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id q2-20020a170902e30200b001c724cd1125si3341538plc.203.2023.10.04.05.36.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Oct 2023 05:36:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=K8cDLMkh; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id D169982164EC; Wed, 4 Oct 2023 05:36:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242491AbjJDMgR (ORCPT + 18 others); Wed, 4 Oct 2023 08:36:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242421AbjJDMfp (ORCPT ); Wed, 4 Oct 2023 08:35:45 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55F1E109 for ; Wed, 4 Oct 2023 05:35:38 -0700 (PDT) From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1696422936; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QACr81bz714MeCGL3gllUl2Nlnw9ik8CMB4MxkBQEQc=; b=K8cDLMkhIl9jSNDDBcAcfHcoA/acgRfw443uKotsXCB7uc3qnguuRXlXa5Kqc3BmdkjVuO nnCkb02YomwFh5lpEv33jfrAX4zsDb/UlGY2OdPS7TVV2KcQDNIVArnsfnwQcJshzjdrgf z5KjhSlTMj29E69cROikykyYCAVPz915PKq/J4A6PqHEO6+5+gy6Z6fRq6vTmUn1DpFDf3 yzHYsGwhdu1eCoMorK0lLGPECTRnvluV8lk71vKtW6/Y0KjltbFQjiOHAnARTHwarKxHgC /vo6NqgtHhgx8aAjCO/T3Sey5zJxVvejS389q64glxIudCkMboAC7cYZ0/5Jkw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1696422936; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QACr81bz714MeCGL3gllUl2Nlnw9ik8CMB4MxkBQEQc=; b=RXiLI0RQZzrhT3E/isMxxk+c+AZeO2+fDmy/adcP9ISHbDsdDhqoMABYMgPukXHa7ndD2W DgstDnFJ8iuZL0CQ== To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , John Stultz , Thomas Gleixner , Eric Dumazet , "Rafael J . Wysocki" , Arjan van de Ven , "Paul E . McKenney" , Frederic Weisbecker , Rik van Riel , Steven Rostedt , Sebastian Siewior , Giovanni Gherdovich , Lukasz Luba , "Gautham R . Shenoy" , Srinivas Pandruvada , K Prateek Nayak , Anna-Maria Behnsen Subject: [PATCH v8 19/25] timer: Add get next timer interrupt functionality for remote CPUs Date: Wed, 4 Oct 2023 14:34:48 +0200 Message-Id: <20231004123454.15691-20-anna-maria@linutronix.de> In-Reply-To: <20231004123454.15691-1-anna-maria@linutronix.de> References: <20231004123454.15691-1-anna-maria@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 04 Oct 2023 05:36:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778828461048284917 X-GMAIL-MSGID: 1778828461048284917 To prepare for the conversion of the NOHZ timer placement to a pull at expiry time model it's required to have functionality available getting the next timer interrupt on a remote CPU. Locking of the timer bases and getting the information for the next timer interrupt functionality is split into separate functions. This is required to be compliant with lock ordering when the new model is in place. Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- v8: - Update comment v7: - Move functions into CONFIG_SMP && CONFIG_NO_HZ_COMMON section - change lock, fetch functions to be unconditional - split out unlock function into a separate function v6: - introduce timer_lock_remote_bases() to fix race --- kernel/time/tick-internal.h | 10 +++++ kernel/time/timer.c | 76 ++++++++++++++++++++++++++++++++++--- 2 files changed, 81 insertions(+), 5 deletions(-) diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h index b035606a6f5e..206010ae2a53 100644 --- a/kernel/time/tick-internal.h +++ b/kernel/time/tick-internal.h @@ -8,6 +8,11 @@ #include "timekeeping.h" #include "tick-sched.h" +struct timer_events { + u64 local; + u64 global; +}; + #ifdef CONFIG_GENERIC_CLOCKEVENTS # define TICK_DO_TIMER_NONE -1 @@ -155,6 +160,11 @@ extern unsigned long tick_nohz_active; extern void timers_update_nohz(void); # ifdef CONFIG_SMP extern struct static_key_false timers_migration_enabled; +extern void fetch_next_timer_interrupt_remote(unsigned long basej, u64 basem, + struct timer_events *tevt, + unsigned int cpu); +extern void timer_lock_remote_bases(unsigned int cpu); +extern void timer_unlock_remote_bases(unsigned int cpu); # endif #else /* CONFIG_NO_HZ_COMMON */ static inline void timers_update_nohz(void) { } diff --git a/kernel/time/timer.c b/kernel/time/timer.c index c3061b28214e..ae4b6f62b082 100644 --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -221,11 +221,6 @@ struct timer_base { static DEFINE_PER_CPU(struct timer_base, timer_bases[NR_BASES]); -struct timer_events { - u64 local; - u64 global; -}; - #ifdef CONFIG_NO_HZ_COMMON static DEFINE_STATIC_KEY_FALSE(timers_nohz_active); @@ -2016,6 +2011,77 @@ static unsigned long fetch_next_timer_interrupt(unsigned long basej, u64 basem, return local_first ? nextevt_local : nextevt_global; } +# ifdef CONFIG_SMP +/** + * fetch_next_timer_interrupt_remote + * @basej: base time jiffies + * @basem: base time clock monotonic + * @tevt: Pointer to the storage for the expiry values + * @cpu: Remote CPU + * + * Stores the next pending local and global timer expiry values in the + * struct pointed to by @tevt. If a queue is empty the corresponding + * field is set to KTIME_MAX. If local event expires before global + * event, global event is set to KTIME_MAX as well. + * + * Caller needs to make sure timer base locks are held (use + * timer_lock_remote_bases() for this purpose). + */ +void fetch_next_timer_interrupt_remote(unsigned long basej, u64 basem, + struct timer_events *tevt, + unsigned int cpu) +{ + struct timer_base *base_local, *base_global; + + /* Preset local / global events */ + tevt->local = tevt->global = KTIME_MAX; + + base_local = per_cpu_ptr(&timer_bases[BASE_LOCAL], cpu); + base_global = per_cpu_ptr(&timer_bases[BASE_GLOBAL], cpu); + + lockdep_assert_held(&base_local->lock); + lockdep_assert_held(&base_global->lock); + + fetch_next_timer_interrupt(basej, basem, base_local, base_global, tevt); +} + +/** + * timer_unlock_remote_bases - unlock timer bases of cpu + * @cpu: Remote CPU + * + * Unlocks the remote timer bases. + */ +void timer_unlock_remote_bases(unsigned int cpu) +{ + struct timer_base *base_local, *base_global; + + base_local = per_cpu_ptr(&timer_bases[BASE_LOCAL], cpu); + base_global = per_cpu_ptr(&timer_bases[BASE_GLOBAL], cpu); + + raw_spin_unlock(&base_global->lock); + raw_spin_unlock(&base_local->lock); +} + +/** + * timer_lock_remote_bases - lock timer bases of cpu + * @cpu: Remote CPU + * + * Locks the remote timer bases. + */ +void timer_lock_remote_bases(unsigned int cpu) +{ + struct timer_base *base_local, *base_global; + + base_local = per_cpu_ptr(&timer_bases[BASE_LOCAL], cpu); + base_global = per_cpu_ptr(&timer_bases[BASE_GLOBAL], cpu); + + lockdep_assert_irqs_disabled(); + + raw_spin_lock(&base_local->lock); + raw_spin_lock_nested(&base_global->lock, SINGLE_DEPTH_NESTING); +} +# endif /* CONFIG_SMP */ + static inline unsigned long __get_next_timer_interrupt(unsigned long basej, u64 basem, struct timer_base *base_local, struct timer_base *base_global,