From patchwork Wed Nov 16 16:21:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 21144 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp232705wru; Wed, 16 Nov 2022 08:24:16 -0800 (PST) X-Google-Smtp-Source: AA0mqf6iFilqKbJE5DWHtGCWaQpnp+wicaQPWYQhb1CET2z8WLXr6y86ShbNLAiJRv9csZU9Ducj X-Received: by 2002:a17:90a:5d81:b0:212:cf2e:2b0b with SMTP id t1-20020a17090a5d8100b00212cf2e2b0bmr4459808pji.169.1668615855971; Wed, 16 Nov 2022 08:24:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668615855; cv=none; d=google.com; s=arc-20160816; b=KxKvpVHFMCYzWzknHy5hnfxRznDmZ8X4gOGffCDBAq/IGaSf1HBFK+QIJkzE29LQ3o ZUsNSicssU+8rpgmcrjvmySjl36uaiT8YR2v9KNqbw/VtXDtWYO13pI6OfJSw6UZRtKv R14ctcZDsGmL5+XnFoUA7MQYmmETC4b3MxPeAeZaLrIWUMIGfU7WfjtRnJ2CJZZTALOJ +dEcX4BHDZm/gaK2H1nUQVK3PS6RjDllM/HSBCXWjnApUQVJ+MvtwVhSuX8KL9n79clc ze50ivGxR9riacTOCNyoLsdKQom8vK6FpcAzGkp9dWbXMaUptdxZjJ6FL4BljJiRkdWx fcoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=1B4/BOJeAGf7Dtoezu6zTfoEVRhsIga8Xj9og7bYHLY=; b=Is4bRESJ6pbDMT3pNvHgC5FgkK15Y5IwIJA5ye2cClY96SjHBp5RDDdbG11AQ+WEe1 QvDLo5paI1j9U2gYiMLCHgfPR3sZqRm4XtPQssnUaS47DMDQDqxbuzzkBVxapNNZ8q+h VM2QkNMyFRkJv7DExT9pBMXZm6DoVajJQM3OyDKz5hpKATHFxsAXS/vjeb6mgVdgbaxx gFoIJogJUFp/CQ8t6it8x4OKmb5BQ1TV7P8XjgmuKC44e6ZqJ/tLcmCKsDpoDs22WxQU B+4yeDvnFqqBAChQaU8pFFVv2AVrewoBqFhg9IV5ZuhVuTkeaapwDOP71f2qm2OwGQAT zYXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=NhfDuPfr; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nl7-20020a17090b384700b00213b10313efsi2756983pjb.175.2022.11.16.08.24.02; Wed, 16 Nov 2022 08:24:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=NhfDuPfr; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234377AbiKPQWH (ORCPT + 99 others); Wed, 16 Nov 2022 11:22:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232974AbiKPQWC (ORCPT ); Wed, 16 Nov 2022 11:22:02 -0500 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58E8C419B4 for ; Wed, 16 Nov 2022 08:22:00 -0800 (PST) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1668615719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1B4/BOJeAGf7Dtoezu6zTfoEVRhsIga8Xj9og7bYHLY=; b=NhfDuPfrTQdAhS2yS2TjTsBngwncnpG132e/pQbGvoRRmBnDOd6bOxR3TRPbxCd64WD+gc 3xQQ5Vks76kND/HCXvDvGUtlOZpYJ/40AgsqSP178m+U4sWZN2SFc2Y5KuLaTUmCz5sCnP VGq90QQLbTBq4x9+vi98VoxWnS9wN4SaJxILO79hhZZuNw6PpkRai26p04wqJRi//dlvUH MKZ7p1cRS9ogwmzMbTR1zK2Fb0bKGhjOr8rYiY1yAqYSJd3oMKK53xSOkfVvVG0FQgj0DH 8jLujte6ocwtLVI9fskRZ2XruudSzRWOHVbFNn9Y6j5b5+DvRWVUbuYuxDS3MQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1668615719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1B4/BOJeAGf7Dtoezu6zTfoEVRhsIga8Xj9og7bYHLY=; b=zICGtHOqKGJha+Es2Kka/4nSFRG/vK4Gjiqee7KTkBwn8W7eNWY7vMpyVUShDkLbPp28+5 xyz9HnRcdXiQcEAw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Miguel Ojeda , Greg Kroah-Hartman , "Paul E . McKenney" Subject: [PATCH printk v5 03/40] printk: Prepare for SRCU console list protection Date: Wed, 16 Nov 2022 17:27:15 +0106 Message-Id: <20221116162152.193147-4-john.ogness@linutronix.de> In-Reply-To: <20221116162152.193147-1-john.ogness@linutronix.de> References: <20221116162152.193147-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749670539566454038?= X-GMAIL-MSGID: =?utf-8?q?1749670539566454038?= Provide an NMI-safe SRCU protected variant to walk the console list. Note that all console fields are now set before adding the console to the list to avoid the console becoming visible by SCRU readers before being fully initialized. This is a preparatory change for a new console infrastructure which operates independent of the console BKL. Suggested-by: Thomas Gleixner Signed-off-by: John Ogness Acked-by: Miguel Ojeda Reviewed-by: Paul E. McKenney Reviewed-by: Petr Mladek Tested-by: John Ogness --- .clang-format | 1 + include/linux/console.h | 28 ++++++++++++- kernel/printk/printk.c | 87 ++++++++++++++++++++++++++++++++++------- 3 files changed, 100 insertions(+), 16 deletions(-) diff --git a/.clang-format b/.clang-format index 1247d54f9e49..04a675b56b57 100644 --- a/.clang-format +++ b/.clang-format @@ -222,6 +222,7 @@ ForEachMacros: - 'for_each_component_dais' - 'for_each_component_dais_safe' - 'for_each_console' + - 'for_each_console_srcu' - 'for_each_cpu' - 'for_each_cpu_and' - 'for_each_cpu_not' diff --git a/include/linux/console.h b/include/linux/console.h index 7b5f21f9e469..f4f0c9523835 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -15,7 +15,7 @@ #define _LINUX_CONSOLE_H_ 1 #include -#include +#include #include struct vc_data; @@ -158,8 +158,34 @@ struct console { struct hlist_node node; }; +#ifdef CONFIG_DEBUG_LOCK_ALLOC +extern bool console_srcu_read_lock_is_held(void); +#else +static inline bool console_srcu_read_lock_is_held(void) +{ + return 1; +} +#endif + +extern int console_srcu_read_lock(void); +extern void console_srcu_read_unlock(int cookie); + extern struct hlist_head console_list; +/** + * for_each_console_srcu() - Iterator over registered consoles + * @con: struct console pointer used as loop cursor + * + * Although SRCU guarantees the console list will be consistent, the + * struct console fields may be updated by other CPUs while iterating. + * + * Requires console_srcu_read_lock to be held. Can be invoked from + * any context. + */ +#define for_each_console_srcu(con) \ + hlist_for_each_entry_srcu(con, &console_list, node, \ + console_srcu_read_lock_is_held()) + /* * for_each_console() allows you to iterate on each console */ diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index e6f0832e71f0..173f46a29252 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -85,6 +85,7 @@ EXPORT_SYMBOL(oops_in_progress); static DEFINE_SEMAPHORE(console_sem); HLIST_HEAD(console_list); EXPORT_SYMBOL_GPL(console_list); +DEFINE_STATIC_SRCU(console_srcu); /* * System may need to suppress printk message under certain @@ -104,6 +105,13 @@ static struct lockdep_map console_lock_dep_map = { }; #endif +#ifdef CONFIG_DEBUG_LOCK_ALLOC +bool console_srcu_read_lock_is_held(void) +{ + return srcu_read_lock_held(&console_srcu); +} +#endif + enum devkmsg_log_bits { __DEVKMSG_LOG_BIT_ON = 0, __DEVKMSG_LOG_BIT_OFF, @@ -219,6 +227,32 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write, } #endif /* CONFIG_PRINTK && CONFIG_SYSCTL */ +/** + * console_srcu_read_lock - Register a new reader for the + * SRCU-protected console list + * + * Use for_each_console_srcu() to iterate the console list + * + * Context: Any context. + */ +int console_srcu_read_lock(void) +{ + return srcu_read_lock_nmisafe(&console_srcu); +} +EXPORT_SYMBOL(console_srcu_read_lock); + +/** + * console_srcu_read_unlock - Unregister an old reader from + * the SRCU-protected console list + * + * Counterpart to console_srcu_read_lock() + */ +void console_srcu_read_unlock(int cookie) +{ + srcu_read_unlock_nmisafe(&console_srcu, cookie); +} +EXPORT_SYMBOL(console_srcu_read_unlock); + /* * Helper macros to handle lockdep when locking/unlocking console_sem. We use * macros instead of functions so that _RET_IP_ contains useful information. @@ -2989,6 +3023,14 @@ void console_stop(struct console *console) console_lock(); console->flags &= ~CON_ENABLED; console_unlock(); + + /* + * Ensure that all SRCU list walks have completed. All contexts must + * be able to see that this console is disabled so that (for example) + * the caller can suspend the port without risk of another context + * using the port. + */ + synchronize_srcu(&console_srcu); } EXPORT_SYMBOL(console_stop); @@ -3179,6 +3221,17 @@ void register_console(struct console *newcon) newcon->flags &= ~CON_PRINTBUFFER; } + newcon->dropped = 0; + if (newcon->flags & CON_PRINTBUFFER) { + /* Get a consistent copy of @syslog_seq. */ + mutex_lock(&syslog_lock); + newcon->seq = syslog_seq; + mutex_unlock(&syslog_lock); + } else { + /* Begin with next message. */ + newcon->seq = prb_next_seq(prb); + } + /* * Put this console in the list - keep the * preferred driver at the head of the list. @@ -3187,28 +3240,24 @@ void register_console(struct console *newcon) if (hlist_empty(&console_list)) { /* Ensure CON_CONSDEV is always set for the head. */ newcon->flags |= CON_CONSDEV; - hlist_add_head(&newcon->node, &console_list); + hlist_add_head_rcu(&newcon->node, &console_list); } else if (newcon->flags & CON_CONSDEV) { /* Only the new head can have CON_CONSDEV set. */ console_first()->flags &= ~CON_CONSDEV; - hlist_add_head(&newcon->node, &console_list); + hlist_add_head_rcu(&newcon->node, &console_list); } else { - hlist_add_behind(&newcon->node, console_list.first); - } - - newcon->dropped = 0; - if (newcon->flags & CON_PRINTBUFFER) { - /* Get a consistent copy of @syslog_seq. */ - mutex_lock(&syslog_lock); - newcon->seq = syslog_seq; - mutex_unlock(&syslog_lock); - } else { - /* Begin with next message. */ - newcon->seq = prb_next_seq(prb); + hlist_add_behind_rcu(&newcon->node, console_list.first); } console_unlock(); + + /* + * No need to synchronize SRCU here! The caller does not rely + * on all contexts being able to see the new console before + * register_console() completes. + */ + console_sysfs_notify(); /* @@ -3254,7 +3303,7 @@ int unregister_console(struct console *console) return -ENODEV; } - hlist_del_init(&console->node); + hlist_del_init_rcu(&console->node); /* * @@ -3269,6 +3318,14 @@ int unregister_console(struct console *console) console_first()->flags |= CON_CONSDEV; console_unlock(); + + /* + * Ensure that all SRCU list walks have completed. All contexts + * must not be able to see this console in the list so that any + * exit/cleanup routines can be performed safely. + */ + synchronize_srcu(&console_srcu); + console_sysfs_notify(); if (console->exit)