From patchwork Fri Jul 28 00:02:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127263 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp94563vqg; Thu, 27 Jul 2023 17:21:00 -0700 (PDT) X-Google-Smtp-Source: APBJJlFSNDoj3WgEGqbGxImfm4yJa8imU+uPEWeEOVFijSePS4wHsnd/hrHWhv9Fk6o11WdkfkWm X-Received: by 2002:a05:6a00:c86:b0:682:4c9f:aa0 with SMTP id a6-20020a056a000c8600b006824c9f0aa0mr264058pfv.29.1690503659809; Thu, 27 Jul 2023 17:20:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690503659; cv=none; d=google.com; s=arc-20160816; b=lN3aAkAXinzFMIuIsH4HD/FPONn2DlXUWGBa6CZ5QDoyCwVbU9bnu2Ce+kknJ2YZ01 ZKLHCbmkTBswsUQ8sedMv2DdOln2ZdFid/AonyQXT6TtGA1rX2Q6YCm3iGFF6/kp3hi1 md5yeQxrSA4QLLJ66PZlTqirymcKnrDK6WB/CW4UYh2/X3otFhgDc18rrzLu2QNroHmO RsgDbri5NqSKhsATGl4lCetRZHK954iQjs9CxvoLtnkmI70puftGh90hJ0jxWxcHDLkK +JLJziino/PsBTtcnnEbSvdnBiih2+tum/8ro5rGdgPcXR2t171ORrVFbcbeKqYHfIsC 0f5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=c1xKGpW2D3rUybleKwKmWrTn8UA1EZe2/GxNjEoe5rU=; fh=I/6bvJz6sWq8p1xoNUgGjc7lme65GQ/wcdq821dg0a8=; b=RTqrzcNJiTsvGpew3ZNeqBQQRamvTTc/PXpY9SQDkARNecPxaPYWlo5Z4dM63Y9WJj NEb9+t+2X6X8e8bgiGZl/wPSaW314ZDat0sU4LuVfgTlmsaWeE9+V2a1+ZQUyIxX+yw5 C1r3Ohe7LIi6osGMrWcXnquPJgjY9bbu7NCTdMjrN/HPTnHyuJgme4Qzyie9eM39iE50 HfYGpwi95X/kmI/5XBS4U7bQSUeLnYhlqXLmXJ7ZhuZGaNk6ucPF8rv8GbGyqc40JDH1 pdlfZYHuuLyM9AyqZYwwTSgBz7c0JDudXqyBW3fZbO5+U0UmaUObdrT1qnobwLiJpEbJ R0yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=LrbaGHeU; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u70-20020a638549000000b005578c6a7645si45151pgd.69.2023.07.27.17.20.46; Thu, 27 Jul 2023 17:20:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=LrbaGHeU; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231703AbjG1ACy (ORCPT + 99 others); Thu, 27 Jul 2023 20:02:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231387AbjG1ACn (ORCPT ); Thu, 27 Jul 2023 20:02:43 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C85B530FF for ; Thu, 27 Jul 2023 17:02:39 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c1xKGpW2D3rUybleKwKmWrTn8UA1EZe2/GxNjEoe5rU=; b=LrbaGHeULndIM49PvMkxcA3aQEuZNE32au8Q4SIbnc3w7FlAkv0aTqzPV+ytJzhFy5E9Zo LEf9d5kC+QMV7NLMxi+pgRCVmyHKe+A6znNzNw++OLHlFiR4DHSNMXD6/SUacygfQmS+pX GgNdSl4W90SWr4d46wuBfB4AKsiAoJfn/BSkyoq/XC/0WJNZh497Gjwm3JxX70UcLlawXz Zv2QxsDXSh0Wn1n/kkarDBWqaMZc6WYHC31VrBJaoU1Bi7ajfd8TyemcQxXWY+TEqlAiCq NEl5uCCkhZxW1OIHGQWNpJa3noich4sFxAEhgmZjmg2WKXPHYvHDh/EN32Q2pw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c1xKGpW2D3rUybleKwKmWrTn8UA1EZe2/GxNjEoe5rU=; b=6P4thcNewSQwvuy8MKkSsZTyY12+vYnGWqzgzN+YkpVsVOPS+UOQ8GqpTHNQcEYqNIAws+ TEg8ABQzzSBkHtAw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 1/8] printk: Add non-BKL (nbcon) console basic infrastructure Date: Fri, 28 Jul 2023 02:08:26 +0206 Message-Id: <20230728000233.50887-2-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772621565675621830 X-GMAIL-MSGID: 1772621565675621830 From: Thomas Gleixner The current console/printk subsystem is protected by a Big Kernel Lock, (aka console_lock) which has ill defined semantics and is more or less stateless. This puts severe limitations on the console subsystem and makes forced takeover and output in emergency and panic situations a fragile endeavour that is based on try and pray. The goal of non-BKL (nbcon) consoles is to break out of the console lock jail and to provide a new infrastructure that avoids the pitfalls and allows console drivers to be gradually converted over. The proposed infrastructure aims for the following properties: - Per console locking instead of global locking - Per console state that allows to make informed decisions - Stateful handover and takeover As a first step, state is added to struct console. The per console state is an atomic_t using a 32bit bit field. Reserve state bits, which will be populated later in the series. Wire it up into the console register/unregister functionality and exclude such consoles from being handled in the legacy console mechanisms. Since the nbcon consoles will not depend on the console lock/unlock dance for printing, only perform said dance if a legacy console is registered. The decision to use a bitfield was made as using a plain u32 with mask/shift operations turned out to result in uncomprehensible code. Note that nbcon consoles are not able to print simultaneously with boot consoles because it is not possible to know if they are using the same hardware. For this reason, nbcon consoles are handled as legacy consoles as long as a boot console is registered. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) --- include/linux/console.h | 31 ++++++++++ kernel/printk/Makefile | 2 +- kernel/printk/internal.h | 11 ++++ kernel/printk/printk.c | 112 +++++++++++++++++++++++++++++++---- kernel/printk/printk_nbcon.c | 74 +++++++++++++++++++++++ 5 files changed, 216 insertions(+), 14 deletions(-) create mode 100644 kernel/printk/printk_nbcon.c diff --git a/include/linux/console.h b/include/linux/console.h index 7de11c763eb3..c99265d82b98 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -156,6 +156,8 @@ static inline int con_debug_leave(void) * /dev/kmesg which requires a larger output buffer. * @CON_SUSPENDED: Indicates if a console is suspended. If true, the * printing callbacks must not be called. + * @CON_NBCON: Console can operate outside of the legacy style console_lock + * constraints. */ enum cons_flags { CON_PRINTBUFFER = BIT(0), @@ -166,8 +168,32 @@ enum cons_flags { CON_BRL = BIT(5), CON_EXTENDED = BIT(6), CON_SUSPENDED = BIT(7), + CON_NBCON = BIT(8), }; +/** + * struct nbcon_state - console state for nbcon consoles + * @atom: Compound of the state fields for atomic operations + * + * To be used for reading and preparing of the value stored in the nbcon + * state variable @console.nbcon_state. + */ +struct nbcon_state { + union { + unsigned int atom; + struct { + }; + }; +}; + +/* + * The nbcon_state struct is used to easily create and interpret values that + * are stored in the console.nbcon_state variable. Make sure this struct stays + * within the size boundaries of that atomic variable's underlying type in + * order to avoid any accidental truncation. + */ +static_assert(sizeof(struct nbcon_state) <= sizeof(int)); + /** * struct console - The console descriptor structure * @name: The name of the console driver @@ -187,6 +213,8 @@ enum cons_flags { * @dropped: Number of unreported dropped ringbuffer records * @data: Driver private data * @node: hlist node for the console list + * + * @nbcon_state: State for nbcon consoles */ struct console { char name[16]; @@ -206,6 +234,9 @@ struct console { unsigned long dropped; void *data; struct hlist_node node; + + /* nbcon console specific members */ + atomic_t __private nbcon_state; }; #ifdef CONFIG_LOCKDEP diff --git a/kernel/printk/Makefile b/kernel/printk/Makefile index f5b388e810b9..552525edf562 100644 --- a/kernel/printk/Makefile +++ b/kernel/printk/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y = printk.o -obj-$(CONFIG_PRINTK) += printk_safe.o +obj-$(CONFIG_PRINTK) += printk_safe.o printk_nbcon.o obj-$(CONFIG_A11Y_BRAILLE_CONSOLE) += braille.o obj-$(CONFIG_PRINTK_INDEX) += index.o diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h index 7d4979d5c3ce..655810f2976e 100644 --- a/kernel/printk/internal.h +++ b/kernel/printk/internal.h @@ -3,6 +3,7 @@ * internal.h - printk internal definitions */ #include +#include #if defined(CONFIG_PRINTK) && defined(CONFIG_SYSCTL) void __init printk_sysctl_init(void); @@ -35,6 +36,9 @@ enum printk_info_flags { LOG_CONT = 8, /* text is a fragment of a continuation line */ }; +extern bool have_legacy_console; +extern bool have_boot_console; + __printf(4, 0) int vprintk_store(int facility, int level, const struct dev_printk_info *dev_info, @@ -61,6 +65,10 @@ void defer_console_output(void); u16 printk_parse_prefix(const char *text, int *level, enum printk_info_flags *flags); + +bool nbcon_init(struct console *con); +void nbcon_cleanup(struct console *con); + #else #define PRINTK_PREFIX_MAX 0 @@ -76,6 +84,9 @@ u16 printk_parse_prefix(const char *text, int *level, #define printk_safe_exit_irqrestore(flags) local_irq_restore(flags) static inline bool printk_percpu_data_ready(void) { return false; } +static inline bool nbcon_init(struct console *con) { return true; } +static inline void nbcon_cleanup(struct console *con) { } + #endif /* CONFIG_PRINTK */ /** diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 8787d3a72114..98b4854c81ea 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -442,6 +442,26 @@ static int console_msg_format = MSG_FORMAT_DEFAULT; /* syslog_lock protects syslog_* variables and write access to clear_seq. */ static DEFINE_MUTEX(syslog_lock); +/* + * Specifies if a legacy console is registered. See serialized_printing + * for details. + */ +bool have_legacy_console; + +/* + * Specifies if a boot console is registered. See serialized_printing + * for details. + */ +bool have_boot_console; + +/* + * Specifies if the console lock/unlock dance is needed for console + * printing. If @have_boot_console is true, the nbcon consoles will + * be printed serially along with the legacy consoles because nbcon + * consoles cannot print simultaneously with boot consoles. + */ +#define serialized_printing (have_legacy_console || have_boot_console) + #ifdef CONFIG_PRINTK DECLARE_WAIT_QUEUE_HEAD(log_wait); /* All 3 protected by @syslog_lock. */ @@ -2286,7 +2306,7 @@ asmlinkage int vprintk_emit(int facility, int level, printed_len = vprintk_store(facility, level, dev_info, fmt, args); /* If called from the scheduler, we can not call up(). */ - if (!in_sched) { + if (!in_sched && serialized_printing) { /* * The caller may be holding system-critical or * timing-sensitive locks. Disable preemption during @@ -2603,7 +2623,7 @@ void resume_console(void) */ static int console_cpu_notify(unsigned int cpu) { - if (!cpuhp_tasks_frozen) { + if (!cpuhp_tasks_frozen && serialized_printing) { /* If trylock fails, someone else is doing the printing */ if (console_trylock()) console_unlock(); @@ -2955,8 +2975,17 @@ static bool console_flush_all(bool do_cond_resched, u64 *next_seq, bool *handove cookie = console_srcu_read_lock(); for_each_console_srcu(con) { + short flags = console_srcu_read_flags(con); bool progress; + /* + * console_flush_all() is only for legacy consoles, + * unless a boot console is registered. See + * serialized_printing for details. + */ + if ((flags & CON_NBCON) && !have_boot_console) + continue; + if (!console_is_usable(con)) continue; any_usable = true; @@ -3075,6 +3104,9 @@ void console_unblank(void) struct console *c; int cookie; + if (!serialized_printing) + return; + /* * First check if there are any consoles implementing the unblank() * callback. If not, there is no reason to continue and take the @@ -3142,6 +3174,9 @@ void console_flush_on_panic(enum con_flush_mode mode) bool handover; u64 next_seq; + if (!serialized_printing) + return; + /* * Ignore the console lock and flush out the messages. Attempting a * trylock would not be useful because: @@ -3324,9 +3359,10 @@ static void try_enable_default_console(struct console *newcon) newcon->flags |= CON_CONSDEV; } -#define con_printk(lvl, con, fmt, ...) \ - printk(lvl pr_fmt("%sconsole [%s%d] " fmt), \ - (con->flags & CON_BOOT) ? "boot" : "", \ +#define con_printk(lvl, con, fmt, ...) \ + printk(lvl pr_fmt("%s%sconsole [%s%d] " fmt), \ + (con->flags & CON_NBCON) ? "" : "legacy ", \ + (con->flags & CON_BOOT) ? "boot" : "", \ con->name, con->index, ##__VA_ARGS__) static void console_init_seq(struct console *newcon, bool bootcon_registered) @@ -3486,6 +3522,15 @@ void register_console(struct console *newcon) newcon->dropped = 0; console_init_seq(newcon, bootcon_registered); + if (!(newcon->flags & CON_NBCON)) { + have_legacy_console = true; + } else if (!nbcon_init(newcon)) { + goto unlock; + } + + if (newcon->flags & CON_BOOT) + have_boot_console = true; + /* * Put this console in the list - keep the * preferred driver at the head of the list. @@ -3538,6 +3583,9 @@ EXPORT_SYMBOL(register_console); /* Must be called under console_list_lock(). */ static int unregister_console_locked(struct console *console) { + bool is_legacy_con = !(console->flags & CON_NBCON); + bool is_boot_con = (console->flags & CON_BOOT); + struct console *c; int res; lockdep_assert_console_list_lock_held(); @@ -3577,11 +3625,34 @@ static int unregister_console_locked(struct console *console) */ synchronize_srcu(&console_srcu); + if (console->flags & CON_NBCON) + nbcon_cleanup(console); + console_sysfs_notify(); if (console->exit) res = console->exit(console); + /* + * If the current console was a boot and/or legacy console, the + * related global flags might need to be updated. + */ + if (is_boot_con || is_legacy_con) { + bool found_boot_con = false; + bool found_legacy_con = false; + + for_each_console(c) { + if (c->flags & CON_BOOT) + found_boot_con = true; + if (!(c->flags & CON_NBCON)) + found_legacy_con = true; + } + if (!found_boot_con) + have_boot_console = false; + if (!found_legacy_con) + have_legacy_console = false; + } + return res; } @@ -3730,6 +3801,7 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre struct console *c; u64 last_diff = 0; u64 printk_seq; + bool locked; int cookie; u64 diff; u64 seq; @@ -3739,13 +3811,17 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre seq = prb_next_seq(prb); for (;;) { + locked = false; diff = 0; - /* - * Hold the console_lock to guarantee safe access to - * console->seq. - */ - console_lock(); + if (serialized_printing) { + /* + * Hold the console_lock to guarantee safe access to + * console->seq. + */ + console_lock(); + locked = true; + } cookie = console_srcu_read_lock(); for_each_console_srcu(c) { @@ -3758,7 +3834,12 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre */ if (!console_is_usable(c)) continue; - printk_seq = c->seq; + + if (locked) + printk_seq = c->seq; + else + continue; + if (printk_seq < seq) diff += seq - printk_seq; } @@ -3767,7 +3848,8 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre if (diff != last_diff && reset_on_progress) remaining = timeout_ms; - console_unlock(); + if (locked) + console_unlock(); /* Note: @diff is 0 if there are no usable consoles. */ if (diff == 0 || remaining == 0) @@ -3893,7 +3975,11 @@ void defer_console_output(void) * New messages may have been added directly to the ringbuffer * using vprintk_store(), so wake any waiters as well. */ - __wake_up_klogd(PRINTK_PENDING_WAKEUP | PRINTK_PENDING_OUTPUT); + int val = PRINTK_PENDING_WAKEUP; + + if (serialized_printing) + val |= PRINTK_PENDING_OUTPUT; + __wake_up_klogd(val); } void printk_trigger_flush(void) diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c new file mode 100644 index 000000000000..bb379a4f6263 --- /dev/null +++ b/kernel/printk/printk_nbcon.c @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright (C) 2022 Linutronix GmbH, John Ogness +// Copyright (C) 2022 Intel, Thomas Gleixner + +#include +#include +#include "internal.h" +/* + * Printk console printing implementation for consoles that do not depend on + * the legacy style console_lock mechanism. + */ + +/** + * nbcon_state_set - Helper function to set the console state + * @con: Console to update + * @new: The new state to write + * + * Only to be used when the console is not yet or no longer visible in the + * system. Otherwise use nbcon_state_try_cmpxchg(). + */ +static inline void nbcon_state_set(struct console *con, struct nbcon_state *new) +{ + atomic_set(&ACCESS_PRIVATE(con, nbcon_state), new->atom); +} + +/** + * nbcon_state_read - Helper function to read the console state + * @con: Console to read + * @state: The state to store the result + */ +static inline void nbcon_state_read(struct console *con, struct nbcon_state *state) +{ + state->atom = atomic_read(&ACCESS_PRIVATE(con, nbcon_state)); +} + +/** + * nbcon_state_try_cmpxchg() - Helper function for atomic_try_cmpxchg() on console state + * @con: Console to update + * @cur: Old/expected state + * @new: New state + * + * Return: True on success. False on fail and @cur is updated. + */ +static inline bool nbcon_state_try_cmpxchg(struct console *con, struct nbcon_state *cur, + struct nbcon_state *new) +{ + return atomic_try_cmpxchg(&ACCESS_PRIVATE(con, nbcon_state), &cur->atom, new->atom); +} + +/** + * nbcon_init - Initialize the nbcon console specific data + * @con: Console to initialize + * + * Return: True on success. False otherwise and the console cannot + * be used. + */ +bool nbcon_init(struct console *con) +{ + struct nbcon_state state = { }; + + nbcon_state_set(con, &state); + return true; +} + +/** + * nbcon_cleanup - Cleanup the nbcon console specific data + * @con: Console to cleanup + */ +void nbcon_cleanup(struct console *con) +{ + struct nbcon_state state = { }; + + nbcon_state_set(con, &state); +} From patchwork Fri Jul 28 00:02:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127258 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp88173vqg; Thu, 27 Jul 2023 17:06:44 -0700 (PDT) X-Google-Smtp-Source: APBJJlHEzqwjzr1JLPqz0xY1HC+ASFl+6wG1adpFEwUO+RT/KzmgtJp6fZxowXVpMvdgsUt2MCKH X-Received: by 2002:a05:6512:3b13:b0:4fe:df2:b82d with SMTP id f19-20020a0565123b1300b004fe0df2b82dmr604479lfv.37.1690502803787; Thu, 27 Jul 2023 17:06:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690502803; cv=none; d=google.com; s=arc-20160816; b=xL8MTe9OVVUQW+cW6sUI0ek34Z+wVA6ieSTRnLhh2mPbxNorJ/mcXriFR1/Hb6LwL8 AaUQXJUVmU7wJAvnir/Ogvg4w1O7NckIkeaITOgjWH6uTQRx5UHoTYz2Dgxst1g3Dr9Q FaqQGNkXWGSQ24YpoVoMYDAxHmNxRYSisyToWnl14DJpAT4gFZQujwrVsLUEQlHHLkNH TY7gw1Y4TKnO0ZwnTpqFQ7E1I4V5Odt8DYA4QWeyvHdvOxuvocouykzFl3H9OXmX0mSi PF3rhXonZGf8xOx+1uItBxgUY2C3/oSDlzuA6hyPwURPp/hGA/KbWgM0mwu5hzMCo7hX GFeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=aN/PZD88PlKxnnAWKpPNKfderKHCjvFqWgQcXb3T0GI=; fh=HVRBJStvoKWbFco6Jta18xe5UAVCwWVs9OvJstciom4=; b=g8xArTpqu1lNREH4h9fhR289FweL0/+fB/xIpZArEjUZQPm4A3GOvxVzidGGJBLw01 0CZC7JLdtRYrizNK8jVqTeiRpZ+UCNvPyaFI5CFQmTagO0ffwqn+TKEsDJo48lzIwolC 0PoIDFr+cyOfDJWiCanep/8ONtrY7JzsYucEYMDBkaGnJpjHmWRaSvR/9Ej52IfutMi7 JTsBkKZQF8PII9hN3wlWgW9IBmJG8Y5dZ7AGoU10TCpAfaND+B0H7+L6Na8iWreafsUO PuzCLWYoE1GOWVKhuWTSi51LHssGv4OsmLfeb/htejyibhsZAs1bnNnK0OjadheFdWvV 9MsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=JdVIc2tn; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=zyrFz+Na; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jo16-20020a170906f6d000b0099bd6fbd6b9si1805696ejb.433.2023.07.27.17.06.20; Thu, 27 Jul 2023 17:06:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=JdVIc2tn; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=zyrFz+Na; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231236AbjG1ACn (ORCPT + 99 others); Thu, 27 Jul 2023 20:02:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229687AbjG1ACl (ORCPT ); Thu, 27 Jul 2023 20:02:41 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6A2F30FA for ; Thu, 27 Jul 2023 17:02:39 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aN/PZD88PlKxnnAWKpPNKfderKHCjvFqWgQcXb3T0GI=; b=JdVIc2tnGLddF4guSyv+W1fRWkdVCzJxI3lOUsfk5IO3LFAb/v2SVjWVzw7JGpk4hkEbgR YMxEkV4bEiy2n9gKVKEyALgwLEEjaI6ldWVsfX/vp0OMn44s5uXmlukHcvmimI0yAIw+JN wyXxDa8PwCD5Nie2M85cJLQ9EavtaAVrCxMjQXd6z3k+LOP0EcA8r38BlZVecCk60uKxJu 6jJMiLjQNV1T9gGOSbzoT+wQEh4bJe+6h08Z9lqMp0Os/+nHuH6C4Iz3rvbasLuHnE8Tnv oa3GZq8kiwOoaBRWMtQzYqbYq8V1Fcumb29YD8y1mIPed5oUcZxma6SUZz4nJQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aN/PZD88PlKxnnAWKpPNKfderKHCjvFqWgQcXb3T0GI=; b=zyrFz+NafiJs7zKAN8tZTy+t4GuTNq+lbyyRYH2GMWuzRAMbvkORDmdJiXI2PriwTnEa1z zyqU+VlP3PRz4yBA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 2/8] printk: Provide debug_store() for nbcon debugging Date: Fri, 28 Jul 2023 02:08:27 +0206 Message-Id: <20230728000233.50887-3-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772620667685057343 X-GMAIL-MSGID: 1772620667685057343 To debug and validate nbcon ownership transitions, provide a new macro debug_store() (enabled with DEBUG_NBCON) to log transition information to the ringbuffer. If enabled, this macro only logs to the ringbuffer and does not trigger any printing. This is to avoid triggering recursive printing in the nbcon consoles. Signed-off-by: John Ogness --- kernel/printk/printk_nbcon.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index bb379a4f6263..f9462b088439 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -10,6 +10,35 @@ * the legacy style console_lock mechanism. */ +/* + * Define DEBUG_NBCON to allow for nbcon ownership transitions to be logged + * to the ringbuffer. The debug_store() macro only logs to the lockless + * ringbuffer and does not trigger any printing. + */ +#undef DEBUG_NBCON + +#ifdef DEBUG_NBCON +/* Only write to ringbuffer. */ +int __debug_store(const char *fmt, ...) +{ + va_list args; + int r; + + va_start(args, fmt); + r = vprintk_store(2, 7, NULL, fmt, args); + va_end(args); + + return r; +} +#define debug_store(cond, fmt, ...) \ + do { \ + if (cond) \ + __debug_store(pr_fmt("DEBUG: " fmt), ##__VA_ARGS__) \ + } while (0) +#else +#define debug_store(cond, fmt, ...) +#endif + /** * nbcon_state_set - Helper function to set the console state * @con: Console to update From patchwork Fri Jul 28 00:02:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127260 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp89126vqg; Thu, 27 Jul 2023 17:08:47 -0700 (PDT) X-Google-Smtp-Source: APBJJlE0J8i9UHMLafiO6M7VJp143zvnX91j/AN3kiucFeCue9D3yVlH6D5ft80HFliCR+JocLE2 X-Received: by 2002:a05:6830:19:b0:6bb:1036:67f2 with SMTP id c25-20020a056830001900b006bb103667f2mr755964otp.6.1690502926950; Thu, 27 Jul 2023 17:08:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690502926; cv=none; d=google.com; s=arc-20160816; b=OjagVW3lh45BIvl5pU1IeD3DXioCocLxD95hziEDzq0ZhVZbWhBdRHECI509BNOfos Oei9QIl3QCmixJwFgbtIzPoGm2xo6YwHXw/9zwLMeeA7Ry/vCQcL5vXH0pXTwh0n6DFA hcxGJVKrn6kFOE7DD7Lu5RwqtfugcoJobrRBV5A5SEv6S6BYiZeUPCTB5heq883CDSgn BMY31wlAHVLhjNmCVDXBUXzSW7QqO8RYD9LcOOfyRMjnRcXMIK3AR9C/vC9fzpc13ZQl HCBsKQ8sjM/ujO29+sFJaVboaMkWtrrGJlXvGCV5Cbhq+mcAi4FsKA/rNHNDil0W+dNl CPKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=+OBt8R0S5ZLGTYLHZc+xPyD6wpLwP3wXA8ac52ked50=; fh=I/6bvJz6sWq8p1xoNUgGjc7lme65GQ/wcdq821dg0a8=; b=reWed4OB7dHrm/goHO+im9hf0XIztKqvuim2nFHgI8G75bi3nUU/5+DtYHpZ5+kjHP w99WPyUMpBKAjplk1MRK6STj8tzb3PBChdy7ITBCihxYH29jnCIj4n81M8Fr4qtdDsQ8 jcZ4qcs5/74vpzDBHChmOnfRsi1X3GtFytiMaqhxRf3UVm78kioTuPs5G7aZ7i32zvCh epSdlU52wexeUzsKKt+NMAtnIbKsS5oU6qoZ9nbRN8hj4seRw1ynSO/wz/Mwo0bcnQr6 Y6xHtFD2J9a1Z8dC9QXgECCrY52gm+ColBLUXQEZ8GCYrL996b11VwF69CwZRbXxbrs9 2LtA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=mX5yNNHU; dkim=neutral (no key) header.i=@linutronix.de header.b=pGf+JUzT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s3-20020a637703000000b005641697e82bsi538316pgc.435.2023.07.27.17.08.33; Thu, 27 Jul 2023 17:08:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=mX5yNNHU; dkim=neutral (no key) header.i=@linutronix.de header.b=pGf+JUzT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231387AbjG1AC6 (ORCPT + 99 others); Thu, 27 Jul 2023 20:02:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229687AbjG1ACn (ORCPT ); Thu, 27 Jul 2023 20:02:43 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6DE830FC for ; Thu, 27 Jul 2023 17:02:39 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+OBt8R0S5ZLGTYLHZc+xPyD6wpLwP3wXA8ac52ked50=; b=mX5yNNHU8v6EAiyTOmgHWzRkWgxRUWgDI5VDwmPGwyhGjtbQBUI6dGC3MBO1WTAx/ItqFB X+l+vZa4xvY5xY8MaQj/PGxT6nbtzG6Nn715N5CaRWkbEOI62xaORUiagU92sbRnfWK2N9 SDErw8KCD1LpQQlMpu6ZnOm9AoR49N2ZzYPe7/6XeckBsBAzik1yF986jFfmy22MZEU5YQ 53rOygmsnOvcjSlfN/xtB7/kjFwPMrJihEpY3uTJ/rtsF/xynd7gVBIpbkgtn7aL/+CRgi mLEK1sJnwLnTgJeYq+K4HT9t83f7ch5vswBn5yqt/MwiNO0/P2AgxNjBp6S+vA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+OBt8R0S5ZLGTYLHZc+xPyD6wpLwP3wXA8ac52ked50=; b=pGf+JUzTPnBX2GgRyBKVgZhISfdZSTZ2Jfv1PgZQ6Quvaikxjm8wcpC68szvM/X6G6RcXu q3M17jVIG1sE3hDQ== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 3/8] printk: nbcon: Add acquire/release logic Date: Fri, 28 Jul 2023 02:08:28 +0206 Message-Id: <20230728000233.50887-4-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772620797261077769 X-GMAIL-MSGID: 1772620797261077769 From: Thomas Gleixner Add per console acquire/release functionality. The console 'locked' state is a combination of multiple state fields: - The 'prio' field contains the severity of the context that owns the console. This field is used for decisions whether to attempt friendly handovers and also prevents takeovers from a less severe context, e.g. to protect the panic CPU. A value of 0 (NBCON_PRIO_NONE) means the console is not locked. - The 'cpu' field denotes on which CPU the console is locked. The acquire mechanism comes with several flavours: - Straight forward acquire when the console is not contended. - Friendly handover mechanism based on a request/grant handshake. The requesting context: 1) Sets its priority into the 'req_prio' field. 2) Waits (with a timeout) for the owning context to unlock the console. 3) Sets the 'prio' field and clears the 'req_prio' field. The owning context: 1) Observes the 'req_prio' field set. 2) Gives up console ownership by clearing the 'prio' field. - Hostile takeover The new owner takes the console over without 'req_prio' handshake. This is required when friendly handovers are not possible, i.e. the higher priority context interrupted the owning context on the same CPU or the owning context is not able to make progress on a remote CPU. The release is the counterpart which either releases the console directly or yields it gracefully over to a requester. All operations on console::nbcon_state are atomic cmpxchg based to handle concurrency. The acquire/release functions implement only minimal policies: - Preference for higher priority contexts. - Protection of the panic CPU. All other policy decisions have to be made at the call sites: - What is marked as an unsafe section. - Whether to spinwait if there is already an owner. - Whether to attempt a hostile takeover when safe. - Whether to attempt a hostile takeover when unsafe. The design allows to implement the well known: acquire() output_one_line() release() algorithm, but also allows to avoid the per line acquire/release for e.g. panic situations by doing the acquire once and then relying on the panic CPU protection for the rest. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) --- include/linux/console.h | 59 +++++ kernel/printk/printk_nbcon.c | 447 +++++++++++++++++++++++++++++++++++ 2 files changed, 506 insertions(+) diff --git a/include/linux/console.h b/include/linux/console.h index c99265d82b98..e06cd1ce3e82 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -175,13 +175,28 @@ enum cons_flags { * struct nbcon_state - console state for nbcon consoles * @atom: Compound of the state fields for atomic operations * + * @req_prio: The priority of a handover request + * @prio: The priority of the current usage + * @unsafe: Console is busy in a non takeover region + * @hostile_unsafe: The @unsafe value before a hostile takeover + * @cpu: The CPU on which the owner runs + * * To be used for reading and preparing of the value stored in the nbcon * state variable @console.nbcon_state. + * + * The @prio and @req_prio fields are particularly important to allow + * spin-waiting to timeout and give up without the risk of a waiter being + * assigned the lock after giving up. */ struct nbcon_state { union { unsigned int atom; struct { + unsigned int prio : 2; + unsigned int req_prio : 2; + unsigned int unsafe : 1; + unsigned int hostile_unsafe : 1; + unsigned int cpu : 24; }; }; }; @@ -194,6 +209,50 @@ struct nbcon_state { */ static_assert(sizeof(struct nbcon_state) <= sizeof(int)); +/** + * nbcon_prio - console owner priority for nbcon consoles + * @NBCON_PRIO_NONE: Unused + * @NBCON_PRIO_NORMAL: Normal (non-emergency) usage + * @NBCON_PRIO_EMERGENCY: Emergency output (WARN/OOPS...) + * @NBCON_PRIO_PANIC: Panic output + * @NBCON_PRIO_MAX: The number of priority levels + * + * A context wanting to produce emergency output can carefully takeover the + * console, even without consent of the owner. Ideally such a takeover is only + * when @nbcon_state::unsafe is not set. However, a context wanting to produce + * panic output can ignore the unsafe flag as a last resort. If panic output + * is active, no takeover is possible until the panic output releases the + * console. + */ +enum nbcon_prio { + NBCON_PRIO_NONE = 0, + NBCON_PRIO_NORMAL, + NBCON_PRIO_EMERGENCY, + NBCON_PRIO_PANIC, + NBCON_PRIO_MAX, +}; + +struct console; + +/** + * struct nbcon_context - Context for console acquire/release + * @console: The associated console + * @spinwait_max_us: Limit for spinwait acquire + * @prio: Priority of the context + * @unsafe: This context is in an unsafe section + * @hostile: Acquire console by hostile takeover + * @takeover_unsafe: Acquire console by hostile takeover even if unsafe + */ +struct nbcon_context { + /* members set by caller */ + struct console *console; + unsigned int spinwait_max_us; + enum nbcon_prio prio; + unsigned int unsafe : 1; + unsigned int hostile : 1; + unsigned int takeover_unsafe : 1; +}; + /** * struct console - The console descriptor structure * @name: The name of the console driver diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index f9462b088439..b0acde0cb949 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -4,10 +4,38 @@ #include #include +#include #include "internal.h" /* * Printk console printing implementation for consoles that do not depend on * the legacy style console_lock mechanism. + * + * Console is locked on a CPU when @nbcon_state::prio is set and + * @nbcon_state:cpu == current CPU. This is valid for the current execution + * context. + * + * Nesting execution contexts on the same CPU can carefully take over if + * the driver allows reentrancy via nbcon_state::unsafe = false. When the + * interrupted context resumes it checks the state before entering an unsafe + * region and aborts the operation if it detects a takeover. + * + * In case of panic the nesting context can take over the console forcefully. + * + * A concurrent writer on a different CPU with a higher priority can request + * to take over the console by: + * + * 1) Carefully writing the desired priority into @nbcon_state::req_prio + * if there is no higher priority request pending. + * + * 2) Carefully spin on @nbcon_state::prio until it is no longer locked. + * + * 3) Attempt to lock the console by carefully writing the desired + * priority to @nbcon_state::prio and carefully removing the desired + * priority from @nbcon_state::req_prio. + * + * In case that the owner does not react on the request and does not make + * observable progress, the waiter will timeout and can then decide to do + * a hostile takeover. */ /* @@ -76,6 +104,425 @@ static inline bool nbcon_state_try_cmpxchg(struct console *con, struct nbcon_sta return atomic_try_cmpxchg(&ACCESS_PRIVATE(con, nbcon_state), &cur->atom, new->atom); } +/** + * nbcon_context_try_acquire_direct - Try to acquire directly + * @ctxt: The context of the caller + * @cur: The current console state + * + * Return: 0 on success and @cur is updated to the new console state. + * Otherwise an error code on failure. + * + * Errors: + * + * -EPERM: A panic is in progress an this is not the panic CPU. + * Retrying the acquire will not be immediately useful. + * + * -EBUSY: The console already has an owner. Retrying the + * acquire will not be immediately useful. + * + * -EAGAIN: An unexpected state change occurred. @cur has been + * updated. The caller should retry the acquire. + * + * The general procedure is to change @prio from unowned to owned. + */ +static int nbcon_context_try_acquire_direct(struct nbcon_context *ctxt, + struct nbcon_state *cur) +{ + unsigned int cpu = smp_processor_id(); + struct console *con = ctxt->console; + struct nbcon_state new = { + .cpu = cpu, + .prio = ctxt->prio, + .unsafe = cur->hostile_unsafe | ctxt->unsafe, + .hostile_unsafe = cur->hostile_unsafe, + }; + + if (other_cpu_in_panic()) + return -EPERM; + + /* + * Direct not possible if owner in unsafe region. If there + * was previously a hostile takeover, the console may be left + * in an unsafe state, even if it is now unowned. + */ + if (cur->unsafe) + return -EBUSY; + + /* Direct is only possible if it is unowned. */ + if (cur->prio != NBCON_PRIO_NONE) + return -EBUSY; + + /* Higher priority waiters are allowed to keep waiting. */ + if (cur->req_prio > ctxt->prio) + new.req_prio = cur->req_prio; + else + new.req_prio = 0; + + /* All looks good. Try to take ownership. */ + + if (!nbcon_state_try_cmpxchg(con, cur, &new)) + return -EAGAIN; + + cur->atom = new.atom; + + return 0; +} + +/** + * nbcon_context_try_acquire_handover - Try to acquire via handover + * @ctxt: The context of the caller + * @cur: The current console state + * + * Return: 0 on success and @cur is updated to the new console state. + * Otherwise an error code on failure. + * + * Errors: + * + * -EPERM: A panic is in progress an this is not the panic CPU. + * Retrying the acquire will not be immediately useful. + * + * -EBUSY: The current owner or waiter is such that this context + * is not able to execute a handover. Retrying the + * acquire will not be immediately useful. + * + * -EAGAIN: An unexpected state change occurred. @cur has been + * updated. The caller should retry the acquire. + * + * The general procedure is to set @req_prio and wait until unowned. Then + * set @prio (claiming ownership) and clearing @req_prio. + */ +static int nbcon_context_try_acquire_handover(struct nbcon_context *ctxt, + struct nbcon_state *cur) +{ + unsigned int cpu = smp_processor_id(); + struct console *con = ctxt->console; + struct nbcon_state new; + int timeout; + int err = 0; + + /* Cannot request handover if owner has same or higher priority. */ + if (cur->prio >= ctxt->prio) + return -EBUSY; + + /* Cannot request handover if a waiter has same or higher priority. */ + if (cur->req_prio >= ctxt->prio) + return -EBUSY; + + /* + * If there is no owner, a handover may be in progress and this + * context (having a higher priority) could jump in directly. + */ + if (cur->prio == NBCON_PRIO_NONE) + return nbcon_context_try_acquire_direct(ctxt, cur); + + /* Cannot handover on same CPU. */ + if (cur->cpu == cpu) + return -EBUSY; + + /* Cannot request handover if caller unwilling to wait. */ + if (ctxt->spinwait_max_us == 0) + return -EBUSY; + + /* Set request for handover. */ + new.atom = cur->atom; + new.req_prio = ctxt->prio; + if (!nbcon_state_try_cmpxchg(con, cur, &new)) + return -EAGAIN; + + cur->atom = new.atom; + + debug_store(1, "handover: cpu%d SPINNING to acquire for prio%d\n", cpu, ctxt->prio); + + /* Wait until there is no owner and then acquire directly. */ + for (timeout = ctxt->spinwait_max_us; timeout >= 0; timeout--) { + /* On successful acquire, this request is cleared. */ + err = nbcon_context_try_acquire_direct(ctxt, cur); + if (!err) + return 0; + + /* + * If another CPU is in panic, the request must be removed + * before returning to the caller. + */ + if (err == -EPERM) + break; + + /* Continue spinwaiting on -EAGAIN and -EBUSY. */ + + udelay(1); + + /* Re-read the state because some time has passed. */ + nbcon_state_read(con, cur); + + /* + * If the waiter unexpectedly changed, this request is + * no longer set. Have the caller try again rather than + * guessing what has happened. + */ + if (cur->req_prio != ctxt->prio) + return -EAGAIN; + } + + /* Timeout. Safely remove handover request and report failure. */ + do { + /* No need to remove request if there is another waiter. */ + if (cur->req_prio != ctxt->prio) { + if (err == -EPERM) + return err; + return -EBUSY; + } + + /* Unset request for handover. */ + new.atom = cur->atom; + new.req_prio = NBCON_PRIO_NONE; + if (nbcon_state_try_cmpxchg(con, cur, &new)) { + debug_store(1, "handover: cpu%d TIMEOUT to acquire for prio%d\n", + cpu, ctxt->prio); + /* + * Request successfully unset. Report failure of + * acquiring via handover. + */ + cur->atom = new.atom; + return -EBUSY; + } + + /* + * Unable to remove request. Try direct acquire. If + * successful, this request is also cleared. + */ + err = nbcon_context_try_acquire_direct(ctxt, cur); + } while (err); + + /* Direct acquire at timeout succeeded! */ + return 0; +} + +/** + * nbcon_context_try_acquire_hostile - Try to acquire via hostile takeover + * @ctxt: The context of the caller + * @cur: The current console state + * + * Return: 0 on success and @cur is updated to the new console state. + * Otherwise an error code on failure. + * + * Errors: + * + * -EPERM: A panic is in progress an this is not the panic CPU + * or this context is not the single panic context. + * Retrying the acquire will not be immediately useful. + * + * -EBUSY: The current owner is in an unsafe region and + * @takeover_unsafe was not set. Retrying the acquire + * is only immediately useful if @takeover_unsafe is + * set. + * + * -EAGAIN: An unexpected state change occurred. @cur has been + * updated. The caller should retry the acquire. + * + * The general procedure is to set @prio (forcing ownership). This method + * must only be used as a final attempt during panic. + */ +static int nbcon_context_try_acquire_hostile(struct nbcon_context *ctxt, + struct nbcon_state *cur) +{ + unsigned int cpu = smp_processor_id(); + struct console *con = ctxt->console; + struct nbcon_state new = { + .cpu = cpu, + .prio = ctxt->prio, + .unsafe = cur->hostile_unsafe | ctxt->unsafe, + .hostile_unsafe = cur->hostile_unsafe | cur->unsafe, + }; + + if (other_cpu_in_panic()) + return -EPERM; + + /* Hostile takeovers must only be in the single panic context! */ + if (WARN_ON_ONCE(ctxt->prio != NBCON_PRIO_PANIC || cur->prio == NBCON_PRIO_PANIC)) + return -EPERM; + + /* + * If a hostile takeover in unsafe regions is wanted, + * this is additionally requested. + */ + if (cur->unsafe && !ctxt->takeover_unsafe) + return -EBUSY; + + if (!nbcon_state_try_cmpxchg(con, cur, &new)) + return -EAGAIN; + + cur->atom = new.atom; + + return 0; +} + +/** + * nbcon_context_try_acquire - Try to acquire nbcon console + * @ctxt: The context of the caller + * + * Return: True if the console was acquired. False otherwise. + * + * The attempts to acquire always begin with the safest methods and only + * upon failure move to more aggressive methods. + * + * If the caller requested a hostile takeover, on success @ctxt is + * updated so that the caller can see if indeed a hostile (and + * possibly unsafe) takeover occurred. + */ +__maybe_unused +static bool nbcon_context_try_acquire(struct nbcon_context *ctxt) +{ + __maybe_unused + unsigned int cpu = smp_processor_id(); + struct console *con = ctxt->console; + struct nbcon_state cur; + bool hostile; + int err; + + /* @takeover_unsafe is only valid when hostile takeover requested. */ + WARN_ON_ONCE(ctxt->takeover_unsafe && !ctxt->hostile); + + nbcon_state_read(con, &cur); + +try_again: + hostile = false; + + /* ACQUIRE DIRECT */ + + debug_store(ctxt->prio > NBCON_PRIO_NORMAL, + "direct: cpu%d trying DIRECT acquire prio%d\n", cpu, ctxt->prio); + err = nbcon_context_try_acquire_direct(ctxt, &cur); + if (!err) { + debug_store(ctxt->prio > NBCON_PRIO_NORMAL, + "direct: cpu%d SUCCESS prio%d\n", cpu, ctxt->prio); + goto success; + } else if (err == -EPERM) { + debug_store(1, "direct: cpu%d FAILED prio%d (non-panic cpu)\n", + cpu, ctxt->prio); + return false; + } else if (err == -EAGAIN) { + debug_store(1, "direct: cpu%d FAILED prio%d (state race)\n", + cpu, ctxt->prio); + goto try_again; + } else { + debug_store(1, "direct: cpu%d FAILED prio%d (already owned by cpu%d)\n", + cpu, ctxt->prio, cur.cpu); + /* Continue to next method. */ + } + + /* ACQUIRE VIA HANDOVER */ + + debug_store(1, "handover: cpu%d REQUESTING prio%d from cpu%d\n", + cpu, ctxt->prio, cur.cpu); + err = nbcon_context_try_acquire_handover(ctxt, &cur); + if (!err) { + debug_store(1, "handover: cpu%d SUCCESS prio%d\n", + cpu, ctxt->prio); + goto success; + } else if (err == -EPERM) { + debug_store(1, "handover: cpu%d FAILED requesting prio%d (non-panic cpu)\n", + cpu, ctxt->prio); + return false; + } else if (err == -EAGAIN) { + debug_store(1, "handover: cpu%d FAILED requesting prio%d (state race)\n", + cpu, ctxt->prio); + goto try_again; + } else { + debug_store(1, "handover: cpu%d FAILED requesting prio%d (cpu%d/prio%d/timeout)\n", + cpu, ctxt->prio, cur.cpu, cur.prio); + /* Continue to next method. */ + } + + /* ACQUIRE VIA HOSTILE TAKEOVER */ + + /* Only attempt hostile takeover if explicitly requested. */ + if (!ctxt->hostile) + return false; + + debug_store(1, + "hostile: cpu%d trying HOSTILE acquire for prio%d from cpu%d (takeover_unsafe=%d)\n", + cpu, ctxt->prio, cur.cpu, ctxt->takeover_unsafe); + err = nbcon_context_try_acquire_hostile(ctxt, &cur); + if (!err) { + debug_store(1, "hostile: cpu%d SUCCESS prio%d\n", cpu, ctxt->prio); + /* Let caller know if the takeover was unsafe. */ + ctxt->takeover_unsafe = cur.hostile_unsafe; + hostile = true; + goto success; + } else if (err == -EPERM) { + debug_store(1, "hostile: cpu%d FAILED acquire prio%d (non-panic cpu)\n", + cpu, ctxt->prio); + return false; + } else if (err == -EAGAIN) { + debug_store(1, "hostile: cpu%d FAILED acquire prio%d (state race)\n", + cpu, ctxt->prio); + goto try_again; + } else { + debug_store(1, "hostile: cpu%d FAILED acquire prio%d (unsafe)\n", + cpu, ctxt->prio); + /* Continue to next method. */ + } + + /* No methods left to try. */ + return false; +success: + if (!hostile) { + /* Let caller know it was not a hostile takeover. */ + ctxt->hostile = 0; + ctxt->takeover_unsafe = 0; + } + return true; +} + +static bool nbcon_owner_matches(struct nbcon_state *cur, int expected_cpu, + int expected_prio) +{ + /* + * Since consoles can only be acquired by higher priorities, + * owning contexts are uniquely identified by @prio. However, + * since contexts can unexpectedly lose ownership, it is + * possible that later another owner appears with the same + * priority. For this reason @cpu is also needed. + */ + + if (cur->prio != expected_prio) + return false; + + if (cur->cpu != expected_cpu) + return false; + + return true; +} + +/** + * nbcon_context_release - Release the console + * @ctxt: The nbcon context from nbcon_context_try_acquire() + */ +__maybe_unused +static void nbcon_context_release(struct nbcon_context *ctxt) +{ + unsigned int cpu = smp_processor_id(); + struct console *con = ctxt->console; + struct nbcon_state cur; + struct nbcon_state new; + + nbcon_state_read(con, &cur); + do { + if (!nbcon_owner_matches(&cur, cpu, ctxt->prio)) + return; + + new.atom = cur.atom; + new.prio = NBCON_PRIO_NONE; + new.unsafe = cur.hostile_unsafe; + + /* + * If @hostile_unsafe is set, it is kept set so that + * the state remains permanently unsafe. + */ + + } while (!nbcon_state_try_cmpxchg(con, &cur, &new)); +} + /** * nbcon_init - Initialize the nbcon console specific data * @con: Console to initialize From patchwork Fri Jul 28 00:02:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127259 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp88551vqg; Thu, 27 Jul 2023 17:07:30 -0700 (PDT) X-Google-Smtp-Source: APBJJlE9N96HPK8eXg0DW9vf8qU3KGFacN2FeXieijJmtjC0vJHNkHOFF4z8tUsK3SgNyyNxS7kP X-Received: by 2002:a17:903:1246:b0:1bb:a367:a70 with SMTP id u6-20020a170903124600b001bba3670a70mr99312plh.17.1690502850118; Thu, 27 Jul 2023 17:07:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690502850; cv=none; d=google.com; s=arc-20160816; b=psOH9hX61Sonw5fmWNDO10AUtAiYQKAS3dOhW80e3KRJVJVCYjgXCtN9p+n+xUKFKR KBcOirOfPvHFmh794dm++dkyV6HmglWnoVXLQ3EIlvGOvxIKbWkjh8HSyDTqqJiOuxow ataLTOfeRUM2PCzsxyC8tZn5OszmD2lbw5Ctb77Vg+LxNU0j8rsRmUHMzHWvdCxPXFZJ uha5jnjfAiUkosGXizoJbiKQ7mN2ZiciiI6s00oQmeVGK11S1+hzQDE0iNTuHa5GkTeF NUwdu4YKUweGv8z4DEMGzbauzM2kG41Lf2hLp7Kk2p7YCLwXOA4bE8sGgjcP2sjCCkkx 6Gww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=zQ+5GFgvTYBrgG/gdG5LVsZvzCc8vSNhvk0jKBm+CCo=; fh=I/6bvJz6sWq8p1xoNUgGjc7lme65GQ/wcdq821dg0a8=; b=V5obHdLlaA1zVcc8D9CU0iwRj83ze0A/JDWHuiip12uXbzyvbeZpY0dHV+yZsa2w/F YiSj8wHpox58C1VLz8i07gsnTlaiwAqxZ01DsjWip9wxyoK/+5X8SjUNXbb8ABU3oATM q3vfeefjAwDa+6kiuM1FoDlNFFwweSCu1tF9nEOndF1lTFOR8fDo8qsi94RwEPPgTXFY 9Jw+lFs1kQbmRwYgk3ZkTcb6rJQg4vbv1RCMx5JEB2HX1uuxxZTPlemmzH63zYMU6m+n oWvDKuyiS54dFz2imoH6gTaNGhYO6P8RXyMkHRGgBxW8HWBsvO0pMbz6nK+mB8emf09b wKvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=uIIPXliW; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kv13-20020a17090328cd00b001b9eb5d1ea2si1966716plb.198.2023.07.27.17.07.17; Thu, 27 Jul 2023 17:07:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=uIIPXliW; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231605AbjG1ACu (ORCPT + 99 others); Thu, 27 Jul 2023 20:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229771AbjG1ACm (ORCPT ); Thu, 27 Jul 2023 20:02:42 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6C3F30FB for ; Thu, 27 Jul 2023 17:02:39 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zQ+5GFgvTYBrgG/gdG5LVsZvzCc8vSNhvk0jKBm+CCo=; b=uIIPXliW+A6EgH2lwlEr1gJeNeuAXEsY3WO8uFMJrkYscgESGYB+lhCWaXYRoO58CZTnwa l2h5meAY/H5+TL6R+znFVDl5drSt18/dXv6TAe+UVeHxR2PgeahI3Lnaa4LhWkZiSJ0LU2 uv3vj+qKYERD3I4r+VAu/JikZ3lt+entASHR2xUl0GoFVbAlD65foUX1RSIJLB+5nh2J9V 3OryIMUi0T3CiF4zkHiEa5CycdmdEEZo0WdOvKwXmgWdGnu2OqTQUjqZUXFP3+MtQ9oYzX SAolP93EPQnjW+o6e92Ymeo7NalwvY2idruM/kv5lv5Cghv94bxOSGcIU867ww== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zQ+5GFgvTYBrgG/gdG5LVsZvzCc8vSNhvk0jKBm+CCo=; b=gPSHpsOiX/R3mhLQZAseHTJvjSYroz99Pnd9DUbBZu5UhZCYBRgMi4zQkGQuBcD/ufPblk pVxwVBqn/ssCI9CA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 4/8] printk: nbcon: Add buffer management Date: Fri, 28 Jul 2023 02:08:29 +0206 Message-Id: <20230728000233.50887-5-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772620716515594067 X-GMAIL-MSGID: 1772620716515594067 From: Thomas Gleixner In case of hostile takeovers it must be ensured that the previous owner cannot scribble over the output buffer of the emergency/panic context. This is achieved by: - Adding a global output buffer instance for the panic context. This is the only situation where hostile takeovers can occur and there is always at most 1 panic context. - Allocating an output buffer per console upon console registration. This buffer is used by the console owner when not in panic context. - Choosing the appropriate buffer is handled in the acquire/release functions. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) Reviewed-by: Petr Mladek --- include/linux/console.h | 7 +++++++ kernel/printk/internal.h | 6 ++++++ kernel/printk/printk.c | 6 ------ kernel/printk/printk_nbcon.c | 26 ++++++++++++++++++++++++-- 4 files changed, 37 insertions(+), 8 deletions(-) diff --git a/include/linux/console.h b/include/linux/console.h index e06cd1ce3e82..d2bcd2c190a7 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -233,6 +233,7 @@ enum nbcon_prio { }; struct console; +struct printk_buffers; /** * struct nbcon_context - Context for console acquire/release @@ -242,6 +243,7 @@ struct console; * @unsafe: This context is in an unsafe section * @hostile: Acquire console by hostile takeover * @takeover_unsafe: Acquire console by hostile takeover even if unsafe + * @pbufs: Pointer to the text buffer for this context */ struct nbcon_context { /* members set by caller */ @@ -251,6 +253,9 @@ struct nbcon_context { unsigned int unsafe : 1; unsigned int hostile : 1; unsigned int takeover_unsafe : 1; + + /* members set by acquire */ + struct printk_buffers *pbufs; }; /** @@ -274,6 +279,7 @@ struct nbcon_context { * @node: hlist node for the console list * * @nbcon_state: State for nbcon consoles + * @pbufs: Pointer to nbcon private buffer */ struct console { char name[16]; @@ -296,6 +302,7 @@ struct console { /* nbcon console specific members */ atomic_t __private nbcon_state; + struct printk_buffers *pbufs; }; #ifdef CONFIG_LOCKDEP diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h index 655810f2976e..0f2be350600e 100644 --- a/kernel/printk/internal.h +++ b/kernel/printk/internal.h @@ -13,6 +13,12 @@ int devkmsg_sysctl_set_loglvl(struct ctl_table *table, int write, #define printk_sysctl_init() do { } while (0) #endif +#define con_printk(lvl, con, fmt, ...) \ + printk(lvl pr_fmt("%s%sconsole [%s%d] " fmt), \ + (con->flags & CON_NBCON) ? "" : "legacy ", \ + (con->flags & CON_BOOT) ? "boot" : "", \ + con->name, con->index, ##__VA_ARGS__) + #ifdef CONFIG_PRINTK #ifdef CONFIG_PRINTK_CALLER diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 98b4854c81ea..582552a96c57 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -3359,12 +3359,6 @@ static void try_enable_default_console(struct console *newcon) newcon->flags |= CON_CONSDEV; } -#define con_printk(lvl, con, fmt, ...) \ - printk(lvl pr_fmt("%s%sconsole [%s%d] " fmt), \ - (con->flags & CON_NBCON) ? "" : "legacy ", \ - (con->flags & CON_BOOT) ? "boot" : "", \ - con->name, con->index, ##__VA_ARGS__) - static void console_init_seq(struct console *newcon, bool bootcon_registered) { struct console *con; diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index b0acde0cb949..39fa64891ec6 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "internal.h" /* * Printk console printing implementation for consoles that do not depend on @@ -20,6 +21,9 @@ * region and aborts the operation if it detects a takeover. * * In case of panic the nesting context can take over the console forcefully. + * If the interrupted context touches the assigned record buffer after + * takeover, it does not cause harm because the interrupting single panic + * context is assigned its own panic record buffer. * * A concurrent writer on a different CPU with a higher priority can request * to take over the console by: @@ -356,6 +360,8 @@ static int nbcon_context_try_acquire_hostile(struct nbcon_context *ctxt, return 0; } +static struct printk_buffers panic_nbcon_pbufs; + /** * nbcon_context_try_acquire - Try to acquire nbcon console * @ctxt: The context of the caller @@ -372,7 +378,6 @@ static int nbcon_context_try_acquire_hostile(struct nbcon_context *ctxt, __maybe_unused static bool nbcon_context_try_acquire(struct nbcon_context *ctxt) { - __maybe_unused unsigned int cpu = smp_processor_id(); struct console *con = ctxt->console; struct nbcon_state cur; @@ -471,6 +476,13 @@ static bool nbcon_context_try_acquire(struct nbcon_context *ctxt) ctxt->hostile = 0; ctxt->takeover_unsafe = 0; } + + /* Assign the appropriate buffer for this context. */ + if (atomic_read(&panic_cpu) == cpu) + ctxt->pbufs = &panic_nbcon_pbufs; + else + ctxt->pbufs = con->pbufs; + return true; } @@ -509,7 +521,7 @@ static void nbcon_context_release(struct nbcon_context *ctxt) nbcon_state_read(con, &cur); do { if (!nbcon_owner_matches(&cur, cpu, ctxt->prio)) - return; + break; new.atom = cur.atom; new.prio = NBCON_PRIO_NONE; @@ -521,6 +533,8 @@ static void nbcon_context_release(struct nbcon_context *ctxt) */ } while (!nbcon_state_try_cmpxchg(con, &cur, &new)); + + ctxt->pbufs = NULL; } /** @@ -534,6 +548,12 @@ bool nbcon_init(struct console *con) { struct nbcon_state state = { }; + con->pbufs = kmalloc(sizeof(*con->pbufs), GFP_KERNEL); + if (!con->pbufs) { + con_printk(KERN_ERR, con, "failed to allocate printing buffer\n"); + return false; + } + nbcon_state_set(con, &state); return true; } @@ -547,4 +567,6 @@ void nbcon_cleanup(struct console *con) struct nbcon_state state = { }; nbcon_state_set(con, &state); + kfree(con->pbufs); + con->pbufs = NULL; } From patchwork Fri Jul 28 00:02:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127268 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp105165vqg; Thu, 27 Jul 2023 17:48:24 -0700 (PDT) X-Google-Smtp-Source: APBJJlFvwbgOodgSqLZO6ERcVXnhrXmxihhqAHYIy9BG6CC9GDSQycpS2GgwRG2O/IBoonZ1KVbq X-Received: by 2002:a17:906:3f49:b0:997:c377:b41f with SMTP id f9-20020a1709063f4900b00997c377b41fmr548963ejj.64.1690505303940; Thu, 27 Jul 2023 17:48:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690505303; cv=none; d=google.com; s=arc-20160816; b=z+dL2ZLYRDxZrp3aBHd7S2/UOWm9rs6iPGi/fcnAh6Kdq0AV3OWWdVG0mcDaQnHFQ0 vR0QPZZ/WHpJyzf4JviWzrZzcMP8hTxq40ECByUyRnw9csnF0kHuOtemI4CIqQw6XXN9 VnpizJBX63q/iukbfz9ZXUlmGSfKPkOShgnzXFf0OcQC+CcG+bdt0D9A8aPTiViShCtd w0qnetsY27bqmEeKtsMxqgokE4NAuUwgNWe2l3ELj/a21vKpdzw8suKN9UIcyuF/Ic85 w8g2EGWJx/HXhr8WEW0ZJffIMk+ywxwpLWdUsbTu2vx1+UchGkaMf+GoBK2JlOHN/Ppp ihMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=j2CQPdz/Xds8UFh3BREFGUGoaQ7ARNwZtS9/ALhBvE0=; fh=I/6bvJz6sWq8p1xoNUgGjc7lme65GQ/wcdq821dg0a8=; b=yoKg5+sbjBkPMaVbULsrKIu03AiAXQRQ4u9u4R1ZL/hUpNyXDDmT486fjO9usP+nYA mQK1ov4lNiuQBtLBLyrWVzgN3MjwTq/20vwgXFYNcsRR9/gJrJ2ALFddCCpBJYj9vR7o SzsmGICXZEAqbrn481jF/Gck4czAjW2ka7LRbLE3DyhcyOmHngCJRQZahJhqPG4AoeDU jbun1isYdLBgsHjrsGFntmt139DuEc24iqnNILGyXkQfnbYvpslS4iy/urlgeCvl46Ax vGK2DyiOEVBnXNusnO1NwPRa17OJ1OENgY3h58WHsV8GVkRBoyAaSI2VDqxZLSUJXxom g96A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="yPNvc/Ax"; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id dv22-20020a170906b81600b00982bd2c070fsi1689269ejb.469.2023.07.27.17.48.00; Thu, 27 Jul 2023 17:48:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="yPNvc/Ax"; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232055AbjG1ADL (ORCPT + 99 others); Thu, 27 Jul 2023 20:03:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231578AbjG1ACp (ORCPT ); Thu, 27 Jul 2023 20:02:45 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01F6EDB for ; Thu, 27 Jul 2023 17:02:42 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j2CQPdz/Xds8UFh3BREFGUGoaQ7ARNwZtS9/ALhBvE0=; b=yPNvc/AxHIy5mN1vjVRYZp5bhzb1pSPtA5IVb36D7S3EaUkaNoZogU/rbJFPTTWpTUluBs qjtOLZ+ZxNGrGSLLwkFeV/ILdpKFCjaoF1w5/XmOLJE00vWX0kI0qVrSR38OkCIz/WriwI L3jLs1O5ndIYlQIRrunOYeFz+ckm0dz+XuKrV+DUH4V1ybl1raWeodrD0sM4a32VViL792 0qMkynegjZMYlMNZtIyEL7e8lk9GxVLbkMMO+2mF5msOLIZHPb6hqkLj6Drjs05YJBNuE6 u9lrcfu3jnj831PdHAnep6BLQuvYyZ6rZEL6iScj1+/MvCkHehdKqQqW1V2kaQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j2CQPdz/Xds8UFh3BREFGUGoaQ7ARNwZtS9/ALhBvE0=; b=BJFhd29SUIjXu9rd7WsFAZLqDPn0RL8jsHvJzgvBSnY1SDvnYZoYEhYWaX8nAritwMlwfr L7zh2WPzHK3McWBA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 5/8] printk: nbcon: Add sequence handling Date: Fri, 28 Jul 2023 02:08:30 +0206 Message-Id: <20230728000233.50887-6-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772623289559945585 X-GMAIL-MSGID: 1772623289559945585 From: Thomas Gleixner Add an atomic_long_t field @nbcon_seq to the console struct to store the sequence number for nbcon consoles. For nbcon consoles this will be used instead of the non-atomic @seq field. The new field allows for safe atomic sequence number updates without requiring any locking. On 64bit systems the new field stores the full sequence number. On 32bit systems the new field stores the lower 32 bits of the sequence number, which are expanded to 64bit as needed by folding the values based on the sequence numbers available in the ringbuffer. For 32bit systems, having a 32bit representation in the console is sufficient. If a console ever gets more than 2^31 records behind the ringbuffer then this is the least of the problems. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) --- include/linux/console.h | 4 ++ kernel/printk/internal.h | 6 ++ kernel/printk/printk.c | 39 +++++++++--- kernel/printk/printk_nbcon.c | 114 +++++++++++++++++++++++++++++++++++ 4 files changed, 153 insertions(+), 10 deletions(-) diff --git a/include/linux/console.h b/include/linux/console.h index d2bcd2c190a7..a50eaa3b8420 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -244,6 +244,7 @@ struct printk_buffers; * @hostile: Acquire console by hostile takeover * @takeover_unsafe: Acquire console by hostile takeover even if unsafe * @pbufs: Pointer to the text buffer for this context + * @seq: The sequence number to print for this context */ struct nbcon_context { /* members set by caller */ @@ -256,6 +257,7 @@ struct nbcon_context { /* members set by acquire */ struct printk_buffers *pbufs; + u64 seq; }; /** @@ -279,6 +281,7 @@ struct nbcon_context { * @node: hlist node for the console list * * @nbcon_state: State for nbcon consoles + * @nbcon_seq: Sequence number of the next record for nbcon to print * @pbufs: Pointer to nbcon private buffer */ struct console { @@ -302,6 +305,7 @@ struct console { /* nbcon console specific members */ atomic_t __private nbcon_state; + atomic_long_t __private nbcon_seq; struct printk_buffers *pbufs; }; diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h index 0f2be350600e..1baccddf83b8 100644 --- a/kernel/printk/internal.h +++ b/kernel/printk/internal.h @@ -4,6 +4,7 @@ */ #include #include +#include "printk_ringbuffer.h" #if defined(CONFIG_PRINTK) && defined(CONFIG_SYSCTL) void __init printk_sysctl_init(void); @@ -42,6 +43,7 @@ enum printk_info_flags { LOG_CONT = 8, /* text is a fragment of a continuation line */ }; +extern struct printk_ringbuffer *prb; extern bool have_legacy_console; extern bool have_boot_console; @@ -72,6 +74,8 @@ void defer_console_output(void); u16 printk_parse_prefix(const char *text, int *level, enum printk_info_flags *flags); +u64 nbcon_seq_read(struct console *con); +void nbcon_seq_force(struct console *con, u64 seq); bool nbcon_init(struct console *con); void nbcon_cleanup(struct console *con); @@ -90,6 +94,8 @@ void nbcon_cleanup(struct console *con); #define printk_safe_exit_irqrestore(flags) local_irq_restore(flags) static inline bool printk_percpu_data_ready(void) { return false; } +static inline u64 nbcon_seq_read(struct console *con) { return 0; } +static inline void nbcon_seq_force(struct console *con, u64 seq) { } static inline bool nbcon_init(struct console *con) { return true; } static inline void nbcon_cleanup(struct console *con) { } diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 582552a96c57..64e404aacbc4 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -512,7 +512,7 @@ _DEFINE_PRINTKRB(printk_rb_static, CONFIG_LOG_BUF_SHIFT - PRB_AVGBITS, static struct printk_ringbuffer printk_rb_dynamic; -static struct printk_ringbuffer *prb = &printk_rb_static; +struct printk_ringbuffer *prb = &printk_rb_static; /* * We cannot access per-CPU data (e.g. per-CPU flush irq_work) before @@ -3171,8 +3171,27 @@ void console_unblank(void) */ void console_flush_on_panic(enum con_flush_mode mode) { + struct console *c; bool handover; - u64 next_seq; + short flags; + int cookie; + u64 seq; + + seq = prb_first_valid_seq(prb); + + /* + * Safely handle the atomic consoles before trying to flush any + * legacy consoles. + */ + if (mode == CONSOLE_REPLAY_ALL) { + cookie = console_srcu_read_lock(); + for_each_console_srcu(c) { + flags = console_srcu_read_flags(c); + if (flags & CON_NBCON) + nbcon_seq_force(c, seq); + } + console_srcu_read_unlock(cookie); + } if (!serialized_printing) return; @@ -3195,12 +3214,6 @@ void console_flush_on_panic(enum con_flush_mode mode) console_may_schedule = 0; if (mode == CONSOLE_REPLAY_ALL) { - struct console *c; - int cookie; - u64 seq; - - seq = prb_first_valid_seq(prb); - cookie = console_srcu_read_lock(); for_each_console_srcu(c) { /* @@ -3212,7 +3225,7 @@ void console_flush_on_panic(enum con_flush_mode mode) console_srcu_read_unlock(cookie); } - console_flush_all(false, &next_seq, &handover); + console_flush_all(false, &seq, &handover); } /* @@ -3795,6 +3808,7 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre struct console *c; u64 last_diff = 0; u64 printk_seq; + short flags; bool locked; int cookie; u64 diff; @@ -3821,6 +3835,9 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre for_each_console_srcu(c) { if (con && con != c) continue; + + flags = console_srcu_read_flags(c); + /* * If consoles are not usable, it cannot be expected * that they make forward progress, so only increment @@ -3829,7 +3846,9 @@ static bool __pr_flush(struct console *con, int timeout_ms, bool reset_on_progre if (!console_is_usable(c)) continue; - if (locked) + if (flags & CON_NBCON) + printk_seq = nbcon_seq_read(c); + else if (locked) printk_seq = c->seq; else continue; diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index 39fa64891ec6..8229a0a00d5b 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -108,6 +108,116 @@ static inline bool nbcon_state_try_cmpxchg(struct console *con, struct nbcon_sta return atomic_try_cmpxchg(&ACCESS_PRIVATE(con, nbcon_state), &cur->atom, new->atom); } +/* Convert sequence from u64 to unsigned long. */ +static inline unsigned long __nbcon_seq_to_stored(u64 full_seq) +{ + /* On 32bit systems only the lower 32 bits are stored. */ + return (unsigned long)full_seq; +} + +/* Convert sequence from unsigned long to u64. */ +static inline u64 __nbcon_seq_to_full(unsigned long stored_seq) +{ +#ifdef CONFIG_64BIT + return stored_seq; +#else + u64 full_seq; + u64 rb_seq; + + /* + * The provided sequence is only the lower 32 bits of the ringbuffer + * sequence. It needs to be expanded to 64bit. Get the next sequence + * number from the ringbuffer and fold it. + */ + rb_seq = prb_next_seq(prb); + full_seq = rb_seq - ((u32)rb_seq - stored_seq); + + return full_seq; +#endif +} + +/** + * nbcon_seq_init - Helper function to initialize the console sequence + * @con: Console to work on + * + * Set @con->nbcon_seq to the starting record (specified with con->seq). + * If the starting record no longer exists, the oldest available record + * is chosen. This is because on 32bit systems only the lower 32 bits of + * the sequence number are stored. The upper 32 bits are derived from the + * sequence numbers available in the ringbuffer. + * + * For init only. Do not use for runtime updates. + */ +static void nbcon_seq_init(struct console *con) +{ + u64 seq = max_t(u64, con->seq, prb_first_valid_seq(prb)); + + atomic_long_set(&ACCESS_PRIVATE(con, nbcon_seq), __nbcon_seq_to_stored(seq)); + + /* Clear con->seq since nbcon consoles use con->nbcon_seq instead. */ + con->seq = 0; +} + +/** + * nbcon_read_seq - Read the current console sequence + * @con: Console to read the sequence of + * + * Return: Sequence number of the next record to print on @con. + */ +u64 nbcon_seq_read(struct console *con) +{ + unsigned long seq = atomic_long_read(&ACCESS_PRIVATE(con, nbcon_seq)); + + return __nbcon_seq_to_full(seq); +} + +/** + * nbcon_seq_force - Force console sequence to a specific value + * @con: Console to work on + * + * Only to be used in extreme situations (such as panic with + * CONSOLE_REPLAY_ALL). + */ +void nbcon_seq_force(struct console *con, u64 seq) +{ + atomic_long_set(&ACCESS_PRIVATE(con, nbcon_seq), __nbcon_seq_to_stored(seq)); +} + +/** + * nbcon_seq_try_update - Try to update the console sequence number + * @ctxt: Pointer to an acquire context that contains + * all information about the acquire mode + * + * Return: True if the console sequence was updated, false otherwise. + * + * On 32bit the sequence in con->nbcon_seq is only the lower 32 bits. + * Therefore it must be expanded to 64bit upon a failed cmpxchg in + * order to correctly verify that the new sequence (ctxt->seq) is + * larger. + * + * In case of fail the console has been likely taken over. However, the + * caller must still assume it has ownership and decide how to proceed. + */ +__maybe_unused +static bool nbcon_seq_try_update(struct nbcon_context *ctxt) +{ + struct console *con = ctxt->console; + u64 con_seq = nbcon_seq_read(con); + + while (con_seq < ctxt->seq) { + unsigned long seq = __nbcon_seq_to_stored(con_seq); + + if (atomic_long_try_cmpxchg(&ACCESS_PRIVATE(con, nbcon_seq), &seq, + __nbcon_seq_to_stored(ctxt->seq))) { + return true; + } + + /* Expand new @seq value for comparing. */ + con_seq = __nbcon_seq_to_full(seq); + } + return false; +} + /** * nbcon_context_try_acquire_direct - Try to acquire directly * @ctxt: The context of the caller @@ -483,6 +593,9 @@ static bool nbcon_context_try_acquire(struct nbcon_context *ctxt) else ctxt->pbufs = con->pbufs; + /* Set the record sequence for this context to print. */ + ctxt->seq = nbcon_seq_read(ctxt->console); + return true; } @@ -554,6 +667,7 @@ bool nbcon_init(struct console *con) return false; } + nbcon_seq_init(con); nbcon_state_set(con, &state); return true; } From patchwork Fri Jul 28 00:02:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127276 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp122903vqg; Thu, 27 Jul 2023 18:34:56 -0700 (PDT) X-Google-Smtp-Source: APBJJlGnO/UFKsQCAJrbQrPzgaOdR8znHVMVfuEOklMylucvjcecoEPh8bje/LLCSuFZC3yG9I6L X-Received: by 2002:a17:906:76d8:b0:974:1e0e:91ee with SMTP id q24-20020a17090676d800b009741e0e91eemr690385ejn.13.1690508095796; Thu, 27 Jul 2023 18:34:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690508095; cv=none; d=google.com; s=arc-20160816; b=SGJuWSfLvDnOXAUoK5ALiwO+kwWgAWbJN1uon8eZdcgNE9n6TOyHtExDnj9LiM/fJl DdIWIiOS4NHZYB0TSahJGpVYDd2esqnHAZq5/7fUWEWJ2eSkvow1Ezu+/2TeZGLhClVJ EyTkKGfJEIdUdcSjd0+KgKIOyKT+cVptUWwoixF6EYkD45qT0RkirRnUak83svm67h3o HZB2x2G06XiMM8F8ARs4MPW2dCh/qfkPrXJj2C+efxbQOJEelh6mMfwlVW5lk1xLfz6w tcAofTWJU9KmztWQhIya0wMnt8oVzlcePISV0HOzNHhHDHGCQA5WOgRL/VlxXzAZj6b7 Vczg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=lzKIhztjxshdapDIJ3AGdBj3vg17a8NsL1htiDJV/M4=; fh=HVRBJStvoKWbFco6Jta18xe5UAVCwWVs9OvJstciom4=; b=s73/k+QPTyDQlKUACmUruzLzSU7Y2jEjgWFiMm8NbaX9Ad52qasR4r+4OWjK9EypIA 5sD6usJ6RWAptwx0zvUkyAgDpgbO5CkzSXGxG8AY/PpZ3gogIEMDahaXtiPozFCWw+y6 p2Zs91JALMfV+Dx6BPvGCk4sSOGrfoZXgrQkMg0uCzU0diDC+JzeM/I3zX/UyfBc53QV CwEBb9iA35FcNN0KdX+id16BoqQuPtmQEkKRDY/66FhQrpX6fTjYpaGxF78nPD4Yl+TO mB6oINCpNlrkNapqGyRDI0eFqFypG/daKMi1iwTC5hhpkuixyQzzLQlfQJC51FAxeGDX UCwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=tHN90uaw; dkim=neutral (no key) header.i=@linutronix.de header.b=iz4gHIhd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mb22-20020a170906eb1600b00997c9f1bdf7si1760946ejb.407.2023.07.27.18.34.32; Thu, 27 Jul 2023 18:34:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=tHN90uaw; dkim=neutral (no key) header.i=@linutronix.de header.b=iz4gHIhd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231741AbjG1ADE (ORCPT + 99 others); Thu, 27 Jul 2023 20:03:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231569AbjG1ACp (ORCPT ); Thu, 27 Jul 2023 20:02:45 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 024BAFA for ; Thu, 27 Jul 2023 17:02:42 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lzKIhztjxshdapDIJ3AGdBj3vg17a8NsL1htiDJV/M4=; b=tHN90uawbFBFjdj0vVJAMNJ+QybnaTFkpLt92HjBaa0/GLBhQqBnQHVRKQZ3pK7n9K31PI sD3V0aEIVNC2d6yD42OZt7FCtff81/8KvxpINHh3yHHi819kYTDxavSJGgB3UVQ5JzA5Di nT3xfo6a6bzuCanhAzpiqvAEDakdys2670nc6YGyB/r6OFptA8CD6zB/FJ96onesCaTwZs BfHzXDfR9bwKPyfx44iK7jUBSggdFwxz4/fwuJy1YMV0GLo0DvMgIUYSGxZTkQMbkzr38U 408YmFJHxVf8pujLV94e3pUjiUKf4MtgP4nfYKJU2VY7ogG8pPgDywikTc1DOw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lzKIhztjxshdapDIJ3AGdBj3vg17a8NsL1htiDJV/M4=; b=iz4gHIhd7TSH/GZ8ZRC+eRBVwFHwCZNwztT2vCiqIAHOClz0TiWOcDRMRQ+SGSYIr+QnGL HJ5gGyyeQK/L47DQ== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org Subject: [PATCH printk v2 6/8] printk: nbcon: Add ownership state functions Date: Fri, 28 Jul 2023 02:08:31 +0206 Message-Id: <20230728000233.50887-7-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772626216947208724 X-GMAIL-MSGID: 1772626216947208724 From: Thomas Gleixner Provide functions that are related to the safe handover mechanism and allow console drivers to dynamically specify unsafe regions: - nbcon_context_can_proceed() Invoked by a console owner to check whether a handover request is pending or whether the console was taken over in a hostile fashion. If a handover request is pending, this function will also perform the handover, thus cancelling its own ownership. - nbcon_context_update_unsafe() Invoked by a console owner to denote that the driver is about to enter or leave a critical region where a hostile take over is unsafe. This function is also a cancellation point where loss of ownership can occur. The unsafe state is stored in the console state and allows a new context to make informed decisions whether to attempt a takeover of such a console. The unsafe state is also available to the driver so that it can make informed decisions about the required actions or take a special emergency path. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) --- kernel/printk/printk_nbcon.c | 113 ++++++++++++++++++++++++++++++++++- 1 file changed, 112 insertions(+), 1 deletion(-) diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index 8229a0a00d5b..e41f2eff5ef6 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -623,7 +623,6 @@ static bool nbcon_owner_matches(struct nbcon_state *cur, int expected_cpu, * nbcon_context_release - Release the console * @ctxt: The nbcon context from nbcon_context_try_acquire() */ -__maybe_unused static void nbcon_context_release(struct nbcon_context *ctxt) { unsigned int cpu = smp_processor_id(); @@ -650,6 +649,118 @@ static void nbcon_context_release(struct nbcon_context *ctxt) ctxt->pbufs = NULL; } +/** + * nbcon_context_can_proceed - Check whether ownership can proceed + * @ctxt: The nbcon context from nbcon_context_try_acquire() + * @cur: The current console state + * + * Return: True if the state is correct. False if ownership was + * handed over or taken. + * + * Must be invoked after the record was dumped into the assigned buffer + * and at appropriate safe places in the driver. + * + * When this function returns false then the calling context is no longer + * the owner and is no longer allowed to go forward. In this case it must + * back out immediately and carefully. The buffer content is also no longer + * trusted since it no longer belongs to the calling context. + */ +static bool nbcon_context_can_proceed(struct nbcon_context *ctxt, struct nbcon_state *cur) +{ + unsigned int cpu = smp_processor_id(); + + /* Make sure this context is still the owner. */ + if (!nbcon_owner_matches(cur, cpu, ctxt->prio)) { + debug_store(1, "handover: cpu%d DETECTED HOSTILE takeover\n", cpu); + return false; + } + + /* The console owner can proceed if there is no waiter. */ + if (cur->req_prio == NBCON_PRIO_NONE) + return true; + + /* + * A console owner within an unsafe region is always allowed to + * proceed, even if there are waiters. It can perform a handover + * when exiting the unsafe region. Otherwise the waiter will + * need to perform an unsafe hostile takeover. + */ + if (cur->unsafe) { + debug_store(cur->req_prio > cur->prio, + "handover: cpu%d IGNORING HANDOVER prio%d -> prio%d (unsafe)\n", + cpu, cur->prio, cur->req_prio); + return true; + } + + /* Waiters always have higher priorities than owners. */ + WARN_ON_ONCE(cur->req_prio <= cur->prio); + + debug_store(1, "handover: cpu%d HANDING OVER prio%d -> prio%d\n", + cpu, cur->prio, cur->req_prio); + + + /* + * Having a safe point for take over and eventually a few + * duplicated characters or a full line is way better than a + * hostile takeover. Post processing can take care of the garbage. + * Release and hand over. + */ + nbcon_context_release(ctxt); + + /* + * It is not known whether the handover succeeded. The outermost + * callsite has to make the final decision whether printing + * should proceed or not (via reacquire, possibly hostile). The + * console is now unlocked so go back all the way instead of + * trying to implement heuristics in tons of places. + */ + return false; +} + +/** + * nbcon_context_update_unsafe - Update the unsafe bit in @con->nbcon_state + * @ctxt: The nbcon context from nbcon_context_try_acquire() + * @unsafe: The new value for the unsafe bit + * + * Return: True if the state is correct. False if ownership was + * handed over or taken. + * + * Typically the unsafe bit is set during acquire. This function allows + * modifying the unsafe status without releasing ownership. + * + * When this function returns false then the calling context is no longer + * the owner and is no longer allowed to go forward. In this case it must + * back out immediately and carefully. The buffer content is also no longer + * trusted since it no longer belongs to the calling context. + * + * Internal helper to avoid duplicated code + */ +__maybe_unused +static bool nbcon_context_update_unsafe(struct nbcon_context *ctxt, bool unsafe) +{ + struct console *con = ctxt->console; + struct nbcon_state cur; + struct nbcon_state new; + + nbcon_state_read(con, &cur); + + /* The unsafe bit must not be cleared if @hostile_unsafe is set. */ + if (!unsafe && cur.hostile_unsafe) + return nbcon_context_can_proceed(ctxt, &cur); + + do { + if (!nbcon_context_can_proceed(ctxt, &cur)) + return false; + + new.atom = cur.atom; + new.unsafe = unsafe; + } while (!nbcon_state_try_cmpxchg(con, &cur, &new)); + + ctxt->unsafe = unsafe; + + return true; +} + /** * nbcon_init - Initialize the nbcon console specific data * @con: Console to initialize From patchwork Fri Jul 28 00:02:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127261 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp93003vqg; Thu, 27 Jul 2023 17:17:36 -0700 (PDT) X-Google-Smtp-Source: APBJJlHTqtnnMeKVsRG5y47OavaJ/J2HWQr3eqgTcdiokYh7LiCh3mTVqHsHaKyVP/bLIpdoAhKC X-Received: by 2002:a05:6402:397:b0:51e:2e39:9003 with SMTP id o23-20020a056402039700b0051e2e399003mr300041edv.40.1690503455776; Thu, 27 Jul 2023 17:17:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690503455; cv=none; d=google.com; s=arc-20160816; b=RY8JpD1e/CqyBQbYDuNYzJuSyuaDXUneJBHew63zcJiMWQNkF8MoemyYNbd+WcSqxN 6bONmq0xOvJZWOyomUI3rCAisPRiFmf9WhTto47IQ4fuuQrNyA2j4MV8Tqv+O1d971iE F2QiGcOr9ZX5l+0PGx3lGfeFReH8oDG+qMIi/5J2xU6Ozk/NO+KTh7iBZEb7NTHyWYR8 TfwBpY40DjBVDhQhEoSGS4X360EuhyxqAwuoslXx8vZflYH9CtrJxH6zN93pB/16u/Rr SbJTwSsfxw1aR+FzbZp1YV4fsKanViPldE7nsWbJL4TeixYBNFB77z6d08ue/QGjLcCN 98Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=+2Y16kupdutHjhiUaBB096BqHV4UeSVGmrxKDvfMgWs=; fh=I/6bvJz6sWq8p1xoNUgGjc7lme65GQ/wcdq821dg0a8=; b=FPQJMUCbzAjEek7VGIbCR2PzOJ0Sn1rMB7sB9n+xsaKD33XDo4fMM5tNYufbGCNyOG Alp6xXw2SbjigfNA6szoBvKBnPr8ohmoGnCTWmfrLphsd/8Am+fcJX5H7LXGbl2RT4Y5 mYZ++brMX0iP/G/qDyF+TsxfOsRL1cW80o/KzmxSyo4pQ7iPVY3jq/G/7GdY0At0hzeY u98OgZbAmI4/anezvZTZc+Pf8luhOt4ZMXiRp49zTJ5FW6PdOv9mBWeY0jN1RRQKKBEz HEIEORGZsSkT+BUHwq9VN1Y/3MNDEqoNdIIyUbAaaJQCbnoHYdO7i4OdTAhFWLHA+75m +3qg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=sgipofk7; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="Ui+XH/mS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r16-20020aa7d590000000b005223f4f74a8si1659091edq.398.2023.07.27.17.17.11; Thu, 27 Jul 2023 17:17:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=sgipofk7; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="Ui+XH/mS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232029AbjG1ADH (ORCPT + 99 others); Thu, 27 Jul 2023 20:03:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231570AbjG1ACp (ORCPT ); Thu, 27 Jul 2023 20:02:45 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 006B1BF for ; Thu, 27 Jul 2023 17:02:42 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+2Y16kupdutHjhiUaBB096BqHV4UeSVGmrxKDvfMgWs=; b=sgipofk7vGitPK8JkqZ6tuBbrENDClmWXVbTPa43JXKL3wJBb8zRCefBwj83Rpnr0D2mnC +ofWrSWlqWlAI1cVj8QU4hIKCjX03YfIvvjFM/yUMotvcn+BurtrxepWdzN+LUSAbLRDMG wk8TQrh/awtK438uUGWAoxIBSZwYJK6hSnwevk9Im/3DTk76wAWIYUhyUDIkTBT2bmcTRX LocX8AJqRFLO9OikS8e4x/tXumI08M4VdnZ/j4pfdRq7As6pHbgtHJYnKgwSKLvs+xCTCP dWYsmiwtojGS/DfX63DdYDcSW5pTsW6I0z7SvFrsq3Kk48SlDAkFMKqx2BAERw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+2Y16kupdutHjhiUaBB096BqHV4UeSVGmrxKDvfMgWs=; b=Ui+XH/mSQ/PmaAfWhLql0P8DmP9eOR63WLUKjroDBUypMeaYhfvNbbqwul4r/A6hNb61hB 5pbHj2pM091ZISDw== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 7/8] printk: nbcon: Add emit function and callback function for atomic printing Date: Fri, 28 Jul 2023 02:08:32 +0206 Message-Id: <20230728000233.50887-8-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772621352098236365 X-GMAIL-MSGID: 1772621352098236365 From: Thomas Gleixner Implement an emit function for nbcon consoles to output printk messages. It utilizes the lockless printk_get_next_message() and console_prepend_dropped() functions to retrieve/build the output message. The emit function includes the required safety points to check for handover/takeover and calls a new write_atomic callback of the console driver to output the message. It also includes proper handling for updating the nbcon console sequence number. A new nbcon_write_context struct is introduced. This is provided to the write_atomic callback and includes only the information necessary for performing atomic writes. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) --- include/linux/console.h | 27 ++++++++ kernel/printk/internal.h | 6 ++ kernel/printk/printk.c | 9 +-- kernel/printk/printk_nbcon.c | 121 ++++++++++++++++++++++++++++++++++- 4 files changed, 156 insertions(+), 7 deletions(-) diff --git a/include/linux/console.h b/include/linux/console.h index a50eaa3b8420..3d129b2b70a1 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -243,6 +243,7 @@ struct printk_buffers; * @unsafe: This context is in an unsafe section * @hostile: Acquire console by hostile takeover * @takeover_unsafe: Acquire console by hostile takeover even if unsafe + * @backlog: Ringbuffer has pending records * @pbufs: Pointer to the text buffer for this context * @seq: The sequence number to print for this context */ @@ -255,11 +256,28 @@ struct nbcon_context { unsigned int hostile : 1; unsigned int takeover_unsafe : 1; + /* members set by emit */ + unsigned int backlog : 1; + /* members set by acquire */ struct printk_buffers *pbufs; u64 seq; }; +/** + * struct nbcon_write_context - Context handed to the nbcon write callbacks + * @ctxt: The core console context + * @outbuf: Pointer to the text buffer for output + * @len: Length to write + * @hostile_unsafe: The @unsafe value before a hostile takeover + */ +struct nbcon_write_context { + struct nbcon_context __private ctxt; + char *outbuf; + unsigned int len; + bool hostile_unsafe; +}; + /** * struct console - The console descriptor structure * @name: The name of the console driver @@ -280,6 +298,7 @@ struct nbcon_context { * @data: Driver private data * @node: hlist node for the console list * + * @write_atomic: Write callback for atomic context * @nbcon_state: State for nbcon consoles * @nbcon_seq: Sequence number of the next record for nbcon to print * @pbufs: Pointer to nbcon private buffer @@ -304,6 +323,8 @@ struct console { struct hlist_node node; /* nbcon console specific members */ + bool (*write_atomic)(struct console *con, + struct nbcon_write_context *wctxt); atomic_t __private nbcon_state; atomic_long_t __private nbcon_seq; struct printk_buffers *pbufs; @@ -433,6 +454,12 @@ static inline bool console_is_registered(const struct console *con) lockdep_assert_console_list_lock_held(); \ hlist_for_each_entry(con, &console_list, node) +#ifdef CONFIG_PRINTK +extern bool nbcon_can_proceed(struct nbcon_write_context *wctxt); +#else +static inline bool nbcon_can_proceed(struct nbcon_write_context *wctxt) { return false; } +#endif + extern int console_set_on_cmdline; extern struct console *early_console; diff --git a/kernel/printk/internal.h b/kernel/printk/internal.h index 1baccddf83b8..96dcb2cc05b1 100644 --- a/kernel/printk/internal.h +++ b/kernel/printk/internal.h @@ -128,3 +128,9 @@ struct printk_message { }; bool other_cpu_in_panic(void); +bool printk_get_next_message(struct printk_message *pmsg, u64 seq, + bool is_extended, bool may_supress); + +#ifdef CONFIG_PRINTK +void console_prepend_dropped(struct printk_message *pmsg, unsigned long dropped); +#endif diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 64e404aacbc4..0124c433da9d 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -716,9 +716,6 @@ static ssize_t msg_print_ext_body(char *buf, size_t size, return len; } -static bool printk_get_next_message(struct printk_message *pmsg, u64 seq, - bool is_extended, bool may_supress); - /* /dev/kmsg - userspace message inject/listen interface */ struct devkmsg_user { atomic64_t seq; @@ -2751,7 +2748,7 @@ static void __console_unlock(void) * If @pmsg->pbufs->outbuf is modified, @pmsg->outbuf_len is updated. */ #ifdef CONFIG_PRINTK -static void console_prepend_dropped(struct printk_message *pmsg, unsigned long dropped) +void console_prepend_dropped(struct printk_message *pmsg, unsigned long dropped) { struct printk_buffers *pbufs = pmsg->pbufs; const size_t scratchbuf_sz = sizeof(pbufs->scratchbuf); @@ -2805,8 +2802,8 @@ static void console_prepend_dropped(struct printk_message *pmsg, unsigned long d * of @pmsg are valid. (See the documentation of struct printk_message * for information about the @pmsg fields.) */ -static bool printk_get_next_message(struct printk_message *pmsg, u64 seq, - bool is_extended, bool may_suppress) +bool printk_get_next_message(struct printk_message *pmsg, u64 seq, + bool is_extended, bool may_suppress) { static int panic_console_dropped; diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index e41f2eff5ef6..e1f0e4278ffa 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -198,7 +198,6 @@ void nbcon_seq_force(struct console *con, u64 seq) * In case of fail the console has been likely taken over. However, the * caller must still assume it has ownership and decide how to proceed. */ -__maybe_unused static bool nbcon_seq_try_update(struct nbcon_context *ctxt) { struct console *con = ctxt->console; @@ -717,6 +716,33 @@ static bool nbcon_context_can_proceed(struct nbcon_context *ctxt, struct nbcon_s return false; } +/** + * nbcon_can_proceed - Check whether ownership can proceed + * @wctxt: The write context that was handed to the write function + * + * Return: True if the state is correct. False if ownership was + * handed over or taken. + * + * Must be invoked after the record was dumped into the assigned buffer + * and at appropriate safe places in the driver. + * + * When this function returns false then the calling context is no longer + * the owner and is no longer allowed to go forward. In this case it must + * back out immediately and carefully. The buffer content is also no longer + * trusted since it no longer belongs to the calling context. + */ +bool nbcon_can_proceed(struct nbcon_write_context *wctxt) +{ + struct nbcon_context *ctxt = &ACCESS_PRIVATE(wctxt, ctxt); + struct console *con = ctxt->console; + struct nbcon_state cur; + + nbcon_state_read(con, &cur); + + return nbcon_context_can_proceed(ctxt, &cur); +} +EXPORT_SYMBOL_GPL(nbcon_can_proceed); + /** * nbcon_context_update_unsafe - Update the unsafe bit in @con->nbcon_state * @ctxt: The nbcon context from nbcon_context_try_acquire() @@ -761,6 +787,99 @@ static bool nbcon_context_update_unsafe(struct nbcon_context *ctxt, bool unsafe) return true; } +/** + * nbcon_emit_next_record - Emit a record in the acquired context + * @wctxt: The write context that will be handed to the write function + * + * Return: True if the state is correct. False if ownership was + * handed over or taken. + * + * When this function returns false then the calling context is no longer + * the owner and is no longer allowed to go forward. In this case it must + * back out immediately and carefully. The buffer content is also no longer + * trusted since it no longer belongs to the calling context. If the caller + * wants to do more it must reacquire the console first. + * + * When true is returned, @wctxt->ctxt.backlog indicates whether there are + * still records pending in the ringbuffer, + */ +__maybe_unused +static bool nbcon_emit_next_record(struct nbcon_write_context *wctxt) +{ + struct nbcon_context *ctxt = &ACCESS_PRIVATE(wctxt, ctxt); + struct console *con = ctxt->console; + bool is_extended = console_srcu_read_flags(con) & CON_EXTENDED; + struct printk_message pmsg = { + .pbufs = ctxt->pbufs, + }; + unsigned long con_dropped; + struct nbcon_state cur; + unsigned long dropped; + bool done; + + ctxt->backlog = printk_get_next_message(&pmsg, ctxt->seq, is_extended, true); + if (!ctxt->backlog) + return true; + + /* + * @con->dropped is not protected in case of hostile takeovers. In + * that situation the update can be racy so annotate it accordingly. + */ + con_dropped = data_race(READ_ONCE(con->dropped)); + + dropped = con_dropped + pmsg.dropped; + if (dropped && !is_extended) + console_prepend_dropped(&pmsg, dropped); + + /* Safety point. Do not touch state in case of takeover. */ + nbcon_state_read(con, &cur); + if (!nbcon_context_can_proceed(ctxt, &cur)) + return false; + + /* For skipped records just update seq/dropped in @con. */ + if (pmsg.outbuf_len == 0) + goto update_con; + + /* Set the write context before calling write callback. */ + wctxt->hostile_unsafe = cur.hostile_unsafe; + wctxt->len = pmsg.outbuf_len; + if (wctxt->len) + wctxt->outbuf = &pmsg.pbufs->outbuf[0]; + else + wctxt->outbuf = NULL; + + if (con->write_atomic) { + done = con->write_atomic(con, wctxt); + } else { + nbcon_context_release(ctxt); + WARN_ON_ONCE(1); + done = false; + } + + /* If not done, the emit was aborted. */ + if (!done) + return false; + + /* + * Since any dropped message was successfully output, reset the + * dropped count for the console. + */ + dropped = 0; +update_con: + if (dropped != con_dropped) { + /* Counterpart to the READ_ONCE() above. */ + WRITE_ONCE(con->dropped, dropped); + } + + ctxt->seq = pmsg.seq + 1; + if (!nbcon_seq_try_update(ctxt)) { + /* This context probably lost ownership. Check. */ + return nbcon_can_proceed(wctxt); + } + + return true; +} + /** * nbcon_init - Initialize the nbcon console specific data * @con: Console to initialize From patchwork Fri Jul 28 00:02:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Ogness X-Patchwork-Id: 127264 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp98072vqg; Thu, 27 Jul 2023 17:30:11 -0700 (PDT) X-Google-Smtp-Source: APBJJlFo4FH3x7b6JrF6ni4GvkoypJlsce/hhzbvLdh7Vekaj5fyLfEMUZIlq3UNy8sG+BxDpmnE X-Received: by 2002:a17:906:cc4b:b0:993:f2b4:13c9 with SMTP id mm11-20020a170906cc4b00b00993f2b413c9mr533571ejb.21.1690504211258; Thu, 27 Jul 2023 17:30:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690504211; cv=none; d=google.com; s=arc-20160816; b=Br4HWhMdBs6Ke2eYT60zz47X2G/gCCV/w9Ce3MDziQHczLIQ3ZVofnqNtrD3MeIbZA FlGzrZ+jzyFMf467MJ0RIUtKaRkcchqYcOHJIiLpe78ncCtlIhYZSlNvZXJJ/FHHOfdf McTxZAvr5BDypPkA1cFrT6TxS0pkExg3dFf1U8A7qBD2N2gEyS1yMfZ/DhtkhvqdXD2Z V5D94Au6Ieei/yPLlWqFmTFU81Vhx9wLOEokwIhB1CVCJ1/x2ftikA/0+zWDdwLM1Osz cMlNaXQ4vDc/5fQzpnjz37OratVw3C/lSK8/jwaJoOIsgNBvoeNkqDFZYIyAdnnqLMQW mvPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:dkim-signature :dkim-signature:from; bh=xb4IARB7ZGs0NFGdahNKOcVuwqCsWQdgRW9ky2NkqP0=; fh=I/6bvJz6sWq8p1xoNUgGjc7lme65GQ/wcdq821dg0a8=; b=wU4p6iOcKrfcmAcIU5TotcbaXqHDpd4gy8LKV0dMlVPd/7ZaGVAMmVZKqYpHYuqhS3 kMA/URb1ZOSrT48VZXVOe2jIkyn1ft3GnDvQTUdYWUB8vvLAiKG7y5V96a/QmuGQ0OgL /+08OB5szoMxLDppybXhfx0BRpBDQ1RNnxNell8wl1E0V9FkJaAT1/RPn92JOh9ImBVD geOMfCvF/Ia/Kz9wYTWZW76YakUiwinH+bhJb97QpPVlPFdYqh8ajVM+BTCfj2is+63C zMcg3Mp2Vf+tDr+V0a+GwXoczWG0d8WJo//vryLs0T2mcLI21n8+lo1vk45SHvB4hNXe zQLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=bHC60CvV; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=OClN8Ozq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ga6-20020a170906b84600b00989027eb30bsi1929120ejb.610.2023.07.27.17.29.46; Thu, 27 Jul 2023 17:30:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=bHC60CvV; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=OClN8Ozq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231697AbjG1ADA (ORCPT + 99 others); Thu, 27 Jul 2023 20:03:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231481AbjG1ACn (ORCPT ); Thu, 27 Jul 2023 20:02:43 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 025E0170D for ; Thu, 27 Jul 2023 17:02:42 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1690502559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xb4IARB7ZGs0NFGdahNKOcVuwqCsWQdgRW9ky2NkqP0=; b=bHC60CvVn0k5V/Ob2wGVruqaoijuC5OhwuGq8OdO28M1v7kmYBNC5Pcqlj112Tv329px04 xoOv0JbDAZi0d03qVzIOkCpF4WQqlZHC5grMzu3eSIEdETyqjHzI4FFpNF4tAECymfMoU/ 1prM90FTfK0afeyH9Y3/MWGPlJWUQ8RpEzLBEje2JR+Dgnci+X/hzJ6zRCI7/1JSuJ7MnM gQS9e8ch2R/Fu0sDtea01twbCChPWz4ci6k472ezTOlD3l01I+CjNir1j30oIDTNLEtWJe N8uEgfZ9p79D7fCuV51NoJidOWaApXHLw0GmUz/EpfFxkjiqifa9L1ptEHhEbA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1690502559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xb4IARB7ZGs0NFGdahNKOcVuwqCsWQdgRW9ky2NkqP0=; b=OClN8Ozq+PUpq9KrNWc9iSN08LxWbFx7BcujZAphNTRhSaICw0IwD2aaHDjeSPsnVeuht6 4AgtmvM4HAAqv/CA== To: Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman Subject: [PATCH printk v2 8/8] printk: nbcon: Add functions for drivers to mark unsafe regions Date: Fri, 28 Jul 2023 02:08:33 +0206 Message-Id: <20230728000233.50887-9-john.ogness@linutronix.de> In-Reply-To: <20230728000233.50887-1-john.ogness@linutronix.de> References: <20230728000233.50887-1-john.ogness@linutronix.de> MIME-Version: 1.0 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772622143657711408 X-GMAIL-MSGID: 1772622143657711408 From: Thomas Gleixner For the write_atomic callback, the console driver may have unsafe regions that need to be appropriately marked. Provide functions that accept the nbcon_write_context struct to allow for the driver to enter and exit unsafe regions. Co-developed-by: John Ogness Signed-off-by: John Ogness Signed-off-by: Thomas Gleixner (Intel) --- include/linux/console.h | 4 ++++ kernel/printk/printk_nbcon.c | 41 +++++++++++++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/include/linux/console.h b/include/linux/console.h index 3d129b2b70a1..e4ce1015627c 100644 --- a/include/linux/console.h +++ b/include/linux/console.h @@ -456,8 +456,12 @@ static inline bool console_is_registered(const struct console *con) #ifdef CONFIG_PRINTK extern bool nbcon_can_proceed(struct nbcon_write_context *wctxt); +extern bool nbcon_enter_unsafe(struct nbcon_write_context *wctxt); +extern bool nbcon_exit_unsafe(struct nbcon_write_context *wctxt); #else static inline bool nbcon_can_proceed(struct nbcon_write_context *wctxt) { return false; } +static inline bool nbcon_enter_unsafe(struct nbcon_write_context *wctxt) { return false; } +static inline bool nbcon_exit_unsafe(struct nbcon_write_context *wctxt) { return false; } #endif extern int console_set_on_cmdline; diff --git a/kernel/printk/printk_nbcon.c b/kernel/printk/printk_nbcon.c index e1f0e4278ffa..57b7539bbbb2 100644 --- a/kernel/printk/printk_nbcon.c +++ b/kernel/printk/printk_nbcon.c @@ -761,7 +761,6 @@ EXPORT_SYMBOL_GPL(nbcon_can_proceed); * * Internal helper to avoid duplicated code */ -__maybe_unused static bool nbcon_context_update_unsafe(struct nbcon_context *ctxt, bool unsafe) { struct console *con = ctxt->console; @@ -787,6 +786,46 @@ static bool nbcon_context_update_unsafe(struct nbcon_context *ctxt, bool unsafe) return true; } +/** + * nbcon_enter_unsafe - Enter an unsafe region in the driver + * @wctxt: The write context that was handed to the write function + * + * Return: True if the state is correct. False if ownership was + * handed over or taken. + * + * When this function returns false then the calling context is no longer + * the owner and is no longer allowed to go forward. In this case it must + * back out immediately and carefully. The buffer content is also no longer + * trusted since it no longer belongs to the calling context. + */ +bool nbcon_enter_unsafe(struct nbcon_write_context *wctxt) +{ + struct nbcon_context *ctxt = &ACCESS_PRIVATE(wctxt, ctxt); + + return nbcon_context_update_unsafe(ctxt, true); +} +EXPORT_SYMBOL_GPL(nbcon_enter_unsafe); + +/** + * nbcon_exit_unsafe - Exit an unsafe region in the driver + * @wctxt: The write context that was handed to the write function + * + * Return: True if the state is correct. False if ownership was + * handed over or taken. + * + * When this function returns false then the calling context is no longer + * the owner and is no longer allowed to go forward. In this case it must + * back out immediately and carefully. The buffer content is also no longer + * trusted since it no longer belongs to the calling context. + */ +bool nbcon_exit_unsafe(struct nbcon_write_context *wctxt) +{ + struct nbcon_context *ctxt = &ACCESS_PRIVATE(wctxt, ctxt); + + return nbcon_context_update_unsafe(ctxt, false); +} +EXPORT_SYMBOL_GPL(nbcon_exit_unsafe); + /** * nbcon_emit_next_record - Emit a record in the acquired context * @wctxt: The write context that will be handed to the write function