From patchwork Thu Dec 21 16:10:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182332 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp515904dyi; Thu, 21 Dec 2023 08:10:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IGGyJO9ix6Bjqr9c//0KOkTOBR9xsHI6pue4AJ3KVQbJX7txyYfuhnkG82hihpMTMHtjMvG X-Received: by 2002:a50:bb44:0:b0:553:2b04:14a2 with SMTP id y62-20020a50bb44000000b005532b0414a2mr4623520ede.14.1703175044654; Thu, 21 Dec 2023 08:10:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175044; cv=none; d=google.com; s=arc-20160816; b=oxO+//RsoGe4iZ9eHuKxHRKuDEAxnGNqcdsdaq2JdcbSYHhaIj0lkQ2xWWrODiLlXW 0moVw0t8FJX1qtA2YEFUOIXDAClDytCHh4uYxbG9mO4BCSN+OnSatCjh23QEX1Pn5VfG tUU/UJ/Ni/+MeNq+voT8hwp0A1fEzgzK6lxY+6uX7SftJrt60h5XGSmGI1Z5jpR18AXL fo5YkolaVbrDaZ5+xUkQBYqbzcdQzJiIubLwX0naH6+FEfBwG+wD15MVwI1AK8t1z38b CFs1wBZSQ5k0DNXmsIrVtGfP513OjIPDDuZc1dcoVLP7hCzhlUGEovgcXlH/mn7CiMQb G5zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=7W4tJDRcxYCqg7goZIztsuHDwmrGC2lzvR1FzyukStA=; fh=vS4GW7DeA1AMEc8SlhVYPDpeY9+vqwFsekYGU4Lsths=; b=COJs0jKPICJPzaPTAZwfBcYF+WrAgqW63vm+rszrFI7hroJKXaUxoG/gTYEPwYvPCH Hl0oa+LUbS6eGbrFjTKfQRwdvRmc9LRF6Zn1RWimeCSineRMkvz4b2zHP+/ZzWkgm4u+ MtsmHreFop1HbKM8Y2L0uJMEy6RAlcSfFc2rCi4HMfQZpgo3x1HAgduYtweoDujRdoL6 gd9ZkDaDan09anrmMeEcJtIRpc+hcNzIPBdbGWBz5wnVMKHhlrJlLbaDeG5A2QBzNeDu GZyM+HlB0Kkf7KtdG0hXPSuuxxbUXDSwtieJ1uosYN/UBzaHD1fQxoUhg28nbujAfHUu KOcA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8724-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8724-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id e11-20020a50a68b000000b00554013391b7si998426edc.45.2023.12.21.08.10.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:10:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8724-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8724-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8724-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 46E2A1F23EEF for ; Thu, 21 Dec 2023 16:10:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8731A59901; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0374E55E4C for ; Thu, 21 Dec 2023 16:09:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D0BB7C433CB; Thu, 21 Dec 2023 16:09:58 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdW-000000041Gb-2oc5; Thu, 21 Dec 2023 11:11:02 -0500 Message-ID: <20231221161102.453129978@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:25 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort , Kent Overstreet , "Tzvetomir Stoyanov (VMware)" Subject: [for-next][PATCH 01/16] ring-buffer: Have ring_buffer_print_page_header() be able to access ring_buffer_iter References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908475637511820 X-GMAIL-MSGID: 1785908475637511820 From: "Tzvetomir Stoyanov (VMware)" In order to introduce sub-buffer size per ring buffer, some internal refactoring is needed. As ring_buffer_print_page_header() will depend on the trace_buffer structure, it is moved after the structure definition. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-2-tz.stoyanov@gmail.com Link: https://lore.kernel.org/linux-trace-kernel/20231219185627.723857541@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 60 +++++++++++++++++++------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index f7dc74e45ebf..2400c8e68fd3 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -379,36 +379,6 @@ static inline bool test_time_stamp(u64 delta) /* Max payload is BUF_PAGE_SIZE - header (8bytes) */ #define BUF_MAX_DATA_SIZE (BUF_PAGE_SIZE - (sizeof(u32) * 2)) -int ring_buffer_print_page_header(struct trace_seq *s) -{ - struct buffer_data_page field; - - trace_seq_printf(s, "\tfield: u64 timestamp;\t" - "offset:0;\tsize:%u;\tsigned:%u;\n", - (unsigned int)sizeof(field.time_stamp), - (unsigned int)is_signed_type(u64)); - - trace_seq_printf(s, "\tfield: local_t commit;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), commit), - (unsigned int)sizeof(field.commit), - (unsigned int)is_signed_type(long)); - - trace_seq_printf(s, "\tfield: int overwrite;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), commit), - 1, - (unsigned int)is_signed_type(long)); - - trace_seq_printf(s, "\tfield: char data;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), data), - (unsigned int)BUF_PAGE_SIZE, - (unsigned int)is_signed_type(char)); - - return !trace_seq_has_overflowed(s); -} - struct rb_irq_work { struct irq_work work; wait_queue_head_t waiters; @@ -556,6 +526,36 @@ struct ring_buffer_iter { int missed_events; }; +int ring_buffer_print_page_header(struct trace_seq *s) +{ + struct buffer_data_page field; + + trace_seq_printf(s, "\tfield: u64 timestamp;\t" + "offset:0;\tsize:%u;\tsigned:%u;\n", + (unsigned int)sizeof(field.time_stamp), + (unsigned int)is_signed_type(u64)); + + trace_seq_printf(s, "\tfield: local_t commit;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), commit), + (unsigned int)sizeof(field.commit), + (unsigned int)is_signed_type(long)); + + trace_seq_printf(s, "\tfield: int overwrite;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), commit), + 1, + (unsigned int)is_signed_type(long)); + + trace_seq_printf(s, "\tfield: char data;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), data), + (unsigned int)BUF_PAGE_SIZE, + (unsigned int)is_signed_type(char)); + + return !trace_seq_has_overflowed(s); +} + static inline void rb_time_read(rb_time_t *t, u64 *ret) { *ret = local64_read(&t->time); From patchwork Thu Dec 21 16:10:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182334 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp516998dyi; Thu, 21 Dec 2023 08:12:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IFjWGuh36oxReiAw07VrIGOZ/KLa3GUS6fRTgYkr21JxBQMxh+ERvMDowYrxDIlcb5ZOvtc X-Received: by 2002:a17:90b:b0b:b0:28b:f9fc:6cd0 with SMTP id bf11-20020a17090b0b0b00b0028bf9fc6cd0mr482656pjb.51.1703175128240; Thu, 21 Dec 2023 08:12:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175128; cv=none; d=google.com; s=arc-20160816; b=qhYaGtS0FB7ae7S3ZMC8PlxTOrpfzwCN9MnqLZnsKbHVDvbVAGkprTnF0A84AQf7uf E1GbCYzRCsMgjXpBB89hQQmzkBFR96h3XqkzrXEQ306Myhj4KcbToPZU9dhngOOdegcl f83y+4tG6M1MXMz6KLa+BmJxjRHkyS7krX3XlF+jZAEFfFLqkOvmzTG3YQGfhnJuO1h0 7hbEwf8q5CvU78KPAddXAGC5lZGKjBKp4wcTaxjE7m9JYTBvt5KeyR6r75FIowOko/hV FrSFBvppgnsQ7osURAXvZ9X9LUxmNqZadEnQ24ETI+8wV/2JC76ecPS3oQd/+hANQSzp sIWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=rUDJDt2nEqH1CCIC1tBQdcqPzyC/RDlTibuj3XNQp/8=; fh=vS4GW7DeA1AMEc8SlhVYPDpeY9+vqwFsekYGU4Lsths=; b=0vQffBqVpWA/i6zO7spelLR3apg6ibGscHitw8YMrGBy3UL3UxNLJp/J5/Vz3IZedB +ncrHFm7PjSH7w7usXn33zI1IAf7JD/kGNqLKtQewNlmHpwBC76iEVasGmzzcLY0otqm ufaZ1CuKGsQgI1Vcu3GIKYpTwh0iv8sPqw/QFLd6StiHzwdIF4SEDXA3oAXwpMw4zC+Z iT4d3s2qO/I5Mn9MWCf52+3fmxc0u1oaf1IaYuUCoyioJeFxkBpxaA2Y/QVVzE9W5YJa THse1PwsFpJCZrmzsM4BlqLLE0oaCad2CwSGg1wiJiZcUGQla17gyMhfR7RmtnxPCsAr op1g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8726-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8726-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id c21-20020a17090a8d1500b0028bec866b0bsi1608377pjo.11.2023.12.21.08.12.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:12:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8726-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8726-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8726-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 4224BB22F71 for ; Thu, 21 Dec 2023 16:11:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4917A627EC; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ECC355E71 for ; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 00DA0C433C9; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdW-000000041H5-3za8; Thu, 21 Dec 2023 11:11:02 -0500 Message-ID: <20231221161102.744146443@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:26 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort , Kent Overstreet , "Tzvetomir Stoyanov (VMware)" Subject: [for-next][PATCH 02/16] ring-buffer: Page size per ring buffer References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908563597116737 X-GMAIL-MSGID: 1785908563597116737 From: "Tzvetomir Stoyanov (VMware)" Currently the size of one sub buffer page is global for all buffers and it is hard coded to one system page. In order to introduce configurable ring buffer sub page size, the internal logic should be refactored to work with sub page size per ring buffer. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-3-tz.stoyanov@gmail.com Link: https://lore.kernel.org/linux-trace-kernel/20231219185628.009147038@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 2 +- kernel/trace/ring_buffer.c | 68 +++++++++++++++++++++---------------- kernel/trace/trace.c | 2 +- kernel/trace/trace.h | 1 + kernel/trace/trace_events.c | 59 ++++++++++++++++++++++++-------- 5 files changed, 86 insertions(+), 46 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index b1b03b2c0f08..ce46218ce46d 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -200,7 +200,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, void **data_page, struct trace_seq; int ring_buffer_print_entry_header(struct trace_seq *s); -int ring_buffer_print_page_header(struct trace_seq *s); +int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s); enum ring_buffer_flags { RB_FL_OVERWRITE = 1 << 0, diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 2400c8e68fd3..d9f656502400 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -374,11 +374,6 @@ static inline bool test_time_stamp(u64 delta) return !!(delta & TS_DELTA_TEST); } -#define BUF_PAGE_SIZE (PAGE_SIZE - BUF_PAGE_HDR_SIZE) - -/* Max payload is BUF_PAGE_SIZE - header (8bytes) */ -#define BUF_MAX_DATA_SIZE (BUF_PAGE_SIZE - (sizeof(u32) * 2)) - struct rb_irq_work { struct irq_work work; wait_queue_head_t waiters; @@ -510,6 +505,9 @@ struct trace_buffer { struct rb_irq_work irq_work; bool time_stamp_abs; + + unsigned int subbuf_size; + unsigned int max_data_size; }; struct ring_buffer_iter { @@ -523,10 +521,11 @@ struct ring_buffer_iter { u64 read_stamp; u64 page_stamp; struct ring_buffer_event *event; + size_t event_size; int missed_events; }; -int ring_buffer_print_page_header(struct trace_seq *s) +int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s) { struct buffer_data_page field; @@ -550,7 +549,7 @@ int ring_buffer_print_page_header(struct trace_seq *s) trace_seq_printf(s, "\tfield: char data;\t" "offset:%u;\tsize:%u;\tsigned:%u;\n", (unsigned int)offsetof(typeof(field), data), - (unsigned int)BUF_PAGE_SIZE, + (unsigned int)buffer->subbuf_size, (unsigned int)is_signed_type(char)); return !trace_seq_has_overflowed(s); @@ -1625,7 +1624,13 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags, if (!zalloc_cpumask_var(&buffer->cpumask, GFP_KERNEL)) goto fail_free_buffer; - nr_pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE); + /* Default buffer page size - one system page */ + buffer->subbuf_size = PAGE_SIZE - BUF_PAGE_HDR_SIZE; + + /* Max payload is buffer page size - header (8bytes) */ + buffer->max_data_size = buffer->subbuf_size - (sizeof(u32) * 2); + + nr_pages = DIV_ROUND_UP(size, buffer->subbuf_size); buffer->flags = flags; buffer->clock = trace_clock_local; buffer->reader_lock_key = key; @@ -1944,7 +1949,7 @@ static void update_pages_handler(struct work_struct *work) * @size: the new size. * @cpu_id: the cpu buffer to resize * - * Minimum size is 2 * BUF_PAGE_SIZE. + * Minimum size is 2 * buffer->subbuf_size. * * Returns 0 on success and < 0 on failure. */ @@ -1966,7 +1971,7 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, !cpumask_test_cpu(cpu_id, buffer->cpumask)) return 0; - nr_pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE); + nr_pages = DIV_ROUND_UP(size, buffer->subbuf_size); /* we need a minimum of two pages */ if (nr_pages < 2) @@ -2213,7 +2218,7 @@ rb_iter_head_event(struct ring_buffer_iter *iter) */ barrier(); - if ((iter->head + length) > commit || length > BUF_PAGE_SIZE) + if ((iter->head + length) > commit || length > iter->event_size) /* Writer corrupted the read? */ goto reset; @@ -2446,6 +2451,7 @@ static inline void rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, unsigned long tail, struct rb_event_info *info) { + unsigned long bsize = READ_ONCE(cpu_buffer->buffer->subbuf_size); struct buffer_page *tail_page = info->tail_page; struct ring_buffer_event *event; unsigned long length = info->length; @@ -2454,13 +2460,13 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * Only the event that crossed the page boundary * must fill the old tail_page with padding. */ - if (tail >= BUF_PAGE_SIZE) { + if (tail >= bsize) { /* * If the page was filled, then we still need * to update the real_end. Reset it to zero * and the reader will ignore it. */ - if (tail == BUF_PAGE_SIZE) + if (tail == bsize) tail_page->real_end = 0; local_sub(length, &tail_page->write); @@ -2488,7 +2494,7 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * If we are less than the minimum size, we don't need to * worry about it. */ - if (tail > (BUF_PAGE_SIZE - RB_EVNT_MIN_SIZE)) { + if (tail > (bsize - RB_EVNT_MIN_SIZE)) { /* No room for any events */ /* Mark the rest of the page with padding */ @@ -2503,19 +2509,19 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, } /* Put in a discarded event */ - event->array[0] = (BUF_PAGE_SIZE - tail) - RB_EVNT_HDR_SIZE; + event->array[0] = (bsize - tail) - RB_EVNT_HDR_SIZE; event->type_len = RINGBUF_TYPE_PADDING; /* time delta must be non zero */ event->time_delta = 1; /* account for padding bytes */ - local_add(BUF_PAGE_SIZE - tail, &cpu_buffer->entries_bytes); + local_add(bsize - tail, &cpu_buffer->entries_bytes); /* Make sure the padding is visible before the tail_page->write update */ smp_wmb(); /* Set write to end of buffer */ - length = (tail + length) - BUF_PAGE_SIZE; + length = (tail + length) - bsize; local_sub(length, &tail_page->write); } @@ -3469,7 +3475,7 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer, tail = write - info->length; /* See if we shot pass the end of this buffer page */ - if (unlikely(write > BUF_PAGE_SIZE)) { + if (unlikely(write > cpu_buffer->buffer->subbuf_size)) { check_buffer(cpu_buffer, info, CHECK_FULL_PAGE); return rb_move_tail(cpu_buffer, tail, info); } @@ -3600,7 +3606,7 @@ rb_reserve_next_event(struct trace_buffer *buffer, if (ring_buffer_time_stamp_abs(cpu_buffer->buffer)) { add_ts_default = RB_ADD_STAMP_ABSOLUTE; info.length += RB_LEN_TIME_EXTEND; - if (info.length > BUF_MAX_DATA_SIZE) + if (info.length > cpu_buffer->buffer->max_data_size) goto out_fail; } else { add_ts_default = RB_ADD_STAMP_NONE; @@ -3675,7 +3681,7 @@ ring_buffer_lock_reserve(struct trace_buffer *buffer, unsigned long length) if (unlikely(atomic_read(&cpu_buffer->record_disabled))) goto out; - if (unlikely(length > BUF_MAX_DATA_SIZE)) + if (unlikely(length > buffer->max_data_size)) goto out; if (unlikely(trace_recursive_lock(cpu_buffer))) @@ -3825,7 +3831,7 @@ int ring_buffer_write(struct trace_buffer *buffer, if (atomic_read(&cpu_buffer->record_disabled)) goto out; - if (length > BUF_MAX_DATA_SIZE) + if (length > buffer->max_data_size) goto out; if (unlikely(trace_recursive_lock(cpu_buffer))) @@ -4405,6 +4411,7 @@ static struct buffer_page * rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) { struct buffer_page *reader = NULL; + unsigned long bsize = READ_ONCE(cpu_buffer->buffer->subbuf_size); unsigned long overwrite; unsigned long flags; int nr_loops = 0; @@ -4540,7 +4547,7 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) #define USECS_WAIT 1000000 for (nr_loops = 0; nr_loops < USECS_WAIT; nr_loops++) { /* If the write is past the end of page, a writer is still updating it */ - if (likely(!reader || rb_page_write(reader) <= BUF_PAGE_SIZE)) + if (likely(!reader || rb_page_write(reader) <= bsize)) break; udelay(1); @@ -4984,7 +4991,8 @@ ring_buffer_read_prepare(struct trace_buffer *buffer, int cpu, gfp_t flags) return NULL; /* Holds the entire event: data and meta data */ - iter->event = kmalloc(BUF_PAGE_SIZE, flags); + iter->event_size = buffer->subbuf_size; + iter->event = kmalloc(iter->event_size, flags); if (!iter->event) { kfree(iter); return NULL; @@ -5102,14 +5110,14 @@ unsigned long ring_buffer_size(struct trace_buffer *buffer, int cpu) { /* * Earlier, this method returned - * BUF_PAGE_SIZE * buffer->nr_pages + * buffer->subbuf_size * buffer->nr_pages * Since the nr_pages field is now removed, we have converted this to * return the per cpu buffer value. */ if (!cpumask_test_cpu(cpu, buffer->cpumask)) return 0; - return BUF_PAGE_SIZE * buffer->buffers[cpu]->nr_pages; + return buffer->subbuf_size * buffer->buffers[cpu]->nr_pages; } EXPORT_SYMBOL_GPL(ring_buffer_size); @@ -5123,8 +5131,8 @@ unsigned long ring_buffer_max_event_size(struct trace_buffer *buffer) { /* If abs timestamp is requested, events have a timestamp too */ if (ring_buffer_time_stamp_abs(buffer)) - return BUF_MAX_DATA_SIZE - RB_LEN_TIME_EXTEND; - return BUF_MAX_DATA_SIZE; + return buffer->max_data_size - RB_LEN_TIME_EXTEND; + return buffer->max_data_size; } EXPORT_SYMBOL_GPL(ring_buffer_max_event_size); @@ -5730,7 +5738,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* If there is room at the end of the page to save the * missed events, then record it there. */ - if (BUF_PAGE_SIZE - commit >= sizeof(missed_events)) { + if (buffer->subbuf_size - commit >= sizeof(missed_events)) { memcpy(&bpage->data[commit], &missed_events, sizeof(missed_events)); local_add(RB_MISSED_STORED, &bpage->commit); @@ -5742,8 +5750,8 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* * This page may be off to user land. Zero it out here. */ - if (commit < BUF_PAGE_SIZE) - memset(&bpage->data[commit], 0, BUF_PAGE_SIZE - commit); + if (commit < buffer->subbuf_size) + memset(&bpage->data[commit], 0, buffer->subbuf_size - commit); out_unlock: raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags); diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 55dabee4c78b..76dd0a4c8cb5 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -5018,7 +5018,7 @@ static int tracing_release(struct inode *inode, struct file *file) return 0; } -static int tracing_release_generic_tr(struct inode *inode, struct file *file) +int tracing_release_generic_tr(struct inode *inode, struct file *file) { struct trace_array *tr = inode->i_private; diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 79180aed13ee..00f873910c5d 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -616,6 +616,7 @@ void tracing_reset_all_online_cpus(void); void tracing_reset_all_online_cpus_unlocked(void); int tracing_open_generic(struct inode *inode, struct file *filp); int tracing_open_generic_tr(struct inode *inode, struct file *filp); +int tracing_release_generic_tr(struct inode *inode, struct file *file); int tracing_open_file_tr(struct inode *inode, struct file *filp); int tracing_release_file_tr(struct inode *inode, struct file *filp); int tracing_single_release_file_tr(struct inode *inode, struct file *filp); diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index b70d03818038..7c364b87352e 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -1893,9 +1893,9 @@ subsystem_filter_write(struct file *filp, const char __user *ubuf, size_t cnt, } static ssize_t -show_header(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +show_header_page_file(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) { - int (*func)(struct trace_seq *s) = filp->private_data; + struct trace_array *tr = filp->private_data; struct trace_seq *s; int r; @@ -1908,7 +1908,31 @@ show_header(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) trace_seq_init(s); - func(s); + ring_buffer_print_page_header(tr->array_buffer.buffer, s); + r = simple_read_from_buffer(ubuf, cnt, ppos, + s->buffer, trace_seq_used(s)); + + kfree(s); + + return r; +} + +static ssize_t +show_header_event_file(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +{ + struct trace_seq *s; + int r; + + if (*ppos) + return 0; + + s = kmalloc(sizeof(*s), GFP_KERNEL); + if (!s) + return -ENOMEM; + + trace_seq_init(s); + + ring_buffer_print_entry_header(s); r = simple_read_from_buffer(ubuf, cnt, ppos, s->buffer, trace_seq_used(s)); @@ -2165,10 +2189,18 @@ static const struct file_operations ftrace_tr_enable_fops = { .release = subsystem_release, }; -static const struct file_operations ftrace_show_header_fops = { - .open = tracing_open_generic, - .read = show_header, +static const struct file_operations ftrace_show_header_page_fops = { + .open = tracing_open_generic_tr, + .read = show_header_page_file, .llseek = default_llseek, + .release = tracing_release_generic_tr, +}; + +static const struct file_operations ftrace_show_header_event_fops = { + .open = tracing_open_generic_tr, + .read = show_header_event_file, + .llseek = default_llseek, + .release = tracing_release_generic_tr, }; static int @@ -3794,17 +3826,16 @@ static int events_callback(const char *name, umode_t *mode, void **data, return 1; } - if (strcmp(name, "header_page") == 0) - *data = ring_buffer_print_page_header; - - else if (strcmp(name, "header_event") == 0) - *data = ring_buffer_print_entry_header; + if (strcmp(name, "header_page") == 0) { + *mode = TRACE_MODE_READ; + *fops = &ftrace_show_header_page_fops; - else + } else if (strcmp(name, "header_event") == 0) { + *mode = TRACE_MODE_READ; + *fops = &ftrace_show_header_event_fops; + } else return 0; - *mode = TRACE_MODE_READ; - *fops = &ftrace_show_header_fops; return 1; } From patchwork Thu Dec 21 16:10:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182333 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp516400dyi; Thu, 21 Dec 2023 08:11:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IEiybwKHipTFbwyb00PQAG3fYui6im78eLt8+1E6e2y4MvIXZyjUdqkXsJfmvr0zlphFas/ X-Received: by 2002:a05:6512:31d2:b0:50e:1fa1:5ffa with SMTP id j18-20020a05651231d200b0050e1fa15ffamr5876703lfe.130.1703175081840; Thu, 21 Dec 2023 08:11:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175081; cv=none; d=google.com; s=arc-20160816; b=aQY6tFZjYBWqPoT9mDKNTDM1wWFfrAT8hg31cSarnNlYqU5U0vDgwdOzhYYkcM7oLI r6wNhIEgStak1/1D3656drE62xwU4UEDwtQKQqsv9kf8icCQpmLo4RkkmtVFotmaj/ft VsFgZqYWtyMN7YQyJxbSs09irAkPdUxKxztM+RJBccVj9PdnXCawgOczNgYCTL0zbOSf yhuoYgL9Habg+Dyhf+NAeSqCE+AJupIwP/P9ZktbEOy8QN/mDXK5wakK4712/ZAmQuVU D3519K706hh4YjFMfyXuhppvgCL2txcGU3JxpjkkbTQoNgIyF1zH2/fh5qj+lONI97dl g2nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=OpvCjBmrWK45nafrz+DoMonmHg6FpJCXSjkBXbkbqng=; fh=vS4GW7DeA1AMEc8SlhVYPDpeY9+vqwFsekYGU4Lsths=; b=xCe8nAB+q/FMZd2pDmRgioPgIqecImGM1hTm5XKYPl8wOguEGnB3tYDzq5+9CPdbAG mYRTjBuPg9gZz2Kt5T1CLsiwVIEic/eru2i7D4no+uTM6wKefup2Ug548GojkFCBrf5H 3qmTigSJalI1FjomiwwyYpDgdrvjMUHaKm39Ee8CNhVXCZBGL+AvlaklErsl2hVN4XwI C+ZFi4Y5o5gBQNIvhphisv3syOpJhkM2CikwqpRph/j02kTtq+aiF55zjtwVUzjX0hUg Nyf56I0c8YNnnyqOKKHKCXk7pQp9SqF7VRf3JeO3MXAVT2pzffqzX1xZi3AByaRNrYvD x0kw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8727-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8727-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id v5-20020a1709064e8500b00a23513d072asi997946eju.870.2023.12.21.08.11.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:11:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8727-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8727-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8727-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 4ADEB1F21CD1 for ; Thu, 21 Dec 2023 16:11:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4D35D627F1; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0B7A55E74 for ; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 43262C433C7; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdX-000000041HZ-13Li; Thu, 21 Dec 2023 11:11:03 -0500 Message-ID: <20231221161103.024976149@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:27 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort , Kent Overstreet , "Tzvetomir Stoyanov (VMware)" Subject: [for-next][PATCH 03/16] ring-buffer: Add interface for configuring trace sub buffer size References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908514239444353 X-GMAIL-MSGID: 1785908514239444353 From: "Tzvetomir Stoyanov (VMware)" The trace ring buffer sub page size can be configured, per trace instance. A new ftrace file "buffer_subbuf_order" is added to get and set the size of the ring buffer sub page for current trace instance. The size must be an order of system page size, that's why the new interface works with system page order, instead of absolute page size: 0 means the ring buffer sub page is equal to 1 system page and so forth: 0 - 1 system page 1 - 2 system pages 2 - 4 system pages ... The ring buffer sub page size is limited between 1 and 128 system pages. The default value is 1 system page. New ring buffer APIs are introduced: ring_buffer_subbuf_order_set() ring_buffer_subbuf_order_get() ring_buffer_subbuf_size_get() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-4-tz.stoyanov@gmail.com Link: https://lore.kernel.org/linux-trace-kernel/20231219185628.298324722@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 4 ++ kernel/trace/ring_buffer.c | 73 +++++++++++++++++++++++++++++++++++++ kernel/trace/trace.c | 48 ++++++++++++++++++++++++ 3 files changed, 125 insertions(+) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index ce46218ce46d..12573306b889 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -202,6 +202,10 @@ struct trace_seq; int ring_buffer_print_entry_header(struct trace_seq *s); int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s); +int ring_buffer_subbuf_order_get(struct trace_buffer *buffer); +int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order); +int ring_buffer_subbuf_size_get(struct trace_buffer *buffer); + enum ring_buffer_flags { RB_FL_OVERWRITE = 1 << 0, }; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index d9f656502400..20fc0121735d 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -507,6 +507,7 @@ struct trace_buffer { bool time_stamp_abs; unsigned int subbuf_size; + unsigned int subbuf_order; unsigned int max_data_size; }; @@ -5761,6 +5762,78 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } EXPORT_SYMBOL_GPL(ring_buffer_read_page); +/** + * ring_buffer_subbuf_size_get - get size of the sub buffer. + * @buffer: the buffer to get the sub buffer size from + * + * Returns size of the sub buffer, in bytes. + */ +int ring_buffer_subbuf_size_get(struct trace_buffer *buffer) +{ + return buffer->subbuf_size + BUF_PAGE_HDR_SIZE; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_size_get); + +/** + * ring_buffer_subbuf_order_get - get order of system sub pages in one buffer page. + * @buffer: The ring_buffer to get the system sub page order from + * + * By default, one ring buffer sub page equals to one system page. This parameter + * is configurable, per ring buffer. The size of the ring buffer sub page can be + * extended, but must be an order of system page size. + * + * Returns the order of buffer sub page size, in system pages: + * 0 means the sub buffer size is 1 system page and so forth. + * In case of an error < 0 is returned. + */ +int ring_buffer_subbuf_order_get(struct trace_buffer *buffer) +{ + if (!buffer) + return -EINVAL; + + return buffer->subbuf_order; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); + +/** + * ring_buffer_subbuf_order_set - set the size of ring buffer sub page. + * @buffer: The ring_buffer to set the new page size. + * @order: Order of the system pages in one sub buffer page + * + * By default, one ring buffer pages equals to one system page. This API can be + * used to set new size of the ring buffer page. The size must be order of + * system page size, that's why the input parameter @order is the order of + * system pages that are allocated for one ring buffer page: + * 0 - 1 system page + * 1 - 2 system pages + * 3 - 4 system pages + * ... + * + * Returns 0 on success or < 0 in case of an error. + */ +int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) +{ + int psize; + + if (!buffer || order < 0) + return -EINVAL; + + if (buffer->subbuf_order == order) + return 0; + + psize = (1 << order) * PAGE_SIZE; + if (psize <= BUF_PAGE_HDR_SIZE) + return -EINVAL; + + buffer->subbuf_order = order; + buffer->subbuf_size = psize - BUF_PAGE_HDR_SIZE; + + /* Todo: reset the buffer with the new page size */ + + return 0; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_set); + /* * We only allocate new buffers, never free them if the CPU goes down. * If we were to free the buffer, then the user would lose any trace that was in diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 76dd0a4c8cb5..a010aba4c4a4 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9358,6 +9358,51 @@ static const struct file_operations buffer_percent_fops = { .llseek = default_llseek, }; +static ssize_t +buffer_order_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +{ + struct trace_array *tr = filp->private_data; + char buf[64]; + int r; + + r = sprintf(buf, "%d\n", ring_buffer_subbuf_order_get(tr->array_buffer.buffer)); + + return simple_read_from_buffer(ubuf, cnt, ppos, buf, r); +} + +static ssize_t +buffer_order_write(struct file *filp, const char __user *ubuf, + size_t cnt, loff_t *ppos) +{ + struct trace_array *tr = filp->private_data; + unsigned long val; + int ret; + + ret = kstrtoul_from_user(ubuf, cnt, 10, &val); + if (ret) + return ret; + + /* limit between 1 and 128 system pages */ + if (val < 0 || val > 7) + return -EINVAL; + + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); + if (ret) + return ret; + + (*ppos)++; + + return cnt; +} + +static const struct file_operations buffer_order_fops = { + .open = tracing_open_generic_tr, + .read = buffer_order_read, + .write = buffer_order_write, + .release = tracing_release_generic_tr, + .llseek = default_llseek, +}; + static struct dentry *trace_instance_dir; static void @@ -9824,6 +9869,9 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) trace_create_file("buffer_percent", TRACE_MODE_WRITE, d_tracer, tr, &buffer_percent_fops); + trace_create_file("buffer_subbuf_order", TRACE_MODE_WRITE, d_tracer, + tr, &buffer_order_fops); + create_trace_options_dir(tr); #ifdef CONFIG_TRACER_MAX_TRACE From patchwork Thu Dec 21 16:10:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182335 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp517314dyi; Thu, 21 Dec 2023 08:12:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IHbEVzaPQZMQ0nUFUiH+LFQdNDplxD3eK8D2p6HXVKxPuN6RDUxd2FK3YcmQts0FQEAv+6j X-Received: by 2002:a17:902:c406:b0:1d0:cf22:c6ca with SMTP id k6-20020a170902c40600b001d0cf22c6camr14394944plk.52.1703175153501; Thu, 21 Dec 2023 08:12:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175153; cv=none; d=google.com; s=arc-20160816; b=ljuZH3G8pxI63+1bcCRQPCJ/2FAFyNCAYdTXu0f1/ZWeL76udHA5neH9JjZbuBN/is QUIBVA8hwgDdthpyAtZ7sDTqqF9q0NwzhmRjAJ9EeKkfxk+qz/6NSOt8ZqasRcMbEj6b 9lN4cHJED3POtBlpej3JQ8E3hX/c0nP85eb3guOFTDzzPwpNFLHuQ1l+xWRDkXGnUgzY hB0OoMwRRTt58oIBYU4EE8EPG+TmOVSSww8WY6RXFdYk8+V0CM5bLjo8Z9GLgOq7dOsh iktE9/dUe79lvzQU0eBbUfkwvstB1y4PvdeUHnZPUb3K1+7VgH1SnWN3ZIjn+pjWw2wY mNVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=JoL77rIAC2yBv/VEHSplcXATLuV4YHqU8CsZyWLeVzQ=; fh=vS4GW7DeA1AMEc8SlhVYPDpeY9+vqwFsekYGU4Lsths=; b=reRL2M5slwuGnUg1nQuEQ/fhq0VXR5rTJ1lcRGLy0sZa0EdcpRVCKI97SDfnsHSasJ TS54djMc1reYep+FFYIjA7JlWRrCqD3GFRofG9NVXSRnvy2vrSrAH32+E/1lrBWne/UK 1K/wykH2o75E0rDQAOYnKaCP8nbNe1wmhkwb9W8Zcg1KrkE88JDrIYZ4uuzUN1H0PRIV +RcesVpWzijLQBXAblhgM/wpXnM7YAfiinXUDh8cYo0FmJucr/0io1huO+SmjTANnbLk EnsvLKcpyT7Xg+ee3GdI/j7stGUKBjqQ3xhF91gPR8CUUHa0t+DE3BsoDg0tusBqkF6W cHKA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8728-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8728-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id jc41-20020a056a006ca900b006d96de65dc4si1564699pfb.353.2023.12.21.08.12.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:12:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8728-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8728-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8728-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id D4A89B24313 for ; Thu, 21 Dec 2023 16:11:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9D95462815; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E4B958211 for ; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9BF13C433B8; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdX-000000041I3-2Hsq; Thu, 21 Dec 2023 11:11:03 -0500 Message-ID: <20231221161103.323451761@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:28 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort , Kent Overstreet , "Tzvetomir Stoyanov (VMware)" Subject: [for-next][PATCH 04/16] ring-buffer: Set new size of the ring buffer sub page References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908589737553527 X-GMAIL-MSGID: 1785908589737553527 From: "Tzvetomir Stoyanov (VMware)" There are two approaches when changing the size of the ring buffer sub page: 1. Destroying all pages and allocating new pages with the new size. 2. Allocating new pages, copying the content of the old pages before destroying them. The first approach is easier, it is selected in the proposed implementation. Changing the ring buffer sub page size is supposed to not happen frequently. Usually, that size should be set only once, when the buffer is not in use yet and is supposed to be empty. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-5-tz.stoyanov@gmail.com Link: https://lore.kernel.org/linux-trace-kernel/20231219185628.588995543@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 80 ++++++++++++++++++++++++++++++++++---- 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 20fc0121735d..b928bd03dd0e 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -332,6 +332,7 @@ struct buffer_page { unsigned read; /* index for next read */ local_t entries; /* entries on this page */ unsigned long real_end; /* real end of data */ + unsigned order; /* order of the page */ struct buffer_data_page *page; /* Actual data page */ }; @@ -362,7 +363,7 @@ static __always_inline unsigned int rb_page_commit(struct buffer_page *bpage) static void free_buffer_page(struct buffer_page *bpage) { - free_page((unsigned long)bpage->page); + free_pages((unsigned long)bpage->page, bpage->order); kfree(bpage); } @@ -1460,10 +1461,12 @@ static int __rb_allocate_pages(struct ring_buffer_per_cpu *cpu_buffer, list_add(&bpage->list, pages); - page = alloc_pages_node(cpu_to_node(cpu_buffer->cpu), mflags, 0); + page = alloc_pages_node(cpu_to_node(cpu_buffer->cpu), mflags, + cpu_buffer->buffer->subbuf_order); if (!page) goto free_pages; bpage->page = page_address(page); + bpage->order = cpu_buffer->buffer->subbuf_order; rb_init_page(bpage->page); if (user_thread && fatal_signal_pending(current)) @@ -1542,7 +1545,8 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, long nr_pages, int cpu) rb_check_bpage(cpu_buffer, bpage); cpu_buffer->reader_page = bpage; - page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL, 0); + + page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL, cpu_buffer->buffer->subbuf_order); if (!page) goto fail_free_reader; bpage->page = page_address(page); @@ -1626,6 +1630,7 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags, goto fail_free_buffer; /* Default buffer page size - one system page */ + buffer->subbuf_order = 0; buffer->subbuf_size = PAGE_SIZE - BUF_PAGE_HDR_SIZE; /* Max payload is buffer page size - header (8bytes) */ @@ -5503,8 +5508,8 @@ void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) if (bpage) goto out; - page = alloc_pages_node(cpu_to_node(cpu), - GFP_KERNEL | __GFP_NORETRY, 0); + page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL | __GFP_NORETRY, + cpu_buffer->buffer->subbuf_order); if (!page) return ERR_PTR(-ENOMEM); @@ -5553,7 +5558,7 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data local_irq_restore(flags); out: - free_page((unsigned long)bpage); + free_pages((unsigned long)bpage, buffer->subbuf_order); } EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); @@ -5813,7 +5818,13 @@ EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); */ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) { + struct ring_buffer_per_cpu **cpu_buffers; + int old_order, old_size; + int nr_pages; int psize; + int bsize; + int err; + int cpu; if (!buffer || order < 0) return -EINVAL; @@ -5825,12 +5836,67 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (psize <= BUF_PAGE_HDR_SIZE) return -EINVAL; + bsize = sizeof(void *) * buffer->cpus; + cpu_buffers = kzalloc(bsize, GFP_KERNEL); + if (!cpu_buffers) + return -ENOMEM; + + old_order = buffer->subbuf_order; + old_size = buffer->subbuf_size; + + /* prevent another thread from changing buffer sizes */ + mutex_lock(&buffer->mutex); + atomic_inc(&buffer->record_disabled); + + /* Make sure all commits have finished */ + synchronize_rcu(); + buffer->subbuf_order = order; buffer->subbuf_size = psize - BUF_PAGE_HDR_SIZE; - /* Todo: reset the buffer with the new page size */ + /* Make sure all new buffers are allocated, before deleting the old ones */ + for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) + continue; + + nr_pages = buffer->buffers[cpu]->nr_pages; + cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); + if (!cpu_buffers[cpu]) { + err = -ENOMEM; + goto error; + } + } + + for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) + continue; + + rb_free_cpu_buffer(buffer->buffers[cpu]); + buffer->buffers[cpu] = cpu_buffers[cpu]; + } + + atomic_dec(&buffer->record_disabled); + mutex_unlock(&buffer->mutex); + + kfree(cpu_buffers); return 0; + +error: + buffer->subbuf_order = old_order; + buffer->subbuf_size = old_size; + + atomic_dec(&buffer->record_disabled); + mutex_unlock(&buffer->mutex); + + for_each_buffer_cpu(buffer, cpu) { + if (!cpu_buffers[cpu]) + continue; + kfree(cpu_buffers[cpu]); + } + kfree(cpu_buffers); + + return err; } EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_set); From patchwork Thu Dec 21 16:10:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182338 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518065dyi; Thu, 21 Dec 2023 08:13:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFgHNwdEMTUjQSSZ9TxgTBjWHsuWDPVdgDeuzu8/lvdJdr5zcXmBngJJKOdb266dDdDWXy7 X-Received: by 2002:a05:6a20:7d9f:b0:18f:ca74:fc0 with SMTP id v31-20020a056a207d9f00b0018fca740fc0mr1804524pzj.50.1703175214166; Thu, 21 Dec 2023 08:13:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175214; cv=none; d=google.com; s=arc-20160816; b=UdmTso9NbxvlingiRwaV7BlZieJRDDTtskhRXTngbXa8Vx0ADysH3qQL3PYa3XKQjS dcdoXAPxc2DPCfsrXQtifGG6+nG0/EOiUnbqeVXWuVrBbjEVUeNuP3xM6jeoMEhBzHq9 ZD/A4SZ4eNmKu0n6D9KcKFR09KO20RAYYgH/TOHVC9eBe09x/u8jqA/1SsBloAaUzsSM z6B2IKvJDzKrH5PCVsYpAjDgi2QmQBUB8nfmOyt8+VPsjVfGsBYcUK9zqnYsx9XHK4o8 R6ZirpSI3cyDv63D70g4NXjuuNqDO4CmErvNAYT3FzqdaEr2YBXYhG26K/miMZCYX/LV kJ9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=fAo5sgk8JT5/ypl2JF9Bxjdo93Pked67NkdrMxGwN0I=; fh=vS4GW7DeA1AMEc8SlhVYPDpeY9+vqwFsekYGU4Lsths=; b=iLHcSt8Mp2Fiur5Ugvvh2Xwv7su70ZDzkgsH7RN0rjtLPbeM5TEBmxRYlhr4dWIlJ6 xyfdKOm/flls5MmLyeoR3LGiUBDyXNNTAINEa6+nr8lSLV/nHBQLAWa9xSxi2o9DBGhk ehxnA9rMnx6HYfm2vyHOHhIwfHB9r+IgQcbfSzix1YvMh5e1jexCwXtys0NAWY0a21XX F3HaxWeGdoj/W8cZQ6liAelxjoczSTTfpkj+f6CqWNDcSgntvcaCBA1HPZCX/1HWCtpG fg87rWCt9TgZvk6Dw13D+j6JfeYPDVdOzYjpgAltkSCjtBnVwnwETg03L2jtgajokJWS wPtA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8729-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8729-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id l124-20020a632582000000b005cdc4415f56si1789764pgl.350.2023.12.21.08.13.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:13:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8729-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8729-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8729-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id C03BCB22FC9 for ; Thu, 21 Dec 2023 16:12:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8CDB964A9F; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1885258214 for ; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CEA8AC433BC; Thu, 21 Dec 2023 16:09:59 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdX-000000041IY-3Vt9; Thu, 21 Dec 2023 11:11:03 -0500 Message-ID: <20231221161103.620318994@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:29 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Vincent Donnefort , Kent Overstreet , "Tzvetomir Stoyanov (VMware)" Subject: [for-next][PATCH 05/16] ring-buffer: Read and write to ring buffers with custom sub buffer size References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908653716926095 X-GMAIL-MSGID: 1785908653716926095 From: "Tzvetomir Stoyanov (VMware)" As the size of the ring sub buffer page can be changed dynamically, the logic that reads and writes to the buffer should be fixed to take that into account. Some internal ring buffer APIs are changed: ring_buffer_alloc_read_page() ring_buffer_free_read_page() ring_buffer_read_page() A new API is introduced: ring_buffer_read_page_data() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-6-tz.stoyanov@gmail.com Link: https://lore.kernel.org/linux-trace-kernel/20231219185628.875145995@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Tzvetomir Stoyanov (VMware) [ Fixed kerneldoc on data_page parameter in ring_buffer_free_read_page() ] Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 11 ++-- kernel/trace/ring_buffer.c | 75 ++++++++++++++++++++-------- kernel/trace/ring_buffer_benchmark.c | 10 ++-- kernel/trace/trace.c | 34 +++++++------ 4 files changed, 89 insertions(+), 41 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index 12573306b889..fa802db216f9 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -192,10 +192,15 @@ bool ring_buffer_time_stamp_abs(struct trace_buffer *buffer); size_t ring_buffer_nr_pages(struct trace_buffer *buffer, int cpu); size_t ring_buffer_nr_dirty_pages(struct trace_buffer *buffer, int cpu); -void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu); -void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data); -int ring_buffer_read_page(struct trace_buffer *buffer, void **data_page, +struct buffer_data_read_page; +struct buffer_data_read_page * +ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu); +void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, + struct buffer_data_read_page *page); +int ring_buffer_read_page(struct trace_buffer *buffer, + struct buffer_data_read_page *data_page, size_t len, int cpu, int full); +void *ring_buffer_read_page_data(struct buffer_data_read_page *page); struct trace_seq; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index b928bd03dd0e..c2afcf98ea91 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -318,6 +318,11 @@ struct buffer_data_page { unsigned char data[] RB_ALIGN_DATA; /* data of buffer page */ }; +struct buffer_data_read_page { + unsigned order; /* order of the page */ + struct buffer_data_page *data; /* actual data, stored in this page */ +}; + /* * Note, the buffer_page list must be first. The buffer pages * are allocated in cache lines, which means that each buffer @@ -5483,40 +5488,48 @@ EXPORT_SYMBOL_GPL(ring_buffer_swap_cpu); * Returns: * The page allocated, or ERR_PTR */ -void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) +struct buffer_data_read_page * +ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) { struct ring_buffer_per_cpu *cpu_buffer; - struct buffer_data_page *bpage = NULL; + struct buffer_data_read_page *bpage = NULL; unsigned long flags; struct page *page; if (!cpumask_test_cpu(cpu, buffer->cpumask)) return ERR_PTR(-ENODEV); + bpage = kzalloc(sizeof(*bpage), GFP_KERNEL); + if (!bpage) + return ERR_PTR(-ENOMEM); + + bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu]; local_irq_save(flags); arch_spin_lock(&cpu_buffer->lock); if (cpu_buffer->free_page) { - bpage = cpu_buffer->free_page; + bpage->data = cpu_buffer->free_page; cpu_buffer->free_page = NULL; } arch_spin_unlock(&cpu_buffer->lock); local_irq_restore(flags); - if (bpage) + if (bpage->data) goto out; page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL | __GFP_NORETRY, cpu_buffer->buffer->subbuf_order); - if (!page) + if (!page) { + kfree(bpage); return ERR_PTR(-ENOMEM); + } - bpage = page_address(page); + bpage->data = page_address(page); out: - rb_init_page(bpage); + rb_init_page(bpage->data); return bpage; } @@ -5526,14 +5539,15 @@ EXPORT_SYMBOL_GPL(ring_buffer_alloc_read_page); * ring_buffer_free_read_page - free an allocated read page * @buffer: the buffer the page was allocate for * @cpu: the cpu buffer the page came from - * @data: the page to free + * @data_page: the page to free * * Free a page allocated from ring_buffer_alloc_read_page. */ -void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data) +void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, + struct buffer_data_read_page *data_page) { struct ring_buffer_per_cpu *cpu_buffer; - struct buffer_data_page *bpage = data; + struct buffer_data_page *bpage = data_page->data; struct page *page = virt_to_page(bpage); unsigned long flags; @@ -5542,8 +5556,12 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data cpu_buffer = buffer->buffers[cpu]; - /* If the page is still in use someplace else, we can't reuse it */ - if (page_ref_count(page) > 1) + /* + * If the page is still in use someplace else, or order of the page + * is different from the subbuffer order of the buffer - + * we can't reuse it + */ + if (page_ref_count(page) > 1 || data_page->order != buffer->subbuf_order) goto out; local_irq_save(flags); @@ -5558,7 +5576,8 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data local_irq_restore(flags); out: - free_pages((unsigned long)bpage, buffer->subbuf_order); + free_pages((unsigned long)bpage, data_page->order); + kfree(data_page); } EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); @@ -5579,9 +5598,10 @@ EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); * rpage = ring_buffer_alloc_read_page(buffer, cpu); * if (IS_ERR(rpage)) * return PTR_ERR(rpage); - * ret = ring_buffer_read_page(buffer, &rpage, len, cpu, 0); + * ret = ring_buffer_read_page(buffer, rpage, len, cpu, 0); * if (ret >= 0) - * process_page(rpage, ret); + * process_page(ring_buffer_read_page_data(rpage), ret); + * ring_buffer_free_read_page(buffer, cpu, rpage); * * When @full is set, the function will not return true unless * the writer is off the reader page. @@ -5596,7 +5616,8 @@ EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); * <0 if no data has been transferred. */ int ring_buffer_read_page(struct trace_buffer *buffer, - void **data_page, size_t len, int cpu, int full) + struct buffer_data_read_page *data_page, + size_t len, int cpu, int full) { struct ring_buffer_per_cpu *cpu_buffer = buffer->buffers[cpu]; struct ring_buffer_event *event; @@ -5621,10 +5642,12 @@ int ring_buffer_read_page(struct trace_buffer *buffer, len -= BUF_PAGE_HDR_SIZE; - if (!data_page) + if (!data_page || !data_page->data) + goto out; + if (data_page->order != buffer->subbuf_order) goto out; - bpage = *data_page; + bpage = data_page->data; if (!bpage) goto out; @@ -5718,11 +5741,11 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* swap the pages */ rb_init_page(bpage); bpage = reader->page; - reader->page = *data_page; + reader->page = data_page->data; local_set(&reader->write, 0); local_set(&reader->entries, 0); reader->read = 0; - *data_page = bpage; + data_page->data = bpage; /* * Use the real_end for the data size, @@ -5767,6 +5790,18 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } EXPORT_SYMBOL_GPL(ring_buffer_read_page); +/** + * ring_buffer_read_page_data - get pointer to the data in the page. + * @page: the page to get the data from + * + * Returns pointer to the actual data in this page. + */ +void *ring_buffer_read_page_data(struct buffer_data_read_page *page) +{ + return page->data; +} +EXPORT_SYMBOL_GPL(ring_buffer_read_page_data); + /** * ring_buffer_subbuf_size_get - get size of the sub buffer. * @buffer: the buffer to get the sub buffer size from diff --git a/kernel/trace/ring_buffer_benchmark.c b/kernel/trace/ring_buffer_benchmark.c index aef34673d79d..008187ebd7fe 100644 --- a/kernel/trace/ring_buffer_benchmark.c +++ b/kernel/trace/ring_buffer_benchmark.c @@ -104,10 +104,11 @@ static enum event_status read_event(int cpu) static enum event_status read_page(int cpu) { + struct buffer_data_read_page *bpage; struct ring_buffer_event *event; struct rb_page *rpage; unsigned long commit; - void *bpage; + int page_size; int *entry; int ret; int inc; @@ -117,14 +118,15 @@ static enum event_status read_page(int cpu) if (IS_ERR(bpage)) return EVENT_DROPPED; - ret = ring_buffer_read_page(buffer, &bpage, PAGE_SIZE, cpu, 1); + page_size = ring_buffer_subbuf_size_get(buffer); + ret = ring_buffer_read_page(buffer, bpage, page_size, cpu, 1); if (ret >= 0) { - rpage = bpage; + rpage = ring_buffer_read_page_data(bpage); /* The commit may have missed event flags set, clear them */ commit = local_read(&rpage->commit) & 0xfffff; for (i = 0; i < commit && !test_error ; i += inc) { - if (i >= (PAGE_SIZE - offsetof(struct rb_page, data))) { + if (i >= (page_size - offsetof(struct rb_page, data))) { TEST_ERROR(); break; } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index a010aba4c4a4..711095aa731d 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -8286,6 +8286,8 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, { struct ftrace_buffer_info *info = filp->private_data; struct trace_iterator *iter = &info->iter; + void *trace_data; + int page_size; ssize_t ret = 0; ssize_t size; @@ -8297,6 +8299,8 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, return -EBUSY; #endif + page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + if (!info->spare) { info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file); @@ -8311,13 +8315,13 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, return ret; /* Do we have previous read data to read? */ - if (info->read < PAGE_SIZE) + if (info->read < page_size) goto read; again: trace_access_lock(iter->cpu_file); ret = ring_buffer_read_page(iter->array_buffer->buffer, - &info->spare, + info->spare, count, iter->cpu_file, 0); trace_access_unlock(iter->cpu_file); @@ -8338,11 +8342,11 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, info->read = 0; read: - size = PAGE_SIZE - info->read; + size = page_size - info->read; if (size > count) size = count; - - ret = copy_to_user(ubuf, info->spare + info->read, size); + trace_data = ring_buffer_read_page_data(info->spare); + ret = copy_to_user(ubuf, trace_data + info->read, size); if (ret == size) return -EFAULT; @@ -8453,6 +8457,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, .spd_release = buffer_spd_release, }; struct buffer_ref *ref; + int page_size; int entries, i; ssize_t ret = 0; @@ -8461,13 +8466,14 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, return -EBUSY; #endif - if (*ppos & (PAGE_SIZE - 1)) + page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + if (*ppos & (page_size - 1)) return -EINVAL; - if (len & (PAGE_SIZE - 1)) { - if (len < PAGE_SIZE) + if (len & (page_size - 1)) { + if (len < page_size) return -EINVAL; - len &= PAGE_MASK; + len &= (~(page_size - 1)); } if (splice_grow_spd(pipe, &spd)) @@ -8477,7 +8483,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, trace_access_lock(iter->cpu_file); entries = ring_buffer_entries_cpu(iter->array_buffer->buffer, iter->cpu_file); - for (i = 0; i < spd.nr_pages_max && len && entries; i++, len -= PAGE_SIZE) { + for (i = 0; i < spd.nr_pages_max && len && entries; i++, len -= page_size) { struct page *page; int r; @@ -8498,7 +8504,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, } ref->cpu = iter->cpu_file; - r = ring_buffer_read_page(ref->buffer, &ref->page, + r = ring_buffer_read_page(ref->buffer, ref->page, len, iter->cpu_file, 1); if (r < 0) { ring_buffer_free_read_page(ref->buffer, ref->cpu, @@ -8507,14 +8513,14 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, break; } - page = virt_to_page(ref->page); + page = virt_to_page(ring_buffer_read_page_data(ref->page)); spd.pages[i] = page; - spd.partial[i].len = PAGE_SIZE; + spd.partial[i].len = page_size; spd.partial[i].offset = 0; spd.partial[i].private = (unsigned long)ref; spd.nr_pages++; - *ppos += PAGE_SIZE; + *ppos += page_size; entries = ring_buffer_entries_cpu(iter->array_buffer->buffer, iter->cpu_file); } From patchwork Thu Dec 21 16:10:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182340 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518105dyi; Thu, 21 Dec 2023 08:13:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IG8p5y8cDOfuyMARLS7fODjRsfTBO+Q2WwcCz/4HhSA0jzdrR/1MtrFG55c49kRAMdNyDY0 X-Received: by 2002:a05:6e02:12ce:b0:35d:59a2:1281 with SMTP id i14-20020a056e0212ce00b0035d59a21281mr31214797ilm.45.1703175216779; Thu, 21 Dec 2023 08:13:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175216; cv=none; d=google.com; s=arc-20160816; b=mHxjbIn+fFhNWrMMemvsmkuiy5Zx5FBja1a928JYSxO0mcl1wF0yQQlbOHveUyaAwz IIGlHIzIetD1CHdTTMhbNejDyqn6j+xJoF5MTLp/6J6Rc3JlwnRJYLVRGFWc0pBOmRRJ A+b2/1Akc4/ig7js82Eq6r3nP4n9VHMBDdUHdNuL2rdR20S+ixWW2zE3JXpqX2hhGK8v POdZ00OPgrdpLouBBGds2DaFl0HqGsPxPyGmwxBuFyAe1Lk0GC2ZdIUrMNYfdb447xVw 5f9iMh1R2SuNB0ZnfSpHhzAWIXmu3iHYxIOAvt4QBPVnyDuasXeLx/tlhkcqExKx73SH Hajg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=5qSTLMqxg4rMWTsxNN+DS9D/jbhLJSuGGJhe4XBOvKc=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=s6auhbaH6OF7HOf9SFXS2oUArLk4MCOTerlpNKGrpnEmLDFizMHLPIKfzrhYkbrtNh Sqyb3zQP9gBH1RNTncLPECRY2Ee1T/SOvQhDp5ltRbO5Hvf523UrHeEVePc4K1sVKcHD x9NIpsF1PZDEum8euIJu3HZaEFVF/j0QbhVmWAi+gJimwA2UJ39FIQg8hzxhmm+pPtwW VgazXlOs3VaHC+83FKj7MeOILbHeNOEYp8M5z2O855ABh94iliFKS2JJmIYA4ctJa1Da EBYyRS1cioZwoCCG/lxs9XvUD7epDAIvcgZOCHb+SMmvwxzrjx1+uWJqb7I/g0kL8P/Q p/5A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8730-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8730-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id h33-20020a63f921000000b005cd8b9dc455si1758352pgi.567.2023.12.21.08.13.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:13:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8730-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8730-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8730-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id E7BCAB22C95 for ; Thu, 21 Dec 2023 16:12:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9591F64AA2; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD8A759910 for ; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F87BC433C8; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdY-000000041J3-0WLL; Thu, 21 Dec 2023 11:11:04 -0500 Message-ID: <20231221161103.912510492@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:30 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 06/16] ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908656029630783 X-GMAIL-MSGID: 1785908656029630783 From: "Steven Rostedt (Google)" On failure to allocate ring buffer pages, the pointer to the CPU buffer pages is freed, but the pages that were allocated previously were not. Make sure they are freed too. Link: https://lore.kernel.org/linux-trace-kernel/20231219185629.179352802@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Fixes: f9b94daa542a ("tracing: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index c2afcf98ea91..3c11e8e811ed 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5927,6 +5927,7 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) for_each_buffer_cpu(buffer, cpu) { if (!cpu_buffers[cpu]) continue; + rb_free_cpu_buffer(cpu_buffers[cpu]); kfree(cpu_buffers[cpu]); } kfree(cpu_buffers); From patchwork Thu Dec 21 16:10:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182336 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp517394dyi; Thu, 21 Dec 2023 08:12:39 -0800 (PST) X-Google-Smtp-Source: AGHT+IFLk6wuX2ohVRQcsgW/c6vRV1VOOeh5LZtmRKOSiq1meRSG0upEiJBxsi1VOvBW6wsW3898 X-Received: by 2002:a05:6512:75:b0:50e:242a:775e with SMTP id i21-20020a056512007500b0050e242a775emr4458340lfo.106.1703175159542; Thu, 21 Dec 2023 08:12:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175159; cv=none; d=google.com; s=arc-20160816; b=gTbEJrtd0ygBpSlTOhPNefQIG4hBGEyJRHTKYckmi13/O0UeDxxuVhjFcPPGt4dBbO X93PNlHVQQH0SJij8i2FWLLmIUhDlDpI9g4MH+QgVDYBaId9OCEIn3oR2VKUfwA45iNB X2brG6OA/5H3pYWQophPNMYoD9rI7qlBf7YQ1bMJnSsRc/CztaGqQJVSwAATEACYmC1j 1GeAdRYkIetHhKmuuqeUVwthFqrW24cXt0JpjDpTMnsXk2Kz3Yznpyqk3dlzR51Ol/tf yv+tIjGe0xnGdcDGOBjd7BK56NP9Bi6OpjK5v+SorOqqEX+X9tx/+4H6NgtGHOYaR+u+ mCXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=4WihiHXInPnGl8lay5hJsHJZq9W0Uab5DK+n0yUp+aI=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=P5K3o24KsToqHtzXK0w8DYRKhe3PVOaASHgA5gPjz2st8ZMb1xzx25gcMyfcjNNpZq tr8dgBXSTRnC4dJznki/mSqYlmzEW2+TgY74WJnEAGt6qr6VRYGsvV564Rpw3kCSoylY HOpyBQfJOANSFI6DTE48uCc8lDsdbLye+PTXnBPCv2K0XC2Fep8RWo1fxghS5SY/KP0r tMwl55lvLqHpxM9Vpj2MJQpsqoCfKE376LUYeIHw+Xh058VhWp+tj3U5FVXwGIb4f/R/ i6RPylbFxBk9mhnhh4O304gBwKw3x6K962xFL2czsFULAmWYyWDjo/p9Fp7oF6TVgxHT hClQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8732-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8732-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id a24-20020a1709065f9800b00a22e6dac442si997175eju.632.2023.12.21.08.12.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:12:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8732-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8732-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8732-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 12C8E1F2495E for ; Thu, 21 Dec 2023 16:12:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BC65664AB8; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02E335992C for ; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76B3CC116B3; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdY-000000041JZ-1nnd; Thu, 21 Dec 2023 11:11:04 -0500 Message-ID: <20231221161104.201002261@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:31 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 07/16] ring-buffer: Do no swap cpu buffers if order is different References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908596196962186 X-GMAIL-MSGID: 1785908596196962186 From: "Steven Rostedt (Google)" As all the subbuffer order (subbuffer sizes) must be the same throughout the ring buffer, check the order of the buffers that are doing a CPU buffer swap in ring_buffer_swap_cpu() to make sure they are the same. If the are not the same, then fail to do the swap, otherwise the ring buffer will think the CPU buffer has a specific subbuffer size when it does not. Link: https://lore.kernel.org/linux-trace-kernel/20231219185629.467894710@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 3c11e8e811ed..fdcd171b09b5 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5417,6 +5417,9 @@ int ring_buffer_swap_cpu(struct trace_buffer *buffer_a, if (cpu_buffer_a->nr_pages != cpu_buffer_b->nr_pages) goto out; + if (buffer_a->subbuf_order != buffer_b->subbuf_order) + goto out; + ret = -EAGAIN; if (atomic_read(&buffer_a->record_disabled)) From patchwork Thu Dec 21 16:10:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182347 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp522267dyi; Thu, 21 Dec 2023 08:19:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IHVjz9TpFTiX08b6wXKS6wJUXOkQjI0yKeqYIXS7STUSirOp45nUczRJM7DjUrZUbRtBpSF X-Received: by 2002:a05:6a00:2311:b0:6ce:f74e:ff4f with SMTP id h17-20020a056a00231100b006cef74eff4fmr12879746pfh.50.1703175573664; Thu, 21 Dec 2023 08:19:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175573; cv=none; d=google.com; s=arc-20160816; b=TdDs4XjdnfaNCyiZatR7IUk6hdU3syAlNX7Yl9o4lTPGGatS9Hf6Wvu1MiOpHTf9II MsnqiFsW8zwgH/nTicHnzjDwU0hN48KS/70OXKZ4OmNzNzPbNYkNpvhhZIDFNFeNWNDl Vc9EE8bnKNx96EhzfQL/9eTFgEktFLgDa2aVV8QPGkcr6LVPVn1NF8TT1UbWCayCevyv K8PM0jKT1s3uq7mjLst3E50JSw8e+yx87EjHir6R/ybghm96aG87hsbr1G5PhuBjumPo j+qDRblggGB8MvzHD7GQ49ISWsRWWme+pf1cSdVwxyBOsXhkGb97ET9IKCNXOz65HEIk XHEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=iu0Q76K9SyTr6h0Kq6xnCxEbvy8hMpXowVN697MoZ0Y=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=Nsi6rDfFBTilURhWjSDH5O+gb7mJsSjuyhH+PbuGjHBIr5Btz5ADzFIhP07UmGvP+9 1LUW1/u1ZpXmorIHSP8DggBlmUZaEcIk8NbQmabEhnrQt6NTmvDJlchfN0vz71ilClj9 s8sP094yfoFwpnbvxUcV48gwqPoujVmyv9gjxy/T/W6R8fDzw6++iMUiQansNx4wys5w Lfp98oiEj6hLLQv5nmf5FYPC1bVKIdRzy9HkYz6eiTFMDZMOQliHtc6uO1FzTYW2Zw/v C9ouUu7T04mc06Elu8x6xPaHvzDKUW9M+F9PWwR3L3chvcJq1jjIxL9UxCmV8APja6Ph dQSw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8731-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8731-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id m16-20020aa78a10000000b006ce7b6f8c7esi1752327pfa.210.2023.12.21.08.19.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:19:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8731-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8731-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8731-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id D010428B40D for ; Thu, 21 Dec 2023 16:12:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A844F64AAE; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA9EC59927 for ; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B51F4C433BF; Thu, 21 Dec 2023 16:10:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdY-000000041K3-31gc; Thu, 21 Dec 2023 11:11:04 -0500 Message-ID: <20231221161104.501992954@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:32 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 08/16] ring-buffer: Make sure the spare sub buffer used for reads has same size References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785909030410635405 X-GMAIL-MSGID: 1785909030410635405 From: "Steven Rostedt (Google)" Now that the ring buffer specifies the size of its sub buffers, they all need to be the same size. When doing a read, a swap is done with a spare page. Make sure they are the same size before doing the swap, otherwise the read will fail. Link: https://lore.kernel.org/linux-trace-kernel/20231219185629.763664788@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 711095aa731d..4dcdc30aa110 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -7582,6 +7582,7 @@ struct ftrace_buffer_info { struct trace_iterator iter; void *spare; unsigned int spare_cpu; + unsigned int spare_size; unsigned int read; }; @@ -8301,6 +8302,15 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + /* Make sure the spare matches the current sub buffer size */ + if (info->spare) { + if (page_size != info->spare_size) { + ring_buffer_free_read_page(iter->array_buffer->buffer, + info->spare_cpu, info->spare); + info->spare = NULL; + } + } + if (!info->spare) { info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file); @@ -8309,6 +8319,7 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, info->spare = NULL; } else { info->spare_cpu = iter->cpu_file; + info->spare_size = page_size; } } if (!info->spare) From patchwork Thu Dec 21 16:10:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182337 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp517563dyi; Thu, 21 Dec 2023 08:12:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IGGI1VhdnlGvbAnjX580P0HASQuDUpcOP8CloEwLq5W35Pgtb/skoOm8mzjxJHst+Cst5bs X-Received: by 2002:a17:906:7490:b0:a23:6e16:310c with SMTP id e16-20020a170906749000b00a236e16310cmr15443ejl.2.1703175170622; Thu, 21 Dec 2023 08:12:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175170; cv=none; d=google.com; s=arc-20160816; b=NEpSP3r9TrIYDhTi1thkwamAdkMo6M+waO0g4PIEuJqkLb+MT9BRoYuYXRWZj0edng rHiSQBZrt9CKlI+uBV4RHhOtBMixGlGAd6/Hwdj/ismBk8jxX6MPFAF8oJZeTuHUy9e4 RGKsGsun5tzgAKgIQilh5Uyv79CP2mjOqEvTXsxSZhI97o72S52wRkegezd8YqUSF2En 36lYl2K9BKfNOq10QuP0HsT12/AN8pB6raRR4fmpnP/ny05Mmiw5WyuDoA5y5IhJdaTU 9trblop0+BRuGMsLallg064YyOSDXII9hD/C2a7rOPjdMMr8XF4h0jjh5NdcxOPP9fuR zIow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=sOIHL+NE1DitOxba89DChClxzv7XQYnkgbMTnwuLRDg=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=OXe760Nat+Yb66frV7sY20cuTmHBndD53iTUVYidvGTeWWcjWAGOuR7iObfbHYuuDc Dhy2B2eAiWAL1GoWTFiSMhmtnLp//UHHPAVAKGU64j3C8xzSBx32aARFMR6Ri4sE7j7b I/357tKHXS2IrwEF9bamDl5N8UPIaspCcL7eSKrTCOrnMuMyqCX5KrtLBE1g9L98oyjz DmoA+HDFTDY9/2DSH29VyEQVgbDu7dbfcuZ8bFcMT1D7UNvEBfa/jnB+aa0ARK6W+bEn /kIGsOv6DN8CQHDEh0gWUlEHJINATRo+GD7UtWJBhmDvlUHZKHyGVRnKEAnCTA31kTRy F1QQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8733-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8733-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id ga25-20020a170906b85900b00a235a606ba2si954081ejb.714.2023.12.21.08.12.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:12:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8733-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8733-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8733-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E7CA51F25DBB for ; Thu, 21 Dec 2023 16:12:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E839065191; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33F7B627E0 for ; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 124A7C433CA; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdZ-000000041KY-03JX; Thu, 21 Dec 2023 11:11:05 -0500 Message-ID: <20231221161104.796933244@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:33 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 09/16] tracing: Update snapshot order along with main buffer order References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908607739923851 X-GMAIL-MSGID: 1785908607739923851 From: "Steven Rostedt (Google)" When updating the order of the sub buffers for the main buffer, make sure that if the snapshot buffer exists, that it gets its order updated as well. Link: https://lore.kernel.org/linux-trace-kernel/20231219185630.054668186@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 45 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 4dcdc30aa110..2439e00aa4ce 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -1263,10 +1263,17 @@ static void set_buffer_entries(struct array_buffer *buf, unsigned long val); int tracing_alloc_snapshot_instance(struct trace_array *tr) { + int order; int ret; if (!tr->allocated_snapshot) { + /* Make the snapshot buffer have the same order as main buffer */ + order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, order); + if (ret < 0) + return ret; + /* allocate spare buffer */ ret = resize_buffer_duplicate_size(&tr->max_buffer, &tr->array_buffer, RING_BUFFER_ALL_CPUS); @@ -1286,6 +1293,7 @@ static void free_snapshot(struct trace_array *tr) * The max_tr ring buffer has some state (e.g. ring->clock) and * we want preserve it. */ + ring_buffer_subbuf_order_set(tr->max_buffer.buffer, 0); ring_buffer_resize(tr->max_buffer.buffer, 1, RING_BUFFER_ALL_CPUS); set_buffer_entries(&tr->max_buffer, 1); tracing_reset_online_cpus(&tr->max_buffer); @@ -9393,6 +9401,7 @@ buffer_order_write(struct file *filp, const char __user *ubuf, { struct trace_array *tr = filp->private_data; unsigned long val; + int old_order; int ret; ret = kstrtoul_from_user(ubuf, cnt, 10, &val); @@ -9403,12 +9412,44 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (val < 0 || val > 7) return -EINVAL; + old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + if (old_order == val) + return 0; + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); if (ret) - return ret; + return 0; - (*ppos)++; +#ifdef CONFIG_TRACER_MAX_TRACE + + if (!tr->allocated_snapshot) + goto out_max; + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, val); + if (ret) { + /* Put back the old order */ + cnt = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, old_order); + if (WARN_ON_ONCE(cnt)) { + /* + * AARGH! We are left with different orders! + * The max buffer is our "snapshot" buffer. + * When a tracer needs a snapshot (one of the + * latency tracers), it swaps the max buffer + * with the saved snap shot. We succeeded to + * update the order of the main buffer, but failed to + * update the order of the max buffer. But when we tried + * to reset the main buffer to the original size, we + * failed there too. This is very unlikely to + * happen, but if it does, warn and kill all + * tracing. + */ + tracing_disabled = 1; + } + return ret; + } + out_max: +#endif + (*ppos)++; return cnt; } From patchwork Thu Dec 21 16:10:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182344 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518716dyi; Thu, 21 Dec 2023 08:14:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IEwdb4sjjJ008i8OM10PJoFAY9lygd9nvhO/gSFk8/xnfn98jvWops9ZiBax75CXhZx8WpZ X-Received: by 2002:a17:902:fc8e:b0:1d3:fbf6:4d94 with SMTP id mf14-20020a170902fc8e00b001d3fbf64d94mr1847697plb.75.1703175273481; Thu, 21 Dec 2023 08:14:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175273; cv=none; d=google.com; s=arc-20160816; b=bBY6JugsiLNwSf4oEP08eHOeBpEPCtIzsDAiweXuJbduAYTIAq3ZmN+gU1cyc4VZg0 k7xZ62h23L1kr8ZEmNtfTM7QKauN+rYH4PXY6S5KaAmrS4xLdkP/lKSZ1B8APbybsTjN tLUmbJzNzMq7xtAe5njoCosrnkb4XmaAEmZeLxq8TrbLsX6KoAp4rKZpgFAQhjurg1v0 xmqSDwkXUDQbmZ7YMclIFRB7hs/E3suKN2VzDRKOgrspB9D7HNYWiXzIUxq1/xrdqxjl CQFYTaoBo5tXE/PTF1n8vCJhcGrBXI4PCFqXxrSdgqJARDWQwkOa6TRk/uhcopUnvQfF pMQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=5hNWDqgSFY9AXxJ8P0I9Rkvn5qtR5h3YHL9ykj1XeeM=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=AD5YdoLmlQLFBH/VEmhgf59nCQuj3Uc2A5UFQvStbdPpQPlFh9uBtBDWwMRHDcWFFg GpYK4S+AiGj8E4a7zc65JMYY46TjbmJL4oKf06xA8M5GQFsK9rFTHM7WHQpXPx9bcZEw w7GsNxNdXyjEW0k/0KmmH5B3xfj3upaRDmAleJAPXEcGwp0NL19t0vvjsUHz+oB/tJHl j4b4/A1KH2Gdj22Gra8SucjolYYXBbe+jn0tndIq92cSZj2kgisCtZvMQexkuVUFLlXL ed9cKrIdcYLJEYBV+i7Nk43BcdlZqhu2sKhPlgz9FPRAI2pkQhpcs8Bl84gfhq4TrWdp G8Fg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8734-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8734-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id o17-20020a170902d4d100b001d0a0ee28e0si1722310plg.288.2023.12.21.08.14.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:14:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8734-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8734-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8734-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id C1E7FB233F7 for ; Thu, 21 Dec 2023 16:13:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8266473182; Thu, 21 Dec 2023 16:10:03 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3AE2634E1 for ; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 522E4C433CA; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdZ-000000041L3-1Jnc; Thu, 21 Dec 2023 11:11:05 -0500 Message-ID: <20231221161105.083563911@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:34 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 10/16] tracing: Stop the tracing while changing the ring buffer subbuf size References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908715540060673 X-GMAIL-MSGID: 1785908715540060673 From: "Steven Rostedt (Google)" Because the main buffer and the snapshot buffer need to be the same for some tracers, otherwise it will fail and disable all tracing, the tracers need to be stopped while updating the sub buffer sizes so that the tracers see the main and snapshot buffers with the same sub buffer size. Link: https://lore.kernel.org/linux-trace-kernel/20231219185630.353222794@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Fixes: 2808e31ec12e ("ring-buffer: Add interface for configuring trace sub buffer size") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 2439e00aa4ce..82303bd2bba1 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9412,13 +9412,16 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (val < 0 || val > 7) return -EINVAL; + /* Do not allow tracing while changing the order of the ring buffer */ + tracing_stop_tr(tr); + old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); if (old_order == val) - return 0; + goto out; ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); if (ret) - return 0; + goto out; #ifdef CONFIG_TRACER_MAX_TRACE @@ -9445,11 +9448,15 @@ buffer_order_write(struct file *filp, const char __user *ubuf, */ tracing_disabled = 1; } - return ret; + goto out; } out_max: #endif (*ppos)++; + out: + if (ret) + cnt = ret; + tracing_start_tr(tr); return cnt; } From patchwork Thu Dec 21 16:10:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182343 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518690dyi; Thu, 21 Dec 2023 08:14:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IHZWQzddN8BuXVDxlRQnXxXafEgVysQWOqghYgPeUJBl3pbxfnevZOSAgocguzZaAloCQxT X-Received: by 2002:a05:6e02:1ba9:b0:35c:e98f:c591 with SMTP id n9-20020a056e021ba900b0035ce98fc591mr21667599ili.3.1703175271720; Thu, 21 Dec 2023 08:14:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175271; cv=none; d=google.com; s=arc-20160816; b=FDS3Bzp7x7P+a5hEgbOqHceL5auW8gHKCAX5XthKRmMjI7Ze1gNmCVzBUOnRpw0xhi /rpLUT5bnTho66uJx+cesdEen31NTMzw9UuIe1IHfgBTrfLWwTVzT0rkgV8TqOpjAXFy nWBOMSNs/Lwuuf8quhFHvNIoDgQDuH6qVhZ2V7jgP8f52pLB/oj6j8yZGY6bNaO0PRiW VJVcLTBOPR7F4+sZjGqwn8GOmB92iFJzF47CI8ybXvu7ve7raMQpZ1oq5n2o9piQ5/NE rNDSaCRDSzyg4yWI364OIAnCPzkXng8CVCkIBWjDyp0Ee/n54zyJSa0eHczoBa+GGa6M Krvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=LivDZpjGlV656DTtofl9jUnIAaYu2QkVpB94alOxgAg=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=C6h0EVXoh3Nc+tB5G5pJQjO8ymcIKhDWZzC0rbw2wwosrwg6nobkytbZCu7Iuht1EN Hbqwas7PVfrXMgQtKZiyXzJPPkhUhW/E5Fg0VQ3sEjOGSmlleBwfUly5D26ekyWdV+Jl mHC9fAIYUFuA0RmIVn3EPIDRbhIX1NIPRZIgryex+1vKhiixr145y8DERRJdWgvGz/+T xzOeyTjRGh5UJ8W7r1Oyh36K4pqYeG2ZNwhzvtthJjsL9cXDciDlJ0nYhnoeJTgLcbWO sWCS6BXddcGKe0iq7Clm9QvpoE1cBueTX3T6x7Np86PJBVL+cz91/GIspHZREPeFjAOw Wflg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8735-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8735-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id h33-20020a63f921000000b005cd8b9dc455si1758352pgi.567.2023.12.21.08.14.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:14:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8735-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8735-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8735-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 94E41B22B17 for ; Thu, 21 Dec 2023 16:13:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8161C6EB77; Thu, 21 Dec 2023 16:10:03 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCE00634E3 for ; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9AA6FC433B9; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdZ-000000041LZ-2YYa; Thu, 21 Dec 2023 11:11:05 -0500 Message-ID: <20231221161105.389855512@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:35 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 11/16] ring-buffer: Keep the same size when updating the order References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908713434020816 X-GMAIL-MSGID: 1785908713434020816 From: "Steven Rostedt (Google)" The function ring_buffer_subbuf_order_set() just updated the sub-buffers to the new size, but this also changes the size of the buffer in doing so. As the size is determined by nr_pages * subbuf_size. If the subbuf_size is increased without decreasing the nr_pages, this causes the total size of the buffer to increase. This broke the latency tracers as the snapshot needs to be the same size as the main buffer. The size of the snapshot buffer is only expanded when needed, and because the order is still the same, the size becomes out of sync with the main buffer, as the main buffer increased in size without the tracing system knowing. Calculate the nr_pages to allocate with the new subbuf_size to be buffer_size / new_subbuf_size. Link: https://lore.kernel.org/linux-trace-kernel/20231219185630.649397785@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Fixes: f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index fdcd171b09b5..23ead7602da0 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5897,7 +5897,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; - nr_pages = buffer->buffers[cpu]->nr_pages; + /* Update the number of pages to match the new size */ + nr_pages = old_size * buffer->buffers[cpu]->nr_pages; + nr_pages = DIV_ROUND_UP(nr_pages, buffer->subbuf_size); + cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); if (!cpu_buffers[cpu]) { err = -ENOMEM; From patchwork Thu Dec 21 16:10:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182339 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518070dyi; Thu, 21 Dec 2023 08:13:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IG3Edj2uCAa0nuHQ0QL/5bTrLQWw3Bs/nXR8HNanTcTiBVPETDruer+K02WTMQI0VGKN1LU X-Received: by 2002:a17:906:844d:b0:a26:8d62:e78d with SMTP id e13-20020a170906844d00b00a268d62e78dmr13008ejy.74.1703175214705; Thu, 21 Dec 2023 08:13:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175214; cv=none; d=google.com; s=arc-20160816; b=sdsVWXMearf10LWAX1xjei35nfsBztS+DVtx9BNyUuFNc7xJwYeC6Wd0Y+wAENL3rF 0/IiBgdCvMc1ate99AvpXZkHmNAuGESA/KrRJa/3Yj8C12noCLstFsVkLx8O0KmvYxJW e18CcRgQ/Uh0qrt2a3RgfviuVtq9ST3dbo7hTezLNbpXxfZZxqfefvLc043gNhZTVlFl cTdwv3b1FqILnQ+otX9afr+H514WVVL21g7BsJiqif08hVtATBBxLupjSIHm8CmsW7Xf zUZcNz6bZr+yDhku5PnLPvud5pxJMlyJDlabQ9lobHMrrtRfXsLDW1jJ1BPKZVpPeTIr iphQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=/ZIXAxE12UkduEy5ria+JA6QT6BIDuw6g1lDfFyRoS0=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=WcdYqjNbgdkRSG/637ioB2n5ly+3iUWSO2hcfPAnabKRa2NLEvI0mFVSg6yp72YIwh b7dVaTysBgchu6Pnb1aS4RiywmUFBfmQx0hw7p5MNa8GLXMlwCCCI/JIgSQoidM+pWX4 /PAZYI+ttV/SxHeX5BuIYHW7WxIeE5XuhqUQW8L+sWgfB/fiTdcKGEevDOXm2bV2W0GN O/iyYdlIcYgIzq40lOanNQd64aM5lq/g4u2A+VobLAD2hHVhtlM2EuUgc/urOQuGgK9m d6+rAskYSrN8R6uOPd6E+Nc/TfXH56bFIQcLV7ns7s+6fYxiF3wbCesVKVuSqHC+bOVN agoQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8736-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8736-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id gg19-20020a170906e29300b00a2362de599bsi927563ejb.1010.2023.12.21.08.13.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:13:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8736-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8736-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8736-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EC92E1F25DCF for ; Thu, 21 Dec 2023 16:13:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E38B774E00; Thu, 21 Dec 2023 16:10:03 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 173EB63509 for ; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5C30C43395; Thu, 21 Dec 2023 16:10:01 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdZ-000000041M5-3mqd; Thu, 21 Dec 2023 11:11:05 -0500 Message-ID: <20231221161105.683206821@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:36 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 12/16] ring-buffer: Just update the subbuffers when changing their allocation order References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908654116850719 X-GMAIL-MSGID: 1785908654116850719 From: "Steven Rostedt (Google)" The ring_buffer_subbuf_order_set() was creating ring_buffer_per_cpu cpu_buffers with the new subbuffers with the updated order, and if they all successfully were created, then they the ring_buffer's per_cpu buffers would be freed and replaced by them. The problem is that the freed per_cpu buffers contains state that would be lost. Running the following commands: 1. # echo 3 > /sys/kernel/tracing/buffer_subbuf_order 2. # echo 0 > /sys/kernel/tracing/tracing_cpumask 3. # echo 1 > /sys/kernel/tracing/snapshot 4. # echo ff > /sys/kernel/tracing/tracing_cpumask 5. # echo test > /sys/kernel/tracing/trace_marker Would result in: -bash: echo: write error: Bad file descriptor That's because the state of the per_cpu buffers of the snapshot buffer is lost when the order is changed (the order of a freed snapshot buffer goes to 0 to save memory, and when the snapshot buffer is allocated again, it goes back to what the main buffer is). In operation 2, the snapshot buffers were set to "disable" (as all the ring buffers CPUs were disabled). In operation 3, the snapshot is allocated and a call to ring_buffer_subbuf_order_set() replaced the per_cpu buffers losing the "record_disable" count. When it was enabled again, the atomic_dec(&cpu_buffer->record_disable) was decrementing a zero, setting it to -1. Writing 1 into the snapshot would swap the snapshot buffer with the main buffer, so now the main buffer is "disabled", and nothing can write to the ring buffer anymore. Instead of creating new per_cpu buffers and losing the state of the old buffers, basically do what the resize does and just allocate new subbuf pages into the new_pages link list of the per_cpu buffer and if they all succeed, then replace the old sub buffers with the new ones. This keeps the per_cpu buffer descriptor in tact and by doing so, keeps its state. Link: https://lore.kernel.org/linux-trace-kernel/20231219185630.944104939@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Fixes: f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 88 ++++++++++++++++++++++++++++++-------- 1 file changed, 71 insertions(+), 17 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 23ead7602da0..7ee6779bf292 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5856,11 +5856,11 @@ EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); */ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) { - struct ring_buffer_per_cpu **cpu_buffers; + struct ring_buffer_per_cpu *cpu_buffer; + struct buffer_page *bpage, *tmp; int old_order, old_size; int nr_pages; int psize; - int bsize; int err; int cpu; @@ -5874,11 +5874,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (psize <= BUF_PAGE_HDR_SIZE) return -EINVAL; - bsize = sizeof(void *) * buffer->cpus; - cpu_buffers = kzalloc(bsize, GFP_KERNEL); - if (!cpu_buffers) - return -ENOMEM; - old_order = buffer->subbuf_order; old_size = buffer->subbuf_size; @@ -5894,33 +5889,88 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) /* Make sure all new buffers are allocated, before deleting the old ones */ for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; + cpu_buffer = buffer->buffers[cpu]; + /* Update the number of pages to match the new size */ nr_pages = old_size * buffer->buffers[cpu]->nr_pages; nr_pages = DIV_ROUND_UP(nr_pages, buffer->subbuf_size); - cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); - if (!cpu_buffers[cpu]) { + /* we need a minimum of two pages */ + if (nr_pages < 2) + nr_pages = 2; + + cpu_buffer->nr_pages_to_update = nr_pages; + + /* Include the reader page */ + nr_pages++; + + /* Allocate the new size buffer */ + INIT_LIST_HEAD(&cpu_buffer->new_pages); + if (__rb_allocate_pages(cpu_buffer, nr_pages, + &cpu_buffer->new_pages)) { + /* not enough memory for new pages */ err = -ENOMEM; goto error; } } for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; - rb_free_cpu_buffer(buffer->buffers[cpu]); - buffer->buffers[cpu] = cpu_buffers[cpu]; + cpu_buffer = buffer->buffers[cpu]; + + /* Clear the head bit to make the link list normal to read */ + rb_head_page_deactivate(cpu_buffer); + + /* Now walk the list and free all the old sub buffers */ + list_for_each_entry_safe(bpage, tmp, cpu_buffer->pages, list) { + list_del_init(&bpage->list); + free_buffer_page(bpage); + } + /* The above loop stopped an the last page needing to be freed */ + bpage = list_entry(cpu_buffer->pages, struct buffer_page, list); + free_buffer_page(bpage); + + /* Free the current reader page */ + free_buffer_page(cpu_buffer->reader_page); + + /* One page was allocated for the reader page */ + cpu_buffer->reader_page = list_entry(cpu_buffer->new_pages.next, + struct buffer_page, list); + list_del_init(&cpu_buffer->reader_page->list); + + /* The cpu_buffer pages are a link list with no head */ + cpu_buffer->pages = cpu_buffer->new_pages.next; + cpu_buffer->new_pages.next->prev = cpu_buffer->new_pages.prev; + cpu_buffer->new_pages.prev->next = cpu_buffer->new_pages.next; + + /* Clear the new_pages list */ + INIT_LIST_HEAD(&cpu_buffer->new_pages); + + cpu_buffer->head_page + = list_entry(cpu_buffer->pages, struct buffer_page, list); + cpu_buffer->tail_page = cpu_buffer->commit_page = cpu_buffer->head_page; + + cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; + cpu_buffer->nr_pages_to_update = 0; + + free_pages((unsigned long)cpu_buffer->free_page, old_order); + cpu_buffer->free_page = NULL; + + rb_head_page_activate(cpu_buffer); + + rb_check_pages(cpu_buffer); } atomic_dec(&buffer->record_disabled); mutex_unlock(&buffer->mutex); - kfree(cpu_buffers); - return 0; error: @@ -5931,12 +5981,16 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) mutex_unlock(&buffer->mutex); for_each_buffer_cpu(buffer, cpu) { - if (!cpu_buffers[cpu]) + cpu_buffer = buffer->buffers[cpu]; + + if (!cpu_buffer->nr_pages_to_update) continue; - rb_free_cpu_buffer(cpu_buffers[cpu]); - kfree(cpu_buffers[cpu]); + + list_for_each_entry_safe(bpage, tmp, &cpu_buffer->new_pages, list) { + list_del_init(&bpage->list); + free_buffer_page(bpage); + } } - kfree(cpu_buffers); return err; } From patchwork Thu Dec 21 16:10:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182341 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518250dyi; Thu, 21 Dec 2023 08:13:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IF1/6q58R1KC0HY93/qToeJiK0sJzrqQHJ+2knmySBn5oBj24VNko4BfkrOjunidCqICME0 X-Received: by 2002:a05:6122:459d:b0:4b6:c771:47e8 with SMTP id de29-20020a056122459d00b004b6c77147e8mr712000vkb.20.1703175232051; Thu, 21 Dec 2023 08:13:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175232; cv=none; d=google.com; s=arc-20160816; b=1G7Z9LwvQ1m5Uv3acFMrke7KoYbHvOoLOturW55w4R9DjizydRtqtXexUV39VpGPFE 1K5cIjtfJPncAa5EtIGbEq1Wlca6j33oMSXEJTwyEaoP8MHuNsOBXvFTm5WVvllgV3D3 TOFSRM3XGo+lGtB4o0roWKMVFHYT92QanzlNXEYcRzl6n2vcPokDbQortQPlRVCVvy/1 AWm1Vl7BRReekwFs+371aoZza+zUTyyLxtlEW9EYZ2WeL8EnhKEFlx+posKxvN78pdKF gtKPzNyO8jqoVRERHovaLh7evYrXSdwlgBZbM2CwN6nXigemdVDE403Wpz74vS539Sja MsQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=McYJKPLfQ+rkepx6eEblxPh4fiijViQKWNHp3/gGEXc=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=U4XLtoan00U/nk86zCMXFGPVuCL61dWaYVZUKanOH6T6250i1573cQaKZ9TtvSVIj1 P0QlnA8I6vfMrq35dMPGKBVZMBwfh1y1ST16X58UBgllgVofmnt43CmMk1P0lWbs0xDU NL47NmIa5AzaHO2cJUs0WXExDAB6E6k/sf/oeUq2WkXcMgXBUCeuLC/3RBXqtVvWwTtZ a2OjVz7Ee4dNyAD+HQAyn9Wxiig4nxQxsiNenzSTjDmPCxfFGwB7G/3su9RVk39FdRYZ 0bi7tkG+EuRGELPwBFoYSG2Tj8+NZ5yqatmUjIR4MDMAcG5no4FaMU6CUTRXAQv+icFM ryEg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8737-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8737-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id b185-20020a1fe4c2000000b004b6e6e149b7si474111vkh.51.2023.12.21.08.13.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:13:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8737-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8737-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8737-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id CBE6E1C22DAD for ; Thu, 21 Dec 2023 16:13:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7507059927; Thu, 21 Dec 2023 16:10:04 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B376364AB3 for ; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34733C433C7; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLda-000000041Ma-0nq3; Thu, 21 Dec 2023 11:11:06 -0500 Message-ID: <20231221161105.976193556@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:37 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 13/16] ring-buffer: Add documentation on the buffer_subbuf_order file References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908672104878608 X-GMAIL-MSGID: 1785908672104878608 From: "Steven Rostedt (Google)" Add to the documentation how to use the buffer_subbuf_order file to change the size and how it affects what events can be added to the ring buffer. Link: https://lore.kernel.org/linux-trace-kernel/20231219185631.230636734@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 23572f6697c0..231d26ceedb0 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -203,6 +203,33 @@ of ftrace. Here is a list of some of the key files: This displays the total combined size of all the trace buffers. + buffer_subbuf_order: + + This sets or displays the sub buffer page size order. The ring buffer + is broken up into several same size "sub buffers". An event can not be + bigger than the size of the sub buffer. Normally, the sub buffer is + the size of the architecture's page (4K on x86). The sub buffer also + contains meta data at the start which also limits the size of an event. + That means when the sub buffer is a page size, no event can be larger + than the page size minus the sub buffer meta data. + + The buffer_subbuf_order allows the user to change the size of the sub + buffer. As the sub buffer is a set of pages by the power of 2, thus + the sub buffer total size is defined by the order: + + order size + ---- ---- + 0 PAGE_SIZE + 1 PAGE_SIZE * 2 + 2 PAGE_SIZE * 4 + 3 PAGE_SIZE * 8 + + Changing the order will change the sub buffer size allowing for events + to be larger than the page size. + + Note: When changing the order, tracing is stopped and any data in the + ring buffer and the snapshot buffer will be discarded. + free_buffer: If a process is performing tracing, and the ring buffer should be From patchwork Thu Dec 21 16:10:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182346 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp522158dyi; Thu, 21 Dec 2023 08:19:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IESlJpyfvXqapSFqk//1JqwVAMM/0poc8EOi4PUOqEyVI3FcbNE589Khc+CPJapv1OJBVox X-Received: by 2002:a05:6a00:2296:b0:6d3:f68b:3439 with SMTP id f22-20020a056a00229600b006d3f68b3439mr10094977pfe.14.1703175560867; Thu, 21 Dec 2023 08:19:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175560; cv=none; d=google.com; s=arc-20160816; b=izhNo5UnlY1GnF3L1mQTprkg77t+6j4cyJ43vEvvhQ8wKZlF5g6brBpurwjUMs2ezh yPLAHAh7F32PnBEL9NLqH14ycTt3VcBIgWjbyxA9Ob8pyL4Qux+t+kc2VMUMP561eoJa RpUxT10VpcvH+0j54m5TWpqkgSN21YHaEaWavIyl4sIsc8sUdb+XYzi+KNjBPINxlBWj S5n/GuDaCERpSuQr4Zo0/fu4gTIOopFSD5jCC1VsIJjGuTyyJZlq4G12tRAOh+cTmRD1 twBhj5QfRcryo2S8kjT65FK9cTukXHDPvff2ZokgFLAaEZfhJ4sDtYNJhFtdbkeC97uE T/1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=lGPPxRdqME4e6a4LKtn6wPEkI6zFpV2yKIOWqhqLYsc=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=l9bnu+O6Xh0P8hq85XkFd7AEQ6ozRicIQ7otSNlQS0MD+q1e4FQHfuWBKCkL6phdam k+9PpEVyb8AFtiChXByFqZ1+0EBvAZ9kqRzqNg6Vo+0DlEwjR8UexTyc719Xv5bdIi/r S1CyyE2kkE3w5c2TdolRXa4YLnc/e9hHtnmVkTOJqgFY79qJBWOOZ93eIHxYbuWBaTHn MaybPWfX43Q9OqCQoFzYpYRBv2Dob+m42Eb72bLAEZgodo8hAtuix9OsgvMtwzxfHhxx NWvEgcpybgLYW10fpPV0vu2leMC8MKDrQVP3KFlTUJgAsirzWIN5qbIUt0XIB1OqrICO ZFQg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8738-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8738-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id f25-20020a631f19000000b005c6ef013589si1732144pgf.641.2023.12.21.08.19.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:19:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8738-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8738-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8738-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3220028CD2E for ; Thu, 21 Dec 2023 16:13:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A9E8F768E0; Thu, 21 Dec 2023 16:10:04 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2C7965188 for ; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 770D2C433CA; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLda-000000041N5-20d0; Thu, 21 Dec 2023 11:11:06 -0500 Message-ID: <20231221161106.268425007@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:38 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 14/16] ringbuffer/selftest: Add basic selftest to test changing subbuf order References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785909017157361865 X-GMAIL-MSGID: 1785909017157361865 From: "Steven Rostedt (Google)" Add a self test that will write into the trace buffer with differ trace sub buffer order sizes. Link: https://lore.kernel.org/linux-trace-kernel/20231219185631.520496304@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Steven Rostedt (Google) --- .../ftrace/test.d/00basic/ringbuffer_order.tc | 95 +++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc diff --git a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc new file mode 100644 index 000000000000..ecbcc810e6c1 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc @@ -0,0 +1,95 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Change the ringbuffer sub-buffer order +# requires: buffer_subbuf_order +# flags: instance + +get_buffer_data_size() { + sed -ne 's/^.*data.*size:\([0-9][0-9]*\).*/\1/p' events/header_page +} + +get_buffer_data_offset() { + sed -ne 's/^.*data.*offset:\([0-9][0-9]*\).*/\1/p' events/header_page +} + +get_event_header_size() { + type_len=`sed -ne 's/^.*type_len.*:[^0-9]*\([0-9][0-9]*\).*/\1/p' events/header_event` + time_len=`sed -ne 's/^.*time_delta.*:[^0-9]*\([0-9][0-9]*\).*/\1/p' events/header_event` + array_len=`sed -ne 's/^.*array.*:[^0-9]*\([0-9][0-9]*\).*/\1/p' events/header_event` + total_bits=$((type_len+time_len+array_len)) + total_bits=$((total_bits+7)) + echo $((total_bits/8)) +} + +get_print_event_buf_offset() { + sed -ne 's/^.*buf.*offset:\([0-9][0-9]*\).*/\1/p' events/ftrace/print/format +} + +event_header_size=`get_event_header_size` +print_header_size=`get_print_event_buf_offset` + +data_offset=`get_buffer_data_offset` + +marker_meta=$((event_header_size+print_header_size)) + +make_str() { + cnt=$1 + printf -- 'X%.0s' $(seq $cnt) +} + +write_buffer() { + size=$1 + + str=`make_str $size` + + # clear the buffer + echo > trace + + # write the string into the marker + echo $str > trace_marker + + echo $str +} + +test_buffer() { + orde=$1 + page_size=$((4096< buffer_subbuf_order + test_buffer $a +done + +echo $ORIG > buffer_subbuf_order + From patchwork Thu Dec 21 16:10:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182342 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp518350dyi; Thu, 21 Dec 2023 08:14:00 -0800 (PST) X-Google-Smtp-Source: AGHT+IGc5pGewW6eKxUOKfsk9aE8gd2bAL0n3eXvNDJKk/uYChG9X9VHjItYEDzwq3kMPTgaumS/ X-Received: by 2002:a17:906:4c4d:b0:a23:52d8:883a with SMTP id d13-20020a1709064c4d00b00a2352d8883amr573ejw.184.1703175240667; Thu, 21 Dec 2023 08:14:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175240; cv=none; d=google.com; s=arc-20160816; b=k0Soe1WXGwJ6qG4etilXesRdE2w386wSICZe1J61L9nzou3rpnsOIi+7Oy86VclDOP uV68ZJX13CD0NPO40VpstaRAPq/Hd8ldmdTzGuSIAR4KERd68BOjWxA5gI8DK3t2DMZj uTVZHggeWVoBwdJeiarrJz9x7E4+QJZK/YNpSMYWhyWgpquDozvep0jXwjTyTGqLXcqG NkO/Fho9HmPhPYx57jNSTIBt4LnQwUcMAf7dPkOAn4NUYRrL3SiXrOuJrOZv9eOL/FMF Rf8csnuwITMfsxvt7oM25rfYatG2T3YlWfNYkY9tcw5VBiFfg0xp5ioPzkjO+s4iJJVV U0Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=cPhI64W+DSiGb2mrH3irV9zjOLWpR8gLDITVYbmjM3I=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=IeN65S/m5Rh1/Zo5uxWs0IQgqBmxTivjVh+YvD0XWdaZUAYt/bzNMFj42VZGp0o2Im sBNvwesGhKwZnTX4IBMnQcLnTfRHrlQWV9qeTXmqjIdHBnV0veAkYZGjfcS5vzjciBsy EGCgiFP547xBOmJvMo3AziTkqcfsHegRImQsLWwLOVlQDL6dfnGDCzqkAQdZYmkC67d2 Uvu8IXQDsTDXblSYPPlMYcCNIJMPXnGMrawyz9mOk/hLZxR1VmomYjmIf0AkHoitM0WK 8UdWDQMdDgUB4+eTKQxt74R3ByNmeIOg/w38aWuvX9OvztdEwIvuPjXFORxOlPbFKlGL AbPw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8739-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8739-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id vz21-20020a17090704d500b00a233dc7143bsi935897ejb.136.2023.12.21.08.14.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:14:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8739-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8739-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8739-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 196C81F248F6 for ; Thu, 21 Dec 2023 16:14:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BB10C55E6F; Thu, 21 Dec 2023 16:10:04 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE74F65193 for ; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CAB33C433C8; Thu, 21 Dec 2023 16:10:02 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLda-000000041Na-3G2C; Thu, 21 Dec 2023 11:11:06 -0500 Message-ID: <20231221161106.554486678@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:39 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 15/16] tracing: Update subbuffer with kilobytes not page order References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908680926185101 X-GMAIL-MSGID: 1785908680926185101 From: "Steven Rostedt (Google)" Using page order for deciding what the size of the ring buffer sub buffers are is exposing a bit too much of the implementation. Although the sub buffers are only allocated in orders of pages, allow the user to specify the minimum size of each sub-buffer via kilobytes like they can with the buffer size itself. If the user specifies 3 via: echo 3 > buffer_subbuf_size_kb Then the sub-buffer size will round up to 4kb (on a 4kb page size system). If they specify: echo 6 > buffer_subbuf_size_kb The sub-buffer size will become 8kb. and so on. Link: https://lore.kernel.org/linux-trace-kernel/20231219185631.809766769@goodmis.org Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 46 ++++++++----------- kernel/trace/trace.c | 38 +++++++++------ ...fer_order.tc => ringbuffer_subbuf_size.tc} | 18 ++++---- 3 files changed, 54 insertions(+), 48 deletions(-) rename tools/testing/selftests/ftrace/test.d/00basic/{ringbuffer_order.tc => ringbuffer_subbuf_size.tc} (85%) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 231d26ceedb0..933e7efb9f1b 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -203,32 +203,26 @@ of ftrace. Here is a list of some of the key files: This displays the total combined size of all the trace buffers. - buffer_subbuf_order: - - This sets or displays the sub buffer page size order. The ring buffer - is broken up into several same size "sub buffers". An event can not be - bigger than the size of the sub buffer. Normally, the sub buffer is - the size of the architecture's page (4K on x86). The sub buffer also - contains meta data at the start which also limits the size of an event. - That means when the sub buffer is a page size, no event can be larger - than the page size minus the sub buffer meta data. - - The buffer_subbuf_order allows the user to change the size of the sub - buffer. As the sub buffer is a set of pages by the power of 2, thus - the sub buffer total size is defined by the order: - - order size - ---- ---- - 0 PAGE_SIZE - 1 PAGE_SIZE * 2 - 2 PAGE_SIZE * 4 - 3 PAGE_SIZE * 8 - - Changing the order will change the sub buffer size allowing for events - to be larger than the page size. - - Note: When changing the order, tracing is stopped and any data in the - ring buffer and the snapshot buffer will be discarded. + buffer_subbuf_size_kb: + + This sets or displays the sub buffer size. The ring buffer is broken up + into several same size "sub buffers". An event can not be bigger than + the size of the sub buffer. Normally, the sub buffer is the size of the + architecture's page (4K on x86). The sub buffer also contains meta data + at the start which also limits the size of an event. That means when + the sub buffer is a page size, no event can be larger than the page + size minus the sub buffer meta data. + + Note, the buffer_subbuf_size_kb is a way for the user to specify the + minimum size of the subbuffer. The kernel may make it bigger due to the + implementation details, or simply fail the operation if the kernel can + not handle the request. + + Changing the sub buffer size allows for events to be larger than the + page size. + + Note: When changing the sub-buffer size, tracing is stopped and any + data in the ring buffer and the snapshot buffer will be discarded. free_buffer: diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 82303bd2bba1..46dbe22121e9 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9384,42 +9384,54 @@ static const struct file_operations buffer_percent_fops = { }; static ssize_t -buffer_order_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +buffer_subbuf_size_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) { struct trace_array *tr = filp->private_data; + size_t size; char buf[64]; + int order; int r; - r = sprintf(buf, "%d\n", ring_buffer_subbuf_order_get(tr->array_buffer.buffer)); + order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + size = (PAGE_SIZE << order) / 1024; + + r = sprintf(buf, "%zd\n", size); return simple_read_from_buffer(ubuf, cnt, ppos, buf, r); } static ssize_t -buffer_order_write(struct file *filp, const char __user *ubuf, - size_t cnt, loff_t *ppos) +buffer_subbuf_size_write(struct file *filp, const char __user *ubuf, + size_t cnt, loff_t *ppos) { struct trace_array *tr = filp->private_data; unsigned long val; int old_order; + int order; + int pages; int ret; ret = kstrtoul_from_user(ubuf, cnt, 10, &val); if (ret) return ret; + val *= 1024; /* value passed in is in KB */ + + pages = DIV_ROUND_UP(val, PAGE_SIZE); + order = fls(pages - 1); + /* limit between 1 and 128 system pages */ - if (val < 0 || val > 7) + if (order < 0 || order > 7) return -EINVAL; /* Do not allow tracing while changing the order of the ring buffer */ tracing_stop_tr(tr); old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); - if (old_order == val) + if (old_order == order) goto out; - ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, order); if (ret) goto out; @@ -9428,7 +9440,7 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (!tr->allocated_snapshot) goto out_max; - ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, val); + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, order); if (ret) { /* Put back the old order */ cnt = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, old_order); @@ -9460,10 +9472,10 @@ buffer_order_write(struct file *filp, const char __user *ubuf, return cnt; } -static const struct file_operations buffer_order_fops = { +static const struct file_operations buffer_subbuf_size_fops = { .open = tracing_open_generic_tr, - .read = buffer_order_read, - .write = buffer_order_write, + .read = buffer_subbuf_size_read, + .write = buffer_subbuf_size_write, .release = tracing_release_generic_tr, .llseek = default_llseek, }; @@ -9934,8 +9946,8 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) trace_create_file("buffer_percent", TRACE_MODE_WRITE, d_tracer, tr, &buffer_percent_fops); - trace_create_file("buffer_subbuf_order", TRACE_MODE_WRITE, d_tracer, - tr, &buffer_order_fops); + trace_create_file("buffer_subbuf_size_kb", TRACE_MODE_WRITE, d_tracer, + tr, &buffer_subbuf_size_fops); create_trace_options_dir(tr); diff --git a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_subbuf_size.tc similarity index 85% rename from tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc rename to tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_subbuf_size.tc index ecbcc810e6c1..d44d09a33a74 100644 --- a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc +++ b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_subbuf_size.tc @@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 -# description: Change the ringbuffer sub-buffer order -# requires: buffer_subbuf_order +# description: Change the ringbuffer sub-buffer size +# requires: buffer_subbuf_size_kb # flags: instance get_buffer_data_size() { @@ -52,8 +52,8 @@ write_buffer() { } test_buffer() { - orde=$1 - page_size=$((4096< buffer_subbuf_order +for a in 4 8 16 32 ; do + echo $a > buffer_subbuf_size_kb test_buffer $a done -echo $ORIG > buffer_subbuf_order +echo $ORIG > buffer_subbuf_size_kb From patchwork Thu Dec 21 16:10:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 182345 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp519579dyi; Thu, 21 Dec 2023 08:15:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IGHbjkUx2H5drZuyfC9Ik6BE6P5y6xLGCxd5lntLE2QBL2NbaCqACIHR6Pq0c+dqDN8pyOj X-Received: by 2002:a17:902:780e:b0:1d0:ae84:6107 with SMTP id p14-20020a170902780e00b001d0ae846107mr12001229pll.38.1703175340350; Thu, 21 Dec 2023 08:15:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703175340; cv=none; d=google.com; s=arc-20160816; b=LaxA4C+QlqCD3id8LO3gKRxurh8zY4zJWrynyyg7YYJFEITWnstr3u2XH1Ej6ehdoO 9ALzMcwe/lZysUy13WA4bGIhTyZoFHxiTB6M4RVrA6KOlNgao6gcEA/WhP4Z+GxQCrFn /P+XjqEFEBGcIQGEURvM4m/F3fS4EVXZwyLPmZjMc/4bzFh/flApk6GeSUoqEOIYxzjl KGYRNp2PMyFi2FGiSnxuuQWVvgli+MDWudbvVOwPJOwbTuRkDmrBxh8cylvWM6dXV0AW ZvKQJC3WkAprfSpjUwbLXGwGVx7BfvqhgbbayXD3EhT4klvEK03p6DVGrDbuI6e6qgKt i9QQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :references:subject:cc:to:from:date:user-agent:message-id; bh=abVV5XX4+XCcDrTPRiIUzBfXP36iBBmzHiAwXpBOFso=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=tNUfKmL0w/ThDsREcKUtQR6bzQufEediDMqifFDcD+ojZP7iEtX9P1/BRtPD4VT610 eq7G3wO26QY7ViyANVFVP7///H2pRopPuUlXRqOqzUL46Q8jgEFSMeRQhUcZx+9YUckY iBQoqyn+MzWXZOO/oyXYTlCKXPY8U36dWrgWOEK4ddLtsHZTWb8n7+ondkdNeHvdSsrL QU7zGsUvbm4PM8xh7PqjOFZ7q8rrOOVglUzJsXmE4nEY4zmabOFOrSrXPfENdoa4Nec+ 4WVAipz+ryZodb1Bi33wSUeRQWg3njaSA/yHwnuAP/k8l1vu5gqzca5tbcTPg1rGH0Kh wcmQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8740-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8740-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id a9-20020aa78e89000000b006d96355329bsi1759910pfr.256.2023.12.21.08.15.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 08:15:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-8740-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-8740-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-8740-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id DB713B23E90 for ; Thu, 21 Dec 2023 16:14:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2B77965193; Thu, 21 Dec 2023 16:10:05 +0000 (UTC) X-Original-To: linux-kernel@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 832AB73190 for ; Thu, 21 Dec 2023 16:10:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 13C46C433CC; Thu, 21 Dec 2023 16:10:03 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rGLdb-000000041O5-0GxN; Thu, 21 Dec 2023 11:11:07 -0500 Message-ID: <20231221161106.846599574@goodmis.org> User-Agent: quilt/0.67 Date: Thu, 21 Dec 2023 11:10:40 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [for-next][PATCH 16/16] ring-buffer: Use subbuf_order for buffer page masking References: <20231221161024.478795180@goodmis.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785908785664323357 X-GMAIL-MSGID: 1785908785664323357 From: "Steven Rostedt (Google)" The comparisons to PAGE_SIZE were all converted to use the buffer->subbuf_order, but the use of PAGE_MASK was missed. Convert all the PAGE_MASK usages over to: (PAGE_SIZE << cpu_buffer->buffer->subbuf_order) - 1 Link: https://lore.kernel.org/linux-trace-kernel/20231219173800.66eefb7a@gandalf.local.home Cc: Masami Hiramatsu Cc: Mark Rutland Cc: Mathieu Desnoyers Cc: Andrew Morton Cc: Tzvetomir Stoyanov Cc: Vincent Donnefort Cc: Kent Overstreet Fixes: 139f84002145 ("ring-buffer: Page size per ring buffer") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 7ee6779bf292..173d2595ce2d 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -2269,11 +2269,13 @@ rb_commit_index(struct ring_buffer_per_cpu *cpu_buffer) } static __always_inline unsigned -rb_event_index(struct ring_buffer_event *event) +rb_event_index(struct ring_buffer_per_cpu *cpu_buffer, struct ring_buffer_event *event) { unsigned long addr = (unsigned long)event; - return (addr & ~PAGE_MASK) - BUF_PAGE_HDR_SIZE; + addr &= (PAGE_SIZE << cpu_buffer->buffer->subbuf_order) - 1; + + return addr - BUF_PAGE_HDR_SIZE; } static void rb_inc_iter(struct ring_buffer_iter *iter) @@ -2646,7 +2648,8 @@ rb_move_tail(struct ring_buffer_per_cpu *cpu_buffer, /* Slow path */ static struct ring_buffer_event * -rb_add_time_stamp(struct ring_buffer_event *event, u64 delta, bool abs) +rb_add_time_stamp(struct ring_buffer_per_cpu *cpu_buffer, + struct ring_buffer_event *event, u64 delta, bool abs) { if (abs) event->type_len = RINGBUF_TYPE_TIME_STAMP; @@ -2654,7 +2657,7 @@ rb_add_time_stamp(struct ring_buffer_event *event, u64 delta, bool abs) event->type_len = RINGBUF_TYPE_TIME_EXTEND; /* Not the first event on the page, or not delta? */ - if (abs || rb_event_index(event)) { + if (abs || rb_event_index(cpu_buffer, event)) { event->time_delta = delta & TS_MASK; event->array[0] = delta >> TS_SHIFT; } else { @@ -2728,7 +2731,7 @@ static void rb_add_timestamp(struct ring_buffer_per_cpu *cpu_buffer, if (!abs) info->delta = 0; } - *event = rb_add_time_stamp(*event, info->delta, abs); + *event = rb_add_time_stamp(cpu_buffer, *event, info->delta, abs); *length -= RB_LEN_TIME_EXTEND; *delta = 0; } @@ -2812,10 +2815,10 @@ rb_try_to_discard(struct ring_buffer_per_cpu *cpu_buffer, struct buffer_page *bpage; unsigned long addr; - new_index = rb_event_index(event); + new_index = rb_event_index(cpu_buffer, event); old_index = new_index + rb_event_ts_length(event); addr = (unsigned long)event; - addr &= PAGE_MASK; + addr &= ~((PAGE_SIZE << cpu_buffer->buffer->subbuf_order) - 1); bpage = READ_ONCE(cpu_buffer->tail_page); @@ -3726,7 +3729,7 @@ rb_decrement_entry(struct ring_buffer_per_cpu *cpu_buffer, struct buffer_page *bpage = cpu_buffer->commit_page; struct buffer_page *start; - addr &= PAGE_MASK; + addr &= ~((PAGE_SIZE << cpu_buffer->buffer->subbuf_order) - 1); /* Do the likely case first */ if (likely(bpage->page == (void *)addr)) {