From patchwork Sun Dec 10 03:54:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176263 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353783vqy; Sat, 9 Dec 2023 20:04:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IHHxnIROJvr90RxzP/1pbUgLg4ARfGbTOGhrk2VGhWKVLJA26YMPCng4/MdYaS9zwETTy13 X-Received: by 2002:a05:6a20:7b12:b0:190:adc:7fd0 with SMTP id s18-20020a056a207b1200b001900adc7fd0mr776257pzh.21.1702181068053; Sat, 09 Dec 2023 20:04:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181068; cv=none; d=google.com; s=arc-20160816; b=G0rGcBvQgj3aBcMyKjfYMnTXS5dLI46OdyXI3pujDIhiRbI1oV86JCNDgJA9DbDyjD nkeDcX6pHS1FPqqDwPgFDBkvHF8nD0ltHA75XPn6bEGZkYcV2Cp17AM8qHyjSxTv+oVR mnRIMST+1sTaFU+SD9u9x55FHkyIKArOf3t0mrblPHl7WehMDdPGqbB9jlb6iJP/umRQ gnCyNZ4VvLK39lyBYkqddrHADS5bQpTC0SxNxcfqNTp2jyPi5sIS7SrZc9uJwPT2FtYl +EmZzirlBM3T0Q61+uHxqrvqHt8hU3acMWbUMqqi6P1MxOIOLK8YzRBC0ndQogopdcTT VRVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=3haQajqhXyxgR+U+Llw5uM+GUSgl3jpHXB2knl5dWFw=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=UuEgufho77kaup8BUZMo1cSlxQMWq581z2JVdGgXbfE9Qqujxjze+mYkVGz/zHdGNw 9rEialaIM3byUGaMLlqFwhHqSD8vj4FiM1hU6xcHjKX/FY32+AgzMSMWZB5PkYY2qBO4 pH/GpsH4GhjTld+GOW3LHH4HFdN6SSHX10/b1jICvd+vxuJGN8nuiqezIA3I4c8/zpu+ 9B9F1+rLmT8+9lYeEK5PHmBeZCRpiGuK9bc8sa4ZJSTj2GWe1wgEy51Wcn++hkzQ6WmV O20dvVYXYllfCzvcywGxlcZPL+OcdiPgeug1k5NLZ4Y/uFI4c+hOYgz0g1KzgQwN7f0T jvXA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id mp5-20020a17090b190500b00286922ce5cdsi4108659pjb.108.2023.12.09.20.04.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id C26AD809501A; Sat, 9 Dec 2023 20:04:26 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231510AbjLJEEI (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33160 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231280AbjLJEEG (ORCPT ); Sat, 9 Dec 2023 23:04:06 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 615F210E for ; Sat, 9 Dec 2023 20:04:12 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C490AC433C9; Sun, 10 Dec 2023 04:04:11 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3g-000000021Lp-3INx; Sat, 09 Dec 2023 23:04:48 -0500 Message-ID: <20231210040448.569462486@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:05 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 01/14] ring-buffer: Refactor ring buffer implementation References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:26 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866215488797695 X-GMAIL-MSGID: 1784866215488797695 From: "Tzvetomir Stoyanov (VMware)" In order to introduce sub-buffer size per ring buffer, some internal refactoring is needed. As ring_buffer_print_page_header() will depend on the trace_buffer structure, it is moved after the structure definition. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-2-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 60 +++++++++++++++++++------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 3a5026702179..d59d14e67da1 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -381,36 +381,6 @@ static inline bool test_time_stamp(u64 delta) /* Events may have a timestamp attached to them */ #define BUF_MAX_EVENT_SIZE (BUF_MAX_DATA_SIZE - RB_LEN_TIME_EXTEND) -int ring_buffer_print_page_header(struct trace_seq *s) -{ - struct buffer_data_page field; - - trace_seq_printf(s, "\tfield: u64 timestamp;\t" - "offset:0;\tsize:%u;\tsigned:%u;\n", - (unsigned int)sizeof(field.time_stamp), - (unsigned int)is_signed_type(u64)); - - trace_seq_printf(s, "\tfield: local_t commit;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), commit), - (unsigned int)sizeof(field.commit), - (unsigned int)is_signed_type(long)); - - trace_seq_printf(s, "\tfield: int overwrite;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), commit), - 1, - (unsigned int)is_signed_type(long)); - - trace_seq_printf(s, "\tfield: char data;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), data), - (unsigned int)BUF_PAGE_SIZE, - (unsigned int)is_signed_type(char)); - - return !trace_seq_has_overflowed(s); -} - struct rb_irq_work { struct irq_work work; wait_queue_head_t waiters; @@ -576,6 +546,36 @@ struct ring_buffer_iter { int missed_events; }; +int ring_buffer_print_page_header(struct trace_seq *s) +{ + struct buffer_data_page field; + + trace_seq_printf(s, "\tfield: u64 timestamp;\t" + "offset:0;\tsize:%u;\tsigned:%u;\n", + (unsigned int)sizeof(field.time_stamp), + (unsigned int)is_signed_type(u64)); + + trace_seq_printf(s, "\tfield: local_t commit;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), commit), + (unsigned int)sizeof(field.commit), + (unsigned int)is_signed_type(long)); + + trace_seq_printf(s, "\tfield: int overwrite;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), commit), + 1, + (unsigned int)is_signed_type(long)); + + trace_seq_printf(s, "\tfield: char data;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), data), + (unsigned int)BUF_PAGE_SIZE, + (unsigned int)is_signed_type(char)); + + return !trace_seq_has_overflowed(s); +} + #ifdef RB_TIME_32 /* From patchwork Sun Dec 10 03:54:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176266 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353840vqy; Sat, 9 Dec 2023 20:04:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFXXp2A5cqB7uLwrl+v5TM96E9YybUkSjGIueQ3+mt0/Ed30V7Hu9mZJT4FqnlPYIEOSiiT X-Received: by 2002:a17:90b:3c02:b0:286:6cc1:866b with SMTP id pb2-20020a17090b3c0200b002866cc1866bmr1144048pjb.80.1702181083329; Sat, 09 Dec 2023 20:04:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181083; cv=none; d=google.com; s=arc-20160816; b=FtzUm7A4s7jMFrNBLh4dVKyBN3xstsREvIbY4R7c8a+TuiUYUzZwUirEFUKFKwtjeP fyiH3YkZjMO6WUtAre0lm2K5ZgiEU/V5W7MUJiybwsmO1RpKMdLiN/7nnE+tOY/geHFK ql15Xh1kt2SzZkbYOApiy7DvaA1lyB1Aq9gUN+8BsHqvg/VlNFG91JVt2gC35podLW4V CnxOs36qZiYDVKYey4SyUdizgarq/k10o9S+qAgcguor2pUDdpBu8FMD6HkawDDKf9W4 73ZE8uCTzuLY3w8IQrYYh0mS645QpevbWyWV0Gy/5UFh71LCdWfZO8RGI8g9nZPPMsss Wy7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=qez4VDLSp/kjgWzlsgMKpjtfRUTlOBqLc9EX7ssUc6M=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=kpVFDGh8rLSSKZ2RiHKEQeiboDW5jZ5gJM5kghCIOdrI9zeXB8zijp3KZ9k8vmmKXp G4g9jFbIV11B0CsuLt8XAVb621IFfzDqX4jr7YeX7Ih5+YVOv4ZCnGyC89IYjOU5hAYl cuzcwirjiY4Y9HzLwHCSHm1w5mI85YAC5RvC2Y2xCbC7A7lPJT/INL/781MI1o4uxpkl WP7pCadrpbvz100qpN7CoLUTD12GTMDyGb4aahZGJ7QS64d/IjSn65qEgyab8ZOrPOSt UoRPh+P41SjqvQcmyhScqvMxOXYdbpBb+AzTRYElVuXDizVg/8F/TnPRkrvkBGyBypKd OMfg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id st15-20020a17090b1fcf00b00283a18d5662si4156659pjb.80.2023.12.09.20.04.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 55EDD80659A8; Sat, 9 Dec 2023 20:04:38 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231549AbjLJEEX (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231489AbjLJEEH (ORCPT ); Sat, 9 Dec 2023 23:04:07 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0C72115 for ; Sat, 9 Dec 2023 20:04:12 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 240DEC433CC; Sun, 10 Dec 2023 04:04:12 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3h-000000021MJ-0IZM; Sat, 09 Dec 2023 23:04:49 -0500 Message-ID: <20231210040448.852632640@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:06 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 02/14] ring-buffer: Page size per ring buffer References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:38 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866231609633129 X-GMAIL-MSGID: 1784866231609633129 From: "Tzvetomir Stoyanov (VMware)" Currently the size of one sub buffer page is global for all buffers and it is hard coded to one system page. In order to introduce configurable ring buffer sub page size, the internal logic should be refactored to work with sub page size per ring buffer. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-3-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 2 +- kernel/trace/ring_buffer.c | 65 +++++++++++++++++++------------------ kernel/trace/trace.c | 2 +- kernel/trace/trace.h | 1 + kernel/trace/trace_events.c | 59 +++++++++++++++++++++++++-------- 5 files changed, 82 insertions(+), 47 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index b1b03b2c0f08..ce46218ce46d 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -200,7 +200,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, void **data_page, struct trace_seq; int ring_buffer_print_entry_header(struct trace_seq *s); -int ring_buffer_print_page_header(struct trace_seq *s); +int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s); enum ring_buffer_flags { RB_FL_OVERWRITE = 1 << 0, diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index d59d14e67da1..a4354287dec3 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -373,14 +373,6 @@ static inline bool test_time_stamp(u64 delta) return !!(delta & TS_DELTA_TEST); } -#define BUF_PAGE_SIZE (PAGE_SIZE - BUF_PAGE_HDR_SIZE) - -/* Max payload is BUF_PAGE_SIZE - header (8bytes) */ -#define BUF_MAX_DATA_SIZE (BUF_PAGE_SIZE - (sizeof(u32) * 2)) - -/* Events may have a timestamp attached to them */ -#define BUF_MAX_EVENT_SIZE (BUF_MAX_DATA_SIZE - RB_LEN_TIME_EXTEND) - struct rb_irq_work { struct irq_work work; wait_queue_head_t waiters; @@ -530,6 +522,9 @@ struct trace_buffer { struct rb_irq_work irq_work; bool time_stamp_abs; + + unsigned int subbuf_size; + unsigned int max_data_size; }; struct ring_buffer_iter { @@ -546,7 +541,7 @@ struct ring_buffer_iter { int missed_events; }; -int ring_buffer_print_page_header(struct trace_seq *s) +int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s) { struct buffer_data_page field; @@ -570,7 +565,7 @@ int ring_buffer_print_page_header(struct trace_seq *s) trace_seq_printf(s, "\tfield: char data;\t" "offset:%u;\tsize:%u;\tsigned:%u;\n", (unsigned int)offsetof(typeof(field), data), - (unsigned int)BUF_PAGE_SIZE, + (unsigned int)buffer->subbuf_size, (unsigned int)is_signed_type(char)); return !trace_seq_has_overflowed(s); @@ -1822,7 +1817,13 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags, if (!zalloc_cpumask_var(&buffer->cpumask, GFP_KERNEL)) goto fail_free_buffer; - nr_pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE); + /* Default buffer page size - one system page */ + buffer->subbuf_size = PAGE_SIZE - BUF_PAGE_HDR_SIZE; + + /* Max payload is buffer page size - header (8bytes) */ + buffer->max_data_size = buffer->subbuf_size - (sizeof(u32) * 2); + + nr_pages = DIV_ROUND_UP(size, buffer->subbuf_size); buffer->flags = flags; buffer->clock = trace_clock_local; buffer->reader_lock_key = key; @@ -2141,7 +2142,7 @@ static void update_pages_handler(struct work_struct *work) * @size: the new size. * @cpu_id: the cpu buffer to resize * - * Minimum size is 2 * BUF_PAGE_SIZE. + * Minimum size is 2 * buffer->subbuf_size. * * Returns 0 on success and < 0 on failure. */ @@ -2163,7 +2164,7 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, !cpumask_test_cpu(cpu_id, buffer->cpumask)) return 0; - nr_pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE); + nr_pages = DIV_ROUND_UP(size, buffer->subbuf_size); /* we need a minimum of two pages */ if (nr_pages < 2) @@ -2410,7 +2411,7 @@ rb_iter_head_event(struct ring_buffer_iter *iter) */ barrier(); - if ((iter->head + length) > commit || length > BUF_MAX_DATA_SIZE) + if ((iter->head + length) > commit || length > iter->cpu_buffer->buffer->max_data_size) /* Writer corrupted the read? */ goto reset; @@ -2643,6 +2644,7 @@ static inline void rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, unsigned long tail, struct rb_event_info *info) { + unsigned long bsize = READ_ONCE(cpu_buffer->buffer->subbuf_size); struct buffer_page *tail_page = info->tail_page; struct ring_buffer_event *event; unsigned long length = info->length; @@ -2651,13 +2653,13 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * Only the event that crossed the page boundary * must fill the old tail_page with padding. */ - if (tail >= BUF_PAGE_SIZE) { + if (tail >= bsize) { /* * If the page was filled, then we still need * to update the real_end. Reset it to zero * and the reader will ignore it. */ - if (tail == BUF_PAGE_SIZE) + if (tail == bsize) tail_page->real_end = 0; local_sub(length, &tail_page->write); @@ -2685,7 +2687,7 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * If we are less than the minimum size, we don't need to * worry about it. */ - if (tail > (BUF_PAGE_SIZE - RB_EVNT_MIN_SIZE)) { + if (tail > (bsize - RB_EVNT_MIN_SIZE)) { /* No room for any events */ /* Mark the rest of the page with padding */ @@ -2700,19 +2702,19 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, } /* Put in a discarded event */ - event->array[0] = (BUF_PAGE_SIZE - tail) - RB_EVNT_HDR_SIZE; + event->array[0] = (bsize - tail) - RB_EVNT_HDR_SIZE; event->type_len = RINGBUF_TYPE_PADDING; /* time delta must be non zero */ event->time_delta = 1; /* account for padding bytes */ - local_add(BUF_PAGE_SIZE - tail, &cpu_buffer->entries_bytes); + local_add(bsize - tail, &cpu_buffer->entries_bytes); /* Make sure the padding is visible before the tail_page->write update */ smp_wmb(); /* Set write to end of buffer */ - length = (tail + length) - BUF_PAGE_SIZE; + length = (tail + length) - bsize; local_sub(length, &tail_page->write); } @@ -3608,7 +3610,7 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer, tail = write - info->length; /* See if we shot pass the end of this buffer page */ - if (unlikely(write > BUF_PAGE_SIZE)) { + if (unlikely(write > cpu_buffer->buffer->subbuf_size)) { /* before and after may now different, fix it up*/ b_ok = rb_time_read(&cpu_buffer->before_stamp, &info->before); a_ok = rb_time_read(&cpu_buffer->write_stamp, &info->after); @@ -3817,7 +3819,7 @@ ring_buffer_lock_reserve(struct trace_buffer *buffer, unsigned long length) if (unlikely(atomic_read(&cpu_buffer->record_disabled))) goto out; - if (unlikely(length > BUF_MAX_EVENT_SIZE)) + if (unlikely(length > buffer->max_data_size - RB_LEN_TIME_EXTEND)) goto out; if (unlikely(trace_recursive_lock(cpu_buffer))) @@ -3967,7 +3969,7 @@ int ring_buffer_write(struct trace_buffer *buffer, if (atomic_read(&cpu_buffer->record_disabled)) goto out; - if (length > BUF_MAX_EVENT_SIZE) + if (length > buffer->max_data_size - RB_LEN_TIME_EXTEND) goto out; if (unlikely(trace_recursive_lock(cpu_buffer))) @@ -4547,6 +4549,7 @@ static struct buffer_page * rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) { struct buffer_page *reader = NULL; + unsigned long bsize = READ_ONCE(cpu_buffer->buffer->subbuf_size); unsigned long overwrite; unsigned long flags; int nr_loops = 0; @@ -4682,7 +4685,7 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) #define USECS_WAIT 1000000 for (nr_loops = 0; nr_loops < USECS_WAIT; nr_loops++) { /* If the write is past the end of page, a writer is still updating it */ - if (likely(!reader || rb_page_write(reader) <= BUF_PAGE_SIZE)) + if (likely(!reader || rb_page_write(reader) <= bsize)) break; udelay(1); @@ -5125,7 +5128,7 @@ ring_buffer_read_prepare(struct trace_buffer *buffer, int cpu, gfp_t flags) if (!iter) return NULL; - iter->event = kmalloc(BUF_MAX_DATA_SIZE, flags); + iter->event = kmalloc(buffer->max_data_size, flags); if (!iter->event) { kfree(iter); return NULL; @@ -5243,14 +5246,14 @@ unsigned long ring_buffer_size(struct trace_buffer *buffer, int cpu) { /* * Earlier, this method returned - * BUF_PAGE_SIZE * buffer->nr_pages + * buffer->subbuf_size * buffer->nr_pages * Since the nr_pages field is now removed, we have converted this to * return the per cpu buffer value. */ if (!cpumask_test_cpu(cpu, buffer->cpumask)) return 0; - return BUF_PAGE_SIZE * buffer->buffers[cpu]->nr_pages; + return buffer->subbuf_size * buffer->buffers[cpu]->nr_pages; } EXPORT_SYMBOL_GPL(ring_buffer_size); @@ -5262,7 +5265,7 @@ EXPORT_SYMBOL_GPL(ring_buffer_size); */ unsigned long ring_buffer_max_event_size(struct trace_buffer *buffer) { - return BUF_MAX_EVENT_SIZE; + return buffer->max_data_size - RB_LEN_TIME_EXTEND; } EXPORT_SYMBOL_GPL(ring_buffer_max_event_size); @@ -5868,7 +5871,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* If there is room at the end of the page to save the * missed events, then record it there. */ - if (BUF_PAGE_SIZE - commit >= sizeof(missed_events)) { + if (buffer->subbuf_size - commit >= sizeof(missed_events)) { memcpy(&bpage->data[commit], &missed_events, sizeof(missed_events)); local_add(RB_MISSED_STORED, &bpage->commit); @@ -5880,8 +5883,8 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* * This page may be off to user land. Zero it out here. */ - if (commit < BUF_PAGE_SIZE) - memset(&bpage->data[commit], 0, BUF_PAGE_SIZE - commit); + if (commit < buffer->subbuf_size) + memset(&bpage->data[commit], 0, buffer->subbuf_size - commit); out_unlock: raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags); diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 077b20e83e7c..90e0ea91521f 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -5012,7 +5012,7 @@ static int tracing_release(struct inode *inode, struct file *file) return 0; } -static int tracing_release_generic_tr(struct inode *inode, struct file *file) +int tracing_release_generic_tr(struct inode *inode, struct file *file) { struct trace_array *tr = inode->i_private; diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 97b01dfd7070..af07194db3d5 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -616,6 +616,7 @@ void tracing_reset_all_online_cpus(void); void tracing_reset_all_online_cpus_unlocked(void); int tracing_open_generic(struct inode *inode, struct file *filp); int tracing_open_generic_tr(struct inode *inode, struct file *filp); +int tracing_release_generic_tr(struct inode *inode, struct file *file); int tracing_open_file_tr(struct inode *inode, struct file *filp); int tracing_release_file_tr(struct inode *inode, struct file *filp); bool tracing_is_disabled(void); diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index b70d03818038..7c364b87352e 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -1893,9 +1893,9 @@ subsystem_filter_write(struct file *filp, const char __user *ubuf, size_t cnt, } static ssize_t -show_header(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +show_header_page_file(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) { - int (*func)(struct trace_seq *s) = filp->private_data; + struct trace_array *tr = filp->private_data; struct trace_seq *s; int r; @@ -1908,7 +1908,31 @@ show_header(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) trace_seq_init(s); - func(s); + ring_buffer_print_page_header(tr->array_buffer.buffer, s); + r = simple_read_from_buffer(ubuf, cnt, ppos, + s->buffer, trace_seq_used(s)); + + kfree(s); + + return r; +} + +static ssize_t +show_header_event_file(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +{ + struct trace_seq *s; + int r; + + if (*ppos) + return 0; + + s = kmalloc(sizeof(*s), GFP_KERNEL); + if (!s) + return -ENOMEM; + + trace_seq_init(s); + + ring_buffer_print_entry_header(s); r = simple_read_from_buffer(ubuf, cnt, ppos, s->buffer, trace_seq_used(s)); @@ -2165,10 +2189,18 @@ static const struct file_operations ftrace_tr_enable_fops = { .release = subsystem_release, }; -static const struct file_operations ftrace_show_header_fops = { - .open = tracing_open_generic, - .read = show_header, +static const struct file_operations ftrace_show_header_page_fops = { + .open = tracing_open_generic_tr, + .read = show_header_page_file, .llseek = default_llseek, + .release = tracing_release_generic_tr, +}; + +static const struct file_operations ftrace_show_header_event_fops = { + .open = tracing_open_generic_tr, + .read = show_header_event_file, + .llseek = default_llseek, + .release = tracing_release_generic_tr, }; static int @@ -3794,17 +3826,16 @@ static int events_callback(const char *name, umode_t *mode, void **data, return 1; } - if (strcmp(name, "header_page") == 0) - *data = ring_buffer_print_page_header; - - else if (strcmp(name, "header_event") == 0) - *data = ring_buffer_print_entry_header; + if (strcmp(name, "header_page") == 0) { + *mode = TRACE_MODE_READ; + *fops = &ftrace_show_header_page_fops; - else + } else if (strcmp(name, "header_event") == 0) { + *mode = TRACE_MODE_READ; + *fops = &ftrace_show_header_event_fops; + } else return 0; - *mode = TRACE_MODE_READ; - *fops = &ftrace_show_header_fops; return 1; } From patchwork Sun Dec 10 03:54:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176265 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353814vqy; Sat, 9 Dec 2023 20:04:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IF735f2Aw+6+ra1JP9izfDPzAiOppSEvizJiulYU0m8h07iDJ6wnpw9ySwDxtmbL7kKpNxw X-Received: by 2002:a17:902:ab14:b0:1d0:ab0f:324 with SMTP id ik20-20020a170902ab1400b001d0ab0f0324mr956556plb.139.1702181074803; Sat, 09 Dec 2023 20:04:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181074; cv=none; d=google.com; s=arc-20160816; b=cVvvcN5FcQOfZyB3v33Je0DuBlIEyJYPev84RJCsa88kwjT77LuTfdJuynuARoYMoa ksI2HjtsmLEmeBOOXWOwUKcaG6LSn7MbFMZ/P7jOE3jBednusLNEyQzze5YrI7lQFXbd xUyZjQXH70j+ZALNRDFUGdSNr83SNmstlY/Uzpw/R6NIhqnkU5A/H7D/eqRUc3d7njHy mYNpo9hEyV3svqIaqvo2QRZvSJeuzkgTRSXB6p2bg6UDMvEM+t7NK1GiCz/nNuBRukn/ BhXLM5mZfVJJ27mvF7Ml8lvVRC1mFVH5wtUbOnyMUipIcwZvMZgt98n1ClLu8wDlLeEY 6HSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=mqcBZO1wKZOG1kcyrcDzFnGofQxPoFcLuRQisbfBbNM=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=q7d6hXalBUVXwiCmVj+5T3IXexYblQjfgT02vnXz9lKYpKJvYOtLpEedV4Ye6ctnkJ FBKNt20lcY4zifIj3YXZf0G6z6Yx/Dy0tC0YX1HFOr1pRxy2E2+EvxqFIdZbgpj/DBHS QggU4uc/E+u/jRkMaCUeGenFd9MgUVKab1C8/JSL3FBIp6DzmuwU9lOoXRRuRVUHVJ/y la64Zjpq3WWfAp2nwl/Rp/yZ0xmcZSQlGvr6ezr9PrlgeRlQtO5a9inDfSiquqkxxBcN zjAcHy79WiuziwVDVjS6upvwdZTPrqk2QORF5TT1o2h8lh1eDsaX7eClgbygJxkx8BYf QAZw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id k9-20020a170902c40900b001d0bcb62f98si4246024plk.368.2023.12.09.20.04.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 333CD8087E0B; Sat, 9 Dec 2023 20:04:29 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231548AbjLJEEP (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231472AbjLJEEH (ORCPT ); Sat, 9 Dec 2023 23:04:07 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D95EE118 for ; Sat, 9 Dec 2023 20:04:12 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6ACDEC433C8; Sun, 10 Dec 2023 04:04:12 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3h-000000021Mn-1UhX; Sat, 09 Dec 2023 23:04:49 -0500 Message-ID: <20231210040449.140717619@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:07 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 03/14] ring-buffer: Add interface for configuring trace sub buffer size References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:29 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866222405819866 X-GMAIL-MSGID: 1784866222405819866 From: "Tzvetomir Stoyanov (VMware)" The trace ring buffer sub page size can be configured, per trace instance. A new ftrace file "buffer_subbuf_order" is added to get and set the size of the ring buffer sub page for current trace instance. The size must be an order of system page size, that's why the new interface works with system page order, instead of absolute page size: 0 means the ring buffer sub page is equal to 1 system page and so forth: 0 - 1 system page 1 - 2 system pages 2 - 4 system pages ... The ring buffer sub page size is limited between 1 and 128 system pages. The default value is 1 system page. New ring buffer APIs are introduced: ring_buffer_subbuf_order_set() ring_buffer_subbuf_order_get() ring_buffer_subbuf_size_get() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-4-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 4 ++ kernel/trace/ring_buffer.c | 73 +++++++++++++++++++++++++++++++++++++ kernel/trace/trace.c | 48 ++++++++++++++++++++++++ 3 files changed, 125 insertions(+) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index ce46218ce46d..12573306b889 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -202,6 +202,10 @@ struct trace_seq; int ring_buffer_print_entry_header(struct trace_seq *s); int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s); +int ring_buffer_subbuf_order_get(struct trace_buffer *buffer); +int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order); +int ring_buffer_subbuf_size_get(struct trace_buffer *buffer); + enum ring_buffer_flags { RB_FL_OVERWRITE = 1 << 0, }; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index a4354287dec3..7c925ce95c96 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -524,6 +524,7 @@ struct trace_buffer { bool time_stamp_abs; unsigned int subbuf_size; + unsigned int subbuf_order; unsigned int max_data_size; }; @@ -5894,6 +5895,78 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } EXPORT_SYMBOL_GPL(ring_buffer_read_page); +/** + * ring_buffer_subbuf_size_get - get size of the sub buffer. + * @buffer: the buffer to get the sub buffer size from + * + * Returns size of the sub buffer, in bytes. + */ +int ring_buffer_subbuf_size_get(struct trace_buffer *buffer) +{ + return buffer->subbuf_size + BUF_PAGE_HDR_SIZE; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_size_get); + +/** + * ring_buffer_subbuf_order_get - get order of system sub pages in one buffer page. + * @buffer: The ring_buffer to get the system sub page order from + * + * By default, one ring buffer sub page equals to one system page. This parameter + * is configurable, per ring buffer. The size of the ring buffer sub page can be + * extended, but must be an order of system page size. + * + * Returns the order of buffer sub page size, in system pages: + * 0 means the sub buffer size is 1 system page and so forth. + * In case of an error < 0 is returned. + */ +int ring_buffer_subbuf_order_get(struct trace_buffer *buffer) +{ + if (!buffer) + return -EINVAL; + + return buffer->subbuf_order; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); + +/** + * ring_buffer_subbuf_order_set - set the size of ring buffer sub page. + * @buffer: The ring_buffer to set the new page size. + * @order: Order of the system pages in one sub buffer page + * + * By default, one ring buffer pages equals to one system page. This API can be + * used to set new size of the ring buffer page. The size must be order of + * system page size, that's why the input parameter @order is the order of + * system pages that are allocated for one ring buffer page: + * 0 - 1 system page + * 1 - 2 system pages + * 3 - 4 system pages + * ... + * + * Returns 0 on success or < 0 in case of an error. + */ +int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) +{ + int psize; + + if (!buffer || order < 0) + return -EINVAL; + + if (buffer->subbuf_order == order) + return 0; + + psize = (1 << order) * PAGE_SIZE; + if (psize <= BUF_PAGE_HDR_SIZE) + return -EINVAL; + + buffer->subbuf_order = order; + buffer->subbuf_size = psize - BUF_PAGE_HDR_SIZE; + + /* Todo: reset the buffer with the new page size */ + + return 0; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_set); + /* * We only allocate new buffers, never free them if the CPU goes down. * If we were to free the buffer, then the user would lose any trace that was in diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 90e0ea91521f..503fd7a57177 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9343,6 +9343,51 @@ static const struct file_operations buffer_percent_fops = { .llseek = default_llseek, }; +static ssize_t +buffer_order_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +{ + struct trace_array *tr = filp->private_data; + char buf[64]; + int r; + + r = sprintf(buf, "%d\n", ring_buffer_subbuf_order_get(tr->array_buffer.buffer)); + + return simple_read_from_buffer(ubuf, cnt, ppos, buf, r); +} + +static ssize_t +buffer_order_write(struct file *filp, const char __user *ubuf, + size_t cnt, loff_t *ppos) +{ + struct trace_array *tr = filp->private_data; + unsigned long val; + int ret; + + ret = kstrtoul_from_user(ubuf, cnt, 10, &val); + if (ret) + return ret; + + /* limit between 1 and 128 system pages */ + if (val < 0 || val > 7) + return -EINVAL; + + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); + if (ret) + return ret; + + (*ppos)++; + + return cnt; +} + +static const struct file_operations buffer_order_fops = { + .open = tracing_open_generic_tr, + .read = buffer_order_read, + .write = buffer_order_write, + .release = tracing_release_generic_tr, + .llseek = default_llseek, +}; + static struct dentry *trace_instance_dir; static void @@ -9809,6 +9854,9 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) trace_create_file("buffer_percent", TRACE_MODE_WRITE, d_tracer, tr, &buffer_percent_fops); + trace_create_file("buffer_subbuf_order", TRACE_MODE_WRITE, d_tracer, + tr, &buffer_order_fops); + create_trace_options_dir(tr); #ifdef CONFIG_TRACER_MAX_TRACE From patchwork Sun Dec 10 03:54:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176264 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353789vqy; Sat, 9 Dec 2023 20:04:29 -0800 (PST) X-Google-Smtp-Source: AGHT+IGcx8B7cKSpHh1By0skgX6F3I6/hegPd933IRQQegf+C6Lp57DbW+HA2ZILiG8Db9rvdfbC X-Received: by 2002:a17:90a:8a91:b0:286:c040:e6cd with SMTP id x17-20020a17090a8a9100b00286c040e6cdmr1706002pjn.46.1702181069725; Sat, 09 Dec 2023 20:04:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181069; cv=none; d=google.com; s=arc-20160816; b=M/QuZE9ueusp50DXLZ/1XtdtYGJdSTeyxajgEk8W6X7H32Ax89krXP2qBi7IMn6iS0 X62BjbUJGu93GAOUekR5y3Y72fVV0h0Rl6U+ACZY//lmOQj0745QMBvI0xUwF+AnvG0z zDZdNvhHHBIlgCPPYjOq1yvOwjAawjcqy7ksOr2/ArUTqdZOQ6Sn1RQnUSRH3MKIYqnN l/SkPoEDWt0xZMWLuSxcCu1vcCViZR+sR4IGqAZ2kYijhxCQp+Mf16UtjvthCwtRUZBH 8yv+55hzGEVF1os7MAuG1vNtqTHjZenarLQIPrrFsCke9O1j+vGgQYAXXVWhJqoBL4ue FXCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=TcpGgp7dNhuNDzNHDQ5iFcXAPUYvjIsQNQb17MEGcIw=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=elpC9SQx5IDGSj+9K3cli1L4DKqslxzSx2vhar+2YtMwstZBcjVA4a7/WhZ5QZzt35 qYMWAyeqHpG27ZKzVf6PtsNUOBc62xSN+wl0QsXGaxpKLMdATA/Suxmdzd9mOMpTc4Le Hibw4lo0rgbOG/iJ5/eXO/5YKwdaHexlGx2dkD8i4j10uMAzes39AOm8EqeNGhgJhH8T BbsfgJnzOuk9Zrh+/Gv40BqqDtlqIBghPQ20MiZL5BdnAFz4NF2Kmrx6+Qsz5P7EkTpR 94EBapScNJdlUcN6P9lxrK5ZOcgaPPcLHi9JhZD51zYasXkS7UMvhVBO7HD5EEchi0PF z1lA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id h5-20020a17090a130500b00286b22b913fsi4130666pja.37.2023.12.09.20.04.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7C84B809819C; Sat, 9 Dec 2023 20:04:28 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231519AbjLJEEL (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231468AbjLJEEG (ORCPT ); Sat, 9 Dec 2023 23:04:06 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11DEB11C for ; Sat, 9 Dec 2023 20:04:13 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6C7DC433D9; Sun, 10 Dec 2023 04:04:12 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3h-000000021NH-2hl3; Sat, 09 Dec 2023 23:04:49 -0500 Message-ID: <20231210040449.427093958@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:08 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 04/14] ring-buffer: Set new size of the ring buffer sub page References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:28 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866217070196135 X-GMAIL-MSGID: 1784866217070196135 From: "Tzvetomir Stoyanov (VMware)" There are two approaches when changing the size of the ring buffer sub page: 1. Destroying all pages and allocating new pages with the new size. 2. Allocating new pages, copying the content of the old pages before destroying them. The first approach is easier, it is selected in the proposed implementation. Changing the ring buffer sub page size is supposed to not happen frequently. Usually, that size should be set only once, when the buffer is not in use yet and is supposed to be empty. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-5-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 80 ++++++++++++++++++++++++++++++++++---- 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 7c925ce95c96..7f18b2fd2514 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -331,6 +331,7 @@ struct buffer_page { unsigned read; /* index for next read */ local_t entries; /* entries on this page */ unsigned long real_end; /* real end of data */ + unsigned order; /* order of the page */ struct buffer_data_page *page; /* Actual data page */ }; @@ -361,7 +362,7 @@ static __always_inline unsigned int rb_page_commit(struct buffer_page *bpage) static void free_buffer_page(struct buffer_page *bpage) { - free_page((unsigned long)bpage->page); + free_pages((unsigned long)bpage->page, bpage->order); kfree(bpage); } @@ -1655,10 +1656,12 @@ static int __rb_allocate_pages(struct ring_buffer_per_cpu *cpu_buffer, list_add(&bpage->list, pages); - page = alloc_pages_node(cpu_to_node(cpu_buffer->cpu), mflags, 0); + page = alloc_pages_node(cpu_to_node(cpu_buffer->cpu), mflags, + cpu_buffer->buffer->subbuf_order); if (!page) goto free_pages; bpage->page = page_address(page); + bpage->order = cpu_buffer->buffer->subbuf_order; rb_init_page(bpage->page); if (user_thread && fatal_signal_pending(current)) @@ -1737,7 +1740,8 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, long nr_pages, int cpu) rb_check_bpage(cpu_buffer, bpage); cpu_buffer->reader_page = bpage; - page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL, 0); + + page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL, cpu_buffer->buffer->subbuf_order); if (!page) goto fail_free_reader; bpage->page = page_address(page); @@ -1819,6 +1823,7 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags, goto fail_free_buffer; /* Default buffer page size - one system page */ + buffer->subbuf_order = 0; buffer->subbuf_size = PAGE_SIZE - BUF_PAGE_HDR_SIZE; /* Max payload is buffer page size - header (8bytes) */ @@ -5636,8 +5641,8 @@ void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) if (bpage) goto out; - page = alloc_pages_node(cpu_to_node(cpu), - GFP_KERNEL | __GFP_NORETRY, 0); + page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL | __GFP_NORETRY, + cpu_buffer->buffer->subbuf_order); if (!page) return ERR_PTR(-ENOMEM); @@ -5686,7 +5691,7 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data local_irq_restore(flags); out: - free_page((unsigned long)bpage); + free_pages((unsigned long)bpage, buffer->subbuf_order); } EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); @@ -5946,7 +5951,13 @@ EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); */ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) { + struct ring_buffer_per_cpu **cpu_buffers; + int old_order, old_size; + int nr_pages; int psize; + int bsize; + int err; + int cpu; if (!buffer || order < 0) return -EINVAL; @@ -5958,12 +5969,67 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (psize <= BUF_PAGE_HDR_SIZE) return -EINVAL; + bsize = sizeof(void *) * buffer->cpus; + cpu_buffers = kzalloc(bsize, GFP_KERNEL); + if (!cpu_buffers) + return -ENOMEM; + + old_order = buffer->subbuf_order; + old_size = buffer->subbuf_size; + + /* prevent another thread from changing buffer sizes */ + mutex_lock(&buffer->mutex); + atomic_inc(&buffer->record_disabled); + + /* Make sure all commits have finished */ + synchronize_rcu(); + buffer->subbuf_order = order; buffer->subbuf_size = psize - BUF_PAGE_HDR_SIZE; - /* Todo: reset the buffer with the new page size */ + /* Make sure all new buffers are allocated, before deleting the old ones */ + for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) + continue; + + nr_pages = buffer->buffers[cpu]->nr_pages; + cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); + if (!cpu_buffers[cpu]) { + err = -ENOMEM; + goto error; + } + } + + for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) + continue; + + rb_free_cpu_buffer(buffer->buffers[cpu]); + buffer->buffers[cpu] = cpu_buffers[cpu]; + } + + atomic_dec(&buffer->record_disabled); + mutex_unlock(&buffer->mutex); + + kfree(cpu_buffers); return 0; + +error: + buffer->subbuf_order = old_order; + buffer->subbuf_size = old_size; + + atomic_dec(&buffer->record_disabled); + mutex_unlock(&buffer->mutex); + + for_each_buffer_cpu(buffer, cpu) { + if (!cpu_buffers[cpu]) + continue; + kfree(cpu_buffers[cpu]); + } + kfree(cpu_buffers); + + return err; } EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_set); From patchwork Sun Dec 10 03:54:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176272 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353932vqy; Sat, 9 Dec 2023 20:05:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IFe/76Rho/HZ+jtLZQMwQa13VxuHZ/Z/ERq3BTEGWEHgLxAeSs8capnoT0zlMf/lzrxI6mY X-Received: by 2002:a05:6870:4728:b0:1fa:e630:85cd with SMTP id b40-20020a056870472800b001fae63085cdmr2437527oaq.52.1702181103643; Sat, 09 Dec 2023 20:05:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181103; cv=none; d=google.com; s=arc-20160816; b=o/OdI9gzgPOKF4Yyc8p+QSrrFvwuQvci85zDENr0BiehYAo4cxWIiTzOh/a7+P+6AA 1VuNU6rm8A8YttwYx/PgnyV771dkCnwvzz4qhFToKN1xSPHe3zbBG/bVRCvvEpD8qJFO QJJnijWWYsV4WZ+cKxegMiy7HnMmPqwXHkAtjGqJ8Evze9aCq8n2zxSJ53FCJcOn1gTc bhhkI0zaOEZBApkRMQ8To0w2fe7ey/8LmIMSQnmQYffFd6ouwCoBOIqQ7QthTRch+PhV +J5mHYJ6vDYVDOn+pNgLnPRM0tkzzol/Hf7Ap1KvjoJp2Pc9MC2Y1mSFQGfYpmjvjP1w QweA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=ycSXRD3bk+pMGgJWXLypd4Mh1axe6SoGj/28vrXtCVo=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=eZdncfuUTTqHh+oAgLokPNNnDezyX5kL9ETZxD6YaVK1elo6NQu4bBVh5xJ1X3iJ/V nNcpNpGMNjS2elS93uQAPVHLWig96dvbDjqs/61qlEljqiydvScS7NYl7a8fzZPnC50B SshM4gt+WS3ZWPC/JxRk+j0J1gY5q+/jfTxuOvKUDRpap7odDsnMALcXL/n+CpmI2z+7 I9RB853+/qsmZX+UrUu8xDqHcdv6wKqjlxp1meh1wVxyKgGN0NVHdb5U2w/CDbWkZ8sE +0nHPtzGZ6slUuJb5Hwek/mYYDVLp14i5esa9ccghkahaIyXjrJ9Vq8nd70gcbpO9LKS 3Y0w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id 19-20020a631753000000b005c61d17e6aasi4026537pgx.122.2023.12.09.20.05.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:05:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 3C479805F0A2; Sat, 9 Dec 2023 20:05:02 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231749AbjLJEEu (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231500AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FA25123 for ; Sat, 9 Dec 2023 20:04:13 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EA9A4C4339A; Sun, 10 Dec 2023 04:04:12 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3h-000000021Nl-3tTk; Sat, 09 Dec 2023 23:04:49 -0500 Message-ID: <20231210040449.711956756@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:09 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 05/14] ring-buffer: Read and write to ring buffers with custom sub buffer size References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:05:02 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866252894224401 X-GMAIL-MSGID: 1784866252894224401 From: "Tzvetomir Stoyanov (VMware)" As the size of the ring sub buffer page can be changed dynamically, the logic that reads and writes to the buffer should be fixed to take that into account. Some internal ring buffer APIs are changed: ring_buffer_alloc_read_page() ring_buffer_free_read_page() ring_buffer_read_page() A new API is introduced: ring_buffer_read_page_data() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-6-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 11 ++-- kernel/trace/ring_buffer.c | 75 ++++++++++++++++++++-------- kernel/trace/ring_buffer_benchmark.c | 10 ++-- kernel/trace/trace.c | 34 +++++++------ 4 files changed, 89 insertions(+), 41 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index 12573306b889..fa802db216f9 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -192,10 +192,15 @@ bool ring_buffer_time_stamp_abs(struct trace_buffer *buffer); size_t ring_buffer_nr_pages(struct trace_buffer *buffer, int cpu); size_t ring_buffer_nr_dirty_pages(struct trace_buffer *buffer, int cpu); -void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu); -void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data); -int ring_buffer_read_page(struct trace_buffer *buffer, void **data_page, +struct buffer_data_read_page; +struct buffer_data_read_page * +ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu); +void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, + struct buffer_data_read_page *page); +int ring_buffer_read_page(struct trace_buffer *buffer, + struct buffer_data_read_page *data_page, size_t len, int cpu, int full); +void *ring_buffer_read_page_data(struct buffer_data_read_page *page); struct trace_seq; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 7f18b2fd2514..4cb03b3b99e6 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -317,6 +317,11 @@ struct buffer_data_page { unsigned char data[] RB_ALIGN_DATA; /* data of buffer page */ }; +struct buffer_data_read_page { + unsigned order; /* order of the page */ + struct buffer_data_page *data; /* actual data, stored in this page */ +}; + /* * Note, the buffer_page list must be first. The buffer pages * are allocated in cache lines, which means that each buffer @@ -5616,40 +5621,48 @@ EXPORT_SYMBOL_GPL(ring_buffer_swap_cpu); * Returns: * The page allocated, or ERR_PTR */ -void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) +struct buffer_data_read_page * +ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) { struct ring_buffer_per_cpu *cpu_buffer; - struct buffer_data_page *bpage = NULL; + struct buffer_data_read_page *bpage = NULL; unsigned long flags; struct page *page; if (!cpumask_test_cpu(cpu, buffer->cpumask)) return ERR_PTR(-ENODEV); + bpage = kzalloc(sizeof(*bpage), GFP_KERNEL); + if (!bpage) + return ERR_PTR(-ENOMEM); + + bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu]; local_irq_save(flags); arch_spin_lock(&cpu_buffer->lock); if (cpu_buffer->free_page) { - bpage = cpu_buffer->free_page; + bpage->data = cpu_buffer->free_page; cpu_buffer->free_page = NULL; } arch_spin_unlock(&cpu_buffer->lock); local_irq_restore(flags); - if (bpage) + if (bpage->data) goto out; page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL | __GFP_NORETRY, cpu_buffer->buffer->subbuf_order); - if (!page) + if (!page) { + kfree(bpage); return ERR_PTR(-ENOMEM); + } - bpage = page_address(page); + bpage->data = page_address(page); out: - rb_init_page(bpage); + rb_init_page(bpage->data); return bpage; } @@ -5659,14 +5672,15 @@ EXPORT_SYMBOL_GPL(ring_buffer_alloc_read_page); * ring_buffer_free_read_page - free an allocated read page * @buffer: the buffer the page was allocate for * @cpu: the cpu buffer the page came from - * @data: the page to free + * @page: the page to free * * Free a page allocated from ring_buffer_alloc_read_page. */ -void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data) +void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, + struct buffer_data_read_page *data_page) { struct ring_buffer_per_cpu *cpu_buffer; - struct buffer_data_page *bpage = data; + struct buffer_data_page *bpage = data_page->data; struct page *page = virt_to_page(bpage); unsigned long flags; @@ -5675,8 +5689,12 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data cpu_buffer = buffer->buffers[cpu]; - /* If the page is still in use someplace else, we can't reuse it */ - if (page_ref_count(page) > 1) + /* + * If the page is still in use someplace else, or order of the page + * is different from the subbuffer order of the buffer - + * we can't reuse it + */ + if (page_ref_count(page) > 1 || data_page->order != buffer->subbuf_order) goto out; local_irq_save(flags); @@ -5691,7 +5709,8 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data local_irq_restore(flags); out: - free_pages((unsigned long)bpage, buffer->subbuf_order); + free_pages((unsigned long)bpage, data_page->order); + kfree(data_page); } EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); @@ -5712,9 +5731,10 @@ EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); * rpage = ring_buffer_alloc_read_page(buffer, cpu); * if (IS_ERR(rpage)) * return PTR_ERR(rpage); - * ret = ring_buffer_read_page(buffer, &rpage, len, cpu, 0); + * ret = ring_buffer_read_page(buffer, rpage, len, cpu, 0); * if (ret >= 0) - * process_page(rpage, ret); + * process_page(ring_buffer_read_page_data(rpage), ret); + * ring_buffer_free_read_page(buffer, cpu, rpage); * * When @full is set, the function will not return true unless * the writer is off the reader page. @@ -5729,7 +5749,8 @@ EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); * <0 if no data has been transferred. */ int ring_buffer_read_page(struct trace_buffer *buffer, - void **data_page, size_t len, int cpu, int full) + struct buffer_data_read_page *data_page, + size_t len, int cpu, int full) { struct ring_buffer_per_cpu *cpu_buffer = buffer->buffers[cpu]; struct ring_buffer_event *event; @@ -5754,10 +5775,12 @@ int ring_buffer_read_page(struct trace_buffer *buffer, len -= BUF_PAGE_HDR_SIZE; - if (!data_page) + if (!data_page || !data_page->data) + goto out; + if (data_page->order != buffer->subbuf_order) goto out; - bpage = *data_page; + bpage = data_page->data; if (!bpage) goto out; @@ -5851,11 +5874,11 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* swap the pages */ rb_init_page(bpage); bpage = reader->page; - reader->page = *data_page; + reader->page = data_page->data; local_set(&reader->write, 0); local_set(&reader->entries, 0); reader->read = 0; - *data_page = bpage; + data_page->data = bpage; /* * Use the real_end for the data size, @@ -5900,6 +5923,18 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } EXPORT_SYMBOL_GPL(ring_buffer_read_page); +/** + * ring_buffer_read_page_data - get pointer to the data in the page. + * @page: the page to get the data from + * + * Returns pointer to the actual data in this page. + */ +void *ring_buffer_read_page_data(struct buffer_data_read_page *page) +{ + return page->data; +} +EXPORT_SYMBOL_GPL(ring_buffer_read_page_data); + /** * ring_buffer_subbuf_size_get - get size of the sub buffer. * @buffer: the buffer to get the sub buffer size from diff --git a/kernel/trace/ring_buffer_benchmark.c b/kernel/trace/ring_buffer_benchmark.c index aef34673d79d..008187ebd7fe 100644 --- a/kernel/trace/ring_buffer_benchmark.c +++ b/kernel/trace/ring_buffer_benchmark.c @@ -104,10 +104,11 @@ static enum event_status read_event(int cpu) static enum event_status read_page(int cpu) { + struct buffer_data_read_page *bpage; struct ring_buffer_event *event; struct rb_page *rpage; unsigned long commit; - void *bpage; + int page_size; int *entry; int ret; int inc; @@ -117,14 +118,15 @@ static enum event_status read_page(int cpu) if (IS_ERR(bpage)) return EVENT_DROPPED; - ret = ring_buffer_read_page(buffer, &bpage, PAGE_SIZE, cpu, 1); + page_size = ring_buffer_subbuf_size_get(buffer); + ret = ring_buffer_read_page(buffer, bpage, page_size, cpu, 1); if (ret >= 0) { - rpage = bpage; + rpage = ring_buffer_read_page_data(bpage); /* The commit may have missed event flags set, clear them */ commit = local_read(&rpage->commit) & 0xfffff; for (i = 0; i < commit && !test_error ; i += inc) { - if (i >= (PAGE_SIZE - offsetof(struct rb_page, data))) { + if (i >= (page_size - offsetof(struct rb_page, data))) { TEST_ERROR(); break; } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 503fd7a57177..8e9853d38c8d 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -8271,6 +8271,8 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, { struct ftrace_buffer_info *info = filp->private_data; struct trace_iterator *iter = &info->iter; + void *trace_data; + int page_size; ssize_t ret = 0; ssize_t size; @@ -8282,6 +8284,8 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, return -EBUSY; #endif + page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + if (!info->spare) { info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file); @@ -8296,13 +8300,13 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, return ret; /* Do we have previous read data to read? */ - if (info->read < PAGE_SIZE) + if (info->read < page_size) goto read; again: trace_access_lock(iter->cpu_file); ret = ring_buffer_read_page(iter->array_buffer->buffer, - &info->spare, + info->spare, count, iter->cpu_file, 0); trace_access_unlock(iter->cpu_file); @@ -8323,11 +8327,11 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, info->read = 0; read: - size = PAGE_SIZE - info->read; + size = page_size - info->read; if (size > count) size = count; - - ret = copy_to_user(ubuf, info->spare + info->read, size); + trace_data = ring_buffer_read_page_data(info->spare); + ret = copy_to_user(ubuf, trace_data + info->read, size); if (ret == size) return -EFAULT; @@ -8438,6 +8442,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, .spd_release = buffer_spd_release, }; struct buffer_ref *ref; + int page_size; int entries, i; ssize_t ret = 0; @@ -8446,13 +8451,14 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, return -EBUSY; #endif - if (*ppos & (PAGE_SIZE - 1)) + page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + if (*ppos & (page_size - 1)) return -EINVAL; - if (len & (PAGE_SIZE - 1)) { - if (len < PAGE_SIZE) + if (len & (page_size - 1)) { + if (len < page_size) return -EINVAL; - len &= PAGE_MASK; + len &= (~(page_size - 1)); } if (splice_grow_spd(pipe, &spd)) @@ -8462,7 +8468,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, trace_access_lock(iter->cpu_file); entries = ring_buffer_entries_cpu(iter->array_buffer->buffer, iter->cpu_file); - for (i = 0; i < spd.nr_pages_max && len && entries; i++, len -= PAGE_SIZE) { + for (i = 0; i < spd.nr_pages_max && len && entries; i++, len -= page_size) { struct page *page; int r; @@ -8483,7 +8489,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, } ref->cpu = iter->cpu_file; - r = ring_buffer_read_page(ref->buffer, &ref->page, + r = ring_buffer_read_page(ref->buffer, ref->page, len, iter->cpu_file, 1); if (r < 0) { ring_buffer_free_read_page(ref->buffer, ref->cpu, @@ -8492,14 +8498,14 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, break; } - page = virt_to_page(ref->page); + page = virt_to_page(ring_buffer_read_page_data(ref->page)); spd.pages[i] = page; - spd.partial[i].len = PAGE_SIZE; + spd.partial[i].len = page_size; spd.partial[i].offset = 0; spd.partial[i].private = (unsigned long)ref; spd.nr_pages++; - *ppos += PAGE_SIZE; + *ppos += page_size; entries = ring_buffer_entries_cpu(iter->array_buffer->buffer, iter->cpu_file); } From patchwork Sun Dec 10 03:54:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176270 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353897vqy; Sat, 9 Dec 2023 20:04:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IGESa33k9EpInn/FDkk0ziq91HOjSq3dXSZPxqghSs1h39mcy/QPW4yPLOYPS9BUgNjo5Jv X-Received: by 2002:a05:6871:a116:b0:1fb:75b:2fdc with SMTP id vs22-20020a056871a11600b001fb075b2fdcmr3232329oab.115.1702181098144; Sat, 09 Dec 2023 20:04:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181098; cv=none; d=google.com; s=arc-20160816; b=eooJoASd+sIzZVbYvjw95qzJWPrP/n5ySYhHBllfarkcQj5SdnjctOcjlbpVIIb3Dr Y8Ar/bkQbinsRaZd2VAk8VsaW8DP/hM9OSFwNwX6vykcOsxcmhKYcQNZa1tAE65M4DJM fnMweD55A1xVvFeCsM0n7L3K9r1BO6nC1TcHnwOB4j6toPlv01Er0m0dPmKc25LsFEf3 kZ398crg3bzLWgUy21Ixw7w+UDPcD4wlrdilzk28wM+QWBsPKIuQd4z4oTiVNBMnicIp cNnm84ORlXXnKOYVrAp3EOM0lM0I+2VgShcZ4qe8zq9KEjBxB26wsqWQ6iAZyIobssPW 4ZBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=ieu3+9d2yMJWO558m3E5V2AZFHf+3LScQkVUXq1naoo=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=BpywXzGK5iMBS2mRgVr2ynNs+ydpYhUktaL67uj0opLhNLNwJo/hc9jPZNE3JXTUQF yw/Ik//mklsqER7ZdtLAOErtDFei7BgTxGi0apTXCJjsJMqanXjcEOLJr6I7YwTiTf9I JXlt8S/H/8d97rnUZg338TBQ2Y5BRXatbzGbztrmea0+R/5CiDOqFBEy1osl3v3hwKpw 9HmMjQ5LXBRgB2H7aQilaCLuQQzvDKGYtNDLAQigWMxcBFVY9QX51mOZDr4bPYq/umsJ UGnDYwIod8LP1sMKGCMU+C97GqaIfvUhbToRb1zDZXqGVDZtePZfyahChJImqP1DbUhT 1BCw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id k1-20020a63d841000000b00584ca25959csi3981326pgj.540.2023.12.09.20.04.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 3FE4980681C9; Sat, 9 Dec 2023 20:04:54 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231565AbjLJEET (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231487AbjLJEEH (ORCPT ); Sat, 9 Dec 2023 23:04:07 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5B3EB4 for ; Sat, 9 Dec 2023 20:04:13 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4AB36C433AB; Sun, 10 Dec 2023 04:04:13 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3i-000000021OH-0soF; Sat, 09 Dec 2023 23:04:50 -0500 Message-ID: <20231210040449.997493720@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:10 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 06/14] ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:54 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866247455023582 X-GMAIL-MSGID: 1784866247455023582 From: "Steven Rostedt (Google)" On failure to allocate ring buffer pages, the pointer to the CPU buffer pages is freed, but the pages that were allocated previously were not. Make sure they are freed too. Fixes: TBD ("tracing: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 4cb03b3b99e6..4e7eb41695f5 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -6060,6 +6060,7 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) for_each_buffer_cpu(buffer, cpu) { if (!cpu_buffers[cpu]) continue; + rb_free_cpu_buffer(cpu_buffers[cpu]); kfree(cpu_buffers[cpu]); } kfree(cpu_buffers); From patchwork Sun Dec 10 03:54:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176269 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353886vqy; Sat, 9 Dec 2023 20:04:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IEy06erx5KpSW6MPiW5LTpyaS5NhqS+MxZ+eUYLluMYnTwPIjpnEnCvsJcey4eFT2Du7lZi X-Received: by 2002:a05:6870:59f:b0:1fa:1719:dce2 with SMTP id m31-20020a056870059f00b001fa1719dce2mr3115783oap.28.1702181096167; Sat, 09 Dec 2023 20:04:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181096; cv=none; d=google.com; s=arc-20160816; b=cXOl7R1KM9RAksnVq12F872rZKYkRyWL1ndZrzTxIFJw+zQuzo3/gpMncYX6X0HoTn IeWO9I5l1JOUaV97AMZTFukC4CnqB10RxxU1azYNt/vnJ5ENTWT1yPGROMVsT4M126Ns tOMRPHUNjUY2qNBjh1Mf4SHZv1+zoUXu0L4kzodpTlDgaZI6O0cX7SQagM2JNzX2imfO TZKTYJuw08Lc3frMxY5qoqwmALbiZy0gHuA7vVF5jD88THGmfra/MtNSonGYLrTG76B+ GEMSmNCouhyvJemKaO5TniWNzQSMlnvw/VMUMpeBiABf6x0uD5I7jtxznA459Ub4oUkf IxNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=1/KTL/MdExsqEVcDaWlHh9guTnY1c/7X/YMUaTA0ra8=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=ypV9LLsx2wlKI56WwODmxZvi0LIQevw7jdEu9y7hayjCWlOifdfCHu6DxA+xBaVgcF FbMmE7bkTdGyDTORsPsFNPCJ4kU14E4l958HL/Z6OBS76QXWwGirqOGJ5Ln+PMioALjn sjorau6IQ4I1U3U81g1GYv24UJGRoqRl4IXDIBpMTPbbILXeNw5LJY77dASWmeGGrq0n dH+Je6lBLLGZJjN/iGj+yEJUcCNODpeb4nADfFbaPIFWPtkDesDxedTDcdI3jsECID5M zYcAtf4mhmfSNdXtYEa0UvHGzwaMD30sscPW+i0a/y9wuPJgTigdVdqzFrQXRtnImsYc M3aA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id a3-20020a654183000000b005b9a1494637si3881390pgq.317.2023.12.09.20.04.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 592BC8067B5B; Sat, 9 Dec 2023 20:04:38 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231588AbjLJEE1 (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231490AbjLJEEH (ORCPT ); Sat, 9 Dec 2023 23:04:07 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D716110E for ; Sat, 9 Dec 2023 20:04:13 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7CCDBC433C9; Sun, 10 Dec 2023 04:04:13 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3i-000000021Om-23pp; Sat, 09 Dec 2023 23:04:50 -0500 Message-ID: <20231210040450.277096110@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:11 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 07/14] ring-buffer: Do no swap cpu buffers if order is different References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:38 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866245212924472 X-GMAIL-MSGID: 1784866245212924472 From: "Steven Rostedt (Google)" As all the subbuffer order (subbuffer sizes) must be the same throughout the ring buffer, check the order of the buffers that are doing a CPU buffer swap in ring_buffer_swap_cpu() to make sure they are the same. If the are not the same, then fail to do the swap, otherwise the ring buffer will think the CPU buffer has a specific subbuffer size when it does not. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 4e7eb41695f5..9725aab1b5eb 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5550,6 +5550,9 @@ int ring_buffer_swap_cpu(struct trace_buffer *buffer_a, if (cpu_buffer_a->nr_pages != cpu_buffer_b->nr_pages) goto out; + if (buffer_a->subbuf_order != buffer_b->subbuf_order) + goto out; + ret = -EAGAIN; if (atomic_read(&buffer_a->record_disabled)) From patchwork Sun Dec 10 03:54:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176267 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353852vqy; Sat, 9 Dec 2023 20:04:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IHoqmsSRRGLHOEohhB41oOF4l/hmjnaXNVAq0kXFYQQu2K1OAM3IqegHPynpIt9lGUa7eNu X-Received: by 2002:a05:6808:208a:b0:3b9:da28:feeb with SMTP id s10-20020a056808208a00b003b9da28feebmr2990406oiw.27.1702181086080; Sat, 09 Dec 2023 20:04:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181086; cv=none; d=google.com; s=arc-20160816; b=iNhiCq6Wa6eUBj8ZIDn6xHp5R1jZLAeiY8Lg4yqMCRdUdn33/8WIj40LD8Sceb38PZ pdt5ixYsjuv93RCUxi1o7JSuOaCAhavIrVDQ65dBbF1ceGP2RdN4hWT4WcJy1+VZJpYl yncGrifd3oZO0ZNlPaeN5Yk//I86Lj76oCdymMnDrHtecp3cBwUVa6oYGooa4Wxda8Jp zBp8Pa2zPn2xZfJDUSt2qmrHMzOJ+vURYbxsKQOxoRRrnbiQNoJiN8ivdD9sN8w5CcF6 OzznHdFEn+hKi1Ef2xbxNpokTdWzpghFhq0Ic7t4ZYnhNK+L9julT/q1g7oPCAPiF3Gh 5KeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=BqBkd0zgilgA0Vzcr+eugYEe1Uzp2a4KYC8aHYjVuCQ=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=vhk64fxxTKV+AvFpEJjjOyaagom7uJU1/GvTf03GGz6fUa1LK0pIG9I6NPxFWr60pc ULW3VV10Kd18mSMUnkbgszT+5ehc47op8e1xekaSu+NehDp8RtrP85pd1KHCUSZXhxRX mYHeWM4PJUpgKDyqJgXiCsi2ath4SuRnJIKu4JU5gx1tKQ9W1eR8pRt14yx0UFxBJcpR zLasYKpBwUG611x+J4y1fyEcU/krWRNOxkQjlKLdVYozUaanNkmdvs2mwBs3c6IQLnNd 4NhV0jDmurd6vFX4HrlG9MqPOd+akp7YORIRvWotN9nZeZxq4ote8zvTX46impGs6BCd Gb4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id y186-20020a6364c3000000b005c6818b5a24si3959702pgb.245.2023.12.09.20.04.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id D356F809848A; Sat, 9 Dec 2023 20:04:44 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231608AbjLJEEc (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231492AbjLJEEH (ORCPT ); Sat, 9 Dec 2023 23:04:07 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 229DA12E for ; Sat, 9 Dec 2023 20:04:14 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BBF91C433CB; Sun, 10 Dec 2023 04:04:13 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3i-000000021PH-3FLz; Sat, 09 Dec 2023 23:04:50 -0500 Message-ID: <20231210040450.561181051@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:12 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 08/14] ring-buffer: Make sure the spare sub buffer used for reads has same size References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866234147142747 X-GMAIL-MSGID: 1784866234147142747 From: "Steven Rostedt (Google)" Now that the ring buffer specifies the size of its sub buffers, they all need to be the same size. When doing a read, a swap is done with a spare page. Make sure they are the same size before doing the swap, otherwise the read will fail. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 8e9853d38c8d..b3b36d7f1201 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -7567,6 +7567,7 @@ struct ftrace_buffer_info { struct trace_iterator iter; void *spare; unsigned int spare_cpu; + unsigned int spare_size; unsigned int read; }; @@ -8286,6 +8287,15 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + /* Make sure the spare matches the current sub buffer size */ + if (info->spare) { + if (page_size != info->spare_size) { + ring_buffer_free_read_page(iter->array_buffer->buffer, + info->spare_cpu, info->spare); + info->spare = NULL; + } + } + if (!info->spare) { info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file); @@ -8294,6 +8304,7 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, info->spare = NULL; } else { info->spare_cpu = iter->cpu_file; + info->spare_size = page_size; } } if (!info->spare) From patchwork Sun Dec 10 03:54:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176271 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353934vqy; Sat, 9 Dec 2023 20:05:04 -0800 (PST) X-Google-Smtp-Source: AGHT+IEr8ccDF5u8JPfkW139TJ0PlRlKtYvG5rvUNdalFTVS1L3+Wa0l8h7sFzl0PZAMV9rETGqG X-Received: by 2002:a17:90a:9a6:b0:286:6cd8:ef05 with SMTP id 35-20020a17090a09a600b002866cd8ef05mr3680421pjo.29.1702181103763; Sat, 09 Dec 2023 20:05:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181103; cv=none; d=google.com; s=arc-20160816; b=UWAgIJSSEN4TOWVg8sYzIOyjPRN4NJhZtML044i08qwLkpmlfF+77Eytp49QaWUs5y BzoADkersjCiLGNZAzc4mVo2qhaQLIxxgqV/mtQNay5YYRqD69I2H1YqDK32IcijZPyA GsgNdWDH+uNsb5f9TNw4GKGlgBON6q8HCLsHsV2dICOQ0qWVtfcmgnM9/zKBNaH0hOlY UQzvvQa0oMfi/byxuoxDZecycGElLUsZX+xIrwUybpK8o03WcHQKlXTQH/pDY73ue5jV aQ+XGTBSMATvMwhqxGKRSveXHWofVHE6mtJnXCwwwC3UUODrBq3k8ykH58j/T/kk6+nn G+Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=5CogPuU2lAhkI7VrUiUj8VeHZB4MBtmCWUCt58CmRzA=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=WQkPNVi81kucp4XJasn5CqtxaekbWIGkCzxOPaF0cdLjhecg1DOzNkiPBazfLmdGjc aZUH/XhEHC6HWMDb5M+24ALBjJ3sJri+HH7F4p7W/gLYMXPqzGC4UjSoH9MNdUwe6bnY QEW/WBvXbfsjLElf1yI/Z9g0E9GEZPVcyl0qZI6xwN4Q24lIX4vXH0l7eDHTaZNNUJ1g c7ibGhFwcr/brQ55XPWK0iq7uDusCNJNsBlgou7Fi4DxxegSeHxq3ME4WS/G+EwDDQ5t UE1Q4cH/HrUNhuZRU4bsLhOuIm3iwjgzp2533OBzGtgSopMTVu0DYqqWbTgSgMxD50nZ UQhA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id f5-20020a17090ace0500b0028644ca706dsi5398190pju.171.2023.12.09.20.05.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:05:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A2D37805F0A5; Sat, 9 Dec 2023 20:05:02 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231522AbjLJEEo (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231496AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 855FC118 for ; Sat, 9 Dec 2023 20:04:14 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B427C433B7; Sun, 10 Dec 2023 04:04:14 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3j-000000021Pm-0Gbc; Sat, 09 Dec 2023 23:04:51 -0500 Message-ID: <20231210040450.846348590@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:13 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 09/14] tracing: Update snapshot order along with main buffer order References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:05:02 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866253095908953 X-GMAIL-MSGID: 1784866253095908953 From: "Steven Rostedt (Google)" When updating the order of the sub buffers for the main buffer, make sure that if the snapshot buffer exists, that it gets its order updated as well. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 45 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index b3b36d7f1201..7faaaf29b504 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -1263,10 +1263,17 @@ static void set_buffer_entries(struct array_buffer *buf, unsigned long val); int tracing_alloc_snapshot_instance(struct trace_array *tr) { + int order; int ret; if (!tr->allocated_snapshot) { + /* Make the snapshot buffer have the same order as main buffer */ + order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, order); + if (ret < 0) + return ret; + /* allocate spare buffer */ ret = resize_buffer_duplicate_size(&tr->max_buffer, &tr->array_buffer, RING_BUFFER_ALL_CPUS); @@ -1286,6 +1293,7 @@ static void free_snapshot(struct trace_array *tr) * The max_tr ring buffer has some state (e.g. ring->clock) and * we want preserve it. */ + ring_buffer_subbuf_order_set(tr->max_buffer.buffer, 0); ring_buffer_resize(tr->max_buffer.buffer, 1, RING_BUFFER_ALL_CPUS); set_buffer_entries(&tr->max_buffer, 1); tracing_reset_online_cpus(&tr->max_buffer); @@ -9378,6 +9386,7 @@ buffer_order_write(struct file *filp, const char __user *ubuf, { struct trace_array *tr = filp->private_data; unsigned long val; + int old_order; int ret; ret = kstrtoul_from_user(ubuf, cnt, 10, &val); @@ -9388,12 +9397,44 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (val < 0 || val > 7) return -EINVAL; + old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + if (old_order == val) + return 0; + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); if (ret) - return ret; + return 0; - (*ppos)++; +#ifdef CONFIG_TRACER_MAX_TRACE + if (!tr->allocated_snapshot) + goto out_max; + + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, val); + if (ret) { + /* Put back the old order */ + cnt = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, old_order); + if (WARN_ON_ONCE(cnt)) { + /* + * AARGH! We are left with different orders! + * The max buffer is our "snapshot" buffer. + * When a tracer needs a snapshot (one of the + * latency tracers), it swaps the max buffer + * with the saved snap shot. We succeeded to + * update the order of the main buffer, but failed to + * update the order of the max buffer. But when we tried + * to reset the main buffer to the original size, we + * failed there too. This is very unlikely to + * happen, but if it does, warn and kill all + * tracing. + */ + tracing_disabled = 1; + } + return ret; + } + out_max: +#endif + (*ppos)++; return cnt; } From patchwork Sun Dec 10 03:54:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176268 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353879vqy; Sat, 9 Dec 2023 20:04:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IEo4o13OSmdyf0hguMhnPTHLf3Q4xT/0JwNk04NscW2CjxfvQ04qXyS3S7/JOScQYDmvl8w X-Received: by 2002:a05:6a20:da84:b0:18f:97c:9287 with SMTP id iy4-20020a056a20da8400b0018f097c9287mr1313003pzb.108.1702181095032; Sat, 09 Dec 2023 20:04:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181095; cv=none; d=google.com; s=arc-20160816; b=VIHjAi2XEmRtwcn5FIx17c0dP6I76xrW3MG0fIug+xVF+bS+lRuD0jhSRRtINH0BvW dBQz6W31RXorYtGXjqLE/KZWY1b5Lrz6RvPn6hi6z7fxJXZjoqay6JRWeiTTJS0Jz9Qk JlI3yB9ytYreGdOuRKt6XIvuadH4kYIM1oYiEyGmQB6SVxjkqXlkQw2XySqX4H3pbVBu G9d0/Ce4EJ1V9aJij+R6Bt501OyxS/Sch+bj+EvMJfgPOeLTQ8UZCV5s4TmWClWxOWs+ 9HfwruK3b8vsXbcvZj3Pl0AIKJLPunvCl9U8/WlICiakLQo+n8fzNCnIiNTCjTMFsr21 yB9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=Eq9sBsrut4+ndQbw6zCNUZb02rHiNejqBiJJbr+Tlao=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=KuOscClw+odZy9Ln6mWmK8K+RWdwSkoANzHg4ydOT/Cu0MtGNs3yZ+rPZE4JVrH32K lIM0VrdJ/7qyjjumuJDq4kvbzVUkGMpFN4VygX38wMFQczmIoMAiK3tmZoRyrDhRaC+u XfPaQxLBZa6eUHbTgNNgL3UAwsN95UuR0C3tOoNRuJeqi/lAN6bgkk5qe/ybdgNAgLkw iqNxTGibOX4gg6cHnIEzTWvnZJYc3Y+wFxs5+2SknvJwj3rAd1X9yXWJHIbzk0U3Ix0A bdiVcLUT53z9onAyAvxF8MtHVXJrJ4q30bIjb2eidSA6qoJ//AkO7kDtbCAlZUeNZVu2 1+CA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id ij29-20020a170902ab5d00b001cfdb94ee48si4015680plb.376.2023.12.09.20.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:04:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 18999809849D; Sat, 9 Dec 2023 20:04:53 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231660AbjLJEEi (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229488AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A02A211C for ; Sat, 9 Dec 2023 20:04:14 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E44FC433D9; Sun, 10 Dec 2023 04:04:14 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3j-000000021QH-1SGg; Sat, 09 Dec 2023 23:04:51 -0500 Message-ID: <20231210040451.136611306@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:14 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 10/14] tracing: Stop the tracing while changing the ring buffer subbuf size References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:04:53 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866244176692296 X-GMAIL-MSGID: 1784866244176692296 From: "Steven Rostedt (Google)" Because the main buffer and the snapshot buffer need to be the same for some tracers, otherwise it will fail and disable all tracing, the tracers need to be stopped while updating the sub buffer sizes so that the tracers see the main and snapshot buffers with the same sub buffer size. Fixes: TBD ("ring-buffer: Add interface for configuring trace sub buffer size") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 7faaaf29b504..9e5f2392d521 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9397,13 +9397,16 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (val < 0 || val > 7) return -EINVAL; + /* Do not allow tracing while changing the order of the ring buffer */ + tracing_stop_tr(tr); + old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); if (old_order == val) - return 0; + goto out; ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); if (ret) - return 0; + goto out; #ifdef CONFIG_TRACER_MAX_TRACE @@ -9430,11 +9433,15 @@ buffer_order_write(struct file *filp, const char __user *ubuf, */ tracing_disabled = 1; } - return ret; + goto out; } out_max: #endif (*ppos)++; + out: + if (ret) + cnt = ret; + tracing_start_tr(tr); return cnt; } From patchwork Sun Dec 10 03:54:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176273 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353953vqy; Sat, 9 Dec 2023 20:05:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IHyjHAKCUJYCMwK67nzkJ1nzPGSB5PYXa3KWXSsZycrxYVrwEiIRAJGOdTFzFbmFFoq1HYY X-Received: by 2002:a05:6358:881:b0:170:6fc7:c2f2 with SMTP id m1-20020a056358088100b001706fc7c2f2mr2622339rwj.14.1702181107266; Sat, 09 Dec 2023 20:05:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181107; cv=none; d=google.com; s=arc-20160816; b=cLnjFE2itKMejFW+oVXhslcbSDFVMHveSMm1ccbSGrD/L8itugBGKHwSobRYRpNUUp JSLyJ0R7JUrYyVbVAsyUblqfpRBLh2AGmLe/P94UNux3sggOlIZdqVflfI6j6+yEWJ7W UVii4HYBHAPrVOZRgGD/KWkZsOgTTbVd4GWN8mHjSAbGbW+D8NwGELM2t/8yRsyoXfLV 29StroHCW+BhpHngpXFYr/NFWhvIMBVBcDxHof8qQxcdtL+7TOMAQlukYl2oArUjBONj JdRLJuwhw4LkudNkdK63EgGA6g+ZC6fDLR5hzw/DykqZ+G1gcWbzc1Oqg9HOQgVKAeto no9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=aLoh1AYSen0LPPbCUh5hXSD2eWDg4oW1oxI+b1cyCsI=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=N3gePS+tudXx4+O542kxdkpdimU9xSM8V/Ou+1UG3nYguD8E0ckV7Dx7z+aGAkX4Oo tsK5Ztkn1QJd1Fr9eAYbwENvqCd4HMrTbkOjt5OHqQGZhh6+TUUrbPOf4nstK+Au1dA0 6q9VO40ducmzL/JMrtmXI37CBJF4QxtoWtHUefls8T75NqvORB771QXhFoRgqSQlAf54 tBZsLFvxRYKJBKTgnplH8jYOO+ee4zdBBeYFK1lEABvaboykwO6kLgikuBdj5S3Fjebj sxN0lhydI8Du5mXu5SMV/XD3RHJDASre2Jqrr6ltVTqJgrOrFzhdIeDT6sl5xB14Y1YZ XksA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id x16-20020a634a10000000b005b96d038729si2960122pga.728.2023.12.09.20.05.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:05:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id EB69880659B7; Sat, 9 Dec 2023 20:05:04 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231487AbjLJEEz (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231503AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F70FB4 for ; Sat, 9 Dec 2023 20:04:15 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B4A1BC433C7; Sun, 10 Dec 2023 04:04:14 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3j-000000021Qm-2g3H; Sat, 09 Dec 2023 23:04:51 -0500 Message-ID: <20231210040451.424991712@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:15 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 11/14] ring-buffer: Keep the same size when updating the order References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:05:05 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866256919688391 X-GMAIL-MSGID: 1784866256919688391 From: "Steven Rostedt (Google)" The function ring_buffer_subbuf_order_set() just updated the sub-buffers to the new size, but this also changes the size of the buffer in doing so. As the size is determined by nr_pages * subbuf_size. If the subbuf_size is increased without decreasing the nr_pages, this causes the total size of the buffer to increase. This broke the latency tracers as the snapshot needs to be the same size as the main buffer. The size of the snapshot buffer is only expanded when needed, and because the order is still the same, the size becomes out of sync with the main buffer, as the main buffer increased in size without the tracing system knowing. Calculate the nr_pages to allocate with the new subbuf_size to be buffer_size / new_subbuf_size. Fixes: TBD ("ring-buffer: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 9725aab1b5eb..4726deccd997 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -6030,7 +6030,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; - nr_pages = buffer->buffers[cpu]->nr_pages; + /* Update the number of pages to match the new size */ + nr_pages = old_size * buffer->buffers[cpu]->nr_pages; + nr_pages = DIV_ROUND_UP(nr_pages, buffer->subbuf_size); + cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); if (!cpu_buffers[cpu]) { err = -ENOMEM; From patchwork Sun Dec 10 03:54:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176275 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6354013vqy; Sat, 9 Dec 2023 20:05:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IFWVtbwGlWPdh2baicJxrjJn5E606v66pT2IC7ZIbTFefQck2yeSpKocWpbIFlXm2lxct+Q X-Received: by 2002:a05:6871:452:b0:1fb:75b:2b92 with SMTP id e18-20020a056871045200b001fb075b2b92mr1243863oag.78.1702181119256; Sat, 09 Dec 2023 20:05:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181119; cv=none; d=google.com; s=arc-20160816; b=Rl5jjKZGYkMTyhS98MlFdyhaMjw7NKy42EMI/EZLD8WS2ddh38Q1n4ZNeqOGrOD2Bw cCht+6bgUPca3E1fLP0VbomjpXQ5APn9VTTbRNOW4d/NP6Ax7MKkzuXxwTCeKaIGMX6m 9qZ05h9bqnYrkbrDAhexNwGRLEILIiiSaD2W9r0sN3YY0OavSnHttCtxL53LS3CDoKCo U36bB8nMiGdIHQxs6w6Cpl9Lo98uQGEUXpGPsrkG+pTloaooSpqbfRknuYplnUBn/h5x 4gCyJ6C15b1U/f054tMmheBK/Sa7iwlvR+A7i/poWWDFbY9GCOw/42wbtDCtFc34SeaX FW2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=fx2/QeR2Cd/LAxfi+tNePBDyZME5RKjxLzw14AG4qJM=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=tydMr9R58TiDVszcolPS5aSZbaVxkYFrks63D8hac2iCviHM5+qbFIC924nPbnjp1W ZQn0LjlehHImtJSHD2c2yVTqJRkQS9jCTZETPmE56WLmY7JJ1r3Ya7voH4SMCPf9CFl2 y/fd3J73yxUydtB9dK0/VMNUdvo3Vtd5AJ0yOTHMpaD9BsI3abGFnCNNqQyOsz/QUEPX q5rbIyAhM5VlYt0NQDGlR+ajfkrfCWVUZbDkW/v8nUHBJ7+BQbs6nCfS63Hra2/y58Ml okUVfAqmXEzO/nOpgEUEVDGcb7BRShbOfSjnepAg/rJy3c0JIEmkqqK+3odJCtC+zqf7 T8aA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id j27-20020a63231b000000b0058572c8d5d5si3834265pgj.233.2023.12.09.20.05.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:05:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 7D0CF806599B; Sat, 9 Dec 2023 20:05:16 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231915AbjLJEFE (ORCPT + 99 others); Sat, 9 Dec 2023 23:05:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231509AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01D6913A for ; Sat, 9 Dec 2023 20:04:15 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DA8EAC433CD; Sun, 10 Dec 2023 04:04:14 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3j-000000021RI-3qmN; Sat, 09 Dec 2023 23:04:51 -0500 Message-ID: <20231210040451.704153084@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:16 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 12/14] ring-buffer: Just update the subbuffers when changing their allocation order References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:05:16 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866269218702671 X-GMAIL-MSGID: 1784866269218702671 From: "Steven Rostedt (Google)" The ring_buffer_subbuf_order_set() was creating ring_buffer_per_cpu cpu_buffers with the new subbuffers with the updated order, and if they all successfully were created, then they the ring_buffer's per_cpu buffers would be freed and replaced by them. The problem is that the freed per_cpu buffers contains state that would be lost. Running the following commands: 1. # echo 3 > /sys/kernel/tracing/buffer_subbuf_order 2. # echo 0 > /sys/kernel/tracing/tracing_cpumask 3. # echo 1 > /sys/kernel/tracing/snapshot 4. # echo ff > /sys/kernel/tracing/tracing_cpumask 5. # echo test > /sys/kernel/tracing/trace_marker Would result in: -bash: echo: write error: Bad file descriptor That's because the state of the per_cpu buffers of the snapshot buffer is lost when the order is changed (the order of a freed snapshot buffer goes to 0 to save memory, and when the snapshot buffer is allocated again, it goes back to what the main buffer is). In operation 2, the snapshot buffers were set to "disable" (as all the ring buffers CPUs were disabled). In operation 3, the snapshot is allocated and a call to ring_buffer_subbuf_order_set() replaced the per_cpu buffers losing the "record_disable" count. When it was enabled again, the atomic_dec(&cpu_buffer->record_disable) was decrementing a zero, setting it to -1. Writing 1 into the snapshot would swap the snapshot buffer with the main buffer, so now the main buffer is "disabled", and nothing can write to the ring buffer anymore. Instead of creating new per_cpu buffers and losing the state of the old buffers, basically do what the resize does and just allocate new subbuf pages into the new_pages link list of the per_cpu buffer and if they all succeed, then replace the old sub buffers with the new ones. This keeps the per_cpu buffer descriptor in tact and by doing so, keeps its state. Fixes: TBD ("ring-buffer: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 82 +++++++++++++++++++++++++++++++------- 1 file changed, 68 insertions(+), 14 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 4726deccd997..bbefb5838e64 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5989,7 +5989,8 @@ EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); */ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) { - struct ring_buffer_per_cpu **cpu_buffers; + struct ring_buffer_per_cpu *cpu_buffer; + struct buffer_page *bpage, *tmp; int old_order, old_size; int nr_pages; int psize; @@ -6008,9 +6009,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) return -EINVAL; bsize = sizeof(void *) * buffer->cpus; - cpu_buffers = kzalloc(bsize, GFP_KERNEL); - if (!cpu_buffers) - return -ENOMEM; old_order = buffer->subbuf_order; old_size = buffer->subbuf_size; @@ -6027,33 +6025,85 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) /* Make sure all new buffers are allocated, before deleting the old ones */ for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; + cpu_buffer = buffer->buffers[cpu]; + /* Update the number of pages to match the new size */ nr_pages = old_size * buffer->buffers[cpu]->nr_pages; nr_pages = DIV_ROUND_UP(nr_pages, buffer->subbuf_size); - cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); - if (!cpu_buffers[cpu]) { + /* we need a minimum of two pages */ + if (nr_pages < 2) + nr_pages = 2; + + cpu_buffer->nr_pages_to_update = nr_pages; + + /* Include the reader page */ + nr_pages++; + + /* Allocate the new size buffer */ + INIT_LIST_HEAD(&cpu_buffer->new_pages); + if (__rb_allocate_pages(cpu_buffer, nr_pages, + &cpu_buffer->new_pages)) { + /* not enough memory for new pages */ err = -ENOMEM; goto error; } } for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; - rb_free_cpu_buffer(buffer->buffers[cpu]); - buffer->buffers[cpu] = cpu_buffers[cpu]; + cpu_buffer = buffer->buffers[cpu]; + + /* Clear the head bit to make the link list normal to read */ + rb_head_page_deactivate(cpu_buffer); + + /* Now walk the list and free all the old sub buffers */ + list_for_each_entry_safe(bpage, tmp, cpu_buffer->pages, list) { + list_del_init(&bpage->list); + free_buffer_page(bpage); + } + /* The above loop stopped an the last page needing to be freed */ + bpage = list_entry(cpu_buffer->pages, struct buffer_page, list); + free_buffer_page(bpage); + + /* Free the current reader page */ + free_buffer_page(cpu_buffer->reader_page); + + /* One page was allocated for the reader page */ + cpu_buffer->reader_page = list_entry(cpu_buffer->new_pages.next, + struct buffer_page, list); + list_del_init(&cpu_buffer->reader_page->list); + + /* The cpu_buffer pages are a link list with no head */ + cpu_buffer->pages = cpu_buffer->new_pages.next; + cpu_buffer->new_pages.next->prev = cpu_buffer->new_pages.prev; + cpu_buffer->new_pages.prev->next = cpu_buffer->new_pages.next; + + /* Clear the new_pages list */ + INIT_LIST_HEAD(&cpu_buffer->new_pages); + + cpu_buffer->head_page + = list_entry(cpu_buffer->pages, struct buffer_page, list); + cpu_buffer->tail_page = cpu_buffer->commit_page = cpu_buffer->head_page; + + cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; + cpu_buffer->nr_pages_to_update = 0; + + rb_head_page_activate(cpu_buffer); + + rb_check_pages(cpu_buffer); } atomic_dec(&buffer->record_disabled); mutex_unlock(&buffer->mutex); - kfree(cpu_buffers); - return 0; error: @@ -6064,12 +6114,16 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) mutex_unlock(&buffer->mutex); for_each_buffer_cpu(buffer, cpu) { - if (!cpu_buffers[cpu]) + cpu_buffer = buffer->buffers[cpu]; + + if (!cpu_buffer->nr_pages_to_update) continue; - rb_free_cpu_buffer(cpu_buffers[cpu]); - kfree(cpu_buffers[cpu]); + + list_for_each_entry_safe(bpage, tmp, &cpu_buffer->new_pages, list) { + list_del_init(&bpage->list); + free_buffer_page(bpage); + } } - kfree(cpu_buffers); return err; } From patchwork Sun Dec 10 03:54:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176274 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6353990vqy; Sat, 9 Dec 2023 20:05:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IGy+dhcFWBHIQNfuQ18O/dWwqoSFOCLqm66Y5FXUW373VvE6Qtq/TBQT9COfTtr08Sqjwly X-Received: by 2002:a05:6a20:918b:b0:18f:97c:b9fa with SMTP id v11-20020a056a20918b00b0018f097cb9famr1274723pzd.84.1702181115623; Sat, 09 Dec 2023 20:05:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181115; cv=none; d=google.com; s=arc-20160816; b=qUQ7bRaievRFT0qwLKNbIlKZVXseY5oytPNLHMcMQEMD1vpma1HOhJiOWP3Q3y6mdk Z6cRPBZ/P84wOfie+qHEfqwNEIAcbQWvoUPzk9rtFxtCyixwxOUQAMkncaoKhG5pp2bK 8qGFDqHWdZ268QHK5zyvwl0UCXd4XJtwr6hixEaCUmA7hN7RBvwAjKcU9cdlr1a+4uiU sc6a8yPJwswfXjAlUqOGmWwrlTHwSpA2JQgvuWahv4DsB3oNbqIGhpI0T9ICoLLQvrao EXQo/odEV8IaFgBBPCGNHjvWkHSgsJQQFzr6UOdHywll7WYHvvwMaiSFFrHHrnjS6vT3 RoAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=tdKtp+q3Qb3XGeC7e6FROAGw+qd8YOirCoNgdW22lfU=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=xWZIxcDN6MpQl+bOawOUyKZpaQTVkGdKiFH8f3279dVJ2TimL2KJkGUT+WN0ecR71a uOV0SpRpYalLkqqOlsictYMIId+KiMc29hEEKjEMJmOwd9Kh6upbW0qgQ0is7iIGveHY 77USaJ2SS3j7Md7A9RKCamKSftD6fJhKTzfcQIGE4/bxUOyhL+H2rlSW5oT6ez3em9km BwPAqbzffoX9zPFRRuC5XVvQda2IVuu1wIRyNu0/eGeHmJJbnoCa3BzcOn5xRLzLxBod gjiJ/hfUrzyOUGibols1Kt4l/Kt/U6lVtXhgu4+RkEwmAZjF5bXPZmqDljBOckVyBQ9d FL8g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id c12-20020a170903234c00b001cfcf1faaa6si4111158plh.206.2023.12.09.20.05.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:05:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 7E51C8069346; Sat, 9 Dec 2023 20:05:11 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231709AbjLJEE7 (ORCPT + 99 others); Sat, 9 Dec 2023 23:04:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231506AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4797E115 for ; Sat, 9 Dec 2023 20:04:15 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BD60C433CB; Sun, 10 Dec 2023 04:04:15 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3k-000000021Rm-0rbT; Sat, 09 Dec 2023 23:04:52 -0500 Message-ID: <20231210040451.982725347@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:17 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 13/14] ring-buffer: Add documentation on the buffer_subbuf_order file References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:05:11 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866265286456453 X-GMAIL-MSGID: 1784866265286456453 From: "Steven Rostedt (Google)" Add to the documentation how to use the buffer_subbuf_order file to change the size and how it affects what events can be added to the ring buffer. Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 23572f6697c0..231d26ceedb0 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -203,6 +203,33 @@ of ftrace. Here is a list of some of the key files: This displays the total combined size of all the trace buffers. + buffer_subbuf_order: + + This sets or displays the sub buffer page size order. The ring buffer + is broken up into several same size "sub buffers". An event can not be + bigger than the size of the sub buffer. Normally, the sub buffer is + the size of the architecture's page (4K on x86). The sub buffer also + contains meta data at the start which also limits the size of an event. + That means when the sub buffer is a page size, no event can be larger + than the page size minus the sub buffer meta data. + + The buffer_subbuf_order allows the user to change the size of the sub + buffer. As the sub buffer is a set of pages by the power of 2, thus + the sub buffer total size is defined by the order: + + order size + ---- ---- + 0 PAGE_SIZE + 1 PAGE_SIZE * 2 + 2 PAGE_SIZE * 4 + 3 PAGE_SIZE * 8 + + Changing the order will change the sub buffer size allowing for events + to be larger than the page size. + + Note: When changing the order, tracing is stopped and any data in the + ring buffer and the snapshot buffer will be discarded. + free_buffer: If a process is performing tracing, and the ring buffer should be From patchwork Sun Dec 10 03:54:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 176276 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp6354085vqy; Sat, 9 Dec 2023 20:05:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IEMo4sZbaviyoQL6qthBNz1m97BVbmOQIhFAcsabiEUgPkDRqrIlheD5QZNj32eYpL/Cvds X-Received: by 2002:a05:6359:608c:b0:170:17eb:9c40 with SMTP id ry12-20020a056359608c00b0017017eb9c40mr2400063rwb.33.1702181138274; Sat, 09 Dec 2023 20:05:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702181138; cv=none; d=google.com; s=arc-20160816; b=AcunfpUsh7eNwho7OxYd6KQJ0LaxP1X8++yD2tIV+xi/8OVcI9XH8hw4mfBrE03Jtt P9TWbfIvuNcd52pyrmRXQoXUL2sAVhs8KrZaHqPA3kcqE9WJ3k4TxZziv7XdJ/W5LoZj ZbyRYoX4deBPjVqJt7u+b2OKY66etg8EgjNuOZZZ7i7ebIkANARPe7GGvJv40UkzMA2l XjRiJyhNDoYYFmS5377R7PX1tUNPA6a8l7TJiwdnU45i/5LzsPJ9YTD9mv1yrNVwzOjN Vt+ZOEv8E7uocpXNgf3CJymrvMYXWsN1N2ZH9tzfDHz/ZnzbKXzOr7r8IoYUxIJ4F8Km QPtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=LL+M6cFVrFxl6tvP0L/d8dotBikTYSybMvyiJ88/0uw=; fh=pL1t9p0RxD5qCgSUniSa3P4+XmpksJYPzX6TE+dndao=; b=UuM997Mg415Iz7pn2B1HRniqDmloBhf8UY8tC9uiyvoxW+JGuOqUuBdHipW1niRYnK B+21+8BbFZen4j/UqYwLPX24BQU5N5uGsg2uEDIS/YYF0e7aj6MyAYLWT2E5Lbde8N6M IbQqVA0nqBxuLKXmoimESRZ4pnmYBx6vYuAA4TaniUKzACSxd62+NCwXn+0w72q/tFSV DhF6dWHUR3VihyvAUORaXjrIvq6p5BheAXnCrgc9TbkiFiATxaWc72ZYg8WIuLSBPWLa FsEG9/dCV73/j4GSeL25rxk9/CkKeM6QH/3ptAb0g4pt5LS7jN3Ioewtw78p4XHDuA8B VXxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id c37-20020a634e25000000b005c645e1ff20si3891169pgb.73.2023.12.09.20.05.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 Dec 2023 20:05:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id E3EEB805FD55; Sat, 9 Dec 2023 20:05:25 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231978AbjLJEFH (ORCPT + 99 others); Sat, 9 Dec 2023 23:05:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231512AbjLJEEI (ORCPT ); Sat, 9 Dec 2023 23:04:08 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8900310E for ; Sat, 9 Dec 2023 20:04:15 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6C84EC433C9; Sun, 10 Dec 2023 04:04:15 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rCB3k-000000021SI-24ci; Sat, 09 Dec 2023 23:04:52 -0500 Message-ID: <20231210040452.274868572@goodmis.org> User-Agent: quilt/0.67 Date: Sat, 09 Dec 2023 22:54:18 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH 14/14] ringbuffer/selftest: Add basic selftest to test chaning subbuf order References: <20231210035404.053677508@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Sat, 09 Dec 2023 20:05:26 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784866288963597598 X-GMAIL-MSGID: 1784866288963597598 From: "Steven Rostedt (Google)" Add a self test that will write into the trace buffer with differ trace sub buffer order sizes. Signed-off-by: Steven Rostedt (Google) --- .../ftrace/test.d/00basic/ringbuffer_order.tc | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc diff --git a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc new file mode 100644 index 000000000000..c0d76dc724d3 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc @@ -0,0 +1,46 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Change the ringbuffer size +# requires: buffer_subbuf_order +# flags: instance + +get_buffer_data_size() { + sed -ne 's/^.*data.*size:\([0-9][0-9]*\).*/\1/p' events/header_page +} + +a="1234567890" + +make_str() { + cnt=$1 + s="" + while [ $cnt -gt 10 ]; do + s="${s}${a}" + cnt=$((cnt-10)) + done + while [ $cnt -gt 0 ]; do + s="${s}X" + cnt=$((cnt-1)) + done + echo -n $s +} + +test_buffer() { + + size=`get_buffer_data_size` + + str=`make_str $size` + + echo $str > trace_marker + + grep -q $a trace +} + +ORIG=`cat buffer_subbuf_order` + +for a in `seq 0 4`; do + echo 0 > buffer_subbuf_order + test_buffer +done + +echo $ORIG > buffer_subbuf_order +