From patchwork Wed Dec 13 02:19:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177740 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133525vqy; Tue, 12 Dec 2023 18:19:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IHNcC93jXjdJv/fjYmkf9q7hh1hm8SRO9X4MLalLGN4Oow6EGpYp/SGDSJbZq9+i+dQtSgB X-Received: by 2002:a17:902:eb88:b0:1d0:265:6a2c with SMTP id q8-20020a170902eb8800b001d002656a2cmr8629821plg.11.1702433975291; Tue, 12 Dec 2023 18:19:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433975; cv=none; d=google.com; s=arc-20160816; b=f18HtkKwYc/VsrsifXaS4GUyo3oeSfpWK6R5kznaGSalPp5vTTOSatKU1eLq+HRzbF bnDuODLHScsJ3+bOGfAi+6n2BS+6FJfdPBuEumoVBpg1t+8i7a/jvN77f6Qyle0Wnk0N saPh41z1RTphde9UbZOLf31njIpOhyMGeD9sS5NZE+UqG7L1tg50O9qAMgR7W6VjwkP/ KX18YSSJ/ONxRjPbjNexhgZCBs19qOihaSt9Vdfrv3nycjUGrISZi1eVKZ9Uheffxtcn Xq8RVJ5tJ9R8ChHNBJ2FxXtXnojdgGh1Y0sjOkaOx5tUgY96x4DwxuwPPMkT8YB9c1Fv qEVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=Zm0XVusRrdQrinbBWqzuajF93HZqR/i5W51sFHIm79g=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=yOY8g3o5hrSa9TRrtDOd/dyl1nBW4Eusnaseofv2GFcGlBIHZi8bn0aMDv7ErxPbWS zL1pxhJzPCEIZ34F0YjvxuAB+wmaYjV0Hyd/R2+/ID5CEIGKfn/46Ug987wubYiG2zaA cmaVQA343RB9FV5E0lE85YQh1wH9bjhFwcHcrUlIAu4wLMKruTZL54pH8bXdh//mh0+S Uud2afwkZviVDxgm8lkB3GI8VnELtu8JV8Canveo+vKP3rzCx4tPLHDhoGzELuMaCsy0 /X7Dk8DpkftoXMhRF2Js31gsYkHvhYQ7lMNyrHiqxfGYoMp+gL5UYBfEgSZgs8swKhG2 chbw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id q14-20020a170902eb8e00b001d0bdb270a7si8885200plg.259.2023.12.12.18.19.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 1E7D780408D6; Tue, 12 Dec 2023 18:19:34 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378319AbjLMCTX (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378299AbjLMCTU (ORCPT ); Tue, 12 Dec 2023 21:19:20 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C4C2CF for ; Tue, 12 Dec 2023 18:19:25 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B72F3C433CA; Wed, 13 Dec 2023 02:19:24 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr2-00000002TlF-2BED; Tue, 12 Dec 2023 21:20:08 -0500 Message-ID: <20231213022008.302634297@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:15 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 01/15] ring-buffer: Refactor ring buffer implementation References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:34 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131407857068036 X-GMAIL-MSGID: 1785131407857068036 From: "Tzvetomir Stoyanov (VMware)" In order to introduce sub-buffer size per ring buffer, some internal refactoring is needed. As ring_buffer_print_page_header() will depend on the trace_buffer structure, it is moved after the structure definition. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-2-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 60 +++++++++++++++++++------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 8f8887f025c9..eee36f90ae25 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -378,36 +378,6 @@ static inline bool test_time_stamp(u64 delta) /* Max payload is BUF_PAGE_SIZE - header (8bytes) */ #define BUF_MAX_DATA_SIZE (BUF_PAGE_SIZE - (sizeof(u32) * 2)) -int ring_buffer_print_page_header(struct trace_seq *s) -{ - struct buffer_data_page field; - - trace_seq_printf(s, "\tfield: u64 timestamp;\t" - "offset:0;\tsize:%u;\tsigned:%u;\n", - (unsigned int)sizeof(field.time_stamp), - (unsigned int)is_signed_type(u64)); - - trace_seq_printf(s, "\tfield: local_t commit;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), commit), - (unsigned int)sizeof(field.commit), - (unsigned int)is_signed_type(long)); - - trace_seq_printf(s, "\tfield: int overwrite;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), commit), - 1, - (unsigned int)is_signed_type(long)); - - trace_seq_printf(s, "\tfield: char data;\t" - "offset:%u;\tsize:%u;\tsigned:%u;\n", - (unsigned int)offsetof(typeof(field), data), - (unsigned int)BUF_PAGE_SIZE, - (unsigned int)is_signed_type(char)); - - return !trace_seq_has_overflowed(s); -} - struct rb_irq_work { struct irq_work work; wait_queue_head_t waiters; @@ -573,6 +543,36 @@ struct ring_buffer_iter { int missed_events; }; +int ring_buffer_print_page_header(struct trace_seq *s) +{ + struct buffer_data_page field; + + trace_seq_printf(s, "\tfield: u64 timestamp;\t" + "offset:0;\tsize:%u;\tsigned:%u;\n", + (unsigned int)sizeof(field.time_stamp), + (unsigned int)is_signed_type(u64)); + + trace_seq_printf(s, "\tfield: local_t commit;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), commit), + (unsigned int)sizeof(field.commit), + (unsigned int)is_signed_type(long)); + + trace_seq_printf(s, "\tfield: int overwrite;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), commit), + 1, + (unsigned int)is_signed_type(long)); + + trace_seq_printf(s, "\tfield: char data;\t" + "offset:%u;\tsize:%u;\tsigned:%u;\n", + (unsigned int)offsetof(typeof(field), data), + (unsigned int)BUF_PAGE_SIZE, + (unsigned int)is_signed_type(char)); + + return !trace_seq_has_overflowed(s); +} + #ifdef RB_TIME_32 /* From patchwork Wed Dec 13 02:19:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177753 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8134049vqy; Tue, 12 Dec 2023 18:20:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IEkRxH1QDUC0n1MyubZwB20DM+n/qVpWy9vc4qZKvHeRZTwDy8E/jSGN7xQ9RZLb8MlY6CJ X-Received: by 2002:a17:902:db0b:b0:1d0:bc5f:ce62 with SMTP id m11-20020a170902db0b00b001d0bc5fce62mr7715347plx.107.1702434050811; Tue, 12 Dec 2023 18:20:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434050; cv=none; d=google.com; s=arc-20160816; b=YQmARJcaHW41awXZE2xc7lP0cxYMpBujmQ9s2fN97HSLcygJQQVmmK0JFKH+N8fJt4 FAaqWzATBzJynrTKrDWn/svbR8nt1SFVppPoqiCWfbWbkB66xdRNShVpZnIUmk5k5c/o 80nvPf4dfk9oBsMzbGBYAalS9BXlQdU2pAYIrCZFQMTt9BTqJa4XzZX7WGSigQfwOm5r l9u7CVinSpwRFtOg+N2y0gzxwc1JlHWQxiIwFzYQrJDjN7daf0tn27SttUXm72PFM0ml +V0t8ooNclZ+FX+ZxlLQazrmTfgTHQLy1KEdxmfjUjjdxPwcPbHCNQzgu6o/sZ+cxcL6 It7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=SuCbqy6UqEX8zn/jlYu/MND8zkpPhrT5soi5KbdN92k=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=udy2dyDhp0Fmt18P0vtlTwpNvclZ8/8VOqJZUT+XZbyYvwoMXPhNKax9Ks5glLW1g5 bp4Lkfsc3HvhYrdPkcKfAthiBNzG8ClGFlPhJZRu70aL1DwNbFwxUIJnHqkCaC/9ovTS KiqRHPM3Pjs5pY8u7uDxY+Vz1gpDfvberF2qEP1TR/u6R7YNzOXNK0lMhUiGLf7hzZTs HWg1PcM9I0wY6U6q43+IwFLvUzdEkRerRa88ckW7KPR7g2dp8GVHEvhazado6aPaqRpD td+Pfg7jpC2l3JB8AZludAIKHzNYiyaXzWambxMc89YHWwizNTy6fXpclKebpu0zo2PL YpXA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id g2-20020a1709026b4200b001d2f0bf3dccsi7691348plt.273.2023.12.12.18.20.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id C2F8D8076669; Tue, 12 Dec 2023 18:20:18 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378353AbjLMCT2 (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378296AbjLMCTV (ORCPT ); Tue, 12 Dec 2023 21:19:21 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8844DC for ; Tue, 12 Dec 2023 18:19:25 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 23D1CC433CC; Wed, 13 Dec 2023 02:19:25 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr2-00000002Tlj-3MJ3; Tue, 12 Dec 2023 21:20:08 -0500 Message-ID: <20231213022008.584287938@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:16 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 02/15] ring-buffer: Page size per ring buffer References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:20:19 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131487061796308 X-GMAIL-MSGID: 1785131487061796308 From: "Tzvetomir Stoyanov (VMware)" Currently the size of one sub buffer page is global for all buffers and it is hard coded to one system page. In order to introduce configurable ring buffer sub page size, the internal logic should be refactored to work with sub page size per ring buffer. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-3-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 2 +- kernel/trace/ring_buffer.c | 68 +++++++++++++++++++++---------------- kernel/trace/trace.c | 2 +- kernel/trace/trace.h | 1 + kernel/trace/trace_events.c | 59 ++++++++++++++++++++++++-------- 5 files changed, 86 insertions(+), 46 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index b1b03b2c0f08..ce46218ce46d 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -200,7 +200,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, void **data_page, struct trace_seq; int ring_buffer_print_entry_header(struct trace_seq *s); -int ring_buffer_print_page_header(struct trace_seq *s); +int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s); enum ring_buffer_flags { RB_FL_OVERWRITE = 1 << 0, diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index eee36f90ae25..7ca97da72538 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -373,11 +373,6 @@ static inline bool test_time_stamp(u64 delta) return !!(delta & TS_DELTA_TEST); } -#define BUF_PAGE_SIZE (PAGE_SIZE - BUF_PAGE_HDR_SIZE) - -/* Max payload is BUF_PAGE_SIZE - header (8bytes) */ -#define BUF_MAX_DATA_SIZE (BUF_PAGE_SIZE - (sizeof(u32) * 2)) - struct rb_irq_work { struct irq_work work; wait_queue_head_t waiters; @@ -527,6 +522,9 @@ struct trace_buffer { struct rb_irq_work irq_work; bool time_stamp_abs; + + unsigned int subbuf_size; + unsigned int max_data_size; }; struct ring_buffer_iter { @@ -540,10 +538,11 @@ struct ring_buffer_iter { u64 read_stamp; u64 page_stamp; struct ring_buffer_event *event; + size_t event_size; int missed_events; }; -int ring_buffer_print_page_header(struct trace_seq *s) +int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s) { struct buffer_data_page field; @@ -567,7 +566,7 @@ int ring_buffer_print_page_header(struct trace_seq *s) trace_seq_printf(s, "\tfield: char data;\t" "offset:%u;\tsize:%u;\tsigned:%u;\n", (unsigned int)offsetof(typeof(field), data), - (unsigned int)BUF_PAGE_SIZE, + (unsigned int)buffer->subbuf_size, (unsigned int)is_signed_type(char)); return !trace_seq_has_overflowed(s); @@ -1823,7 +1822,13 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags, if (!zalloc_cpumask_var(&buffer->cpumask, GFP_KERNEL)) goto fail_free_buffer; - nr_pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE); + /* Default buffer page size - one system page */ + buffer->subbuf_size = PAGE_SIZE - BUF_PAGE_HDR_SIZE; + + /* Max payload is buffer page size - header (8bytes) */ + buffer->max_data_size = buffer->subbuf_size - (sizeof(u32) * 2); + + nr_pages = DIV_ROUND_UP(size, buffer->subbuf_size); buffer->flags = flags; buffer->clock = trace_clock_local; buffer->reader_lock_key = key; @@ -2142,7 +2147,7 @@ static void update_pages_handler(struct work_struct *work) * @size: the new size. * @cpu_id: the cpu buffer to resize * - * Minimum size is 2 * BUF_PAGE_SIZE. + * Minimum size is 2 * buffer->subbuf_size. * * Returns 0 on success and < 0 on failure. */ @@ -2164,7 +2169,7 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, !cpumask_test_cpu(cpu_id, buffer->cpumask)) return 0; - nr_pages = DIV_ROUND_UP(size, BUF_PAGE_SIZE); + nr_pages = DIV_ROUND_UP(size, buffer->subbuf_size); /* we need a minimum of two pages */ if (nr_pages < 2) @@ -2411,7 +2416,7 @@ rb_iter_head_event(struct ring_buffer_iter *iter) */ barrier(); - if ((iter->head + length) > commit || length > BUF_PAGE_SIZE) + if ((iter->head + length) > commit || length > iter->event_size) /* Writer corrupted the read? */ goto reset; @@ -2644,6 +2649,7 @@ static inline void rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, unsigned long tail, struct rb_event_info *info) { + unsigned long bsize = READ_ONCE(cpu_buffer->buffer->subbuf_size); struct buffer_page *tail_page = info->tail_page; struct ring_buffer_event *event; unsigned long length = info->length; @@ -2652,13 +2658,13 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * Only the event that crossed the page boundary * must fill the old tail_page with padding. */ - if (tail >= BUF_PAGE_SIZE) { + if (tail >= bsize) { /* * If the page was filled, then we still need * to update the real_end. Reset it to zero * and the reader will ignore it. */ - if (tail == BUF_PAGE_SIZE) + if (tail == bsize) tail_page->real_end = 0; local_sub(length, &tail_page->write); @@ -2686,7 +2692,7 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, * If we are less than the minimum size, we don't need to * worry about it. */ - if (tail > (BUF_PAGE_SIZE - RB_EVNT_MIN_SIZE)) { + if (tail > (bsize - RB_EVNT_MIN_SIZE)) { /* No room for any events */ /* Mark the rest of the page with padding */ @@ -2701,19 +2707,19 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, } /* Put in a discarded event */ - event->array[0] = (BUF_PAGE_SIZE - tail) - RB_EVNT_HDR_SIZE; + event->array[0] = (bsize - tail) - RB_EVNT_HDR_SIZE; event->type_len = RINGBUF_TYPE_PADDING; /* time delta must be non zero */ event->time_delta = 1; /* account for padding bytes */ - local_add(BUF_PAGE_SIZE - tail, &cpu_buffer->entries_bytes); + local_add(bsize - tail, &cpu_buffer->entries_bytes); /* Make sure the padding is visible before the tail_page->write update */ smp_wmb(); /* Set write to end of buffer */ - length = (tail + length) - BUF_PAGE_SIZE; + length = (tail + length) - bsize; local_sub(length, &tail_page->write); } @@ -3613,7 +3619,7 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer, tail = write - info->length; /* See if we shot pass the end of this buffer page */ - if (unlikely(write > BUF_PAGE_SIZE)) { + if (unlikely(write > cpu_buffer->buffer->subbuf_size)) { check_buffer(cpu_buffer, info, CHECK_FULL_PAGE); return rb_move_tail(cpu_buffer, tail, info); } @@ -3742,7 +3748,7 @@ rb_reserve_next_event(struct trace_buffer *buffer, if (ring_buffer_time_stamp_abs(cpu_buffer->buffer)) { add_ts_default = RB_ADD_STAMP_ABSOLUTE; info.length += RB_LEN_TIME_EXTEND; - if (info.length > BUF_MAX_DATA_SIZE) + if (info.length > cpu_buffer->buffer->max_data_size) goto out_fail; } else { add_ts_default = RB_ADD_STAMP_NONE; @@ -3817,7 +3823,7 @@ ring_buffer_lock_reserve(struct trace_buffer *buffer, unsigned long length) if (unlikely(atomic_read(&cpu_buffer->record_disabled))) goto out; - if (unlikely(length > BUF_MAX_DATA_SIZE)) + if (unlikely(length > buffer->max_data_size)) goto out; if (unlikely(trace_recursive_lock(cpu_buffer))) @@ -3967,7 +3973,7 @@ int ring_buffer_write(struct trace_buffer *buffer, if (atomic_read(&cpu_buffer->record_disabled)) goto out; - if (length > BUF_MAX_DATA_SIZE) + if (length > buffer->max_data_size) goto out; if (unlikely(trace_recursive_lock(cpu_buffer))) @@ -4547,6 +4553,7 @@ static struct buffer_page * rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) { struct buffer_page *reader = NULL; + unsigned long bsize = READ_ONCE(cpu_buffer->buffer->subbuf_size); unsigned long overwrite; unsigned long flags; int nr_loops = 0; @@ -4682,7 +4689,7 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) #define USECS_WAIT 1000000 for (nr_loops = 0; nr_loops < USECS_WAIT; nr_loops++) { /* If the write is past the end of page, a writer is still updating it */ - if (likely(!reader || rb_page_write(reader) <= BUF_PAGE_SIZE)) + if (likely(!reader || rb_page_write(reader) <= bsize)) break; udelay(1); @@ -5126,7 +5133,8 @@ ring_buffer_read_prepare(struct trace_buffer *buffer, int cpu, gfp_t flags) return NULL; /* Holds the entire event: data and meta data */ - iter->event = kmalloc(BUF_PAGE_SIZE, flags); + iter->event_size = buffer->subbuf_size; + iter->event = kmalloc(iter->event_size, flags); if (!iter->event) { kfree(iter); return NULL; @@ -5244,14 +5252,14 @@ unsigned long ring_buffer_size(struct trace_buffer *buffer, int cpu) { /* * Earlier, this method returned - * BUF_PAGE_SIZE * buffer->nr_pages + * buffer->subbuf_size * buffer->nr_pages * Since the nr_pages field is now removed, we have converted this to * return the per cpu buffer value. */ if (!cpumask_test_cpu(cpu, buffer->cpumask)) return 0; - return BUF_PAGE_SIZE * buffer->buffers[cpu]->nr_pages; + return buffer->subbuf_size * buffer->buffers[cpu]->nr_pages; } EXPORT_SYMBOL_GPL(ring_buffer_size); @@ -5265,8 +5273,8 @@ unsigned long ring_buffer_max_event_size(struct trace_buffer *buffer) { /* If abs timestamp is requested, events have a timestamp too */ if (ring_buffer_time_stamp_abs(buffer)) - return BUF_MAX_DATA_SIZE - RB_LEN_TIME_EXTEND; - return BUF_MAX_DATA_SIZE; + return buffer->max_data_size - RB_LEN_TIME_EXTEND; + return buffer->max_data_size; } EXPORT_SYMBOL_GPL(ring_buffer_max_event_size); @@ -5872,7 +5880,7 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* If there is room at the end of the page to save the * missed events, then record it there. */ - if (BUF_PAGE_SIZE - commit >= sizeof(missed_events)) { + if (buffer->subbuf_size - commit >= sizeof(missed_events)) { memcpy(&bpage->data[commit], &missed_events, sizeof(missed_events)); local_add(RB_MISSED_STORED, &bpage->commit); @@ -5884,8 +5892,8 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* * This page may be off to user land. Zero it out here. */ - if (commit < BUF_PAGE_SIZE) - memset(&bpage->data[commit], 0, BUF_PAGE_SIZE - commit); + if (commit < buffer->subbuf_size) + memset(&bpage->data[commit], 0, buffer->subbuf_size - commit); out_unlock: raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags); diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 69e58d09b165..7b10ade68561 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -5012,7 +5012,7 @@ static int tracing_release(struct inode *inode, struct file *file) return 0; } -static int tracing_release_generic_tr(struct inode *inode, struct file *file) +int tracing_release_generic_tr(struct inode *inode, struct file *file) { struct trace_array *tr = inode->i_private; diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index 97b01dfd7070..af07194db3d5 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h @@ -616,6 +616,7 @@ void tracing_reset_all_online_cpus(void); void tracing_reset_all_online_cpus_unlocked(void); int tracing_open_generic(struct inode *inode, struct file *filp); int tracing_open_generic_tr(struct inode *inode, struct file *filp); +int tracing_release_generic_tr(struct inode *inode, struct file *file); int tracing_open_file_tr(struct inode *inode, struct file *filp); int tracing_release_file_tr(struct inode *inode, struct file *filp); bool tracing_is_disabled(void); diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index b70d03818038..7c364b87352e 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c @@ -1893,9 +1893,9 @@ subsystem_filter_write(struct file *filp, const char __user *ubuf, size_t cnt, } static ssize_t -show_header(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +show_header_page_file(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) { - int (*func)(struct trace_seq *s) = filp->private_data; + struct trace_array *tr = filp->private_data; struct trace_seq *s; int r; @@ -1908,7 +1908,31 @@ show_header(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) trace_seq_init(s); - func(s); + ring_buffer_print_page_header(tr->array_buffer.buffer, s); + r = simple_read_from_buffer(ubuf, cnt, ppos, + s->buffer, trace_seq_used(s)); + + kfree(s); + + return r; +} + +static ssize_t +show_header_event_file(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +{ + struct trace_seq *s; + int r; + + if (*ppos) + return 0; + + s = kmalloc(sizeof(*s), GFP_KERNEL); + if (!s) + return -ENOMEM; + + trace_seq_init(s); + + ring_buffer_print_entry_header(s); r = simple_read_from_buffer(ubuf, cnt, ppos, s->buffer, trace_seq_used(s)); @@ -2165,10 +2189,18 @@ static const struct file_operations ftrace_tr_enable_fops = { .release = subsystem_release, }; -static const struct file_operations ftrace_show_header_fops = { - .open = tracing_open_generic, - .read = show_header, +static const struct file_operations ftrace_show_header_page_fops = { + .open = tracing_open_generic_tr, + .read = show_header_page_file, .llseek = default_llseek, + .release = tracing_release_generic_tr, +}; + +static const struct file_operations ftrace_show_header_event_fops = { + .open = tracing_open_generic_tr, + .read = show_header_event_file, + .llseek = default_llseek, + .release = tracing_release_generic_tr, }; static int @@ -3794,17 +3826,16 @@ static int events_callback(const char *name, umode_t *mode, void **data, return 1; } - if (strcmp(name, "header_page") == 0) - *data = ring_buffer_print_page_header; - - else if (strcmp(name, "header_event") == 0) - *data = ring_buffer_print_entry_header; + if (strcmp(name, "header_page") == 0) { + *mode = TRACE_MODE_READ; + *fops = &ftrace_show_header_page_fops; - else + } else if (strcmp(name, "header_event") == 0) { + *mode = TRACE_MODE_READ; + *fops = &ftrace_show_header_event_fops; + } else return 0; - *mode = TRACE_MODE_READ; - *fops = &ftrace_show_header_fops; return 1; } From patchwork Wed Dec 13 02:19:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177742 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133552vqy; Tue, 12 Dec 2023 18:19:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IHBsm4sRtGzHSSZz1Bf5uDGZ15txUqqP8sig98fMN3wddMwJ97roegF271LZt1qGjhz4Ely X-Received: by 2002:a05:6a00:4613:b0:6cd:dc48:1fff with SMTP id ko19-20020a056a00461300b006cddc481fffmr10715735pfb.0.1702433980084; Tue, 12 Dec 2023 18:19:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433980; cv=none; d=google.com; s=arc-20160816; b=xIpyD2P5EqBWa/cj+wt5hWiMOrpu/KidjcojaF/6JuNRTBP9HmEe+rfOMhRkhAZ0hL TdWSvudDs73ojGEm32rzToOS+bUEV7W5w0SPXTM4W7/hPaSEHtSyWaP33Dhp8wm6RCMU aPnu6GUC5YzW0kbrmyIJ+g/rPBlDOeL0qDiK2JTaD2G6t4ZKz13dtk2/zIj9UUzgqDQN +0Dfbs2wRcMC3/X1MfysXzaHsVGwLxnBjS1UKbT51PZlUWL/1MaO/jl7SbTxglr3CFSo 47BTicJChCiFfE/ZfEbyWvke10JkZntbvps1PQ5XbLeLJp5LaPxMhYNxGeECcLI02QK0 WdZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=6Sbrsr6jaDzjCanjPx1kb98t/STqLz8Wt3Dz8dOYMzs=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=XRaoSl/ggzk6H+w8+eJ9S+CfMcWxzHPJnoScEUYy2M+uI6KDJ+rbRiEPwAmUXnyIa3 lqMshjrfoA5jOPfiisFJQ14KUvST11UVEtcL/P+mx+Z7uPVjHUZdL/spMEuhSSN24PNg aNowT8V08cM8gLCIIdmYvAhhcf9H2fQ90UxNja4HKfj7vTdEsRwSBEWGCzFH5sP008iv PnV81ddtjkFA1mF14tkZhrjxsMtIap0MG1RkKcFmeT10uToWKnaAskGgA0sO3vafi2gb /P1KJP9XOjkKalrs4CUQ5r9W6QWojnb1oXU5dBEY+FxPOBlMWQH/hY8lke5xdWTyMM/3 ILOg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id u11-20020a62d44b000000b006ce2efa4422si8454843pfl.405.2023.12.12.18.19.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id CF72D80408DD; Tue, 12 Dec 2023 18:19:38 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378345AbjLMCT0 (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378303AbjLMCTU (ORCPT ); Tue, 12 Dec 2023 21:19:20 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C432AE8 for ; Tue, 12 Dec 2023 18:19:25 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53EB4C433CB; Wed, 13 Dec 2023 02:19:25 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr3-00000002TmD-0LM0; Tue, 12 Dec 2023 21:20:09 -0500 Message-ID: <20231213022008.868252922@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:17 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 03/15] ring-buffer: Add interface for configuring trace sub buffer size References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:39 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131413113298065 X-GMAIL-MSGID: 1785131413113298065 From: "Tzvetomir Stoyanov (VMware)" The trace ring buffer sub page size can be configured, per trace instance. A new ftrace file "buffer_subbuf_order" is added to get and set the size of the ring buffer sub page for current trace instance. The size must be an order of system page size, that's why the new interface works with system page order, instead of absolute page size: 0 means the ring buffer sub page is equal to 1 system page and so forth: 0 - 1 system page 1 - 2 system pages 2 - 4 system pages ... The ring buffer sub page size is limited between 1 and 128 system pages. The default value is 1 system page. New ring buffer APIs are introduced: ring_buffer_subbuf_order_set() ring_buffer_subbuf_order_get() ring_buffer_subbuf_size_get() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-4-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 4 ++ kernel/trace/ring_buffer.c | 73 +++++++++++++++++++++++++++++++++++++ kernel/trace/trace.c | 48 ++++++++++++++++++++++++ 3 files changed, 125 insertions(+) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index ce46218ce46d..12573306b889 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -202,6 +202,10 @@ struct trace_seq; int ring_buffer_print_entry_header(struct trace_seq *s); int ring_buffer_print_page_header(struct trace_buffer *buffer, struct trace_seq *s); +int ring_buffer_subbuf_order_get(struct trace_buffer *buffer); +int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order); +int ring_buffer_subbuf_size_get(struct trace_buffer *buffer); + enum ring_buffer_flags { RB_FL_OVERWRITE = 1 << 0, }; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 7ca97da72538..8ba7a45f7c21 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -524,6 +524,7 @@ struct trace_buffer { bool time_stamp_abs; unsigned int subbuf_size; + unsigned int subbuf_order; unsigned int max_data_size; }; @@ -5903,6 +5904,78 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } EXPORT_SYMBOL_GPL(ring_buffer_read_page); +/** + * ring_buffer_subbuf_size_get - get size of the sub buffer. + * @buffer: the buffer to get the sub buffer size from + * + * Returns size of the sub buffer, in bytes. + */ +int ring_buffer_subbuf_size_get(struct trace_buffer *buffer) +{ + return buffer->subbuf_size + BUF_PAGE_HDR_SIZE; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_size_get); + +/** + * ring_buffer_subbuf_order_get - get order of system sub pages in one buffer page. + * @buffer: The ring_buffer to get the system sub page order from + * + * By default, one ring buffer sub page equals to one system page. This parameter + * is configurable, per ring buffer. The size of the ring buffer sub page can be + * extended, but must be an order of system page size. + * + * Returns the order of buffer sub page size, in system pages: + * 0 means the sub buffer size is 1 system page and so forth. + * In case of an error < 0 is returned. + */ +int ring_buffer_subbuf_order_get(struct trace_buffer *buffer) +{ + if (!buffer) + return -EINVAL; + + return buffer->subbuf_order; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); + +/** + * ring_buffer_subbuf_order_set - set the size of ring buffer sub page. + * @buffer: The ring_buffer to set the new page size. + * @order: Order of the system pages in one sub buffer page + * + * By default, one ring buffer pages equals to one system page. This API can be + * used to set new size of the ring buffer page. The size must be order of + * system page size, that's why the input parameter @order is the order of + * system pages that are allocated for one ring buffer page: + * 0 - 1 system page + * 1 - 2 system pages + * 3 - 4 system pages + * ... + * + * Returns 0 on success or < 0 in case of an error. + */ +int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) +{ + int psize; + + if (!buffer || order < 0) + return -EINVAL; + + if (buffer->subbuf_order == order) + return 0; + + psize = (1 << order) * PAGE_SIZE; + if (psize <= BUF_PAGE_HDR_SIZE) + return -EINVAL; + + buffer->subbuf_order = order; + buffer->subbuf_size = psize - BUF_PAGE_HDR_SIZE; + + /* Todo: reset the buffer with the new page size */ + + return 0; +} +EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_set); + /* * We only allocate new buffers, never free them if the CPU goes down. * If we were to free the buffer, then the user would lose any trace that was in diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 7b10ade68561..4ac9da316f68 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9351,6 +9351,51 @@ static const struct file_operations buffer_percent_fops = { .llseek = default_llseek, }; +static ssize_t +buffer_order_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +{ + struct trace_array *tr = filp->private_data; + char buf[64]; + int r; + + r = sprintf(buf, "%d\n", ring_buffer_subbuf_order_get(tr->array_buffer.buffer)); + + return simple_read_from_buffer(ubuf, cnt, ppos, buf, r); +} + +static ssize_t +buffer_order_write(struct file *filp, const char __user *ubuf, + size_t cnt, loff_t *ppos) +{ + struct trace_array *tr = filp->private_data; + unsigned long val; + int ret; + + ret = kstrtoul_from_user(ubuf, cnt, 10, &val); + if (ret) + return ret; + + /* limit between 1 and 128 system pages */ + if (val < 0 || val > 7) + return -EINVAL; + + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); + if (ret) + return ret; + + (*ppos)++; + + return cnt; +} + +static const struct file_operations buffer_order_fops = { + .open = tracing_open_generic_tr, + .read = buffer_order_read, + .write = buffer_order_write, + .release = tracing_release_generic_tr, + .llseek = default_llseek, +}; + static struct dentry *trace_instance_dir; static void @@ -9817,6 +9862,9 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) trace_create_file("buffer_percent", TRACE_MODE_WRITE, d_tracer, tr, &buffer_percent_fops); + trace_create_file("buffer_subbuf_order", TRACE_MODE_WRITE, d_tracer, + tr, &buffer_order_fops); + create_trace_options_dir(tr); #ifdef CONFIG_TRACER_MAX_TRACE From patchwork Wed Dec 13 02:19:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177741 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133543vqy; Tue, 12 Dec 2023 18:19:39 -0800 (PST) X-Google-Smtp-Source: AGHT+IE5nDGM+AQoYWQJa72M2Aqi0fMPQxU3zm5AWGZVqb7110Wrf227NMzrTiGNZWV7iryGAnwC X-Received: by 2002:a05:6e02:1be5:b0:35d:59a2:a32c with SMTP id y5-20020a056e021be500b0035d59a2a32cmr7450625ilv.46.1702433978923; Tue, 12 Dec 2023 18:19:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433978; cv=none; d=google.com; s=arc-20160816; b=NyIRK/Nj8u6TXEAvyfTsCmKRb96p7xm/a8PEc2apXw7vzmjExR2uB6xYDVaIdmlm3t DJanq3l06tuQA+9vJKHKiX7yZ4wSO//KSVlU/e8emvlEHrRHCIbKRuRf1f5CCO3tkOTZ DrAw3VahHZ6lS+waITTHnyrRMjV01cxRnzVZBSFgVFPTmFoub4hpJ6YcYw4cKk8y+ZA5 nn8pIH/UyHNHoTFgjU0P1Jgx3euObiMkX3gVmtMEOhVLWIuPowWoV8vDpr23SCLpXb86 ii4hZPlhYrREpimNPKst+wquH0MjJcKQazjRZeh0bN6dbi4pTokHX9/eJjV54ivxSZA6 z0VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=EVtwl0+VwwpdB16kVt2Z1hF7xysua6AV6MtjyYFO618=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=QxE9MShuuvld9xY0Dmbe/M9i2MGF2X4g63Uvy7jVStxAWua+3sOP47MPSS1N6qdhiy HWpzW6OD3d3MMPeaYpAs/lUcO5yXSLkz+/rXjW58giMZw0YksRWspo/51IBNUaJiiygR wwVzfK3Whk1gPZ+1sV4QcqkqaeLrJL8ffkIBRUF/IiF9xXziF3fjytNozX3stQJTU+C9 IDrz4NQKfoOzdP8UfQOML7xbmCpQyHr7wuK4MozNcBs7AzPLftWAZftKlVkcDrAOU66m Di1qNk5KU4jc49l0y1p/MyWTjMacH7EJZiIeddiznwTHBLHETzpJKSZe/vdkNPeuHkEJ 8gig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id p3-20020a1709028a8300b001cfbd3f391bsi8431747plo.226.2023.12.12.18.19.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 7881280560F8; Tue, 12 Dec 2023 18:19:35 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378338AbjLMCTZ (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378302AbjLMCTU (ORCPT ); Tue, 12 Dec 2023 21:19:20 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3774EA for ; Tue, 12 Dec 2023 18:19:25 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95C56C433C9; Wed, 13 Dec 2023 02:19:25 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr3-00000002Tmn-1a3V; Tue, 12 Dec 2023 21:20:09 -0500 Message-ID: <20231213022009.153898232@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:18 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 04/15] ring-buffer: Set new size of the ring buffer sub page References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:35 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131412309487333 X-GMAIL-MSGID: 1785131412309487333 From: "Tzvetomir Stoyanov (VMware)" There are two approaches when changing the size of the ring buffer sub page: 1. Destroying all pages and allocating new pages with the new size. 2. Allocating new pages, copying the content of the old pages before destroying them. The first approach is easier, it is selected in the proposed implementation. Changing the ring buffer sub page size is supposed to not happen frequently. Usually, that size should be set only once, when the buffer is not in use yet and is supposed to be empty. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-5-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 80 ++++++++++++++++++++++++++++++++++---- 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 8ba7a45f7c21..6c780d7204ba 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -331,6 +331,7 @@ struct buffer_page { unsigned read; /* index for next read */ local_t entries; /* entries on this page */ unsigned long real_end; /* real end of data */ + unsigned order; /* order of the page */ struct buffer_data_page *page; /* Actual data page */ }; @@ -361,7 +362,7 @@ static __always_inline unsigned int rb_page_commit(struct buffer_page *bpage) static void free_buffer_page(struct buffer_page *bpage) { - free_page((unsigned long)bpage->page); + free_pages((unsigned long)bpage->page, bpage->order); kfree(bpage); } @@ -1658,10 +1659,12 @@ static int __rb_allocate_pages(struct ring_buffer_per_cpu *cpu_buffer, list_add(&bpage->list, pages); - page = alloc_pages_node(cpu_to_node(cpu_buffer->cpu), mflags, 0); + page = alloc_pages_node(cpu_to_node(cpu_buffer->cpu), mflags, + cpu_buffer->buffer->subbuf_order); if (!page) goto free_pages; bpage->page = page_address(page); + bpage->order = cpu_buffer->buffer->subbuf_order; rb_init_page(bpage->page); if (user_thread && fatal_signal_pending(current)) @@ -1740,7 +1743,8 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, long nr_pages, int cpu) rb_check_bpage(cpu_buffer, bpage); cpu_buffer->reader_page = bpage; - page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL, 0); + + page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL, cpu_buffer->buffer->subbuf_order); if (!page) goto fail_free_reader; bpage->page = page_address(page); @@ -1824,6 +1828,7 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags, goto fail_free_buffer; /* Default buffer page size - one system page */ + buffer->subbuf_order = 0; buffer->subbuf_size = PAGE_SIZE - BUF_PAGE_HDR_SIZE; /* Max payload is buffer page size - header (8bytes) */ @@ -5645,8 +5650,8 @@ void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) if (bpage) goto out; - page = alloc_pages_node(cpu_to_node(cpu), - GFP_KERNEL | __GFP_NORETRY, 0); + page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL | __GFP_NORETRY, + cpu_buffer->buffer->subbuf_order); if (!page) return ERR_PTR(-ENOMEM); @@ -5695,7 +5700,7 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data local_irq_restore(flags); out: - free_page((unsigned long)bpage); + free_pages((unsigned long)bpage, buffer->subbuf_order); } EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); @@ -5955,7 +5960,13 @@ EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); */ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) { + struct ring_buffer_per_cpu **cpu_buffers; + int old_order, old_size; + int nr_pages; int psize; + int bsize; + int err; + int cpu; if (!buffer || order < 0) return -EINVAL; @@ -5967,12 +5978,67 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (psize <= BUF_PAGE_HDR_SIZE) return -EINVAL; + bsize = sizeof(void *) * buffer->cpus; + cpu_buffers = kzalloc(bsize, GFP_KERNEL); + if (!cpu_buffers) + return -ENOMEM; + + old_order = buffer->subbuf_order; + old_size = buffer->subbuf_size; + + /* prevent another thread from changing buffer sizes */ + mutex_lock(&buffer->mutex); + atomic_inc(&buffer->record_disabled); + + /* Make sure all commits have finished */ + synchronize_rcu(); + buffer->subbuf_order = order; buffer->subbuf_size = psize - BUF_PAGE_HDR_SIZE; - /* Todo: reset the buffer with the new page size */ + /* Make sure all new buffers are allocated, before deleting the old ones */ + for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) + continue; + + nr_pages = buffer->buffers[cpu]->nr_pages; + cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); + if (!cpu_buffers[cpu]) { + err = -ENOMEM; + goto error; + } + } + + for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) + continue; + + rb_free_cpu_buffer(buffer->buffers[cpu]); + buffer->buffers[cpu] = cpu_buffers[cpu]; + } + + atomic_dec(&buffer->record_disabled); + mutex_unlock(&buffer->mutex); + + kfree(cpu_buffers); return 0; + +error: + buffer->subbuf_order = old_order; + buffer->subbuf_size = old_size; + + atomic_dec(&buffer->record_disabled); + mutex_unlock(&buffer->mutex); + + for_each_buffer_cpu(buffer, cpu) { + if (!cpu_buffers[cpu]) + continue; + kfree(cpu_buffers[cpu]); + } + kfree(cpu_buffers); + + return err; } EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_set); From patchwork Wed Dec 13 02:19:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177747 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133673vqy; Tue, 12 Dec 2023 18:19:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IHyoPknny9kksMY4OY/2etO3AyEU15srD5QpYM4UA1aWq54heAz0wlxEk0ZWPxH6/IGaS3W X-Received: by 2002:a05:6808:1a1c:b0:3b8:b4c6:ce0d with SMTP id bk28-20020a0568081a1c00b003b8b4c6ce0dmr7500304oib.81.1702433996634; Tue, 12 Dec 2023 18:19:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433996; cv=none; d=google.com; s=arc-20160816; b=HbYwNWKpS7K2fPBuWGnpIFDQNqox8QtUUI8R8famrfoAAhop/bxdjIzeSGDhDqzvPr P1DXou8/AbABwaUyH4IV3498ZCz0Tl4fkS2+MSoQ6k0DJJsDrWG/3IRT0W3ijgs68xXs oHhW2XBfYFHExvA/20OayK0tm7APBhmzzU4w9pFdaf9BJC/GZvmJw4awKnNCqMDlo6hs eTlsQ6aVzVqyl0LcUR94ObceEWp7jCqYXbp2WjybBnBmZAg/6oB12IRzNAnILZD2IyRr 278gkkLMYiMTFtRqWav3NzWCVObHRCRs2jDvYwKKDKrkeN8QEwE7EJMFTJbUuDipXrGC aMxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=mVPGlb39DxPNSu265Fcn3s2E5Mz6SN9PI19gcdLBKF0=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=apnGLAA78YpRnPSzZgYIb7pkQTPQv1yXb+XO2USsa0YF1g3I9tL65Ftix4bX1QITz8 bkdhm2y8xCm3dU/e/aq+EKfcORygWv8z6MbhE3GseIoRgqG6VGFzuH+8WIyMc0JwDDhs YUpIEiJ9S3cPza0zGmFVMQP82h0i+lXq6c/4fJPlrSs604NnsFFtS09F7/pkuIsOYSbi ax0zaaPpItN+Ukz/Ln7gHBCsqcr9tEm4Ha6GqyZbg3HxZBZAERkQrHZyvKoMC9PoLvBz ePfJIWXZnqBya2fIF021rkjqj1ayZ95ryZdXiNQUYFDsXznfy/fQFzsm8n5b2iJMNfDU s99A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id n123-20020a632781000000b005b91192c628si8998522pgn.369.2023.12.12.18.19.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id E475C8057990; Tue, 12 Dec 2023 18:19:51 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378383AbjLMCTf (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378314AbjLMCTW (ORCPT ); Tue, 12 Dec 2023 21:19:22 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C93D2100 for ; Tue, 12 Dec 2023 18:19:27 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9FD8C433C8; Wed, 13 Dec 2023 02:19:25 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr3-00000002TnI-2m7g; Tue, 12 Dec 2023 21:20:09 -0500 Message-ID: <20231213022009.441698612@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:19 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 05/15] ring-buffer: Read and write to ring buffers with custom sub buffer size References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:52 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131430422385563 X-GMAIL-MSGID: 1785131430422385563 From: "Tzvetomir Stoyanov (VMware)" As the size of the ring sub buffer page can be changed dynamically, the logic that reads and writes to the buffer should be fixed to take that into account. Some internal ring buffer APIs are changed: ring_buffer_alloc_read_page() ring_buffer_free_read_page() ring_buffer_read_page() A new API is introduced: ring_buffer_read_page_data() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-6-tz.stoyanov@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 11 ++-- kernel/trace/ring_buffer.c | 75 ++++++++++++++++++++-------- kernel/trace/ring_buffer_benchmark.c | 10 ++-- kernel/trace/trace.c | 34 +++++++------ 4 files changed, 89 insertions(+), 41 deletions(-) diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h index 12573306b889..fa802db216f9 100644 --- a/include/linux/ring_buffer.h +++ b/include/linux/ring_buffer.h @@ -192,10 +192,15 @@ bool ring_buffer_time_stamp_abs(struct trace_buffer *buffer); size_t ring_buffer_nr_pages(struct trace_buffer *buffer, int cpu); size_t ring_buffer_nr_dirty_pages(struct trace_buffer *buffer, int cpu); -void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu); -void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data); -int ring_buffer_read_page(struct trace_buffer *buffer, void **data_page, +struct buffer_data_read_page; +struct buffer_data_read_page * +ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu); +void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, + struct buffer_data_read_page *page); +int ring_buffer_read_page(struct trace_buffer *buffer, + struct buffer_data_read_page *data_page, size_t len, int cpu, int full); +void *ring_buffer_read_page_data(struct buffer_data_read_page *page); struct trace_seq; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 6c780d7204ba..729fd4f07c1e 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -317,6 +317,11 @@ struct buffer_data_page { unsigned char data[] RB_ALIGN_DATA; /* data of buffer page */ }; +struct buffer_data_read_page { + unsigned order; /* order of the page */ + struct buffer_data_page *data; /* actual data, stored in this page */ +}; + /* * Note, the buffer_page list must be first. The buffer pages * are allocated in cache lines, which means that each buffer @@ -5625,40 +5630,48 @@ EXPORT_SYMBOL_GPL(ring_buffer_swap_cpu); * Returns: * The page allocated, or ERR_PTR */ -void *ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) +struct buffer_data_read_page * +ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) { struct ring_buffer_per_cpu *cpu_buffer; - struct buffer_data_page *bpage = NULL; + struct buffer_data_read_page *bpage = NULL; unsigned long flags; struct page *page; if (!cpumask_test_cpu(cpu, buffer->cpumask)) return ERR_PTR(-ENODEV); + bpage = kzalloc(sizeof(*bpage), GFP_KERNEL); + if (!bpage) + return ERR_PTR(-ENOMEM); + + bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu]; local_irq_save(flags); arch_spin_lock(&cpu_buffer->lock); if (cpu_buffer->free_page) { - bpage = cpu_buffer->free_page; + bpage->data = cpu_buffer->free_page; cpu_buffer->free_page = NULL; } arch_spin_unlock(&cpu_buffer->lock); local_irq_restore(flags); - if (bpage) + if (bpage->data) goto out; page = alloc_pages_node(cpu_to_node(cpu), GFP_KERNEL | __GFP_NORETRY, cpu_buffer->buffer->subbuf_order); - if (!page) + if (!page) { + kfree(bpage); return ERR_PTR(-ENOMEM); + } - bpage = page_address(page); + bpage->data = page_address(page); out: - rb_init_page(bpage); + rb_init_page(bpage->data); return bpage; } @@ -5668,14 +5681,15 @@ EXPORT_SYMBOL_GPL(ring_buffer_alloc_read_page); * ring_buffer_free_read_page - free an allocated read page * @buffer: the buffer the page was allocate for * @cpu: the cpu buffer the page came from - * @data: the page to free + * @page: the page to free * * Free a page allocated from ring_buffer_alloc_read_page. */ -void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data) +void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, + struct buffer_data_read_page *data_page) { struct ring_buffer_per_cpu *cpu_buffer; - struct buffer_data_page *bpage = data; + struct buffer_data_page *bpage = data_page->data; struct page *page = virt_to_page(bpage); unsigned long flags; @@ -5684,8 +5698,12 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data cpu_buffer = buffer->buffers[cpu]; - /* If the page is still in use someplace else, we can't reuse it */ - if (page_ref_count(page) > 1) + /* + * If the page is still in use someplace else, or order of the page + * is different from the subbuffer order of the buffer - + * we can't reuse it + */ + if (page_ref_count(page) > 1 || data_page->order != buffer->subbuf_order) goto out; local_irq_save(flags); @@ -5700,7 +5718,8 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, void *data local_irq_restore(flags); out: - free_pages((unsigned long)bpage, buffer->subbuf_order); + free_pages((unsigned long)bpage, data_page->order); + kfree(data_page); } EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); @@ -5721,9 +5740,10 @@ EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); * rpage = ring_buffer_alloc_read_page(buffer, cpu); * if (IS_ERR(rpage)) * return PTR_ERR(rpage); - * ret = ring_buffer_read_page(buffer, &rpage, len, cpu, 0); + * ret = ring_buffer_read_page(buffer, rpage, len, cpu, 0); * if (ret >= 0) - * process_page(rpage, ret); + * process_page(ring_buffer_read_page_data(rpage), ret); + * ring_buffer_free_read_page(buffer, cpu, rpage); * * When @full is set, the function will not return true unless * the writer is off the reader page. @@ -5738,7 +5758,8 @@ EXPORT_SYMBOL_GPL(ring_buffer_free_read_page); * <0 if no data has been transferred. */ int ring_buffer_read_page(struct trace_buffer *buffer, - void **data_page, size_t len, int cpu, int full) + struct buffer_data_read_page *data_page, + size_t len, int cpu, int full) { struct ring_buffer_per_cpu *cpu_buffer = buffer->buffers[cpu]; struct ring_buffer_event *event; @@ -5763,10 +5784,12 @@ int ring_buffer_read_page(struct trace_buffer *buffer, len -= BUF_PAGE_HDR_SIZE; - if (!data_page) + if (!data_page || !data_page->data) + goto out; + if (data_page->order != buffer->subbuf_order) goto out; - bpage = *data_page; + bpage = data_page->data; if (!bpage) goto out; @@ -5860,11 +5883,11 @@ int ring_buffer_read_page(struct trace_buffer *buffer, /* swap the pages */ rb_init_page(bpage); bpage = reader->page; - reader->page = *data_page; + reader->page = data_page->data; local_set(&reader->write, 0); local_set(&reader->entries, 0); reader->read = 0; - *data_page = bpage; + data_page->data = bpage; /* * Use the real_end for the data size, @@ -5909,6 +5932,18 @@ int ring_buffer_read_page(struct trace_buffer *buffer, } EXPORT_SYMBOL_GPL(ring_buffer_read_page); +/** + * ring_buffer_read_page_data - get pointer to the data in the page. + * @page: the page to get the data from + * + * Returns pointer to the actual data in this page. + */ +void *ring_buffer_read_page_data(struct buffer_data_read_page *page) +{ + return page->data; +} +EXPORT_SYMBOL_GPL(ring_buffer_read_page_data); + /** * ring_buffer_subbuf_size_get - get size of the sub buffer. * @buffer: the buffer to get the sub buffer size from diff --git a/kernel/trace/ring_buffer_benchmark.c b/kernel/trace/ring_buffer_benchmark.c index aef34673d79d..008187ebd7fe 100644 --- a/kernel/trace/ring_buffer_benchmark.c +++ b/kernel/trace/ring_buffer_benchmark.c @@ -104,10 +104,11 @@ static enum event_status read_event(int cpu) static enum event_status read_page(int cpu) { + struct buffer_data_read_page *bpage; struct ring_buffer_event *event; struct rb_page *rpage; unsigned long commit; - void *bpage; + int page_size; int *entry; int ret; int inc; @@ -117,14 +118,15 @@ static enum event_status read_page(int cpu) if (IS_ERR(bpage)) return EVENT_DROPPED; - ret = ring_buffer_read_page(buffer, &bpage, PAGE_SIZE, cpu, 1); + page_size = ring_buffer_subbuf_size_get(buffer); + ret = ring_buffer_read_page(buffer, bpage, page_size, cpu, 1); if (ret >= 0) { - rpage = bpage; + rpage = ring_buffer_read_page_data(bpage); /* The commit may have missed event flags set, clear them */ commit = local_read(&rpage->commit) & 0xfffff; for (i = 0; i < commit && !test_error ; i += inc) { - if (i >= (PAGE_SIZE - offsetof(struct rb_page, data))) { + if (i >= (page_size - offsetof(struct rb_page, data))) { TEST_ERROR(); break; } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 4ac9da316f68..bb43605aacf6 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -8279,6 +8279,8 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, { struct ftrace_buffer_info *info = filp->private_data; struct trace_iterator *iter = &info->iter; + void *trace_data; + int page_size; ssize_t ret = 0; ssize_t size; @@ -8290,6 +8292,8 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, return -EBUSY; #endif + page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + if (!info->spare) { info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file); @@ -8304,13 +8308,13 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, return ret; /* Do we have previous read data to read? */ - if (info->read < PAGE_SIZE) + if (info->read < page_size) goto read; again: trace_access_lock(iter->cpu_file); ret = ring_buffer_read_page(iter->array_buffer->buffer, - &info->spare, + info->spare, count, iter->cpu_file, 0); trace_access_unlock(iter->cpu_file); @@ -8331,11 +8335,11 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, info->read = 0; read: - size = PAGE_SIZE - info->read; + size = page_size - info->read; if (size > count) size = count; - - ret = copy_to_user(ubuf, info->spare + info->read, size); + trace_data = ring_buffer_read_page_data(info->spare); + ret = copy_to_user(ubuf, trace_data + info->read, size); if (ret == size) return -EFAULT; @@ -8446,6 +8450,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, .spd_release = buffer_spd_release, }; struct buffer_ref *ref; + int page_size; int entries, i; ssize_t ret = 0; @@ -8454,13 +8459,14 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, return -EBUSY; #endif - if (*ppos & (PAGE_SIZE - 1)) + page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + if (*ppos & (page_size - 1)) return -EINVAL; - if (len & (PAGE_SIZE - 1)) { - if (len < PAGE_SIZE) + if (len & (page_size - 1)) { + if (len < page_size) return -EINVAL; - len &= PAGE_MASK; + len &= (~(page_size - 1)); } if (splice_grow_spd(pipe, &spd)) @@ -8470,7 +8476,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, trace_access_lock(iter->cpu_file); entries = ring_buffer_entries_cpu(iter->array_buffer->buffer, iter->cpu_file); - for (i = 0; i < spd.nr_pages_max && len && entries; i++, len -= PAGE_SIZE) { + for (i = 0; i < spd.nr_pages_max && len && entries; i++, len -= page_size) { struct page *page; int r; @@ -8491,7 +8497,7 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, } ref->cpu = iter->cpu_file; - r = ring_buffer_read_page(ref->buffer, &ref->page, + r = ring_buffer_read_page(ref->buffer, ref->page, len, iter->cpu_file, 1); if (r < 0) { ring_buffer_free_read_page(ref->buffer, ref->cpu, @@ -8500,14 +8506,14 @@ tracing_buffers_splice_read(struct file *file, loff_t *ppos, break; } - page = virt_to_page(ref->page); + page = virt_to_page(ring_buffer_read_page_data(ref->page)); spd.pages[i] = page; - spd.partial[i].len = PAGE_SIZE; + spd.partial[i].len = page_size; spd.partial[i].offset = 0; spd.partial[i].private = (unsigned long)ref; spd.nr_pages++; - *ppos += PAGE_SIZE; + *ppos += page_size; entries = ring_buffer_entries_cpu(iter->array_buffer->buffer, iter->cpu_file); } From patchwork Wed Dec 13 02:19:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177744 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133600vqy; Tue, 12 Dec 2023 18:19:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IHPwDo7V2J70lbwags5alNcS/W2c0KZ5pb8igev2Cbd22W/lLKfAqWNK1H3bc5iYJPQgH+X X-Received: by 2002:a05:690c:2c13:b0:5e2:3ef8:d91c with SMTP id eo19-20020a05690c2c1300b005e23ef8d91cmr1140229ywb.13.1702433986947; Tue, 12 Dec 2023 18:19:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433986; cv=none; d=google.com; s=arc-20160816; b=ncY2Vr45vkYG91VrS2za3ntkK4Z7QqnRWYzK1CRvlcKA1y9tCw0RgukEn5GR1AITyL hKf9hF3LrH0Uh9SmcR+qgKluIhhS33CK6l0CEMdV5rBHJ9obr6WTwqtauaqSoVTcs8XB nep1fiH+LKyzE0GtuGi8zX9j3DNCWkaDCJ8k2H5Tj8eKY+GssEXt3rNG/Sbua2/IXG4f J0uhfZAgrw6UP71AJuzJ9WGBmoQ244e+lNgBwQyHh/fVoMZhyP4tA7GqS/N738yf8Qvs TshNhXNSO19Yqee1juzzqQUiLreGB6tZvSB19wHB6cf3QZVEAkKsXl8alQaTeTybPiJp giKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=oD1U99Rnv3qrpuGvCu9nS4gHJid3Suzmm3jAUPP20HY=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=l2WSqqDYs92anHJzR6m7Y6IKOjF9R7NS2UiXaXVuz4lfBLiu9aYD6DzBJKt3GKPLj+ YRwgRwugZ7B/e3Yy21hTXY+q5gMRwelK7kPDxqP9rhIKdOjeoOIGccHU/8zTYtPHhCrW O+vTPPLqhhZWBdM2SrYG2NSFtpzgMWpI8z8bMSw+yEU7XfssF5LlsNn0bYga9FstZAl8 lKl6EA5XKEGT12o7nWsh7PSZ1fl0dNeSX1it0mGcScD0uUT9Z/5GasUn9dVzmEjTM+m0 1deyDlrNHobons070c7gJCFTXinW1xYDGWQXfF8suP2jRz19OBJ+WQTRwI8aqvkU/IhZ tTCw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id w13-20020a170902e88d00b001d348627d4csi819240plg.239.2023.12.12.18.19.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id F212880408E5; Tue, 12 Dec 2023 18:19:44 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378374AbjLMCTc (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378297AbjLMCTW (ORCPT ); Tue, 12 Dec 2023 21:19:22 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C914EFD for ; Tue, 12 Dec 2023 18:19:27 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F7F4C43397; Wed, 13 Dec 2023 02:19:26 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr3-00000002Tnn-3w4M; Tue, 12 Dec 2023 21:20:09 -0500 Message-ID: <20231213022009.725077654@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:20 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 06/15] ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131420307392940 X-GMAIL-MSGID: 1785131420307392940 From: "Steven Rostedt (Google)" On failure to allocate ring buffer pages, the pointer to the CPU buffer pages is freed, but the pages that were allocated previously were not. Make sure they are freed too. Fixes: TBD ("tracing: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 729fd4f07c1e..ea3c217e4e43 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -6069,6 +6069,7 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) for_each_buffer_cpu(buffer, cpu) { if (!cpu_buffers[cpu]) continue; + rb_free_cpu_buffer(cpu_buffers[cpu]); kfree(cpu_buffers[cpu]); } kfree(cpu_buffers); From patchwork Wed Dec 13 02:19:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177743 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133586vqy; Tue, 12 Dec 2023 18:19:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IEOk513NBC5vGo5j+ybG4mutYDfb5SmJeV7Gf35yA3aOGy/iqAaxBbpsuBqYqMIRSDFXXde X-Received: by 2002:a05:6808:114e:b0:3b8:b063:ade5 with SMTP id u14-20020a056808114e00b003b8b063ade5mr9941298oiu.66.1702433984809; Tue, 12 Dec 2023 18:19:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433984; cv=none; d=google.com; s=arc-20160816; b=qCzM/aaI4yE+eTjWAh0KF28Ap4RcAMkk1ZRiPzjTXPLKImWB0ppqNSFHYPspbkQ6OC 4vZG7GP+y1FG10CYcqG/nPhCYrJM4UaUux1Yvz5Rto/jsuCzDy+mAQxsKIDR+IAz5AqZ CmxGCQSYOvhZZs7LGsia2YLts1EffZD6+HpRafIS9N6M+6I89wxRmic/bPD3cZ2K2nh9 G8hlejv0CZkO9Mcn92C6Q7rI/hsP4BqLUKxvezfVX5RdX7J/OEa4/59IC1PPrGaV4Sgl Qr7O/VKHcr8GFPkB4boL68Q+CYrinrWlSeP25K7PB2Vy7ywGlHZAIEXNSt4hLLSB/PeN W8Xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=yvr2LzUtSlz+BM+2LFJ5ob5HMyEs2GYNx64qf4q/pj0=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=hT43D6htsM4Yn72Rsf8ija84h3+SGNDh48iGZHjmpAmld/uKHE8uVL2zVSBDNy/MqP U/neZOXKXTNPyeAyWaVXadvvvo7Q5DPPj1POFof096ew3IZxO0hDQgqr+PU1Ul3I1nKo qUtNeKX8u9cGSlv0y08CVz0fVtSMNtbBKiMpB6XqBBuhb5Y5cHbUNfun9WOIAtorD2rH uJJ93uThPhF3lus+jFSRwuKVg+/VMeQfSyo7F8UVQU6yNnrOYlLbRi3S2oqhhDLWmZjP EwK8W1ODUx29wX5I0OC/IjYXZa9kk3DRI4q87348EPqmRgMSDrCl6uJoSpHlp5EQkd/B fIRg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id n4-20020a17090ade8400b00286e36b62efsi8679963pjv.74.2023.12.12.18.19.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 21BC480E994B; Tue, 12 Dec 2023 18:19:42 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378364AbjLMCTb (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378305AbjLMCTV (ORCPT ); Tue, 12 Dec 2023 21:19:21 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C917FFE for ; Tue, 12 Dec 2023 18:19:27 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5ED36C433D9; Wed, 13 Dec 2023 02:19:26 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr4-00000002ToH-0vr6; Tue, 12 Dec 2023 21:20:10 -0500 Message-ID: <20231213022010.001371013@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:21 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 07/15] ring-buffer: Do no swap cpu buffers if order is different References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:42 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131417954910559 X-GMAIL-MSGID: 1785131417954910559 From: "Steven Rostedt (Google)" As all the subbuffer order (subbuffer sizes) must be the same throughout the ring buffer, check the order of the buffers that are doing a CPU buffer swap in ring_buffer_swap_cpu() to make sure they are the same. If the are not the same, then fail to do the swap, otherwise the ring buffer will think the CPU buffer has a specific subbuffer size when it does not. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index ea3c217e4e43..2c908b4f3f68 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5559,6 +5559,9 @@ int ring_buffer_swap_cpu(struct trace_buffer *buffer_a, if (cpu_buffer_a->nr_pages != cpu_buffer_b->nr_pages) goto out; + if (buffer_a->subbuf_order != buffer_b->subbuf_order) + goto out; + ret = -EAGAIN; if (atomic_read(&buffer_a->record_disabled)) From patchwork Wed Dec 13 02:19:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177745 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133623vqy; Tue, 12 Dec 2023 18:19:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IHWznpubcrkAIZMTfBHGGNUYeLX3r+RhtDzY0V6H6iv58Bgch+iooceK9AMooPKql0N2nVb X-Received: by 2002:a05:6808:1526:b0:3b9:ef59:42f8 with SMTP id u38-20020a056808152600b003b9ef5942f8mr9533877oiw.21.1702433990194; Tue, 12 Dec 2023 18:19:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433990; cv=none; d=google.com; s=arc-20160816; b=Z7WUFt1qA8GB1+7Tkadi7ssk9Pd/317rDnSHmNq+4K9QoU4d1GgnHtp99IvmExIqH3 A0Ql7D63+teDoTF9pROXDt8pBiVDCkB8gfgLbXFHcm+J6KaYqy76SVV2+EaQt7z1hAvg f4MBotW3CHWnE32NL5Qp1WI9KdjundNAlrpw60yS3Oige1RZOKylS9bQ/rx7/yAssOEY K1A6JOY/enIxeSRMsp/UFS8Nq8+pAO1ND0D6GNsYNXyBMMY5w38awEQeC7NqivaVBVE0 /yB01B8Ud28QT4lSeZX2iWF09TOk8Y5dv8sbtwZluFAHet+1rKik/tLX9V/7QG9fmMBu ERrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=U0WZfufljikYzDBLMq+8YQjwi+MojDmBWUJfgUcpqBc=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=WDABoAGskKkHFWtc4hTsp1r+aQ31YwNjhMfexOYHWuffMHpZ7VGeHi6y9TZNrUI9s1 guC/YcupSiceQzQ6BuOL02z4Mj2rrQCzQ7hbuaIU7GIpzRZtECgxRxAv290VxcDH6pXX pK59z2Ui7qQ4FSt73V6F/KOvhtrqMJULVRdoKw8wcnXUhYz++APlJysVhwGo0IF05VOb Yn5ZfIVuWRl/5ThLPZIcJ3gDENdqNNkOlqnqSzRCXpw2YLAOH/LKHtmCC4I3TeqCmcSg JQF2lAKbpxCzRUrFeAIRI80rc23D/0l4ZGYryEm7Pzxd4ALsQM7xmmi21mt2BqBgcioO uF4g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id k25-20020a6568d9000000b005c65f10c216si8618939pgt.287.2023.12.12.18.19.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 4418080A9E01; Tue, 12 Dec 2023 18:19:45 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378272AbjLMCT3 (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378300AbjLMCTV (ORCPT ); Tue, 12 Dec 2023 21:19:21 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C803EF7 for ; Tue, 12 Dec 2023 18:19:27 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4D60C433CD; Wed, 13 Dec 2023 02:19:26 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr4-00000002Ton-26uq; Tue, 12 Dec 2023 21:20:10 -0500 Message-ID: <20231213022010.289709857@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:22 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 08/15] ring-buffer: Make sure the spare sub buffer used for reads has same size References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131423751700883 X-GMAIL-MSGID: 1785131423751700883 From: "Steven Rostedt (Google)" Now that the ring buffer specifies the size of its sub buffers, they all need to be the same size. When doing a read, a swap is done with a spare page. Make sure they are the same size before doing the swap, otherwise the read will fail. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index bb43605aacf6..9fc5caecf2f6 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -7575,6 +7575,7 @@ struct ftrace_buffer_info { struct trace_iterator iter; void *spare; unsigned int spare_cpu; + unsigned int spare_size; unsigned int read; }; @@ -8294,6 +8295,15 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, page_size = ring_buffer_subbuf_size_get(iter->array_buffer->buffer); + /* Make sure the spare matches the current sub buffer size */ + if (info->spare) { + if (page_size != info->spare_size) { + ring_buffer_free_read_page(iter->array_buffer->buffer, + info->spare_cpu, info->spare); + info->spare = NULL; + } + } + if (!info->spare) { info->spare = ring_buffer_alloc_read_page(iter->array_buffer->buffer, iter->cpu_file); @@ -8302,6 +8312,7 @@ tracing_buffers_read(struct file *filp, char __user *ubuf, info->spare = NULL; } else { info->spare_cpu = iter->cpu_file; + info->spare_size = page_size; } } if (!info->spare) From patchwork Wed Dec 13 02:19:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177749 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133728vqy; Tue, 12 Dec 2023 18:20:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdjrt8KP9+0oUJfqH2anLAhbuF+KwDI0NJSmMtqrlKn+w1Ox5GGqH9Pasrj401qvENvdg9 X-Received: by 2002:a05:6870:3926:b0:203:230e:6969 with SMTP id b38-20020a056870392600b00203230e6969mr270601oap.25.1702434002523; Tue, 12 Dec 2023 18:20:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434002; cv=none; d=google.com; s=arc-20160816; b=V/PWNGplXnwwvd1VXzKvPrMRHCKtGcm0HcBO6DY40OsxBGjk+lcONh5HMHo2rB9BCd EoXn9BWcx4R7TBhKmt8kvBKtzIl6TjRJqMSQxSmo/KR2NYmZwmivoS1TE3EUMnOOU1V4 MIwXJyjMHxsEUaNt/TwuPL8RThITX0n+/j647l9Uc6RjC6EzoS9FqUhD26hZUHMi7I1k cQRca/LdNfZTYJAUl+tf97xs8r7549MQYILmF5Bscz5aXSKdDi0NWYQTX0PN+R5H+RlY iWh/qEO7Mh2co+5eWZNoYF2v7OeOhI1vZHs5sQeAZoLzcCpAK1LhVXrR78ViFKViazcO J2ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=99HiCPbjPqPLgPgYnGt0YUYzthKLwJ6+MreU/q2rfcw=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=bIH3tiIhzgDZ//MxJKjhxWklIlzLYOudmYtY4ZTledL9MelLf8LtYyA27D6XWAlxxi BtHAP6cdMljM0idr+hzW1IoYBDNCIahMzGcVQyVrQOEudfTEEN5gbH+8/0UR3K+mdNXR hpfuzdQsmCa2hb0AnvxN/z4pqaIf5x+6E1BCBgZVmFg2pnJAANo9FwXvGLrA0CUGJ4iC ZM3nBol0y+TTaZzc4Zo1deaN1K+j3b/dpsv54UUw54Y8M+L1lLxGr8oXfuFvxT4zhT2o 3fbGOnewKbzEqd460uhmKDS2mCrr8x/jdjX9BLfrAg25+/gqVswZA80kzoCc3GjRWanH l6Bg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id 80-20020a630253000000b005b8ee1c0c68si8674322pgc.605.2023.12.12.18.20.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 3B2E980E9957; Tue, 12 Dec 2023 18:19:58 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378410AbjLMCTn (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378315AbjLMCTW (ORCPT ); Tue, 12 Dec 2023 21:19:22 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55FDFBE for ; Tue, 12 Dec 2023 18:19:28 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 00C95C43391; Wed, 13 Dec 2023 02:19:27 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr4-00000002TpI-3HCb; Tue, 12 Dec 2023 21:20:10 -0500 Message-ID: <20231213022010.576288314@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:23 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 09/15] tracing: Update snapshot order along with main buffer order References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:58 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131436713130639 X-GMAIL-MSGID: 1785131436713130639 From: "Steven Rostedt (Google)" When updating the order of the sub buffers for the main buffer, make sure that if the snapshot buffer exists, that it gets its order updated as well. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 45 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 9fc5caecf2f6..7c5ef3f33d81 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -1263,10 +1263,17 @@ static void set_buffer_entries(struct array_buffer *buf, unsigned long val); int tracing_alloc_snapshot_instance(struct trace_array *tr) { + int order; int ret; if (!tr->allocated_snapshot) { + /* Make the snapshot buffer have the same order as main buffer */ + order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, order); + if (ret < 0) + return ret; + /* allocate spare buffer */ ret = resize_buffer_duplicate_size(&tr->max_buffer, &tr->array_buffer, RING_BUFFER_ALL_CPUS); @@ -1286,6 +1293,7 @@ static void free_snapshot(struct trace_array *tr) * The max_tr ring buffer has some state (e.g. ring->clock) and * we want preserve it. */ + ring_buffer_subbuf_order_set(tr->max_buffer.buffer, 0); ring_buffer_resize(tr->max_buffer.buffer, 1, RING_BUFFER_ALL_CPUS); set_buffer_entries(&tr->max_buffer, 1); tracing_reset_online_cpus(&tr->max_buffer); @@ -9386,6 +9394,7 @@ buffer_order_write(struct file *filp, const char __user *ubuf, { struct trace_array *tr = filp->private_data; unsigned long val; + int old_order; int ret; ret = kstrtoul_from_user(ubuf, cnt, 10, &val); @@ -9396,12 +9405,44 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (val < 0 || val > 7) return -EINVAL; + old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + if (old_order == val) + return 0; + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); if (ret) - return ret; + return 0; - (*ppos)++; +#ifdef CONFIG_TRACER_MAX_TRACE + + if (!tr->allocated_snapshot) + goto out_max; + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, val); + if (ret) { + /* Put back the old order */ + cnt = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, old_order); + if (WARN_ON_ONCE(cnt)) { + /* + * AARGH! We are left with different orders! + * The max buffer is our "snapshot" buffer. + * When a tracer needs a snapshot (one of the + * latency tracers), it swaps the max buffer + * with the saved snap shot. We succeeded to + * update the order of the main buffer, but failed to + * update the order of the max buffer. But when we tried + * to reset the main buffer to the original size, we + * failed there too. This is very unlikely to + * happen, but if it does, warn and kill all + * tracing. + */ + tracing_disabled = 1; + } + return ret; + } + out_max: +#endif + (*ppos)++; return cnt; } From patchwork Wed Dec 13 02:19:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177746 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133625vqy; Tue, 12 Dec 2023 18:19:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IGELf/ZwQfusUVFD5Jzw5XT9DrkiYpUD5l3a1FabO5wMRZ9CH0W8VbkP0wNdy5T7jFcvAxc X-Received: by 2002:a05:6e02:1c2f:b0:35e:6aed:e523 with SMTP id m15-20020a056e021c2f00b0035e6aede523mr10574580ilh.13.1702433990399; Tue, 12 Dec 2023 18:19:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702433990; cv=none; d=google.com; s=arc-20160816; b=0mwkosOrKpV+ImUs2kfb658XKxOUBL0Lj1ZKqrWz/0MB1+hJopxHo6ixp9xtjddEpE bRv0h53C/vzwS4z1tY+32SGAafS/JIiFjTtMo0t1L0xV9GRL847emMIajiaPQFXda3ID nYfMRM684tcjo0NohSZcQSQVjSJsXcAgSzpyzFEiLKnFfXQD4cV58l2le65zuNurN1vi p55XhK8nvtx7Xq0V4hIhmVw2+Lad1HOAIkw18waBhblZr5OGudkxBEXTSAHQ2wF9lwq6 PIQEiYYpcd5qU8Yggchspvad+2YtQLTGrlJXzs/EvG4QVIKLGXLLN2wJ3L+g8JhhZ1/U gCLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=2RKrxOddHZAx4at3BnWyT0vTv5IcOCQvvdYGGr7nGMc=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=N6AzntnidRXyRAkqalQMNy1g6QFt0GXfXfGiOh/bDWOp8eanJhu2tPaHDYS2nR710e 7Vt+BEDkulCRfMkQ9z+JkzqqU/p1HfZcWzRIQZ8WkmUETffpJSkpOqTm9/hpPRwneCdl vc+auLbKpY+fuZcFmkjbfotMz6mNDihW4E4aPw41lsROVbNiXNOowKFigysNx+l/4bZx kt2sWQQz3krAlUEEbhwdjQB4cTdQift58zq+cxLtR3tSuL9OGZ6NyqQJN2ZwWxbIs8WI P7UcHl63PpWuaEj65vTehwnhj66CxDBL7D4f5yLi4C6wVHm/KTPOGFxRnYNjLRXDfzxc o65A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id bb8-20020a170902bc8800b001cfd66f31ccsi8804234plb.55.2023.12.12.18.19.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:19:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 19FF180B8E56; Tue, 12 Dec 2023 18:19:49 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378395AbjLMCTj (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378308AbjLMCTW (ORCPT ); Tue, 12 Dec 2023 21:19:22 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 550C0BC for ; Tue, 12 Dec 2023 18:19:28 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46190C433A9; Wed, 13 Dec 2023 02:19:27 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr5-00000002Tpn-0JKL; Tue, 12 Dec 2023 21:20:11 -0500 Message-ID: <20231213022010.851142884@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:24 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 10/15] tracing: Stop the tracing while changing the ring buffer subbuf size References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:49 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131424025925454 X-GMAIL-MSGID: 1785131424025925454 From: "Steven Rostedt (Google)" Because the main buffer and the snapshot buffer need to be the same for some tracers, otherwise it will fail and disable all tracing, the tracers need to be stopped while updating the sub buffer sizes so that the tracers see the main and snapshot buffers with the same sub buffer size. Fixes: TBD ("ring-buffer: Add interface for configuring trace sub buffer size") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 7c5ef3f33d81..0cb5e1a50e65 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9405,13 +9405,16 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (val < 0 || val > 7) return -EINVAL; + /* Do not allow tracing while changing the order of the ring buffer */ + tracing_stop_tr(tr); + old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); if (old_order == val) - return 0; + goto out; ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); if (ret) - return 0; + goto out; #ifdef CONFIG_TRACER_MAX_TRACE @@ -9438,11 +9441,15 @@ buffer_order_write(struct file *filp, const char __user *ubuf, */ tracing_disabled = 1; } - return ret; + goto out; } out_max: #endif (*ppos)++; + out: + if (ret) + cnt = ret; + tracing_start_tr(tr); return cnt; } From patchwork Wed Dec 13 02:19:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177755 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8134070vqy; Tue, 12 Dec 2023 18:20:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IHY4KIRLuPjvOkCxA2ipCjPHqpI4rAEKr7u1vSZ4gbL3BYVlsYsxxu7fBLxshQ4QLUhQAUC X-Received: by 2002:a05:6808:1793:b0:3b2:ef72:f59e with SMTP id bg19-20020a056808179300b003b2ef72f59emr8999142oib.24.1702434055208; Tue, 12 Dec 2023 18:20:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434055; cv=none; d=google.com; s=arc-20160816; b=yX4CZW5igoIYSZK+Mtx3msWMvY5wX58XaJ1LaPNptEy237NWxvozSl4vUr2/cLfdwD 7pV4rYpIkMs3zGmwhpMWVkvB4m7i7i5B8+PPrj+VlT6QQY6E4tyK0UQ5pHzHzMwiqO0Z YyohNkLfi7RfURbDZHplA3rrIkjXt+gMSYbFmjuCoq8PfuxGH4wCtEjC13ea0HWbz8Dz knQ/ZAKG/1G2l82knqrHOJcn9KZtBwzfE05GXxaEO8T9jH/71v4sUTMvavg1TxcGtKxo aPXn1kOgT9WQtR0t48ymUiPY5BY/j7tsJBUj3CpjArHjtVAsoACh1O3+i0oQdtvCA3Rk 4b5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=ZobDNIVy76n8TCtsWB9gVbVLeKlpwMA0LZQrV6oENA0=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=bUWwhAIcfJ8jIg8yr3klH72l2r+IVjCdrNYMUuWj3f3mzVFqHZlYOQ9O0ldITcwdMN 3Yocx+JWegQ8EGXEbCgJNYIT+tgc7zA+i8XNdkt/3oAsmnKpKwUZOais+0X5U9rJmb71 /78lTI8FsWtjRxlnSD61vOmq0SzXt1rkqMT7Q81q97l7mZVzQ6vY70c1O25O5TYGAuDY EDgzNrsUfgYCNam1+f1Xkifro4H6BqV66JhlMDzRMNbPNDAkRxcNdCTbS/oNho2n62Ag fN//L5AA52rQN/bVIGT1RgP8hGBL1AcxNsIqHnYtUhMeahH4Q2iJiY14kz4bRucuFXdq RWHQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id bq15-20020a056a000e0f00b006cef66f8c03si6307765pfb.53.2023.12.12.18.20.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id DDB348083AF3; Tue, 12 Dec 2023 18:20:35 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378374AbjLMCTm (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378316AbjLMCTW (ORCPT ); Tue, 12 Dec 2023 21:19:22 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5686BCF for ; Tue, 12 Dec 2023 18:19:28 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EC20C433CA; Wed, 13 Dec 2023 02:19:27 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr5-00000002TqJ-1VjB; Tue, 12 Dec 2023 21:20:11 -0500 Message-ID: <20231213022011.141490099@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:25 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 11/15] ring-buffer: Keep the same size when updating the order References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:20:36 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131492225215963 X-GMAIL-MSGID: 1785131492225215963 From: "Steven Rostedt (Google)" The function ring_buffer_subbuf_order_set() just updated the sub-buffers to the new size, but this also changes the size of the buffer in doing so. As the size is determined by nr_pages * subbuf_size. If the subbuf_size is increased without decreasing the nr_pages, this causes the total size of the buffer to increase. This broke the latency tracers as the snapshot needs to be the same size as the main buffer. The size of the snapshot buffer is only expanded when needed, and because the order is still the same, the size becomes out of sync with the main buffer, as the main buffer increased in size without the tracing system knowing. Calculate the nr_pages to allocate with the new subbuf_size to be buffer_size / new_subbuf_size. Fixes: TBD ("ring-buffer: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 2c908b4f3f68..86d3cac2a877 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -6039,7 +6039,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; - nr_pages = buffer->buffers[cpu]->nr_pages; + /* Update the number of pages to match the new size */ + nr_pages = old_size * buffer->buffers[cpu]->nr_pages; + nr_pages = DIV_ROUND_UP(nr_pages, buffer->subbuf_size); + cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); if (!cpu_buffers[cpu]) { err = -ENOMEM; From patchwork Wed Dec 13 02:19:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177750 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133742vqy; Tue, 12 Dec 2023 18:20:04 -0800 (PST) X-Google-Smtp-Source: AGHT+IHlct0YcdVVpujlcAgHJErwXprXUOyRPyxP7VNN9RBSsPRyzxh74naF5BCzn+nDQd4OqeMZ X-Received: by 2002:a05:6870:c10c:b0:1fb:75c:3ffd with SMTP id f12-20020a056870c10c00b001fb075c3ffdmr9143127oad.93.1702434004199; Tue, 12 Dec 2023 18:20:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434004; cv=none; d=google.com; s=arc-20160816; b=yseHU5wNCc1sZG2iUnb8NX01lKIV3yBPaZg9WuujxDFWM4+94Qb0o2gqp7fbl8D1+/ RlV3XfUbCxWecgMLX1R2BV4gvaEtkS5cGC9+yNwKvO2XiWLwWai0LasS8tguiXNOVPZB 6lKUyDfdJtG9FC96igJwbD3mzt6bBqHGAdtLMRuvjaYzZ40WaYxqELuM3achNyJ6n7sS SfxBFwP5DjdlyOe83doTyj7TTgw2X+zpPN+XVSatx61gqy1DUvUVZaUPPHmUHFOdcRyB CCUh/3mhsrOYxJ8K3vXyuDuf8J10R/bHJPDDItJnP/obQAabrTWyKp8U2ZXxqlJp+gHw gnKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=L38wi1rCn3fHkaMj+wYjvFW/6K5ixWuyEwaLm46Q2w4=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=wdm/REVbN2LTKc5u31ckrO4AU8k7kG19JjGV7Gb4Ec9WYZSoxDY6WKqxRgNfhV9K6t foRxMiHb/Ow6ZkzVHztiLQfeqvFk1nLk5LNtIlmpPvp0n6/J+JrB2iIg8yyXLJvr4gas UhCQLs5aJk76r6PI760eeHJ1Vmj79BURkrHSHWaKiIvR07BBpWhAKEWNchdOwRgGfeDR t7Q3/V1DrajiS+iqsUt/eDtLozuk7Jdq4CYX5JkzimTunmv35F1wukagb8hFz1B+5LSb kHlMO+53X0Qn6ILnqfMoYxfIwpNH5iHFJ1AlyTzziN80J6t5djve6RgoDurwGmoH2KX2 a2Ag== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id 17-20020a631651000000b005c668a5a90esi8324088pgw.596.2023.12.12.18.20.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id E1AC8801B336; Tue, 12 Dec 2023 18:19:56 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378335AbjLMCTr (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378307AbjLMCTX (ORCPT ); Tue, 12 Dec 2023 21:19:23 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B18CEA for ; Tue, 12 Dec 2023 18:19:28 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8B5FC433C7; Wed, 13 Dec 2023 02:19:27 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr5-00000002Tqn-2fDi; Tue, 12 Dec 2023 21:20:11 -0500 Message-ID: <20231213022011.432211428@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:26 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 12/15] ring-buffer: Just update the subbuffers when changing their allocation order References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:19:57 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131438200833595 X-GMAIL-MSGID: 1785131438200833595 From: "Steven Rostedt (Google)" The ring_buffer_subbuf_order_set() was creating ring_buffer_per_cpu cpu_buffers with the new subbuffers with the updated order, and if they all successfully were created, then they the ring_buffer's per_cpu buffers would be freed and replaced by them. The problem is that the freed per_cpu buffers contains state that would be lost. Running the following commands: 1. # echo 3 > /sys/kernel/tracing/buffer_subbuf_order 2. # echo 0 > /sys/kernel/tracing/tracing_cpumask 3. # echo 1 > /sys/kernel/tracing/snapshot 4. # echo ff > /sys/kernel/tracing/tracing_cpumask 5. # echo test > /sys/kernel/tracing/trace_marker Would result in: -bash: echo: write error: Bad file descriptor That's because the state of the per_cpu buffers of the snapshot buffer is lost when the order is changed (the order of a freed snapshot buffer goes to 0 to save memory, and when the snapshot buffer is allocated again, it goes back to what the main buffer is). In operation 2, the snapshot buffers were set to "disable" (as all the ring buffers CPUs were disabled). In operation 3, the snapshot is allocated and a call to ring_buffer_subbuf_order_set() replaced the per_cpu buffers losing the "record_disable" count. When it was enabled again, the atomic_dec(&cpu_buffer->record_disable) was decrementing a zero, setting it to -1. Writing 1 into the snapshot would swap the snapshot buffer with the main buffer, so now the main buffer is "disabled", and nothing can write to the ring buffer anymore. Instead of creating new per_cpu buffers and losing the state of the old buffers, basically do what the resize does and just allocate new subbuf pages into the new_pages link list of the per_cpu buffer and if they all succeed, then replace the old sub buffers with the new ones. This keeps the per_cpu buffer descriptor in tact and by doing so, keeps its state. Fixes: TBD ("ring-buffer: Set new size of the ring buffer sub page") Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 88 ++++++++++++++++++++++++++++++-------- 1 file changed, 71 insertions(+), 17 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 86d3cac2a877..e08e646c4277 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -5998,11 +5998,11 @@ EXPORT_SYMBOL_GPL(ring_buffer_subbuf_order_get); */ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) { - struct ring_buffer_per_cpu **cpu_buffers; + struct ring_buffer_per_cpu *cpu_buffer; + struct buffer_page *bpage, *tmp; int old_order, old_size; int nr_pages; int psize; - int bsize; int err; int cpu; @@ -6016,11 +6016,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) if (psize <= BUF_PAGE_HDR_SIZE) return -EINVAL; - bsize = sizeof(void *) * buffer->cpus; - cpu_buffers = kzalloc(bsize, GFP_KERNEL); - if (!cpu_buffers) - return -ENOMEM; - old_order = buffer->subbuf_order; old_size = buffer->subbuf_size; @@ -6036,33 +6031,88 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) /* Make sure all new buffers are allocated, before deleting the old ones */ for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; + cpu_buffer = buffer->buffers[cpu]; + /* Update the number of pages to match the new size */ nr_pages = old_size * buffer->buffers[cpu]->nr_pages; nr_pages = DIV_ROUND_UP(nr_pages, buffer->subbuf_size); - cpu_buffers[cpu] = rb_allocate_cpu_buffer(buffer, nr_pages, cpu); - if (!cpu_buffers[cpu]) { + /* we need a minimum of two pages */ + if (nr_pages < 2) + nr_pages = 2; + + cpu_buffer->nr_pages_to_update = nr_pages; + + /* Include the reader page */ + nr_pages++; + + /* Allocate the new size buffer */ + INIT_LIST_HEAD(&cpu_buffer->new_pages); + if (__rb_allocate_pages(cpu_buffer, nr_pages, + &cpu_buffer->new_pages)) { + /* not enough memory for new pages */ err = -ENOMEM; goto error; } } for_each_buffer_cpu(buffer, cpu) { + if (!cpumask_test_cpu(cpu, buffer->cpumask)) continue; - rb_free_cpu_buffer(buffer->buffers[cpu]); - buffer->buffers[cpu] = cpu_buffers[cpu]; + cpu_buffer = buffer->buffers[cpu]; + + /* Clear the head bit to make the link list normal to read */ + rb_head_page_deactivate(cpu_buffer); + + /* Now walk the list and free all the old sub buffers */ + list_for_each_entry_safe(bpage, tmp, cpu_buffer->pages, list) { + list_del_init(&bpage->list); + free_buffer_page(bpage); + } + /* The above loop stopped an the last page needing to be freed */ + bpage = list_entry(cpu_buffer->pages, struct buffer_page, list); + free_buffer_page(bpage); + + /* Free the current reader page */ + free_buffer_page(cpu_buffer->reader_page); + + /* One page was allocated for the reader page */ + cpu_buffer->reader_page = list_entry(cpu_buffer->new_pages.next, + struct buffer_page, list); + list_del_init(&cpu_buffer->reader_page->list); + + /* The cpu_buffer pages are a link list with no head */ + cpu_buffer->pages = cpu_buffer->new_pages.next; + cpu_buffer->new_pages.next->prev = cpu_buffer->new_pages.prev; + cpu_buffer->new_pages.prev->next = cpu_buffer->new_pages.next; + + /* Clear the new_pages list */ + INIT_LIST_HEAD(&cpu_buffer->new_pages); + + cpu_buffer->head_page + = list_entry(cpu_buffer->pages, struct buffer_page, list); + cpu_buffer->tail_page = cpu_buffer->commit_page = cpu_buffer->head_page; + + cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; + cpu_buffer->nr_pages_to_update = 0; + + free_pages((unsigned long)cpu_buffer->free_page, old_order); + cpu_buffer->free_page = NULL; + + rb_head_page_activate(cpu_buffer); + + rb_check_pages(cpu_buffer); } atomic_dec(&buffer->record_disabled); mutex_unlock(&buffer->mutex); - kfree(cpu_buffers); - return 0; error: @@ -6073,12 +6123,16 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) mutex_unlock(&buffer->mutex); for_each_buffer_cpu(buffer, cpu) { - if (!cpu_buffers[cpu]) + cpu_buffer = buffer->buffers[cpu]; + + if (!cpu_buffer->nr_pages_to_update) continue; - rb_free_cpu_buffer(cpu_buffers[cpu]); - kfree(cpu_buffers[cpu]); + + list_for_each_entry_safe(bpage, tmp, &cpu_buffer->new_pages, list) { + list_del_init(&bpage->list); + free_buffer_page(bpage); + } } - kfree(cpu_buffers); return err; } From patchwork Wed Dec 13 02:19:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177748 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133725vqy; Tue, 12 Dec 2023 18:20:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IEI9hqgpe8TQ/m9aE0JfVcCQzrkqzTKsT6P0cJxn66xUzWqc4CI7IGFMF/mKPVWYfGiM8EB X-Received: by 2002:a05:6e02:1949:b0:35d:7200:916e with SMTP id x9-20020a056e02194900b0035d7200916emr7420744ilu.50.1702434002181; Tue, 12 Dec 2023 18:20:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434002; cv=none; d=google.com; s=arc-20160816; b=Y0Lb2tGpVl2U+S89xaFPc/daExb7fiEDxEcUB9x5e9pJ1eY74nnHrVCd9peuBzzrpu 0OrebEuIWLalZcG4Grwyy+eZYKkTf80jdNNfUHKYXZZqOVrKlk9ZvPIsWu3T1gTp7bO3 Eyy4tRm4TC6qbl8HON5+vB+DAs4WBx98Ls87jV31z7pKUL1nI6uGsjn1pxqUG0fx22I+ w0lb9W/s96YfMW7oM4X3yOTlUDiuaG/wz811m1s74hMp9BO2vqmnT17kzvzCrNCtq5R8 6RTZVwyo8/5ESM053ZqNbNffOCrFd7VB3W0OG4iATB16lOReQW0N3W0OE8rRm/3Yyw+R g2Dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=tdKtp+q3Qb3XGeC7e6FROAGw+qd8YOirCoNgdW22lfU=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=A9FJy5MbiUiFK+vTUAm2vzOGRr+xsGRT9Qwx6MATVY4FWwJbEef9vridHpxTnd/q7t YbJtR8PM6KzIIGldJrSlHQxG3swvWVYteLNysuGLyWQ1clyzDE8pSQ0axdbB4AmouUv1 OWfdzM0IPuDxnfLrhq1cw3QEpDmBrzZASGOQFh8LO2bmiGZokuqRXUGbJvEdibZE1Ac0 j0WuD0Zq6qtH3nOBpw32gPz/PMicz2M8MccHvt0apUL1Kiy4RNpD5pLrciyu5WFy60bM nBYhwu47/Mr9+2gtdY8CZUr9H/V/VHceenVGg021rzYDNVf7jdABQCGo7qnA0UDqo16+ Z6FA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id iz9-20020a170902ef8900b001d2f6d7f089si7251006plb.254.2023.12.12.18.20.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id E2BF680B8E56; Tue, 12 Dec 2023 18:20:00 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378344AbjLMCTv (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378309AbjLMCTX (ORCPT ); Tue, 12 Dec 2023 21:19:23 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3CACDC for ; Tue, 12 Dec 2023 18:19:28 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 241DAC433C9; Wed, 13 Dec 2023 02:19:28 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr5-00000002TrI-3rF1; Tue, 12 Dec 2023 21:20:11 -0500 Message-ID: <20231213022011.701293241@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:27 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 13/15] ring-buffer: Add documentation on the buffer_subbuf_order file References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:20:01 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131436480478782 X-GMAIL-MSGID: 1785131436480478782 From: "Steven Rostedt (Google)" Add to the documentation how to use the buffer_subbuf_order file to change the size and how it affects what events can be added to the ring buffer. Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 23572f6697c0..231d26ceedb0 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -203,6 +203,33 @@ of ftrace. Here is a list of some of the key files: This displays the total combined size of all the trace buffers. + buffer_subbuf_order: + + This sets or displays the sub buffer page size order. The ring buffer + is broken up into several same size "sub buffers". An event can not be + bigger than the size of the sub buffer. Normally, the sub buffer is + the size of the architecture's page (4K on x86). The sub buffer also + contains meta data at the start which also limits the size of an event. + That means when the sub buffer is a page size, no event can be larger + than the page size minus the sub buffer meta data. + + The buffer_subbuf_order allows the user to change the size of the sub + buffer. As the sub buffer is a set of pages by the power of 2, thus + the sub buffer total size is defined by the order: + + order size + ---- ---- + 0 PAGE_SIZE + 1 PAGE_SIZE * 2 + 2 PAGE_SIZE * 4 + 3 PAGE_SIZE * 8 + + Changing the order will change the sub buffer size allowing for events + to be larger than the page size. + + Note: When changing the order, tracing is stopped and any data in the + ring buffer and the snapshot buffer will be discarded. + free_buffer: If a process is performing tracing, and the ring buffer should be From patchwork Wed Dec 13 02:19:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177751 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8133802vqy; Tue, 12 Dec 2023 18:20:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IGf1hbKrSAHlFxFYX8jPf3fsJOJWksc4VCUQJmXc4sV3LxNWMlIyNIDfCeUgiacQUG6CkiH X-Received: by 2002:a05:6e02:b44:b0:359:6e32:70ae with SMTP id f4-20020a056e020b4400b003596e3270aemr7840998ilu.4.1702434012040; Tue, 12 Dec 2023 18:20:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434012; cv=none; d=google.com; s=arc-20160816; b=wKxkiJ09p9YoOnbyaAiQp5BpY0JAkT+46ThxKFqIVrGzJeLcE5hAfKD4M5O21UCXFK grEda8CNZhkvD2s/dMX+Hdndnf7O3cgdBZ4LA9YhrZWNgrEGrLkG0ZlPKibYI1t+th+J Prn1rjV74BdQX+l3DDQ0JV1z2fT3JUIoMvBbLxsEBm4KlaLJswrs3NaFlJVhE+/9zRpg ifAhkvnJrEB7TSr2WmzbM6K4J6jwB3k6333aLz6clJHIc+/B5/VAn0UanjslOHaBdd3R kxsl89Di7uTHzXovRepVI4khC/04BOySzkMBbLozM1MjLDRS/sB4JDHFemVMOigchtA/ nOqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=dvPv41ieuApqqhojcvKXbRsEGPZCimaxwfryOBCSi6w=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=XJU7ZdNTPsdheG6+oIwe3taoNxg61ZsbY+bpCcyZLzt/L3qxavKi5F76r2iV+NonoF A1nPttk8tWNF/7NXwBOUlDJPWA+LxuyR4zkDlpsO/vhDImInNoNcAMjx6DlclhGsC73Q AyA3ne7b3otmRcqKyUpPQ32Ato67z5JzH4/F9wuDLQfz3e1XiUAb6hGBghM+Pfiyosqr m/h1E4a1EwNbKQWWCODL7JuHv07fi6nAYml3A38l23dlEagoavRjl2VUJk62yN7Wi2DR BorbLqvWB5kKgHHeX5ow9FsfJ4z14EHMLdJZyiZYwJmMjHGXHKUcR8+wbgy+TTS8jbVm pZYA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id i16-20020a170902c95000b001d0b1392c7fsi8924479pla.350.2023.12.12.18.20.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7F7E580B8E77; Tue, 12 Dec 2023 18:20:08 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378436AbjLMCTy (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378313AbjLMCTX (ORCPT ); Tue, 12 Dec 2023 21:19:23 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E573107 for ; Tue, 12 Dec 2023 18:19:29 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E676C433CB; Wed, 13 Dec 2023 02:19:28 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr6-00000002Trn-0psh; Tue, 12 Dec 2023 21:20:12 -0500 Message-ID: <20231213022011.985384265@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:28 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 14/15] ringbuffer/selftest: Add basic selftest to test changing subbuf order References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:20:08 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131446711340102 X-GMAIL-MSGID: 1785131446711340102 From: "Steven Rostedt (Google)" Add a self test that will write into the trace buffer with differ trace sub buffer order sizes. Signed-off-by: Steven Rostedt (Google) --- .../ftrace/test.d/00basic/ringbuffer_order.tc | 95 +++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc diff --git a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc new file mode 100644 index 000000000000..5204c48a82f9 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc @@ -0,0 +1,95 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: Change the ringbuffer sub-buffer order +# requires: buffer_subbuf_order +# flags: instance + +get_buffer_data_size() { + sed -ne 's/^.*data.*size:\([0-9][0-9]*\).*/\1/p' events/header_page +} + +get_buffer_data_offset() { + sed -ne 's/^.*data.*offset:\([0-9][0-9]*\).*/\1/p' events/header_page +} + +get_event_header_size() { + type_len=`sed -ne 's/^.*type_len.*:[^0-9]*\([0-9][0-9]*\).*/\1/p' events/header_event` + time_len=`sed -ne 's/^.*time_delta.*:[^0-9]*\([0-9][0-9]*\).*/\1/p' events/header_event` + array_len=`sed -ne 's/^.*array.*:[^0-9]*\([0-9][0-9]*\).*/\1/p' events/header_event` + total_bits=$((type_len+time_len+array_len)) + total_bits=$((total_bits+7)) + echo $((total_bits/8)) +} + +get_print_event_buf_offset() { + sed -ne 's/^.*buf.*offset:\([0-9][0-9]*\).*/\1/p' events/ftrace/print/format +} + +event_header_size=`get_event_header_size` +print_header_size=`get_print_event_buf_offset` + +data_offset=`get_buffer_data_offset` + +marker_meta=$((event_header_size+print_header_size)) + +make_str() { + cnt=$1 + printf -- 'X%.0s' $(seq $cnt) +} + +write_buffer() { + size=$1 + + str=`make_str $size` + + # clear the buffer + echo > trace + + # write the string into the marker + echo $str > trace_marker + + echo $str +} + +test_buffer() { + orde=$1 + page_size=$((4096< buffer_subbuf_order + test_buffer $a +done + +echo $ORIG > buffer_subbuf_order + From patchwork Wed Dec 13 02:19:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 177754 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp8134062vqy; Tue, 12 Dec 2023 18:20:53 -0800 (PST) X-Google-Smtp-Source: AGHT+IFnjAXXSs9/+v2Fz6BogpyfUU4nSBZfYlLyDbs5KXtXoVs7DqLRIu1Kxle4jhfBveB7OPNN X-Received: by 2002:a05:6e02:1a07:b0:35d:578e:407d with SMTP id s7-20020a056e021a0700b0035d578e407dmr9187211ild.0.1702434053501; Tue, 12 Dec 2023 18:20:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702434053; cv=none; d=google.com; s=arc-20160816; b=n4IuYQrKPYOcgaM+JRIj6jVceVOzU6GqS+yPaVTRFyU8+TepB4Eqf9q8BHyHJDyp3a K0Q2CMIDx5hL8KElY3Od4407p5O2i8KvL48DzpRpoXH0EYbN9TaWcBP9E2WgJ7um8WzK Dnnkhz6OYYVNa8XC0Ufm65gPtPFhj6k3KwfbEPWPqLtki5XIpx3rhzkHOdHShfDP/gM/ IzRJJ2xrcVRZwEn5eLxSWcDa2USMbwWpl2ItA/mrgk/iRI4k7Yej5Y+8iMSeMVKUJzst 4M3SaGMJLjAV4RvRmw+v7nmPQBMT3wMQeB8SxWbSAuOeo358kmcRZIZYGe7WD/x2g/7+ eymQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=FFsyqJzd/Up+VUudOOifrojtmq2tVsQ0dgh0yfNbm6E=; fh=e3UoLP0GGIVG/JuDivUt2G6X10h9sIOfzt1IJTPiLuw=; b=x/iP3lZvOUCjNxJBNEIuDJWxLGclJ20pRtiQdH/RDYoCKtD5EI3Ux6uVtJxZp49cjy wgMo5fxE+MAUigWEPSxNbycrYoqdUvELj7uM8ovL5RSwLQew89Nv6Q9CVnmaHzn6HrIf hGLOLvDgPj4RBTo/AFFLtydSe6Vnjmj2DyYrXo1vDY/imA1tN4LhyOTWyJ/LEF+KbNbn nVJtFDq21y+uojDQhmrw91WpjE7aBNyJBwqKwazzfOb2Mr6qEFI5ctHQ3mq188VG5c6q hCTdeg69+EtuCmDHm8BmEh+YTFE5vcnvfCEgx2uJasZBQdptm05NTVEzUth4o/y3tFNs LoIA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id e1-20020a656881000000b005c278464e08si8572415pgt.275.2023.12.12.18.20.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 18:20:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 4479F80E9953; Tue, 12 Dec 2023 18:20:12 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378305AbjLMCT6 (ORCPT + 99 others); Tue, 12 Dec 2023 21:19:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378321AbjLMCTY (ORCPT ); Tue, 12 Dec 2023 21:19:24 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C8F1E8 for ; Tue, 12 Dec 2023 18:19:29 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2EA8C433CC; Wed, 13 Dec 2023 02:19:28 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.97) (envelope-from ) id 1rDEr6-00000002TsJ-22KG; Tue, 12 Dec 2023 21:20:12 -0500 Message-ID: <20231213022012.267049115@goodmis.org> User-Agent: quilt/0.67 Date: Tue, 12 Dec 2023 21:19:29 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Tzvetomir Stoyanov , Vincent Donnefort , Kent Overstreet Subject: [PATCH v2 15/15] tracing: Update subbuffer with kilobytes not page order References: <20231213021914.361709558@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 12 Dec 2023 18:20:12 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785131489956878729 X-GMAIL-MSGID: 1785131489956878729 From: "Steven Rostedt (Google)" Using page order for deciding what the size of the ring buffer sub buffers are is exposing a bit too much of the implementation. Although the sub buffers are only allocated in orders of pages, allow the user to specify the minimum size of each sub-buffer via kilobytes like they can with the buffer size itself. If the user specifies 3 via: echo 3 > buffer_subbuf_size_kb Then the sub-buffer size will round up to 4kb (on a 4kb page size system). If they specify: echo 6 > buffer_subbuf_size_kb The sub-buffer size will become 8kb. and so on. Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 46 ++++++++----------- kernel/trace/trace.c | 38 +++++++++------ ...fer_order.tc => ringbuffer_subbuf_size.tc} | 18 ++++---- 3 files changed, 54 insertions(+), 48 deletions(-) rename tools/testing/selftests/ftrace/test.d/00basic/{ringbuffer_order.tc => ringbuffer_subbuf_size.tc} (85%) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 231d26ceedb0..933e7efb9f1b 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -203,32 +203,26 @@ of ftrace. Here is a list of some of the key files: This displays the total combined size of all the trace buffers. - buffer_subbuf_order: - - This sets or displays the sub buffer page size order. The ring buffer - is broken up into several same size "sub buffers". An event can not be - bigger than the size of the sub buffer. Normally, the sub buffer is - the size of the architecture's page (4K on x86). The sub buffer also - contains meta data at the start which also limits the size of an event. - That means when the sub buffer is a page size, no event can be larger - than the page size minus the sub buffer meta data. - - The buffer_subbuf_order allows the user to change the size of the sub - buffer. As the sub buffer is a set of pages by the power of 2, thus - the sub buffer total size is defined by the order: - - order size - ---- ---- - 0 PAGE_SIZE - 1 PAGE_SIZE * 2 - 2 PAGE_SIZE * 4 - 3 PAGE_SIZE * 8 - - Changing the order will change the sub buffer size allowing for events - to be larger than the page size. - - Note: When changing the order, tracing is stopped and any data in the - ring buffer and the snapshot buffer will be discarded. + buffer_subbuf_size_kb: + + This sets or displays the sub buffer size. The ring buffer is broken up + into several same size "sub buffers". An event can not be bigger than + the size of the sub buffer. Normally, the sub buffer is the size of the + architecture's page (4K on x86). The sub buffer also contains meta data + at the start which also limits the size of an event. That means when + the sub buffer is a page size, no event can be larger than the page + size minus the sub buffer meta data. + + Note, the buffer_subbuf_size_kb is a way for the user to specify the + minimum size of the subbuffer. The kernel may make it bigger due to the + implementation details, or simply fail the operation if the kernel can + not handle the request. + + Changing the sub buffer size allows for events to be larger than the + page size. + + Note: When changing the sub-buffer size, tracing is stopped and any + data in the ring buffer and the snapshot buffer will be discarded. free_buffer: diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 0cb5e1a50e65..d121f3742792 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -9377,42 +9377,54 @@ static const struct file_operations buffer_percent_fops = { }; static ssize_t -buffer_order_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) +buffer_subbuf_size_read(struct file *filp, char __user *ubuf, size_t cnt, loff_t *ppos) { struct trace_array *tr = filp->private_data; + size_t size; char buf[64]; + int order; int r; - r = sprintf(buf, "%d\n", ring_buffer_subbuf_order_get(tr->array_buffer.buffer)); + order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); + size = (PAGE_SIZE << order) / 1024; + + r = sprintf(buf, "%zd\n", size); return simple_read_from_buffer(ubuf, cnt, ppos, buf, r); } static ssize_t -buffer_order_write(struct file *filp, const char __user *ubuf, - size_t cnt, loff_t *ppos) +buffer_subbuf_size_write(struct file *filp, const char __user *ubuf, + size_t cnt, loff_t *ppos) { struct trace_array *tr = filp->private_data; unsigned long val; int old_order; + int order; + int pages; int ret; ret = kstrtoul_from_user(ubuf, cnt, 10, &val); if (ret) return ret; + val *= 1024; /* value passed in is in KB */ + + pages = DIV_ROUND_UP(val, PAGE_SIZE); + order = fls(pages - 1); + /* limit between 1 and 128 system pages */ - if (val < 0 || val > 7) + if (order < 0 || order > 7) return -EINVAL; /* Do not allow tracing while changing the order of the ring buffer */ tracing_stop_tr(tr); old_order = ring_buffer_subbuf_order_get(tr->array_buffer.buffer); - if (old_order == val) + if (old_order == order) goto out; - ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, val); + ret = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, order); if (ret) goto out; @@ -9421,7 +9433,7 @@ buffer_order_write(struct file *filp, const char __user *ubuf, if (!tr->allocated_snapshot) goto out_max; - ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, val); + ret = ring_buffer_subbuf_order_set(tr->max_buffer.buffer, order); if (ret) { /* Put back the old order */ cnt = ring_buffer_subbuf_order_set(tr->array_buffer.buffer, old_order); @@ -9453,10 +9465,10 @@ buffer_order_write(struct file *filp, const char __user *ubuf, return cnt; } -static const struct file_operations buffer_order_fops = { +static const struct file_operations buffer_subbuf_size_fops = { .open = tracing_open_generic_tr, - .read = buffer_order_read, - .write = buffer_order_write, + .read = buffer_subbuf_size_read, + .write = buffer_subbuf_size_write, .release = tracing_release_generic_tr, .llseek = default_llseek, }; @@ -9927,8 +9939,8 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer) trace_create_file("buffer_percent", TRACE_MODE_WRITE, d_tracer, tr, &buffer_percent_fops); - trace_create_file("buffer_subbuf_order", TRACE_MODE_WRITE, d_tracer, - tr, &buffer_order_fops); + trace_create_file("buffer_subbuf_size_kb", TRACE_MODE_WRITE, d_tracer, + tr, &buffer_subbuf_size_fops); create_trace_options_dir(tr); diff --git a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_subbuf_size.tc similarity index 85% rename from tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc rename to tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_subbuf_size.tc index 5204c48a82f9..1abf5c39952a 100644 --- a/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_order.tc +++ b/tools/testing/selftests/ftrace/test.d/00basic/ringbuffer_subbuf_size.tc @@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 -# description: Change the ringbuffer sub-buffer order -# requires: buffer_subbuf_order +# description: Change the ringbuffer sub-buffer size +# requires: buffer_subbuf_size_kb # flags: instance get_buffer_data_size() { @@ -52,8 +52,8 @@ write_buffer() { } test_buffer() { - orde=$1 - page_size=$((4096< buffer_subbuf_order +for a in 4 8 16 32 ; do + echo $a > buffer_subbuf_size_kb test_buffer $a done -echo $ORIG > buffer_subbuf_order +echo $ORIG > buffer_subbuf_size_kb