[08/15] tracing: Introduce names for ring buffers
Commit Message
With KHO (Kexec HandOver), we want to preserve trace buffers across
kexec. To carry over their state between kernels, the kernel needs a
common handle for them that exists on both sides. As handle we introduce
names for ring buffers. In a follow-up patch, the kernel can then use
these names to recover buffer contents for specific ring buffers.
Signed-off-by: Alexander Graf <graf@amazon.com>
---
include/linux/ring_buffer.h | 7 ++++---
kernel/trace/ring_buffer.c | 5 ++++-
kernel/trace/trace.c | 7 ++++---
3 files changed, 12 insertions(+), 7 deletions(-)
Comments
On Wed, 13 Dec 2023 00:04:45 +0000
Alexander Graf <graf@amazon.com> wrote:
> With KHO (Kexec HandOver), we want to preserve trace buffers across
> kexec. To carry over their state between kernels, the kernel needs a
> common handle for them that exists on both sides. As handle we introduce
> names for ring buffers. In a follow-up patch, the kernel can then use
> these names to recover buffer contents for specific ring buffers.
>
Is there a way to use the trace_array name instead?
The trace_array is the structure that represents each tracing instance. And
it already has a name field. And if you can get the associated ring buffer
from that too.
struct trace_array *tr;
tr->array_buffer.buffer
tr->name
When you do: mkdir /sys/kernel/tracing/instance/foo
You create a new trace_array instance where tr->name = "foo" and allocates
the buffer for it as well.
-- Steve
Hi Steve,
On 13.12.23 01:15, Steven Rostedt wrote:
>
> On Wed, 13 Dec 2023 00:04:45 +0000
> Alexander Graf <graf@amazon.com> wrote:
>
>> With KHO (Kexec HandOver), we want to preserve trace buffers across
>> kexec. To carry over their state between kernels, the kernel needs a
>> common handle for them that exists on both sides. As handle we introduce
>> names for ring buffers. In a follow-up patch, the kernel can then use
>> these names to recover buffer contents for specific ring buffers.
>>
> Is there a way to use the trace_array name instead?
>
> The trace_array is the structure that represents each tracing instance. And
> it already has a name field. And if you can get the associated ring buffer
> from that too.
>
> struct trace_array *tr;
>
> tr->array_buffer.buffer
>
> tr->name
>
> When you do: mkdir /sys/kernel/tracing/instance/foo
>
> You create a new trace_array instance where tr->name = "foo" and allocates
> the buffer for it as well.
The name in the ring buffer is pretty much just a copy of the trace
array name. I use it to reconstruct which buffer we're actually
referring to inside __ring_buffer_alloc().
I'm all ears for alternative suggestions. I suppose we could pass tr as
argument to ring_buffer_alloc() instead of the name?
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
On Wed, 13 Dec 2023 01:35:16 +0100
Alexander Graf <graf@amazon.com> wrote:
> > The trace_array is the structure that represents each tracing instance. And
> > it already has a name field. And if you can get the associated ring buffer
> > from that too.
> >
> > struct trace_array *tr;
> >
> > tr->array_buffer.buffer
> >
> > tr->name
> >
> > When you do: mkdir /sys/kernel/tracing/instance/foo
> >
> > You create a new trace_array instance where tr->name = "foo" and allocates
> > the buffer for it as well.
>
> The name in the ring buffer is pretty much just a copy of the trace
> array name. I use it to reconstruct which buffer we're actually
> referring to inside __ring_buffer_alloc().
No, I rather not tie the ring buffer to the trace_array.
>
> I'm all ears for alternative suggestions. I suppose we could pass tr as
> argument to ring_buffer_alloc() instead of the name?
I'll have to spend some time (that I don't currently have :-( ) on looking
at this more. I really don't like the copying of the name into the ring
buffer allocation, as it may be an unneeded burden to maintain, not to
mention the duplicate field.
-- Steve
Hi Alexander,
kernel test robot noticed the following build errors:
[auto build test ERROR on tip/x86/core]
[also build test ERROR on arm64/for-next/core akpm-mm/mm-everything linus/master v6.7-rc5 next-20231213]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Alexander-Graf/mm-memblock-Add-support-for-scratch-memory/20231213-080941
base: tip/x86/core
patch link: https://lore.kernel.org/r/20231213000452.88295-9-graf%40amazon.com
patch subject: [PATCH 08/15] tracing: Introduce names for ring buffers
config: i386-buildonly-randconfig-003-20231213 (https://download.01.org/0day-ci/archive/20231213/202312131922.5dJORQu4-lkp@intel.com/config)
compiler: clang version 16.0.4 (https://github.com/llvm/llvm-project.git ae42196bc493ffe877a7e3dff8be32035dea4d07)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231213/202312131922.5dJORQu4-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312131922.5dJORQu4-lkp@intel.com/
All errors (new ones prefixed by >>):
>> kernel/trace/ring_buffer_benchmark.c:435:53: error: too few arguments provided to function-like macro invocation
buffer = ring_buffer_alloc(1000000, RB_FL_OVERWRITE);
^
include/linux/ring_buffer.h:96:9: note: macro 'ring_buffer_alloc' defined here
#define ring_buffer_alloc(name, size, flags) \
^
>> kernel/trace/ring_buffer_benchmark.c:435:11: error: use of undeclared identifier 'ring_buffer_alloc'; did you mean '__ring_buffer_alloc'?
buffer = ring_buffer_alloc(1000000, RB_FL_OVERWRITE);
^~~~~~~~~~~~~~~~~
__ring_buffer_alloc
include/linux/ring_buffer.h:88:1: note: '__ring_buffer_alloc' declared here
__ring_buffer_alloc(const char *name, unsigned long size, unsigned flags,
^
2 errors generated.
--
>> kernel/trace/ring_buffer.c:6096:65: error: too few arguments provided to function-like macro invocation
buffer = ring_buffer_alloc(RB_TEST_BUFFER_SIZE, RB_FL_OVERWRITE);
^
include/linux/ring_buffer.h:96:9: note: macro 'ring_buffer_alloc' defined here
#define ring_buffer_alloc(name, size, flags) \
^
>> kernel/trace/ring_buffer.c:6096:11: error: use of undeclared identifier 'ring_buffer_alloc'; did you mean '__ring_buffer_alloc'?
buffer = ring_buffer_alloc(RB_TEST_BUFFER_SIZE, RB_FL_OVERWRITE);
^~~~~~~~~~~~~~~~~
__ring_buffer_alloc
kernel/trace/ring_buffer.c:1873:19: note: '__ring_buffer_alloc' declared here
EXPORT_SYMBOL_GPL(__ring_buffer_alloc);
^
2 errors generated.
vim +435 kernel/trace/ring_buffer_benchmark.c
5092dbc96f3acd Steven Rostedt 2009-05-05 429
5092dbc96f3acd Steven Rostedt 2009-05-05 430 static int __init ring_buffer_benchmark_init(void)
5092dbc96f3acd Steven Rostedt 2009-05-05 431 {
5092dbc96f3acd Steven Rostedt 2009-05-05 432 int ret;
5092dbc96f3acd Steven Rostedt 2009-05-05 433
5092dbc96f3acd Steven Rostedt 2009-05-05 434 /* make a one meg buffer in overwite mode */
5092dbc96f3acd Steven Rostedt 2009-05-05 @435 buffer = ring_buffer_alloc(1000000, RB_FL_OVERWRITE);
5092dbc96f3acd Steven Rostedt 2009-05-05 436 if (!buffer)
5092dbc96f3acd Steven Rostedt 2009-05-05 437 return -ENOMEM;
5092dbc96f3acd Steven Rostedt 2009-05-05 438
5092dbc96f3acd Steven Rostedt 2009-05-05 439 if (!disable_reader) {
5092dbc96f3acd Steven Rostedt 2009-05-05 440 consumer = kthread_create(ring_buffer_consumer_thread,
5092dbc96f3acd Steven Rostedt 2009-05-05 441 NULL, "rb_consumer");
5092dbc96f3acd Steven Rostedt 2009-05-05 442 ret = PTR_ERR(consumer);
5092dbc96f3acd Steven Rostedt 2009-05-05 443 if (IS_ERR(consumer))
5092dbc96f3acd Steven Rostedt 2009-05-05 444 goto out_fail;
5092dbc96f3acd Steven Rostedt 2009-05-05 445 }
5092dbc96f3acd Steven Rostedt 2009-05-05 446
5092dbc96f3acd Steven Rostedt 2009-05-05 447 producer = kthread_run(ring_buffer_producer_thread,
5092dbc96f3acd Steven Rostedt 2009-05-05 448 NULL, "rb_producer");
5092dbc96f3acd Steven Rostedt 2009-05-05 449 ret = PTR_ERR(producer);
5092dbc96f3acd Steven Rostedt 2009-05-05 450
5092dbc96f3acd Steven Rostedt 2009-05-05 451 if (IS_ERR(producer))
5092dbc96f3acd Steven Rostedt 2009-05-05 452 goto out_kill;
5092dbc96f3acd Steven Rostedt 2009-05-05 453
98e4833ba3c314 Ingo Molnar 2009-11-23 454 /*
98e4833ba3c314 Ingo Molnar 2009-11-23 455 * Run them as low-prio background tasks by default:
98e4833ba3c314 Ingo Molnar 2009-11-23 456 */
7ac07434048001 Steven Rostedt 2009-11-25 457 if (!disable_reader) {
4fd5750af02ab7 Peter Zijlstra 2020-07-20 458 if (consumer_fifo >= 2)
4fd5750af02ab7 Peter Zijlstra 2020-07-20 459 sched_set_fifo(consumer);
4fd5750af02ab7 Peter Zijlstra 2020-07-20 460 else if (consumer_fifo == 1)
4fd5750af02ab7 Peter Zijlstra 2020-07-20 461 sched_set_fifo_low(consumer);
4fd5750af02ab7 Peter Zijlstra 2020-07-20 462 else
7ac07434048001 Steven Rostedt 2009-11-25 463 set_user_nice(consumer, consumer_nice);
7ac07434048001 Steven Rostedt 2009-11-25 464 }
7ac07434048001 Steven Rostedt 2009-11-25 465
4fd5750af02ab7 Peter Zijlstra 2020-07-20 466 if (producer_fifo >= 2)
4fd5750af02ab7 Peter Zijlstra 2020-07-20 467 sched_set_fifo(producer);
4fd5750af02ab7 Peter Zijlstra 2020-07-20 468 else if (producer_fifo == 1)
4fd5750af02ab7 Peter Zijlstra 2020-07-20 469 sched_set_fifo_low(producer);
4fd5750af02ab7 Peter Zijlstra 2020-07-20 470 else
7ac07434048001 Steven Rostedt 2009-11-25 471 set_user_nice(producer, producer_nice);
98e4833ba3c314 Ingo Molnar 2009-11-23 472
5092dbc96f3acd Steven Rostedt 2009-05-05 473 return 0;
5092dbc96f3acd Steven Rostedt 2009-05-05 474
5092dbc96f3acd Steven Rostedt 2009-05-05 475 out_kill:
5092dbc96f3acd Steven Rostedt 2009-05-05 476 if (consumer)
5092dbc96f3acd Steven Rostedt 2009-05-05 477 kthread_stop(consumer);
5092dbc96f3acd Steven Rostedt 2009-05-05 478
5092dbc96f3acd Steven Rostedt 2009-05-05 479 out_fail:
5092dbc96f3acd Steven Rostedt 2009-05-05 480 ring_buffer_free(buffer);
5092dbc96f3acd Steven Rostedt 2009-05-05 481 return ret;
5092dbc96f3acd Steven Rostedt 2009-05-05 482 }
5092dbc96f3acd Steven Rostedt 2009-05-05 483
@@ -85,17 +85,18 @@ void ring_buffer_discard_commit(struct trace_buffer *buffer,
* size is in bytes for each per CPU buffer.
*/
struct trace_buffer *
-__ring_buffer_alloc(unsigned long size, unsigned flags, struct lock_class_key *key);
+__ring_buffer_alloc(const char *name, unsigned long size, unsigned flags,
+ struct lock_class_key *key);
/*
* Because the ring buffer is generic, if other users of the ring buffer get
* traced by ftrace, it can produce lockdep warnings. We need to keep each
* ring buffer's lock class separate.
*/
-#define ring_buffer_alloc(size, flags) \
+#define ring_buffer_alloc(name, size, flags) \
({ \
static struct lock_class_key __key; \
- __ring_buffer_alloc((size), (flags), &__key); \
+ __ring_buffer_alloc((name), (size), (flags), &__key); \
})
int ring_buffer_wait(struct trace_buffer *buffer, int cpu, int full);
@@ -557,6 +557,7 @@ struct trace_buffer {
struct rb_irq_work irq_work;
bool time_stamp_abs;
+ const char *name;
};
struct ring_buffer_iter {
@@ -1801,7 +1802,8 @@ static void rb_free_cpu_buffer(struct ring_buffer_per_cpu *cpu_buffer)
* when the buffer wraps. If this flag is not set, the buffer will
* drop data when the tail hits the head.
*/
-struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags,
+struct trace_buffer *__ring_buffer_alloc(const char *name,
+ unsigned long size, unsigned flags,
struct lock_class_key *key)
{
struct trace_buffer *buffer;
@@ -1823,6 +1825,7 @@ struct trace_buffer *__ring_buffer_alloc(unsigned long size, unsigned flags,
buffer->flags = flags;
buffer->clock = trace_clock_local;
buffer->reader_lock_key = key;
+ buffer->name = name;
init_irq_work(&buffer->irq_work.work, rb_wake_up_waiters);
init_waitqueue_head(&buffer->irq_work.waiters);
@@ -9384,7 +9384,8 @@ allocate_trace_buffer(struct trace_array *tr, struct array_buffer *buf, int size
buf->tr = tr;
- buf->buffer = ring_buffer_alloc(size, rb_flags);
+ buf->buffer = ring_buffer_alloc(tr->name ? tr->name : "global_trace",
+ size, rb_flags);
if (!buf->buffer)
return -ENOMEM;
@@ -9421,7 +9422,7 @@ static int allocate_trace_buffers(struct trace_array *tr, int size)
return ret;
#ifdef CONFIG_TRACER_MAX_TRACE
- ret = allocate_trace_buffer(tr, &tr->max_buffer,
+ ret = allocate_trace_buffer(NULL, &tr->max_buffer,
allocate_snapshot ? size : 1);
if (MEM_FAIL(ret, "Failed to allocate trace buffer\n")) {
free_trace_buffer(&tr->array_buffer);
@@ -10473,7 +10474,7 @@ __init static int tracer_alloc_buffers(void)
goto out_free_cpumask;
/* Used for event triggers */
ret = -ENOMEM;
- temp_buffer = ring_buffer_alloc(PAGE_SIZE, RB_FL_OVERWRITE);
+ temp_buffer = ring_buffer_alloc("temp_buffer", PAGE_SIZE, RB_FL_OVERWRITE);
if (!temp_buffer)
goto out_rm_hp_state;