From patchwork Fri Dec 2 15:58:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28950 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp937580wrr; Fri, 2 Dec 2022 08:01:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf4tKRKer41WIjnDJ3GxgdSzGXN2HcTeHTRz/nx6fos+20lObrfrHMRhVRdjh3bnlHJ65pSL X-Received: by 2002:a63:5d58:0:b0:476:ffc7:f7b with SMTP id o24-20020a635d58000000b00476ffc70f7bmr46377230pgm.271.1669996879673; Fri, 02 Dec 2022 08:01:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996879; cv=none; d=google.com; s=arc-20160816; b=pTAUWHvFXSldLZw2CsL0mIXcwy/OjjJ0f74TG57qShDGSxOQK5ZuM1FStdkiEp+tAN UQsHZcJIhjJkQPWyvT8lDeKT1CraPDy/ELMhlTStOvBkEEs+6wRJrBc7aktTeiAi8Drw 4ojaa3goey1mG75P5nh17cdpIn7/mfB/21GDKdU2KG1s+8x7zlEjW9lb5yDBS9ZrmV9o 1GDhwlwTOsJU0ln+4S7q/rUZ6NteW9JK6tSBEBgq9NUDxc5GPN7OP6pEq+OMVTJb5peL EGvQs/6tThGF4QPjLJVkLlCJBExdTGQG60CbS1Cm3omNgm1Zh2hVD21LWtYbjb+diB4j AGow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=q8YngGgRcmlRV/MB12zFfR3CA+LXaloiHhszoenRMmY=; b=o3lsyGISnLHqI0n7vfWookCqBA8Z54FMECG7TWRcIT/uBlm1aYwvxhcXvpf4UfcRXi G9Ea+1GYVb/RPaLMcIMkJGBP2orDUg4PxODtAsV7oGJpbig2BKo3KD7gmuuJcxqo8L8d FlZ9JPvCE+9Jebfb5s53ljtm9RFNfDaVQoXog7RNSYS7wyr2xLJBvkSDj5y6j6YAK+hB PMoCm4D5aVDNooyqK/OQzeubFZS36ExN0j+Z3Oal0wy8z7CUjzG6aVVjDmSTSfrjarGS ar4zl8N/Z+Dx5/XPvrtP5tQMefDP5HbsbMJiLdyYKdBetyMYCw8mJIdxYv89PLh4peHR 6uIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hlv+Qb+0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nu7-20020a17090b1b0700b0020ab20c54fesi12399171pjb.114.2022.12.02.08.01.05; Fri, 02 Dec 2022 08:01:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hlv+Qb+0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233795AbiLBQAL (ORCPT + 99 others); Fri, 2 Dec 2022 11:00:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233714AbiLBQAG (ORCPT ); Fri, 2 Dec 2022 11:00:06 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB8194219E for ; Fri, 2 Dec 2022 07:59:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996743; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q8YngGgRcmlRV/MB12zFfR3CA+LXaloiHhszoenRMmY=; b=hlv+Qb+0afx479qf9Cc0M8k7vXc4FP8iDdGCgn30sgJGFN9hn5LjopLgi9Kkrv6Kx/uovF RGHPUERYiEvUe7c0t7SC7lkGepK/ULvdleGGlfSC5qgVtVJjZfaVjesmlNYdmzappzRR7d TJppZ3ogkKe0HhIHu6w4I3K3nCfirAk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-509-1wkZ57ENMJC4qSZWzWfbjQ-1; Fri, 02 Dec 2022 10:59:00 -0500 X-MC-Unique: 1wkZ57ENMJC4qSZWzWfbjQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 66D1D811E81; Fri, 2 Dec 2022 15:58:58 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id ECD502028E96; Fri, 2 Dec 2022 15:58:52 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: "Steven Rostedt (Google)" , "Paul E. McKenney" , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 1/8] DO-NOT-MERGE: tracing: Add __cpumask to denote a trace event field that is a cpumask_t Date: Fri, 2 Dec 2022 15:58:10 +0000 Message-Id: <20221202155817.2102944-2-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118647975994081?= X-GMAIL-MSGID: =?utf-8?q?1751118647975994081?= From: "Steven Rostedt (Google)" The trace events have a __bitmask field that can be used for anything that requires bitmasks. Although currently it is only used for CPU masks, it could be used in the future for any type of bitmasks. There is some user space tooling that wants to know if a field is a CPU mask and not just some random unsigned long bitmask. Introduce "__cpumask()" helper functions that work the same as the current __bitmask() helpers but displays in the format file: field:__data_loc cpumask_t *[] mask; offset:36; size:4; signed:0; Instead of: field:__data_loc unsigned long[] mask; offset:32; size:4; signed:0; The main difference is the type. Instead of "unsigned long" it is "cpumask_t *". Note, this type field needs to be a real type in the __dynamic_array() logic that both __cpumask and__bitmask use, but the comparison field requires it to be a scalar type whereas cpumask_t is a structure (non-scalar). But everything works when making it a pointer. Valentin added changes to remove the need of passing in "nr_bits" and the __cpumask will always use nr_cpumask_bits as its size. Link: https://lkml.kernel.org/r/20221014080456.1d32b989@rorschach.local.home Requested-by: Valentin Schneider Reviewed-by: Valentin Schneider Signed-off-by: Valentin Schneider Signed-off-by: Steven Rostedt (Google) --- include/trace/bpf_probe.h | 6 ++++ include/trace/perf.h | 6 ++++ include/trace/stages/stage1_struct_define.h | 6 ++++ include/trace/stages/stage2_data_offsets.h | 6 ++++ include/trace/stages/stage3_trace_output.h | 6 ++++ include/trace/stages/stage4_event_fields.h | 6 ++++ include/trace/stages/stage5_get_offsets.h | 6 ++++ include/trace/stages/stage6_event_callback.h | 20 ++++++++++++ include/trace/stages/stage7_class_define.h | 2 ++ samples/trace_events/trace-events-sample.c | 2 +- samples/trace_events/trace-events-sample.h | 34 +++++++++++++++----- 11 files changed, 91 insertions(+), 9 deletions(-) -- 2.31.1 diff --git a/include/trace/bpf_probe.h b/include/trace/bpf_probe.h index 6a13220d2d27b..155c495b89ead 100644 --- a/include/trace/bpf_probe.h +++ b/include/trace/bpf_probe.h @@ -21,6 +21,9 @@ #undef __get_bitmask #define __get_bitmask(field) (char *)__get_dynamic_array(field) +#undef __get_cpumask +#define __get_cpumask(field) (char *)__get_dynamic_array(field) + #undef __get_sockaddr #define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field)) @@ -40,6 +43,9 @@ #undef __get_rel_bitmask #define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field) +#undef __get_rel_cpumask +#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field) + #undef __get_rel_sockaddr #define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field)) diff --git a/include/trace/perf.h b/include/trace/perf.h index 5800d13146c3d..8f3bf1e177070 100644 --- a/include/trace/perf.h +++ b/include/trace/perf.h @@ -21,6 +21,9 @@ #undef __get_bitmask #define __get_bitmask(field) (char *)__get_dynamic_array(field) +#undef __get_cpumask +#define __get_cpumask(field) (char *)__get_dynamic_array(field) + #undef __get_sockaddr #define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field)) @@ -41,6 +44,9 @@ #undef __get_rel_bitmask #define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field) +#undef __get_rel_cpumask +#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field) + #undef __get_rel_sockaddr #define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field)) diff --git a/include/trace/stages/stage1_struct_define.h b/include/trace/stages/stage1_struct_define.h index 1b7bab60434c1..69e0dae453bfa 100644 --- a/include/trace/stages/stage1_struct_define.h +++ b/include/trace/stages/stage1_struct_define.h @@ -32,6 +32,9 @@ #undef __bitmask #define __bitmask(item, nr_bits) __dynamic_array(char, item, -1) +#undef __cpumask +#define __cpumask(item) __dynamic_array(char, item, -1) + #undef __sockaddr #define __sockaddr(field, len) __dynamic_array(u8, field, len) @@ -47,6 +50,9 @@ #undef __rel_bitmask #define __rel_bitmask(item, nr_bits) __rel_dynamic_array(char, item, -1) +#undef __rel_cpumask +#define __rel_cpumask(item) __rel_dynamic_array(char, item, -1) + #undef __rel_sockaddr #define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len) diff --git a/include/trace/stages/stage2_data_offsets.h b/include/trace/stages/stage2_data_offsets.h index 1b7a8f764fddd..469b6a64293de 100644 --- a/include/trace/stages/stage2_data_offsets.h +++ b/include/trace/stages/stage2_data_offsets.h @@ -38,6 +38,9 @@ #undef __bitmask #define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1) +#undef __cpumask +#define __cpumask(item) __dynamic_array(unsigned long, item, -1) + #undef __sockaddr #define __sockaddr(field, len) __dynamic_array(u8, field, len) @@ -53,5 +56,8 @@ #undef __rel_bitmask #define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1) +#undef __rel_cpumask +#define __rel_cpumask(item) __rel_dynamic_array(unsigned long, item, -1) + #undef __rel_sockaddr #define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len) diff --git a/include/trace/stages/stage3_trace_output.h b/include/trace/stages/stage3_trace_output.h index e3b183e9d18ea..66374df61ed30 100644 --- a/include/trace/stages/stage3_trace_output.h +++ b/include/trace/stages/stage3_trace_output.h @@ -42,6 +42,9 @@ trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \ }) +#undef __get_cpumask +#define __get_cpumask(field) __get_bitmask(field) + #undef __get_rel_bitmask #define __get_rel_bitmask(field) \ ({ \ @@ -51,6 +54,9 @@ trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \ }) +#undef __get_rel_cpumask +#define __get_rel_cpumask(field) __get_rel_bitmask(field) + #undef __get_sockaddr #define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field)) diff --git a/include/trace/stages/stage4_event_fields.h b/include/trace/stages/stage4_event_fields.h index a8fb25f39a99d..f2990d22313cf 100644 --- a/include/trace/stages/stage4_event_fields.h +++ b/include/trace/stages/stage4_event_fields.h @@ -46,6 +46,9 @@ #undef __bitmask #define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1) +#undef __cpumask +#define __cpumask(item) __dynamic_array(cpumask_t *, item, -1) + #undef __sockaddr #define __sockaddr(field, len) __dynamic_array(u8, field, len) @@ -64,5 +67,8 @@ #undef __rel_bitmask #define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1) +#undef __rel_cpumask +#define __rel_cpumask(item) __rel_dynamic_array(cpumask_t *, item, -1) + #undef __rel_sockaddr #define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len) diff --git a/include/trace/stages/stage5_get_offsets.h b/include/trace/stages/stage5_get_offsets.h index fba4c24ed9e60..ac5c24d3beeb2 100644 --- a/include/trace/stages/stage5_get_offsets.h +++ b/include/trace/stages/stage5_get_offsets.h @@ -82,10 +82,16 @@ #define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, \ __bitmask_size_in_longs(nr_bits)) +#undef __cpumask +#define __cpumask(item) __bitmask(item, nr_cpumask_bits) + #undef __rel_bitmask #define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, \ __bitmask_size_in_longs(nr_bits)) +#undef __rel_cpumask +#define __rel_cpumask(item) __rel_bitmask(item, nr_cpumask_bits) + #undef __sockaddr #define __sockaddr(field, len) __dynamic_array(u8, field, len) diff --git a/include/trace/stages/stage6_event_callback.h b/include/trace/stages/stage6_event_callback.h index 3c554a5853204..49c32394b53fb 100644 --- a/include/trace/stages/stage6_event_callback.h +++ b/include/trace/stages/stage6_event_callback.h @@ -57,6 +57,16 @@ #define __assign_bitmask(dst, src, nr_bits) \ memcpy(__get_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits)) +#undef __cpumask +#define __cpumask(item) __dynamic_array(unsigned long, item, -1) + +#undef __get_cpumask +#define __get_cpumask(field) (char *)__get_dynamic_array(field) + +#undef __assign_cpumask +#define __assign_cpumask(dst, src) \ + memcpy(__get_cpumask(dst), (src), __bitmask_size_in_bytes(nr_cpumask_bits)) + #undef __sockaddr #define __sockaddr(field, len) __dynamic_array(u8, field, len) @@ -98,6 +108,16 @@ #define __assign_rel_bitmask(dst, src, nr_bits) \ memcpy(__get_rel_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits)) +#undef __rel_cpumask +#define __rel_cpumask(item) __rel_dynamic_array(unsigned long, item, -1) + +#undef __get_rel_cpumask +#define __get_rel_cpumask(field) (char *)__get_rel_dynamic_array(field) + +#undef __assign_rel_cpumask +#define __assign_rel_cpumask(dst, src) \ + memcpy(__get_rel_cpumask(dst), (src), __bitmask_size_in_bytes(nr_cpumask_bits)) + #undef __rel_sockaddr #define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len) diff --git a/include/trace/stages/stage7_class_define.h b/include/trace/stages/stage7_class_define.h index 8a7ec24c246dd..8795429f388b0 100644 --- a/include/trace/stages/stage7_class_define.h +++ b/include/trace/stages/stage7_class_define.h @@ -13,11 +13,13 @@ #undef __get_dynamic_array_len #undef __get_str #undef __get_bitmask +#undef __get_cpumask #undef __get_sockaddr #undef __get_rel_dynamic_array #undef __get_rel_dynamic_array_len #undef __get_rel_str #undef __get_rel_bitmask +#undef __get_rel_cpumask #undef __get_rel_sockaddr #undef __print_array #undef __print_hex_dump diff --git a/samples/trace_events/trace-events-sample.c b/samples/trace_events/trace-events-sample.c index 608c4ae3b08a3..ecc7db237f2ef 100644 --- a/samples/trace_events/trace-events-sample.c +++ b/samples/trace_events/trace-events-sample.c @@ -50,7 +50,7 @@ static void do_simple_thread_func(int cnt, const char *fmt, ...) trace_foo_with_template_print("I have to be different", cnt); - trace_foo_rel_loc("Hello __rel_loc", cnt, bitmask); + trace_foo_rel_loc("Hello __rel_loc", cnt, bitmask, current->cpus_ptr); } static void simple_thread_func(int cnt) diff --git a/samples/trace_events/trace-events-sample.h b/samples/trace_events/trace-events-sample.h index 1a92226202fc5..fb4548a44153c 100644 --- a/samples/trace_events/trace-events-sample.h +++ b/samples/trace_events/trace-events-sample.h @@ -200,6 +200,16 @@ * * __assign_bitmask(target_cpus, cpumask_bits(bar), nr_cpumask_bits); * + * __cpumask: This is pretty much the same as __bitmask but is specific for + * CPU masks. The type displayed to the user via the format files will + * be "cpumaks_t" such that user space may deal with them differently + * if they choose to do so, and the bits is always set to nr_cpumask_bits. + * + * __cpumask(target_cpu) + * + * To assign a cpumask, use the __assign_cpumask() helper macro. + * + * __assign_cpumask(target_cpus, cpumask_bits(bar)); * * fast_assign: This is a C like function that is used to store the items * into the ring buffer. A special variable called "__entry" will be the @@ -212,8 +222,8 @@ * This is also used to print out the data from the trace files. * Again, the __entry macro is used to access the data from the ring buffer. * - * Note, __dynamic_array, __string, and __bitmask require special helpers - * to access the data. + * Note, __dynamic_array, __string, __bitmask and __cpumask require special + * helpers to access the data. * * For __dynamic_array(int, foo, bar) use __get_dynamic_array(foo) * Use __get_dynamic_array_len(foo) to get the length of the array @@ -226,6 +236,8 @@ * * For __bitmask(target_cpus, nr_cpumask_bits) use __get_bitmask(target_cpus) * + * For __cpumask(target_cpus) use __get_cpumask(target_cpus) + * * * Note, that for both the assign and the printk, __entry is the handler * to the data structure in the ring buffer, and is defined by the @@ -288,6 +300,7 @@ TRACE_EVENT(foo_bar, __dynamic_array(int, list, __length_of(lst)) __string( str, string ) __bitmask( cpus, num_possible_cpus() ) + __cpumask( cpum ) __vstring( vstr, fmt, va ) ), @@ -299,9 +312,10 @@ TRACE_EVENT(foo_bar, __assign_str(str, string); __assign_vstr(vstr, fmt, va); __assign_bitmask(cpus, cpumask_bits(mask), num_possible_cpus()); + __assign_cpumask(cpum, cpumask_bits(mask)); ), - TP_printk("foo %s %d %s %s %s %s (%s) %s", __entry->foo, __entry->bar, + TP_printk("foo %s %d %s %s %s %s (%s) (%s) %s", __entry->foo, __entry->bar, /* * Notice here the use of some helper functions. This includes: @@ -345,7 +359,8 @@ TRACE_EVENT(foo_bar, __print_array(__get_dynamic_array(list), __get_dynamic_array_len(list) / sizeof(int), sizeof(int)), - __get_str(str), __get_bitmask(cpus), __get_str(vstr)) + __get_str(str), __get_bitmask(cpus), __get_cpumask(cpus), + __get_str(vstr)) ); /* @@ -542,15 +557,16 @@ DEFINE_EVENT_PRINT(foo_template, foo_with_template_print, TRACE_EVENT(foo_rel_loc, - TP_PROTO(const char *foo, int bar, unsigned long *mask), + TP_PROTO(const char *foo, int bar, unsigned long *mask, const cpumask_t *cpus), - TP_ARGS(foo, bar, mask), + TP_ARGS(foo, bar, mask, cpus), TP_STRUCT__entry( __rel_string( foo, foo ) __field( int, bar ) __rel_bitmask( bitmask, BITS_PER_BYTE * sizeof(unsigned long) ) + __rel_cpumask( cpumask ) ), TP_fast_assign( @@ -558,10 +574,12 @@ TRACE_EVENT(foo_rel_loc, __entry->bar = bar; __assign_rel_bitmask(bitmask, mask, BITS_PER_BYTE * sizeof(unsigned long)); + __assign_rel_cpumask(cpumask, cpus); ), - TP_printk("foo_rel_loc %s, %d, %s", __get_rel_str(foo), __entry->bar, - __get_rel_bitmask(bitmask)) + TP_printk("foo_rel_loc %s, %d, %s, %s", __get_rel_str(foo), __entry->bar, + __get_rel_bitmask(bitmask), + __get_rel_cpumask(cpumask)) ); #endif From patchwork Fri Dec 2 15:58:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28949 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp937576wrr; Fri, 2 Dec 2022 08:01:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf5aNNwAcpzbrubGd/A9dfvFmpV59jHQEkoDt+qQaBaaMnc0uBZuzNfU05EpR2D0L1xpuMPA X-Received: by 2002:a63:ce10:0:b0:478:5e53:4742 with SMTP id y16-20020a63ce10000000b004785e534742mr11711395pgf.471.1669996879403; Fri, 02 Dec 2022 08:01:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996879; cv=none; d=google.com; s=arc-20160816; b=RiDGqz31rXu+e/V3N7QR1CHiB9VNz9ZLgbPx6GudSWcWQaAQ9jjb6kFdz/9RO7y1Az 2Qlb3Q82VnrWNZ/ZgA41aZ/fPjxXSRoZtMn3uwLjpiDmduyIIEn17S935yFVOrO2tI37 YFB/mZpnkXjRPb2wHN+lK0mte9br7ISm1QP+xEUkWNwW8QPP33u+ZaKYCW+k99RZSoV2 guoqMDPF6DELd3UglRPYrzkAiNeDwpNbZ4oFpRiIWOcW5+eyUtdxONCgdzrvPCOzLfEe Y1et7ev8WQXd9qA6+8UzR5Oe82R07VXVypwXs9VL8XimFfsBuNjSsf1ygt4vGNXPlmUw cUXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zkIMvc7EhOE64u6GFhS7x1Bfp7Rm8YufbRtsmhjcZ2o=; b=ZlJsziBQj8KAtESLz7n2kCO9Uu9kbe2lbImY7la4bNea9nCHKPdPD8iUBcEzAwyLmx 1A+8S7lvGauYOf0UJiWs45hpU5z190+0YRL4nIHgxXVcuXiiL68VPZ2Dt0LeqJduGz+h 7Tc0qTFoCwTZA+GHInAokLIAnU1UeWqGTWiOtERPzb1stOwz+Ng4FkKPQWL0D8PuQCBv Hf7EieYstPwNTBTaXiShkT3zcEx+6bUl1zWSrdGv2dn1CbytuZ0h5ELUTWxoJ3hp3LSj fuxVpon/qkZpbly1KmSyFGYzkvj965bPohAizl8eaSmTDTkzjS8lsPc4jcTM6hkGY7y6 hgXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=MJmVNIZe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z9-20020a170902834900b00186ae54007dsi6997202pln.499.2022.12.02.08.01.04; Fri, 02 Dec 2022 08:01:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=MJmVNIZe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233751AbiLBQAQ (ORCPT + 99 others); Fri, 2 Dec 2022 11:00:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233806AbiLBQAI (ORCPT ); Fri, 2 Dec 2022 11:00:08 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 718669A4E9 for ; Fri, 2 Dec 2022 07:59:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996747; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zkIMvc7EhOE64u6GFhS7x1Bfp7Rm8YufbRtsmhjcZ2o=; b=MJmVNIZevE+AqqJ4fFkUyd4UzuWzI989NlXVgJLYRtlu0EqdUBm+Mf+W26ML/AVBQfM89D W/YR24XIgf67QLdXzKFNvaWv3boO6gb/PPmUtrzYRbxlFg6Q8PvI2IC9GF8M1zYXh6C3n0 qUJaWVo4qlRz/GYFe5HOPZak4fQICow= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-530-lYV170HcN-CcrO2NjCca_Q-1; Fri, 02 Dec 2022 10:59:04 -0500 X-MC-Unique: lYV170HcN-CcrO2NjCca_Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E4E05833A0E; Fri, 2 Dec 2022 15:59:02 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9BFEE20290A5; Fri, 2 Dec 2022 15:58:58 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: Steven Rostedt , "Paul E. McKenney" , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 2/8] trace: Add trace_ipi_send_cpumask() Date: Fri, 2 Dec 2022 15:58:11 +0000 Message-Id: <20221202155817.2102944-3-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118647792865672?= X-GMAIL-MSGID: =?utf-8?q?1751118647792865672?= trace_ipi_raise() is unsuitable for generically tracing IPI sources due to its "reason" argument being an uninformative string (on arm64 all you get is "Function call interrupts" for SMP calls). Add a variant of it that exports a target CPU, a callsite and a callback. Signed-off-by: Valentin Schneider Reviewed-by: Steven Rostedt (Google) --- include/trace/events/ipi.h | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/include/trace/events/ipi.h b/include/trace/events/ipi.h index 0be71dad6ec03..b1125dc27682c 100644 --- a/include/trace/events/ipi.h +++ b/include/trace/events/ipi.h @@ -35,6 +35,28 @@ TRACE_EVENT(ipi_raise, TP_printk("target_mask=%s (%s)", __get_bitmask(target_cpus), __entry->reason) ); +TRACE_EVENT(ipi_send_cpumask, + + TP_PROTO(const struct cpumask *cpumask, unsigned long callsite, void *callback), + + TP_ARGS(cpumask, callsite, callback), + + TP_STRUCT__entry( + __cpumask(cpumask) + __field(void *, callsite) + __field(void *, callback) + ), + + TP_fast_assign( + __assign_cpumask(cpumask, cpumask_bits(cpumask)); + __entry->callsite = (void *)callsite; + __entry->callback = callback; + ), + + TP_printk("cpumask=%s callsite=%pS callback=%pS", + __get_cpumask(cpumask), __entry->callsite, __entry->callback) +); + DECLARE_EVENT_CLASS(ipi_handler, TP_PROTO(const char *reason), From patchwork Fri Dec 2 15:58:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28952 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp937936wrr; Fri, 2 Dec 2022 08:01:49 -0800 (PST) X-Google-Smtp-Source: AA0mqf5Pe9n9LlHppS+Oyg2crW7t+HDSjJR5eOPpz+Au2RqvynCf5ax6lznn/6aFvmJdinC/QQUW X-Received: by 2002:a17:902:ced1:b0:179:ee31:1527 with SMTP id d17-20020a170902ced100b00179ee311527mr22002260plg.138.1669996908980; Fri, 02 Dec 2022 08:01:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996908; cv=none; d=google.com; s=arc-20160816; b=EjX/yBkFrBr8wh6G57JALia/gHi61ZC3CfLSoNV44ofiUt/qDjKsec8pn9qCzZxMJU wMT0rvCgxrfr9A/o7JEovdtX0tyfUWXWTzTZL8MZeJoBADY6OuK9H8Yj0fVhKGTPkBeQ DnTh/+R8clde+dOvhs8Mv9qDlJAOL3jPioir0AqxEQ5iLM4m/PEL0q8BRMpMgJ5C1ASk 51CQGnRfChjuRsf7INOhmisnD3zzwO6M6xKhV593kKfQGfAtmKiMsHPBB4IvsDptUj/E 9CIo9VjZtTinzmh/cK6NfDIV9UMK7rM68GcP+vYG+xIEBXwibMUg+dD74vnHDTMrhM97 KHjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uzgc0mUISh4RQwmAXJRmvlcbcfqhtS4nRVialK+bVYE=; b=qxhVcxP97Z64vykbP1jszGPIz1EQ/Ve9H4HhaOgV99KQj2FmqZKYH2OAHWjq+fsvqb EZTX/wX9Q10dvZIvMU0PFCqKJqkP4nPOOuIV6zIQmyl7P9nb6LiXmEyJt8/UUbAOKyhR woK38w9rPHXkQW+F6+qnTI9RFzaRGEvs/BLeSK4ey0mnAUqa1F/BxQMWMb05H6/qWwju +M8jeqsTQSTcjLdyhFJST/YThW2KxTtkiiuvdnIDGQ9AAemNEywTZonmfMZNyXklP97s VkFuie1dBEwTH5uy5jC+8dnCsJXG2igaHJ95GZpoi8M29ZoeC4++lUnSFtzbPdhA48sc wxLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=igZd6YV1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i8-20020a635408000000b00476a08c5d87si7187564pgb.713.2022.12.02.08.01.33; Fri, 02 Dec 2022 08:01:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=igZd6YV1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233873AbiLBQBH (ORCPT + 99 others); Fri, 2 Dec 2022 11:01:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233871AbiLBQAZ (ORCPT ); Fri, 2 Dec 2022 11:00:25 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06ED5A9590 for ; Fri, 2 Dec 2022 07:59:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996760; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uzgc0mUISh4RQwmAXJRmvlcbcfqhtS4nRVialK+bVYE=; b=igZd6YV18H1Ry7aBCi1UP9SGrpFFRLa55sEpT8Yjbe1OKZFk0hP4Rd2PMeNLfDWTHUSsl4 0A/X92DqIqeoFm0Qh8dNz4aNWvf4IwhA/4/3a/FuQ4yxOp4Bna41k6pkLY/E3iX4waiWzj foehYRj7AZ5wrkmTQMFF6ob1ApHevVs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-207-bTpaOTExOIuFIHa5yC3C0A-1; Fri, 02 Dec 2022 10:59:10 -0500 X-MC-Unique: bTpaOTExOIuFIHa5yC3C0A-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 708C0833A0F; Fri, 2 Dec 2022 15:59:07 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 255752028E96; Fri, 2 Dec 2022 15:59:03 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: Steven Rostedt , "Paul E. McKenney" , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 3/8] sched, smp: Trace IPIs sent via send_call_function_single_ipi() Date: Fri, 2 Dec 2022 15:58:12 +0000 Message-Id: <20221202155817.2102944-4-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118678706670891?= X-GMAIL-MSGID: =?utf-8?q?1751118678706670891?= send_call_function_single_ipi() is the thing that sends IPIs at the bottom of smp_call_function*() via either generic_exec_single() or smp_call_function_many_cond(). Give it an IPI-related tracepoint. Note that this ends up tracing any IPI sent via __smp_call_single_queue(), which covers __ttwu_queue_wakelist() and irq_work_queue_on() "for free". Signed-off-by: Valentin Schneider Reviewed-by: Steven Rostedt (Google) Acked-by: Ingo Molnar --- arch/arm/kernel/smp.c | 3 --- arch/arm64/kernel/smp.c | 1 - kernel/sched/core.c | 7 +++++-- kernel/smp.c | 4 ++++ 4 files changed, 9 insertions(+), 6 deletions(-) diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index 978db2d96b446..3b280d55c1c40 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -48,9 +48,6 @@ #include #include -#define CREATE_TRACE_POINTS -#include - /* * as from 2.5, kernels no longer have an init_tasks structure * so we need some other way of telling a new secondary core diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index ffc5d76cf6955..937d2623e06ba 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -51,7 +51,6 @@ #include #include -#define CREATE_TRACE_POINTS #include DEFINE_PER_CPU_READ_MOSTLY(int, cpu_number); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index daff72f003858..40587b0d99329 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -81,6 +81,7 @@ #include #include #undef CREATE_TRACE_POINTS +#include #include "sched.h" #include "stats.h" @@ -3746,10 +3747,12 @@ void send_call_function_single_ipi(int cpu) { struct rq *rq = cpu_rq(cpu); - if (!set_nr_if_polling(rq->idle)) + if (!set_nr_if_polling(rq->idle)) { + trace_ipi_send_cpumask(cpumask_of(cpu), _RET_IP_, NULL); arch_send_call_function_single_ipi(cpu); - else + } else { trace_sched_wake_idle_without_ipi(cpu); + } } /* diff --git a/kernel/smp.c b/kernel/smp.c index 06a413987a14a..e2ca1e2f31274 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -26,6 +26,10 @@ #include #include +#define CREATE_TRACE_POINTS +#include +#undef CREATE_TRACE_POINTS + #include "smpboot.h" #include "sched/smp.h" From patchwork Fri Dec 2 15:58:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28951 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp937750wrr; Fri, 2 Dec 2022 08:01:35 -0800 (PST) X-Google-Smtp-Source: AA0mqf5r3N22OyeQv87dIlbtjrVBlerHM32WlDU8bDcU6KUuychhJsb5SsJF72xBgWpv7zRd6/Pk X-Received: by 2002:a17:902:e849:b0:186:ee56:4a1e with SMTP id t9-20020a170902e84900b00186ee564a1emr56728976plg.167.1669996894711; Fri, 02 Dec 2022 08:01:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996894; cv=none; d=google.com; s=arc-20160816; b=TBbnIV2IpXCmMltayc4vmqsVL3giZ+KkTA81Bm6zPm1jbMuPQWWwt4ZainZuwR1a3B zXZNjnJqR+QPrWgP7KTMdc5FVDS4ryIZ9uBIH2NwvZgK/wT3BsfiSlk2usqnPsNcg0Cc 9BjYq7NRPlhm1Sv2Mzm5oZMV8XK3Ld+sN1Temk7rBiuD5fxOj7TNyNHwkX2Zjla7/3gj +BYaIVLQp6p6AGSt5a8R5uPNksGRgugWMimNSsv5u1Zq3jl+1wwc1ZItZZDFVRh0NZdu bOhiejWApfMdcV4L4cmuaHRQL5tcx0ElMLv8zywrqian0FdfY+RwCDngZb5l+8VvxtGG DurA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EP4LqydUSiQVulPj8M4fIZioGJfqcQn9EHXEOk9M9cI=; b=aglNigJOXWndYpitBSMKm6OHWrkDO6dumRWu76AWtEIdcAcXxsGUDGrRgb2YuWciP/ 9fc6oaYGHu1mXvg2Gbvge0czd51khcZZapUclE4OZL2R9r7brRG1NDmzaMjIoH73dDA9 zBA7CCIIKdo5BLUN1dyVsxyTDTucLvMRCjepAxV1bhbwr8ir03ok08YgWp4pP228KHK7 xiLjadF76uepEkpH8MzSZT75eLXOtth2bmGfCo25bCYwOABQNUkWIPQfnm6fw0ip7Lwd xYwP8/mMTrhtJ5l7A868b3fYiZIYmkRSq8fEcFaDx8ciyiKytrL9/wF3Wyyn71uT5nZM s4/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=M1d1bNLB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 71-20020a63014a000000b0047853fba938si7697309pgb.390.2022.12.02.08.01.20; Fri, 02 Dec 2022 08:01:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=M1d1bNLB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233654AbiLBQA5 (ORCPT + 99 others); Fri, 2 Dec 2022 11:00:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233861AbiLBQAV (ORCPT ); Fri, 2 Dec 2022 11:00:21 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D20359A4F0 for ; Fri, 2 Dec 2022 07:59:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EP4LqydUSiQVulPj8M4fIZioGJfqcQn9EHXEOk9M9cI=; b=M1d1bNLBzHanB/uSa16264CESg4qqHh+WirBs1whhtZhP+tQa1OUHVStI0+SDDvowvrY7O sCYTqLJZkC9am2pXS/WSoUAdJ1lBwE39O7lD79pFesF3qZv/io5IKDMcyLWRMn3TKVjLPe 64GYVSKd5aq94vgWGse3XR/t4LXcZdo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-563-Bg0y_EcoMPeHLIEf6idOGw-1; Fri, 02 Dec 2022 10:59:13 -0500 X-MC-Unique: Bg0y_EcoMPeHLIEf6idOGw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BB89085A5A6; Fri, 2 Dec 2022 15:59:11 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A4E3F202903F; Fri, 2 Dec 2022 15:59:07 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: Steven Rostedt , "Paul E. McKenney" , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 4/8] smp: Trace IPIs sent via arch_send_call_function_ipi_mask() Date: Fri, 2 Dec 2022 15:58:13 +0000 Message-Id: <20221202155817.2102944-5-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118663434618047?= X-GMAIL-MSGID: =?utf-8?q?1751118663434618047?= This simply wraps around the arch function and prepends it with a tracepoint, similar to send_call_function_single_ipi(). Signed-off-by: Valentin Schneider Reviewed-by: Steven Rostedt (Google) --- kernel/smp.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/kernel/smp.c b/kernel/smp.c index e2ca1e2f31274..93b4386cd3096 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -160,6 +160,13 @@ void __init call_function_init(void) smpcfd_prepare_cpu(smp_processor_id()); } +static __always_inline void +send_call_function_ipi_mask(const struct cpumask *mask) +{ + trace_ipi_send_cpumask(mask, _RET_IP_, NULL); + arch_send_call_function_ipi_mask(mask); +} + #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG static DEFINE_STATIC_KEY_FALSE(csdlock_debug_enabled); @@ -970,7 +977,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask, if (nr_cpus == 1) send_call_function_single_ipi(last_cpu); else if (likely(nr_cpus > 1)) - arch_send_call_function_ipi_mask(cfd->cpumask_ipi); + send_call_function_ipi_mask(cfd->cpumask_ipi); cfd_seq_store(this_cpu_ptr(&cfd_seq_local)->pinged, this_cpu, CFD_SEQ_NOCPU, CFD_SEQ_PINGED); } From patchwork Fri Dec 2 15:58:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28953 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp937984wrr; Fri, 2 Dec 2022 08:01:52 -0800 (PST) X-Google-Smtp-Source: AA0mqf6kp6x3WYTQcX5CAdI+Mz/nXlyNmnqFNWsJ2c6MEO9TI5E0kOffIL4TNl41QJibCPkSPB+F X-Received: by 2002:a05:6a00:21c2:b0:56b:bba4:650a with SMTP id t2-20020a056a0021c200b0056bbba4650amr53335461pfj.4.1669996911890; Fri, 02 Dec 2022 08:01:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996911; cv=none; d=google.com; s=arc-20160816; b=NYR6B0LhRqRMPszhJIKwJOGejKdtkpkOjW+pnKsPXJGNdUKhLOTn4hSXjsi4xMrQsI PjhCFZ7Cds35iGFGumfbVj9KGDUmszVkpp6qOSPru1aU0enFdp8H9Mr9JLyyZpohr8u5 J27qWdGWww/dHSDUFBChtAbeynGLQ5x4QN7uiGTQFWE55flw6N3VsleN7sq+XLocA5CK rMIFkDHu+8HikOKu/59m5N/rqOTxEdHbmOxenZO2ZchD3aqbL8zxYTY3I3nMeLo5Gg8p B3/ajBfN4OkL9WnqDgjHmUR915Z/yaszm2/cT0753i95L3kM83g2UiDvsblBmSUrvclN Tidw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6o4aDoMnOyw6oIKqxXD/63VXzreRgu7lohjEhFx0NNg=; b=FTawqG5tHwrpxiNOOGNEstC4hFQ2mFWPjNq5o3hmUryh+Mp+HVoZToL910HIedzhbe wrgcF9sdK6gEswICzLZ1E3CHjbufovlUJVupve8mgaDCHrACYDhcV6f8eWr2xCoNQdRB R0yK2CGTbIfzFU2ncNPnoSZlriIhu4av7vKyObRf+5e7Vkj70GgAueeURLwF823QrIOc vyTvHiZhxEcQtA9qWQ2OOJJZz1/YzUMRWUcfcCojgPs0lerG6/YviYYkJPieImy8qssL PlUDbP05f4hwQh4kTYXjQL342Aj0e8xQa57z02ZTtcRGtJ1+SY//QOrybAI5FEMDHWJS h8Ag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=OzouMb8y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w7-20020a056a0014c700b0056aafb43783si8479751pfu.155.2022.12.02.08.01.37; Fri, 02 Dec 2022 08:01:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=OzouMb8y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233884AbiLBQBL (ORCPT + 99 others); Fri, 2 Dec 2022 11:01:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233878AbiLBQA0 (ORCPT ); Fri, 2 Dec 2022 11:00:26 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4E09D1581 for ; Fri, 2 Dec 2022 07:59:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996762; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6o4aDoMnOyw6oIKqxXD/63VXzreRgu7lohjEhFx0NNg=; b=OzouMb8yq8tuKd2KsA6W7WeRnK4mbl9f7FSgRmjVtOg6F4VOm7fsy58Kqc4ISmSeFBGYkH 27AWcdnSkk6dWrFw0yx78L6k+PwZexhTSpk0eG7THjQBtPHNVbUN6/KRf2qChlW8LA7Dj3 pvf6mPoHKPBXBXR6TBIwR1HRpaAYfR4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-594-Okj2luc0MBCEEYR4jqlogg-1; Fri, 02 Dec 2022 10:59:19 -0500 X-MC-Unique: Okj2luc0MBCEEYR4jqlogg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 92E14101A528; Fri, 2 Dec 2022 15:59:17 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EAE2920290A5; Fri, 2 Dec 2022 15:59:11 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: Steven Rostedt , "Paul E. McKenney" , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 5/8] irq_work: Trace self-IPIs sent via arch_irq_work_raise() Date: Fri, 2 Dec 2022 15:58:14 +0000 Message-Id: <20221202155817.2102944-6-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118681765472493?= X-GMAIL-MSGID: =?utf-8?q?1751118681765472493?= IPIs sent to remote CPUs via irq_work_queue_on() are now covered by trace_ipi_send_cpumask(), add another instance of the tracepoint to cover self-IPIs. Signed-off-by: Valentin Schneider Reviewed-by: Steven Rostedt (Google) --- kernel/irq_work.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/kernel/irq_work.c b/kernel/irq_work.c index 7afa40fe5cc43..c33e88e32a67a 100644 --- a/kernel/irq_work.c +++ b/kernel/irq_work.c @@ -22,6 +22,8 @@ #include #include +#include + static DEFINE_PER_CPU(struct llist_head, raised_list); static DEFINE_PER_CPU(struct llist_head, lazy_list); static DEFINE_PER_CPU(struct task_struct *, irq_workd); @@ -74,6 +76,16 @@ void __weak arch_irq_work_raise(void) */ } +static __always_inline void irq_work_raise(struct irq_work *work) +{ + if (trace_ipi_send_cpumask_enabled() && arch_irq_work_has_interrupt()) + trace_ipi_send_cpumask(cpumask_of(smp_processor_id()), + _RET_IP_, + work->func); + + arch_irq_work_raise(); +} + /* Enqueue on current CPU, work must already be claimed and preempt disabled */ static void __irq_work_queue_local(struct irq_work *work) { @@ -99,7 +111,7 @@ static void __irq_work_queue_local(struct irq_work *work) /* If the work is "lazy", handle it from next tick if any */ if (!lazy_work || tick_nohz_tick_stopped()) - arch_irq_work_raise(); + irq_work_raise(work); } /* Enqueue the irq work @work on the current CPU */ From patchwork Fri Dec 2 15:58:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28956 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp938465wrr; Fri, 2 Dec 2022 08:02:32 -0800 (PST) X-Google-Smtp-Source: AA0mqf60IAXUGO1GEbjJRdkqMxBwcXnY2ZsIT+1NY5kkIw5G6MJvjofHt/N/CraXF/507296e0Dx X-Received: by 2002:a17:906:c24e:b0:778:d81d:b1f0 with SMTP id bl14-20020a170906c24e00b00778d81db1f0mr60914009ejb.731.1669996952713; Fri, 02 Dec 2022 08:02:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996952; cv=none; d=google.com; s=arc-20160816; b=hwrFepCNC4wIsbwX/zez4AhRCnEdpcy9gKvn58RvRwjKuADXQ00hGt/X4vrawgkJfE RQK0kyA+ctejhn1JmSn2OIHkh/u6VMeg/8vlGbWsfg030fn8SCdDkbNtLWKr9NkhLbKm EJ4oQaraZ4TF/lw6EH133n5w6q556s21xC3JKyXMU9rDDJ/mgJifyK3mMwMon+kNlRpA 7Uikxq1EbSWMwbpwI0OZ+Y5+G8AQMGHjOEKP7Wu/1VlXfq3QoxkxfqUklgO4L5HcMywO RrbbJzajLsh6zn5WgArbQyVvWsrblKxK8YjF7oO0o1QQcFVJ0smvsXPMU1usW42LJz7G bp4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=S0rn6ibspmsa/u6Ppq7nR0uXkFaRdafaQGMWAeCDkWQ=; b=XGLSMk/PUfTad40I573Yk7/VG68USjqEmguIHUwB43lTo4AayV61EfQ0ItTHFU6iGE sYceRkHJenxhz9J3uBG/1LHgM2kzDQSOe/0wV32Ft0bhuvsVv9oVlYmMXQeAtDrVRd8R nesgyMblInVKL3rLNNuNr3ypvdlx9nKCxPlR2DZnvUq2rD7uP6DUFUDQQq8l22eDd1KF ptRURwBG4Bo45waeikGqxMKL0iiCh2oc/AtlLiixVgGMZLLPt5qZaQ26jBVBUz2/qpxW RSFCS58RWfbzXP+REK4dZWUHOFsVx/RGLPHAqobDbml4GWmisPcoy2TZDy+i0qu91REZ C4+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QeoutyHO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jg36-20020a170907972400b00787b9cd2fe3si6558405ejc.844.2022.12.02.08.02.06; Fri, 02 Dec 2022 08:02:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QeoutyHO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233962AbiLBQB1 (ORCPT + 99 others); Fri, 2 Dec 2022 11:01:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233892AbiLBQAb (ORCPT ); Fri, 2 Dec 2022 11:00:31 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF859DF610 for ; Fri, 2 Dec 2022 07:59:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S0rn6ibspmsa/u6Ppq7nR0uXkFaRdafaQGMWAeCDkWQ=; b=QeoutyHOEAx1DOpk93EHULKfNdAWA9WtrpM3nSiwjadhaH0aag1+UM4UOQmub7ng1/FoRF ftPzEB08Q3Ika2IYzCdQCtbIS0YbXAle52MjbmYZs9SJc5amuFg4kWfBFrKrkaD4SajLXB +crn5Zoyadd3rbb2KKE5tsPQ5qMgQHQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-126-k2L3t5y8Nr-GYd2qSxoR8w-1; Fri, 02 Dec 2022 10:59:23 -0500 X-MC-Unique: k2L3t5y8Nr-GYd2qSxoR8w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 405EB800B23; Fri, 2 Dec 2022 15:59:22 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D1AEE2028E96; Fri, 2 Dec 2022 15:59:17 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: Guo Ren , "Paul E. McKenney" , Steven Rostedt , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , "David S. Miller" Subject: [PATCH v3 6/8] treewide: Trace IPIs sent via smp_send_reschedule() Date: Fri, 2 Dec 2022 15:58:15 +0000 Message-Id: <20221202155817.2102944-7-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118724896585346?= X-GMAIL-MSGID: =?utf-8?q?1751118724896585346?= To be able to trace invocations of smp_send_reschedule(), rename the arch-specific definitions of it to arch_smp_send_reschedule() and wrap it into an smp_send_reschedule() that contains a tracepoint. Changes to include the declaration of the tracepoint were driven by the following coccinelle script: @func_use@ @@ smp_send_reschedule(...); @include@ @@ #include @no_include depends on func_use && !include@ @@ #include <...> + + #include Signed-off-by: Valentin Schneider [csky bits] Acked-by: Guo Ren --- arch/alpha/kernel/smp.c | 2 +- arch/arc/kernel/smp.c | 2 +- arch/arm/kernel/smp.c | 2 +- arch/arm/mach-actions/platsmp.c | 2 ++ arch/arm64/kernel/smp.c | 2 +- arch/csky/kernel/smp.c | 2 +- arch/hexagon/kernel/smp.c | 2 +- arch/ia64/kernel/smp.c | 4 ++-- arch/loongarch/include/asm/smp.h | 2 +- arch/mips/include/asm/smp.h | 2 +- arch/mips/kernel/rtlx-cmp.c | 2 ++ arch/openrisc/kernel/smp.c | 2 +- arch/parisc/kernel/smp.c | 4 ++-- arch/powerpc/kernel/smp.c | 6 ++++-- arch/powerpc/kvm/book3s_hv.c | 3 +++ arch/powerpc/platforms/powernv/subcore.c | 2 ++ arch/riscv/kernel/smp.c | 4 ++-- arch/s390/kernel/smp.c | 2 +- arch/sh/kernel/smp.c | 2 +- arch/sparc/kernel/smp_32.c | 2 +- arch/sparc/kernel/smp_64.c | 2 +- arch/x86/include/asm/smp.h | 2 +- arch/x86/kvm/svm/svm.c | 4 ++++ arch/x86/kvm/x86.c | 2 ++ arch/xtensa/kernel/smp.c | 2 +- include/linux/smp.h | 8 ++++++-- virt/kvm/kvm_main.c | 1 + 27 files changed, 47 insertions(+), 25 deletions(-) diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c index f4e20f75438f8..38637eb9eebd5 100644 --- a/arch/alpha/kernel/smp.c +++ b/arch/alpha/kernel/smp.c @@ -562,7 +562,7 @@ handle_ipi(struct pt_regs *regs) } void -smp_send_reschedule(int cpu) +arch_smp_send_reschedule(int cpu) { #ifdef DEBUG_IPI_MSG if (cpu == hard_smp_processor_id()) diff --git a/arch/arc/kernel/smp.c b/arch/arc/kernel/smp.c index ad93fe6e4b77d..409cfa4675b40 100644 --- a/arch/arc/kernel/smp.c +++ b/arch/arc/kernel/smp.c @@ -292,7 +292,7 @@ static void ipi_send_msg(const struct cpumask *callmap, enum ipi_msg_type msg) ipi_send_msg_one(cpu, msg); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { ipi_send_msg_one(cpu, IPI_RESCHEDULE); } diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index 3b280d55c1c40..f216ac890b6f9 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -745,7 +745,7 @@ void __init set_smp_ipi_range(int ipi_base, int n) ipi_setup(smp_processor_id()); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/arch/arm/mach-actions/platsmp.c b/arch/arm/mach-actions/platsmp.c index f26618b435145..7b208e96fbb67 100644 --- a/arch/arm/mach-actions/platsmp.c +++ b/arch/arm/mach-actions/platsmp.c @@ -20,6 +20,8 @@ #include #include +#include + #define OWL_CPU1_ADDR 0x50 #define OWL_CPU1_FLAG 0x5c diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 937d2623e06ba..8d108edc4a89f 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -976,7 +976,7 @@ void __init set_smp_ipi_range(int ipi_base, int n) ipi_setup(smp_processor_id()); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/arch/csky/kernel/smp.c b/arch/csky/kernel/smp.c index 4b605aa2e1d65..fd7f81be16dd6 100644 --- a/arch/csky/kernel/smp.c +++ b/arch/csky/kernel/smp.c @@ -140,7 +140,7 @@ void smp_send_stop(void) on_each_cpu(ipi_stop, NULL, 1); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/arch/hexagon/kernel/smp.c b/arch/hexagon/kernel/smp.c index 4ba93e59370c4..4e8bee25b8c68 100644 --- a/arch/hexagon/kernel/smp.c +++ b/arch/hexagon/kernel/smp.c @@ -217,7 +217,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus) } } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { send_ipi(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/arch/ia64/kernel/smp.c b/arch/ia64/kernel/smp.c index e2cc59db86bc2..ea4f009a232b4 100644 --- a/arch/ia64/kernel/smp.c +++ b/arch/ia64/kernel/smp.c @@ -220,11 +220,11 @@ kdump_smp_send_init(void) * Called with preemption disabled. */ void -smp_send_reschedule (int cpu) +arch_smp_send_reschedule (int cpu) { ia64_send_ipi(cpu, IA64_IPI_RESCHEDULE, IA64_IPI_DM_INT, 0); } -EXPORT_SYMBOL_GPL(smp_send_reschedule); +EXPORT_SYMBOL_GPL(arch_smp_send_reschedule); /* * Called with preemption disabled. diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h index 3dd172d9ffea0..99f6f6d51d33a 100644 --- a/arch/loongarch/include/asm/smp.h +++ b/arch/loongarch/include/asm/smp.h @@ -83,7 +83,7 @@ extern void show_ipi_list(struct seq_file *p, int prec); * it goes straight through and wastes no time serializing * anything. Worst case is that we lose a reschedule ... */ -static inline void smp_send_reschedule(int cpu) +static inline void arch_smp_send_reschedule(int cpu) { loongson_send_ipi_single(cpu, SMP_RESCHEDULE); } diff --git a/arch/mips/include/asm/smp.h b/arch/mips/include/asm/smp.h index 5d9ff61004ca7..9806e79895d99 100644 --- a/arch/mips/include/asm/smp.h +++ b/arch/mips/include/asm/smp.h @@ -66,7 +66,7 @@ extern void calculate_cpu_foreign_map(void); * it goes straight through and wastes no time serializing * anything. Worst case is that we lose a reschedule ... */ -static inline void smp_send_reschedule(int cpu) +static inline void arch_smp_send_reschedule(int cpu) { extern const struct plat_smp_ops *mp_ops; /* private */ diff --git a/arch/mips/kernel/rtlx-cmp.c b/arch/mips/kernel/rtlx-cmp.c index d26dcc4b46e74..e991cc936c1cd 100644 --- a/arch/mips/kernel/rtlx-cmp.c +++ b/arch/mips/kernel/rtlx-cmp.c @@ -17,6 +17,8 @@ #include #include +#include + static int major; static void rtlx_interrupt(void) diff --git a/arch/openrisc/kernel/smp.c b/arch/openrisc/kernel/smp.c index e1419095a6f0a..0a7a059e2dff4 100644 --- a/arch/openrisc/kernel/smp.c +++ b/arch/openrisc/kernel/smp.c @@ -173,7 +173,7 @@ void handle_IPI(unsigned int ipi_msg) } } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c index 7dbd92cafae38..b7fc859fa87db 100644 --- a/arch/parisc/kernel/smp.c +++ b/arch/parisc/kernel/smp.c @@ -246,8 +246,8 @@ void kgdb_roundup_cpus(void) inline void smp_send_stop(void) { send_IPI_allbutself(IPI_CPU_STOP); } -void -smp_send_reschedule(int cpu) { send_IPI_single(cpu, IPI_RESCHEDULE); } +void +arch_smp_send_reschedule(int cpu) { send_IPI_single(cpu, IPI_RESCHEDULE); } void smp_send_all_nop(void) diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c index 0da6e59161cd4..11cfa76fd3699 100644 --- a/arch/powerpc/kernel/smp.c +++ b/arch/powerpc/kernel/smp.c @@ -61,6 +61,8 @@ #include #include +#include + #ifdef DEBUG #include #define DBG(fmt...) udbg_printf(fmt) @@ -364,12 +366,12 @@ static inline void do_message_pass(int cpu, int msg) #endif } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { if (likely(smp_ops)) do_message_pass(cpu, PPC_MSG_RESCHEDULE); } -EXPORT_SYMBOL_GPL(smp_send_reschedule); +EXPORT_SYMBOL_GPL(arch_smp_send_reschedule); void arch_send_call_function_single_ipi(int cpu) { diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 6ba68dd6190bd..3b70b5f80bd56 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include @@ -80,6 +81,8 @@ #include #include +#include + #include "book3s.h" #include "book3s_hv.h" diff --git a/arch/powerpc/platforms/powernv/subcore.c b/arch/powerpc/platforms/powernv/subcore.c index 7e98b00ea2e84..c53c4c7977680 100644 --- a/arch/powerpc/platforms/powernv/subcore.c +++ b/arch/powerpc/platforms/powernv/subcore.c @@ -20,6 +20,8 @@ #include #include +#include + #include "subcore.h" #include "powernv.h" diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c index 760a64518c585..213602e89a8b2 100644 --- a/arch/riscv/kernel/smp.c +++ b/arch/riscv/kernel/smp.c @@ -235,8 +235,8 @@ void smp_send_stop(void) cpumask_pr_args(cpu_online_mask)); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { send_ipi_single(cpu, IPI_RESCHEDULE); } -EXPORT_SYMBOL_GPL(smp_send_reschedule); +EXPORT_SYMBOL_GPL(arch_smp_send_reschedule); diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c index 0031325ce4bc9..6c4da1e26e568 100644 --- a/arch/s390/kernel/smp.c +++ b/arch/s390/kernel/smp.c @@ -553,7 +553,7 @@ void arch_send_call_function_single_ipi(int cpu) * it goes straight through and wastes no time serializing * anything. Worst case is that we lose a reschedule ... */ -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { pcpu_ec_call(pcpu_devices + cpu, ec_schedule); } diff --git a/arch/sh/kernel/smp.c b/arch/sh/kernel/smp.c index 65924d9ec2459..5cf35a774dc70 100644 --- a/arch/sh/kernel/smp.c +++ b/arch/sh/kernel/smp.c @@ -256,7 +256,7 @@ void __init smp_cpus_done(unsigned int max_cpus) (bogosum / (5000/HZ)) % 100); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { mp_ops->send_ipi(cpu, SMP_MSG_RESCHEDULE); } diff --git a/arch/sparc/kernel/smp_32.c b/arch/sparc/kernel/smp_32.c index ad8094d955eba..87eaa7719fa27 100644 --- a/arch/sparc/kernel/smp_32.c +++ b/arch/sparc/kernel/smp_32.c @@ -120,7 +120,7 @@ void cpu_panic(void) struct linux_prom_registers smp_penguin_ctable = { 0 }; -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { /* * CPU model dependent way of implementing IPI generation targeting diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c index a55295d1b9244..e5964d1d8b37d 100644 --- a/arch/sparc/kernel/smp_64.c +++ b/arch/sparc/kernel/smp_64.c @@ -1430,7 +1430,7 @@ static unsigned long send_cpu_poke(int cpu) return hv_err; } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { if (cpu == smp_processor_id()) { WARN_ON_ONCE(preemptible()); diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index a73bced40e241..5ff5815149bd3 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -99,7 +99,7 @@ static inline void play_dead(void) smp_ops.play_dead(); } -static inline void smp_send_reschedule(int cpu) +static inline void arch_smp_send_reschedule(int cpu) { smp_ops.smp_send_reschedule(cpu); } diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index ce362e88a5676..cfc622d8fc2d8 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -40,6 +41,9 @@ #include #include + +#include + #include "trace.h" #include "svm.h" diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2835bd7966391..4c2baf8090bdc 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -58,7 +58,9 @@ #include #include #include +#include +#include #include #include diff --git a/arch/xtensa/kernel/smp.c b/arch/xtensa/kernel/smp.c index 4dc109dd6214e..d95907b8e4d38 100644 --- a/arch/xtensa/kernel/smp.c +++ b/arch/xtensa/kernel/smp.c @@ -389,7 +389,7 @@ void arch_send_call_function_single_ipi(int cpu) send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC); } -void smp_send_reschedule(int cpu) +void arch_smp_send_reschedule(int cpu) { send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/include/linux/smp.h b/include/linux/smp.h index a80ab58ae3f1d..96836af927893 100644 --- a/include/linux/smp.h +++ b/include/linux/smp.h @@ -125,8 +125,12 @@ extern void smp_send_stop(void); /* * sends a 'reschedule' event to another CPU: */ -extern void smp_send_reschedule(int cpu); - +extern void arch_smp_send_reschedule(int cpu); +#define smp_send_reschedule(cpu) ({ \ + /* XXX scheduler_ipi is inline :/ */ \ + trace_ipi_send_cpumask(cpumask_of(cpu), _RET_IP_, NULL); \ + arch_smp_send_reschedule(cpu); \ +}) /* * Prepare machine for booting other CPUs. diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index fab4d37905785..a1fc2c947edfe 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -62,6 +62,7 @@ #include "kvm_mm.h" #include "vfio.h" +#include #define CREATE_TRACE_POINTS #include From patchwork Fri Dec 2 15:58:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28955 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp938373wrr; Fri, 2 Dec 2022 08:02:25 -0800 (PST) X-Google-Smtp-Source: AA0mqf4BvpeBHndzplPe4T+eyjkBQyJQyH5aAL7jjvgJjxvWWpGIXJ8Hr18zdeoEjUcwz5XQixqU X-Received: by 2002:a17:903:1311:b0:189:1e5:8422 with SMTP id iy17-20020a170903131100b0018901e58422mr53304078plb.17.1669996944741; Fri, 02 Dec 2022 08:02:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996944; cv=none; d=google.com; s=arc-20160816; b=O2hOdma9OKRZKAmatDfvx5OzQrJB68B2y1VeKvlmGBZbSm7zoXrHo0BBDMffxtI3mV Bk0KWM7CMsMksQyiyBuZvBSrOH1PfFoBcZe32ttaxH9mGzQfbB1bFuDbdfmdn6YTfYeh rtTYGJYQCrtmdH3a5ss/u+TAC6L3T4hAjhgFKSY0J/XDbo6p0NcIbCqv8ivZwuGZWgA9 LVihwowF7OD7TVPRtlk6AlIzSOFl9IrRd6Crkco6jOqHwv6R17wNcHbaGar3V8u3L5hW RMJTchDoCj1mDluwpmeYCjx40zKzEYdfA6I6k2WzAOpU20zRVfxSwrZaGjmqodRlzs9b 9dLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QbSxuBYwyBhZhp7hgOeOy88U7e5YyI7vjVO9jj/5+zE=; b=fkgCYa0JxoKzBS/IuKTVzKAE6ahQ2Ce5TfH5qwYNXbS4dqceRP+P2d2RWepWuN9Yzn mobStAkrJ0iXqTFhC8SXFvxVmfjlBtOunmn7oHVLuGVSQJfUZ4gi1syypa/s9JD+x0l8 twQ2dPvfVachkeh6A5lvO+fs9RhqxPOyhRWPsEtW8TrdkejVCC+M+i5W2gBoTOup9Osg MS1ggqaFaRUYemJUIyemDzhivfl48hdQ2S2ijuUaUBKcyagwUEUtTjU9+I4YoV6qPkpf ml6xjOM7fhUbIzZM/XfxqSDqu/LtzhGt/KSAamGpLBgPvaVgYg6LBrawVHrDQYpsO2AY i0nA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=g1+hmoe+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p34-20020a635b22000000b00473c36e3b75si7431424pgb.608.2022.12.02.08.02.10; Fri, 02 Dec 2022 08:02:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=g1+hmoe+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233896AbiLBQBv (ORCPT + 99 others); Fri, 2 Dec 2022 11:01:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233850AbiLBQAw (ORCPT ); Fri, 2 Dec 2022 11:00:52 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 424C2DF63C for ; Fri, 2 Dec 2022 07:59:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QbSxuBYwyBhZhp7hgOeOy88U7e5YyI7vjVO9jj/5+zE=; b=g1+hmoe+IQ5Q25/+4K3+hynisY5BWNqHkSMiOAqrBkTKMuIl0/5cDxZxVx5/BrXdYm7okt RiJZi1L4JaSuwtl2qKGD5h8+n7yDVsKtnaoviNFiRW5nXqOTdIm5IPgu9SuEm+o8XeGEbn BNvPqQZzn1zhVKSA7cDOXoxgb/jTqCk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-643-PZIbC1qRM0uYzAeFKyhbPQ-1; Fri, 02 Dec 2022 10:59:28 -0500 X-MC-Unique: PZIbC1qRM0uYzAeFKyhbPQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62F0E833A06; Fri, 2 Dec 2022 15:59:26 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 712A220290A5; Fri, 2 Dec 2022 15:59:22 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: "Paul E. McKenney" , Steven Rostedt , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 7/8] smp: reword smp call IPI comment Date: Fri, 2 Dec 2022 15:58:16 +0000 Message-Id: <20221202155817.2102944-8-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118716450602333?= X-GMAIL-MSGID: =?utf-8?q?1751118716450602333?= Accessing the call_single_queue hasn't involved a spinlock since 2014: 6897fc22ea01 ("kernel: use lockless list for smp_call_function_single") The llist operations (namely cmpxchg() and xchg()) provide similar ordering guarantees, update the comment to lessen confusion. Signed-off-by: Valentin Schneider --- kernel/smp.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/kernel/smp.c b/kernel/smp.c index 93b4386cd3096..821b5986721ac 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -495,9 +495,10 @@ void __smp_call_single_queue(int cpu, struct llist_node *node) #endif /* - * The list addition should be visible before sending the IPI - * handler locks the list to pull the entry off it because of - * normal cache coherency rules implied by spinlocks. + * The list addition should be visible to the target CPU when it pops + * the head of the list to pull the entry off it in the IPI handler + * because of normal cache coherency rules implied by the underlying + * llist ops. * * If IPIs can go out of order to the cache coherency protocol * in an architecture, sufficient synchronisation should be added From patchwork Fri Dec 2 15:58:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 28957 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp938667wrr; Fri, 2 Dec 2022 08:02:50 -0800 (PST) X-Google-Smtp-Source: AA0mqf6iTxJtVoF4O/GIqGXXz9aAmgskOoy0WBe8BOjkbpZA5HzA9vre+kfb13tSLlpiVDxWrDcp X-Received: by 2002:a17:903:100c:b0:189:a682:c5a6 with SMTP id a12-20020a170903100c00b00189a682c5a6mr15539749plb.78.1669996970177; Fri, 02 Dec 2022 08:02:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669996970; cv=none; d=google.com; s=arc-20160816; b=HAZKxTvCRbilIwK/vVXlOPGSpKYrPZ0NgNJYnF1ne7Pvrjwg6d6uXIC2IfTyTiCtdC OOjmZbiCFQ3gQRocxKV1StwB9xRZ9BU1pclByshQffoPJxiNohgZcnGhzkEIpxs+1lRJ RVe0bNpdDz1Cmo50Vh+ClUod1TJIj0Stn/oegP/rabf+oq0zji/ILlh+GMzW6yOQFnDX 4q23huo6sRg2kxgn92TKqyUSI0Pgt6pQ5R/FZMhP/eVu9DdpXyjq8cF2nLD+uVtsAfub z7V6e6qvMxhmBGiIRgvfi31HYq7vmiHcfZPaGIsULk6vYlYuxw3mUVQ0Xx0sCc9E6CxH UIqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=imKWtJjnLxywPeyj1k0cz/LH0f5N0UttG38/bDrYO7s=; b=tGwBmKFmT7X6nNbseq6pGEPxDbaniPFgT2MCcWfJb2WcBhZ1u2A+g50B87cbHOL8Db ULY3IOnfhnpDpm/bIK+bnuf1ltfSjvoXuDJI9qsyv5MRQScs02w1oQL0GdNLVKOMV1SO J36fR5BZ+ihOxssmITdnpakVxLceeOkoMbUfW2kX0TLRk5xphgKB67kdW6N7e3sc7QEd a08lj9pQ1EffZY452B3WKe7mxj+wiYk37c+nDR4KsrsS5TDb3bU3NXrxC+W4ik0Ct3LX n+LqWfrVzGntI6evko1jDqio0icOyqE6FeGSmYDd8D0HJSlkuX1mM7QChvVjculYAbGz mzHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Iga6Wad8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b10-20020a6541ca000000b0047769403088si7731971pgq.627.2022.12.02.08.02.35; Fri, 02 Dec 2022 08:02:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Iga6Wad8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233933AbiLBQCG (ORCPT + 99 others); Fri, 2 Dec 2022 11:02:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233882AbiLBQBK (ORCPT ); Fri, 2 Dec 2022 11:01:10 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 696E2E51C9 for ; Fri, 2 Dec 2022 07:59:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669996775; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=imKWtJjnLxywPeyj1k0cz/LH0f5N0UttG38/bDrYO7s=; b=Iga6Wad8+eHNc05Vsjk66LXzmEpfA2SOy6MZuxXqD88tkTBhbYnrxco9lzX7o+CDoPvl+s 7dPmchRbagGzh5sxiWLSq/2/AAH9Ivg30ioujnx+CYnaCpFyTcufj/PZ7pEyjdNNAgUnS9 lbgGlmmE4CvuhoH/DmaIT/GBEN6CVas= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-232-AJnbpNZiOu2OJHlZZZSQxA-1; Fri, 02 Dec 2022 10:59:32 -0500 X-MC-Unique: AJnbpNZiOu2OJHlZZZSQxA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1D316833A09; Fri, 2 Dec 2022 15:59:31 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.77]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AD20020290A5; Fri, 2 Dec 2022 15:59:26 +0000 (UTC) From: Valentin Schneider To: linux-alpha@vger.kernel.org, linux-kernel@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, openrisc@lists.librecores.org, linux-parisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, x86@kernel.org Cc: "Paul E. McKenney" , Steven Rostedt , Peter Zijlstra , Thomas Gleixner , Sebastian Andrzej Siewior , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Frederic Weisbecker , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Marc Zyngier , Mark Rutland , Russell King , Nicholas Piggin , Guo Ren , "David S. Miller" Subject: [PATCH v3 8/8] sched, smp: Trace smp callback causing an IPI Date: Fri, 2 Dec 2022 15:58:17 +0000 Message-Id: <20221202155817.2102944-9-vschneid@redhat.com> In-Reply-To: <20221202155817.2102944-1-vschneid@redhat.com> References: <20221202155817.2102944-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751118743283838408?= X-GMAIL-MSGID: =?utf-8?q?1751118743283838408?= Context ======= The newly-introduced ipi_send_cpumask tracepoint has a "callback" parameter which so far has only been fed with NULL. While CSD_TYPE_SYNC/ASYNC and CSD_TYPE_IRQ_WORK share a similar backing struct layout (meaning their callback func can be accessed without caring about the actual CSD type), CSD_TYPE_TTWU doesn't even have a function attached to its struct. This means we need to check the type of a CSD before eventually dereferencing its associated callback. This isn't as trivial as it sounds: the CSD type is stored in __call_single_node.u_flags, which get cleared right before the callback is executed via csd_unlock(). This implies checking the CSD type before it is enqueued on the call_single_queue, as the target CPU's queue can be flushed before we get to sending an IPI. Furthermore, send_call_function_single_ipi() only has a CPU parameter, and would need to have an additional argument to trickle down the invoked function. This is somewhat silly, as the extra argument will always be pushed down to the function even when nothing is being traced, which is unnecessary overhead. Changes ======= send_call_function_single_ipi() is only used by smp.c, and is defined in sched/core.c as it contains scheduler-specific ops (set_nr_if_polling() of a CPU's idle task). Split it into two parts: the scheduler bits remain in sched/core.c, and the actual IPI emission is moved into smp.c. This lets us define an __always_inline helper function that can take the related callback as parameter without creating useless register pressure in the non-traced path which only gains a (disabled) static branch. Do the same thing for the multi IPI case. Signed-off-by: Valentin Schneider --- kernel/sched/core.c | 18 +++++++----- kernel/sched/smp.h | 2 +- kernel/smp.c | 72 +++++++++++++++++++++++++++++++++------------ 3 files changed, 66 insertions(+), 26 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 40587b0d99329..e59aac936dcb9 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3743,16 +3743,20 @@ void sched_ttwu_pending(void *arg) rq_unlock_irqrestore(rq, &rf); } -void send_call_function_single_ipi(int cpu) +/* + * Prepare the scene for sending an IPI for a remote smp_call + * + * Returns true if the caller can proceed with sending the IPI. + * Returns false otherwise. + */ +bool call_function_single_prep_ipi(int cpu) { - struct rq *rq = cpu_rq(cpu); - - if (!set_nr_if_polling(rq->idle)) { - trace_ipi_send_cpumask(cpumask_of(cpu), _RET_IP_, NULL); - arch_send_call_function_single_ipi(cpu); - } else { + if (set_nr_if_polling(cpu_rq(cpu)->idle)) { trace_sched_wake_idle_without_ipi(cpu); + return false; } + + return true; } /* diff --git a/kernel/sched/smp.h b/kernel/sched/smp.h index 2eb23dd0f2856..21ac44428bb02 100644 --- a/kernel/sched/smp.h +++ b/kernel/sched/smp.h @@ -6,7 +6,7 @@ extern void sched_ttwu_pending(void *arg); -extern void send_call_function_single_ipi(int cpu); +extern bool call_function_single_prep_ipi(int cpu); #ifdef CONFIG_SMP extern void flush_smp_call_function_queue(void); diff --git a/kernel/smp.c b/kernel/smp.c index 821b5986721ac..5cd680a7e78ef 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -161,9 +161,18 @@ void __init call_function_init(void) } static __always_inline void -send_call_function_ipi_mask(const struct cpumask *mask) +send_call_function_single_ipi(int cpu, smp_call_func_t func) { - trace_ipi_send_cpumask(mask, _RET_IP_, NULL); + if (call_function_single_prep_ipi(cpu)) { + trace_ipi_send_cpumask(cpumask_of(cpu), _RET_IP_, func); + arch_send_call_function_single_ipi(cpu); + } +} + +static __always_inline void +send_call_function_ipi_mask(const struct cpumask *mask, smp_call_func_t func) +{ + trace_ipi_send_cpumask(mask, _RET_IP_, func); arch_send_call_function_ipi_mask(mask); } @@ -430,12 +439,16 @@ static void __smp_call_single_queue_debug(int cpu, struct llist_node *node) struct cfd_seq_local *seq = this_cpu_ptr(&cfd_seq_local); struct call_function_data *cfd = this_cpu_ptr(&cfd_data); struct cfd_percpu *pcpu = per_cpu_ptr(cfd->pcpu, cpu); + struct __call_single_data *csd; + + csd = container_of(node, call_single_data_t, node.llist); + WARN_ON_ONCE(!(CSD_TYPE(csd) & (CSD_TYPE_SYNC | CSD_TYPE_ASYNC))); cfd_seq_store(pcpu->seq_queue, this_cpu, cpu, CFD_SEQ_QUEUE); if (llist_add(node, &per_cpu(call_single_queue, cpu))) { cfd_seq_store(pcpu->seq_ipi, this_cpu, cpu, CFD_SEQ_IPI); cfd_seq_store(seq->ping, this_cpu, cpu, CFD_SEQ_PING); - send_call_function_single_ipi(cpu); + send_call_function_single_ipi(cpu, csd->func); cfd_seq_store(seq->pinged, this_cpu, cpu, CFD_SEQ_PINGED); } else { cfd_seq_store(pcpu->seq_noipi, this_cpu, cpu, CFD_SEQ_NOIPI); @@ -477,6 +490,25 @@ static __always_inline void csd_unlock(struct __call_single_data *csd) smp_store_release(&csd->node.u_flags, 0); } +static __always_inline void +raw_smp_call_single_queue(int cpu, struct llist_node *node, smp_call_func_t func) +{ + /* + * The list addition should be visible to the target CPU when it pops + * the head of the list to pull the entry off it in the IPI handler + * because of normal cache coherency rules implied by the underlying + * llist ops. + * + * If IPIs can go out of order to the cache coherency protocol + * in an architecture, sufficient synchronisation should be added + * to arch code to make it appear to obey cache coherency WRT + * locking and barrier primitives. Generic code isn't really + * equipped to do the right thing... + */ + if (llist_add(node, &per_cpu(call_single_queue, cpu))) + send_call_function_single_ipi(cpu, func); +} + static DEFINE_PER_CPU_SHARED_ALIGNED(call_single_data_t, csd_data); void __smp_call_single_queue(int cpu, struct llist_node *node) @@ -493,21 +525,25 @@ void __smp_call_single_queue(int cpu, struct llist_node *node) } } #endif - /* - * The list addition should be visible to the target CPU when it pops - * the head of the list to pull the entry off it in the IPI handler - * because of normal cache coherency rules implied by the underlying - * llist ops. - * - * If IPIs can go out of order to the cache coherency protocol - * in an architecture, sufficient synchronisation should be added - * to arch code to make it appear to obey cache coherency WRT - * locking and barrier primitives. Generic code isn't really - * equipped to do the right thing... + * We have to check the type of the CSD before queueing it, because + * once queued it can have its flags cleared by + * flush_smp_call_function_queue() + * even if we haven't sent the smp_call IPI yet (e.g. the stopper + * executes migration_cpu_stop() on the remote CPU). */ - if (llist_add(node, &per_cpu(call_single_queue, cpu))) - send_call_function_single_ipi(cpu); + if (trace_ipi_send_cpumask_enabled()) { + call_single_data_t *csd; + smp_call_func_t func; + + csd = container_of(node, call_single_data_t, node.llist); + func = CSD_TYPE(csd) == CSD_TYPE_TTWU ? + sched_ttwu_pending : csd->func; + + raw_smp_call_single_queue(cpu, node, func); + } else { + raw_smp_call_single_queue(cpu, node, NULL); + } } /* @@ -976,9 +1012,9 @@ static void smp_call_function_many_cond(const struct cpumask *mask, * provided mask. */ if (nr_cpus == 1) - send_call_function_single_ipi(last_cpu); + send_call_function_single_ipi(last_cpu, func); else if (likely(nr_cpus > 1)) - send_call_function_ipi_mask(cfd->cpumask_ipi); + send_call_function_ipi_mask(cfd->cpumask_ipi, func); cfd_seq_store(this_cpu_ptr(&cfd_seq_local)->pinged, this_cpu, CFD_SEQ_NOCPU, CFD_SEQ_PINGED); }