From patchwork Mon Jun 19 10:49:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 109929 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2937873vqr; Mon, 19 Jun 2023 04:29:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5m6nJOzpBICxEK0rUAjt4tWoLpDqxIAnOvA42vTyRDNpffFH5QyBmypaf/10r9sgxaLgBI X-Received: by 2002:a05:6830:13cd:b0:6b5:8689:4c78 with SMTP id e13-20020a05683013cd00b006b586894c78mr1054458otq.13.1687174168389; Mon, 19 Jun 2023 04:29:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687174168; cv=none; d=google.com; s=arc-20160816; b=z/pgr6FHI48uA3Grgz89jEm74G7Crx2njVgE1f/Bglbl5eZZdtlVo3MmFWaXDXCgUY wFGUg6zQT3oM1ihWmQXF2hmMORtzZem7Y8o7QiM0l+WpYQ2jYzRH45yX8zjS6PliwSyY 1JoVK8k4DL903lqR90UDERPIuUR+CoQYw0uJvHEmUwKa4LXTl42V4pnk8icRfWAzIwpk Wd/R2OO520onrmpbf2TeS9rPkwbKsnItwqi6krt/0Z9je3zS2IWBFkMKZ+nkYJp2a1sL 0XNdkbJ7SRRA4auLAvQ61GuSbfAqjPrT1a75LgCDPO9mpEAnv459WSiMDf2/21L+pcAE C4SQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=wImM+p33UZ20jm9bf7CgfuXOmXaME53jmn4uQACAOXc=; b=w7xBJ/MocE+6rYQuWb8WQV0gkHwHlSDjqy8puutrRrY2V/3FHzB9fs0IW0TSGT9QE7 C+58cNH9aINZ24I2n/Ad16PC7N9KBtpKR6VCo9KdVGS0qGiIwD/BwJ2keTZi/mtp/EdX BM0pGFqH7GghPoSoMW8SPoYl33ukdRuFCU0eaM7ZU2D8CYC5/rumJ4Qhng3Yw8Hi6q/2 SsodFci9tvnPdzMfnLtkPNKQlnK1ai/+cXWW/W9+eOQwy4GByDGiuJEMNc5WDmuYq6Gl 0oP9DGBd7tr5bn+0xotJFj3xQpbgXWgQmnIeTUii/iFrwdBZ01MnHHCTgfMPS35nTdS2 xmmw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=tlLpx0nh; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="i6n6D/8r"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l64-20020a639143000000b00552198fe2afsi8496456pge.480.2023.06.19.04.29.14; Mon, 19 Jun 2023 04:29:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=tlLpx0nh; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b="i6n6D/8r"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232180AbjFSKuc (ORCPT + 99 others); Mon, 19 Jun 2023 06:50:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232157AbjFSKtc (ORCPT ); Mon, 19 Jun 2023 06:49:32 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F23FB198B; Mon, 19 Jun 2023 03:49:26 -0700 (PDT) Date: Mon, 19 Jun 2023 10:49:24 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1687171765; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wImM+p33UZ20jm9bf7CgfuXOmXaME53jmn4uQACAOXc=; b=tlLpx0nhI+ax4T5nIZvbEEPlpLEpCTei40PPJx94XEtVtGcuZMDvLdlkE4S/E284svq0ha oYJlW82oHFpxZMFasI3LXhd4LFCvHKIik8flnYyb9shYxnHoxqNLThGP7WACUEUQjnWg3h PMCA+iPSMuKAbAARf5Eu0v/UJdsJcRm7c7eNDPKpuX5CXJ35MVQWrPV7O+D7Y2nvLXJX8x /y18Gq3S2LWyTRfmYSK1pw8pxi/VF+VQAaC0SNo5m0vCTuzH+Ph30lCMj33+69e0AQJofD Th2CaJZWwAZyUHVV1cjEVTFbe5pMxu36/WfyzpgnSMRICwvXR6GpsRrAbk9iKQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1687171765; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wImM+p33UZ20jm9bf7CgfuXOmXaME53jmn4uQACAOXc=; b=i6n6D/8rja9F5ZrA8h2BXghOv9vNjG0zt1UtgE7yr6D0AML3jOFy1y6Y2RNKDFoSHMVK4X FpAHrwn9mdiSIoAg== From: "tip-bot2 for Leonardo Bras" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: smp/core] trace,smp: Add tracepoints around remotelly called functions Cc: Leonardo Bras , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20230615065944.188876-5-leobras@redhat.com> References: <20230615065944.188876-5-leobras@redhat.com> MIME-Version: 1.0 Message-ID: <168717176494.404.16420010294801824681.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768752806499025319?= X-GMAIL-MSGID: =?utf-8?q?1769130340550880651?= The following commit has been merged into the smp/core branch of tip: Commit-ID: 949fa3f11ced2a5c8e3737e73b09676adf4b322b Gitweb: https://git.kernel.org/tip/949fa3f11ced2a5c8e3737e73b09676adf4b322b Author: Leonardo Bras AuthorDate: Thu, 15 Jun 2023 03:59:45 -03:00 Committer: Peter Zijlstra CommitterDate: Fri, 16 Jun 2023 22:08:09 +02:00 trace,smp: Add tracepoints around remotelly called functions The recently added ipi_send_{cpu,cpumask} tracepoints allow finding sources of IPIs targeting CPUs running latency-sensitive applications. For NOHZ_FULL CPUs, all IPIs are interference, and those tracepoints are sufficient to find them and work on getting rid of them. In some setups however, not *all* IPIs are to be suppressed, but long-running IPI callbacks can still be problematic. Add a pair of tracepoints to mark the start and end of processing a CSD IPI callback, similar to what exists for softirq, workqueue or timer callbacks. Signed-off-by: Leonardo Bras Tested-and-reviewed-by: Valentin Schneider Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20230615065944.188876-5-leobras@redhat.com --- include/trace/events/csd.h | 45 +++++++++++++++++++++++++++++++++++++- kernel/smp.c | 25 +++++++++++++++------ 2 files changed, 64 insertions(+), 6 deletions(-) create mode 100644 include/trace/events/csd.h diff --git a/include/trace/events/csd.h b/include/trace/events/csd.h new file mode 100644 index 0000000..af1df52 --- /dev/null +++ b/include/trace/events/csd.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM csd + +#if !defined(_TRACE_CSD_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_CSD_H + +#include + +/* + * Tracepoints for a function which is called as an effect of smp_call_function.* + */ +DECLARE_EVENT_CLASS(csd_function, + + TP_PROTO(smp_call_func_t func, struct __call_single_data *csd), + + TP_ARGS(func, csd), + + TP_STRUCT__entry( + __field(void *, func) + __field(void *, csd) + ), + + TP_fast_assign( + __entry->func = func; + __entry->csd = csd; + ), + + TP_printk("func=%ps, csd=%p", __entry->func, __entry->csd) +); + +DEFINE_EVENT(csd_function, csd_function_entry, + TP_PROTO(smp_call_func_t func, struct __call_single_data *csd), + TP_ARGS(func, csd) +); + +DEFINE_EVENT(csd_function, csd_function_exit, + TP_PROTO(smp_call_func_t func, struct __call_single_data *csd), + TP_ARGS(func, csd) +); + +#endif /* _TRACE_CSD_H */ + +/* This part must be outside protection */ +#include diff --git a/kernel/smp.c b/kernel/smp.c index 71dce74..1fa01a8 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -27,6 +27,9 @@ #include #include +#define CREATE_TRACE_POINTS +#include +#undef CREATE_TRACE_POINTS #include "smpboot.h" #include "sched/smp.h" @@ -121,6 +124,14 @@ send_call_function_ipi_mask(struct cpumask *mask) arch_send_call_function_ipi_mask(mask); } +static __always_inline void +csd_do_func(smp_call_func_t func, void *info, struct __call_single_data *csd) +{ + trace_csd_function_entry(func, csd); + func(info); + trace_csd_function_exit(func, csd); +} + #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG static DEFINE_STATIC_KEY_MAYBE(CONFIG_CSD_LOCK_WAIT_DEBUG_DEFAULT, csdlock_debug_enabled); @@ -375,7 +386,7 @@ static int generic_exec_single(int cpu, struct __call_single_data *csd) csd_lock_record(csd); csd_unlock(csd); local_irq_save(flags); - func(info); + csd_do_func(func, info, NULL); csd_lock_record(NULL); local_irq_restore(flags); return 0; @@ -477,7 +488,7 @@ static void __flush_smp_call_function_queue(bool warn_cpu_offline) } csd_lock_record(csd); - func(info); + csd_do_func(func, info, csd); csd_unlock(csd); csd_lock_record(NULL); } else { @@ -508,7 +519,7 @@ static void __flush_smp_call_function_queue(bool warn_cpu_offline) csd_lock_record(csd); csd_unlock(csd); - func(info); + csd_do_func(func, info, csd); csd_lock_record(NULL); } else if (type == CSD_TYPE_IRQ_WORK) { irq_work_single(csd); @@ -522,8 +533,10 @@ static void __flush_smp_call_function_queue(bool warn_cpu_offline) /* * Third; only CSD_TYPE_TTWU is left, issue those. */ - if (entry) - sched_ttwu_pending(entry); + if (entry) { + csd = llist_entry(entry, typeof(*csd), node.llist); + csd_do_func(sched_ttwu_pending, entry, csd); + } } @@ -816,7 +829,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask, unsigned long flags; local_irq_save(flags); - func(info); + csd_do_func(func, info, NULL); local_irq_restore(flags); }