From patchwork Wed Jun 21 16:29:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 111193 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp4504554vqr; Wed, 21 Jun 2023 09:53:26 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5b0EQ28PX8ynoE7DaoBdz4cfIo+yohOmWB7OBy6ZbekHySTnPEPdP706as1AcxcHcJVrTZ X-Received: by 2002:a05:6830:8a:b0:6b4:6d5c:61dc with SMTP id a10-20020a056830008a00b006b46d5c61dcmr10734070oto.17.1687366406360; Wed, 21 Jun 2023 09:53:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687366406; cv=none; d=google.com; s=arc-20160816; b=r/DGIzVYLZFDC0dQLIOSuV3iNXoCFPI0SUeNDrZ1GVi7nm5anTMV8XCM6D23lZ7t3o Dadmm4wyAP/FU1xgAYKPm6SAUp+wkxjWzI3+2AoTRFMxgdZdxlNXLdO/X3bn21QkfTgF 6q1rXHZKXHv7rpsy1WeIpdVFdIxuTi0SckkRrThf3GvM9gAbqnk3qhr6JZqwVqVuSjsp 4uamcHqtYOb8IlMt6CNxbo3/hYlMLfPOdKvsE+z7FfQt72sqDlFOIrUifTiYiW3at6yb ANQSXRA0qrZMAIK4/9RXPpn1TDI1qeij/GGbeFbi33duD2wKRCIhoqwQ06cvd1u3y18n cbcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:subject:cc:to:from:date :user-agent:message-id; bh=A9f6ryfvPdtAJNVaptmObjZ/gLC0/h9z+0E5Dxl8ApQ=; b=t0SwU0tvpGgvCPbG5+Xc/+rMZzOe/jchSMPaf6wTCPFCs0xTGxHeyk7LaTv+wMhsdF sCmBBcDwVXg+1jsMVQcG7EWDCa3JIIiAmoVhc/d39rihH9uVWofP0D1/SeSbWi7F7Sz9 X3UhllfejzUqIvyvi77xR+5RwvYr24f32aB7S+CSf142KPCMcPyjnuqaVrXXOSma0Za8 1YvDUs/LeKbpaYaqLrSKqCaW6WWh+bZSmReLBEDe1IIKAmRf7c96sXDP86UQzn2NPW7X lbS5F6hCfs+yVfvk7JXoe75afdF7aVyT3LWAnuXXzMkF26KaoCMjoONTBuN2APupZqGg g4sA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k135-20020a633d8d000000b00534913a5303si4623415pga.540.2023.06.21.09.53.12; Wed, 21 Jun 2023 09:53:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232285AbjFUQag (ORCPT + 99 others); Wed, 21 Jun 2023 12:30:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232376AbjFUQ34 (ORCPT ); Wed, 21 Jun 2023 12:29:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E087710CE for ; Wed, 21 Jun 2023 09:29:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 673EC61625 for ; Wed, 21 Jun 2023 16:29:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4185AC433C9; Wed, 21 Jun 2023 16:29:47 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.96) (envelope-from ) id 1qC0iI-001ynJ-0y; Wed, 21 Jun 2023 12:29:46 -0400 Message-ID: <20230621162946.120464083@goodmis.org> User-Agent: quilt/0.66 Date: Wed, 21 Jun 2023 12:29:32 -0400 From: Steven Rostedt To: linux-kernel@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Andrew Morton , Juri Lelli , William White , Daniel Bristot de Oliveira , Jonathan Corbet Subject: [for-next][PATCH 09/13] tracing/osnoise: Switch from PF_NO_SETAFFINITY to migrate_disable References: <20230621162923.953123395@goodmis.org> MIME-Version: 1.0 X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769331916946367007?= X-GMAIL-MSGID: =?utf-8?q?1769331916946367007?= From: Daniel Bristot de Oliveira Currently, osnoise/timerlat threads run with PF_NO_SETAFFINITY set. It works well, however, cgroups do not allow PF_NO_SETAFFINITY threads to be accepted, and this creates a limitation to osnoise/timerlat. To avoid this limitation, disable migration of the threads as soon as they start to run, and then clean the PF_NO_SETAFFINITY flag (still) used during thread creation. If for some reason a thread migration is requested, e.g., via sched_settafinity, the tracer thread will notice and exit. Link: https://lkml.kernel.org/r/8ba8bc9c15b3ea40cf73cf67a9bc061a264609f0.1686063934.git.bristot@kernel.org Cc: Juri Lelli Cc: William White Cc: Daniel Bristot de Oliveira Cc: Masami Hiramatsu Cc: Jonathan Corbet Signed-off-by: Daniel Bristot de Oliveira Signed-off-by: Steven Rostedt (Google) --- kernel/trace/trace_osnoise.c | 68 ++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c index e97e3fa5cbed..c265ec5f1726 100644 --- a/kernel/trace/trace_osnoise.c +++ b/kernel/trace/trace_osnoise.c @@ -1545,6 +1545,39 @@ static void osnoise_sleep(void) } } +/* + * osnoise_migration_pending - checks if the task needs to migrate + * + * osnoise/timerlat threads are per-cpu. If there is a pending request to + * migrate the thread away from the current CPU, something bad has happened. + * Play the good citizen and leave. + * + * Returns 0 if it is safe to continue, 1 otherwise. + */ +static inline int osnoise_migration_pending(void) +{ + if (!current->migration_pending) + return 0; + + /* + * If migration is pending, there is a task waiting for the + * tracer to enable migration. The tracer does not allow migration, + * thus: taint and leave to unblock the blocked thread. + */ + osnoise_taint("migration requested to osnoise threads, leaving."); + + /* + * Unset this thread from the threads managed by the interface. + * The tracers are responsible for cleaning their env before + * exiting. + */ + mutex_lock(&interface_lock); + this_cpu_osn_var()->kthread = NULL; + mutex_unlock(&interface_lock); + + return 1; +} + /* * osnoise_main - The osnoise detection kernel thread * @@ -1553,12 +1586,29 @@ static void osnoise_sleep(void) */ static int osnoise_main(void *data) { + unsigned long flags; + + /* + * This thread was created pinned to the CPU using PF_NO_SETAFFINITY. + * The problem is that cgroup does not allow PF_NO_SETAFFINITY thread. + * + * To work around this limitation, disable migration and remove the + * flag. + */ + migrate_disable(); + raw_spin_lock_irqsave(¤t->pi_lock, flags); + current->flags &= ~(PF_NO_SETAFFINITY); + raw_spin_unlock_irqrestore(¤t->pi_lock, flags); while (!kthread_should_stop()) { + if (osnoise_migration_pending()) + break; + run_osnoise(); osnoise_sleep(); } + migrate_enable(); return 0; } @@ -1706,6 +1756,7 @@ static int timerlat_main(void *data) struct timerlat_variables *tlat = this_cpu_tmr_var(); struct timerlat_sample s; struct sched_param sp; + unsigned long flags; u64 now, diff; /* @@ -1714,6 +1765,18 @@ static int timerlat_main(void *data) sp.sched_priority = DEFAULT_TIMERLAT_PRIO; sched_setscheduler_nocheck(current, SCHED_FIFO, &sp); + /* + * This thread was created pinned to the CPU using PF_NO_SETAFFINITY. + * The problem is that cgroup does not allow PF_NO_SETAFFINITY thread. + * + * To work around this limitation, disable migration and remove the + * flag. + */ + migrate_disable(); + raw_spin_lock_irqsave(¤t->pi_lock, flags); + current->flags &= ~(PF_NO_SETAFFINITY); + raw_spin_unlock_irqrestore(¤t->pi_lock, flags); + tlat->count = 0; tlat->tracing_thread = false; @@ -1731,6 +1794,7 @@ static int timerlat_main(void *data) osn_var->sampling = 1; while (!kthread_should_stop()) { + now = ktime_to_ns(hrtimer_cb_get_time(&tlat->timer)); diff = now - tlat->abs_period; @@ -1749,10 +1813,14 @@ static int timerlat_main(void *data) if (time_to_us(diff) >= osnoise_data.stop_tracing_total) osnoise_stop_tracing(); + if (osnoise_migration_pending()) + break; + wait_next_period(tlat); } hrtimer_cancel(&tlat->timer); + migrate_enable(); return 0; } #else /* CONFIG_TIMERLAT_TRACER */