From patchwork Fri Feb 10 14:09:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55415 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp976720wrn; Fri, 10 Feb 2023 06:18:34 -0800 (PST) X-Google-Smtp-Source: AK7set9qV0l1ng73VbN+MOI/KFm8UdFDMHwT9JEHs/ELgpykuckgx8qHq/qAGKygDqkkeV1bYvAk X-Received: by 2002:a17:906:1511:b0:88d:d304:3432 with SMTP id b17-20020a170906151100b0088dd3043432mr15623021ejd.60.1676038714006; Fri, 10 Feb 2023 06:18:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038713; cv=none; d=google.com; s=arc-20160816; b=igOqVfoKVZgm+mldomz6LaUiU7AGTLmxwpeeD3gXupDsSM4QSpDBvHdkbxWC/uYSe5 cDQVbmjP0hZfPSF16IeXULv4E72k+JmUbyo12lONLLKU5niyDx/BfgodZfdqgFplHc35 SBF6GK41VgB6dU90z3oW82j4ZXO1WSuJRtrgiPSu7HS++JQcNkg/rP6r8egtzJaZPXca AgHk1KN2MTXAtan8075ViB532Sae+1Uf3ZL1/zUnvgJ1ukWsZwEqfBhVZZadp1sNUyhv 7dufbHE5jsEwcjhR3sB5+KyurFA+WOw4BThqXxfMkSjCsn6OlBNJMrRu2MekLwMttLrE Kj6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QZOYb2vW2FDOq/YvNHG9EKnn3TIFIHHCx+GGrib8vN4=; b=m5wgFGr07U2TD4/IGex+wYtww8afgs4LDj+FxIKn7xXfWqZXXzjjCmhznNW/bxZHFD pd6cQa7lTy8L/0lZXlYg79lTENx7QH1u7nyF12NQpU1sXDnwvPbOsxaGsha02JeRDvJK 3VB7LT1gWI6qgDNNHsvweuWjKDIU5eNJkrNMz3DfLWJZc6J15qkayRl5+lEF1scqJn3G 3uGfiPO3aO7xAplYaHycHhfan+/KYmMx4gGUnKjTOslYmYidfy/vDI9N5LjbApAxM5kQ sA8PkOzK9h08j08O+ft4eRpO6Q9TCW3JdFAzMzvM1Fvel1u6UXihfkgMIFaWt//uTpFr PArg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lvIYljSO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fq14-20020a1709069d8e00b0087bdac7c672si5986528ejc.938.2023.02.10.06.18.10; Fri, 10 Feb 2023 06:18:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lvIYljSO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232441AbjBJOJl (ORCPT + 99 others); Fri, 10 Feb 2023 09:09:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232443AbjBJOJh (ORCPT ); Fri, 10 Feb 2023 09:09:37 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38C911B570 for ; Fri, 10 Feb 2023 06:09:32 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A6A74B82280 for ; Fri, 10 Feb 2023 14:09:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42292C433EF; Fri, 10 Feb 2023 14:09:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038169; bh=DTwnO69RuMGiGrcCioikHuzFr2i7tzmHe97x2bj6jfw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lvIYljSOpFSc1gXt4msCSasQgiigspNSl7oK8mjEVdmfTZGTqzy55GSupermnme7B 3ULYZB4r2PrYFQy85Mx92pTSkyQ8SaZGCOhYPFsFq9h+Q3zjBFVcP1jo2w10TMUINb NJKo90JkRRLiOLP/VgJss8s9MczqNVnNOl5QspxINpyNIj8d456HJ0nGpXJzoD87fg nylbp1qcUdhReF/ZU0g4jiGSEIPyi7dNno9iHtDWBccv1YB2YSegzXgyQpIXsKzr5g WjBQd8Y/M+lMBNpNY4Nmj9osrsNUTpOD0i0Lh3CnVl+CdsbaahcqRWX8gZCmSLaJ7u RS/26oZNYJIZg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 2/6] timers/nohz: Only ever update sleeptime from idle exit Date: Fri, 10 Feb 2023 15:09:13 +0100 Message-Id: <20230210140917.279062-3-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757453970634983707?= X-GMAIL-MSGID: =?utf-8?q?1757453970634983707?= The idle and io sleeptime statistics appearing in /proc/stat can be currently updated from two sites: locally on idle exit and remotely by cpufreq. However there is no synchronization mechanism protecting concurrent updates. It is therefore possible to account the sleeptime twice, among all the possible broken scenarios. To prevent from breaking the sleeptime accounting source, restrict the sleeptime updates to the local idle exit site. If there is a delta to add since the last update, IO/Idle sleep time readers will now only compute the delta without actually writing it back to the internal idle statistic fields. This fixes a writer VS writer race. Note there are still two known reader VS writer races to handle. A subsequent patch will fix one. Reported-by: Yu Liao Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 104 ++++++++++++++++----------------------- 1 file changed, 42 insertions(+), 62 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index b0e3c9205946..db22342f8948 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -637,31 +637,21 @@ static void tick_nohz_update_jiffies(ktime_t now) touch_softlockup_watchdog_sched(); } -/* - * Updates the per-CPU time idle statistics counters - */ -static void -update_ts_time_stats(int cpu, struct tick_sched *ts, ktime_t now, u64 *last_update_time) -{ - ktime_t delta; - - if (ts->idle_active) { - delta = ktime_sub(now, ts->idle_entrytime); - if (nr_iowait_cpu(cpu) > 0) - ts->iowait_sleeptime = ktime_add(ts->iowait_sleeptime, delta); - else - ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); - ts->idle_entrytime = now; - } - - if (last_update_time) - *last_update_time = ktime_to_us(now); - -} - static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) { - update_ts_time_stats(smp_processor_id(), ts, now, NULL); + ktime_t delta; + + if (WARN_ON_ONCE(!ts->idle_active)) + return; + + delta = ktime_sub(now, ts->idle_entrytime); + + if (nr_iowait_cpu(smp_processor_id()) > 0) + ts->iowait_sleeptime = ktime_add(ts->iowait_sleeptime, delta); + else + ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); + + ts->idle_entrytime = now; ts->idle_active = 0; sched_clock_idle_wakeup_event(); @@ -674,6 +664,31 @@ static void tick_nohz_start_idle(struct tick_sched *ts) sched_clock_idle_sleep_event(); } +static u64 get_cpu_sleep_time_us(int cpu, ktime_t *sleeptime, + bool compute_delta, u64 *last_update_time) +{ + struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); + ktime_t now, idle; + + if (!tick_nohz_active) + return -1; + + now = ktime_get(); + if (last_update_time) + *last_update_time = ktime_to_us(now); + + if (ts->idle_active && !nr_iowait_cpu(cpu)) { + ktime_t delta = ktime_sub(now, ts->idle_entrytime); + + idle = ktime_add(*sleeptime, delta); + } else { + idle = *sleeptime; + } + + return ktime_to_us(idle); + +} + /** * get_cpu_idle_time_us - get the total idle time of a CPU * @cpu: CPU number to query @@ -691,27 +706,9 @@ static void tick_nohz_start_idle(struct tick_sched *ts) u64 get_cpu_idle_time_us(int cpu, u64 *last_update_time) { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); - ktime_t now, idle; - - if (!tick_nohz_active) - return -1; - - now = ktime_get(); - if (last_update_time) { - update_ts_time_stats(cpu, ts, now, last_update_time); - idle = ts->idle_sleeptime; - } else { - if (ts->idle_active && !nr_iowait_cpu(cpu)) { - ktime_t delta = ktime_sub(now, ts->idle_entrytime); - - idle = ktime_add(ts->idle_sleeptime, delta); - } else { - idle = ts->idle_sleeptime; - } - } - - return ktime_to_us(idle); + return get_cpu_sleep_time_us(cpu, &ts->idle_sleeptime, + !nr_iowait_cpu(cpu), last_update_time); } EXPORT_SYMBOL_GPL(get_cpu_idle_time_us); @@ -732,26 +729,9 @@ EXPORT_SYMBOL_GPL(get_cpu_idle_time_us); u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time) { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); - ktime_t now, iowait; - if (!tick_nohz_active) - return -1; - - now = ktime_get(); - if (last_update_time) { - update_ts_time_stats(cpu, ts, now, last_update_time); - iowait = ts->iowait_sleeptime; - } else { - if (ts->idle_active && nr_iowait_cpu(cpu) > 0) { - ktime_t delta = ktime_sub(now, ts->idle_entrytime); - - iowait = ktime_add(ts->iowait_sleeptime, delta); - } else { - iowait = ts->iowait_sleeptime; - } - } - - return ktime_to_us(iowait); + return get_cpu_sleep_time_us(cpu, &ts->iowait_sleeptime, + nr_iowait_cpu(cpu), last_update_time); } EXPORT_SYMBOL_GPL(get_cpu_iowait_time_us);