From patchwork Fri Feb 10 14:09:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55414 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp976197wrn; Fri, 10 Feb 2023 06:17:40 -0800 (PST) X-Google-Smtp-Source: AK7set8QeBHFu9hNP9VlHGau+gewMjV9cEU1BTM1IUZa4IWv1jkoQGiCf9f9IgFEojBxHlf42Z98 X-Received: by 2002:a17:907:8810:b0:8a9:e031:c4ae with SMTP id ro16-20020a170907881000b008a9e031c4aemr16968014ejc.2.1676038660110; Fri, 10 Feb 2023 06:17:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038660; cv=none; d=google.com; s=arc-20160816; b=l2hKRJNC7iSkmLUa0CgdggH7IH4tMxH7F/cEJ23AdUs7fkSIQ3V4PVNgjqIt8jaA1F rodfLGhEQrroD84Eq/QuyyLpZKpufhnG4lyqdfqgINRFz0e2YWbjj6YYDnbgtFsEo0cd D2iE3u+JdYvO4Z1xAb8Bvs1nVaoOefJy/BQOCRdrHA6TdvRNSjl3hS+URAsWTieMJ/qk xYwavsKKgBHTbMrXkfuDcXhJngQlrBc4JPORDdT348RMDO8mlH6v8L4zq+LTgLi56eO+ dOTeejKsl2uo5wlQe7NW3yYV4lo0ercIGaCNGs2WcFHU+oN6DawcKkr5iDdsnkxuHKSg klMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=jIysAK2gsHywpDPJToTwx5ZYxE7w+TcG3IV0GL9jlQI=; b=XbHuleslNaQY2HFWXuBODYf6YxeAYgbvpzUKo5vo4U0EdJD6c6ljYlrbcdx12VgiJV zttR6I3xg2oZD6TVXP5COfWj5VddrNZZp6/FVPAjJG+8DoqSnGnS2EsgIDHw4j4r+ylN d4+N7opo0vJdCgQ1tpM0SgD1Brikq9HK3dE+/5mJjedyZ7miYHU1GrnVH7/hfmo6AuGc YAQg0pO8Ntx5pE5WXguv8Wvyp3gen98G6j06D5CG9Oy1ilpn0mo9rHPVQ1ECrACITIJV MeIsIsVHKjOXCjksfLDIqPLL8jNlWOmIgm5i26H1hPbbOech0fA0zskwjyVqwIDnfx0d ho8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=UELP+A9o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 9-20020a170906020900b0089dcdd2c208si5204358ejd.921.2023.02.10.06.17.16; Fri, 10 Feb 2023 06:17:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=UELP+A9o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232253AbjBJOJd (ORCPT + 99 others); Fri, 10 Feb 2023 09:09:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232429AbjBJOJ3 (ORCPT ); Fri, 10 Feb 2023 09:09:29 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E73E72195A for ; Fri, 10 Feb 2023 06:09:27 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5991A61DCC for ; Fri, 10 Feb 2023 14:09:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B65CC433D2; Fri, 10 Feb 2023 14:09:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038166; bh=FaVCj67P0RIyRZ9SOpAzVW9SW/d7Prf5e3bxlBOh2/o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UELP+A9oXEuDHgvQT4Xnlimx8oOHWewth4dIVn8V4qqePtaa/ZXu7V3shqHKtChaS RzaZ/LWoUWrmNvH9XjuWU3BFP2HVX6vy5Do9A3v1HMm9H4E92OnlPKxM1Jj3i31MTb G82c6Kbqsmnei+EGq7X+vI54fzZIh6jrl1VhmQbVdF8eZajz8ohT3SSa99OAEmbB88 gJcolUJZfUK9wK41WSZAHigUU7EduSYF/haH/a5/xH/o717V6rkk0RHGcOdmxvzXTF ZjlVx60HL42E0JxO5SXuYWjdAnRqHAvRzKIAk1zccJOYNCd82gQUTpn2aNdg/WFoSQ 5US2UPnpqtYdg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 1/6] timers/nohz: Restructure and reshuffle struct tick_sched Date: Fri, 10 Feb 2023 15:09:12 +0100 Message-Id: <20230210140917.279062-2-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757453913535083339?= X-GMAIL-MSGID: =?utf-8?q?1757453913535083339?= Restructure and group fields by access in order to optimize cache layout. While at it, also add missing kernel doc for two fields: @last_jiffies and @idle_expires. Reported-by: Thomas Gleixner Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.h | 66 +++++++++++++++++++++++++--------------- 1 file changed, 41 insertions(+), 25 deletions(-) diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index 504649513399..c6663254d17d 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -22,65 +22,81 @@ enum tick_nohz_mode { /** * struct tick_sched - sched tick emulation and no idle tick control/stats - * @sched_timer: hrtimer to schedule the periodic tick in high - * resolution mode - * @check_clocks: Notification mechanism about clocksource changes - * @nohz_mode: Mode - one state of tick_nohz_mode + * * @inidle: Indicator that the CPU is in the tick idle mode * @tick_stopped: Indicator that the idle tick has been stopped * @idle_active: Indicator that the CPU is actively in the tick idle mode; * it is reset during irq handling phases. - * @do_timer_lst: CPU was the last one doing do_timer before going idle + * @do_timer_last: CPU was the last one doing do_timer before going idle * @got_idle_tick: Tick timer function has run with @inidle set + * @stalled_jiffies: Number of stalled jiffies detected across ticks + * @last_tick_jiffies: Value of jiffies seen on last tick + * @sched_timer: hrtimer to schedule the periodic tick in high + * resolution mode * @last_tick: Store the last tick expiry time when the tick * timer is modified for nohz sleeps. This is necessary * to resume the tick timer operation in the timeline * when the CPU returns from nohz sleep. * @next_tick: Next tick to be fired when in dynticks mode. * @idle_jiffies: jiffies at the entry to idle for idle time accounting + * @idle_waketime: Time when the idle was interrupted + * @idle_entrytime: Time when the idle call was entered + * @nohz_mode: Mode - one state of tick_nohz_mode + * @last_jiffies: Base jiffies snapshot when next event was last computed + * @timer_expires_base: Base time clock monotonic for @timer_expires + * @timer_expires: Anticipated timer expiration time (in case sched tick is stopped) + * @next_timer: Expiry time of next expiring timer for debugging purpose only + * @idle_expires: Next tick in idle, for debugging purpose only * @idle_calls: Total number of idle calls * @idle_sleeps: Number of idle calls, where the sched tick was stopped - * @idle_entrytime: Time when the idle call was entered - * @idle_waketime: Time when the idle was interrupted * @idle_exittime: Time when the idle state was left * @idle_sleeptime: Sum of the time slept in idle with sched tick stopped * @iowait_sleeptime: Sum of the time slept in idle with sched tick stopped, with IO outstanding - * @timer_expires: Anticipated timer expiration time (in case sched tick is stopped) - * @timer_expires_base: Base time clock monotonic for @timer_expires - * @next_timer: Expiry time of next expiring timer for debugging purpose only * @tick_dep_mask: Tick dependency mask - is set, if someone needs the tick - * @last_tick_jiffies: Value of jiffies seen on last tick - * @stalled_jiffies: Number of stalled jiffies detected across ticks + * @check_clocks: Notification mechanism about clocksource changes */ struct tick_sched { - struct hrtimer sched_timer; - unsigned long check_clocks; - enum tick_nohz_mode nohz_mode; - + /* Common flags */ unsigned int inidle : 1; unsigned int tick_stopped : 1; unsigned int idle_active : 1; unsigned int do_timer_last : 1; unsigned int got_idle_tick : 1; + /* Tick handling: jiffies stall check */ + unsigned int stalled_jiffies; + unsigned long last_tick_jiffies; + + /* Tick handling */ + struct hrtimer sched_timer; ktime_t last_tick; ktime_t next_tick; unsigned long idle_jiffies; - unsigned long idle_calls; - unsigned long idle_sleeps; - ktime_t idle_entrytime; ktime_t idle_waketime; - ktime_t idle_exittime; - ktime_t idle_sleeptime; - ktime_t iowait_sleeptime; + + /* Idle entry */ + ktime_t idle_entrytime; + + /* Tick stop */ + enum tick_nohz_mode nohz_mode; unsigned long last_jiffies; - u64 timer_expires; u64 timer_expires_base; + u64 timer_expires; u64 next_timer; ktime_t idle_expires; + unsigned long idle_calls; + unsigned long idle_sleeps; + + /* Idle exit */ + ktime_t idle_exittime; + ktime_t idle_sleeptime; + ktime_t iowait_sleeptime; + + /* Full dynticks handling */ atomic_t tick_dep_mask; - unsigned long last_tick_jiffies; - unsigned int stalled_jiffies; + + /* Clocksource changes */ + unsigned long check_clocks; }; extern struct tick_sched *tick_get_tick_sched(int cpu); From patchwork Fri Feb 10 14:09:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55415 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp976720wrn; Fri, 10 Feb 2023 06:18:34 -0800 (PST) X-Google-Smtp-Source: AK7set9qV0l1ng73VbN+MOI/KFm8UdFDMHwT9JEHs/ELgpykuckgx8qHq/qAGKygDqkkeV1bYvAk X-Received: by 2002:a17:906:1511:b0:88d:d304:3432 with SMTP id b17-20020a170906151100b0088dd3043432mr15623021ejd.60.1676038714006; Fri, 10 Feb 2023 06:18:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038713; cv=none; d=google.com; s=arc-20160816; b=igOqVfoKVZgm+mldomz6LaUiU7AGTLmxwpeeD3gXupDsSM4QSpDBvHdkbxWC/uYSe5 cDQVbmjP0hZfPSF16IeXULv4E72k+JmUbyo12lONLLKU5niyDx/BfgodZfdqgFplHc35 SBF6GK41VgB6dU90z3oW82j4ZXO1WSuJRtrgiPSu7HS++JQcNkg/rP6r8egtzJaZPXca AgHk1KN2MTXAtan8075ViB532Sae+1Uf3ZL1/zUnvgJ1ukWsZwEqfBhVZZadp1sNUyhv 7dufbHE5jsEwcjhR3sB5+KyurFA+WOw4BThqXxfMkSjCsn6OlBNJMrRu2MekLwMttLrE Kj6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QZOYb2vW2FDOq/YvNHG9EKnn3TIFIHHCx+GGrib8vN4=; b=m5wgFGr07U2TD4/IGex+wYtww8afgs4LDj+FxIKn7xXfWqZXXzjjCmhznNW/bxZHFD pd6cQa7lTy8L/0lZXlYg79lTENx7QH1u7nyF12NQpU1sXDnwvPbOsxaGsha02JeRDvJK 3VB7LT1gWI6qgDNNHsvweuWjKDIU5eNJkrNMz3DfLWJZc6J15qkayRl5+lEF1scqJn3G 3uGfiPO3aO7xAplYaHycHhfan+/KYmMx4gGUnKjTOslYmYidfy/vDI9N5LjbApAxM5kQ sA8PkOzK9h08j08O+ft4eRpO6Q9TCW3JdFAzMzvM1Fvel1u6UXihfkgMIFaWt//uTpFr PArg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lvIYljSO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fq14-20020a1709069d8e00b0087bdac7c672si5986528ejc.938.2023.02.10.06.18.10; Fri, 10 Feb 2023 06:18:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=lvIYljSO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232441AbjBJOJl (ORCPT + 99 others); Fri, 10 Feb 2023 09:09:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232443AbjBJOJh (ORCPT ); Fri, 10 Feb 2023 09:09:37 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38C911B570 for ; Fri, 10 Feb 2023 06:09:32 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A6A74B82280 for ; Fri, 10 Feb 2023 14:09:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 42292C433EF; Fri, 10 Feb 2023 14:09:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038169; bh=DTwnO69RuMGiGrcCioikHuzFr2i7tzmHe97x2bj6jfw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lvIYljSOpFSc1gXt4msCSasQgiigspNSl7oK8mjEVdmfTZGTqzy55GSupermnme7B 3ULYZB4r2PrYFQy85Mx92pTSkyQ8SaZGCOhYPFsFq9h+Q3zjBFVcP1jo2w10TMUINb NJKo90JkRRLiOLP/VgJss8s9MczqNVnNOl5QspxINpyNIj8d456HJ0nGpXJzoD87fg nylbp1qcUdhReF/ZU0g4jiGSEIPyi7dNno9iHtDWBccv1YB2YSegzXgyQpIXsKzr5g WjBQd8Y/M+lMBNpNY4Nmj9osrsNUTpOD0i0Lh3CnVl+CdsbaahcqRWX8gZCmSLaJ7u RS/26oZNYJIZg== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 2/6] timers/nohz: Only ever update sleeptime from idle exit Date: Fri, 10 Feb 2023 15:09:13 +0100 Message-Id: <20230210140917.279062-3-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757453970634983707?= X-GMAIL-MSGID: =?utf-8?q?1757453970634983707?= The idle and io sleeptime statistics appearing in /proc/stat can be currently updated from two sites: locally on idle exit and remotely by cpufreq. However there is no synchronization mechanism protecting concurrent updates. It is therefore possible to account the sleeptime twice, among all the possible broken scenarios. To prevent from breaking the sleeptime accounting source, restrict the sleeptime updates to the local idle exit site. If there is a delta to add since the last update, IO/Idle sleep time readers will now only compute the delta without actually writing it back to the internal idle statistic fields. This fixes a writer VS writer race. Note there are still two known reader VS writer races to handle. A subsequent patch will fix one. Reported-by: Yu Liao Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 104 ++++++++++++++++----------------------- 1 file changed, 42 insertions(+), 62 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index b0e3c9205946..db22342f8948 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -637,31 +637,21 @@ static void tick_nohz_update_jiffies(ktime_t now) touch_softlockup_watchdog_sched(); } -/* - * Updates the per-CPU time idle statistics counters - */ -static void -update_ts_time_stats(int cpu, struct tick_sched *ts, ktime_t now, u64 *last_update_time) -{ - ktime_t delta; - - if (ts->idle_active) { - delta = ktime_sub(now, ts->idle_entrytime); - if (nr_iowait_cpu(cpu) > 0) - ts->iowait_sleeptime = ktime_add(ts->iowait_sleeptime, delta); - else - ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); - ts->idle_entrytime = now; - } - - if (last_update_time) - *last_update_time = ktime_to_us(now); - -} - static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) { - update_ts_time_stats(smp_processor_id(), ts, now, NULL); + ktime_t delta; + + if (WARN_ON_ONCE(!ts->idle_active)) + return; + + delta = ktime_sub(now, ts->idle_entrytime); + + if (nr_iowait_cpu(smp_processor_id()) > 0) + ts->iowait_sleeptime = ktime_add(ts->iowait_sleeptime, delta); + else + ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); + + ts->idle_entrytime = now; ts->idle_active = 0; sched_clock_idle_wakeup_event(); @@ -674,6 +664,31 @@ static void tick_nohz_start_idle(struct tick_sched *ts) sched_clock_idle_sleep_event(); } +static u64 get_cpu_sleep_time_us(int cpu, ktime_t *sleeptime, + bool compute_delta, u64 *last_update_time) +{ + struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); + ktime_t now, idle; + + if (!tick_nohz_active) + return -1; + + now = ktime_get(); + if (last_update_time) + *last_update_time = ktime_to_us(now); + + if (ts->idle_active && !nr_iowait_cpu(cpu)) { + ktime_t delta = ktime_sub(now, ts->idle_entrytime); + + idle = ktime_add(*sleeptime, delta); + } else { + idle = *sleeptime; + } + + return ktime_to_us(idle); + +} + /** * get_cpu_idle_time_us - get the total idle time of a CPU * @cpu: CPU number to query @@ -691,27 +706,9 @@ static void tick_nohz_start_idle(struct tick_sched *ts) u64 get_cpu_idle_time_us(int cpu, u64 *last_update_time) { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); - ktime_t now, idle; - - if (!tick_nohz_active) - return -1; - - now = ktime_get(); - if (last_update_time) { - update_ts_time_stats(cpu, ts, now, last_update_time); - idle = ts->idle_sleeptime; - } else { - if (ts->idle_active && !nr_iowait_cpu(cpu)) { - ktime_t delta = ktime_sub(now, ts->idle_entrytime); - - idle = ktime_add(ts->idle_sleeptime, delta); - } else { - idle = ts->idle_sleeptime; - } - } - - return ktime_to_us(idle); + return get_cpu_sleep_time_us(cpu, &ts->idle_sleeptime, + !nr_iowait_cpu(cpu), last_update_time); } EXPORT_SYMBOL_GPL(get_cpu_idle_time_us); @@ -732,26 +729,9 @@ EXPORT_SYMBOL_GPL(get_cpu_idle_time_us); u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time) { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); - ktime_t now, iowait; - if (!tick_nohz_active) - return -1; - - now = ktime_get(); - if (last_update_time) { - update_ts_time_stats(cpu, ts, now, last_update_time); - iowait = ts->iowait_sleeptime; - } else { - if (ts->idle_active && nr_iowait_cpu(cpu) > 0) { - ktime_t delta = ktime_sub(now, ts->idle_entrytime); - - iowait = ktime_add(ts->iowait_sleeptime, delta); - } else { - iowait = ts->iowait_sleeptime; - } - } - - return ktime_to_us(iowait); + return get_cpu_sleep_time_us(cpu, &ts->iowait_sleeptime, + nr_iowait_cpu(cpu), last_update_time); } EXPORT_SYMBOL_GPL(get_cpu_iowait_time_us); From patchwork Fri Feb 10 14:09:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55416 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp977247wrn; Fri, 10 Feb 2023 06:19:28 -0800 (PST) X-Google-Smtp-Source: AK7set/xIHSWrlse5hEcVLPprz5yyOeUPRs6WOWheHWqJWePvm9daa4wjDPy3/XiAOKvpoj6+9wx X-Received: by 2002:a50:c314:0:b0:4ab:1681:b99e with SMTP id a20-20020a50c314000000b004ab1681b99emr991160edb.4.1676038768013; Fri, 10 Feb 2023 06:19:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038768; cv=none; d=google.com; s=arc-20160816; b=AyDPJ6g/DHYMI9cAB2fr2YPvD5vPegToZhpGtBN8xxE26fCMV0d/rnyybbBYhg3JqU lrYJjmJmHebqM7Xz9iU+9r7M9PFiK4loQR8UUjYhoHlpehkxoCVde9LzaRs7uaYDcV6/ v56rBZrFhb3834dcXc0FL9eCovstQsBuEUJ+xUTCFm0XEjlwYJAWK1YLfKpgGabC7goc Cy4dLELMrmyaWBHhjJU4Kp+g6Bh4pam/9Tg5aXvmTLdESUmpMHVQD5v+kfc1EtNTiL0l XFrPLalLmYQ8/N9kdlnTztYLAM+RaslKcvIMHIc1Aee91ipB/LTLm+ReaJbM/IBXOZ+Z w8Bg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=QmcmgrZNsJ8p0kTiJtSaDH3JPYXXqL2AEHwxbqkk8sY=; b=owqpMMPS5GZwwrlBX7ExV5uQW4lQPe7H2P8lHk044V4ad4vjaX9sq4ACfsX46PlQRX 7stqlJ/bRcHfuDTdRkBubKKmFbtZSZjNEpujDpzEQEorc2jJg2ClEXGIu7g+YzT0mjZh aoHZWSBlJBEBl5VDCQI1IYxB3y71UpHmzZ1v05DTyYvQSjuOrWCQs1SdoKyVzB7FUOwE x89f/cdsU1e+MYVxkhYOkOZfKoysDHWKkejcmKNeUzSXeMwq3gdr9jKq/M2FtwFq4uA5 W9pBoz04U/KzPCCslLV5gvE0UCBwcTn/Uepyo2lhag3aj/QmNEoPBNYFSc5K/qiR+FpB 5fMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=G10o5gZn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f14-20020a05640214ce00b004aa694bbc43si5077661edx.68.2023.02.10.06.19.04; Fri, 10 Feb 2023 06:19:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=G10o5gZn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232462AbjBJOJq (ORCPT + 99 others); Fri, 10 Feb 2023 09:09:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232460AbjBJOJn (ORCPT ); Fri, 10 Feb 2023 09:09:43 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08B4A7407D for ; Fri, 10 Feb 2023 06:09:34 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 436CFB8251D for ; Fri, 10 Feb 2023 14:09:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DA9E2C4339B; Fri, 10 Feb 2023 14:09:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038172; bh=uGFctYYXWtKjvXXubFVYxBpGfYhc7n33Hu43EUXs6NM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=G10o5gZnxPa9cdACbV9apoHHnkc6ZHxJPdHxwQyzqZzTKAf+d+shez4QNAGy6KrIa CABaCi4TORQERtSoPhOwdxg0h+WjfP8bcnrRe/XTGogYvOKeIkUphZRK42U0XpUrS6 ARilPOTRz1SGmDemN2WkSiLZ35EOwDfJ8EuGkf++OIg8WITUhoR+GEQfxPaitd2OoE nkPvWt7Cn/7dXQBs7GFd6SWMaCCuk0IikYF8/YrcHr1WraLSkn75m1+ftkhdEN5Cvi o5vkTR/Z4ZFv7ndvVTLxIX1YTjTSVFmTq/EqNK9FdXZXeoztCcLFBaCW9YneBZ+z+9 N726M+2CSpEaA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 3/6] timers/nohz: Protect idle/iowait sleep time under seqcount Date: Fri, 10 Feb 2023 15:09:14 +0100 Message-Id: <20230210140917.279062-4-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757454026863636684?= X-GMAIL-MSGID: =?utf-8?q?1757454026863636684?= Reading idle/io sleep time (eg: from /proc/stat) can race with idle exit updates because the state machine handling the stats is not atomic and requires a coherent read batch. As a result reading the sleep time may report irrelevant or backward values. Fix this with protecting the simple state machine within a seqcount. This is expected to be cheap enough not to add measurable performance impact on the idle path. Note this only fixes reader VS writer condition partitially. A race remains that involves remote updates of the CPU iowait task counter. It can hardly be fixed. Reported-by: Yu Liao Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 22 ++++++++++++++++------ kernel/time/tick-sched.h | 1 + 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index db22342f8948..757e03ef4409 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -646,6 +646,7 @@ static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) delta = ktime_sub(now, ts->idle_entrytime); + write_seqcount_begin(&ts->idle_sleeptime_seq); if (nr_iowait_cpu(smp_processor_id()) > 0) ts->iowait_sleeptime = ktime_add(ts->iowait_sleeptime, delta); else @@ -653,14 +654,18 @@ static void tick_nohz_stop_idle(struct tick_sched *ts, ktime_t now) ts->idle_entrytime = now; ts->idle_active = 0; + write_seqcount_end(&ts->idle_sleeptime_seq); sched_clock_idle_wakeup_event(); } static void tick_nohz_start_idle(struct tick_sched *ts) { + write_seqcount_begin(&ts->idle_sleeptime_seq); ts->idle_entrytime = ktime_get(); ts->idle_active = 1; + write_seqcount_end(&ts->idle_sleeptime_seq); + sched_clock_idle_sleep_event(); } @@ -669,6 +674,7 @@ static u64 get_cpu_sleep_time_us(int cpu, ktime_t *sleeptime, { struct tick_sched *ts = &per_cpu(tick_cpu_sched, cpu); ktime_t now, idle; + unsigned int seq; if (!tick_nohz_active) return -1; @@ -677,13 +683,17 @@ static u64 get_cpu_sleep_time_us(int cpu, ktime_t *sleeptime, if (last_update_time) *last_update_time = ktime_to_us(now); - if (ts->idle_active && !nr_iowait_cpu(cpu)) { - ktime_t delta = ktime_sub(now, ts->idle_entrytime); + do { + seq = read_seqcount_begin(&ts->idle_sleeptime_seq); - idle = ktime_add(*sleeptime, delta); - } else { - idle = *sleeptime; - } + if (ts->idle_active && !nr_iowait_cpu(cpu)) { + ktime_t delta = ktime_sub(now, ts->idle_entrytime); + + idle = ktime_add(*sleeptime, delta); + } else { + idle = *sleeptime; + } + } while (read_seqcount_retry(&ts->idle_sleeptime_seq, seq)); return ktime_to_us(idle); diff --git a/kernel/time/tick-sched.h b/kernel/time/tick-sched.h index c6663254d17d..5ed5a9d41d5a 100644 --- a/kernel/time/tick-sched.h +++ b/kernel/time/tick-sched.h @@ -75,6 +75,7 @@ struct tick_sched { ktime_t idle_waketime; /* Idle entry */ + seqcount_t idle_sleeptime_seq; ktime_t idle_entrytime; /* Tick stop */ From patchwork Fri Feb 10 14:09:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55417 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp977476wrn; Fri, 10 Feb 2023 06:19:52 -0800 (PST) X-Google-Smtp-Source: AK7set/zcSH/nJ/I3AH4EYQ33O0dS7mEqkzQMnWXsrhjC47Mb+swpYKxmLeQxnKlZ+URdJm0P770 X-Received: by 2002:a17:906:1399:b0:7c0:be5d:59a9 with SMTP id f25-20020a170906139900b007c0be5d59a9mr10990899ejc.20.1676038792769; Fri, 10 Feb 2023 06:19:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038792; cv=none; d=google.com; s=arc-20160816; b=SIv8wHQG0asqC2CUU09wYSqpScY4WJTC0feBym/PmHWa3cnBX6jGLu1KF7jL0O1cTp IlGW7Pme+32gx9dlTmlZioAKxzwCZf6TBW1Pz1ksae7rSO7wM9lCrjsXqb/tAckSmj4u VmJsJcqdo3TZq/EE1+JTak384ZtCMDvbj3ko3kkVkTRTWaR314ishkDQeswrLizwYPTj ebFX8DvOPSzRtq/zatTB59hdIqPb+ckQnSUl9ZaC7Tp5Mc6nh+4k/1HHbeeyvddCuirv n7OT7/IjvlK90lgZcEGJICT1/ApzojhsKkBgCNcVSDbWCipwkY08ZiVrMq3QnOBL42u5 Z15w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yKn6Ab7Ocx7nypdVf2acgNm0BcUStnHbNhy0tJWpBjM=; b=aaPp7PakNC0Xxq/UHKl33wsOMZue5Az4IgC3W09n2FVI/1edLWBBe1zM5Ew41tUL5J mCqmwuYrjek6gojiIhkTboq7+XB90ipyFHpje7cPov70wYswKlmQ2ex1klSycRTBPO7Z wz4UKy/1zn6Khi0v6OtrEovVMGe+SGnAKk3q/VLPJ/HI6boLDE8/wxwTXnmIystJjhwn O1QhXEdSkPz/AGlsNqSQswoYGtzDyGcC/LospJiegylOOgLwLgemMsPNcTM9WFBCZBAy fimiwm0/gU9DbSsqHiMGCqE36LtNe8mg4TXkyEDNVcJGxXh9RjYzZDAdeaFrl/wc/cBz VNvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=DDFnJHQO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ft29-20020a170907801d00b00879d438501dsi5362537ejc.431.2023.02.10.06.19.29; Fri, 10 Feb 2023 06:19:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=DDFnJHQO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231540AbjBJOKF (ORCPT + 99 others); Fri, 10 Feb 2023 09:10:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232465AbjBJOJr (ORCPT ); Fri, 10 Feb 2023 09:09:47 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E64674998 for ; Fri, 10 Feb 2023 06:09:37 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1C466B824CF for ; Fri, 10 Feb 2023 14:09:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7CE88C433EF; Fri, 10 Feb 2023 14:09:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038174; bh=H1/1wQpakm7Rwe+HqpYOF44S3z7tTyE+Intx0NLRpx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DDFnJHQOZ66Z8Tl3ybE0nX4k4L/0RQNDRp2D04Wg023oD/b/WgkR/ryauQ70mOTJ6 45qevKmTRI7WuMliaLFNQVIEmLo7ozbu8cUMYP4mNGgIIOdokUmWR5vrzZNqngvaTk lcEtMzqME/TABgcQjOSAMWr1oM2XPW1NoNH+0xlpa8VR6jZHsquv4s0Mtg4SyvVWSn VRuKc9pscKhM40MDPYz+ErJ+ujq1YgKXCgvCuUf0dNVc8Vo3gA/oQs8PWHcnRhAnO/ 9qslSokLvZDWXpIa+c7auLCdX0bJn3g1vPocg1jQPBDFOam9Ex1sEF9i5OINpplvne x6BmcZmSCQumQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 4/6] timers/nohz: Add a comment about broken iowait counter update race Date: Fri, 10 Feb 2023 15:09:15 +0100 Message-Id: <20230210140917.279062-5-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757454053634690196?= X-GMAIL-MSGID: =?utf-8?q?1757454053634690196?= The per-cpu iowait task counter is incremented locally upon sleeping. But since the task can be woken to (and by) another CPU, the counter may then be decremented remotely. This is the source of a race involving readers VS writer of idle/iowait sleeptime. The following scenario shows an example where a /proc/stat reader observes a pending sleep time as IO whereas that pending sleep time later eventually gets accounted as non-IO. CPU 0 CPU 1 CPU 2 ----- ----- ------ //io_schedule() TASK A current->in_iowait = 1 rq(0)->nr_iowait++ //switch to idle // READ /proc/stat // See nr_iowait_cpu(0) == 1 return ts->iowait_sleeptime + ktime_sub(ktime_get(), ts->idle_entrytime) //try_to_wake_up(TASK A) rq(0)->nr_iowait-- //idle exit // See nr_iowait_cpu(0) == 0 ts->idle_sleeptime += ktime_sub(ktime_get(), ts->idle_entrytime) As a result subsequent reads on /proc/stat may expose backward progress. This is unfortunately hardly fixable. Just add a comment about that condition. Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 757e03ef4409..bd9499fb15b3 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -706,7 +706,10 @@ static u64 get_cpu_sleep_time_us(int cpu, ktime_t *sleeptime, * counters if NULL. * * Return the cumulative idle time (since boot) for a given - * CPU, in microseconds. + * CPU, in microseconds. Note this is partially broken due to + * the counter of iowait tasks that can be remotely updated without + * any synchronization. Therefore it is possible to observe backward + * values within two consecutive reads. * * This time is measured via accounting rather than sampling, * and is as accurate as ktime_get() is. @@ -729,7 +732,10 @@ EXPORT_SYMBOL_GPL(get_cpu_idle_time_us); * counters if NULL. * * Return the cumulative iowait time (since boot) for a given - * CPU, in microseconds. + * CPU, in microseconds. Note this is partially broken due to + * the counter of iowait tasks that can be remotely updated without + * any synchronization. Therefore it is possible to observe backward + * values within two consecutive reads. * * This time is measured via accounting rather than sampling, * and is as accurate as ktime_get() is. From patchwork Fri Feb 10 14:09:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55418 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp977641wrn; Fri, 10 Feb 2023 06:20:11 -0800 (PST) X-Google-Smtp-Source: AK7set8xAmBkpuwyki9yHLM68JfmYjFaRtey2eTbQLxTc3Q5NrjBjz2AKAPXWzhiKhFZRGB6cmJo X-Received: by 2002:a17:907:6e8f:b0:88a:d760:19ac with SMTP id sh15-20020a1709076e8f00b0088ad76019acmr22010093ejc.1.1676038811519; Fri, 10 Feb 2023 06:20:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038811; cv=none; d=google.com; s=arc-20160816; b=BIfYhoVEPOrMnN8kEWK9RtM9l8vT2onulVnJVhwrGQuRaTeuIT2GHSectoriPEDPQy jspymdRNokJn9UHq70PhnqfFIbKK6WBS3zqm+Hr3VUoN3++02KkOF/6wABBlz+CBRDiA nEEw40kARZv57CkCv6FhVR5KD78O+RtARH1YcBsv8e0Yqoq+AVKg2cMGViOFauieUAfM ly5DChoq6QVl40oF5lGdeVMsEPF/BBV23q3fdoyOTHCK5LNUSn5GNSYhbH3jwpRThXV5 V08Ag3Eo3zBx+ide/G8HmYxKzKsBTMl2SICbZIoDJey18/7Lg3AUYWhmSWM7fmsl/IGo miOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Inuo+DccMpte9YIXtwQ9ihZlrHO32Y9h4Zg41uQmAmo=; b=we9tKeQNW13e4eHScZ3jZ6Gj6hOLhF2eQiBxIb2K7nCVuglYTYbb2tMD/r4H432gaa R0taVyFzJwdCQYYPknPtwC3w4Vr7FK9cIK8dMbWf/RVy6ByH33cg/+44/YcJHKNkIx5q l5MWARvkw/Q5FTs6SafZX/zdEy9S0j1MVFAEnJxxXWGf29dG3TUahBzJbS3sJFAUsiMp K673vz95yzVfm7nMysFsmRJj4H3FvqHtnVUCpMnMKJocYsxia00AiZz1Dtqr4vZsXknx GfzHpc73l3BPrVKERBvIJB+959HVWj73A63azm+NlZyB8HyHV2WIaI038/VpH6SmqUoK vuoA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="K/0jBDC/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n29-20020a17090673dd00b0088eefdc75c8si6379496ejl.732.2023.02.10.06.19.48; Fri, 10 Feb 2023 06:20:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="K/0jBDC/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232491AbjBJOKR (ORCPT + 99 others); Fri, 10 Feb 2023 09:10:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232489AbjBJOJx (ORCPT ); Fri, 10 Feb 2023 09:09:53 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F01B74077 for ; Fri, 10 Feb 2023 06:09:40 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 9E4E3B8252A for ; Fri, 10 Feb 2023 14:09:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21966C4339B; Fri, 10 Feb 2023 14:09:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038177; bh=/jXn/vnDl0cDz/KxBmavjhz+/0Xao8NYmx2pabP5Zx4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K/0jBDC/fMsnlIotZfR27XyUnoZAGtly6nQErhHRgQGnqqJKeNqcwk4AifODzILik kAKP+wCFxOBpg8K7GGp48xIXbzDFrmmS6IbNgnYFH0F5iB3DUg02eTjc0xi/YLHbt7 Fp5IDGzrrXA6YVu2fiqpcTa/JSZkR8I467GGH59b1zgQSHmKQrY0ZMGTCw1QK+zu6p bT4dkVZGhq7YrXFv4dxexV95Rylakm7gf3RqcBFSLBz9kvtlPffJ1pnKh7pXozchBd RUX8WwYq4wE8/s6gYsYcntQXf/vuqwmUvLGETk+SYEMrS5EdFw6I2S026Oa23G8pBc TMzu2JVON+NvQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 5/6] timers/nohz: Remove middle-function __tick_nohz_idle_stop_tick() Date: Fri, 10 Feb 2023 15:09:16 +0100 Message-Id: <20230210140917.279062-6-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757454072620207299?= X-GMAIL-MSGID: =?utf-8?q?1757454072620207299?= There is no need for the __tick_nohz_idle_stop_tick() function between tick_nohz_idle_stop_tick() and its implementation. Remove that unnecessary step. Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- kernel/time/tick-sched.c | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index bd9499fb15b3..7065c6e373e9 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -1080,10 +1080,16 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) return true; } -static void __tick_nohz_idle_stop_tick(struct tick_sched *ts) +/** + * tick_nohz_idle_stop_tick - stop the idle tick from the idle task + * + * When the next event is more than a tick into the future, stop the idle tick + */ +void tick_nohz_idle_stop_tick(void) { + struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); + int cpu = smp_processor_id(); ktime_t expires; - int cpu = smp_processor_id(); /* * If tick_nohz_get_sleep_length() ran tick_nohz_next_event(), the @@ -1115,16 +1121,6 @@ static void __tick_nohz_idle_stop_tick(struct tick_sched *ts) } } -/** - * tick_nohz_idle_stop_tick - stop the idle tick from the idle task - * - * When the next event is more than a tick into the future, stop the idle tick - */ -void tick_nohz_idle_stop_tick(void) -{ - __tick_nohz_idle_stop_tick(this_cpu_ptr(&tick_cpu_sched)); -} - void tick_nohz_idle_retain_tick(void) { tick_nohz_retain_tick(this_cpu_ptr(&tick_cpu_sched)); From patchwork Fri Feb 10 14:09:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Weisbecker X-Patchwork-Id: 55419 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp977816wrn; Fri, 10 Feb 2023 06:20:32 -0800 (PST) X-Google-Smtp-Source: AK7set8vU4/DmFlsuIF0AmyMVJLq4yIJtJAC15RCgK6IbsPKW9EPP5US1LTF7wxU6ZF5upYlK2+Q X-Received: by 2002:a50:871a:0:b0:4ab:1686:e90c with SMTP id i26-20020a50871a000000b004ab1686e90cmr8161859edb.15.1676038832012; Fri, 10 Feb 2023 06:20:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676038831; cv=none; d=google.com; s=arc-20160816; b=xLQ65RiXftGoFLvXNqqXXHOFC269DXEeHmMetRLn1+E6szknqbZPps+tGgPdVWrc97 +9Gfzl8aDCLXQSi3KpDpmCqW30FdoIdI4h+BqAiWI7qfLvnmENFhCjnbzmbcxKIBnOXC +TAvVvA6lU8IvjdrLolP22579S/JKVTTjMgiLpcVRiBxvNtkTDlYf4v3hCEpocArcrkE eiRAm0nyHMPvP9W1W/rS2HZtmMEL5v4MGtaiGrhXi0K03BlKzRDcovpmSI6bY3X10Zl1 oCyNwWWL5k8dLzxaEIYzeQkluVuwkEtkNtQsTTIjBDBjVQKcZCeNO8D/4e733kUR3sOP rgPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tuPCigz2r7EepdqDQX0dC6S/zNlFxwTf1mLwV19C4Ys=; b=fhuNhPfo6yFiyW0dbP0qbCIVSNUwR+nqCzS+t5tJGXgBiZ6qe8ZWAp3AwqEMlUnSw2 E/cuJwaKy2TC2KqmY7VtK3Nv9pWa1IWiULFlB7xUylAYirdJgePjMkG0g0Z8nZLqko8+ 6SzWSpYUq8fqMXFy58S3K8y5ab3L4fpD+kv4jJI1vjRmdNJcdJ11jON8+2ZIRV8LzUsg eEy1tx/y8TqEYq89s98N1mfUMq8snRQ6rciUkCi4a5yLejkK5dROCQf9LP041/qt8ykk 7FlxEsai3paZfAUlplMX0EcHmuwoIlnWcYbff7hT0JXuW/vcoUOByl2oG37z2G31Cxyr eEqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=SQXYyBgD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j16-20020aa7de90000000b004ab498f69c4si2138633edv.242.2023.02.10.06.20.08; Fri, 10 Feb 2023 06:20:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=SQXYyBgD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232504AbjBJOK1 (ORCPT + 99 others); Fri, 10 Feb 2023 09:10:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232500AbjBJOJz (ORCPT ); Fri, 10 Feb 2023 09:09:55 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31A0E74064 for ; Fri, 10 Feb 2023 06:09:41 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7F7B361DD2 for ; Fri, 10 Feb 2023 14:09:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB131C43445; Fri, 10 Feb 2023 14:09:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1676038179; bh=gtKeflSOf/kRVKW9lvZG7UlWcNwX9gYbrJOAj1/IZRk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SQXYyBgD/vKrwdGKQ0x0+iUltbLAg5bsSTaW72Oos3ne4vHjakkeJNoKHMoLrsMR3 JO9WGdZmWyo3n8tGhl8kSk3vEDGgaRrKzhWpde8OfJohw1Aoi0VXqQ5I+wng3XwWDS pG5AkMRXEoq2JiK9/GO1bRrXGazyvq00egHl69bOPtoeLuYtliEPEaJWGqDN0SBs6X 23uZRwjunYcxr8Ekf0BVaTjGrdX8kXsza51ASUc0//fOGkULnR5nz/6URabnxDwmRo ORj0p4IT/A3Ah95EB8iMKgA9wwHy6wA7YssnD/TeqXGDtMdEGK9sLu8UY3IFANdA0F OWSNLhZO7yqVA== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Alexey Dobriyan , Peter Zijlstra , Wei Li , Mirsad Goran Todorovac , Thomas Gleixner , Yu Liao , Hillf Danton , Ingo Molnar Subject: [PATCH 6/6] MAINTAINERS: Remove stale email address Date: Fri, 10 Feb 2023 15:09:17 +0100 Message-Id: <20230210140917.279062-7-frederic@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230210140917.279062-1-frederic@kernel.org> References: <20230210140917.279062-1-frederic@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757454094297045129?= X-GMAIL-MSGID: =?utf-8?q?1757454094297045129?= Cc: Hillf Danton Cc: Yu Liao Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Wei Li Cc: Alexey Dobriyan Cc: Mirsad Goran Todorovac Cc: Peter Zijlstra Signed-off-by: Frederic Weisbecker --- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index fb1471cb5ed3..300ca61fa0bc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14794,7 +14794,7 @@ F: include/uapi/linux/nitro_enclaves.h F: samples/nitro_enclaves/ NOHZ, DYNTICKS SUPPORT -M: Frederic Weisbecker +M: Frederic Weisbecker M: Thomas Gleixner M: Ingo Molnar L: linux-kernel@vger.kernel.org