From patchwork Tue Apr 18 14:53:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 84905 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2912709vqo; Tue, 18 Apr 2023 07:54:50 -0700 (PDT) X-Google-Smtp-Source: AKy350ZFgKi3apW7m5RQQgYbqEnYN0XPKKVNfkZbdTA+62Nz18RTP39j6bRjcJRkol4tDq2XRSgq X-Received: by 2002:a05:6a20:9145:b0:d6:7264:f44e with SMTP id x5-20020a056a20914500b000d67264f44emr119861pzc.3.1681829690399; Tue, 18 Apr 2023 07:54:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681829690; cv=none; d=google.com; s=arc-20160816; b=XXWlD+j2rb3VvyaUDT0thdz60spfUDFCm7gmXBSaxO4nCSXq/jBhvNIYSfZ9/l85ML o66yiZ9TAeaCLlEEVbeC6WSApm9RiSavbh97gwdm3BGQ/wk/Sh8G3EFNV5wMzb0y/4mj UTjpnOY99HJABGu0UwqMgE9S9Dv8AYa7xmfNdEV+j+IEM8puTD8JO3NpV12QBPgDRZoU IQzWT1NhRJNUs61odxepQhBzUVTYiCx6jZ3zr7EJW5swsKyjjpWD+b1vflNMf+P8fI/I iesj4qZ7ENH26pXm9r9sBno/tp4BxLC1dEE/2ZYcFiy3XGcC5iGFfgLjUB7/bkzW+ZXt n/hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=idUFnLwc9DEicaszUNU0vcf7I7imSGPqvv4DdMdr8TA=; b=vs7dnUl28tUbtE4d9pJd6IM44A9aF5zKOVkQbAvoNNTYfbZKYwM3s7+6Ke61xMt9Kg 0BmJ2ivxtb//GXZt1o6xHO1jt/uJekxL+VN6IVY/NxA6vo38uVc/+EyLdrLfPHoMI4Ko zRsMNfGryyQxh98TNmQcZLDJCA/579rf5ZpwBXzFsNDhuUgzHVJKhfT6ctPtxPFK1jTL 1XBVIquJIHttFTXred/GSc5DGYwZzpkHbq4ptr2QfHxUVwpGvAlUprUUm7Ytqn46B7aG BHYmQIOMA7nPBxcuWZlOltPL82Iqj4GmF0PCdkhYyoSvZFNLi5RWMizAZ5b+svQDWaYj RqBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=mV2CO4dm; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k12-20020a17090a62cc00b00247ad6e4188si3308414pjs.51.2023.04.18.07.54.34; Tue, 18 Apr 2023 07:54:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=mV2CO4dm; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232100AbjDROxt (ORCPT + 99 others); Tue, 18 Apr 2023 10:53:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231194AbjDROxe (ORCPT ); Tue, 18 Apr 2023 10:53:34 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FE2FAD2A; Tue, 18 Apr 2023 07:53:31 -0700 (PDT) Date: Tue, 18 Apr 2023 14:53:29 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1681829609; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=idUFnLwc9DEicaszUNU0vcf7I7imSGPqvv4DdMdr8TA=; b=mV2CO4dmI6IEPEdsbfIHwEROgvvjDutm5oXeKCuSPCD9nBdriYVBIblnu8rharWNHnmYE3 Su6JU9JBwoVt7M/vgEvJ1eeifnzT4eVXJDPHsApMx2NsfbGPlpxY33spA9AEYWHQXAPII+ kNoM9ZI/JTYPT0xqR0x89TiqGEVWMY39pv3Ax+oZZr7E4Nw+xhgiyg70te8bzcKULMMtoo 2P0qKQvGs6Mx++zDWeIDYfMHMvX6BwdaGPU37rjZF22gwDK37JVhKnjceg9IICs+LLWp0T YnROErOEHr5TBIfFqYMgQhSi/UXJU+HgGV3/PXtL4EiqMwdLJIrOgGxJ+GSkyQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1681829609; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=idUFnLwc9DEicaszUNU0vcf7I7imSGPqvv4DdMdr8TA=; b=hejqBdWfk07hDMeXqdHKjDzZ6XFbBu7X9RWV8G29DlTc6xy2gSs28kwXyMhmYtywx2g2cS oeRKITnDX1iPs7CA== From: "tip-bot2 for Frederic Weisbecker" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: timers/core] timers/nohz: Add a comment about broken iowait counter update race Cc: Frederic Weisbecker , Thomas Gleixner , "Peter Zijlstra (Intel)" , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20230222144649.624380-5-frederic@kernel.org> References: <20230222144649.624380-5-frederic@kernel.org> MIME-Version: 1.0 Message-ID: <168182960920.404.6558173122716747219.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1758543252374747903?= X-GMAIL-MSGID: =?utf-8?q?1763526249523839185?= The following commit has been merged into the timers/core branch of tip: Commit-ID: ead70b75237371c735a481a9843b411cfbb18404 Gitweb: https://git.kernel.org/tip/ead70b75237371c735a481a9843b411cfbb18404 Author: Frederic Weisbecker AuthorDate: Wed, 22 Feb 2023 15:46:45 +01:00 Committer: Thomas Gleixner CommitterDate: Tue, 18 Apr 2023 16:35:12 +02:00 timers/nohz: Add a comment about broken iowait counter update race The per-cpu iowait task counter is incremented locally upon sleeping. But since the task can be woken to (and by) another CPU, the counter may then be decremented remotely. This is the source of a race involving readers VS writer of idle/iowait sleeptime. The following scenario shows an example where a /proc/stat reader observes a pending sleep time as IO whereas that pending sleep time later eventually gets accounted as non-IO. CPU 0 CPU 1 CPU 2 ----- ----- ------ //io_schedule() TASK A current->in_iowait = 1 rq(0)->nr_iowait++ //switch to idle // READ /proc/stat // See nr_iowait_cpu(0) == 1 return ts->iowait_sleeptime + ktime_sub(ktime_get(), ts->idle_entrytime) //try_to_wake_up(TASK A) rq(0)->nr_iowait-- //idle exit // See nr_iowait_cpu(0) == 0 ts->idle_sleeptime += ktime_sub(ktime_get(), ts->idle_entrytime) As a result subsequent reads on /proc/stat may expose backward progress. This is unfortunately hardly fixable. Just add a comment about that condition. Signed-off-by: Frederic Weisbecker Signed-off-by: Thomas Gleixner Acked-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/20230222144649.624380-5-frederic@kernel.org --- kernel/time/tick-sched.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 90d9b7b..edd6e9f 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -705,7 +705,10 @@ static u64 get_cpu_sleep_time_us(struct tick_sched *ts, ktime_t *sleeptime, * counters if NULL. * * Return the cumulative idle time (since boot) for a given - * CPU, in microseconds. + * CPU, in microseconds. Note this is partially broken due to + * the counter of iowait tasks that can be remotely updated without + * any synchronization. Therefore it is possible to observe backward + * values within two consecutive reads. * * This time is measured via accounting rather than sampling, * and is as accurate as ktime_get() is. @@ -728,7 +731,10 @@ EXPORT_SYMBOL_GPL(get_cpu_idle_time_us); * counters if NULL. * * Return the cumulative iowait time (since boot) for a given - * CPU, in microseconds. + * CPU, in microseconds. Note this is partially broken due to + * the counter of iowait tasks that can be remotely updated without + * any synchronization. Therefore it is possible to observe backward + * values within two consecutive reads. * * This time is measured via accounting rather than sampling, * and is as accurate as ktime_get() is.