From patchwork Mon Oct 31 10:53:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tip-bot2 for Thomas Gleixner X-Patchwork-Id: 13200 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2238153wru; Mon, 31 Oct 2022 03:54:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5e/6BtSnGc0bN71rMFoFMNWa3VWY4yfoXtEkZZjKHabw68o3vAn2MqH/DFBjJp8AeDFqyw X-Received: by 2002:a05:6a00:84b:b0:56d:3cf5:1031 with SMTP id q11-20020a056a00084b00b0056d3cf51031mr8892034pfk.75.1667213676089; Mon, 31 Oct 2022 03:54:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667213676; cv=none; d=google.com; s=arc-20160816; b=nlFRaCHMrD0cBlyO9wc82eV2znLrkhgChb/ni3eDmiH5N8200r45o/Th18H+QIfoN4 e/EL0iTKjHqkXfCp4Aygn0wVI9m1yJONga88BC953Z+HlXXUjskVuBzh/j+pSsMPjJot sG3yUrw0vS4O6q74M1AkZpwAxOBTwUWcddUX9u1dXa2o3GpDWhtcMuMGpinYvxdyB8EY opuidJCXjWnFPQEtQUeZppQGVLJWg7OkFm+KPGlcfaqVV3O+CNPaU1gO6i/eOoe5sfma mVs3FpPQpc41RjOMmkCY4UfX8JU2OUYV8N0jYdtgi4RfFwKvxXRHJspA//lDzN2hj/5q 6uMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:robot-unsubscribe :robot-id:message-id:mime-version:references:in-reply-to:cc:subject :to:reply-to:sender:from:dkim-signature:dkim-signature:date; bh=oaNurCJbtcwOCKgbGDmdD1pyoT5KwJzQ0tjIxKc/eqo=; b=I8vAdLnrU7eeuB6DJYTuCxx5ZRnpUHvng1h/xWaNty4R+osnna/QvdtrZJkdBoj39T P5N3EHD/xW1EXWgxsZJ/qjsLdT52RCGyFXfAz0Vx85kbc6TgHiQhpeb8pL6qqeAMPpaz s84GtNNWjXIUq4dQtq8Mnbrojse8Zb1B+FNml3q1KAlMs3iAoPHuSLjGKv/P6adjZ4G0 RV7pPF/yXEN0hBrwBAKZmh0sdRKroeX9GB2rwyQqBP8Mu2tOH5crXfW4E9l8Vj5Sh70A YqdGJpGaOg3c0YYFz1mx5GaSWvsk2UE1kAM2jpXs2CBIKzc/1MOH7kdWDTZ1AadWqwbm +rQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=lRtlelk4; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q6-20020a635046000000b004541962a9basi8149176pgl.701.2022.10.31.03.54.23; Mon, 31 Oct 2022 03:54:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=lRtlelk4; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230049AbiJaKyI (ORCPT + 99 others); Mon, 31 Oct 2022 06:54:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230097AbiJaKyC (ORCPT ); Mon, 31 Oct 2022 06:54:02 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1B623A6; Mon, 31 Oct 2022 03:54:00 -0700 (PDT) Date: Mon, 31 Oct 2022 10:53:56 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1667213638; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oaNurCJbtcwOCKgbGDmdD1pyoT5KwJzQ0tjIxKc/eqo=; b=lRtlelk4CsmPqa3Q8irKCRH0pJZCWWmB6UWpqOrSPXmzFyh6bvAHnBCchvC+RputDwJC0c TUhyjBGO6gVilO4K8Km3PdfWErSq6zya75VR5Che2yFaLnOMY55BRZjkP63wFIxlYGCgVt f172eteAr/FY0lPkz2IXdnTrJJmmiG02DasMqKOiFEgTtbrk3w200Mme+1qXscRhmxZVYj 1s3ROQvog3q4UFZdgbY/SINNf57QksLAQRZr1hfI+kUIUNK1dpDJEpwmKiObVjZI5NUEqS sYKNRU+P8iZSlSyh9TlLlFkFO9lWQo/9pKEDSvdWtNQEYMz95a1cpvlNcJROvQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1667213638; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oaNurCJbtcwOCKgbGDmdD1pyoT5KwJzQ0tjIxKc/eqo=; b=PK8Wo3/GlBuqjkCNuVV568nsSsmniyFjhVnSxX4X3TJJAaq/XgFfmZM0KlGFxasabNZ8rt /1KXMGEmVSmN7qBw== From: "tip-bot2 for Chengming Zhou" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/psi: Use task->psi_flags to clear in CPU migration Cc: Chengming Zhou , "Peter Zijlstra (Intel)" , Johannes Weiner , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20220926081931.45420-1-zhouchengming@bytedance.com> References: <20220926081931.45420-1-zhouchengming@bytedance.com> MIME-Version: 1.0 Message-ID: <166721363659.7716.10522729624614709628.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748200247694190203?= X-GMAIL-MSGID: =?utf-8?q?1748200247694190203?= The following commit has been merged into the sched/core branch of tip: Commit-ID: 52b33d87b9197c51e8ffdc61873739d90dd0a16f Gitweb: https://git.kernel.org/tip/52b33d87b9197c51e8ffdc61873739d90dd0a16f Author: Chengming Zhou AuthorDate: Mon, 26 Sep 2022 16:19:31 +08:00 Committer: Peter Zijlstra CommitterDate: Sun, 30 Oct 2022 10:12:15 +01:00 sched/psi: Use task->psi_flags to clear in CPU migration The commit d583d360a620 ("psi: Fix psi state corruption when schedule() races with cgroup move") fixed a race problem by making cgroup_move_task() use task->psi_flags instead of looking at the scheduler state. We can extend task->psi_flags usage to CPU migration, which should be a minor optimization for performance and code simplicity. Signed-off-by: Chengming Zhou Signed-off-by: Peter Zijlstra (Intel) Acked-by: Johannes Weiner Link: https://lore.kernel.org/r/20220926081931.45420-1-zhouchengming@bytedance.com --- include/linux/sched.h | 3 --- kernel/sched/core.c | 2 +- kernel/sched/stats.h | 22 ++++------------------ 3 files changed, 5 insertions(+), 22 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index ffb6eb5..23de7fe 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -888,9 +888,6 @@ struct task_struct { unsigned sched_reset_on_fork:1; unsigned sched_contributes_to_load:1; unsigned sched_migrated:1; -#ifdef CONFIG_PSI - unsigned sched_psi_wake_requeue:1; -#endif /* Force alignment to the next boundary: */ unsigned :0; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 87c9cdf..07ac08c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2053,7 +2053,7 @@ static inline void enqueue_task(struct rq *rq, struct task_struct *p, int flags) if (!(flags & ENQUEUE_RESTORE)) { sched_info_enqueue(rq, p); - psi_enqueue(p, flags & ENQUEUE_WAKEUP); + psi_enqueue(p, (flags & ENQUEUE_WAKEUP) && !(flags & ENQUEUE_MIGRATED)); } uclamp_rq_inc(rq, p); diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h index 84a1889..38f3698 100644 --- a/kernel/sched/stats.h +++ b/kernel/sched/stats.h @@ -128,11 +128,9 @@ static inline void psi_enqueue(struct task_struct *p, bool wakeup) if (p->in_memstall) set |= TSK_MEMSTALL_RUNNING; - if (!wakeup || p->sched_psi_wake_requeue) { + if (!wakeup) { if (p->in_memstall) set |= TSK_MEMSTALL; - if (p->sched_psi_wake_requeue) - p->sched_psi_wake_requeue = 0; } else { if (p->in_iowait) clear |= TSK_IOWAIT; @@ -143,8 +141,6 @@ static inline void psi_enqueue(struct task_struct *p, bool wakeup) static inline void psi_dequeue(struct task_struct *p, bool sleep) { - int clear = TSK_RUNNING; - if (static_branch_likely(&psi_disabled)) return; @@ -157,10 +153,7 @@ static inline void psi_dequeue(struct task_struct *p, bool sleep) if (sleep) return; - if (p->in_memstall) - clear |= (TSK_MEMSTALL | TSK_MEMSTALL_RUNNING); - - psi_task_change(p, clear, 0); + psi_task_change(p, p->psi_flags, 0); } static inline void psi_ttwu_dequeue(struct task_struct *p) @@ -172,19 +165,12 @@ static inline void psi_ttwu_dequeue(struct task_struct *p) * deregister its sleep-persistent psi states from the old * queue, and let psi_enqueue() know it has to requeue. */ - if (unlikely(p->in_iowait || p->in_memstall)) { + if (unlikely(p->psi_flags)) { struct rq_flags rf; struct rq *rq; - int clear = 0; - - if (p->in_iowait) - clear |= TSK_IOWAIT; - if (p->in_memstall) - clear |= TSK_MEMSTALL; rq = __task_rq_lock(p, &rf); - psi_task_change(p, clear, 0); - p->sched_psi_wake_requeue = 1; + psi_task_change(p, p->psi_flags, 0); __task_rq_unlock(rq, &rf); } }