From patchwork Wed Dec 6 06:33:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yajun Deng X-Patchwork-Id: 174318 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3917371vqy; Tue, 5 Dec 2023 22:34:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IGHYKiPC2HVXPskca7IXS3GbsLEly6i2V/euzQBK9AJgLefzwQIUnYrmRVXWtKCHySV1aKa X-Received: by 2002:a05:6358:9d85:b0:170:1d75:a85e with SMTP id d5-20020a0563589d8500b001701d75a85emr545648rwo.48.1701844483080; Tue, 05 Dec 2023 22:34:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701844483; cv=none; d=google.com; s=arc-20160816; b=Z9BMAlwjFC7pDplxP+T6vRgLBnfiHSkoyrW6eHzS2qqIBs5kSMASpQ5PPLaHBqLJOC 9xK/34780LT713eIDKrTpgvdXdQLaxX5qGUo049oTfZ0f4FioJ6CY3qy79GJKn60TkfH QUdNcHr+F0U61NeQDu6DoOvivvqQMaeeaq3sqIIsMfIT7Nod9E7Bq8497rZNrIgqCEzk NR2CLvMxE5QgyDBvMRtUMjH7PKRubtGVjYrpEGfbvH9II0B2zO0hdSPKkaSeygRoyID1 3O8IkgTppr5wFOI9+Dm0kNkUYZLVaip6A7Bl261Wuv1r70MSO4V+lDZ8ygl3ZBQd9d3X amkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3+ur4a0hWU+7jS1cQdkoVwB5xk0PqQxzIYs0iU9W36Q=; fh=kY5sJNvn29qhEh2SatEVpXE2cTkcQ8AnU+xu9Bnn8Ic=; b=RanIBjDUUHzvG4wUcwxBEA+EolM0aYKaX8iMqT8MS6RHnPnSe1X+6Y5lsc53Dww6Qj 4RA85pcJtCnM/Dy24p+hnpCEOC+bkebQVO1UxGuQrUwGZ92BXWquk9FzEhZ5G2qc9g2a fSMXl+SRzutOzuab93U/wCzfjGxrKxUTCXanKzBuyk0yW4012vOHk57QtWMMIg6p/z4m jwGzQ9k/odl4AMeuxJsyzMI46Uhw/vOtXFInMDXlHtXjAfY2NPnS6knVfAP9PDg3Bl0f hr6LJuXibr3JcDr73Q826ha/ByX7FHv7JhTRZUiY+ge2Dw0sHfK2PPMN5FLLr3+I9NQ1 SGrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="THWr/WsI"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id e19-20020a631e13000000b00589884fef91si7112507pge.740.2023.12.05.22.34.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Dec 2023 22:34:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b="THWr/WsI"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 1610E802FB82; Tue, 5 Dec 2023 22:34:40 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376797AbjLFGe1 (ORCPT + 99 others); Wed, 6 Dec 2023 01:34:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231374AbjLFGe0 (ORCPT ); Wed, 6 Dec 2023 01:34:26 -0500 Received: from out-172.mta0.migadu.com (out-172.mta0.migadu.com [91.218.175.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D040D1BD for ; Tue, 5 Dec 2023 22:34:31 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1701844469; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3+ur4a0hWU+7jS1cQdkoVwB5xk0PqQxzIYs0iU9W36Q=; b=THWr/WsIUQYJV3nh7/K0bWCOiqU/xSaMB1QTvbGUaM8Xrm6iojxUSSW0cF8hUwqCTicbn2 HYKEqJPLH3cDWODr4m7Ph7tt/gohXtZiPp/Hed0RSj8srIVAuDqmJ62G7hrtNgBeHj8T6+ 512YdlsETVcVx7Y8YX4J3WPDz+7UGmQ= From: Yajun Deng To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yajun Deng Subject: [PATCH v2 1/2] sched/fair: Return NULL when entity isn't a task in task_of() Date: Wed, 6 Dec 2023 14:33:59 +0800 Message-Id: <20231206063400.3206346-2-yajun.deng@linux.dev> In-Reply-To: <20231206063400.3206346-1-yajun.deng@linux.dev> References: <20231206063400.3206346-1-yajun.deng@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 05 Dec 2023 22:34:40 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784513280672556102 X-GMAIL-MSGID: 1784513280672556102 Before calling task_of(), we need to make sure that the entity is a task. There is also a warning in task_of() if the entity isn't a task. That means we need to check the entity twice. If the entity isn't a task, return the task struct is meaningless. Return NULL when entity isn't a task in task_of(), and call task_of() instead of entity_is_task() when we need a task_struct. Signed-off-by: Yajun Deng --- kernel/sched/fair.c | 50 +++++++++++++++++++++----------------------- kernel/sched/sched.h | 4 +++- 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index bcd0f230e21f..5003c8501398 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -470,8 +470,10 @@ static int cfs_rq_is_idle(struct cfs_rq *cfs_rq) static int se_is_idle(struct sched_entity *se) { - if (entity_is_task(se)) - return task_has_idle_policy(task_of(se)); + struct task_struct *p = task_of(se); + + if (p) + return task_has_idle_policy(p); return cfs_rq_is_idle(group_cfs_rq(se)); } @@ -1156,6 +1158,7 @@ s64 update_curr_common(struct rq *rq) static void update_curr(struct cfs_rq *cfs_rq) { struct sched_entity *curr = cfs_rq->curr; + struct task_struct *p; s64 delta_exec; if (unlikely(!curr)) @@ -1169,8 +1172,9 @@ static void update_curr(struct cfs_rq *cfs_rq) update_deadline(cfs_rq, curr); update_min_vruntime(cfs_rq); - if (entity_is_task(curr)) - update_curr_task(task_of(curr), delta_exec); + p = task_of(curr); + if (p) + update_curr_task(p, delta_exec); account_cfs_rq_runtime(cfs_rq, delta_exec); } @@ -1184,24 +1188,19 @@ static inline void update_stats_wait_start_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct sched_statistics *stats; - struct task_struct *p = NULL; if (!schedstat_enabled()) return; stats = __schedstats_from_se(se); - if (entity_is_task(se)) - p = task_of(se); - - __update_stats_wait_start(rq_of(cfs_rq), p, stats); + __update_stats_wait_start(rq_of(cfs_rq), task_of(se), stats); } static inline void update_stats_wait_end_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct sched_statistics *stats; - struct task_struct *p = NULL; if (!schedstat_enabled()) return; @@ -1217,27 +1216,20 @@ update_stats_wait_end_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) if (unlikely(!schedstat_val(stats->wait_start))) return; - if (entity_is_task(se)) - p = task_of(se); - - __update_stats_wait_end(rq_of(cfs_rq), p, stats); + __update_stats_wait_end(rq_of(cfs_rq), task_of(se), stats); } static inline void update_stats_enqueue_sleeper_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct sched_statistics *stats; - struct task_struct *tsk = NULL; if (!schedstat_enabled()) return; stats = __schedstats_from_se(se); - if (entity_is_task(se)) - tsk = task_of(se); - - __update_stats_enqueue_sleeper(rq_of(cfs_rq), tsk, stats); + __update_stats_enqueue_sleeper(rq_of(cfs_rq), task_of(se), stats); } /* @@ -1263,6 +1255,7 @@ update_stats_enqueue_fair(struct cfs_rq *cfs_rq, struct sched_entity *se, int fl static inline void update_stats_dequeue_fair(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { + struct task_struct *tsk; if (!schedstat_enabled()) return; @@ -1274,8 +1267,8 @@ update_stats_dequeue_fair(struct cfs_rq *cfs_rq, struct sched_entity *se, int fl if (se != cfs_rq->curr) update_stats_wait_end_fair(cfs_rq, se); - if ((flags & DEQUEUE_SLEEP) && entity_is_task(se)) { - struct task_struct *tsk = task_of(se); + tsk = task_of(se); + if ((flags & DEQUEUE_SLEEP) && tsk) { unsigned int state; /* XXX racy against TTWU */ @@ -3569,12 +3562,14 @@ static inline void update_scan_period(struct task_struct *p, int new_cpu) static void account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) { + struct task_struct *p = task_of(se); + update_load_add(&cfs_rq->load, se->load.weight); #ifdef CONFIG_SMP - if (entity_is_task(se)) { + if (p) { struct rq *rq = rq_of(cfs_rq); - account_numa_enqueue(rq, task_of(se)); + account_numa_enqueue(rq, p); list_add(&se->group_node, &rq->cfs_tasks); } #endif @@ -3586,10 +3581,12 @@ account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) static void account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) { + struct task_struct *p = task_of(se); + update_load_sub(&cfs_rq->load, se->load.weight); #ifdef CONFIG_SMP - if (entity_is_task(se)) { - account_numa_dequeue(rq_of(cfs_rq), task_of(se)); + if (p) { + account_numa_dequeue(rq_of(cfs_rq), p); list_del_init(&se->group_node); } #endif @@ -5323,9 +5320,10 @@ static __always_inline void return_cfs_rq_runtime(struct cfs_rq *cfs_rq); static void dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { + struct task_struct *p = task_of(se); int action = UPDATE_TG; - if (entity_is_task(se) && task_on_rq_migrating(task_of(se))) + if (p && task_on_rq_migrating(p)) action |= DO_DETACH; /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 001fe047bd5d..6baba9ebcde1 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1432,7 +1432,9 @@ static inline void update_idle_core(struct rq *rq) { } #ifdef CONFIG_FAIR_GROUP_SCHED static inline struct task_struct *task_of(struct sched_entity *se) { - SCHED_WARN_ON(!entity_is_task(se)); + if (!entity_is_task(se)) + return NULL; + return container_of(se, struct task_struct, se); } From patchwork Wed Dec 6 06:34:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yajun Deng X-Patchwork-Id: 174319 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp3917408vqy; Tue, 5 Dec 2023 22:34:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IF8lqZ476GDlQETOJJjQUL8QQomxj9eCiuvQoWkCxg6Tj9BS+lbdmQr7x5Vkjaj07Sql+cm X-Received: by 2002:a17:902:7443:b0:1d0:7908:ab6d with SMTP id e3-20020a170902744300b001d07908ab6dmr143965plt.11.1701844488926; Tue, 05 Dec 2023 22:34:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701844488; cv=none; d=google.com; s=arc-20160816; b=DKnoZH6ZDjjznNnNti/i85iqm0036kNfQEkaz3jAsu0KpEovTAaKlFlYF96f4hAia0 En2JjLPLy8XN11rbZmTXUBMP75p+zJs4ethEeivTyKDQkix7vCspnBld74zsxedFWg1a 2aE9GkvJbdya725O234/OlskXNoBC9NdaHvo23dW28wwE8AnChswVXgPCKON0fEV4HOh TsfTH6F7uVcNYfPvMNHyiiZ92JkCsO1cv/vnZaT/tsUjMJGQesr/0olAbep6aeFlBE5n jKFCsVCL7ETucu1AVPDUn5PJHHFOytGj8jNDQCrJvVaWC3K5ImRt8c3Av+2GhPvgSHMf gXHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0lDBFEoz+SUMLKMlohSt70DLpSaHMM5qAM0YknDYPRc=; fh=kY5sJNvn29qhEh2SatEVpXE2cTkcQ8AnU+xu9Bnn8Ic=; b=MwBJyqDcMOj6aLnYEFfILwYwdnwb7D00wfJJx27hkQwxG9sGUIxo+PnQEwJbC19O49 KUgLgO/VVg/ox6BQ8gLsqotGpP38Gfh8GVsuEIDGPZXZJCYygAxqOIzvz7JDKu2XDqtz dpAsV3oRL+K2Y56Z1XkmX8bX56+pe1GUaU3j/uU5bwvJzNlgdsFAP8ehE5jislIgDE9d nxZEhAUmIac0jypUwurcodr1pjxu+lS422OxkK6LdBzuBmeV1pfyRaU/ojQZHXgY8BGu 9KadgS1QGTS2vsWwrCJQql+o/Cc1MBa+IbKig1Sw9dm5QaEKj1Cz/QqO87nKrpmhZdwE 5V1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Jki4iV4V; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id q2-20020a170902edc200b001d0b4bfcbb9si3845171plk.574.2023.12.05.22.34.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Dec 2023 22:34:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=Jki4iV4V; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 024338043026; Tue, 5 Dec 2023 22:34:45 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376812AbjLFGef (ORCPT + 99 others); Wed, 6 Dec 2023 01:34:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376801AbjLFGee (ORCPT ); Wed, 6 Dec 2023 01:34:34 -0500 Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC868D4F for ; Tue, 5 Dec 2023 22:34:39 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1701844478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0lDBFEoz+SUMLKMlohSt70DLpSaHMM5qAM0YknDYPRc=; b=Jki4iV4V+jHYrHoOxnxxlqEsZzBkU+FuKWila9Z99n4hUwLNHQz0EzSQUNjYBhRnHw23yW CUMJhz/RW0ZCwzxGFYA7Yk1DyIAXDz1Ed0NWI5KHEaWVs4uDdRijRVZyDyzgaagPe9iJKp o0VTbdgdDU9d5P5u4enkXVbOxZI+2CQ= From: Yajun Deng To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yajun Deng Subject: [PATCH v2 2/2] sched/rt: Return NULL when rt entity isn't a task in rt_task_of() Date: Wed, 6 Dec 2023 14:34:00 +0800 Message-Id: <20231206063400.3206346-3-yajun.deng@linux.dev> In-Reply-To: <20231206063400.3206346-1-yajun.deng@linux.dev> References: <20231206063400.3206346-1-yajun.deng@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 05 Dec 2023 22:34:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784513286854633577 X-GMAIL-MSGID: 1784513286854633577 Before calling rt_task_of(), we need to make sure that the rt entity is a task. There is also a warning in rt_task_of() if the rt entity isn't a task. That means we need to check the rt entity twice. If the rt entity isn't a task, return the task struct is meaningless. Return NULL when rt entity isn't a task in rt_task_of(), and call rt_task_of() instead of rt_entity_is_task() when we need a task_struct. Signed-off-by: Yajun Deng --- v2: fix 'struct rt_rq' no member named 'highest_prio'. v1: https://lore.kernel.org/all/20231201022704.3526377-1-yajun.deng@linux.dev/ --- kernel/sched/rt.c | 60 ++++++++++++++--------------------------------- 1 file changed, 17 insertions(+), 43 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 40d24f64b444..061f5f005c35 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -169,9 +169,9 @@ static void destroy_rt_bandwidth(struct rt_bandwidth *rt_b) static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_SCHED_DEBUG - WARN_ON_ONCE(!rt_entity_is_task(rt_se)); -#endif + if (!rt_entity_is_task(rt_se)) + return NULL; + return container_of(rt_se, struct task_struct, rt); } @@ -1266,54 +1266,34 @@ static void __delist_rt_entity(struct sched_rt_entity *rt_se, struct rt_prio_arr rt_se->on_list = 0; } -static inline struct sched_statistics * -__schedstats_from_rt_se(struct sched_rt_entity *rt_se) -{ -#ifdef CONFIG_RT_GROUP_SCHED - /* schedstats is not supported for rt group. */ - if (!rt_entity_is_task(rt_se)) - return NULL; -#endif - - return &rt_task_of(rt_se)->stats; -} - static inline void update_stats_wait_start_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se) { - struct sched_statistics *stats; - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - - stats = __schedstats_from_rt_se(rt_se); - if (!stats) + p = rt_task_of(rt_se); + if (!p) return; - __update_stats_wait_start(rq_of_rt_rq(rt_rq), p, stats); + __update_stats_wait_start(rq_of_rt_rq(rt_rq), p, &p->stats); } static inline void update_stats_enqueue_sleeper_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se) { - struct sched_statistics *stats; - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - - stats = __schedstats_from_rt_se(rt_se); - if (!stats) + p = rt_task_of(rt_se); + if (!p) return; - __update_stats_enqueue_sleeper(rq_of_rt_rq(rt_rq), p, stats); + __update_stats_enqueue_sleeper(rq_of_rt_rq(rt_rq), p, &p->stats); } static inline void @@ -1330,34 +1310,28 @@ update_stats_enqueue_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se, static inline void update_stats_wait_end_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se) { - struct sched_statistics *stats; - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - - stats = __schedstats_from_rt_se(rt_se); - if (!stats) + p = rt_task_of(rt_se); + if (!p) return; - __update_stats_wait_end(rq_of_rt_rq(rt_rq), p, stats); + __update_stats_wait_end(rq_of_rt_rq(rt_rq), p, &p->stats); } static inline void update_stats_dequeue_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se, int flags) { - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - + p = rt_task_of(rt_se); if ((flags & DEQUEUE_SLEEP) && p) { unsigned int state;