From patchwork Fri Dec 1 02:27:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yajun Deng X-Patchwork-Id: 172218 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp833967vqy; Thu, 30 Nov 2023 18:27:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IE7Opr3+0Td5OZ5CXD+KkN9tato2phNnaC5GBD8KrmamJVS26qnA7D+zE7uGsDB2z8pmQYd X-Received: by 2002:a05:6a20:2449:b0:187:5db4:168c with SMTP id t9-20020a056a20244900b001875db4168cmr30479314pzc.31.1701397672406; Thu, 30 Nov 2023 18:27:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701397672; cv=none; d=google.com; s=arc-20160816; b=bhpW78beiIAejSv4o+Teu3Qlwm30XAl9vKcd4aFKt+ihZanLW4nYPTMymft5+KqcuL K9EfcC8UGMAYXMm6sS8mQU71zDncG/neTf4sQuq2xjfPxTQqWYb6poOadt0hDQ+jG994 DC4oD6lqZmXV2z8ODPHURzLcz/oi3sTsPS8LjylmdmbQqycFyP/F2s7KYFY+Xba3Y1xI 53AOJjB1azHF5G5DOUrqMlSHpjg11g5TY0Y73rP0ziMUDcgFSI9whlA/ktEnbpK4scU8 XooZSkGDx4GNcy5KahK140DzzIgI+ucGwYp7MLQ2q+jMf7rSU6abVecDl0PmA2i4/nFN hOVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lWZqmO3qkmxououtRVmaXMPnVef7A+w+wBdufvjjzbQ=; fh=kY5sJNvn29qhEh2SatEVpXE2cTkcQ8AnU+xu9Bnn8Ic=; b=pKu67OVLhNmDEfq1mGv8VdfzG1r8hEHucP7FTJHuvHm6KWep2cIcElC8czOe04ubqa 6GpkH21A6GOwKQ2JQXcdarrs5dE9NY9GUAVDUOD6WQqvOH0fII8+2i8hwWAQb79F1rdD jeMIUlJ7EhN55v8MYkej1Gx8vzpq0BHbou6PosmVDEGmCfoj9OBd/2LGcd9zboH6bOPx YnAaZsjvPGiilFGHNYqhGkaFLKAcRMZTNFVING0O1P3840lL/CjSqYOaNTouwAuiDRy9 v58WEyk+fPcLSdcyXpBTj6hznpb8CXQ8zHIqG+LI8LC+1dTzWpE05uV6vcObTn1R8NW9 AtZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=sZvCPDKk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id x8-20020a056a00188800b006c4b01d5f85si2488507pfh.16.2023.11.30.18.27.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 18:27:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=sZvCPDKk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id A182782DF4A2; Thu, 30 Nov 2023 18:27:45 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376736AbjLAC1h (ORCPT + 99 others); Thu, 30 Nov 2023 21:27:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229523AbjLAC1g (ORCPT ); Thu, 30 Nov 2023 21:27:36 -0500 Received: from out-184.mta1.migadu.com (out-184.mta1.migadu.com [IPv6:2001:41d0:203:375::b8]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED0CE1707 for ; Thu, 30 Nov 2023 18:27:41 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1701397658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lWZqmO3qkmxououtRVmaXMPnVef7A+w+wBdufvjjzbQ=; b=sZvCPDKknSawjBRoM59gZX3xK/xndbvFX0Ytjm/rtsmNHnnd08H2oAE6ku1lcx6nvW3r9W mcLlxiConj9Jmnp7eZELKi9tcPIW2TwyxNfY6jgE7v7zEkRF8vxv5FTO2Bgdz/7ENEsjTX 8YajYexpI+kCAEMt+IopzJGe+ptwAbM= From: Yajun Deng To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yajun Deng Subject: [PATCH 1/2] sched/fair: Return NULL when entity isn't a task in task_of() Date: Fri, 1 Dec 2023 10:27:03 +0800 Message-Id: <20231201022704.3526377-2-yajun.deng@linux.dev> In-Reply-To: <20231201022704.3526377-1-yajun.deng@linux.dev> References: <20231201022704.3526377-1-yajun.deng@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 30 Nov 2023 18:27:46 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784044765383469549 X-GMAIL-MSGID: 1784044765383469549 Before calling task_of(), we need to make sure that the entity is a task. There is also a warning in task_of() if the entity isn't a task. That means we need to check the entity twice. If the entity isn't a task, return the task struct is meaningless. Return NULL when entity isn't a task in task_of(), and call task_of() instead of entity_is_task() when we need a task_struct. Signed-off-by: Yajun Deng --- kernel/sched/fair.c | 50 +++++++++++++++++++++----------------------- kernel/sched/sched.h | 4 +++- 2 files changed, 27 insertions(+), 27 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 34e23a8984ac..e8d444fad2d6 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -470,8 +470,10 @@ static int cfs_rq_is_idle(struct cfs_rq *cfs_rq) static int se_is_idle(struct sched_entity *se) { - if (entity_is_task(se)) - return task_has_idle_policy(task_of(se)); + struct task_struct *p = task_of(se); + + if (p) + return task_has_idle_policy(p); return cfs_rq_is_idle(group_cfs_rq(se)); } @@ -1156,6 +1158,7 @@ s64 update_curr_common(struct rq *rq) static void update_curr(struct cfs_rq *cfs_rq) { struct sched_entity *curr = cfs_rq->curr; + struct task_struct *p; s64 delta_exec; if (unlikely(!curr)) @@ -1169,8 +1172,9 @@ static void update_curr(struct cfs_rq *cfs_rq) update_deadline(cfs_rq, curr); update_min_vruntime(cfs_rq); - if (entity_is_task(curr)) - update_curr_task(task_of(curr), delta_exec); + p = task_of(curr); + if (p) + update_curr_task(p, delta_exec); account_cfs_rq_runtime(cfs_rq, delta_exec); } @@ -1184,24 +1188,19 @@ static inline void update_stats_wait_start_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct sched_statistics *stats; - struct task_struct *p = NULL; if (!schedstat_enabled()) return; stats = __schedstats_from_se(se); - if (entity_is_task(se)) - p = task_of(se); - - __update_stats_wait_start(rq_of(cfs_rq), p, stats); + __update_stats_wait_start(rq_of(cfs_rq), task_of(se), stats); } static inline void update_stats_wait_end_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct sched_statistics *stats; - struct task_struct *p = NULL; if (!schedstat_enabled()) return; @@ -1217,27 +1216,20 @@ update_stats_wait_end_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) if (unlikely(!schedstat_val(stats->wait_start))) return; - if (entity_is_task(se)) - p = task_of(se); - - __update_stats_wait_end(rq_of(cfs_rq), p, stats); + __update_stats_wait_end(rq_of(cfs_rq), task_of(se), stats); } static inline void update_stats_enqueue_sleeper_fair(struct cfs_rq *cfs_rq, struct sched_entity *se) { struct sched_statistics *stats; - struct task_struct *tsk = NULL; if (!schedstat_enabled()) return; stats = __schedstats_from_se(se); - if (entity_is_task(se)) - tsk = task_of(se); - - __update_stats_enqueue_sleeper(rq_of(cfs_rq), tsk, stats); + __update_stats_enqueue_sleeper(rq_of(cfs_rq), task_of(se), stats); } /* @@ -1263,6 +1255,7 @@ update_stats_enqueue_fair(struct cfs_rq *cfs_rq, struct sched_entity *se, int fl static inline void update_stats_dequeue_fair(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { + struct task_struct *tsk; if (!schedstat_enabled()) return; @@ -1274,8 +1267,8 @@ update_stats_dequeue_fair(struct cfs_rq *cfs_rq, struct sched_entity *se, int fl if (se != cfs_rq->curr) update_stats_wait_end_fair(cfs_rq, se); - if ((flags & DEQUEUE_SLEEP) && entity_is_task(se)) { - struct task_struct *tsk = task_of(se); + tsk = task_of(se); + if ((flags & DEQUEUE_SLEEP) && tsk) { unsigned int state; /* XXX racy against TTWU */ @@ -3569,12 +3562,14 @@ static inline void update_scan_period(struct task_struct *p, int new_cpu) static void account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) { + struct task_struct *p = task_of(se); + update_load_add(&cfs_rq->load, se->load.weight); #ifdef CONFIG_SMP - if (entity_is_task(se)) { + if (p) { struct rq *rq = rq_of(cfs_rq); - account_numa_enqueue(rq, task_of(se)); + account_numa_enqueue(rq, p); list_add(&se->group_node, &rq->cfs_tasks); } #endif @@ -3586,10 +3581,12 @@ account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) static void account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) { + struct task_struct *p = task_of(se); + update_load_sub(&cfs_rq->load, se->load.weight); #ifdef CONFIG_SMP - if (entity_is_task(se)) { - account_numa_dequeue(rq_of(cfs_rq), task_of(se)); + if (p) { + account_numa_dequeue(rq_of(cfs_rq), p); list_del_init(&se->group_node); } #endif @@ -5323,9 +5320,10 @@ static __always_inline void return_cfs_rq_runtime(struct cfs_rq *cfs_rq); static void dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { + struct task_struct *p = task_of(se); int action = UPDATE_TG; - if (entity_is_task(se) && task_on_rq_migrating(task_of(se))) + if (p && task_on_rq_migrating(p)) action |= DO_DETACH; /* diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 001fe047bd5d..6baba9ebcde1 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1432,7 +1432,9 @@ static inline void update_idle_core(struct rq *rq) { } #ifdef CONFIG_FAIR_GROUP_SCHED static inline struct task_struct *task_of(struct sched_entity *se) { - SCHED_WARN_ON(!entity_is_task(se)); + if (!entity_is_task(se)) + return NULL; + return container_of(se, struct task_struct, se); } From patchwork Fri Dec 1 02:27:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yajun Deng X-Patchwork-Id: 172219 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:bcd1:0:b0:403:3b70:6f57 with SMTP id r17csp834064vqy; Thu, 30 Nov 2023 18:28:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IHEH0qdme5k59mNV9Ah+XP9oOE9sTavyMwf/Wu3b7tUsFATJPki0Sp1sQMRCLDllbRxhRdN X-Received: by 2002:a05:6358:9043:b0:16c:499:bbbd with SMTP id f3-20020a056358904300b0016c0499bbbdmr25062274rwf.21.1701397692473; Thu, 30 Nov 2023 18:28:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701397692; cv=none; d=google.com; s=arc-20160816; b=yYsTNvlmCwLNnzeWfwovD3SMw0/P32SRS4ZjHxhgl0lKry62LUCvYzk9GMqpA58WjQ 6LK74rW3gqwgVX6rjyFjrdPFd+em62zOroYUwAU4Wnvyv/EzO8YMLtlN7lwTorUz5U8N KHr3/LaRlLWOGzrYKjlVad0fVi1umjUNHiGyCOaZHP/iq0M1llRwgoLWiF/hl6GnwjBw uargLqMadFPml0H5hIA5RVCFgCobELjeqqfYp9fNyo7a+HB5NZhNM4lRLt0OqE8BZIYL 92jxt+RI3qfYMUD2OAXdhvK2yx2iTVjvMhxSYaZU+RrolMa2B2B77AHkuzwg7rwb0sEm F8fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fH2a7qGhcZXIGgkMnI78AIz2EcPccxN2AHWONEyfKRs=; fh=kY5sJNvn29qhEh2SatEVpXE2cTkcQ8AnU+xu9Bnn8Ic=; b=hqxKJ7OAqqaCd01NY9ZY0+i2ddE+zvoQ/0QJej4yBEiXjFuXIBO+wu/4oTtAODZrnR t8284NQYBVvJVdEOat11LE16O0hcXDeL/QArab3kZayvisKIKKUsByKUkLTPUAmYsAXa KF8Pr+scxr2kqy2ZJYnJI1UbSHHhSgawjzRTu8Y41fc3BV2UOHRVxYDWbKQYDU5pHQ+Q /YzYR8XG/Fd7AMKGpmyyLPNKV2yLHOAIHCMEGaCUEKpEbiGWspBXP0U3yyUyxcDnYHw5 9Xt+O1vKXTxoHKxvR/154mEGj4qvOJ3Pt/kPxDoiBBjkscj79WKnIMVA63Nks/Wk8xq1 g42Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=T6evYVBw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id x13-20020a63fe4d000000b005c1cd1ba05csi2306516pgj.240.2023.11.30.18.28.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Nov 2023 18:28:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=T6evYVBw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 896778705E9C; Thu, 30 Nov 2023 18:28:06 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376818AbjLAC1r (ORCPT + 99 others); Thu, 30 Nov 2023 21:27:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229523AbjLAC1p (ORCPT ); Thu, 30 Nov 2023 21:27:45 -0500 Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [IPv6:2001:41d0:203:375::ae]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 249CD1729 for ; Thu, 30 Nov 2023 18:27:48 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1701397666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fH2a7qGhcZXIGgkMnI78AIz2EcPccxN2AHWONEyfKRs=; b=T6evYVBwsNGTj0sOvAKv/gA4H1/48MNtmuovLzHY9YTRHZcsdOx4aGQetuphmYvEg2dvZV 5t44FPvtWgQIYe+0N3X+yPJ9W08gwDmptvmT0NDQL+AbPE1llnDScD+VZE8Eg2op5WzpyO ZORbL1+/QGvlvP6O+XuJZ/Dvg7K5mMM= From: Yajun Deng To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, Yajun Deng Subject: [PATCH 2/2] sched/rt: Return NULL when rt entity isn't a task in rt_task_of() Date: Fri, 1 Dec 2023 10:27:04 +0800 Message-Id: <20231201022704.3526377-3-yajun.deng@linux.dev> In-Reply-To: <20231201022704.3526377-1-yajun.deng@linux.dev> References: <20231201022704.3526377-1-yajun.deng@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 30 Nov 2023 18:28:06 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1784044787021372850 X-GMAIL-MSGID: 1784044787021372850 Before calling rt_task_of(), we need to make sure that the rt entity is a task. There is also a warning in rt_task_of() if the rt entity isn't a task. That means we need to check the rt entity twice. If the rt entity isn't a task, return the task struct is meaningless. Return NULL when rt entity isn't a task in rt_task_of(), and call rt_task_of() instead of rt_entity_is_task() when we need a task_struct. Signed-off-by: Yajun Deng --- kernel/sched/rt.c | 62 +++++++++++++---------------------------------- 1 file changed, 17 insertions(+), 45 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 3261b067b67e..be4d65e05af1 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -169,9 +169,9 @@ static void destroy_rt_bandwidth(struct rt_bandwidth *rt_b) static inline struct task_struct *rt_task_of(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_SCHED_DEBUG - WARN_ON_ONCE(!rt_entity_is_task(rt_se)); -#endif + if (!rt_entity_is_task(rt_se)) + return NULL; + return container_of(rt_se, struct task_struct, rt); } @@ -941,12 +941,10 @@ static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun) static inline int rt_se_prio(struct sched_rt_entity *rt_se) { -#ifdef CONFIG_RT_GROUP_SCHED struct rt_rq *rt_rq = group_rt_rq(rt_se); if (rt_rq) return rt_rq->highest_prio.curr; -#endif return rt_task_of(rt_se)->prio; } @@ -1266,54 +1264,34 @@ static void __delist_rt_entity(struct sched_rt_entity *rt_se, struct rt_prio_arr rt_se->on_list = 0; } -static inline struct sched_statistics * -__schedstats_from_rt_se(struct sched_rt_entity *rt_se) -{ -#ifdef CONFIG_RT_GROUP_SCHED - /* schedstats is not supported for rt group. */ - if (!rt_entity_is_task(rt_se)) - return NULL; -#endif - - return &rt_task_of(rt_se)->stats; -} - static inline void update_stats_wait_start_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se) { - struct sched_statistics *stats; - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - - stats = __schedstats_from_rt_se(rt_se); - if (!stats) + p = rt_task_of(rt_se); + if (!p) return; - __update_stats_wait_start(rq_of_rt_rq(rt_rq), p, stats); + __update_stats_wait_start(rq_of_rt_rq(rt_rq), p, &p->stats); } static inline void update_stats_enqueue_sleeper_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se) { - struct sched_statistics *stats; - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - - stats = __schedstats_from_rt_se(rt_se); - if (!stats) + p = rt_task_of(rt_se); + if (!p) return; - __update_stats_enqueue_sleeper(rq_of_rt_rq(rt_rq), p, stats); + __update_stats_enqueue_sleeper(rq_of_rt_rq(rt_rq), p, &p->stats); } static inline void @@ -1330,34 +1308,28 @@ update_stats_enqueue_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se, static inline void update_stats_wait_end_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se) { - struct sched_statistics *stats; - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - - stats = __schedstats_from_rt_se(rt_se); - if (!stats) + p = rt_task_of(rt_se); + if (!p) return; - __update_stats_wait_end(rq_of_rt_rq(rt_rq), p, stats); + __update_stats_wait_end(rq_of_rt_rq(rt_rq), p, &p->stats); } static inline void update_stats_dequeue_rt(struct rt_rq *rt_rq, struct sched_rt_entity *rt_se, int flags) { - struct task_struct *p = NULL; + struct task_struct *p; if (!schedstat_enabled()) return; - if (rt_entity_is_task(rt_se)) - p = rt_task_of(rt_se); - + p = rt_task_of(rt_se); if ((flags & DEQUEUE_SLEEP) && p) { unsigned int state;