From patchwork Thu May 18 01:34:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Don X-Patchwork-Id: 95610 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp173865vqo; Wed, 17 May 2023 18:45:11 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4dNbqxmH/wN8d7UjpzPYkkrfMwD15Xjuez+3XjEBoHzR142wZHUAuHM8TinaeH3SxjBoTn X-Received: by 2002:a05:6a20:9144:b0:ff:ca91:68ee with SMTP id x4-20020a056a20914400b000ffca9168eemr148033pzc.9.1684374311087; Wed, 17 May 2023 18:45:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684374311; cv=none; d=google.com; s=arc-20160816; b=GN8ui/eSibWPfjJCX/bI7sRvHPjycsJZf0T0WxXLUYE9rs1etQBG9F9wRyxJRT34nd aHkdEIDNDTZhsRhnPwD5kT1upA9bFHkJsRyXcC2h2V2CT0Z14MRmGQ9pPECHKnvh+6Rb o+TBZBXvWzKNXFbpgAlF9m0nV4VdwT5/xaSY1ehE/yjsgNs/4HqU5sSNwNXhTYiZ+jh2 vCoLfhpXiOa3vY9H5/AE7mshyxo5+YjwzNkoTqZPz05jOJ7aapNsaoAmigwMpADurYZ8 klSDYMwghOhTi46I8w6r21Earh4dHtbvRBRNoYtl2UgaBKRdqALgz6RfsfkQtphPviOW mO0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=YCkOGnN5VV+G0vc4oBgEXNSPP/cS4uflXB2VdW0nTDc=; b=uaj8bBbkhvOfudkLjKwMymIgScHMnqqiIdJO9Tw7n/gLBQb73wbX+PkEzH4VwbPpPn VbBrI/0xtH9kVW3tmvtTdCxWqIfHlVfI3IS/KUwY008G2w3u8JlAVGBBwSbIARM4Fe/x l9N3/BmzgXSUuU+XBcMq72ImSg9YfszWzQIgrBEwtWcmrN7HFyztgXAQobJE5MvwBNP6 H2tVo5p3SCg1i52SCakBHU62ygmcm10iGNZR1H7RWPtHj9Gofhq3nbgUr9WpTjZ/9+/5 pdxZjDUlbBVtEMroIYDJn0wEvzh9qmH31hXNk2+0E3T//IRcsyKeZ46gQit0NLMweb/5 PHvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=LV9SKqc9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 3-20020a621703000000b006435b9bd9f0si452291pfx.45.2023.05.17.18.44.56; Wed, 17 May 2023 18:45:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=LV9SKqc9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229676AbjERBem (ORCPT + 99 others); Wed, 17 May 2023 21:34:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229621AbjERBek (ORCPT ); Wed, 17 May 2023 21:34:40 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 987D535BF for ; Wed, 17 May 2023 18:34:37 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-561abf855afso15762157b3.2 for ; Wed, 17 May 2023 18:34:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684373677; x=1686965677; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=YCkOGnN5VV+G0vc4oBgEXNSPP/cS4uflXB2VdW0nTDc=; b=LV9SKqc9LLji/8d1D1nfFc5i5jRcnJLn+AfdqPiE8q/ZsW+08nHc91HNb4jIFg5d6H jgdmF4AsVk+sjFNbe6dNmj/yjWzAvlQoPbqzFzG1yAFkYJTm0tSE4JRPp2pkAbpk8Yzl sMwx5yX3xjLDUYEbTRyn1YgUs8/4CyPkDnd5i1xEg8U1TP4klX+n1KZZQP7nQ2rdYwLc sIkYfAezE/CJwAl1KHldUuPcIRJf1cXLC5Q2taC1S6LpVTa83YOcpVTVkwZz0l9o+njR AndVecG6UkuOC+ZGe3GQEYiXzhMOPnOp8lV1m5u//s0N+M6h7dNTW4jeFc87BXnBz6yA wdNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684373677; x=1686965677; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=YCkOGnN5VV+G0vc4oBgEXNSPP/cS4uflXB2VdW0nTDc=; b=kI2c7SMIZzfwqPmR1IvP9sLx9OjG7I36bndLiZYyyZmizg4EVb+nPLEv6WsdFxvCby 2u0futUAKUR+Vw3zqLXa+xcklGQhc7SWNfikNEZEmvv36kYS+61p11AtjgDgczNK8u4i mJSoRWOFbdUaSB4s2VSrBH6gNACylgfimXR4mBEvHUQc1OVIGQmlUwNqp2i+AJE9F30/ vnojCtHPGxNve1D+Gg1u+n6dE2tRgkdu1qJ47SvhtQJZNr+hNrq+0YHLvwoxV6qUpQWh VY0C0wjPuCVD0rEEztrbZcj3uQBO12lKgK4AFniuesgBPBqEZEYywQZJhBS4bptdpeQ1 t77g== X-Gm-Message-State: AC+VfDymT0+9j45GWoX+Pp3CO1nzeVenvhvKWCyC30WQT8Oifqt7+BBj fJLd05gyyATQATsB0LwJ6xCX1PY4/T78 X-Received: from joshdon-desktop.svl.corp.google.com ([2620:15c:2d4:203:53e2:12a7:c986:1d79]) (user=joshdon job=sendgmr) by 2002:a81:ae64:0:b0:561:2d82:7f08 with SMTP id g36-20020a81ae64000000b005612d827f08mr131225ywk.0.1684373676864; Wed, 17 May 2023 18:34:36 -0700 (PDT) Date: Wed, 17 May 2023 18:34:13 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog Message-ID: <20230518013414.3053254-1-joshdon@google.com> Subject: [PATCH 1/2] sched: don't account throttle time for empty groups From: Josh Don To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Tejun Heo , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , linux-kernel@vger.kernel.org, Xiangling Kong , Josh Don X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766194477971609020?= X-GMAIL-MSGID: =?utf-8?q?1766194477971609020?= It is easy for a cfs_rq to become throttled even when it has no enqueued entities (for example, if we have just put_prev()'d the last runnable task of the cfs_rq, and the cfs_rq is out of quota). Avoid accounting this time towards total throttle time, since it otherwise falsely inflates the stats. Signed-off-by: Josh Don --- kernel/sched/fair.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f4b8b906d30a..85c2c0c3cab6 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4768,6 +4768,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { bool renorm = !(flags & ENQUEUE_WAKEUP) || (flags & ENQUEUE_MIGRATED); bool curr = cfs_rq->curr == se; + struct rq *rq = rq_of(cfs_rq); /* * If we're the current task, we must renormalise before calling @@ -4816,8 +4817,12 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) if (cfs_rq->nr_running == 1) { check_enqueue_throttle(cfs_rq); - if (!throttled_hierarchy(cfs_rq)) + if (!throttled_hierarchy(cfs_rq)) { list_add_leaf_cfs_rq(cfs_rq); + } else { + if (!cfs_rq->throttled_clock) + cfs_rq->throttled_clock = rq_clock(rq); + } } } @@ -5423,7 +5428,9 @@ static bool throttle_cfs_rq(struct cfs_rq *cfs_rq) * throttled-list. rq->lock protects completion. */ cfs_rq->throttled = 1; - cfs_rq->throttled_clock = rq_clock(rq); + SCHED_WARN_ON(cfs_rq->throttled_clock); + if (cfs_rq->nr_running) + cfs_rq->throttled_clock = rq_clock(rq); return true; } @@ -5441,7 +5448,10 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) update_rq_clock(rq); raw_spin_lock(&cfs_b->lock); - cfs_b->throttled_time += rq_clock(rq) - cfs_rq->throttled_clock; + if (cfs_rq->throttled_clock) { + cfs_b->throttled_time += rq_clock(rq) - cfs_rq->throttled_clock; + cfs_rq->throttled_clock = 0; + } list_del_rcu(&cfs_rq->throttled_list); raw_spin_unlock(&cfs_b->lock);