From patchwork Thu Jun 15 20:12:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Don X-Patchwork-Id: 108710 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp902811vqr; Thu, 15 Jun 2023 13:39:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7wsiWb5uhS9FnM8QzC/X9Stt8ZOBJ9kpy4sYmwKZDGvnohu7jGhUrnhUvrU3V4w7qUba14 X-Received: by 2002:a05:6a00:2d9d:b0:656:5182:5e61 with SMTP id fb29-20020a056a002d9d00b0065651825e61mr8106665pfb.2.1686861573439; Thu, 15 Jun 2023 13:39:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686861573; cv=none; d=google.com; s=arc-20160816; b=AC6GPXjQsPKXp/5ji4DjBtv4tBWAhm2O8av4r8VJOERmyVl7ZafpNBNctr+hW6dOpD VAuBxwVCV1qDDZ1Dh3fKXJaavxHAnsvgFqK91LdeHnb0dzUsR7ecrExfOuqolwtDtFgN iR/ASd4zrhNnw7HqvhTERQ50NwrmheDR8aFP4kvQBzhbZ2ziz6YUC1/ZTEUm3EVCPwEM AKOseCjr+MU/X71NMLlFNJWGO96CGGfrh3um0BGRsc7yhc/oKfnYclEAw/K5A7GTmybv FdKGEUU7KdOIT0pOxwBs9lFivUMKvaFnJ15lFWuEyogxNav/8xHE6nByKaGmCPPjlwkx Cfzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=CEZv/6sGhvOMhQWxnDExbwTM685+qPRmRKEGe7ClWdI=; b=gHneDzHwNMGw27q0n2Bz/H9jM+32ehUZRbViDaX47fCVVYvyMZPh5an9sggmg3DD4q Yq1pVeaZdDXCTU7yvOZxaKR8sm37U/+GMtUz/s0rc/eN/vfkwrXq/g1WH04UWjIacTFD sZ+w9paTKo1gQDJMlA5pGmp1m+E/LSgYw5gJjlYfHZ8dLHKHTSwLZBh1LwRVBPTjSlep HW8N52pkDYd+pRAJSeBKU8cWxPIYmlrqHa2iQ6ETT91odWH+VEruFbGM1gqmXAN6OlXV lvbESh2uTdkNob6OTBBPaiuKqpTPTRR+1zM0YaT2Rsw9RlZrzi13L7rd4bHVcAmdDf29 LPzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=txXDxOuA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q10-20020aa7842a000000b0064d3cf07fdcsi1095291pfn.88.2023.06.15.13.39.20; Thu, 15 Jun 2023 13:39:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=txXDxOuA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237675AbjFOUOV (ORCPT + 99 others); Thu, 15 Jun 2023 16:14:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230176AbjFOUON (ORCPT ); Thu, 15 Jun 2023 16:14:13 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0343130FC for ; Thu, 15 Jun 2023 13:13:47 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-565de4b5be5so951407b3.1 for ; Thu, 15 Jun 2023 13:13:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686860018; x=1689452018; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=CEZv/6sGhvOMhQWxnDExbwTM685+qPRmRKEGe7ClWdI=; b=txXDxOuAgvISduwhNORlog6PD3uTMSCFN9bEfs929D/4zo30iziM6RQp6/EPYtLr6w MirYt4C82nwhJ/MjoHs1UuLMiiPMkHNDKBql+WfzmmKrkcO8E/lusmOAlPmuJBL0OAd7 loM13cEvQW8fpCXG3Vx7cXVGSrmiN5q2SHKHeRVoxHkN8CwtkSIYMLwsSgXKVv7OFsL6 +vQ1/aT36BMArNIVLBtWMEKU69RDSoGoOD9RtDemIT1orr7S11CetMrXv4QVSTn1j40f VZG2cDcYJOaxyh7dvS7Jxp/t/yJ04pYxaXXILbV+U0fxOJA8ppCY+wOcY4QoikCNbWJ0 O4Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686860018; x=1689452018; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CEZv/6sGhvOMhQWxnDExbwTM685+qPRmRKEGe7ClWdI=; b=gFuTekO5COeq84UvGT2vzJnyO3M18h3gT4mWPDr7iEULKrZvSkZOurJN7yoNjAPk+J kJhR9MMwBzzOUuIXiHB5o4tT2QtAL1TbuXCNsMIHYrdf6dyosaUmBE2bYjjT+PCbNjot 39m/UAUfpVSx5xJrwQc0MZIwnV8pIEkFha9Pf3/yg/b75irS72bZ5jUe9+suPrH+IjE/ C7NihRFo8Gv4TIEY4r+9ziytGwK6EMSTqwbTpmvNL5YUy1IJmCgL+UbeBcwFltE1I04E /PeWlOqzcdJEgWWZjv8lbW83QfnPTTjc+j2gcEJi3izL96AHyYZLOtXwOtQF1HhSVDY0 4SHg== X-Gm-Message-State: AC+VfDwpZ2dNzB3J9cn+mbI6Yx45otTzzjGLlQe05B0bKMWTtpqNXFYQ QpbPm/i5cVyWCn3ndI3HgMbdwmQkr+eE X-Received: from joshdon-desktop.svl.corp.google.com ([2620:15c:2d4:203:5083:2e33:474b:405c]) (user=joshdon job=sendgmr) by 2002:a81:ad1f:0:b0:56d:255:e2b6 with SMTP id l31-20020a81ad1f000000b0056d0255e2b6mr2752990ywh.7.1686860018034; Thu, 15 Jun 2023 13:13:38 -0700 (PDT) Date: Thu, 15 Jun 2023 13:12:52 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230615201252.1009678-1-joshdon@google.com> Subject: [PATCH] sched: fix throttle accounting with nested bandwidth limits From: Josh Don To: Peter Zijlstra Cc: Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , linux-kernel@vger.kernel.org, Josh Don X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768802561253798297?= X-GMAIL-MSGID: =?utf-8?q?1768802561253798297?= This fixes two issues: - throttled_clock should only be set on the group that is actually getting throttled - self-throttled time should only be accounted on entry/exit to throttled state when we have nested limits Fixes: 88cb2868250c ("sched: add throttled time stat for throttled children") Fixes: 3ab150d011da ("sched: don't account throttle time for empty groups") Signed-off-by: Josh Don --- kernel/sched/fair.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0219cf870cef..a5fc825a8d70 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4787,6 +4787,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial) } static void check_enqueue_throttle(struct cfs_rq *cfs_rq); +static inline int cfs_rq_throttled(struct cfs_rq *cfs_rq); static inline bool cfs_bandwidth_used(void); @@ -4879,7 +4880,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) #ifdef CONFIG_CFS_BANDWIDTH struct rq *rq = rq_of(cfs_rq); - if (!cfs_rq->throttled_clock) + if (cfs_rq_throttled(cfs_rq) && !cfs_rq->throttled_clock) cfs_rq->throttled_clock = rq_clock(rq); if (!cfs_rq->throttled_clock_self) cfs_rq->throttled_clock_self = rq_clock(rq); @@ -5387,17 +5388,17 @@ static int tg_unthrottle_up(struct task_group *tg, void *data) /* Add cfs_rq with load or one or more already running entities to the list */ if (!cfs_rq_is_decayed(cfs_rq)) list_add_leaf_cfs_rq(cfs_rq); - } - if (cfs_rq->throttled_clock_self) { - u64 delta = rq_clock(rq) - cfs_rq->throttled_clock_self; + if (cfs_rq->throttled_clock_self) { + u64 delta = rq_clock(rq) - cfs_rq->throttled_clock_self; - cfs_rq->throttled_clock_self = 0; + cfs_rq->throttled_clock_self = 0; - if (SCHED_WARN_ON((s64)delta < 0)) - delta = 0; + if (SCHED_WARN_ON((s64)delta < 0)) + delta = 0; - cfs_rq->throttled_clock_self_time += delta; + cfs_rq->throttled_clock_self_time += delta; + } } return 0; @@ -5412,13 +5413,13 @@ static int tg_throttle_down(struct task_group *tg, void *data) if (!cfs_rq->throttle_count) { cfs_rq->throttled_clock_pelt = rq_clock_pelt(rq); list_del_leaf_cfs_rq(cfs_rq); + + SCHED_WARN_ON(cfs_rq->throttled_clock_self); + if (cfs_rq->nr_running) + cfs_rq->throttled_clock_self = rq_clock(rq); } cfs_rq->throttle_count++; - SCHED_WARN_ON(cfs_rq->throttled_clock_self); - if (cfs_rq->nr_running) - cfs_rq->throttled_clock_self = rq_clock(rq); - return 0; }