From patchwork Fri Dec 16 06:24:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Jia X-Patchwork-Id: 33874 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp806594wrn; Thu, 15 Dec 2022 22:47:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf6DkG+l8lyTV9a34PWerlj2JEdBc2wypBsh45iDFGLkeZZpkyTi/uj7EyrPHwzho50IA1Z9 X-Received: by 2002:a17:906:face:b0:7c0:e98d:b27 with SMTP id lu14-20020a170906face00b007c0e98d0b27mr30154895ejb.58.1671173239777; Thu, 15 Dec 2022 22:47:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671173239; cv=none; d=google.com; s=arc-20160816; b=wmgcRKh+1UXj2bFOlnDstRtHIyoO987KdVygrtx+khb6OwBg64RW/R4VMwMZ22K0oR Sjm3HOxJErDCGHy5a1sPSTjNLxTeHEofsbc6hvkD5scwZZH0VSNbAzLRrICJXFmIC2Lw ZAsWdyAMxu0hN4HZK24QL8O7aKIxhIGp8MhujInqx4D9ffGVYWezFfUzUxUflMOS1Kbe DrLxFA9jBV2gHuZ847LXCB2vl8wW3W2c+pRfr1qaDJnGOxYONv0L7qi2P6VxJLWb18QY NE2tFT5uGc9mJb+bVjnCSyHuuPGe6MER4hwJg0UONurh7WhtCVWh8UbFPh3nIDtiRUyx O2BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=rEBn3rIORKhKVuqIN/cMMOVkHVPj5Ym5EqGWb/wIQVk=; b=KnfEE6Jf1T9lV/3tvyR4g5afRl7+rqze8J2w9SYT9xyr2Jozn7jM+f23a2WMKVhT2f ERxhfLnJ9IwVOJAXFhLOTXbftUkv7sMRx7fT+HWBoJZi4TJJdtDSfhSpIMo/o71yeYUJ 5r91X4RN8ZkMFKIWH9HKPHfSedqs0Vz/TOBZUWsKyxrbBoPThRdYVyA9rB1KyqQuoybm cMxuWkRncQ06+6FwtVNBCF9CbYjl98IbmKjekoZDfX2XYFT7wcCBp7/P1k7/C2dLZAPZ n/6I+Q+2GH59kOQpVbUnKHghF/0FdiYCpwSgvlClJEfeaI6wXc1SJFSsmit1ovnehgja VPxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="Ypv/XyC6"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c10-20020a170906528a00b007adeb7f8e5bsi1133620ejm.913.2022.12.15.22.46.52; Thu, 15 Dec 2022 22:47:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="Ypv/XyC6"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229852AbiLPGYd (ORCPT + 99 others); Fri, 16 Dec 2022 01:24:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229704AbiLPGY3 (ORCPT ); Fri, 16 Dec 2022 01:24:29 -0500 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9F9B654EE for ; Thu, 15 Dec 2022 22:24:28 -0800 (PST) Received: by mail-pj1-x1033.google.com with SMTP id z8-20020a17090abd8800b00219ed30ce47so5057569pjr.3 for ; Thu, 15 Dec 2022 22:24:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rEBn3rIORKhKVuqIN/cMMOVkHVPj5Ym5EqGWb/wIQVk=; b=Ypv/XyC62kjv9HAqqcaPhJIxQx9wsgVqq+EI46a3l1IPfn4ashDi1YifxmVQ9zyaVC eBEhkkfSiP8xNDVkbyUl3xH1CVEX52HhFHVd4G7xLceGkNjgFcCaXQWHwKCM3zBwsIUi 25xJqTAxwVt9ldwxBU+/D5pDGqu+ChU2VKqsX153LKLtbJWRn7DBxXxWZ9hIqnKQgXci f6Ux1Ik5ExddTVYC2P9rIb1U/xqR/h6z9WzbCRlj6TiHa61/dgMSifzrF7E3QUO/C9CF yTlM/2U3oP5rqQZGkg3kywwY12EduCyBixeyp+wa88kSG40utoaAdeIJVNkrCGVhWhcW DQUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rEBn3rIORKhKVuqIN/cMMOVkHVPj5Ym5EqGWb/wIQVk=; b=WJZpGZHEOrRTolN9Tvvvd8IvQko0WLVSHEellmueZt454ssu2JL5TKJ58dWe+BEGHB 5j3+0Dv6ECMOHnCGYHCgBSfmOxwTmr5MGLsG54rX5vA7w2MXhXdukbrHdTv+U3ycBBMS AtRWCS2zBTvbUpyoCWzxWe1cZJML60w5dUAh7vZ7G03L0r4a64e7mwDmD+bPGlIvZjNp tneJoDBpp5HdIIvbFSlRTBrOaNiA89H4f3ZOutMqsxLRjn3SSBjEP+7Kzi1fIXdRm3iC 8kUu0RsQHfCrOxvba85G4PWyM3RLRU1Q3LFPjAjyxE5+tdYYosJrBOSdeYgKyCCWSLKP l/Hg== X-Gm-Message-State: ANoB5plffaC/hhex7/PlA0jzlglX2vMFUJ1m3rFm8ppax66ZQtJWtlrE ts3v1rqOjwZ8vTYD7BEYKW+ajQ== X-Received: by 2002:a05:6a20:bf17:b0:ac:6543:d515 with SMTP id gc23-20020a056a20bf1700b000ac6543d515mr34111225pzb.42.1671171868243; Thu, 15 Dec 2022 22:24:28 -0800 (PST) Received: from C02G87K0MD6R.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id s5-20020a656445000000b0046feca0883fsm681819pgv.64.2022.12.15.22.24.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Dec 2022 22:24:27 -0800 (PST) From: Hao Jia To: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mgorman@techsingularity.net Cc: linux-kernel@vger.kernel.org, Hao Jia Subject: [PATCH v3 1/2] sched/numa: Stop an exhastive search if an idle core is found Date: Fri, 16 Dec 2022 14:24:05 +0800 Message-Id: <20221216062406.7812-2-jiahao.os@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20221216062406.7812-1-jiahao.os@bytedance.com> References: <20221216062406.7812-1-jiahao.os@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752352150658384187?= X-GMAIL-MSGID: =?utf-8?q?1752352150658384187?= In update_numa_stats() we try to find an idle cpu on the NUMA node, preferably an idle core. we can stop looking for the next idle core or idle cpu after finding an idle core. But we can't stop the whole loop of scanning the CPU, because we need to calculate approximate NUMA stats at a point in time. For example, the src and dst nr_running is needed by task_numa_find_cpu(). Signed-off-by: Hao Jia Acked-by: Mel Gorman --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c36aa54ae071..643cbcb61b49 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1804,7 +1804,7 @@ static void update_numa_stats(struct task_numa_env *env, ns->nr_running += rq->cfs.h_nr_running; ns->compute_capacity += capacity_of(cpu); - if (find_idle && !rq->nr_running && idle_cpu(cpu)) { + if (find_idle && idle_core < 0 && !rq->nr_running && idle_cpu(cpu)) { if (READ_ONCE(rq->numa_migrate_on) || !cpumask_test_cpu(cpu, env->p->cpus_ptr)) continue; From patchwork Fri Dec 16 06:24:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Jia X-Patchwork-Id: 33875 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp806754wrn; Thu, 15 Dec 2022 22:47:44 -0800 (PST) X-Google-Smtp-Source: AA0mqf4mkZKq8kzmRmnsKkRwVN7xZqUfiD/+3eDBe5AA/sS1BvHyTrRCxya+4+BLbLQi/yQBqNFn X-Received: by 2002:a17:906:5a54:b0:7c0:eba2:f9dd with SMTP id my20-20020a1709065a5400b007c0eba2f9ddmr27634251ejc.53.1671173264208; Thu, 15 Dec 2022 22:47:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671173264; cv=none; d=google.com; s=arc-20160816; b=vfRYmzxLpjBuLU4s/G/9jHf1QzTYxMKKg7cmCj9vr7HEBnyooBbYm1wtYl+q1yx+Xm 2r56G8sSWxQnG+B22gE2aNSQrbD/jdnqW1ocZlI5pRVdUsIgUhElvz5IxVSLqgICyMOv /bOUpEUpbQaLfCBwV9mMNiKFQv2KbAHon1Hmt4iJYMHoq4OPwCI/oKsf9oGzqGQwjahe HEV+8SITz9CC32snk0/NRcUJz1M1P2zYfRCuJvZkW9akcOWjzERIdF6FZY4lJ1+q8rw3 aI9wbEQ+qpHA8BL6Snd6qS74tmVoVVOA1tA9CQyraM4083exsuK8m3ySsUy6nVfm3iEM /3BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0JvWppdepGKuCmZdK+T4x0HrLho3kQxHEXyInOIABdw=; b=Mf+v3JdlEKdcpNQtLCrofbd4Thm07gklad3Nz0eZkoinTxL1JaHrYWpkMiTLow0one T2JbZKx7rMbJZmlc3f+irBPE09Ivnw2vDVCT7E1PN8i+R9Z4ouLh3zs0fblkRVGQxDis 8lhxaUv6eO5DuSj1mlxgWEZctVL4CUO1fP4f+7GR+yfI/VQfM7D5lNom9+Vqcj7K/heW kozQdO6pcUIxsnaJE3arm5GuXUkr4hoSDabbxjUIHudTcu53AMp/2VuKCmE9e5Vh6kj6 +tk6aY4qFUE4mpbCwd7CBGFCsTAn/TQusOrLfm3aUajFVL2do0fGmxqdz1QdeshcwX2K 5dew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=ZLRQsDIa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id js1-20020a17090797c100b00780e022dfb8si2017407ejc.494.2022.12.15.22.47.21; Thu, 15 Dec 2022 22:47:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=ZLRQsDIa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229665AbiLPGYo (ORCPT + 99 others); Fri, 16 Dec 2022 01:24:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229897AbiLPGYk (ORCPT ); Fri, 16 Dec 2022 01:24:40 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0608E6BCA2 for ; Thu, 15 Dec 2022 22:24:38 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id 3-20020a17090a098300b00219041dcbe9so1430206pjo.3 for ; Thu, 15 Dec 2022 22:24:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0JvWppdepGKuCmZdK+T4x0HrLho3kQxHEXyInOIABdw=; b=ZLRQsDIaEc7I4ixWaVvP1F3HTLEthQeeNLtaMh7Ktn1wlWDLMyCqsbcbnCBaXmjW62 KsFFGwGUzPVpOUI8uJFu1YHQUogRl16vxmktW0lI+MyWw18yfVz9GVAH13MDatVGmCQm btMgceRIrcHIdzEE1DdJmDxVH59tQqFSKPh1Exm7snRZtSB0Rzf+7noxtFVnduY3kMLg JpJvZ5p++xvAjCnXqrf30gkis6G0/CwPr4meDvGfXTs4lWrpcAh6ZxnI1dvyxCyAM0/O Kx8aOfHP6gHdl2lCtS4gC21O7GA2m7wnc/hjHLb461KwWPEscD8VX/uyPY9B8bAdnDOx mL7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0JvWppdepGKuCmZdK+T4x0HrLho3kQxHEXyInOIABdw=; b=M5BsupKS1nau7T4U1GP3Ev1dCD9+vt2+4Fr4iiwk3GFGP5orkKQxNRZCaywjQtbs/t jskIbujt5qN0lXSAxdDLoZrCUi7FmOkZVlfg4f42CuxuJA3a58kR53d0TG6IvuG29+bZ zXIFx5FdST1LPQyV8TX39I9TTIBV/FpiiInb8U36Fs44txNGh+h3T/u5Nhkc/IZKD6KP FVhKS6Jt015hoHyXPm1Eef3kyqoahev1JaLrObuOg3iypp18GsezGMcxRXGFqjQW+le3 sXvHQ9qQJHHDoPxIXLln1xsOH9+cfaxHln4HaO+kUgeutaMpb8aLoIMRZciwcuSiMiC8 qLvA== X-Gm-Message-State: ANoB5pmreluGRyWLiicu3GCtfPR3gBvyY4HMv5r920/IJ9XgGIZAOpQR M6pPTSx5DUzQeWv/dWOjXRXV5g== X-Received: by 2002:a05:6a21:394b:b0:ac:3f3f:9fbd with SMTP id ac11-20020a056a21394b00b000ac3f3f9fbdmr36790086pzc.48.1671171877541; Thu, 15 Dec 2022 22:24:37 -0800 (PST) Received: from C02G87K0MD6R.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id s5-20020a656445000000b0046feca0883fsm681819pgv.64.2022.12.15.22.24.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Dec 2022 22:24:36 -0800 (PST) From: Hao Jia To: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mgorman@techsingularity.net Cc: linux-kernel@vger.kernel.org, Hao Jia Subject: [PATCH v3 2/2] sched/core: Adjusting the order of scanning CPU Date: Fri, 16 Dec 2022 14:24:06 +0800 Message-Id: <20221216062406.7812-3-jiahao.os@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20221216062406.7812-1-jiahao.os@bytedance.com> References: <20221216062406.7812-1-jiahao.os@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752352176842537921?= X-GMAIL-MSGID: =?utf-8?q?1752352176842537921?= When select_idle_capacity() starts scanning for an idle CPU, it starts with target CPU that has already been checked in select_idle_sibling(). So we start checking from the next CPU and try the target CPU at the end. Similarly for task_numa_assign(), we have just checked numa_migrate_on of dst_cpu, so start from the next CPU. This also works for steal_cookie_task(), the first scan must fail and start directly from the next one. Signed-off-by: Hao Jia Acked-by: Mel Gorman Reviewed-by: Vincent Guittot --- kernel/sched/core.c | 2 +- kernel/sched/fair.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 25b582b6ee5f..40149ff68f14 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6206,7 +6206,7 @@ static bool steal_cookie_task(int cpu, struct sched_domain *sd) { int i; - for_each_cpu_wrap(i, sched_domain_span(sd), cpu) { + for_each_cpu_wrap(i, sched_domain_span(sd), cpu + 1) { if (i == cpu) continue; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 643cbcb61b49..e910df06b779 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1836,7 +1836,7 @@ static void task_numa_assign(struct task_numa_env *env, int start = env->dst_cpu; /* Find alternative idle CPU. */ - for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start) { + for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start + 1) { if (cpu == env->best_cpu || !idle_cpu(cpu) || !cpumask_test_cpu(cpu, env->p->cpus_ptr)) { continue; @@ -6811,7 +6811,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) util_min = uclamp_eff_value(p, UCLAMP_MIN); util_max = uclamp_eff_value(p, UCLAMP_MAX); - for_each_cpu_wrap(cpu, cpus, target) { + for_each_cpu_wrap(cpu, cpus, target + 1) { unsigned long cpu_cap = capacity_of(cpu); if (!available_idle_cpu(cpu) && !sched_idle_cpu(cpu))