From patchwork Wed Oct 26 06:42:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Jia X-Patchwork-Id: 11083 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp96964wru; Wed, 26 Oct 2022 00:01:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4N7U55Ywz3bdSH226p93imUkfqaIvPuq80FfU7UN/Bhu3pUL94nzGlOQOC+RmBjsuU/Rha X-Received: by 2002:a65:6c08:0:b0:448:c216:fe9 with SMTP id y8-20020a656c08000000b00448c2160fe9mr35687036pgu.243.1666767674112; Wed, 26 Oct 2022 00:01:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666767674; cv=none; d=google.com; s=arc-20160816; b=l7/mc0Q11G1lVgoGyOVScJLeZCxTPzvVVJGRGl//sVxN6VeBRNydDjceWNfFgD63b/ hfhDq40k8QtY9rAwHNtw+7+lRewepErJ/UZAH5D4JDMxYOWj9Q1X3VGciay+1wSJ4Gue R/GWvvGuxahhncI3w9Q9y8Fk3Hod3F9ci7MXnJbhSGApdOUqITg/ZCFc2Uj4t9uLVMYl BYGcII8NaRvvjmPT5l2+Dw8Wzo/lb954kV1pHy9PDO8HhxVUsF9qniWpqrIMnEV3cSHh mX51j5zFxJKuCyRf4sBpwqdrfxtHlmErlb5Z83fawF6GxvQwhciepqAqZL/4jaQp9FUF lmSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HIlTSDWbVtqkeVttUcwmGvIfocdqv0kcWKfN4GKoIZ8=; b=PobmtvYfWkXYSWelW0p3US6FagYGwMWivlcDVR7EOpK4vn8s9VBiPi8mHHFLbGJvAK zzrRxx9nRyBUUGcFrrbNrMLe8tYMm6asJtw54rabHRJk9uSN7pqlYkxLMHtMw99fgpiL b8YFhVDeHIO9yhCzlZgshuanIu6jCjZTxgQPha1zhnEXZD/KjtciMXXK9YGpJn2OwX4Q xhPHXZ9AFu3hYPQg9FxcOOLBo45r8NEGS7tsxKXUwutrLHcNG715iOO5UbTTehlLD596 atSAWpGgrX7b5b42s5Kfwex8fNe8yO6Fs64vCFaLkf6mXmFCSIatW2QDAlL3496QwwPg aUdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=uaadtIEG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y3-20020a62ce03000000b0056bf7935ac1si6002588pfg.145.2022.10.26.00.00.33; Wed, 26 Oct 2022 00:01:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=uaadtIEG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233084AbiJZGnb (ORCPT + 99 others); Wed, 26 Oct 2022 02:43:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232981AbiJZGnU (ORCPT ); Wed, 26 Oct 2022 02:43:20 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79407AE851 for ; Tue, 25 Oct 2022 23:43:18 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id e4so10457238pfl.2 for ; Tue, 25 Oct 2022 23:43:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HIlTSDWbVtqkeVttUcwmGvIfocdqv0kcWKfN4GKoIZ8=; b=uaadtIEGpfWSDRPGzX2Ka4GTJgjPlMo5p28PF2vQeBzSIzSSz0Ox1TrE239LPGLjZj LTY2wizD6N0vBeZ8ZY+dvZPeYIpiqkjTey5L5yHwRz8/UsLboW0ZOwxnOIIPw1C77jqZ JFTluMO9P4NxRWnUqe/EbELATGUOmdK5V6SkOSgrFzSwlWXUm8EA8URFCTQMV8b8n8WK bpXe5fmTMBZ6/70pOdXuSmpdWvFaDvt+YDlgJtvqiLtYEnbqguIF6dxy8/VDl+8AKer/ 2lrvrXGEdF68wLKMUNVeU2hLLaWuph/WmGLcT59HI1PwqWzhLFqG3GwBBPACjgSpwFJd 23Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HIlTSDWbVtqkeVttUcwmGvIfocdqv0kcWKfN4GKoIZ8=; b=Zp+J/10E7dPym7M5CKB8EbyRBpWLgQc9hpKPIU5PDM/NsRLMSIox6gHTFZD/izfCSK Imcsa1SqfgUWWFsGPwJvsFmAr/L68Sh/JXwPL8EHkl/Lbc/OADN+0Y7Np8gLLTiH2arV hXkupVpwcT8ll6J8IiAoARk2O4h3BjipGWNs29eI6q1NzfKtsgEHKTF19IRNOAbcnjTL LITyce5STUFeQq/iUKEmZIjmDelQFzNTjKUzWb+iSKYV44wxqy+ANl66lSxxMdwn0Dtj Ge2EEXBLMR0zaSAEnR2+Ek88d8jP3C9h4ytCNlTspf2sPGQuM3LNdpxaxiZ05lHAULA1 +Lvg== X-Gm-Message-State: ACrzQf26foSH5H1oFA15HoJ3fNUym04LdS+TyeFncu67AZnK/YabvAsC +vQ0yfS0XTDhZSkH016ox86rnQ== X-Received: by 2002:a63:90c1:0:b0:45f:c9a7:15c3 with SMTP id a184-20020a6390c1000000b0045fc9a715c3mr36038681pge.304.1666766597869; Tue, 25 Oct 2022 23:43:17 -0700 (PDT) Received: from C02G87K0MD6R.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id b62-20020a621b41000000b0056b932f3280sm2363187pfb.103.2022.10.25.23.43.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Oct 2022 23:43:17 -0700 (PDT) From: Hao Jia To: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mgorman@techsingularity.net Cc: linux-kernel@vger.kernel.org, Hao Jia Subject: [PATCH v2 1/2] sched/numa: Stop an exhastive search if an idle core is found Date: Wed, 26 Oct 2022 14:42:59 +0800 Message-Id: <20221026064300.78869-2-jiahao.os@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20221026064300.78869-1-jiahao.os@bytedance.com> References: <20221026064300.78869-1-jiahao.os@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747732580558996488?= X-GMAIL-MSGID: =?utf-8?q?1747732580558996488?= In update_numa_stats() we try to find an idle cpu on the NUMA node, preferably an idle core. we can stop looking for the next idle core or idle cpu after finding an idle core. But we can't stop the whole loop of scanning the CPU, because we need to calculate approximate NUMA stats at a point in time. For example, the src and dst nr_running is needed by task_numa_find_cpu(). Signed-off-by: Hao Jia Acked-by: Mel Gorman --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e4a0b8bd941c..dfcb620bfe50 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1792,7 +1792,7 @@ static void update_numa_stats(struct task_numa_env *env, ns->nr_running += rq->cfs.h_nr_running; ns->compute_capacity += capacity_of(cpu); - if (find_idle && !rq->nr_running && idle_cpu(cpu)) { + if (find_idle && idle_core < 0 && !rq->nr_running && idle_cpu(cpu)) { if (READ_ONCE(rq->numa_migrate_on) || !cpumask_test_cpu(cpu, env->p->cpus_ptr)) continue; From patchwork Wed Oct 26 06:43:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Jia X-Patchwork-Id: 11079 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp93584wru; Tue, 25 Oct 2022 23:49:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4OZG8H7zxeUsrtjT9DT5a6p7f8XmmnMt2ombG1PbiuzelgU7IF+9p4HiZScqIPxnbY63A+ X-Received: by 2002:a05:6402:e9d:b0:443:7833:3d7b with SMTP id h29-20020a0564020e9d00b0044378333d7bmr1614707eda.151.1666766999054; Tue, 25 Oct 2022 23:49:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666766999; cv=none; d=google.com; s=arc-20160816; b=XRo6pp6UAdN5TB6EkBDjlfGhisQQXsGl5PMhBikdTIL0+03TXYgP65R0GXi78TUKiX ssnro8xgOfCrKrrpYYUl4iA0FtCWpS5eaCeQGo97Wq1Tg4SdOpA/r1h2CZJe7KYPB0Y0 gN5bzFM6jlAY9GF2BzIPZbJWp6y+VtZXGUxSMFm74sxS2R/a32A/A7a2g4QA9TcJIuFQ yt5w41fsclWzqPtn+hjS+6jblzpzJbNdvmqumazFdlYH0XmLWu9/1dXalCn3TT6zETYX 3eot/QGYo5xtBDvBiKwcsr+4A3e/WzAOtEX8fzpWLWOCVwYaEfAg/oyNh14WworYLuxL QCeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2fjwpVVv8NNDnWDvTOJej+inL4k7BmCXyJpK0M4DMs4=; b=x6fxm+cOphEGmNcnR0ycnK9oiXZ/njCwhUXbZZNGvjHxmEK5/nwBAjYnowLf/QvXLy SaYTbsssYTwbk+HUPDdtScMi4exLDBJTb/MmGNDHSkIIid5qJf/sBNRD+Gxjw6C/ma3l f8kW07CqslU5jOEy82JQJCrR450QJdDQCZTwXLAAXC2UG7ET2Sv0E7dU7w2R6ds6a+xJ RP6QwHD+Snsun7zTgyafLQFir1fb2iZBCRzGsNGa5Th/G8eEW8kP06rqAlBs9cPEM35f D0ucT3eLAXhQciGKllX3k5CbaLHCrQtFMs6Ov9JcY7ES2AwX7rDYfYuBpBGA2o3Zwk9B pCAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=k7ku7Elr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ec28-20020a0564020d5c00b004571a669230si3885778edb.469.2022.10.25.23.49.33; Tue, 25 Oct 2022 23:49:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=k7ku7Elr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232698AbiJZGng (ORCPT + 99 others); Wed, 26 Oct 2022 02:43:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233027AbiJZGna (ORCPT ); Wed, 26 Oct 2022 02:43:30 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0AE5AD99C for ; Tue, 25 Oct 2022 23:43:24 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id y13so9841496pfp.7 for ; Tue, 25 Oct 2022 23:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2fjwpVVv8NNDnWDvTOJej+inL4k7BmCXyJpK0M4DMs4=; b=k7ku7Elr6sQpsfdbeabwVWXePRIEmsJH2g8MvGOJVv9XSTu83/tFBtHYnmFjlwovxG lomJjjQJtTvCcozuz907T/VmwAhTYpQxwvKLIaS9xFP8li3tBVTTMCq+EMZ2OuEEKa3c aoPmgWkPbGH1FcCYlYbFlEsvBLqEmvJ3PfDcvfc/rTEpe0yCU2IF7Ogolz99HHYmr2U/ SyNvCtGCp0rwdKhXA8AsaryifFskUXcAiSqgcFhZTgvDAmlTZOsCoze9tRR9tmBn2D5q YSRBHIvEF6MZkXmZg8RytyDGfD1WOyqfFJZYdkqHpJurWvvb29jv3vj5nZfEUa/mLggG 9tNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2fjwpVVv8NNDnWDvTOJej+inL4k7BmCXyJpK0M4DMs4=; b=l0VJwg34JkoKDVMjWVxLR/7lwlM2K4kzdOVf4YKTQN33+q5YU6Y7ns9pX/lhRkSsld /PiHYUQ72MERwn4Gcp7eA90yhw6qUvlwJGa/JFlk9VegIUb2BOFR87Zn0n1ZqxecA+fa 3gBJD6pxArWe83HCc06PFS5gRYCzLaWqD4vcrPh2X9ISD4hEUVmBamkPrtVdNK0qH3OJ cJZOvfHd8XOEA1vlasz1X+Axtoa52OejE03lYCxzpfY89vBzgeHfoUUGZu6tFSVchtGe 7eB/te7SALNFHoVbrGi7BNzcMFMJqDxKy7KLvLjpXsiYLHOaberp7Ipy+yNrR14s4/YW d7Nw== X-Gm-Message-State: ACrzQf0XZp2loBfZJKDCgz5s8SYW8Ah6XLCZPCug6EtRWAEZ8DmRcdmV DcSCZUZ9b6c27GaG4Aqb55dF5g== X-Received: by 2002:a65:6bc7:0:b0:44a:dcee:18a5 with SMTP id e7-20020a656bc7000000b0044adcee18a5mr35855576pgw.413.1666766604344; Tue, 25 Oct 2022 23:43:24 -0700 (PDT) Received: from C02G87K0MD6R.bytedance.net ([139.177.225.228]) by smtp.gmail.com with ESMTPSA id b62-20020a621b41000000b0056b932f3280sm2363187pfb.103.2022.10.25.23.43.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Oct 2022 23:43:23 -0700 (PDT) From: Hao Jia To: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mgorman@techsingularity.net Cc: linux-kernel@vger.kernel.org, Hao Jia Subject: [PATCH v2 2/2] sched/core: Adjusting the order of scanning CPU Date: Wed, 26 Oct 2022 14:43:00 +0800 Message-Id: <20221026064300.78869-3-jiahao.os@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20221026064300.78869-1-jiahao.os@bytedance.com> References: <20221026064300.78869-1-jiahao.os@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747731873301547733?= X-GMAIL-MSGID: =?utf-8?q?1747731873301547733?= When select_idle_capacity() starts scanning for an idle CPU, it starts with target CPU that has already been checked in select_idle_sibling(). So we start checking from the next CPU and try the target CPU at the end. Similarly for task_numa_assign(), we have just checked numa_migrate_on of dst_cpu, so start from the next CPU. This also works for steal_cookie_task(), the first scan must fail and start directly from the next one. Signed-off-by: Hao Jia Acked-by: Mel Gorman Reviewed-by: Vincent Guittot --- kernel/sched/core.c | 2 +- kernel/sched/fair.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index cb2aa2b54c7a..5c3c539e1712 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6154,7 +6154,7 @@ static bool steal_cookie_task(int cpu, struct sched_domain *sd) { int i; - for_each_cpu_wrap(i, sched_domain_span(sd), cpu) { + for_each_cpu_wrap(i, sched_domain_span(sd), cpu + 1) { if (i == cpu) continue; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index dfcb620bfe50..ba91d4478260 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1824,7 +1824,7 @@ static void task_numa_assign(struct task_numa_env *env, int start = env->dst_cpu; /* Find alternative idle CPU. */ - for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start) { + for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start + 1) { if (cpu == env->best_cpu || !idle_cpu(cpu) || !cpumask_test_cpu(cpu, env->p->cpus_ptr)) { continue; @@ -6663,7 +6663,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) task_util = uclamp_task_util(p); - for_each_cpu_wrap(cpu, cpus, target) { + for_each_cpu_wrap(cpu, cpus, target + 1) { unsigned long cpu_cap = capacity_of(cpu); if (!available_idle_cpu(cpu) && !sched_idle_cpu(cpu))