From patchwork Fri Oct 21 06:15:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Jia X-Patchwork-Id: 6524 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp521408wrr; Thu, 20 Oct 2022 23:32:54 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6ly3MZ678ZV+y0wvMQgV/SKtfUxryDdtTzJyDomIk+fNT/dhe+/yrDhGVtYVcQvGjTxzE2 X-Received: by 2002:a05:6402:520f:b0:45f:b7a0:a31 with SMTP id s15-20020a056402520f00b0045fb7a00a31mr8138731edd.379.1666333974651; Thu, 20 Oct 2022 23:32:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666333974; cv=none; d=google.com; s=arc-20160816; b=xun0AzmeLLr0n3vQm6N0WiHzWh2w+6Z4joQEK1XXo1456pD/+WSAbjIFc9XkyMPV5c YruH/im4HPfk5E9AfXxXaY/vnEo6HQwmbGEljWkHHjdnaJdbTr2EybcAdETnYE8/4Xxc +6HoTJn/3fcAP46Us/ioPKrtDBv58CxG4kpCWya2X+rjvxsExs1xd3EVFYaQVfiG2tJP OwOzMK643txHh3o35VUkkWJY2Y3vjxrf/+VB30Oi/WCX/jLGfBuZMxknWJSnnzeQzFO1 G5NILRjGUf2nx/u7XTq+1oNVPyeJr/GXJVpHba2qgOlrGz/9PzwXrdERK4RVKH02WM/W 43VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=i19y5QR9mtVVOPA0wEofADzlXW2aN27SSyV8zYQJio8=; b=fQUbKjFWGz1dySatZfGKI6SmALzFq1AWlOrXvs9frBPvpuwzbincfGolEXnCtibXEm u9o4ekpV/xzLtvVD0znLLB1x4BDDIakK/lgVOuSPX+1G4gJ382ohxbJH4Uuy6qnAAow1 1GOP9budJz6EtlkJhDBuEnpTa+s0HKnE2y57goutHowW27kdgrX0Ui6Wh3Qac81WUjpI KyZsF5NLq1waIlpom2GPn5xrWghihGAar2ePpf9Mo8018S8Ps2ryWJ7zN6VY4joYxnGL AtGQRAGbnEi3uNaGyKQV+WBVbQb1Q8KC3IqwvmK9+RSuQCoIizgYPsXvGvUqbP2UdDX0 NyWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=u1ImNYfd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qb7-20020a1709077e8700b007878f030816si18886607ejc.109.2022.10.20.23.32.30; Thu, 20 Oct 2022 23:32:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=u1ImNYfd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229763AbiJUGQ0 (ORCPT + 99 others); Fri, 21 Oct 2022 02:16:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230123AbiJUGQU (ORCPT ); Fri, 21 Oct 2022 02:16:20 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EA7D237960 for ; Thu, 20 Oct 2022 23:16:19 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id g8-20020a17090a128800b0020c79f987ceso5711938pja.5 for ; Thu, 20 Oct 2022 23:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i19y5QR9mtVVOPA0wEofADzlXW2aN27SSyV8zYQJio8=; b=u1ImNYfdUbWLriSGwT46wNN9Zle5dKFToW2gR/VcF1g2jLV1fk/3baGCIQJH1A/qHW 1JTbFXqdbvsDobztBPBy4yKK94U7TPmgYBPa1J+PTLkBOGD/KGHvzoQjyViTsLqPNBIM cC6L3nXkvhkY/jfftsf1rvrJoUGSg1MKMfmLEF5Lz1Kvn7o/h9VICAGgRrgUQ1uZTdIO Zce3wdZYmxrSGz4IC9ZI9BVPfk+gcnA7Wv0C+tH+zctYJ3FYSkc/qK/uTqve9BR6qLtb mnSIxAin35S7/lAr565wadFXn88M9dJqvIshpGqhVrBI/QSgp2Xw8VQBHZ4TDEnm9buC dQig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i19y5QR9mtVVOPA0wEofADzlXW2aN27SSyV8zYQJio8=; b=qSKgqhBxPSF9LxS4/GKgIN8T/WvbJrh6uqkQRW+4MRaPf4CnFsdXRMzirPJjtSOftl tGgrR8/yVLl0TsLZgjLKp/j3Ig3bScNr7i7XCu9cMezHqOMDiQkHgFS+NyhIdjUIliK7 wcIO5jDKpZYfLgXjlzdCWuQeFuTh6pnNSrGmeZFo5fv/VHsk365VixH3xvwOix35ihuk uT6VymAC4aUw8GYwlpBRCnP9tqNvhVwaeOL3ysdkOjZg9XsCHhlHZGvmuf3y2H1xVUvb Cox/wp9AbT6wkqe3Sjp9boClnlXhzj84pfEI2o5Tis2kKFCFq1W3EHMfPEw8QwN96trg 1A/A== X-Gm-Message-State: ACrzQf1aoFa2sSAgtEETnshYDvQYI/O4pVg+48YGTlyps74g7aBurJW7 5dBvEk/Thg7f+BOl6NjoeqD4m+Cu21pAJmGD X-Received: by 2002:a17:90b:384e:b0:20d:4b52:ddf0 with SMTP id nl14-20020a17090b384e00b0020d4b52ddf0mr20742540pjb.113.1666332978010; Thu, 20 Oct 2022 23:16:18 -0700 (PDT) Received: from C02G87K0MD6R.bytedance.net ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id o11-20020a17090a4b4b00b0020d45a155d9sm327532pjl.35.2022.10.20.23.16.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Oct 2022 23:16:17 -0700 (PDT) From: Hao Jia To: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mgorman@techsingularity.net Cc: linux-kernel@vger.kernel.org, Hao Jia Subject: [PATCH 1/2] sched/numa: Stop an exhastive search if an idle core is found Date: Fri, 21 Oct 2022 14:15:57 +0800 Message-Id: <20221021061558.34767-2-jiahao.os@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20221021061558.34767-1-jiahao.os@bytedance.com> References: <20221021061558.34767-1-jiahao.os@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747277814025412699?= X-GMAIL-MSGID: =?utf-8?q?1747277814025412699?= In update_numa_stats() we try to find an idle cpu on the NUMA node, preferably an idle core. When we find an idle core, we can stop searching. Signed-off-by: Hao Jia Acked-by: Mel Gorman --- kernel/sched/fair.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e4a0b8bd941c..b7cbec539c77 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1792,7 +1792,7 @@ static void update_numa_stats(struct task_numa_env *env, ns->nr_running += rq->cfs.h_nr_running; ns->compute_capacity += capacity_of(cpu); - if (find_idle && !rq->nr_running && idle_cpu(cpu)) { + if (find_idle && idle_core < 0 && !rq->nr_running && idle_cpu(cpu)) { if (READ_ONCE(rq->numa_migrate_on) || !cpumask_test_cpu(cpu, env->p->cpus_ptr)) continue; @@ -1801,6 +1801,9 @@ static void update_numa_stats(struct task_numa_env *env, ns->idle_cpu = cpu; idle_core = numa_idle_core(idle_core, cpu); + /* If we find an idle core, stop searching. */ + if (idle_core >= 0) + ns->idle_cpu = idle_core; } } rcu_read_unlock(); @@ -1808,9 +1811,6 @@ static void update_numa_stats(struct task_numa_env *env, ns->weight = cpumask_weight(cpumask_of_node(nid)); ns->node_type = numa_classify(env->imbalance_pct, ns); - - if (idle_core >= 0) - ns->idle_cpu = idle_core; } static void task_numa_assign(struct task_numa_env *env, From patchwork Fri Oct 21 06:15:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Jia X-Patchwork-Id: 6523 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp521396wrr; Thu, 20 Oct 2022 23:32:52 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7FSbwy1DhCDB9Sj9+Yd0qwdS9rM+nkOZ76300w6pQmyTzSe4lCOIJkKiU5SIQzQboEiA9K X-Received: by 2002:a17:907:608f:b0:78e:1b60:60e2 with SMTP id ht15-20020a170907608f00b0078e1b6060e2mr14320235ejc.382.1666333972208; Thu, 20 Oct 2022 23:32:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666333972; cv=none; d=google.com; s=arc-20160816; b=0hBmv/DuKwife3Uuj1RU8bk9zvBsdV45pXQIr4H4cZfZwtH245U9V8+Y41ONgxj5XC 9Gi+D4I6Wf+CM5qmNhb/zN3geq7gJWZg1ZuBWwCUt+f5AlDspfsqMy6SV+mSmYPFAT6n lPfT2Tdeu39a1TlryUV79+NBOUF4AcEj+m1K/gVjlgS+stsd69TFeNrf55Egw8y7p0u1 yIqFWh5IVM5w++53q+rXvZY1P9vbSik0vAuVpl4st1gSJzWK/N/o2Y+t11oFLl9HmUvS u6rwhTJr4oyxw8z9cXcSuMkbmK7dg49/8+XYG3tOMu5U3nqdxNr7FQTiwp47GpE8yRJa kBmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hVI308eXyS8Lbu1XFDgJY09yiy/y64U8mWrCVAZtfNM=; b=TJTMLzZ2R4+hfaXysfcEs0Gudd35YAJPixfDlaht25OXXZ0TLHsxtt8APos7Z3kT7m qmDu6qiMxIGhMf3CTcHVg5TLMX/i8dyP8+ETRSnnhgVOAMVZplbc1bMU3bIuk8w3+rGV nRXcehx7T1thYrzMughvGCZBnm+55X2LBsnXWjhTwRc0xFX2AdBXFHiyqvvYqR4RvEoJ fXXCnWr9qefOcSpe7wpbc/YEInpGVCKDtsuAnnTjqqSDPgE02Ild+J1HaJ3l43PKK5Ap ZY/x7Z8VXe+C22nrgISUNOZilkNKtWW80FzFwo3yz8o3U2mFugpJLlhfOM0d2XuMJdH0 /0xQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=ApOsebNM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hu8-20020a170907a08800b0078d202b7a3bsi2050669ejc.421.2022.10.20.23.32.27; Thu, 20 Oct 2022 23:32:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=ApOsebNM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230132AbiJUGQ3 (ORCPT + 99 others); Fri, 21 Oct 2022 02:16:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230126AbiJUGQ1 (ORCPT ); Fri, 21 Oct 2022 02:16:27 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 802F4102DF6 for ; Thu, 20 Oct 2022 23:16:26 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id f8-20020a17090a664800b00212464cd49fso1883435pjm.4 for ; Thu, 20 Oct 2022 23:16:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hVI308eXyS8Lbu1XFDgJY09yiy/y64U8mWrCVAZtfNM=; b=ApOsebNMMrUy1nIx9uaw7MCe9ozsGFebYEhBLz0bsBGKXFN129y/H60GbCNJZsA9iX hwsUE9Z2hqIjcaf8wEsh4qz6GtRIbp6UK2WD0PeZ0gnEFZojbgAsAjRGBYeBsZzQY/rk mdA+TPES9mVRCvQ3uBb77fJAq7rfD5iqvrmB9GZUcsTkC2nlJcJ6NWenQDAvFktb/zA9 4iS7qgImoGOZcEdP5Q/t0LgUISEoSSXgvwjpuch/meb4HXpzSM8/75GBDzTBZQrl9DLu gecBtwLI7dxO/7Wzp7s43WQYUw17KPE68jK2Fd/D+9UR7Adf5blpFf/TixbL/cY1yPPl enWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hVI308eXyS8Lbu1XFDgJY09yiy/y64U8mWrCVAZtfNM=; b=jyZRBLjOEW9oeQPIgm41VZUxlY3Jsobhetfp8SnQZbiL6Y9Ij8IOWE2iY5NrEUaHPa JZyECnKB6jFODJJsCTqAFLTUg738jTIilxhf+3rqeYpZN3hYasNvLg0SjH3H5HKBYhYx MGDd14zXLF7U2qRvbJvShJ/vua9711sLpO8T95qVqj8PvFd4ZhSBx94njU/G2v1RssMu 5QtAJcr1VxMaXoViI4rSd1hSimvOumDtZrQ6z4FrH58tsjSPDw0+8k0AXqYPaZSQHlFg UA8FsWUrJKmsb7m4cRJf/BjktFTRlHqE9lw7TQTltDn8n9u6gZ3G+R9nssF05BAF6YfT flZw== X-Gm-Message-State: ACrzQf0eBY91qTJmm/YCUUrVWhdHj+bL6k1HoQIPViC7ic5MIorwsmwR 82qlA2gg1+KrL5iebALnLvlp+Q== X-Received: by 2002:a17:90b:1b0b:b0:20d:7c31:e75d with SMTP id nu11-20020a17090b1b0b00b0020d7c31e75dmr19924620pjb.101.1666332985621; Thu, 20 Oct 2022 23:16:25 -0700 (PDT) Received: from C02G87K0MD6R.bytedance.net ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id o11-20020a17090a4b4b00b0020d45a155d9sm327532pjl.35.2022.10.20.23.16.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Oct 2022 23:16:24 -0700 (PDT) From: Hao Jia To: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mgorman@techsingularity.net Cc: linux-kernel@vger.kernel.org, Hao Jia Subject: [PATCH 2/2] sched/core: Optimize the order of scanning CPU Date: Fri, 21 Oct 2022 14:15:58 +0800 Message-Id: <20221021061558.34767-3-jiahao.os@bytedance.com> X-Mailer: git-send-email 2.37.0 (Apple Git-136) In-Reply-To: <20221021061558.34767-1-jiahao.os@bytedance.com> References: <20221021061558.34767-1-jiahao.os@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747277811033885201?= X-GMAIL-MSGID: =?utf-8?q?1747277811033885201?= When select_idle_capacity() starts scanning for an idle CPU, it starts with target CPU that has already been checked in select_idle_sibling(). So, we finally try the target CPU. Similarly for task_numa_assign(), we have just checked numa_migrate_on of dst_cpu, so start from the next CPU. This also works for steal_cookie_task(), the first scan must fail and start directly from the next one. Signed-off-by: Hao Jia --- kernel/sched/core.c | 2 +- kernel/sched/fair.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5800b0623ff3..f6ad68714546 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6154,7 +6154,7 @@ static bool steal_cookie_task(int cpu, struct sched_domain *sd) { int i; - for_each_cpu_wrap(i, sched_domain_span(sd), cpu) { + for_each_cpu_wrap(i, sched_domain_span(sd), cpu + 1) { if (i == cpu) continue; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b7cbec539c77..d95ee285734e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1824,7 +1824,7 @@ static void task_numa_assign(struct task_numa_env *env, int start = env->dst_cpu; /* Find alternative idle CPU. */ - for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start) { + for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), start + 1) { if (cpu == env->best_cpu || !idle_cpu(cpu) || !cpumask_test_cpu(cpu, env->p->cpus_ptr)) { continue; @@ -6663,7 +6663,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) task_util = uclamp_task_util(p); - for_each_cpu_wrap(cpu, cpus, target) { + for_each_cpu_wrap(cpu, cpus, target + 1) { unsigned long cpu_cap = capacity_of(cpu); if (!available_idle_cpu(cpu) && !sched_idle_cpu(cpu))