From patchwork Tue Nov 22 20:35:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24546 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2424806wrr; Tue, 22 Nov 2022 12:30:03 -0800 (PST) X-Google-Smtp-Source: AA0mqf7ZKz12JDURRYsZqedI7DCFF+vmtRa2J6SK8SrfYYh+uB7HBR+GosMPi5Fdo3x2bcsAqIr6 X-Received: by 2002:a05:6402:3d1:b0:46a:fd6:207 with SMTP id t17-20020a05640203d100b0046a0fd60207mr1008564edw.28.1669149002870; Tue, 22 Nov 2022 12:30:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149002; cv=none; d=google.com; s=arc-20160816; b=FuKrua6MDmVX2PaevjpJio47CqmbJH0O1EXBS5uSYn1yRlB7nh4FJq+Fy27/lZBETW MCGkhTya5Yc7BR0MrKMZH5ygU5+fmKcUfvhjuzkPjSwdxkHTgcePV9EBvQDgf3OSZjEB mST3K55fO90XSU806EExWCut2MCFLuAhAusVyly0kl/FsrU0DNy8V53Rr8g5DZrz9IDA ICNjMEqSXEifhZ8QDXsg/TGHd4fFI4dWcXZIJHdBu0OhdWEVdu8S8KkcHxCZWdPZiFVQ ifi56ZODGH7H5VhY0lPkD2TSxNbmHPYJIGclH8zTc79pPjZg38w6WXYczp0FWnODVFSR MipQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=0FmmaYWUUzyBYDYSIO8jz1OqA5s3Tbt6NxESnfIcMjM=; b=YfoFjaHNn+xkgiR1YydKRuaVI660h/7lJ27FgtOcZWKDV7SXuwtM3fHop598OvIP6w tdpxyo2NDHOooxlDIGyfGj9+qq3vRVUtUBJ6gH5ZXwMULEofQ7JaehuQDqcOxPQLpOCL GRPxJipKPmC/lYHe64nt7ncO+u9Gc9lnxECvHyQd7q4pJYUJGnWOG6lgM989+3J0A4GY jaRRkvN/Mahcb+GefsJ4j7yfL/G4TzVfkLLSkPVaFiRaJtPhNgf5iejz5VZ0KbptCnpp 6YVswf+g8E6MIypOCbC6hqyXw6hzH0GskJVw+wRFXz4iwHVmrm87mLG/FVkaLecr4qrr ZuVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VixAVveT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t21-20020a056402525500b00458d1c48708si13557790edd.303.2022.11.22.12.29.38; Tue, 22 Nov 2022 12:30:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VixAVveT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234791AbiKVU2Z (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232554AbiKVU2O (ORCPT ); Tue, 22 Nov 2022 15:28:14 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E11EE1AD83 for ; Tue, 22 Nov 2022 12:28:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148893; x=1700684893; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=0yHmKdRsV591tHGMnKRYVefaIOCtc6d0I/pJdG+LlvU=; b=VixAVveTTgBILU537x5lOBygOGbEtNITiTtO12/uMiymItMal26+/PfT EWaE9taAOryiZQ6bDPqaWr3nfsCf0VphmrraexHIxFL8sTO/zJld679p9 g+BtA8xRQgcjsCVwlYe5hV/mBO56OETLCkqBk3XEIvP81yfF7wt5Z7WNG 4B7c3G1gLvUn9oT0eIuM7Ai8jn4qPUVeJoOhz7pXI5HjeiHiRbq6lW7VZ OcLRgnPwvKooEJTPH1hpsI2JW6MGrIgGRQLxgRexjNnc954PMgPISKr+B abdROcQZJWxzIgBvFXP66hfbPPHvYVOS91j28vHgdmyQjzgT8TcHZL+LG Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616500" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616500" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228121" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228121" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:12 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 1/7] sched/fair: Generalize asym_packing logic for SMT local sched group Date: Tue, 22 Nov 2022 12:35:26 -0800 Message-Id: <20221122203532.15013-2-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229584893947737?= X-GMAIL-MSGID: =?utf-8?q?1750229584893947737?= When balancing load between two physical cores, an idle destination CPU can help another core only if all of its SMT siblings are also idle. Otherwise, there is not increase in throughput. It does not matter whether the other core is composed of SMT siblings. Simply check if there are any tasks running on the local group and the other core has exactly one busy CPU before proceeding. Let find_busiest_group() handle the case of more than one busy CPU. This is true for SMT2, SMT4, SMT8, etc. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Len Brown Signed-off-by: Ricardo Neri --- Changes since v1: * Reworded commit message and inline comments for clarity. * Stated that this changeset does not impact STM4 or SMT8. --- kernel/sched/fair.c | 29 +++++++++-------------------- 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e4a0b8bd941c..18c672ff39ef 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8900,12 +8900,10 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, struct sched_group *sg) { #ifdef CONFIG_SCHED_SMT - bool local_is_smt, sg_is_smt; + bool local_is_smt; int sg_busy_cpus; local_is_smt = sds->local->flags & SD_SHARE_CPUCAPACITY; - sg_is_smt = sg->flags & SD_SHARE_CPUCAPACITY; - sg_busy_cpus = sgs->group_weight - sgs->idle_cpus; if (!local_is_smt) { @@ -8926,25 +8924,16 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); } - /* @dst_cpu has SMT siblings. */ - - if (sg_is_smt) { - int local_busy_cpus = sds->local->group_weight - - sds->local_stat.idle_cpus; - int busy_cpus_delta = sg_busy_cpus - local_busy_cpus; - - if (busy_cpus_delta == 1) - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); - - return false; - } - /* - * @sg does not have SMT siblings. Ensure that @sds::local does not end - * up with more than one busy SMT sibling and only pull tasks if there - * are not busy CPUs (i.e., no CPU has running tasks). + * @dst_cpu has SMT siblings. Do asym_packing load balancing only if + * all its siblings are idle (moving tasks between physical cores in + * which some SMT siblings are busy results in the same throughput). + * + * If the difference in the number of busy CPUs is two or more, let + * find_busiest_group() take care of it. We only care if @sg has + * exactly one busy CPU. This covers SMT and non-SMT sched groups. */ - if (!sds->local_stat.sum_nr_running) + if (sg_busy_cpus == 1 && !sds->local_stat.sum_nr_running) return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); return false; From patchwork Tue Nov 22 20:35:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24548 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2424905wrr; Tue, 22 Nov 2022 12:30:15 -0800 (PST) X-Google-Smtp-Source: AA0mqf4bu/nnp4rXW4fEpLjmZjch8XdsU0eVtcA3zqWHEzBUyPUpQPSWEJY0dbPSxCvirbWsHiId X-Received: by 2002:a50:ff0d:0:b0:461:c6e8:452e with SMTP id a13-20020a50ff0d000000b00461c6e8452emr12627199edu.298.1669149015080; Tue, 22 Nov 2022 12:30:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149015; cv=none; d=google.com; s=arc-20160816; b=x91ZWMyRiz9b3SlrOWeNfN3by9kF6IM2XS2pCXhTQBwBxxjnG8rDYyPdYRSY9iMC5H AubpM75s42GFS6CKoAanToZ6McadU761CS0mRUhr3KDHD/1KXHSoNXjj/WBMoxvZe7Kp 5HzED1uu3XeL9LkDKxfDYlnTDZVTahLvwedu1PghJjx1tiHRjA7niUMpGis5ggSARqi+ Hqq6fKPzx+08uQdxqV+u2uns1R1lS76Er2Jct3vq+JY4lT9Sf6E/b9HprtMlgko+lXwF 4WAQont/sHr4ZLRzliOjhL5J39xNPXzChjDdsBsVw9GErJ/pTbOP0VePLDXm+uSZomhj 4h+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=qPGeA7JlzuAV3zVWS3gSd6xHntcrcFh/0jv3Qbp5gnI=; b=LfigE0yQPldWYHHMIZDo1ovM6l/q4qfx+s1xTtvYagZiPPwQCdyev20SG/j13hrp1w xRzJkaImRxccrs2qNb3S0Zg236x5nBywG9O/rrTMbH7SnWh3xOXxdxMztFEu5rBIbWPO X13rhrV6ws6ytW+QvxEC/5wypiLZFQHzftN7jUUJBdahTpFeUDwNj7/pu0BIFPAu7mXQ cCINYTyiQNYtFOcYJkasYXOGyprI6EyfBESEYoEQdJT9zp4aY87WUcPUTy9A2R4yR6X0 rSmDG2snYgYHZrxZcYyn0sIkHfWRlqFDG7H17JkTMmq/zJPS6wxeeTzR8GQz63TsqPUU MyBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MxSPRBvh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jg27-20020a170907971b00b0073d81935416si15299232ejc.749.2022.11.22.12.29.50; Tue, 22 Nov 2022 12:30:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MxSPRBvh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234861AbiKVU2e (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234490AbiKVU2O (ORCPT ); Tue, 22 Nov 2022 15:28:14 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FBB6165A6 for ; Tue, 22 Nov 2022 12:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148894; x=1700684894; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=y/1MAtHpQYdgN3xn2hBOyK3a06TFyZ6nWBTd2NULu+4=; b=MxSPRBvh003O+2p68QIC1ZbnVoGu99y4zO9Yy24dMEH3kuTwg1Icre7+ uDKuvzxf3eCuDeyF9YK1i07kq+IddE+20oF8s7oJPrV7CCYBOzeKkWgQj FMqOqvYzzyOF/3GUulFs+iBLkZskZKp2GnSDeEslNnZ2TG6OSV/A18wku K6PFVnT+sBUZeZCwwjo1yUdvuIwZzxpNKSXXsy/iB/ufuEKPoecm48iDm 3F7/PFXNPKveqHQacpxV0IeK2bzGpzo9CeCcWW8rJ/k8VXzVij18OvEZ+ D831iiw0WsFamIkkW66TfCIuW0FOPLsrh9V5XmEG6WJHw/SRt+zleXnyn w==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616506" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616506" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228127" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228127" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:12 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 2/7] sched: Prepare sched_asym_prefer() to handle idle state of SMT siblings Date: Tue, 22 Nov 2022 12:35:27 -0800 Message-Id: <20221122203532.15013-3-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229597858373862?= X-GMAIL-MSGID: =?utf-8?q?1750229597858373862?= The throughput of an SMT sibling decreases if one or more of its siblings are also busy. Idle, lower-priority cores can help. Thus, it is necessary to consider the idle state of the SMT siblings of CPUs when selecting by priority. In some cases, sched_asym_prefer() does not care about the idle state (when building sched domains or looking at the priority of the preferred CPU in a sched group). Add a new parameter to check the state of the SMT siblings of a CPU when applicable. While here, remove a spurious newline. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch --- kernel/sched/fair.c | 17 ++++++++++------- kernel/sched/sched.h | 8 ++++++-- kernel/sched/topology.c | 6 +++++- 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 18c672ff39ef..d18947a9c03e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8921,7 +8921,7 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, * can help if it has higher priority and is idle (i.e., * it has no running tasks). */ - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); + return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu, false); } /* @@ -8934,7 +8934,7 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, * exactly one busy CPU. This covers SMT and non-SMT sched groups. */ if (sg_busy_cpus == 1 && !sds->local_stat.sum_nr_running) - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); + return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu, false); return false; #else @@ -8952,7 +8952,8 @@ sched_asym(struct lb_env *env, struct sd_lb_stats *sds, struct sg_lb_stats *sgs (group->flags & SD_SHARE_CPUCAPACITY)) return asym_smt_can_pull_tasks(env->dst_cpu, sds, sgs, group); - return sched_asym_prefer(env->dst_cpu, group->asym_prefer_cpu); + /* Neither env::dst_cpu nor group::asym_prefer_cpu have SMT siblings. */ + return sched_asym_prefer(env->dst_cpu, group->asym_prefer_cpu, false); } static inline bool @@ -9118,7 +9119,9 @@ static bool update_sd_pick_busiest(struct lb_env *env, case group_asym_packing: /* Prefer to move from lowest priority CPU's work */ - if (sched_asym_prefer(sg->asym_prefer_cpu, sds->busiest->asym_prefer_cpu)) + if (sched_asym_prefer(sg->asym_prefer_cpu, + sds->busiest->asym_prefer_cpu, + false)) return false; break; @@ -10060,7 +10063,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, /* Make sure we only pull tasks from a CPU of lower priority */ if ((env->sd->flags & SD_ASYM_PACKING) && - sched_asym_prefer(i, env->dst_cpu) && + sched_asym_prefer(i, env->dst_cpu, true) && nr_running == 1) continue; @@ -10153,7 +10156,7 @@ asym_active_balance(struct lb_env *env) * highest priority CPUs. */ return env->idle != CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING) && - sched_asym_prefer(env->dst_cpu, env->src_cpu); + sched_asym_prefer(env->dst_cpu, env->src_cpu, true); } static inline bool @@ -10889,7 +10892,7 @@ static void nohz_balancer_kick(struct rq *rq) * around. */ for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { - if (sched_asym_prefer(i, cpu)) { + if (sched_asym_prefer(i, cpu, true)) { flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; goto unlock; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index a4a20046e586..0fc7c0130755 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -795,8 +795,12 @@ static inline long se_weight(struct sched_entity *se) return scale_load_down(se->load.weight); } - -static inline bool sched_asym_prefer(int a, int b) +/* + * Used to compare specific CPUs. Also when comparing the preferred CPU of a + * sched group or building the sched domains; in such cases checking the state + * of SMT siblings, if any, is not needed. + */ +static inline bool sched_asym_prefer(int a, int b, bool check_smt) { return arch_asym_cpu_priority(a) > arch_asym_cpu_priority(b); } diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 8739c2a5a54e..8154ef590b9f 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1282,7 +1282,11 @@ static void init_sched_groups_capacity(int cpu, struct sched_domain *sd) for_each_cpu(cpu, sched_group_span(sg)) { if (max_cpu < 0) max_cpu = cpu; - else if (sched_asym_prefer(cpu, max_cpu)) + /* + * We want the CPU priorities unaffected by the idle + * state of its SMT siblings, if any. + */ + else if (sched_asym_prefer(cpu, max_cpu, false)) max_cpu = cpu; } sg->asym_prefer_cpu = max_cpu; From patchwork Tue Nov 22 20:35:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24547 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2424812wrr; Tue, 22 Nov 2022 12:30:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf4jw3VM9XGltc32Huq/2ahLbr3xnUs4ywuB2EH/uYr/uRPlX9j9Ekj10K8UYSdsu/bCrKB7 X-Received: by 2002:aa7:cd8d:0:b0:463:19ca:a573 with SMTP id x13-20020aa7cd8d000000b0046319caa573mr22490675edv.31.1669149003849; Tue, 22 Nov 2022 12:30:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149003; cv=none; d=google.com; s=arc-20160816; b=UHVbdfXF35J5ZbH80qXT8qdx7ba7Pq/V1GUWAdnnqCJQ0hUTgy3+ypq7sX6xTeLyKT JNUPtXQuxI5CXvIyZYJfooyGN3uCpGQ0oAETsNnG1slxyemcIjhxwPP/dJDrSx+ep+tB tUbv5gc4BuZ2U6P2RVnUHtjT14YVzu13d/q+WwnsjdO7eXBBaIRjZLLdKCXmUI/sFsg0 setRsMjeOCdizhaAUs482u2W2S6FFzJ4ldxv4qKjBBFUPPHUrdar7RTxVHBueHXq7SiE TquE1KfIYy267Q4xfBmzuJciumEmbR9VR91GGe+r7jeDTtTCBCWZqNud8HUT0bVCvksn i3Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=qfdLMp6C2mQQHbjZslPrJ3feswPbF4MER/ytrchrErw=; b=sf5M0T3XaqPStNegI9HgnMpllx9Deb63lRLDzZKQAjFGDxPf2Yjvz6yflMvRPMuplW X9MaZr5vEDhKNmR2BRJ7ForZWUAVOGStmK0XjXW7Dbwje7oEQ2f3KoU9ZMWo9eY+wXXc AhJ0wWx4mCnQ+DMQmFBXfI0HQiijBRV3jXNmOSefbdJVLsdIIyNMpQ7Le0CwEtnpcd+x m2UDp4wjUqzbY6aT3lNlu0N840Y3eEGO3v4dpXPQ/x5CfrM8SC1lgXt83/9n4zdafDgV L7JryFeupV+0AoWfe8VkM5mOi6m0QhV22oVShxNLcshVo16N1HNHRpyVH3o70OHFWTms HVWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="YptDn/a3"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n11-20020a170906118b00b0077e04f856a2si6140945eja.541.2022.11.22.12.29.40; Tue, 22 Nov 2022 12:30:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="YptDn/a3"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234853AbiKVU2a (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234457AbiKVU2P (ORCPT ); Tue, 22 Nov 2022 15:28:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3B1F1ADB9 for ; Tue, 22 Nov 2022 12:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148894; x=1700684894; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Tu8yb4rTFtiG5/ROXb+qs9Ym1yY05u5ArJRxVYfGmFo=; b=YptDn/a3/nVFOVtzeQfSQeV98c6tT1NwvCbizeZq55KsOlAwv4ge4FUe 8aIQAfVMfwBhZo3q+3Kh+36152bdNfkn2358g6SNsOcKl0wFhO0MzZGS5 qFrKDTXl5sg4/VMENHGAbzwUkLOT/gYLSP2rI2etx8Swn9cFru1tBsoMC zqD6sAD2A7EdQyhKrAdJCGqjzX0DyPQNG1Ak5v700hb6Y7V/RJb0deiGZ EZQiAKhcVC9NscnX/CG3ZuQudCv4pcAukhWC3ya5FxmaHKfIWcMVXWdNj IU8PDE03v+VaojhDTTG908ULMkBDBPBDmlVPjY+6mo8aF6kMTARamaUYn Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616507" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616507" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228132" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228132" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:13 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 3/7] sched: Teach arch_asym_cpu_priority() the idle state of SMT siblings Date: Tue, 22 Nov 2022 12:35:28 -0800 Message-Id: <20221122203532.15013-4-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229586105324686?= X-GMAIL-MSGID: =?utf-8?q?1750229586105324686?= Some processors (e.g., Intel processors with ITMT) use asym_packing to balance load between physical cores with SMT. In such case, a core with all its SMT siblings idle is more desirable than another with one or more busy siblings. Other processors (e.g, Power7 with SMT8) use asym_packing to balance load among SMT siblings of different priority, regardless of their idle state. Add a new parameter, check_smt, that architectures can use as needed. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- arch/x86/kernel/itmt.c | 2 +- include/linux/sched/topology.h | 2 +- kernel/sched/fair.c | 5 ++++- kernel/sched/sched.h | 3 ++- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 9ff480e94511..4cb5a5e4fa47 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -167,7 +167,7 @@ void sched_clear_itmt_support(void) mutex_unlock(&itmt_update_mutex); } -int arch_asym_cpu_priority(int cpu) +int arch_asym_cpu_priority(int cpu, bool check_smt) { return per_cpu(sched_core_priority, cpu); } diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 816df6cc444e..87b64b9776f6 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -63,7 +63,7 @@ static inline int cpu_numa_flags(void) } #endif -extern int arch_asym_cpu_priority(int cpu); +extern int arch_asym_cpu_priority(int cpu, bool check_smt); struct sched_domain_attr { int relax_domain_level; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d18947a9c03e..0e4251f83807 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -142,8 +142,11 @@ __setup("sched_thermal_decay_shift=", setup_sched_thermal_decay_shift); #ifdef CONFIG_SMP /* * For asym packing, by default the lower numbered CPU has higher priority. + * + * When doing ASYM_PACKING at the "MC" or higher domains, architectures may + * want to check the idle state of the SMT siblngs of @cpu. */ -int __weak arch_asym_cpu_priority(int cpu) +int __weak arch_asym_cpu_priority(int cpu, bool check_smt) { return -cpu; } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 0fc7c0130755..e5e52c2e82de 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -802,7 +802,8 @@ static inline long se_weight(struct sched_entity *se) */ static inline bool sched_asym_prefer(int a, int b, bool check_smt) { - return arch_asym_cpu_priority(a) > arch_asym_cpu_priority(b); + return arch_asym_cpu_priority(a, check_smt) > + arch_asym_cpu_priority(b, check_smt); } struct perf_domain { From patchwork Tue Nov 22 20:35:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24549 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2425056wrr; Tue, 22 Nov 2022 12:30:35 -0800 (PST) X-Google-Smtp-Source: AA0mqf6+iRSDjPzdmXMw969XFqvFh2T7wrl/8YeBCKpHpMTx+D741FbUQAgAG2TR+sYw97eDbVKi X-Received: by 2002:a05:6402:1a2e:b0:461:2915:e41d with SMTP id be14-20020a0564021a2e00b004612915e41dmr22301700edb.184.1669149035320; Tue, 22 Nov 2022 12:30:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149035; cv=none; d=google.com; s=arc-20160816; b=MRBf2tVJB7JRlmnpwLadhN/saTOKW/VhL6++HjJDbIANFxTwghqeTXJ2y3MfiWFgT8 w2CXFcVafDb2sv4Q2HDxJh/Rs62O8i83PBiai4v1Jv9dBTPU6mx4D3qB0fUd3fTbRYU8 Fk2RiKDUjiACxl3j6iwo7h8LKpOQH/cN7x8fCGWspuZccbQq1v4/PrMLns6VNmaVACsh 0znZQJQ5wyLH3rM7OiXAynUFbLHc8SkX91TElBa0LsWXd68thXAknmtTQAIq95GhSDHM D89KNz/nrrTEtIpMaU/7WBK5ghjTXDeCzhV+hB4EfeMPr2SsZGalvB6EASdMRb1eWTKp YByQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=+hZ/T+LG2SEQUHOc50V28ZpAG9z7f3Qnk7J/1+X9MBw=; b=i6dJtMhgIcfQZZsj9sMzMlASU6tpMzQU2Z5MPfOrzVT3RDD7tI3GZ1dHqLeB4l59v0 Z1h7EDguWpLhI0y/jKcsPbpn+Fwvf1Uspp3pBgQr2W9YVhheerDX3SFUMNoD2uq8VoG8 Z7aitWWF6wlKDeegUkkP/Woc2U5eTmSsV4ea5pMNX8M24+csITW++0S2GOAGgS7hjO3t lLaSoUpkn7Wyzif9LOOxhC90RLbYpS35nH0fhOsm3CZhtSXYW+Vpsz7l2VliWyNgnXUT OtzArmEHlHTapmFHiPzn11UouzOc5SxpKts08o4sf/GcCjPmSc8OPNIcvj9GupX97MQE tS6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YO5abYmc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i21-20020a170906251500b007ae52a5707bsi9192881ejb.95.2022.11.22.12.30.11; Tue, 22 Nov 2022 12:30:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YO5abYmc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234880AbiKVU2h (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234402AbiKVU2P (ORCPT ); Tue, 22 Nov 2022 15:28:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC03C1B1D8 for ; Tue, 22 Nov 2022 12:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148894; x=1700684894; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ZxiRvtsJwz9dFNPg5MHP4zjCzO9YSRx6bjDINyhvJ5o=; b=YO5abYmcCWkxOVmwE0RoQ8gdQ02Gl9vS8d2LzhQcaNrhIaclRTjZDDva w5ZimWTsceDPCaNUq56YuyLrJ8tzVeStPDv6caiGQTFbfd2pNFRc6Zy2D reQHuqADcqYYODzbEyaRuUVGxwJiw7DSV7WqyPBLunxBEVkDHmPrcLA1I UUY3hn5dNJFqTXvHa/SUhP3KGvaJaV4b1fi2wwDzyIm8bhFTnUresYOk5 j2whgaklJBIUugC1QQ6HyQ9yf+en30WsRKYO3LyW8ReX5RPzsZGvzdgp0 5SJWvlE6+5jr2zf4Ay+e+/CIo4vLXHms8hlce/MztAlwoGMTNeqowluFK g==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616509" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616509" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228136" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228136" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:14 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 4/7] sched/fair: Introduce sched_smt_siblings_idle() Date: Tue, 22 Nov 2022 12:35:29 -0800 Message-Id: <20221122203532.15013-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229618591661866?= X-GMAIL-MSGID: =?utf-8?q?1750229618591661866?= Architectures that implement arch_asym_cpu_priority() may need to know the idle state of the SMT siblings of a CPU. The scheduler has this information and functionality. Expose it. Move the existing functionality outside of the NUMA code. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- include/linux/sched.h | 2 ++ kernel/sched/fair.c | 39 ++++++++++++++++++++++----------------- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index ffb6eb55cd13..0d01c64ac737 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2426,4 +2426,6 @@ static inline void sched_core_fork(struct task_struct *p) { } extern void sched_set_stop_task(int cpu, struct task_struct *stop); +extern bool sched_smt_siblings_idle(int cpu); + #endif diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0e4251f83807..9517c48df50e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1052,6 +1052,28 @@ update_stats_curr_start(struct cfs_rq *cfs_rq, struct sched_entity *se) * Scheduling class queueing methods: */ +static inline bool is_core_idle(int cpu) +{ +#ifdef CONFIG_SCHED_SMT + int sibling; + + for_each_cpu(sibling, cpu_smt_mask(cpu)) { + if (cpu == sibling) + continue; + + if (!idle_cpu(sibling)) + return false; + } +#endif + + return true; +} + +bool sched_smt_siblings_idle(int cpu) +{ + return is_core_idle(cpu); +} + #ifdef CONFIG_NUMA #define NUMA_IMBALANCE_MIN 2 @@ -1691,23 +1713,6 @@ struct numa_stats { int idle_cpu; }; -static inline bool is_core_idle(int cpu) -{ -#ifdef CONFIG_SCHED_SMT - int sibling; - - for_each_cpu(sibling, cpu_smt_mask(cpu)) { - if (cpu == sibling) - continue; - - if (!idle_cpu(sibling)) - return false; - } -#endif - - return true; -} - struct task_numa_env { struct task_struct *p; From patchwork Tue Nov 22 20:35:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24550 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2425153wrr; Tue, 22 Nov 2022 12:30:48 -0800 (PST) X-Google-Smtp-Source: AA0mqf75bCnO7+0zM9FFiFeI3nS7WbCQEoFCXuMLlZOTOhia5+Z50BZ9ju0jGKToWfe/j2XlG1hk X-Received: by 2002:a17:906:c40d:b0:7af:1534:73c9 with SMTP id u13-20020a170906c40d00b007af153473c9mr5005708ejz.558.1669149047891; Tue, 22 Nov 2022 12:30:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149047; cv=none; d=google.com; s=arc-20160816; b=ICBVTJfTkurAdUwguKZlWjXLKYhOK62NxTJY647UfegIUPIxbWrjTiOCI4iT3G9CLq g9kRzt7HqrdMiU1s1DBrIBAUTJyqc1KhhViOH++f+Czh4K3kGT7/Ao6EFvasx0XwXeWj cRvWL6Guz+cK145d/Xvc+9+sDMUOaO8ch6odnkst+rtZ9bBGvfeVyD5MsMLEnhCyXaDW Wx9c97yYos+9MQmouJvB7gL9nbzTR3cvfdQXlwenXjjZLEbE9VQkGUC8eCzokTkE4ZmC IuN8NYvfqFg0xkWtXNyigMOkhNkMXcl5k2FBoEunu1EPplzepaRDrmPAvZKHY6mEW3AC VUAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=MmqyR15v2ZN53QVWLOcGu+qn2Cn4MiE8w7DDSiR+oT0=; b=agayLeWzEWh8ipZklgfSMC6CittNtPmqv/ANSJHT6RSZenefPNYsWl56DkprQw/azh MlXqt58Dehy78jP9Ue+PbxdjBhkx0bN+B/rFDvwqnHzXBMYrAAyGeOW3XptXCEIeBWEm uBJjSFeiVdWnaHEpt6MQ0qSPsk/GdpdVhklCwHoyUw8RigHjYT5Q25iRP56miogJFlJ7 1VtbC67DdttgJ5PQWHw4itbuQZ8UYuMp+byuB2PPfe0gZKSjvsmCs3Z/oBVhyOA6cPuM yn4PquqXABH9XZaclm5c2vTcf9eMDP77YCrj3E5G6wDG0TmyjdqtrJonAlvpHsyfdfyb 2UUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ezwAMMGq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c17-20020aa7c751000000b0046329e2724dsi11267264eds.86.2022.11.22.12.30.23; Tue, 22 Nov 2022 12:30:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ezwAMMGq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234889AbiKVU2l (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234470AbiKVU2P (ORCPT ); Tue, 22 Nov 2022 15:28:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5576A17055 for ; Tue, 22 Nov 2022 12:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148895; x=1700684895; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=vXIaD/zsIWPHn5n3olE8jagfMylwb+X/0KOXnCS5E6Q=; b=ezwAMMGqxlaicR1khS+T/6eMe09a2oG4+ubhca8mbuxeAjebbBBQvccE yzAEvaI1egytNfUP8SgWJBb7VEL8ioI+Tz1KLvCUnlhHstuikCyyrzj7P ziiLnTyZ2qqndpeHd7fW90As0qVFZKL9ayyWp6s4rnQfvVsYS1GPU1ScP tMKKJds2SCMB58oLW3toHRRzpxH9YwcTbI2jGCtxXd+kKO5ZxvH/NwzdI NZgL9e+EEUdUUislcc9xH6c+EExe3QlcVv07D3Lz3K2YFBGJ0WonXyCXs 8Neit6x8iKNal3IW2uBcWWK2VMF6J5l0k5wIZVsj/BwshL8RHETA7PZA1 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616511" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616511" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228141" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228141" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:14 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 5/7] x86/sched: Remove SD_ASYM_PACKING from the "SMT" domain Date: Tue, 22 Nov 2022 12:35:30 -0800 Message-Id: <20221122203532.15013-6-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229632240157968?= X-GMAIL-MSGID: =?utf-8?q?1750229632240157968?= There is no difference between any of the SMT siblings of a physical core. asym_packing load balancing is not needed among siblings. When balancing load among physical cores, the scheduler now considers the state of the siblings when checking the priority of a CPU. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- arch/x86/kernel/smpboot.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 3f3ea0287f69..c3de98224cb4 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -545,7 +545,7 @@ static int x86_core_flags(void) #ifdef CONFIG_SCHED_SMT static int x86_smt_flags(void) { - return cpu_smt_flags() | x86_sched_itmt_flags(); + return cpu_smt_flags(); } #endif #ifdef CONFIG_SCHED_CLUSTER From patchwork Tue Nov 22 20:35:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24551 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2425292wrr; Tue, 22 Nov 2022 12:31:09 -0800 (PST) X-Google-Smtp-Source: AA0mqf4K/aJIVwki5AoAcjPZLauX8oKkaO0CocwKi2FTUkhyg47mlfIrK08axXQR6cfazGn5Za+O X-Received: by 2002:a17:906:b250:b0:7b2:86d5:8b14 with SMTP id ce16-20020a170906b25000b007b286d58b14mr21512698ejb.230.1669149069009; Tue, 22 Nov 2022 12:31:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149069; cv=none; d=google.com; s=arc-20160816; b=OlchyDqLKzS7d0hOtA/mX+0A87qdohP6TWvsnI8sa32Pz9vnZFhtrTdlJBnZQj5Xor u013elNa8iQqNs6p8aPPOeOhvxZvaALMtKRCKziN/AzaGL53EGEG+tq1hRHRa+fAR2gf 96tqjNc3G1JVy/Tx8ljS6eFzv9s6By5Rd/GQyOOVPsrbj0Ir7B/9v46eyHeMqE4IyI3d 36yv1Ffe6QxF3poarBlHw4t+e078sPk7ixANzFFExfXVNpTnriOYIQdIQIrdkpJUa75p KlEsJDQyMJUrAcNX6SD0Lk13gkfZjXNBa86bemSzJJNVmLkoDMS5oLSO1oWIz49iiCbl GYLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=O2VA3TrA3wMitC3cActLADpuw7WbICZ5lqU3DMkyj2c=; b=BSdR/58M87gjysdtvMyUiTULG0nPOWEZ26CL0uVq0TGAQlQFBIKvfsj+xD1E8pbfFq Ty4rjoY3btb0V9of3JX1xOqukcGHiF8Qv5iWpoGoEuQFnp394D2npN3X9KQVwreqgEYS IX9m2Yu2G/whCXCBR5O2FwNmVPD3MiBpuqtl5tGdVj707LuL8j7D1ytPHCGtD+TJX6W9 x4xEAN7Mc2w8Rz6UpqeOOWTE++HSScIi2OdllqI5nGo9rvedKHkiBdWD5wlX4JdesVkg l81dTjZtbVSzxPpVhf3Jr/J+nwLzhj0MIXs4azEeD8LbcvZZi6VcrEMwPjkZvrZR2kth PNzw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UsM+efA2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y15-20020a056402270f00b0045d4a88c735si13382561edd.220.2022.11.22.12.30.45; Tue, 22 Nov 2022 12:31:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UsM+efA2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234865AbiKVU2p (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234528AbiKVU2Q (ORCPT ); Tue, 22 Nov 2022 15:28:16 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDEBE14D07 for ; Tue, 22 Nov 2022 12:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148895; x=1700684895; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=bhiwDeK4ICYY9tIKImH8FX+AqBh+ctstEwbC4wBb3kI=; b=UsM+efA2znobHYPGHWvHjZfJetf7YQSNUD2EqNJq44+DhN4OJsRSU38t q7u0pB+hGpumF4AJMOtyMZoB5Nm0FhTY2GaVyNaod7DK1RI7bLcWk1n06 nFpRgTJTEIUi2jzsS3X8tL6t1MWWzCVx3Ol5ujOGZTINcWU/YAPaj79dy irEktVOVkX2qEyAbRV8G+f+dgVIsosVSQ7kNqg7h1/cyBduRPJKzXJLuY PD/PFfdIivrjP0tXRoP/WLfFskbBfTAIUmHsLeNv74GVQAB7IlMwt3DMg dyWLRouVx+RVBAEI8rqCLvTCZiOdduA8oGUr6A/bX3H/UkyUZRjLQLsra w==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616514" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616514" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228144" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228144" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:15 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 6/7] x86/sched/itmt: Give all SMT siblings of a core the same priority Date: Tue, 22 Nov 2022 12:35:31 -0800 Message-Id: <20221122203532.15013-7-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229653987626546?= X-GMAIL-MSGID: =?utf-8?q?1750229653987626546?= Currently, each SMT sibling is given a priority that ensures that in partially busy systems load is evenly spread among cores. There is, however, no difference in performance among the SMT siblings of a core. Having different priorities for each SMT sibling triggers unnecessary load balancing towards the higher-priority sibling. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Len Brown Signed-off-by: Ricardo Neri --- Changes since v1: * Reworded commit message for clarity. --- arch/x86/kernel/itmt.c | 23 +++++------------------ 1 file changed, 5 insertions(+), 18 deletions(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 4cb5a5e4fa47..2be49ce4f94a 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -174,32 +174,19 @@ int arch_asym_cpu_priority(int cpu, bool check_smt) /** * sched_set_itmt_core_prio() - Set CPU priority based on ITMT - * @prio: Priority of cpu core - * @core_cpu: The cpu number associated with the core + * @prio: Priority of @cpu + * @cpu: The CPU number * * The pstate driver will find out the max boost frequency * and call this function to set a priority proportional - * to the max boost frequency. CPU with higher boost + * to the max boost frequency. CPUs with higher boost * frequency will receive higher priority. * * No need to rebuild sched domain after updating * the CPU priorities. The sched domains have no * dependency on CPU priorities. */ -void sched_set_itmt_core_prio(int prio, int core_cpu) +void sched_set_itmt_core_prio(int prio, int cpu) { - int cpu, i = 1; - - for_each_cpu(cpu, topology_sibling_cpumask(core_cpu)) { - int smt_prio; - - /* - * Ensure that the siblings are moved to the end - * of the priority chain and only used when - * all other high priority cpus are out of capacity. - */ - smt_prio = prio * smp_num_siblings / (i * i); - per_cpu(sched_core_priority, cpu) = smt_prio; - i++; - } + per_cpu(sched_core_priority, cpu) = prio; } From patchwork Tue Nov 22 20:35:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 24552 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2425648wrr; Tue, 22 Nov 2022 12:31:55 -0800 (PST) X-Google-Smtp-Source: AA0mqf6XcFD0oxKQRhk3wwO02AnwUE5+E79v+c9SGzXrwUwPRNAJmBdPqxiBTszx1f7gModArfeT X-Received: by 2002:a17:906:c40d:b0:7af:1534:73c9 with SMTP id u13-20020a170906c40d00b007af153473c9mr5008889ejz.558.1669149115291; Tue, 22 Nov 2022 12:31:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149115; cv=none; d=google.com; s=arc-20160816; b=bT4N9zgTLM3uWWK+8r1M2X0Pf5de1x6iQWSTtjpbyVKPBLg2oBe6zVTJ/tPi+EM1ip ZRAWk6MfyiQLeDh1rwfgmsf07P4xi7yq9DFOUlAKxeHS05tAwUH0YDwyNkP1eNLIaEhd J+2EEHQpuuCJTmmfnEHYEvcZYdewZ+G2iRFlwZ9FJkFf4/09levsNbRyWHhDGCy8/MZ0 TM7XX3v/ip9RMehaEC7fXVZAJ93kZ48h5FPf8THMqIP5XE0XplYLyMit38o+dAM4KUs8 pRTlrBv7UxTg48hSSttPosG/JuGjKHRaHnHt6er95XRIOdVwIU09fo8230qXaoDnIoWe tOMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=4DUKlbW0j5sTgrSqzx8AqNnWpJYrLD/1IZEdUnnGrHc=; b=o3lRHfPbMkZQUhZEMftEtVRgU0E2oNz44wgl2kpmtc2QruYec9v3mVvEExjGNZuuaT d0C55YE++faaQwHzZ3G4uQoZ0csDx85NFYOWkoOCwFRkG2WFGF+z8W2ZPEP1fHVusrku phw+vn3KgZBhTmsyHVhtgwr6AJ04EsgHXq5DglN8obCBPfhdLN55a9yCnWj+tk1yy5ae qoo7wVH2+Z0IEMyuZQYkLIcrQz5idnRAU7S08eM8M+TpED8UW+AMeRnTbEU2R7BW/f/V IFt4xHIGr6XA926R4kel09Dei7A2vv087eAvSfzVNo9hNH5jqF5ib6xUBe3i3J3C08su rrhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=NKkxiZ5F; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g14-20020a1709065d0e00b007ae9abf1990si14607889ejt.352.2022.11.22.12.31.31; Tue, 22 Nov 2022 12:31:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=NKkxiZ5F; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232734AbiKVU2t (ORCPT + 99 others); Tue, 22 Nov 2022 15:28:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234546AbiKVU2R (ORCPT ); Tue, 22 Nov 2022 15:28:17 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DAD515805 for ; Tue, 22 Nov 2022 12:28:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669148896; x=1700684896; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=jiYnvM57QdQau/H42shFzT9M5SGTODsNBjdPxNUJhmI=; b=NKkxiZ5Fji9Mkd6TVOXWtDrKNQtE8xJFI7uZ6aSxJFzMgHhILdr0ssmV r5ep01bB53BDuAEVNBvek40Lsgc/1t53rkw4UyJay949WgIc9kRau3h2/ yAPe2fA3H8F1eNzGMTb3v6q0i/PtPLYeqhX3EKw0azbWMt9lakzI2JFP+ 0za1zHOj8Qb/eQwJJumNo3w7VaqG3koknHzmLvOhp7uvbnY+u/awOkiAW cblmNU1rCO7pJAhb//e/iwl17pqffbyr4gghdJ0mWabcLNr88942yt6kS zOr3zB5oeAV8TCcZwTAW7KnTL/CN3WSYIRoqj0teW/MWe3NfCI+YHJuoI w==; X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="293616515" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="293616515" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Nov 2022 12:28:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10539"; a="816228147" X-IronPort-AV: E=Sophos;i="5.96,185,1665471600"; d="scan'208";a="816228147" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 22 Nov 2022 12:28:15 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 7/7] x86/sched/itmt: Consider the idle state of SMT siblings Date: Tue, 22 Nov 2022 12:35:32 -0800 Message-Id: <20221122203532.15013-8-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> References: <20221122203532.15013-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750229702463060866?= X-GMAIL-MSGID: =?utf-8?q?1750229702463060866?= Cores with more than one busy SMT sibling need help from fully idle cores, even if they have lower priority. Return a priority of 0 if any of the SMT siblings of a CPU are busy. This lets lower-priority idle cores pull tasks from the CPU. Only do this if the scheduler cares about the idle state of the siblings. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Introduced this patch. --- arch/x86/kernel/itmt.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 2be49ce4f94a..cb22944969a1 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -169,7 +169,10 @@ void sched_clear_itmt_support(void) int arch_asym_cpu_priority(int cpu, bool check_smt) { - return per_cpu(sched_core_priority, cpu); + if (!check_smt || sched_smt_siblings_idle(cpu)) + return per_cpu(sched_core_priority, cpu); + + return 0; } /**