From patchwork Tue Feb 7 04:58:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53658 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649117wrn; Mon, 6 Feb 2023 20:52:06 -0800 (PST) X-Google-Smtp-Source: AK7set96aXov4c01GAaJtqoLj1efGo8dvnUciakVFVTuE1IapBcmXJbkjyQ2WGiPu/maM/W9U5jp X-Received: by 2002:a05:6a20:8e29:b0:be:972a:f5e9 with SMTP id y41-20020a056a208e2900b000be972af5e9mr2275888pzj.34.1675745526669; Mon, 06 Feb 2023 20:52:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745526; cv=none; d=google.com; s=arc-20160816; b=dRiQSsKvR265zzUp3hsV3XPJFcB/AHgHqY79AWJp2g4qxAmluN52LzQk6mhT6drDIC 6mg+DE/T7jcimXLCBcNa+3A5JsEdy6cFTZGxN0nPLLFaI+AVCOx1gqK9k026DYhvYCSP XMRaXmR1kkS6Hlz3Q0I1skdzL5PhUBMvjc+Xf735tOnFfbyQcMb6pbuy+wIhcxjNjoJ3 1/DCtRff8Izr7C2BpsaFXDlQ4O1w07j0TmQtz+6Gol0PE2JU3YuzIN1lv64Y8sgSJy4i UG9u4qng32ftEv83Ma9oIMgyxNBC48igbtDoghhwUKl+Xt0JbUDBXacyb7aqmnvRvoT/ zK4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=jhRAiUPCTSX99Thbijw92BjhRF7IZb6ZJjQHcTrw2Hs=; b=y8KwV71MPx+l+st90at0d4r88NY14M1PIEUaXJhdYkoQxH8uz7zwK7PJxmacjFkaNw rm5VmB7MNltLVUuwY9kxUL+yqUl0uARwEgRpYGUqT/mjJIDbPZpyfBZvBSCWAK7uW88H dLfg7x1eva+bRYJvkN77fzkxqu5KbCxa08XmrcQoVJPnKDtwg4mkh4L9PdC5OE7zaHbt LUBuBbu8SPOvGFJwMNUnvxfGIdP4jnnuQfotCeDtVhKxeSgIWof+riwiu3S9hhQkrlFN SlQObdkIzRO5f+onVyrknTBNuHsBGHrtwg1/yX2ZvZi+q40Lwj+13jodDIJFyQbBtrTc ksSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YHminuoJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b12-20020a621b0c000000b00593df8c5449si12903064pfb.381.2023.02.06.20.51.54; Mon, 06 Feb 2023 20:52:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YHminuoJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229755AbjBGEux (ORCPT + 99 others); Mon, 6 Feb 2023 23:50:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229670AbjBGEus (ORCPT ); Mon, 6 Feb 2023 23:50:48 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C793E4224 for ; Mon, 6 Feb 2023 20:50:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745446; x=1707281446; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=NHwRQom2Hnt0ENynqej3UWEaK2X2qGGXakLIKRH9HWo=; b=YHminuoJPXwQ006sZy5NSdn0kTaGQqHBJjCzInTDnixDyFOXIY4jurnE Y4hNRN7xYCRpigLCpuNTWbpyrFnggAf5nXVsHzpKvs9O4RV3kIeSoCmL7 tK9RHlD9zhUpvSFj5WuvWahdlK1VIW2zTPqBEZN9c8o/SnLWOdoxZ1mPq US4O5kxrtcgDUKcr+NgSPMiuMEtCMtyW5z8hIAS5DZ+PEzhVdZF7ABuNT 6liUYq3SoDosbkQfHTIfEXcFa5EZK4tp6CJIwtDQp2TR1dHqrGPXdJ/If KfEWQ0dx2KhOm73KrgkNXtLmtGp3rbloa1RG4Gep/oLjYJpT0d54xrR+R w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415623976" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415623976" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653779" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653779" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:45 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 01/10] sched/fair: Generalize asym_packing logic for SMT cores Date: Mon, 6 Feb 2023 20:58:29 -0800 Message-Id: <20230207045838.11243-2-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146541241928752?= X-GMAIL-MSGID: =?utf-8?q?1757146541241928752?= When doing asym_packing load balancing between cores, all we care is that the destination core is fully idle (including SMT siblings, if any) and that the busiest candidate scheduling group has exactly one busy CPU. It is irrelevant whether the candidate busiest core is non-SMT, SMT2, SMT4, SMT8, etc. Do not handle the candidate busiest non-SMT vs SMT cases separately. Simply do the two checks described above. Let find_busiest_group() handle bigger imbalances in the number of idle CPUs. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Len Brown Signed-off-by: Ricardo Neri --- Changes since v2: * Updated documentation of the function to reflect the new behavior. (Dietmar) Changes since v1: * Reworded commit message and inline comments for clarity. * Stated that this changeset does not impact SMT4 or SMT8. --- kernel/sched/fair.c | 41 ++++++++++++++--------------------------- 1 file changed, 14 insertions(+), 27 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7c46485d65d7..df46e06c9a3e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9254,13 +9254,11 @@ group_type group_classify(unsigned int imbalance_pct, * the SMT siblings of @sg are busy. If only one CPU in @sg is busy, pull tasks * only if @dst_cpu has higher priority. * - * If both @dst_cpu and @sg have SMT siblings, and @sg has exactly one more - * busy CPU than @sds::local, let @dst_cpu pull tasks if it has higher priority. - * Bigger imbalances in the number of busy CPUs will be dealt with in - * update_sd_pick_busiest(). - * - * If @sg does not have SMT siblings, only pull tasks if all of the SMT siblings - * of @dst_cpu are idle and @sg has lower priority. + * If @dst_cpu has SMT siblings, check if there are no running tasks in + * @sds::local. In such case, decide based on the priority of @sg. Do it only + * if @sg has exactly one busy CPU (i.e., one more than @sds::local). Bigger + * imbalances in the number of busy CPUs will be dealt with in + * find_busiest_group(). * * Return: true if @dst_cpu can pull tasks, false otherwise. */ @@ -9269,12 +9267,10 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, struct sched_group *sg) { #ifdef CONFIG_SCHED_SMT - bool local_is_smt, sg_is_smt; + bool local_is_smt; int sg_busy_cpus; local_is_smt = sds->local->flags & SD_SHARE_CPUCAPACITY; - sg_is_smt = sg->flags & SD_SHARE_CPUCAPACITY; - sg_busy_cpus = sgs->group_weight - sgs->idle_cpus; if (!local_is_smt) { @@ -9295,25 +9291,16 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); } - /* @dst_cpu has SMT siblings. */ - - if (sg_is_smt) { - int local_busy_cpus = sds->local->group_weight - - sds->local_stat.idle_cpus; - int busy_cpus_delta = sg_busy_cpus - local_busy_cpus; - - if (busy_cpus_delta == 1) - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); - - return false; - } - /* - * @sg does not have SMT siblings. Ensure that @sds::local does not end - * up with more than one busy SMT sibling and only pull tasks if there - * are not busy CPUs (i.e., no CPU has running tasks). + * @dst_cpu has SMT siblings. Do asym_packing load balancing only if + * all its siblings are idle (moving tasks between physical cores in + * which some SMT siblings are busy results in the same throughput). + * + * If the difference in the number of busy CPUs is two or more, let + * find_busiest_group() take care of it. We only care if @sg has + * exactly one busy CPU. This covers SMT and non-SMT sched groups. */ - if (!sds->local_stat.sum_nr_running) + if (sg_busy_cpus == 1 && !sds->local_stat.sum_nr_running) return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); return false; From patchwork Tue Feb 7 04:58:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53659 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649133wrn; Mon, 6 Feb 2023 20:52:10 -0800 (PST) X-Google-Smtp-Source: AK7set+dLHVQz3MqL/b+h3fOjDVLBwxUytvVuOEynHMHvtXIDzh9S3IyR/6dF9MjOTIIaBlgjQY1 X-Received: by 2002:a05:6a20:3d03:b0:be:8e8d:330c with SMTP id y3-20020a056a203d0300b000be8e8d330cmr2447033pzi.27.1675745530074; Mon, 06 Feb 2023 20:52:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745530; cv=none; d=google.com; s=arc-20160816; b=JtyVDfVdC8T5b/lS1DsvIzCqBD4XdicPsNMeeBZCYZXdoe9CUPeL9UUspPbei0V3nd MfHLr/P2zxwFJh6UXTrbp42LDVwwoN7YB9rGpbnkyZ6WxCuEGVKRZ93U2bO5vu83O+bJ ed976Rv1KYOBy4hgidzG4qnhTrhFNrnH7371lQlxRHGbCg9gTsOcqPy4Du4eRWX6kndG 87i/5gf07NnzEk/fNyZf6mHOZq9i4JZ55r+QUwlTku6IV/HjUnpeNSMfEVMZ0UoikYwi wqxJZK3IRS+hn8AeE+Gpfn4L4p+E/p4MNkT8EGvQP+nHiYpqJCwYGQU+zFvJrAw5afdp HvWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=IWL+dejvclVrWaOwRIQW+7T4EUTsGhayFHO/8YjWK6A=; b=hCrbsXWWLVnIkKeOaS0xR9V54t5wg9bbM8gMGfxulOzy+NaXOleQrJE3iDG7fLpkdT YkCp6F7HjiGp5wGaCgszOribcN3olsU6lobBZQcYjinwYZpNEYx+9ywtJBC8FHt4dnku /cBdCDKHu8zL+bZMJktkKIHoSLj8XbpaKu5eKrEVIDqEIwI1NL6/eXZJCoJifcMlqF5N YYGa/KeCfZXTC0IgjV1hBk+mcz8LnOUY+DooMCe+bZhXqXrP2vVUhB09/27fdlKn5t1B +1omSCHXxaqbgFZnCEYkML6deaiGK1Ou+EH2ZgS7sL7rHZSvC8Z6oGJATOct0XCMbVBV AGRw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=m+QHtUxy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t71-20020a63814a000000b004f2d18b40dbsi14068746pgd.79.2023.02.06.20.51.57; Mon, 06 Feb 2023 20:52:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=m+QHtUxy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229944AbjBGEu5 (ORCPT + 99 others); Mon, 6 Feb 2023 23:50:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229625AbjBGEus (ORCPT ); Mon, 6 Feb 2023 23:50:48 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4697E4236 for ; Mon, 6 Feb 2023 20:50:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745447; x=1707281447; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=fFIXmz2jRoZi+M1l2J6uT/aihpD4LyOEaKM5OhhEq7Y=; b=m+QHtUxyPZrmE6PrVxYqy+114rJq8NQRahcbGxK1WkKLm6C3qj4XncPd CfQeKVLWC+z/pB2xZDJtWnQty11GBvC50D5dvgIMQKFrR/HCPBJvaAG97 rPVSTgWrTP5h+3nO1zRxUgBUg3sZp5feiJrgSy9b9yB1pU2ouMs+ush6m eGaPmsbxz1r4xmP9obo10xQjvwNl7pp9AQjjx6Lm+ODJZOcNvGlHphXyJ Q7RE4XyY5mSbEImWSOEG1kgTuWgAAq08FqjVp2dS9MB9zwrj71Lqlp69B VoFeokdbEMsYUdd3Y1YsfNzTVb6AXDbh8N3aFBTWZDnLnp+S1dLG9mnVs w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415623986" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415623986" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653782" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653782" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:45 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 02/10] sched/fair: Move is_core_idle() out of CONFIG_NUMA Date: Mon, 6 Feb 2023 20:58:30 -0800 Message-Id: <20230207045838.11243-3-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146545060686163?= X-GMAIL-MSGID: =?utf-8?q?1757146545060686163?= asym_packing needs this function to determine whether an SMT core is a suitable destination for load balancing. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index df46e06c9a3e..767bec7789ac 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1064,6 +1064,23 @@ update_stats_curr_start(struct cfs_rq *cfs_rq, struct sched_entity *se) * Scheduling class queueing methods: */ +static inline bool is_core_idle(int cpu) +{ +#ifdef CONFIG_SCHED_SMT + int sibling; + + for_each_cpu(sibling, cpu_smt_mask(cpu)) { + if (cpu == sibling) + continue; + + if (!idle_cpu(sibling)) + return false; + } +#endif + + return true; +} + #ifdef CONFIG_NUMA #define NUMA_IMBALANCE_MIN 2 @@ -1700,23 +1717,6 @@ struct numa_stats { int idle_cpu; }; -static inline bool is_core_idle(int cpu) -{ -#ifdef CONFIG_SCHED_SMT - int sibling; - - for_each_cpu(sibling, cpu_smt_mask(cpu)) { - if (cpu == sibling) - continue; - - if (!idle_cpu(sibling)) - return false; - } -#endif - - return true; -} - struct task_numa_env { struct task_struct *p; From patchwork Tue Feb 7 04:58:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53661 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649186wrn; Mon, 6 Feb 2023 20:52:22 -0800 (PST) X-Google-Smtp-Source: AK7set+jgh+mFUZx8gJuWQHy07dx3Y/Cu3na3mLMmLYJfMIot4PcpJlCo12ODi24eLQw4bIQg1kZ X-Received: by 2002:a17:90b:1e45:b0:230:c467:1c7e with SMTP id pi5-20020a17090b1e4500b00230c4671c7emr2569081pjb.11.1675745542170; Mon, 06 Feb 2023 20:52:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745542; cv=none; d=google.com; s=arc-20160816; b=AF1VRoHSDDOm4L63W/L6zK3G9f5/qAe/bqQ4BGRm0M5URamVn6lPFmqsyvymPxEcE5 6okMWsPr5OOxvM99wbQ3B/D2zJJc9W6woO2CvoFDx8Uo6ynf5wX5OeEqi1iBhlD6FhMa 1Enbn6HuUoDAWwYztJg9gpVnlB5Y2hIN7t5VtIkFcClqzrQ0uf2segAkDGr9HYj+U+AT QDhORLojn/SQamgkzvsgebptMM/7Wt9a49f3AXuYJO9DdZQhmZhhfP3/EwHbuz4Hh575 npioZpUTMg2XfyndJt5C7PPo1iBNopu3cofRF5UgUvj02vGkXw/09IW7uQazkWiMpwB8 a4Sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=x6fNv3OZZ2F5Z9PN3mPLVbgfmnXZWwzDX/SbzZTleBU=; b=bL0l3NiodkVmv+yiQnbT/nLTtYDPRfT76LAMHKFECemBGEevrYWPQuSW/H6td8O8EC F9iB1JlBgtW/kYmAad77EuROgS1j4jhnL6m/25cYf73m/vG96KPxhpTR/g3APcIJGRwI tsNmS4/61hZn9TfSkkLjv7QmBaU/v754FkT5H35QnuNMyCV7Qql+trEckFfIcBhztTdp RmuvQLdEpMpWh4aYu7sQc+LFKNswr3ZbDNEOaAsCQs48d0YJ3jRtaCQR/jEcI3K1ngBN /WP+gZ8pmMArm3cdkocT80/K27UtObVFB7Fo5YWURu4i9eKNEBmKCaMbrRECftADM90n kYqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YHYjq52Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 99-20020a17090a0fec00b00230d6230ffasi2561369pjz.119.2023.02.06.20.52.09; Mon, 06 Feb 2023 20:52:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YHYjq52Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229667AbjBGEvE (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229828AbjBGEus (ORCPT ); Mon, 6 Feb 2023 23:50:48 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0204FD3 for ; Mon, 6 Feb 2023 20:50:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745447; x=1707281447; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=W6TuYGPZ71rk8F3QYL8M9R3AwD+pJaLK+sDkNMpMUBo=; b=YHYjq52ZX3LEnC+1eAoCUQfYaa7KZQeKg31l1h2r1WsbCvUzmyu8FQgO jm5KM5ASexmN+yucnn2vwZ1lonHeU/QnE8RVj+P2VI1iOVDxaF53ZkQWZ CXTqo/EuYreSuFxtEF8kCkBGdZNjlwzdudl3Ia8E/dxYBMi0keigkB5mJ h47mU/F8HG1yXLadpay4x+LqPF16LoXZ4rQALcLDD4Wf5klRA8+kqpDdl A06SFIOTK3spdfkvS6oLB/gzIhlYtBILT8CH3vWxnkGnU5znDmPwwmt1L 6LiIv5Kr9yPgVqyT1KNwwATi6rsEN4itUNY9dGx+NjELaq1UW/upOgpYM A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415623995" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415623995" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653786" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653786" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 03/10] sched/fair: Only do asym_packing load balancing from fully idle SMT cores Date: Mon, 6 Feb 2023 20:58:31 -0800 Message-Id: <20230207045838.11243-4-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146557649138404?= X-GMAIL-MSGID: =?utf-8?q?1757146557649138404?= When balancing load between cores, all the SMT siblings of the destination CPU, if any, must be idle. Otherwise, pulling new tasks degrades the throughput of the busy SMT siblings. The overall throughput of the system remains the same. When balancing load within an SMT core this consideration is not relevant relevant. Follow the priorities that hardware indicates. Using is_core_idle() renders checking !sds->local_stat.sum_nr_running redundant. Remove it. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Valentin Schneider Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 34 +++++++++++++++++++++++++--------- 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 767bec7789ac..80c86462c6f6 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9250,12 +9250,14 @@ group_type group_classify(unsigned int imbalance_pct, * Check the state of the SMT siblings of both @sds::local and @sg and decide * if @dst_cpu can pull tasks. * + * This function must be called only if all the SMT siblings of @dst_cpu are + * idle, if any. + * * If @dst_cpu does not have SMT siblings, it can pull tasks if two or more of * the SMT siblings of @sg are busy. If only one CPU in @sg is busy, pull tasks * only if @dst_cpu has higher priority. * - * If @dst_cpu has SMT siblings, check if there are no running tasks in - * @sds::local. In such case, decide based on the priority of @sg. Do it only + * If @dst_cpu has SMT siblings, decide based on the priority of @sg. Do it only * if @sg has exactly one busy CPU (i.e., one more than @sds::local). Bigger * imbalances in the number of busy CPUs will be dealt with in * find_busiest_group(). @@ -9292,15 +9294,13 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, } /* - * @dst_cpu has SMT siblings. Do asym_packing load balancing only if - * all its siblings are idle (moving tasks between physical cores in - * which some SMT siblings are busy results in the same throughput). + * @dst_cpu has SMT siblings and are also idle. * * If the difference in the number of busy CPUs is two or more, let * find_busiest_group() take care of it. We only care if @sg has * exactly one busy CPU. This covers SMT and non-SMT sched groups. */ - if (sg_busy_cpus == 1 && !sds->local_stat.sum_nr_running) + if (sg_busy_cpus == 1) return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); return false; @@ -9314,7 +9314,14 @@ static inline bool sched_asym(struct lb_env *env, struct sd_lb_stats *sds, struct sg_lb_stats *sgs, struct sched_group *group) { - /* Only do SMT checks if either local or candidate have SMT siblings */ + /* + * If the destination CPU has SMT siblings, env->idle != CPU_NOT_IDLE + * is not sufficient. We need to make sure the whole core is idle. + */ + if (sds->local->flags & SD_SHARE_CPUCAPACITY && !is_core_idle(env->dst_cpu)) + return false; + + /* Only do SMT checks if either local or candidate have SMT siblings. */ if ((sds->local->flags & SD_SHARE_CPUCAPACITY) || (group->flags & SD_SHARE_CPUCAPACITY)) return asym_smt_can_pull_tasks(env->dst_cpu, sds, sgs, group); @@ -11261,8 +11268,17 @@ static void nohz_balancer_kick(struct rq *rq) */ for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { if (sched_asym_prefer(i, cpu)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + /* + * Always do ASYM_PACKING balance in the SMT + * domain. In upper domains, the core must be + * fully idle. + */ + if (sd->flags & SD_SHARE_CPUCAPACITY || + (!(sd->flags & SD_SHARE_CPUCAPACITY) && + is_core_idle(i))) { + flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto unlock; + } } } } From patchwork Tue Feb 7 04:58:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53660 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649184wrn; Mon, 6 Feb 2023 20:52:20 -0800 (PST) X-Google-Smtp-Source: AK7set/8MCLLVMCtUVSjoGsZBT6kVoCY+TsIQmXFTuPnElc51pBoT45HNze3vXT0/ed3Ufxaa/nq X-Received: by 2002:a05:6a20:a01d:b0:bb:c422:809f with SMTP id p29-20020a056a20a01d00b000bbc422809fmr2288307pzj.4.1675745540350; Mon, 06 Feb 2023 20:52:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745540; cv=none; d=google.com; s=arc-20160816; b=XYmHj4NKRhOOg4tZs10oNycMuGTjojFRUJ8076Tw05iE74Eo9oaWRsN1Qd4SI+NiD9 7tRqY7hZuNBuH3H64hI+MHVKAJavojyXCkS/tO+/z0gBEe62i/lN3TTrdkIFS30V9sUp WH79AjNfmXX8L4ucTbGsfNpvBxmCL5qTuD51rUfp2QKmFxwUDGCgxLwRuMkeNTGlsVNp lFAnC+Urv+b0vVGnVvgUujpx9TuiZ1PcbWIybtLxT7834ciPbyN15biD2UJWj4OVzVcx IIlZ54LSD574PjLFyoEdcBvBzhU+UwTvIRAao5SfNjVG+N9vQuyzNJ0EM11qaChhY6J+ wITw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=pdgOdU5fNY2lTAuA2Zxw82DFQ2kJyOkfJeIV/iI0mYQ=; b=LzZrVbjM7vuOyqb5Y5/JuAQuEQHvsNGuVdnC4lGVRVL5g07oPvPNfgL93zsgAradxO z+BNaZm31YYkMSP0M8d+vvQGXzncpc/1OVSU2+H6DsjRtwj5dN1znHbfUXEZ9E3SkNWg jouG7ry393wA5b+2a8PtBT3tRVh/EzzUA7N+mvU8yEKZm38Gdl0wY7DsBGhGiSHihVOB 458Hrf1GeloyN4q1UY4/0Y4x/CwRu5G8CGDKLmu8KMF9Bzew1zj5lARluW82far53Rmu pOvuDCb0//iX3biDBW2Qz8tsOlGOPjqUtov/9fKx7hLoG5hox6wuzm3elxAZ7OkNqhLs dwSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YDiE51cF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b5-20020a637145000000b004df3908855csi15556227pgn.421.2023.02.06.20.52.07; Mon, 06 Feb 2023 20:52:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YDiE51cF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229994AbjBGEvB (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229832AbjBGEus (ORCPT ); Mon, 6 Feb 2023 23:50:48 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10DF044BC for ; Mon, 6 Feb 2023 20:50:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745448; x=1707281448; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=BhYkRtUf49Glw2ZU0a3Pjs3bki0DXfT9COx0nHL/qCw=; b=YDiE51cFVqLf62JsGMdpJchxCPazPEa9taxciml+UxNWEVPIcA387Y9j UH7SiGW7dUPaLVD4FNTHSz7wcEV4HQwAntBYK/vvsXcTfcy7znMlRqdLz /bAXyt6wxsLMuFoGXXt0Hd8y+XpOHfgUZwgMw8XevWJUGC1QJvjjZ6v0M 0cxLIt21/6FKXhOnMrRjd5cGUsO1mCc85vjjv9F8uAGWpYckXHI07tglT S4OsdxcvJusnDo/fPLWUiPgOhSq9LvSTujYic9YdZw+WbZ0+4p4A59SYW vew3fVbz6tnRXI2Jzn8UDs3pePGt/g+fz9HmBQpit6AuaRNix0yWWlsQ9 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624005" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624005" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653789" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653789" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 04/10] sched/fair: Let low-priority cores help high-priority busy SMT cores Date: Mon, 6 Feb 2023 20:58:32 -0800 Message-Id: <20230207045838.11243-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146555912228703?= X-GMAIL-MSGID: =?utf-8?q?1757146555912228703?= Using asym_packing priorities within an SMT core is straightforward. Just follow the priorities that hardware indicates. When balancing load from an SMT core, also consider the idle of its siblings. Priorities do not reflect that an SMT core divides its throughput among all its busy siblings. They only makes sense when exactly one sibling is busy. Indicate that active balance is needed if the destination CPU has lower priority than the source CPU but the latter has busy SMT siblings. Make find_busiest_queue() not skip higher-priority SMT cores with more than busy sibling. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Valentin Schneider Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 80c86462c6f6..c9d0ddfd11f2 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10436,11 +10436,20 @@ static struct rq *find_busiest_queue(struct lb_env *env, nr_running == 1) continue; - /* Make sure we only pull tasks from a CPU of lower priority */ + /* + * Make sure we only pull tasks from a CPU of lower priority + * when balancing between SMT siblings. + * + * If balancing between cores, let lower priority CPUs help + * SMT cores with more than one busy sibling. + */ if ((env->sd->flags & SD_ASYM_PACKING) && sched_asym_prefer(i, env->dst_cpu) && - nr_running == 1) - continue; + nr_running == 1) { + if (env->sd->flags & SD_SHARE_CPUCAPACITY || + (!(env->sd->flags & SD_SHARE_CPUCAPACITY) && is_core_idle(i))) + continue; + } switch (env->migration_type) { case migrate_load: @@ -10530,8 +10539,20 @@ asym_active_balance(struct lb_env *env) * lower priority CPUs in order to pack all tasks in the * highest priority CPUs. */ - return env->idle != CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING) && - sched_asym_prefer(env->dst_cpu, env->src_cpu); + if (env->idle != CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING)) { + /* Always obey priorities between SMT siblings. */ + if (env->sd->flags & SD_SHARE_CPUCAPACITY) + return sched_asym_prefer(env->dst_cpu, env->src_cpu); + + /* + * A lower priority CPU can help an SMT core with more than one + * busy sibling. + */ + return sched_asym_prefer(env->dst_cpu, env->src_cpu) || + !is_core_idle(env->src_cpu); + } + + return false; } static inline bool From patchwork Tue Feb 7 04:58:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53662 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649265wrn; Mon, 6 Feb 2023 20:52:39 -0800 (PST) X-Google-Smtp-Source: AK7set9GssFpeAJSRHmz0+LYM5z5wEu3rTN2r5+SFC/5ZIhqrVMWdkH5Ue9vX2kUh6rJvlU/n5af X-Received: by 2002:a17:903:32c9:b0:196:82d2:93a with SMTP id i9-20020a17090332c900b0019682d2093amr2187808plr.11.1675745558728; Mon, 06 Feb 2023 20:52:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745558; cv=none; d=google.com; s=arc-20160816; b=EE2JJEpboYia1a+Q+UlNwdR3g4KFdqb2ibqDLFp/noiYt6OTbGcTtq7nFv6mL1LxK8 rn2LrgRppS4y6IIv/FJMx2BJMDQDNrmPlUa83gkZdPFPbNO/M72vlW9yz1mTM0LcuoKE 0HWaXXVLsPbGJ+AiL9ok+P1o8JXhgIqcsj/h+Ul4mfBVRMGPwK4qp7N7HOqZcUFxYO7s PrardwPed4PIczG+FW/PF0sjmJskDNY/iyX8JC2SGkjOejjZxolSoWbJ+sxyZhO7WDUJ SXlEyEbreaIl+bvKmwj+7nU2uMScITxuQKqrfUUamd+ptNPzBGo9s80va9F2/HYifskP 2XIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=OaCKbvvEOY6M1j3D70iNgPeZok4a89QP4HOOehL8fqw=; b=bJRVXPfcnG3PTfGRNQ5mU6kXK115bWOQxK4iJpdC3G+kdfkvLz1zLOJXRkD4szXZ1M 4V8UErPBpy9utgPekhUydg/K54lP3zypuc6t+4IQC4NMDgHXBeWp+wjEvVcWl/wtxV5G 6gWvPafsPf0qYpzwQbqvrqOIsJrWF20BWypYpqun0NXhWdL2DWRR11cF87LSZo2ORclM vSjmt9VXVQVygBjiynW/CjBALhY/oH7XRHxFZkPmb0lGWFpIzBvL4AngRQn3NSvHm3Tj R8J90QVvh5NFJNqcvT5heTQDZ2LFW3JrIDBph4gK+FRaC1E1mg0exCL1dXj35sNNjU/L UpTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SzkE43ST; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bi6-20020a170902bf0600b00199049b687bsi6912813plb.180.2023.02.06.20.52.26; Mon, 06 Feb 2023 20:52:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SzkE43ST; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230060AbjBGEvJ (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229924AbjBGEut (ORCPT ); Mon, 6 Feb 2023 23:50:49 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14E344ECC for ; Mon, 6 Feb 2023 20:50:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745448; x=1707281448; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ulLn873Z56iFQvxm4YOPwOFsRDQ7WudOQQX77xthURc=; b=SzkE43STwpMyQxRVXh1tqpIqgXttsqg5ePJ5eHfmPIEUGXPrrKS4B6fJ 7aQALzxucFatxuMoEd9qzkLhSOKpFVn83NMhDa4C7VrydQoHLMG3Pma4d kik/WQkjKVXSYYmBYdk9TtfuRnXd5w8BRNlSlcml6KQItel0s8YZAbB/J dJbt8dgzFh/i9LrG+g0LlT09oxmCQcTyHOTrXkdJUkSWtCUfRjF1qiuKs vdyKT3yDTCQBqF+rCeoqLJiJKT9AVjJoqqQQWe+bPwfOuedn/ftc9FQ8v LBe3R4fkrT0nIacAadNbrimb5ZYe6hdtGs+G6hJT+REbElVRsNFsdx1Hg w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624014" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624014" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653792" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653792" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 05/10] sched/fair: Keep a fully_busy SMT sched group as busiest Date: Mon, 6 Feb 2023 20:58:33 -0800 Message-Id: <20230207045838.11243-6-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146575060799422?= X-GMAIL-MSGID: =?utf-8?q?1757146575060799422?= When comparing two fully_busy scheduling groups, keep the current busiest group if it represents an SMT core. Tasks in such scheduling group share CPU resources and need more help than tasks in a non-SMT fully_busy group. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c9d0ddfd11f2..df7bcbf634a8 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9514,10 +9514,22 @@ static bool update_sd_pick_busiest(struct lb_env *env, * contention when accessing shared HW resources. * * XXX for now avg_load is not computed and always 0 so we - * select the 1st one. + * select the 1st one, except if @sg is composed of SMT + * siblings. */ - if (sgs->avg_load <= busiest->avg_load) + + if (sgs->avg_load < busiest->avg_load) return false; + + if (sgs->avg_load == busiest->avg_load) { + /* + * SMT sched groups need more help than non-SMT groups. + * If @sg happens to also be SMT, either choice is good. + */ + if (sds->busiest->flags & SD_SHARE_CPUCAPACITY) + return false; + } + break; case group_has_spare: From patchwork Tue Feb 7 04:58:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53663 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649479wrn; Mon, 6 Feb 2023 20:53:13 -0800 (PST) X-Google-Smtp-Source: AK7set8s0U/t578pXZBwLnq8Qdkiq4v4llH3+FJgI0c5gGznenahWhWLg89Lo6KsvauCB40DKE3G X-Received: by 2002:a17:90b:1b50:b0:230:ba03:f322 with SMTP id nv16-20020a17090b1b5000b00230ba03f322mr2591814pjb.21.1675745593057; Mon, 06 Feb 2023 20:53:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745593; cv=none; d=google.com; s=arc-20160816; b=PSxsZnzjXiD8jHuejqT4226I8XhlKlUB5rq5LqvzdJp3nSwBWHvGybLBeyAD6n1Qk4 qq/JwC62bUZupL48y4xescQ2lzaw/5UO+6P1+HWEmHWfAoUlwHp8oKDdY2UuBmb7kmoC YxjSu3oY/fldj5oSRB+jzyz7BW5JDNWedFRA5NyMbZ2c4gewEsb8QjKqLbKzB5fgG6K9 2rbZXwAtKFPix09AF5fPHTWhjgERCqprjnA9/Viz3L5lWTkNkZ8cxWpV/Z6NBfiNuO9h yR/7j3sU1P3Q7wU9NyN0QspISNYxRZNdAqVrtH3Huhk2sBxRKRdfzpVxuRPALBfnn8q2 FR/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=ww4/UVZg7pule0+hKsxP3kqOsZ8YGPIEAXnT8F2GSHQ=; b=Vt+S15DFN6vm0jXNY02d8IV5YyfItlc4SbbE/sDUuttaE1PZP6Q8WIvnN+/puBNnU1 Yao9mQkR2nE0Eu2ZR5LT0PNShJO0EyQ08FJG5IrH4QOoDYYq3W7/ir+22qntmwPHVUHz rDDYYkhcAhWhwP74/NvYDPE71fXikbciufSa53Zrc2LxMGCNiX/AUgfHHtOvXek3fGGt R9vML0PKRzPI0agD233PeGf9t4gNJeeyRl1LjpUqd0QMw9UhhzelHA8SVg38iJP8NVeq zV2hhmmjVKhscQldAuf1tlyO8QGrOQ36pkTzIJKGtO+idt2FtxGABsKx0gL/R5E87B0l wzIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SsRNjdSS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s8-20020a17090aad8800b00227132d2a4esi8260772pjq.101.2023.02.06.20.52.59; Mon, 06 Feb 2023 20:53:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SsRNjdSS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230092AbjBGEvN (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229939AbjBGEut (ORCPT ); Mon, 6 Feb 2023 23:50:49 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC3EE4224 for ; Mon, 6 Feb 2023 20:50:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745448; x=1707281448; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=YXCyBdAQ0Q8XIVxjs4jHd98vK+N1vGYP5jLh9y5xDlw=; b=SsRNjdSSrN7/Pechmo0lvFWLJlMbcut3thdz6lEyVulzntW2fqLGpEoe MMkA8ZZaZm7UY5JjrC3MgQurO7pLd8nchhy2nvRcdbeSRX/+H1zyNOROI m/oeS9/hUPgDoB9lr/ygETgGQkP908ws2naKl6ZxTqurrtVUMfBWe2i46 lJDNrkgebccArGTGSv199qzuQe86DMEIoH/6b2XuSIyNOMG52aCZyEVsS dlytJghiMrliO3eIkbWhw8XncPTJIacKdqsAwW+HPED8dmWR3pNrPhpAk pKm7+WaPb14KjKcFqrvnpWvLkyO+qIZkhId8cBKdNNsNBkaRWonDW2VnD A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624023" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624023" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653796" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653796" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:47 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 06/10] sched/fair: Use the prefer_sibling flag of the current sched domain Date: Mon, 6 Feb 2023 20:58:34 -0800 Message-Id: <20230207045838.11243-7-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146611214728574?= X-GMAIL-MSGID: =?utf-8?q?1757146611214728574?= SD_PREFER_SIBLING is set from the SMT scheduling domain up to the first non-NUMA domain (the exception is systems with SD_ASYM_CPUCAPACITY). Above the SMT sched domain, all domains have a child. The SD_PREFER_ SIBLING is honored always regardless of the scheduling domain at which the load balance takes place. There are cases, however, in which the busiest CPU's sched domain has child but the destination CPU's does not. Consider, for instance a non-SMT core (or an SMT core with only one online sibling) doing load balance with an SMT core at the MC level. SD_PREFER_SIBLING will not be honored. We are left with a fully busy SMT core and an idle non-SMT core. Avoid inconsistent behavior. Use the prefer_sibling behavior at the current scheduling domain, not its child. The NUMA sched domain does not have the SD_PREFER_SIBLING flag. Thus, we will not spread load among NUMA sched groups, as desired. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Valentin Schneider Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index df7bcbf634a8..a37ad59f20ea 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10004,7 +10004,6 @@ static void update_idle_cpu_scan(struct lb_env *env, static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sds) { - struct sched_domain *child = env->sd->child; struct sched_group *sg = env->sd->groups; struct sg_lb_stats *local = &sds->local_stat; struct sg_lb_stats tmp_sgs; @@ -10045,9 +10044,11 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd sg = sg->next; } while (sg != env->sd->groups); - /* Tag domain that child domain prefers tasks go to siblings first */ - sds->prefer_sibling = child && child->flags & SD_PREFER_SIBLING; - + /* + * Tag domain that @env::sd prefers to spread excess tasks among + * sibling sched groups. + */ + sds->prefer_sibling = env->sd->flags & SD_PREFER_SIBLING; if (env->sd->flags & SD_NUMA) env->fbq_type = fbq_classify_group(&sds->busiest_stat); @@ -10346,7 +10347,6 @@ static struct sched_group *find_busiest_group(struct lb_env *env) goto out_balanced; } - /* Try to move all excess tasks to child's sibling domain */ if (sds.prefer_sibling && local->group_type == group_has_spare && busiest->sum_nr_running > local->sum_nr_running + 1) goto force_balance; From patchwork Tue Feb 7 04:58:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53664 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2651205wrn; Mon, 6 Feb 2023 20:58:41 -0800 (PST) X-Google-Smtp-Source: AK7set978CPAKlP+eMOB5vghIxe6bs4f+tQ9+TjPuy5BLGmz7HQoBVwaWnBO71HKOy2Sk/SNiUBE X-Received: by 2002:a62:1452:0:b0:593:c7d4:22b0 with SMTP id 79-20020a621452000000b00593c7d422b0mr1906720pfu.2.1675745920977; Mon, 06 Feb 2023 20:58:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745920; cv=none; d=google.com; s=arc-20160816; b=uiwk1IyDqg50R/Be83fDbyOSqr27g5nuOWJC5rE47IeqxHvyjMBmeX5MBMyGuilhDO evdNSlnnBBMyDsztmMwAhk8thUeqc/onchYh6SM90Fy6jAWZp+etFJkdfgtBFOHTzLKG olLrV+fwHRNjoJ0u/lwIOZ0wd1ONR7S9h2PqrbxEEFxiliQZutqS4oRNDeDcuSxbIdzn JCV+kT/5njep0mi//IdUg8obSwfA2I/1ci8x0EXuHS9u8JkZmDoNkWVTDiUgdoJSSxGy wrDD4+8Hn8np2B8++7h0KNjBjpiQ2IOwT1eDtdx4j3CtsTnXgGWmRgV3diApUWSpPeYp WCYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=NcexxxgltvupAzpSG+jcPumY1sgB48H/9vbN1oZYGXc=; b=FzAhZa3q0MEcEJ+GXbKZcJxqiV752ZYEWaLBYpzSKTpYNEZrosMUJnxTHCuVFIeZtQ RsaeC+uSi+NB7Nzw5qaMieEFe9mrLdiNPR+BhyvkNTHqiaehv1X2RppOexs/vzUpuAFg qP584Covq3gYMKYtomE2cKLe8s1VGqsBOE4/mFK5OKYl2/Jx9WwJMLuVwiu9hbYi1FSr GLeYbXybqgjkuNQ1x1goe4qFn32qwqaeTfWzrIlPow7cvZsQZ6IFrOun7h4mjhc4EnVA z4WWjVtKd2lU+zAgtGaR6rr/zGDe6ijFp7FtPolNJK6jQNXnjSLPQ4e2iLfU4A3u8Fhi mNhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Addoqy/W"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q25-20020a056a0002b900b00592d5c262aasi12540667pfs.137.2023.02.06.20.58.27; Mon, 06 Feb 2023 20:58:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Addoqy/W"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229854AbjBGEvW (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229535AbjBGEuu (ORCPT ); Mon, 6 Feb 2023 23:50:50 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 784574236 for ; Mon, 6 Feb 2023 20:50:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745449; x=1707281449; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=fNzFE8GHMV6lK6aBMJxB809+PnuCf6ZrTheiPPiWiEE=; b=Addoqy/WC6wa3n4JwXifYlMPFB/MmaXzJTJtb0yCk84Kj0cRFkcFazm3 aXO6LUMxUKGY7yARJUDwVOXMFkgfqRPN4cKo7MlJnOqeq2jTnx/xsuBsO qMKjo2izp3EyTXVVHOzD3JkVLEK50TjdXaeNmmWZo7hWXD+9wngNjmRpl NmCwSPCdpz9zWE6Ee35qVNxJRtPR0auGsa+EBJW3zO5+rF/X2twrH4vgm oQ25dcqFhx69CZHi8X2zUumCXs7pjwoZIu0Nqiafi/A3JhXiwrNTx6b01 EzE4PDD3RUYBMZ5ZUrQWTaRczij6TqkTkWYj1U3Fljc6Me5mGYE6h1Y4S g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624032" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624032" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653799" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653799" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:47 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 07/10] sched/fair: Do not even the number of busy CPUs via asym_packing Date: Mon, 6 Feb 2023 20:58:35 -0800 Message-Id: <20230207045838.11243-8-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146954494086986?= X-GMAIL-MSGID: =?utf-8?q?1757146954494086986?= Now that find_busiest_group() triggers load balancing between a fully_ busy SMT2 core and an idle non-SMT core, it is no longer needed to force balancing via asym_packing. Use asym_packing only as intended: when there is high-priority CPU that is idle. After this change, the same logic apply to SMT and non-SMT local groups. Simplify asym_smt_can_pull_tasks() accordingly. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 37 +++++-------------------------------- 1 file changed, 5 insertions(+), 32 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a37ad59f20ea..0ada2d18b934 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9247,20 +9247,15 @@ group_type group_classify(unsigned int imbalance_pct, * @sgs: Load-balancing statistics of the candidate busiest group * @sg: The candidate busiest group * - * Check the state of the SMT siblings of both @sds::local and @sg and decide - * if @dst_cpu can pull tasks. + * Check the state of the SMT siblings of @sg and decide if @dst_cpu can pull + * tasks. * * This function must be called only if all the SMT siblings of @dst_cpu are * idle, if any. * - * If @dst_cpu does not have SMT siblings, it can pull tasks if two or more of - * the SMT siblings of @sg are busy. If only one CPU in @sg is busy, pull tasks - * only if @dst_cpu has higher priority. - * - * If @dst_cpu has SMT siblings, decide based on the priority of @sg. Do it only - * if @sg has exactly one busy CPU (i.e., one more than @sds::local). Bigger - * imbalances in the number of busy CPUs will be dealt with in - * find_busiest_group(). + * @dst_cpu can pull tasks if @sg has exactly one busy CPU (i.e., one more than + * @sds::local) and has lower group priority than @sds::local. Bigger imbalances + * in the number of busy CPUs will be dealt with in find_busiest_group(). * * Return: true if @dst_cpu can pull tasks, false otherwise. */ @@ -9269,33 +9264,11 @@ static bool asym_smt_can_pull_tasks(int dst_cpu, struct sd_lb_stats *sds, struct sched_group *sg) { #ifdef CONFIG_SCHED_SMT - bool local_is_smt; int sg_busy_cpus; - local_is_smt = sds->local->flags & SD_SHARE_CPUCAPACITY; sg_busy_cpus = sgs->group_weight - sgs->idle_cpus; - if (!local_is_smt) { - /* - * If we are here, @dst_cpu is idle and does not have SMT - * siblings. Pull tasks if candidate group has two or more - * busy CPUs. - */ - if (sg_busy_cpus >= 2) /* implies sg_is_smt */ - return true; - - /* - * @dst_cpu does not have SMT siblings. @sg may have SMT - * siblings and only one is busy. In such case, @dst_cpu - * can help if it has higher priority and is idle (i.e., - * it has no running tasks). - */ - return sched_asym_prefer(dst_cpu, sg->asym_prefer_cpu); - } - /* - * @dst_cpu has SMT siblings and are also idle. - * * If the difference in the number of busy CPUs is two or more, let * find_busiest_group() take care of it. We only care if @sg has * exactly one busy CPU. This covers SMT and non-SMT sched groups. From patchwork Tue Feb 7 04:58:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53667 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2652296wrn; Mon, 6 Feb 2023 21:01:36 -0800 (PST) X-Google-Smtp-Source: AK7set9b4oPt71HrkWIrgK/MNTfdSGr6kxmjrxZ8d09kQPNaGh1zhOpDqfhvDxxNVix5MIZZVAdH X-Received: by 2002:a62:7bd2:0:b0:593:e0d2:cda2 with SMTP id w201-20020a627bd2000000b00593e0d2cda2mr2069382pfc.4.1675746096047; Mon, 06 Feb 2023 21:01:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746096; cv=none; d=google.com; s=arc-20160816; b=05Dp2eCTgDXXAEnwttHmjhze4wF0vvX02u6X0NA2m+UqcaYlE/NHyhgoNBjoRmcl08 pKRVtCSsNutNzqc4zLjL3n9Lrt0B+NxFBczmmyrnfBYuJqmDmUL/d4zS/f9v+rajQrxl /ImQNksa3w4lQkjafJZxsBMUBgfDkMC8P06Yu7y3+pND+TIH7O0/Fv0wspcLSrFb1E7e nZ7G3AzICPp5iu9xhSMfhk6AeNs1jF4zo07cT29j81d94nX9kNFga8yyyTwpNScAQDA4 XuP4nKTrkgXlxNQA/xLLd4J9LTL5NBqHtpP2E5a9ViyguXPKk6J5pYuvTdj7cLi5T9LJ WiMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=JqQOA/tLRvNKWcZJbfAI8eWIDo3GrqTOxSvbJyiAZOw=; b=vJew9Vptv/lXS7WRMv/ZiRBLHMsQRwbsTXe21LGG2oscvrdpo561ZVTfjegtOzTD6A blNgBseAylfzwPeiXQPNWK7V3joyq3ZcZQSDucyxqq2KwlsE/S89krP1n37lpM7CAycs zhfp9vLljzXFISALdzs+KOpzkzWZAvq8z7+OnZpwMLAq9CokSjLsuFOuSwpUcd2n1xFx 42Bf8vVCAD0WprD9mW5N3As4WL3CDqGyc4Bw2ct/woIDFSNsnKJJkJfJq5HEQ9Geoe1P 3jxnhM2yRtwsWh6h6Ugu5xyZ5gdXdICmi5Z88MDIjAidOOFOf59vxVbfWdvOnkLItHAy UXow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=i7xL4ERp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cn15-20020a056a00340f00b0058dbb06c94dsi14135665pfb.291.2023.02.06.21.01.22; Mon, 06 Feb 2023 21:01:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=i7xL4ERp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230168AbjBGEvR (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229647AbjBGEuu (ORCPT ); Mon, 6 Feb 2023 23:50:50 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B873B44BC for ; Mon, 6 Feb 2023 20:50:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745449; x=1707281449; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=gCAF3pOvYu6INyXHaG52RuEfLVgO2TXZcOSdaT+lLxQ=; b=i7xL4ERpfSQxN0FM17Jn4FDvGddQB0cbWybKTC36I9Bxopqo6Djxlth1 rY/kArRBYkugTUHFt0ZgTANUhRgfFR7lMrAiIXZy2QKtyh/fQrW5tKWYO 4+mW7EZeeZyF1d0nl6dZUglbWppevqgTjl60OA21QqTHLtDgz6i0p3kiw 9gCpI93vZwLOV24ixa1tuSmMX/Nsgi6uGiG42mGPKhwSc0vkrdVRa21f1 JqTOR+T2U9uyGrkw1/R42gQtNSSLGLXo+zjtTwI4I36TxbRt3GkvRsb3g H7kVvmabjF/sZeE+Y5QLO5r+V4cAEnp263NaMZ7/uPH109n0GG2mJhLfi g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624041" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624041" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653802" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653802" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:48 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 08/10] sched/topology: Remove SHARED_CHILD from ASYM_PACKING Date: Mon, 6 Feb 2023 20:58:36 -0800 Message-Id: <20230207045838.11243-9-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147138444351078?= X-GMAIL-MSGID: =?utf-8?q?1757147138444351078?= Only x86 and Power7 use ASYM_PACKING. They use it differently. Power7 has cores of equal priority, but the SMT siblings of a core have different priorities. Parent scheduling domains do not need (nor have) the ASYM_PACKING flag. SHARED_CHILD is not needed. Using SHARED_PARENT would cause the topology debug code to complain. X86 has cores of different priority, but all the SMT siblings of the core have equal priority. It needs ASYM_PACKING at the MC level, but not at the SMT level (it also needs it at upper levels if they have scheduling groups of different priority). Removing ASYM_PACKING from the SMT domain causes the topology debug code to complain. Remove SHARED_CHILD for now. We still need a topology check that satisfies both architectures. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Valentin Schneider Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- include/linux/sched/sd_flags.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h index 57bde66d95f7..800238854ba5 100644 --- a/include/linux/sched/sd_flags.h +++ b/include/linux/sched/sd_flags.h @@ -132,12 +132,9 @@ SD_FLAG(SD_SERIALIZE, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS) /* * Place busy tasks earlier in the domain * - * SHARED_CHILD: Usually set on the SMT level. Technically could be set further - * up, but currently assumed to be set from the base domain - * upwards (see update_top_cache_domain()). * NEEDS_GROUPS: Load balancing flag. */ -SD_FLAG(SD_ASYM_PACKING, SDF_SHARED_CHILD | SDF_NEEDS_GROUPS) +SD_FLAG(SD_ASYM_PACKING, SDF_NEEDS_GROUPS) /* * Prefer to place tasks in a sibling domain From patchwork Tue Feb 7 04:58:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53665 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2651327wrn; Mon, 6 Feb 2023 20:59:02 -0800 (PST) X-Google-Smtp-Source: AK7set/ijvYcmNlISmkM6EsQe/kJ01GGOTwgFHZMrTmC8QmDXOJO9LoyCENFP9IlGpBro9aVBkbu X-Received: by 2002:a17:902:ce92:b0:198:e966:66e with SMTP id f18-20020a170902ce9200b00198e966066emr1918696plg.4.1675745941796; Mon, 06 Feb 2023 20:59:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745941; cv=none; d=google.com; s=arc-20160816; b=XrtREUdUdIB0oudfyO19CZpaqvTFUhkg0w/E6CRG9HjHgV07AHmh4+SFmo0oJH2Wrw SRddk1CjybzI/aMjPZgqJn6tZV76gO8Nn7K2jrAxHCuIDVI1Yhh6eI4LeDroWrN/XwKk 6kVMm0XTsAqlc7rY+aHf4KJ6uVrPxybBcWgT3zXZdSM4Agk6SA3xX0ZVL0SIC3tP9Dyp V60z6gpBpDFPn+99CtV0RJinn7Z7K0AZQvmwcBDe4UOdh5pFZI0FzBt3V1CKHh0RP7oI Z/xeDx8qGc7kSUrFD+YNxIxoePzOlhXkBYSqr/b10ruxbQp+gYzGye3CeRr8dvdsl9Iq NVVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=h+PrE5Dh2eo+6hY4VGWk5md7uGRyA/r/oQ3MwpDu1Qc=; b=iwotZcAp8vOA8PD8upigHkPgY02g95Z34zDS3htjzfudgHpRiq0sAI5d68a08K3REM y8y4LxPuCO46Vol1xj87mIU+RONteKgpjoe2RZvkXjv62T6tAveLXlHb9jPlootHFWnA p0nF2dmHbWxLbzhn3HJ75xIy5e4a4ToxYcF0nAeJtI4xa0jIsAdyEQFM7IrKBzGwoAsk ukvklaqhVqBg6k1q6WZxbHuXJXPPqAEVRozOslrAj5Z9gUsCJLGjV6ENxOXjyaczRwui POslZZKP+KFkKjfeTxQiWBIVXOWYGFddUuu/qPqHpqb4BXXBnenBlXhAJQIzj+ZQpG33 IUVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JocOryHR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z15-20020a1709027e8f00b001991b226a3bsi4182068pla.422.2023.02.06.20.58.49; Mon, 06 Feb 2023 20:59:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JocOryHR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229525AbjBGEv1 (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229850AbjBGEuv (ORCPT ); Mon, 6 Feb 2023 23:50:51 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D52B8D3 for ; Mon, 6 Feb 2023 20:50:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745449; x=1707281449; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=7VjPXZk06n0CTxUjfuNsgpOoSCtgvNDjYZVfPD67F1s=; b=JocOryHRJNA9O8K9MfgNXuKDETp3g/aV2onT4uf/G9thXm4JLKfGVmbP xgVq75Fj8FWjBZclwxPJIs0sAEzxhK3C2egbLdRKFyn49rnaFfyhab/6k fpWsLPMAdWBDFiHjS0rfwQSTq2qx8FS9pfy9tTx7f07McvxQZB0MvFtlB LfUWEYNv/rRNsN8RlHPrdecL6dexkP248SsMguHgW+7+so3FRnIdn3Uh0 Z7l6/A0yVXnPQdMsj11Ws6dYfWYpr5VcmF4mV6ue/XsNv6sN3BZ9xAXI1 xI76NGP8utdazmB1OlZ9yG0sw9D58PVZXs+dLf2+/SPYL064POF1DEHXk w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624052" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624052" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653805" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653805" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:48 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 09/10] x86/sched: Remove SD_ASYM_PACKING from the SMT domain flags Date: Mon, 6 Feb 2023 20:58:37 -0800 Message-Id: <20230207045838.11243-10-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146976555912244?= X-GMAIL-MSGID: =?utf-8?q?1757146976555912244?= There is no difference between any of the SMT siblings of a physical core. Do not do asym_packing load balancing at this level. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None. Changes since v1: * Introduced this patch. --- arch/x86/kernel/smpboot.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 55cad72715d9..0213d066a9a9 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -547,7 +547,7 @@ static int x86_core_flags(void) #ifdef CONFIG_SCHED_SMT static int x86_smt_flags(void) { - return cpu_smt_flags() | x86_sched_itmt_flags(); + return cpu_smt_flags(); } #endif #ifdef CONFIG_SCHED_CLUSTER From patchwork Tue Feb 7 04:58:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53666 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2651379wrn; Mon, 6 Feb 2023 20:59:14 -0800 (PST) X-Google-Smtp-Source: AK7set90JjXBrubXs2vDUTvtzlFfiq99cXR/1uZ5Cdl9JPGJivGbv0nXaRCMVjsozZD++d8VyY+O X-Received: by 2002:a17:90a:1a52:b0:22c:1b3:b72b with SMTP id 18-20020a17090a1a5200b0022c01b3b72bmr2628406pjl.14.1675745954087; Mon, 06 Feb 2023 20:59:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745954; cv=none; d=google.com; s=arc-20160816; b=m4XkUwHQ3sOFj83rfdse7px1sNteJwUoGcFKpaaAxx0aFxSgzoVBhCgY7um3eYn5+J 4ObqQgr99X3s1jnvg6cVimtLAPPgrpQ1dYS7H9xap/8wT8ZWVrRJDZEUGByNrKLVZsGo X8+wPZk04atqx+Krr9XGH8GcvWmxNH3ccOPdyO1XXijTL3euaYS3s0UeFf1EDiuzzwf6 kuLGMygG4N8GCgUrNnNrJ/WEfElAtUcOLCKIa+KtuNWlK6Igm0Dpk0SDcuTM0Oj81sCM MtPAWyLpd23XeGd1/Wam3WasLcgBSc2o1DqSJ1pVZLzJgHkrqurA2tg+JdQ1z/R/I3E3 nDiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=EZvvdlOEyRJ0EwVSISNqydnTDZXTfN+iRHqLHoIVyIQ=; b=qqXhtFV7EiYm+XJnsgymnKKMdeRz5uRlIs7hLWKEiqQV8Qe4wOf4c82yqzzOhWA3tz dQi+/IqO2WuE+SnNZNmxPq06f4QGtamOTn+zRoYgdcWqLEEyfxuv19KmLAKsIGzYUgZC wSRfEVyHtdn1wC0fM/oDa2rJOfztjDhmak9gFUskHIPLVkuOSaXlrsvkjCpsTGHoLfS6 gI13wNvpwJEF4wLl5ErFOxl8oSQaxEnhM/LWWDSh9zjjceD6MO6g6KlCEHpMQ3La7Fr3 H/b6nWS/24gX3ulTkUUNW7LUF/j5oeKPcxxLD252j81HEDR9CTo1V9KTxeQpQ0bOvz79 oz+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=bItoG2aA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ls11-20020a17090b350b00b0023082ccac58si13467579pjb.7.2023.02.06.20.59.01; Mon, 06 Feb 2023 20:59:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=bItoG2aA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230010AbjBGEva (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229953AbjBGEuv (ORCPT ); Mon, 6 Feb 2023 23:50:51 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2B384224 for ; Mon, 6 Feb 2023 20:50:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745450; x=1707281450; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=V6FYmmT4588dHQyb2k4T7y1sGWHIXNL1S/nAV83f7Kc=; b=bItoG2aADFROUGdSb2VH4jrC46brvF3ikHNj8OPk60rTl3tqoRZqr/Zr /9Dj4KrZpwmDCPPG4jx+l/OHLJoEGLQhC8HmNTRYFEekfFJEeEJtS6xtY EIObkfX+FqfjbP3AUrAR8XTSlEX8P95FceKsYlrnAp0lJSblV9gsupVet jWz/1aViJVasfUOQPGJKoy17irsWgdfPGgiuF/ZAY4h2hCmQgaj9U5KxQ i+JB29OenZssRNAk+prtey3McNTeVtDbKGJcPhtGLI7bm/F7Kx5+YHiRa elPdtxUIFvt5qVD0xZwYKswhNe1rpK11rGtFgQ/tLvK1B1HzG/MN9dI7r g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624062" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624062" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653808" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653808" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:48 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 10/10] x86/sched/itmt: Give all SMT siblings of a core the same priority Date: Mon, 6 Feb 2023 20:58:38 -0800 Message-Id: <20230207045838.11243-11-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146989766267970?= X-GMAIL-MSGID: =?utf-8?q?1757146989766267970?= X86 does not have the SD_ASYM_PACKING flag in the SMT domain. The scheduler knows how to handle SMT and non-SMT cores of different priority. There is no reason for SMT siblings of a core to have different priorities. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Len Brown Signed-off-by: Ricardo Neri --- Changes since v2: * None. Changes since v1: * Reworded commit message for clarity. --- arch/x86/kernel/itmt.c | 23 +++++------------------ 1 file changed, 5 insertions(+), 18 deletions(-) diff --git a/arch/x86/kernel/itmt.c b/arch/x86/kernel/itmt.c index 9ff480e94511..6510883c5e81 100644 --- a/arch/x86/kernel/itmt.c +++ b/arch/x86/kernel/itmt.c @@ -174,32 +174,19 @@ int arch_asym_cpu_priority(int cpu) /** * sched_set_itmt_core_prio() - Set CPU priority based on ITMT - * @prio: Priority of cpu core - * @core_cpu: The cpu number associated with the core + * @prio: Priority of @cpu + * @cpu: The CPU number * * The pstate driver will find out the max boost frequency * and call this function to set a priority proportional - * to the max boost frequency. CPU with higher boost + * to the max boost frequency. CPUs with higher boost * frequency will receive higher priority. * * No need to rebuild sched domain after updating * the CPU priorities. The sched domains have no * dependency on CPU priorities. */ -void sched_set_itmt_core_prio(int prio, int core_cpu) +void sched_set_itmt_core_prio(int prio, int cpu) { - int cpu, i = 1; - - for_each_cpu(cpu, topology_sibling_cpumask(core_cpu)) { - int smt_prio; - - /* - * Ensure that the siblings are moved to the end - * of the priority chain and only used when - * all other high priority cpus are out of capacity. - */ - smt_prio = prio * smp_num_siblings / (i * i); - per_cpu(sched_core_priority, cpu) = smt_prio; - i++; - } + per_cpu(sched_core_priority, cpu) = prio; }