From patchwork Tue Feb 7 04:58:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53660 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2649184wrn; Mon, 6 Feb 2023 20:52:20 -0800 (PST) X-Google-Smtp-Source: AK7set/8MCLLVMCtUVSjoGsZBT6kVoCY+TsIQmXFTuPnElc51pBoT45HNze3vXT0/ed3Ufxaa/nq X-Received: by 2002:a05:6a20:a01d:b0:bb:c422:809f with SMTP id p29-20020a056a20a01d00b000bbc422809fmr2288307pzj.4.1675745540350; Mon, 06 Feb 2023 20:52:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675745540; cv=none; d=google.com; s=arc-20160816; b=XYmHj4NKRhOOg4tZs10oNycMuGTjojFRUJ8076Tw05iE74Eo9oaWRsN1Qd4SI+NiD9 7tRqY7hZuNBuH3H64hI+MHVKAJavojyXCkS/tO+/z0gBEe62i/lN3TTrdkIFS30V9sUp WH79AjNfmXX8L4ucTbGsfNpvBxmCL5qTuD51rUfp2QKmFxwUDGCgxLwRuMkeNTGlsVNp lFAnC+Urv+b0vVGnVvgUujpx9TuiZ1PcbWIybtLxT7834ciPbyN15biD2UJWj4OVzVcx IIlZ54LSD574PjLFyoEdcBvBzhU+UwTvIRAao5SfNjVG+N9vQuyzNJ0EM11qaChhY6J+ wITw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=pdgOdU5fNY2lTAuA2Zxw82DFQ2kJyOkfJeIV/iI0mYQ=; b=LzZrVbjM7vuOyqb5Y5/JuAQuEQHvsNGuVdnC4lGVRVL5g07oPvPNfgL93zsgAradxO z+BNaZm31YYkMSP0M8d+vvQGXzncpc/1OVSU2+H6DsjRtwj5dN1znHbfUXEZ9E3SkNWg jouG7ry393wA5b+2a8PtBT3tRVh/EzzUA7N+mvU8yEKZm38Gdl0wY7DsBGhGiSHihVOB 458Hrf1GeloyN4q1UY4/0Y4x/CwRu5G8CGDKLmu8KMF9Bzew1zj5lARluW82far53Rmu pOvuDCb0//iX3biDBW2Qz8tsOlGOPjqUtov/9fKx7hLoG5hox6wuzm3elxAZ7OkNqhLs dwSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YDiE51cF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b5-20020a637145000000b004df3908855csi15556227pgn.421.2023.02.06.20.52.07; Mon, 06 Feb 2023 20:52:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YDiE51cF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229994AbjBGEvB (ORCPT + 99 others); Mon, 6 Feb 2023 23:51:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229832AbjBGEus (ORCPT ); Mon, 6 Feb 2023 23:50:48 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10DF044BC for ; Mon, 6 Feb 2023 20:50:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675745448; x=1707281448; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=BhYkRtUf49Glw2ZU0a3Pjs3bki0DXfT9COx0nHL/qCw=; b=YDiE51cFVqLf62JsGMdpJchxCPazPEa9taxciml+UxNWEVPIcA387Y9j UH7SiGW7dUPaLVD4FNTHSz7wcEV4HQwAntBYK/vvsXcTfcy7znMlRqdLz /bAXyt6wxsLMuFoGXXt0Hd8y+XpOHfgUZwgMw8XevWJUGC1QJvjjZ6v0M 0cxLIt21/6FKXhOnMrRjd5cGUsO1mCc85vjjv9F8uAGWpYckXHI07tglT S4OsdxcvJusnDo/fPLWUiPgOhSq9LvSTujYic9YdZw+WbZ0+4p4A59SYW vew3fVbz6tnRXI2Jzn8UDs3pePGt/g+fz9HmBQpit6AuaRNix0yWWlsQ9 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415624005" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415624005" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 20:50:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668653789" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668653789" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 20:50:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 04/10] sched/fair: Let low-priority cores help high-priority busy SMT cores Date: Mon, 6 Feb 2023 20:58:32 -0800 Message-Id: <20230207045838.11243-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> References: <20230207045838.11243-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757146555912228703?= X-GMAIL-MSGID: =?utf-8?q?1757146555912228703?= Using asym_packing priorities within an SMT core is straightforward. Just follow the priorities that hardware indicates. When balancing load from an SMT core, also consider the idle of its siblings. Priorities do not reflect that an SMT core divides its throughput among all its busy siblings. They only makes sense when exactly one sibling is busy. Indicate that active balance is needed if the destination CPU has lower priority than the source CPU but the latter has busy SMT siblings. Make find_busiest_queue() not skip higher-priority SMT cores with more than busy sibling. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Valentin Schneider Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 80c86462c6f6..c9d0ddfd11f2 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10436,11 +10436,20 @@ static struct rq *find_busiest_queue(struct lb_env *env, nr_running == 1) continue; - /* Make sure we only pull tasks from a CPU of lower priority */ + /* + * Make sure we only pull tasks from a CPU of lower priority + * when balancing between SMT siblings. + * + * If balancing between cores, let lower priority CPUs help + * SMT cores with more than one busy sibling. + */ if ((env->sd->flags & SD_ASYM_PACKING) && sched_asym_prefer(i, env->dst_cpu) && - nr_running == 1) - continue; + nr_running == 1) { + if (env->sd->flags & SD_SHARE_CPUCAPACITY || + (!(env->sd->flags & SD_SHARE_CPUCAPACITY) && is_core_idle(i))) + continue; + } switch (env->migration_type) { case migrate_load: @@ -10530,8 +10539,20 @@ asym_active_balance(struct lb_env *env) * lower priority CPUs in order to pack all tasks in the * highest priority CPUs. */ - return env->idle != CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING) && - sched_asym_prefer(env->dst_cpu, env->src_cpu); + if (env->idle != CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING)) { + /* Always obey priorities between SMT siblings. */ + if (env->sd->flags & SD_SHARE_CPUCAPACITY) + return sched_asym_prefer(env->dst_cpu, env->src_cpu); + + /* + * A lower priority CPU can help an SMT core with more than one + * busy sibling. + */ + return sched_asym_prefer(env->dst_cpu, env->src_cpu) || + !is_core_idle(env->src_cpu); + } + + return false; } static inline bool