From patchwork Tue Feb 7 05:10:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53669 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2652814wrn; Mon, 6 Feb 2023 21:02:47 -0800 (PST) X-Google-Smtp-Source: AK7set+nIwkGmvANtWZUnrRuSsGw/3EWKKCpvXKCvuuwXJvkG5ZlSBNmU+wa7KKNSwb7U6PRxe2V X-Received: by 2002:a17:902:f809:b0:196:32fa:69a9 with SMTP id ix9-20020a170902f80900b0019632fa69a9mr1370956plb.51.1675746167593; Mon, 06 Feb 2023 21:02:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746167; cv=none; d=google.com; s=arc-20160816; b=iYtBwf1DyWniPxJlWBf4+g/wNopqnE1l7r/SJhHHO23t60xxpsw37pbl9tC8IgaqR0 DJg4Iuuu7gMkTyvwo5TqS3NLB9iG59+nu/pk0/4hSP8VLi3ZpvS3ZA7Fv4Xm1y+DykbI AFhr49gxYe82vGF0N7j5BBlrg62XN7QsOd953+AaL8eU02E9o+rrB7rLLu7JX/KQ8CM5 KZzO+pD9CFieP9yOMGME+S+DysW2cgLoeYfD7FF//jtY0RcmuP9rAMHD1R9zWDcGZF0L b7XP7ABfko2B9kKDtju4tfDE8iWipRUoKot/ZM5A1FvhSLwOv61Fmc9a6qhxR7+xR4DD xC1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=4tpXAC6ZOL2TtTbMN+BSq5ZEVnqY/mOkGbKRH+3t6I8=; b=jgmqB/f5/lZOy93jNqDYT8+O4u2ZA7H+GzrL37848e/kkm4qIu57yj0rvkmLg9yt1q fSzuy+mQjrHPb+gD5oGC1z5WQUKFvGslLM6oVbkme4bD3OZVbspsvKfJhqFgGo43NtZO 8Hz1RoDuwDHzOmc6AlQ4rdJXZl8CFTKxgrx5+nq6MakwX4NVpxYQGjoypq1hyAkYepkK pr2rDEWQxxKeY0Zkc3HH6IQS71CLUJE7RO04I7DhZVF7srPg4ZBoItStSH1vXVBRzwIT PHLFZBVKUwiVzofTjdCTLVMAnmqbaJL4v44nXMYdlHl+vhg9eFlfuwVnwy8N3y1InR94 e/FA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=l6lE2P4k; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u2-20020a170902a60200b00196711a94f6si12480844plq.353.2023.02.06.21.02.35; Mon, 06 Feb 2023 21:02:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=l6lE2P4k; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230167AbjBGFCB (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230040AbjBGFBt (ORCPT ); Tue, 7 Feb 2023 00:01:49 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C78A22B2BC; Mon, 6 Feb 2023 21:01:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746101; x=1707282101; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=lZUt9bTzhG5+Fn6a2/x++iylr7jp1U1p86qFgAgn3yU=; b=l6lE2P4kbnF0kklERSzoSHvU0x1byi/pE8aw7AtzyiS93pW2d3z/oh6k zg/ryjKJvt6n2Gw5J54AlcUJSO0nNDtHryL7C5GQHO3FPYdHUqKSLGKDQ 3B+EzhDjmh5IRDGVGYfCGcz4jC9akVS0R96Afkys4lJuHx19dhshaui4t UwJMt3oPJ9RkPk5Kg7NbAEwrJc+z3Ja9As0/RC8U83D7PmXPiPMZBQYlT 8AlQJoNt9PSV0rpn4xOJlq0cAkJKBKU/0VZP3QmK6cv2TTGJotUN2xmz2 EaQdOl4u6rP+ZIBfae8+La3ukKKGNDMVKhEV+RWWLHW4MqPBWlHOnRfA7 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625738" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625738" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657688" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657688" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:39 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 01/24] sched/task_struct: Introduce IPC classes of tasks Date: Mon, 6 Feb 2023 21:10:42 -0800 Message-Id: <20230207051105.11575-2-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147213156012399?= X-GMAIL-MSGID: =?utf-8?q?1757147213156012399?= On hybrid processors, the architecture differences between the types of CPUs lead to different instructions-per-cycle (IPC) on each type of CPU. IPCs may differ further by the type of instructions. Instructions can be grouped into classes of similar IPCs. Hence, tasks can be classified into groups based on the type of instructions they execute. Add a new member task_struct::ipcc to associate a particular task to an IPC class that depends on the instructions it executes. The scheduler may use the IPC class of a task and data about the performance among CPUs of a given IPC class to improve throughput. It may, for instance, place certain classes of tasks on CPUs of higher performance. The methods to determine the classification of a task and its relative IPC score are specific to each CPU architecture. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Changed the type of task_struct::ipcc to unsigned short. A subsequent patch uses bit fields to use 9 bits, along with other auxiliary members. Changes since v1: * Renamed task_struct::class as task_struct::ipcc. (Joel) * Use task_struct::ipcc = 0 for unclassified tasks. (PeterZ) * Renamed CONFIG_SCHED_TASK_CLASSES as CONFIG_IPC_CLASSES. (PeterZ, Joel) --- include/linux/sched.h | 10 ++++++++++ init/Kconfig | 12 ++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 4df2b3e76b30..98f84f90a01d 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -127,6 +127,8 @@ struct task_group; __TASK_TRACED | EXIT_DEAD | EXIT_ZOMBIE | \ TASK_PARKED) +#define IPC_CLASS_UNCLASSIFIED 0 + #define task_is_running(task) (READ_ONCE((task)->__state) == TASK_RUNNING) #define task_is_traced(task) ((READ_ONCE(task->jobctl) & JOBCTL_TRACED) != 0) @@ -1528,6 +1530,14 @@ struct task_struct { union rv_task_monitor rv[RV_PER_TASK_MONITORS]; #endif +#ifdef CONFIG_IPC_CLASSES + /* + * A hardware-defined classification of task that reflects but is + * not identical to the number of instructions per cycle. + */ + unsigned short ipcc; +#endif + /* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct. diff --git a/init/Kconfig b/init/Kconfig index e76dc579cfa2..731a4b652030 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -867,6 +867,18 @@ config UCLAMP_BUCKETS_COUNT If in doubt, use the default value. +config IPC_CLASSES + bool "IPC classes of tasks" + depends on SMP + help + If selected, each task is assigned a classification value that + reflects the type of instructions that the task executes. This + classification reflects but is not equal to the number of + instructions retired per cycle. + + The scheduler uses the classification value to improve the placement + of tasks. + endmenu # From patchwork Tue Feb 7 05:10:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53670 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2652840wrn; Mon, 6 Feb 2023 21:02:52 -0800 (PST) X-Google-Smtp-Source: AK7set+vPg6hEQtAccJRL6Y3IiMoJAnmyIII2njvA92f55jAwWl4mf8UhPGB1R7x2CUro+mjs8+N X-Received: by 2002:a17:90b:1d90:b0:230:e9ff:7292 with SMTP id pf16-20020a17090b1d9000b00230e9ff7292mr1659742pjb.40.1675746172083; Mon, 06 Feb 2023 21:02:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746172; cv=none; d=google.com; s=arc-20160816; b=VvqetAWZlelVZVZL+q1FyR1O1ZSowqlL6D/xO69RyL8Wd5AYr2Zkn6sCicOrrvZ7e4 FhTK6zZ8zne8eI3JaDs0sR19gOyiY3eDS8G4wN9yifVcp8R1Nz6dSJbW1mMvMut0IZ5T jndLasUNAithEupBGOld+cyyA8s4KRxk3Dlbhcfnos3dhajCUm/n5ecVuIqwJO15Ndgo P8YUzsYKTuTzkoYNkB2X5wQLL9j8zAH/cBI2KzjeVEhWnisG4BGbDLxZ4inL844K264G 2eQ4kqPgbzqLV+d7a6A/XHOTcjFsgRh7/7NwAf3bqldc8RtKbpVOJfoJdfT0C0PWGm8W 6uXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=mMPXXoeUcn/v2osC2m+S2uqKdk26hXrEcX//nmPwYTw=; b=Dqeeb671FNS4ifjgAhRXrmIkpXb3SRzXyEk7VYkn97E3eiL/DsqPVYH45HFH03aLPv lUCqO/nt5ktMxIDMTtsihP4akY3QXAWa/YgEvV3N6xn9oYTpUJr2ks3G8AFJZLK3vBQo 2j2dner8c+J818oSYyDZD2cw/wxYMf3+XLtrlH/97CYsfcelC/hK4OEi0MhGbuFkKLz5 PFN1wgb4JAvMY9iwgeDQ9zVw9Acl0DGO+5akuPXDW3W+4dmd419XC6IXDw4WvVegu3g+ HjocN1QrSXhj4nvkdBK5H9cSlfz8E7ddJplc9MHAIFO1k7vu/ZLqI1mOayOWQXzZHkCa O8vg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WZdoyckU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b23-20020a17090a101700b002262a74af32si13182553pja.40.2023.02.06.21.02.39; Mon, 06 Feb 2023 21:02:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WZdoyckU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229868AbjBGFCF (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230132AbjBGFBv (ORCPT ); Tue, 7 Feb 2023 00:01:51 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 795FC2B084; Mon, 6 Feb 2023 21:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746103; x=1707282103; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Ks6Wx9Pqszz2YvypOx73VVPt8pwQu0X0WqROcgHGPFg=; b=WZdoyckUVTPTLZ2SvcnaawYxkYhotga/TCp5v4N7a2+bLqWFEwZmIjlk WoWar9WG52JEq7WxCX/2bNAesLGgDxBYbocBFWpoy4y/xMiu/MEq/PjWe cCO0WmtY9GowklBREZ399+wQ2MVjzrTqQLKmN4XWXX9PU2CLnV5OREUN0 VhPmsYet5dPMJFBVEslZ1wfq4kFG+XPwEXk3JqvukrC+rDHvUQC7PWE2+ rF/PVCsHvI4lRJUXluceKGroS4y0UpdI3OIGzuEvyggnP+Z/DxSRFoVw/ kNEmureFao+KWih2O8kBXEhAeQICNolkr+1CWB/9lGWPUC+zJeNXsVZ/x w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625749" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625749" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657693" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657693" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:40 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 02/24] sched: Add interfaces for IPC classes Date: Mon, 6 Feb 2023 21:10:43 -0800 Message-Id: <20230207051105.11575-3-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147218285451213?= X-GMAIL-MSGID: =?utf-8?q?1757147218285451213?= Add the interfaces that architectures shall implement to convey the data to support IPC classes. arch_update_ipcc() updates the IPC classification of the current task as given by hardware. arch_get_ipcc_score() provides a performance score for a given IPC class when placed on a specific CPU. Higher scores indicate higher performance. When a driver or equivalent enablement code has configured the necessary hardware to support IPC classes, it should call sched_enable_ipc_classes() to notify the scheduler that it can start using IPC classes data. The number of classes and the score of each class of task are determined by hardware. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Clarified the properties of the IPC score: abstract, and linear. It can normalized when needed. (Ionela) * Selected a better default IPC score. (Ionela) * Removed arch_has_ipc_classes(). It is not suitable for hardware that is not ready to support IPC classes after boot. (Lukasz) * Added a new sched_enable_ipc_classes() interface that drivers or enablement code can call when ready to support IPC classes. (Lukasz) Changes since v1: * Shortened the names of the IPCC interfaces (PeterZ): sched_task_classes_enabled >> sched_ipcc_enabled arch_has_task_classes >> arch_has_ipc_classes arch_update_task_class >> arch_update_ipcc arch_get_task_class_score >> arch_get_ipcc_score * Removed smt_siblings_idle argument from arch_update_ipcc(). (PeterZ) --- include/linux/sched/topology.h | 6 ++++ kernel/sched/sched.h | 66 ++++++++++++++++++++++++++++++++++ kernel/sched/topology.c | 9 +++++ 3 files changed, 81 insertions(+) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index 816df6cc444e..5b084d3c9ad1 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -280,4 +280,10 @@ static inline int task_node(const struct task_struct *p) return cpu_to_node(task_cpu(p)); } +#ifdef CONFIG_IPC_CLASSES +extern void sched_enable_ipc_classes(void); +#else +static inline void sched_enable_ipc_classes(void) { } +#endif + #endif /* _LINUX_SCHED_TOPOLOGY_H */ diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 1072502976df..0a9c3024326d 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2532,6 +2532,72 @@ void arch_scale_freq_tick(void) } #endif +#ifdef CONFIG_IPC_CLASSES +DECLARE_STATIC_KEY_FALSE(sched_ipcc); + +static inline bool sched_ipcc_enabled(void) +{ + return static_branch_unlikely(&sched_ipcc); +} + +#ifndef arch_update_ipcc +/** + * arch_update_ipcc() - Update the IPC class of the current task + * @curr: The current task + * + * Request that the IPC classification of @curr is updated. + * + * Returns: none + */ +static __always_inline +void arch_update_ipcc(struct task_struct *curr) +{ +} +#endif + +#ifndef arch_get_ipcc_score + +#define SCHED_IPCC_SCORE_SCALE (1L << SCHED_FIXEDPOINT_SHIFT) +/** + * arch_get_ipcc_score() - Get the IPC score of a class of task + * @ipcc: The IPC class + * @cpu: A CPU number + * + * The IPC performance scores reflects (but it is not identical to) the number + * of instructions retired per cycle for a given IPC class. It is a linear and + * abstract metric. Higher scores reflect better performance. + * + * The IPC score can be normalized with respect to the class, i, with the + * highest IPC score on the CPU, c, with highest performance: + * + * IPC(i, c) + * ------------------------------------ * SCHED_IPCC_SCORE_SCALE + * max(IPC(i, c) : (i, c)) + * + * Scheduling schemes that want to use the IPC score along with other + * normalized metrics for scheduling (e.g., CPU capacity) may need to normalize + * it. + * + * Other scheduling schemes (e.g., asym_packing) do not need normalization. + * + * Returns the performance score of an IPC class, @ipcc, when running on @cpu. + * Error when either @ipcc or @cpu are invalid. + */ +static __always_inline +unsigned long arch_get_ipcc_score(unsigned short ipcc, int cpu) +{ + return SCHED_IPCC_SCORE_SCALE; +} +#endif +#else /* CONFIG_IPC_CLASSES */ + +#define arch_get_ipcc_score(ipcc, cpu) (-EINVAL) +#define arch_update_ipcc(curr) + +static inline bool sched_ipcc_enabled(void) { return false; } + +#endif /* CONFIG_IPC_CLASSES */ + #ifndef arch_scale_freq_capacity /** * arch_scale_freq_capacity - get the frequency scale factor of a given CPU. diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index d93c3379e901..8380bb7f0cd9 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -670,6 +670,15 @@ DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing); DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity); DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity); +#ifdef CONFIG_IPC_CLASSES +DEFINE_STATIC_KEY_FALSE(sched_ipcc); + +void sched_enable_ipc_classes(void) +{ + static_branch_enable_cpuslocked(&sched_ipcc); +} +#endif + static void update_top_cache_domain(int cpu) { struct sched_domain_shared *sds = NULL; From patchwork Tue Feb 7 05:10:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53671 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2652865wrn; Mon, 6 Feb 2023 21:02:55 -0800 (PST) X-Google-Smtp-Source: AK7set8L4Nj3T5ZdR2DPH93yWMGIZaJglbKbrjajdtFyDnvjI/N//qwcb4RDg4tcog4uYWPqBmZe X-Received: by 2002:a17:902:ec85:b0:199:2ae2:5842 with SMTP id x5-20020a170902ec8500b001992ae25842mr1809473plg.17.1675746175294; Mon, 06 Feb 2023 21:02:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746175; cv=none; d=google.com; s=arc-20160816; b=FHiGfVTPRwMyVR2JjMoMBE3mT27CL4/dXnJHfZvXVyD6RB2vIkR7nu/tMLMfNVaPZE ChCmg1zreRAs+EQKBcjqGRJtLDT/4nw2gGm93nRwhGYVvIDNZg1S2GmFBC6ZUocDwJ0x CwUmkQWSZYLHhP8dV0nzYaz8g8pEXKKZo7l/QN+p0PApmf1Yy22q9k4KBSvg46MxiWG1 Zw+jDgMX8FGPLc6Nl4xZaTDAytjaEPehvdWN8dU7NZLa3LRCixn0P7bvzQuUM1xTh1UT x6vV6FImkB5pvT8tU9QmuD7nB6UUBMMPclHvUqDx8O9kXO5Pn7QcMf1RviJpGWpi4eSS OBSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=Iz2qwbPrQ7JpBrZTMbknLbeIspToohjVhIhGf2oRC98=; b=P0KJcVFIbuPMUtVKvlNjDU2W9lu79NEoMaBZ1vS9ZP/0C31iSCvo7nLGOd+jOzL952 /RT3CcliIAuR3fngHxsYiym7ODuYzjGHx5rsW2gVhXwY5CIP4+NCm0HEaHEL4bD4XXge aUnAAB/4vhxGc8pexfDOPxMQdibvKK/bI4aktIort/R+dUsXdPemmbqz5Yy3Iai1s3Fx B6UM8szNl2hwcRvAeaEDav3/9qgkbo/mlIzrd//ZArHzyf/dkH2D2mTLh5JHkesyVt6s YX1Ce6Y8nnh5/cTNrPz6Mi5qTqK5NpBNzqwIKsEd8e7W46f0QtJ6sldFGmcKROFPEcuJ f8aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=RdDab7YV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l18-20020a170903121200b00194d0ae0eedsi9070816plh.546.2023.02.06.21.02.42; Mon, 06 Feb 2023 21:02:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=RdDab7YV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230199AbjBGFCH (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229462AbjBGFBv (ORCPT ); Tue, 7 Feb 2023 00:01:51 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08D912709; Mon, 6 Feb 2023 21:01:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746105; x=1707282105; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=aj6JFQKyLnSw7eTFaXzl39JTMTTiJ20FhjGa4KuYwkI=; b=RdDab7YVICyTccSdq8g41dZrfMgwXUNmpwT9xowLHToq8U5S8IjpowyJ 2PEr86LBp2l25Bek0y8B3/M/WB1PftNqGb5G0oVjzaHizwh6X9TYiyj4U oIKhyjywiTZs7f8Ro+TR0xRoT512tABTt/q68q7+CdtCshDf0SY5a9P5v Sfddl1YuDN3OX2u8L/FEE4ZaQAcC7ldPCUG/fN77n102+sLLqxW3tqV8T eCjO2Xu5+WDHl74d3IX0iBPgyEaZJq8UrhhNSm2brbNDO3Z8+vooo1xbC I6FXXBkdzDSUqShxG6Z5Yg7OSaTKpewWNEsjNQsSH6Fn3iC4un8o4ahNY A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625761" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625761" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657696" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657696" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:40 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 03/24] sched/core: Initialize the IPC class of a new task Date: Mon, 6 Feb 2023 21:10:44 -0800 Message-Id: <20230207051105.11575-4-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147221450242771?= X-GMAIL-MSGID: =?utf-8?q?1757147221450242771?= New tasks shall start life as unclassified. They will be classified by hardware when they run. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * None --- kernel/sched/core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 4580fe3e1d0c..ed1549d28090 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4428,6 +4428,9 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p) p->se.prev_sum_exec_runtime = 0; p->se.nr_migrations = 0; p->se.vruntime = 0; +#ifdef CONFIG_IPC_CLASSES + p->ipcc = IPC_CLASS_UNCLASSIFIED; +#endif INIT_LIST_HEAD(&p->se.group_node); #ifdef CONFIG_FAIR_GROUP_SCHED From patchwork Tue Feb 7 05:10:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53672 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2652960wrn; Mon, 6 Feb 2023 21:03:09 -0800 (PST) X-Google-Smtp-Source: AK7set+zskyIYCvCgu2yxA8HA7AAj6J5y42fI5jBEPPr2DssiSHUW8icJOyOFIZlYw5kHTzbBbLJ X-Received: by 2002:a17:903:2448:b0:192:5ec4:6656 with SMTP id l8-20020a170903244800b001925ec46656mr2119766pls.3.1675746189616; Mon, 06 Feb 2023 21:03:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746189; cv=none; d=google.com; s=arc-20160816; b=Hzm0EvlQhuV/17KhwhD1JbMISOay2e/aj6g2E7DimlkxwEW28/Fp/u98r1JW1S47DG +hVRZCqsDthsc4sBJlat7NAVrEQkp1Lim0F4de6K/Y/PHIL+73NBQEkdUvW1L3NVPacU rfZsyJNm6zsgREBjJjlsZR15+x6v4o8SqbaBMwawvx9X2aV+Y3yreJ3Oek4Z5GTz6DSd WM+UiTNvhuQC1NxdSpxNMHGY5vfBHE/Cux89A1fiHY6eqDUIZG39vR5OVPIjUdG26AP2 hg25JcgeCHWByg5Fu428GcpmKYCoMKO2xuIqnhI4ftRT8cswdb5OutimLaW1Zaa4ieJ1 NaTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=JCXEkSzUDRJIl8C22FBIvM+fDWc7IyJTBoI1Su50E8A=; b=pTlNj99IFm4nkjDKSL7ZLkWDMH+O+3mFObhqJqB506pW/AGHlVdln/ekh0LfhEeKRD k8QrTUoyw6dXYfOb3KpR8u8YAPL3/eDfzMShXoJnlPrwmCeoDhrxzngBI5txD89ynKM5 NBLzur+yq2152crcgz/KHbs8UZe2AC393M2n1WUnVDa6fMzymXcVav4t02wEbETBTZYb 3nzMQxCDoe94WWtxpn4HTjW5i5h1oIi2dynucWVtg/J9+aWRzqRUDZAZKm7sbepN5toy kogt4QqcuyM18craqcEwr/GlZrdLcHBCuzjZUruwYnmiusBKk1TwddtTWx/9+zsU+U5B Ss1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="ep/FwozE"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z8-20020a170903018800b001968124dc22si16163516plg.560.2023.02.06.21.02.54; Mon, 06 Feb 2023 21:03:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="ep/FwozE"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230257AbjBGFCY (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230152AbjBGFBx (ORCPT ); Tue, 7 Feb 2023 00:01:53 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAF374C0A; Mon, 6 Feb 2023 21:01:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746109; x=1707282109; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=iDkGE/mIiTsNfeFOnUGlfr0oeLSL2C0UHOku/Uu39Qo=; b=ep/FwozEgPY3+GMxNFBQI7DFYoIlSGA70lZmUH/CqwfRw4+z9+c6iLY0 qw3YJw+vjvQqbe8YZ3NucEpnWmJDcS3Q1osfUPoU4+75i7tKjzhgNcnxo y9VJWF+4v/zKyJOqHcO4PQnyEHZigA5nMwYUuE1pbuEx7l9fM4PMzGuBc ipo0hQ8/lbMeugQxE5DAoepBYKL2GCo+1Pjs6OaOaOUGgMLeS7wzaNPmo HiMbNsbxb9FtzqTP2T0MUfoEdnAMo0v8AhzCurAn7bhez6Cf0KAsoaZT6 05BFJy61P30OuxiJgsxtNWYgw38oyKW1EwLDPz7Ae9fh0RoL4ej2TXL2K A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625771" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625771" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657699" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657699" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:40 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 04/24] sched/core: Add user_tick as argument to scheduler_tick() Date: Mon, 6 Feb 2023 21:10:45 -0800 Message-Id: <20230207051105.11575-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147236492223638?= X-GMAIL-MSGID: =?utf-8?q?1757147236492223638?= Differentiate between user and kernel ticks so that the scheduler updates the IPC class of the current task during the former. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Corrected error in the changeset description: the IPC class of the current task is updated at user tick. (Dietmar) Changes since v1: * None --- include/linux/sched.h | 2 +- kernel/sched/core.c | 2 +- kernel/time/timer.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 98f84f90a01d..10c6abdc3465 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -293,7 +293,7 @@ enum { TASK_COMM_LEN = 16, }; -extern void scheduler_tick(void); +extern void scheduler_tick(bool user_tick); #define MAX_SCHEDULE_TIMEOUT LONG_MAX diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ed1549d28090..39d218a2f243 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5555,7 +5555,7 @@ static inline u64 cpu_resched_latency(struct rq *rq) { return 0; } * This function gets called by the timer code, with HZ frequency. * We call it with interrupts disabled. */ -void scheduler_tick(void) +void scheduler_tick(bool user_tick) { int cpu = smp_processor_id(); struct rq *rq = cpu_rq(cpu); diff --git a/kernel/time/timer.c b/kernel/time/timer.c index 63a8ce7177dd..e15e24105891 100644 --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -2073,7 +2073,7 @@ void update_process_times(int user_tick) if (in_irq()) irq_work_tick(); #endif - scheduler_tick(); + scheduler_tick(user_tick); if (IS_ENABLED(CONFIG_POSIX_TIMERS)) run_posix_cpu_timers(); } From patchwork Tue Feb 7 05:10:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53673 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653017wrn; Mon, 6 Feb 2023 21:03:18 -0800 (PST) X-Google-Smtp-Source: AK7set/dXCTDWgmvKt6MRpwIsQago6j62apIEW8kEKz91dR9kY6bFfCDNYVcF3jA4AdKzMgmHFrQ X-Received: by 2002:a05:6a20:8f03:b0:b8:71b7:b903 with SMTP id b3-20020a056a208f0300b000b871b7b903mr2674358pzk.31.1675746198288; Mon, 06 Feb 2023 21:03:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746198; cv=none; d=google.com; s=arc-20160816; b=ezQcg9uCj8PokclLUQnIHLeH6sjo3V7H6WrsMQKC6EBaEZlVbxm/ZJG82r3GxDJHbi EBfLTxzPzFMG62p89+gBts9sH6kP0O/9UkKLF51NaHrhnhMb2w06ZY2x7oqxEbRvRBNG vUwgwoIXotzUNJJ7Mi4DIxwU5ExsXx/qF58Wbhzn6hjORYgXZaRwbTtmsNI0kXTtmZUj oiIL/nDvM/bU7V0tK3dnfTZHSjdP9F0EykQfUcT3V3CnuJvEWV1isZcRYs8kb7e9FPka ZunnFbLgR0/GMh+l+ff/FBCpmv92CbLve8Xi3T9L6IAzVwnY2kp8T/9BOrlyNqHXorlH 0H6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=gN0OgzuqdNMVjhAbHf3DQ6NuXOfSgPMfa6XOy4wAB5E=; b=OD/a03VfmnG+Zv5Z37+81gsq5wQJ2u+rYcXr+EF/CP9A3uanRmLz5fGWr5AnpFO0nB kHIAC/80aPUBHkP1MTN704xWrP9ofYUqvOmgnRvkfRl5miiWWKyFkNRzzD2MVKjBppJp tXyw2DWfJK5P0ZubEjBTG3gqbddoeRWpLgCMQrv5V/lYrug0r/ncSMdmj9Us+bwJrDnD 7BjQ9BfrzRj2YDkUVjNY1tDBECIjuRTEJGmdTxPbpaDEpqtBafD65Qu/76yKFJCI90nM Fu1SDB0lgkAYUPdA46oQJtRsZd2SsA0ZV+iOmr2dO1NQqCSURhneyWbKGGqQ7buIUrNv OpmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QjCCsi4Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c7-20020a056a00008700b0057fdee40d48si13585852pfj.173.2023.02.06.21.03.03; Mon, 06 Feb 2023 21:03:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=QjCCsi4Z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230294AbjBGFC2 (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230156AbjBGFBy (ORCPT ); Tue, 7 Feb 2023 00:01:54 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F40DE5273; Mon, 6 Feb 2023 21:01:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746110; x=1707282110; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=HuUFjeqT6UwPE0HFZg/3PfgKeD3R8aChweTbYJKxkt4=; b=QjCCsi4ZUGb6//og4Z/aoHF0/yhqe5A+FtwDlMIglvBgQHi0cXi4wzEJ Jynw7Ln5n6tP7vtOiA4e2DI2FWRkstmy+dsWhzA/71LphXv9pMCfDhGaD jN6gha8DnEde9QjaRdiuhycLBUHnY+50YhmrAbWUMQb2japJ31Lx2hjTK d5OMLkg6eiMxfxj0fvOSMIhfGFyhOx3iDpk8QFH/lHSEYmvustbPosyCv wxUzq9aDWABZCda6HcHDI5i3Qk7iU5GEpZAXvuQ9/IZqdVrhbBLLIcSj5 OwEwn3QU+5YD7NmkU7qUhsQsqICtTxkcrDjEcSLJlRmRBhMoUsoXUrDFW w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625782" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625782" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657706" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657706" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:40 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 05/24] sched/core: Update the IPC class of the current task Date: Mon, 6 Feb 2023 21:10:46 -0800 Message-Id: <20230207051105.11575-6-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147245409623801?= X-GMAIL-MSGID: =?utf-8?q?1757147245409623801?= When supported, hardware monitors the instruction stream to classify the current task. Hence, at userspace tick, we are ready to read the most recent classification result for the current task. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Removed argument smt_siblings_idle from call to arch_ipcc_update(). * Used the new IPCC interfaces names. --- kernel/sched/core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 39d218a2f243..9f4e9cc16df8 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5567,6 +5567,9 @@ void scheduler_tick(bool user_tick) if (housekeeping_cpu(cpu, HK_TYPE_TICK)) arch_scale_freq_tick(); + if (sched_ipcc_enabled() && user_tick) + arch_update_ipcc(curr); + sched_clock_tick(); rq_lock(rq, &rf); From patchwork Tue Feb 7 05:10:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53674 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653050wrn; Mon, 6 Feb 2023 21:03:23 -0800 (PST) X-Google-Smtp-Source: AK7set8KQRvTGogZb2UmFlwK9MsGZv8yd6AILKKe/GbxEjM0Ueu5XRh13pdFHowu+DJ1t6E+uMmh X-Received: by 2002:a17:90a:f2d4:b0:230:d3a1:ba03 with SMTP id gt20-20020a17090af2d400b00230d3a1ba03mr2409547pjb.43.1675746203406; Mon, 06 Feb 2023 21:03:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746203; cv=none; d=google.com; s=arc-20160816; b=OZ+a96t3MxErANZCSyJjqVyl42UIWBkJl13vzXkkb71bkhv9C2eVnjLy+mn6GLxjEg Xry6SVOvcXront4z4lKrWqZ8Bhl62IVwrYAApz1Pi/L5g/2/hZ4aNCvvP2FX/v0/1D6y GXarfwRKQVh5OCuVuDt/RL3xbbHvOIVTFIuRii15L/o/lAO9alyQ8ju/nGXKRlilNlaI u6CIEZo8gUaqNFXGgN+jHX7DJetTmiQ3VRzqGXtWE/ZOUkOxY+61gK5n2rHdVtNRSmdv bqTpjuJLndsKONkpD6gsUbSY/6O3O8dS8Ufa+FV9lD5x+MORbqVJvHqEEylFGl/VcToa ZW9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=oiYpGJEy7oIkg3GAl6RWaDX7DmozZlsfr0aejHVKgn0=; b=HbBoL6b5OP77kL20w42Oimgby1DNzZRUzO/ADCxH1MfvSjeq28AMcWduYu1AQ/WDaU q8FV5rDUw9uohTXZjIUTWPIOT2P51eEzxxGFC6q52dSjW5KfQ8vuZaHTN6LnM/1wzQa8 LEaENEiOldIkFOlM8XCVoXiSvIl4ipgSTxayERRDpN/RQ23aqNNLJRnZ4AraLKSwdLNR 5D2XTlMeygJEWHuj4lBV9fh4BieKty8i661YCsZyFH4w6gVc7Xv2PiBIL+q/1AP24rQI rh7lapQDnSu7kQS7IL5KPE214KA6XkrQF3rrLtLi4oOTtvWuymQXZZ+SLsQCwOrcBFZL uwsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=aC0QiTkN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v202-20020a6361d3000000b004fb11698712si1311540pgb.332.2023.02.06.21.03.07; Mon, 06 Feb 2023 21:03:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=aC0QiTkN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230321AbjBGFCb (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229730AbjBGFBy (ORCPT ); Tue, 7 Feb 2023 00:01:54 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9462E55A2; Mon, 6 Feb 2023 21:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746111; x=1707282111; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=9U3HT2fPkvVUOBW50CYIJ3B+L4hO5S4Nc1uYuLdQPpY=; b=aC0QiTkNKtnMvNl23FvuS1tVrjynTfR9uEtl8VzdUW+9Y/2E5f2tORgs 414kjOfSowGsHPg/CX3gTgBzYlJk+5r4hOGMZeaWBMhmlisHG7SNRVK+K cNZUs0BhJC/oZM422gUvFhP9W9M7uQaZRw07TRlxEIE1evtZeCmlFSKim gSDNOYgWQxz2Sn1BCfbmAwQOZAXsY/58ONzAtIxFeQ6RS2fFK6O1un5Xv kojvIHxSnaZa+JTTsHXe7mCgbA16TVdyYKInn75Ub4XG7WIGfSehJaWrF U8xInG+XqJTd3lazXpDkwsaY/3H9uTXdU2ojsY2RiCkt7p6wZO6HCxnvH g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625795" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625795" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657713" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657713" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:41 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 06/24] sched/fair: Collect load-balancing stats for IPC classes Date: Mon, 6 Feb 2023 21:10:47 -0800 Message-Id: <20230207051105.11575-7-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147250957194350?= X-GMAIL-MSGID: =?utf-8?q?1757147250957194350?= When selecting a busiest scheduling group, the IPC class of the current task can be used to select between two scheduling groups of types asym_ packing or fully_busy that are otherwise identical. Compute the IPC class performance score for a scheduling group. It is the sum of the scores of the current tasks of all the runqueues. Also, keep track of the class of the task with the lowest IPC class score in the scheduling group. These two metrics will be used during idle load balancing to compute the current and the prospective IPC class score of a scheduling group. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Also excluded deadline and realtime tasks from IPCC stats. (Dietmar) * Also excluded tasks that cannot run on the destination CPU from the IPCC stats. * Folded struct sg_lb_ipcc_stats into struct sg_lb_stats. (Dietmar) * Reworded description sg_lb_stats::min_ipcc. (Ionela) * Handle errors of arch_get_ipcc_score(). (Ionela) Changes since v1: * Implemented cleanups and reworks from PeterZ. Thanks! * Used the new interface names. --- kernel/sched/fair.c | 61 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 0ada2d18b934..d773380a95b3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8897,6 +8897,11 @@ struct sg_lb_stats { unsigned int nr_numa_running; unsigned int nr_preferred_running; #endif +#ifdef CONFIG_IPC_CLASSES + unsigned long min_score; /* Min(score(rq->curr->ipcc)) */ + unsigned short min_ipcc; /* Class of the task with the minimum IPCC score in the rq */ + unsigned long sum_score; /* Sum(score(rq->curr->ipcc)) */ +#endif }; /* @@ -9240,6 +9245,59 @@ group_type group_classify(unsigned int imbalance_pct, return group_has_spare; } +#ifdef CONFIG_IPC_CLASSES +static void init_rq_ipcc_stats(struct sg_lb_stats *sgs) +{ + /* All IPCC stats have been set to zero in update_sg_lb_stats(). */ + sgs->min_score = ULONG_MAX; +} + +/* Called only if cpu_of(@rq) is not idle and has tasks running. */ +static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, + struct rq *rq) +{ + struct task_struct *curr; + unsigned short ipcc; + unsigned long score; + + if (!sched_ipcc_enabled()) + return; + + curr = rcu_dereference(rq->curr); + if (!curr || (curr->flags & PF_EXITING) || is_idle_task(curr) || + task_is_realtime(curr) || + !cpumask_test_cpu(dst_cpu, curr->cpus_ptr)) + return; + + ipcc = curr->ipcc; + score = arch_get_ipcc_score(ipcc, cpu_of(rq)); + + /* + * Ignore tasks with invalid scores. When finding the busiest group, we + * prefer those with higher sum_score. This group will not be selected. + */ + if (IS_ERR_VALUE(score)) + return; + + sgs->sum_score += score; + + if (score < sgs->min_score) { + sgs->min_score = score; + sgs->min_ipcc = ipcc; + } +} + +#else /* CONFIG_IPC_CLASSES */ +static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, + struct rq *rq) +{ +} + +static void init_rq_ipcc_stats(struct sg_lb_stats *sgs) +{ +} +#endif /* CONFIG_IPC_CLASSES */ + /** * asym_smt_can_pull_tasks - Check whether the load balancing CPU can pull tasks * @dst_cpu: Destination CPU of the load balancing @@ -9332,6 +9390,7 @@ static inline void update_sg_lb_stats(struct lb_env *env, int i, nr_running, local_group; memset(sgs, 0, sizeof(*sgs)); + init_rq_ipcc_stats(sgs); local_group = group == sds->local; @@ -9381,6 +9440,8 @@ static inline void update_sg_lb_stats(struct lb_env *env, if (sgs->group_misfit_task_load < load) sgs->group_misfit_task_load = load; } + + update_sg_lb_ipcc_stats(env->dst_cpu, sgs, rq); } sgs->group_capacity = group->sgc->capacity; From patchwork Tue Feb 7 05:10:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53675 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653061wrn; Mon, 6 Feb 2023 21:03:25 -0800 (PST) X-Google-Smtp-Source: AK7set+iQ5cAjZRmyJQQ5IqGiZvyePr7IvagLpKs/9e9KEGlfDCuWwKbJnKl5miowho3kf2yB4A4 X-Received: by 2002:a05:6a21:29ca:b0:bd:3a3:98cb with SMTP id tv10-20020a056a2129ca00b000bd03a398cbmr1578758pzb.1.1675746205656; Mon, 06 Feb 2023 21:03:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746205; cv=none; d=google.com; s=arc-20160816; b=gxa4ZNfSAHwI0AAwsd4Wd4stOwggeievK5cbIv7IJCpUz9MrW3kSmlXR11VNwwaMK4 /RgB3uEipzdUWlC7JdmP9diid1ysVH1khjAMrMaeuvSfinmBpE/HkPdIg6wdxCV9Dacb uqPgTrsG9ddTaS9XfUT0UmcntIG78WU3nnBZpDAd8a8WLXgQD0HNqfYcoQTlBGLTR7zz KR2kull+bJQRIN2lp7mVv4xsbqk137F5tQ1M66e7CNijLFUC+4Q2lDs2QHMqxSmcYHjy 1WXIaTCSnFcRemzfc9p5YB2gBbwzPZQN+zokYGPvQF+tiloaNAdQesetoqRj5OGdsboT Slvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=h6EaEXZh2HGlXeSYetNBRdT6CNqxqR5lBLc/2fdcdzU=; b=Bmw6cK9Z/5qNNzC8uK4HZzoL367rt6dtgn6kLkSyYOBqcgxLCHcF2Nau9B0xVFtOMy ToCUlndPANpQruaDdDQdwoRv+AUvLh1v+9vpPn0EYpFGmBHngfs1rxr52QqPl1KqTzWf N5fksGHjs/BHY3iKBg4TQYGi+09KmJJGo0x0xCWGybt7wO68T4QKLyKjB48626Gk7qnO 10MsjSHvHHRKmw1Sj4RrqcN5aFwxXfXBuOoi2bFc6VrTQN74ufAVrRdNIaKchjmZmZdV V2FC55XkRrn5TQqVvF8/YD0QubmISjNZOibLxSq49zCLfo64r0cSSTjQdmSn9bmZtdSV R7pQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Sv+DsJxd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i16-20020a17090332d000b001963440c24bsi15870159plr.254.2023.02.06.21.03.11; Mon, 06 Feb 2023 21:03:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Sv+DsJxd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230328AbjBGFCe (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230075AbjBGFBz (ORCPT ); Tue, 7 Feb 2023 00:01:55 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6472E6A56; Mon, 6 Feb 2023 21:01:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746112; x=1707282112; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=s23DDV+a9X4aP0kA4U7st9P3jUXY+WdjiUqKmcmbNZo=; b=Sv+DsJxd21ekmcvCS9JQXGV8feTCQx8c9HGtN/ZIuR/YpRcDacT2EfAs +2YDbamk127Uqfu//18/Gl2XGmohoQ7ofFyyrMfjieT6oaeYNPzCQ8Yn6 cU9RM1/TqLOQvasG4/JTjGOb85/DIBOA+3DJEWR47yW9LOlK0boUv/doM JOGbwP9WuxFggbMK9f5FiqrtWObLgFVtdWnXL7EzWWWWnNnYMAWwDaSUM wVLmDrw3wVGQd8cCNjL36Ch0+031pQb5Cc8uNaR4sstEY6/0uIXk3LE2K 2zMWPmIWc+UU/fhG73oapQTCqfQNu35NM7g70AYpAJiIaZOlDAWXVM1sB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625809" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625809" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657717" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657717" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:42 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 07/24] sched/fair: Compute IPC class scores for load balancing Date: Mon, 6 Feb 2023 21:10:48 -0800 Message-Id: <20230207051105.11575-8-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147253360771055?= X-GMAIL-MSGID: =?utf-8?q?1757147253360771055?= Compute the joint total (both current and prospective) IPC class score of a scheduling group and the local scheduling group. These IPCC statistics are used during idle load balancing. The candidate scheduling group will have one fewer busy CPU after load balancing. This observation is important for cores with SMT support. The IPCC score of scheduling groups composed of SMT siblings needs to consider that the siblings share CPU resources. When computing the total IPCC score of the scheduling group, divide score of each sibling by the number of busy siblings. Collect IPCC statistics for asym_packing and fully_busy scheduling groups. When picking a busiest group, they are used to break ties between otherwise identical groups. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Also collect IPCC stats for fully_busy sched groups. * Restrict use of IPCC stats to SD_ASYM_PACKING. (Ionela) * Handle errors of arch_get_ipcc_score(). (Ionela) Changes since v1: * Implemented cleanups and reworks from PeterZ. I took all his suggestions, except the computation of the IPC score before and after load balancing. We are computing not the average score, but the *total*. * Check for the SD_SHARE_CPUCAPACITY to compute the throughput of the SMT siblings of a physical core. * Used the new interface names. * Reworded commit message for clarity. --- kernel/sched/fair.c | 68 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d773380a95b3..b6165aa8a376 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8901,6 +8901,8 @@ struct sg_lb_stats { unsigned long min_score; /* Min(score(rq->curr->ipcc)) */ unsigned short min_ipcc; /* Class of the task with the minimum IPCC score in the rq */ unsigned long sum_score; /* Sum(score(rq->curr->ipcc)) */ + long ipcc_score_after; /* Prospective IPCC score after load balancing */ + unsigned long ipcc_score_before; /* IPCC score before load balancing */ #endif }; @@ -9287,6 +9289,62 @@ static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, } } +static void update_sg_lb_stats_scores(struct sg_lb_stats *sgs, + struct sched_group *sg, + struct lb_env *env) +{ + unsigned long score_on_dst_cpu, before; + int busy_cpus; + long after; + + if (!sched_ipcc_enabled()) + return; + + /* + * IPCC scores are only useful during idle load balancing. For now, + * only asym_packing uses IPCC scores. + */ + if (!(env->sd->flags & SD_ASYM_PACKING) || + env->idle == CPU_NOT_IDLE) + return; + + /* + * IPCC scores are used to break ties only between these types of + * groups. + */ + if (sgs->group_type != group_fully_busy && + sgs->group_type != group_asym_packing) + return; + + busy_cpus = sgs->group_weight - sgs->idle_cpus; + + /* No busy CPUs in the group. No tasks to move. */ + if (!busy_cpus) + return; + + score_on_dst_cpu = arch_get_ipcc_score(sgs->min_ipcc, env->dst_cpu); + + /* + * Do not use IPC scores. sgs::ipcc_score_{after, before} will be zero + * and not used. + */ + if (IS_ERR_VALUE(score_on_dst_cpu)) + return; + + before = sgs->sum_score; + after = before - sgs->min_score; + + /* SMT siblings share throughput. */ + if (busy_cpus > 1 && sg->flags & SD_SHARE_CPUCAPACITY) { + before /= busy_cpus; + /* One sibling will become idle after load balance. */ + after /= busy_cpus - 1; + } + + sgs->ipcc_score_after = after + score_on_dst_cpu; + sgs->ipcc_score_before = before; +} + #else /* CONFIG_IPC_CLASSES */ static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, struct rq *rq) @@ -9296,6 +9354,13 @@ static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, static void init_rq_ipcc_stats(struct sg_lb_stats *sgs) { } + +static void update_sg_lb_stats_scores(struct sg_lb_stats *sgs, + struct sched_group *sg, + struct lb_env *env) +{ +} + #endif /* CONFIG_IPC_CLASSES */ /** @@ -9457,6 +9522,9 @@ static inline void update_sg_lb_stats(struct lb_env *env, sgs->group_type = group_classify(env->sd->imbalance_pct, group, sgs); + if (!local_group) + update_sg_lb_stats_scores(sgs, group, env); + /* Computing avg_load makes sense only when group is overloaded */ if (sgs->group_type == group_overloaded) sgs->avg_load = (sgs->group_load * SCHED_CAPACITY_SCALE) / From patchwork Tue Feb 7 05:10:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53677 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653080wrn; Mon, 6 Feb 2023 21:03:30 -0800 (PST) X-Google-Smtp-Source: AK7set8csunDe//oHt7gKX+uCuR6ogbsjboP+uyShs8/q7IcwOlgaw486zBifDIe5q918FoCLCvQ X-Received: by 2002:a17:90a:195:b0:230:a427:291f with SMTP id 21-20020a17090a019500b00230a427291fmr2508344pjc.13.1675746210185; Mon, 06 Feb 2023 21:03:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746210; cv=none; d=google.com; s=arc-20160816; b=vS2IlRn53dsoW8OKcIwRhOPusj5RxMmiQ1c7Qs8hj4OS1i4bYw7Y3t3Hz6uGN4S8jt grOvvQyRpMUvjvgWxMCg92Ib9Ywc1z8p/1JS0GxUBOe7RZkN6uOO6ugAEshg2gO1hooE Xo9zWancEy8CSrGBG6wQSob/wBuw2XW/6zapgbryENMDl7qkHqqF0i3F1Fsm9hGXdKBJ ehkcx3Na5R5irZtQvyXqb5uCi9inxLpkWSyq1iTiztYLIJIFc0PMZOzF/NaAkVCATUoy h8A3gNRx8wTvclSeq8F0zwtICgFvX1DJBAo4l0FBTLqy+E5zfUrxkHvv9+4fXQEI+F3t dTHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=AN195jAoezokXtzLvgQ6OoMD+EwBl+BrAVTyhOnYl6E=; b=0cM7YfQSktWdew53+TCtBTlv7pi/zuik3jZah8XXu3HnUQLrBxq0YHkbsXfl2OSHqJ VJsRItTyLG8v9/Zcp/OZ6rXOkXcIl1rDRy7MjiDl1TNgXBBoWypjvNUiWAkrDHcyowA/ p/buC0+3er1NHs/uSEsQB2JQkuPZiRxdUePqBQRwk26buV4n3iDjdetwy4Mp47zV2+9J pS3Xz0t7FZyFOPW0qA/vl0Z1dkgyZGvNUKn90B2/gfoPwRr5IZDqUjXf4+VX6sl+sNO/ X3Yk4hV2C4n0cPTfY3/AZ9JSwNsZRYvZNKrfGlsczaWHqppCWhXJ2HGosUq8Y1RfE/FO y/Ow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=RLeOi2k2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e125-20020a636983000000b004facb801aedsi7359907pgc.378.2023.02.06.21.03.16; Mon, 06 Feb 2023 21:03:30 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=RLeOi2k2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230351AbjBGFCm (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229743AbjBGFB4 (ORCPT ); Tue, 7 Feb 2023 00:01:56 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D302A901E; Mon, 6 Feb 2023 21:01:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746113; x=1707282113; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=aOX48G3IsFHDTBbZEmk9IpTBJOBpnREOfkswBIUTlho=; b=RLeOi2k2COE/zVoVK2CzAPc1wrUepDlkJUycQaQHSSR1n/8qq/0sJlNZ GPYShstWgk8b5Hy9AmRz+aBmQh7NA+pz4GGd8QkkbSKGmfwN+UCzrk526 i/gGkySJivGiS2ha1Gc6hmdZHYqWfhv3/XF9u3+gTwQBtxXxxo0tI2Yff 6JZB2SHgXRlGAzLRUwGAGtLZIaD+iJG8uaQeCc+d5Z+Lrm2hE//h9iFuO NSr7jbawJR/1fYl3d8+eUnHSncWS8BbbHppBztLlLekXu+zeHxCVq/wlX X8NQ33ekK3pVM5e5nGNyIPufIMnbY078qGGFWDIVXVqrur40i825mAhCv A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625820" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625820" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657720" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657720" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:42 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 08/24] sched/fair: Use IPCC stats to break ties between asym_packing sched groups Date: Mon, 6 Feb 2023 21:10:49 -0800 Message-Id: <20230207051105.11575-9-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147257951634364?= X-GMAIL-MSGID: =?utf-8?q?1757147257951634364?= As it iterates, update_sd_pick_busiest() keeps on selecting as busiest sched groups of identical priority. Since both groups have the same priority, either group is a good choice. The IPCC statistics provide a measure of the throughput before and after load balance. Use them to pick a busiest scheduling group from otherwise identical asym_packing scheduling groups. Pick as busiest the scheduling group that yields a higher IPCC score after load balancing. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Added a comment to clarify why sched_asym_prefer() needs a tie breaker only in update_sd_pick_busiest(). (PeterZ) * Renamed functions for accuracy: sched_asym_class_prefer() >> sched_asym_ipcc_prefer() sched_asym_class_pick() >> sched_asym_ipcc_pick() * Reworded commit message for clarity. --- kernel/sched/fair.c | 72 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b6165aa8a376..841927b9b192 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9345,6 +9345,60 @@ static void update_sg_lb_stats_scores(struct sg_lb_stats *sgs, sgs->ipcc_score_before = before; } +/** + * sched_asym_ipcc_prefer - Select a sched group based on its IPCC score + * @a: Load balancing statistics of a sched group + * @b: Load balancing statistics of a second sched group + * + * Returns: true if @a has a higher IPCC score than @b after load balance. + * False otherwise. + */ +static bool sched_asym_ipcc_prefer(struct sg_lb_stats *a, + struct sg_lb_stats *b) +{ + if (!sched_ipcc_enabled()) + return false; + + /* @a increases overall throughput after load balance. */ + if (a->ipcc_score_after > b->ipcc_score_after) + return true; + + /* + * If @a and @b yield the same overall throughput, pick @a if + * its current throughput is lower than that of @b. + */ + if (a->ipcc_score_after == b->ipcc_score_after) + return a->ipcc_score_before < b->ipcc_score_before; + + return false; +} + +/** + * sched_asym_ipcc_pick - Select a sched group based on its IPCC score + * @a: A scheduling group + * @b: A second scheduling group + * @a_stats: Load balancing statistics of @a + * @b_stats: Load balancing statistics of @b + * + * Returns: true if @a has the same priority and @a has tasks with IPC classes + * that yield higher overall throughput after load balance. False otherwise. + */ +static bool sched_asym_ipcc_pick(struct sched_group *a, + struct sched_group *b, + struct sg_lb_stats *a_stats, + struct sg_lb_stats *b_stats) +{ + /* + * Only use the class-specific preference selection if both sched + * groups have the same priority. + */ + if (arch_asym_cpu_priority(a->asym_prefer_cpu) != + arch_asym_cpu_priority(b->asym_prefer_cpu)) + return false; + + return sched_asym_ipcc_prefer(a_stats, b_stats); +} + #else /* CONFIG_IPC_CLASSES */ static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, struct rq *rq) @@ -9361,6 +9415,14 @@ static void update_sg_lb_stats_scores(struct sg_lb_stats *sgs, { } +static bool sched_asym_ipcc_pick(struct sched_group *a, + struct sched_group *b, + struct sg_lb_stats *a_stats, + struct sg_lb_stats *b_stats) +{ + return false; +} + #endif /* CONFIG_IPC_CLASSES */ /** @@ -9596,6 +9658,16 @@ static bool update_sd_pick_busiest(struct lb_env *env, /* Prefer to move from lowest priority CPU's work */ if (sched_asym_prefer(sg->asym_prefer_cpu, sds->busiest->asym_prefer_cpu)) return false; + + /* + * Unlike other callers of sched_asym_prefer(), here both @sg + * and @sds::busiest have tasks running. When they have equal + * priority, their IPC class scores can be used to select a + * better busiest. + */ + if (sched_asym_ipcc_pick(sds->busiest, sg, &sds->busiest_stat, sgs)) + return false; + break; case group_misfit_task: From patchwork Tue Feb 7 05:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53676 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653068wrn; Mon, 6 Feb 2023 21:03:26 -0800 (PST) X-Google-Smtp-Source: AK7set/fDQoHtWkhvEKeZ++vUsG/evq8x580Ny32b4bW85fdzC7mR5tKWXfLaTwr6CP+2zKY/D6b X-Received: by 2002:a62:1b05:0:b0:593:92b3:b757 with SMTP id b5-20020a621b05000000b0059392b3b757mr2005983pfb.3.1675746206669; Mon, 06 Feb 2023 21:03:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746206; cv=none; d=google.com; s=arc-20160816; b=sHq9id39n3E3tNiXAYj8Cu7y+E0+IvkBVJxDYakQYrEGIfWnuK5jXyj6xpzVVS9a7c dVLdJzy6XYzbf1IOdym2BQJbvKO19asKEaHCnLHBk5fVILUh26N3MDmjJ4YcPniS0vFK +O6319QHxenv20q0ZLj+QdfNlAj3qs6ojZPauPooqqnsgHorinEsODZOWx+ZJlzJm21m kb0RcD9tGkz584RqUUvFCCkUjteck1wmlLU3GVNwWoFXx9xYhm4i7rOoIDHtJHxpS0Sf DR9beVopDVEv3LQJNbyghZnAhBOZ9TgbzGH4gyV5ymfoMZ244njLvyYmDkgsAbAXxwc6 +zhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=ZzrEy83LY5oTNovx4i2my1dVT3cKurU/QzvH4Lq0/po=; b=Xn1viPXxH5ICcDDKzS4awPmLots5JDSKML0akhYlCxyMJn9e/Op7IwnrEGr/R2aGx3 BZqGnjEM+8YUwvfgYnZXW0j/InCSnAe3yGo0UBYznl2G/obPf+21AWkQeUFFSE8TabS4 swjdsyZLVj81YZm4i5DpSnz0DxPR/gnDFM6nOosZ+ToL4deuqdFFpY8ZqW5ebzqO+lvW MdYEfEPoqKK0NEuLzOJTvY5i1uldQnPKThhjw26eb90l8FPFPk2/XU1NJE/JfnBrhXPA l+m+zMkdJMTI+OPqNoeZJvWeSZhAlzCHt7URc1y6bE7Z30eRgRd2E5QkRCiFfPdmD/qC ZpcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="dhD9I/7m"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r29-20020aa79edd000000b00592591f1972si13255304pfq.64.2023.02.06.21.03.12; Mon, 06 Feb 2023 21:03:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="dhD9I/7m"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230340AbjBGFCi (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230161AbjBGFBz (ORCPT ); Tue, 7 Feb 2023 00:01:55 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D43E4901F; Mon, 6 Feb 2023 21:01:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746113; x=1707282113; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=XBeyrfeapZSrbuAay+AhGt8TNoIKKt2H5iHcCn0WtSs=; b=dhD9I/7mLVj2Pggklg/grywvghBoSKaHwulbTuRAAHYuSm415n4FAEfV tYcSr+xDP+sso3ijlWudcK+qfujjfWET/RZHe5mWc7Rcarq7kwPwyaejx 47TbMWIgSkkkWrwRxyQj36YO2mQVrMudO8wQ6VkRhb3nBeHdigjm46e/P UcpJL10fI5HT6suN3NwaKLGJj+pvhcYvFoR/acBhYiGK6LiYaVum9sCkr 0eA0Q3jPQDP1zUSOpT3mVTJ6y95Mj7Ri7Ecd6mFaDJPs4JiLYb/8bPE2h Gacexu2m7mQUSjj3I1fzWPF+F+YLmpc4k+EivlWM2dlB1gDdZCQDiMkc1 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625837" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625837" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657723" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657723" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:42 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 09/24] sched/fair: Use IPCC stats to break ties between fully_busy SMT groups Date: Mon, 6 Feb 2023 21:10:50 -0800 Message-Id: <20230207051105.11575-10-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147253970345175?= X-GMAIL-MSGID: =?utf-8?q?1757147253970345175?= IPCC statistics are used during idle load balancing. After balancing one of the siblings of an SMT core will become idle. The rest of the busy siblings will enjoy increased throughput. The IPCC statistics provide a measure of the increased throughput. Use them to pick a busiest group from otherwise identical fully_busy scheduling groups (of which the avg_load is equal - and zero). Using IPCC scores to break ties with non-SMT fully_busy sched groups is not necessary. SMT sched groups always need more help. Add a stub sched_asym_ipcc_prefer() for !CONFIG_IPC_CLASSES. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 841927b9b192..72d88270b320 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9415,6 +9415,12 @@ static void update_sg_lb_stats_scores(struct sg_lb_stats *sgs, { } +static bool sched_asym_ipcc_prefer(struct sg_lb_stats *a, + struct sg_lb_stats *b) +{ + return false; +} + static bool sched_asym_ipcc_pick(struct sched_group *a, struct sched_group *b, struct sg_lb_stats *a_stats, @@ -9698,10 +9704,21 @@ static bool update_sd_pick_busiest(struct lb_env *env, if (sgs->avg_load == busiest->avg_load) { /* * SMT sched groups need more help than non-SMT groups. - * If @sg happens to also be SMT, either choice is good. */ - if (sds->busiest->flags & SD_SHARE_CPUCAPACITY) - return false; + if (sds->busiest->flags & SD_SHARE_CPUCAPACITY) { + if (!(sg->flags & SD_SHARE_CPUCAPACITY)) + return false; + + /* + * Between two SMT groups, use IPCC scores to pick the + * one that would improve throughput the most (only + * asym_packing uses IPCC scores for now). + */ + if (sched_ipcc_enabled() && + env->sd->flags & SD_ASYM_PACKING && + sched_asym_ipcc_prefer(busiest, sgs)) + return false; + } } break; From patchwork Tue Feb 7 05:10:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53678 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653092wrn; Mon, 6 Feb 2023 21:03:31 -0800 (PST) X-Google-Smtp-Source: AK7set/gnXxsbYLIoo4cwSW3XXU8GSzmF5zLK1ZwMBCS7eoXN5tWzyt8gvnRj+pPJiE8isMR+V5O X-Received: by 2002:a17:902:b10b:b0:199:1924:41b with SMTP id q11-20020a170902b10b00b001991924041bmr1155659plr.41.1675746211503; Mon, 06 Feb 2023 21:03:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746211; cv=none; d=google.com; s=arc-20160816; b=A1PIj3dt5aVk+p3nHyfnV5ZvqDCHQjos5+G1xaqK9T7UzKMrl3b01WJADGq7QzWWXH XH7WzQkRR1SJx+lj9PPDh1UmiuZfZkbZHZ075vo1tWE/r4XdW8ePX1NAIcNOkkHrmd53 eqha0tTSDCOkErPxdDvFKJ2CUxuESqv4QuZE7kqpqh91LoEyN/WUgIJ1K4WpAvRcrHc9 ZG/lP/W1JeiNRwnl0HX7XYIeftkSuI/DT4iQRKdFOanlaQVhe0LOsT2fiupLAFzeoNjA 8g+2dxY4oaGhLl3JK+CFHODymAKIBHx/GfqqXMDlft4+ytMyBP+JQCXNkGSZ9ndhqfbp CtVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=74cpbkAmGv/ceW2TWGIHICgO2MkhxzMcEovLSm9/nP4=; b=m9YSgtmcjKfb0EuZGnzPJtW17+ZxViQAUZ950xEeEWOizXjcOw2eiIYvEaZcHB0teD 7MsGjfl5aYZJHQmSStZDkIjyQQ4ioRunBp3IvBItzUCAsIUffiGK6wbjFiT8viGgOgZ3 A0qd5c3rr3aEMVQn/2MCKh9+A1ALzvbQqsVnAS9BwRoAGM++ot1P4jBH4qmEcd/qqekQ rj9LJTSKXLi2gN/Uu09vjHOi6hBQRphcheFBGF67EQLSsDJfupkvj1XmfrAmR/KUZfwu Qa9TeNRAjfInKraI1wVk48OE2k7ptXnIz3ZCvACQy1VhRkx5xMLPHUlbw0pOcllOSIcw eJPg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ZGnjxVpa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jw15-20020a170903278f00b0019735fc6a65si1375706plb.129.2023.02.06.21.03.17; Mon, 06 Feb 2023 21:03:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ZGnjxVpa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230356AbjBGFCr (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229843AbjBGFB4 (ORCPT ); Tue, 7 Feb 2023 00:01:56 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E45BC643; Mon, 6 Feb 2023 21:01:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746114; x=1707282114; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=BrjwTRAQcpfuOeAV7Rg7Zm/X0IgwzgEOiAKjPiNKh1M=; b=ZGnjxVpa8iBCrxYykm4azWnUE82ILNVXXrmb3XL1r1GVuqN/eJz9Fv6+ BG5KtJsW2kvHDhW8u15yamKhtbsHFWIPtI5bZy6b62TNAOHUIfg8BTl6q U+sxFHy32zkOO96x2ls6L01fxLkJfrLFYAAqUmWnvEcC3BtRfXWzNNNnD wP+BBoG13YwvVkI85fwTRqaZp3LSWQKqYSjGYX+zBXadW27ShXC/Q0wv5 6eEhN8XbB0PTIvYMag6ip7fv7IhxmdWx61s09inzXdO9nfMYk/YOUB8lI R8WIEstc3TnFW/izgAh1XJg4cMWsQLMitCr/4HASdp/tYqyQKsmegaFA6 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625849" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625849" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657730" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657730" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:42 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 10/24] sched/fair: Use IPCC scores to select a busiest runqueue Date: Mon, 6 Feb 2023 21:10:51 -0800 Message-Id: <20230207051105.11575-11-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147259622651912?= X-GMAIL-MSGID: =?utf-8?q?1757147259622651912?= For two runqueues of equal priority and equal number of running of tasks, select the one whose current task would have the highest IPC class score if placed on the destination CPU. For now, use IPCC scores only for scheduling domains with the SD_ASYM_PACKING flag. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Only use IPCC scores to break ties if the sched domain uses asym_packing. (Ionela) * Handle errors of arch_get_ipcc_score(). (Ionela) Changes since v1: * Fixed a bug when selecting a busiest runqueue: when comparing two runqueues with equal nr_running, we must compute the IPCC score delta of both. * Renamed local variables to improve the layout of the code block. (PeterZ) * Used the new interface names. --- kernel/sched/fair.c | 64 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 72d88270b320..d3c22dc145f7 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9399,6 +9399,37 @@ static bool sched_asym_ipcc_pick(struct sched_group *a, return sched_asym_ipcc_prefer(a_stats, b_stats); } +/** + * ipcc_score_delta - Get the IPCC score delta wrt the load balance's dst_cpu + * @p: A task + * @env: Load balancing environment + * + * Returns: The IPCC score delta that @p would get if placed in the destination + * CPU of @env. LONG_MIN to indicate that the delta should not be used. + */ +static long ipcc_score_delta(struct task_struct *p, struct lb_env *env) +{ + unsigned long score_src, score_dst; + unsigned short ipcc = p->ipcc; + + if (!sched_ipcc_enabled()) + return LONG_MIN; + + /* Only asym_packing uses IPCC scores at the moment. */ + if (!(env->sd->flags & SD_ASYM_PACKING)) + return LONG_MIN; + + score_dst = arch_get_ipcc_score(ipcc, env->dst_cpu); + if (IS_ERR_VALUE(score_dst)) + return LONG_MIN; + + score_src = arch_get_ipcc_score(ipcc, task_cpu(p)); + if (IS_ERR_VALUE(score_src)) + return LONG_MIN; + + return score_dst - score_src; +} + #else /* CONFIG_IPC_CLASSES */ static void update_sg_lb_ipcc_stats(int dst_cpu, struct sg_lb_stats *sgs, struct rq *rq) @@ -9429,6 +9460,11 @@ static bool sched_asym_ipcc_pick(struct sched_group *a, return false; } +static long ipcc_score_delta(struct task_struct *p, struct lb_env *env) +{ + return LONG_MIN; +} + #endif /* CONFIG_IPC_CLASSES */ /** @@ -10589,6 +10625,7 @@ static struct rq *find_busiest_queue(struct lb_env *env, { struct rq *busiest = NULL, *rq; unsigned long busiest_util = 0, busiest_load = 0, busiest_capacity = 1; + long busiest_ipcc_delta = LONG_MIN; unsigned int busiest_nr = 0; int i; @@ -10705,8 +10742,35 @@ static struct rq *find_busiest_queue(struct lb_env *env, case migrate_task: if (busiest_nr < nr_running) { + struct task_struct *curr; + busiest_nr = nr_running; busiest = rq; + + /* + * Remember the IPCC score delta of busiest::curr. + * We may need it to break a tie with other queues + * with equal nr_running. + */ + curr = rcu_dereference(busiest->curr); + busiest_ipcc_delta = ipcc_score_delta(curr, env); + /* + * If rq and busiest have the same number of running + * tasks and IPC classes are supported, pick rq if doing + * so would give rq::curr a bigger IPC boost on dst_cpu. + */ + } else if (busiest_nr == nr_running) { + struct task_struct *curr; + long delta; + + curr = rcu_dereference(rq->curr); + delta = ipcc_score_delta(curr, env); + + if (busiest_ipcc_delta < delta) { + busiest_ipcc_delta = delta; + busiest_nr = nr_running; + busiest = rq; + } } break; From patchwork Tue Feb 7 05:10:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53680 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653165wrn; Mon, 6 Feb 2023 21:03:40 -0800 (PST) X-Google-Smtp-Source: AK7set8mIxW74/IYUgDSM6RU1aoRiOldlLbtIXxYnXkVc1qWYhCn9iIgbO25eJNyY6BuiS1P+bBI X-Received: by 2002:a05:6a20:e412:b0:be:96d4:83af with SMTP id nh18-20020a056a20e41200b000be96d483afmr1533322pzb.50.1675746220596; Mon, 06 Feb 2023 21:03:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746220; cv=none; d=google.com; s=arc-20160816; b=ZURzq7OeNIMAMknqgxxPkvLBbMVjqPypvxMKheXPFF8A35BnqLar1mcP3G9nsYmvxc nnEwbMtiq5c/xBgS4hsgjUq/T7N1CK4/Ed22SCglCayl+tYVlsF0sw/tY14cDfh/19b2 FcvVb/tVtSjVwiuSVK4fUyZ7y0dEWPLgOVTeMVSMjsYmzlufY+g0ezyRUyKLMUzUcEi6 sizO1LUhe6fNG/4Ri7QS7MWhiCDgndidEJohmY0pX7/62yyQHpmoqciHi86cpfLiRg4I 3z8TDQikr4uKeOgpqVLCJaYE5TyqeV5JJl7GCZoBswlDfKgoYAisEBqp4G85tmxZUKmy utcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=RfqtC3vidJVWSx1599f/CKQEJTYwRuMlalQ7ccFCtjo=; b=EsnrHulKjR8lcnDh6e3ZezU9c3Z3zylwZxN1laZaLqplTPw6jf8kMdU7FLE790HGJ9 mNRUvITnAu/k/I3AYXQ3Yav1PrRIcHPKna+4KrvAv1mDtNhGRopVQMLYwdpk+8GQvfRW tnIyfmoXNZ8VbDCdd7wYFLZ30jLA4SsaD7Sxr75fZhrJqx+I28qeCJissPGqbNWwRi0k Z1envc5TEmidClmGYrcfeakQHXVWE+03i405Qv1Mmhthd02W0FU4ha232/rwZ+xEskln gTL67/KTj7WFXLxmbS/TfCXw5pl+YdOx6SbrDCCVm//QfodUW+vzEgTSC/f/+fivWfP0 lt3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KgwOpYS9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k9-20020a63ab49000000b004ad057fcf5bsi13977888pgp.728.2023.02.06.21.03.27; Mon, 06 Feb 2023 21:03:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KgwOpYS9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230390AbjBGFCz (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229878AbjBGFB5 (ORCPT ); Tue, 7 Feb 2023 00:01:57 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 901C7C678; Mon, 6 Feb 2023 21:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746115; x=1707282115; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=d7YOpMlf1P1ORwPsFp8Wf2z9+WUpcXCD+VCa+NWGzJ0=; b=KgwOpYS9wwI5sG+q542pWz5u/vqrl00gdbipfL2/6e4zuRlSjEa2a1FL EByuNyYUqIIm1c1vhNPO/wgqf+J91afi2g6JOobXDlY4H+cSFYwbpilRE k2W035GmwVnSMmxflsN0H6BOW+8rw4OyOtxAf4PArdcu1YtuXJQjLuDxU UdfT1ID5O/f8u7Lo1tBAdnH3XwyjQoLIOC1CwkU7FvRHl+99J5lfaDCKx KinJoe/DUXiPwN55nUT5Qw8qoJhb7j5HVmHuNplFkU+0dkhqn8Rc+jMg8 msjsuc6nY5h3tVppu8rYD5ZtiImDWUCq9i95V9D/HTtapQu0W+SMGgVrg Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625861" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625861" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657733" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657733" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:43 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 11/24] thermal: intel: hfi: Introduce Intel Thread Director classes Date: Mon, 6 Feb 2023 21:10:52 -0800 Message-Id: <20230207051105.11575-12-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147269395496967?= X-GMAIL-MSGID: =?utf-8?q?1757147269395496967?= On Intel hybrid parts, each type of CPU has specific performance and energy efficiency capabilities. The Intel Thread Director technology extends the Hardware Feedback Interface (HFI) to provide performance and energy efficiency data for advanced classes of instructions. Add support to parse per-class capabilities. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri Acked-by: Rafael J. Wysocki --- Changes since v2: * None Changes since v1: * Removed a now obsolete comment. --- drivers/thermal/intel/intel_hfi.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 6e604bda2b93..2527ae3836c7 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -77,7 +77,7 @@ union cpuid6_edx { * @ee_cap: Energy efficiency capability * * Capabilities of a logical processor in the HFI table. These capabilities are - * unitless. + * unitless and specific to each HFI class. */ struct hfi_cpu_data { u8 perf_cap; @@ -89,7 +89,8 @@ struct hfi_cpu_data { * @perf_updated: Hardware updated performance capabilities * @ee_updated: Hardware updated energy efficiency capabilities * - * Properties of the data in an HFI table. + * Properties of the data in an HFI table. There exists one header per each + * HFI class. */ struct hfi_hdr { u8 perf_updated; @@ -127,16 +128,21 @@ struct hfi_instance { /** * struct hfi_features - Supported HFI features + * @nr_classes: Number of classes supported * @nr_table_pages: Size of the HFI table in 4KB pages * @cpu_stride: Stride size to locate the capability data of a logical * processor within the table (i.e., row stride) + * @class_stride: Stride size to locate a class within the capability + * data of a logical processor or the HFI table header * @hdr_size: Size of the table header * * Parameters and supported features that are common to all HFI instances */ struct hfi_features { + unsigned int nr_classes; size_t nr_table_pages; unsigned int cpu_stride; + unsigned int class_stride; unsigned int hdr_size; }; @@ -333,8 +339,8 @@ static void init_hfi_cpu_index(struct hfi_cpu_info *info) } /* - * The format of the HFI table depends on the number of capabilities that the - * hardware supports. Keep a data structure to navigate the table. + * The format of the HFI table depends on the number of capabilities and classes + * that the hardware supports. Keep a data structure to navigate the table. */ static void init_hfi_instance(struct hfi_instance *hfi_instance) { @@ -515,18 +521,30 @@ static __init int hfi_parse_features(void) /* The number of 4KB pages required by the table */ hfi_features.nr_table_pages = edx.split.table_pages + 1; + /* + * Capability fields of an HFI class are grouped together. Classes are + * contiguous in memory. Hence, use the number of supported features to + * locate a specific class. + */ + hfi_features.class_stride = nr_capabilities; + + /* For now, use only one class of the HFI table */ + hfi_features.nr_classes = 1; + /* * The header contains change indications for each supported feature. * The size of the table header is rounded up to be a multiple of 8 * bytes. */ - hfi_features.hdr_size = DIV_ROUND_UP(nr_capabilities, 8) * 8; + hfi_features.hdr_size = DIV_ROUND_UP(nr_capabilities * + hfi_features.nr_classes, 8) * 8; /* * Data of each logical processor is also rounded up to be a multiple * of 8 bytes. */ - hfi_features.cpu_stride = DIV_ROUND_UP(nr_capabilities, 8) * 8; + hfi_features.cpu_stride = DIV_ROUND_UP(nr_capabilities * + hfi_features.nr_classes, 8) * 8; return 0; } From patchwork Tue Feb 7 05:10:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53679 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653100wrn; Mon, 6 Feb 2023 21:03:33 -0800 (PST) X-Google-Smtp-Source: AK7set+fbsrvFyuT8LxS8ekqo3xbI1eHSPynt6+uAMYFwCMqM0Vq+KLG3DW4qrXR3xByZhAqKicb X-Received: by 2002:a17:903:182:b0:196:869c:8c5c with SMTP id z2-20020a170903018200b00196869c8c5cmr1838986plg.51.1675746213488; Mon, 06 Feb 2023 21:03:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746213; cv=none; d=google.com; s=arc-20160816; b=M/xOAk6DLZ2MlH9bTn3j5eHDH++5Vfi+j1l9oau8gwU6mfLKGcfad/UIRLU7M6L3AN PcC5iIESMhmgjTAxuuoFikKdCtMG0GVQCa0vujT0IDGEaFS56FMUsX5VDaLDzZYhZqcs Arfo+iFgOjGqunVr46xsI8OwLblnIFypkKgDqjzrUV0/9IaRO9WigvZQNBm0DA2LGIGy ozvnbJhiSA+VnBcqHJN/QHTpIcXnWXhldADZB9p8ptoaqql7GZ+COYrL8kferT1v4B3D VZj8kmM68WrzCnZUqvv2iRBgZz0za/bK/CtMT2CRj7oa0K0K/c0qvGhu9Xh3LwDvL6Sg s2Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=+VCHhMadcKyoMnhI9XdJOCGNpEhAul0g1q3jkl3w9Ag=; b=dv9Y9m9Nt+zBSQU5IPP/H7cJjNuXWORnK3W5N52LXFmlOkKqHyA/2mclo8TGSHR9Bc qQEKxPpxazRk1cpLbQqc98XYLlofFnSZD9uofkuPOYYNcIEK7KV69lPjFcrQ4Ry8m4NZ yG+Xe3aYsorKGzv7i6fAYAuO92Cy1QN0UVOcGrpnMEH89k8wWa16+xrsqK0jyIfKnqQE 1DVHI1bsrpBJGIRimAq7iSvfRXvCLdqvIUBD5NE2nRwUgJ5jFKF/I8/Z0qIpK5HzAKvH +Uumg9v++W0S8QMp8tJMrK9VmRmTJElIBIhzeKS8b3HGpLcKr8bvw36MmT/fjndkVGmy HmvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="FT6uX/2Z"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o5-20020a1709026b0500b00189d63396c2si12125998plk.35.2023.02.06.21.03.20; Mon, 06 Feb 2023 21:03:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="FT6uX/2Z"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230364AbjBGFCu (ORCPT + 99 others); Tue, 7 Feb 2023 00:02:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229874AbjBGFB5 (ORCPT ); Tue, 7 Feb 2023 00:01:57 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90246C67C; Mon, 6 Feb 2023 21:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746115; x=1707282115; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=+1DtJmYoUbMLlAjJEQ1d3h7xcYK1w4mnpef+1fxd8fs=; b=FT6uX/2Z2cJDqxB5UQdPg6yPU4mrWILNCMAbkcAW/F+e/3uy6Qs/0kEF 0XfIcNOwyLA75YRzVxpsWrrf+3UZ3uXHqSvt5KobjKrmL/EsQRSPjSI1h M6BrDUgLkm2WB1VHRiPl2q6HDPof2pE362N5D8yj6dTRGbKjFfAmUHoDk 2vpL+iUZ0xIbh56yd9qHd6WZMi509xW0C8O00Fz6W8XETeQhaDwlzaLKE tIDSGHVgWdEEBwXmBVdeIZdv7uYy+a4H64M9rtpmM2v9NtMPl/4OlZOkW JuNtzcQi/sNh/azzkLMTbA8AP69uc2Flsg1o/GHUXjJ6xRrZEY0YMbB4b A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625878" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625878" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657740" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657740" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:43 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 12/24] x86/cpufeatures: Add the Intel Thread Director feature definitions Date: Mon, 6 Feb 2023 21:10:53 -0800 Message-Id: <20230207051105.11575-13-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147261746507871?= X-GMAIL-MSGID: =?utf-8?q?1757147261746507871?= Intel Thread Director (ITD) provides hardware resources to classify the current task. The classification reflects the type of instructions that a task currently executes. ITD extends the Hardware Feedback Interface table to provide performance and energy efficiency capabilities for each of the supported classes of tasks. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Removed dependency on CONFIG_INTEL_THREAD_DIRECTOR. Instead, depend on CONFIG_IPC_CLASSES. * Added DISABLE_ITD to the correct DISABLE_MASK: 14 instead of 13. --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 +++++++- arch/x86/kernel/cpu/cpuid-deps.c | 1 + 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index fdb8e09234ba..8a6261a5dbbf 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -349,6 +349,7 @@ #define X86_FEATURE_HWP_EPP (14*32+10) /* HWP Energy Perf. Preference */ #define X86_FEATURE_HWP_PKG_REQ (14*32+11) /* HWP Package Level Request */ #define X86_FEATURE_HFI (14*32+19) /* Hardware Feedback Interface */ +#define X86_FEATURE_ITD (14*32+23) /* Intel Thread Director */ /* AMD SVM Feature Identification, CPUID level 0x8000000a (EDX), word 15 */ #define X86_FEATURE_NPT (15*32+ 0) /* Nested Page Table support */ diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 5dfa4fb76f4b..f8e145a8c5dd 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -99,6 +99,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_IPC_CLASSES +# define DISABLE_ITD 0 +#else +# define DISABLE_ITD (1 << (X86_FEATURE_ITD & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -117,7 +123,7 @@ DISABLE_CALL_DEPTH_TRACKING) #define DISABLED_MASK12 0 #define DISABLED_MASK13 0 -#define DISABLED_MASK14 0 +#define DISABLED_MASK14 (DISABLE_ITD) #define DISABLED_MASK15 0 #define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP| \ DISABLE_ENQCMD) diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-deps.c index f6748c8bd647..7a87b823eef3 100644 --- a/arch/x86/kernel/cpu/cpuid-deps.c +++ b/arch/x86/kernel/cpu/cpuid-deps.c @@ -81,6 +81,7 @@ static const struct cpuid_dep cpuid_deps[] = { { X86_FEATURE_XFD, X86_FEATURE_XSAVES }, { X86_FEATURE_XFD, X86_FEATURE_XGETBV1 }, { X86_FEATURE_AMX_TILE, X86_FEATURE_XFD }, + { X86_FEATURE_ITD, X86_FEATURE_HFI }, {} }; From patchwork Tue Feb 7 05:10:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53682 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653207wrn; Mon, 6 Feb 2023 21:03:49 -0800 (PST) X-Google-Smtp-Source: AK7set+/kVIMbHnJ8n7IKsTtm3JePJZ5W9dgt0hvxeU4UoUf3zwLBJ39BnBctWlV5P4xd2/JUzqs X-Received: by 2002:a17:902:e403:b0:198:f9a0:8e2c with SMTP id m3-20020a170902e40300b00198f9a08e2cmr1158063ple.69.1675746229619; Mon, 06 Feb 2023 21:03:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746229; cv=none; d=google.com; s=arc-20160816; b=xyefmnOjZZRXRFv1S1hUT3c0tnl8KIGm6S4O77WNGiCU22oCdAEO8Jd4v5nxgZkl8I epjQ159lZo23E0IcKZ6qo9DFTKSMrpJiARZlaBK/Y6MdCvMg2Q4Kz6xGxLhFSlsbKEvA oAG8J96o+H4WoA+5ikKQ0+deRwHDAZk4YKxvDP1Ohg+YpP0j7NsHKFVUfBBB6bH0iAI7 VEKpC7SxpU2i9pdjVYXJhg6o9NofYiYuWMy3a2HZUadZz+JUUoJTM3DJWaZQD2+wSHrD 6HHLVNzTnlTjn/kYkcYhZ/Fd2QtK3bNHMbfV/CY83YDio/088y+Vt2p01cz/q+0vA/lS WfEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=qYnS17U6FDBb3Zx/j3Lh7Ivt4uoFS14mVif/CQ4tFXQ=; b=Q2W5z3RxZOWP1/3GUQ+JXMZ7lcyam/AXlrBxj2kPjwNcMP5F1D7xbnc/qv7pOFyrWJ ZV8/GMgYlWUMtycqzBY9/fGfBMdQryssMwJ48r1iKtlYNi90bpS4gZbT6jY5UdcHaVU5 ifu0QAxQlXzDhjHj6MFmTBuNhI6UURUsl5Hdz8aPePJfRdEEJ7h2szDK8flN79i89rJ7 q/w3Gni7GOm/dE7R/MtW1C7E2VtOFEy/Hlh1gi3748glenl5M6qDO2lCilyIv8n4GA5c tTWFnkDUO4+ShF6V869qir08kaau/4uZlvxWYcr3lWHFPlySsTdwwWm3Lu4eVljpeY6X oHEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="KRH57/e/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g12-20020a170902fe0c00b00199051ac8f7si6984569plj.188.2023.02.06.21.03.37; Mon, 06 Feb 2023 21:03:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="KRH57/e/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230421AbjBGFDE (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229949AbjBGFB5 (ORCPT ); Tue, 7 Feb 2023 00:01:57 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05AC7CA05; Mon, 6 Feb 2023 21:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746115; x=1707282115; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=vDnh1C/cq9wjWDStRuJO1Z3DAdsnd8sqeD4zFb0EIc0=; b=KRH57/e/ImFV0sKKTP80CpIj4dO4n+IZ5L2AQHQEqx+QQjlj1auDvaS6 ImagWXbEZaydTT+vz3o4cHHtECTMXZ9lEnCS6V6Hcj3VwbKk1Npu5lEO2 d8Z8HO7RpvcId73r9XM16gwpA0G7BF3Pq4hXwy12hYctZs1kDnhIEHDZN Hlu2iTzt/oW87/2icGmfrUTU9Frt7SXNYyOCH+qAtLEycVSfbMqeq+aWx ABGC5shFVOvjV+lmwlJO+8IPVkNvYlIalOTBsQSTx7cntDLEXMA4CrNWe bz6J+pEXY6e+tiAvUzLFLXcr4nN/oH75swM+fJrYPAthgRzDAJSSEVxoi A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625884" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625884" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657744" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657744" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:43 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 13/24] thermal: intel: hfi: Store per-CPU IPCC scores Date: Mon, 6 Feb 2023 21:10:54 -0800 Message-Id: <20230207051105.11575-14-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147278212081469?= X-GMAIL-MSGID: =?utf-8?q?1757147278212081469?= The scheduler reads the IPCC scores when balancing load. These reads can be quite frequent. Hardware can also update the HFI table frequently. Concurrent access may cause a lot of lock contention. It gets worse as the number of CPUs increases. Instead, create separate per-CPU IPCC scores that the scheduler can read without the HFI table lock. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Ricardo Neri --- Changes since v2: * Only create these per-CPU variables when Intel Thread Director is supported. Changes since v1: * Added this patch. --- drivers/thermal/intel/intel_hfi.c | 46 +++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 2527ae3836c7..b06021828892 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -170,6 +171,43 @@ static struct workqueue_struct *hfi_updates_wq; #define HFI_UPDATE_INTERVAL HZ #define HFI_MAX_THERM_NOTIFY_COUNT 16 +#ifdef CONFIG_IPC_CLASSES +static int __percpu *hfi_ipcc_scores; + +static int alloc_hfi_ipcc_scores(void) +{ + if (!cpu_feature_enabled(X86_FEATURE_ITD)) + return 0; + + hfi_ipcc_scores = __alloc_percpu(sizeof(*hfi_ipcc_scores) * + hfi_features.nr_classes, + sizeof(*hfi_ipcc_scores)); + + return !hfi_ipcc_scores; +} + +static void set_hfi_ipcc_score(void *caps, int cpu) +{ + int i, *hfi_class; + + if (!cpu_feature_enabled(X86_FEATURE_ITD)) + return; + + hfi_class = per_cpu_ptr(hfi_ipcc_scores, cpu); + + for (i = 0; i < hfi_features.nr_classes; i++) { + struct hfi_cpu_data *class_caps; + + class_caps = caps + i * hfi_features.class_stride; + WRITE_ONCE(hfi_class[i], class_caps->perf_cap); + } +} + +#else +static int alloc_hfi_ipcc_scores(void) { return 0; } +static void set_hfi_ipcc_score(void *caps, int cpu) { } +#endif /* CONFIG_IPC_CLASSES */ + static void get_hfi_caps(struct hfi_instance *hfi_instance, struct thermal_genl_cpu_caps *cpu_caps) { @@ -192,6 +230,8 @@ static void get_hfi_caps(struct hfi_instance *hfi_instance, cpu_caps[i].efficiency = caps->ee_cap << 2; ++i; + + set_hfi_ipcc_score(caps, cpu); } raw_spin_unlock_irq(&hfi_instance->table_lock); } @@ -580,8 +620,14 @@ void __init intel_hfi_init(void) if (!hfi_updates_wq) goto err_nomem; + if (alloc_hfi_ipcc_scores()) + goto err_ipcc; + return; +err_ipcc: + destroy_workqueue(hfi_updates_wq); + err_nomem: for (j = 0; j < i; ++j) { hfi_instance = &hfi_instances[j]; From patchwork Tue Feb 7 05:10:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53681 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653175wrn; Mon, 6 Feb 2023 21:03:42 -0800 (PST) X-Google-Smtp-Source: AK7set89M7zRRV1hW/55oWeK2KCPIf6wBRuz7Add5O527a9sVdSTWfIgp/7wQ9eWNBe9KYceDLZz X-Received: by 2002:a05:6a20:8e0f:b0:be:c7a3:2b91 with SMTP id y15-20020a056a208e0f00b000bec7a32b91mr2653475pzj.0.1675746221881; Mon, 06 Feb 2023 21:03:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746221; cv=none; d=google.com; s=arc-20160816; b=qmJVzlGLpxdBfVRICmWP56chd5zCXfDwmK2Ne3h5t7uW3ie0kh+wmHbyfDMx9KSuWr 13Q52NV5Br3jPoKvlPBQxUOForLZa860PgQghh9sBiX6Na9W5KVdv1Bpp8Btyi4zKUOz CQ20qHoqg1VfgkElpcxDBCilEO2gwurpRYvDZcbWK0uCcvUxxeT+TIv9p3tizWeTzyM5 r3EkUWnAaJVX1kdxKlMpodhgpbSD2JpPUn9PFrEYrxuvS6hSoCECiTPOMXIbsUjPdIZx QrauUd8HGsv4YsLyT7SIriskY8Y+Sty21i6s5s3lR906toI1mUPZF/jlb/vd33SKfvsR Kojg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=uO8sRwIJx7OV10qizAS3GAJFXrLtF6lBcGwFdjOAbuE=; b=Mnj0AjEks1HyxZYJ0F2tFI9nIKuknnBEH41OnrbN35vuH1vkn/PxPt+ILEP5zEp1Yd JuguBhWl5fWZdWlDo2ojVoWBi+pSBYIKCfNDZ9Eyx/ULzoq8JNfNd7G85RwBTyRdq++s pciidCHRXpVgBCZ/ejt+40i/a9NCwR0sR7q95fElfmAFJv+KRa5COa6JTeLtV1MRQ3EP 4F6zPYGTp7rrd7MnILfilh3RKT8sl6nU5nP2qih4u1Rj8El9ksQO/pyA9GLtN1whBiNc 5GMBzoZV64r9r60jfAul6qN2siwB4Gf3OlvT+0jT+/MsPOT1nglvZoVY/VXzB8fYY+lq yECw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=W4bVfXgZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j135-20020a636e8d000000b004fadb22ea88si5883680pgc.569.2023.02.06.21.03.29; Mon, 06 Feb 2023 21:03:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=W4bVfXgZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230407AbjBGFDA (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229960AbjBGFB5 (ORCPT ); Tue, 7 Feb 2023 00:01:57 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 518F9CDCB; Mon, 6 Feb 2023 21:01:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746116; x=1707282116; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=3mEF+X1OQu4dffb2wbCkr1lz7Fm3ISrkb07VTK8J8lk=; b=W4bVfXgZ4TDooAdFk0rCdotTO+9FgnsoPB4KQuc1cHFEbK6hlQBY2ZCt LymnpowYpI6ElxLMO2Y4We9FDf9WOlAwChKUXGZVvm1IEShyajI+RjXQc ZryVBGaY5kIRPfmdDsofC+uy4oIHnHvGZ2Km7D/0bR2s9dxjdYZ4Pbiio e7wxwopXaIoB7/Lxff+oNuzh79YrPYLy8OJR7oW5Fp2QendG2bYPDNf6L I3bnIxtMrUiRnTOXGY/9J9bpAy3h7S+Fr05c3RPfPnrGkhLNRMNq7uYnB rWdG6LmTSqDZ5rAHWhfx4Uwbi4P4jNDLQF6mDBdVYk+J6zLS3SpBTCfUq g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625888" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625888" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657749" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657749" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:44 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 14/24] thermal: intel: hfi: Update the IPC class of the current task Date: Mon, 6 Feb 2023 21:10:55 -0800 Message-Id: <20230207051105.11575-15-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147270119788911?= X-GMAIL-MSGID: =?utf-8?q?1757147270119788911?= Use Intel Thread Director classification to update the IPC class of a task. Implement the arch_update_ipcc() interface of the scheduler. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * Removed the implementation of arch_has_ipc_classes(). Changes since v1: * Adjusted the result the classification of Intel Thread Director to start at class 1. Class 0 for the scheduler means that the task is unclassified. * Redefined union hfi_thread_feedback_char_msr to ensure all bit-fields are packed. (PeterZ) * Removed CONFIG_INTEL_THREAD_DIRECTOR. (PeterZ) * Shortened the names of the functions that implement IPC classes. * Removed argument smt_siblings_idle from intel_hfi_update_ipcc(). (PeterZ) --- arch/x86/include/asm/topology.h | 6 ++++++ drivers/thermal/intel/intel_hfi.c | 32 +++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index 458c891a8273..ffcdac3f398f 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -227,4 +227,10 @@ void init_freq_invariance_cppc(void); #define arch_init_invariance_cppc init_freq_invariance_cppc #endif +#if defined(CONFIG_IPC_CLASSES) && defined(CONFIG_INTEL_HFI_THERMAL) +void intel_hfi_update_ipcc(struct task_struct *curr); + +#define arch_update_ipcc intel_hfi_update_ipcc +#endif /* defined(CONFIG_IPC_CLASSES) && defined(CONFIG_INTEL_HFI_THERMAL) */ + #endif /* _ASM_X86_TOPOLOGY_H */ diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index b06021828892..530dcf57e06e 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -72,6 +72,17 @@ union cpuid6_edx { u32 full; }; +#ifdef CONFIG_IPC_CLASSES +union hfi_thread_feedback_char_msr { + struct { + u64 classid : 8; + u64 __reserved : 55; + u64 valid : 1; + } split; + u64 full; +}; +#endif + /** * struct hfi_cpu_data - HFI capabilities per CPU * @perf_cap: Performance capability @@ -174,6 +185,27 @@ static struct workqueue_struct *hfi_updates_wq; #ifdef CONFIG_IPC_CLASSES static int __percpu *hfi_ipcc_scores; +void intel_hfi_update_ipcc(struct task_struct *curr) +{ + union hfi_thread_feedback_char_msr msr; + + /* We should not be here if ITD is not supported. */ + if (!cpu_feature_enabled(X86_FEATURE_ITD)) { + pr_warn_once("task classification requested but not supported!"); + return; + } + + rdmsrl(MSR_IA32_HW_FEEDBACK_CHAR, msr.full); + if (!msr.split.valid) + return; + + /* + * 0 is a valid classification for Intel Thread Director. A scheduler + * IPCC class of 0 means that the task is unclassified. Adjust. + */ + curr->ipcc = msr.split.classid + 1; +} + static int alloc_hfi_ipcc_scores(void) { if (!cpu_feature_enabled(X86_FEATURE_ITD)) From patchwork Tue Feb 7 05:10:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53684 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653245wrn; Mon, 6 Feb 2023 21:03:57 -0800 (PST) X-Google-Smtp-Source: AK7set85tokJbfkqyaZrCgmu7G645TIXd8ek9BFn3pdoebeQQ4idzC5IGwd4Ae4i14rRI0awg++8 X-Received: by 2002:a05:6a20:4403:b0:b8:9f05:5216 with SMTP id ce3-20020a056a20440300b000b89f055216mr2742404pzb.57.1675746237602; Mon, 06 Feb 2023 21:03:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746237; cv=none; d=google.com; s=arc-20160816; b=FhtPUbq+dqLWkYJxXCWL9ujQtlrzNHsKZYZhFZ4FieyxycIB8RkEP+3/RpK85WYo+Y ph7LEyKWwugFMu6isdbPDbfk3H/xU52WE7Pqrl2sDmR6gTbsYCGSvCbrhyhqxeOD1ss8 G6PSBjQEgBwWiQZZksCN8c7W36fEV2FkTTBYkhRGKLY207d5LM5m/o4cgGCc57K/WNGt uOv3UN1VXjUhMcP7a7LxEKi39S8RKET5FmKkOQX/bbb6FTFBXlP/do9ItcYKVcS5tm65 3Q040EMmlDHOIgnTa0MeNF3v5i70bmPjyIFTLrCoEEeJ/u6Mx6W1Fr6h/gcbPBAWI/pd B3YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=i+SN5OCvmzohmhBTeVPYyxFoLY0hJTRGrepOFVPuVNw=; b=Qj/IaPltpHdALlaRWimPZKAmNHL9GnmSrFRLf3YUuRwViOag7orgESrpmMBzv5r3ZY +4GZBblz3FjSsYJ0hcg3kJ6BZ5a+Q35h1x5u4yHLrV1IMHAQxJj/Lt7Vsr1y/v3tmOsD ahXZCsGdEvASifw1Uxaqagz+e97WyENftp9NSYdmOatgT6cc95xkUw7ehnwQH4EMF344 OMkFv9O7i4mEITG7aqvLCnP3L7BNPlPw7bKrwl/IZUaZIc/Cc2QvKYoRB64mvRJJ5Ui6 VY9n3dp8hteMtv7TCqUzQVwMKs9NbJfmVyK0P+DrCWmXiGPTK05KJji51Y2hn0T2iSSA eXYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=fCIWpLnW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z27-20020a63b91b000000b0047776c01d71si14309196pge.375.2023.02.06.21.03.43; Mon, 06 Feb 2023 21:03:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=fCIWpLnW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230440AbjBGFDK (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230092AbjBGFB7 (ORCPT ); Tue, 7 Feb 2023 00:01:59 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C74CCDF1; Mon, 6 Feb 2023 21:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746117; x=1707282117; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=A+SMOZzgEsVxAgLjF6sCJ/1gpwfA0tMbTNw8B4DLAIM=; b=fCIWpLnWvVOZblBGvUfz1mfcwQWd0Wj6VTVW0bEwvGTCk23O+ZGxeuld jjtcNSGAX6l4iZ/W9PZlReroJ2vApzJ3USF1g8Lm5qUnqlG7JT1qMWDI0 H6FNfi+ZqU1aCCQh8BNlveiPN+RIkkMHbCJ10uUqyWtocDETvtzI67hYH hj9Z0//AAgnDnozgPFdehOz5Miy7HNxQ9O9luBHuopvI18PATDNOP04hv 0kmhNvELgkcUgxUUwU9wzBR/rmZt6fH1Go/E3a8qZeLjprt4QZbQHVhjZ uunENef9sTQ0wGONgCJh2cT7ncG+dxubU8Opat1VDUcw4tNLBdVseObBe g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625902" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625902" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657752" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657752" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:44 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 15/24] thermal: intel: hfi: Report the IPC class score of a CPU Date: Mon, 6 Feb 2023 21:10:56 -0800 Message-Id: <20230207051105.11575-16-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147287072824953?= X-GMAIL-MSGID: =?utf-8?q?1757147287072824953?= Implement the arch_get_ipcc_score() interface of the scheduler. Use the performance capabilities of the extended Hardware Feedback Interface table as the IPC score. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Adjusted the returned HFI class (which starts at 0) to match the scheduler IPCC class (which starts at 1). (PeterZ) * Used the new interface names. --- arch/x86/include/asm/topology.h | 2 ++ drivers/thermal/intel/intel_hfi.c | 27 +++++++++++++++++++++++++++ 2 files changed, 29 insertions(+) diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index ffcdac3f398f..c4fcd9c3c634 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -229,8 +229,10 @@ void init_freq_invariance_cppc(void); #if defined(CONFIG_IPC_CLASSES) && defined(CONFIG_INTEL_HFI_THERMAL) void intel_hfi_update_ipcc(struct task_struct *curr); +unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu); #define arch_update_ipcc intel_hfi_update_ipcc +#define arch_get_ipcc_score intel_hfi_get_ipcc_score #endif /* defined(CONFIG_IPC_CLASSES) && defined(CONFIG_INTEL_HFI_THERMAL) */ #endif /* _ASM_X86_TOPOLOGY_H */ diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 530dcf57e06e..fa9b4a678d92 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -206,6 +206,33 @@ void intel_hfi_update_ipcc(struct task_struct *curr) curr->ipcc = msr.split.classid + 1; } +unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) +{ + unsigned short hfi_class; + int *scores; + + if (cpu < 0 || cpu >= nr_cpu_ids) + return -EINVAL; + + if (ipcc == IPC_CLASS_UNCLASSIFIED) + return -EINVAL; + + /* + * Scheduler IPC classes start at 1. HFI classes start at 0. + * See note intel_hfi_update_ipcc(). + */ + hfi_class = ipcc - 1; + + if (hfi_class >= hfi_features.nr_classes) + return -EINVAL; + + scores = per_cpu_ptr(hfi_ipcc_scores, cpu); + if (!scores) + return -ENODEV; + + return READ_ONCE(scores[hfi_class]); +} + static int alloc_hfi_ipcc_scores(void) { if (!cpu_feature_enabled(X86_FEATURE_ITD)) From patchwork Tue Feb 7 05:10:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53683 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653244wrn; Mon, 6 Feb 2023 21:03:57 -0800 (PST) X-Google-Smtp-Source: AK7set8j2kwlLhVgNWL7FOmBW7Y9GgdfX3nj3PsFRIhtY8w3/0LVu+r4/BIwcMZoXLBJbAjEV7VS X-Received: by 2002:a05:6a20:8f03:b0:be:a5bf:4c5f with SMTP id b3-20020a056a208f0300b000bea5bf4c5fmr2684688pzk.10.1675746237173; Mon, 06 Feb 2023 21:03:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746237; cv=none; d=google.com; s=arc-20160816; b=Kop2pldgNrsWcZvgRjfkS4WqwXWFtju035yT5I3H96p5saCuFFO9K9f6MGhxwsh+MD x9iCsJL0HO7JFe2U2bsWYBVx0Xl0+C7hnYiUDTS0fgPnb/rvBfKNOkF5bgLGfRfSwIZU 2vQ/c29kDVvubVZQcVqIvl0lzpHokLbLQ1WN/THJ/farvDDOvpy0QDSe3vVRXzSeWlF6 5UyDfgz/1hFnA5pf36z3fHgb6y6pS0C1y1WiuuIgy7KcHrReFn4Cim4c61TtMGPL1HGh uU7JfU91eBcKbfKw7gtRqabHPTo1vQB663YVSXqZqf5aVFdvv+PCcCUXtG/DhGfY8xSI fdBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=W7+3P862tRPrlCXnIouOuYMaNnhfkrfzcIWJ3H2FOTQ=; b=Lw5zUF/ms3WyrJx60E30YYtprIRQaRtDlkoqQLXKe5IZRIU534qsNq4oJQbxA2+DDx LFVAtw5jSHRfTg1dBk2OQ/WKVdQAeiCXgd4U8wT878mrC0Pdv+WZegafuvFux4ghZlma rqWN15oPpYC63ryIPFHXaevzeH2WbxuCA6lrYLx78wm/UK4svwJo4AlQ6coL8TNsCLUK 0Ymnavvi4IxNsH7bnvpNH19hcR/kxyFbbY860tg1l6TTBW38nWwC9Zl/KuXuHDMkq6Xg KEPhtdbens+f5KCWxNh+liRYWNciG4QVqO84OehO4sKDCBPd0r+YeApjkgFL6/O5DzSJ hzGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=k8lpySZE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k184-20020a6384c1000000b004e850715f99si14846047pgd.259.2023.02.06.21.03.42; Mon, 06 Feb 2023 21:03:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=k8lpySZE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230430AbjBGFDH (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230082AbjBGFB7 (ORCPT ); Tue, 7 Feb 2023 00:01:59 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C7C7D518; Mon, 6 Feb 2023 21:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746117; x=1707282117; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=L0pxmqCmeAEPVOWhuC7xe5KQw/uWTSvYwYWe6PdvQds=; b=k8lpySZEZuKpnEK6BNh3MqDFNj8noa8cmCiWoDb0FpmCUQuYh2NwIloX Na7gEE8syVSHC4CF6E4fkewSmjcUZLevai8p0s3bodM2YHP2DQxiUqHkS X9kfu8jHMvU0GzwCtoaqkd3YwvzKT/keEfNVk5M+n07Pa+NFwiyz4Qltb kt/XUIaH2tkpyNJd2zHEs7tQZh6oFOUv+jcguXtrEZO4ge9KVVHssA3md EjGRan8X8RDlLc192EpYjS0vRv59HNbF6HJ6gmjJq5DnylQWOYhlBNm2K 0Yws1c81pAawAKeAwqd36Nf5SEHv4K9ewHpLMVw5BA6YUnB8/MwlZOxrN A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625907" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625907" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657755" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657755" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:44 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 16/24] thermal: intel: hfi: Define a default class for unclassified tasks Date: Mon, 6 Feb 2023 21:10:57 -0800 Message-Id: <20230207051105.11575-17-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147286587636134?= X-GMAIL-MSGID: =?utf-8?q?1757147286587636134?= A task may be unclassified if it has been recently created, spend most of its lifetime sleeping, or hardware has not provided a classification. Most tasks will be eventually classified as scheduler's IPC class 1 (HFI class 0). This class corresponds to the capabilities in the legacy, classless, HFI table. IPC class 1 is a reasonable choice until hardware provides an actual classification. Meanwhile, the scheduler will place classes of tasks with higher IPC scores on higher-performance CPUs. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri Acked-by: Rafael J. Wysocki --- Changes since v2: * None Changes since v1: * Now the default class is 1. --- drivers/thermal/intel/intel_hfi.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index fa9b4a678d92..7ea6acce7107 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -185,6 +185,19 @@ static struct workqueue_struct *hfi_updates_wq; #ifdef CONFIG_IPC_CLASSES static int __percpu *hfi_ipcc_scores; +/* + * A task may be unclassified if it has been recently created, spend most of + * its lifetime sleeping, or hardware has not provided a classification. + * + * Most tasks will be classified as scheduler's IPC class 1 (HFI class 0) + * eventually. Meanwhile, the scheduler will place classes of tasks with higher + * IPC scores on higher-performance CPUs. + * + * IPC class 1 is a reasonable choice. It matches the performance capability + * of the legacy, classless, HFI table. + */ +#define HFI_UNCLASSIFIED_DEFAULT 1 + void intel_hfi_update_ipcc(struct task_struct *curr) { union hfi_thread_feedback_char_msr msr; @@ -215,7 +228,7 @@ unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) return -EINVAL; if (ipcc == IPC_CLASS_UNCLASSIFIED) - return -EINVAL; + ipcc = HFI_UNCLASSIFIED_DEFAULT; /* * Scheduler IPC classes start at 1. HFI classes start at 0. From patchwork Tue Feb 7 05:10:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53686 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653705wrn; Mon, 6 Feb 2023 21:05:10 -0800 (PST) X-Google-Smtp-Source: AK7set9VwInUe4GZv5OFzgWpaj5zWhbv+BcV3UZbHqgeP4YLi0CqIUPmPW+FeEtstp+1Ow1Gc2GH X-Received: by 2002:a17:902:cad3:b0:198:f907:2ab2 with SMTP id y19-20020a170902cad300b00198f9072ab2mr1324352pld.24.1675746310193; Mon, 06 Feb 2023 21:05:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746310; cv=none; d=google.com; s=arc-20160816; b=06We7ohn5223Yza9XSVpXaI2sK54r1MYHEQj6q75yxwKtKaVz7PsyPZCe67wxcdN/+ MWVYCxGiQDzrxwfUARZzhU5YGwqrORHGSQwweLYUp5HCNB/JATIopOCqo2r+kdJyu/f0 Nzw01wkp2WVOZFb8ROYm2H7gkqD/v/laV3c37nXlxZrcnwwnXAy8aZAwSq9V454Sl9yU MZP3k0ALbF2Lfuiugo86bdGY1Na/C2xzfCViZm/OuMskJg25il81iMWmAOdLYQZn/2cd I8S1+3hl0Z5dhYEUs8mYTXisFv4HtY6PrCAIa5HnBBcl8Z0tndD4yPHPX18sCMrHaWRh jmlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=mvtp3NtmdsbP9cxHlLTKQJAt6w/ph/ggUCn4aJF1YsM=; b=RCvZFgvyyzICuS6JwpFweOSOSLw1rPKIlhKV2ytCgYMkozniugS2UPeirWM/Vq0D6V 7SVVOXWerqSuWzjW+nQERjRnMnVV4o/vpNw1LJ0joo72NypN1MOhVHlcWYIEut/9ZdVX Seb0BoY9k6FPQOho30azSeB+SpjceHnUEKZLuhE/cAUFGQwRSxNhBDiE9+BtYsEtCi39 laLyJWFtAzhW+daIbS5P5cmaan4uYokdzdu8jsT7r1D73bYVRtZE7Hycng+pzFvgPJ8D cqgj6wRljPjvav9JUzc70k4jOdDs4/4h+/wKu0m9K74YM6erufE0X7F+0tw8D2PnluAm Rc+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cf3RcTOk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q20-20020a638c54000000b004f1b37656a9si14664636pgn.389.2023.02.06.21.04.56; Mon, 06 Feb 2023 21:05:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cf3RcTOk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230454AbjBGFDQ (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230096AbjBGFB7 (ORCPT ); Tue, 7 Feb 2023 00:01:59 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C78A26EBF; Mon, 6 Feb 2023 21:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746117; x=1707282117; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=3ZodASqqSSM2NIj1LSnSMKhTz+ez7n5TchQdgqVyKMk=; b=cf3RcTOkwDxK/NR3BWE33SHlTo7EtZxQEsk/ll545VQT14y2JO6ZaclE FP+2ti8Bzb61Sst5OGfjNqF23wahIOHdpU6++RAAsvcssP0J4pn6d4/Cb D7O7oYuFeRapk0b96g2ehWZj/YxfsCQU+avvB65ooJTzxmLLVYc67Xpc8 DGmU+eOTBhyAweTDJHMgr1bqc5lErPud88NxWOd93bTZO0c8rdObktfsf fAJQD/AH7uEOd6scEKRIjli/Ksi6ntnHo4WOTcsGWJrHVVOtu0ZVUqMZw A1mtrQKDfki2gCri0mN2zGkTRDc8y86fWRwgkI+8kWO2QWht0csLSO9Pu g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625917" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625917" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657758" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657758" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:44 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 17/24] thermal: intel: hfi: Enable the Intel Thread Director Date: Mon, 6 Feb 2023 21:10:58 -0800 Message-Id: <20230207051105.11575-18-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147363191969274?= X-GMAIL-MSGID: =?utf-8?q?1757147363191969274?= Enable Intel Thread Director from the CPU hotplug callback: globally from CPU0 and then enable the thread-classification hardware in each logical processor individually. Also, initialize the number of classes supported. Let the scheduler know that it can start using IPC classes. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri Acked-by: Rafael J. Wysocki --- Changes since v2: * Use the new sched_enable_ipc_classes() interface to enable the use of IPC classes in the scheduler. Changes since v1: * None --- arch/x86/include/asm/msr-index.h | 2 ++ drivers/thermal/intel/intel_hfi.c | 40 +++++++++++++++++++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index ad35355ee43e..0ea25cc9c621 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1106,6 +1106,8 @@ /* Hardware Feedback Interface */ #define MSR_IA32_HW_FEEDBACK_PTR 0x17d0 #define MSR_IA32_HW_FEEDBACK_CONFIG 0x17d1 +#define MSR_IA32_HW_FEEDBACK_THREAD_CONFIG 0x17d4 +#define MSR_IA32_HW_FEEDBACK_CHAR 0x17d2 /* x2APIC locked status */ #define MSR_IA32_XAPIC_DISABLE_STATUS 0xBD diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 7ea6acce7107..35d947f47550 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -48,6 +48,8 @@ /* Hardware Feedback Interface MSR configuration bits */ #define HW_FEEDBACK_PTR_VALID_BIT BIT(0) #define HW_FEEDBACK_CONFIG_HFI_ENABLE_BIT BIT(0) +#define HW_FEEDBACK_CONFIG_ITD_ENABLE_BIT BIT(1) +#define HW_FEEDBACK_THREAD_CONFIG_ENABLE_BIT BIT(0) /* CPUID detection and enumeration definitions for HFI */ @@ -72,6 +74,15 @@ union cpuid6_edx { u32 full; }; +union cpuid6_ecx { + struct { + u32 dont_care0:8; + u32 nr_classes:8; + u32 dont_care1:16; + } split; + u32 full; +}; + #ifdef CONFIG_IPC_CLASSES union hfi_thread_feedback_char_msr { struct { @@ -506,6 +517,11 @@ void intel_hfi_online(unsigned int cpu) init_hfi_cpu_index(info); + if (cpu_feature_enabled(X86_FEATURE_ITD)) { + msr_val = HW_FEEDBACK_THREAD_CONFIG_ENABLE_BIT; + wrmsrl(MSR_IA32_HW_FEEDBACK_THREAD_CONFIG, msr_val); + } + /* * Now check if the HFI instance of the package/die of @cpu has been * initialized (by checking its header). In such case, all we have to @@ -561,8 +577,22 @@ void intel_hfi_online(unsigned int cpu) */ rdmsrl(MSR_IA32_HW_FEEDBACK_CONFIG, msr_val); msr_val |= HW_FEEDBACK_CONFIG_HFI_ENABLE_BIT; + + if (cpu_feature_enabled(X86_FEATURE_ITD)) + msr_val |= HW_FEEDBACK_CONFIG_ITD_ENABLE_BIT; + wrmsrl(MSR_IA32_HW_FEEDBACK_CONFIG, msr_val); + /* + * We have all we need to support IPC classes. Task classification is + * now working. + * + * All class scores are zero until after the first HFI update. That is + * OK. The scheduler queries these scores at every load balance. + */ + if (cpu_feature_enabled(X86_FEATURE_ITD)) + sched_enable_ipc_classes(); + unlock: mutex_unlock(&hfi_instance_lock); return; @@ -640,8 +670,14 @@ static __init int hfi_parse_features(void) */ hfi_features.class_stride = nr_capabilities; - /* For now, use only one class of the HFI table */ - hfi_features.nr_classes = 1; + if (cpu_feature_enabled(X86_FEATURE_ITD)) { + union cpuid6_ecx ecx; + + ecx.full = cpuid_ecx(CPUID_HFI_LEAF); + hfi_features.nr_classes = ecx.split.nr_classes; + } else { + hfi_features.nr_classes = 1; + } /* * The header contains change indications for each supported feature. From patchwork Tue Feb 7 05:10:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53687 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653770wrn; Mon, 6 Feb 2023 21:05:21 -0800 (PST) X-Google-Smtp-Source: AK7set8CZXg+9IOpzqFYNSjPzTNULSgIftdtvWLggiEYD6DzTrJDGL+5ofd8SyqeEiIt1NisqdWH X-Received: by 2002:a05:6a20:e688:b0:bc:92cd:1536 with SMTP id mz8-20020a056a20e68800b000bc92cd1536mr1639322pzb.61.1675746320717; Mon, 06 Feb 2023 21:05:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746320; cv=none; d=google.com; s=arc-20160816; b=GfCz50XKxnaLhiKcdUtlwA/gvoaIlB9hjfKF0rbK6QlIyS3jjPPgmziWAXxXgrzmHL hafdC7RFqWv3bfPADK7Fb0mX12ToeV+1ZInWvdOJzKaMtuBWlKUA6PKqYe0b/PjNcCj9 Gyp9pVwRJpg+5sHLbV3Y1CA7kZ7ymP3C1D0kbeITsMy6ciNofcaLcZ0pzfEqK2rKUqkI d4rJEhvVG71mm5jZEvmQreX7bVumsQK+ejnDPUBv8WBfcd5y9Zn/VPucL2evK172An7Z BAfvPut/Vax57860Kckvx+nu+adKjKQbJTbjkI15vuI+ODvURuYYF2P4FOTqt/zmrAgc 21bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=ckKyDS/DVgXv2nJNeJkaNn1E+yuAQTisHgsPvHQS5P4=; b=Jtp41TwolLUWmOQ+4vJna52W8lByr0G1Q6oeMVRIwD6euPqycKvC59h10gITRp0mbL 9Tqq5elzZLIH6H3SbR4aiAAi5FKb9uQNp7kSPOtz6FD4sTtWE71oa5DzLtrirzQoVGIt Zy3TSvKBo2dbTf0m3gImWL5rhGBnMrNUhKbMdAY91//p+R8Ce59nAKEC28bjEG9OjWWK h/G+RV1Xyt7MvVoxKltIBnaTIwiOB1P+LZMo/YF7E4kwO5S8zhw+iId7g7fmRS8zm+Hx rHpESrGFxYpKZ9YWvVoR+4dlxzyYywtSKY4Hn27NpYMBjpCNmCb/ohTU8Ha9ZOXVbNz4 iUKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KzWpAtfm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w8-20020a637b08000000b00438766f903asi14481773pgc.490.2023.02.06.21.05.08; Mon, 06 Feb 2023 21:05:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KzWpAtfm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230461AbjBGFDS (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230036AbjBGFB7 (ORCPT ); Tue, 7 Feb 2023 00:01:59 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C96BD52A; Mon, 6 Feb 2023 21:01:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746117; x=1707282117; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=JgRkYVFr3x01km8FwYfHaTTQU7Y65rdIHc+lrY4vxao=; b=KzWpAtfmumtSM1eYljIE9LJ+HZG/fiLSQ6asQ9NEnvVsdrO86a44bIfR ucZx3mq/QlB3WBkhbmbJFfi27J2EPn9boEGMkfRQZ2aP1JET4yecVEKcF TXVScFQ/duJxsTDlaZg54Yz1IRcUNo4RBOGaJn52atMXvwPR4gWe4boFi 8O6PoK6H8gm0RLERj2ekOd0QPboQV2P4/dHaMGC8QtCQrbKuZe5FC5ipM Ueq3+i+6BAARmBdj7z8FuckEOPwDz8MV4jndHw+TcXCta18DMYarnom1l NVybexqFUzS8wfsDcRZpCUoXncthXd+zRZPjYjjR4GgYE4WhgJL0GlhkA Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625927" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625927" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657762" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657762" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:45 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 18/24] sched/task_struct: Add helpers for IPC classification Date: Mon, 6 Feb 2023 21:10:59 -0800 Message-Id: <20230207051105.11575-19-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147374086632845?= X-GMAIL-MSGID: =?utf-8?q?1757147374086632845?= The unprocessed classification that hardware provides for a task may not be usable by the scheduler: the classification may change too frequently or architectures may want to consider extra factors. For instance, some processors with Intel Thread Director need to consider the state of the SMT siblings of a core. Provide per-task helper variables that architectures can use to post- process the classification that hardware provides. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Used bit-fields to fit all the IPC class data in 4 bytes. (PeterZ) * Shortened names of the helpers. * Renamed helpers with the ipcc_ prefix. * Reworded commit message for clarity --- include/linux/sched.h | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 10c6abdc3465..45f28a601b3d 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1535,7 +1535,17 @@ struct task_struct { * A hardware-defined classification of task that reflects but is * not identical to the number of instructions per cycle. */ - unsigned short ipcc; + unsigned int ipcc : 9; + /* + * A candidate classification that arch-specific implementations + * qualify for correctness. + */ + unsigned int ipcc_tmp : 9; + /* + * Counter to filter out transient candidate classifications + * of a task. + */ + unsigned int ipcc_cntr : 14; #endif /* From patchwork Tue Feb 7 05:11:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53685 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2653251wrn; Mon, 6 Feb 2023 21:03:59 -0800 (PST) X-Google-Smtp-Source: AK7set8KIoXr4pPin8+LEN6tOf5vH6znTpZ+y0M0SlvKT/ZtGSE3DSQAxMZPf53TJOlY/zu7sj7k X-Received: by 2002:a05:6a21:38c4:b0:bc:b018:4341 with SMTP id yk4-20020a056a2138c400b000bcb0184341mr1586204pzb.7.1675746239310; Mon, 06 Feb 2023 21:03:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746239; cv=none; d=google.com; s=arc-20160816; b=e/HXpXsDC3BJoiimg8mwo0SQ3/IPPfPiwA8U26t4HMVaxzH8eoDQWZTkUKf9Ant8lZ DPMXxp3NtdLC4Lji/NnwQZLMihejzU5ANxyaeiSxFp2zIN7l9QMcn8kLQU5oqiC7kYgh K1fT0ZnuEagGpnZrU2PkPGIrqi+TaYp55YOLBfxB8sCPvkrZCeFWG3Y/pTbvPB6eAiF1 A2hoZ6g3XfZNE6eYK6lDetpnX4LE3knsoiIBl1dh7k6PZDzzdoSy+p57xoO/Re5RO+Em NNINQY/pHJgLKNQfyT97cTUyCzLHeQsoA9PUvPAIcNj2nKKleKftzSOlMHPLMglPcQRU ceYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=j++C+XSoDc7Kvfj8ysRD116SisAsoxY4mEZHOszdi0E=; b=meFA9G8884dbxjIli3m5ECtSGoLlub5ddRmwyXtcjIlyknro7UKcAWpFX3S3mnl60W Vqas+f9WMUkwAv1yCp85yZOzN1HmB5deQ7KbK2nh6SRhrmQZHUXdUSuV6SG0aYVKwh4+ +PGUHkukGDgnm4Mn5/PFQlaObeOOs8fcA9Civak0xtuZ4yIrZu562S4shVjS5r1lraYW 03MFoSIvmOvEBvI99nEiCIe5K5kDoIHVOpxXHyY7uG8lfji3TxiAH30pROY/A3hyIyAT h/KG0MmRg0KfEDI9pg1VvAmIpguh8Y4BoFKavCYvUcXgSwl5dpEFV9l5PqmQGL/3FnuO Go7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kYtZL5d7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w185-20020a6382c2000000b004e14d7638c0si14316816pgd.44.2023.02.06.21.03.46; Mon, 06 Feb 2023 21:03:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=kYtZL5d7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230447AbjBGFDO (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230034AbjBGFCA (ORCPT ); Tue, 7 Feb 2023 00:02:00 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0433F10ABF; Mon, 6 Feb 2023 21:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746118; x=1707282118; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=MnwH95WqrXx0YKjXk9ndqrfgMIqXB+pUulyYTmcEQjs=; b=kYtZL5d7ayHswTaAQ5gMhx0Nug0SLsiXIVIDCSc6wbcXjqbAv0T8xaVL Pmtku8+ZXtdvlmlNDwu8OMjZcKjwqADLokBiQhzDqwCKuFJaiTaw+mUwm hadIQlc2SIrM7S2FBELUTkITH3xGmaaz9rgbpaPVvgFjBkfLc9BZRXekH 7yakHEhYNbUfxxIEeZO2eB7/s7jzWB7UM1SpTmr6mLb09afy7TEsyDVDa kjv3sYUPGbHW0/r+3xRDB66MQCZj+cs/ntE+AtWo13n1X8I2p7uXgKdAH 5algSyVlqMJM4m4IOKejY0wjCYxHZgPQYrvee/kCD+oPNnv0s4YU15FF6 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625938" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625938" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657768" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657768" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:45 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 19/24] sched/core: Initialize helpers of task classification Date: Mon, 6 Feb 2023 21:11:00 -0800 Message-Id: <20230207051105.11575-20-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147288295322469?= X-GMAIL-MSGID: =?utf-8?q?1757147288295322469?= Just as tasks start life unclassified, initialize the classification auxiliar variables. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * None --- kernel/sched/core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9f4e9cc16df8..71b4af7ae496 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4430,6 +4430,8 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p) p->se.vruntime = 0; #ifdef CONFIG_IPC_CLASSES p->ipcc = IPC_CLASS_UNCLASSIFIED; + p->ipcc_tmp = IPC_CLASS_UNCLASSIFIED; + p->ipcc_cntr = 0; #endif INIT_LIST_HEAD(&p->se.group_node); From patchwork Tue Feb 7 05:11:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53689 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2656090wrn; Mon, 6 Feb 2023 21:12:48 -0800 (PST) X-Google-Smtp-Source: AK7set9vHHU2b2+oIBAjJdpAm61z8YfDIdeaGUeu/coT8A1Y2avLGKDUYamrdH5ciTrhJdRQYdIM X-Received: by 2002:a17:907:2142:b0:888:7d50:33b4 with SMTP id rk2-20020a170907214200b008887d5033b4mr2034770ejb.34.1675746767855; Mon, 06 Feb 2023 21:12:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746767; cv=none; d=google.com; s=arc-20160816; b=N6LaQ0G6+JVdi5TTRxMO49ePdpA0UvuMMUY/UtEViqpXCxyG4xRmkKqmr3f6uzbi0I nGFuQrqVmzeWHKZzrGw3TSysh2fV9PkPvodaHyclHHqZr87QWM4S8UH1BRXleDOoq5DG h9nYjnZ1sykxu8gf+gc4JtWnhC7qeVqGTytf1+hTOSmbgO6OshZzuKHKAdmlC+OKDB/t JVC4VsPWZs1Ff1tVPbEdgE/ALl9JUic0O0Mvm5Z0BfBZrJOaW5dx3j/Oek5u6IBNdH2t 2JlD+g1/u1bBzeesqIte78yYAjunlZJvJ64Xvg3d2Q68Qb5jbQjLyHnbKomk0oNe+WA5 tAtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=T5ejqQv0W9l5ss/jpff0oXUjEy6+OpBYwaydc1s9Abg=; b=zi7t6OlKTfYc7bp/5rVc4H2HzSVbrxCpLc7iEn8Vff+0YZfokvUuPs950Gex2j04/O Q+1lEWyI3d/4kuPh6CzXfrcgTTzwx0ehJ4rfnTGvtlnqw5sosJ821krrAZFS+ZHGIkZx ucQm1dwdgwmdLADi8ZH4aFFZV60n0U89a2wMazPi66lZRiDeRoUSRH2iAQ5OVfZG7hBG JL2TpP3BHWSjOO3QfH/A8+4t5TBwlP2+DyCe20+6lCqazFq0PBXO3ve1Dqt5i1J+w1ra X/PPHOZytRobUE/uotYKS57eAgHNjI79Dgme7l1OfIo6JhHnmYmURfXgUMMwXpVgaiG7 7WwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=T5FDN758; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fi2-20020a1709073ac200b008850d9f0ad4si13955493ejc.1.2023.02.06.21.12.23; Mon, 06 Feb 2023 21:12:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=T5FDN758; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230163AbjBGFDg (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230124AbjBGFCB (ORCPT ); Tue, 7 Feb 2023 00:02:01 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22F9F1165B; Mon, 6 Feb 2023 21:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746119; x=1707282119; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ptfoaHFvByfojwrO5m9wFLhJI66qhikYpEJ1HsiQosE=; b=T5FDN758/KNrKvZ6v/rCdtf0U8IQ81E3whmmtmtiqcv5uBjFKiXQHKEy NXYezTjbhG3XLAEbsQyQs+/AXTbO3ol8+IzvbbOVfRD3yk9D6yVoA5o0W Yu3cE5GqLgXw+ZI+9QNkPnbLkPbjZlh3oKFKuYItC1xwzOaK/LdSHfAph BBXOQHWmePCUCzL2t6nUfvd3z1REkB+VroHcQN9EzRFj3VJ/TPAzdmbHK nqqwQbY7lqjtj1LjXIsyEKpRTaVkro54a2xVOONVd0BrJV5R3nKIZGxKN jYpUPlhg/XtEOUEP/q6MNBfM5V0FQeXQAuNo7YJzuuu1iVFFYUXbe1COk A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625950" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625950" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657773" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657773" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:45 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 20/24] sched/fair: Introduce sched_smt_siblings_idle() Date: Mon, 6 Feb 2023 21:11:01 -0800 Message-Id: <20230207051105.11575-21-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147842359627468?= X-GMAIL-MSGID: =?utf-8?q?1757147842359627468?= X86 needs to know the idle state of the SMT siblings of a CPU to improve the accuracy of IPCC classification. X86 implements support for IPC classes in the thermal HFI driver. Rename is_core_idle() as sched_smt_siblings_idle() and make it available outside the scheduler code. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- is_core_idle() is no longer an inline function after this patch. To rule out performance degradation, I compared the execution time of the inline and non-inline versions on a 4-socket Cascade Lake system using the NUMA stressor of stress-ng: $ stress-ng --numa 1500 -t 10m is_core_idle() was called ~200,000 times. I measured the value of the TSC counter before and after calling is_core_idle() and computed the delta value. I arbitrarily removed outliers (defined as any delta larger than 5000 counts). This required removing ~40 samples. The table below summarizes the difference in execution time. All quantities are expressed in TSC counts, except the standard deviation, expressed as a percentage of the average. Average Median Std(%) Mode TSCdelta inline 668.76 626 67.24 42 TSCdelta non-inline 677.64 624 67.67 46 All metrics are similar for the inline and non-inline cases. --- Changes since v2: * Brought back this previously dropped patch. * Profiled inline vs non-inline is_core_idle(). I found not major penalty. * Merged is_core_idle() and sched_smt_siblings_idle() into a single function. (Dietmar) Changes since v1: * Dropped this patch. --- include/linux/sched.h | 2 ++ kernel/sched/fair.c | 21 +++++++++++++++------ 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 45f28a601b3d..7ef9fd84e7ad 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2449,4 +2449,6 @@ static inline void sched_core_fork(struct task_struct *p) { } extern void sched_set_stop_task(int cpu, struct task_struct *stop); +extern bool sched_smt_siblings_idle(int cpu); + #endif diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d3c22dc145f7..a66d86c5cb5c 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1064,7 +1064,14 @@ update_stats_curr_start(struct cfs_rq *cfs_rq, struct sched_entity *se) * Scheduling class queueing methods: */ -static inline bool is_core_idle(int cpu) +/** + * sched_smt_siblings_idle - Check whether SMT siblings of a CPU are idle + * @cpu: The CPU to check + * + * Returns true if all the SMT siblings of @cpu are idle or @cpu does not have + * SMT siblings. The idle state of @cpu is not considered. + */ +bool sched_smt_siblings_idle(int cpu) { #ifdef CONFIG_SCHED_SMT int sibling; @@ -1767,7 +1774,7 @@ static inline int numa_idle_core(int idle_core, int cpu) * Prefer cores instead of packing HT siblings * and triggering future load balancing. */ - if (is_core_idle(cpu)) + if (sched_smt_siblings_idle(cpu)) idle_core = cpu; return idle_core; @@ -9518,7 +9525,8 @@ sched_asym(struct lb_env *env, struct sd_lb_stats *sds, struct sg_lb_stats *sgs * If the destination CPU has SMT siblings, env->idle != CPU_NOT_IDLE * is not sufficient. We need to make sure the whole core is idle. */ - if (sds->local->flags & SD_SHARE_CPUCAPACITY && !is_core_idle(env->dst_cpu)) + if (sds->local->flags & SD_SHARE_CPUCAPACITY && + !sched_smt_siblings_idle(env->dst_cpu)) return false; /* Only do SMT checks if either local or candidate have SMT siblings. */ @@ -10687,7 +10695,8 @@ static struct rq *find_busiest_queue(struct lb_env *env, sched_asym_prefer(i, env->dst_cpu) && nr_running == 1) { if (env->sd->flags & SD_SHARE_CPUCAPACITY || - (!(env->sd->flags & SD_SHARE_CPUCAPACITY) && is_core_idle(i))) + (!(env->sd->flags & SD_SHARE_CPUCAPACITY) && + sched_smt_siblings_idle(i))) continue; } @@ -10816,7 +10825,7 @@ asym_active_balance(struct lb_env *env) * busy sibling. */ return sched_asym_prefer(env->dst_cpu, env->src_cpu) || - !is_core_idle(env->src_cpu); + !sched_smt_siblings_idle(env->src_cpu); } return false; @@ -11563,7 +11572,7 @@ static void nohz_balancer_kick(struct rq *rq) */ if (sd->flags & SD_SHARE_CPUCAPACITY || (!(sd->flags & SD_SHARE_CPUCAPACITY) && - is_core_idle(i))) { + sched_smt_siblings_idle(i))) { flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; goto unlock; } From patchwork Tue Feb 7 05:11:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53692 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2661419wrn; Mon, 6 Feb 2023 21:29:08 -0800 (PST) X-Google-Smtp-Source: AK7set8IWZH+0L3Qk1HIaoHyZ002q9KNgyVO8YJIsGJ95EsmRxqlENssmurpUVRcOeN8p7APH4cg X-Received: by 2002:a17:90a:18e:b0:22b:ee57:724a with SMTP id 14-20020a17090a018e00b0022bee57724amr2646555pjc.25.1675747747732; Mon, 06 Feb 2023 21:29:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675747747; cv=none; d=google.com; s=arc-20160816; b=nE7A+SfQgKytZ7IeMs9kGQgJvRQ5HeCekoI4JTf9SOCqWlmyGv/eI9tUNGRsQFRPA1 vms6qmX2hV0JMe/rq4zWG9pbpNQ9vXwG+IBgjS8ZbQ7cItvBSdM0BqBTLQeqfYiFSY+e 1bmEyWhiHKkEqW0pwz7ZbWDa8JMfS87vI/LSWeQrO6Axm+2INiPgvVRnUvdPekf5v0Ii U172m9QFs6agOEPBDZtMqPxwONcfrShS3cyWD4cQmK0dntC7N23s8Ys/OoK4mzchURyV qNjBz4AU9G12sAu4AvVsowKh1p5A5FxAlQgOa8IjjC4VNl6c7OFtDBRiVi7Jr6ZSx1/9 BMtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=CEVQhbt0/rd8FUj1u0OGoR+RA9iD7fUrkqAKwfR08Zs=; b=hEIK4lvWaHpqvdLn8iywvez0ZprARDUegrYfSLtCUb7KiMdaMDqgPqIFB5KMLNUvWm ZN/lDAAxF+HewUjLwKCTg0M3mYpsuefTXw+5wxoIR+otuq7Vojh5rqnu72LNDAKUW+kA 0RSrEXERqtSZzsGPs1u6rXjpFtfpHyPutdk8keos+YJT8NzE3KLBQ3e38YUXgTc8qlS8 NUxeDNXwoYNtEbOvTNDvBP0Sjz3lZLQiydFtc6ot0rHbjuMx/EXFLyjATSkpjvabfTWb dTLL4gLC5Bd0FxBy48cVDFmIkp9+hCwWzoQHuNolbI4ragJWi7TJbsbt13xMbM/4OVFj cRQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lIT7rH82; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fv5-20020a17090b0e8500b00226613377a2si13684442pjb.167.2023.02.06.21.28.54; Mon, 06 Feb 2023 21:29:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lIT7rH82; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230467AbjBGFDW (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230039AbjBGFCA (ORCPT ); Tue, 7 Feb 2023 00:02:00 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26BF511EBA; Mon, 6 Feb 2023 21:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746119; x=1707282119; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=LRruVzFjzKpVUadVEuZKwelOSSQHqDO6Cu2BUXGn/i0=; b=lIT7rH82m7j2ydTUSyTFAOE1oQoG9CuZrVnEL911ibaoIZCKxF2LVrU7 ELMtviurQo4F/kRMQBuVTfeD2+O6BexIzlNousae+dIaMBft3Yx57FoFi DcOjS3KpHPTj/QALr8DzEJ6SG90f8o+w1fCj+A1Anq3LCSMNXPVAqNpPe sJCuL4sfzXsD3izaZRm0Yyh0nyrX7XNcf7Z+piTmw76JpC+rxSi1Ejn05 aSpt6gKezZDx2D1tCtISx0KgudexZSC4H35g0a3JTBEtwYv4nqeXx+CCg ULxy6wm4D+9SjrXXaeXehv07wq32/SNFuNyy3HC2lNpFQc07g/27rUbOo w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625961" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625961" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657779" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657779" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 21/24] thermal: intel: hfi: Implement model-specific checks for task classification Date: Mon, 6 Feb 2023 21:11:02 -0800 Message-Id: <20230207051105.11575-22-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757148869933557906?= X-GMAIL-MSGID: =?utf-8?q?1757148869933557906?= In Alder Lake and Raptor Lake, the result of thread classification is more accurate when only one SMT sibling is busy. Classification results for class 2 and 3 are always reliable. To avoid unnecessary migrations, only update the class of a task if it has been the same during 4 consecutive user ticks. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Adjusted the result the classification of Intel Thread Director to start at class 1. Class 0 for the scheduler means that the task is unclassified. * Used the new names of the IPC classes members in task_struct. * Reworked helper functions to use sched_smt_siblings_idle() to query the idle state of the SMT siblings of a CPU. --- drivers/thermal/intel/intel_hfi.c | 60 ++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 35d947f47550..fdb53e4cabc1 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -40,6 +40,7 @@ #include #include +#include #include "../thermal_core.h" #include "intel_hfi.h" @@ -209,9 +210,64 @@ static int __percpu *hfi_ipcc_scores; */ #define HFI_UNCLASSIFIED_DEFAULT 1 +#define CLASS_DEBOUNCER_SKIPS 4 + +/** + * debounce_and_update_class() - Process and update a task's classification + * + * @p: The task of which the classification will be updated + * @new_ipcc: The new IPC classification + * + * Update the classification of @p with the new value that hardware provides. + * Only update the classification of @p if it has been the same during + * CLASS_DEBOUNCER_SKIPS consecutive ticks. + */ +static void debounce_and_update_class(struct task_struct *p, u8 new_ipcc) +{ + u16 debounce_skip; + + /* The class of @p changed. Only restart the debounce counter. */ + if (p->ipcc_tmp != new_ipcc) { + p->ipcc_cntr = 1; + goto out; + } + + /* + * The class of @p did not change. Update it if it has been the same + * for CLASS_DEBOUNCER_SKIPS user ticks. + */ + debounce_skip = p->ipcc_cntr + 1; + if (debounce_skip < CLASS_DEBOUNCER_SKIPS) + p->ipcc_cntr++; + else + p->ipcc = new_ipcc; + +out: + p->ipcc_tmp = new_ipcc; +} + +static bool classification_is_accurate(u8 hfi_class, bool smt_siblings_idle) +{ + switch (boot_cpu_data.x86_model) { + case INTEL_FAM6_ALDERLAKE: + case INTEL_FAM6_ALDERLAKE_L: + case INTEL_FAM6_RAPTORLAKE: + case INTEL_FAM6_RAPTORLAKE_P: + case INTEL_FAM6_RAPTORLAKE_S: + if (hfi_class == 3 || hfi_class == 2 || smt_siblings_idle) + return true; + + return false; + + default: + return true; + } +} + void intel_hfi_update_ipcc(struct task_struct *curr) { union hfi_thread_feedback_char_msr msr; + bool idle; /* We should not be here if ITD is not supported. */ if (!cpu_feature_enabled(X86_FEATURE_ITD)) { @@ -227,7 +283,9 @@ void intel_hfi_update_ipcc(struct task_struct *curr) * 0 is a valid classification for Intel Thread Director. A scheduler * IPCC class of 0 means that the task is unclassified. Adjust. */ - curr->ipcc = msr.split.classid + 1; + idle = sched_smt_siblings_idle(task_cpu(curr)); + if (classification_is_accurate(msr.split.classid, idle)) + debounce_and_update_class(curr, msr.split.classid + 1); } unsigned long intel_hfi_get_ipcc_score(unsigned short ipcc, int cpu) From patchwork Tue Feb 7 05:11:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53688 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2655631wrn; Mon, 6 Feb 2023 21:11:15 -0800 (PST) X-Google-Smtp-Source: AK7set8rlVBFELUu0crMjfDwXpO/KSp5jwazQUJ8AveoIW/1SWJMRcT6RpEHB0GoKhTlSimjxZop X-Received: by 2002:a17:907:9709:b0:888:1613:49d4 with SMTP id jg9-20020a170907970900b00888161349d4mr2881097ejc.0.1675746675157; Mon, 06 Feb 2023 21:11:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675746675; cv=none; d=google.com; s=arc-20160816; b=Ja+8htmAiIUCjYpG8fpAfmebC9V+7T98FzZEz4n37xxjj5bx5LBWcbcesOT1jJzIyq TiDFrnjPrvThUxcZ4cl7EUtsoB5AYL9X1+Lw++fSZfppEeM1b+G/pt7gVWDicltl7Fb/ NFd/uZFwOE/tMpP2p54lH06rD4dEkONE4Z3upjrMQnp+Ma3qWnx79BwMyjzDii/N0ZMu yCgOwhuuPhFhWC7KoMoVzDT3NpftNZOg+pd8hKVfxOaPGmgbptoRPljNUSPi+oh97Bqm S/w2FdbVbhVHZUv08Go61CMJGuF7jf9YUar6MmCfW0VJvccfxQ30Z4CoEvbu3BumyQ59 Qfjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=+x0Do1rPr3r7stjSqsz+yBHjiXNqX+3vBJ08PqHTpx8=; b=yQqOzfGPzwsKwBxiEDL1gA54wXhFL1+6tZqSAsfFBMR2/Fxh8foYDV7qZuAe01mVws oQrlI08m/cd5OElQxSzv7myIdry5HaVTMQdfAUadSk/1CCkwdNXw3aQ3QPEvKh2U88W9 rAzEOnf1O6bJUjKpAb+yNdcwyF5U0TjaGVzLtlxCT82CJ/Xj63PmJ8/4dv5rrl6S0WfH /ypFlyutpD8TG0Kfcf0+02bbOsK0z4Y6mt1h2IrUhxxMvVeWwWwBA8MEv3x+hK8FWXTb vtzuqs0Nj71FXUoZK4WMnf/XYVDyc//cBv2Ts4/Wpuzn+lJUxgKGB7xyQWv67PcMbISI a74Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WpC0sW0p; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id zx14-20020a170907348e00b007ae9f42f86esi14430146ejb.383.2023.02.06.21.10.50; Mon, 06 Feb 2023 21:11:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WpC0sW0p; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230483AbjBGFDm (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230176AbjBGFCC (ORCPT ); Tue, 7 Feb 2023 00:02:02 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BD801E9F5; Mon, 6 Feb 2023 21:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746121; x=1707282121; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=goElNzOu/cMaBcsm8/fX9KbVxksjtxFjfuJIL57mdrg=; b=WpC0sW0pWEipNNeTvEKSOjPIUQuuytPxTSJoQTE1NqlGPAH4vgDuToLZ 5VYDwr2L5bwivqAu7eXIDwl/llPbnTLCFP1O7rcxdsLdDkg+6sgNnNlXx LbTqYXMp7GZ7AFBpj6kpHFd1woQMrqYW5B4pRtWenUHEcpGNCbhgJzslC OvLCDU5Dn9qoeGWuqMGEhG1rqIt+wvj1rlqkBHnfnyQTBJSgHJFJEVe08 Aey3i3l3V11bWCUl7AyGfkqdKmppGHIkjmwjo3l/57BAEGAnQIuh/8/sK L/Vci0tKsojEYaRyr0i+eGjx+jx7K/ay2wheXK4U9k9xRZkwjOS6j9jVl g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625971" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625971" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657783" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657783" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 22/24] x86/cpufeatures: Add feature bit for HRESET Date: Mon, 6 Feb 2023 21:11:03 -0800 Message-Id: <20230207051105.11575-23-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757147745676894380?= X-GMAIL-MSGID: =?utf-8?q?1757147745676894380?= The HRESET instruction prevents the classification of the current task from influencing the classification of the next task when running serially on the same logical processor. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * None --- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/msr-index.h | 4 +++- arch/x86/kernel/cpu/scattered.c | 1 + 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h index 8a6261a5dbbf..eb859a82b22a 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -309,6 +309,7 @@ #define X86_FEATURE_MSR_TSX_CTRL (11*32+20) /* "" MSR IA32_TSX_CTRL (Intel) implemented */ #define X86_FEATURE_SMBA (11*32+21) /* "" Slow Memory Bandwidth Allocation */ #define X86_FEATURE_BMEC (11*32+22) /* "" Bandwidth Monitoring Event Configuration */ +#define X86_FEATURE_HRESET (11*32+23) /* Hardware history reset instruction */ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 0ea25cc9c621..dc96944d61a6 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1109,6 +1109,9 @@ #define MSR_IA32_HW_FEEDBACK_THREAD_CONFIG 0x17d4 #define MSR_IA32_HW_FEEDBACK_CHAR 0x17d2 +/* Hardware History Reset */ +#define MSR_IA32_HW_HRESET_ENABLE 0x17da + /* x2APIC locked status */ #define MSR_IA32_XAPIC_DISABLE_STATUS 0xBD #define LEGACY_XAPIC_DISABLED BIT(0) /* @@ -1116,5 +1119,4 @@ * disabling x2APIC will cause * a #GP */ - #endif /* _ASM_X86_MSR_INDEX_H */ diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c index 0dad49a09b7a..cb8a0e7a4fdb 100644 --- a/arch/x86/kernel/cpu/scattered.c +++ b/arch/x86/kernel/cpu/scattered.c @@ -28,6 +28,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_EPB, CPUID_ECX, 3, 0x00000006, 0 }, { X86_FEATURE_INTEL_PPIN, CPUID_EBX, 0, 0x00000007, 1 }, { X86_FEATURE_RRSBA_CTRL, CPUID_EDX, 2, 0x00000007, 2 }, + { X86_FEATURE_HRESET, CPUID_EAX, 22, 0x00000007, 1 }, { X86_FEATURE_CQM_LLC, CPUID_EDX, 1, 0x0000000f, 0 }, { X86_FEATURE_CQM_OCCUP_LLC, CPUID_EDX, 0, 0x0000000f, 1 }, { X86_FEATURE_CQM_MBM_TOTAL, CPUID_EDX, 1, 0x0000000f, 1 }, From patchwork Tue Feb 7 05:11:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53691 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2660067wrn; Mon, 6 Feb 2023 21:25:07 -0800 (PST) X-Google-Smtp-Source: AK7set9SvzbKE4R22m4rtBK1FK6HyNo/WS52howLXv9NV3w291O4XNyn6Gcc8MCq0B5MZBqavd8y X-Received: by 2002:a50:9318:0:b0:4a2:254f:6b1a with SMTP id m24-20020a509318000000b004a2254f6b1amr1993275eda.11.1675747507058; Mon, 06 Feb 2023 21:25:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675747507; cv=none; d=google.com; s=arc-20160816; b=siASkkYSgepEEoz/C1VOIoTZO/aOVllmnEWfqSx0JTs4PezapLiCmldnhEbZxW5C0H 10D24meyLckUyJ0AZJAt4ChqiSTH5oTP8jlaCYKNKhDtSJNhjyZlftg+JFlrZsEM+sS4 oaQMKtJjCo58G8X0Z3/XxIjHkh0OrOw6uZaOjHEU9XH1yeq7tNJQz07wyITBsJ0Ftedw 9SZJcbhFOQgH7YbNnpFV3P5XApmhpI2PfW15J3nPyx9d8lg8gzUB7HJc9CmtHmyPjBBL cTpLuw0vLX7EN/FDkDwQkOP8VwMC6V6Y4qtR9JcpCtm+i2bGYcP+M4DT0q6q5iT8wgiI Iq0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=lGXWeD5LHUZVaBjG53ZCiWuhIljSjVOQsfBe7w7vIZw=; b=kx24sM5G6jvoGR9HOBbkjQOrNsBYToD2RYum9JRsctyTCekntrGK48BTyiRqijEyq5 HImh/H97lUrpzNy2fUV4Rbhl1UWkuT0/vIdBlY2u7UP0uhp8Y6o0PmmsL6GwOknQkFTO wOF7IiNcwGDKcerOp2vo3rd0bptMRtThd6qdTRMEQ2Qkp/l50/79D60lHSzypuFmi9Pz VyjUCnyYF6vwMtcn/c+syo+ZLXdJ0Bda9ne3jox2XkA6rWXIWHn+UfAK/4vKZcewxETL SppNYUBAqTm4HhBPlascDxsRaXqKsXZcEvyokpN81dpvzJQKEUCGnk7vi7IF1IKjMljb 0q+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JTCe7QWJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u24-20020aa7d0d8000000b004a230263841si14340831edo.302.2023.02.06.21.24.41; Mon, 06 Feb 2023 21:25:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=JTCe7QWJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230473AbjBGFD1 (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230163AbjBGFCA (ORCPT ); Tue, 7 Feb 2023 00:02:00 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78AED126F6; Mon, 6 Feb 2023 21:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746119; x=1707282119; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=4NcODw8HpUR3D/VhBDlDDOdaVqVUmE0eJRth2Ko7xT0=; b=JTCe7QWJrZ43mMmzQe7vBNEHjQFNVR9x/ficCxEVsfVMfpTJmx1GDxWd kgkLGIUWGmLtwiHUAuW5BARF7HcmYgzS3G06+6y+2xwbrdqpBl1s7ujJU m9xsizrNkL0lKmavliD8wQkmzlDm6Pme5DKI2nbcWTksUxGqkfAonv0A+ mvX8F89+N681tfAR245Dp4g9Mj5RdTgZPUHD5K7CaD5f3ecqDl1bR3Cd/ ONitshFa7Yrd9GNCqmGbKmDFTfEhtSCJydcAH3f9hsCXQZdaIYOwRpa23 K0TeVSHFxByECjRDK8fIkppkGs8LVbujhL74xJRtJrGfQ5UBhf/axgC9e Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625983" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625983" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657790" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657790" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:46 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 23/24] x86/hreset: Configure history reset Date: Mon, 6 Feb 2023 21:11:04 -0800 Message-Id: <20230207051105.11575-24-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757148617760139080?= X-GMAIL-MSGID: =?utf-8?q?1757148617760139080?= Configure the MSR that controls the behavior of HRESET on each logical processor. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Marked hardware_history_features as __ro_after_init instead of __read_mostly. (PeterZ) --- arch/x86/kernel/cpu/common.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 831a1a07d357..f3f936f7de5f 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -412,6 +412,26 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c) cr4_clear_bits(X86_CR4_UMIP); } +static u32 hardware_history_features __ro_after_init; + +static __always_inline void setup_hreset(struct cpuinfo_x86 *c) +{ + if (!cpu_feature_enabled(X86_FEATURE_HRESET)) + return; + + /* + * Use on all CPUs the hardware history features that the boot + * CPU supports. + */ + if (c == &boot_cpu_data) + hardware_history_features = cpuid_ebx(0x20); + + if (!hardware_history_features) + return; + + wrmsrl(MSR_IA32_HW_HRESET_ENABLE, hardware_history_features); +} + /* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | @@ -1848,10 +1868,11 @@ static void identify_cpu(struct cpuinfo_x86 *c) /* Disable the PN if appropriate */ squash_the_stupid_serial_number(c); - /* Set up SMEP/SMAP/UMIP */ + /* Set up SMEP/SMAP/UMIP/HRESET */ setup_smep(c); setup_smap(c); setup_umip(c); + setup_hreset(c); /* Enable FSGSBASE instructions if available. */ if (cpu_has(c, X86_FEATURE_FSGSBASE)) { From patchwork Tue Feb 7 05:11:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 53690 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2657576wrn; Mon, 6 Feb 2023 21:17:47 -0800 (PST) X-Google-Smtp-Source: AK7set9pdffDfwJQ+OX1EKQ6x/Ou5wnzBxXFFNwsUt+CfsjHOwW49yHof3kxog+YVai6MMF/SpU1 X-Received: by 2002:a17:90b:1b4a:b0:230:af67:b847 with SMTP id nv10-20020a17090b1b4a00b00230af67b847mr2437188pjb.31.1675747066928; Mon, 06 Feb 2023 21:17:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675747066; cv=none; d=google.com; s=arc-20160816; b=1IbsNcnW9bQ6RyyiGVUTUjc0kLl+ejtqQXX4pRNbvSMItD2yW0bETBGDc5dp0uimyz SrQ2vyMwnUSQL370LM5wpmzTj/IFJXFr/br3IfOlG8+G46jC3DuWkfBdS4L8cHzg8rgW KQ5bd1ATpa0jiCG/2aEdDzDSn26PsLhJuyrmgCCPmSpOvCWBj4l8l1fuxiTtG0thRwMm gz26eEmnLnMVb4LW8E7NXjDa/oOgwaK16Rs/yX3aWoQq0CSSviOY9Ov7wzza9mx8DWfN hmfwF1vdhn0q22OF5cNZ5lBlkKUMQu6PbFMelbkrUARHuaY+2czIY/vBK1xuRffFZB6g XUPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature; bh=wyRjluUQ3mMp6+tD3VZp7S9wkdxsE2V3t2zIyzGz4s8=; b=vyvjSOv77Nj9lP4KY3LJg+Yvo8AMZe/tQ2D32I47/4OY9KaCaPA3nrQk/0CGyhrJn5 8p0458MQKKFWGz93A2Aoyk+cMcINP8x9LpC5DEGKYnVfphwxSVSp58zsTs/+pumnOVHo xXiz7Q250UPHMkkqTPIffv7op/HVPRByZsPpPNj2UXasDw890oKXKI3y1XjRyNiUISB1 H1InZVx1+qL68dZAIf6BTEi8xDBwJKw8GX51IHT03ThwlXU9xME+FGv48k6wkON+UoLV 1wiIiNB90MOS7zEJnN6NqRgJGH0RRSZJQvLpLTph0V34GbmBWOo98JRwgBL2JtlF3lfc l66A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=A1K+TFOm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n7-20020a637207000000b004d0506f7037si12712653pgc.253.2023.02.06.21.17.33; Mon, 06 Feb 2023 21:17:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=A1K+TFOm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230489AbjBGFDp (ORCPT + 99 others); Tue, 7 Feb 2023 00:03:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230137AbjBGFCC (ORCPT ); Tue, 7 Feb 2023 00:02:02 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64BAD196BA; Mon, 6 Feb 2023 21:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675746120; x=1707282120; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=un8n4Xz9yXThF7EWOBFRrgSk/o9VXPxzUfeccAO2+YA=; b=A1K+TFOmKRVUM96AxuUVd4h3fifqUoc5xeYFhTBA5F+eu+psGs+Z+nHv KTqb2ir9HdMKJ8pZOw75N+KgLixp8pfAxpjXDGqcb6OSRjnLIVUvPhkdt Z2obMioz5pNcPZ/icAu87MBHUTCZuU0jRVrJMapqDpKzSsLNjmy+H44nQ OgRLhcOYgOVewXrILdyMssmxxF3ku+hwmS4ZXbepNjk3c+ae8kahRFg9c 3xNrHq3qDFSk5AvF5DhiizIhlqsJszoxtwFS+ho0WBH2gbNT2vobQp34W gHcNUaqVj5ROf2WOsNByAgKvYwYkS8xtQiMPt8Qn7G1ZPcV8iQDpOOQuH g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="415625995" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="415625995" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 21:01:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="668657794" X-IronPort-AV: E=Sophos;i="5.97,278,1669104000"; d="scan'208";a="668657794" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga007.fm.intel.com with ESMTP; 06 Feb 2023 21:01:47 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Lukasz Luba , Ionela Voinescu , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v3 24/24] x86/process: Reset hardware history in context switch Date: Mon, 6 Feb 2023 21:11:05 -0800 Message-Id: <20230207051105.11575-25-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> References: <20230207051105.11575-1-ricardo.neri-calderon@linux.intel.com> X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757148156792318261?= X-GMAIL-MSGID: =?utf-8?q?1757148156792318261?= Reset the classification history of the current task when switching to the next task. Hardware will start the classification of the next task from scratch. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Lukasz Luba Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v2: * None Changes since v1: * Measurements of the cost of the HRESET instruction Methodology: I created a tight loop with interrupts and preemption disabled. I recorded the value of the TSC counter before and after executing HRESET or RDTSC. I repeated the measurement 100,000 times. I performed the experiment using an Alder Lake S system. I set the frequency of the CPUs at a fixed value. The table below compares the cost of HRESET with RDTSC (expressed in the elapsed TSC count). The cost of the two instructions is comparable. PCore ECore Frequency (GHz) 5.0 3.8 HRESET (avg) 28.5 44.7 HRESET (stdev %) 3.6 2.3 RDTSC (avg) 25.2 35.7 RDTSC (stdev %) 3.9 2.6 * Used an ALTERNATIVE macro instead of static_cpu_has() to execute HRESET when supported. (PeterZ) --- arch/x86/include/asm/hreset.h | 30 ++++++++++++++++++++++++++++++ arch/x86/kernel/cpu/common.c | 7 +++++++ arch/x86/kernel/process_32.c | 3 +++ arch/x86/kernel/process_64.c | 3 +++ 4 files changed, 43 insertions(+) create mode 100644 arch/x86/include/asm/hreset.h diff --git a/arch/x86/include/asm/hreset.h b/arch/x86/include/asm/hreset.h new file mode 100644 index 000000000000..d68ca2fb8642 --- /dev/null +++ b/arch/x86/include/asm/hreset.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_HRESET_H + +/** + * HRESET - History reset. Available since binutils v2.36. + * + * Request the processor to reset the history of task classification on the + * current logical processor. The history components to be + * reset are specified in %eax. Only bits specified in CPUID(0x20).EBX + * and enabled in the IA32_HRESET_ENABLE MSR can be selected. + * + * The assembly code looks like: + * + * hreset %eax + * + * The corresponding machine code looks like: + * + * F3 0F 3A F0 ModRM Imm + * + * The value of ModRM is 0xc0 to specify %eax register addressing. + * The ignored immediate operand is set to 0. + * + * The instruction is documented in the Intel SDM. + */ + +#define __ASM_HRESET ".byte 0xf3, 0xf, 0x3a, 0xf0, 0xc0, 0x0" + +void reset_hardware_history(void); + +#endif /* _ASM_X86_HRESET_H */ diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index f3f936f7de5f..17e2068530b0 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include #include @@ -414,6 +415,12 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c) static u32 hardware_history_features __ro_after_init; +void reset_hardware_history(void) +{ + asm_inline volatile (ALTERNATIVE("", __ASM_HRESET, X86_FEATURE_HRESET) + : : "a" (hardware_history_features) : "memory"); +} + static __always_inline void setup_hreset(struct cpuinfo_x86 *c) { if (!cpu_feature_enabled(X86_FEATURE_HRESET)) diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c index 470c128759ea..397a6e6f4e61 100644 --- a/arch/x86/kernel/process_32.c +++ b/arch/x86/kernel/process_32.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include "process.h" @@ -214,6 +215,8 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) /* Load the Intel cache allocation PQR MSR. */ resctrl_sched_in(); + reset_hardware_history(); + return prev_p; } diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 4e34b3b68ebd..6176044ecc16 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include #ifdef CONFIG_IA32_EMULATION @@ -658,6 +659,8 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) /* Load the Intel cache allocation PQR MSR. */ resctrl_sched_in(); + reset_hardware_history(); + return prev_p; }