[RFC,01/22] arch_topology: Make register_cpu_capacity_sysctl() tolerant to late CPUs
Message ID | E1r0JKl-00CTwT-Hx@rmk-PC.armlinux.org.uk |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:aa0b:0:b0:403:3b70:6f57 with SMTP id k11csp139562vqo; Tue, 7 Nov 2023 02:29:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFCR8f9vIWzluJiiFVSVA7GKCPJMiSZcM14ujs1skjIMlng4mWVG/iX4TD/yThn6hlyQ0Nn X-Received: by 2002:a9d:6d0b:0:b0:6c4:a036:cc1b with SMTP id o11-20020a9d6d0b000000b006c4a036cc1bmr31070104otp.35.1699352982754; Tue, 07 Nov 2023 02:29:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699352982; cv=none; d=google.com; s=arc-20160816; b=ztNhwCC6ni6Ul5ZaZRxVUHPePqn1A8ehbYPm1M8SPcu5V4+XOj38ImMgWpBc0KWiB8 VqQ0trz6nH8PxCVKVhnxtWxa4lO9zF4Mjk1iUWnI4OE1XLVSRdJdPoMZKg6GpRUQ0VwL wdzdEOa9KYMHbgyv1OjAq1gUTY+oYK86QA7Oxvnm/2vnjJj4mT5JwSOdnezJYHU/519+ pqnk0sUaQcqF3j1oHiO/bDIbJjuR6qZXtV0ibdO6y4uUnnfVp/tbybcj9jx9tS7wGwmS fxuVXIbFuxyEnbtBe8o3y7jwz7G6NrfHwozSUJB0Iy5ylInqm/0t8t/CiM+1Nf2eKfIc Ti1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:sender:message-id:content-transfer-encoding :content-disposition:mime-version:subject:cc:to:from:references :in-reply-to:dkim-signature; bh=XGhpKk09pdERl3TLuSkcjp8wddtxgCnM/F8wNa+d4Ds=; fh=5x3d26Qz6ERs/5V0XcGUEhmIwiM9O4PWyRYwE+rmnhk=; b=zXMBWAk1vtDto7i1QMo64SRVDXg2wI+8DsVyqHLpYVXyENXAw/tY7ufPVGOGnNE/HY xgJgcINaC8/couoV16utk9dO6ZEXdg7nhmY6bZ2DTEO3dMpsxKRU/lZ0mxJZtXdHstDR oBnazlS1L9HMFUldcEMUonoe/o3CtOeksfu5f0p2KPC+qtEGMGYTvzmQpjYXRQMih9EG +vsWUN+DlL0ttt3sJHGo4g6i4//bV0trgJKFTQH5sTEALWVSooA86CuEkLTpfq6u5aEC IsCqv6txcASVE9Lf3F/8Q6sDX2JJsTgiWC+Q3WmE+714uvFQHIMRrmMPZTQOZHAYpvTu 6ueg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail (test mode) header.i=@armlinux.org.uk header.s=pandora-2019 header.b=q+446HBS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=armlinux.org.uk Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id c9-20020a056a00248900b006b9fd40d6cfsi10382402pfv.216.2023.11.07.02.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Nov 2023 02:29:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=fail (test mode) header.i=@armlinux.org.uk header.s=pandora-2019 header.b=q+446HBS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=armlinux.org.uk Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 9E28F8028A55; Tue, 7 Nov 2023 02:29:40 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233859AbjKGK3d (ORCPT <rfc822;lhua1029@gmail.com> + 32 others); Tue, 7 Nov 2023 05:29:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233850AbjKGK3b (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 7 Nov 2023 05:29:31 -0500 Received: from pandora.armlinux.org.uk (pandora.armlinux.org.uk [IPv6:2001:4d48:ad52:32c8:5054:ff:fe00:142]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA8C2122; Tue, 7 Nov 2023 02:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2019; h=Date:Sender:Message-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:Subject:Cc:To:From:References: In-Reply-To:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=XGhpKk09pdERl3TLuSkcjp8wddtxgCnM/F8wNa+d4Ds=; b=q+446HBSaqDASp2DnGSpsalYDH pEuuqbj+B5pSmcX0VDJ+qGo58o7uSeMuiLOYxUQrm0Wkih2A3Ky189UPnhCg834f6nmAlW8BHpu6H zmvap4iTUECvSPaMtHSu5vdB/QurXGDPNvSGKEhAg3Nn51eifgYPV2OqtuAnHQFr18lBLFcIzKzmv Hy8RNRf8FAqUQNat3Jr3povSccaGlwjcGInBALMOWoNXvc+8V7k6IgxxJXcW0E2qWnwSyh+VLF/L5 6nGpBHGGVVySCpWYM/fLTZonlnYqoyQ49FDeC4o1c1P53Mv4wYPnmDRRNKv7EYOPw3DzXxQ3GCg8D 8rDkb9nQ==; Received: from e0022681537dd.dyn.armlinux.org.uk ([fd8f:7570:feb6:1:222:68ff:fe15:37dd]:40856 helo=rmk-PC.armlinux.org.uk) by pandora.armlinux.org.uk with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from <rmk@armlinux.org.uk>) id 1r0JKj-0000DW-2W; Tue, 07 Nov 2023 10:29:21 +0000 Received: from rmk by rmk-PC.armlinux.org.uk with local (Exim 4.94.2) (envelope-from <rmk@rmk-PC.armlinux.org.uk>) id 1r0JKl-00CTwT-Hx; Tue, 07 Nov 2023 10:29:23 +0000 In-Reply-To: <ZUoRY33AAHMc5ThW@shell.armlinux.org.uk> References: <ZUoRY33AAHMc5ThW@shell.armlinux.org.uk> From: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> To: linux-pm@vger.kernel.org, loongarch@lists.linux.dev, linux-acpi@vger.kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, kvmarm@lists.linux.dev, x86@kernel.org, linux-csky@vger.kernel.org, linux-doc@vger.kernel.org, linux-ia64@vger.kernel.org, linux-parisc@vger.kernel.org Cc: Salil Mehta <salil.mehta@huawei.com>, Jean-Philippe Brucker <jean-philippe@linaro.org>, jianyong.wu@arm.com, justin.he@arm.com, James Morse <james.morse@arm.com>, Sudeep Holla <sudeep.holla@arm.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Rafael J. Wysocki" <rafael@kernel.org> Subject: [PATCH RFC 01/22] arch_topology: Make register_cpu_capacity_sysctl() tolerant to late CPUs MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" Message-Id: <E1r0JKl-00CTwT-Hx@rmk-PC.armlinux.org.uk> Sender: Russell King <rmk@armlinux.org.uk> Date: Tue, 07 Nov 2023 10:29:23 +0000 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 07 Nov 2023 02:29:40 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781900753232416608 X-GMAIL-MSGID: 1781900753232416608 |
Series |
Initial cleanups for vCPU hotplug
|
|
Commit Message
Russell King (Oracle)
Nov. 7, 2023, 10:29 a.m. UTC
From: James Morse <james.morse@arm.com> register_cpu_capacity_sysctl() adds a property to sysfs that describes the CPUs capacity. This is done from a subsys_initcall() that assumes all possible CPUs are registered. With CPU hotplug, possible CPUs aren't registered until they become present, (or for arm64 enabled). This leads to messages during boot: | register_cpu_capacity_sysctl: too early to get CPU1 device! and once these CPUs are added to the system, the file is missing. Move this to a cpuhp callback, so that the file is created once CPUs are brought online. This covers CPUs that are added late by mechanisms like hotplug. One observable difference is the file is now missing for offline CPUs. Signed-off-by: James Morse <james.morse@arm.com> --- If the offline CPUs thing is a problem for the tools that consume this value, we'd need to move cpu_capacity to be part of cpu.c's common_cpu_attr_groups. --- drivers/base/arch_topology.c | 38 ++++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 12 deletions(-)
Comments
On 11/7/23 20:29, Russell King (Oracle) wrote: > From: James Morse <james.morse@arm.com> > > register_cpu_capacity_sysctl() adds a property to sysfs that describes > the CPUs capacity. This is done from a subsys_initcall() that assumes > all possible CPUs are registered. > > With CPU hotplug, possible CPUs aren't registered until they become > present, (or for arm64 enabled). This leads to messages during boot: > | register_cpu_capacity_sysctl: too early to get CPU1 device! > and once these CPUs are added to the system, the file is missing. > > Move this to a cpuhp callback, so that the file is created once > CPUs are brought online. This covers CPUs that are added late by > mechanisms like hotplug. > One observable difference is the file is now missing for offline CPUs. > > Signed-off-by: James Morse <james.morse@arm.com> > --- > If the offline CPUs thing is a problem for the tools that consume > this value, we'd need to move cpu_capacity to be part of cpu.c's > common_cpu_attr_groups. > --- > drivers/base/arch_topology.c | 38 ++++++++++++++++++++++++------------ > 1 file changed, 26 insertions(+), 12 deletions(-) > Reviewed-by: Gavin Shan <gshan@redhat.com>
On Tue, 07 Nov 2023 10:29:23 +0000 Russell King <rmk+kernel@armlinux.org.uk> wrote: > From: James Morse <james.morse@arm.com> > > register_cpu_capacity_sysctl() adds a property to sysfs that describes > the CPUs capacity. This is done from a subsys_initcall() that assumes > all possible CPUs are registered. > > With CPU hotplug, possible CPUs aren't registered until they become > present, (or for arm64 enabled). This leads to messages during boot: > | register_cpu_capacity_sysctl: too early to get CPU1 device! > and once these CPUs are added to the system, the file is missing. > > Move this to a cpuhp callback, so that the file is created once > CPUs are brought online. This covers CPUs that are added late by > mechanisms like hotplug. > One observable difference is the file is now missing for offline CPUs. > > Signed-off-by: James Morse <james.morse@arm.com> > --- > If the offline CPUs thing is a problem for the tools that consume > this value, we'd need to move cpu_capacity to be part of cpu.c's > common_cpu_attr_groups. I'm not keen on squirting sysfs files in from code so might be nice to do that anyway and use is_visible() / sysfs_update_group() but that would be a job for another day if at all. Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
On Tue, Nov 28, 2023 at 02:37:22PM +0000, Jonathan Cameron wrote: > On Tue, 07 Nov 2023 10:29:23 +0000 > Russell King <rmk+kernel@armlinux.org.uk> wrote: > > > From: James Morse <james.morse@arm.com> > > > > register_cpu_capacity_sysctl() adds a property to sysfs that describes > > the CPUs capacity. This is done from a subsys_initcall() that assumes > > all possible CPUs are registered. > > > > With CPU hotplug, possible CPUs aren't registered until they become > > present, (or for arm64 enabled). This leads to messages during boot: > > | register_cpu_capacity_sysctl: too early to get CPU1 device! > > and once these CPUs are added to the system, the file is missing. > > > > Move this to a cpuhp callback, so that the file is created once > > CPUs are brought online. This covers CPUs that are added late by > > mechanisms like hotplug. > > One observable difference is the file is now missing for offline CPUs. > > > > Signed-off-by: James Morse <james.morse@arm.com> > > --- > > If the offline CPUs thing is a problem for the tools that consume > > this value, we'd need to move cpu_capacity to be part of cpu.c's > > common_cpu_attr_groups. > > I'm not keen on squirting sysfs files in from code so > might be nice to do that anyway and use is_visible() / sysfs_update_group() > but that would be a job for another day if at all. I'm doing my best, but it's really not helped by the dysfunctional nature of some parts of the kernel community. I have now decided that this is not possible to implement. So while it's a nice idea, I don't think we'll ever see it. As I mentioned on the 14th November, complete with a patch (and got no response from anyone): > Looking into doing this, the easy bit is adding the attribute group > with an appropriate .is_visible dependent on cpu_present(), but we > need to be able to call sysfs_update_groups() when the state of the > .is_visible() changes. > > Given the comment in sysfs_update_groups() about "if an error occurs", > rather than making this part of common_cpu_attr_groups, would it be > better that it's part of its own set of groups, thus limiting the > damage from a possible error? I suspect, however, that any error at > that point means that the system is rather fatally wounded. > > This is what I have so far to implement your idea, less the necessary > sysfs_update_groups() call when we need to change the visibility of > the attributes.
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index b741b5ba82bd..9ccb7daee78e 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -220,20 +220,34 @@ static DECLARE_WORK(update_topology_flags_work, update_topology_flags_workfn); static DEVICE_ATTR_RO(cpu_capacity); -static int register_cpu_capacity_sysctl(void) +static int cpu_capacity_sysctl_add(unsigned int cpu) { - int i; - struct device *cpu; + struct device *cpu_dev = get_cpu_device(cpu); - for_each_possible_cpu(i) { - cpu = get_cpu_device(i); - if (!cpu) { - pr_err("%s: too early to get CPU%d device!\n", - __func__, i); - continue; - } - device_create_file(cpu, &dev_attr_cpu_capacity); - } + if (!cpu_dev) + return -ENOENT; + + device_create_file(cpu_dev, &dev_attr_cpu_capacity); + + return 0; +} + +static int cpu_capacity_sysctl_remove(unsigned int cpu) +{ + struct device *cpu_dev = get_cpu_device(cpu); + + if (!cpu_dev) + return -ENOENT; + + device_remove_file(cpu_dev, &dev_attr_cpu_capacity); + + return 0; +} + +static int register_cpu_capacity_sysctl(void) +{ + cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "topology/cpu-capacity", + cpu_capacity_sysctl_add, cpu_capacity_sysctl_remove); return 0; }